engram-echo 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- engram_echo-0.1.0/LICENSE +21 -0
- engram_echo-0.1.0/MANIFEST.in +3 -0
- engram_echo-0.1.0/PKG-INFO +305 -0
- engram_echo-0.1.0/README.md +271 -0
- engram_echo-0.1.0/engram/__init__.py +6 -0
- engram_echo-0.1.0/engram/graph.py +615 -0
- engram_echo-0.1.0/engram_cli.py +1406 -0
- engram_echo-0.1.0/engram_echo.egg-info/PKG-INFO +305 -0
- engram_echo-0.1.0/engram_echo.egg-info/SOURCES.txt +28 -0
- engram_echo-0.1.0/engram_echo.egg-info/dependency_links.txt +1 -0
- engram_echo-0.1.0/engram_echo.egg-info/entry_points.txt +2 -0
- engram_echo-0.1.0/engram_echo.egg-info/requires.txt +1 -0
- engram_echo-0.1.0/engram_echo.egg-info/top_level.txt +2 -0
- engram_echo-0.1.0/plugin/.claude-plugin/plugin.json +7 -0
- engram_echo-0.1.0/plugin/bin/engram-hook +70 -0
- engram_echo-0.1.0/plugin/bin/engram-pretool +146 -0
- engram_echo-0.1.0/plugin/commands/engram-community.md +39 -0
- engram_echo-0.1.0/plugin/commands/engram-feedback.md +38 -0
- engram_echo-0.1.0/plugin/commands/engram-full.md +50 -0
- engram_echo-0.1.0/plugin/commands/engram-off.md +5 -0
- engram_echo-0.1.0/plugin/commands/engram-on.md +5 -0
- engram_echo-0.1.0/plugin/commands/engram-query.md +14 -0
- engram_echo-0.1.0/plugin/commands/engram-status.md +11 -0
- engram_echo-0.1.0/plugin/commands/engram.md +106 -0
- engram_echo-0.1.0/plugin/hooks/hooks.json +29 -0
- engram_echo-0.1.0/setup.cfg +4 -0
- engram_echo-0.1.0/setup.py +33 -0
- engram_echo-0.1.0/tests/test_cli_e2e.py +467 -0
- engram_echo-0.1.0/tests/test_features.py +1150 -0
- engram_echo-0.1.0/tests/test_smoke.py +65 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 qianheng
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,305 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: engram-echo
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Persistent memory for Claude Code — knowledge graph in Obsidian vault
|
|
5
|
+
Home-page: https://github.com/qianheng-aws/engram
|
|
6
|
+
Author: qianheng
|
|
7
|
+
Author-email: hengqian.qh@gmail.com
|
|
8
|
+
License: MIT
|
|
9
|
+
Keywords: claude-code memory knowledge-graph obsidian
|
|
10
|
+
Classifier: Development Status :: 3 - Alpha
|
|
11
|
+
Classifier: Intended Audience :: Developers
|
|
12
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
17
|
+
Classifier: Topic :: Software Development :: Libraries
|
|
18
|
+
Requires-Python: >=3.10
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
License-File: LICENSE
|
|
21
|
+
Requires-Dist: networkx
|
|
22
|
+
Dynamic: author
|
|
23
|
+
Dynamic: author-email
|
|
24
|
+
Dynamic: classifier
|
|
25
|
+
Dynamic: description
|
|
26
|
+
Dynamic: description-content-type
|
|
27
|
+
Dynamic: home-page
|
|
28
|
+
Dynamic: keywords
|
|
29
|
+
Dynamic: license
|
|
30
|
+
Dynamic: license-file
|
|
31
|
+
Dynamic: requires-dist
|
|
32
|
+
Dynamic: requires-python
|
|
33
|
+
Dynamic: summary
|
|
34
|
+
|
|
35
|
+
<div align="center">
|
|
36
|
+
|
|
37
|
+
# 🧠 Engram
|
|
38
|
+
|
|
39
|
+
**Persistent memory for Claude Code — your coding sessions become a knowledge graph**
|
|
40
|
+
|
|
41
|
+
[](https://opensource.org/licenses/MIT)
|
|
42
|
+
[](https://www.python.org/downloads/)
|
|
43
|
+
[](https://docs.anthropic.com/en/docs/claude-code)
|
|
44
|
+
[-green.svg)](#installation)
|
|
45
|
+
|
|
46
|
+
*Like sleep consolidates human memory, `/engram` consolidates your coding sessions into reusable knowledge.*
|
|
47
|
+
|
|
48
|
+
</div>
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
## ✨ What It Does
|
|
53
|
+
|
|
54
|
+
You work in Claude Code as usual. When you're done, run `/engram`. That's it.
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
You: /engram
|
|
58
|
+
|
|
59
|
+
CC: Analyzing session... Found 5 entities, 3 relations.
|
|
60
|
+
✅ Saved to vault: STDERR_PIPE_BLOCKING, CLAUDE_SLACK_BRIDGE, ...
|
|
61
|
+
📝 Daily note: 2026-04-06.md
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
Behind the scenes:
|
|
65
|
+
|
|
66
|
+
```
|
|
67
|
+
CC Session → /engram → Entity Extraction → Knowledge Graph → Obsidian Vault
|
|
68
|
+
(CC as LLM) (NetworkX) (Markdown + [[wikilinks]])
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Your knowledge accumulates across sessions. Query it anytime:
|
|
72
|
+
|
|
73
|
+
```
|
|
74
|
+
You: /engram-query how did I fix the stderr bug?
|
|
75
|
+
|
|
76
|
+
CC: Found STDERR_PIPE_BLOCKING → Bug where claude process stderr fills 64KB
|
|
77
|
+
pipe buffer, blocking stdout. Fixed by adding _drain_stderr async task.
|
|
78
|
+
Related: [[CLAUDE_SLACK_BRIDGE]]
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
## 🏗️ Architecture
|
|
82
|
+
|
|
83
|
+
Built on the **OODA loop** — the same decision framework used by fighter pilots:
|
|
84
|
+
|
|
85
|
+
| Phase | What | How |
|
|
86
|
+
|:------|:-----|:----|
|
|
87
|
+
| **🔍 Observe** | Capture session conversations | CC session JSONL parser |
|
|
88
|
+
| **🧭 Orient** | Extract entities & relations | CC does entity extraction — no external API |
|
|
89
|
+
| **🎯 Decide** | Consolidate memory | 7-stage: replay → feedback → integrate → prune → community → abstract → lint |
|
|
90
|
+
| **⚡ Act** | Persist to vault | Obsidian markdown + NetworkX GraphML |
|
|
91
|
+
|
|
92
|
+
### Zero External Dependencies
|
|
93
|
+
|
|
94
|
+
- **No API keys** — CC itself is the LLM
|
|
95
|
+
- **No vector database** — graph-only retrieval with CC entity routing
|
|
96
|
+
- **No Docker** — just Python + networkx
|
|
97
|
+
- **No cloud services** — everything runs locally
|
|
98
|
+
|
|
99
|
+
## 📦 Installation
|
|
100
|
+
|
|
101
|
+
### Prerequisites
|
|
102
|
+
|
|
103
|
+
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code) installed
|
|
104
|
+
- Python 3.10+ with `networkx`: `pip install networkx`
|
|
105
|
+
|
|
106
|
+
### Setup
|
|
107
|
+
|
|
108
|
+
```bash
|
|
109
|
+
# 1. Clone and install
|
|
110
|
+
git clone https://github.com/qianheng-aws/engram.git
|
|
111
|
+
cd engram
|
|
112
|
+
pip install -e .
|
|
113
|
+
|
|
114
|
+
# 2. Initialize vault
|
|
115
|
+
engram init ~/.engram/vault
|
|
116
|
+
|
|
117
|
+
# 3. Register as CC plugin
|
|
118
|
+
claude plugin marketplace add ./
|
|
119
|
+
claude plugin install engram
|
|
120
|
+
|
|
121
|
+
# 4. (Optional) Enable auto-capture on session end
|
|
122
|
+
engram auto on
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
## 🎮 Commands
|
|
126
|
+
|
|
127
|
+
| Command | Description |
|
|
128
|
+
|:--------|:------------|
|
|
129
|
+
| `/engram` | Extract entities and relations from current session |
|
|
130
|
+
| `/engram-full` | Full consolidation: replay → feedback → integrate → prune → community → abstract → lint |
|
|
131
|
+
| `/engram-feedback` | Process human corrections from Obsidian callouts |
|
|
132
|
+
| `/engram-community` | Detect and summarize knowledge clusters (Louvain) |
|
|
133
|
+
| `/engram-status` | Show vault statistics, graph analysis, and pending sessions |
|
|
134
|
+
| `/engram-query <question>` | Search knowledge graph (keyword + graph traversal) |
|
|
135
|
+
| `/engram-on` | Enable auto-capture on session end |
|
|
136
|
+
| `/engram-off` | Disable auto-capture |
|
|
137
|
+
|
|
138
|
+
### `/engram` vs `/engram-full`
|
|
139
|
+
|
|
140
|
+
| | `/engram` | `/engram-full` |
|
|
141
|
+
|:--|:---------|:--------------|
|
|
142
|
+
| Stages | Replay only | All 6 stages + lint |
|
|
143
|
+
| Speed | Fast (one extraction) | Slower (multi-step) |
|
|
144
|
+
| When | Every session | Auto-triggered when needed |
|
|
145
|
+
|
|
146
|
+
**Auto-consolidation:** After each `/engram`, the CLI tracks how many replays have occurred since the last full run. When thresholds are reached, `/engram` will remind you (tier 1) or automatically escalate to a full consolidation (tier 2). Thresholds are configurable in `~/.engram/config.json`:
|
|
147
|
+
|
|
148
|
+
```json
|
|
149
|
+
{
|
|
150
|
+
"consolidation": {
|
|
151
|
+
"remind_after_replays": 10,
|
|
152
|
+
"remind_after_days": 7,
|
|
153
|
+
"force_after_replays": 15,
|
|
154
|
+
"force_after_days": 14
|
|
155
|
+
}
|
|
156
|
+
}
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
## 🌙 Consolidation Stages
|
|
160
|
+
|
|
161
|
+
```
|
|
162
|
+
┌──────────┐ ┌──────────┐ ┌───────────┐ ┌─────────┐ ┌───────────┐ ┌──────────┐ ┌────────┐
|
|
163
|
+
│ Replay │ → │ Feedback │ → │ Integrate │ → │ Prune │ → │ Community │ → │ Abstract │ → │ Lint │
|
|
164
|
+
│ │ │ │ │ │ │ │ │ │ │ │ │ │
|
|
165
|
+
│ Extract │ │ Human │ │ Merge │ │ Decay │ │ Cluster │ │ Discover │ │ Verify │
|
|
166
|
+
│ entities │ │ review │ │ dupes │ │ old │ │ & label │ │ patterns │ │ vault │
|
|
167
|
+
└──────────┘ └──────────┘ └───────────┘ └─────────┘ └───────────┘ └──────────┘ └────────┘
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
- **Replay** — CC extracts entities/relations from session → writes to graph + daily note
|
|
171
|
+
- **Feedback** — Scans entity files for `[!correction]`/`[!merge]`/`[!delete]` callouts left by the user in Obsidian → CC applies fixes
|
|
172
|
+
- **Integrate** — Detects duplicate entities (token similarity) → CC decides merge
|
|
173
|
+
- **Prune** — Scores entities by decay (30-day half-life) → archives stale ones
|
|
174
|
+
- **Community** — Louvain clustering on the graph → CC titles and summarizes each cluster
|
|
175
|
+
- **Abstract** — Analyzes daily notes → discovers behavioral patterns (e.g., "user always debugs by observe → hypothesize → verify")
|
|
176
|
+
- **Lint** — Validates vault consistency: GraphML ↔ markdown sync, dead wikilinks, orphan nodes, frontmatter completeness
|
|
177
|
+
|
|
178
|
+
### Human Feedback (Obsidian)
|
|
179
|
+
|
|
180
|
+
The Feedback stage processes corrections you leave directly in entity files. In Obsidian:
|
|
181
|
+
|
|
182
|
+
1. Open any entity file (e.g., `entities/concepts/STDERR_PIPE_BLOCKING.md`)
|
|
183
|
+
2. Add a callout — either `Cmd+P` → search "callout", or type the syntax directly:
|
|
184
|
+
|
|
185
|
+
```markdown
|
|
186
|
+
> [!correction] More accurate description
|
|
187
|
+
> Actually the buffer limit is 64KB, not 32KB
|
|
188
|
+
|
|
189
|
+
> [!merge] Should be same as ASYNC_DRAIN_STDERR
|
|
190
|
+
|
|
191
|
+
> [!delete] False extraction, not a real entity
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
3. Run `/engram-feedback` (or `/engram-full`) — CC reads the callouts, applies fixes, and removes them
|
|
195
|
+
|
|
196
|
+
## 🗂️ Vault Structure
|
|
197
|
+
|
|
198
|
+
Open the vault in [Obsidian](https://obsidian.md) to get an interactive knowledge graph:
|
|
199
|
+
|
|
200
|
+
1. Install Obsidian from [obsidian.md](https://obsidian.md)
|
|
201
|
+
2. Open Obsidian → **Open folder as vault** → In the file picker, press `Cmd+Shift+G` and type `~/.engram/vault`
|
|
202
|
+
- Alternatively, use a visible path: `engram init ~/engram/vault`
|
|
203
|
+
3. Toggle **Graph View** (Ctrl/Cmd + G) to see your knowledge graph
|
|
204
|
+
|
|
205
|
+
<div align="center">
|
|
206
|
+
<img src="docs/obsidian-graph.png" alt="Engram knowledge graph in Obsidian" width="700">
|
|
207
|
+
|
|
208
|
+
*Entity nodes colored by type, with tags, wikilinks, and Graph View visualization*
|
|
209
|
+
</div>
|
|
210
|
+
|
|
211
|
+
```
|
|
212
|
+
~/.engram/vault/
|
|
213
|
+
├── 📁 entities/ # Knowledge graph nodes
|
|
214
|
+
│ ├── people/ # PERSON entities
|
|
215
|
+
│ ├── concepts/ # Bugs, patterns, designs
|
|
216
|
+
│ ├── projects/ # Repos, packages
|
|
217
|
+
│ ├── tools/ # Libraries, frameworks
|
|
218
|
+
│ └── orgs/ # Teams, companies
|
|
219
|
+
├── 📁 relations/ # Edge table with weights
|
|
220
|
+
├── 📁 communities/ # Louvain cluster summaries
|
|
221
|
+
├── 📁 groups/ # Hyperedge MOC (Map of Content) files
|
|
222
|
+
├── 📁 daily/ # Session summaries by date
|
|
223
|
+
├── 📁 patterns/ # Discovered behavioral patterns
|
|
224
|
+
└── 📁 _meta/ # System data (GraphML, queue, lock)
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
### Entity Example
|
|
228
|
+
|
|
229
|
+
```markdown
|
|
230
|
+
---
|
|
231
|
+
entity_type: CONCEPT
|
|
232
|
+
tags:
|
|
233
|
+
- entity/concept
|
|
234
|
+
aliases:
|
|
235
|
+
- "Stderr Pipe Blocking"
|
|
236
|
+
created: 2026-04-03
|
|
237
|
+
last_updated: 2026-04-03
|
|
238
|
+
degree: 1
|
|
239
|
+
cssclasses:
|
|
240
|
+
- entity
|
|
241
|
+
- concept
|
|
242
|
+
---
|
|
243
|
+
|
|
244
|
+
# STDERR_PIPE_BLOCKING
|
|
245
|
+
|
|
246
|
+
Bug in claude-slack-bridge where claude process writes verbose logs
|
|
247
|
+
to stderr but daemon never reads it, causing 64KB buffer to fill
|
|
248
|
+
and block the entire process. Fixed by adding _drain_stderr task.
|
|
249
|
+
|
|
250
|
+
## Relations
|
|
251
|
+
|
|
252
|
+
- [[CLAUDE_SLACK_BRIDGE]] `PROJECT` — Bridge had this bug causing sessions to hang (weight: 0.8)
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
Every entity links to related entities via `[[wikilinks]]` — Obsidian renders these as an interactive graph. Tags, aliases, and cssclasses enable Dataview queries and Graph View styling.
|
|
256
|
+
|
|
257
|
+
## 🔧 Design Decisions
|
|
258
|
+
|
|
259
|
+
| Decision | Why |
|
|
260
|
+
|:---------|:----|
|
|
261
|
+
| **CC as LLM** | No API keys needed. CC extracts entities directly. |
|
|
262
|
+
| **Graph-only retrieval** | No embeddings. CC picks relevant entities from the full list. Scales to ~2000 entities. |
|
|
263
|
+
| **nano-graphrag reference** | Reused prompt templates and storage format, not runtime. |
|
|
264
|
+
| **Obsidian-native** | All output is valid Obsidian markdown. Open vault → instant graph view. |
|
|
265
|
+
| **Description cap** | Keep first + latest description only. Prevents infinite growth. |
|
|
266
|
+
| **File lock** | `fcntl.LOCK_EX` + read-merge-write prevents concurrent corruption. |
|
|
267
|
+
|
|
268
|
+
## 🛠️ CLI Reference
|
|
269
|
+
|
|
270
|
+
```bash
|
|
271
|
+
engram init [PATH] # Initialize vault + register in ~/.claude/CLAUDE.md
|
|
272
|
+
engram auto [on|off|status] # Toggle auto-capture on session end
|
|
273
|
+
engram status # Vault statistics + hub entities + density
|
|
274
|
+
engram query --question "..." # Search graph
|
|
275
|
+
engram replay --stdin # Process extracted entity/relation JSON
|
|
276
|
+
engram integrate # Detect duplicate entities
|
|
277
|
+
echo '<json>' | engram integrate --stdin # Execute merges
|
|
278
|
+
engram prune # Report decay scores
|
|
279
|
+
echo '<json>' | engram prune --stdin # Archive entities
|
|
280
|
+
engram community # Detect knowledge clusters
|
|
281
|
+
echo '<json>' | engram community --stdin # Save community summaries
|
|
282
|
+
engram abstract # Gather data for pattern discovery
|
|
283
|
+
echo '<json>' | engram save-pattern --stdin # Save discovered patterns
|
|
284
|
+
engram feedback # Scan entity files for correction callouts
|
|
285
|
+
echo '<json>' | engram feedback --stdin # Apply corrections/merges/deletes
|
|
286
|
+
engram context # Compact summary for system prompt injection
|
|
287
|
+
engram lint # Validate vault consistency
|
|
288
|
+
engram consolidation # Show consolidation tracking state
|
|
289
|
+
engram consolidation --reset # Reset counter (after full consolidation)
|
|
290
|
+
engram install # Re-register in ~/.claude/CLAUDE.md (auto on init)
|
|
291
|
+
engram uninstall # Remove from ~/.claude/CLAUDE.md
|
|
292
|
+
# All commands use the vault from last `engram init`. Override with --vault PATH.
|
|
293
|
+
```
|
|
294
|
+
|
|
295
|
+
## 📄 License
|
|
296
|
+
|
|
297
|
+
[MIT](LICENSE) — do whatever you want with it.
|
|
298
|
+
|
|
299
|
+
---
|
|
300
|
+
|
|
301
|
+
<div align="center">
|
|
302
|
+
|
|
303
|
+
*Built in one afternoon with Claude Code. The tool that remembers itself.* 🐾
|
|
304
|
+
|
|
305
|
+
</div>
|
|
@@ -0,0 +1,271 @@
|
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
3
|
+
# 🧠 Engram
|
|
4
|
+
|
|
5
|
+
**Persistent memory for Claude Code — your coding sessions become a knowledge graph**
|
|
6
|
+
|
|
7
|
+
[](https://opensource.org/licenses/MIT)
|
|
8
|
+
[](https://www.python.org/downloads/)
|
|
9
|
+
[](https://docs.anthropic.com/en/docs/claude-code)
|
|
10
|
+
[-green.svg)](#installation)
|
|
11
|
+
|
|
12
|
+
*Like sleep consolidates human memory, `/engram` consolidates your coding sessions into reusable knowledge.*
|
|
13
|
+
|
|
14
|
+
</div>
|
|
15
|
+
|
|
16
|
+
---
|
|
17
|
+
|
|
18
|
+
## ✨ What It Does
|
|
19
|
+
|
|
20
|
+
You work in Claude Code as usual. When you're done, run `/engram`. That's it.
|
|
21
|
+
|
|
22
|
+
```
|
|
23
|
+
You: /engram
|
|
24
|
+
|
|
25
|
+
CC: Analyzing session... Found 5 entities, 3 relations.
|
|
26
|
+
✅ Saved to vault: STDERR_PIPE_BLOCKING, CLAUDE_SLACK_BRIDGE, ...
|
|
27
|
+
📝 Daily note: 2026-04-06.md
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
Behind the scenes:
|
|
31
|
+
|
|
32
|
+
```
|
|
33
|
+
CC Session → /engram → Entity Extraction → Knowledge Graph → Obsidian Vault
|
|
34
|
+
(CC as LLM) (NetworkX) (Markdown + [[wikilinks]])
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
Your knowledge accumulates across sessions. Query it anytime:
|
|
38
|
+
|
|
39
|
+
```
|
|
40
|
+
You: /engram-query how did I fix the stderr bug?
|
|
41
|
+
|
|
42
|
+
CC: Found STDERR_PIPE_BLOCKING → Bug where claude process stderr fills 64KB
|
|
43
|
+
pipe buffer, blocking stdout. Fixed by adding _drain_stderr async task.
|
|
44
|
+
Related: [[CLAUDE_SLACK_BRIDGE]]
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
## 🏗️ Architecture
|
|
48
|
+
|
|
49
|
+
Built on the **OODA loop** — the same decision framework used by fighter pilots:
|
|
50
|
+
|
|
51
|
+
| Phase | What | How |
|
|
52
|
+
|:------|:-----|:----|
|
|
53
|
+
| **🔍 Observe** | Capture session conversations | CC session JSONL parser |
|
|
54
|
+
| **🧭 Orient** | Extract entities & relations | CC does entity extraction — no external API |
|
|
55
|
+
| **🎯 Decide** | Consolidate memory | 7-stage: replay → feedback → integrate → prune → community → abstract → lint |
|
|
56
|
+
| **⚡ Act** | Persist to vault | Obsidian markdown + NetworkX GraphML |
|
|
57
|
+
|
|
58
|
+
### Zero External Dependencies
|
|
59
|
+
|
|
60
|
+
- **No API keys** — CC itself is the LLM
|
|
61
|
+
- **No vector database** — graph-only retrieval with CC entity routing
|
|
62
|
+
- **No Docker** — just Python + networkx
|
|
63
|
+
- **No cloud services** — everything runs locally
|
|
64
|
+
|
|
65
|
+
## 📦 Installation
|
|
66
|
+
|
|
67
|
+
### Prerequisites
|
|
68
|
+
|
|
69
|
+
- [Claude Code](https://docs.anthropic.com/en/docs/claude-code) installed
|
|
70
|
+
- Python 3.10+ with `networkx`: `pip install networkx`
|
|
71
|
+
|
|
72
|
+
### Setup
|
|
73
|
+
|
|
74
|
+
```bash
|
|
75
|
+
# 1. Clone and install
|
|
76
|
+
git clone https://github.com/qianheng-aws/engram.git
|
|
77
|
+
cd engram
|
|
78
|
+
pip install -e .
|
|
79
|
+
|
|
80
|
+
# 2. Initialize vault
|
|
81
|
+
engram init ~/.engram/vault
|
|
82
|
+
|
|
83
|
+
# 3. Register as CC plugin
|
|
84
|
+
claude plugin marketplace add ./
|
|
85
|
+
claude plugin install engram
|
|
86
|
+
|
|
87
|
+
# 4. (Optional) Enable auto-capture on session end
|
|
88
|
+
engram auto on
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
## 🎮 Commands
|
|
92
|
+
|
|
93
|
+
| Command | Description |
|
|
94
|
+
|:--------|:------------|
|
|
95
|
+
| `/engram` | Extract entities and relations from current session |
|
|
96
|
+
| `/engram-full` | Full consolidation: replay → feedback → integrate → prune → community → abstract → lint |
|
|
97
|
+
| `/engram-feedback` | Process human corrections from Obsidian callouts |
|
|
98
|
+
| `/engram-community` | Detect and summarize knowledge clusters (Louvain) |
|
|
99
|
+
| `/engram-status` | Show vault statistics, graph analysis, and pending sessions |
|
|
100
|
+
| `/engram-query <question>` | Search knowledge graph (keyword + graph traversal) |
|
|
101
|
+
| `/engram-on` | Enable auto-capture on session end |
|
|
102
|
+
| `/engram-off` | Disable auto-capture |
|
|
103
|
+
|
|
104
|
+
### `/engram` vs `/engram-full`
|
|
105
|
+
|
|
106
|
+
| | `/engram` | `/engram-full` |
|
|
107
|
+
|:--|:---------|:--------------|
|
|
108
|
+
| Stages | Replay only | All 6 stages + lint |
|
|
109
|
+
| Speed | Fast (one extraction) | Slower (multi-step) |
|
|
110
|
+
| When | Every session | Auto-triggered when needed |
|
|
111
|
+
|
|
112
|
+
**Auto-consolidation:** After each `/engram`, the CLI tracks how many replays have occurred since the last full run. When thresholds are reached, `/engram` will remind you (tier 1) or automatically escalate to a full consolidation (tier 2). Thresholds are configurable in `~/.engram/config.json`:
|
|
113
|
+
|
|
114
|
+
```json
|
|
115
|
+
{
|
|
116
|
+
"consolidation": {
|
|
117
|
+
"remind_after_replays": 10,
|
|
118
|
+
"remind_after_days": 7,
|
|
119
|
+
"force_after_replays": 15,
|
|
120
|
+
"force_after_days": 14
|
|
121
|
+
}
|
|
122
|
+
}
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
## 🌙 Consolidation Stages
|
|
126
|
+
|
|
127
|
+
```
|
|
128
|
+
┌──────────┐ ┌──────────┐ ┌───────────┐ ┌─────────┐ ┌───────────┐ ┌──────────┐ ┌────────┐
|
|
129
|
+
│ Replay │ → │ Feedback │ → │ Integrate │ → │ Prune │ → │ Community │ → │ Abstract │ → │ Lint │
|
|
130
|
+
│ │ │ │ │ │ │ │ │ │ │ │ │ │
|
|
131
|
+
│ Extract │ │ Human │ │ Merge │ │ Decay │ │ Cluster │ │ Discover │ │ Verify │
|
|
132
|
+
│ entities │ │ review │ │ dupes │ │ old │ │ & label │ │ patterns │ │ vault │
|
|
133
|
+
└──────────┘ └──────────┘ └───────────┘ └─────────┘ └───────────┘ └──────────┘ └────────┘
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
- **Replay** — CC extracts entities/relations from session → writes to graph + daily note
|
|
137
|
+
- **Feedback** — Scans entity files for `[!correction]`/`[!merge]`/`[!delete]` callouts left by the user in Obsidian → CC applies fixes
|
|
138
|
+
- **Integrate** — Detects duplicate entities (token similarity) → CC decides merge
|
|
139
|
+
- **Prune** — Scores entities by decay (30-day half-life) → archives stale ones
|
|
140
|
+
- **Community** — Louvain clustering on the graph → CC titles and summarizes each cluster
|
|
141
|
+
- **Abstract** — Analyzes daily notes → discovers behavioral patterns (e.g., "user always debugs by observe → hypothesize → verify")
|
|
142
|
+
- **Lint** — Validates vault consistency: GraphML ↔ markdown sync, dead wikilinks, orphan nodes, frontmatter completeness
|
|
143
|
+
|
|
144
|
+
### Human Feedback (Obsidian)
|
|
145
|
+
|
|
146
|
+
The Feedback stage processes corrections you leave directly in entity files. In Obsidian:
|
|
147
|
+
|
|
148
|
+
1. Open any entity file (e.g., `entities/concepts/STDERR_PIPE_BLOCKING.md`)
|
|
149
|
+
2. Add a callout — either `Cmd+P` → search "callout", or type the syntax directly:
|
|
150
|
+
|
|
151
|
+
```markdown
|
|
152
|
+
> [!correction] More accurate description
|
|
153
|
+
> Actually the buffer limit is 64KB, not 32KB
|
|
154
|
+
|
|
155
|
+
> [!merge] Should be same as ASYNC_DRAIN_STDERR
|
|
156
|
+
|
|
157
|
+
> [!delete] False extraction, not a real entity
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
3. Run `/engram-feedback` (or `/engram-full`) — CC reads the callouts, applies fixes, and removes them
|
|
161
|
+
|
|
162
|
+
## 🗂️ Vault Structure
|
|
163
|
+
|
|
164
|
+
Open the vault in [Obsidian](https://obsidian.md) to get an interactive knowledge graph:
|
|
165
|
+
|
|
166
|
+
1. Install Obsidian from [obsidian.md](https://obsidian.md)
|
|
167
|
+
2. Open Obsidian → **Open folder as vault** → In the file picker, press `Cmd+Shift+G` and type `~/.engram/vault`
|
|
168
|
+
- Alternatively, use a visible path: `engram init ~/engram/vault`
|
|
169
|
+
3. Toggle **Graph View** (Ctrl/Cmd + G) to see your knowledge graph
|
|
170
|
+
|
|
171
|
+
<div align="center">
|
|
172
|
+
<img src="docs/obsidian-graph.png" alt="Engram knowledge graph in Obsidian" width="700">
|
|
173
|
+
|
|
174
|
+
*Entity nodes colored by type, with tags, wikilinks, and Graph View visualization*
|
|
175
|
+
</div>
|
|
176
|
+
|
|
177
|
+
```
|
|
178
|
+
~/.engram/vault/
|
|
179
|
+
├── 📁 entities/ # Knowledge graph nodes
|
|
180
|
+
│ ├── people/ # PERSON entities
|
|
181
|
+
│ ├── concepts/ # Bugs, patterns, designs
|
|
182
|
+
│ ├── projects/ # Repos, packages
|
|
183
|
+
│ ├── tools/ # Libraries, frameworks
|
|
184
|
+
│ └── orgs/ # Teams, companies
|
|
185
|
+
├── 📁 relations/ # Edge table with weights
|
|
186
|
+
├── 📁 communities/ # Louvain cluster summaries
|
|
187
|
+
├── 📁 groups/ # Hyperedge MOC (Map of Content) files
|
|
188
|
+
├── 📁 daily/ # Session summaries by date
|
|
189
|
+
├── 📁 patterns/ # Discovered behavioral patterns
|
|
190
|
+
└── 📁 _meta/ # System data (GraphML, queue, lock)
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
### Entity Example
|
|
194
|
+
|
|
195
|
+
```markdown
|
|
196
|
+
---
|
|
197
|
+
entity_type: CONCEPT
|
|
198
|
+
tags:
|
|
199
|
+
- entity/concept
|
|
200
|
+
aliases:
|
|
201
|
+
- "Stderr Pipe Blocking"
|
|
202
|
+
created: 2026-04-03
|
|
203
|
+
last_updated: 2026-04-03
|
|
204
|
+
degree: 1
|
|
205
|
+
cssclasses:
|
|
206
|
+
- entity
|
|
207
|
+
- concept
|
|
208
|
+
---
|
|
209
|
+
|
|
210
|
+
# STDERR_PIPE_BLOCKING
|
|
211
|
+
|
|
212
|
+
Bug in claude-slack-bridge where claude process writes verbose logs
|
|
213
|
+
to stderr but daemon never reads it, causing 64KB buffer to fill
|
|
214
|
+
and block the entire process. Fixed by adding _drain_stderr task.
|
|
215
|
+
|
|
216
|
+
## Relations
|
|
217
|
+
|
|
218
|
+
- [[CLAUDE_SLACK_BRIDGE]] `PROJECT` — Bridge had this bug causing sessions to hang (weight: 0.8)
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
Every entity links to related entities via `[[wikilinks]]` — Obsidian renders these as an interactive graph. Tags, aliases, and cssclasses enable Dataview queries and Graph View styling.
|
|
222
|
+
|
|
223
|
+
## 🔧 Design Decisions
|
|
224
|
+
|
|
225
|
+
| Decision | Why |
|
|
226
|
+
|:---------|:----|
|
|
227
|
+
| **CC as LLM** | No API keys needed. CC extracts entities directly. |
|
|
228
|
+
| **Graph-only retrieval** | No embeddings. CC picks relevant entities from the full list. Scales to ~2000 entities. |
|
|
229
|
+
| **nano-graphrag reference** | Reused prompt templates and storage format, not runtime. |
|
|
230
|
+
| **Obsidian-native** | All output is valid Obsidian markdown. Open vault → instant graph view. |
|
|
231
|
+
| **Description cap** | Keep first + latest description only. Prevents infinite growth. |
|
|
232
|
+
| **File lock** | `fcntl.LOCK_EX` + read-merge-write prevents concurrent corruption. |
|
|
233
|
+
|
|
234
|
+
## 🛠️ CLI Reference
|
|
235
|
+
|
|
236
|
+
```bash
|
|
237
|
+
engram init [PATH] # Initialize vault + register in ~/.claude/CLAUDE.md
|
|
238
|
+
engram auto [on|off|status] # Toggle auto-capture on session end
|
|
239
|
+
engram status # Vault statistics + hub entities + density
|
|
240
|
+
engram query --question "..." # Search graph
|
|
241
|
+
engram replay --stdin # Process extracted entity/relation JSON
|
|
242
|
+
engram integrate # Detect duplicate entities
|
|
243
|
+
echo '<json>' | engram integrate --stdin # Execute merges
|
|
244
|
+
engram prune # Report decay scores
|
|
245
|
+
echo '<json>' | engram prune --stdin # Archive entities
|
|
246
|
+
engram community # Detect knowledge clusters
|
|
247
|
+
echo '<json>' | engram community --stdin # Save community summaries
|
|
248
|
+
engram abstract # Gather data for pattern discovery
|
|
249
|
+
echo '<json>' | engram save-pattern --stdin # Save discovered patterns
|
|
250
|
+
engram feedback # Scan entity files for correction callouts
|
|
251
|
+
echo '<json>' | engram feedback --stdin # Apply corrections/merges/deletes
|
|
252
|
+
engram context # Compact summary for system prompt injection
|
|
253
|
+
engram lint # Validate vault consistency
|
|
254
|
+
engram consolidation # Show consolidation tracking state
|
|
255
|
+
engram consolidation --reset # Reset counter (after full consolidation)
|
|
256
|
+
engram install # Re-register in ~/.claude/CLAUDE.md (auto on init)
|
|
257
|
+
engram uninstall # Remove from ~/.claude/CLAUDE.md
|
|
258
|
+
# All commands use the vault from last `engram init`. Override with --vault PATH.
|
|
259
|
+
```
|
|
260
|
+
|
|
261
|
+
## 📄 License
|
|
262
|
+
|
|
263
|
+
[MIT](LICENSE) — do whatever you want with it.
|
|
264
|
+
|
|
265
|
+
---
|
|
266
|
+
|
|
267
|
+
<div align="center">
|
|
268
|
+
|
|
269
|
+
*Built in one afternoon with Claude Code. The tool that remembers itself.* 🐾
|
|
270
|
+
|
|
271
|
+
</div>
|