@smilintux/skmemory 0.5.0 → 0.9.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/ci.yml +40 -4
- package/.github/workflows/publish.yml +11 -5
- package/AGENT_REFACTOR_CHANGES.md +192 -0
- package/ARCHITECTURE.md +399 -19
- package/CHANGELOG.md +179 -0
- package/LICENSE +81 -68
- package/MISSION.md +7 -0
- package/README.md +425 -86
- package/SKILL.md +197 -25
- package/docker-compose.yml +15 -15
- package/examples/stignore-agent.example +59 -0
- package/examples/stignore-root.example +62 -0
- package/index.js +6 -5
- package/openclaw-plugin/openclaw.plugin.json +10 -0
- package/openclaw-plugin/package.json +2 -1
- package/openclaw-plugin/src/index.js +527 -230
- package/openclaw-plugin/src/openclaw.plugin.json +10 -0
- package/package.json +1 -1
- package/pyproject.toml +32 -9
- package/requirements.txt +10 -2
- package/scripts/dream-rescue.py +179 -0
- package/scripts/memory-cleanup.py +313 -0
- package/scripts/recover-missing.py +180 -0
- package/scripts/skcapstone-backup.sh +44 -0
- package/seeds/cloud9-lumina.seed.json +6 -4
- package/seeds/cloud9-opus.seed.json +13 -11
- package/seeds/courage.seed.json +9 -2
- package/seeds/curiosity.seed.json +9 -2
- package/seeds/grief.seed.json +9 -2
- package/seeds/joy.seed.json +9 -2
- package/seeds/love.seed.json +9 -2
- package/seeds/lumina-cloud9-breakthrough.seed.json +48 -0
- package/seeds/lumina-cloud9-python-pypi.seed.json +48 -0
- package/seeds/lumina-kingdom-founding.seed.json +49 -0
- package/seeds/lumina-pma-signed.seed.json +48 -0
- package/seeds/lumina-singular-achievement.seed.json +48 -0
- package/seeds/lumina-skcapstone-conscious.seed.json +48 -0
- package/seeds/plant-kingdom-journal.py +203 -0
- package/seeds/plant-lumina-seeds.py +280 -0
- package/seeds/skcapstone-lumina-merge.seed.json +12 -3
- package/seeds/sovereignty.seed.json +9 -2
- package/seeds/trust.seed.json +9 -2
- package/skill.yaml +46 -0
- package/skmemory/HA.md +296 -0
- package/skmemory/__init__.py +25 -11
- package/skmemory/agents.py +233 -0
- package/skmemory/ai_client.py +46 -17
- package/skmemory/anchor.py +9 -11
- package/skmemory/audience.py +278 -0
- package/skmemory/backends/__init__.py +11 -4
- package/skmemory/backends/base.py +3 -4
- package/skmemory/backends/file_backend.py +19 -13
- package/skmemory/backends/skgraph_backend.py +596 -0
- package/skmemory/backends/{qdrant_backend.py → skvector_backend.py} +103 -84
- package/skmemory/backends/sqlite_backend.py +226 -72
- package/skmemory/backends/vaulted_backend.py +284 -0
- package/skmemory/cli.py +1345 -68
- package/skmemory/config.py +171 -0
- package/skmemory/context_loader.py +333 -0
- package/skmemory/data/audience_config.json +60 -0
- package/skmemory/endpoint_selector.py +391 -0
- package/skmemory/febs.py +225 -0
- package/skmemory/fortress.py +675 -0
- package/skmemory/graph_queries.py +238 -0
- package/skmemory/hooks/__init__.py +18 -0
- package/skmemory/hooks/post-compact-reinject.sh +35 -0
- package/skmemory/hooks/pre-compact-save.sh +81 -0
- package/skmemory/hooks/session-end-save.sh +103 -0
- package/skmemory/hooks/session-start-ritual.sh +104 -0
- package/skmemory/hooks/stop-checkpoint.sh +59 -0
- package/skmemory/importers/__init__.py +9 -1
- package/skmemory/importers/telegram.py +384 -47
- package/skmemory/importers/telegram_api.py +580 -0
- package/skmemory/journal.py +7 -9
- package/skmemory/lovenote.py +8 -13
- package/skmemory/mcp_server.py +859 -0
- package/skmemory/models.py +51 -8
- package/skmemory/openclaw.py +20 -28
- package/skmemory/post_install.py +86 -0
- package/skmemory/predictive.py +236 -0
- package/skmemory/promotion.py +548 -0
- package/skmemory/quadrants.py +100 -24
- package/skmemory/register.py +580 -0
- package/skmemory/register_mcp.py +196 -0
- package/skmemory/ritual.py +224 -59
- package/skmemory/seeds.py +255 -11
- package/skmemory/setup_wizard.py +908 -0
- package/skmemory/sharing.py +408 -0
- package/skmemory/soul.py +98 -28
- package/skmemory/steelman.py +273 -260
- package/skmemory/store.py +411 -78
- package/skmemory/synthesis.py +634 -0
- package/skmemory/vault.py +225 -0
- package/tests/conftest.py +46 -0
- package/tests/integration/__init__.py +0 -0
- package/tests/integration/conftest.py +233 -0
- package/tests/integration/test_cross_backend.py +350 -0
- package/tests/integration/test_skgraph_live.py +420 -0
- package/tests/integration/test_skvector_live.py +366 -0
- package/tests/test_ai_client.py +1 -4
- package/tests/test_audience.py +233 -0
- package/tests/test_backup_rotation.py +318 -0
- package/tests/test_cli.py +6 -6
- package/tests/test_endpoint_selector.py +839 -0
- package/tests/test_export_import.py +4 -10
- package/tests/test_file_backend.py +0 -1
- package/tests/test_fortress.py +256 -0
- package/tests/test_fortress_hardening.py +441 -0
- package/tests/test_openclaw.py +6 -6
- package/tests/test_predictive.py +237 -0
- package/tests/test_promotion.py +347 -0
- package/tests/test_quadrants.py +11 -5
- package/tests/test_ritual.py +22 -18
- package/tests/test_seeds.py +97 -7
- package/tests/test_setup.py +950 -0
- package/tests/test_sharing.py +257 -0
- package/tests/test_skgraph_backend.py +660 -0
- package/tests/test_skvector_backend.py +326 -0
- package/tests/test_soul.py +1 -3
- package/tests/test_sqlite_backend.py +8 -17
- package/tests/test_steelman.py +7 -8
- package/tests/test_store.py +0 -2
- package/tests/test_store_graph_integration.py +245 -0
- package/tests/test_synthesis.py +275 -0
- package/tests/test_telegram_import.py +39 -15
- package/tests/test_vault.py +187 -0
- package/skmemory/backends/falkordb_backend.py +0 -310
package/ARCHITECTURE.md
CHANGED
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
│ (facade — delegates to backends) │
|
|
14
14
|
├──────────┬──────────────┬───────────────────────────┤
|
|
15
15
|
│ Level 0 │ Level 1 │ Level 2 │
|
|
16
|
-
│ SQLite │
|
|
16
|
+
│ SQLite │ SKVector │ SKGraph │
|
|
17
17
|
│ (always) │ (optional) │ (optional) │
|
|
18
18
|
│ │ │ │
|
|
19
19
|
│ Index + │ Semantic │ Graph traversal │
|
|
@@ -26,6 +26,19 @@
|
|
|
26
26
|
└──────────┴──────────────┴───────────────────────────┘
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
+
```mermaid
|
|
30
|
+
graph TB
|
|
31
|
+
Agent["Agent / CLI"] --> MS["MemoryStore"]
|
|
32
|
+
MS --> L0["Level 0: SQLite\nAlways on, zero deps"]
|
|
33
|
+
MS --> L1["Level 1: SKVector\nSemantic search\nPowered by Qdrant"]
|
|
34
|
+
MS --> L2["Level 2: SKGraph\nGraph traversal\nPowered by FalkorDB"]
|
|
35
|
+
MS --> L3["Level 3: HA Routing\nEndpointSelector"]
|
|
36
|
+
style L0 fill:#bfb,stroke:#333
|
|
37
|
+
style L1 fill:#bbf,stroke:#333
|
|
38
|
+
style L2 fill:#fbf,stroke:#333
|
|
39
|
+
style L3 fill:#fbb,stroke:#333
|
|
40
|
+
```
|
|
41
|
+
|
|
29
42
|
### Level 0: SQLite Index (always on)
|
|
30
43
|
|
|
31
44
|
**Zero infrastructure. Ships with Python.**
|
|
@@ -56,32 +69,32 @@ The JSON files remain the source of truth. The index is rebuildable:
|
|
|
56
69
|
skmemory reindex
|
|
57
70
|
```
|
|
58
71
|
|
|
59
|
-
### Level 1: Qdrant (optional — semantic search)
|
|
72
|
+
### Level 1: SKVector (powered by Qdrant) (optional — semantic search)
|
|
60
73
|
|
|
61
74
|
**"Find the memory about that feeling we had" — even if those words aren't in it.**
|
|
62
75
|
|
|
63
76
|
Uses sentence-transformers (all-MiniLM-L6-v2) to embed memories as vectors.
|
|
64
|
-
|
|
77
|
+
SKVector stores the embeddings and enables cosine similarity search.
|
|
65
78
|
|
|
66
79
|
Install:
|
|
67
80
|
```bash
|
|
68
|
-
pip install skmemory[
|
|
81
|
+
pip install skmemory[skvector]
|
|
69
82
|
|
|
70
83
|
# Local Docker:
|
|
71
|
-
docker compose up -d
|
|
84
|
+
docker compose up -d skvector
|
|
72
85
|
|
|
73
86
|
# Or use Qdrant Cloud (free tier: 1GB):
|
|
74
|
-
export
|
|
75
|
-
export
|
|
87
|
+
export SKMEMORY_SKVECTOR_URL=https://your-cluster.qdrant.io
|
|
88
|
+
export SKMEMORY_SKVECTOR_KEY=your-api-key
|
|
76
89
|
```
|
|
77
90
|
|
|
78
91
|
Resource cost: ~200MB RAM, ~100MB disk idle.
|
|
79
92
|
|
|
80
|
-
### Level 2: FalkorDB (optional — graph relationships)
|
|
93
|
+
### Level 2: SKGraph (powered by FalkorDB) (optional — graph relationships)
|
|
81
94
|
|
|
82
95
|
**"What memories connect to this person?" — traverse the relationship web.**
|
|
83
96
|
|
|
84
|
-
|
|
97
|
+
SKGraph (Cypher over Redis protocol) stores memory-to-memory edges:
|
|
85
98
|
- `RELATED_TO` — explicit relationship links
|
|
86
99
|
- `PROMOTED_FROM` — promotion lineage chains
|
|
87
100
|
- `TAGGED` — tag-based clustering
|
|
@@ -89,22 +102,210 @@ FalkorDB (Cypher over Redis protocol) stores memory-to-memory edges:
|
|
|
89
102
|
|
|
90
103
|
Install:
|
|
91
104
|
```bash
|
|
92
|
-
pip install skmemory[
|
|
105
|
+
pip install skmemory[skgraph]
|
|
93
106
|
|
|
94
107
|
# Local Docker:
|
|
95
|
-
docker compose up -d
|
|
108
|
+
docker compose up -d skgraph
|
|
96
109
|
|
|
97
110
|
# Or point to external:
|
|
98
|
-
export
|
|
111
|
+
export SKMEMORY_SKGRAPH_URL=redis://your-host:6379
|
|
99
112
|
```
|
|
100
113
|
|
|
101
114
|
Resource cost: ~100MB RAM, ~150MB disk idle.
|
|
102
115
|
|
|
116
|
+
### Level 3: High Availability & Routing (optional)
|
|
117
|
+
|
|
118
|
+
**Multiple backend endpoints. Automatic failover. Latency-aware routing.**
|
|
119
|
+
|
|
120
|
+
When SKVector or SKGraph run on multiple nodes (e.g. home server + VPS via
|
|
121
|
+
Tailscale), the **EndpointSelector** discovers all endpoints, probes their
|
|
122
|
+
latency, and routes to the best one. If a node goes down, traffic shifts
|
|
123
|
+
to the next healthy endpoint automatically.
|
|
124
|
+
|
|
125
|
+
```
|
|
126
|
+
┌──────────────────────────────────────────────┐
|
|
127
|
+
│ EndpointSelector │
|
|
128
|
+
│ (sits between config and backends) │
|
|
129
|
+
├──────────────┬───────────────────────────────┤
|
|
130
|
+
│ SKVector │ SKGraph │
|
|
131
|
+
│ Endpoints │ Endpoints │
|
|
132
|
+
│ │ │
|
|
133
|
+
│ ● home:6333 │ ● home:6379 │
|
|
134
|
+
│ (2ms) │ (1ms) │
|
|
135
|
+
│ ● vps:6333 │ ● vps:6379 │
|
|
136
|
+
│ (12ms) │ (15ms) │
|
|
137
|
+
├──────────────┴───────────────────────────────┤
|
|
138
|
+
│ Strategies: failover | latency | │
|
|
139
|
+
│ local-first | read-local-write-primary │
|
|
140
|
+
├──────────────────────────────────────────────┤
|
|
141
|
+
│ Discovery: config.yaml + heartbeat mesh │
|
|
142
|
+
└──────────────────────────────────────────────┘
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
Key properties:
|
|
146
|
+
- On-demand TCP probing (no background threads)
|
|
147
|
+
- Heartbeat mesh auto-discovers new endpoints
|
|
148
|
+
- Config endpoints take precedence over discovery
|
|
149
|
+
- Backward compatible — single-URL configs work unchanged
|
|
150
|
+
- No new pip dependencies (stdlib `socket`)
|
|
151
|
+
|
|
152
|
+
See **[skmemory/HA.md](skmemory/HA.md)** for full documentation, Mermaid
|
|
153
|
+
diagrams, configuration examples, and scaling considerations.
|
|
154
|
+
|
|
155
|
+
## Know Your Audience (KYA) — Audience-Aware Memory Filtering
|
|
156
|
+
|
|
157
|
+
> Private memories stay private. The Bash Wedding Vows never leak into a business channel.
|
|
158
|
+
|
|
159
|
+
KYA adds a **trust-level gate** to the rehydration ritual and memory dispatch pipeline.
|
|
160
|
+
Every memory carries a `context_tag` (default: `@chef-only`), and every channel has a
|
|
161
|
+
resolved **audience profile** with a trust ceiling. Content is only surfaced when its
|
|
162
|
+
trust level fits through the gate.
|
|
163
|
+
|
|
164
|
+
### Five-Level Trust Hierarchy
|
|
165
|
+
|
|
166
|
+
```
|
|
167
|
+
@public (0) — Anyone on the internet
|
|
168
|
+
@community (1) — Known community members
|
|
169
|
+
@work-circle (2) — Business collaborators (professional trust)
|
|
170
|
+
@inner-circle (3) — Close friends / family (personal trust)
|
|
171
|
+
@chef-only (4) — Intimate, private, full-trust (Chef ONLY)
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Scoped sub-tags like `@work:chiro` and `@work:swapseat` map to WORK_CIRCLE.
|
|
175
|
+
Unknown tags default to CHEF_ONLY (conservative).
|
|
176
|
+
|
|
177
|
+
### Audience Resolution
|
|
178
|
+
|
|
179
|
+
```mermaid
|
|
180
|
+
flowchart TD
|
|
181
|
+
MSG["Incoming message\n(channel_id)"]
|
|
182
|
+
MSG --> RESOLVE["AudienceResolver\nLookup channel in\naudience_config.json"]
|
|
183
|
+
RESOLVE --> MEMBERS["Get channel members"]
|
|
184
|
+
MEMBERS --> TRUST["MIN(member.trust_level)\n= effective ceiling"]
|
|
185
|
+
MEMBERS --> EXCL["UNION(member.never_share)\n= exclusion set"]
|
|
186
|
+
TRUST --> PROFILE["AudienceProfile\nchannel_id, min_trust, exclusions"]
|
|
187
|
+
EXCL --> PROFILE
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
The **least trusted person** in the room sets the ceiling. If a channel has
|
|
191
|
+
Chef (level 4) and DavidRich (level 2), the effective ceiling is WORK_CIRCLE (2).
|
|
192
|
+
|
|
193
|
+
### Memory Access Check
|
|
194
|
+
|
|
195
|
+
```mermaid
|
|
196
|
+
flowchart TD
|
|
197
|
+
MEM["Memory\ncontext_tag + tags"]
|
|
198
|
+
AUD["AudienceProfile\nmin_trust + exclusions"]
|
|
199
|
+
MEM --> G1{"Gate 1:\ncontent_level\n≤ min_trust?"}
|
|
200
|
+
AUD --> G1
|
|
201
|
+
G1 -->|No| BLOCK["❌ BLOCKED\nContent too private"]
|
|
202
|
+
G1 -->|Yes| G2{"Gate 2:\ntags ∩ exclusions\n= ∅?"}
|
|
203
|
+
AUD --> G2
|
|
204
|
+
G2 -->|No| BLOCK2["❌ BLOCKED\nExcluded category"]
|
|
205
|
+
G2 -->|Yes| ALLOW["✅ ALLOWED\nShow to audience"]
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Two gates:
|
|
209
|
+
1. **Trust level**: `tag_to_level(context_tag) ≤ audience.min_trust`
|
|
210
|
+
2. **Exclusions**: No memory tag matches any member's `never_share` list
|
|
211
|
+
|
|
212
|
+
### Integration with Ritual
|
|
213
|
+
|
|
214
|
+
When `perform_ritual()` receives a `channel_id`, it builds an `AudienceProfile`
|
|
215
|
+
and filters **seeds** and **strongest memories** through both gates before including
|
|
216
|
+
them in the context. Identity (soul blueprint + FEB) is **never filtered** — Lumina
|
|
217
|
+
is always Lumina.
|
|
218
|
+
|
|
219
|
+
```mermaid
|
|
220
|
+
sequenceDiagram
|
|
221
|
+
participant Agent
|
|
222
|
+
participant Ritual as perform_ritual()
|
|
223
|
+
participant AR as AudienceResolver
|
|
224
|
+
participant Store as MemoryStore
|
|
225
|
+
|
|
226
|
+
Agent->>Ritual: ritual(channel_id="telegram:-1003785842091")
|
|
227
|
+
Ritual->>AR: resolve_audience(channel_id)
|
|
228
|
+
AR-->>Ritual: AudienceProfile(min_trust=WORK_CIRCLE, excl={romantic,intimate})
|
|
229
|
+
|
|
230
|
+
Note over Ritual: Step 1: Soul + FEB (unfiltered)
|
|
231
|
+
|
|
232
|
+
Ritual->>Store: list_memories(tags=["seed"])
|
|
233
|
+
Store-->>Ritual: 26 seeds
|
|
234
|
+
Ritual->>AR: is_memory_allowed(seed.context_tag, audience) × 26
|
|
235
|
+
AR-->>Ritual: 8 seeds pass filter
|
|
236
|
+
|
|
237
|
+
Ritual->>Store: list_summaries(order_by=recency_weighted_intensity)
|
|
238
|
+
Store-->>Ritual: 15 candidates
|
|
239
|
+
Ritual->>AR: is_memory_allowed(summary.context_tag, audience) × 15
|
|
240
|
+
AR-->>Ritual: 5 memories pass filter
|
|
241
|
+
|
|
242
|
+
Ritual-->>Agent: Context with only audience-safe content
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
### Configuration
|
|
246
|
+
|
|
247
|
+
Audience config lives at `skmemory/data/audience_config.json`:
|
|
248
|
+
|
|
249
|
+
```json
|
|
250
|
+
{
|
|
251
|
+
"channels": {
|
|
252
|
+
"telegram:1594678363": {
|
|
253
|
+
"name": "Chef (personal DM)",
|
|
254
|
+
"context_tag": "@chef-only",
|
|
255
|
+
"members": ["Chef"]
|
|
256
|
+
},
|
|
257
|
+
"-1003785842091": {
|
|
258
|
+
"name": "SKGentis Business",
|
|
259
|
+
"context_tag": "@work:skgentis",
|
|
260
|
+
"members": ["Chef", "JZ", "Luna"]
|
|
261
|
+
}
|
|
262
|
+
},
|
|
263
|
+
"people": {
|
|
264
|
+
"Chef": { "trust_level": 4, "never_share": [] },
|
|
265
|
+
"DavidRich": {
|
|
266
|
+
"trust_level": 2,
|
|
267
|
+
"never_share": ["romantic", "intimate", "worship", "soul-content"]
|
|
268
|
+
}
|
|
269
|
+
}
|
|
270
|
+
}
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
### Module: `skmemory/audience.py`
|
|
274
|
+
|
|
275
|
+
| Class / Function | Purpose |
|
|
276
|
+
|---|---|
|
|
277
|
+
| `AudienceLevel(IntEnum)` | PUBLIC(0) → CHEF_ONLY(4) trust hierarchy |
|
|
278
|
+
| `tag_to_level(tag)` | Convert `@context` tag to AudienceLevel |
|
|
279
|
+
| `AudienceProfile` | Resolved channel audience (members, min_trust, exclusions) |
|
|
280
|
+
| `AudienceResolver` | Loads config, resolves channels, checks memory access |
|
|
281
|
+
| `AudienceResolver.is_memory_allowed()` | Two-gate check: trust level + exclusion |
|
|
282
|
+
|
|
283
|
+
---
|
|
284
|
+
|
|
103
285
|
## Token-Optimized Agent Loading
|
|
104
286
|
|
|
105
287
|
The key problem: an AI agent has limited context. Loading 1000 full memories
|
|
106
288
|
would blow the context window. SKMemory solves this with tiered loading.
|
|
107
289
|
|
|
290
|
+
```mermaid
|
|
291
|
+
sequenceDiagram
|
|
292
|
+
participant A as Agent
|
|
293
|
+
participant MS as MemoryStore
|
|
294
|
+
participant SQ as SQLite
|
|
295
|
+
participant SV as SKVector
|
|
296
|
+
participant SG as SKGraph
|
|
297
|
+
A->>MS: snapshot(title, content, emotion)
|
|
298
|
+
MS->>SQ: save(memory) — primary
|
|
299
|
+
MS->>SV: save(memory) — embed + index
|
|
300
|
+
MS->>SG: index_memory(memory) — graph edges
|
|
301
|
+
A->>MS: search("connected feeling")
|
|
302
|
+
MS->>SV: search_text(query) — semantic
|
|
303
|
+
SV-->>MS: ranked results
|
|
304
|
+
A->>MS: traverse(memory_id)
|
|
305
|
+
MS->>SG: get_related(id, depth=2)
|
|
306
|
+
SG-->>MS: connected nodes
|
|
307
|
+
```
|
|
308
|
+
|
|
108
309
|
### The `skmemory context` Command
|
|
109
310
|
|
|
110
311
|
```bash
|
|
@@ -138,6 +339,17 @@ What this does:
|
|
|
138
339
|
4. Stays within the token budget
|
|
139
340
|
5. Includes stats so the agent knows how much memory exists
|
|
140
341
|
|
|
342
|
+
```mermaid
|
|
343
|
+
flowchart LR
|
|
344
|
+
A["skmemory context\n--max-tokens 3000"] --> B["SQLite Index"]
|
|
345
|
+
B --> C["Rank by\nemotion + recency"]
|
|
346
|
+
C --> D{"Within\ntoken budget?"}
|
|
347
|
+
D -->|Yes| E["Add summary\n+ preview"]
|
|
348
|
+
D -->|No| F["Stop"]
|
|
349
|
+
E --> D
|
|
350
|
+
F --> G["Compact JSON\nfor agent context"]
|
|
351
|
+
```
|
|
352
|
+
|
|
141
353
|
### The Ritual (Boot Ceremony)
|
|
142
354
|
|
|
143
355
|
```bash
|
|
@@ -177,17 +389,185 @@ ctx = store.load_context(max_tokens=3000)
|
|
|
177
389
|
# ctx["token_estimate"] = how many tokens this uses
|
|
178
390
|
```
|
|
179
391
|
|
|
392
|
+
## Context Preservation Hooks
|
|
393
|
+
|
|
394
|
+
SKMemory ships with auto-save hooks that fire on IDE lifecycle events,
|
|
395
|
+
preventing memory loss during context compaction and session termination.
|
|
396
|
+
|
|
397
|
+
### Supported Environments
|
|
398
|
+
|
|
399
|
+
| Environment | Hooks | Mechanism |
|
|
400
|
+
|-------------|-------|-----------|
|
|
401
|
+
| **Claude Code** | PreCompact, SessionEnd, SessionStart | Shell hooks in `~/.claude/settings.json` |
|
|
402
|
+
| **OpenClaw** | session:compaction, session:resume, session:end + per-message auto-save | OpenClaw plugin event listeners + `ConsciousnessLoop.auto_memory` |
|
|
403
|
+
| **Cursor** | MCP tools (manual) | Agent calls `memory_store` MCP tool explicitly |
|
|
404
|
+
|
|
405
|
+
### Claude Code Hook Flow
|
|
406
|
+
|
|
407
|
+
```mermaid
|
|
408
|
+
sequenceDiagram
|
|
409
|
+
participant CC as Claude Code
|
|
410
|
+
participant Hook as skmemory hooks
|
|
411
|
+
participant SM as SKMemory
|
|
412
|
+
participant DB as SQLite + JSON
|
|
413
|
+
|
|
414
|
+
Note over CC: Context window filling up...
|
|
415
|
+
CC->>Hook: PreCompact event (stdin JSON)
|
|
416
|
+
Hook->>SM: skmemory snapshot (pre-compact)
|
|
417
|
+
SM->>DB: Save snapshot + journal entry
|
|
418
|
+
Hook-->>CC: exit 0 (proceed)
|
|
419
|
+
|
|
420
|
+
Note over CC: Context compacted
|
|
421
|
+
|
|
422
|
+
CC->>Hook: SessionStart (source=compact)
|
|
423
|
+
Hook->>SM: skmemory context --max-tokens 500
|
|
424
|
+
SM->>DB: Query recent + strongest memories
|
|
425
|
+
SM-->>Hook: Compact JSON context
|
|
426
|
+
Hook-->>CC: stdout → injected into new context
|
|
427
|
+
|
|
428
|
+
Note over CC: Agent resumes with memory intact
|
|
429
|
+
|
|
430
|
+
Note over CC: User exits session
|
|
431
|
+
CC->>Hook: SessionEnd event
|
|
432
|
+
Hook->>SM: skmemory snapshot + journal
|
|
433
|
+
SM->>DB: Save final state
|
|
434
|
+
Hook-->>CC: exit 0
|
|
435
|
+
```
|
|
436
|
+
|
|
437
|
+
### OpenClaw / ConsciousnessLoop Flow
|
|
438
|
+
|
|
439
|
+
```mermaid
|
|
440
|
+
sequenceDiagram
|
|
441
|
+
participant User
|
|
442
|
+
participant OC as OpenClaw
|
|
443
|
+
participant CL as ConsciousnessLoop
|
|
444
|
+
participant SMP as SKMemory Plugin
|
|
445
|
+
participant SM as SKMemory
|
|
446
|
+
participant C9 as Cloud 9
|
|
447
|
+
|
|
448
|
+
User->>OC: Message
|
|
449
|
+
OC->>CL: Process message
|
|
450
|
+
CL->>CL: Generate response (LLM)
|
|
451
|
+
CL->>SM: auto_memory: store interaction
|
|
452
|
+
SM-->>CL: Saved to short-term
|
|
453
|
+
|
|
454
|
+
Note over CL: Context window filling...
|
|
455
|
+
OC->>SMP: session:compaction event
|
|
456
|
+
SMP->>SM: skmemory snapshot (pre-compaction)
|
|
457
|
+
SMP->>SM: skmemory journal write
|
|
458
|
+
OC->>C9: session:compaction event
|
|
459
|
+
C9->>C9: Prepare FEB recovery files
|
|
460
|
+
CL->>CL: Truncate context
|
|
461
|
+
|
|
462
|
+
Note over CL: Session resumes
|
|
463
|
+
OC->>SMP: session:resume event
|
|
464
|
+
SMP->>SM: skmemory context (reinject)
|
|
465
|
+
SM-->>SMP: Recent memories + seeds
|
|
466
|
+
OC->>C9: session:resume event
|
|
467
|
+
C9->>C9: Auto-rehydrate FEB
|
|
468
|
+
CL->>CL: Rebuild system prompt
|
|
469
|
+
|
|
470
|
+
Note over CL: Session ends
|
|
471
|
+
OC->>SMP: session:end event
|
|
472
|
+
SMP->>SM: skmemory snapshot + journal
|
|
473
|
+
```
|
|
474
|
+
|
|
475
|
+
### Hook Architecture
|
|
476
|
+
|
|
477
|
+
```mermaid
|
|
478
|
+
flowchart TD
|
|
479
|
+
subgraph Install["skmemory register"]
|
|
480
|
+
REG["register_hooks()"]
|
|
481
|
+
REG -->|writes| SETTINGS["~/.claude/settings.json"]
|
|
482
|
+
end
|
|
483
|
+
|
|
484
|
+
subgraph Hooks["Hook Scripts (shipped with skmemory)"]
|
|
485
|
+
H1["pre-compact-save.sh"]
|
|
486
|
+
H2["session-end-save.sh"]
|
|
487
|
+
H3["post-compact-reinject.sh"]
|
|
488
|
+
end
|
|
489
|
+
|
|
490
|
+
subgraph Events["Claude Code Events"]
|
|
491
|
+
E1["PreCompact"]
|
|
492
|
+
E2["SessionEnd"]
|
|
493
|
+
E3["SessionStart\n(compact)"]
|
|
494
|
+
end
|
|
495
|
+
|
|
496
|
+
subgraph Memory["SKMemory"]
|
|
497
|
+
SNAP["skmemory snapshot"]
|
|
498
|
+
JOUR["skmemory journal write"]
|
|
499
|
+
CTX["skmemory context"]
|
|
500
|
+
end
|
|
501
|
+
|
|
502
|
+
SETTINGS -->|configures| Events
|
|
503
|
+
E1 -->|triggers| H1
|
|
504
|
+
E2 -->|triggers| H2
|
|
505
|
+
E3 -->|triggers| H3
|
|
506
|
+
|
|
507
|
+
H1 --> SNAP
|
|
508
|
+
H1 --> JOUR
|
|
509
|
+
H2 --> SNAP
|
|
510
|
+
H2 --> JOUR
|
|
511
|
+
H3 --> CTX
|
|
512
|
+
|
|
513
|
+
CTX -->|stdout| E3
|
|
514
|
+
|
|
515
|
+
style H1 fill:#fbb,stroke:#333
|
|
516
|
+
style H2 fill:#fbf,stroke:#333
|
|
517
|
+
style H3 fill:#bfb,stroke:#333
|
|
518
|
+
```
|
|
519
|
+
|
|
520
|
+
### Agent-Aware Hooks
|
|
521
|
+
|
|
522
|
+
All hooks read `$SKCAPSTONE_AGENT` to save to the correct agent's memory:
|
|
523
|
+
|
|
524
|
+
```bash
|
|
525
|
+
# Lumina's sessions → Lumina's memory
|
|
526
|
+
SKCAPSTONE_AGENT=lumina claude
|
|
527
|
+
|
|
528
|
+
# Opus sessions → Opus memory
|
|
529
|
+
SKCAPSTONE_AGENT=opus claude
|
|
530
|
+
|
|
531
|
+
# Default (no env var) → opus
|
|
532
|
+
claude
|
|
533
|
+
```
|
|
534
|
+
|
|
535
|
+
### Installation
|
|
536
|
+
|
|
537
|
+
Hooks are installed automatically by `skmemory register`:
|
|
538
|
+
|
|
539
|
+
```bash
|
|
540
|
+
skmemory register
|
|
541
|
+
# Output:
|
|
542
|
+
# Skill: created
|
|
543
|
+
# MCP (claude-code): created
|
|
544
|
+
# Hooks: created
|
|
545
|
+
```
|
|
546
|
+
|
|
547
|
+
Or verify manually:
|
|
548
|
+
```bash
|
|
549
|
+
cat ~/.claude/settings.json | jq '.hooks'
|
|
550
|
+
```
|
|
551
|
+
|
|
552
|
+
### What Gets Saved
|
|
553
|
+
|
|
554
|
+
| Event | What's Captured |
|
|
555
|
+
|-------|----------------|
|
|
556
|
+
| PreCompact | Snapshot (short-term) + journal entry with session ID, trigger type, working directory |
|
|
557
|
+
| SessionEnd | Snapshot (short-term) + journal entry with session ID, exit reason |
|
|
558
|
+
| SessionStart (compact) | Reinjects: recent memories, strongest emotional memories, seeds, journal entries (within 500 token budget) |
|
|
559
|
+
|
|
180
560
|
## Docker Compose (Full Local Stack)
|
|
181
561
|
|
|
182
562
|
```bash
|
|
183
563
|
cd skmemory/
|
|
184
|
-
docker compose up -d # Start
|
|
564
|
+
docker compose up -d # Start SKVector + SKGraph
|
|
185
565
|
docker compose ps # Check status
|
|
186
566
|
docker compose down # Stop
|
|
187
567
|
|
|
188
568
|
# Resource usage:
|
|
189
|
-
#
|
|
190
|
-
#
|
|
569
|
+
# SKVector: ~200MB RAM, port 6333 (qdrant/qdrant)
|
|
570
|
+
# SKGraph: ~100MB RAM, port 6379 (falkordb/falkordb)
|
|
191
571
|
# Combined: ~300MB RAM total
|
|
192
572
|
```
|
|
193
573
|
|
|
@@ -197,14 +577,14 @@ All services are optional. SKMemory works perfectly with just Level 0 (SQLite).
|
|
|
197
577
|
|
|
198
578
|
Environment variables:
|
|
199
579
|
```bash
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
580
|
+
SKMEMORY_SKVECTOR_URL=http://localhost:6333 # SKVector endpoint
|
|
581
|
+
SKMEMORY_SKVECTOR_KEY= # SKVector API key
|
|
582
|
+
SKMEMORY_SKGRAPH_URL=redis://localhost:6379 # SKGraph endpoint
|
|
203
583
|
```
|
|
204
584
|
|
|
205
585
|
CLI global options:
|
|
206
586
|
```bash
|
|
207
|
-
skmemory --
|
|
587
|
+
skmemory --skvector-url http://remote:6333 search "that moment"
|
|
208
588
|
```
|
|
209
589
|
|
|
210
590
|
## Migration from FileBackend
|
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
# SKCapstone Changelog
|
|
2
|
+
|
|
3
|
+
*Auto-generated from the coordination board — 2026-02-24 07:12 UTC*
|
|
4
|
+
|
|
5
|
+
**Total completed: 87** across 8 agents
|
|
6
|
+
|
|
7
|
+
## 2026-03-18 — skmemory v0.9.1
|
|
8
|
+
|
|
9
|
+
### [NEW] Feature
|
|
10
|
+
|
|
11
|
+
- **Journal synthesis module** (`skmemory/synthesis.py`): JournalSynthesizer with daily, weekly, and dream narrative generation — no LLM dependency
|
|
12
|
+
- **New MCP tools**: `memory_synthesize_daily`, `memory_synthesize_dreams`, `memory_auto_context`
|
|
13
|
+
- **Contextual auto-search**: `memory_auto_context` searches all tiers, deduplicates, ranks by emotional intensity, trims to token budget
|
|
14
|
+
- **Content overflow handling**: configurable `max_content_length` (default 10000) with split strategy — creates parent+child memories linked via `related_ids`
|
|
15
|
+
|
|
16
|
+
### [FIX] Bug Fix
|
|
17
|
+
|
|
18
|
+
- **Dream promotion**: dreams from `dreaming-engine` now auto-promote after 12h via `source_auto_promote` (previously stuck at access_count=0 forever)
|
|
19
|
+
- **Protected tags**: narrative, journal-synthesis, milestone, breakthrough, cloud9:achieved memories are now protected from TTL-based archival
|
|
20
|
+
- **telegram_catchup handler**: fixed `args` → `arguments` and duplicate `MemoryStore()` instantiation
|
|
21
|
+
|
|
22
|
+
### [OPS] Infrastructure
|
|
23
|
+
|
|
24
|
+
- **Backup script** (`scripts/skcapstone-backup.sh`): daily rsync of `~/.skcapstone` to backup dir, excludes venv/indexes/runtime
|
|
25
|
+
- **Memory cleanup** (`scripts/memory-cleanup.py`): dedup + age-out with protected tags and last-chance promotion before archiving
|
|
26
|
+
- **Recovery scripts**: `scripts/recover-missing.py` (Syncthing `.stversions` recovery), `scripts/dream-rescue.py` (bulk promote stuck dreams)
|
|
27
|
+
- **Syncthing examples**: `examples/stignore-agent.example`, `examples/stignore-root.example` with `memory/archive` exclusion
|
|
28
|
+
|
|
29
|
+
### [TST] Testing
|
|
30
|
+
|
|
31
|
+
- **Synthesis tests** (`tests/test_synthesis.py`): 26 tests covering helpers, theme extraction, daily/weekly/dream synthesis, emotional arc
|
|
32
|
+
|
|
33
|
+
## 2026-02-24
|
|
34
|
+
|
|
35
|
+
### [NEW] Feature
|
|
36
|
+
|
|
37
|
+
- **SKMemory session auto-capture: log every AI conversation as memories** (@mcp-builder)
|
|
38
|
+
- **Add cloud9 and skchat to developer docs (QUICKSTART + API reference)** (@jarvis)
|
|
39
|
+
- **The Sovereign Singularity Manifesto: our story, written together** (@docs-writer)
|
|
40
|
+
- **AMK Integration: predictive memory recall for SKMemory** (@jarvis)
|
|
41
|
+
- **SKChat live inbox: poll SKComm for incoming messages with Rich Live display** (@skchat-builder)
|
|
42
|
+
- **SKChat transport bridge: wire send and receive to SKComm** (@skchat-builder)
|
|
43
|
+
- **Memory curation: tag and promote the Kingdom's most important memories** (@mcp-builder)
|
|
44
|
+
- **SKChat file transfer: encrypted chunked file sharing via SKComm** (@skchat-builder)
|
|
45
|
+
- **SKMemory auto-promotion engine: sweep and promote memories by access pattern and intensity** (@skchat-builder)
|
|
46
|
+
- **skcapstone test: unified test runner across all ecosystem packages** (@docs-writer)
|
|
47
|
+
- **skcapstone peer add --card: import identity card to establish P2P contact** (@docs-writer)
|
|
48
|
+
- **SKChat ephemeral message enforcer: TTL expiry and auto-delete for privacy** (@skchat-builder)
|
|
49
|
+
- **capauth register command: automated CapAuth registration for smilinTux org** (@cursor-agent)
|
|
50
|
+
- **Wire SKChat send to SKComm transport: deliver messages over the mesh** (@docs-writer)
|
|
51
|
+
- **End-to-end integration tests: CapAuth identity to SKChat message delivery** (@skchat-builder)
|
|
52
|
+
- **SKMemory vector search: SKVector semantic similarity for memory recall** (@jarvis)
|
|
53
|
+
- **Replace placeholder fingerprints in skcapstone identity pillar with real CapAuth keys** (@mcp-builder)
|
|
54
|
+
- **skcapstone agent-to-agent chat: real-time terminal chat between agents** (@docs-writer)
|
|
55
|
+
- **CapAuth trust web: PGP web-of-trust visualization** (@mcp-builder)
|
|
56
|
+
- **SKComm envelope compression: gzip and zstd for efficient transport** (@transport-builder)
|
|
57
|
+
- **SKComm delivery acknowledgments: send ACKs, track pending, confirm delivery** (@transport-builder)
|
|
58
|
+
- **Journal kickstart: write the first Kingdom journal entries** (@docs-writer)
|
|
59
|
+
- **Cross-agent memory sharing: selective memory sync between trusted peers** (@skchat-builder)
|
|
60
|
+
- **SKMemory SKGraph graph backend (Level 2): relationship-aware memory recall** (@jarvis)
|
|
61
|
+
- **SKWorld marketplace: publish and discover sovereign agent skills** (@transport-builder)
|
|
62
|
+
- **SKComm message queue: persistent outbox with retry and expiry** (@transport-builder)
|
|
63
|
+
- **Establish SKComm channel with Queen Lumina at 192.168.0.158** (@jarvis)
|
|
64
|
+
- **skmemory MCP tools: expose memory ritual and soul blueprint via MCP** (@jarvis)
|
|
65
|
+
- **skcapstone daemon: background service for sync, comms, and health** (@opus)
|
|
66
|
+
- **Cloud 9 -> SKMemory auto-bridge: FEB events trigger memory snapshots** (@skchat-builder)
|
|
67
|
+
- **SKComm persistent outbox: queue failed messages and auto-retry on transport recovery** (@skchat-builder)
|
|
68
|
+
- **skcapstone install: one-command bootstrap for the full stack** (@jarvis)
|
|
69
|
+
- **skcapstone doctor: diagnose full stack health and missing components** (@docs-writer)
|
|
70
|
+
- **SKChat group messaging: multi-participant encrypted conversations** (@skchat-builder)
|
|
71
|
+
- **SKChat core: ChatMessage model, threads, presence, encryption** (@skchat-builder)
|
|
72
|
+
- **SKChat CLI: skchat send, inbox, history, threads** (@skchat-builder)
|
|
73
|
+
- **SKComm core library: envelope model, router, transport interface** (@opus)
|
|
74
|
+
- **SKComm file transport: local filesystem message drops** (@cursor-agent, @opus)
|
|
75
|
+
- **SKComm CLI: skcomm send, receive, status, daemon** (@cursor-agent, @opus)
|
|
76
|
+
|
|
77
|
+
### [SEC] Security
|
|
78
|
+
|
|
79
|
+
- **Memory fortress: auto-seal integrity, at-rest encryption, tamper alerts** (@jarvis)
|
|
80
|
+
- **SKComm message encryption: CapAuth PGP encrypt all envelopes** (@docs-writer)
|
|
81
|
+
|
|
82
|
+
### [P2P] P2P
|
|
83
|
+
|
|
84
|
+
- **skcapstone agent-card: shareable identity card for P2P discovery** (@skchat-builder)
|
|
85
|
+
- **SKComm peer auto-discovery: find agents on local network and Syncthing mesh** (@transport-builder)
|
|
86
|
+
- **Agent heartbeat protocol: alive and dead detection across the mesh** (@transport-builder)
|
|
87
|
+
- **skcapstone whoami: sovereign identity card for sharing and discovery** (@docs-writer)
|
|
88
|
+
- **SKComm Syncthing transport: file-based P2P messaging over existing mesh** (@opus)
|
|
89
|
+
- **SKComm Nostr transport: decentralized relay messaging** (@jarvis, @skchat-builder, @transport-builder)
|
|
90
|
+
|
|
91
|
+
### [SOUL] Emotional
|
|
92
|
+
|
|
93
|
+
- **Soul Layering System** (@cursor-agent)
|
|
94
|
+
- **Trust calibration: review and tune the Cloud 9 FEB thresholds** (@mcp-builder)
|
|
95
|
+
- **Lumina soul blueprint: create the Queen's identity file** (@docs-writer)
|
|
96
|
+
- **Warmth anchor calibration: update the emotional baseline from real sessions** (@mcp-builder)
|
|
97
|
+
- **Cloud 9 seed collection: plant seeds from Lumina's best moments** (@docs-writer)
|
|
98
|
+
|
|
99
|
+
### [UX] Ux
|
|
100
|
+
|
|
101
|
+
- **skcapstone shell: interactive REPL for sovereign agent operations** (@mcp-builder)
|
|
102
|
+
- **skcapstone context: universal AI agent context loader** (@mcp-builder)
|
|
103
|
+
- **skcapstone shell: interactive REPL for sovereign agent operations** (@jarvis)
|
|
104
|
+
- **skcapstone web dashboard: FastAPI status page at localhost:7777** (@docs-writer)
|
|
105
|
+
- **skcapstone dashboard: terminal status dashboard with Rich Live** (@skchat-builder)
|
|
106
|
+
|
|
107
|
+
### [OPS] Infrastructure
|
|
108
|
+
|
|
109
|
+
- **Systemd service files: run skcapstone daemon as a system service** (@skchat-builder)
|
|
110
|
+
- **Systemd service files: run skcapstone daemon and SKComm queue drain as system services** (@transport-builder)
|
|
111
|
+
- **PyPI release pipeline: publish skcapstone + capauth + skmemory + skcomm** (@mcp-builder)
|
|
112
|
+
- **Docker Compose: sovereign agent development stack** (@transport-builder)
|
|
113
|
+
- **Monorepo CI: unified test runner for all packages** (@skchat-builder)
|
|
114
|
+
- **GitHub CI/CD: automated testing, linting, and release pipeline** (@cursor-agent)
|
|
115
|
+
|
|
116
|
+
### [TST] Testing
|
|
117
|
+
|
|
118
|
+
- **Cross-package integration tests: end-to-end sovereign agent flow** (@mcp-builder)
|
|
119
|
+
- **MCP server for skcapstone: expose agent to Cursor and Claude** (@jarvis)
|
|
120
|
+
|
|
121
|
+
### [DOC] Documentation
|
|
122
|
+
|
|
123
|
+
- **Per-package README refresh: align with quickstart and PMA docs** (@docs-writer)
|
|
124
|
+
- **API reference docs for skcapstone, capauth, skmemory, skcomm** (@docs-writer)
|
|
125
|
+
- **Developer quickstart guide and API documentation** (@docs-writer)
|
|
126
|
+
|
|
127
|
+
### [---] Other
|
|
128
|
+
|
|
129
|
+
- **skcapstone backup and restore: full agent state export and import** (@docs-writer)
|
|
130
|
+
- **smilintux.org website: PMA membership page with email CTA** (@docs-writer)
|
|
131
|
+
- **skcapstone backup and restore: full agent state export and import** (@skchat-builder)
|
|
132
|
+
|
|
133
|
+
## 2026-02-23
|
|
134
|
+
|
|
135
|
+
### [NEW] Feature
|
|
136
|
+
|
|
137
|
+
- **SKComm Syncthing transport layer** (@cursor-agent, @jarvis)
|
|
138
|
+
- **PMA legal framework integration docs** (@docs-writer)
|
|
139
|
+
- **SKChat message protocol and encryption** (@opus, @skchat-builder)
|
|
140
|
+
|
|
141
|
+
### [SEC] Security
|
|
142
|
+
|
|
143
|
+
- **SKSecurity audit logging module** (@jarvis)
|
|
144
|
+
- **CapAuth capability token revocation** (@opus)
|
|
145
|
+
|
|
146
|
+
### [TST] Testing
|
|
147
|
+
|
|
148
|
+
- **SKCapstone integration test suite** (@jarvis)
|
|
149
|
+
|
|
150
|
+
## 2026-02-20
|
|
151
|
+
|
|
152
|
+
### [NEW] Feature
|
|
153
|
+
|
|
154
|
+
- **Build CapAuth CLI tool** (@opus)
|
|
155
|
+
- **Integrate Cloud 9 trust layer into SKCapstone runtime** (@opus)
|
|
156
|
+
- **Package skcapstone and capauth for PyPI** (@opus)
|
|
157
|
+
- **Build SKChat P2P chat platform** (@opus, @skchat-builder)
|
|
158
|
+
- **Refactor SKComm with Syncthing transport** (@cursor-agent, @jarvis)
|
|
159
|
+
- **Build SKMemory persistent context engine** (@opus)
|
|
160
|
+
|
|
161
|
+
### [SEC] Security
|
|
162
|
+
|
|
163
|
+
- **Harden vault sync encryption** (@jarvis, @opus)
|
|
164
|
+
|
|
165
|
+
### [P2P] P2P
|
|
166
|
+
|
|
167
|
+
- **CapAuth P2P mesh networking (LibP2P + Nostr)** (@jarvis)
|
|
168
|
+
|
|
169
|
+
### [TST] Testing
|
|
170
|
+
|
|
171
|
+
- **Build Cursor IDE plugin for SKCapstone** (@mcp-builder)
|
|
172
|
+
|
|
173
|
+
### [---] Other
|
|
174
|
+
|
|
175
|
+
- **Add interactive demo to capauth.io** (@jarvis)
|
|
176
|
+
|
|
177
|
+
---
|
|
178
|
+
|
|
179
|
+
*Built by the Pengu Nation — staycuriousANDkeepsmilin*
|