@parallelclaw/memex-openclaw 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +135 -0
- package/index.js +202 -0
- package/lib/conv_id.js +0 -0
- package/lib/corpus_supplement.js +131 -0
- package/lib/store.js +227 -0
- package/openclaw.plugin.json +17 -0
- package/package.json +66 -0
package/README.md
ADDED
|
@@ -0,0 +1,135 @@
|
|
|
1
|
+
# memex-openclaw
|
|
2
|
+
|
|
3
|
+
**OpenClaw plugin that captures every turn verbatim into the [memex](https://memex.parallelclaw.ai) unified SQLite corpus.**
|
|
4
|
+
|
|
5
|
+
Replaces the v0.11.x `memex-sync` file-watcher daemon. Captures via OpenClaw's native plugin lifecycle hooks — no file watching, no JSON parsing, no daemon to manage.
|
|
6
|
+
|
|
7
|
+
> [!IMPORTANT]
|
|
8
|
+
> **memex-openclaw is a bridge plugin, not a memory replacement.** It's most useful when you ALSO run other clients (Claude Code / Hermes / Cursor / Telegram exports) captured into the same [memex-mvp](https://www.npmjs.com/package/memex-mvp) corpus. If you only use OpenClaw with built-in memory-core / Memoria / Mem0 — that's already a complete memory stack; memex-openclaw mostly earns its place when you want **unified search across multiple AI clients**.
|
|
9
|
+
|
|
10
|
+
## What it does
|
|
11
|
+
|
|
12
|
+
Three lifecycle hooks + one corpus supplement:
|
|
13
|
+
|
|
14
|
+
| Hook | What we do |
|
|
15
|
+
|---|---|
|
|
16
|
+
| `agent_end` | Insert the just-completed turn's user + assistant messages into memex.db, verbatim. Channel comes from `ctx.messageProvider` — no parsing. |
|
|
17
|
+
| `before_compaction` | Save messages that are about to be dropped from active context. They become searchable from memex even after OpenClaw forgets them. |
|
|
18
|
+
| `session_end` | Update conversation last_ts. Safety-net marker. |
|
|
19
|
+
| `registerMemoryCorpusSupplement` | Memex contents are surfaced through OpenClaw's built-in `memory_search` / `memory_get` tools — the model sees memex rows alongside workspace memory in one search result. |
|
|
20
|
+
|
|
21
|
+
Storage: `~/.memex/data/memex.db` (override via plugin config `dbPath`). Same SQLite schema as memex-mvp (npm) and memex-hermes (pip) — all three can write to the same DB concurrently.
|
|
22
|
+
|
|
23
|
+
## Install
|
|
24
|
+
|
|
25
|
+
Once published to npm:
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
openclaw plugins install @parallelclaw/memex-openclaw
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
For development / local install:
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
git clone https://github.com/parallelclaw/memex-mvp.git
|
|
35
|
+
cd memex-mvp/plugins/memex-openclaw
|
|
36
|
+
npm install
|
|
37
|
+
openclaw plugins install --link "$(pwd)"
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Enable in `~/.openclaw/openclaw.json`:
|
|
41
|
+
|
|
42
|
+
```json
|
|
43
|
+
{
|
|
44
|
+
"plugins": {
|
|
45
|
+
"entries": {
|
|
46
|
+
"memex-openclaw": {
|
|
47
|
+
"enabled": true
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
}
|
|
51
|
+
}
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Optional config (default db path is `~/.memex/data/memex.db`):
|
|
55
|
+
|
|
56
|
+
```json
|
|
57
|
+
{
|
|
58
|
+
"plugins": {
|
|
59
|
+
"entries": {
|
|
60
|
+
"memex-openclaw": {
|
|
61
|
+
"enabled": true,
|
|
62
|
+
"config": {
|
|
63
|
+
"dbPath": "/some/other/path/memex.db"
|
|
64
|
+
}
|
|
65
|
+
}
|
|
66
|
+
}
|
|
67
|
+
}
|
|
68
|
+
}
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Restart OpenClaw:
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
openclaw gateway restart
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Conversation routing
|
|
78
|
+
|
|
79
|
+
| OpenClaw context | memex conversation_id |
|
|
80
|
+
|---|---|
|
|
81
|
+
| `messageProvider="telegram", channelId="97592799"` | `openclaw-telegram-97592799` |
|
|
82
|
+
| `messageProvider="discord", channelId="..."` | `openclaw-discord-<channelId>` |
|
|
83
|
+
| `messageProvider="cli"` (no channelId) | `openclaw-cli-<session8>` |
|
|
84
|
+
| `messageProvider="cron"` | `openclaw-cron-<session8>` |
|
|
85
|
+
|
|
86
|
+
Per-user threading: same Telegram chat across multiple OpenClaw sessions ends up in **one** memex conversation, just like memex-hermes does with Hermes.
|
|
87
|
+
|
|
88
|
+
## Verify
|
|
89
|
+
|
|
90
|
+
```bash
|
|
91
|
+
# Hermes / OpenClaw / memex-mvp can all coexist — check OpenClaw rows:
|
|
92
|
+
sqlite3 ~/.memex/data/memex.db \
|
|
93
|
+
"SELECT COUNT(*), MIN(date(ts,'unixepoch')), MAX(date(ts,'unixepoch'))
|
|
94
|
+
FROM messages WHERE source='openclaw'"
|
|
95
|
+
|
|
96
|
+
# Or via memex-mvp CLI (npm i -g memex-mvp):
|
|
97
|
+
memex recent --source openclaw
|
|
98
|
+
memex search "your test query" --source openclaw
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
## What this plugin is NOT
|
|
102
|
+
|
|
103
|
+
- ❌ Not a replacement for OpenClaw's built-in memory (memory-core / Active Memory). It augments — registers as a corpus supplement.
|
|
104
|
+
- ❌ Not a fact extractor. Stores raw turns. Use Mem0 / Memoria alongside if you want extraction.
|
|
105
|
+
- ❌ Not a vector store (yet). FTS5 lexical search only.
|
|
106
|
+
- ❌ Not a single-client product. Earns its value when paired with memex-mvp + at least one other captured client.
|
|
107
|
+
|
|
108
|
+
## Testing
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
npm install
|
|
112
|
+
npm test # 40 tests, ~1 second
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
## Architecture vs old memex-sync daemon (v0.11.x)
|
|
116
|
+
|
|
117
|
+
| | Old (v0.11.x file-watcher) | New (this plugin) |
|
|
118
|
+
|---|---|---|
|
|
119
|
+
| Where it runs | Separate `memex-sync` Node daemon | Inside OpenClaw runtime |
|
|
120
|
+
| How it captures | Polls `~/.openclaw/agents/main/sessions/*.jsonl` | Subscribes to `agent_end` hook |
|
|
121
|
+
| Channel detection | Regex on message text + sessions.json parsing | `ctx.messageProvider` — no parsing |
|
|
122
|
+
| Knowledge of OpenClaw file format | Hardcoded (paths, naming, `.reset.`, `.checkpoint.`) | None |
|
|
123
|
+
| Lines of code | ~1000 (in memex-mvp lib/) | ~250 (this package) |
|
|
124
|
+
| What breaks when OpenClaw changes its file format | Everything | Nothing |
|
|
125
|
+
| What breaks when OpenClaw changes its plugin API | This plugin (one file to update) | — |
|
|
126
|
+
|
|
127
|
+
## License
|
|
128
|
+
|
|
129
|
+
MIT. See parent repository.
|
|
130
|
+
|
|
131
|
+
## Related
|
|
132
|
+
|
|
133
|
+
- [memex-mvp](https://www.npmjs.com/package/memex-mvp) — Node.js CLI + MCP server for the same `memex.db`. Install for search tools, dashboard, and capturing other clients (Claude Code, Cursor, Telegram).
|
|
134
|
+
- [memex-hermes](https://pypi.org/project/memex-hermes/) — Python plugin doing the same for [Hermes Agent](https://github.com/NousResearch/hermes-agent).
|
|
135
|
+
- [install-memex-claw](https://clawhub.ai/sedelev/install-memex-claw) — ClawHub skill that walks through the install.
|
package/index.js
ADDED
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* memex-openclaw — OpenClaw plugin that captures every turn verbatim
|
|
3
|
+
* into the memex unified SQLite corpus.
|
|
4
|
+
*
|
|
5
|
+
* Replaces the v0.11.x file-watcher daemon approach. This plugin:
|
|
6
|
+
* • Subscribes to `agent_end` for per-turn capture — no file watching
|
|
7
|
+
* • Subscribes to `before_compaction` to preserve messages before
|
|
8
|
+
* they're dropped from active context
|
|
9
|
+
* • Subscribes to `session_end` as a safety-net flush
|
|
10
|
+
* • Registers a `MemoryCorpusSupplement` so the model's built-in
|
|
11
|
+
* `memory_search` tool sees memex content alongside workspace memory
|
|
12
|
+
*
|
|
13
|
+
* Channel detection: zero parsing. OpenClaw 2026.5+ hands us
|
|
14
|
+
* `ctx.messageProvider` (e.g. "telegram") and `ctx.channelId` (e.g.
|
|
15
|
+
* "97592799") directly in the hook context.
|
|
16
|
+
*
|
|
17
|
+
* Storage: ~/.memex/data/memex.db (override via plugin config db_path).
|
|
18
|
+
* Schema parity with memex-mvp (npm) and memex-hermes (pip) — all three
|
|
19
|
+
* write to the same database with the same UNIQUE-constraint dedup.
|
|
20
|
+
*
|
|
21
|
+
* @see openclaw.plugin.json for manifest
|
|
22
|
+
* @see package.json for npm distribution
|
|
23
|
+
*/
|
|
24
|
+
|
|
25
|
+
import {
|
|
26
|
+
definePluginEntry,
|
|
27
|
+
registerMemoryCorpusSupplement,
|
|
28
|
+
} from 'openclaw/plugin-sdk/core';
|
|
29
|
+
|
|
30
|
+
import { MemexStore } from './lib/store.js';
|
|
31
|
+
import { deriveConvId, deriveMsgId, extractText } from './lib/conv_id.js';
|
|
32
|
+
import { buildCorpusSupplement } from './lib/corpus_supplement.js';
|
|
33
|
+
|
|
34
|
+
export default definePluginEntry({
|
|
35
|
+
id: 'memex-openclaw',
|
|
36
|
+
name: 'Memex',
|
|
37
|
+
description:
|
|
38
|
+
'Captures every OpenClaw turn verbatim into the memex unified SQLite corpus. ' +
|
|
39
|
+
'Pair with memex-mvp (npm) to search OpenClaw + Hermes + Claude Code + Telegram from one place.',
|
|
40
|
+
kind: 'memory',
|
|
41
|
+
|
|
42
|
+
register(api) {
|
|
43
|
+
const logger = api.logger;
|
|
44
|
+
const cfg = api.pluginConfig || {};
|
|
45
|
+
let store;
|
|
46
|
+
|
|
47
|
+
try {
|
|
48
|
+
store = new MemexStore(cfg.dbPath);
|
|
49
|
+
logger.info(`memex-openclaw: opened ${store.dbPath} (current rows: ${store.count()})`);
|
|
50
|
+
} catch (err) {
|
|
51
|
+
logger.error(`memex-openclaw: failed to open memex.db: ${err.message}`);
|
|
52
|
+
return; // can't operate without store
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
// -------------------------------------------------------------
|
|
56
|
+
// 1. Primary capture — every turn that completes successfully
|
|
57
|
+
// -------------------------------------------------------------
|
|
58
|
+
api.on('agent_end', async (event, ctx) => {
|
|
59
|
+
if (!event?.success) return; // failed turns are skipped (LLM error, etc.)
|
|
60
|
+
try {
|
|
61
|
+
const platform = ctx?.messageProvider || 'unknown';
|
|
62
|
+
const channelId = ctx?.channelId;
|
|
63
|
+
const sessionId = ctx?.sessionId;
|
|
64
|
+
const agentId = ctx?.agentId || 'main';
|
|
65
|
+
const convId = deriveConvId({ messageProvider: platform, channelId, sessionId });
|
|
66
|
+
|
|
67
|
+
// Capture only the LAST TURN's user + assistant messages —
|
|
68
|
+
// earlier history was captured by prior agent_end invocations.
|
|
69
|
+
// (OpenClaw passes full conversation history but most of it is
|
|
70
|
+
// already in memex.db from previous turns; UNIQUE dedup makes
|
|
71
|
+
// re-inserting harmless but wasteful.)
|
|
72
|
+
const messages = Array.isArray(event.messages) ? event.messages : [];
|
|
73
|
+
const lastTurn = messages.slice(-2);
|
|
74
|
+
|
|
75
|
+
const baseTs = Math.floor(Date.now() / 1000);
|
|
76
|
+
for (let i = 0; i < lastTurn.length; i++) {
|
|
77
|
+
const msg = lastTurn[i];
|
|
78
|
+
if (!msg || (msg.role !== 'user' && msg.role !== 'assistant')) continue;
|
|
79
|
+
const text = extractText(msg);
|
|
80
|
+
if (!text || !text.trim()) continue;
|
|
81
|
+
|
|
82
|
+
const msgId = deriveMsgId({ role: msg.role, text, convId });
|
|
83
|
+
store.insertMessage({
|
|
84
|
+
conversationId: convId,
|
|
85
|
+
msgId,
|
|
86
|
+
role: msg.role,
|
|
87
|
+
text,
|
|
88
|
+
ts: baseTs + i, // tiny offset to preserve order
|
|
89
|
+
channel: platform,
|
|
90
|
+
metadata: {
|
|
91
|
+
raw_type: 'openclaw-agent-end',
|
|
92
|
+
session_id: sessionId,
|
|
93
|
+
agent_id: agentId,
|
|
94
|
+
platform,
|
|
95
|
+
channel_id: channelId,
|
|
96
|
+
model_provider: ctx?.modelProviderId,
|
|
97
|
+
model_id: ctx?.modelId,
|
|
98
|
+
run_id: event.runId,
|
|
99
|
+
},
|
|
100
|
+
});
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
// Keep conversations.last_ts current.
|
|
104
|
+
store.upsertConversation({
|
|
105
|
+
conversationId: convId,
|
|
106
|
+
title: convId,
|
|
107
|
+
firstTs: baseTs,
|
|
108
|
+
lastTs: baseTs,
|
|
109
|
+
});
|
|
110
|
+
} catch (err) {
|
|
111
|
+
logger.error(`memex-openclaw: agent_end capture failed: ${err.message}`);
|
|
112
|
+
}
|
|
113
|
+
});
|
|
114
|
+
|
|
115
|
+
// -------------------------------------------------------------
|
|
116
|
+
// 2. Preserve messages before they're compacted out of context
|
|
117
|
+
// -------------------------------------------------------------
|
|
118
|
+
api.on('before_compaction', async (event, ctx) => {
|
|
119
|
+
try {
|
|
120
|
+
const messages = Array.isArray(event?.messages) ? event.messages : [];
|
|
121
|
+
if (messages.length === 0) return;
|
|
122
|
+
|
|
123
|
+
const platform = ctx?.messageProvider || 'unknown';
|
|
124
|
+
const channelId = ctx?.channelId;
|
|
125
|
+
const sessionId = ctx?.sessionId;
|
|
126
|
+
const convId = deriveConvId({ messageProvider: platform, channelId, sessionId });
|
|
127
|
+
|
|
128
|
+
const baseTs = Math.floor(Date.now() / 1000);
|
|
129
|
+
let saved = 0;
|
|
130
|
+
for (let i = 0; i < messages.length; i++) {
|
|
131
|
+
const msg = messages[i];
|
|
132
|
+
if (!msg || (msg.role !== 'user' && msg.role !== 'assistant')) continue;
|
|
133
|
+
const text = extractText(msg);
|
|
134
|
+
if (!text || !text.trim()) continue;
|
|
135
|
+
const wrote = store.insertMessage({
|
|
136
|
+
conversationId: convId,
|
|
137
|
+
msgId: deriveMsgId({ role: msg.role, text, convId }),
|
|
138
|
+
role: msg.role,
|
|
139
|
+
text,
|
|
140
|
+
ts: baseTs + i,
|
|
141
|
+
channel: platform,
|
|
142
|
+
metadata: {
|
|
143
|
+
raw_type: 'openclaw-pre-compaction',
|
|
144
|
+
session_id: sessionId,
|
|
145
|
+
platform,
|
|
146
|
+
channel_id: channelId,
|
|
147
|
+
},
|
|
148
|
+
});
|
|
149
|
+
if (wrote) saved++;
|
|
150
|
+
}
|
|
151
|
+
if (saved > 0) {
|
|
152
|
+
logger.info(
|
|
153
|
+
`memex-openclaw: preserved ${saved} message(s) before compaction (conv=${convId})`,
|
|
154
|
+
);
|
|
155
|
+
}
|
|
156
|
+
} catch (err) {
|
|
157
|
+
logger.error(`memex-openclaw: before_compaction failed: ${err.message}`);
|
|
158
|
+
}
|
|
159
|
+
});
|
|
160
|
+
|
|
161
|
+
// -------------------------------------------------------------
|
|
162
|
+
// 3. Session-end safety net — flush the full final history
|
|
163
|
+
// -------------------------------------------------------------
|
|
164
|
+
api.on('session_end', async (event, ctx) => {
|
|
165
|
+
try {
|
|
166
|
+
// session_end doesn't carry messages[] in OpenClaw 2026.5 —
|
|
167
|
+
// we have sessionId + sessionKey + reason. The hook serves as
|
|
168
|
+
// a marker that this conv is "done"; we update conv last_ts
|
|
169
|
+
// and let agent_end captures already-in-DB stand.
|
|
170
|
+
const platform = ctx?.messageProvider || 'unknown';
|
|
171
|
+
const channelId = ctx?.channelId;
|
|
172
|
+
const sessionId = event?.sessionId || ctx?.sessionId;
|
|
173
|
+
const convId = deriveConvId({ messageProvider: platform, channelId, sessionId });
|
|
174
|
+
store.upsertConversation({
|
|
175
|
+
conversationId: convId,
|
|
176
|
+
title: convId,
|
|
177
|
+
lastTs: Math.floor(Date.now() / 1000),
|
|
178
|
+
});
|
|
179
|
+
logger.debug(`memex-openclaw: session_end conv=${convId} reason=${event?.reason}`);
|
|
180
|
+
} catch (err) {
|
|
181
|
+
logger.error(`memex-openclaw: session_end failed: ${err.message}`);
|
|
182
|
+
}
|
|
183
|
+
});
|
|
184
|
+
|
|
185
|
+
// -------------------------------------------------------------
|
|
186
|
+
// 4. Expose memex contents to OpenClaw's built-in memory_search
|
|
187
|
+
// -------------------------------------------------------------
|
|
188
|
+
try {
|
|
189
|
+
const supplement = buildCorpusSupplement(store, logger);
|
|
190
|
+
registerMemoryCorpusSupplement('memex-openclaw', supplement);
|
|
191
|
+
logger.info('memex-openclaw: registered as memory corpus supplement');
|
|
192
|
+
} catch (err) {
|
|
193
|
+
// Non-fatal — capture still works without supplement registration.
|
|
194
|
+
logger.warn(
|
|
195
|
+
`memex-openclaw: could not register corpus supplement: ${err.message} ` +
|
|
196
|
+
'(capture still active; built-in memory_search will not surface memex rows)',
|
|
197
|
+
);
|
|
198
|
+
}
|
|
199
|
+
|
|
200
|
+
logger.info('memex-openclaw: plugin activated');
|
|
201
|
+
},
|
|
202
|
+
});
|
package/lib/conv_id.js
ADDED
|
Binary file
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* MemoryCorpusSupplement adapter — exposes memex contents to OpenClaw's
|
|
3
|
+
* built-in `memory_search` and `memory_get` tools.
|
|
4
|
+
*
|
|
5
|
+
* The OpenClaw runtime ships memory-core, which provides workspace-based
|
|
6
|
+
* memory (MEMORY.md / USER.md / memory/YYYY-MM-DD.md). Plugins can
|
|
7
|
+
* REGISTER A SUPPLEMENT — additional rows that show up in the same tool
|
|
8
|
+
* output, prefixed/labelled so the model knows which corpus a row came
|
|
9
|
+
* from.
|
|
10
|
+
*
|
|
11
|
+
* This is the strategic positioning: memex doesn't REPLACE memory-core,
|
|
12
|
+
* it ADDS to it. The model sees a single `memory_search` tool, gets
|
|
13
|
+
* results from BOTH built-in workspace memory AND memex's verbatim
|
|
14
|
+
* cross-client corpus, in one shot.
|
|
15
|
+
*
|
|
16
|
+
* Multi-plugin coexistence: Mem0, Memoria, memex can all register
|
|
17
|
+
* their own supplements. OpenClaw merges results.
|
|
18
|
+
*/
|
|
19
|
+
|
|
20
|
+
const CORPUS_LABEL = 'memex';
|
|
21
|
+
|
|
22
|
+
/**
|
|
23
|
+
* Build the supplement object that gets handed to OpenClaw's
|
|
24
|
+
* `registerMemoryCorpusSupplement(pluginId, supplement)` call.
|
|
25
|
+
*
|
|
26
|
+
* Contract (from OpenClaw 2026.5.4 plugin-sdk/memory-state.d.ts):
|
|
27
|
+
*
|
|
28
|
+
* supplement.search({ query, maxResults, agentSessionKey })
|
|
29
|
+
* → Promise<MemoryCorpusSearchResult[]>
|
|
30
|
+
*
|
|
31
|
+
* supplement.get({ lookup, fromLine, lineCount, agentSessionKey })
|
|
32
|
+
* → Promise<MemoryCorpusGetResult | null>
|
|
33
|
+
*/
|
|
34
|
+
export function buildCorpusSupplement(store, logger) {
|
|
35
|
+
return {
|
|
36
|
+
async search({ query, maxResults = 10 } = {}) {
|
|
37
|
+
try {
|
|
38
|
+
const rows = store.search(query || '', maxResults);
|
|
39
|
+
return rows.map((r) => toSearchResult(r));
|
|
40
|
+
} catch (err) {
|
|
41
|
+
logger?.warn?.(`memex-openclaw: corpus search failed: ${err.message}`);
|
|
42
|
+
return [];
|
|
43
|
+
}
|
|
44
|
+
},
|
|
45
|
+
|
|
46
|
+
async get({ lookup }) {
|
|
47
|
+
// `lookup` is the id we returned in search results' `id` field.
|
|
48
|
+
// We use the message row id as a string for portability.
|
|
49
|
+
try {
|
|
50
|
+
const numericId = parseInt(String(lookup).replace(/^memex:/, ''), 10);
|
|
51
|
+
if (!Number.isFinite(numericId)) return null;
|
|
52
|
+
const row = store.getById(numericId);
|
|
53
|
+
return row ? toGetResult(row) : null;
|
|
54
|
+
} catch (err) {
|
|
55
|
+
logger?.warn?.(`memex-openclaw: corpus get failed: ${err.message}`);
|
|
56
|
+
return null;
|
|
57
|
+
}
|
|
58
|
+
},
|
|
59
|
+
};
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
/**
|
|
63
|
+
* Map a memex row (from store.search) to OpenClaw's
|
|
64
|
+
* MemoryCorpusSearchResult shape:
|
|
65
|
+
*
|
|
66
|
+
* { corpus, path, title?, kind?, score, snippet, id?,
|
|
67
|
+
* startLine?, endLine?, citation?, source?,
|
|
68
|
+
* provenanceLabel?, sourceType?, sourcePath?, updatedAt? }
|
|
69
|
+
*/
|
|
70
|
+
function toSearchResult(row) {
|
|
71
|
+
const date = row.ts ? new Date(row.ts * 1000).toISOString() : undefined;
|
|
72
|
+
return {
|
|
73
|
+
corpus: CORPUS_LABEL,
|
|
74
|
+
id: `memex:${row.id}`,
|
|
75
|
+
path: `memex://${row.conversation_id}/#msg-${row.id}`,
|
|
76
|
+
title: titleFromRow(row),
|
|
77
|
+
kind: row.channel || 'openclaw',
|
|
78
|
+
score: 1.0, // memex returns ranked-by-ts; bm25 inside FTS would refine
|
|
79
|
+
snippet: row.preview || '',
|
|
80
|
+
source: 'openclaw',
|
|
81
|
+
provenanceLabel: row.channel
|
|
82
|
+
? `memex • ${row.channel}`
|
|
83
|
+
: 'memex • openclaw',
|
|
84
|
+
sourceType: 'verbatim',
|
|
85
|
+
sourcePath: row.conversation_id,
|
|
86
|
+
updatedAt: date,
|
|
87
|
+
};
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
/**
|
|
91
|
+
* Map a memex row (from store.getById) to OpenClaw's
|
|
92
|
+
* MemoryCorpusGetResult shape:
|
|
93
|
+
*
|
|
94
|
+
* { corpus, path, title?, kind?, content, fromLine, lineCount,
|
|
95
|
+
* id?, provenanceLabel?, sourceType?, sourcePath?, updatedAt? }
|
|
96
|
+
*/
|
|
97
|
+
function toGetResult(row) {
|
|
98
|
+
const date = row.ts ? new Date(row.ts * 1000).toISOString() : undefined;
|
|
99
|
+
const lineCount = row.text ? row.text.split('\n').length : 1;
|
|
100
|
+
return {
|
|
101
|
+
corpus: CORPUS_LABEL,
|
|
102
|
+
id: `memex:${row.id}`,
|
|
103
|
+
path: `memex://${row.conversation_id}/#msg-${row.id}`,
|
|
104
|
+
title: titleFromRow(row),
|
|
105
|
+
kind: row.channel || 'openclaw',
|
|
106
|
+
content: row.text || '',
|
|
107
|
+
fromLine: 1,
|
|
108
|
+
lineCount,
|
|
109
|
+
provenanceLabel: row.channel
|
|
110
|
+
? `memex • ${row.channel}`
|
|
111
|
+
: 'memex • openclaw',
|
|
112
|
+
sourceType: 'verbatim',
|
|
113
|
+
sourcePath: row.conversation_id,
|
|
114
|
+
updatedAt: date,
|
|
115
|
+
};
|
|
116
|
+
}
|
|
117
|
+
|
|
118
|
+
function titleFromRow(row) {
|
|
119
|
+
const role = row.role || '?';
|
|
120
|
+
const when = row.ts
|
|
121
|
+
? new Date(row.ts * 1000).toISOString().slice(0, 10)
|
|
122
|
+
: '?';
|
|
123
|
+
// First ~40 chars of text → title-ish
|
|
124
|
+
const preview = (row.text || row.preview || '')
|
|
125
|
+
.replace(/\s+/g, ' ')
|
|
126
|
+
.trim()
|
|
127
|
+
.slice(0, 40);
|
|
128
|
+
return preview
|
|
129
|
+
? `[${when} ${role}] ${preview}${preview.length === 40 ? '…' : ''}`
|
|
130
|
+
: `${role} @ ${when}`;
|
|
131
|
+
}
|
package/lib/store.js
ADDED
|
@@ -0,0 +1,227 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* SQLite layer for memex-openclaw.
|
|
3
|
+
*
|
|
4
|
+
* Opens the shared memex.db at ~/.memex/data/memex.db (override via
|
|
5
|
+
* plugin config db_path), creates the schema if absent, exposes
|
|
6
|
+
* insert/search/get operations.
|
|
7
|
+
*
|
|
8
|
+
* Schema parity with memex-mvp (Node) and memex-hermes (Python) — all
|
|
9
|
+
* three write to the SAME memex.db using identical tables, columns,
|
|
10
|
+
* and the UNIQUE(source, conversation_id, msg_id) constraint that
|
|
11
|
+
* makes inserts idempotent.
|
|
12
|
+
*
|
|
13
|
+
* WAL mode → concurrent reads from memex CLI / MCP server are safe
|
|
14
|
+
* while we write.
|
|
15
|
+
*/
|
|
16
|
+
|
|
17
|
+
import Database from 'better-sqlite3';
|
|
18
|
+
import { mkdirSync } from 'node:fs';
|
|
19
|
+
import { dirname } from 'node:path';
|
|
20
|
+
import { homedir } from 'node:os';
|
|
21
|
+
import { resolve } from 'node:path';
|
|
22
|
+
|
|
23
|
+
export const DEFAULT_DB_PATH = '~/.memex/data/memex.db';
|
|
24
|
+
|
|
25
|
+
/**
|
|
26
|
+
* Expand `~` and turn a possibly-relative path into an absolute one.
|
|
27
|
+
*/
|
|
28
|
+
export function resolveDbPath(p) {
|
|
29
|
+
if (!p || typeof p !== 'string') p = DEFAULT_DB_PATH;
|
|
30
|
+
let s = p.trim();
|
|
31
|
+
if (s.startsWith('~/')) s = homedir() + s.slice(1);
|
|
32
|
+
else if (s === '~') s = homedir();
|
|
33
|
+
return resolve(s);
|
|
34
|
+
}
|
|
35
|
+
|
|
36
|
+
function safeAlter(db, sql) {
|
|
37
|
+
try {
|
|
38
|
+
db.exec(sql);
|
|
39
|
+
} catch (err) {
|
|
40
|
+
if (!String(err.message).toLowerCase().includes('duplicate column')) throw err;
|
|
41
|
+
}
|
|
42
|
+
}
|
|
43
|
+
|
|
44
|
+
/**
|
|
45
|
+
* Apply the memex.db schema. Idempotent — every CREATE uses IF NOT
|
|
46
|
+
* EXISTS, ALTERs swallow "duplicate column" errors. Safe to call
|
|
47
|
+
* against an existing memex.db created by memex-mvp or memex-hermes.
|
|
48
|
+
*/
|
|
49
|
+
export function initialiseSchema(db) {
|
|
50
|
+
db.pragma('journal_mode = WAL');
|
|
51
|
+
db.pragma('synchronous = NORMAL');
|
|
52
|
+
|
|
53
|
+
db.exec(`
|
|
54
|
+
CREATE TABLE IF NOT EXISTS messages (
|
|
55
|
+
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
56
|
+
source TEXT NOT NULL,
|
|
57
|
+
conversation_id TEXT NOT NULL,
|
|
58
|
+
msg_id TEXT,
|
|
59
|
+
role TEXT,
|
|
60
|
+
sender TEXT,
|
|
61
|
+
text TEXT,
|
|
62
|
+
ts INTEGER,
|
|
63
|
+
metadata TEXT,
|
|
64
|
+
UNIQUE(source, conversation_id, msg_id)
|
|
65
|
+
);
|
|
66
|
+
CREATE INDEX IF NOT EXISTS idx_messages_ts ON messages(ts);
|
|
67
|
+
CREATE INDEX IF NOT EXISTS idx_messages_conv ON messages(conversation_id);
|
|
68
|
+
CREATE INDEX IF NOT EXISTS idx_messages_source ON messages(source);
|
|
69
|
+
|
|
70
|
+
CREATE VIRTUAL TABLE IF NOT EXISTS messages_fts USING fts5(
|
|
71
|
+
text, sender, conversation_id, source,
|
|
72
|
+
content=messages, content_rowid=id,
|
|
73
|
+
tokenize='unicode61 remove_diacritics 2'
|
|
74
|
+
);
|
|
75
|
+
|
|
76
|
+
CREATE TABLE IF NOT EXISTS conversations (
|
|
77
|
+
conversation_id TEXT PRIMARY KEY,
|
|
78
|
+
source TEXT NOT NULL,
|
|
79
|
+
title TEXT,
|
|
80
|
+
first_ts INTEGER,
|
|
81
|
+
last_ts INTEGER,
|
|
82
|
+
message_count INTEGER DEFAULT 0
|
|
83
|
+
);
|
|
84
|
+
`);
|
|
85
|
+
|
|
86
|
+
safeAlter(db, 'ALTER TABLE messages ADD COLUMN edited_at INTEGER');
|
|
87
|
+
safeAlter(db, 'ALTER TABLE messages ADD COLUMN uuid TEXT');
|
|
88
|
+
safeAlter(db, 'ALTER TABLE messages ADD COLUMN channel TEXT');
|
|
89
|
+
safeAlter(db, 'ALTER TABLE conversations ADD COLUMN archived_at INTEGER');
|
|
90
|
+
safeAlter(db, 'ALTER TABLE conversations ADD COLUMN parent_conversation_id TEXT');
|
|
91
|
+
safeAlter(db, 'ALTER TABLE conversations ADD COLUMN project_path TEXT');
|
|
92
|
+
|
|
93
|
+
db.exec(`
|
|
94
|
+
CREATE INDEX IF NOT EXISTS idx_messages_uuid
|
|
95
|
+
ON messages(uuid) WHERE uuid IS NOT NULL;
|
|
96
|
+
CREATE INDEX IF NOT EXISTS idx_messages_channel
|
|
97
|
+
ON messages(channel) WHERE channel IS NOT NULL;
|
|
98
|
+
|
|
99
|
+
DROP TRIGGER IF EXISTS messages_fts_ai;
|
|
100
|
+
DROP TRIGGER IF EXISTS messages_fts_ad;
|
|
101
|
+
DROP TRIGGER IF EXISTS messages_fts_au;
|
|
102
|
+
CREATE TRIGGER messages_fts_ai AFTER INSERT ON messages
|
|
103
|
+
WHEN new.role != 'summary' BEGIN
|
|
104
|
+
INSERT INTO messages_fts(rowid, text, sender, conversation_id, source)
|
|
105
|
+
VALUES (new.id, new.text, new.sender, new.conversation_id, new.source);
|
|
106
|
+
END;
|
|
107
|
+
CREATE TRIGGER messages_fts_ad AFTER DELETE ON messages BEGIN
|
|
108
|
+
DELETE FROM messages_fts WHERE rowid = old.id;
|
|
109
|
+
END;
|
|
110
|
+
CREATE TRIGGER messages_fts_au AFTER UPDATE ON messages BEGIN
|
|
111
|
+
DELETE FROM messages_fts WHERE rowid = old.id;
|
|
112
|
+
INSERT INTO messages_fts(rowid, text, sender, conversation_id, source)
|
|
113
|
+
SELECT new.id, new.text, new.sender, new.conversation_id, new.source
|
|
114
|
+
WHERE new.role != 'summary';
|
|
115
|
+
END;
|
|
116
|
+
`);
|
|
117
|
+
}
|
|
118
|
+
|
|
119
|
+
export class MemexStore {
|
|
120
|
+
constructor(dbPath) {
|
|
121
|
+
this.dbPath = resolveDbPath(dbPath);
|
|
122
|
+
mkdirSync(dirname(this.dbPath), { recursive: true });
|
|
123
|
+
this.db = new Database(this.dbPath);
|
|
124
|
+
initialiseSchema(this.db);
|
|
125
|
+
|
|
126
|
+
// Prepared statements (faster on the hot path).
|
|
127
|
+
this._insertMsg = this.db.prepare(`
|
|
128
|
+
INSERT OR IGNORE INTO messages
|
|
129
|
+
(source, conversation_id, msg_id, role, sender, text, ts, metadata, channel)
|
|
130
|
+
VALUES ('openclaw', @conversationId, @msgId, @role, @sender, @text, @ts, @metadata, @channel)
|
|
131
|
+
`);
|
|
132
|
+
|
|
133
|
+
this._upsertConv = this.db.prepare(`
|
|
134
|
+
INSERT INTO conversations
|
|
135
|
+
(conversation_id, source, title, first_ts, last_ts, message_count)
|
|
136
|
+
VALUES (@conversationId, 'openclaw', @title, @firstTs, @lastTs, 0)
|
|
137
|
+
ON CONFLICT(conversation_id) DO UPDATE SET
|
|
138
|
+
title = COALESCE(conversations.title, excluded.title),
|
|
139
|
+
first_ts = MIN(COALESCE(conversations.first_ts, excluded.first_ts), excluded.first_ts),
|
|
140
|
+
last_ts = MAX(COALESCE(conversations.last_ts, excluded.last_ts), excluded.last_ts),
|
|
141
|
+
message_count = (
|
|
142
|
+
SELECT COUNT(*) FROM messages
|
|
143
|
+
WHERE messages.conversation_id = conversations.conversation_id
|
|
144
|
+
)
|
|
145
|
+
`);
|
|
146
|
+
|
|
147
|
+
this._searchStmt = this.db.prepare(`
|
|
148
|
+
SELECT m.id, m.ts, m.role, m.conversation_id, m.channel,
|
|
149
|
+
substr(m.text, 1, 100) AS preview
|
|
150
|
+
FROM messages_fts f
|
|
151
|
+
JOIN messages m ON m.id = f.rowid
|
|
152
|
+
WHERE f.text MATCH ? AND m.source = 'openclaw'
|
|
153
|
+
ORDER BY m.ts DESC
|
|
154
|
+
LIMIT ?
|
|
155
|
+
`);
|
|
156
|
+
|
|
157
|
+
this._getStmt = this.db.prepare(`
|
|
158
|
+
SELECT id, ts, role, sender, conversation_id, channel, text, metadata
|
|
159
|
+
FROM messages
|
|
160
|
+
WHERE id = ? AND source = 'openclaw'
|
|
161
|
+
`);
|
|
162
|
+
|
|
163
|
+
this._countStmt = this.db.prepare(`
|
|
164
|
+
SELECT COUNT(*) AS n FROM messages WHERE source = 'openclaw'
|
|
165
|
+
`);
|
|
166
|
+
}
|
|
167
|
+
|
|
168
|
+
close() {
|
|
169
|
+
try { this.db.close(); } catch { /* already closed */ }
|
|
170
|
+
}
|
|
171
|
+
|
|
172
|
+
/**
|
|
173
|
+
* Insert one verbatim message. Returns true if a new row was created,
|
|
174
|
+
* false if the UNIQUE constraint deduped it (idempotent).
|
|
175
|
+
*/
|
|
176
|
+
insertMessage({ conversationId, msgId, role, text, ts, channel, sender, metadata }) {
|
|
177
|
+
if (!text || !String(text).trim()) return false;
|
|
178
|
+
const senderNorm = sender || (role === 'user' ? 'me' : 'openclaw');
|
|
179
|
+
const result = this._insertMsg.run({
|
|
180
|
+
conversationId,
|
|
181
|
+
msgId,
|
|
182
|
+
role,
|
|
183
|
+
sender: senderNorm,
|
|
184
|
+
text: String(text),
|
|
185
|
+
ts: ts || Math.floor(Date.now() / 1000),
|
|
186
|
+
metadata: JSON.stringify(metadata || {}),
|
|
187
|
+
channel: channel ?? null,
|
|
188
|
+
});
|
|
189
|
+
return result.changes > 0;
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
upsertConversation({ conversationId, title, firstTs, lastTs }) {
|
|
193
|
+
this._upsertConv.run({
|
|
194
|
+
conversationId,
|
|
195
|
+
title: title || conversationId,
|
|
196
|
+
firstTs: firstTs ?? null,
|
|
197
|
+
lastTs: lastTs ?? null,
|
|
198
|
+
});
|
|
199
|
+
}
|
|
200
|
+
|
|
201
|
+
/**
|
|
202
|
+
* FTS5 search restricted to source='openclaw'. Returns abbreviated rows
|
|
203
|
+
* (id + ts + role + 100-char preview) — full text is fetched via getById()
|
|
204
|
+
* to support the progressive-disclosure pattern.
|
|
205
|
+
*/
|
|
206
|
+
search(query, limit = 10) {
|
|
207
|
+
if (!query || !String(query).trim()) return [];
|
|
208
|
+
try {
|
|
209
|
+
return this._searchStmt.all(String(query).trim(), Math.min(Math.max(Number(limit) || 10, 1), 50));
|
|
210
|
+
} catch (err) {
|
|
211
|
+
// Malformed FTS5 syntax — return empty rather than crash.
|
|
212
|
+
return [];
|
|
213
|
+
}
|
|
214
|
+
}
|
|
215
|
+
|
|
216
|
+
getById(id) {
|
|
217
|
+
const row = this._getStmt.get(Number(id));
|
|
218
|
+
if (!row) return null;
|
|
219
|
+
try { row.metadata = JSON.parse(row.metadata || '{}'); }
|
|
220
|
+
catch { row.metadata = {}; }
|
|
221
|
+
return row;
|
|
222
|
+
}
|
|
223
|
+
|
|
224
|
+
count() {
|
|
225
|
+
return this._countStmt.get().n;
|
|
226
|
+
}
|
|
227
|
+
}
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
{
|
|
2
|
+
"id": "memex-openclaw",
|
|
3
|
+
"name": "Memex",
|
|
4
|
+
"description": "Captures every OpenClaw turn verbatim into the memex unified SQLite corpus. Pair with memex-mvp (npm) to search OpenClaw + Hermes + Claude Code + Telegram from one place.",
|
|
5
|
+
"version": "0.1.0",
|
|
6
|
+
"kind": "memory",
|
|
7
|
+
"activation": {
|
|
8
|
+
"onStartup": true
|
|
9
|
+
},
|
|
10
|
+
"contracts": {
|
|
11
|
+
"memoryCorpusSupplements": ["memex-openclaw"]
|
|
12
|
+
},
|
|
13
|
+
"uiHints": {
|
|
14
|
+
"category": "memory",
|
|
15
|
+
"displayName": "Memex (verbatim corpus)"
|
|
16
|
+
}
|
|
17
|
+
}
|
package/package.json
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@parallelclaw/memex-openclaw",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "OpenClaw plugin that captures every turn verbatim into the memex unified SQLite corpus. Pair with memex-mvp for one searchable history across OpenClaw, Hermes, Claude Code, Telegram, and more.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"main": "index.js",
|
|
7
|
+
"exports": {
|
|
8
|
+
".": "./index.js"
|
|
9
|
+
},
|
|
10
|
+
"files": [
|
|
11
|
+
"index.js",
|
|
12
|
+
"openclaw.plugin.json",
|
|
13
|
+
"lib/",
|
|
14
|
+
"README.md",
|
|
15
|
+
"LICENSE"
|
|
16
|
+
],
|
|
17
|
+
"scripts": {
|
|
18
|
+
"test": "node --test tests/*.test.js"
|
|
19
|
+
},
|
|
20
|
+
"openclaw": {
|
|
21
|
+
"extensions": [
|
|
22
|
+
"./index.js"
|
|
23
|
+
]
|
|
24
|
+
},
|
|
25
|
+
"engines": {
|
|
26
|
+
"node": ">=22.12.0"
|
|
27
|
+
},
|
|
28
|
+
"dependencies": {
|
|
29
|
+
"better-sqlite3": "^11.0.0"
|
|
30
|
+
},
|
|
31
|
+
"peerDependencies": {
|
|
32
|
+
"openclaw": ">=2026.5.0"
|
|
33
|
+
},
|
|
34
|
+
"peerDependenciesMeta": {
|
|
35
|
+
"openclaw": {
|
|
36
|
+
"optional": false
|
|
37
|
+
}
|
|
38
|
+
},
|
|
39
|
+
"keywords": [
|
|
40
|
+
"openclaw",
|
|
41
|
+
"openclaw-plugin",
|
|
42
|
+
"memex",
|
|
43
|
+
"memory",
|
|
44
|
+
"verbatim",
|
|
45
|
+
"local-first",
|
|
46
|
+
"sqlite",
|
|
47
|
+
"fts5",
|
|
48
|
+
"parallelclaw",
|
|
49
|
+
"ai-memory"
|
|
50
|
+
],
|
|
51
|
+
"author": {
|
|
52
|
+
"name": "parallelclaw",
|
|
53
|
+
"email": "sedelev@gmail.com",
|
|
54
|
+
"url": "https://memex.parallelclaw.ai"
|
|
55
|
+
},
|
|
56
|
+
"license": "MIT",
|
|
57
|
+
"homepage": "https://memex.parallelclaw.ai",
|
|
58
|
+
"repository": {
|
|
59
|
+
"type": "git",
|
|
60
|
+
"url": "git+https://github.com/parallelclaw/memex-mvp.git",
|
|
61
|
+
"directory": "plugins/memex-openclaw"
|
|
62
|
+
},
|
|
63
|
+
"bugs": {
|
|
64
|
+
"url": "https://github.com/parallelclaw/memex-mvp/issues"
|
|
65
|
+
}
|
|
66
|
+
}
|