ei-tui 1.2.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +16 -0
- package/package.json +1 -1
- package/src/cli/README.md +12 -2
- package/src/cli/mcp.ts +12 -4
- package/src/cli/retrieval.ts +162 -0
- package/src/cli.ts +7 -1
- package/src/core/handlers/document-segmentation.ts +3 -4
- package/src/core/handlers/knowledge-synthesis.ts +1 -3
- package/src/core/heartbeat-manager.ts +10 -0
- package/src/core/orchestrators/ceremony.ts +1 -9
- package/src/core/processor.ts +122 -9
- package/src/core/tools/builtin/fetch-message.ts +27 -1
- package/src/core/tools/builtin/find-memory.ts +11 -3
- package/src/core/tools/index.ts +2 -2
- package/src/core/types/llm.ts +0 -9
- package/src/core/utils/message-id.ts +114 -0
- package/src/integrations/claude-code/importer.ts +6 -5
- package/src/integrations/cursor/importer.ts +6 -5
- package/src/integrations/document/importer.ts +1 -1
- package/src/integrations/document/unsource.ts +6 -11
- package/src/integrations/opencode/importer.ts +6 -5
- package/src/integrations/opencode/json-reader.ts +65 -0
- package/src/integrations/opencode/sqlite-reader.ts +33 -0
- package/src/integrations/opencode/types.ts +8 -0
- package/src/prompts/heartbeat/check.ts +5 -2
- package/src/prompts/heartbeat/ei.ts +7 -0
- package/src/prompts/heartbeat/types.ts +5 -0
- package/src/prompts/response/sections.ts +30 -16
- package/src/prompts/room/sections.ts +28 -6
- package/src/prompts/trait-utils.ts +33 -0
- package/tui/README.md +2 -0
- package/tui/src/util/help-content.ts +11 -0
package/README.md
CHANGED
|
@@ -185,6 +185,22 @@ Ei splits the document into segments, runs them through the extraction pipeline,
|
|
|
185
185
|
|
|
186
186
|
Both surfaces show you which documents have been imported and let you remove their extracted knowledge (web: Delete button in the Documents tab; TUI: `/unsource <source_tag>`).
|
|
187
187
|
|
|
188
|
+
## Knowledge Share
|
|
189
|
+
|
|
190
|
+
Sometimes you want to take what Ei knows and turn it into something you can hand to another human. A new teammate joining a project. A briefing doc before a meeting. A brain dump before a vacation.
|
|
191
|
+
|
|
192
|
+
**TUI**:
|
|
193
|
+
```bash
|
|
194
|
+
/generate everything about the Uniform project
|
|
195
|
+
/generate comprehensive runbook for the IDP deployment issues
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
Give it a subject description and Ei searches its memory, chases down related entities, and synthesizes a clean markdown document using your `rewrite_model` (an Opus-class model is recommended — it'll warn you if you're about to use something less capable). The document lands in `$EI_DATA_PATH/docs/` automatically.
|
|
199
|
+
|
|
200
|
+
`/generate` with no args opens a list of your existing generated documents where you can re-run, re-export, or delete them.
|
|
201
|
+
|
|
202
|
+
**Web**: Open **☰ menu** → **My Data** → **Documents** tab. Click **Generate**, enter a subject, and Ei does the rest. Generated docs appear in the same list as imported ones.
|
|
203
|
+
|
|
188
204
|
## Built-in Tool Integrations
|
|
189
205
|
|
|
190
206
|
Personas can use tools. Not just read-from-memory tools — *actual* tools. Web search. Your music. Your filesystem. Here's what ships with Ei out of the box:
|
package/package.json
CHANGED
package/src/cli/README.md
CHANGED
|
@@ -13,7 +13,7 @@ ei personas -n 5 "query string" # Return up to 5 personas (name match)
|
|
|
13
13
|
ei --persona "Beta" "query string" # Filter results to what Beta has learned
|
|
14
14
|
ei --recent # Most recently mentioned items (no query needed)
|
|
15
15
|
ei --persona "Beta" --recent # Most recently mentioned items Beta has learned
|
|
16
|
-
ei --id <id> # Look up a
|
|
16
|
+
ei --id <id> # Look up entity by ID — or fetch a message by FQ ID
|
|
17
17
|
echo <id> | ei --id # Look up entity by ID from stdin
|
|
18
18
|
ei --install # Register Ei with OpenCode, Claude Code, and Cursor
|
|
19
19
|
ei mcp # Start the Ei MCP stdio server (for Cursor/Claude Desktop)
|
|
@@ -29,6 +29,16 @@ The `--id` flag is designed for piping. For example, search for a topic and then
|
|
|
29
29
|
ei "memory leak" | jq '.[0].id' | ei --id
|
|
30
30
|
```
|
|
31
31
|
|
|
32
|
+
It also resolves fully-qualified message IDs from any supported integration, returning the original message content and session context:
|
|
33
|
+
|
|
34
|
+
```sh
|
|
35
|
+
ei --id "opencode:jeremys-macbook-pro:ses_38a7...:msg_c75b..."
|
|
36
|
+
ei --id "claudecode:my-machine:session-uuid:message-uuid"
|
|
37
|
+
ei --id "cursor:my-machine:composer-uuid:bubble-uuid"
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Quotes surfaced by `ei_search` or `ei_find_memory` include a `message_id` field in this format — pipe it to `ei --id` to read the original conversation.
|
|
41
|
+
|
|
32
42
|
# OpenCode Integration
|
|
33
43
|
|
|
34
44
|
## Quick Install
|
|
@@ -156,7 +166,7 @@ The MCP server exposes these tools to Claude Code, Cursor, and OpenCode:
|
|
|
156
166
|
| `ei_lookup` | Full-record lookup for any entity by ID (facts, topics, people, quotes, personas). |
|
|
157
167
|
| `ei_find_memory` | Grouped human-data search — facts, topics, people, quotes. Returns results grouped by type. Mirrors the persona `find_memory` tool interface. |
|
|
158
168
|
| `ei_fetch_memory` | Full-record lookup for a human entity (Fact, Topic, Person, or Quote) by ID. Returns the complete record including all fields. |
|
|
159
|
-
| `ei_fetch_message` | Retrieve a specific message by ID with optional `before`/`after` context window.
|
|
169
|
+
| `ei_fetch_message` | Retrieve a specific message by fully-qualified ID with optional `before`/`after` context window. Routes to the correct source: `ei:uuid` searches Ei state, `opencode:machine:session:id` queries the OpenCode SQLite DB, `claudecode:...` scans Claude Code JSONL files, `cursor:...` reads the Cursor global DB. Returns message content, surrounding context, and session metadata. |
|
|
160
170
|
|
|
161
171
|
### `ei_search` / `ei_find_memory` arguments
|
|
162
172
|
|
package/src/cli/mcp.ts
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
|
|
2
2
|
import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
|
|
3
3
|
import { z } from "zod";
|
|
4
|
-
import { retrieveBalanced, lookupById, loadLatestState, type BalancedResult } from "./retrieval.js";
|
|
4
|
+
import { retrieveBalanced, lookupById, resolveExternalMessage, loadLatestState, type BalancedResult } from "./retrieval.js";
|
|
5
5
|
import type { StorageState } from "../core/types.js";
|
|
6
6
|
import type { Message } from "../core/types.js";
|
|
7
7
|
import type { RoomMessage } from "../core/types/rooms.js";
|
|
@@ -270,6 +270,17 @@ export function createMcpServer(): McpServer {
|
|
|
270
270
|
},
|
|
271
271
|
},
|
|
272
272
|
async ({ id, before: beforeCount, after: afterCount }) => {
|
|
273
|
+
const beforeN = Math.max(0, Math.floor(beforeCount ?? 0));
|
|
274
|
+
const afterN = Math.max(0, Math.floor(afterCount ?? 0));
|
|
275
|
+
|
|
276
|
+
const externalResult = await resolveExternalMessage(id, beforeN, afterN);
|
|
277
|
+
if (externalResult) {
|
|
278
|
+
if ("error" in externalResult) {
|
|
279
|
+
return { content: [{ type: "text" as const, text: String(externalResult.error) }] };
|
|
280
|
+
}
|
|
281
|
+
return { content: [{ type: "text" as const, text: JSON.stringify(externalResult, null, 2) }] };
|
|
282
|
+
}
|
|
283
|
+
|
|
273
284
|
const state = await loadLatestState();
|
|
274
285
|
if (!state) {
|
|
275
286
|
return {
|
|
@@ -277,9 +288,6 @@ export function createMcpServer(): McpServer {
|
|
|
277
288
|
};
|
|
278
289
|
}
|
|
279
290
|
|
|
280
|
-
const beforeN = Math.max(0, Math.floor(beforeCount ?? 0));
|
|
281
|
-
const afterN = Math.max(0, Math.floor(afterCount ?? 0));
|
|
282
|
-
|
|
283
291
|
const stripPersonaMessage = (m: Message) => ({
|
|
284
292
|
id: m.id,
|
|
285
293
|
role: m.role,
|
package/src/cli/retrieval.ts
CHANGED
|
@@ -6,6 +6,8 @@ import { crossFind } from "../core/utils/index.ts";
|
|
|
6
6
|
import { join } from "path";
|
|
7
7
|
import { readFile } from "fs/promises";
|
|
8
8
|
import { getEmbeddingService, findTopK } from "../core/embedding-service";
|
|
9
|
+
import { parseMessageId } from "../core/utils/message-id.js";
|
|
10
|
+
import { getMachineId } from "../integrations/machine-id.js";
|
|
9
11
|
|
|
10
12
|
const STATE_FILE = "state.json";
|
|
11
13
|
const BACKUP_FILE = "state.backup.json";
|
|
@@ -86,6 +88,7 @@ export interface QuoteResult {
|
|
|
86
88
|
text: string;
|
|
87
89
|
speaker: string;
|
|
88
90
|
timestamp: string;
|
|
91
|
+
message_id: string | null;
|
|
89
92
|
linked_items: LinkedItem[];
|
|
90
93
|
}
|
|
91
94
|
|
|
@@ -164,6 +167,7 @@ export function mapQuote(quote: Quote, state: StorageState): QuoteResult {
|
|
|
164
167
|
text: quote.text,
|
|
165
168
|
speaker: quote.speaker,
|
|
166
169
|
timestamp: quote.timestamp,
|
|
170
|
+
message_id: quote.message_id,
|
|
167
171
|
linked_items: resolveLinkedItems(quote.data_item_ids, state),
|
|
168
172
|
};
|
|
169
173
|
}
|
|
@@ -394,6 +398,164 @@ export async function retrieveBalanced(
|
|
|
394
398
|
return embeddingFinal;
|
|
395
399
|
}
|
|
396
400
|
|
|
401
|
+
const OPENCODE_MESSAGE_ID = /^msg_[a-zA-Z0-9]+$/;
|
|
402
|
+
|
|
403
|
+
/** @deprecated Use resolveExternalMessage */
|
|
404
|
+
export async function resolveOpenCodeMessage(
|
|
405
|
+
id: string,
|
|
406
|
+
before = 0,
|
|
407
|
+
after = 0
|
|
408
|
+
): Promise<Record<string, unknown> | null> {
|
|
409
|
+
return resolveExternalMessage(id, before, after);
|
|
410
|
+
}
|
|
411
|
+
|
|
412
|
+
export async function resolveExternalMessage(
|
|
413
|
+
id: string,
|
|
414
|
+
before = 0,
|
|
415
|
+
after = 0
|
|
416
|
+
): Promise<Record<string, unknown> | null> {
|
|
417
|
+
const parsed = parseMessageId(id);
|
|
418
|
+
|
|
419
|
+
switch (parsed.integration) {
|
|
420
|
+
case "ei": {
|
|
421
|
+
const state = await loadLatestState();
|
|
422
|
+
if (!state) return null;
|
|
423
|
+
|
|
424
|
+
for (const { entity: persona, messages } of Object.values(state.personas)) {
|
|
425
|
+
const idx = messages.findIndex(m => m.id === id);
|
|
426
|
+
if (idx === -1) continue;
|
|
427
|
+
const msg = messages[idx];
|
|
428
|
+
return {
|
|
429
|
+
type: "opencode_message",
|
|
430
|
+
message: { id: msg.id, role: msg.role, content: msg.content ?? "", timestamp: msg.timestamp },
|
|
431
|
+
before: messages.slice(Math.max(0, idx - before), idx).map(m => ({ id: m.id, role: m.role, content: m.content ?? "", timestamp: m.timestamp })),
|
|
432
|
+
after: messages.slice(idx + 1, idx + 1 + after).map(m => ({ id: m.id, role: m.role, content: m.content ?? "", timestamp: m.timestamp })),
|
|
433
|
+
session: { id: persona.id, title: persona.display_name, directory: "" },
|
|
434
|
+
source: "ei",
|
|
435
|
+
};
|
|
436
|
+
}
|
|
437
|
+
|
|
438
|
+
for (const room of Object.values(state.rooms ?? {})) {
|
|
439
|
+
const idx = room.messages.findIndex(m => m.id === id);
|
|
440
|
+
if (idx === -1) continue;
|
|
441
|
+
const msg = room.messages[idx];
|
|
442
|
+
return {
|
|
443
|
+
type: "opencode_message",
|
|
444
|
+
message: { id: msg.id, role: msg.role, content: msg.content ?? "", timestamp: msg.timestamp },
|
|
445
|
+
before: room.messages.slice(Math.max(0, idx - before), idx).map(m => ({ id: m.id, role: m.role, content: m.content ?? "", timestamp: m.timestamp })),
|
|
446
|
+
after: room.messages.slice(idx + 1, idx + 1 + after).map(m => ({ id: m.id, role: m.role, content: m.content ?? "", timestamp: m.timestamp })),
|
|
447
|
+
session: { id: room.id, title: room.display_name, directory: "" },
|
|
448
|
+
source: "ei",
|
|
449
|
+
};
|
|
450
|
+
}
|
|
451
|
+
|
|
452
|
+
return null;
|
|
453
|
+
}
|
|
454
|
+
|
|
455
|
+
case "opencode": {
|
|
456
|
+
if (parsed.machine !== getMachineId()) {
|
|
457
|
+
return { error: `Message is from machine '${parsed.machine}', not available on this machine (${getMachineId()})` };
|
|
458
|
+
}
|
|
459
|
+
try {
|
|
460
|
+
const { createOpenCodeReader } = await import("../integrations/opencode/reader-factory.js");
|
|
461
|
+
const reader = await createOpenCodeReader();
|
|
462
|
+
const win = await reader.getMessageById(parsed.nativeId, before, after);
|
|
463
|
+
if (!win) return null;
|
|
464
|
+
return {
|
|
465
|
+
type: "opencode_message",
|
|
466
|
+
message: { id: win.message.id, role: win.message.role, content: win.message.content, timestamp: win.message.timestamp, agent: win.message.agent },
|
|
467
|
+
before: win.before.map(m => ({ id: m.id, role: m.role, content: m.content, timestamp: m.timestamp, agent: m.agent })),
|
|
468
|
+
after: win.after.map(m => ({ id: m.id, role: m.role, content: m.content, timestamp: m.timestamp, agent: m.agent })),
|
|
469
|
+
session: { id: win.session.id, title: win.session.title, directory: win.session.directory },
|
|
470
|
+
source: "opencode",
|
|
471
|
+
};
|
|
472
|
+
} catch {
|
|
473
|
+
return null;
|
|
474
|
+
}
|
|
475
|
+
}
|
|
476
|
+
|
|
477
|
+
case "claudecode": {
|
|
478
|
+
if (parsed.machine !== getMachineId()) {
|
|
479
|
+
return { error: `Message is from machine '${parsed.machine}', not available on this machine (${getMachineId()})` };
|
|
480
|
+
}
|
|
481
|
+
try {
|
|
482
|
+
const { ClaudeCodeReader } = await import("../integrations/claude-code/reader.js");
|
|
483
|
+
const reader = new ClaudeCodeReader();
|
|
484
|
+
const messages = await reader.getMessagesForSession(parsed.session!);
|
|
485
|
+
const idx = messages.findIndex(m => m.id === parsed.nativeId);
|
|
486
|
+
if (idx === -1) return null;
|
|
487
|
+
const msg = messages[idx];
|
|
488
|
+
return {
|
|
489
|
+
type: "opencode_message",
|
|
490
|
+
message: { id: msg.id, role: msg.role, content: msg.content, timestamp: msg.timestamp },
|
|
491
|
+
before: messages.slice(Math.max(0, idx - before), idx).map(m => ({ id: m.id, role: m.role, content: m.content, timestamp: m.timestamp })),
|
|
492
|
+
after: messages.slice(idx + 1, idx + 1 + after).map(m => ({ id: m.id, role: m.role, content: m.content, timestamp: m.timestamp })),
|
|
493
|
+
session: { id: parsed.session!, title: parsed.session!, directory: "" },
|
|
494
|
+
source: "claudecode",
|
|
495
|
+
};
|
|
496
|
+
} catch {
|
|
497
|
+
return null;
|
|
498
|
+
}
|
|
499
|
+
}
|
|
500
|
+
|
|
501
|
+
case "cursor": {
|
|
502
|
+
if (parsed.machine !== getMachineId()) {
|
|
503
|
+
return { error: `Message is from machine '${parsed.machine}', not available on this machine (${getMachineId()})` };
|
|
504
|
+
}
|
|
505
|
+
try {
|
|
506
|
+
const { CursorReader } = await import("../integrations/cursor/reader.js");
|
|
507
|
+
const reader = new CursorReader();
|
|
508
|
+
const sessions = await reader.getSessions();
|
|
509
|
+
const session = sessions.find(s => s.id === parsed.session);
|
|
510
|
+
if (!session) return null;
|
|
511
|
+
const idx = session.messages.findIndex(m => m.id === parsed.nativeId);
|
|
512
|
+
if (idx === -1) return null;
|
|
513
|
+
const msg = session.messages[idx];
|
|
514
|
+
const mapCursor = (m: { id: string; type: 1 | 2; text: string; timestamp: string }) => ({
|
|
515
|
+
id: m.id,
|
|
516
|
+
role: (m.type === 1 ? "user" : "assistant") as "user" | "assistant",
|
|
517
|
+
content: m.text,
|
|
518
|
+
timestamp: m.timestamp,
|
|
519
|
+
});
|
|
520
|
+
return {
|
|
521
|
+
type: "opencode_message",
|
|
522
|
+
message: mapCursor(msg),
|
|
523
|
+
before: session.messages.slice(Math.max(0, idx - before), idx).map(mapCursor),
|
|
524
|
+
after: session.messages.slice(idx + 1, idx + 1 + after).map(mapCursor),
|
|
525
|
+
session: { id: session.id, title: session.name, directory: session.workspacePath },
|
|
526
|
+
source: "cursor",
|
|
527
|
+
};
|
|
528
|
+
} catch {
|
|
529
|
+
return null;
|
|
530
|
+
}
|
|
531
|
+
}
|
|
532
|
+
|
|
533
|
+
case "unknown":
|
|
534
|
+
default: {
|
|
535
|
+
// Backward compat: bare msg_xxx → treat as opencode (no machine qualifier)
|
|
536
|
+
if (OPENCODE_MESSAGE_ID.test(id)) {
|
|
537
|
+
try {
|
|
538
|
+
const { createOpenCodeReader } = await import("../integrations/opencode/reader-factory.js");
|
|
539
|
+
const reader = await createOpenCodeReader();
|
|
540
|
+
const win = await reader.getMessageById(id, before, after);
|
|
541
|
+
if (!win) return null;
|
|
542
|
+
return {
|
|
543
|
+
type: "opencode_message",
|
|
544
|
+
message: { id: win.message.id, role: win.message.role, content: win.message.content, timestamp: win.message.timestamp, agent: win.message.agent },
|
|
545
|
+
before: win.before.map(m => ({ id: m.id, role: m.role, content: m.content, timestamp: m.timestamp, agent: m.agent })),
|
|
546
|
+
after: win.after.map(m => ({ id: m.id, role: m.role, content: m.content, timestamp: m.timestamp, agent: m.agent })),
|
|
547
|
+
session: { id: win.session.id, title: win.session.title, directory: win.session.directory },
|
|
548
|
+
source: "opencode",
|
|
549
|
+
};
|
|
550
|
+
} catch {
|
|
551
|
+
return null;
|
|
552
|
+
}
|
|
553
|
+
}
|
|
554
|
+
return null;
|
|
555
|
+
}
|
|
556
|
+
}
|
|
557
|
+
}
|
|
558
|
+
|
|
397
559
|
export async function lookupById(id: string): Promise<({ type: string } & Record<string, unknown>) | null> {
|
|
398
560
|
const state = await loadLatestState();
|
|
399
561
|
if (!state) {
|
package/src/cli.ts
CHANGED
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
|
|
14
14
|
import { parseArgs } from "util";
|
|
15
15
|
import { join } from "path";
|
|
16
|
-
import { retrieveBalanced, lookupById, loadLatestState } from "./cli/retrieval";
|
|
16
|
+
import { retrieveBalanced, lookupById, resolveExternalMessage, loadLatestState } from "./cli/retrieval";
|
|
17
17
|
import type { StorageState } from "./core/types";
|
|
18
18
|
import { resolvePersonaId, filterByPersona, filterTypeSpecificByPersona, filterBySource, filterTypeSpecificBySource } from "./cli/persona-filter.js";
|
|
19
19
|
import pkg from "../package.json" assert { type: "json" };
|
|
@@ -247,6 +247,12 @@ mentions a person, or corrects something you assumed.
|
|
|
247
247
|
// Strip surrounding quotes (from jq output or shell quoting)
|
|
248
248
|
id = id.replace(/^["']|["']$/g, "");
|
|
249
249
|
|
|
250
|
+
const ocMessage = await resolveExternalMessage(id);
|
|
251
|
+
if (ocMessage) {
|
|
252
|
+
console.log(JSON.stringify(ocMessage, null, 2));
|
|
253
|
+
process.exit(0);
|
|
254
|
+
}
|
|
255
|
+
|
|
250
256
|
const entity = await lookupById(id);
|
|
251
257
|
if (!entity) {
|
|
252
258
|
console.error(`No entity found with ID: ${id}`);
|
|
@@ -5,6 +5,7 @@ import {
|
|
|
5
5
|
queueAllScans,
|
|
6
6
|
type ExtractionContext,
|
|
7
7
|
} from "../orchestrators/human-extraction.js";
|
|
8
|
+
import { qualifyDocumentMessage } from "../utils/message-id.js";
|
|
8
9
|
|
|
9
10
|
function parseSegmentArray(content: string): string[] | null {
|
|
10
11
|
const jsonMatch = content.match(/```json\s*([\s\S]*?)```/) ?? content.match(/```\s*([\s\S]*?)```/);
|
|
@@ -48,18 +49,16 @@ export function handleDocumentSegmentation(response: LLMResponse, state: StateMa
|
|
|
48
49
|
}
|
|
49
50
|
|
|
50
51
|
const now = new Date().toISOString();
|
|
51
|
-
const sourceTag = `import:document:${filename}`;
|
|
52
52
|
|
|
53
53
|
for (const segment of segments) {
|
|
54
54
|
const message: Message = {
|
|
55
|
-
id: crypto.randomUUID(),
|
|
55
|
+
id: qualifyDocumentMessage(filename, crypto.randomUUID()),
|
|
56
56
|
role: "system",
|
|
57
57
|
content: segment,
|
|
58
58
|
timestamp: now,
|
|
59
59
|
read: true,
|
|
60
60
|
context_status: ContextStatus.Always,
|
|
61
61
|
external: true,
|
|
62
|
-
source_tag: sourceTag,
|
|
63
62
|
};
|
|
64
63
|
state.messages_append("emmet", message);
|
|
65
64
|
}
|
|
@@ -71,7 +70,7 @@ export function finishDocumentBatch(batchId: string, filename: string, state: St
|
|
|
71
70
|
const sourceTag = `import:document:${filename}`;
|
|
72
71
|
|
|
73
72
|
const emmettMessages = state.messages_get("emmet");
|
|
74
|
-
const docMessages = emmettMessages.filter(m => m.external === true && m.
|
|
73
|
+
const docMessages = emmettMessages.filter(m => m.external === true && m.id.startsWith(`${sourceTag}:`));
|
|
75
74
|
|
|
76
75
|
if (docMessages.length === 0) {
|
|
77
76
|
console.warn(`[finishDocumentBatch] No messages found for ${sourceTag} — skipping extraction`);
|
|
@@ -18,17 +18,15 @@ export function handleKnowledgeSynthesis(response: LLMResponse, state: StateMana
|
|
|
18
18
|
}
|
|
19
19
|
|
|
20
20
|
const now = new Date().toISOString();
|
|
21
|
-
const sourceTag = `generate:document:${slug}`;
|
|
22
21
|
|
|
23
22
|
const message: Message = {
|
|
24
|
-
id: crypto.randomUUID()
|
|
23
|
+
id: `generate:document:${slug}:${crypto.randomUUID()}`,
|
|
25
24
|
role: "system",
|
|
26
25
|
content,
|
|
27
26
|
timestamp: now,
|
|
28
27
|
read: true,
|
|
29
28
|
context_status: ContextStatus.Always,
|
|
30
29
|
external: true,
|
|
31
|
-
source_tag: sourceTag,
|
|
32
30
|
};
|
|
33
31
|
|
|
34
32
|
state.messages_append("emmet", message);
|
|
@@ -168,6 +168,16 @@ export async function queueEiHeartbeat(
|
|
|
168
168
|
});
|
|
169
169
|
}
|
|
170
170
|
|
|
171
|
+
// Ei's own pending reflection — separate item type so she can introspect on herself
|
|
172
|
+
const eiPersona = personas.find((p) => p.id === "ei");
|
|
173
|
+
if (eiPersona?.pending_update?.critique) {
|
|
174
|
+
items.push({
|
|
175
|
+
id: eiPersona.id,
|
|
176
|
+
type: "Self Reflection Alert",
|
|
177
|
+
critique: eiPersona.pending_update.critique,
|
|
178
|
+
});
|
|
179
|
+
}
|
|
180
|
+
|
|
171
181
|
const personasWithPendingUpdate = personas.filter(
|
|
172
182
|
(p) => !p.is_archived && !p.is_paused && !p.is_static && p.id !== "ei" && p.pending_update?.critique
|
|
173
183
|
);
|
|
@@ -264,14 +264,6 @@ export function handleCeremonyProgress(state: StateManager, lastPhase: number):
|
|
|
264
264
|
!p.is_static
|
|
265
265
|
);
|
|
266
266
|
|
|
267
|
-
const eiIndex = activePersonas.findIndex(p =>
|
|
268
|
-
(p.aliases?.[0] ?? "").toLowerCase() === "ei"
|
|
269
|
-
);
|
|
270
|
-
|
|
271
|
-
// Ei's topics don't change
|
|
272
|
-
if (eiIndex > -1) {
|
|
273
|
-
activePersonas.splice(eiIndex, 1);
|
|
274
|
-
}
|
|
275
267
|
// Decay phase: apply decay + prune for ALL active personas
|
|
276
268
|
for (const persona of activePersonas) {
|
|
277
269
|
applyDecayPhase(persona.id, state);
|
|
@@ -623,7 +615,7 @@ function queueEventSummaryForAll(state: StateManager, options?: ExtractionOption
|
|
|
623
615
|
|
|
624
616
|
function queueReflectionPhase(state: StateManager): void {
|
|
625
617
|
const personas = state.persona_getAll().filter(p =>
|
|
626
|
-
!p.is_paused && !p.is_archived && !p.is_static
|
|
618
|
+
!p.is_paused && !p.is_archived && !p.is_static
|
|
627
619
|
);
|
|
628
620
|
|
|
629
621
|
let queued = 0;
|
package/src/core/processor.ts
CHANGED
|
@@ -139,6 +139,9 @@ import {
|
|
|
139
139
|
import type { RoomCreationInput, RoomEntity, RoomMessage, RoomSummary } from "./types.js";
|
|
140
140
|
import { previewUnsource as _previewUnsource } from "../integrations/document/unsource.js";
|
|
141
141
|
import type { UnsourcePreview, UnsourceResult } from "../integrations/document/unsource.js";
|
|
142
|
+
import { isQualifiedMessageId, qualifyEiMessage, qualifyOpenCodeMessage } from "./utils/message-id.js";
|
|
143
|
+
|
|
144
|
+
import type { IOpenCodeReader } from "../integrations/opencode/types.js";
|
|
142
145
|
|
|
143
146
|
const DEFAULT_LOOP_INTERVAL_MS = 100;
|
|
144
147
|
const DEFAULT_OPENCODE_POLLING_MS = 60000;
|
|
@@ -242,18 +245,30 @@ export class Processor {
|
|
|
242
245
|
this.bootstrapTools();
|
|
243
246
|
this.seedBuiltinFacts();
|
|
244
247
|
this.migrateLearnedOn();
|
|
248
|
+
await this.migrateMessageIds();
|
|
245
249
|
this.seedSettings();
|
|
246
250
|
registerFindMemoryExecutor(createFindMemoryExecutor(this.searchHumanData.bind(this), this.getPersonaList.bind(this), this.stateManager.getHuman.bind(this.stateManager)));
|
|
247
251
|
registerFetchMemoryExecutor(createFetchMemoryExecutor(this.stateManager.getHuman.bind(this.stateManager)));
|
|
248
|
-
registerFetchMessageExecutor(createFetchMessageExecutor(
|
|
249
|
-
this.stateManager.persona_getAll.bind(this.stateManager),
|
|
250
|
-
this.stateManager.messages_get.bind(this.stateManager),
|
|
251
|
-
this.stateManager.getRoomList.bind(this.stateManager),
|
|
252
|
-
this.stateManager.getRoomMessages.bind(this.stateManager),
|
|
253
|
-
(roomId: string) => this.stateManager.getRoom(roomId)?.display_name ?? null
|
|
254
|
-
));
|
|
255
252
|
if (this.isTUI) {
|
|
256
253
|
await registerFileReadExecutor();
|
|
254
|
+
const { createOpenCodeReader } = await import("../integrations/opencode/reader-factory.js");
|
|
255
|
+
const openCodeReader = await createOpenCodeReader().catch(() => null);
|
|
256
|
+
registerFetchMessageExecutor(createFetchMessageExecutor(
|
|
257
|
+
this.stateManager.persona_getAll.bind(this.stateManager),
|
|
258
|
+
this.stateManager.messages_get.bind(this.stateManager),
|
|
259
|
+
this.stateManager.getRoomList.bind(this.stateManager),
|
|
260
|
+
this.stateManager.getRoomMessages.bind(this.stateManager),
|
|
261
|
+
(roomId: string) => this.stateManager.getRoom(roomId)?.display_name ?? null,
|
|
262
|
+
openCodeReader ? (id, before, after) => openCodeReader.getMessageById(id, before, after) : undefined
|
|
263
|
+
));
|
|
264
|
+
} else {
|
|
265
|
+
registerFetchMessageExecutor(createFetchMessageExecutor(
|
|
266
|
+
this.stateManager.persona_getAll.bind(this.stateManager),
|
|
267
|
+
this.stateManager.messages_get.bind(this.stateManager),
|
|
268
|
+
this.stateManager.getRoomList.bind(this.stateManager),
|
|
269
|
+
this.stateManager.getRoomMessages.bind(this.stateManager),
|
|
270
|
+
(roomId: string) => this.stateManager.getRoom(roomId)?.display_name ?? null
|
|
271
|
+
));
|
|
257
272
|
}
|
|
258
273
|
this.running = true;
|
|
259
274
|
console.log(`[Processor ${this.instanceId}] initialized, starting loop`);
|
|
@@ -454,7 +469,7 @@ export class Processor {
|
|
|
454
469
|
async getGeneratedDocumentContent(slug: string): Promise<string | null> {
|
|
455
470
|
const messages = this.stateManager.messages_get("emmet");
|
|
456
471
|
const target = `generate:document:${slug}`;
|
|
457
|
-
const message = messages.find(m => m.
|
|
472
|
+
const message = messages.find(m => m.id.startsWith(`${target}:`));
|
|
458
473
|
return message?.content ?? null;
|
|
459
474
|
}
|
|
460
475
|
|
|
@@ -1009,6 +1024,104 @@ export class Processor {
|
|
|
1009
1024
|
}
|
|
1010
1025
|
}
|
|
1011
1026
|
|
|
1027
|
+
private async migrateMessageIds(): Promise<void> {
|
|
1028
|
+
try {
|
|
1029
|
+
let msgRewrites = 0;
|
|
1030
|
+
let quoteRewrites = 0;
|
|
1031
|
+
|
|
1032
|
+
const UUID_PATTERN = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i;
|
|
1033
|
+
|
|
1034
|
+
const personas = this.stateManager.persona_getAll();
|
|
1035
|
+
for (const persona of personas) {
|
|
1036
|
+
for (const msg of this.stateManager.messages_get(persona.id)) {
|
|
1037
|
+
if (!msg.external && UUID_PATTERN.test(msg.id)) {
|
|
1038
|
+
this.stateManager.messages_update(persona.id, msg.id, { id: qualifyEiMessage(msg.id) });
|
|
1039
|
+
msgRewrites++;
|
|
1040
|
+
}
|
|
1041
|
+
}
|
|
1042
|
+
}
|
|
1043
|
+
|
|
1044
|
+
const rooms = this.stateManager.getRoomList();
|
|
1045
|
+
for (const room of rooms) {
|
|
1046
|
+
for (const msg of this.stateManager.getRoomMessages(room.id).slice()) {
|
|
1047
|
+
if (UUID_PATTERN.test(msg.id)) {
|
|
1048
|
+
this.stateManager.updateRoomMessage(room.id, msg.id, { id: qualifyEiMessage(msg.id) });
|
|
1049
|
+
msgRewrites++;
|
|
1050
|
+
}
|
|
1051
|
+
}
|
|
1052
|
+
}
|
|
1053
|
+
|
|
1054
|
+
const human = this.stateManager.getHuman();
|
|
1055
|
+
const quotes = human.quotes ?? [];
|
|
1056
|
+
|
|
1057
|
+
const eiUuidMap = new Map<string, string>();
|
|
1058
|
+
for (const persona of personas) {
|
|
1059
|
+
for (const msg of this.stateManager.messages_get(persona.id)) {
|
|
1060
|
+
if (msg.id.startsWith("ei:")) eiUuidMap.set(msg.id.slice(3), msg.id);
|
|
1061
|
+
}
|
|
1062
|
+
}
|
|
1063
|
+
for (const room of rooms) {
|
|
1064
|
+
for (const msg of this.stateManager.getRoomMessages(room.id)) {
|
|
1065
|
+
if (msg.id.startsWith("ei:")) eiUuidMap.set(msg.id.slice(3), msg.id);
|
|
1066
|
+
}
|
|
1067
|
+
}
|
|
1068
|
+
|
|
1069
|
+
const MSG_PATTERN = /^msg_[a-zA-Z0-9]+$/;
|
|
1070
|
+
|
|
1071
|
+
let openCodeReader: IOpenCodeReader | null = null;
|
|
1072
|
+
if (this.isTUI) {
|
|
1073
|
+
const { createOpenCodeReader } = await import("../integrations/opencode/reader-factory.js");
|
|
1074
|
+
openCodeReader = await createOpenCodeReader().catch(() => null);
|
|
1075
|
+
}
|
|
1076
|
+
|
|
1077
|
+
const updatedQuotes: typeof quotes = [];
|
|
1078
|
+
for (const quote of quotes) {
|
|
1079
|
+
const mid = quote.message_id;
|
|
1080
|
+
if (!mid || isQualifiedMessageId(mid)) {
|
|
1081
|
+
updatedQuotes.push(quote);
|
|
1082
|
+
continue;
|
|
1083
|
+
}
|
|
1084
|
+
|
|
1085
|
+
if (MSG_PATTERN.test(mid)) {
|
|
1086
|
+
if (openCodeReader) {
|
|
1087
|
+
const ocWindow = await openCodeReader.getMessageById(mid).catch(() => null);
|
|
1088
|
+
if (ocWindow) {
|
|
1089
|
+
const { getMachineId } = await import("../integrations/machine-id.js");
|
|
1090
|
+
updatedQuotes.push({ ...quote, message_id: qualifyOpenCodeMessage(getMachineId(), ocWindow.session.id, mid) });
|
|
1091
|
+
quoteRewrites++;
|
|
1092
|
+
continue;
|
|
1093
|
+
}
|
|
1094
|
+
}
|
|
1095
|
+
updatedQuotes.push(quote);
|
|
1096
|
+
continue;
|
|
1097
|
+
}
|
|
1098
|
+
|
|
1099
|
+
if (UUID_PATTERN.test(mid)) {
|
|
1100
|
+
const fqId = eiUuidMap.get(mid);
|
|
1101
|
+
if (fqId) {
|
|
1102
|
+
updatedQuotes.push({ ...quote, message_id: fqId });
|
|
1103
|
+
quoteRewrites++;
|
|
1104
|
+
continue;
|
|
1105
|
+
}
|
|
1106
|
+
updatedQuotes.push(quote);
|
|
1107
|
+
continue;
|
|
1108
|
+
}
|
|
1109
|
+
|
|
1110
|
+
updatedQuotes.push(quote);
|
|
1111
|
+
}
|
|
1112
|
+
|
|
1113
|
+
if (quoteRewrites > 0) {
|
|
1114
|
+
this.stateManager.setHuman({ ...human, quotes: updatedQuotes });
|
|
1115
|
+
}
|
|
1116
|
+
|
|
1117
|
+
if (msgRewrites > 0 || quoteRewrites > 0) {
|
|
1118
|
+
console.log(`[Processor] migrateMessageIds: rewrote ${msgRewrites} message IDs, ${quoteRewrites} quote message_ids`);
|
|
1119
|
+
}
|
|
1120
|
+
} catch (err) {
|
|
1121
|
+
console.error("[Processor] migrateMessageIds failed, continuing:", err);
|
|
1122
|
+
}
|
|
1123
|
+
}
|
|
1124
|
+
|
|
1012
1125
|
private seedSettings(): void {
|
|
1013
1126
|
const human = this.stateManager.getHuman();
|
|
1014
1127
|
let modified = false;
|
|
@@ -1912,7 +2025,7 @@ const toolNextSteps = new Set([
|
|
|
1912
2025
|
if (isSynthesisCompletion) {
|
|
1913
2026
|
const slug = response.request.data.slug as string;
|
|
1914
2027
|
const hasContent = slug && this.stateManager.messages_get("emmet")
|
|
1915
|
-
.some(m => m.
|
|
2028
|
+
.some(m => m.id.startsWith(`generate:document:${slug}:`));
|
|
1916
2029
|
if (hasContent) this.interface.onDocumentGenerated?.(slug);
|
|
1917
2030
|
}
|
|
1918
2031
|
} catch (err) {
|
|
@@ -2,6 +2,7 @@ import type { ToolExecutor } from "../types.js";
|
|
|
2
2
|
import type { Message } from "../../types.js";
|
|
3
3
|
import type { RoomMessage, RoomSummary } from "../../types/rooms.js";
|
|
4
4
|
import type { PersonaEntity } from "../../types/entities.js";
|
|
5
|
+
import type { OpenCodeMessageWindow } from "../../../integrations/opencode/types.js";
|
|
5
6
|
|
|
6
7
|
interface CleanMessage {
|
|
7
8
|
id: string;
|
|
@@ -17,6 +18,9 @@ type GetPersonaMessages = (personaId: string) => Message[];
|
|
|
17
18
|
type GetRoomList = () => RoomSummary[];
|
|
18
19
|
type GetRoomMessages = (roomId: string) => RoomMessage[];
|
|
19
20
|
type GetRoomDisplayName = (roomId: string) => string | null;
|
|
21
|
+
type GetOpenCodeMessage = (id: string, before: number, after: number) => Promise<OpenCodeMessageWindow | null>;
|
|
22
|
+
|
|
23
|
+
const OPENCODE_MESSAGE_ID = /^msg_[a-zA-Z0-9]+$/;
|
|
20
24
|
|
|
21
25
|
function stripMessage(m: Message): CleanMessage {
|
|
22
26
|
return {
|
|
@@ -45,7 +49,8 @@ export function createFetchMessageExecutor(
|
|
|
45
49
|
getPersonaMessages: GetPersonaMessages,
|
|
46
50
|
getRoomList: GetRoomList,
|
|
47
51
|
getRoomMessages: GetRoomMessages,
|
|
48
|
-
getRoomDisplayName: GetRoomDisplayName
|
|
52
|
+
getRoomDisplayName: GetRoomDisplayName,
|
|
53
|
+
getOpenCodeMessage?: GetOpenCodeMessage
|
|
49
54
|
): ToolExecutor {
|
|
50
55
|
return {
|
|
51
56
|
name: "fetch_message",
|
|
@@ -62,6 +67,27 @@ export function createFetchMessageExecutor(
|
|
|
62
67
|
return JSON.stringify({ error: "Missing required argument: id" });
|
|
63
68
|
}
|
|
64
69
|
|
|
70
|
+
if (OPENCODE_MESSAGE_ID.test(id)) {
|
|
71
|
+
if (!getOpenCodeMessage) {
|
|
72
|
+
return JSON.stringify({ error: "OpenCode message lookup not available in this runtime", id });
|
|
73
|
+
}
|
|
74
|
+
const window = await getOpenCodeMessage(id, before, after);
|
|
75
|
+
if (!window) {
|
|
76
|
+
return JSON.stringify({
|
|
77
|
+
error: "OpenCode message not found on this machine. It may exist on another device.",
|
|
78
|
+
id,
|
|
79
|
+
hint: "Check the linked topic's sources for the originating machine and session.",
|
|
80
|
+
});
|
|
81
|
+
}
|
|
82
|
+
return JSON.stringify({
|
|
83
|
+
message: { id: window.message.id, role: window.message.role, content: window.message.content, timestamp: window.message.timestamp, agent: window.message.agent },
|
|
84
|
+
before: window.before.map(m => ({ id: m.id, role: m.role, content: m.content, timestamp: m.timestamp, agent: m.agent })),
|
|
85
|
+
after: window.after.map(m => ({ id: m.id, role: m.role, content: m.content, timestamp: m.timestamp, agent: m.agent })),
|
|
86
|
+
session: { id: window.session.id, title: window.session.title, directory: window.session.directory },
|
|
87
|
+
source: "opencode",
|
|
88
|
+
});
|
|
89
|
+
}
|
|
90
|
+
|
|
65
91
|
const personas = getAllPersonas();
|
|
66
92
|
|
|
67
93
|
// TODO: add persona access gate when calling context is available —
|