@kisev/memomatic 1.0.0-dev.72.g0dfe98eca874 → 1.0.0-dev.74.ge851c073615d

package/README.md

@@ -4,8 +4,8 @@
 
 `@kisev/memomatic` is a personal learning memory for agents following the OpenClaw
 memory architecture: a tiered Markdown corpus you can read as plain files, a
-rebuildable SQLite index with FTS5 and optional local embeddings, and a nightly
-`dream` consolidation sweep.
+rebuildable SQLite index with FTS5 and optional local embeddings, a `sessions`
+extraction pass, and a nightly `dream` consolidation sweep.
 
 ## Install
 
@@ -21,24 +21,32 @@ automatic context injection is used. For background processing, run the CLI:
 
 ```bash
 memomatic process    # validate the inbox and index accepted entries (no model)
-memomatic dream      # full sweep: inbox + sessions + consolidation
+memomatic sessions   # extract episodic memory from OpenCode sessions (model)
+memomatic dream      # consolidation: inbox + promotion + bounded rewrite + archive
 ```
 
-The nightly sweep is scheduled by the systemd user units in `assets/systemd/`
-(`memomatic-dream.service` and `memomatic-dream.timer`).
+Nightly runs are scheduled by the systemd user units in `assets/systemd/`
+(`memomatic-sessions.service`/`.timer` for extraction and
+`memomatic-dream.service`/`.timer` for consolidation).
 
-Configure `dream.model` and an OpenCode provider before expecting session
-extraction; without a model, the session watermark is preserved. Follow the
+Configure `sessions.model` (or keep a legacy `dream.model`) and an OpenCode
+provider before expecting session extraction; without a model, the session
+watermark is preserved. Follow the
 [setup, first-entry, and scheduling guide](../../docs/how-to/memomatic.md).
 
-## CLI and Dream
+## CLI, Sessions, and Dream
 
 See [CLI conventions](../../docs/reference/cli.md) and
 [Dream controls](../../docs/how-to/memomatic.md#observe-limit-and-resume-dream).
-`dream --plan` inspects pending work without a model; `dream --dry-run` may call
-models. Normal Dream drains the eligible snapshot with per-fragment checkpoints,
-reusing one OpenCode server and unchanged embeddings. The first legacy-cursor
-migration checks prior history once without discarding existing memory.
+`sessions --plan` inspects pending session work without a model; `dream --plan`
+counts pending inbox files and promotion candidates. `--dry-run` previews
+either command and may still call models. Sessions drain the eligible snapshot
+with per-fragment checkpoints, reusing one OpenCode server and unchanged
+embeddings; Dream then promotes usage-gated episodic entries into `MEMORY.md`
+through bounded consolidation and archives old entries. The first
+legacy-cursor migration checks prior history once without discarding existing
+memory. `status` shows the queue: pending inbox files, session backlog,
+promotion candidates, and the last run of each sweep.
 
 ## Inbox
 
@@ -63,13 +71,14 @@ is exposed in search responses.
 
 ## Surfaces
 
-- `memomatic` CLI: `process`, `dream`, `search`, `status`, `index`.
+- `memomatic` CLI: `process`, `sessions`, `dream`, `search`, `status`, `index`.
 - MCP stdio server with `memory_search`, `memory_get`, `memory_write`, and
   `memory_forget` tools.
 
-The model initiates memory searches through visible MCP tool calls. Dream
-processes OpenCode history separately; connecting another host does not import
-its session history. Agentomatic and memomatic are installed independently.
+The model initiates memory searches through visible MCP tool calls. The
+`sessions` command processes OpenCode history separately; connecting another
+host does not import its session history. Agentomatic and memomatic are
+installed independently.
 
 Nothing is deleted without the explicit directives documented in
 `MEMORY_RULES.md`.

package/README.ru.md

@@ -4,8 +4,8 @@
 
 `@kisev/memomatic` — персональная обучающая память для агентов по архитектуре
 памяти OpenClaw: ярусный Markdown-корпус, который можно читать как обычные файлы,
-пересобираемый SQLite-индекс с FTS5 и опциональными локальными эмбеддингами и
-ночной проход консолидации `dream`.
+пересобираемый SQLite-индекс с FTS5 и опциональными локальными эмбеддингами,
+проход извлечения `sessions` и ночной проход консолидации `dream`.
 
 ## Установка
 
@@ -21,24 +21,32 @@ npm install --global @kisev/memomatic
 
 ```bash
 memomatic process    # валидация inbox и индексация принятого (без модели)
-memomatic dream      # полный проход: inbox + сессии + консолидация
+memomatic sessions   # извлечение эпизодической памяти из сессий OpenCode (модель)
+memomatic dream      # консолидация: inbox + продвижение + перезапись + архив
 ```
 
-Ночной проход планируют user-юниты systemd из `assets/systemd/`
-(`memomatic-dream.service` и `memomatic-dream.timer`).
+Ночные проходы планируют user-юниты systemd из `assets/systemd/`
+(`memomatic-sessions.service`/`.timer` для извлечения и
+`memomatic-dream.service`/`.timer` для консолидации).
 
-Перед извлечением сессий настройте `dream.model` и провайдера OpenCode;
-без модели отметка обработанных сессий сохраняется. См.
+Перед извлечением сессий настройте `sessions.model` (или оставьте старый
+`dream.model`) и провайдера OpenCode; без модели отметка обработанных сессий
+сохраняется. См.
 [настройку, первую запись и расписание](../../docs/ru/how-to/memomatic.md).
 
-## CLI и Dream
+## CLI, Sessions и Dream
 
 См. [соглашения CLI](../../docs/ru/reference/cli.md) и
 [управление Dream](../../docs/ru/how-to/memomatic.md#наблюдение-ограничения-и-продолжение-dream).
-`dream --plan` читает очередь без модели; `dream --dry-run` может вызывать модели.
-Обычный Dream обрабатывает весь подходящий снимок с контрольными точками фрагментов,
-переиспользуя один сервер OpenCode и неизменённые эмбеддинги. Первая миграция старого
-курсора перепроверяет историю один раз, сохраняя накопленную память.
+`sessions --plan` читает очередь сессий без модели; `dream --plan` считает
+ожидающие файлы inbox и кандидатов на продвижение. `--dry-run` показывает
+превью любой команды и может вызывать модели. Sessions обрабатывают весь
+подходящий снимок с контрольными точками фрагментов, переиспользуя один сервер
+OpenCode и неизменённые эмбеддинги; Dream затем продвигает эпизодические записи,
+прошедшие пороги использования, в `MEMORY.md` через ограниченную консолидацию и
+архивирует старое. Первая миграция старого курсора перепроверяет историю один
+раз, сохраняя накопленную память. `status` показывает очередь: ожидающие файлы
+inbox, бэклог сессий, кандидатов на продвижение и время последнего прохода.
 
 ## Inbox
 
@@ -63,12 +71,12 @@ memomatic dream      # полный проход: inbox + сессии + кон
 
 ## Поверхности
 
-- CLI `memomatic`: `process`, `dream`, `search`, `status`, `index`.
+- CLI `memomatic`: `process`, `sessions`, `dream`, `search`, `status`, `index`.
 - MCP-сервер stdio с инструментами `memory_search`, `memory_get`,
   `memory_write` и `memory_forget`.
 
-Модель сама инициирует поиск памяти через видимые MCP-вызовы. Dream отдельно
-обрабатывает историю OpenCode; подключение другого хоста не импортирует его
-сессии. Agentomatic и memomatic устанавливаются независимо.
+Модель сама инициирует поиск памяти через видимые MCP-вызовы. Команда
+`sessions` отдельно обрабатывает историю OpenCode; подключение другого хоста
+не импортирует его сессии. Agentomatic и memomatic устанавливаются независимо.
 
 Без явных директив, описанных в `MEMORY_RULES.md`, ничего не удаляется.

package/assets/systemd/memomatic-dream.timer

@@ -1,8 +1,8 @@
 [Unit]
-Description=Run the Memomatic dream sweep nightly
+Description=Run the Memomatic dream sweep nightly after session extraction
 
 [Timer]
-OnCalendar=*-*-* 03:30:00
+OnCalendar=*-*-* 04:30:00
 RandomizedDelaySec=30m
 Persistent=true
 

package/assets/systemd/memomatic-sessions.service

@@ -0,0 +1,9 @@
+[Unit]
+Description=Memomatic sessions: extract episodic memory from OpenCode sessions
+Documentation=https://github.com/kisev/skills#memomatic
+After=network-online.target
+
+[Service]
+Type=oneshot
+ExecStart=/bin/sh -lc 'exec memomatic sessions'
+Environment=MEMOMATIC_SESSIONS=1

package/assets/systemd/memomatic-sessions.timer

@@ -0,0 +1,10 @@
+[Unit]
+Description=Run the Memomatic session extraction early nightly
+
+[Timer]
+OnCalendar=*-*-* 02:00:00
+RandomizedDelaySec=30m
+Persistent=true
+
+[Install]
+WantedBy=timers.target

package/dist/cli.js

@@ -1,13 +1,16 @@
 #!/usr/bin/env node
 import { readFileSync } from "node:fs";
+import { readdir, stat } from "node:fs/promises";
 import { isAbsolute } from "node:path";
 import { openMemomatic, rebuildIndex, searchMemory } from "./service.js";
 import { runDream } from "./dream.js";
+import { runSessions } from "./sessions.js";
 import { processInbox, withRunLock } from "./inbox.js";
 import { OpenCodeExecutor } from "./executor.js";
 import { normalizeSettings } from "./settings.js";
 import { planIngestion } from "./ingestion.js";
 import { opencodeDatabasePath } from "./ingest.js";
+import { promotionCandidates, relevanceByFts } from "./gates.js";
 import { memomaticPaths } from "./paths.js";
 import { program, common, Option, configFile, applyConfig, reporter, cancellation, duration, integer, fail, } from "./generated/cli.js";
 const version = JSON.parse(readFileSync(new URL("../package.json", import.meta.url), "utf8")).version;
@@ -49,26 +52,35 @@ const run = async (command, action) => {
         context = await openMemomatic({
             readOnly: opts.dryRun || opts.plan || command.name() === "status",
             configure: (settings) => {
+                const sessions = { ...settings.sessions };
                 const dream = { ...settings.dream };
                 for (const [flag, key] of [
                     ["model", "model"],
                     ["variant", "variant"],
                     ["timeout", "timeoutMs"],
                     ["maxDuration", "maxDurationMs"],
+                    ["retries", "retries"],
+                ]) {
+                    if (opts[flag] !== undefined) {
+                        Object.assign(sessions, { [key]: opts[flag] });
+                        Object.assign(dream, { [key]: opts[flag] });
+                    }
+                }
+                for (const [flag, key] of [
                     ["maxSessions", "maxSessions"],
                     ["chunkChars", "maxChars"],
-                    ["retries", "retries"],
                     ["idle", "idleMs"],
                     ["opencodeUrl", "opencodeUrl"],
                 ])
                     if (opts[flag] !== undefined)
-                        Object.assign(dream, { [key]: opts[flag] });
-                return normalizeSettings({ ...settings, dream });
+                        Object.assign(sessions, { [key]: opts[flag] });
+                return normalizeSettings({ ...settings, sessions, dream });
             },
         });
-        const signal = context.settings.dream.maxDurationMs && !opts.maxDuration
-            ? AbortSignal.any([limited, AbortSignal.timeout(context.settings.dream.maxDurationMs)])
+        const deadline = (section) => context.settings[section].maxDurationMs && !opts.maxDuration
+            ? AbortSignal.any([limited, AbortSignal.timeout(context.settings[section].maxDurationMs)])
             : limited;
+        const signal = command.name() === "sessions" ? deadline("sessions") : deadline("dream");
         await action(context, opts, { signal, observe: log.emit });
     }
     finally {
@@ -78,39 +90,61 @@ const run = async (command, action) => {
     }
 };
 const output = (value, opts, summary) => console.log(opts.json || !process.stdout.isTTY ? JSON.stringify(value, null, 2) : summary);
-const dream = add("dream", "Drain the session snapshot, checkpointing completed fragments")
-    .option("--plan", "count pending work without model calls or writes")
-    .option("--dry-run", "preview without persistent writes (model calls still occur)")
-    .addOption(new Option("--model <provider/model>", "OpenCode model; defaults to settings.dream.model").env("MEMOMATIC_MODEL"))
+const modelOptions = (command) => command
+    .addOption(new Option("--model <provider/model>", "OpenCode model; defaults to settings").env("MEMOMATIC_MODEL"))
     .addOption(new Option("--variant <name>", "provider reasoning variant").env("MEMOMATIC_VARIANT"))
     .addOption(new Option("--timeout <duration>", "per-model-call timeout (default 3m)")
     .argParser(duration)
     .env("MEMOMATIC_TIMEOUT"))
-    .addOption(new Option("--max-duration <duration>", "whole-run deadline; 0 means drain the snapshot")
+    .addOption(new Option("--max-duration <duration>", "whole-run deadline; 0 means unlimited")
     .argParser(duration)
     .env("MEMOMATIC_MAX_DURATION"))
+    .addOption(new Option("--retries <count>", "additional model attempts (default 1)")
+    .argParser(integer)
+    .env("MEMOMATIC_RETRIES"));
+const executor = (context, runtime) => {
+    const sections = {
+        sessions: context.settings.sessions,
+        dream: context.settings.dream,
+    };
+    const create = (section) => {
+        const config = sections[section];
+        return config.model
+            ? new OpenCodeExecutor({
+                model: config.model,
+                variant: config.variant,
+                url: "opencodeUrl" in config ? config.opencodeUrl : null,
+                timeoutMs: config.timeoutMs,
+                retries: config.retries,
+                ...runtime,
+            })
+            : null;
+    };
+    return create;
+};
+const sessions = add("sessions", "Drain the OpenCode session snapshot into episodic memory, checkpointing fragments")
+    .option("--plan", "count pending work without model calls or writes")
+    .option("--dry-run", "preview without persistent writes (model calls still occur)");
+modelOptions(sessions)
     .addOption(new Option("--max-sessions <count>", "optional session limit; 0 means all")
     .argParser(integer)
     .env("MEMOMATIC_MAX_SESSIONS"))
     .addOption(new Option("--chunk-chars <count>", "maximum new text per fragment (default 24000)")
     .argParser(integer)
     .env("MEMOMATIC_CHUNK_CHARS"))
-    .addOption(new Option("--retries <count>", "additional model attempts (default 1)")
-    .argParser(integer)
-    .env("MEMOMATIC_RETRIES"))
     .addOption(new Option("--idle <duration>", "minimum session idle age (default 10m)")
     .argParser(duration)
     .env("MEMOMATIC_IDLE"))
     .addOption(new Option("--opencode-url <url>", "reuse an existing OpenCode server; otherwise start one per run").env("MEMOMATIC_OPENCODE_URL"))
     .addOption(new Option("--database <path>", "OpenCode SQLite session database").env("MEMOMATIC_DATABASE"))
-    .addHelpText("after", "\nExamples:\n  memomatic dream\n  memomatic dream --max-duration 15m --log-format json\n  memomatic dream --dry-run --max-sessions 2\n\nCompleted fragments survive cancellation. Logs use stderr; --json keeps stdout machine-readable.");
-dream.action(async () => run(dream, async (context, opts, runtime) => {
+    .addHelpText("after", "\nExamples:\n  memomatic sessions\n  memomatic sessions --max-duration 15m --log-format json\n  memomatic sessions --dry-run --max-sessions 2\n\nCompleted fragments survive cancellation. Logs use stderr; --json keeps stdout machine-readable.");
+sessions.action(async () => run(sessions, async (context, opts, runtime) => {
     if (opts.plan) {
         const start = Date.now();
         const plan = planIngestion(opts.database ?? opencodeDatabasePath(), context.store, {
-            before: Date.now() - context.settings.dream.idleMs,
-            maxChars: context.settings.dream.maxChars,
-            maxSessions: context.settings.dream.maxSessions,
+            before: Date.now() - context.settings.sessions.idleMs,
+            maxChars: context.settings.sessions.maxChars,
+            maxSessions: context.settings.sessions.maxSessions,
         });
         const summary = {
             sessions: plan.sessions,
@@ -125,27 +159,43 @@ dream.action(async () => run(dream, async (context, opts, runtime) => {
         output(summary, opts, `${summary.sessions} sessions, ${summary.fragments} fragments, ${summary.characters} characters (${summary.scanMs}ms scan). No model calls.`);
         return;
     }
-    const executor = context.settings.dream.model
-        ? new OpenCodeExecutor({
-            model: context.settings.dream.model,
-            variant: context.settings.dream.variant,
-            url: context.settings.dream.opencodeUrl,
-            timeoutMs: context.settings.dream.timeoutMs,
-            retries: context.settings.dream.retries,
-            ...runtime,
-        })
-        : null;
+    const create = executor(context, runtime);
+    const model = create("sessions");
     try {
-        const execute = () => runDream(context, executor, {
+        const execute = () => runSessions(context, model, {
             ...runtime,
             dryRun: opts.dryRun,
             databaseFile: opts.database,
         });
         const result = opts.dryRun ? await execute() : await withRunLock(context.paths, execute);
-        output({ ...result, modelUsage: executor?.usage ?? null }, opts, `Dream complete: ${result.sessionsIngested} sessions, ${result.candidatesExtracted} candidates, ${result.elapsedMs}ms`);
+        output({ ...result, modelUsage: model?.usage ?? null }, opts, `Sessions complete: ${result.sessionsIngested} sessions, ${result.candidatesExtracted} candidates, ${result.elapsedMs}ms`);
     }
     finally {
-        await executor?.close();
+        await model?.close();
+    }
+}));
+const dream = add("dream", "Consolidate memory: inbox, promotion, bounded rewrite, archive")
+    .option("--plan", "count pending work without model calls or writes")
+    .option("--dry-run", "preview without persistent writes (model calls still occur)");
+modelOptions(dream).addHelpText("after", "\nExamples:\n  memomatic dream\n  memomatic dream --max-duration 15m --log-format json\n  memomatic dream --dry-run\n\nSession extraction lives in `memomatic sessions`. Logs use stderr; --json keeps stdout machine-readable.");
+dream.action(async () => run(dream, async (context, opts, runtime) => {
+    if (opts.plan) {
+        const relevance = relevanceByFts(context.store, context.settings);
+        const candidates = promotionCandidates(context.store, context.settings, relevance);
+        const inboxFiles = (await readdir(context.paths.inboxDir).catch(() => [])).filter((name) => name.endsWith(".md")).length;
+        const summary = { inboxFiles, promotionCandidates: candidates.length };
+        output(summary, opts, `${summary.inboxFiles} inbox files, ${summary.promotionCandidates} promotion candidates. No model calls.`);
+        return;
+    }
+    const create = executor(context, runtime);
+    const model = create("dream");
+    try {
+        const execute = () => runDream(context, model, { ...runtime, dryRun: opts.dryRun });
+        const result = opts.dryRun ? await execute() : await withRunLock(context.paths, execute);
+        output({ ...result, modelUsage: model?.usage ?? null }, opts, `Dream complete: ${result.promoted.length} promoted, ${result.superseded.length} superseded, ${result.elapsedMs}ms`);
+    }
+    finally {
+        await model?.close();
     }
 }));
 const processCommand = add("process", "Process explicit inbox entries without LLM extraction").option("--dry-run", "preview without writes");
@@ -162,12 +212,54 @@ index.action(() => run(index, async (context, opts, runtime) => {
     else
         console.log(`${entries} entries indexed`);
 }));
-const status = add("status", "Read corpus/index status without indexing or model calls");
-status.action(() => run(status, async (context, opts) => output({
-    entries: context.store.allEntries().length,
-    fts5: context.store.hasFts(),
-    stateRoot: context.paths.stateRoot,
-}, opts, `${context.store.allEntries().length} indexed entries in ${context.paths.stateRoot}`)));
+const status = add("status", "Read corpus/index/queue status without indexing or model calls");
+status.action(() => run(status, async (context, opts, runtime) => {
+    const inboxFiles = (await readdir(context.paths.inboxDir).catch(() => [])).filter((name) => name.endsWith(".md")).length;
+    let sessionBacklog = null;
+    const databaseFile = opencodeDatabasePath();
+    if (await stat(databaseFile).catch(() => null)) {
+        try {
+            const plan = planIngestion(databaseFile, context.store, {
+                before: Date.now() - context.settings.sessions.idleMs,
+                maxChars: context.settings.sessions.maxChars,
+                maxSessions: context.settings.sessions.maxSessions,
+            });
+            sessionBacklog = {
+                sessions: plan.sessions,
+                messages: plan.messages,
+                fragments: plan.fragments.length,
+                characters: plan.characters,
+                cachedFragments: plan.cached,
+                cachedSessions: plan.cachedSessions,
+                unscannedSessions: plan.unscannedSessions,
+            };
+        }
+        catch (error) {
+            runtime.observe({
+                phase: "status.sessions",
+                message: `Cannot read OpenCode sessions: ${error.message}`,
+                level: "warn",
+            });
+        }
+    }
+    const relevance = relevanceByFts(context.store, context.settings);
+    const candidates = promotionCandidates(context.store, context.settings, relevance);
+    const report = {
+        entries: context.store.allEntries().length,
+        fts5: context.store.hasFts(),
+        stateRoot: context.paths.stateRoot,
+        queue: {
+            inboxFiles,
+            sessions: sessionBacklog,
+            promotionCandidates: candidates.length,
+        },
+        lastRun: {
+            dream: context.store.getMeta("last-dream-at"),
+            sessions: context.store.getMeta("last-sessions-at"),
+        },
+    };
+    output(report, opts, `${report.entries} indexed entries, ${inboxFiles} inbox files, ${sessionBacklog ? `${sessionBacklog.fragments} pending fragments` : "no session database"}, ${candidates.length} promotion candidates in ${context.paths.stateRoot}`);
+}));
 const search = add("search <query...>", "Find relevant memory or return no matches")
     .option("--explain", "include relevance diagnostics")
     .addOption(new Option("--project <name>", "restrict to a project and user-level memory").env("MEMOMATIC_PROJECT"));

package/dist/dream.d.ts

@@ -2,42 +2,20 @@ import { type InboxReport } from "./inbox.js";
 import { type ModelExecutor } from "./executor.js";
 import { type OperationOptions } from "./operations.js";
 import type { MemomaticContext } from "./service.js";
+export declare const LAST_RUN_KEY = "last-dream-at";
 export type DreamReport = {
     inbox: InboxReport;
-    sessionsIngested: number;
-    candidatesExtracted: number;
     promoted: string[];
     superseded: string[];
     dropped: string[];
-    rejected: Array<{
-        text: string;
-        reason: string;
-    }>;
     archived: string[];
     appendOnlyFallback: boolean;
     dryRun: boolean;
-    ingestion?: {
-        messages: number;
-        fragments: number;
-        completed: number;
-        cached: number;
-        characters: number;
-        remaining: number;
-        cachedSessions: number;
-        unscannedSessions: number;
-    };
     elapsedMs?: number;
 };
 export type DreamOptions = OperationOptions & {
     dryRun?: boolean;
-    databaseFile?: string;
 };
-export type ExtractedCandidate = {
-    text: string;
-    key: string | null;
-    reason: string;
-};
-export declare function parseExtraction(response: string): ExtractedCandidate[];
 export type ConsolidationOperation = {
     op: "add";
     line: string;