memoryrouter-codex 1.2.1 → 1.4.0

package/README.md

@@ -128,9 +128,21 @@ Automatic memories use an envelope containing project handle, stable memory hand
 
 Set `retrievalScope` to `"vault"` in the private config only if you intentionally want unscoped automatic recall.
 
-## Historical import boundary
+## Historical import (agent-driven)
 
-The Codex package captures future work after installation. It does not ship a deterministic importer for ChatGPT exports or prior Codex conversations. A local Codex workflow can inspect an extracted export and curate selected facts, but that is experimental guided curation, not checkpointed/resumable/reconciled archive ingestion. Remote MCP OAuth connection by itself never imports history.
+Prior Codex conversations on this machine can be imported with a strict division of labor: the server owns the extraction prompt (versioned, fetched at import time), Codex itself owns discovery/parsing of its own session files, and this package owns transport. The agent never uploads; this CLI never parses history.
+
+```bash
+memoryrouter-codex import --instructions        # print the server's extraction prompt
+memoryrouter-codex import --instructions --run  # or run it headless via codex exec
+memoryrouter-codex import --file extract.jsonl  # validate, preview, quote, approve, upload
+memoryrouter-codex import --file extract.jsonl --yes   # non-interactive approval
+memoryrouter-codex import --dismiss             # permanently silence the import nudge
+```
+
+Guarantees: the extract is validated locally (four fields per record, `codex:` prefixed ids, epoch-ms timestamps); the CLI shows record count, date range, estimated tokens, and the server price quote and requires explicit approval before any write; the import id is content-derived and every batch carries an `Idempotency-Key`, so re-running after any interruption replays committed batches for free and resumes. Duplicates are never re-billed. The API key never appears in the prompt or the extract file. When a vault is new, injected context includes a one-line import suggestion until an import completes or you `--dismiss`.
+
+ChatGPT archive imports remain a separate flow (the `memoryrouter` ChatGPT CLI). Remote MCP OAuth connection by itself never imports history.
 
 ## Handles, deduplication, retries, and forgetting
 
@@ -153,6 +165,8 @@ MemoryRouter's Reflection Hierarchy consolidates raw memories into higher-tier r
 
 Reflection prompts are always server-authored. This package never writes its own consolidation instructions and refuses to reflect if the server response carries none. See https://docs.memoryrouter.ai/reflections for endpoint semantics, lease rules, and billing.
 
+**Automatic consolidation (default on):** when a hook's retrieval response reports debt above the server's threshold, the hooks spawn the same reflect loop as a detached background process (`codex exec` headless, your own inference). Rails: max 3 batches per trigger with a debt re-check between batches, a machine-wide lockfile at `~/.memoryrouter/consolidate.lock` (stale after 20 minutes), a 30 minute cooldown after each completed run, and silent failure with logs at `~/.memoryrouter/consolidate.log`. The threshold always comes from the server response. Opt out with `"autoConsolidate": false` in `~/.memoryrouter/codex/config.json`; the suggestion line, stdio tools, and manual `reflect` keep working.
+
 ## Context budget and failure behavior
 
 - Default injected context cap: 12,000 characters, assembled at memory boundaries.

package/bin/cli.cjs

@@ -23,7 +23,7 @@ function parseArgs(argv) {
 }
 
 function usage() {
-  return `memoryrouter-codex ${INTEGRATION_VERSION}\n\nUsage:\n  memoryrouter-codex init [--scope user|project] [--auth oauth|api-key]\n                          [--project-dir PATH] [--api-key-stdin]\n                          [--no-hooks] [--no-agents]\n  memoryrouter-codex doctor [--scope user|project] [--project-dir PATH] [--offline]\n  memoryrouter-codex uninstall [--scope user|project] [--project-dir PATH]\n  memoryrouter-codex reflect [--tier 1|2] [--batches N|all] [--max-tokens N] [--dry-run]\n  memoryrouter-codex mcp-server\n\nReflect:\n  Consolidates unconsolidated memories into reflections using Codex itself as\n  the model (codex exec, you pay your own inference). Server-authored\n  instructions; resumable; no local state. --dry-run prints the checkout and\n  lets the lease expire without committing.\n\nAuth modes:\n  oauth    Current remote MCP at ${'https://mcp.memoryrouter.ai/mcp'} (default).\n           Run \`codex mcp login memoryrouter\` after init. Hooks additionally\n           require MEMORYROUTER_API_KEY or a key saved with --api-key-stdin.\n  api-key  Local stdio MCP fallback plus automatic hooks. The key is read only\n           from the environment/private config, never from project files.\n\nSecure key setup:\n  printf '%s' "$MEMORYROUTER_API_KEY" | memoryrouter-codex init --api-key-stdin\n`;
+  return `memoryrouter-codex ${INTEGRATION_VERSION}\n\nUsage:\n  memoryrouter-codex init [--scope user|project] [--auth oauth|api-key]\n                          [--project-dir PATH] [--api-key-stdin]\n                          [--no-hooks] [--no-agents]\n  memoryrouter-codex doctor [--scope user|project] [--project-dir PATH] [--offline]\n  memoryrouter-codex uninstall [--scope user|project] [--project-dir PATH]\n  memoryrouter-codex reflect [--tier 1|2] [--batches N|all] [--max-tokens N] [--dry-run]\n  memoryrouter-codex import [--instructions [--run]] [--file PATH] [--dismiss] [--yes]\n  memoryrouter-codex mcp-server\n\nImport:\n  Import historical Codex conversations into your vault. --instructions\n  fetches the server's extraction prompt for the host agent to execute\n  (--run spawns it via codex exec); --file uploads a JSONL extract with a\n  preview, price quote, explicit approval (--yes for non-interactive),\n  idempotent batches, and resume; --dismiss silences the import nudge\n  permanently. Re-running --file is always safe: committed batches replay\n  for free and duplicates are never re-billed.\n\nReflect:\n  Consolidates unconsolidated memories into reflections using Codex itself as\n  the model (codex exec, you pay your own inference). Server-authored\n  instructions; resumable; no local state. --dry-run prints the checkout and\n  lets the lease expire without committing.\n  Automatic mode: when the server reports consolidation debt above its\n  threshold, the hooks spawn this same loop in the background (max 3 batches\n  per trigger, 30 minute cooldown, machine-wide lockfile at\n  ~/.memoryrouter/consolidate.lock). Opt out with "autoConsolidate": false in\n  ~/.memoryrouter/codex/config.json. Logs: ~/.memoryrouter/consolidate.log\n\nAuth modes:\n  oauth    Current remote MCP at ${'https://mcp.memoryrouter.ai/mcp'} (default).\n           Run \`codex mcp login memoryrouter\` after init. Hooks additionally\n           require MEMORYROUTER_API_KEY or a key saved with --api-key-stdin.\n  api-key  Local stdio MCP fallback plus automatic hooks. The key is read only\n           from the environment/private config, never from project files.\n\nSecure key setup:\n  printf '%s' "$MEMORYROUTER_API_KEY" | memoryrouter-codex init --api-key-stdin\n`;
 }
 
 function optionsFromArgs(args) {
@@ -77,6 +77,38 @@ async function main() {
     process.exitCode = result.ok ? 0 : 1;
     return;
   }
+  if (command === 'reflect-auto') {
+    // Internal entrypoint for the detached automatic consolidation run
+    // spawned by the hooks. Silent to the user's session; logs to
+    // ~/.memoryrouter/consolidate.log. Always exits 0.
+    const { runAutoConsolidate } = require('../runtime/auto.cjs');
+    try { await runAutoConsolidate(); } catch { /* logged */ }
+    process.exitCode = 0;
+    return;
+  }
+  if (command === 'import') {
+    const { loadConfig, getApiKey } = require('../runtime/config.cjs');
+    const { runInstructions, runDismiss, runImportFile } = require('../runtime/import.cjs');
+    const config = loadConfig(process.env);
+    const key = getApiKey(process.env, config);
+    if (!key) throw new Error('MemoryRouter API key is not configured. Run `memoryrouter-codex init` or set MEMORYROUTER_API_KEY.');
+    const modes = [Boolean(args.instructions), args.file !== undefined, Boolean(args.dismiss)].filter(Boolean).length;
+    if (modes > 1) throw new Error('Choose one of --instructions, --file <path>, or --dismiss');
+    if (args.dismiss) {
+      await runDismiss({ key });
+      return;
+    }
+    if (args.file) {
+      await runImportFile({ key, file: String(args.file), yes: Boolean(args.yes) });
+      return;
+    }
+    await runInstructions({ key, spawn: Boolean(args.run) });
+    if (!args.run) {
+      process.stdout.write('\nExecute the prompt above with your agent (or re-run with --instructions --run to spawn codex exec).\n');
+      process.stdout.write('Then upload the extract: memoryrouter-codex import --file <path>\n');
+    }
+    return;
+  }
   const options = optionsFromArgs(args);
   if (command === 'init') {
     const result = init(options);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "memoryrouter-codex",
-  "version": "1.2.1",
+  "version": "1.4.0",
   "description": "Automatic, project-scoped MemoryRouter memory for OpenAI Codex",
   "license": "MIT",
   "type": "commonjs",

package/runtime/api.cjs

@@ -8,6 +8,7 @@ async function request(method, endpoint, options = {}) {
   const {
     key,
     body,
+    rawBody,
     headers = {},
     timeoutMs = 7000,
     retries = method === 'GET' || endpoint.endsWith('/search') ? 2 : 0,
@@ -26,7 +27,7 @@ async function request(method, endpoint, options = {}) {
           'User-Agent': `memoryrouter-codex/${INTEGRATION_VERSION}`,
           ...headers,
         },
-        body: body === undefined ? undefined : JSON.stringify(body),
+        body: rawBody !== undefined ? rawBody : body === undefined ? undefined : JSON.stringify(body),
         signal: controller.signal,
       });
       let data = null;
@@ -146,6 +147,59 @@ function debtProbe(key, options = {}) {
   }, options);
 }
 
+// Historical import wrappers. The server owns the extraction prompt; the CLI
+// owns transport. Create/approve/batches run with retries: 0 at this layer;
+// the import loop applies its own bounded backoff around batch uploads.
+
+function importInstructions(key, platform, options = {}) {
+  const query = `platform=${encodeURIComponent(platform)}`
+    + (options.outputPath ? `&output_path=${encodeURIComponent(options.outputPath)}` : '');
+  return request('GET', `/v1/memory/imports/instructions?${query}`, {
+    ...options, key, timeoutMs: options.timeoutMs || 15000, retries: 1,
+  });
+}
+
+function importCreate(key, body, options = {}) {
+  return request('POST', '/v1/memory/imports', {
+    ...options, key, body, timeoutMs: options.timeoutMs || 20000, retries: 0,
+  });
+}
+
+function importApprove(key, importId, approvalDigest, options = {}) {
+  return request('POST', `/v1/memory/imports/${encodeURIComponent(importId)}/approve`, {
+    ...options, key, body: { approval_digest: approvalDigest }, timeoutMs: options.timeoutMs || 20000, retries: 0,
+  });
+}
+
+function importBatch(key, importId, batchIndex, jsonl, idempotencyKey, options = {}) {
+  return request('POST', `/v1/memory/imports/${encodeURIComponent(importId)}/batches/${batchIndex}`, {
+    ...options,
+    key,
+    rawBody: jsonl,
+    headers: { 'Content-Type': 'application/x-ndjson', 'Idempotency-Key': idempotencyKey },
+    timeoutMs: options.timeoutMs || 120000,
+    retries: 0,
+  });
+}
+
+function importStatus(key, importId, options = {}) {
+  return request('GET', `/v1/memory/imports/${encodeURIComponent(importId)}`, {
+    ...options, key, timeoutMs: options.timeoutMs || 15000, retries: 1,
+  });
+}
+
+function importReceipt(key, importId, options = {}) {
+  return request('GET', `/v1/memory/imports/${encodeURIComponent(importId)}/receipt`, {
+    ...options, key, timeoutMs: options.timeoutMs || 15000, retries: 1,
+  });
+}
+
+function importDismiss(key, platform, options = {}) {
+  return request('POST', '/v1/memory/imports/dismiss-suggestion', {
+    ...options, key, body: { platform }, timeoutMs: options.timeoutMs || 10000, retries: 0,
+  });
+}
+
 module.exports = {
   sleep,
   request,
@@ -160,4 +214,11 @@ module.exports = {
   deleteIds,
   dateSearch,
   debtProbe,
+  importInstructions,
+  importCreate,
+  importApprove,
+  importBatch,
+  importStatus,
+  importReceipt,
+  importDismiss,
 };

package/runtime/auto.cjs

@@ -0,0 +1,235 @@
+'use strict';
+// Automatic threshold-activated consolidation for the codex connector.
+//
+// When a hook's search/prepare response carries a non-null
+// reflection.suggestion (the server's own threshold decision; no threshold is
+// ever hardcoded client-side), the hook spawns the existing reflect loop
+// (runtime/reflect.cjs, codex exec headless) as a detached background
+// process. The user's session is never blocked; failures are silent to the
+// session and logged to ~/.memoryrouter/consolidate.log.
+//
+// Rails: machine-wide lockfile (~/.memoryrouter/consolidate.lock, pid +
+// timestamp, stale after 20 minutes, shared with the other MemoryRouter
+// connectors), max 3 batches per trigger with a debt re-check between
+// batches, 30 minute cooldown after a completed run (plus a 10 minute
+// attempt cooldown so persistent failures cannot spam the host model), and
+// config autoConsolidate: false (default true) to opt out.
+
+const fs = require('fs');
+const os = require('os');
+const path = require('path');
+const childProcess = require('child_process');
+
+const LOCK_STALE_MS = 20 * 60 * 1000;
+const COOLDOWN_MS = 30 * 60 * 1000;
+const ATTEMPT_COOLDOWN_MS = 10 * 60 * 1000;
+const MAX_BATCHES_PER_TRIGGER = 3;
+
+// The lock/cooldown directory is intentionally the shared ~/.memoryrouter
+// root (not the codex-specific subdirectory) so all MemoryRouter connectors
+// on one machine coordinate through the same lockfile.
+function autoDir(env = process.env) {
+  return env.MEMORYROUTER_AUTO_DIR || path.join(os.homedir(), '.memoryrouter');
+}
+
+function lockPath(env) { return path.join(autoDir(env), 'consolidate.lock'); }
+function statePath(env) { return path.join(autoDir(env), 'consolidate-state.json'); }
+function logPath(env) { return path.join(autoDir(env), 'consolidate.log'); }
+
+function ensureDir(dir) {
+  fs.mkdirSync(dir, { recursive: true, mode: 0o700 });
+}
+
+function appendLog(line, env = process.env) {
+  try {
+    ensureDir(autoDir(env));
+    const file = logPath(env);
+    try {
+      if (fs.statSync(file).size > 1024 * 1024) fs.renameSync(file, `${file}.1`);
+    } catch { /* new file */ }
+    fs.appendFileSync(file, `[${new Date().toISOString()}] ${line}\n`, { mode: 0o600 });
+  } catch { /* logging never throws */ }
+}
+
+function readState(env = process.env) {
+  try { return JSON.parse(fs.readFileSync(statePath(env), 'utf8')) || {}; } catch { return {}; }
+}
+
+function writeState(patch, env = process.env) {
+  try {
+    ensureDir(autoDir(env));
+    const next = { ...readState(env), ...patch };
+    fs.writeFileSync(statePath(env), JSON.stringify(next, null, 2) + '\n', { mode: 0o600 });
+  } catch { /* state is best-effort */ }
+}
+
+function lockFresh(env = process.env, now = Date.now()) {
+  try {
+    const info = JSON.parse(fs.readFileSync(lockPath(env), 'utf8'));
+    return now - Number(info.at) <= LOCK_STALE_MS;
+  } catch (err) {
+    if (err && err.code === 'ENOENT') return false;
+    try { return now - fs.statSync(lockPath(env)).mtimeMs <= LOCK_STALE_MS; } catch { return false; }
+  }
+}
+
+function acquireLock(env = process.env, now = Date.now()) {
+  ensureDir(autoDir(env));
+  for (let attempt = 0; attempt < 2; attempt += 1) {
+    try {
+      const fd = fs.openSync(lockPath(env), 'wx', 0o600);
+      fs.writeFileSync(fd, JSON.stringify({ pid: process.pid, at: now }) + '\n');
+      fs.closeSync(fd);
+      return true;
+    } catch (err) {
+      if (!err || err.code !== 'EEXIST') return false;
+      if (lockFresh(env, now)) return false;
+      try { fs.unlinkSync(lockPath(env)); } catch { return false; }
+    }
+  }
+  return false;
+}
+
+function releaseLock(env = process.env) {
+  try { fs.unlinkSync(lockPath(env)); } catch { /* already gone */ }
+}
+
+function cooldownActive(env = process.env, now = Date.now()) {
+  const state = readState(env);
+  const completed = Number(state.lastCompletedAt);
+  if (Number.isFinite(completed) && now - completed < COOLDOWN_MS) return true;
+  const attempted = Number(state.lastAttemptAt);
+  if (Number.isFinite(attempted) && now - attempted < ATTEMPT_COOLDOWN_MS) return true;
+  return false;
+}
+
+// Tier choice comes entirely from the server's debt block: per-tier debt and
+// the threshold are both server-reported.
+function pickTier(debt) {
+  const threshold = Number(debt && debt.threshold_importance);
+  const rawEquiv = Number(debt && debt.unconsolidated && debt.unconsolidated.raw
+    && debt.unconsolidated.raw.importance_equiv);
+  const tier2 = Number(debt && debt.unconsolidated && debt.unconsolidated.tier2
+    && debt.unconsolidated.tier2.importance);
+  if (Number.isFinite(threshold) && threshold > 0) {
+    if (Number.isFinite(rawEquiv) && rawEquiv >= threshold) return 1;
+    if (Number.isFinite(tier2) && tier2 >= threshold) return 2;
+  }
+  return /tier 2/i.test(String(debt && debt.suggestion || '')) ? 2 : 1;
+}
+
+function maxBatches(env = process.env) {
+  const raw = parseInt(env.MEMORYROUTER_AUTO_MAX_BATCHES, 10);
+  if (Number.isInteger(raw) && raw >= 1) return Math.min(raw, MAX_BATCHES_PER_TRIGGER);
+  return MAX_BATCHES_PER_TRIGGER;
+}
+
+/**
+ * Trigger check + detached spawn, called from the hook path with the
+ * reflection debt block the hook already received. Never blocks, never
+ * throws into the hook.
+ */
+function maybeSpawnAutoConsolidate(reflection, config = {}, options = {}) {
+  const env = options.env || process.env;
+  try {
+    if (env.MEMORYROUTER_DISABLE === '1') return false;
+    if (config.autoConsolidate === false) return false;
+    if (!reflection || !reflection.suggestion) return false;
+    const now = options.now || Date.now();
+    if (cooldownActive(env, now)) return false;
+    if (lockFresh(env, now)) return false;
+
+    const spawner = options.spawn || childProcess.spawn;
+    const cliPath = options.cliPath || path.resolve(__dirname, '..', 'bin', 'cli.cjs');
+    ensureDir(autoDir(env));
+    const out = fs.openSync(logPath(env), 'a');
+    try {
+      const child = spawner(process.execPath, [cliPath, 'reflect-auto'], {
+        detached: true,
+        stdio: ['ignore', out, out],
+        env: { ...env },
+      });
+      if (child && child.unref) child.unref();
+    } finally {
+      try { fs.closeSync(out); } catch { /* fd handed to child */ }
+    }
+    appendLog(`trigger: spawned reflect-auto (suggestion: ${reflection.suggestion})`, env);
+    return true;
+  } catch (err) {
+    appendLog(`trigger error: ${err && err.message}`, env);
+    return false;
+  }
+}
+
+/**
+ * The detached background run: lock, consolidate up to 3 batches re-checking
+ * debt between each via the API, record completion, release. Reuses the
+ * EXISTING reflect loop (runtime/reflect.cjs); nothing is duplicated.
+ */
+async function runAutoConsolidate(options = {}) {
+  const env = options.env || process.env;
+  const now = options.now || (() => Date.now());
+  const api = options.api || require('./api.cjs');
+  const runReflect = options.runReflect || require('./reflect.cjs').runReflect;
+  const { loadConfig, getApiKey } = require('./config.cjs');
+  const config = options.config || loadConfig(env);
+  const key = options.key || getApiKey(env, config);
+  const log = (line) => appendLog(line, env);
+  if (!key) { log('auto: no API key available; skipping'); return { committed: 0, reason: 'no-key' }; }
+
+  if (cooldownActive(env, now())) { log('auto: cooldown active; skipping'); return { committed: 0, reason: 'cooldown' }; }
+  if (!acquireLock(env, now())) { log('auto: another consolidation run holds the lock; skipping'); return { committed: 0, reason: 'locked' }; }
+
+  writeState({ lastAttemptAt: now() }, env);
+  let committed = 0;
+  try {
+    const bound = maxBatches(env);
+    for (let batch = 0; batch < bound; batch += 1) {
+      const probe = await api.debtProbe(key, { apiBase: config.apiBase });
+      const debt = probe && probe.ok && probe.data && probe.data.reflection;
+      if (!debt || !debt.suggestion) {
+        log(`auto: debt below threshold after ${committed} batch(es); stopping`);
+        break;
+      }
+      const tier = pickTier(debt);
+      log(`auto: batch ${batch + 1}/${bound} starting (tier ${tier}; ${debt.suggestion})`);
+      let result;
+      try {
+        result = await runReflect({ key, tier, batches: '1', env, config, log });
+      } catch (err) {
+        log(`auto: reflect failed: ${err && err.message}`);
+        break;
+      }
+      if (!result || !result.completed) {
+        log('auto: no batch committed; stopping');
+        break;
+      }
+      committed += result.completed;
+    }
+    if (committed > 0) {
+      writeState({ lastCompletedAt: now() }, env);
+      log(`auto: run complete, ${committed} batch(es) committed`);
+    }
+  } finally {
+    releaseLock(env);
+  }
+  return { committed };
+}
+
+module.exports = {
+  maybeSpawnAutoConsolidate,
+  runAutoConsolidate,
+  acquireLock,
+  releaseLock,
+  lockFresh,
+  cooldownActive,
+  pickTier,
+  appendLog,
+  readState,
+  writeState,
+  autoDir,
+  LOCK_STALE_MS,
+  COOLDOWN_MS,
+  ATTEMPT_COOLDOWN_MS,
+  MAX_BATCHES_PER_TRIGGER,
+};

package/runtime/constants.cjs

@@ -1,6 +1,6 @@
 'use strict';
 
-const INTEGRATION_VERSION = '1.1.0';
+const INTEGRATION_VERSION = '1.4.0';
 const RUNTIME_SCHEMA_VERSION = 1;
 const API_BASE = 'https://api.memoryrouter.ai';
 const MCP_ENDPOINT = 'https://mcp.memoryrouter.ai/mcp';

package/runtime/hook.cjs

@@ -4,6 +4,7 @@
 const { loadConfig, getApiKey, loadSessionState, saveSessionState, pruneState, recordHealth } = require('./config.cjs');
 const { resolveProject } = require('./project.cjs');
 const { searchScoped, storeScoped, memoryHandle } = require('./memory.cjs');
+const { maybeSpawnAutoConsolidate } = require('./auto.cjs');
 
 function debug(...values) {
   if (process.env.MEMORYROUTER_DEBUG) console.error('[memoryrouter-codex]', ...values);
@@ -46,13 +47,21 @@ function estimateDebtTokens(reflection) {
 // only when a context is being emitted anyway, and counted inside the
 // existing context char cap so it can never push memory content out.
 // Fires on every run with a non-null suggestion.
-function maybeAppendNudge(context, reflection, sessionId, hookContext) {
-  if (!context || !reflection || !reflection.suggestion || !sessionId) return context;
-  const tokens = estimateDebtTokens(reflection);
-  const nudge = `\n\n[MemoryRouter] Consolidation suggested: ${Math.round(tokens / 1000)}k unconsolidated tokens. Run: memoryrouter-codex reflect`;
+function maybeAppendNudge(context, reflection, sessionId, hookContext, importState) {
+  if (!context || !sessionId) return context;
   const maxChars = Number(hookContext.config?.contextChars || 12000);
-  if (context.length + nudge.length > maxChars) return context;
-  return context + nudge;
+  if (reflection && reflection.suggestion) {
+    const tokens = estimateDebtTokens(reflection);
+    const nudge = `\n\n[MemoryRouter] Consolidation suggested: ${Math.round(tokens / 1000)}k unconsolidated tokens. Run: memoryrouter-codex reflect`;
+    if (context.length + nudge.length <= maxChars) context += nudge;
+  }
+  // Import nudge: same server-owned contract (import.suggested on the search
+  // response), same cap rules, no rate limit; clears on import or dismiss.
+  if (importState && importState.suggested) {
+    const nudge = '\n\n[MemoryRouter] Historical import suggested: this machine has local conversation history not yet in your vault. Run: memoryrouter-codex import --instructions';
+    if (context.length + nudge.length <= maxChars) context += nudge;
+  }
+  return context;
 }
 
 function turnContent(input, state) {
@@ -87,7 +96,11 @@ async function handleInject(input, context) {
     memories: result.memories?.length || 0,
     status: result.status,
   }, context.env);
-  emitContext('UserPromptSubmit', maybeAppendNudge(result.context, result.reflection, input.session_id, context));
+  // Automatic consolidation (default on; config autoConsolidate: false opts
+  // out): detached fire-and-forget background run when the server says
+  // should-consolidate. Never blocks the hook.
+  maybeSpawnAutoConsolidate(result.reflection, context.config, { env: context.env });
+  emitContext('UserPromptSubmit', maybeAppendNudge(result.context, result.reflection, input.session_id, context, result.importState));
 }
 
 async function handleSessionStart(input, context) {
@@ -102,7 +115,8 @@ async function handleSessionStart(input, context) {
     project: context.project.handle,
     memories: result.memories?.length || 0,
   }, context.env);
-  emitContext('SessionStart', maybeAppendNudge(result.context, result.reflection, input.session_id, context));
+  maybeSpawnAutoConsolidate(result.reflection, context.config, { env: context.env });
+  emitContext('SessionStart', maybeAppendNudge(result.context, result.reflection, input.session_id, context, result.importState));
 }
 
 async function captureTurn(input, context, partial = false) {

package/runtime/import-identity.cjs

@@ -0,0 +1,113 @@
+'use strict';
+// Client-side mirror of the server import identity math
+// (workers/src/services/import-identity.ts). Byte-for-byte: the server
+// recomputes every hash from the wire fields and rejects any mismatch, so
+// these formulas MUST stay in lockstep with the server. If either side
+// changes, change the other and bump PARSER_VERSION.
+//
+// IMPORTANT: none of these functions ever incorporate the raw Memory Key.
+// The destination is represented only by an opaque fingerprint.
+
+const { createHash } = require('crypto');
+
+const PARSER_VERSION = 1;
+const CHUNK_VERSION = 1;
+const SOURCE_PLATFORM = 'codex';
+
+function sha256hex(input) {
+  return createHash('sha256').update(input, 'utf8').digest('hex');
+}
+
+function sortKeys(value) {
+  if (Array.isArray(value)) return value.map(sortKeys);
+  if (value && typeof value === 'object') {
+    const out = {};
+    for (const key of Object.keys(value).sort()) out[key] = sortKeys(value[key]);
+    return out;
+  }
+  return value;
+}
+
+/** Canonical JSON: recursively key-sorted. Identical to the server. */
+function canonicalJson(value) {
+  return JSON.stringify(sortKeys(value));
+}
+
+/** Deterministic text normalization. Identical to the server normalizeText. */
+function normalizeText(input) {
+  return input.normalize('NFC').replace(/\r\n?/g, '\n').trim();
+}
+
+/** Opaque, non-reversible destination fingerprint. */
+function vaultFingerprint(memoryKey) {
+  return `mrvault_${sha256hex(memoryKey).slice(0, 16)}`;
+}
+
+function unescapeIdPart(s) {
+  return s.replace(/%3A/g, ':').replace(/%25/g, '%');
+}
+
+/** Recover (platform, conversationId, messageId, parserVersion) or null. */
+function parseExternalId(id) {
+  const parts = id.split(':');
+  if (parts.length !== 4) return null;
+  const [platform, conv, msg, ver] = parts;
+  if (!/^v\d+$/.test(ver)) return null;
+  return {
+    platform,
+    conversationId: unescapeIdPart(conv),
+    messageId: unescapeIdPart(msg),
+    parserVersion: parseInt(ver.slice(1), 10),
+  };
+}
+
+/** Content-bound record hash. Mirrors the server computeRecordHash exactly. */
+function recordHash(fields) {
+  return sha256hex(
+    canonicalJson({
+      conversationId: fields.conversationId,
+      messageId: fields.messageId,
+      role: fields.role,
+      text: normalizeText(fields.text),
+      timestamp: fields.timestamp,
+      v: PARSER_VERSION,
+    }),
+  ).slice(0, 32);
+}
+
+/** Per-conversation content hash. Mirrors the chatgpt CLI conversationContentHash. */
+function conversationContentHash(fields) {
+  return sha256hex(canonicalJson(fields)).slice(0, 32);
+}
+
+/** Content identity of the whole extract, independent of read order. */
+function archiveContentId(conversationContentHashes) {
+  const canonical = [...conversationContentHashes].sort().join('\n');
+  return `arc_${sha256hex(`v${PARSER_VERSION}\n${canonical}`).slice(0, 24)}`;
+}
+
+/** Deterministic import id: same extract to the same vault = same import. */
+function deriveImportId(archiveId, fingerprint) {
+  return `imp_${sha256hex(`${archiveId}|${fingerprint}|p${PARSER_VERSION}|c${CHUNK_VERSION}`).slice(0, 24)}`;
+}
+
+/** Client-computed preview digest; the server recomputes and must agree. */
+function previewDigest(preview) {
+  return `pvd_${sha256hex(canonicalJson(preview)).slice(0, 32)}`;
+}
+
+module.exports = {
+  PARSER_VERSION,
+  CHUNK_VERSION,
+  SOURCE_PLATFORM,
+  sha256hex,
+  canonicalJson,
+  normalizeText,
+  vaultFingerprint,
+  parseExternalId,
+  recordHash,
+  conversationContentHash,
+  archiveContentId,
+  deriveImportId,
+  previewDigest,
+};

package/runtime/import.cjs

@@ -0,0 +1,421 @@
+'use strict';
+// Agent-driven historical import: the transport half.
+//
+// Division of labor (locked): the SERVER owns the extraction prompt
+// (GET /v1/memory/imports/instructions, versioned like reflection_contract);
+// the HOST AGENT executes that prompt locally and writes a JSONL extract; THIS
+// MODULE owns transport only: local validation, deterministic identity math
+// (byte-for-byte with the server), create/approve, idempotent batch upload
+// with resume, and the receipt. The agent never touches HTTP for upload and
+// this module never parses conversation history.
+//
+// The Memory Key never appears in the prompt or the extract file; it rides
+// only in the Authorization header via lib/api.js.
+
+const fs = require('fs');
+const { spawnSync } = require('child_process');
+const readline = require('readline');
+const defaultApi = require('./api.cjs');
+const ids = require('./import-identity.cjs');
+
+const PLATFORM = 'codex';
+const BATCH_SIZE = 2000; // server maximum per batch
+const MAX_CONTENT_CHARS = 1000000;
+const MAX_UPLOAD_ATTEMPTS = 5;
+const INITIAL_BACKOFF_MS = 2000;
+const MAX_BACKOFF_MS = 30000;
+
+function sleep(ms) {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+
+function fmtDate(ms) {
+  if (!Number.isFinite(ms) || ms <= 0) return 'unknown';
+  return new Date(ms).toISOString().slice(0, 10);
+}
+
+function fmtTokens(n) {
+  if (n >= 1000000) return `${(n / 1000000).toFixed(1)}M`;
+  if (n >= 1000) return `${(n / 1000).toFixed(1)}K`;
+  return String(n);
+}
+
+function apiErrorDetail(res) {
+  const data = res && res.data;
+  if (data && typeof data === 'object') {
+    const parts = [data.error, data.message].filter(Boolean);
+    if (parts.length) return parts.join(': ');
+  }
+  return `HTTP ${res && res.status}`;
+}
+
+/**
+ * Read + validate the JSONL extract. Four fields per record, platform-prefixed
+ * external_id, numeric epoch-ms timestamps, role user/assistant, non-empty
+ * content. Returns { records } or { error } (first offending line reported).
+ */
+function readExtract(filePath, platform = PLATFORM) {
+  let raw;
+  try {
+    raw = fs.readFileSync(filePath, 'utf8');
+  } catch (err) {
+    return { error: `could not read ${filePath}: ${err.message}` };
+  }
+  const records = [];
+  const seen = new Set();
+  const lines = raw.split('\n');
+  for (let i = 0; i < lines.length; i++) {
+    const line = lines[i].trim();
+    if (!line) continue;
+    const lineNo = i + 1;
+    let p;
+    try {
+      p = JSON.parse(line);
+    } catch {
+      return { error: `line ${lineNo}: not valid JSON` };
+    }
+    if (!p || typeof p !== 'object' || Array.isArray(p)) return { error: `line ${lineNo}: record is not an object` };
+    const extras = Object.keys(p).filter((k) => !['external_id', 'content', 'timestamp', 'role'].includes(k));
+    if (extras.length) return { error: `line ${lineNo}: unexpected fields: ${extras.join(', ')}` };
+    if (typeof p.external_id !== 'string' || !p.external_id) return { error: `line ${lineNo}: external_id must be a string` };
+    const parsed = ids.parseExternalId(p.external_id);
+    if (!parsed) return { error: `line ${lineNo}: malformed external_id (expected ${platform}:<conversation>:<message>:v${ids.PARSER_VERSION})` };
+    if (parsed.platform !== platform) return { error: `line ${lineNo}: external_id platform must be ${platform}, got ${parsed.platform}` };
+    if (parsed.parserVersion !== ids.PARSER_VERSION) return { error: `line ${lineNo}: external_id version must be v${ids.PARSER_VERSION}` };
+    if (typeof p.content !== 'string' || p.content.length === 0) return { error: `line ${lineNo}: content must be a non-empty string` };
+    if (p.content.length > MAX_CONTENT_CHARS) return { error: `line ${lineNo}: content exceeds ${MAX_CONTENT_CHARS} chars` };
+    if (typeof p.timestamp !== 'number' || !Number.isFinite(p.timestamp) || p.timestamp <= 0) {
+      return { error: `line ${lineNo}: timestamp must be a positive epoch-milliseconds number` };
+    }
+    if (p.timestamp < 1000000000000) return { error: `line ${lineNo}: timestamp looks like seconds, not milliseconds (${p.timestamp})` };
+    if (p.role !== 'user' && p.role !== 'assistant') return { error: `line ${lineNo}: role must be "user" or "assistant"` };
+    if (seen.has(p.external_id)) return { error: `line ${lineNo}: duplicate external_id ${p.external_id}` };
+    seen.add(p.external_id);
+    records.push({
+      externalId: p.external_id,
+      conversationId: parsed.conversationId,
+      messageId: parsed.messageId,
+      content: p.content,
+      role: p.role,
+      timestamp: p.timestamp,
+    });
+  }
+  if (records.length === 0) return { error: 'the extract file contains no records' };
+  return { records };
+}
+
+/**
+ * Deterministic identity for the whole extract: per-record hashes, sorted
+ * per-conversation content hashes, archive id, and the vault-bound import id.
+ * Same math as the chatgpt CLI (cli/src/chatgpt/ids.ts), so re-running the
+ * command replays the same import and committed batches return for free.
+ */
+function computeIdentity(records, memoryKey, platform = PLATFORM) {
+  const byConversation = new Map();
+  for (const r of records) {
+    r.recordSha256 = ids.recordHash({
+      conversationId: r.conversationId,
+      messageId: r.messageId,
+      role: r.role,
+      text: r.content,
+      timestamp: r.timestamp,
+    });
+    if (!byConversation.has(r.conversationId)) byConversation.set(r.conversationId, []);
+    byConversation.get(r.conversationId).push(r);
+  }
+  const convHashes = [];
+  for (const [conversationId, convRecords] of byConversation) {
+    convHashes.push(ids.conversationContentHash({
+      conversationId,
+      canonicalPolicy: `agent-extract-${platform}`,
+      canonicalLeaf: null,
+      records: convRecords.map((r) => ({
+        externalId: r.externalId,
+        recordHash: r.recordSha256,
+        role: r.role,
+        timestamp: r.timestamp,
+      })),
+    }));
+  }
+  const archiveId = ids.archiveContentId(convHashes);
+  const fingerprint = ids.vaultFingerprint(memoryKey);
+  const importId = ids.deriveImportId(archiveId, fingerprint);
+  return { archiveId, fingerprint, importId, conversations: byConversation.size };
+}
+
+function estimateTokens(records) {
+  let total = 0;
+  for (const r of records) total += Math.ceil(r.content.length / 4);
+  return total;
+}
+
+function buildPreviewSummary(records) {
+  let earliest = Number.POSITIVE_INFINITY;
+  let latest = 0;
+  let user = 0;
+  for (const r of records) {
+    if (r.role === 'user') user++;
+    if (r.timestamp < earliest) earliest = r.timestamp;
+    if (r.timestamp > latest) latest = r.timestamp;
+  }
+  return {
+    records: records.length,
+    userRecords: user,
+    assistantRecords: records.length - user,
+    earliestMs: Number.isFinite(earliest) ? earliest : 0,
+    latestMs: latest,
+    estimatedTokens: estimateTokens(records),
+  };
+}
+
+function toWireRecord(r, platform = PLATFORM) {
+  return JSON.stringify({
+    external_id: r.externalId,
+    record_sha256: r.recordSha256,
+    content: r.content,
+    role: r.role,
+    timestamp: r.timestamp,
+    source_platform: platform,
+    parser_version: ids.PARSER_VERSION,
+    chunk_version: ids.CHUNK_VERSION,
+  });
+}
+
+function chunkArray(arr, size) {
+  const out = [];
+  for (let i = 0; i < arr.length; i += size) out.push(arr.slice(i, i + size));
+  return out;
+}
+
+function ask(question) {
+  const rl = readline.createInterface({ input: process.stdin, output: process.stdout });
+  return new Promise((resolve) => rl.question(question, (answer) => {
+    rl.close();
+    resolve(answer.trim().toLowerCase());
+  }));
+}
+
+/**
+ * Fetch and print (or execute) the server-side extraction prompt. With
+ * spawn=true the prompt runs through the local Codex CLI (codex exec),
+ * following the reflect verb's spawn pattern; the model does the extraction
+ * work and the user then runs `import --file <path>`.
+ */
+async function runInstructions(options = {}) {
+  const key = options.key;
+  if (!key) throw new Error('a Memory Key is required');
+  const api = options.api || defaultApi;
+  const log = options.log || ((...args) => console.log(...args));
+  const res = await api.importInstructions(key, options.platform || PLATFORM, { outputPath: options.outputPath });
+  if (!res.ok) throw new Error(`could not fetch import instructions (${apiErrorDetail(res)})`);
+  const data = res.data || {};
+  if (!data.prompt) throw new Error('instructions response carried no prompt');
+
+  if (options.spawn) {
+    const spawn = options.spawnSync || spawnSync;
+    log(`Running the extraction prompt through codex exec (import prompt v${data.import_prompt_version})...`);
+    const result = spawn('codex', ['exec', '--skip-git-repo-check', data.prompt], {
+      encoding: 'utf8',
+      maxBuffer: 64 * 1024 * 1024,
+      timeout: 30 * 60 * 1000,
+    });
+    if (result.error) throw new Error(`could not run the codex CLI: ${result.error.message}`);
+    log(String(result.stdout || ''));
+    if (result.status !== 0) {
+      throw new Error(`codex CLI exited ${result.status}: ${String(result.stderr || '').slice(0, 400)}`);
+    }
+    log('');
+    log(`Extraction finished. Next: ${data.next_command}`);
+    return { spawned: true, version: data.import_prompt_version };
+  }
+
+  log(data.prompt);
+  return { spawned: false, version: data.import_prompt_version };
+}
+
+/** Call the dismiss endpoint so the server-side nudge stops permanently. */
+async function runDismiss(options = {}) {
+  const key = options.key;
+  if (!key) throw new Error('a Memory Key is required');
+  const api = options.api || defaultApi;
+  const log = options.log || ((...args) => console.log(...args));
+  const res = await api.importDismiss(key, options.platform || PLATFORM);
+  if (!res.ok) throw new Error(`dismiss failed (${apiErrorDetail(res)})`);
+  log('Import suggestion dismissed permanently for this vault.');
+  return res.data;
+}
+
+/**
+ * The transport: validate the extract, compute identity, create the import
+ * (deterministic import_id -> re-runs resume for free), show the preview +
+ * server price quote, require approval, upload batches with Idempotency-Keys
+ * and bounded backoff, then print the receipt.
+ */
+async function runImportFile(options = {}) {
+  const key = options.key;
+  if (!key) throw new Error('a Memory Key is required');
+  if (!options.file) throw new Error('--file <path> is required');
+  const api = options.api || defaultApi;
+  const log = options.log || ((...args) => console.log(...args));
+  const platform = options.platform || PLATFORM;
+
+  // 1. Validate locally (fail fast, never partially upload a malformed file).
+  const parsed = readExtract(options.file, platform);
+  if (parsed.error) throw new Error(`invalid extract: ${parsed.error}`);
+  const records = parsed.records;
+  const identity = computeIdentity(records, key, platform);
+  const preview = buildPreviewSummary(records);
+
+  // 2. Create / preflight (idempotent: same content + vault = same import).
+  const previewDigest = ids.previewDigest({
+    importId: identity.importId,
+    archiveId: identity.archiveId,
+    fingerprint: identity.fingerprint,
+    parserVersion: ids.PARSER_VERSION,
+    chunkVersion: ids.CHUNK_VERSION,
+    expectedRecords: preview.records,
+    expectedChunks: preview.records,
+    estimatedTokens: preview.estimatedTokens,
+    selection: null,
+    categories: [],
+  });
+  const createRes = await api.importCreate(key, {
+    import_id: identity.importId,
+    archive_content_id: identity.archiveId,
+    archive_sha256: null,
+    parser_version: ids.PARSER_VERSION,
+    chunk_version: ids.CHUNK_VERSION,
+    source_platform: platform,
+    vault_fingerprint: identity.fingerprint,
+    preview_digest: previewDigest,
+    expected_records: preview.records,
+    expected_chunks: preview.records,
+    estimated_tokens: preview.estimatedTokens,
+    selection: null,
+    categories: [],
+  });
+  if (!createRes.ok) throw new Error(`import preflight failed (${apiErrorDetail(createRes)})`);
+  const created = createRes.data || {};
+  if (created.preview_digest && created.preview_digest !== previewDigest) {
+    throw new Error('client/server preview digest mismatch (fail closed)');
+  }
+  const quote = created.cost_quote || {};
+
+  // 3. Preview + quote for the user.
+  log('');
+  log('Import preview');
+  log('----------------------------');
+  log(`Platform:          ${platform}`);
+  log(`Destination vault: ${identity.fingerprint} (opaque fingerprint)`);
+  log(`Conversations:     ${identity.conversations}`);
+  log(`Records:           ${preview.records} (user ${preview.userRecords}, assistant ${preview.assistantRecords})`);
+  log(`Date range:        ${fmtDate(preview.earliestMs)} to ${fmtDate(preview.latestMs)}`);
+  log(`Estimated tokens:  ${fmtTokens(preview.estimatedTokens)}`);
+  if (quote.available) {
+    log(`Estimated cost:    $${((quote.estimated_cost_cents || 0) / 100).toFixed(2)} (rate $${quote.rate_per_million_usd}/1M paid tokens, free remaining ${fmtTokens(quote.free_tokens_remaining || 0)})`);
+  } else {
+    log(`Estimated cost:    unavailable (${quote.reason || 'unknown'}); the server will refuse approval (fail closed)`);
+  }
+  log(`Import id:         ${identity.importId}`);
+  log('');
+  log('Re-running this command is always safe: committed batches replay for free and duplicates are never re-billed.');
+  log('');
+
+  // 4. Explicit approval.
+  if (!options.yes) {
+    if (options.nonInteractive || !process.stdin.isTTY) {
+      throw new Error('approval required: re-run with --yes to confirm the upload');
+    }
+    const answer = await ask('Proceed with the upload? [y/N]: ');
+    if (answer !== 'y' && answer !== 'yes') {
+      log('Aborted. Nothing was uploaded.');
+      return { aborted: true };
+    }
+  }
+
+  const approveRes = await api.importApprove(key, identity.importId, created.approval_digest);
+  if (!approveRes.ok) {
+    // A completed import cannot be re-approved; that is the fully-settled
+    // resume case. Print the receipt instead of failing.
+    const reason = approveRes.data && approveRes.data.error;
+    if (reason === 'import_completed') {
+      const receiptRes = await api.importReceipt(key, identity.importId);
+      const receipt = receiptRes.ok && receiptRes.data ? receiptRes.data.receipt || null : null;
+      log('This exact extract was already fully imported into this vault. Nothing to upload.');
+      if (receipt) {
+        log('Receipt:');
+        log(JSON.stringify(receipt, null, 2));
+      }
+      return { importId: identity.importId, alreadyComplete: true, complete: true, receipt };
+    }
+    throw new Error(`approval refused (${apiErrorDetail(approveRes)})`);
+  }
+
+  // 5. Idempotent batch upload with bounded backoff. Resume is free: a batch
+  // index that was already durably committed replays its stored response.
+  const batches = chunkArray(records, options.batchSize || BATCH_SIZE);
+  const totals = { stored: 0, duplicate: 0, skipped: 0, failed: 0, conflict: 0 };
+  log(`Uploading ${records.length} record(s) in ${batches.length} batch(es)...`);
+  for (let i = 0; i < batches.length; i++) {
+    const jsonl = batches[i].map((r) => toWireRecord(r, platform)).join('\n');
+    const idempotencyKey = `${identity.importId}:b${i}`;
+    let res = null;
+    let backoff = INITIAL_BACKOFF_MS;
+    for (let attempt = 1; attempt <= MAX_UPLOAD_ATTEMPTS; attempt++) {
+      try {
+        res = await api.importBatch(key, identity.importId, i, jsonl, idempotencyKey);
+      } catch (err) {
+        res = null;
+        if (attempt === MAX_UPLOAD_ATTEMPTS) throw new Error(`batch ${i} failed after ${MAX_UPLOAD_ATTEMPTS} attempts: ${err.message}`);
+      }
+      if (res && (res.ok || (res.status < 500 && res.status !== 429))) break;
+      if (attempt === MAX_UPLOAD_ATTEMPTS) break;
+      await sleep(backoff);
+      backoff = Math.min(backoff * 2, MAX_BACKOFF_MS);
+    }
+    if (!res) throw new Error(`batch ${i} received no response`);
+    if (res.status === 402) {
+      throw new Error('payment required: out of tokens/credits. Add credits at https://app.memoryrouter.ai and re-run to resume.');
+    }
+    if (!res.ok) throw new Error(`batch ${i} failed (${apiErrorDetail(res)}). Re-run to resume; committed batches replay for free.`);
+    const body = res.data || {};
+    const s = body.summary || {};
+    totals.stored += s.stored || 0;
+    totals.duplicate += s.duplicate || 0;
+    totals.skipped += s.skipped || 0;
+    totals.failed += s.failed || 0;
+    totals.conflict += s.conflict || 0;
+    log(`  batch ${i + 1}/${batches.length}${body.replayed ? ' (replayed, free)' : ''}: stored ${s.stored || 0}, duplicate ${s.duplicate || 0}, failed ${s.failed || 0}`);
+  }
+
+  // 6. Receipt.
+  const statusRes = await api.importStatus(key, identity.importId);
+  const complete = Boolean(statusRes.ok && statusRes.data && statusRes.data.complete);
+  let receipt = null;
+  if (complete) {
+    const receiptRes = await api.importReceipt(key, identity.importId);
+    if (receiptRes.ok && receiptRes.data) receipt = receiptRes.data.receipt || null;
+  }
+  log('');
+  log(`Done. stored=${totals.stored} duplicate=${totals.duplicate} skipped=${totals.skipped} failed=${totals.failed} conflict=${totals.conflict}`);
+  if (receipt) {
+    log('Receipt:');
+    log(JSON.stringify(receipt, null, 2));
+  } else if (!complete) {
+    log('The import is not fully settled yet; re-run this command to resume/reconcile.');
+  }
+  return { importId: identity.importId, totals, complete, receipt };
+}
+
+module.exports = {
+  PLATFORM,
+  BATCH_SIZE,
+  readExtract,
+  computeIdentity,
+  buildPreviewSummary,
+  estimateTokens,
+  toWireRecord,
+  runInstructions,
+  runDismiss,
+  runImportFile,
+};

package/runtime/memory.cjs

@@ -132,6 +132,7 @@ async function searchScoped(query, options = {}) {
     project,
     memories,
     reflection: response.data?.reflection || null,
+    importState: response.data?.import || null,
     context: formatContext(memories, project.handle, Number(config.contextChars || 12000)),
   };
 }