memex-mvp 0.10.10 → 0.10.12

package/HELP.md

@@ -337,6 +337,13 @@ memex telegram open-pending --in terminal          # forcefully в Terminal с
 memex telegram scan                                # one-shot ре-скан Downloads
 memex telegram status                              # all decisions
 
+# Импорт файла из ЛЮБОГО пути (v0.10.12+) — без танцев с inbox/pending:
+memex import ~/projects/memex/result.json          # автодетект формата
+memex import ~/Downloads/ChatExport_2026-05-18/    # директория Telegram HTML
+memex import ~/path/to/session.jsonl               # Claude Code JSONL
+memex import ~/Downloads/result.json --force       # пропустить privacy-gate
+memex import file --format claude-jsonl --json     # явный формат + JSON-вывод
+
 memex help                                         # эта инструкция в терминале
 memex --help                                       # справка по командам
 memex --version
@@ -363,6 +370,27 @@ memex get web-1582ab51a7b7 --json > backup.json
 
 ---
 
+## 📥 Импорт файла из произвольного пути (v0.10.8+ → v0.10.12+ MCP)
+
+Если у тебя файл `result.json` (Telegram export) или `.jsonl` (Claude/Cowork-сессия) лежит где-нибудь в `~/Downloads/`, `~/Desktop/`, `~/projects/foo/` — **не нужно** копировать его руками в `~/.memex/inbox/`. Одна команда:
+
+```bash
+memex import ~/projects/memex/result.json
+```
+
+Что произойдёт:
+- Авто-детект формата (Telegram JSON / HTML / Claude Code JSONL / Cowork JSONL)
+- Для Telegram: проверяет твои decision-предпочтения (`allow` / `skip` / `block`)
+  - Новый чат → выводит preview (title, кол-во сообщений, диапазон дат, senders) и спрашивает подтверждения (`--force` чтобы импортить)
+  - Уже в `allowed` → импортит сразу
+  - В `skipped` / `blocked` → отказывается (или `--force` чтобы переопределить)
+- Для Claude/Cowork JSONL — импортит сразу (своя память, нет privacy-gate)
+- Идемпотентно — повторный импорт того же файла не задвоит (UNIQUE constraint)
+
+**То же самое через AI-агента:** любой агент с MCP-соединением может вызвать `memex_import_file({path: "..."})` одним tool-call'ом, получить структурированный ответ. Это **в 100 раз дешевле по токенам** чем агент через bash руками таскает файлы в inbox и парсит логи.
+
+---
+
 ## 🖥 Web-дашборд (v0.10.8+) — посмотреть память глазами
 
 Когда хочется не звать AI, а **самому полистать** что у тебя в memex'е — есть локальный read-only веб-UI. Та же SQLite, другая поверхность.

package/README.md

@@ -131,10 +131,23 @@ memex list --source web                     # all saved URLs
 memex get web-1582ab51a7b7                  # full content of one conversation
 memex overview                              # snapshot of corpus + v0.8.1: capture streak
 memex projects                              # distinct project_paths captured
+memex import ~/projects/memex/result.json  # v0.10.12: ingest any file from any path
 memex help                                  # full user guide (HELP.md)
 memex --help                                # command reference
 ```
 
+### Ingest from any path (v0.10.12+)
+
+```sh
+memex import ~/projects/memex/result.json           # auto-detects Telegram JSON
+memex import ~/Downloads/ChatExport_2026-05-18/     # Telegram HTML export directory
+memex import ~/path/to/session.jsonl                # Claude Code JSONL
+memex import ~/Downloads/result.json --force        # skip Telegram privacy gate
+memex import some-file --format claude-jsonl        # explicit format override
+```
+
+For Telegram, the privacy gate fires for any chat that isn't on your allow-list — the command exits with a preview (title, message count, date range, senders) so you can review before re-running with `--force`. Same path via MCP: any AI agent can call `memex_import_file({path: "..."})` to ingest one file in one tool call (instead of ~10k tokens of bash `mv`-shuffling).
+
 Every query supports `--json` for machine-readable output: `memex search foo --json | jq '.results[].snippet'`. The DB is opened **read-only** — safe to run while `memex-sync` daemon is writing.
 
 When called **without arguments** (`memex`), the binary still runs as an MCP stdio server (the way Claude Code / Cursor / Cline launch it). CLI mode and MCP mode are the same package — no extra install.

package/README.ru.md

@@ -523,9 +523,22 @@ One file with all your AI conversations — sounds scarier than it is.
 1. Telegram **Desktop** (mobile не умеет export)
 2. Чат → меню → **Export chat history**
 3. **Format: JSON** (не HTML)
-4. **Path:** `~/.memex/inbox/`
+4. **Path:** `~/.memex/inbox/` — или, начиная с v0.10.12, **любой путь куда угодно**, и потом `memex import <путь>` ([см. ниже](#импорт-из-любого-пути-v01012))
 5. Готово. Memex подхватит автоматически.
 
+## Импорт из любого пути (v0.10.12+)
+
+Если файл лежит не в `~/.memex/inbox/`, а в обычном месте на диске (`~/Downloads/`, `~/Desktop/`, `~/projects/foo/`) — одна команда:
+
+```bash
+memex import ~/projects/memex/result.json
+memex import ~/Downloads/ChatExport_2026-05-18/         # Telegram HTML-папка
+memex import ~/path/to/session.jsonl                    # Claude Code сессия
+memex import ~/Downloads/result.json --force            # пропустить privacy-gate
+```
+
+То же через AI-агента: `memex_import_file({path: "..."})` — один MCP tool-call. Не нужно вручную переносить файлы.
+
 ---
 
 ## Web-дашборд (v0.10.8+) — увидеть свою память без AI

package/lib/cli/index.js

@@ -38,7 +38,7 @@ import {
 // ---------- Subcommand registry ----------
 export const CLI_SUBCOMMAND_NAMES = [
   'search', 'recent', 'list', 'get', 'overview',
-  'projects', 'context', 'hook', 'when', 'telegram', 'web',
+  'projects', 'context', 'hook', 'when', 'telegram', 'web', 'import',
   'help', '-h', '--help', '-v', '--version',
 ];
 
@@ -1691,6 +1691,134 @@ function tgCmdScan(opts, discovery, pending) {
   console.log(c.dim('Review: memex telegram pending'));
 }
 
+// =============================================================
+// `memex import <path>` — single-command file ingest from any path
+// =============================================================
+async function cmdImport(args) {
+  let path = null;
+  let format = 'auto';
+  let force = false;
+  let json = false;
+
+  for (let i = 0; i < args.length; i++) {
+    const a = args[i];
+    if (a === '--format') format = args[++i] || 'auto';
+    else if (a === '--force' || a === '-f') force = true;
+    else if (a === '--json') json = true;
+    else if (a === '--help' || a === '-h') {
+      console.log(`memex import — ingest a chat file from any path on disk
+
+Usage:
+  memex import <path> [options]
+
+Arguments:
+  <path>            Absolute or ~-relative path to the file or directory.
+                    Supported: Telegram JSON / HTML / dir, Claude Code JSONL,
+                    Cowork JSONL.
+
+Options:
+  --format <fmt>    Override auto-detection. One of: telegram-json,
+                    telegram-html, claude-jsonl, cowork-jsonl, auto (default).
+  --force, -f       Bypass Telegram privacy gate (import new chat without
+                    asking, or override a previously-skipped chat). Only set
+                    this when the user has explicitly approved.
+  --json            Output JSON instead of human text.
+  -h, --help        Show this help.
+
+Examples:
+  memex import ~/projects/memex/result.json
+  memex import ~/Downloads/ChatExport_2026-05-18/
+  memex import ~/path/to/session.jsonl --format claude-jsonl
+  memex import ~/Downloads/result.json --force      # skip privacy gate
+`);
+      return;
+    }
+    else if (a.startsWith('-')) {
+      console.error(`Unknown flag: ${a}`);
+      process.exit(2);
+    }
+    else if (!path) path = a;
+    else { console.error(`Unexpected positional: ${a}`); process.exit(2); }
+  }
+
+  if (!path) {
+    console.error('Usage: memex import <path> [--format X] [--force]');
+    console.error('Run `memex import --help` for details.');
+    process.exit(2);
+  }
+
+  // Open writable DB so the helper can do upserts directly.
+  if (!existsSync(DB_PATH)) {
+    console.error(`memex.db not found at ${DB_PATH}`);
+    console.error(`Run 'memex-sync install' first.`);
+    process.exit(1);
+  }
+  const db = new Database(DB_PATH);
+  db.pragma('journal_mode = WAL');
+  db.pragma('synchronous = NORMAL');
+
+  const { ingestFile } = await import('../ingest-file.js');
+  let result;
+  try {
+    result = await ingestFile(db, path, { format, force });
+  } finally {
+    db.close();
+  }
+
+  if (json) {
+    console.log(JSON.stringify(result, null, 2));
+    return;
+  }
+
+  // Human-readable summary
+  switch (result.status) {
+    case 'imported': {
+      if (result.chats) {
+        // Telegram path
+        console.log(c.green('✓') + ` Imported ${result.total_imported.toLocaleString()} messages from ${result.chats.length} chat(s):`);
+        for (const ch of result.chats) {
+          console.log(`  • ${ch.title} — ${ch.msg_count.toLocaleString()} msgs · ${ch.date_first} → ${ch.date_last}`);
+        }
+      } else {
+        // Claude/Cowork JSONL path
+        console.log(c.green('✓') + ` Imported ${result.total_imported.toLocaleString()} records into "${result.title}"`);
+        console.log(c.dim(`  conversation_id: ${result.conversation_id}`));
+      }
+      console.log('');
+      console.log(c.dim('Searchable from any MCP client now.'));
+      break;
+    }
+    case 'needs_consent': {
+      console.log(c.yellow('⚠ ') + `New Telegram chat requires your consent:`);
+      console.log(`  Title:     ${c.bold(result.chat_title)}`);
+      console.log(`  Type:      ${result.chat_type}`);
+      console.log(`  Messages:  ${result.message_count.toLocaleString()}`);
+      console.log(`  Dates:     ${result.date_first} → ${result.date_last}`);
+      if (result.senders_sample?.length) {
+        console.log(`  Senders:   ${result.senders_sample.join(', ')}`);
+      }
+      console.log('');
+      console.log(c.dim('To import this chat:'));
+      console.log(`  ${c.cyan('memex import')} ${path} ${c.cyan('--force')}`);
+      console.log('');
+      console.log(c.dim('To never import this chat:'));
+      console.log(`  ${c.cyan('memex telegram skip')} "${result.chat_title}"`);
+      process.exit(0);
+    }
+    case 'skipped': {
+      console.log(c.yellow('⏭  Skipped: ') + result.chat_title);
+      console.log(c.dim(`  Reason: ${result.reason}`));
+      if (result.message) console.log(c.dim('  ' + result.message));
+      process.exit(0);
+    }
+    case 'error':
+    default: {
+      console.error(c.yellow('✗ ') + 'Import failed: ' + (result.error || 'unknown'));
+      process.exit(1);
+    }
+  }
+}
+
 // =============================================================
 // `memex web` — opt-in local dashboard
 // =============================================================
@@ -1798,6 +1926,7 @@ export async function runCli(sub, args) {
       case 'hook':       await cmdHook(args); break;
       case 'telegram':   await cmdTelegram(args); break;
       case 'web':        await cmdWeb(args); break;
+      case 'import':     await cmdImport(args); break;
       case 'help':       await cmdHelp(); break;
       case '--help':
       case '-h':         await cmdUsage(); break;

package/lib/ingest-file.js

@@ -0,0 +1,366 @@
+/**
+ * Single-call ingest of an arbitrary file path.
+ *
+ * Solves the v0.10.x onboarding gap: users put their Telegram/Claude exports
+ * in *natural* places like ~/projects/memex/ or ~/Desktop/ rather than the
+ * memex-magic paths (~/.memex/inbox/, ~/Downloads/Telegram Desktop/). The
+ * old answer was "drop it where memex watches" — but users don't know those
+ * paths and AI agents had to fall back to ~10k tokens of bash file-ops
+ * (mv to inbox, poll ingest.log, check DB count, …) just to get one file in.
+ *
+ * This helper lets the agent (or CLI) pass any path and get a single
+ * structured response. Auto-detects format, respects Telegram privacy
+ * decisions, returns precise insert/duplicate counts.
+ *
+ * Returns one of:
+ *   { status: 'imported',     ... }   — records landed in DB
+ *   { status: 'needs_consent',... }   — new Telegram chat, ask user before forcing
+ *   { status: 'skipped',      ... }   — user previously skipped/blocked this chat
+ *   { status: 'error',        ... }   — file unreadable / format unknown / parse failed
+ */
+
+import { existsSync, readFileSync, statSync } from 'node:fs';
+import { homedir } from 'node:os';
+import { basename, isAbsolute, join } from 'node:path';
+
+const SUPPORTED_FORMATS = ['telegram-json', 'telegram-html', 'claude-jsonl', 'cowork-jsonl'];
+
+/**
+ * Expand ~ and resolve relative paths against cwd. We DON'T realpath here —
+ * symlinks in the user's home are valid (e.g. iCloud-synced ~/Downloads).
+ */
+export function resolvePath(p) {
+  if (!p || typeof p !== 'string') return null;
+  let s = p.trim();
+  if (s.startsWith('~/')) s = join(homedir(), s.slice(2));
+  else if (s === '~') s = homedir();
+  if (!isAbsolute(s)) s = join(process.cwd(), s);
+  return s;
+}
+
+/**
+ * Sniff format from path + content. Returns null when we can't tell.
+ *
+ * Heuristics:
+ *   - directory containing messages.html → telegram-html (Desktop export dir)
+ *   - .html / .htm → telegram-html
+ *   - .json with Telegram markers in first 8KB → telegram-json
+ *   - .jsonl starting with cowork- → cowork-jsonl
+ *   - other .jsonl → claude-jsonl
+ */
+export function detectFormat(absPath) {
+  if (!existsSync(absPath)) return null;
+  const stat = statSync(absPath);
+
+  if (stat.isDirectory()) {
+    // Telegram Desktop HTML export = directory with messages.html at root
+    if (existsSync(join(absPath, 'messages.html'))) return 'telegram-html';
+    // Some exports nest result.json inside
+    if (existsSync(join(absPath, 'result.json'))) return 'telegram-json-in-dir';
+    return null;
+  }
+
+  const lower = absPath.toLowerCase();
+  if (lower.endsWith('.html') || lower.endsWith('.htm')) return 'telegram-html';
+
+  if (lower.endsWith('.json')) {
+    let head = '';
+    try { head = readFileSync(absPath, 'utf-8').slice(0, 8192); }
+    catch (_) { return null; }
+    if (
+      head.includes('"messages"') ||
+      head.includes('"chats"') ||
+      head.includes('"personal_information"')
+    ) return 'telegram-json';
+    return null;
+  }
+
+  if (lower.endsWith('.jsonl')) {
+    const name = basename(lower);
+    if (name.startsWith('cowork-')) return 'cowork-jsonl';
+    return 'claude-jsonl';
+  }
+
+  return null;
+}
+
+/**
+ * Main entry. Caller is responsible for opening a writable better-sqlite3 Database
+ * handle and passing it in — we don't open/close the DB here.
+ *
+ * opts:
+ *   format:  one of SUPPORTED_FORMATS, or 'auto' (default).
+ *   force:   bypass Telegram privacy gate (skip/block decisions). Default false.
+ */
+export async function ingestFile(db, rawPath, opts = {}) {
+  const path = resolvePath(rawPath);
+  if (!path) return { status: 'error', error: 'path required' };
+  if (!existsSync(path)) return { status: 'error', error: `file not found: ${path}` };
+
+  const format = opts.format && opts.format !== 'auto' ? opts.format : detectFormat(path);
+  if (!format) {
+    return {
+      status: 'error',
+      error: `unable to detect format; supported: ${SUPPORTED_FORMATS.join(', ')}. ` +
+             `Pass an explicit format= if the file has no extension hint.`,
+    };
+  }
+
+  if (format === 'telegram-json' || format === 'telegram-json-in-dir' || format === 'telegram-html') {
+    return ingestTelegram(db, path, format, opts);
+  }
+  if (format === 'claude-jsonl' || format === 'cowork-jsonl') {
+    return ingestClaudeJsonl(db, path, format, opts);
+  }
+  return { status: 'error', error: `unsupported format: ${format}` };
+}
+
+// ----- Telegram path -----
+
+async function ingestTelegram(db, path, format, opts = {}) {
+  // Parse raw → { chats: { list: [...] }, ... }
+  let raw;
+  try {
+    if (format === 'telegram-html') {
+      const { parseTelegramHtmlExport } = await import('./parse-telegram-html.js');
+      raw = parseTelegramHtmlExport(path);
+    } else if (format === 'telegram-json-in-dir') {
+      raw = JSON.parse(readFileSync(join(path, 'result.json'), 'utf-8'));
+    } else {
+      raw = JSON.parse(readFileSync(path, 'utf-8'));
+    }
+  } catch (e) {
+    return { status: 'error', error: `parse failed: ${e.message}` };
+  }
+  if (!raw) return { status: 'error', error: 'parse returned empty' };
+
+  // Find the chat title — for single-chat exports, it's the only chat in the list
+  const chats = Array.isArray(raw.chats?.list)
+    ? raw.chats.list
+    : Array.isArray(raw.list)
+    ? raw.list
+    : raw.messages
+    ? [raw]
+    : [];
+  if (chats.length === 0) {
+    return { status: 'error', error: 'no chats found in file' };
+  }
+
+  // Privacy gate — single chat exports route through user's decision state;
+  // multi-chat exports (whole-Telegram-archive case, rare) skip the gate and
+  // import everything by default (the agent already had to coordinate that).
+  const decisionsMod = await import('./telegram-decisions.js');
+  const state = decisionsMod.loadDecisions();
+
+  if (chats.length === 1 && !opts.force) {
+    const chat = chats[0];
+    const title =
+      chat.name ||
+      (chat.type === 'saved_messages' ? 'Saved Messages' : `Telegram chat ${chat.id}`);
+
+    if (decisionsMod.isBlocked(state, title)) {
+      return {
+        status: 'skipped',
+        format,
+        chat_title: title,
+        reason: 'matches a block pattern in your decision state',
+        message: `Chat "${title}" matches a block pattern. To override, call again with force=true (and consider unblocking via memex telegram unblock <pattern>).`,
+      };
+    }
+    if (decisionsMod.isSkipped(state, title)) {
+      return {
+        status: 'skipped',
+        format,
+        chat_title: title,
+        reason: 'previously skipped by user',
+        message: `Chat "${title}" was previously skipped. To override, call again with force=true.`,
+      };
+    }
+    if (!decisionsMod.isAllowed(state, title)) {
+      // New chat — needs consent. Surface preview so the agent can ask the user.
+      const msgs = Array.isArray(chat.messages) ? chat.messages : [];
+      const realMsgs = msgs.filter((m) => m.type === 'message');
+      const senders = new Set();
+      let firstTs = Infinity, lastTs = 0;
+      for (const m of realMsgs.slice(0, 200)) {
+        if (m.from) senders.add(m.from);
+        const ts = parseInt(m.date_unixtime || '0', 10);
+        if (ts) { firstTs = Math.min(firstTs, ts); lastTs = Math.max(lastTs, ts); }
+      }
+      return {
+        status: 'needs_consent',
+        format,
+        chat_title: title,
+        chat_type: chat.type || 'unknown',
+        message_count: realMsgs.length,
+        senders_sample: Array.from(senders).slice(0, 5),
+        date_first: isFinite(firstTs) ? new Date(firstTs * 1000).toISOString().slice(0, 10) : null,
+        date_last: lastTs ? new Date(lastTs * 1000).toISOString().slice(0, 10) : null,
+        message:
+          `New Telegram chat "${title}" (${realMsgs.length} msgs). ` +
+          `Ask the user before importing — privacy gate. ` +
+          `To proceed: call memex_import_file again with force=true. ` +
+          `To never auto-import this chat: memex telegram skip "${title}".`,
+      };
+    }
+  }
+
+  // Allowed (or forced, or multi-chat archive) → run the upsert.
+  const { importTelegramRaw } = await import('./import-telegram.js');
+  let result;
+  try {
+    result = importTelegramRaw(db, raw);
+  } catch (e) {
+    return { status: 'error', error: `import failed: ${e.message}` };
+  }
+
+  // Mark chats as allowed for future re-exports
+  for (const c of result.chats) {
+    decisionsMod.allowChat(state, c.title);
+  }
+  decisionsMod.saveDecisions(state);
+
+  return {
+    status: 'imported',
+    format,
+    chats: result.chats.map((c) => ({
+      conversation_id: c.conversation_id,
+      title: c.title,
+      msg_count: c.msg_count,
+      date_first: c.first_ts ? new Date(c.first_ts * 1000).toISOString().slice(0, 10) : null,
+      date_last: c.last_ts ? new Date(c.last_ts * 1000).toISOString().slice(0, 10) : null,
+    })),
+    total_imported: result.totalImported,
+  };
+}
+
+// ----- Claude Code / Cowork JSONL path -----
+
+async function ingestClaudeJsonl(db, path, format, _opts) {
+  const source = format === 'cowork-jsonl' ? 'claude-cowork' : 'claude-code';
+  const fileName = basename(path, '.jsonl');
+  const conversationId = `${source}-${fileName}`;
+
+  // We replicate the inner logic of server.js's importClaudeCodeJsonl so this
+  // module stays self-contained. (Extracting that function from server.js is
+  // a bigger refactor we don't need right now.)
+  const { extractMessageFromRecord, extractCompactBoundary, isContinuationBoilerplate } =
+    await import('./parse.js');
+
+  const insertMessage = db.prepare(`
+    INSERT INTO messages (source, conversation_id, msg_id, role, sender, text, ts, metadata, edited_at, uuid)
+    VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
+    ON CONFLICT(source, conversation_id, msg_id) DO UPDATE SET
+      text = excluded.text,
+      uuid = COALESCE(messages.uuid, excluded.uuid)
+  `);
+  const upsertConversation = db.prepare(`
+    INSERT INTO conversations (conversation_id, source, title, first_ts, last_ts, message_count, parent_conversation_id, project_path)
+    VALUES (?, ?, ?, ?, ?, ?, ?, ?)
+    ON CONFLICT(conversation_id) DO UPDATE SET
+      title = COALESCE(excluded.title, title),
+      first_ts = MIN(first_ts, excluded.first_ts),
+      last_ts = MAX(last_ts, excluded.last_ts),
+      project_path = COALESCE(excluded.project_path, project_path),
+      message_count = (
+        SELECT COUNT(*) FROM messages WHERE messages.conversation_id = conversations.conversation_id
+      )
+  `);
+
+  let lines;
+  try {
+    lines = readFileSync(path, 'utf-8').split('\n').filter(Boolean);
+  } catch (e) {
+    return { status: 'error', error: `read failed: ${e.message}` };
+  }
+
+  let aiTitle = null;
+  let firstUserText = null;
+  let projectPath = null;
+  let first_ts = Infinity, last_ts = 0;
+  let imported = 0;
+
+  const tx = db.transaction(() => {
+    for (const line of lines) {
+      let obj;
+      try { obj = JSON.parse(line); } catch (_) { continue; }
+
+      if (obj.type === 'ai-title' && typeof obj.aiTitle === 'string' && obj.aiTitle.trim()) {
+        aiTitle = obj.aiTitle.trim();
+        continue;
+      }
+      if (obj.cwd && !projectPath) projectPath = obj.cwd;
+
+      const boundary = extractCompactBoundary(obj);
+      if (boundary) {
+        const ts = boundary.timestamp ? Math.floor(new Date(boundary.timestamp).getTime() / 1000) : 0;
+        if (ts) { first_ts = Math.min(first_ts, ts); last_ts = Math.max(last_ts, ts); }
+        const msgId =
+          boundary.id ||
+          (boundary.uuid ? `boundary-${boundary.uuid}` : null) ||
+          (boundary.timestamp ? `boundary-${boundary.timestamp}` : 'boundary-unknown');
+        try {
+          insertMessage.run(
+            source, conversationId, msgId, 'boundary', 'compact',
+            JSON.stringify(boundary.metadata || {}), ts,
+            JSON.stringify({ raw_type: 'compact_boundary', parentUuid: boundary.parentUuid || null }),
+            null, boundary.uuid || null,
+          );
+          imported++;
+        } catch (_) { /* dupe */ }
+        continue;
+      }
+
+      const m = extractMessageFromRecord(obj);
+      if (!m) continue;
+      if (!['user', 'assistant', 'summary'].includes(m.role)) continue;
+
+      const ts = m.timestamp ? Math.floor(new Date(m.timestamp).getTime() / 1000) : 0;
+      if (ts) { first_ts = Math.min(first_ts, ts); last_ts = Math.max(last_ts, ts); }
+
+      if (m.role === 'user' && !firstUserText) {
+        const text = m.text.trim().replace(/\s+/g, ' ');
+        if (text && !isContinuationBoilerplate(text)) firstUserText = text.slice(0, 80);
+      }
+
+      const sender =
+        m.role === 'user' ? 'me'
+        : m.role === 'summary' ? 'compact-summary'
+        : source;
+
+      try {
+        insertMessage.run(
+          source, conversationId,
+          obj.id || `${source}-${fileName}-${(m.uuid || ts).toString().slice(0, 16)}`,
+          m.role, sender, m.text, ts,
+          JSON.stringify({ raw_type: obj.type || null, parentUuid: m.parentUuid || null }),
+          null, m.uuid || null,
+        );
+        imported++;
+      } catch (_) { /* dupe */ }
+    }
+
+    const baseTitle = aiTitle || firstUserText || fileName;
+    upsertConversation.run(
+      conversationId, source, baseTitle,
+      isFinite(first_ts) ? first_ts : null,
+      last_ts || null,
+      0, // message_count gets recomputed by the trigger expression above
+      null, projectPath,
+    );
+  });
+
+  try {
+    tx();
+  } catch (e) {
+    return { status: 'error', error: `transaction failed: ${e.message}` };
+  }
+
+  return {
+    status: 'imported',
+    format,
+    conversation_id: conversationId,
+    total_imported: imported,
+    title: aiTitle || firstUserText || fileName,
+  };
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "memex-mvp",
-  "version": "0.10.10",
+  "version": "0.10.12",
   "description": "Local-first MCP server for cross-agent AI memory. One SQLite + FTS5 corpus across Claude Code, Cowork, Cursor, Continue, Zed, Obsidian, and Telegram — passively captured, verbatim, searchable from any MCP-compatible client.",
   "type": "module",
   "main": "server.js",
@@ -26,7 +26,7 @@
     "sync": "node ingest.js",
     "ingest": "node ingest.js",
     "bot": "node bot/index.js",
-    "test": "node test/parser.test.js && node test/bot-inbox.test.js && node test/search-sort.test.js && node test/store-document.test.js && node test/cli.test.js && node test/hook.test.js && node test/telegram-html.test.js && node test/telegram-decisions.test.js && node test/telegram-pending.test.js && node test/telegram-notify.test.js && node test/notify-click-action.test.js",
+    "test": "node test/parser.test.js && node test/bot-inbox.test.js && node test/search-sort.test.js && node test/store-document.test.js && node test/cli.test.js && node test/hook.test.js && node test/telegram-html.test.js && node test/telegram-decisions.test.js && node test/telegram-pending.test.js && node test/telegram-notify.test.js && node test/notify-click-action.test.js && node test/inbox-watcher.test.js && node test/e2e-inbox.test.js && node test/ingest-file.test.js",
     "prepublishOnly": "npm test"
   },
   "engines": {

package/server.js

@@ -810,18 +810,23 @@ function importFile(filePath) {
       Math.floor(Date.now() / 1000),
       imported
     );
+    log(`imported ${imported} messages from ${basename(filePath)} (${source})`);
+  } else {
+    log(`no NEW messages from ${basename(filePath)} (all dupes)`);
+  }
 
-    // Move processed file to archive
+  // Move processed file to archive regardless of imported count. If we only
+  // archive when imported>0, a fully-deduplicated snapshot stays in inbox.
+  // Daemon then overwrites that file periodically — and on filesystems where
+  // rename-over-existing only fires chokidar 'change' (not 'add'), the
+  // 'change' listener above re-imports, fine; but it also means a wasted
+  // file accumulates in inbox if for any reason the listener didn't catch.
+  // Archiving always keeps inbox a clean "what's new" queue.
+  if (source !== 'unknown') {
     const targetDir = join(ARCHIVE, source);
     mkdirSync(targetDir, { recursive: true });
     const target = join(targetDir, basename(filePath));
-    try {
-      renameSync(filePath, target);
-    } catch (_) {}
-
-    log(`imported ${imported} messages from ${basename(filePath)} (${source})`);
-  } else {
-    log(`no messages imported from ${basename(filePath)}`);
+    try { renameSync(filePath, target); } catch (_) {}
   }
   return imported;
 }
@@ -852,6 +857,19 @@ chokidar
     log('inbox detected (file):', basename(filePath));
     importFile(filePath);
   })
+  // 'change' is critical: the ingest daemon overwrites the inbox snapshot
+  // file every few seconds as the underlying Claude Code / Cowork JSONL
+  // grows (the snapshot is a full re-serialisation, not a delta append).
+  // Without a 'change' listener, chokidar only fires 'add' once when the
+  // file first appears — every subsequent overwrite is silent, the inbox
+  // file accumulates new content on disk but server.js never re-imports
+  // it. UNIQUE(source, conv, msg_id) + INSERT OR IGNORE keep repeated
+  // imports idempotent, so re-processing the whole file on every change
+  // is correct (and cheap: SQLite handles ~10k rows in tens of ms).
+  .on('change', (filePath) => {
+    log('inbox changed (file):', basename(filePath));
+    importFile(filePath);
+  })
   .on('addDir', (dirPath) => {
     // Skip the inbox itself
     if (dirPath === INBOX) return;
@@ -1820,6 +1838,60 @@ const TOOLS = [
       required: ['content'],
     },
   },
+  {
+    name: 'memex_import_file',
+    description:
+      'Ingest a chat/conversation file at any path on disk into memex. ' +
+      'Use this when the user gives you a file path (e.g. "~/projects/memex/result.json", ' +
+      '"~/Downloads/ChatExport_2026-05-18/", or a Claude Code .jsonl) and wants its content ' +
+      'in memex. Auto-detects format (Telegram JSON / Telegram HTML directory / Claude Code ' +
+      'JSONL / Cowork JSONL).\n\n' +
+      'WHEN TO USE:\n' +
+      '  • User: "загрузи мой файл из ~/projects/memex/result.json"\n' +
+      '  • User: "вот файл с историей перпплексити, импортируй"\n' +
+      '  • Anywhere a path is given outside the auto-watched directories.\n\n' +
+      'PRIVACY GATE: for Telegram chats not yet on the user\'s allow-list, the tool returns ' +
+      'status="needs_consent" with a preview (chat title, msg count, date range, senders). ' +
+      'Surface that to the user, confirm, then call again with force=true. Skipped/blocked ' +
+      'chats are refused similarly (override with force=true only after explicit "yes").\n\n' +
+      'IDEMPOTENT: re-importing the same file is safe — UNIQUE(source, conversation_id, msg_id) ' +
+      'dedups under the hood. Re-importing a Telegram export with new messages adds the delta.\n\n' +
+      'DO NOT use this for URLs / web pages — use memex_store_document for those. ' +
+      'DO NOT use bash file ops (mv, cp) to move files into ~/.memex/inbox/ — that\'s the ' +
+      'old workflow this tool replaces. Pass the path directly here and get a structured response.',
+    inputSchema: {
+      type: 'object',
+      properties: {
+        path: {
+          type: 'string',
+          description:
+            'Absolute or ~-relative path to the file or directory. Supported: ' +
+            '.json (Telegram Desktop JSON export), .html (Telegram HTML export, single file), ' +
+            'directory (Telegram HTML export root with messages.html inside, or a Telegram dir ' +
+            'with nested result.json), .jsonl (Claude Code / Cowork session log — prefix cowork- ' +
+            'is treated as Cowork).',
+        },
+        format: {
+          type: 'string',
+          enum: ['auto', 'telegram-json', 'telegram-html', 'claude-jsonl', 'cowork-jsonl'],
+          default: 'auto',
+          description:
+            'Override format detection. Default "auto" sniffs the file content/extension. ' +
+            'Pass an explicit value when the file lacks an extension or auto-detection fails.',
+        },
+        force: {
+          type: 'boolean',
+          default: false,
+          description:
+            'For Telegram: bypass the privacy gate — import even if the chat is new (no prior ' +
+            '"allow" decision), or if it was previously skipped/blocked. Default false: the tool ' +
+            'returns "needs_consent" or "skipped" so the agent can confirm with the user first. ' +
+            'Set true only AFTER the user has explicitly approved this chat\'s import.',
+        },
+      },
+      required: ['path'],
+    },
+  },
   // ---------------------------------------------------------------------
   // Telegram capture flow (v0.10+) — proactive agent-driven import path.
   //
@@ -3113,6 +3185,25 @@ server.setRequestHandler(CallToolRequestSchema, async (req) => {
       });
     }
 
+    // ============================================================
+    //  ARBITRARY-PATH FILE INGEST (v0.10.12+)
+    //
+    //  Closes the onboarding gap: a user puts an export at a path
+    //  memex doesn't watch by default (~/projects/foo/result.json,
+    //  ~/Desktop/, etc.) — agent calls this with the path and gets a
+    //  structured result. No more 10k-token bash file-shuffling.
+    // ============================================================
+    if (name === 'memex_import_file') {
+      const { ingestFile } = await import('./lib/ingest-file.js');
+      const path = typeof args.path === 'string' ? args.path : '';
+      if (!path) return jsonResult({ status: 'error', error: 'path is required' });
+      const result = await ingestFile(db, path, {
+        format: args.format || 'auto',
+        force: args.force === true,
+      });
+      return jsonResult(result);
+    }
+
     // ============================================================
     //  TELEGRAM CAPTURE FLOW (v0.10+)
     // ============================================================

package/skills/install-memex/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: install-memex
 description: Make Claude, Cursor, Cline, Continue, and Zed remember every AI conversation forever — one local SQLite corpus shared across all of them. Installs memex (local-first MCP server) in ~60 seconds via curl one-liner. Includes auto-capture daemon for Claude Code / Cowork / Cursor / Obsidian; v0.10 Telegram auto-detect (export from Desktop → memex stages it → AI proactively asks which to import, privacy-first per-chat consent); v0.8 SessionStart hook for the Brian Chesky moment ("Claude already knows what you were doing"); URL / Perplexity / AI chat share capture via memex_store_document. 18 MCP tools, no cloud, no account, verbatim storage. Use when the user says "install memex", "set up memex", "add memory to my AI", "make my agent remember across sessions", "сохрани мои чаты", or similar.
-version: 1.4.1
+version: 1.5.0
 metadata:
   openclaw:
     emoji: "📚"
@@ -290,7 +290,13 @@ If the user declines Telegram setup ("not now" / "skip"): say "OK, I'll skip Tel
 
 Tell the user to fully quit and reopen the MCP client (Cmd+Q on macOS) so it picks up the new memex tools.
 
-**🌟 FIRST, STRONGLY RECOMMEND THIS (v0.10.8+):** offer to run `memex web --open` for them right now. It's the single most magical post-install moment — a local browser tab opens at `http://127.0.0.1:8765` with 5 pages showing every conversation memex has captured, dialogue rendered as chat-bubbles, full-text searchable. Verbatim — not AI-summarized. Read-only, localhost-only, Ctrl+C to stop. Phrase it naturally, e.g.:
+**🌟 OFFER TO IMPORT THEIR EXISTING DATA RIGHT NOW (v0.10.12+):** before anything else, ask the user if they have a Telegram export or a chat-history file sitting somewhere on disk that they'd like to load. Many users come with a `result.json` they exported yesterday, sitting in `~/Downloads/` or a project folder. Don't make them figure out memex's "magic paths" — just ask:
+
+> "Do you have any chat exports already on disk you want loaded into memex right now? Tell me the path (e.g. `~/Downloads/result.json` or `~/projects/memex/result.json`) — I'll handle it."
+
+Then call `memex_import_file({path: "<their-path>"})`. The tool auto-detects format (Telegram JSON / HTML / Claude JSONL / Cowork JSONL), respects the privacy gate for Telegram (returns `needs_consent` with a preview — surface it, get user OK, retry with `force: true`), and gives you a structured result in one call. **Do not use bash `mv`/`cp` to shuffle files into `~/.memex/inbox/`** — that was the old workflow and burned ~10k tokens per file. Use `memex_import_file` instead.
+
+**🌟 ALSO STRONGLY RECOMMEND THIS (v0.10.8+):** offer to run `memex web --open` for them right now. It's the single most magical post-install moment — a local browser tab opens at `http://127.0.0.1:8765` with 5 pages showing every conversation memex has captured, dialogue rendered as chat-bubbles, full-text searchable. Verbatim — not AI-summarized. Read-only, localhost-only, Ctrl+C to stop. Phrase it naturally, e.g.:
 
 > "I can open a local web dashboard that lets you actually *see* what memex captured — every message, in chat-bubble form, searchable. It's just a browser tab on `localhost:8765`. Run it now? It's a way better demo than asking me search queries blind."