@getlore/cli 0.5.1 → 0.6.0

package/LICENSE

@@ -1,8 +1,16 @@
-Copyright (c) 2026 Mishkin Faustini. All rights reserved.
+MIT License
 
-This software is proprietary and confidential. No part of this software
-may be reproduced, distributed, or transmitted in any form or by any means
-without the prior written permission of the copyright holder.
+Copyright (c) 2026 Mishkin Faustini
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
 
 THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,

package/README.md

@@ -1,6 +1,8 @@
 # Lore
 
-> The lore behind your projects.
+> The lore behind your projects. — [getlore.ai](https://getlore.ai) · [npm](https://www.npmjs.com/package/@getlore/cli)
+
+Every project accumulates lore — the decisions, conversations, research, and context that explain why things are the way they are. Most of it gets lost between chat threads and forgotten docs. Lore keeps it searchable and citable.
 
 A research knowledge repository with **semantic search** and **citations**. Unlike memory systems that store processed facts, Lore preserves your original sources and lets you cite exactly what was said, by whom, and when.
 
@@ -31,14 +33,23 @@ lore search "user pain points"
 
 ## MCP Configuration
 
-Add to your Claude Code or Claude Desktop MCP config:
+**One-click install:**
+
+[![Install in Cursor](https://cursor.com/deeplink/mcp-install-dark.svg)](https://cursor.com/en-US/install-mcp?name=lore&config=eyJjb21tYW5kIjoibnB4IiwiYXJncyI6WyIteSIsIkBnZXRsb3JlL2NsaSIsIm1jcCJdfQ%3D%3D)
+[![Install in VS Code](https://img.shields.io/badge/VS_Code-Install_Server-0098FF?style=flat-square&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=lore&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40getlore%2Fcli%22%2C%22mcp%22%5D%7D)
+[![Install in VS Code Insiders](https://img.shields.io/badge/VS_Code_Insiders-Install_Server-24bfa5?style=flat-square&logo=visualstudiocode&logoColor=white)](https://insiders.vscode.dev/redirect/mcp/install?name=lore&config=%7B%22command%22%3A%22npx%22%2C%22args%22%3A%5B%22-y%22%2C%22%40getlore%2Fcli%22%2C%22mcp%22%5D%7D&quality=insiders)
+[![Install in Goose](https://img.shields.io/badge/Goose-Install_Extension-F97316?style=flat-square&logoColor=white)](goose://extension?cmd=npx&arg=-y&arg=%40getlore%2Fcli&arg=mcp&timeout=300&id=lore-mcp&name=Lore&description=Research%20knowledge%20repository%20with%20semantic%20search%20and%20citations)
+
+After installing, run `npx @getlore/cli setup` to configure API keys and sign in.
+
+**Manual config** — add to your MCP client config (`.mcp.json`, `.cursor/mcp.json`, etc.):
 
 ```json
 {
   "mcpServers": {
     "lore": {
-      "command": "lore",
-      "args": ["mcp"]
+      "command": "npx",
+      "args": ["-y", "@getlore/cli", "mcp"]
     }
   }
 }
@@ -75,6 +86,56 @@ If the MCP host doesn't inherit your shell environment (e.g. Claude Desktop), ad
 
 Same content on different machines produces the same hash — no duplicate processing.
 
+## Agent Platform Install
+
+Lore works with any agent that supports MCP. Use `lore skills install` or install directly from your platform's registry.
+
+### Claude Code
+
+```bash
+# From plugin directory (once approved)
+/plugin install lore
+
+# Or install directly from GitHub
+/plugin install https://github.com/getlore-ai/lore/tree/main/plugins/claude-code
+
+# Or via Lore CLI
+lore skills install claude-code
+```
+
+### Gemini CLI
+
+```bash
+# From Extensions Gallery
+gemini extensions install lore
+
+# Or install directly from GitHub
+gemini extensions install https://github.com/getlore-ai/lore --path plugins/gemini
+
+# Or via Lore CLI
+lore skills install gemini
+```
+
+### Codex CLI
+
+```bash
+# Add MCP server
+codex mcp add lore -- npx -y @getlore/cli mcp
+
+# Install skill
+lore skills install codex
+```
+
+### OpenClaw
+
+```bash
+# From ClawHub
+clawhub install lore
+
+# Or via Lore CLI
+lore skills install openclaw
+```
+
 ## License
 
-Proprietary. All rights reserved.
+MIT

package/dist/cli/commands/sync.js

@@ -205,7 +205,7 @@ export function registerSyncCommand(program, defaultDataDir) {
                 console.log(`  ... and ${result.processing.titles.length - 10} more`);
             }
             if (result.processing.errors > 0) {
-                console.log(`  Errors: ${result.processing.errors}`);
+                console.log(`  ⚠ ${result.processing.errors} file(s) failed to process (check logs above)`);
             }
         }
         if (result.sources_found > 0 || result.sources_indexed > 0) {
@@ -214,6 +214,9 @@ export function registerSyncCommand(program, defaultDataDir) {
             console.log(`  Newly indexed: ${result.sources_indexed}`);
             console.log(`  Already indexed: ${result.already_indexed}`);
         }
+        if (result.reconciled > 0) {
+            console.log(`\nReconciled ${result.reconciled} source(s) missing local content`);
+        }
         if (result.git_pushed) {
             console.log('\n✓ Pushed changes to git');
         }

package/dist/core/git.js

@@ -51,11 +51,43 @@ export async function gitPull(dir) {
         if (!(await hasRemote(dir))) {
             return { success: false, error: 'No remote configured' };
         }
-        // Stash any local changes
-        await execAsync('git stash', { cwd: dir }).catch(() => { });
+        // Stash any local changes before pulling
+        let didStash = false;
+        if (await hasChanges(dir)) {
+            try {
+                const { stdout: stashOut } = await execAsync('git stash', { cwd: dir });
+                didStash = !stashOut.includes('No local changes');
+            }
+            catch (stashErr) {
+                console.error(`[git] Stash failed: ${stashErr}`);
+            }
+        }
         // Pull with rebase
-        const { stdout } = await execAsync('git pull --rebase', { cwd: dir });
-        const pulled = !stdout.includes('Already up to date');
+        let pullOutput;
+        try {
+            const { stdout } = await execAsync('git pull --rebase', { cwd: dir });
+            pullOutput = stdout;
+        }
+        catch (pullErr) {
+            // Restore stashed changes before returning error
+            if (didStash) {
+                await execAsync('git stash pop', { cwd: dir }).catch((popErr) => {
+                    console.error(`[git] Stash pop failed after pull error: ${popErr}`);
+                });
+            }
+            throw pullErr;
+        }
+        // Restore stashed changes after successful pull
+        if (didStash) {
+            try {
+                await execAsync('git stash pop', { cwd: dir });
+            }
+            catch (popErr) {
+                console.error(`[git] Stash pop failed (possible conflict): ${popErr}`);
+                // Don't fail the pull — stashed content is still in `git stash list`
+            }
+        }
+        const pulled = !pullOutput.includes('Already up to date');
         return {
             success: true,
             message: pulled ? 'Pulled new changes' : 'Already up to date'

package/dist/core/vector-store.d.ts

@@ -79,6 +79,7 @@ export declare function getAllSources(_dbPath: string, options?: {
     project?: string;
     source_type?: SourceType;
     limit?: number;
+    sort_by?: 'indexed_at' | 'created_at';
 }): Promise<Array<{
     id: string;
     title: string;
@@ -86,8 +87,19 @@ export declare function getAllSources(_dbPath: string, options?: {
     content_type: ContentType;
     projects: string[];
     created_at: string;
+    indexed_at: string;
     summary: string;
 }>>;
+/**
+ * Get all sources that have a source_path set.
+ * Used by reconciliation to ensure local content.md files exist.
+ */
+export declare function getSourcesWithPaths(_dbPath: string): Promise<Array<{
+    id: string;
+    title: string;
+    summary: string;
+    source_path: string;
+}>>;
 export declare function getSourceById(_dbPath: string, sourceId: string): Promise<{
     id: string;
     title: string;
@@ -101,6 +113,7 @@ export declare function getSourceById(_dbPath: string, sourceId: string): Promis
     quotes: Quote[];
     source_url?: string;
     source_name?: string;
+    source_path?: string;
 } | null>;
 export declare function deleteSource(_dbPath: string, sourceId: string): Promise<{
     deleted: boolean;

package/dist/core/vector-store.js

@@ -321,12 +321,12 @@ export async function searchSources(_dbPath, queryVector, options = {}) {
 // Retrieval Operations
 // ============================================================================
 export async function getAllSources(_dbPath, options = {}) {
-    const { project, source_type, limit } = options;
+    const { project, source_type, limit, sort_by = 'indexed_at' } = options;
     const client = await getSupabase();
     let query = client
         .from('sources')
-        .select('id, title, source_type, content_type, projects, created_at, summary')
-        .order('created_at', { ascending: false });
+        .select('id, title, source_type, content_type, projects, created_at, indexed_at, summary')
+        .order(sort_by, { ascending: false });
     if (source_type) {
         query = query.eq('source_type', source_type);
     }
@@ -348,9 +348,33 @@ export async function getAllSources(_dbPath, options = {}) {
         content_type: row.content_type,
         projects: row.projects,
         created_at: row.created_at,
+        indexed_at: row.indexed_at || row.created_at,
         summary: row.summary,
     }));
 }
+/**
+ * Get all sources that have a source_path set.
+ * Used by reconciliation to ensure local content.md files exist.
+ */
+export async function getSourcesWithPaths(_dbPath) {
+    const client = await getSupabase();
+    const { data, error } = await client
+        .from('sources')
+        .select('id, title, summary, source_path')
+        .not('source_path', 'is', null);
+    if (error) {
+        console.error('Error getting sources with paths:', error);
+        return [];
+    }
+    return (data || [])
+        .filter((row) => row.source_path)
+        .map((row) => ({
+        id: row.id,
+        title: row.title,
+        summary: row.summary || '',
+        source_path: row.source_path,
+    }));
+}
 export async function getSourceById(_dbPath, sourceId) {
     const client = await getSupabase();
     const { data, error } = await client
@@ -375,6 +399,7 @@ export async function getSourceById(_dbPath, sourceId) {
         quotes: data.quotes_json || [],
         source_url: data.source_url || undefined,
         source_name: data.source_name || undefined,
+        source_path: data.source_path || undefined,
     };
 }
 export async function deleteSource(_dbPath, sourceId) {

package/dist/mcp/handlers/sync.d.ts

@@ -38,6 +38,7 @@ interface SyncResult {
         errors: number;
         titles: string[];
     };
+    reconciled: number;
 }
 export declare function handleSync(dbPath: string, dataDir: string, args: SyncArgs, options?: {
     hookContext?: {

package/dist/mcp/handlers/sync.js

@@ -11,10 +11,10 @@
  *   - Generate embeddings
  *   - Store in Supabase + local data dir
  */
-import { readdir, readFile } from 'fs/promises';
+import { readdir, readFile, mkdir, writeFile } from 'fs/promises';
 import { existsSync } from 'fs';
 import path from 'path';
-import { getAllSources, addSource, resetDatabaseConnection, } from '../../core/vector-store.js';
+import { getAllSources, addSource, getSourcesWithPaths, resetDatabaseConnection, } from '../../core/vector-store.js';
 import { generateEmbedding, createSearchableText } from '../../core/embedder.js';
 import { gitPull, gitCommitAndPush } from '../../core/git.js';
 import { loadSyncConfig, getEnabledSources } from '../../sync/config.js';
@@ -106,6 +106,65 @@ async function legacyDiskSync(dbPath, dataDir) {
     return result;
 }
 // ============================================================================
+// Local Content Reconciliation
+// ============================================================================
+/**
+ * Ensures every source in Supabase with a source_path has a local
+ * ~/.lore/sources/{id}/content.md file. This handles:
+ * - Sources indexed before storeSourceToDisk was implemented
+ * - Sources from other machines (in shared Supabase but no local content)
+ * - Any edge case where Supabase write succeeded but disk write failed
+ *
+ * Cost: One Supabase query + local filesystem checks. No LLM calls.
+ */
+async function reconcileLocalContent(dataDir) {
+    const sourcesDir = path.join(dataDir, 'sources');
+    const textExts = ['.md', '.txt', '.json', '.jsonl', '.csv', '.xml', '.yaml', '.yml', '.html', '.log'];
+    // Get all sources that have a source_path in Supabase
+    const sourcesWithPaths = await getSourcesWithPaths('');
+    if (sourcesWithPaths.length === 0)
+        return 0;
+    let reconciled = 0;
+    for (const source of sourcesWithPaths) {
+        const sourceDir = path.join(sourcesDir, source.id);
+        const contentPath = path.join(sourceDir, 'content.md');
+        // Skip if content.md already exists
+        if (existsSync(contentPath))
+            continue;
+        // Try to create content.md from the original source_path
+        let content = null;
+        if (existsSync(source.source_path)) {
+            const ext = path.extname(source.source_path).toLowerCase();
+            if (textExts.includes(ext)) {
+                try {
+                    content = await readFile(source.source_path, 'utf-8');
+                }
+                catch {
+                    // File can't be read — fall through to summary
+                }
+            }
+        }
+        // If we couldn't read the original file, use the summary from Supabase
+        if (!content) {
+            content = [
+                `# ${source.title}`,
+                '',
+                source.summary,
+            ].join('\n');
+        }
+        // Create the source directory and content.md
+        try {
+            await mkdir(sourceDir, { recursive: true });
+            await writeFile(contentPath, content);
+            reconciled++;
+        }
+        catch {
+            // Skip on write failure — will retry on next sync
+        }
+    }
+    return reconciled;
+}
+// ============================================================================
 // Universal Sync (new system)
 // ============================================================================
 async function universalSync(dataDir, dryRun, hookContext) {
@@ -172,6 +231,7 @@ export async function handleSync(dbPath, dataDir, args, options = {}) {
         sources_found: 0,
         sources_indexed: 0,
         already_indexed: 0,
+        reconciled: 0,
     };
     // 1. Git pull
     if (doPull) {
@@ -198,10 +258,12 @@ export async function handleSync(dbPath, dataDir, args, options = {}) {
         result.sources_found = legacyResult.sources_found;
         result.sources_indexed = legacyResult.sources_indexed;
         result.already_indexed = legacyResult.already_indexed;
+        // Reconcile: ensure every Supabase source has local content.md
+        result.reconciled = await reconcileLocalContent(dataDir);
     }
     // 3. Git push
     if (doPush && !dryRun) {
-        const totalNew = (result.processing?.processed || 0) + result.sources_indexed;
+        const totalNew = (result.processing?.processed || 0) + result.sources_indexed + result.reconciled;
         if (totalNew > 0) {
             const pushResult = await gitCommitAndPush(dataDir, `Sync: Added ${totalNew} source(s)`);
             result.git_pushed = pushResult.success && (pushResult.message?.includes('pushed') || false);

package/dist/sync/process.d.ts

@@ -16,6 +16,7 @@ import { type ImageMediaType } from './processors.js';
 export interface ExtractedMetadata {
     title: string;
     summary: string;
+    description?: string;
     date: string | null;
     participants: string[];
     content_type: ContentType;
@@ -38,6 +39,13 @@ export declare function extractMetadata(content: string, filePath: string, optio
         base64: string;
         mediaType: ImageMediaType;
     };
+    fileMetadata?: {
+        filename: string;
+        sizeBytes: number;
+        createdAt: string;
+        modifiedAt: string;
+        exif?: Record<string, unknown>;
+    };
 }): Promise<ExtractedMetadata>;
 export declare function processFiles(files: DiscoveredFile[], dataDir: string, options?: {
     onProgress?: (completed: number, total: number, title: string) => void;

package/dist/sync/process.js

@@ -55,12 +55,24 @@ Content type guidelines:
 
 Be specific in the summary. Include concrete details, names, numbers when present.`;
 export async function extractMetadata(content, filePath, options = {}) {
-    const { model = 'claude-sonnet-4-20250514', image } = options;
+    const { model = 'claude-sonnet-4-20250514', image, fileMetadata } = options;
     const client = getAnthropic();
     // Build message content based on whether we have an image or text
     let messageContent;
     if (image) {
-        // Image analysis with Claude Vision
+        // Image analysis with Claude Vision — extract metadata AND a detailed text description
+        const imagePrompt = `Analyze this image and return ONLY valid JSON with these fields:
+
+{
+  "title": "A descriptive title for this image",
+  "summary": "2-4 sentences capturing the key takeaway or purpose of this image",
+  "description": "A comprehensive text description of everything in this image. Include all text, data, labels, numbers, charts, diagrams, and visual elements. Transcribe any visible text verbatim. For charts/graphs, describe the data points and trends. For screenshots, describe the UI elements and content. Be thorough — this description replaces the image in a text-only knowledge base.",
+  "date": "ISO date string (YYYY-MM-DD) if mentioned, otherwise null",
+  "participants": ["list", "of", "names"] if people are mentioned, otherwise [],
+  "content_type": "one of: interview|meeting|conversation|document|note|analysis"
+}
+
+Be specific and thorough in the description. Include ALL visible text, numbers, and data.`;
         messageContent = [
             {
                 type: 'image',
@@ -72,7 +84,7 @@ export async function extractMetadata(content, filePath, options = {}) {
             },
             {
                 type: 'text',
-                text: `${EXTRACTION_PROMPT}\n\nFile: ${path.basename(filePath)}\n\nAnalyze this image and extract metadata. Describe what's in the image in detail in the summary.`,
+                text: `${imagePrompt}\n\nFile: ${path.basename(filePath)}${fileMetadata ? `\nFile size: ${(fileMetadata.sizeBytes / 1024).toFixed(0)} KB\nFile created: ${fileMetadata.createdAt}\nFile modified: ${fileMetadata.modifiedAt}${fileMetadata.exif ? `\nEXIF data: ${JSON.stringify(fileMetadata.exif)}` : ''}` : ''}`,
             },
         ];
     }
@@ -86,7 +98,7 @@ export async function extractMetadata(content, filePath, options = {}) {
     }
     const response = await client.messages.create({
         model,
-        max_tokens: 1000,
+        max_tokens: image ? 4000 : 1000,
         messages: [
             {
                 role: 'user',
@@ -111,6 +123,7 @@ export async function extractMetadata(content, filePath, options = {}) {
         return {
             title: parsed.title || path.basename(filePath),
             summary: parsed.summary || 'No summary available',
+            description: parsed.description || undefined,
             date: parsed.date || null,
             participants: Array.isArray(parsed.participants) ? parsed.participants : [],
             content_type: validateContentType(parsed.content_type),
@@ -150,9 +163,12 @@ async function storeSourceToDisk(sourceId, file, metadata, processedContent, dat
     const sourceDir = path.join(sourcesDir, sourceId);
     // Create source directory
     await mkdir(sourceDir, { recursive: true });
-    // Copy original file
-    const originalExt = path.extname(file.absolutePath);
-    await copyFile(file.absolutePath, path.join(sourceDir, `original${originalExt}`));
+    // Copy original file (skip binary formats — knowledge store is text-based)
+    const originalExt = path.extname(file.absolutePath).toLowerCase();
+    const binaryExts = ['.jpg', '.jpeg', '.png', '.gif', '.webp', '.bmp', '.tiff', '.ico', '.svg'];
+    if (!binaryExts.includes(originalExt)) {
+        await copyFile(file.absolutePath, path.join(sourceDir, `original${originalExt}`));
+    }
     // Save processed content
     await writeFile(path.join(sourceDir, 'content.md'), processedContent);
     // Save metadata
@@ -223,17 +239,59 @@ export async function processFiles(files, dataDir, options = {}) {
             // 1. Read and preprocess file
             const processed = await processFile(file.absolutePath);
             // 2. Extract metadata with Claude (handles both text and images)
-            const metadata = await extractMetadata(processed.text, file.absolutePath, { model, image: processed.image });
-            // For images, use the summary as the text content
-            const contentText = processed.image
-                ? `# ${metadata.title}\n\n${metadata.summary}`
-                : processed.text;
+            const metadata = await extractMetadata(processed.text, file.absolutePath, { model, image: processed.image, fileMetadata: processed.fileMetadata });
+            // For images, use the detailed description as the text content
+            let contentText;
+            if (processed.image) {
+                const lines = [
+                    `# ${metadata.title}`,
+                    '',
+                    metadata.description || metadata.summary,
+                    '',
+                    '---',
+                    '',
+                    `*Original file: ${path.basename(file.absolutePath)}*`,
+                    `*Synced from: ${file.sourceName}*`,
+                    metadata.date ? `*Date: ${metadata.date}*` : '',
+                ];
+                // Append EXIF metadata if available
+                const exif = processed.fileMetadata?.exif;
+                if (exif && Object.keys(exif).length > 0) {
+                    lines.push('');
+                    lines.push('## Image Metadata');
+                    for (const [key, value] of Object.entries(exif)) {
+                        if (value != null && value !== '') {
+                            const label = key.replace(/([A-Z])/g, ' $1').replace(/^./, s => s.toUpperCase()).trim();
+                            lines.push(`- **${label}:** ${Array.isArray(value) ? value.join(', ') : String(value)}`);
+                        }
+                    }
+                }
+                contentText = lines.filter(Boolean).join('\n');
+            }
+            else {
+                contentText = processed.text;
+            }
             // 3. Use existing ID for edits, generate new ID for new files
             const sourceId = file.existingId || generateSourceId();
-            // 4. Index in Supabase FIRST (may fail on duplicate content_hash)
-            await indexSource(sourceId, file, metadata, dbPath);
-            // 5. Store source to disk ONLY if Supabase succeeded
-            await storeSourceToDisk(sourceId, file, metadata, contentText, dataDir);
+            // 4. Store to disk FIRST — ensures content.md always exists
+            //    If this fails, we skip Supabase so the file stays "new" for retry.
+            try {
+                await storeSourceToDisk(sourceId, file, metadata, contentText, dataDir);
+            }
+            catch (diskError) {
+                console.error(`[process] Disk write failed for ${file.relativePath}: ${diskError}`);
+                throw new Error(`Disk write failed for ${file.relativePath}: ${diskError}`);
+            }
+            // 5. Index in Supabase — if this fails, disk content still exists
+            //    and legacy sync will pick it up on the next run.
+            try {
+                await indexSource(sourceId, file, metadata, dbPath);
+            }
+            catch (supabaseError) {
+                console.error(`[process] Supabase index failed for ${file.relativePath}: ${supabaseError}`);
+                console.error(`[process] Content saved to disk — will be indexed on next sync via legacy path`);
+                // Don't re-throw: disk write succeeded, source is safe
+            }
             if (extensionRegistry && hookContext) {
                 await extensionRegistry.runHook('onSourceCreated', {
                     id: sourceId,
@@ -265,9 +323,11 @@ export async function processFiles(files, dataDir, options = {}) {
                 onProgress?.(result.processed.length + result.errors.length, files.length, batchResult.value.metadata.title);
             }
             else {
+                const errorMsg = batchResult.reason?.message || String(batchResult.reason);
+                console.error(`[process] Failed to process ${file.relativePath}: ${errorMsg}`);
                 result.errors.push({
                     file,
-                    error: batchResult.reason?.message || String(batchResult.reason),
+                    error: errorMsg,
                 });
                 onProgress?.(result.processed.length + result.errors.length, files.length, `Error: ${file.relativePath}`);
             }

package/dist/sync/processors.d.ts

@@ -17,6 +17,13 @@ export interface ProcessedContent {
         base64: string;
         mediaType: ImageMediaType;
     };
+    fileMetadata?: {
+        filename: string;
+        sizeBytes: number;
+        createdAt: string;
+        modifiedAt: string;
+        exif?: Record<string, unknown>;
+    };
 }
 export declare function processFile(filePath: string): Promise<ProcessedContent>;
 export declare function preprocessFiles(filePaths: string[], options?: {

package/dist/sync/processors.js

@@ -4,7 +4,7 @@
  * Converts various file formats to plain text for Claude analysis.
  * All processing is IN MEMORY ONLY - original files are never modified.
  */
-import { readFile } from 'fs/promises';
+import { readFile, stat } from 'fs/promises';
 import path from 'path';
 let pdfParser = null;
 async function getPdfParser() {
@@ -196,13 +196,107 @@ async function processImage(filePath) {
     }
     const buffer = await readFile(filePath);
     const base64 = buffer.toString('base64');
+    // Extract file-level metadata
+    const fileStat = await stat(filePath);
+    const filename = path.basename(filePath);
+    // Try to parse date from common filename patterns (e.g. WhatsApp, screenshots)
+    let dateFromFilename;
+    const whatsappMatch = filename.match(/(\d{4}-\d{2}-\d{2})/);
+    if (whatsappMatch) {
+        dateFromFilename = whatsappMatch[1];
+    }
+    // Extract EXIF metadata (GPS, camera, date, etc.)
+    let exifData;
+    try {
+        const exifr = await import('exifr');
+        const raw = await exifr.default.parse(buffer, {
+            // Request all available tags
+            tiff: true,
+            exif: true,
+            gps: true,
+            icc: false, // Skip color profile (not useful for knowledge)
+            iptc: true, // Keywords, captions, copyright
+            xmp: true, // Extended metadata
+        });
+        if (raw) {
+            // Extract the most useful fields
+            exifData = {};
+            // Camera info
+            if (raw.Make)
+                exifData.cameraMake = raw.Make;
+            if (raw.Model)
+                exifData.cameraModel = raw.Model;
+            if (raw.LensModel)
+                exifData.lens = raw.LensModel;
+            // Date
+            if (raw.DateTimeOriginal)
+                exifData.dateTaken = raw.DateTimeOriginal instanceof Date ? raw.DateTimeOriginal.toISOString() : String(raw.DateTimeOriginal);
+            if (raw.CreateDate)
+                exifData.dateCreated = raw.CreateDate instanceof Date ? raw.CreateDate.toISOString() : String(raw.CreateDate);
+            // GPS
+            if (raw.latitude != null && raw.longitude != null) {
+                exifData.gpsLatitude = raw.latitude;
+                exifData.gpsLongitude = raw.longitude;
+            }
+            if (raw.GPSAltitude != null)
+                exifData.gpsAltitude = raw.GPSAltitude;
+            // Image dimensions
+            if (raw.ImageWidth)
+                exifData.width = raw.ImageWidth;
+            if (raw.ImageHeight)
+                exifData.height = raw.ImageHeight;
+            if (raw.ExifImageWidth)
+                exifData.width = raw.ExifImageWidth;
+            if (raw.ExifImageHeight)
+                exifData.height = raw.ExifImageHeight;
+            // Software / source
+            if (raw.Software)
+                exifData.software = raw.Software;
+            if (raw.Artist)
+                exifData.artist = raw.Artist;
+            if (raw.Copyright)
+                exifData.copyright = raw.Copyright;
+            // IPTC/XMP tags
+            if (raw.Keywords)
+                exifData.keywords = raw.Keywords;
+            if (raw.Description)
+                exifData.description = raw.Description;
+            if (raw.Caption)
+                exifData.caption = raw.Caption;
+            if (raw.Subject)
+                exifData.subject = raw.Subject;
+            if (raw.Title)
+                exifData.title = raw.Title;
+            // Use EXIF date if no filename date
+            if (!dateFromFilename && exifData.dateTaken) {
+                const d = new Date(exifData.dateTaken);
+                if (!isNaN(d.getTime())) {
+                    dateFromFilename = d.toISOString().split('T')[0];
+                }
+            }
+            // Drop empty objects
+            if (Object.keys(exifData).length === 0)
+                exifData = undefined;
+        }
+    }
+    catch (exifError) {
+        console.error(`[processors] EXIF extraction failed for ${path.basename(filePath)}: ${exifError}`);
+    }
     return {
         text: '', // Will be filled by Claude vision
         format: 'image',
+        metadata: dateFromFilename ? { date: dateFromFilename } : undefined,
         image: {
             base64,
             mediaType,
         },
+        fileMetadata: {
+            filename,
+            sizeBytes: fileStat.size,
+            createdAt: fileStat.birthtime.toISOString(),
+            modifiedAt: fileStat.mtime.toISOString(),
+            ...(exifData ? { exif: exifData } : {}),
+        },
     };
 }
 // ============================================================================

package/dist/tui/browse-handlers.js

@@ -90,47 +90,85 @@ export async function loadFullContent(state, ui, dbPath, sourcesDir) {
     const source = getSelectedSource(state);
     if (!source)
         return;
-    // Try to load from disk first
-    const contentPath = path.join(sourcesDir, source.id, 'content.md');
+    // Try to load from disk first (content.md, then original file)
+    const sourceDir = path.join(sourcesDir, source.id);
+    const contentPath = path.join(sourceDir, 'content.md');
     try {
         const { readFile } = await import('fs/promises');
         state.fullContent = await readFile(contentPath, 'utf-8');
     }
     catch {
-        // Fall back to database source details
-        const details = await getSourceById(dbPath, source.id);
-        if (details) {
-            state.fullContent = [
-                `# ${details.title}`,
-                '',
-                `**Type:** ${details.source_type} · ${details.content_type}`,
-                `**Date:** ${formatDate(details.created_at)}`,
-                `**Projects:** ${details.projects.join(', ') || '(none)'}`,
-                '',
-                '## Summary',
-                details.summary,
-                '',
-            ].join('\n');
-            if (details.themes && details.themes.length > 0) {
-                state.fullContent += '## Themes\n';
-                for (const theme of details.themes) {
-                    state.fullContent += `- **${theme.name}**`;
-                    if (theme.summary)
-                        state.fullContent += `: ${theme.summary}`;
-                    state.fullContent += '\n';
+        // content.md not found — try to find and read an original text file
+        let foundOriginal = false;
+        try {
+            const { readFile, readdir } = await import('fs/promises');
+            const files = await readdir(sourceDir);
+            const originalFile = files.find(f => f.startsWith('original.'));
+            if (originalFile) {
+                const textExts = ['.md', '.txt', '.json', '.jsonl', '.csv', '.xml', '.yaml', '.yml', '.html', '.log'];
+                const ext = path.extname(originalFile).toLowerCase();
+                if (textExts.includes(ext)) {
+                    state.fullContent = await readFile(path.join(sourceDir, originalFile), 'utf-8');
+                    foundOriginal = true;
                 }
-                state.fullContent += '\n';
             }
-            if (details.quotes && details.quotes.length > 0) {
-                state.fullContent += '## Key Quotes\n';
-                for (const quote of details.quotes.slice(0, 10)) {
-                    const speaker = quote.speaker === 'user' ? '[You]' : `[${quote.speaker_name || 'Participant'}]`;
-                    state.fullContent += `> ${speaker} "${quote.text}"\n\n`;
+        }
+        catch {
+            // Source directory doesn't exist locally — fall through to DB
+        }
+        if (!foundOriginal) {
+            // Try reading from source_path (original file in sync directory)
+            const details = await getSourceById(dbPath, source.id);
+            if (details?.source_path) {
+                try {
+                    const { readFile } = await import('fs/promises');
+                    const ext = path.extname(details.source_path).toLowerCase();
+                    const textExts = ['.md', '.txt', '.json', '.jsonl', '.csv', '.xml', '.yaml', '.yml', '.html', '.log'];
+                    if (textExts.includes(ext)) {
+                        state.fullContent = await readFile(details.source_path, 'utf-8');
+                        foundOriginal = true;
+                    }
+                }
+                catch {
+                    // source_path file doesn't exist or can't be read
+                }
+            }
+            if (!foundOriginal) {
+                // Final fallback: database summary view
+                if (details) {
+                    state.fullContent = [
+                        `# ${details.title}`,
+                        '',
+                        `**Type:** ${details.source_type} · ${details.content_type}`,
+                        `**Date:** ${formatDate(details.created_at)}`,
+                        `**Projects:** ${details.projects.join(', ') || '(none)'}`,
+                        '',
+                        '## Summary',
+                        details.summary,
+                        '',
+                    ].join('\n');
+                    if (details.themes && details.themes.length > 0) {
+                        state.fullContent += '## Themes\n';
+                        for (const theme of details.themes) {
+                            state.fullContent += `- **${theme.name}**`;
+                            if (theme.summary)
+                                state.fullContent += `: ${theme.summary}`;
+                            state.fullContent += '\n';
+                        }
+                        state.fullContent += '\n';
+                    }
+                    if (details.quotes && details.quotes.length > 0) {
+                        state.fullContent += '## Key Quotes\n';
+                        for (const quote of details.quotes.slice(0, 10)) {
+                            const speaker = quote.speaker === 'user' ? '[You]' : `[${quote.speaker_name || 'Participant'}]`;
+                            state.fullContent += `> ${speaker} "${quote.text}"\n\n`;
+                        }
+                    }
+                }
+                else {
+                    state.fullContent = `Could not load content for ${source.title}`;
                 }
             }
-        }
-        else {
-            state.fullContent = `Could not load content for ${source.title}`;
         }
     }
     // Store raw lines for searching
@@ -370,6 +408,7 @@ export async function applyFilter(state, ui, query, filterMode, dbPath, dataDir,
                 content_type: r.content_type,
                 projects: r.projects,
                 created_at: r.created_at,
+                indexed_at: r.created_at,
                 summary: r.summary,
                 score: r.score,
             }));

package/dist/tui/browse-render.js

@@ -289,7 +289,7 @@ export function getSelectedSource(state) {
  */
 export function renderList(ui, state) {
     const width = ui.listContent.width - 2;
-    const height = ui.listContent.height - 1;
+    const height = ui.listContent.height;
     const lines = [];
     if (state.filtered.length === 0) {
         lines.push('');
@@ -328,8 +328,12 @@ function renderFlatList(ui, state, width, height, lines) {
     if (state.selectedIndex >= itemsVisible) {
         visibleStart = state.selectedIndex - itemsVisible + 1;
     }
-    const visibleEnd = Math.min(state.filtered.length, visibleStart + itemsVisible);
-    for (let i = visibleStart; i < visibleEnd; i++) {
+    // Render items until we fill the viewport
+    let linesUsed = 0;
+    for (let i = visibleStart; i < state.filtered.length; i++) {
+        if (linesUsed + linesPerItem > height)
+            break;
+        linesUsed += linesPerItem;
         const source = state.filtered[i];
         const isSelected = i === state.selectedIndex;
         renderDocumentItem(source, isSelected, width, lines, true);
@@ -339,15 +343,27 @@ function renderFlatList(ui, state, width, height, lines) {
  * Render grouped list with collapsible project folders
  */
 function renderGroupedList(ui, state, width, height, lines) {
-    // Calculate lines per item (headers take 2, docs take 3)
-    const avgLinesPerItem = 2.5;
-    const itemsVisible = Math.floor(height / avgLinesPerItem);
+    // Calculate line height for each item type
+    const itemLineHeight = (i) => state.listItems[i]?.type === 'header' ? 2 : 3;
+    // Find visibleStart: scroll so selectedIndex is visible
     let visibleStart = 0;
-    if (state.selectedIndex >= itemsVisible) {
-        visibleStart = state.selectedIndex - itemsVisible + 1;
-    }
-    const visibleEnd = Math.min(state.listItems.length, visibleStart + itemsVisible);
-    for (let i = visibleStart; i < visibleEnd; i++) {
+    // Count lines from visibleStart to selectedIndex (inclusive)
+    let linesFromStartToSelected = 0;
+    for (let i = 0; i <= state.selectedIndex; i++) {
+        linesFromStartToSelected += itemLineHeight(i);
+    }
+    // If selectedIndex doesn't fit, scroll forward
+    while (linesFromStartToSelected > height && visibleStart < state.selectedIndex) {
+        linesFromStartToSelected -= itemLineHeight(visibleStart);
+        visibleStart++;
+    }
+    // Render items until we fill the viewport
+    let linesUsed = 0;
+    for (let i = visibleStart; i < state.listItems.length; i++) {
+        const h = itemLineHeight(i);
+        if (linesUsed + h > height)
+            break;
+        linesUsed += h;
         const item = state.listItems[i];
         const isSelected = i === state.selectedIndex;
         if (item.type === 'header') {
@@ -377,7 +393,7 @@ function renderProjectHeader(item, isSelected, width, lines) {
  * Render a document item row
  */
 function renderDocumentItem(source, isSelected, width, lines, showProject) {
-    const date = formatDate(source.created_at);
+    const date = formatDate(source.indexed_at || source.created_at);
     const contentType = source.content_type || 'document';
     const project = source.projects[0] || '';
     // Format content type as a tag

package/dist/tui/browse-types.d.ts

@@ -10,6 +10,7 @@ export interface SourceItem {
     content_type: ContentType;
     projects: string[];
     created_at: string;
+    indexed_at: string;
     summary: string;
     score?: number;
 }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@getlore/cli",
-  "version": "0.5.1",
+  "version": "0.6.0",
   "description": "Research knowledge repository with semantic search, citations, and project lineage tracking",
   "type": "module",
   "main": "dist/index.js",
@@ -44,7 +44,7 @@
   "bugs": {
     "url": "https://github.com/getlore-ai/lore/issues"
   },
-  "license": "UNLICENSED",
+  "license": "MIT",
   "engines": {
     "node": ">=18"
   },
@@ -58,6 +58,7 @@
     "chokidar": "^5.0.0",
     "commander": "^12.1.0",
     "dotenv": "^17.2.3",
+    "exifr": "^7.1.3",
     "openai": "^4.77.0",
     "pdf-parse": "^2.4.5",
     "zod": "^3.24.0"

package/plugins/claude-code/skills/lore/SKILL.md

@@ -10,15 +10,23 @@ Lore is a research knowledge repository available via MCP. It stores documents,
 
 ## First-Time Setup
 
-If Lore is not installed yet, you can set it up for the user. The process requires their input at two points.
+If Lore is not installed yet, you can set it up for the user.
 
 1. **Install**: `npm install -g @getlore/cli` (requires Node.js 18+)
-2. **Ask the user** for their **email address** and their **OpenAI** and **Anthropic API keys**
-3. **Send OTP**: Run `lore setup --openai-key <key> --anthropic-key <key> --email <email> --data-dir ~/.lore` — this sends a 6-digit code to their email and exits
-4. **Ask the user** for the **6-digit code** from their email
-5. **Complete setup**: Re-run the same command with `--code <code>` appended — this verifies and finishes setup
-
-After setup, Lore works autonomously. No further user input is needed for searching, ingesting, or syncing.
+2. **Ask the user** for their **email address**
+3. **Ask for API keys** — present these two options:
+   - **Recommended**: Tell the user to run these commands themselves (keys stay out of chat history):
+     ```
+     export OPENAI_API_KEY="sk-..."
+     export ANTHROPIC_API_KEY="sk-ant-..."
+     ```
+     Then run: `lore setup --openai-key $OPENAI_API_KEY --anthropic-key $ANTHROPIC_API_KEY --email <email> --data-dir ~/.lore`
+   - **Convenient but riskier**: The user can paste keys directly into this chat and you run setup with them. Warn the user that keys shared in chat may be stored in conversation history.
+4. **Send OTP**: Run the setup command — this sends a 6-digit code to their email and exits
+5. **Ask the user** for the **6-digit code** from their email
+6. **Complete setup**: Re-run the same command with `--code <code>` appended
+
+After setup, Lore works autonomously.
 
 ## MCP Tools
 

package/plugins/codex/SKILL.md

@@ -9,15 +9,23 @@ Lore is a research knowledge repository available via MCP. It stores documents,
 
 ## First-Time Setup
 
-If Lore is not installed yet, you can set it up for the user. The process requires their input at two points.
+If Lore is not installed yet, you can set it up for the user.
 
 1. **Install**: `npm install -g @getlore/cli` (requires Node.js 18+)
-2. **Ask the user** for their **email address** and their **OpenAI** and **Anthropic API keys**
-3. **Send OTP**: Run `lore setup --openai-key <key> --anthropic-key <key> --email <email> --data-dir ~/.lore` — this sends a 6-digit code to their email and exits
-4. **Ask the user** for the **6-digit code** from their email
-5. **Complete setup**: Re-run the same command with `--code <code>` appended — this verifies and finishes setup
-
-After setup, Lore works autonomously. No further user input is needed for searching, ingesting, or syncing.
+2. **Ask the user** for their **email address**
+3. **Ask for API keys** — present these two options:
+   - **Recommended**: Tell the user to run these commands themselves (keys stay out of chat history):
+     ```
+     export OPENAI_API_KEY="sk-..."
+     export ANTHROPIC_API_KEY="sk-ant-..."
+     ```
+     Then run: `lore setup --openai-key $OPENAI_API_KEY --anthropic-key $ANTHROPIC_API_KEY --email <email> --data-dir ~/.lore`
+   - **Convenient but riskier**: The user can paste keys directly into this chat and you run setup with them. Warn the user that keys shared in chat may be stored in conversation history.
+4. **Send OTP**: Run the setup command — this sends a 6-digit code to their email and exits
+5. **Ask the user** for the **6-digit code** from their email
+6. **Complete setup**: Re-run the same command with `--code <code>` appended
+
+After setup, Lore works autonomously.
 
 ## MCP Tools
 

package/plugins/gemini/GEMINI.md

@@ -4,15 +4,23 @@ Lore is a research knowledge repository available via MCP. It stores documents,
 
 ## First-Time Setup
 
-If Lore is not installed yet, you can set it up for the user. The process requires their input at two points.
+If Lore is not installed yet, you can set it up for the user.
 
 1. **Install**: `npm install -g @getlore/cli` (requires Node.js 18+)
-2. **Ask the user** for their **email address** and their **OpenAI** and **Anthropic API keys**
-3. **Send OTP**: Run `lore setup --openai-key <key> --anthropic-key <key> --email <email> --data-dir ~/.lore` — this sends a 6-digit code to their email and exits
-4. **Ask the user** for the **6-digit code** from their email
-5. **Complete setup**: Re-run the same command with `--code <code>` appended — this verifies and finishes setup
-
-After setup, Lore works autonomously. No further user input is needed for searching, ingesting, or syncing.
+2. **Ask the user** for their **email address**
+3. **Ask for API keys** — present these two options:
+   - **Recommended**: Tell the user to run these commands themselves (keys stay out of chat history):
+     ```
+     export OPENAI_API_KEY="sk-..."
+     export ANTHROPIC_API_KEY="sk-ant-..."
+     ```
+     Then run: `lore setup --openai-key $OPENAI_API_KEY --anthropic-key $ANTHROPIC_API_KEY --email <email> --data-dir ~/.lore`
+   - **Convenient but riskier**: The user can paste keys directly into this chat and you run setup with them. Warn the user that keys shared in chat may be stored in conversation history.
+4. **Send OTP**: Run the setup command — this sends a 6-digit code to their email and exits
+5. **Ask the user** for the **6-digit code** from their email
+6. **Complete setup**: Re-run the same command with `--code <code>` appended
+
+After setup, Lore works autonomously.
 
 ## MCP Tools
 

package/skills/generic-agent.md

@@ -4,15 +4,23 @@ Lore is a research knowledge repository accessible via MCP (Model Context Protoc
 
 ## First-Time Setup
 
-If Lore is not installed yet, you can set it up for the user. The process requires their input at two points.
+If Lore is not installed yet, you can set it up for the user.
 
 1. **Install**: `npm install -g @getlore/cli` (requires Node.js 18+)
-2. **Ask the user** for their **email address** and their **OpenAI** and **Anthropic API keys**
-3. **Send OTP**: Run `lore setup --openai-key <key> --anthropic-key <key> --email <email> --data-dir ~/.lore` — this sends a 6-digit code to their email and exits
-4. **Ask the user** for the **6-digit code** from their email
-5. **Complete setup**: Re-run the same command with `--code <code>` appended — this verifies and finishes setup
-
-After setup, Lore works autonomously. No further user input is needed for searching, ingesting, or syncing.
+2. **Ask the user** for their **email address**
+3. **Ask for API keys** — present these two options:
+   - **Recommended**: Tell the user to run these commands themselves (keys stay out of chat history):
+     ```
+     export OPENAI_API_KEY="sk-..."
+     export ANTHROPIC_API_KEY="sk-ant-..."
+     ```
+     Then run: `lore setup --openai-key $OPENAI_API_KEY --anthropic-key $ANTHROPIC_API_KEY --email <email> --data-dir ~/.lore`
+   - **Convenient but riskier**: The user can paste keys directly into this chat and you run setup with them. Warn the user that keys shared in chat may be stored in conversation history.
+4. **Send OTP**: Run the setup command — this sends a 6-digit code to their email and exits
+5. **Ask the user** for the **6-digit code** from their email
+6. **Complete setup**: Re-run the same command with `--code <code>` appended
+
+After setup, Lore works autonomously.
 
 ## Core Concepts