@lore-ai/cli 0.1.7 → 0.1.9

package/README.md

@@ -2,7 +2,33 @@
 
 **Automated project knowledge that writes itself.**
 
-Lore is a CLI that runs periodic agentic scans over your development activity -- git history, AI coding conversations (Cursor, Claude Code) -- and distills them into living, always-current project documentation.
+Lore is a CLI that scans your development activity -- git history, AI coding conversations (Cursor, Claude Code) -- and distills them into living, always-current project documentation. It works globally across all your repos from a single `~/.lore/` installation.
+
+## Prerequisites
+
+### AI Agent CLI (required -- pick one)
+
+Lore is an orchestrator. All intelligence -- scanning, classifying, distilling, and generating documentation -- is delegated to an AI agent CLI. **Without one of these installed and authenticated, Lore cannot do anything.**
+
+| Agent | Command | Install |
+|-------|---------|---------|
+| **Claude Code CLI** | `claude` | [docs.anthropic.com/en/docs/claude-code](https://docs.anthropic.com/en/docs/claude-code/overview) |
+| **Cursor Agent CLI** | `cursor` | Built into [Cursor IDE](https://cursor.com) |
+
+By default, `lore sync` tries Claude Code first, then falls back to Cursor Agent. You can lock to one:
+
+```bash
+lore sync --cli claude    # use only Claude Code
+lore sync --cli cursor    # use only Cursor Agent
+```
+
+Or set it globally in `~/.lore/config.json` under `llm.cli` (`"auto"` | `"claude"` | `"cursor"`).
+
+### Other Dependencies
+
+- **Node.js 18+** -- runtime ([nodejs.org](https://nodejs.org))
+- **Git** -- for scanning commit history
+- **GitHub CLI** (`gh`) -- for PR metadata scanning and `lore pr` ([cli.github.com](https://cli.github.com/))
 
 ## Installation
 
@@ -18,106 +44,125 @@ npx @lore-ai/cli
 
 ### Editor Extension
 
-Search **"Lore"** in the VS Code or Cursor extension marketplace, or install manually from a [GitHub Release](https://github.com/matan1542/lore/releases):
+Search **"Lore"** in the Cursor extension marketplace, or install manually from a [GitHub Release](https://github.com/matan1542/lore/releases):
 
 ```bash
-# VS Code
-code --install-extension lore-cursor-extension-<version>.vsix
-
-# Cursor
 cursor --install-extension lore-cursor-extension-<version>.vsix
 ```
 
-### Prerequisites
+### From Source (contributors)
 
-- **Node.js 18+**
-- **Git** -- for scanning commit history
-- **Claude Code CLI** (`claude`) -- for AI-powered distillation ([install](https://docs.anthropic.com/en/docs/claude-code/overview))
-- **GitHub CLI** (`gh`) -- optional, for PR metadata scanning ([install](https://cli.github.com/))
+```bash
+git clone https://github.com/matan1542/lore.git
+cd lore
+npm install
+npm run build
+npm link
+```
 
 ## Quick Start
 
 ```bash
-# Initialize Lore in your project
+# 1. Initialize Lore globally
 lore init
 
-# Run your first scan
+# 2. Select which repos to scan
+lore repos
+
+# 3. Run your first scan
 lore sync
 
-# Check what was captured
+# 4. Check what was captured
 lore status
 
-# Review proposed documentation updates
-lore review
+# 5. Launch the web dashboard
+lore ui
 
-# Ask questions about your project
-lore recall "why did we choose TypeScript?"
+# 6. Generate a SKILL.md for a specific repo
+lore skills -r my-project
 
-# Start automatic periodic scanning
-lore watch
+# 7. Generate a PR description
+lore pr
 ```
 
 ## How It Works
 
 ```
-Developer writes code
-        |
-        v
-Git commits + AI conversations accumulate
+AI conversations accumulate (Cursor sessions, Claude Code transcripts)
+Git commits accumulate (commits, diffs, PRs)
         |
         v
    [ lore sync ]  <-- manual or cron trigger
         |
-        +---> Scans git log (commits, diffs)
-        +---> Scans GitHub PRs (titles, reviews, comments)
-        +---> Scans Claude Code sessions (JSONL transcripts)
-        +---> Scans Cursor conversations (workspace storage)
-        +---> Reads current documentation state
-        |
-        v
-   [ Alignment Engine ]
-        |
-        +---> Matches commits to conversations (by file overlap, branch, timing)
-        +---> Produces: matched pairs, unmatched commits, unmatched conversations
+        +---> Phase 1: Summarize
+        |     Reads all AI conversations, produces concise summaries
         |
-        v
-   [ Agentic Distillation ] (via Claude Code CLI + Octocode MCP)
+        +---> Phase 2: Classify
+        |     Matches conversations to repos (file overlap, branch, timing)
         |
-        +---> Extracts what changed and WHY
-        +---> Identifies architectural decisions
-        +---> Detects emerging patterns
-        +---> Finds stale documentation
+        +---> Phase 3: Distill
+        |     Extracts what changed and WHY, architectural decisions,
+        |     emerging patterns, stale documentation
         |
-        v
-   [ Documentation Output ]
+        +---> Phase 4: Verdict
+        |     Checks relevance, decides what to update
         |
-        +---> Updates CHANGELOG.md
-        +---> Creates Architecture Decision Records (ADRs)
-        +---> Updates architecture docs
-        +---> Queues items for human review
+        +---> Phase 5: Skill Generate
+              Writes/updates SKILL.md files per repo
+              (.cursor/skills/{repo}-guide/SKILL.md)
 ```
 
 ## Commands
 
 | Command | Description |
 |---------|-------------|
-| `lore init` | Set up Lore for a repository. Auto-detects available sources. |
-| `lore sync` | Run a one-off scan, distill, and write cycle. |
+| `lore init` | Set up Lore globally (`~/.lore/`). Auto-detects available sources and CLI agents. |
+| `lore sync` | Run a one-off scan of all AI conversations. |
 | `lore watch` | Start periodic scanning on a cron schedule. |
-| `lore status` | Show scan history, pending reviews, and source status. |
-| `lore recall <topic>` | Query accumulated knowledge about a topic. |
-| `lore review` | Interactively review and approve proposed doc updates. |
+| `lore status` | Show what has been captured and what is pending. |
+| `lore recall <topic>` | Query accumulated knowledge about a topic (progressive disclosure). |
+| `lore review` | Show proposed documentation updates awaiting approval. |
 | `lore config [get\|set] <key> [value]` | View or edit configuration. |
+| `lore repos [select\|list\|add\|remove]` | Manage which repos Lore processes. |
+| `lore ui [-p port]` | Launch the web dashboard (default: port 3333). |
+| `lore summarize` | Summarize all AI conversations and cluster by repo. |
+| `lore skills [-r repo]` | Generate per-repo SKILL.md guides from conversations and git history. |
+| `lore debug [sources\|scan\|conversations]` | Run pipeline diagnostics to troubleshoot scan issues. |
+| `lore pr [-t tier]` | Generate a PR description from git history and cached conversation summaries. |
+
+## Web Dashboard
+
+Launch the built-in dashboard to explore everything Lore has captured:
+
+```bash
+lore ui                # starts at http://localhost:3333
+lore ui -p 4000        # custom port
+```
+
+Requires `lore init` and at least one `lore sync` to have data to display.
+
+**Available pages:**
+
+| Page | What it shows |
+|------|---------------|
+| Overview | Summary of all scanned repos, recent activity |
+| Conversations | All AI conversations with summaries and metadata |
+| Skills | Generated SKILL.md files per repo |
+| Scans | Scan history and timeline |
+| Repo Scans | Per-repo scan details |
+| Analytics | Charts and metrics across scans |
+| Vetting | Skill quality vetting pipeline |
+| Review | Pending documentation updates |
+| Phase Review | Drill into individual pipeline phases |
+| Tool Usage | AI tool call analytics |
 
 ## Configuration
 
-After `lore init`, configuration lives in `.lore/config.json`:
+After `lore init`, configuration lives in `~/.lore/config.json`:
 
 ```json
 {
   "sources": {
-    "git": { "enabled": true },
-    "github": { "enabled": true, "repo": "owner/repo" },
     "cursor": { "enabled": true },
     "claudeCode": { "enabled": true }
   },
@@ -126,53 +171,79 @@ After `lore init`, configuration lives in `.lore/config.json`:
     "adr": "docs/decisions/",
     "architecture": "docs/architecture.md"
   },
+  "repos": {
+    "mode": "auto",
+    "include": [],
+    "exclude": [],
+    "auto": {
+      "maxRepos": 15,
+      "activityWindowDays": 14
+    }
+  },
+  "llm": {
+    "cli": "auto"
+  },
   "schedule": "0 */4 * * *",
   "reviewMode": "propose"
 }
 ```
 
-### Review Modes
-
-- **`propose`** (default): All doc updates go to `.lore/pending/` for human review via `lore review`.
-- **`auto`**: Changes are applied directly to documentation files.
+### Key Settings
+
+| Key | Default | Description |
+|-----|---------|-------------|
+| `sources.cursor.enabled` | `true` | Scan Cursor AI conversations |
+| `sources.claudeCode.enabled` | `true` | Scan Claude Code transcripts |
+| `repos.mode` | `"auto"` | `auto` (top N by activity), `whitelist` (only `include` list), or `all` |
+| `repos.include` | `[]` | Always process these repos (names, org/repo, or path substrings) |
+| `repos.exclude` | `[]` | Never process these repos |
+| `repos.auto.maxRepos` | `15` | Max repos to process in auto mode |
+| `repos.auto.activityWindowDays` | `14` | Only consider repos with recent activity |
+| `llm.cli` | `"auto"` | AI agent: `auto` (Claude first, then Cursor), `claude`, or `cursor` |
+| `schedule` | `"0 */4 * * *"` | Cron schedule for `lore watch` |
+| `reviewMode` | `"propose"` | `propose` (queue for review) or `auto` (apply directly) |
 
 ## Data Sources
 
-### Git History
-Scans commits, diffs, and changed files since the last sync.
-
-### GitHub PRs
-Uses `gh` CLI to fetch merged PRs, review comments, and discussions.
+### AI Conversations
 
-### Claude Code Sessions
-Reads JSONL transcripts from `~/.claude/projects/`. Extracts conversation turns, tool usage, and files modified.
+**Cursor** -- Reads conversation metadata from Cursor's workspace SQLite storage.
 
-### Cursor Conversations
-Reads conversation metadata from Cursor's workspace SQLite storage.
+**Claude Code** -- Reads JSONL transcripts from `~/.claude/projects/`. Extracts conversation turns, tool usage, and files modified.
 
-## Alignment Engine
+### Git History
 
-The key insight: **conversations explain the "why" behind commits**. Lore aligns them using:
+Scans commits, diffs, and changed files since the last sync.
 
-1. **File overlap** (strongest): files edited in conversation match files in commit diff
-2. **Branch + time window**: same branch, commit within session timeframe
-3. **Commit message match**: session contains a git commit command with matching message
+### GitHub PRs
 
-Unmatched conversations (planning, research, debugging sessions) are also captured -- they contain decisions and knowledge even when no code was committed.
+Uses `gh` CLI to fetch merged PRs, review comments, and discussions.
 
 ## Architecture
 
 Built with:
+
+**CLI & Pipeline:**
 - **TypeScript** with strict mode
 - **Commander** for CLI framework
-- **Clack** for beautiful interactive prompts
+- **Clack** for interactive prompts
 - **Zod** for runtime validation
 - **Simple-git** for git operations
 - **Execa** for subprocess management
-- **Claude Code CLI** for AI-powered distillation
-- **Octocode MCP** for code research during distillation
+- **better-sqlite3** / **sql.js** for local state
+- **js-tiktoken** for token counting
+
+**Web Dashboard:**
+- **Hono** for API server
+- **React 19** with **React Router 7**
+- **TanStack Query** for data fetching
+- **Recharts** for analytics charts
+- **Vite** for build tooling
+- **Tailwind CSS** for styling
+
+**AI Integration:**
+- **Claude Code CLI** or **Cursor Agent CLI** for all AI-powered phases
 
 ## License
 
 MIT
-# lore

package/dist/bin/lore.js

@@ -505,6 +505,44 @@ function isRateLimited(output) {
   const lower = output.toLowerCase();
   return lower.includes("you've hit your limit") || lower.includes("hit your limit") || lower.includes("rate limit") || lower.includes("rate_limit") || lower.includes("too many requests") || lower.includes("quota exceeded") || lower.includes("overloaded") || lower.includes("usage limit reached");
 }
+function safeExeca(cmd, args, options) {
+  const { signal, ...execaOpts } = options;
+  const child = execa(cmd, args, {
+    ...execaOpts,
+    // Disable execa's cleanup handler that registers signal-exit listeners
+    // on the parent process. We manage the lifecycle manually.
+    cleanup: false
+  });
+  if (signal) {
+    let forceKillTimer = null;
+    const onAbort = () => {
+      try {
+        child.kill("SIGTERM");
+      } catch {
+      }
+      forceKillTimer = setTimeout(() => {
+        try {
+          child.kill("SIGKILL");
+        } catch {
+        }
+      }, FORCE_KILL_TIMEOUT_MS);
+    };
+    if (signal.aborted) {
+      onAbort();
+    } else {
+      signal.addEventListener("abort", onAbort, { once: true });
+    }
+    const cleanup = () => {
+      signal.removeEventListener("abort", onAbort);
+      if (forceKillTimer) {
+        clearTimeout(forceKillTimer);
+        forceKillTimer = null;
+      }
+    };
+    child.then(cleanup, cleanup);
+  }
+  return child;
+}
 async function invokeCliOnce(target, opts) {
   const { cmd, name } = target;
   const vendor = getVendorConfig(name);
@@ -543,11 +581,11 @@ ${input}`;
     }
     args = filtered;
   }
-  const child = execa(cmd, args, {
+  const child = safeExeca(cmd, args, {
     input,
     cwd: opts.cwd,
     timeout: opts.timeout,
-    cancelSignal: opts.signal,
+    signal: opts.signal,
     ...opts.env && { env: { ...process.env, ...opts.env } }
   });
   let rawOutput = "";
@@ -713,7 +751,7 @@ async function invokeWithFallbackAndTag(opts, label, phase, runId) {
   }
   return result;
 }
-var LORE_MARKER_PREFIX, LORE_PROCESS_MARKER, CLI_VENDOR_CONFIG, CLI_CHAIN, BACKOFF_DELAYS, JITTER_FRACTION, MODEL_TIER_MAP, rateLimitStats, cliPreference, resolvedCli;
+var LORE_MARKER_PREFIX, LORE_PROCESS_MARKER, CLI_VENDOR_CONFIG, CLI_CHAIN, BACKOFF_DELAYS, JITTER_FRACTION, MODEL_TIER_MAP, rateLimitStats, cliPreference, resolvedCli, FORCE_KILL_TIMEOUT_MS;
 var init_cli = __esm({
   "src/utils/cli.ts"() {
     "use strict";
@@ -777,6 +815,7 @@ var init_cli = __esm({
     };
     cliPreference = "auto";
     resolvedCli = null;
+    FORCE_KILL_TIMEOUT_MS = 5e3;
   }
 });
 
@@ -7700,14 +7739,14 @@ ${userPrompt}`;
         // auto-approve MCPs in headless mode (needed for Octocode)
       ]
     ];
-    const child = execa3(
+    const child = safeExeca(
       cli.cmd,
       baseArgs,
       {
         cwd,
         input: researchInput,
         timeout: SKILL_TIMEOUT_MS,
-        cancelSignal: options.signal
+        signal: options.signal
       }
     );
     const result = isClaude ? await streamAndCollect(child, `research:${repoName}`, options) : await collectPlainOutput(child);
@@ -7807,14 +7846,14 @@ ${userPrompt}`;
     ];
     const startTime = Date.now();
     let lastActivityTime = Date.now();
-    const child = execa3(
+    const child = safeExeca(
       cli.cmd,
       baseArgs,
       {
         input: genInput,
         timeout: WRITING_HARD_CEILING_MS,
         // absolute safety net only
-        cancelSignal: options.signal,
+        signal: options.signal,
         env: { ...process.env, ...WRITING_ENV_OVERRIDES }
       }
     );
@@ -7835,7 +7874,10 @@ ${userPrompt}`;
       if (stallMs > WRITING_STALL_TIMEOUT_MS) {
         const stallSec = Math.round(stallMs / 1e3);
         options.onOutput?.(`[skill-gen] [gen:${repoName}] No progress for ${stallSec}s \u2014 aborting stalled agent (${elapsed}s total)`);
-        child.kill();
+        try {
+          child.kill();
+        } catch {
+        }
         return;
       }
       options.onOutput?.(`[skill-gen] [gen:${repoName}] Writing...${progressStr} (${elapsed}s elapsed)`);
@@ -7974,14 +8016,14 @@ ${userPrompt}`;
     ];
     const startTime = Date.now();
     let lastActivityTime = Date.now();
-    const child = execa3(
+    const child = safeExeca(
       cli.cmd,
       baseArgs,
       {
         input: genInput,
         timeout: WRITING_HARD_CEILING_MS,
         // absolute safety net only
-        cancelSignal: options.signal,
+        signal: options.signal,
         env: { ...process.env, ...WRITING_ENV_OVERRIDES }
       }
     );
@@ -8002,7 +8044,10 @@ ${userPrompt}`;
       if (stallMs > WRITING_STALL_TIMEOUT_MS) {
         const stallSec = Math.round(stallMs / 1e3);
         options.onOutput?.(`[skill-gen] [delta:${repoName}] No progress for ${stallSec}s \u2014 aborting stalled agent (${elapsed}s total)`);
-        child.kill();
+        try {
+          child.kill();
+        } catch {
+        }
         return;
       }
       options.onOutput?.(`[skill-gen] [delta:${repoName}] Writing...${progressStr} (${elapsed}s elapsed)`);