ai-lens 0.8.112 → 0.8.114

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/.commithash CHANGED
@@ -1 +1 @@
1
- 1411e25
1
+ 507ff10
package/CHANGELOG.md CHANGED
@@ -2,6 +2,22 @@
2
2
 
3
3
  History of changes to the `ai-lens` CLI package on npm. New entries go on top. Format: `## X.Y.Z — YYYY-MM-DD`, followed by user-facing bullets.
4
4
 
5
+ ## 0.8.114 — 2026-07-01
6
+ - fix(client): normalize Windows /c:/ drive paths so Cursor git metadata is captured
7
+ - feat(team-memory): SessionStart recall + injection (T6)
8
+ - feat(cli): `ai-lens local-server up|down|status` — one-command private local server
9
+ - feat(dx): local-first compose, npx docs, DATA_DIR-aware config path
10
+ - fix(import): read Cursor CLI store WAL — fix `no such table: meta`
11
+ - fix(import): surface missing identity in cursor --dry-run + name hint
12
+ - feat(client): nudge old-Node Cursor users to upgrade so import cursor works
13
+ - feat(import): add `ai-lens import cursor` (IDE + CLI), at parity with claude-code
14
+ - feat(cli): --use-repo-path implies --project-hooks
15
+ - fix(cli): make --no-hooks an inert deprecated no-op so /setup installs hooks
16
+
17
+ ## 0.8.113 — 2026-06-24
18
+ - feat: track working-directory changes mid-session (Claude Code `CwdChanged`) so events are attributed to the project you `cd` into, not the one you started in
19
+ - feat: capture the assistant's response text and thinking from the Claude Code transcript, not just token counts — the full turn-by-turn dialogue is now recorded
20
+
5
21
  ## 0.8.112 — 2026-06-23
6
22
  - feat: re-running `init --yes` no longer changes your MCP registration. On an already-set-up install it's left exactly as-is — if MCP is registered it stays (same scope), if it isn't it's left off — instead of being removed-and-re-added at user scope (which could migrate its scope, force it on, or drop it if the re-add failed). Fresh installs still register MCP; use `init --mcp-only` to deliberately (re)register.
7
23
 
package/README.md CHANGED
@@ -25,6 +25,8 @@ This will:
25
25
 
26
26
  Re-running is safe — it updates outdated hooks and skips current ones.
27
27
 
28
+ > **No global install — always use `npx`.** The CLI is never placed on your `PATH`; run every command as `npx -y ai-lens <command>` (each run uses the latest published version). A bare `ai-lens …` will be "command not found". Note: `init` *does* install the **runtime** (the capture client) into `~/.ai-lens/client/` — only the CLI itself stays npx-only.
29
+
28
30
  ### Deploying hooks in a specific project (project-level hooks)
29
31
 
30
32
  To write hooks into the project directory (`.cursor/hooks.json` and `.claude/settings.json`) instead of global `~/.cursor/` and `~/.claude/`, run from the project root:
@@ -38,10 +40,17 @@ Add `--use-repo-path` to run `capture.js` directly from the package (repo or npx
38
40
  ### CLI commands
39
41
 
40
42
  ```bash
41
- npx ai-lens init # Setup wizard — detect tools, install hooks, configure MCP
42
- npx ai-lens status # Run health checks and generate a diagnostic report
43
- npx ai-lens remove # Remove hooks, client files, and MCP config
44
- npx ai-lens version # Show installed version
43
+ npx ai-lens init # Setup wizard — detect tools, install hooks, configure MCP
44
+ npx ai-lens status # Run health checks and generate a diagnostic report
45
+ npx ai-lens import <source> # Backfill history. <source> = claude-code | cursor
46
+ # [--days N | --since YYYY-MM-DD | --from --to …] [--projects A,B] [--dry-run]
47
+ npx ai-lens list-sessions # List captured sessions
48
+ npx ai-lens find-session <query> # Find a captured session
49
+ npx ai-lens delete-sessions <…> # Delete captured sessions
50
+ npx ai-lens local-server up # Run a PRIVATE local server in Docker (data stays on your machine)
51
+ # · down [--purge] | status · then offers to point the client at it
52
+ npx ai-lens remove # Remove hooks, client files, and MCP config
53
+ npx ai-lens version # Show installed version
45
54
  ```
46
55
 
47
56
  ### CLI options
@@ -64,6 +73,35 @@ export AI_LENS_SERVER_URL=https://ai-lens.rantsports.com
64
73
  export AI_LENS_PROJECTS="~/meta/, ~/meta-cursor/" # optional, default: all
65
74
  ```
66
75
 
76
+ ### Private local server (your data stays on your machine)
77
+
78
+ For a private trial or solo use — your sessions never leave your machine. **Prerequisite: Docker Desktop** (on Windows, with WSL2). Then one command stands up a local server (Postgres+pgvector + app) and offers to point the client at it:
79
+
80
+ ```bash
81
+ npx -y ai-lens local-server up # check Docker → pull → start → wait healthy → offer `init`
82
+ # → server at http://localhost:3000
83
+ npx -y ai-lens local-server status # container + health status
84
+ npx -y ai-lens local-server down # stop (add --purge to also delete the data volume)
85
+ ```
86
+
87
+ It uses **public images only** (`pgvector/pgvector:pg16` + `ghcr.io/r-ms/ai-lens/app`) — no login, no build. The server runs in personal (self-host) mode and attributes events by git identity, so set BOTH (the server 400s without name):
88
+
89
+ ```bash
90
+ git config --global user.email "you@example.com"
91
+ git config --global user.name "Your Name"
92
+ ```
93
+
94
+ <details>
95
+ <summary>Alternative: run from a repo checkout (for AI Lens development)</summary>
96
+
97
+ ```bash
98
+ docker compose -f docker-compose.yml -f docker-compose.local.yml up -d postgres app
99
+ AI_LENS_SERVER_URL=http://localhost:3000 npx -y ai-lens import claude-code --days 7
100
+ ```
101
+ </details>
102
+
103
+ Use a separate `AI_LENS_DATA_DIR` (e.g. `~/.ai-lens-local`) to isolate a local profile — its own `config.json` + spool — from your prod setup.
104
+
67
105
  <details>
68
106
  <summary>Manual hook setup</summary>
69
107
 
@@ -328,6 +366,6 @@ Build jobs trigger only when relevant files change (Dockerfile, server/**, dashb
328
366
 
329
367
  ## Requirements
330
368
 
331
- - Node.js 20+
369
+ - Node.js 22.5+ (the CLI reads Cursor history via the built-in `node:sqlite`, available from 22.5; the server container runs Node 20 and is unaffected)
332
370
  - Docker + Docker Compose (for production deployment)
333
371
  - PostgreSQL 16 (for local development without Docker)
package/bin/ai-lens.js CHANGED
@@ -38,6 +38,14 @@ switch (command) {
38
38
  await deleteSessions();
39
39
  break;
40
40
  }
41
+ case 'local-server':
42
+ case 'install-local-server': {
43
+ // `install-local-server` is an alias that always means `local-server up`.
44
+ if (command === 'install-local-server') process.env.AI_LENS_LOCAL_SUBVERB = 'up';
45
+ const { default: localServer } = await import('../cli/local-server.js');
46
+ await localServer();
47
+ break;
48
+ }
41
49
  case 'version':
42
50
  case '--version':
43
51
  case '-v': {
@@ -67,18 +75,25 @@ switch (command) {
67
75
  console.log(' --use-repo-path Run capture.js from this package; skip copy to ~/.ai-lens/client/');
68
76
  console.log(' remove Remove AI Lens hooks and client files');
69
77
  console.log(' status Run diagnostics and generate a status report');
70
- console.log(' import <source> Import local history (source: claude-code)');
71
- console.log(' --days N Window in days (default 30; 0 = all history)');
78
+ console.log(' import <source> Import local history (sources: claude-code, cursor)');
79
+ console.log(' --days N Window in days (default 90; 0 = all history)');
72
80
  console.log(' --since DATE Import from YYYY-MM-DD instead of --days');
73
81
  console.log(' --from D --to D Import a bounded YYYY-MM-DD..YYYY-MM-DD window (idempotent re-import)');
74
82
  console.log(' --dry-run Scan + count, write nothing');
75
83
  console.log(' --projects LIST Only these project paths (comma-separated)');
84
+ console.log(' --cursor-db PATH Cursor IDE state.vscdb override (default: auto-detect)');
85
+ console.log(' --cursor-chats PATH Cursor CLI chats dir override (default: ~/.cursor/chats)');
76
86
  console.log(' list-sessions List your own sessions');
77
87
  console.log(' --days N --source S --limit N --json');
78
88
  console.log(' find-session <q> Find your sessions by id / project / source substring');
79
89
  console.log(' --days N --source S --limit N --json');
80
90
  console.log(' delete-sessions Delete your own sessions (dry-run unless --yes)');
81
91
  console.log(' <id…> | --from D --to D [--source S] [--days N] [--yes] [--json]');
92
+ console.log(' local-server <up|down|status> Run a private local server in Docker (your data stays local)');
93
+ console.log(' up [--port N] [--yes] [--recreate] Bring it up, then offer to point the client at it');
94
+ console.log(' down [--purge] Stop it (--purge also deletes the data volume)');
95
+ console.log(' status [--port N] Show container + health status');
96
+ console.log(' install-local-server Alias for `local-server up`');
82
97
  console.log(' version Show package version and commit hash');
83
98
  process.exit(command ? 1 : 0);
84
99
  }
package/cli/hooks.js CHANGED
@@ -3,6 +3,7 @@ import { join, dirname } from 'node:path';
3
3
  import { homedir, release } from 'node:os';
4
4
  import { fileURLToPath } from 'node:url';
5
5
  import { execFileSync } from 'node:child_process';
6
+ import { CONFIG_PATH } from '../client/config.js';
6
7
 
7
8
  const __dirname = dirname(fileURLToPath(import.meta.url));
8
9
  const PKG_ROOT = join(__dirname, '..');
@@ -24,9 +25,12 @@ export function getVersionInfo() {
24
25
  return { version, commit };
25
26
  }
26
27
 
27
- // Stable install location for client files
28
+ // Stable install location for client files. The install dir stays a fixed
29
+ // ~/.ai-lens (its path is baked into hook configs and must not move), but
30
+ // CONFIG_PATH is imported from client/config.js so config reads/writes honor
31
+ // AI_LENS_DATA_DIR — same as the client runtime (was previously hardcoded here,
32
+ // which broke DATA_DIR-based isolation).
28
33
  const CLIENT_INSTALL_DIR = join(homedir(), '.ai-lens', 'client');
29
- const CONFIG_PATH = join(homedir(), '.ai-lens', 'config.json');
30
34
 
31
35
  // Hooks always point to the installed copy at ~/.ai-lens/client/capture.js
32
36
  export const CAPTURE_PATH = join(CLIENT_INSTALL_DIR, 'capture.js');
@@ -502,6 +506,11 @@ const CLAUDE_HOOK_SPEC = {
502
506
  TaskCompleted: { matcher: '' },
503
507
  InstructionsLoaded: { matcher: '' },
504
508
  UserPromptExpansion: { matcher: '' },
509
+ // Project attribution: fires when the session's cwd changes mid-flight.
510
+ // capture.js updates the session→project cache from new_cwd so subsequent
511
+ // events attribute to the right project (cwd otherwise only arrives at
512
+ // SessionStart). Metric-neutral server-side (OBSERVABILITY_EVENT_TYPES).
513
+ CwdChanged: { matcher: '' },
505
514
  };
506
515
 
507
516
  const CURSOR_HOOK_NAMES = [
@@ -7,6 +7,11 @@
7
7
  * existing sender ship them to POST /api/events. Historical timestamps are kept
8
8
  * verbatim.
9
9
  *
10
+ * Source-agnostic plumbing (window/flag resolution, coverage de-overlap, project
11
+ * filter, ledger, unified-event build + spool delivery) lives in ./common.js and
12
+ * is shared with the other import sources; this module keeps only the Claude Code
13
+ * reader, the claude_code event_id namespace, and the local-history preview.
14
+ *
10
15
  * Re-import is idempotent: every event carries a DETERMINISTIC event_id
11
16
  * (computeEventId below) and the server inserts with ON CONFLICT DO NOTHING
12
17
  * against the `events.event_id` UNIQUE constraint, so re-running over an
@@ -19,22 +24,34 @@
19
24
  * on the fast path; a bounded `--from/--to` re-import bypasses the ledger and
20
25
  * leans entirely on the event_id dedup.
21
26
  */
22
- import { createReadStream, existsSync, readdirSync, statSync, readFileSync, writeFileSync, mkdirSync, renameSync, realpathSync } from 'node:fs';
27
+ import { createReadStream, existsSync, readdirSync, statSync } from 'node:fs';
23
28
  import { join, basename } from 'node:path';
24
29
  import { homedir } from 'node:os';
25
30
  import { createInterface } from 'node:readline';
26
31
  import { createHash } from 'node:crypto';
27
- import { spawn } from 'node:child_process';
28
- import { fileURLToPath } from 'node:url';
29
32
 
30
- import { writeToSpool, canonicalizeProjectPath, deterministicEventId } from '../../client/capture.js';
31
- import { PENDING_DIR, SENDING_DIR, DATA_DIR, ensureDataDir, getGitIdentity, getGitMetadata, getServerUrl, getAuthToken, getMonitoredProjects } from '../../client/config.js';
33
+ import { canonicalizeProjectPath, deterministicEventId } from '../../client/capture.js';
34
+ import { DATA_DIR, ensureDataDir, getGitIdentity, getGitMetadata, getServerUrl, getAuthToken, getMonitoredProjects } from '../../client/config.js';
32
35
  import { mapTranscript } from './transcript-map.js';
33
- import { info, success, warn, error, heading, detail, blank, progress, progressDone } from '../logger.js';
36
+ import {
37
+ TERMINAL_DORMANT_MS,
38
+ validateFlags, resolveCutoff, resolveLowerBound, resolveUpperBound,
39
+ resolveProjectFilter, projectMatches,
40
+ fetchCoverage, hasDeliverableIdentity, sliceEvents,
41
+ ledgerCovers, clipWindow, createImportShipper,
42
+ } from './common.js';
43
+ import { info, success, warn, error, heading, detail, blank, progress } from '../logger.js';
44
+
45
+ // Re-export the source-agnostic helpers the test-suite (and callers) import from
46
+ // this module — the stable public surface after the Phase-0 split into common.js.
47
+ export {
48
+ validateFlags, resolveCutoff, resolveLowerBound, resolveUpperBound,
49
+ resolveProjectFilter, projectMatches, sliceEvents, hasDeliverableIdentity, ledgerCovers,
50
+ };
34
51
 
35
52
  const PROJECTS_DIR = join(homedir(), '.claude', 'projects');
36
53
  const LEDGER_PATH = join(DATA_DIR, 'import-state', 'claude-code.json');
37
- const __dirname = fileURLToPath(new URL('.', import.meta.url));
54
+ const DAY_MS = 86400_000;
38
55
 
39
56
  // TokenUsage gets a LIVE-compatible event_id (sha of the assistant line uuid,
40
57
  // exactly as client/capture.js:1363) so an imported call dedups against the same
@@ -46,52 +63,6 @@ export function computeEventId(ev) {
46
63
  return deterministicEventId(`claude_code:import:v1:${ev._seed}`);
47
64
  }
48
65
 
49
- const DAY_MS = 86400_000;
50
- // Drain the spool to the server every DRAIN_BATCH spooled events. MUST stay well
51
- // under sender.js MAX_QUEUE_SIZE (10_000) — once pending exceeds that, the sender
52
- // DROPS the oldest overflow files (data loss). Draining in batches also lets us
53
- // commit the ledger only AFTER a batch's events have actually shipped, so a failed
54
- // send never leaves files marked covered. (sender.js can't be imported for the
55
- // constant — it self-invokes main() on import.)
56
- const DRAIN_BATCH = 4000;
57
- // A transcript untouched for ≥ this long is treated as a finished session, so its
58
- // terminal Stop/SubagentStop marker is safe to emit (matches the 5h chain-gap).
59
- const TERMINAL_DORMANT_MS = 5 * 60 * 60 * 1000;
60
-
61
- const isValidDate = (s) => typeof s === 'string' && !Number.isNaN(Date.parse(s));
62
-
63
- /** Validate flags. Returns an error string, or null when ok. */
64
- export function validateFlags({ days, since, from, to }) {
65
- // --from/--to define an explicit [from, to] window (lower = from, upper = to).
66
- if (from != null || to != null) {
67
- if (since != null) return 'Use either --since or --from/--to, not both.';
68
- if (from != null && !isValidDate(from)) return `Invalid --from "${from}". Use YYYY-MM-DD.`;
69
- if (to != null && !isValidDate(to)) return `Invalid --to "${to}". Use YYYY-MM-DD.`;
70
- if (from != null && to != null && Date.parse(from) > Date.parse(to)) {
71
- return `--from "${from}" is after --to "${to}".`;
72
- }
73
- return null;
74
- }
75
- if (since != null) {
76
- if (!isValidDate(since)) return `Invalid --since "${since}". Use YYYY-MM-DD.`;
77
- } else if (!Number.isInteger(days) || days < 0) {
78
- return `Invalid --days "${days}". Use a non-negative integer (0 = all history).`;
79
- }
80
- return null;
81
- }
82
-
83
- /**
84
- * Resolve the lower-bound cutoff ISO string from flags. `--from` and `--since`
85
- * are equivalent lower bounds; `--days 0` / no window ⇒ epoch (all history).
86
- */
87
- export function resolveCutoff({ days, since, from }, now = new Date()) {
88
- const lower = from || since;
89
- if (lower) return new Date(Date.parse(lower)).toISOString();
90
- if (days === 0) return new Date(0).toISOString();
91
- const d = Number.isInteger(days) && days >= 0 ? days : 90;
92
- return new Date(now.getTime() - d * DAY_MS).toISOString();
93
- }
94
-
95
66
  /**
96
67
  * Cheap preview for the init import offer — how many local transcripts (≈ sessions)
97
68
  * fall within the window and their date span, by file mtime (no file reads). Lets
@@ -117,122 +88,6 @@ export function previewClaudeCode({ days = 90, dir = PROJECTS_DIR } = {}, now =
117
88
  };
118
89
  }
119
90
 
120
- /**
121
- * Resolve the INCLUSIVE lower bound from `--from` (or null when open-ended).
122
- * Unlike the file-level `cutoff` (which only decides whether to read a file),
123
- * this clips PER EVENT so an explicit `--from/--to` window means exactly the
124
- * events in [from 00:00, to+1day) — a file touched inside the window no longer
125
- * drags in its older events. Only `--from`/`--to` engage this precise clipping;
126
- * `--days`/`--since` keep their looser file-level semantics.
127
- */
128
- export function resolveLowerBound({ from }) {
129
- if (!from || !isValidDate(from)) return null;
130
- return new Date(Date.parse(from)).toISOString();
131
- }
132
-
133
- /**
134
- * Resolve the EXCLUSIVE upper bound from `--to` (or null when open-ended).
135
- * `--to 2026-06-07` includes the whole of June 7 ⇒ exclusive bound is the next
136
- * midnight (2026-06-08T00:00Z).
137
- */
138
- export function resolveUpperBound({ to }) {
139
- if (!to || !isValidDate(to)) return null;
140
- return new Date(Date.parse(to) + DAY_MS).toISOString();
141
- }
142
-
143
- /** Expand a leading ~ and resolve to a canonical project (git) root. */
144
- function normalizeProjectArg(p) {
145
- let s = (p || '').trim();
146
- if (!s) return null;
147
- if (s === '~' || s.startsWith('~/')) s = join(homedir(), s.slice(1));
148
- let real = s;
149
- try { real = realpathSync(s); } catch { /* path may not exist on disk — keep as given */ }
150
- return canonicalizeProjectPath(real) || real;
151
- }
152
-
153
- /**
154
- * Resolve the effective project filter, mirroring live capture by default:
155
- * --projects A,B → explicit list (each entry ~-expanded + git-root canonicalized)
156
- * --projects all → no filter, even when one is configured (explicit override)
157
- * (no flag) → the live-capture filter (AI_LENS_PROJECTS / config
158
- * `projects`), so import never ships history that live
159
- * capture would have dropped via project_filter.
160
- * `monitored` is getMonitoredProjects() output: normalized absolute paths or
161
- * null. Matching semantics are identical to live capture's pathContains
162
- * (entry === path || path inside entry), so no extra canonicalization needed.
163
- * Returns { filter: string[]|null, source: 'flag'|'config'|null }.
164
- */
165
- export function resolveProjectFilter(projectsFlag, monitored) {
166
- if (typeof projectsFlag === 'string' && projectsFlag.trim()) {
167
- if (projectsFlag.trim().toLowerCase() === 'all') return { filter: null, source: null };
168
- return { filter: projectsFlag.split(',').map(normalizeProjectArg).filter(Boolean), source: 'flag' };
169
- }
170
- if (Array.isArray(monitored) && monitored.length > 0) return { filter: monitored, source: 'config' };
171
- return { filter: null, source: null };
172
- }
173
-
174
- /** Path-boundary match (so /repo does NOT match /repo2). */
175
- export function projectMatches(projectPath, filters) {
176
- if (!filters) return true;
177
- if (!projectPath) return false;
178
- return filters.some((pf) => projectPath === pf || projectPath.startsWith(pf + '/'));
179
- }
180
-
181
- /**
182
- * Ask the server, for these session_ids, the earliest LIVE (non-import) event
183
- * timestamp it already has → { sessionId: minIso }. Best-effort: on any failure
184
- * (old server without the route, network) returns {} so we import everything
185
- * (TokenUsage still dedups on its live-compatible id; nothing is lost).
186
- */
187
- export async function fetchCoverage(sessionIds, { fetchImpl = globalThis.fetch } = {}) {
188
- const out = {};
189
- if (!sessionIds.length || !fetchImpl) return out;
190
- const base = getServerUrl();
191
- const token = getAuthToken();
192
- // Auth: token preferred, else git-identity headers so coverage also works in
193
- // self-host personal mode (no token) — same fallback the ingest path uses.
194
- const ident = token ? null : getGitIdentity(process.cwd());
195
- const authHeaders = token
196
- ? { 'X-Auth-Token': token }
197
- : (ident?.email ? { 'X-Developer-Git-Email': ident.email, ...(ident.name ? { 'X-Developer-Name': ident.name } : {}) } : {});
198
- const CHUNK = 400;
199
- for (let i = 0; i < sessionIds.length; i += CHUNK) {
200
- const batch = sessionIds.slice(i, i + CHUNK);
201
- try {
202
- const res = await fetchImpl(new URL('/api/events/coverage', base), {
203
- method: 'POST',
204
- headers: { 'Content-Type': 'application/json', ...authHeaders },
205
- body: JSON.stringify({ session_ids: batch }),
206
- });
207
- if (res.ok) Object.assign(out, await res.json());
208
- } catch { /* best-effort */ }
209
- }
210
- return out;
211
- }
212
-
213
- /**
214
- * Can the importer actually deliver events? A personal auth token OR a resolvable
215
- * git email (the ingest path accepts either). Pure-ish; takes the two resolvers so
216
- * it's testable without touching real git/env.
217
- */
218
- export function hasDeliverableIdentity(token, gitEmail) {
219
- return !!token || !!gitEmail;
220
- }
221
-
222
- /**
223
- * Drop events at/after the live-coverage boundary for their session, so import
224
- * only adds the pre-live backlog. No boundary ⇒ keep all. (Pure, for tests.)
225
- */
226
- export function sliceEvents(events, coverage) {
227
- return events.filter((ev) => {
228
- const boundary = coverage[ev.session_id];
229
- if (!boundary) return true;
230
- // Compare instants, not strings — server timestamps may lack the ms ('…00Z'
231
- // vs '…00.000Z'), which would mis-sort a boundary-equal event lexically.
232
- return Date.parse(ev.timestamp) < Date.parse(boundary);
233
- });
234
- }
235
-
236
91
  function walkJsonl(dir) {
237
92
  const out = [];
238
93
  let entries;
@@ -245,23 +100,6 @@ function walkJsonl(dir) {
245
100
  return out;
246
101
  }
247
102
 
248
- function loadLedger() {
249
- try { return JSON.parse(readFileSync(LEDGER_PATH, 'utf-8')); } catch { return {}; }
250
- }
251
- function saveLedger(ledger) {
252
- mkdirSync(join(DATA_DIR, 'import-state'), { recursive: true });
253
- const tmp = LEDGER_PATH + '.tmp';
254
- writeFileSync(tmp, JSON.stringify(ledger));
255
- renameSync(tmp, LEDGER_PATH);
256
- }
257
-
258
- /** A file is already covered if a prior run imported a wider-or-equal window AND the file is unchanged. */
259
- export function ledgerCovers(entry, cutoff, fp) {
260
- if (!entry || !entry.fingerprint) return false;
261
- if (entry.fingerprint.mtimeMs !== fp.mtimeMs || entry.fingerprint.size !== fp.size) return false;
262
- return entry.complete_all || (entry.covered_cutoff && entry.covered_cutoff <= cutoff);
263
- }
264
-
265
103
  /** Read one transcript file fully (line-by-line, never JSON.parse(whole_file)). */
266
104
  async function readTranscript(filePath) {
267
105
  const lines = [];
@@ -290,23 +128,6 @@ function firstSessionId(lines, fallback) {
290
128
  return fallback;
291
129
  }
292
130
 
293
- async function drainSpool({ timeoutMs = 120_000 } = {}) {
294
- const senderPath = join(__dirname, '..', '..', 'client', 'sender.js');
295
- const pendingCount = () => { try { return readdirSync(PENDING_DIR).filter((f) => f.endsWith('.json')).length; } catch { return 0; } };
296
- const sendingCount = () => { try { return readdirSync(SENDING_DIR).filter((f) => f.endsWith('.json')).length; } catch { return 0; } };
297
- const deadline = Date.now() + timeoutMs;
298
- while (Date.now() < deadline) {
299
- if (pendingCount() === 0 && sendingCount() === 0) return true;
300
- await new Promise((resolve) => {
301
- const child = spawn(process.execPath, [senderPath], { stdio: 'ignore' });
302
- child.on('exit', resolve);
303
- child.on('error', resolve);
304
- });
305
- await new Promise((r) => setTimeout(r, 400));
306
- }
307
- return pendingCount() === 0 && sendingCount() === 0;
308
- }
309
-
310
131
  export default async function importClaudeCode(flags) {
311
132
  const {
312
133
  days = 90, since = null, from = null, to = null, dryRun = false, projects = null,
@@ -320,10 +141,17 @@ export default async function importClaudeCode(flags) {
320
141
  return;
321
142
  }
322
143
  // Need a way to attribute events: a token, OR a git email (self-host personal
323
- // mode ships via the git-identity fallback, just like live capture).
324
- if (!dryRun && !hasDeliverableIdentity(getAuthToken(), getGitIdentity(process.cwd())?.email)) {
325
- error('No identity to attribute events. Run `npx ai-lens init` first, or set a git user.email (or AI_LENS_AUTH_TOKEN).');
326
- return;
144
+ // mode ships via the git-identity fallback, just like live capture). Checked in
145
+ // dry-run too (as a warning, not a hard stop) so a green dry-run never hides a
146
+ // delivery blocker the real run would hit.
147
+ if (!hasDeliverableIdentity(getAuthToken(), getGitIdentity(process.cwd())?.email)) {
148
+ const idMsg = 'No identity to attribute events. Set git user.email AND user.name (self-host ingest needs both), or AI_LENS_AUTH_TOKEN, or run `npx ai-lens init`.';
149
+ if (dryRun) {
150
+ warn(`${idMsg} (dry-run continues, but a real import would NOT deliver until identity resolves.)`);
151
+ } else {
152
+ error(idMsg);
153
+ return;
154
+ }
327
155
  }
328
156
  if (noRedact) warn('⚠ --no-redact: secrets in raw transcripts are NOT redacted client-side and persist locally in the spool if a send fails. Use only for debugging.');
329
157
 
@@ -351,7 +179,11 @@ export default async function importClaudeCode(flags) {
351
179
  detail('Pass --projects all to import everything, or --projects A,B to override.');
352
180
  }
353
181
  ensureDataDir();
354
- const ledger = loadLedger();
182
+ // Shared ship engine: spool → batched drain → ledger-commit-after-ship.
183
+ const shipper = createImportShipper({
184
+ ledgerPath: LEDGER_PATH, dryRun, source: 'claude_code', computeEventId, noRedact,
185
+ beforeDrain: (n) => progress(`shipping ${n} event(s) to the server…`),
186
+ });
355
187
  // Human-readable window label for the progress lines.
356
188
  const windowLabel = (from || to)
357
189
  ? `${from || 'start'}…${to || 'now'}`
@@ -366,23 +198,6 @@ export default async function importClaudeCode(flags) {
366
198
  let lastDate = null;
367
199
  const nowMs = Date.now();
368
200
 
369
- // Batched shipping: spool events, and every DRAIN_BATCH events drain the spool
370
- // to the server, committing the ledger only for files whose events have shipped.
371
- let inFlight = 0; // events spooled since the last successful drain
372
- const pendingCommit = []; // {filePath, entry} for fully-written, not-yet-committed files
373
- let drainFailed = false;
374
- const flush = async () => {
375
- if (dryRun) { inFlight = 0; return true; }
376
- progress(`shipping ${inFlight} event(s) to the server…`);
377
- const ok = await drainSpool();
378
- if (!ok) { drainFailed = true; return false; }
379
- inFlight = 0;
380
- for (const c of pendingCommit) ledger[c.filePath] = c.entry;
381
- if (pendingCommit.length) saveLedger(ledger);
382
- pendingCommit.length = 0;
383
- return true;
384
- };
385
-
386
201
  // Pass 1 — cheap shortlist (mtime + ledger), no file reads. A transcript's
387
202
  // filename basename IS the session UUID, so we can gather session ids for the
388
203
  // coverage query without reading anything.
@@ -393,7 +208,7 @@ export default async function importClaudeCode(flags) {
393
208
  try { st = statSync(filePath); } catch { continue; }
394
209
  const fp = { mtimeMs: Math.trunc(st.mtimeMs), size: st.size };
395
210
  if (st.mtime.toISOString() < cutoff) { filesSkipped++; continue; } // last write before window
396
- if (!rangeMode && ledgerCovers(ledger[filePath], cutoff, fp)) { filesSkipped++; continue; }
211
+ if (!rangeMode && ledgerCovers(shipper.ledgerEntry(filePath), cutoff, fp)) { filesSkipped++; continue; }
397
212
  candidates.push({ filePath, fp });
398
213
  }
399
214
 
@@ -407,7 +222,7 @@ export default async function importClaudeCode(flags) {
407
222
  // Pass 2 — read, map, de-overlap, ship.
408
223
  let filesProcessed = 0;
409
224
  for (const { filePath, fp } of candidates) {
410
- if (drainFailed) break;
225
+ if (shipper.failed) break;
411
226
  filesProcessed++;
412
227
  progress(`${dryRun ? 'scanning' : 'importing'}… ${filesProcessed}/${candidates.length} transcript(s) · ${eventCount} event(s) from ${sessionCount} session(s)`);
413
228
  const { lines, lastTs } = await readTranscript(filePath);
@@ -429,20 +244,14 @@ export default async function importClaudeCode(flags) {
429
244
  const mapped = mapTranscript(lines, { sessionId, projectPath, fileId, isSubagentFile, agentId, agentSlug: null, emitTerminal });
430
245
  if (mapped.length === 0) { filesSkipped++; continue; }
431
246
  // De-overlap with live coverage, then clip to the explicit [from, to] window.
432
- const events = sliceEvents(mapped, coverage).filter((ev) => {
433
- if (lowerMs == null && upperMs == null) return true;
434
- const t = Date.parse(ev.timestamp);
435
- if (lowerMs != null && t < lowerMs) return false;
436
- if (upperMs != null && t >= upperMs) return false;
437
- return true;
438
- });
247
+ const events = clipWindow(sliceEvents(mapped, coverage), lowerMs, upperMs);
439
248
  if (events.length === 0) {
440
249
  // Nothing left for this file: either live already owns the session, or every
441
250
  // event fell outside the [from, to] window. Only the former is "live skipped";
442
251
  // only a full (unbounded) import may mark the file covered in the ledger.
443
252
  if (!rangeMode) {
444
253
  liveSkipped++;
445
- if (!dryRun) pendingCommit.push({ filePath, entry: { covered_cutoff: cutoff, complete_all: days === 0, fingerprint: fp } });
254
+ if (!dryRun) shipper.deferCommit(filePath, { covered_cutoff: cutoff, complete_all: days === 0, fingerprint: fp });
446
255
  }
447
256
  continue;
448
257
  }
@@ -450,40 +259,18 @@ export default async function importClaudeCode(flags) {
450
259
  const ident = getGitIdentity(cwd);
451
260
  const gitMeta = getGitMetadata(projectPath);
452
261
 
453
- for (const ev of events) {
454
- const unified = {
455
- event_id: computeEventId(ev),
456
- source: 'claude_code',
457
- session_id: ev.session_id,
458
- type: ev.type,
459
- project_path: ev.project_path,
460
- timestamp: ev.timestamp,
461
- data: { ...ev.data, _import: true }, // marks import rows so /coverage ignores them
462
- raw: ev.raw,
463
- developer_email: ident.email || null,
464
- developer_name: ident.name || null,
465
- git_remote: gitMeta.git_remote || null,
466
- git_branch: gitMeta.git_branch || null,
467
- git_commit: gitMeta.git_commit || null,
468
- };
469
- if (!dryRun) {
470
- if (noRedact) writeRaw(unified); else writeToSpool(unified);
471
- inFlight++;
472
- // Drain BEFORE pending can approach the sender's overflow cap. Checked per
473
- // event so a single huge transcript can't blow past it mid-file.
474
- if (inFlight >= DRAIN_BATCH) { if (!(await flush())) break; }
475
- }
476
- eventCount++;
477
- }
478
- if (drainFailed) break;
262
+ // Drain BEFORE pending can approach the sender's overflow cap (handled inside
263
+ // shipper.ship per DRAIN_BATCH), so a single huge transcript can't blow past it.
264
+ eventCount += await shipper.ship(events, { ident, gitMeta });
265
+ if (shipper.failed) break;
479
266
 
480
267
  filesIncluded++;
481
268
  sessionCount++;
482
- if (!isSubagentFile) (lastTs >= new Date(nowMs - analysisMaxAgeDays * 86400_000).toISOString() ? withinWindow++ : older++);
269
+ if (!isSubagentFile) (lastTs >= new Date(nowMs - analysisMaxAgeDays * DAY_MS).toISOString() ? withinWindow++ : older++);
483
270
  if (!lastDate || lastTs > lastDate) lastDate = lastTs;
484
271
  // Defer the ledger commit until this file's events have actually shipped (flush()).
485
272
  // A bounded `--from/--to` re-import never records coverage (it didn't read to "now").
486
- if (!dryRun && !rangeMode) pendingCommit.push({ filePath, entry: { covered_cutoff: cutoff, complete_all: days === 0, fingerprint: fp } });
273
+ if (!dryRun && !rangeMode) shipper.deferCommit(filePath, { covered_cutoff: cutoff, complete_all: days === 0, fingerprint: fp });
487
274
  if (filesIncluded % 25 === 0) info(` …${filesIncluded} files, ${eventCount} events — last ${(lastDate || '').slice(0, 10)}`);
488
275
  }
489
276
 
@@ -495,18 +282,10 @@ export default async function importClaudeCode(flags) {
495
282
  }
496
283
 
497
284
  info(`Spooled ${eventCount} event(s) from ${filesIncluded} session(s). Shipping…`);
498
- const drained = await flush(); // ship the final partial batch + commit remaining ledger
285
+ const drained = await shipper.flush(); // ship the final partial batch + commit remaining ledger
499
286
  if (drained) success(`Imported ${sessionCount} session(s), ${eventCount} event(s).`);
500
287
  else warn(`Some events couldn't be shipped — their sessions were NOT marked imported and will retry on the next run. Check \`ai-lens status\`.`);
501
288
 
502
289
  detail(`Within ${analysisMaxAgeDays}d: ${withinWindow} session(s) eligible for auto-analysis. Older: ${older} (skipped by max-age ${analysisMaxAgeDays}, available on-demand).`);
503
290
  detail('Open /me to see your imported history.');
504
291
  }
505
-
506
- /** --no-redact path: write the event to pending/ WITHOUT client redaction. */
507
- function writeRaw(unified) {
508
- const filename = `${unified.event_id}.json`;
509
- const tmp = join(PENDING_DIR, filename + '.tmp.' + process.pid);
510
- writeFileSync(tmp, JSON.stringify(unified));
511
- renameSync(tmp, join(PENDING_DIR, filename));
512
- }