pi-fast-resume 1.1.3 → 1.3.0

package/README.md

@@ -4,7 +4,7 @@
 
 **Instant session picker for [pi](https://github.com/earendil-works/pi-coding-agent)**
 
-_Reads the head + tail of each file instead of the full JSONL — first results in **6ms**._
+_Reads just enough of each file to show the title row — first results in **6ms**._
 
 [![pi extension](https://img.shields.io/badge/pi-extension-blueviolet)](https://github.com/earendil-works/pi-coding-agent)
 [![license](https://img.shields.io/badge/license-MIT-blue)](./LICENSE)
@@ -16,7 +16,7 @@ _Reads the head + tail of each file instead of the full JSONL — first results
 > **`/resume` takes 5.6 seconds** when you have 1,700+ sessions.
 > pi-fast-resume's `/fast-resume` takes **6 milliseconds**.
 
-Same picker UI and keybindings as `/resume`. The difference is pi-fast-resume reads at most ~24KB of each file (a 16KB head + up to 8KB tail) instead of the full JSONL. The head carries the header and first user message; the tail recovers the latest session name, which pi appends at EOF on `/rename`. Everything between is full message history the picker never shows. Search matches against the first message only (see [Known Limitations](#known-limitations)).
+Same picker UI and keybindings as `/resume`. The difference is pi-fast-resume never parses the full JSONL. For each file it streams complete lines forward just until the first user message (the title), then reads a bounded tail near EOF to recover the latest session name (which pi appends on `/rename`). Everything between is full message history the picker never shows. Search matches against the first message only (see [Known Limitations](#known-limitations)).
 
 ```
 ──────────────────────────────────────────────────────────
@@ -54,7 +54,7 @@ Tested with **1,771 sessions, 1.46 GB** of JSONL data on disk.
 | ------------------------------------------- | --------- | ------------------------------------------ |
 | `SessionManager.listAll()` (current)        | ~5,600 ms | Full parse of every file                   |
 | DuckDB `read_ndjson` full query             | ~2,560 ms | Still reads all 1.46 GB, but multithreaded |
-| Node.js partial read (16 KB/file)           | ~730 ms   | All 1,771 sessions                         |
+| Node.js streaming partial read (this extension) | ~370 ms | All ~2,550 sessions; stops at first user message + bounded tail |
 | DuckDB persistent index (query all)         | ~49 ms    | After one-time build                       |
 | `node:sqlite` persistent index (query)      | ~52 ms    | Zero external deps                         |
 | **pi-fast-resume, first 30 sessions**       | **~6 ms** | **Streaming display**                      |
@@ -165,7 +165,7 @@ Press `Tab` to switch to **all sessions** — shows every session pi knows about
 ## How it works
 
 ```
-stat() all .jsonl files ──────► sort by mtime ──────► read 16KB of top 30
+stat() all .jsonl files ──────► sort by mtime ──────► stream top 30 forward
       (~100ms)                   (recent first)          (~6ms)
                                                             │
                                                             ▼
@@ -180,7 +180,7 @@ stat() all .jsonl files ──────► sort by mtime ──────
 
 1. **`stat()` all session files** — collect paths and mtimes (~100 ms for 1,700 files)
 2. **Sort by mtime descending** — most recent sessions first
-3. **Read head (16KB) + tail (≤8KB)** of the top 30 files — extract header, first user message, and latest session name (~6 ms)
+3. **Stream each file forward** line by line until the first user message, then read a bounded tail at EOF for the latest rename name (~6 ms)
 4. **Show picker** — user can navigate, filter, and select immediately
 5. **Background load** — remaining sessions stream in batches of 50, non-blocking
 6. **Tab to switch scope** — filter to current project or show everything
@@ -248,7 +248,7 @@ None optimize the `/resume` picker itself — they either still fully parse ever
 
 ## Known Limitations
 
-The 16KB partial-read tradeoff that gives pi-fast-resume its speed comes with one functional gap vs. the built-in `/resume`:
+The partial-read tradeoff that gives pi-fast-resume its speed comes with functional gaps vs. the built-in `/resume`. The forward stream reads exactly as many bytes as the first user message needs (no fixed window), so oversized first messages — `<skill>` injections, long pastes, base64 images — are parsed correctly. A rename is recovered only if it lives within a bounded tail near EOF (32 KB by default); a rename buried under more continued activity than that falls back to `firstMessage`.
 
 | Area | Built-in `/resume` | pi-fast-resume | Impact |
 | ---- | ------------------ | -------------- | ------ |

package/fast-resume.ts

@@ -1,9 +1,10 @@
 /**
  * pi-fast-resume — Fast session picker for pi
  *
- * Reads only the first 16KB of each session file (header + first messages)
- * instead of parsing the entire JSONL. Shows results instantly with
- * incremental background loading.
+ * Reads only enough of each session file to render the title row — streaming
+ * forward line by line until the first user message, plus a bounded tail near
+ * EOF for the latest rename name — instead of parsing the entire JSONL. Shows
+ * results instantly with incremental background loading.
  *
  * Mirrors the exact TUI layout and keybindings of pi's built-in /resume.
  *
@@ -36,10 +37,10 @@
  *   exact phrase           "node cve"       case-insensitive substring
  *   regex                  re:<pattern>      RegExp search (case-insensitive)
  *
- * Note on search depth: pi-fast-resume only reads the first 16KB of each
- * session file, so search matches against id + name + firstMessage + cwd.
+ * Note on search depth: pi-fast-resume stops reading each file at the first
+ * user message, so search matches against id + name + firstMessage + cwd.
  * Upstream /resume matches against all messages (allMessagesText). This
- * tradeoff is by design — the 6ms load time depends on partial reads.
+ * tradeoff is by design — the fast load time depends on partial reads.
  */
 
 import type { ExtensionAPI, ExtensionCommandContext } from "@earendil-works/pi-coding-agent";
@@ -71,6 +72,8 @@ import {
   scanAllSessionDirs,
   scanSessionDir,
   loadSessionHeaders,
+  loadSessionHeadersForward,
+  resolveSessionName,
   sortByModified,
   sortByModifiedDesc,
   filterByCwd,
@@ -132,7 +135,7 @@ function loadCurrentSessionsImmediate(
 ): SessionHeader[] {
   if (!sessionDir) return [];
   const metas = sortByModifiedDesc(scanSessionDir(sessionDir));
-  let headers = loadSessionHeaders(metas);
+  let headers = loadSessionHeadersForward(metas);
   if (!usesDefaultSessionDir) {
     // Custom session dirs may contain sessions from multiple cwds; filter to
     // the current one, matching SessionManager.list behavior.
@@ -674,6 +677,17 @@ class FastResumePicker extends Container {
   private loadingAbort: AbortController | null = null;
   private allLoadSeq = 0;
 
+  // Deferred rename-name resolution. The picker displays rows immediately
+  // with forward-only headers (fast: ~80ms for 2.5k sessions); the latest rename
+  // name (which pi appends at EOF, past the forward stop) is resolved per file
+  // in the background and applied in-place, so a row's name pops in without
+  // blocking the initial render. See resolveSessionName in scanner.ts.
+  private metaByPath = new Map<string, SessionFileMeta>();
+  private nameResolveQueue: SessionFileMeta[] = [];
+  private nameResolveScheduled = false;
+  private nameResolveSeq = 0;
+  private nameResolvedPaths = new Set<string>();
+
   private mode: "list" | "rename" = "list";
   private renameTargetPath: string | null = null;
 
@@ -756,16 +770,19 @@ class FastResumePicker extends Container {
     this.sessionList.onSelect = (sessionPath) => {
       this.header.clearStatusTimeout();
       this.loadingAbort?.abort();
+      this.nameResolveSeq++; // cancel any pending name-resolution ticks
       this.done({ sessionPath, cancelled: false });
     };
     this.sessionList.onCancel = () => {
       this.header.clearStatusTimeout();
       this.loadingAbort?.abort();
+      this.nameResolveSeq++;
       this.done({ cancelled: true });
     };
     this.sessionList.onExit = () => {
       this.header.clearStatusTimeout();
       this.loadingAbort?.abort();
+      this.nameResolveSeq++;
       this.done({ cancelled: true });
     };
     this.sessionList.onToggleScope = () => this.toggleScope();
@@ -817,6 +834,17 @@ class FastResumePicker extends Container {
     // Build layout
     this.buildBaseLayout(this.sessionList);
 
+    // Build the path → meta lookup from allMetas (which is a superset of the
+    // current-scope metas both for the default and custom-dir cases), then
+    // enqueue the current-scope sessions for background rename-name resolution.
+    // Rows are already visible with the correct firstMessage; names populate
+    // in-place as their tails resolve.
+    for (const m of allMetas) this.metaByPath.set(m.path, m);
+    const currentMetas = initialCurrentSessions
+      .map((s) => this.metaByPath.get(s.path))
+      .filter((m): m is SessionFileMeta => !!m);
+    this.enqueueNameResolution(currentMetas);
+
     // Start loading current sessions (mark as loaded since we already have them)
     this.currentLoading = false;
     this.header.loading = false;
@@ -956,7 +984,10 @@ class FastResumePicker extends Container {
 
       let headers: SessionHeader[];
       try {
-        headers = loadSessionHeaders(batch);
+        // Forward-only: the rename name will resolve in the background via
+        // resolveSessionName (enqueued below), so rows appear with the correct
+        // firstMessage immediately and names populate in-place.
+        headers = loadSessionHeadersForward(batch);
       } catch (err) {
         const message = err instanceof Error ? err.message : String(err);
         this.allLoading = false;
@@ -969,6 +1000,10 @@ class FastResumePicker extends Container {
       }
 
       allParsed.push(...headers);
+      // Enqueue this batch's metas for background rename-name resolution.
+      // Names will be applied in-place as they resolve; if the user is viewing
+      // "all" scope, newly-named rows also reflect in the active list.
+      this.enqueueNameResolution(batch);
 
       // If we're currently showing "all" scope, update progress
       if (this.scope === "all") {
@@ -985,6 +1020,76 @@ class FastResumePicker extends Container {
     setImmediate(loadBatch);
   }
 
+  // Enqueue session file metas for background rename-name resolution. Each
+  // path is resolved at most once (deduped via nameResolvedPaths); repeated
+  // enqueues for the same path are no-ops. Safe to call for the current-scope
+  // sessions at construction and for each batch of the all-scope background load.
+  private enqueueNameResolution(metas: SessionFileMeta[]): void {
+    for (const m of metas) {
+      if (this.nameResolvedPaths.has(m.path)) continue;
+      this.nameResolvedPaths.add(m.path);
+      this.nameResolveQueue.push(m);
+    }
+    this.scheduleNameResolution();
+  }
+
+  private scheduleNameResolution(): void {
+    if (this.nameResolveScheduled) return;
+    this.nameResolveScheduled = true;
+    setImmediate(() => this.drainNameResolution());
+  }
+
+  // Resolve one cooperative batch of rename names (up to 50 per tick), apply
+  // any found names in-place, and re-render once for the whole batch. Yields
+  // between batches so input stays responsive even while thousands of tail
+  // reads resolve. Aborts cleanly on select/cancel/exit via nameResolveSeq.
+  private drainNameResolution(): void {
+    this.nameResolveScheduled = false;
+    if (this.loadingAbort?.signal.aborted) return;
+    const seq = this.nameResolveSeq;
+    const BATCH = 50;
+    const batch = this.nameResolveQueue.splice(0, BATCH);
+    if (batch.length === 0) return;
+
+    let updatedAny = false;
+    for (const meta of batch) {
+      if (seq !== this.nameResolveSeq) return; // stale — picker exited/aborted
+      const result = resolveSessionName(meta);
+      if (result.found && this.applyNameUpdate(meta.path, result.name)) {
+        updatedAny = true;
+      }
+    }
+
+    if (updatedAny) {
+      const sessions = this.scope === "all" ? (this.allSessions ?? []) : (this.currentSessions ?? []);
+      const showCwd = this.scope === "all";
+      this.sessionList.setSessions(sessions, showCwd);
+      this.tuiRequestRender();
+    }
+
+    if (this.nameResolveQueue.length > 0) this.scheduleNameResolution();
+  }
+
+  // Apply a resolved name to the session with the given path in both the
+  // current- and all-scope caches. The same logical session may appear as
+  // distinct objects in the two caches, so both are updated. Returns whether a
+  // session was found and updated (so the caller can batch re-renders).
+  private applyNameUpdate(path: string, name: string | undefined): boolean {
+    let updated = false;
+    const updateArr = (arr: SessionHeader[] | null) => {
+      if (!arr) return;
+      for (const s of arr) {
+        if (s.path === path) {
+          s.name = name;
+          updated = true;
+        }
+      }
+    };
+    updateArr(this.currentSessions);
+    updateArr(this.allSessions);
+    return updated;
+  }
+
   private toggleScope(): void {
     if (this.scope === "current") {
       this.scope = "all";

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-fast-resume",
-  "version": "1.1.3",
+  "version": "1.3.0",
   "description": "Fast session picker for pi — reads headers + first messages from 16KB partial reads instead of full-file parsing",
   "type": "module",
   "author": "Tom X Nguyen",

package/src/index.ts

@@ -1,5 +1,20 @@
-export { parseSessionFromBuffer, loadSessionHeader, loadSessionHeaders, scanAllSessionDirs, scanSessionDir, sortByModified, sortByModifiedDesc, filterByCwd, matchQuery, canonicalizePath } from "./scanner.js";
-export type { SessionHeader, SessionFileMeta } from "./scanner.js";
+export {
+  parseSessionFromBuffer,
+  scanTailForSessionInfo,
+  loadSessionHeader,
+  loadSessionHeaderForward,
+  loadSessionHeaders,
+  loadSessionHeadersForward,
+  resolveSessionName,
+  scanAllSessionDirs,
+  scanSessionDir,
+  sortByModified,
+  sortByModifiedDesc,
+  filterByCwd,
+  matchQuery,
+  canonicalizePath,
+} from "./scanner.js";
+export type { SessionHeader, SessionFileMeta, TailSessionInfo } from "./scanner.js";
 export {
   parseSearchQuery,
   matchSession,

package/src/scanner.ts

@@ -21,27 +21,76 @@ export interface SessionFileMeta {
   size: number;
 }
 
-const PARTIAL_READ_SIZE = 16_384;
-
-// Size of the trailing read used to recover session_info entries (session
-// names set via /rename or programmatically). These are appended at EOF by
-// SessionManager.appendSessionInfo, so for any session larger than the head
-// window the latest name lands past the partial read and would be invisible —
-// showing "(no messages)" for renamed large sessions. The tail read recovers
-// it, matching pi-core's getSessionName() semantics (latest session_info wins,
-// including explicit name clears).
-const TAIL_READ_SIZE = 8_192;
+// I/O granularity for the streaming forward reader. This is pure performance
+// tuning — it does NOT affect correctness. The reader assembles complete lines
+// across chunks (a line is never truncated mid-JSON), so a line of any size is
+// parsed correctly regardless of this value. It only controls how many bytes
+// each readSync call fetches; smaller = more syscalls for big reads, larger =
+// over-read for tiny sessions that stop early. 16KB is a reasonable middle.
+const READ_CHUNK_SIZE = 16_384;
+
+// Tail read for recovering the latest session_info (the rename name) near EOF.
+//
+// Why a tail read exists at all: pi appends session_info as a new line on every
+// /rename (appendSessionInfo → appendFileSync). The forward pass stops at the
+// first user message, which can be very early in a large file — so the latest
+// rename often lives past the forward pass's stop point and must be recovered
+// from the end of the file.
+//
+// Why the bound is this size and not "scan to EOF": scanning the whole file
+// backward to find a session_info that may not exist (98% of sessions are never
+// renamed) collapses pi-fast-resume's perf to pi-core's (~100× slower). So the
+// tail is bounded. The bound only needs to cover "the rename line itself plus
+// any continued activity written after the rename before reopening."
+//
+// Measurement (across 47 real renamed sessions on this system): the latest
+// session_info was at EOF in 100% of cases — the rename was the last write
+// before the session was reopened, every single time. 32KB therefore covers all
+// observed renames with ~32,000× margin, and also covers a rename followed by
+// up to ~32KB of continued activity (dozens of typical message turns) before
+// reopening. A rename followed by more than this much continued activity is
+// missed and falls back to firstMessage — the documented tradeoff vs a
+// full-file scan.
+const TAIL_READ_SIZE = 32_768;
+
+// Defensive guard against a single pathological line with no newline (e.g. a
+// corrupted file). Real pi sessions are newline-terminated JSONL, so this never
+// triggers on valid input. It only caps memory for malformed input.
+const MAX_LINE_BYTES = 256 * 1024 * 1024;
 
 // Result of scanning a file tail for session_info entries.
-//   found: false  → no session_info seen in the tail; fall back to the head's name.
+//   found: false  → no session_info seen in the tail; fall back to the forward name.
 //   found: true   → a session_info was seen (later in file order than anything
-//                   in the head, since the tail starts past the head window);
-//                   name is undefined if that entry explicitly cleared the name.
+//                   the forward pass saw, since the tail starts past the forward
+//                   stop); name is undefined if that entry explicitly cleared it.
 export interface TailSessionInfo {
   found: boolean;
   name?: string;
 }
 
+// Accumulator state while processing complete session entries line by line.
+// Shared by the pure parseSessionFromBuffer and the streaming loadSessionHeader
+// so the per-entry logic exists in exactly one place.
+interface SessionAccumulator {
+  header: { id: string; timestamp: string; cwd?: string; parentSession?: string } | null;
+  firstUserMessage: string;
+  messageCount: number;
+  name: string | undefined;
+  lastActivityTime: number | undefined;
+  foundFirstUser: boolean;
+}
+
+function newAccumulator(): SessionAccumulator {
+  return {
+    header: null,
+    firstUserMessage: "",
+    messageCount: 0,
+    name: undefined,
+    lastActivityTime: undefined,
+    foundFirstUser: false,
+  };
+}
+
 function extractTextFromContent(
   content: string | Array<{ type: string; text?: string }>,
 ): string {
@@ -52,96 +101,88 @@ function extractTextFromContent(
     .join(" ");
 }
 
-export function parseSessionFromBuffer(
-  buf: Buffer,
-  bytesRead: number,
-  filePath: string,
-  mtimeMs: number,
-  partial = false,
-  tailInfo?: TailSessionInfo,
-): SessionHeader | null {
-  const decoder = new StringDecoder("utf8");
-  const text = decoder.write(buf.subarray(0, bytesRead)) + decoder.end();
-  const lines = text.split("\n");
-
-  let header: { id: string; timestamp: string; cwd?: string; parentSession?: string } | null = null;
-  let firstUserMsg = "";
-  let name: string | undefined;
-  let msgCount = 0;
-  let lastActivityTime: number | undefined;
-
-  for (const line of lines) {
-    const trimmed = line.trim();
-    if (!trimmed) continue;
-    try {
-      const entry = JSON.parse(trimmed);
-      if (entry.type === "session") {
-        header = entry;
-        continue;
-      }
-      if (entry.type === "session_info") {
-        name = entry.name?.trim() || undefined;
-      }
-      if (entry.type === "message") {
-        msgCount++;
-
-        // Track last activity time from user/assistant messages
-        // Matches pi-core's getMessageActivityTime priority:
-        //   message.timestamp (number) > entry.timestamp (date string)
-        const msg = entry.message;
-        if (msg?.role === "user" || msg?.role === "assistant") {
-          const msgTimestamp = msg.timestamp;
-          if (typeof msgTimestamp === "number" && msgTimestamp > 0) {
-            lastActivityTime = Math.max(lastActivityTime ?? 0, msgTimestamp);
-          } else if (typeof entry.timestamp === "string") {
-            const t = new Date(entry.timestamp).getTime();
-            if (!Number.isNaN(t)) {
-              lastActivityTime = Math.max(lastActivityTime ?? 0, t);
-            }
-          }
-        }
+// Process one complete entry line. Pure: mutates only `acc`. Malformed JSON (a
+// line truncated at a read boundary, or genuinely corrupt input) is swallowed
+// by the try/catch — callers only ever feed complete lines, so a parse failure
+// here means the line is malformed and skipping it is correct.
+function processEntry(acc: SessionAccumulator, line: string): void {
+  const trimmed = line.trim();
+  if (!trimmed) return;
+  let entry: any;
+  try {
+    entry = JSON.parse(trimmed);
+  } catch {
+    return;
+  }
+  if (typeof entry !== "object" || entry === null) return;
 
-        if (!firstUserMsg && msg?.role === "user") {
-          try {
-            firstUserMsg = extractTextFromContent(msg.content);
-          } catch {
-            // Malformed content, skip
-          }
+  if (entry.type === "session") {
+    acc.header = entry;
+    return;
+  }
+  if (entry.type === "session_info") {
+    // Latest session_info in file order wins, including explicit clears
+    // (empty/whitespace name → undefined). Matches pi-core's getSessionName().
+    acc.name = entry.name?.trim() || undefined;
+    return;
+  }
+  if (entry.type === "message") {
+    acc.messageCount++;
+
+    // Track last activity time. Matches pi-core's getMessageActivityTime
+    // priority: message.timestamp (number) > entry.timestamp (date string).
+    const msg = entry.message;
+    if (msg?.role === "user" || msg?.role === "assistant") {
+      const msgTimestamp = msg.timestamp;
+      if (typeof msgTimestamp === "number" && msgTimestamp > 0) {
+        acc.lastActivityTime = Math.max(acc.lastActivityTime ?? 0, msgTimestamp);
+      } else if (typeof entry.timestamp === "string") {
+        const t = Date.parse(entry.timestamp);
+        if (!Number.isNaN(t)) {
+          acc.lastActivityTime = Math.max(acc.lastActivityTime ?? 0, t);
         }
       }
-    } catch {
-      // Incomplete/truncated JSON at buffer boundary, skip
+    }
+
+    if (!acc.foundFirstUser && msg?.role === "user") {
+      acc.firstUserMessage = extractTextFromContent(msg.content);
+      acc.foundFirstUser = true;
     }
   }
+}
 
+// Build the SessionHeader from an accumulator. `reachedEof` is whether the
+// forward pass consumed all input — when false (it stopped early at the first
+// user message), lastActivityTime only reflects entries seen and is unreliable,
+// so stat mtime is used instead (pi updates it on every append, so it tracks
+// the true last write time). `tailInfo`, if present, carries the latest
+// session_info from a tail read and wins over the forward name (later in file
+// order), including explicit name clears.
+function buildHeader(
+  acc: SessionAccumulator,
+  filePath: string,
+  mtimeMs: number,
+  reachedEof: boolean,
+  tailInfo?: TailSessionInfo,
+): SessionHeader | null {
+  const header = acc.header;
   if (!header) return null;
 
-  // Determine modified time:
-  //   - Full read: use pi-core's priority (message timestamp > header timestamp > stat mtime).
-  //     lastActivityTime is accurate since we saw every message.
-  //   - Partial read: stat mtime is more reliable than a partial lastActivityTime,
-  //     which only reflects messages in the first 16KB and may severely underestimate
-  //     the true last activity for large sessions.
-  const headerTime = typeof header.timestamp === "string" ? new Date(header.timestamp).getTime() : NaN;
+  const name = tailInfo?.found ? tailInfo.name : acc.name;
+
+  const headerTime = Date.parse(header.timestamp);
   let modified: Date;
-  if (partial) {
-    // Partial read — stat mtime is the most reliable signal
+  if (!reachedEof) {
+    // Partial read — stat mtime is the only reliable signal.
     modified = new Date(mtimeMs);
+  } else if (typeof acc.lastActivityTime === "number" && acc.lastActivityTime > 0) {
+    modified = new Date(acc.lastActivityTime);
+  } else if (!Number.isNaN(headerTime)) {
+    modified = new Date(headerTime);
   } else {
-    // Full read — pi-core's priority chain
-    modified =
-      typeof lastActivityTime === "number" && lastActivityTime > 0
-        ? new Date(lastActivityTime)
-        : !Number.isNaN(headerTime)
-          ? new Date(headerTime)
-          : new Date(mtimeMs);
+    modified = new Date(mtimeMs);
   }
 
-  // The tail scan (if any) sees session_info entries at EOF, which are later
-  // in file order than anything in the head window — so it wins over the
-  // head-derived name, including explicit clears (empty name → undefined).
-  const finalName = tailInfo?.found ? tailInfo.name : name;
-
   return {
     path: filePath,
     id: header.id,
@@ -149,12 +190,39 @@ export function parseSessionFromBuffer(
     parentSessionPath: header.parentSession || undefined,
     created: new Date(header.timestamp),
     modified,
-    messageCount: msgCount,
-    firstMessage: firstUserMsg || "(no messages)",
-    name: finalName,
+    messageCount: acc.messageCount,
+    firstMessage: acc.firstUserMessage || "(no messages)",
+    name,
   };
 }
 
+// Pure parser over a buffer containing complete (or complete-prefix) entry
+// lines. Splits on \n and runs processEntry on each line; the last line may be
+// truncated at the buffer boundary (its JSON.parse fails and it is skipped).
+// `partial` means the buffer does not contain the whole file — when true,
+// modified time falls back to stat mtime (the buffer's lastActivityTime only
+// reflects the prefix). `tailInfo` carries a tail-read latest session_info that
+// overrides the buffer's name.
+//
+// Kept as a pure, synchronous, fd-free function for direct testing and
+// callers that already have the bytes. loadSessionHeader (the production path)
+// uses the streaming reader below so it never truncates a line mid-JSON.
+export function parseSessionFromBuffer(
+  buf: Buffer,
+  bytesRead: number,
+  filePath: string,
+  mtimeMs: number,
+  partial = false,
+  tailInfo?: TailSessionInfo,
+): SessionHeader | null {
+  const decoder = new StringDecoder("utf8");
+  const text = decoder.write(buf.subarray(0, bytesRead)) + decoder.end();
+  const lines = text.split("\n");
+  const acc = newAccumulator();
+  for (const line of lines) processEntry(acc, line);
+  return buildHeader(acc, filePath, mtimeMs, !partial, tailInfo);
+}
+
 // Scan a tail chunk (read from near EOF) for session_info entries and return
 // the latest name, matching pi-core's getSessionName() semantics: the latest
 // session_info in file order wins, including explicit clears (empty name).
@@ -174,7 +242,7 @@ export function scanTailForSessionInfo(
     if (!trimmed) continue;
     try {
       const entry = JSON.parse(trimmed);
-      if (entry.type === "session_info") {
+      if (typeof entry === "object" && entry !== null && entry.type === "session_info") {
         found = true;
         name = entry.name?.trim() || undefined;
       }
@@ -185,6 +253,68 @@ export function scanTailForSessionInfo(
   return { found, name };
 }
 
+// Read complete lines forward from `fd` starting at offset 0. Calls onLine for
+// each complete (newline-terminated) line. Reading stops when onLine returns
+// false, or at EOF. Returns whether EOF was reached (i.e. the caller did not
+// stop early) and the byte offset just past the last emitted line's newline —
+// the caller uses this as the lower bound for any tail scan so the tail never
+// re-reads already-covered bytes.
+//
+// Chunk-based I/O with a StringDecoder so multi-byte UTF-8 sequences split
+// across chunk boundaries decode correctly. The in-memory line buffer grows to
+// fit the longest single line (bounded by MAX_LINE_BYTES against malformed
+// input); real entries are newline-terminated so this is unbounded only for
+// corrupt files.
+function forEachLineForward(
+  fd: number,
+  size: number,
+  onLine: (line: string) => boolean | void,
+): { reachedEof: boolean; consumedBytes: number } {
+  const decoder = new StringDecoder("utf8");
+  const chunk = Buffer.alloc(READ_CHUNK_SIZE);
+  let lineBuf = "";
+  let offset = 0;
+  let consumedBytes = 0;
+
+  const flushLine = (line: string): boolean | void => {
+    consumedBytes += Buffer.byteLength(line, "utf8") + 1; // +1 for \n
+    return onLine(line);
+  };
+
+  while (offset < size) {
+    const toRead = Math.min(READ_CHUNK_SIZE, size - offset);
+    const bytesRead = readSync(fd, chunk, 0, toRead, offset);
+    if (bytesRead <= 0) break;
+    offset += bytesRead;
+
+    const text = decoder.write(chunk.subarray(0, bytesRead));
+    let start = 0;
+    let nl: number;
+    while ((nl = text.indexOf("\n", start)) !== -1) {
+      const line = lineBuf + text.slice(start, nl);
+      lineBuf = "";
+      if (flushLine(line) === false) {
+        return { reachedEof: false, consumedBytes };
+      }
+      start = nl + 1;
+    }
+    lineBuf += text.slice(start);
+
+    // Defensive: bound memory for a single pathological line.
+    if (lineBuf.length > MAX_LINE_BYTES) {
+      lineBuf = "";
+    }
+  }
+  // Flush decoder + any trailing line without a final newline.
+  const tail = decoder.end();
+  if (tail) lineBuf += tail;
+  if (lineBuf.length > 0) {
+    consumedBytes += Buffer.byteLength(lineBuf, "utf8"); // no trailing \n
+    onLine(lineBuf);
+  }
+  return { reachedEof: true, consumedBytes };
+}
+
 const HOME = homedir();
 const DEFAULT_SESSIONS_DIR = join(HOME, ".pi", "agent", "sessions");
 
@@ -267,44 +397,109 @@ export function scanSessionDir(
   return results;
 }
 
+// Forward-only load: reads complete lines from the start and stops at the
+// first user message (which is all the title row needs). This reads exactly as
+// many bytes as the first user message requires — a few KB for a normal
+// session, ~19KB for a <skill> injection, more for a base64 image — and never
+// truncates a line mid-JSON the way a fixed byte window would. So oversized
+// first user messages (the cases that used to show "(no messages)") are parsed
+// correctly.
+//
+// No tail read — the returned header's name reflects only session_info entries
+// seen within the forward window. For sessions whose latest rename lives past
+// the forward stop point (the common case for renamed large sessions), pair
+// this with resolveSessionName() run in the background; the name then populates
+// in-place without blocking the picker's initial render.
+export function loadSessionHeaderForward(
+  meta: SessionFileMeta,
+): SessionHeader | null {
+  let fd: number | undefined;
+  try {
+    fd = openSync(meta.path, "r");
+    const acc = newAccumulator();
+    const { reachedEof } = forEachLineForward(fd, meta.size, (line) => {
+      processEntry(acc, line);
+      if (acc.header && acc.foundFirstUser) return false;
+      return true;
+    });
+    return buildHeader(acc, meta.path, meta.mtimeMs, reachedEof);
+  } catch {
+    return null;
+  } finally {
+    if (fd !== undefined) closeSync(fd);
+  }
+}
+
+// Resolve the latest session_info (the rename name) from a bounded tail at EOF,
+// independent of any forward pass. Returns found:false when no session_info
+// lives in the tail region (keep whatever name the forward pass produced);
+// found:true means a session_info was seen — its name (or explicit clear)
+// overrides the forward name (it is later in file order).
+//
+// This is the deferred half of loadSessionHeader, exposed so callers can show
+// a row immediately with the forward name and resolve the rename name in the
+// background. Reading up to TAIL_READ_SIZE bytes from EOF may overlap the
+// forward region for small files; that is a redundant re-read of a small range
+// (no correctness impact — the latest session_info wins either way).
+export function resolveSessionName(meta: SessionFileMeta): TailSessionInfo {
+  if (meta.size <= 0) return { found: false };
+  let fd: number | undefined;
+  try {
+    fd = openSync(meta.path, "r");
+    const tailReadSize = Math.min(TAIL_READ_SIZE, meta.size);
+    const tailBuf = Buffer.alloc(tailReadSize);
+    const tailOffset = meta.size - tailReadSize;
+    const tailBytesRead = readSync(fd, tailBuf, 0, tailReadSize, tailOffset);
+    return scanTailForSessionInfo(tailBuf, tailBytesRead);
+  } catch {
+    return { found: false };
+  } finally {
+    if (fd !== undefined) closeSync(fd);
+  }
+}
+
+// Load a session header using a streaming forward read plus a bounded tail read
+// — forward + tail in one shared fd. Equivalent to loadSessionHeaderForward
+// followed by resolveSessionName, but bounds the tail below by the forward stop
+// offset so it never re-reads already-covered bytes. Use this when the full
+// header (including rename name) is needed synchronously.
 export function loadSessionHeader(
   meta: SessionFileMeta,
 ): SessionHeader | null {
   let fd: number | undefined;
   try {
     fd = openSync(meta.path, "r");
-    const readSize = Math.min(PARTIAL_READ_SIZE, meta.size);
-    const buf = Buffer.alloc(readSize);
-    const bytesRead = readSync(fd, buf, 0, readSize, 0);
-
-    // Recover the latest session name from EOF. session_info entries are
-    // appended at EOF by SessionManager.appendSessionInfo, so for any session
-    // larger than the head window they live past the 16KB read and would be
-    // invisible. The tail read covers `meta.size - PARTIAL_READ_SIZE` bytes
-    // (capped at TAIL_READ_SIZE), starting past the head window so it never
-    // overlaps. A failure here must not lose the whole header — fall back to
-    // head-only parsing by leaving tailInfo undefined.
+    const acc = newAccumulator();
+
+    // Forward pass: read complete lines, stopping at the first user message.
+    const { reachedEof: forwardReachedEof, consumedBytes } = forEachLineForward(
+      fd,
+      meta.size,
+      (line) => {
+        processEntry(acc, line);
+        if (acc.header && acc.foundFirstUser) return false;
+        return true;
+      },
+    );
+
+    // If the forward pass stopped before EOF, recover the latest session_info
+    // from a bounded tail at EOF. Bounded below by consumedBytes so it never
+    // re-parses already-seen entries; the tail wins over the forward name
+    // (later in file order). A failure here falls back to the forward name.
     let tailInfo: TailSessionInfo | undefined;
-    if (meta.size > PARTIAL_READ_SIZE) {
+    if (!forwardReachedEof) {
       try {
-        const tailReadSize = Math.min(TAIL_READ_SIZE, meta.size - PARTIAL_READ_SIZE);
+        const tailReadSize = Math.min(TAIL_READ_SIZE, meta.size - consumedBytes);
         const tailBuf = Buffer.alloc(tailReadSize);
         const tailOffset = meta.size - tailReadSize;
         const tailBytesRead = readSync(fd, tailBuf, 0, tailReadSize, tailOffset);
         tailInfo = scanTailForSessionInfo(tailBuf, tailBytesRead);
       } catch {
-        // Tail read failed — fall back to head-only parse
+        // Tail read failed — fall back to forward-only name
       }
     }
 
-    return parseSessionFromBuffer(
-      buf,
-      bytesRead,
-      meta.path,
-      meta.mtimeMs,
-      meta.size > PARTIAL_READ_SIZE,
-      tailInfo,
-    );
+    return buildHeader(acc, meta.path, meta.mtimeMs, forwardReachedEof, tailInfo);
   } catch {
     return null;
   } finally {
@@ -323,6 +518,20 @@ export function loadSessionHeaders(
   return results;
 }
 
+// Forward-only batch load — see loadSessionHeaderForward. Use for the picker's
+// immediate display path: rows appear instantly with the correct firstMessage,
+// and rename names resolve in the background via resolveSessionName().
+export function loadSessionHeadersForward(
+  metas: SessionFileMeta[],
+): SessionHeader[] {
+  const results: SessionHeader[] = [];
+  for (const meta of metas) {
+    const header = loadSessionHeaderForward(meta);
+    if (header) results.push(header);
+  }
+  return results;
+}
+
 export function sortByModified(sessions: SessionHeader[]): SessionHeader[] {
   return sessions.sort(
     (a, b) => b.modified.getTime() - a.modified.getTime(),
@@ -370,4 +579,4 @@ export function matchQuery(
   if (session.cwd.toLowerCase().includes(q)) return true;
   if (session.id.toLowerCase().includes(q)) return true;
   return false;
-}
+}

package/src/search.ts

@@ -38,10 +38,11 @@ function normalizeWhitespaceLower(text: string): string {
  * Build the searchable text for a session.
  *
  * IMPORTANT LIMITATION: pi-core's /resume builds `allMessagesText` from the
- * full file, so search matches against every message in the session. We only
- * read the first 16KB, so we search against `firstMessage` instead. This means
- * queries like `fix oauth` won't match a session where that phrase appears in
- * the 5th message but not the 1st. Name, id, and cwd matches are unaffected.
+ * full file, so search matches against every message in the session. We stop
+ * reading at the first user message, so we search against `firstMessage`
+ * instead. This means queries like `fix oauth` won't match a session where
+ * that phrase appears in the 5th message but not the 1st. Name, id, and cwd
+ * matches are unaffected.
  *
  * See README "Known Limitations" section for the user-facing explanation.
  */