@thurstonsand/pi-librarian 0.2.1 → 0.3.1

package/README.md

@@ -8,14 +8,21 @@ A GitHub research subagent for the [pi coding agent](https://github.com/badlogic
 
 ## How it works
 
-The `librarian` tool spawns a nested research agent with purpose-built tools:
+The `librarian` tool spawns a research subagent with purpose-built tools:
 
-- **`checkout_repo`** — clone into a local cache; pi's own `grep`/`read`/`find` then work on real files, and `git log -S`/`blame`/`diff` cover history.
-- **`search_repos`** — GitHub repository discovery (stars, topics, language).
+- **`checkout_repo`** — clone into a local cache, where it can be read locally, and `git log -S`/`blame`/`diff` covers history.
+- **`search_repos`** — GitHub repository discovery (stars, topics, other metadata).
 - **`search_code`** — cross-repo public code search via [Grep](https://grep.app/) (regex, global discovery, repo/language/path filters).
 - **`search_github_code`** — GitHub REST code search over public code and private repositories your configured GitHub auth can access.
 - **`read_github_file`** — single-file API reads for quick peeks without cloning.
 
+## Usage
+
+- Ask pi a question involving other repos; it asks the `librarian`.
+- Ask follow-up questions to earlier librarian runs.
+- `/librarian` attaches the research tools directly to your session for direct tool usage.
+- Recommended: install [pi-web-access](https://pi.dev/packages/pi-web-access) (or your web search of choice, tho this one is zero-config to get started) and add `web_search`, `fetch_content`, `get_search_content` to `librarian.tools` to expand past GitHub-only research.
+
 ## GitHub auth for private repos
 
 Public GitHub reads work without configuration. To let the librarian search and read private GitHub repositories, provide a token in one of these ways:
@@ -23,13 +30,7 @@ Public GitHub reads work without configuration. To let the librarian search and
 1. Set `GITHUB_TOKEN` or `GH_TOKEN` in the environment before starting pi.
 2. Or authenticate the GitHub CLI so `gh auth token` returns a token:
 
-The token is loaded once per pi session and passed to GitHub REST calls used by `search_repos`, `search_github_code`, `checkout_repo`, and `read_github_file`. For private repositories, use a token with read access to the target repos.
-
-## Usage
-
-- Ask pi a question involving other repos; it delegates to the `librarian` tool.
-- Ask follow-up questions to earlier librarian runs.
-- `/librarian` attaches the research tools directly to your session for manual lookups.
+The token is loaded once per pi session and used for github access in `search_repos`, `search_github_code`, `checkout_repo`, and `read_github_file`. For private repositories, use a token with read access to the target repos.
 
 ## Configuration
 
@@ -40,7 +41,7 @@ In pi's global `settings.json`:
   "librarian": {
     "model": "openai-codex/gpt-5.5",
     "thinkingLevel": "off",
-    "tools": ["search_web", "fetch_web"],
+    "tools": ["web_search", "fetch_content", "get_search_content"],
     "extensions": ["~/.pi/agent/extensions/parallel-web-tools"],
     "cacheDir": "/tmp/pi-librarian",
     "debug": { "persistRuns": false },
@@ -48,16 +49,16 @@ In pi's global `settings.json`:
 }
 ```
 
-| Setting             | Recommended                                     | Default                        |
-| ------------------- | ----------------------------------------------- | ------------------------------ |
-| `model`             | `openai-codex/gpt-5.5`                          | current session model          |
-| `thinkingLevel`     | `off`                                           | current session thinking level |
-| `tools`             | names of extra tools to activate, when needed   | `[]`                           |
-| `extensions`        | escape hatch paths for tools not loaded in pi   | `[]`                           |
-| `cacheDir`          | `/tmp/pi-librarian`                             | `/tmp/pi-librarian`            |
-| `debug.persistRuns` | persist nested session file paths for debugging | `false`                        |
+| Setting             | Recommended                                      | Default                        |
+| ------------------- | ------------------------------------------------ | ------------------------------ |
+| `model`             | `openai-codex/gpt-5.5`                           | current session model          |
+| `thinkingLevel`     | `off`                                            | current session thinking level |
+| `tools`             | names of extra tools to provide the librarian    | `[]`                           |
+| `extensions`        | extra paths to load extensions for the librarian | `[]`                           |
+| `cacheDir`          | `/tmp/pi-librarian`                              | `/tmp/pi-librarian`            |
+| `debug.persistRuns` | persist nested session file paths for debugging  | `false`                        |
 
-`librarian.tools` is the activation gate for extra tools. `librarian.extensions` only adds extension paths to the search space when a named tool is not already loaded in the main pi session; listing an extension path does not activate every tool in that bundle. Librarian runs exclude `write` and `edit`.
+`librarian.extensions` dynamically loads extra extensions just for the librarian. Add any tools from that extension to `librarian.tools` for the librarian to actually be able to use them. Librarian excludes `write` and `edit`.
 
 ## Development
 

package/RELEASE.md

@@ -2,6 +2,21 @@
 
 # Release notes
 
+## 0.3.1
+
+### Changed
+
+- Sharpened the research tools' prompt snippets, guidelines, and descriptions so the librarian names each tool explicitly and reads its guidance as directive instructions.
+
+## 0.3.0
+
+Hardens librarian research tools and aligns GitHub file reads with pi's native `read` tool.
+
+### Changed
+
+- Changed `read_github_file` to use `offset` and `limit` instead of `range`, matching pi's native `read` tool semantics.
+- Hardened `checkout_repo` cache reuse so mismatched or malformed cached paths are discarded, while reused checkouts are reset and cleaned before research.
+
 ## 0.2.1
 
 Fixes TUI rendering corruption.

package/docs/designs/03-librarian-research-tool-hardening.md

@@ -0,0 +1,136 @@
+# Librarian research-tool hardening
+
+## Status
+
+Partially implemented
+
+## Decision Summary
+
+Capture the hardening work implemented for librarian runs: disposable checkout-cache mechanics and remote file reads that behave like pi's native `read` tool. AST-aware local-code tools and heavier result metadata remain deferred ideas, not active implementation pressure.
+
+## Problem Statement / Background
+
+The current librarian design already makes the main reliability move: within-repo research happens from local blob-less clones, while search APIs are candidate finders. A survey of adjacent implementations suggested several refinements worth preserving before the details are lost:
+
+- Mitsuhiko's librarian skill treats checkouts as stable cached infrastructure: repo references normalize to a predictable cache path, clones are reused, and refreshes are throttled.
+- Code-research tools such as Octocode and codebase-research-agent emphasize exact regions, line anchors, and smaller useful code chunks rather than raw search dumps.
+- This project should borrow those patterns only where they simplify the librarian's operating surface. It should not become an API-first search tool, a heavy MCP service, or a semantic index whose snippets are mistaken for evidence.
+
+The useful scenario is a repeated question about the same repo: the librarian should reuse the existing clone, force it back to the requested upstream state subject to fetch debounce, clean away any cache-local debris, and let the agent read GitHub files with the same mental model it uses for local files.
+
+## Goals
+
+- Reuse cached clones safely while preserving the librarian's read-only research contract.
+- Treat checkout-cache contents as disposable infrastructure, not user-owned worktrees.
+- Make `read_github_file` operationally match pi's native `read` tool where possible.
+
+## Non-Goals
+
+- No dependency on Octocode or any external MCP service.
+- No shift away from clone-first within-repo research.
+- No semantic/vector index as part of this design.
+- No AST-aware local-code primitive for now; useful, but not part of this hardening pass.
+- No broad tool-result metadata expansion unless a concrete enforcement feature needs it.
+- No repo-profile cache for now; the value is less clear than the hardening work captured here.
+
+## Exposed Shape
+
+These additions would preserve the current user-facing `librarian` tool and `/librarian` attach command. The exposed changes would be inside the repo-tool surface:
+
+- `checkout_repo` becomes stricter about cache reuse by resetting and cleaning cache entries deterministically.
+- `read_github_file` uses `offset`/`limit` semantics and output conventions that closely mirror pi's native `read` tool.
+- Existing `provide_results.locations` remains the final citation surface. Search results are still candidates by prompt discipline, not by a new metadata framework.
+
+## Design Decisions
+
+### 1. Cached checkouts are disposable infrastructure, not worktrees
+
+A reused checkout should be forced back to the requested upstream state, subject only to the fetch debounce. Unlike a human working clone, librarian checkouts are cache infrastructure. If a previous run, aborted process, or manual edit left local changes, generated files, or an old branch behind, `checkout_repo` should not preserve them. Manual edits inside the checkout cache are undefined behavior.
+
+For default-branch checkouts, the desired shape is:
+
+1. verify the cache path is a git repo for the requested normalized remote;
+2. fetch from origin unless the fetch debounce says the clone is fresh enough;
+3. checkout the default branch;
+4. hard reset to `origin/<defaultBranch>`;
+5. remove untracked files with `git clean -fdx`;
+6. return the pinned HEAD sha and cache status.
+
+For explicit refs, the tool should fetch the requested ref when needed, force checkout/detach to the resolved commit, hard reset to that commit, and clean the tree. If the cached remote does not match the requested repo, or if the cache path is malformed, the safe behavior is to discard the cache path and clone again.
+
+**Tradeoff:** this destroys any local modifications inside the checkout cache. That is acceptable because librarian runs are read-only by contract; cache contents are not user-owned work.
+
+### 2. Remote file reads should behave like local reads
+
+`read_github_file` is most useful when it feels like pi's native `read` tool pointed at GitHub. The agent should not need one mental model for local checkout files and another for remote GitHub files.
+
+The tool should use `offset` and `limit` parameters rather than a bespoke `[startLine, endLine]` range:
+
+- `offset` is a 1-indexed starting line;
+- `limit` is the maximum number of lines to return;
+- default behavior should match native `read` as closely as practical;
+- clipped output should tell the agent how to continue with `offset`/`limit`;
+- file output should be directly returned as text, not written to disk;
+
+Directory listings may remain GitHub-specific, but file reads should be operationally interchangeable with local `read` output. This is more useful than extra metadata because it changes the agent's actual operating surface.
+
+**Tradeoff:** changing from `range` to `offset`/`limit` breaks the current tool schema. That is acceptable if it makes the tool more pi-native and easier for agents to use consistently.
+
+### 3. AST and metadata expansion remain deferred
+
+Plain grep plus `read` remains the right baseline because it is universal, fast, and transparent. A future AST-aware addition could still be useful, probably as `read_enclosing_node(path, line, language?)` for TypeScript/JavaScript first, but it should wait until the simpler read and checkout semantics are solid.
+
+Likewise, broad result metadata such as `evidenceKind`, `complete`, `truncated`, `continuation`, and `recoveryHint` should not be added speculatively. Agents primarily use tool text and prompt instructions; hidden metadata only matters when a renderer or validator consumes it. If citation honesty needs more structure later, the better first step is validating `provide_results.locations` against the run trace: files checked out, files read, line ranges observed, and search hits that were only candidates.
+
+**Tradeoff:** this leaves candidate-vs-proof enforcement mostly in the prompt for now. That is acceptable under KISS until there is a concrete validation feature to support.
+
+## Edge Cases & Failure Modes
+
+- **Cached repo remote mismatch:** discard and reclone rather than trying to repair an ambiguous cache entry.
+- **Dirty cached checkout:** hard reset and `git clean -fdx`; do not preserve changes.
+- **Fetch fails but stale clone exists:** report a structured error instead of silently researching stale code, unless a future explicit offline mode exists.
+- **Explicit commit not reachable after fetch:** return a ref-resolution error with a recovery hint to verify the ref or repo.
+- **Large GitHub file:** return a clipped line window using native-read-like semantics and tell the agent the next `offset`/`limit` to request.
+- **Search result clipped by limits:** keep the text honest about limits; do not add metadata unless a concrete consumer needs it.
+
+## Alternatives
+
+### Keep checkout cache reuse as-is
+
+- **Status:** Rejected
+- **Decision:** The current implementation already does a force checkout/reset for the default branch and force checkout for explicit refs, but it does not fully encode the disposable-cache policy described here.
+- **Discussion:** The stricter policy is easier to reason about and matches the contract: checkout-cache contents are not user-owned work.
+
+### Add Octocode as a dependency
+
+- **Status:** Rejected
+- **Decision**: Octocode is an external code-research/MCP-style system, not a pi-native tool. Its useful contribution here is contract design: exact regions, line anchors, pagination, bulk/error semantics, and candidate-vs-proof discipline.
+- **Discussion:** Borrow only the interaction ideas that make pi-native tools easier to use. Do not import the infrastructure or add metadata for its own sake.
+
+### Add repo profiles
+
+- **Status:** Rejected
+- **Decision**: Repo profiles are less compelling for now than cache correctness and native-read-like remote file access.
+- **Discussion:** A profile cache could store package manager, source roots, manifests, and test commands, but it risks becoming stale orientation data that agents over-trust. Evidence should continue to come from actual file reads.
+
+## Implementation Plan
+
+- [x] Phase 1: Checkout cache hardening
+  - Goal: Make reused clones deterministic disposable infrastructure that match the requested upstream ref, subject to fetch debounce.
+  - Files: `extensions/librarian/checkout.ts`, `extensions/librarian/tools/checkout-repo.ts`, checkout tests.
+  - Work: Verify cached remote identity; discard/reclone on mismatch or malformed cache; fetch with existing debounce; hard reset and `git clean -fdx` for default branch and explicit refs.
+  - Validation: Unit tests for dirty checkout cleanup, untracked file cleanup, remote mismatch reclone, explicit ref checkout, malformed cache replacement, and fetch failure behavior; `npm run check`.
+
+- [x] Phase 2: Native-read-like `read_github_file`
+  - Goal: Make remote GitHub file reads operationally match pi's native `read` tool.
+  - Files: `extensions/librarian/tools/read-github-file.ts`, prompt/tests.
+  - Work: Replace `range` with `offset`/`limit`; match native read truncation and continuation language as closely as practical; keep direct text output and directory listing behavior.
+  - Validation: Tool tests for default truncation, offset/limit reads, clipped continuation text, invalid ranges, directory listings, and pinned `ref` reads; `npm run check`.
+
+- [ ] Deferred: citation validation
+  - Goal: If citation honesty needs structural enforcement, validate `provide_results.locations` against the run trace instead of adding broad metadata first.
+  - Work: Track which checked-out or GitHub files were actually read, including covered line ranges; reject or warn on locations sourced only from search candidates.
+
+- [ ] Deferred: AST-aware local-code primitive
+  - Goal: Provide a syntactic-region helper only if grep/read chains prove too wasteful.
+  - Work: Consider `read_enclosing_node(path, line, language?)`, likely TypeScript/JavaScript first.

package/extensions/librarian/checkout.ts

@@ -148,6 +148,17 @@ async function pathExists(target: string): Promise<boolean> {
   }
 }
 
+async function currentRemoteKey(
+  dest: string,
+  signal: AbortSignal | undefined,
+): Promise<string | undefined> {
+  try {
+    return parseRepoReference(await git(["remote", "get-url", "origin"], dest, signal)).key;
+  } catch {
+    return undefined;
+  }
+}
+
 async function clone(
   reference: RepoReference,
   dest: string,
@@ -207,7 +218,13 @@ export async function checkoutRepo(
   const repo = parseRepoReference(repoInput);
   const dest = repoCachePath(cacheDir, repo.key);
 
-  const reusedClone = await pathExists(path.join(dest, ".git"));
+  const reusedClone =
+    (await pathExists(path.join(dest, ".git"))) &&
+    (await currentRemoteKey(dest, signal)) === repo.key;
+  if (!reusedClone) {
+    await fs.rm(dest, { recursive: true, force: true });
+  }
+
   if (reusedClone) {
     await fetchIfStale(dest, signal);
   } else {
@@ -223,6 +240,7 @@ export async function checkoutRepo(
       // Refs already fetched (e.g. a sha reachable from an earlier fetch) check out directly.
       await git(["checkout", "--force", "--detach", ref], dest, signal);
     }
+    await git(["reset", "--hard", "HEAD"], dest, signal);
     checkedOutRef = ref;
   } else {
     const branch = await defaultBranch(dest, signal);
@@ -231,6 +249,8 @@ export async function checkoutRepo(
     checkedOutRef = branch;
   }
 
+  await git(["clean", "-fdx"], dest, signal);
+
   const headSha = await git(["rev-parse", "HEAD"], dest, signal);
   const files = await git(["ls-files"], dest, signal);
   const fileCount = files.length === 0 ? 0 : files.split("\n").length;

package/extensions/librarian/github.ts

@@ -28,7 +28,14 @@ function repoFullName(repo: GitHubRepoSlug): string {
   return `${repo.owner}/${repo.repo}`;
 }
 
-export type GitHubClientProvider = () => Promise<GitHubClient>;
+export interface GitHubClientApi {
+  searchRepositories(params: SearchRepositoriesParams): Promise<RepoSearchResult>;
+  searchCode(params: SearchGitHubCodeParams): Promise<GitHubCodeSearchResult>;
+  readContents(params: ReadContentsParams): Promise<FileContents | DirectoryContents>;
+  getRepo(repo: GitHubRepoSlug): Promise<RepoMeta>;
+}
+
+export type GitHubClientProvider = () => Promise<GitHubClientApi>;
 
 export async function resolveGitHubToken(): Promise<string | undefined> {
   const envToken = process.env.GITHUB_TOKEN?.trim() || process.env.GH_TOKEN?.trim();
@@ -249,7 +256,7 @@ function buildGitHubCodeQuery(params: SearchGitHubCodeParams): string {
   return parts.join(" ");
 }
 
-export class GitHubClient {
+export class GitHubClient implements GitHubClientApi {
   constructor(private readonly octokit: Octokit) {}
 
   async searchRepositories(params: SearchRepositoriesParams): Promise<RepoSearchResult> {

package/extensions/librarian/tools/checkout-repo.ts

@@ -29,10 +29,10 @@ export function createCheckoutRepoTool(cacheDir: string) {
     name: LIBRARIAN_TOOL_NAMES.checkoutRepo,
     label: "Checkout repo",
     description: "Clone a repo (blob-less partial clone, cached locally).",
-    promptSnippet: "Clone a repo",
+    promptSnippet: "Clone a repo locally",
     promptGuidelines: [
-      "Use when you want to deep dive on a specific repo. Follow up using grep/read/find/ls on the returned path, and `git -C <path> log/blame/diff` for history.",
-      "Do not checkout repos directly using `git`. Always use this tool instead.",
+      "Use checkout_repo to deep dive on a specific repo, then grep/read/find/ls on the returned path and `git -C <path> log/blame/diff` for history.",
+      "Do not clone repos directly with `git`; always use checkout_repo.",
     ],
     parameters: CheckoutRepoParams,
 

package/extensions/librarian/tools/provide-results.ts

@@ -46,10 +46,9 @@ export function createProvideResultsTool(onFindings: (findings: Findings) => voi
     name: LIBRARIAN_TOOL_NAMES.provideResults,
     label: "Provide results",
     description: "Report your findings in structured form.",
-    promptSnippet: "Provide results",
+    promptSnippet: "Report findings",
     promptGuidelines: [
-      "Tool must be called before the turn ends.",
-      "After calling this tool, the turn will end.",
+      "provide_results ends the turn; call it once, after you've gathered your findings.",
     ],
     parameters: FindingsSchema,
 

package/extensions/librarian/tools/read-github-file.ts

@@ -1,10 +1,13 @@
-import { defineTool } from "@earendil-works/pi-coding-agent";
+import {
+  DEFAULT_MAX_BYTES,
+  defineTool,
+  formatSize,
+  truncateHead,
+} from "@earendil-works/pi-coding-agent";
 import { Type } from "typebox";
 import { GitHubApiError, type GitHubClientProvider } from "../github.ts";
 import { LIBRARIAN_TOOL_NAMES } from "./names.ts";
 
-const MAX_LINES_WITHOUT_RANGE = 1200;
-
 export interface ReadGitHubFileDetails {
   kind: "read_github_file";
   owner: string;
@@ -17,15 +20,24 @@ export interface ReadGitHubFileDetails {
 const ReadGitHubFileParams = Type.Object({
   owner: Type.String({ description: "Repository owner or organization." }),
   repo: Type.String({ description: "Repository name." }),
-  path: Type.String({ description: "File or directory path within the repository." }),
+  path: Type.String({
+    description: "File or directory path within the repository.",
+  }),
   ref: Type.Optional(
-    Type.String({ description: "Branch, tag, or commit sha. Default: the default branch." }),
+    Type.String({
+      description: "Branch, tag, or commit sha. Default: the default branch.",
+    }),
+  ),
+  offset: Type.Optional(
+    Type.Number({
+      minimum: 1,
+      description: "Line number to start reading from (1-indexed).",
+    }),
   ),
-  range: Type.Optional(
-    Type.Array(Type.Number({ minimum: 1 }), {
-      description: "[startLine, endLine] (1-based, inclusive) for large files.",
-      minItems: 2,
-      maxItems: 2,
+  limit: Type.Optional(
+    Type.Number({
+      minimum: 1,
+      description: "Maximum number of lines to read.",
     }),
   ),
 });
@@ -34,19 +46,22 @@ export function createReadGitHubFileTool(githubClient: GitHubClientProvider) {
   return defineTool<typeof ReadGitHubFileParams, ReadGitHubFileDetails>({
     name: LIBRARIAN_TOOL_NAMES.readGitHubFile,
     label: "Read GitHub file",
-    description:
-      "Read a single file (or list a directory) from a GitHub repo via the API, without cloning.",
-    promptSnippet: "Read Github file",
+    description: `Read a single file (or list a directory) from a GitHub repo via the API, without cloning.`,
+    promptSnippet: "Read a file/directory from GitHub",
     promptGuidelines: [
-      "For quick peeks — package.json, a README, one source file.",
-      "For multi-file exploration prefer checkout_repo.",
+      "Use read_github_file for quick peeks — package.json, a README, one source file.",
+      "For multi-file exploration, prefer checkout_repo over read_github_file.",
+      "Use read_github_file's offset/limit for larger files.",
     ],
     parameters: ReadGitHubFileParams,
 
     async execute(_toolCallId, params) {
-      const range = params.range ? (params.range as [number, number]) : undefined;
-      if (range && range[1] < range[0]) {
-        throw new Error("range end must be greater than or equal to range start.");
+      const offset = params.offset ?? 1;
+      if (offset < 1) {
+        throw new Error("offset must be greater than or equal to 1.");
+      }
+      if (params.limit !== undefined && params.limit < 1) {
+        throw new Error("limit must be greater than or equal to 1.");
       }
 
       try {
@@ -75,32 +90,52 @@ export function createReadGitHubFileTool(githubClient: GitHubClientProvider) {
         }
 
         const allLines = contents.text.split("\n");
-        let start = 1;
-        let end = allLines.length;
-        let clipNote = "";
-
-        if (range) {
-          start = Math.min(range[0], allLines.length);
-          end = Math.min(range[1], allLines.length);
-        } else if (allLines.length > MAX_LINES_WITHOUT_RANGE) {
-          end = MAX_LINES_WITHOUT_RANGE;
-          clipNote = `\n... clipped at line ${MAX_LINES_WITHOUT_RANGE} of ${allLines.length}; pass range to read further.`;
+        const startLine = offset - 1;
+        if (startLine >= allLines.length) {
+          throw new Error(
+            `Offset ${offset} is beyond end of file (${allLines.length} lines total)`,
+          );
         }
 
-        const width = String(end).length;
-        const numbered = allLines
-          .slice(start - 1, end)
-          .map((line, index) => `${String(start + index).padStart(width)}\t${line}`)
-          .join("\n");
+        const selectedLines =
+          params.limit === undefined
+            ? allLines.slice(startLine)
+            : allLines.slice(startLine, Math.min(startLine + params.limit, allLines.length));
+        const selectedContent = selectedLines.join("\n");
+        const truncated = truncateHead(selectedContent);
+        let text: string;
+        let lineCount = truncated.outputLines;
+
+        if (truncated.firstLineExceedsLimit) {
+          const firstLineSize = formatSize(Buffer.byteLength(allLines[startLine] ?? "", "utf-8"));
+          text = `[Line ${offset} is ${firstLineSize}, exceeds ${formatSize(DEFAULT_MAX_BYTES)} limit.]`;
+        } else if (truncated.truncated) {
+          const endLine = offset + truncated.outputLines - 1;
+          const nextOffset = endLine + 1;
+          const limitNote =
+            truncated.truncatedBy === "lines" ? "" : ` (${formatSize(DEFAULT_MAX_BYTES)} limit)`;
+          text = `${truncated.content}\n\n[Showing lines ${offset}-${endLine} of ${allLines.length}${limitNote}. Use offset=${nextOffset} to continue.]`;
+        } else if (
+          params.limit !== undefined &&
+          startLine + selectedLines.length < allLines.length
+        ) {
+          const remaining = allLines.length - (startLine + selectedLines.length);
+          const nextOffset = startLine + selectedLines.length + 1;
+          text = `${truncated.content}\n\n[${remaining} more lines in file. Use offset=${nextOffset} to continue.]`;
+          lineCount = selectedLines.length;
+        } else {
+          text = truncated.content;
+          lineCount = selectedLines.length;
+        }
 
         return {
-          content: [{ type: "text", text: `${numbered}${clipNote}` }],
+          content: [{ type: "text", text }],
           details: {
             kind: "read_github_file",
             owner: params.owner,
             repo: params.repo,
             path: params.path,
-            lineCount: end - start + 1,
+            lineCount,
             isDirectory: false,
           },
         };

package/extensions/librarian/tools/search-code.ts

@@ -18,7 +18,8 @@ const SearchCodeParams = Type.Object({
   }),
   regex: Type.Optional(
     Type.Boolean({
-      description: "Treat `pattern` as a regular expression instead of literals.",
+      description:
+        "Treat `pattern` as a regular expression instead of literals. Supports `(?s)` for multi-line patterns.",
     }),
   ),
   repo: Type.Optional(
@@ -59,11 +60,10 @@ export const searchCodeTool = defineTool<typeof SearchCodeParams, SearchCodeDeta
   name: LIBRARIAN_TOOL_NAMES.searchCode,
   label: "Search code across repos",
   description: "Cross-repo code search over public source code.",
-  promptSnippet: "Search code across repos",
+  promptSnippet: "Search public code across repos",
   promptGuidelines: [
-    "When `regex` is true, `pattern` can also use `(?s)` for multi-line patterns.",
-    "Results are CANDIDATES to verify via checkout_repo/read_github_file — never cite them directly.",
-    "This searches public GitHub code only. Use search_github_code when private repository access matters.",
+    "search_code results are candidates to verify via checkout_repo/read_github_file — never cite them directly.",
+    "search_code covers public GitHub code only; use search_github_code when private repository access is required.",
   ],
   parameters: SearchCodeParams,
 

package/extensions/librarian/tools/search-github-code.ts

@@ -89,10 +89,9 @@ export function createSearchGitHubCodeTool(githubClient: GitHubClientProvider) {
     label: "Search GitHub code",
     description:
       "GitHub REST code search over public code and private repositories your configured GitHub auth can access.",
-    promptSnippet: "Search GitHub code",
+    promptSnippet: "Search GitHub code (incl. private repos)",
     promptGuidelines: [
-      "Results are CANDIDATES to verify via checkout_repo/read_github_file — never cite them directly.",
-      "GitHub REST code search is literal/tokenized and does not support regex; use search_code for public regex/global code search.",
+      "search_github_code is literal/tokenized and does not support regex; use search_code for public regex/global search.",
     ],
     parameters: SearchGitHubCodeParams,
 

package/extensions/librarian/tools/search-repos.ts

@@ -34,8 +34,8 @@ export function createSearchReposTool(githubClient: GitHubClientProvider) {
     name: LIBRARIAN_TOOL_NAMES.searchRepos,
     label: "Search repos metadata",
     description: "Discover GitHub repositories.",
-    promptSnippet: "Search repos metadata",
-    promptGuidelines: ["Use for questions like 'what are the popular X libraries'."],
+    promptSnippet: "Search GitHub repos by metadata",
+    promptGuidelines: ['Use search_repos for questions like "what are the popular X libraries".'],
     parameters: SearchReposParams,
 
     async execute(_toolCallId, params) {

package/extensions/librarian.ts

@@ -61,7 +61,6 @@ export default function librarianExtension(pi: ExtensionAPI): void {
   for (const tool of attachableTools) {
     pi.registerTool({
       ...tool,
-      description: `${tool.description} (Librarian tool, attached via /librarian: use for quick single lookups; delegate multi-step research to the librarian tool.)`,
       renderCall(args, theme) {
         const { verb, subject } = formatTraceLine(
           { name: tool.name, args, id: "", startedAt: 0 } satisfies TraceCall,
@@ -81,10 +80,9 @@ export default function librarianExtension(pi: ExtensionAPI): void {
     label: "Librarian",
     description:
       "Understand complex, multi-repo codebases, exploring cross-repo relationships, analyzing architectural patterns, finding implementations across codebases, understanding code evolution/commit history, diving deep on specific code bases, getting comprehensive feature explanations, and exploring end-to-end system design across within or across repositories.",
-    promptSnippet:
-      "librarian: research GitHub repos (implementation questions, ecosystem comparisons, usage examples) and return cited findings",
+    promptSnippet: "Research GitHub repos and codebases, returning cited findings",
     promptGuidelines: [
-      "If a single file or reference must be located, you may use other means (search for downloaded dependencies, web search, etc), but for deeper queries that will take multiple steps to resolve, prefer the librarian.",
+      "Use librarian for deep, multi-step research across repos; for a single file or reference, prefer cheaper means (downloaded dependencies, web search or fetch).",
     ],
     parameters: LibrarianParams,
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@thurstonsand/pi-librarian",
-  "version": "0.2.1",
+  "version": "0.3.1",
   "type": "module",
   "description": "GitHub research subagent for pi: deep-dive specific repos, discover across the ecosystem",
   "license": "MIT",