@hicaru/pi-rlm 0.1.7 → 0.1.9

package/README.md

@@ -100,8 +100,8 @@ sub-LLM calls, hence the name.
 
 While a run is active, a **live tree** shows the root orchestrator and every sub-LLM /
 recursive child with status, model, cost, tokens, and duration. The final answer is posted
-to the chat as markdown; any code edits are collected as diffs and reviewed via a popup
-(unless `yolo` is on).
+to the chat as markdown. File changes use Pi's native `edit` / `write` tools (with their
+built-in diff preview).
 
 ## Sandbox API
 
@@ -117,11 +117,44 @@ These functions are injected into the model's Python namespace inside the REPL:
 | `rlm_query_batched` | `(prompts, model=None) -> list[str]` | Concurrent recursive child RLMs |
 | `todo` | `(action, **kwargs) -> str` | Task list: `create`/`update`/`list`/`get`/`delete`/`clear` |
 | `ask_user_question` | `(questions) -> list[dict]` | Ask the user structured questions (depth 0 only) |
-| `stage_edit` | `(path, old_text, new_text) -> str` | Stage a file edit; relayed to the host's native edit flow |
-| `advance_phase` | `(phase, summary=None) -> str` | Move the root pipeline to a new phase |
+| `load_library` | `(source) -> dict \| str` | Append an external dir, file, or git URL into `context` under `lib/<id>/` |
+| `save_artifact` | `(kind, content) -> str` | Persist a stage artifact (`clarification` / `research` / `plan` / `validation`) under `.rlm/artifacts/` (root depth only). Returns preflight gate critique. |
+| `advance_phase` | `(phase, summary=None) -> str` | Advance one step in order `clarify → research → blueprint → validate` (clarify skipped when `askUserQuestion` is off). **Engine-gated** on the latest artifact + interview rounds. Rejected transitions return the gate error. |
 | `SHOW_VARS` | `() -> str` | List currently defined variables & their types |
 | `answer` | `dict` | Set `answer["content"]=...; answer["ready"]=True` to finalize |
 
+### Loading external libraries
+
+When the task needs an **external library, another source tree, or standalone docs** that are
+not in the packed repo `context`, the model calls `load_library(source)` mid-run:
+
+```python
+info = load_library("../some-lib")                # local directory → packed + appended
+info = load_library("docs/api.md")                # single file → one entry in context
+info = load_library("https://github.com/x/y.git") # shallow clone, then pack + append
+# Files land in the SAME `context` list under lib/<source_id>/…
+# info == {"source_id", "path_prefix", "files", "chars", "context_len", "already_loaded", …}
+lib = [f for f in context if f["path"].startswith(info["path_prefix"])]
+```
+
+There is no `context_1` / `context_2` — only `context`. Paths are namespaced so multiple
+libraries do not collide. Toggle via `/rlm-config` → **Library loader** (`libraryLoader`,
+default on). On headless runs with persistence, each load writes a resume sidecar
+(`context.<N>.json`) that is **merged back into `context`** on resume.
+
+### Artifact-gated pipeline (opt-in via `pipeline: true`)
+
+When enabled at root depth:
+
+1. **Goal capture** — the brief is written verbatim to `.rlm/artifacts/goal/goal-<ts>.md` with a pre-run dirty-tree baseline.
+2. **Stages** — `clarify → research → blueprint → validate` (**read-only** — produces a validated plan; does not write code). Each produces a durable markdown artifact with frontmatter contracts; chat history is **reset** at every phase boundary (artifacts are the only channel; REPL vars persist).
+3. **Clarify (intake)** — interviews the user via `ask_user_question` (intent first, then evidence-confirmed decisions). Writes `.rlm/artifacts/clarifications/*` with `decisions_count` / `open_questions_count`. Engine gate: **≥1 serviced ask round** + artifact contract. When **`askUserQuestion` is off**, clarify is skipped and the run starts at research.
+4. **Gates (TypeScript, never LLM judgment)** — `status: ready`; clarify structure; plan `phases:` ≡ fence-aware `## Phase N:` headings; every `file:line` citation resolves; validate carries `blockers_count` + `verdict`. Preflight critique runs on every `save_artifact`.
+5. **Validate** — adversarial plan review against the tree (not a post-implementation diff check). Final answer is the validated plan.
+6. **Corrective loop** — `blockers_count > 0` re-enters blueprint (superseded plan kept for context), bounded by `maxBackwardJumps` (default 2).
+
+Native RLM mode authors file changes with Pi's native `edit` / `write` tools. Sub-LLMs extract and locate; they never ship code.
+
 ## Settings (`/rlm-config`)
 
 | Setting | Default | Meaning |
@@ -137,11 +170,15 @@ These functions are injected into the model's Python namespace inside the REPL:
 | Token ceiling | none | total input+output token cap for the whole recursive tree |
 | Max consecutive errors | `5` | stop after N consecutive failing turns (none = off) |
 | Orchestrator addendum | on | divide-and-conquer guidance in the root system prompt |
+| Phase pipeline | off | artifact-gated clarify→research→blueprint→validate (read-only plan pipeline) |
+| Max validate→blueprint loops | `2` | bounded corrective re-entries when validation reports blockers |
+| Ask user question | on | when pipeline is on, enables clarify intake; when off, pipeline starts at research |
 | Trajectory compaction | on (0.65) | summarize old turns when history nears the context window |
 | Root model output cap (tok) | `16384` | max output tokens per root-model turn |
 | Sandbox init timeout | `30000` ms | how long to wait for the Python worker to start |
 | `askUserQuestion` | on | expose `ask_user_question()` to the model |
 | `todo` | on | expose `todo()` to the model |
+| Library loader | on | expose `load_library()` for external dirs/files/git repos |
 
 > **Concurrency note:** each `rlm_query` child spawns its own `python3` worker (~50–150 ms
 > cold start). Worst-case concurrent interpreters ≈ `maxConcurrentSubcalls`^(depth−1); at

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@hicaru/pi-rlm",
-  "version": "0.1.7",
+  "version": "0.1.9",
   "type": "module",
   "description": "Save 99% tokens, Recursive Language Model (RLM) for the Pi",
   "license": "MIT",
@@ -27,6 +27,7 @@
   ],
   "scripts": {
     "check": "tsc --noEmit",
+    "test": "bun run test/smoke.ts",
     "prepublishOnly": "npm run check"
   },
   "pi": {

package/src/bridge/library.ts

@@ -0,0 +1,155 @@
+/**
+ * Shared load_library handler for headless engine and native repl() mode.
+ *
+ * Host packs the source via resolveLibrarySource (namespaced under lib/<id>/),
+ * assigns a resume-sidecar index, and returns the payload for the worker to
+ * append into the single `context` list.
+ *
+ * Idempotency is host-side: re-loading a source that was already packed does
+ * not consume an index, write a sidecar, or re-clone/pack. That keeps resume
+ * trails free of duplicate library slots.
+ *
+ * Late-bound deps (getCwd / getEmitter) keep a single handler closure correct
+ * across native repl() calls — getOrCreate only installs handlers at spawn.
+ */
+
+import type { RlmEmitter } from "../tool/rlm-events.ts";
+import type { SubLlmHandlers } from "../sandbox/sandbox.ts";
+import {
+  libraryNamespace,
+  resolveLibrarySource,
+} from "../context/library-context.ts";
+import { previewText } from "../text/preview.ts";
+
+export interface LibraryBridgeOpts {
+  /** Fixed cwd (headless). Prefer getCwd when the sandbox outlives a single invocation. */
+  readonly cwd?: string;
+  /** Late-bound cwd (native mode — sandbox handlers outlive a single repl()). */
+  readonly getCwd?: () => string;
+  readonly emitter?: RlmEmitter;
+  /** Native mode: read the live emitter each call. */
+  readonly getEmitter?: () => RlmEmitter | null | undefined;
+  readonly parentId?: string;
+  readonly signal?: AbortSignal;
+  /** First resume-sidecar index (slot 0 = repo). Resume passes 1 + max restored. */
+  readonly startIndex: number;
+  /**
+   * Prefixes already present in context (e.g. restored from sidecars).
+   * Seeded so re-load after resume is still a no-op without re-packing.
+   */
+  readonly loadedPrefixes?: readonly string[];
+  /** Post-load hook — the engine writes the resume sidecar here; native mode omits it. */
+  readonly onLoaded?: (index: number, payload: unknown) => void | Promise<void>;
+}
+
+export interface LibraryHandlerBundle {
+  readonly handlers: Pick<SubLlmHandlers, "loadLibrary">;
+  /** Reset the sidecar index counter (call when the sandbox is discarded and will re-spawn). */
+  readonly reset: () => void;
+  /** Prefixes loaded in this sandbox lifetime (for tests). */
+  readonly loadedPrefixes: () => ReadonlySet<string>;
+}
+
+export function buildLibraryHandler(opts: LibraryBridgeOpts): LibraryHandlerBundle {
+  let nextIndex = opts.startIndex;
+  /** Prefixes already loaded in this sandbox — mirrors the worker's context state. */
+  const loaded = new Set<string>(opts.loadedPrefixes ?? []);
+  return {
+    reset: () => {
+      nextIndex = opts.startIndex;
+      loaded.clear();
+    },
+    loadedPrefixes: () => loaded,
+    handlers: {
+      async loadLibrary(source, depth) {
+        const emitter = opts.getEmitter?.() ?? opts.emitter;
+        const cwd = opts.getCwd?.() ?? opts.cwd;
+        if (cwd === undefined || cwd === "") {
+          throw new Error("load_library: no cwd configured");
+        }
+        const id = emitter?.emitSubcallCreated({
+          kind: "tool", parentId: opts.parentId,
+          label: "load_library",
+          args: previewText(source, 80),
+          depth,
+        });
+        try {
+          // Cheap pre-check BEFORE cloning/packing: same namespace ⇒ nothing to do.
+          const { sourceId: preId, pathPrefix: prefix } = libraryNamespace(source, cwd);
+          if (loaded.has(prefix)) {
+            if (id) {
+              emitter?.emitSubcallUpdated({
+                id,
+                status: "done",
+                resultPreview: `already loaded (${prefix}*)`,
+              });
+            }
+            // No index consumed, no sidecar written — resume stays consistent.
+            return {
+              payload: Object.freeze([]),
+              index: -1,
+              files: 0,
+              chars: 0,
+              sourceId: preId,
+              pathPrefix: prefix,
+              alreadyLoaded: true,
+            };
+          }
+
+          const resolved = await resolveLibrarySource(source, cwd, opts.signal);
+          if (!resolved.ok) throw new Error(resolved.error);
+          const { payload, files, chars, sourceId, pathPrefix } = resolved.value;
+
+          // Race: another concurrent load of the same prefix finished while we packed.
+          if (loaded.has(pathPrefix)) {
+            if (id) {
+              emitter?.emitSubcallUpdated({
+                id,
+                status: "done",
+                resultPreview: `already loaded (${pathPrefix}*)`,
+              });
+            }
+            return {
+              payload: Object.freeze([]),
+              index: -1,
+              files: 0,
+              chars: 0,
+              sourceId,
+              pathPrefix,
+              alreadyLoaded: true,
+            };
+          }
+
+          // Increment only after a successful sidecar write (or when no hook is set).
+          const index = nextIndex;
+          if (opts.onLoaded) {
+            await opts.onLoaded(index, payload);
+          }
+          nextIndex = index + 1;
+          loaded.add(pathPrefix);
+
+          if (id) {
+            emitter?.emitSubcallUpdated({
+              id,
+              status: "done",
+              resultPreview:
+                `+${files} file(s) → context (${pathPrefix}*, ${chars.toLocaleString()} chars)`,
+            });
+          }
+          return {
+            payload,
+            index,
+            files,
+            chars,
+            sourceId,
+            pathPrefix,
+            alreadyLoaded: false,
+          };
+        } catch (err) {
+          if (id) emitter?.emitSubcallUpdated({ id, status: "error", detail: String(err) });
+          throw err;   // serviceInterrupt catch → {error} reply → "Error: …" in the REPL
+        }
+      },
+    },
+  };
+}

package/src/bridge/llm-query.ts

@@ -125,6 +125,7 @@ export function createLlmBridge(opts: LlmBridgeOptions): LlmBridge {
       if (emitter && id !== undefined) emitter.emitSubcallUpdated({ id,
         status: error ? "error" : "done", costUsd: cost, tokens,
         resultPreview, detail: error,
+        failedCount: failed, totalCount: out.length,
       });
       return out;
     },

package/src/bridge/rlm-query.ts

@@ -4,18 +4,30 @@
  * A child RLM gets its own sandbox and iterates over the prompt as its context. At/over the
  * depth cap it degrades to a plain `llm_query` (ported from rlm/core/rlm.py `_subcall`). The
  * concurrency pool bounds parallel children for `rlm_query_batched`.
+ *
+ * `childRun` is the single spawn path shared by `rlmQuery` / `rlmQueryBatched`.
  */
 
-import type { RunRlm } from "../core/types.ts";
+import type { RlmResult, RunRlm } from "../core/types.ts";
 import type { LlmBridge } from "./llm-query.ts";
 import type { RlmEmitter } from "../tool/rlm-events.ts";
 import { checkResourceLimits } from "../core/resource-limits.ts";
 import { formatError } from "../util/errors.ts";
 import { mapPool } from "../util/concurrency.ts";
 
+export interface ChildRunInput {
+  readonly rootPrompt: string;
+  readonly context: unknown;
+  readonly depth: number;
+  readonly label?: string;
+  readonly model?: string | null;
+}
+
 export interface RlmHandlers {
   rlmQuery(prompt: string, model: string | null, depth: number): Promise<string>;
   rlmQueryBatched(prompts: string[], model: string | null, depth: number): Promise<string[]>;
+  /** Full child-run result (answer + usage) for recursive spawns. */
+  childRun(input: ChildRunInput): Promise<RlmResult>;
 }
 
 export interface RlmBridgeOptions {
@@ -33,29 +45,50 @@ export interface RlmBridgeOptions {
   readonly onChildUsage?: (costUsd: number, inputTokens: number, outputTokens: number) => void;
 }
 
+function emptyResult(answer: string): RlmResult {
+  return {
+    answer,
+    iterations: 0,
+    costUsd: 0,
+    inputTokens: 0,
+    outputTokens: 0,
+    durationMs: 0,
+  };
+}
+
 export function createRlmHandlers(opts: RlmBridgeOptions): RlmHandlers {
-  async function child(prompt: string, model: string | null, depth: number): Promise<string> {
-    const childDepth = depth + 1;
+  async function childRun(input: ChildRunInput): Promise<RlmResult> {
+    const childDepth = input.depth;
     // At the cap, a child RLM would just be an LM — short-circuit to a one-shot llm_query.
-    if (childDepth >= opts.maxDepth) return opts.llm.llmQuery(prompt, model, depth);
+    // (Callers pass the absolute child depth; rlmQuery wraps with depth+1.)
+    if (childDepth >= opts.maxDepth) {
+      const answer = await opts.llm.llmQuery(
+        input.rootPrompt || String(input.context),
+        input.model ?? null,
+        childDepth - 1,
+      );
+      return emptyResult(answer);
+    }
     let subId: string | undefined;
     try {
       const rem = opts.remainingBudget?.() ?? {};
       // Pre-spawn guard: refuse if the parent's budget or timeout is already exhausted
       // (reference: _subcall checks remaining_budget/timeout before spawning).
       const limitError = checkResourceLimits(rem);
-      if (limitError) return limitError;
+      if (limitError) return emptyResult(limitError);
+      const label = input.label ?? "rlm_query";
+      const detailSource = input.rootPrompt || String(input.context);
       subId = opts.emitter.emitSubcallCreated({
-        kind: "rlm", parentId: opts.parentNodeId, label: "rlm_query",
-        model: model ?? undefined, detail: prompt.slice(0, 60),
+        kind: "rlm", parentId: opts.parentNodeId, label,
+        model: input.model ?? undefined, detail: detailSource.slice(0, 60),
         depth: childDepth,
       });
       const res = await opts.run({
-        rootPrompt: "",
-        context: prompt,
+        rootPrompt: input.rootPrompt,
+        context: input.context,
         depth: childDepth,
         parentNodeId: subId,
-        modelOverride: model ?? undefined,
+        modelOverride: input.model ?? undefined,
         remainingBudgetUsd: rem.budgetUsd,
         remainingTimeoutMs: rem.timeoutMs,
       });
@@ -63,16 +96,27 @@ export function createRlmHandlers(opts: RlmBridgeOptions): RlmHandlers {
       opts.emitter.emitSubcallUpdated({ id: subId,
         status: "done", resultPreview: res.answer.slice(0, 200),
       });
-      return res.answer;
+      return res;
     } catch (err) {
       const msg = err instanceof Error ? err.message : String(err);
       if (subId) opts.emitter.emitSubcallUpdated({ id: subId, status: "error", detail: msg });
-      return formatError(`child RLM failed - ${msg}`);
+      return emptyResult(formatError(`child RLM failed - ${msg}`));
     }
   }
 
+  async function child(prompt: string, model: string | null, depth: number): Promise<string> {
+    const res = await childRun({
+      rootPrompt: "",
+      context: prompt,
+      depth: depth + 1,
+      model,
+    });
+    return res.answer;
+  }
+
   return {
     rlmQuery: (prompt, model, depth) => child(prompt, model, depth),
     rlmQueryBatched: (prompts, model, depth) => mapPool(prompts, opts.maxConcurrent, (p) => child(p, model, depth)),
+    childRun,
   };
 }

package/src/config/defaults.ts

@@ -21,12 +21,14 @@ export const DEFAULT_CONFIG: Readonly<RlmConfig> = Object.freeze({
   maxErrors: 5,
   orchestrator: true,
   pipeline: false,
+  maxBackwardJumps: 2,
   compaction: true,
   compactionThresholdPct: 0.65,
   python: "python3",
   sandboxInitTimeoutMs: 30_000,
   askUserQuestion: true,
   todo: true,
+  libraryLoader: true,
   rootSampling: Object.freeze({ maxTokens: 16_384 }),
   subSystemPrompt: DEFAULT_SUB_SYSTEM_PROMPT,
   subSampling: Object.freeze({ maxTokens: 8192 }),

package/src/config/settings.ts

@@ -75,6 +75,8 @@ function validateConfig(raw: unknown): Partial<RlmConfig> {
   if (orchestrator !== undefined) out.orchestrator = orchestrator;
   const pipeline = validateBoolean(r.pipeline);
   if (pipeline !== undefined) out.pipeline = pipeline;
+  const maxBackwardJumps = validateNumber(r.maxBackwardJumps, 0);
+  if (maxBackwardJumps !== undefined) out.maxBackwardJumps = maxBackwardJumps;
   const compaction = validateBoolean(r.compaction);
   if (compaction !== undefined) out.compaction = compaction;
   const compactionThresholdPct = validateNumber(r.compactionThresholdPct, 0);
@@ -92,6 +94,8 @@ function validateConfig(raw: unknown): Partial<RlmConfig> {
   if (askUserQuestion !== undefined) out.askUserQuestion = askUserQuestion;
   const todo = validateBoolean(r.todo);
   if (todo !== undefined) out.todo = todo;
+  const libraryLoader = validateBoolean(r.libraryLoader);
+  if (libraryLoader !== undefined) out.libraryLoader = libraryLoader;
   if (typeof r.subSampling === "object" && r.subSampling !== null) {
     const ss = r.subSampling as Record<string, unknown>;
     const sampling: { maxTokens?: number; temperature?: number; reasoning?: ThinkingLevel } = {};

package/src/context/library-context.ts

@@ -0,0 +1,266 @@
+/**
+ * Resolve load_library(source) into a sandbox-ready payload.
+ *
+ * Sources: local directory (repomix-packed), single file (utf-8), or remote git URL
+ * (shallow clone then pack). Host-side only — never runs in the sandbox.
+ *
+ * Every successful payload is a namespaced list of ContextFile under
+ * `lib/<source_id>/…` so the worker can append into the single `context` variable.
+ * Source ids include a short content fingerprint so two libraries that share a
+ * basename never collide.
+ */
+
+import { createHash } from "node:crypto";
+import { execFile } from "node:child_process";
+import { mkdtemp, readFile, rm, stat } from "node:fs/promises";
+import { basename, isAbsolute, join, resolve } from "node:path";
+import { tmpdir } from "node:os";
+import { promisify } from "node:util";
+import {
+  packRepository,
+  serializeForSandbox,
+  type ContextBundle,
+  type ContextFile,
+} from "./repomix-context.ts";
+import { estimateTokens } from "../text/tokens.ts";
+import type { Result } from "../util/errors.ts";
+import { errorMessage } from "../util/errors.ts";
+
+const execFileP = promisify(execFile);
+
+/** Single-file sources above this must use open() + llm_query_chunked in the REPL. */
+export const MAX_LIBRARY_FILE_BYTES = 8 * 1024 * 1024;
+
+/** Legacy catch-all prefix for pre-namespace string sidecars — never an identity key. */
+const LEGACY_UNKNOWN_PREFIX = "lib/unknown/";
+
+export interface LibrarySource {
+  /** Always a namespaced file list (dirs, single files, and git clones). */
+  readonly payload: readonly ContextFile[];
+  readonly files: number;
+  /** Sum of raw content lengths — what the model should size batches against. */
+  readonly chars: number;
+  readonly sourceId: string;
+  readonly pathPrefix: string;
+}
+
+export interface LibraryNamespace {
+  readonly sourceId: string;
+  readonly pathPrefix: string;
+}
+
+/** https://host/… or git@host:… — option-injection safe (never starts with "-"). */
+const GIT_URL = /^(https:\/\/|git@)[\w.-]+[:/]\S+$/;
+
+/** Short, stable discriminator so two sources never share a namespace. */
+function sourceFingerprint(canonical: string): string {
+  return createHash("sha256").update(canonical).digest("hex").slice(0, 8);
+}
+
+/**
+ * Sanitize a path/url basename into a stable, filesystem-safe source id.
+ * `resolvedPath` (absolute path) or the git URL is fingerprinted into the id so
+ * distinct sources sharing a basename get distinct namespaces
+ * (`lib/utils-3f9a1c02/` vs `lib/utils-a1b2c3d4/`).
+ */
+export function librarySourceId(source: string, resolvedPath?: string): string {
+  const trimmed = source.trim();
+  const isGit = GIT_URL.test(trimmed);
+  let raw: string;
+  if (isGit) {
+    const m = trimmed.match(/(?:\/|:)([\w.-]+?)(?:\.git)?\/?\s*$/);
+    raw = m?.[1] ?? "repo";
+  } else {
+    raw = basename(resolvedPath ?? trimmed);
+  }
+  const cleaned = raw
+    .replace(/\.git$/i, "")
+    .replace(/[^\w.-]+/g, "-")
+    .replace(/^-+|-+$/g, "")
+    .slice(0, 80);
+  const canonical = isGit ? trimmed : (resolvedPath ?? trimmed);
+  return `${cleaned.length > 0 ? cleaned : "lib"}-${sourceFingerprint(canonical)}`;
+}
+
+export function pathPrefixFor(sourceId: string): string {
+  return `lib/${sourceId}/`;
+}
+
+/**
+ * Derive the namespace for a source string without packing (host-side idempotency).
+ * Returns both sourceId and pathPrefix so callers never un-parse the prefix.
+ */
+export function libraryNamespace(source: string, cwd: string): LibraryNamespace {
+  const trimmed = source.trim();
+  const sourceId = GIT_URL.test(trimmed)
+    ? librarySourceId(trimmed)
+    : librarySourceId(trimmed, isAbsolute(trimmed) ? trimmed : resolve(cwd, trimmed));
+  return Object.freeze({ sourceId, pathPrefix: pathPrefixFor(sourceId) });
+}
+
+/** Derive the path prefix for a source string without packing. */
+export function libraryPathPrefix(source: string, cwd: string): string {
+  return libraryNamespace(source, cwd).pathPrefix;
+}
+
+/** Namespaced files plus summed content chars (one pass). */
+export function namespaceLibraryFilesWithChars(
+  payload: unknown,
+  sourceId: string,
+): { readonly files: readonly ContextFile[]; readonly chars: number } {
+  const prefix = pathPrefixFor(sourceId);
+  if (typeof payload === "string") {
+    return Object.freeze({
+      files: Object.freeze([
+        Object.freeze({
+          path: `${prefix}content`,
+          content: payload,
+          tokens: Math.max(1, estimateTokens(payload.length)),
+        }),
+      ]),
+      chars: payload.length,
+    });
+  }
+  if (!Array.isArray(payload)) return Object.freeze({ files: Object.freeze([]), chars: 0 });
+  const out = new Array<ContextFile>(payload.length);
+  let n = 0;
+  let chars = 0;
+  for (const item of payload) {
+    if (item === null || typeof item !== "object") continue;
+    const rec = item as Record<string, unknown>;
+    const content = typeof rec.content === "string" ? rec.content : String(rec.content ?? "");
+    let path = typeof rec.path === "string" ? rec.path : "unknown";
+    path = path.replace(/^\/+/, "");
+    if (!path.startsWith(prefix)) path = `${prefix}${path}`;
+    const tokens = typeof rec.tokens === "number" && Number.isFinite(rec.tokens)
+      ? Math.max(0, Math.floor(rec.tokens))
+      : Math.max(1, estimateTokens(content.length));
+    out[n++] = Object.freeze({ path, content, tokens });
+    chars += content.length;
+  }
+  out.length = n;
+  return Object.freeze({ files: Object.freeze(out), chars });
+}
+
+/** Namespace file entries under `lib/<sourceId>/…` (single shared implementation). */
+export function namespaceLibraryFiles(
+  payload: unknown,
+  sourceId: string,
+): readonly ContextFile[] {
+  return namespaceLibraryFilesWithChars(payload, sourceId).files;
+}
+
+/** First `lib/<id>/` prefix found in the payload, or undefined. Skips `lib/unknown/`. */
+export function payloadPrefix(payload: readonly unknown[]): string | undefined {
+  for (let i = 0; i < payload.length; i++) {
+    const item = payload[i];
+    if (item === null || typeof item !== "object") continue;
+    const path = (item as { path?: unknown }).path;
+    if (typeof path !== "string") continue;
+    const m = /^(lib\/[^/]+\/)/.exec(path);
+    // `lib/unknown/` is the legacy catch-all: never treat it as an identity.
+    if (m?.[1] !== undefined && m[1] !== LEGACY_UNKNOWN_PREFIX) return m[1];
+  }
+  return undefined;
+}
+
+/**
+ * Append a library payload into an existing list context (host-side resume merge).
+ * Skips a payload whose path prefix is already present (resume-safe dedup).
+ */
+export function mergeLibraryIntoContext(base: unknown, libraryPayload: unknown): unknown {
+  if (!Array.isArray(base)) return base;
+  if (Array.isArray(libraryPayload)) {
+    if (libraryPayload.length === 0) return base;
+    const prefix = payloadPrefix(libraryPayload);
+    if (prefix !== undefined) {
+      for (let i = 0; i < base.length; i++) {
+        const item = base[i];
+        if (item !== null && typeof item === "object"
+          && typeof (item as { path?: unknown }).path === "string"
+          && (item as { path: string }).path.startsWith(prefix)) {
+          return base; // already present
+        }
+      }
+    }
+    const merged = new Array<unknown>(base.length + libraryPayload.length);
+    for (let i = 0; i < base.length; i++) merged[i] = base[i];
+    for (let i = 0; i < libraryPayload.length; i++) merged[base.length + i] = libraryPayload[i];
+    return merged;
+  }
+  if (typeof libraryPayload === "string") {
+    // Legacy string sidecars: wrap once under an unknown prefix.
+    return mergeLibraryIntoContext(base, namespaceLibraryFiles(libraryPayload, "unknown"));
+  }
+  return base;
+}
+
+function toLibrarySource(payload: unknown, sourceId: string): LibrarySource {
+  const { files, chars } = namespaceLibraryFilesWithChars(payload, sourceId);
+  return {
+    payload: files,
+    files: files.length,
+    chars,
+    sourceId,
+    pathPrefix: pathPrefixFor(sourceId),
+  };
+}
+
+export async function resolveLibrarySource(
+  source: string,
+  cwd: string,
+  signal?: AbortSignal,
+): Promise<Result<LibrarySource, string>> {
+  const trimmed = source.trim();
+  if (trimmed === "") return { ok: false, error: "load_library: empty source" };
+  if (GIT_URL.test(trimmed)) return await cloneAndPack(trimmed, signal);
+  if (/^[a-z][a-z0-9+.-]*:\/\//i.test(trimmed)) {
+    return { ok: false, error: `unsupported URL scheme (only https:// and git@ are allowed): ${trimmed}` };
+  }
+  const path = isAbsolute(trimmed) ? trimmed : resolve(cwd, trimmed);
+  let s: Awaited<ReturnType<typeof stat>>;
+  try {
+    s = await stat(path);
+  } catch {
+    return { ok: false, error: `load_library: path not found: ${path}` };
+  }
+  const sourceId = librarySourceId(trimmed, path);
+  if (s.isDirectory()) return await packDir(path, sourceId, signal);
+  if (s.size > MAX_LIBRARY_FILE_BYTES) {
+    return {
+      ok: false,
+      error: `load_library: ${path} is ${s.size.toLocaleString()} bytes `
+        + `(limit ${MAX_LIBRARY_FILE_BYTES.toLocaleString()}) — `
+        + "open() it in the REPL and delegate with llm_query_chunked instead",
+    };
+  }
+  const text = await readFile(path, "utf-8");
+  return { ok: true, value: toLibrarySource(text, sourceId) };
+}
+
+async function packDir(
+  dir: string,
+  sourceId: string,
+  signal?: AbortSignal,
+): Promise<Result<LibrarySource, string>> {
+  const packed = await packRepository(dir, signal);
+  if (!packed.ok) return { ok: false, error: `pack failed for ${dir} — ${packed.error}` };
+  return { ok: true, value: bundleToSource(packed.value, sourceId) };
+}
+
+function bundleToSource(bundle: ContextBundle, sourceId: string): LibrarySource {
+  return toLibrarySource(serializeForSandbox(bundle), sourceId);
+}
+
+async function cloneAndPack(url: string, signal?: AbortSignal): Promise<Result<LibrarySource, string>> {
+  const dir = await mkdtemp(join(tmpdir(), "rlm-lib-"));
+  const sourceId = librarySourceId(url);
+  try {
+    await execFileP("git", ["clone", "--depth", "1", "--", url, dir], { signal, timeout: 120_000 });
+    return await packDir(dir, sourceId, signal);
+  } catch (err: unknown) {
+    return { ok: false, error: `git clone failed for ${url} — ${errorMessage(err)}` };
+  } finally {
+    await rm(dir, { recursive: true, force: true }).catch(() => {});
+  }
+}