@autorag/librarian 2.10.1 → 2.10.3

package/NOTICE

@@ -63,9 +63,6 @@ License: Apache-2.0
 typebox@1.3.37
 License: MIT
 
-yaml@2.9.1
-License: ISC
-
 Embedding runtime and model notices:
 licenses/llama.cpp-MIT.txt
 licenses/qwen3-embedding-notice.txt

package/README.md

@@ -136,7 +136,7 @@ does, and searches them right alongside your files:
   File contents are never sent to a remote embedder.
 - **Keys stay encrypted.** Sign-in tokens and API keys live in the system's
   encrypted store, never in a config file.
-- **Bring your own model.** Use the [hosted AutoRAG plan](#hosted-autorag-plan),
+- **Bring your own model.** Use the [hosted Dazzi Models plan](#hosted-dazzi-models-plan),
   or connect your own provider key and pay the provider directly.
 
 > [!TIP]
@@ -186,7 +186,7 @@ Three principles drive every design decision in AutoRAG Agent:
 
 1. **Never migrate your data to search it.** Traditional RAG systems force you to upload, ETL, and duplicate your files into a centralized vector database. AutoRAG Agent federates your data **in place**, querying CLI-native stores (`lazykatok`, `discrawl`, `slacrawl`, `mailcrawl`, `rclone`, `qmd`) where your data already lives. Results retain opaque, source-native identities (`/kakao/...`, `/slack/...`) that preserve local access control and privacy. *(See our [Competitive Landscape Study](docs/competitive-landscape-2026-09.md) on why in-place federation is the durable differentiator).*
 
-2. **Just works — no RAG degree required.** No pipeline tuning, no vector-DB maintenance, and no remote embedding keys required. AutoRAG Agent automatically manages [MinSync](docs/minsync-setup.md) for incremental Change Data Capture (CDC) chunking and provides a local [embedding gateway](docs/embedding-runtime.md) out of the box with zero external telemetry. A [remote rerank model](docs/rerank.md) (OpenRouter, `voyageai/rerank-3-lite` by default) is an opt-in extra.
+2. **Just works — no RAG degree required.** No pipeline tuning, no vector-DB maintenance, and no remote embedding keys required. AutoRAG Agent automatically manages [MinSync](docs/minsync-setup.md) for incremental Change Data Capture (CDC) chunking and provides a local [embedding gateway](docs/embedding-runtime.md) out of the box with zero external telemetry. A [remote rerank model](docs/rerank.md) (the hosted Dazzi Models reranker when signed in, else OpenRouter `voyageai/rerank-3-lite`) is an opt-in extra.
 
 3. **Fast by design.** Rather than coordinating slow multi-agent hierarchies, a single configured model owns the entire retrieval, direct-read, and curation loop. Local CDC chunks (BM25, vector, and hybrid modes) deliver rapid, low-latency turnaround across multi-turn research queries.
 
@@ -304,6 +304,26 @@ Tesseract. Set `parserOptions.ocr.cachePath` to store the Tesseract models
 elsewhere, or `KORDOC_OFFLINE=1` to block the built-in model download. An
 explicit `parserOptions.kordoc.ocr` overrides the language-based choice.
 
+### Bounding and cancelling a parse
+
+`kordoc` (PDF, HWP/HWPX, DOCX, XLSX/XLS, images), `.pptx` and `.eml` parse in a worker
+thread during a refresh, so one corrupt or hostile file cannot hang or crash the run. A
+`Promise.race` deadline cannot stop a synchronous parse; terminating the worker can.
+
+| `parserOptions` key | Default | Effect |
+|---|---|---|
+| `fileTimeoutMs` | `120000` | Per-file deadline. A parse still running is killed with its worker. |
+| `maxHeapMb` | Node's default | V8 old-generation limit for the worker. An out-of-memory parse is caught, not a host crash. |
+
+A timeout, a heap overrun and a worker crash each become one `parser-failed` skip for that
+file; the diagnostic message says which. The rest of the corpus is still indexed. Plain-text
+files and parsers configured with a function (`parserOptions.kordoc.parse`, a custom OCR
+provider) parse in-process.
+
+```json
+{ "parserOptions": { "fileTimeoutMs": 60000, "maxHeapMb": 2048 } }
+```
+
 Photos and videos are also reachable through two other paths that are not part
 of this switch: the `clawgallery` datasource (captions and visual search over its
 own folders) and Jikji's media index (`jikji.enableMediaIndex`, file facts such as
@@ -375,6 +395,32 @@ await run;
 
 `steer()` reaches the search's next model request (also between phases), cuts in-flight retrieval and the baseline prefetch short, and always runs verification, whose prompt restates the correction. It returns `false` when no search is running, the search was stopped, or it is past its last model request; ask the text as a new question then.
 
+A refresh can be cancelled and observed from code:
+
+```typescript
+const controller = new AbortController();
+try {
+  await agent.refresh(false, {
+    signal: controller.signal, // controller.abort() stops it
+    onProgress: ({ phase, processed, total, currentVirtualPath }) => {
+      console.log(`${phase} ${processed}/${total}`, currentVirtualPath ?? "");
+    },
+  });
+} catch (error) {
+  if (controller.signal.aborted) console.log("refresh cancelled");
+  else throw error;
+}
+```
+
+`signal` is checked between files and between phases, kills an in-flight parse worker, and
+terminates the MinSync, Jikji and datasource child processes (and their descendants). The
+refresh then rejects with the signal's reason and is recorded as `failed`; files already
+mirrored are kept and not re-parsed on the next run. `onProgress` reports `parsed`
+(`processed`/`total` are files, `currentVirtualPath` is the file about to be processed) and
+`datasources` (`processed`/`total` are datasource skills). During the parse phase
+`.autorag/refresh-progress.json` also carries the running `sourceFiles.total` and
+`parsedCounts`. Without these options a refresh behaves as before.
+
 ---
 
 ## CLI Command Reference
@@ -504,7 +550,7 @@ Do not copy or retain the removed `autorag-lite-search` skill.
    ```
 4. **Safety Guidelines for Agents:**
    - **Never delete, move, or modify original user documents.**
-   - AutoRAG writes index artifacts strictly into `<workspace>/.autorag/` and `.jikji/` caches.
+   - AutoRAG writes index artifacts into `<workspace>/.autorag/` and Jikji's central store (`<OS data dir>/jikji/`, override with `JIKJI_DATA_DIR`). Jikji also drops a `.jikji_agent_map.md` guide into each indexed root.
    - Never print or leak API keys, tokens, or credential values into stdout or logs.
 
 ---
@@ -536,73 +582,69 @@ For configuration syntax and connector details, see [docs/datasource-skills.md](
 
 ---
 
-## Hosted AutoRAG Plan
+## Hosted Dazzi Models Plan
 
-AutoRAG can run on the hosted **AutoRAG** plan as a first-class model provider named
-`autorag`. The provider speaks the server's OpenAI Responses API surface (`<root>/v1`),
+AutoRAG can run on the hosted **Dazzi Models** plan as a first-class model provider named
+`dazzi`. The provider speaks the server's OpenAI Responses API surface (`<root>/v1`),
 so streaming, tool calls, and usage are handled by Pi's built-in `openai-responses`
 implementation, and the model catalog is discovered live from the server — no model id
 is baked into AutoRAG.
 
-Sign in from the interactive TUI:
+Sign in from the interactive TUI. Login is the only way to authenticate; there is no API-key
+environment variable:
 
 ```text
 autorag tui
-/login          # choose "AutoRAG"; the browser completes a PKCE login
+/login          # choose "Dazzi Models"; the browser completes a PKCE login
 ```
 
-Headless (no browser) with a long-lived API key:
-
-```bash
-export AUTORAG_API_KEY=dz_...
-```
+The provider, Jev, and rerank all use the credential `/login` stores. A login flow for
+machines with no browser is not available yet and is tracked in [#1913](https://github.com/Marker-Inc-Korea/AutoRAG/issues/1913).
 
 On a fresh agent home, pin the model with an explicit endpoint so it resolves
-offline without a catalog snapshot — this is the headless configuration:
+offline without a catalog snapshot:
 
 ```json
 {
   "model": {
-    "provider": "autorag",
+    "provider": "dazzi",
     "id": "anthropic/claude-haiku-5.5",
     "baseUrl": "https://api.dazziapp.com/v1",
-    "api": "openai-responses",
-    "apiKeyEnv": "AUTORAG_API_KEY"
+    "api": "openai-responses"
   }
 }
 ```
 
-The `provider`/`id`-only shorthand (`{ "provider": "autorag", "id": "anthropic/claude-haiku-5.5" }`)
+The `provider`/`id`-only shorthand (`{ "provider": "dazzi", "id": "anthropic/claude-haiku-5.5" }`)
 only resolves once a TUI `/login` or a model refresh has persisted the catalog
 snapshot; with no snapshot there is nothing to restore offline, and resolution
-fails with `Unknown configured model: autorag/<id>`.
+fails with `Unknown configured model: dazzi/<id>`.
 
 | Variable | Default | Meaning |
 |---|---|---|
 | `AUTORAG_BASE_URL` | `https://api.dazziapp.com` | Server **root** (not the `/v1` path); the API base is `<root>/v1`. |
-| `AUTORAG_API_KEY` | – | Headless API key. A stored `/login` credential takes precedence. |
 
 The catalog is persisted after a `/login` (or any model refresh), so the
-`provider`/`id` shorthand `model: { provider: "autorag", id }`
+`provider`/`id` shorthand `model: { provider: "dazzi", id }`
 keeps resolving offline. When you are not signed in the provider still registers — it appears under
 `/login`, contributes no models, and makes no network call. See
-[Hosted AutoRAG provider](docs/cloud-provider.md) for the full flow.
+[Hosted Dazzi Models provider](docs/cloud-provider.md) for the full flow.
 
-The same plan can also serve the **Jev** judgment model and **rerank** stage, so
+The same sign-in can also serve the **Jev** judgment model and **rerank** stage, so
 the whole librarian loop runs on the hosted account:
 
 ```json
 {
-  "jev": { "backend": "autorag", "model": "jev-1.13" },
-  "rerank": { "provider": "autorag", "model": "qwen/qwen3-reranker-8b" }
+  "jev": { "backend": "dazzi", "model": "jev-1.13" },
+  "rerank": { "provider": "dazzi", "model": "qwen/qwen3-reranker-8b" }
 }
 ```
 
-`jev.backend: "autorag"` posts Jev's native dialect to `<root>/v1/jev`
-(default model `jev-1.13`); `rerank.provider: "autorag"` reranks against
-`<root>/v1/rerank` (default model `qwen/qwen3-reranker-8b`). Both read the
-account key from `AUTORAG_API_KEY`, and an explicit `rerank.baseUrl`,
-`rerank.apiKeyEnv`, or `rerank.model` still overrides the defaults.
+`jev.backend: "dazzi"` posts Jev's native dialect to `<root>/v1/jev`
+(default model `jev-1.13`); `rerank.provider: "dazzi"` reranks against
+`<root>/v1/rerank` (default model `qwen/qwen3-reranker-8b`). Both use the stored
+`/login` credential, and an explicit `rerank.baseUrl` or `rerank.model` still overrides
+the defaults.
 
 ---
 
@@ -612,9 +654,9 @@ Deep dive into AutoRAG Agent's architecture, security, and integration guides:
 
 - **[MinSync Setup & Embedding QA](docs/minsync-setup.md):** Automatic binary installation, CDC chunking, and local embedding verification.
 - **[Local Embedding Runtime & Gateway](docs/embedding-runtime.md):** AutoRAG-owned local gateway, model prefetching, and zero-egress semantic search. Model card: [`qwen3-embedding-0.6b`](docs/model-cards/qwen3-embedding-0.6b.md).
-- **[Hosted AutoRAG Provider](docs/cloud-provider.md):** The hosted `autorag` model provider — PKCE `/login`, `AUTORAG_API_KEY`, `AUTORAG_BASE_URL`, live model discovery, and offline catalog snapshots.
+- **[Hosted Dazzi Models Provider](docs/cloud-provider.md):** The hosted `dazzi` model provider — PKCE `/login`, `AUTORAG_BASE_URL`, live model discovery, and offline catalog snapshots.
 - **[Datasource Skills Reference](docs/datasource-skills.md):** Full configuration contracts, connection aliases, and connector options.
-- **[Jev Decisions](docs/jev-decisions.md):** Jev query pipeline, **on by default** via OpenRouter: it routes each question to a direct answer (intrinsic knowledge) or local search (everything needing more information; web search is left to the agent after the fast answer), splits multi-part questions into up to five parallel searches (`openrouter/qwen/qwen3.7-flash`), picks which registered datasources to search before the fast answer, and ends the run after the fast answer when that answer is complete. Also covers the `jev` judgment tool, backends (OpenRouter, TypeSafe, Vercel AI Gateway), and how to opt out.
+- **[Jev Decisions](docs/jev-decisions.md):** Jev query pipeline, **on by default** via Dazzi Models when signed in (OpenRouter otherwise): it routes each question to a direct answer (intrinsic knowledge) or local search (everything needing more information; web search is left to the agent after the fast answer), splits multi-part questions into up to five parallel searches (`dazzi/openai/gpt-6-luna` when signed in, else `openrouter/qwen/qwen3.7-flash`), picks which registered datasources to search before the fast answer, and ends the run after the fast answer when that answer is complete. Also covers the `jev` judgment tool, backends (OpenRouter, TypeSafe, Vercel AI Gateway), and how to opt out.
 - **[Manual QA & Datasource Test Harnesses](docs/manual-qa-datasources.md):** Real-world testing guides for Discord, KakaoTalk, Slack, Notion, and email.
 - **[P2P SimpleX Sharing & Path Standard](docs/p2p-path-standard.md):** Decentralized peer query sharing with SimpleX, PII redaction, and approval queues.
 - **[Supply Chain Security & License Audits](docs/supply-chain.md):** Software bill of materials (SBOM) and dependency gate policies.
@@ -684,8 +726,9 @@ Common failures and their fix:
 | A datasource errors during refresh | `datasource-index-failed` | run that CLI's own `doctor` |
 | MinSync binary missing (search and refresh fail with `MinSync is required ...`) | — | `cargo install minsync`, or leave `minSync.autoInstall` on and re-run refresh |
 | Jikji missing | `jikji-unavailable` | check the Rust toolchain, re-run refresh |
+| Jikji prepare of a large root times out (`the Jikji prepare run timed out`) | `jikji-prepare-failed` | the refresh still finishes and the other roots are indexed. There is no fixed total wall: the per-root hang guard is `jikji.timeoutMs` (60 s base) plus `jikji.prepareTimeoutMsPerFile` ms per file under the root (default: `jikji.parseTimeout`, 5 s), uncapped, so a huge root is not cut by its size and only a run stuck beyond every file hitting its parse limit is killed. Set `prepareTimeoutMsPerFile: 0` for a flat `timeoutMs` wall. `jikji.parseTimeout` (seconds) is the per-file limit, forwarded to `jikji prepare --parse-timeout`; jikji skips a file that exceeds it. A timeout kills jikji's whole process group |
 | Windows file-name search fails during refresh | `everything-index-failed` | read the ES exit code and stderr in the message, then `autorag refresh --method everything --json` |
-| Every search does a full local search and verification, even for small talk | `query-route-fallback` | set `OPENROUTER_API_KEY` (Jev routing and decomposition are on by default) |
+| Every search does a full local search and verification, even for small talk | `query-route-fallback` | sign in to Dazzi Models with `/login`, or set `OPENROUTER_API_KEY` (Jev routing and decomposition are on by default) |
 
 Native datasource stores stay owned by their CLIs — AutoRAG never rebuilds them. Fix a broken archive with `lazykatok doctor`, `discrawl --json metadata`, `slacrawl --json doctor`, `wacrawl --json doctor`, `telecrawl --json doctor`, `notcrawl doctor`, `qmd status`, or `mailcrawl doctor`, then re-run `autorag refresh --method datasources --json`.
 

package/dist/agent/agent.d.ts

@@ -1,14 +1,15 @@
 import type { Agent, AgentTool } from "@earendil-works/pi-agent-core";
 import type { Api, ImageContent, Model } from "@earendil-works/pi-ai";
+import { type DatasourceRetryOptions } from "../datasource/retry.js";
 import type { DatasourceIndexResult, DatasourceSkill } from "../datasource/types.js";
 import { type DupeyCliOptions } from "../dupey/index.js";
 import { type Embedder } from "../embedding-runtime/gateway-embedder.js";
 import { type EverythingClientOptions, type EverythingSearchRequest, type EverythingSearchResult } from "../everything/index.js";
 import { type FSearchClientOptions, type FSearchFailureReason, type FSearchSearchRequest, type FSearchSearchResult } from "../fsearch/index.js";
-import { type JikjiFailureReason, type JikjiOptions } from "../jikji/index.js";
+import { type JikjiFailureReason, type JikjiOptions, type JikjiPrepareOptions } from "../jikji/index.js";
 import { type LanguageTag } from "../language.js";
 import { type MinSyncSyncResult, type MinSyncVectorMethodOptions } from "../minsync/index.js";
-import { type ParsedMirrorSyncResult } from "../mirror/sync.js";
+import { type ParsedMirrorSyncOptions, type ParsedMirrorSyncResult } from "../mirror/sync.js";
 import type { PolicyResolver } from "../p2p/policy-filter.js";
 import type { SimplexQueryState } from "../p2p/simplex-query-store.js";
 import { type SimplexTransport } from "../p2p/simplex-transport.js";
@@ -26,9 +27,28 @@ import { type AutoRAGPiInteractiveRuntime, type AutoRAGPiSession } from "./pi-se
 import { type RefreshProgressCounts, type RefreshProgressPhase } from "./refresh-progress.js";
 import { type SearchDocumentDiagnostic, type SearchDocumentsResponse, type SearchDocumentsStreamEvent } from "./search-documents.js";
 import { type SelfConfigOptions } from "./self-config.js";
+import { type ToolBurstGuardOptions } from "./tool-burst-guard.js";
 import { type WatcherFactory, type WatchRefreshHandle } from "./watch-refresh.js";
 import { type WebFetchToolOptions } from "./web-fetch-tool.js";
 import { type WebSearchToolOptions } from "./web-search-tool.js";
+/**
+ * Safety limits for the parsed-mirror refresh. Every field is optional; an
+ * omitted field keeps the shipped default.
+ */
+export interface AutoRAGRefreshLimits {
+    /**
+     * Largest share (0–1) of indexed documents one refresh may delete because
+     * their source disappeared; a bigger deletion above 100 documents is
+     * suppressed unless the refresh is forced. Default 0.3.
+     */
+    readonly maxDeleteRatio?: number;
+    /** Directory entries examined per search root before the walk stops. Default 2,000,000. */
+    readonly maxScannedEntries?: number;
+    /** Deepest directory level walked below a search root. Default 64. */
+    readonly maxDepth?: number;
+    /** Retries for a transient per-file I/O error (EMFILE/EBUSY/EIO/ETIMEDOUT) before the file is skipped. Default 3. */
+    readonly transientRetries?: number;
+}
 /**
  * Hard caps on retrieval, baseline prefetch, and the candidate lists handed to
  * the model. Every field is optional: an omitted field keeps the shipped
@@ -64,9 +84,40 @@ export interface AutoRefreshOptions {
 }
 /** Methods that `refresh` can selectively run. Defaults to all when omitted. */
 export type RefreshMethod = "parsed" | "minsync" | "datasources" | "jikji" | "everything" | "fsearch";
+/**
+ * Progress of a refresh phase that can count its work. `parsed` counts source files
+ * (`currentVirtualPath` is the opaque virtual path of the file about to be processed);
+ * `datasources` counts configured datasource skills. Other phases do not report counts.
+ */
+export interface AutoRAGRefreshProgress {
+    readonly phase: "parsed" | "datasources";
+    /** Items finished so far. Advances by one per item and ends equal to `total`. */
+    readonly processed: number;
+    readonly total: number;
+    readonly currentVirtualPath?: string;
+}
 export interface AutoRAGRefreshOptions {
     /** Restrict refresh to specific methods. Defaults to all when undefined. */
     readonly methods?: readonly RefreshMethod[];
+    /**
+     * Retry sources previously skipped as parser-failed or source-unreadable once, ignoring their
+     * skip records. Unlike `force`, healthy unchanged mirrors are not re-parsed.
+     */
+    readonly retrySkipped?: boolean;
+    /**
+     * Cancels the refresh. It is checked between files and between phases, and it kills an in-flight parse
+     * worker and the MinSync, Jikji and datasource child processes. `refresh` then rejects with the signal's
+     * reason and records the run as failed; mirror entries written before the abort are kept.
+     */
+    readonly signal?: AbortSignal;
+    /** Called while the parse and datasource phases run. A throwing callback is ignored; it never fails the refresh. */
+    readonly onProgress?: (progress: AutoRAGRefreshProgress) => void;
+    /**
+     * Per-root `jikji prepare` controls: a hard `timeoutMs` wall per root and a
+     * `parseTimeout` (seconds, forwarded to `--parse-timeout`). Unset values fall
+     * back to the `jikji` client options. Cancel through the refresh `signal`.
+     */
+    readonly jikjiPrepare?: Omit<JikjiPrepareOptions, "signal">;
 }
 export interface AutoRAGMinSyncRefreshResult {
     readonly ok: boolean;
@@ -185,7 +236,6 @@ export interface AutoRAGAgentOptions {
     providerApiKeys?: Readonly<Record<string, string>>;
     searchPaths: string[];
     languages?: readonly LanguageTag[];
-    manifestDir?: string;
     memoryPath?: string;
     workspacePath?: string;
     tools?: AgentTool[];
@@ -278,13 +328,23 @@ export interface AutoRAGAgentOptions {
      * {@link AutoRAGRetrievalLimits}. Values must be positive integers.
      */
     limits?: AutoRAGRetrievalLimits;
+    /** Parsed-mirror refresh limits and retry tuning. See {@link AutoRAGRefreshLimits}. */
+    refresh?: AutoRAGRefreshLimits;
     datasourceSkills?: readonly DatasourceSkill[];
+    /**
+     * Retry/backoff for datasource `index()` during refresh: 2 retries with
+     * jittered exponential backoff by default, applied only to failures the
+     * skill marks retryable. `maxRetries: 0` disables retrying.
+     */
+    datasourceRetry?: DatasourceRetryOptions;
     /** Non-fatal diagnostics from config/agent construction (e.g. skipped unknown datasources). */
     startupDiagnostics?: readonly SearchDocumentDiagnostic[];
-    /** Maximum time a model/tool search may run before it is aborted. */
-    searchTimeoutMs?: number;
-    /** Maximum number of retrieval/tool executions allowed in one search. */
-    maxSearchToolCalls?: number;
+    /**
+     * Runaway-loop guard: the run is aborted when one tool is called `maxCalls`
+     * times within `windowMs` (default 60 calls in 60 s). `false` disables it.
+     * There is no limit on a run's duration or its total number of tool calls.
+     */
+    toolCallBurstGuard?: ToolBurstGuardOptions | false;
     /**
      * Outbound SimpleX queries to trusted peer AutoRAG agents. Remote sessions
      * never receive this tool.
@@ -313,9 +373,9 @@ export interface RerankAgentOptions {
     provider?: string;
     /** Wire model id. @default "voyageai/rerank-3-lite" */
     model?: string;
-    /** API key. Prefer `apiKeyEnv`; this is a programmatic seam. */
+    /** API key. Prefer `apiKeyEnv`; this is a programmatic seam. Not used by the `dazzi` provider. */
     apiKey?: string;
-    /** Environment variable holding the provider API key. */
+    /** Environment variable holding the provider API key. Not used by the `dazzi` provider, which uses the `/login` credential. */
     apiKeyEnv?: string;
     /** Override the provider base URL. */
     baseUrl?: string;
@@ -368,7 +428,11 @@ export declare class AutoRAGAgent {
     private activeRunAbort;
     private modelNativeSearchAuth;
     private retrievalTrace;
-    /** Evidence every retrieval tool returned this run; the answer's inline `[eN]` citations resolve against it. */
+    /**
+     * Evidence the retrieval tools returned; the answer's inline `[eN]` citations resolve against it. A bound
+     * interactive session keeps it for the whole conversation and persists it with the transcript; any other
+     * run starts from an empty ledger.
+     */
     private readonly evidenceLedger;
     private preliminaryCallback;
     /** Per-phase thinking levels of the two-phase search. */
@@ -378,6 +442,8 @@ export declare class AutoRAGAgent {
     private refreshing;
     private refreshState;
     private readonly searchPaths;
+    /** Roots handed to Jikji: duplicates and roots nested in another root are dropped (see `outermostJikjiRoots`). */
+    private readonly jikjiRoots;
     private readonly configuredSearchPaths;
     readonly languages: readonly LanguageTag[];
     private retrievalScopeBindings;
@@ -393,6 +459,7 @@ export declare class AutoRAGAgent {
     private readonly everythingClient;
     private readonly fsearchClient;
     private readonly datasourceSkills;
+    private readonly datasourceRetry;
     private readonly startupDiagnostics;
     private readonly datasourceAgentSkills;
     private readonly parserOptions;
@@ -412,8 +479,6 @@ export declare class AutoRAGAgent {
     private readonly excludePaths;
     private readonly baseSystemPromptConfig;
     private readonly droppedCallerToolNames;
-    private readonly searchTimeoutMs;
-    private readonly maxSearchToolCalls;
     private readonly piAgentDir;
     private readonly piSessionDir;
     private readonly persistPiSessions;
@@ -433,8 +498,9 @@ export declare class AutoRAGAgent {
     /** True when this agent was constructed for an untrusted remote peer. */
     readonly remoteSession: boolean;
     private activeRetrievalOptions;
-    private searchToolCallCount;
+    private readonly toolBurstGuard;
     private readonly limits;
+    private readonly refreshLimits;
     constructor(options: AutoRAGAgentOptions);
     scanDuplicateDocuments(): Promise<ScanDuplicateDocumentsDetails>;
     /**
@@ -646,6 +712,12 @@ export declare class AutoRAGAgent {
      * as plain text, without any further tool calls.
      */
     buildFastAnswerPrompt(query: string, options: RetrievalOptions, baseline: string): string;
+    /**
+     * True when the bound interactive session already holds an assistant turn:
+     * the query continues a conversation instead of opening one. Read from the
+     * transcript, so a fresh session (`/new`) starts over with the full pipeline.
+     */
+    private isConversationTurn;
     /**
      * Prompt for a question Jev routed to a direct answer: general knowledge or
      * small talk. No retrieval ran and no verification phase follows, so the
@@ -695,7 +767,7 @@ export declare class AutoRAGAgent {
      */
     startWatchRefresh(options?: AutoRAGWatchRefreshOptions): AutoRAGWatchRefreshHandle;
     private defaultWatcherFactory;
-    syncParsedMirrors(force?: boolean): Promise<ParsedMirrorSyncResult>;
+    syncParsedMirrors(force?: boolean, options?: Pick<ParsedMirrorSyncOptions, "signal" | "onProgress" | "retrySkipped">): Promise<ParsedMirrorSyncResult>;
     /**
      * Lightweight stat-only staleness scan used when `refresh` is called with
      * methods that exclude parsed mirrors (e.g. only `datasources` or `jikji`).
@@ -704,7 +776,7 @@ export declare class AutoRAGAgent {
      */
     private scanMirrorStaleness;
     private exactDuplicateExclusions;
-    syncMinSync(force?: boolean): Promise<MinSyncSyncResult>;
+    syncMinSync(force?: boolean, signal?: AbortSignal): Promise<MinSyncSyncResult>;
     /** Provider for the Windows-only everything_search tool. */
     searchEverything(request: EverythingSearchRequest): Promise<EverythingSearchResult>;
     /**
@@ -722,12 +794,18 @@ export declare class AutoRAGAgent {
      * macOS/Linux.
      */
     stopFsearch(): Promise<void>;
-    prepareJikji(): Promise<readonly AutoRAGJikjiPrepareResult[] | undefined>;
+    prepareJikji(options?: JikjiPrepareOptions): Promise<readonly AutoRAGJikjiPrepareResult[] | undefined>;
     /**
      * Build or incrementally update every root's Jikji index. Called only by
      * refresh/watch (and the explicit `prepareJikji()` API), never by a query:
-     * `jikji prepare` reuses unchanged documents, and roots run in parallel so
-     * one slow root does not serialize the rest.
+     * `jikji prepare` reuses unchanged documents. Roots are submitted together
+     * but `JikjiClient` runs them one at a time, because every root shares one
+     * central SQLite file that fails with "database is locked" under concurrent
+     * writers. Nested and duplicate roots are skipped: upstream rejects
+     * overlapping roots and the outer root's index already covers them. A failed
+     * or timed-out root is a degraded result (reported as a diagnostic), never a
+     * rejection, so the refresh still finishes. `jikji.parseTimeout` /
+     * `jikji.timeoutMs` / `jikji.prepareTimeoutMsPerFile` apply to every root.
      */
     private executeJikjiPrepare;
     private sanitizeJikjiPrepareResult;

package/dist/agent/evidence-ledger.d.ts

@@ -1,3 +1,4 @@
+import { type Static, Type } from "typebox";
 import type { RetrievalResult } from "../retrieval/types.js";
 import type { AutoRAGEvidenceRef } from "./results.js";
 export interface LedgerEvidenceInput {
@@ -13,28 +14,65 @@ export interface LedgerEvidenceInput {
     readonly evidenceType?: string;
     readonly evidenceLocation?: string;
 }
+/** Custom session entry type under which a conversation persists its evidence ledger. */
+export declare const EVIDENCE_ENTRY_TYPE = "autorag.evidence";
+declare const ledgerEntrySchema: Type.TObject<{
+    id: Type.TString;
+    /** Source plus a digest of the full chunk: the same chunk retrieved again keeps its id. */
+    key: Type.TString;
+    method: Type.TString;
+    source: Type.TString;
+    content: Type.TString;
+    retrieverMix: Type.TArray<Type.TString>;
+    retrievalResultId: Type.TOptional<Type.TString>;
+    chunkIndex: Type.TOptional<Type.TNumber>;
+    lineNumber: Type.TOptional<Type.TNumber>;
+    parserType: Type.TOptional<Type.TString>;
+    documentType: Type.TOptional<Type.TString>;
+    documentArea: Type.TOptional<Type.TString>;
+    evidenceType: Type.TOptional<Type.TString>;
+    evidenceLocation: Type.TOptional<Type.TString>;
+}>;
+/** One recorded piece of evidence: what a retrieval tool returned, under the id it printed. */
+export type LedgerEntry = Static<typeof ledgerEntrySchema>;
+/**
+ * Entries a session persisted under {@link EVIDENCE_ENTRY_TYPE}. A malformed
+ * entry is skipped: its id then resolves to nothing and the citation is
+ * dropped with a diagnostic, never mis-resolved.
+ */
+export declare function parseLedgerEntries(data: unknown): LedgerEntry[];
+/** Highest evidence id ordinal among `entries` (`e12` is 12); 0 when there are none. */
+export declare function highestOrdinal(entries: readonly LedgerEntry[]): number;
 export interface LookupEvidenceOptions {
     /** False in remote sessions: only evidence a tool returned this run may be cited. */
     readonly allowLocalFiles: boolean;
 }
 export declare function isRegularFile(path: string): boolean;
 /**
- * Per-run record of every piece of evidence a retrieval tool returned.
+ * Record of every piece of evidence the retrieval tools returned in one
+ * conversation.
  *
  * Tools hand the model a short id (`e1`, `e2`, ...) next to each result. The
  * model cites evidence by id; the harness resolves the id back to the exact
  * source, method, content, and chunk metadata it recorded. The model never
  * retypes a path or a chunk, so it cannot misquote or invent one, and the
  * stable evidence id stored in memory is always derived from real evidence.
+ *
+ * Ids count up for as long as the process lives and are never reissued, so an
+ * id the model saw in any earlier turn keeps naming the chunk it named. The
+ * hosting session persists what {@link takeUnsaved} returns and rebuilds the
+ * ledger with {@link restore} when a conversation is resumed.
  */
 export declare class EvidenceLedger {
     private entries;
     private readonly byKey;
     private readonly byId;
+    /** Entries recorded or changed since the last {@link takeUnsaved}. */
+    private readonly unsaved;
     /**
-     * Ordinal of the last issued id. Survives {@link clear}: a hosted (TUI)
-     * conversation keeps earlier runs' tool output in context, so restarting at
-     * e1 would let a stale id silently resolve to a different chunk.
+     * Ordinal of the last issued id. Survives {@link clear} and {@link restore}:
+     * a conversation keeps earlier turns' tool output in context, so restarting
+     * at e1 would let a stale id silently resolve to a different chunk.
      */
     private lastOrdinal;
     /** Record evidence and return its id. The same chunk always gets the same id. */
@@ -49,6 +87,17 @@ export declare class EvidenceLedger {
      * citation instead of failing the run.
      */
     lookup(raw: string, options: LookupEvidenceOptions): AutoRAGEvidenceRef[];
-    /** Forget every recorded entry. Ids are never reissued, so a previous run's id stays unresolvable. */
+    /** Forget every recorded entry. Ids are never reissued, so a previous conversation's id stays unresolvable. */
     clear(): void;
+    /** Entries recorded or changed since the last call, for the hosting session to persist. */
+    takeUnsaved(): LedgerEntry[];
+    /**
+     * Rebuild the ledger from persisted entries, such as the ones on the resumed
+     * conversation branch; a later entry with the same id replaces an earlier
+     * one. Ids keep counting past every restored id and past `ordinalFloor`, the
+     * highest id issued anywhere in the conversation (other branches included),
+     * so a restored ledger never reissues an id a transcript may still show.
+     */
+    restore(entries: readonly LedgerEntry[], ordinalFloor?: number): void;
 }
+export {};

package/dist/agent/evidence-persistence.d.ts

@@ -0,0 +1,24 @@
+import type { ExtensionFactory, SessionEntry } from "@earendil-works/pi-coding-agent";
+import { type EvidenceLedger } from "./evidence-ledger.js";
+/** The part of pi's session manager the ledger is rebuilt from. */
+interface SessionEntries {
+    getBranch(): SessionEntry[];
+    getEntries(): SessionEntry[];
+}
+/**
+ * Rebuild `ledger` for the conversation branch `session` is on. Evidence from
+ * other branches stays out of reach (their transcripts are not in context), but
+ * its ids still count: new evidence never reuses an id issued anywhere in the
+ * conversation.
+ */
+export declare function restoreEvidenceLedger(ledger: EvidenceLedger, session: SessionEntries): void;
+/**
+ * pi extension that keeps the conversation's evidence ledger in the session
+ * file, next to the transcript that shows the model its `[eN]` ids. Evidence is
+ * appended as the tools return it; switching, resuming, forking, or navigating
+ * the session tree rebuilds the ledger from the branch now in view. An id the
+ * model saw therefore resolves, and is never reissued, whatever turns, tool
+ * calls, resumes, or restarts happened in between.
+ */
+export declare function createEvidencePersistence(ledger: EvidenceLedger): ExtensionFactory;
+export {};

package/dist/agent/jev-extension.d.ts

@@ -4,10 +4,10 @@ import { type JevBackend, type Judgment, type Question, type State } from "jev-u
 export declare const JEV_TOOL_NAME = "jev";
 /**
  * Jev backends the `jev-use` engine resolves from the environment, in
- * auto-select order: TypeSafe -> OpenRouter -> Vercel AI Gateway. `autorag` is
- * the hosted AutoRAG (Dazzi) plan, built here rather than resolved by jev-use.
+ * auto-select order: TypeSafe -> OpenRouter -> Vercel AI Gateway. `dazzi` is
+ * the hosted Dazzi Models plan, built here rather than resolved by jev-use.
  */
-export type JevBackendName = "typesafe" | "openrouter" | "vercel" | "autorag";
+export type JevBackendName = "typesafe" | "openrouter" | "vercel" | "dazzi";
 /**
  * OpenRouter's live Jev model id. `jev-use` 0.8.0 ships `typesafe/jev-latest`,
  * which OpenRouter rejects with HTTP 400 ("Model ... does not exist"), so the
@@ -40,6 +40,8 @@ export interface JevToolOptions {
 export interface JevJudgeOptions extends JevToolOptions {
     /** Environment consulted for the backend credential. Default `process.env`. */
     readonly env?: Record<string, string | undefined>;
+    /** Pi agent home whose `auth.json` holds the Dazzi `/login` credential. Default: Pi's agent dir. */
+    readonly agentDir?: string;
 }
 /**
  * One batched Jev judgment. Rejects only when no backend can be built (for

package/dist/agent/search-documents.d.ts

@@ -9,7 +9,7 @@ export type SearchDocumentDiagnosticSeverity = "info" | "warning" | "error";
  * (#21/#22) does not require another source-breaking type change; codes not yet
  * emitted are still valid members.
  */
-export type SearchDocumentDiagnosticCode = "empty-query" | "no-verified-results" | "unknown-warning" | "caller-tool-dropped" | "minsync-staging-excluded" | "minsync-sync-failed" | "embedder-unavailable" | "embedding-identity-mismatch" | "parser-skipped" | "parser-failed" | "source-vanished" | "source-unreadable" | "parser-warning" | "duplicate-excluded" | "unsupported-file" | "stale-index" | "deleted-mirror" | "pdf-extract-thin" | "retrieval-method-failed" | "rerank-failed" | "jikji-unavailable" | "jikji-prepare-failed" | "jikji-find-failed" | "everything-index-failed" | "fsearch-index-failed" | "fsearch-binary-missing" | "refresh-failed" | "refresh-interrupted" | "watch-failed" | "watch-limited" | "unknown-datasource-skill" | "no-final-answer" | "model-request-failed" | "search-timeout" | "query-routed" | "query-route-fallback" | "query-decomposition-failed" | "follow-up-skipped" | "follow-up-check-fallback" | "datasources-selected" | "datasource-selection-fallback" | "citation-without-result" | "evidence-judged" | "evidence-judgment-fallback" | "memory-search-fallback" | "self-config-unavailable" | "self-config-rolled-back";
+export type SearchDocumentDiagnosticCode = "empty-query" | "no-verified-results" | "unknown-warning" | "caller-tool-dropped" | "minsync-staging-excluded" | "minsync-file-skipped" | "minsync-sync-failed" | "minsync-lock-held" | "embedder-unavailable" | "embedding-identity-mismatch" | "parser-skipped" | "parser-failed" | "parser-crashed" | "source-vanished" | "source-unreadable" | "source-read-failed" | "mirror-write-failed" | "parser-warning" | "duplicate-excluded" | "unsupported-file" | "stale-index" | "deleted-mirror" | "search-root-unavailable" | "mass-delete-suppressed" | "walk-truncated" | "directory-unreadable" | "pdf-extract-thin" | "retrieval-method-failed" | "rerank-failed" | "jikji-unavailable" | "jikji-prepare-failed" | "jikji-find-failed" | "everything-index-failed" | "fsearch-index-failed" | "fsearch-binary-missing" | "refresh-failed" | "refresh-interrupted" | "watch-failed" | "watch-limited" | "unknown-datasource-skill" | "no-final-answer" | "model-request-failed" | "tool-call-burst" | "query-routed" | "query-route-fallback" | "query-decomposition-failed" | "follow-up-skipped" | "follow-up-check-fallback" | "datasources-selected" | "datasource-selection-fallback" | "citation-without-result" | "evidence-judged" | "evidence-judgment-fallback" | "memory-search-fallback" | "self-config-unavailable" | "self-config-rolled-back";
 export interface SearchDocumentDiagnostic {
     readonly code: SearchDocumentDiagnosticCode;
     readonly severity: SearchDocumentDiagnosticSeverity;

package/dist/agent/system-prompt.d.ts

@@ -1,10 +1,8 @@
-import type { StoreManifest } from "../manifest/types.js";
 import type { DatasourceAgentSkill } from "./datasource-skill.js";
 export interface SystemPromptConfig {
     toolNames: string[];
     modelId?: string;
     memoryEntries?: readonly unknown[];
-    manifests: StoreManifest[];
     jikjiIndexingEnabled?: boolean;
     datasourceSkills?: readonly DatasourceAgentSkill[];
     retrievedContentGuard?: boolean;