@autorag/librarian 2.9.0 → 2.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +57 -2
- package/dist/agent/agent.d.ts +27 -21
- package/dist/agent/answer-citations.d.ts +22 -0
- package/dist/agent/answer-guidelines.d.ts +9 -9
- package/dist/agent/citations.d.ts +35 -23
- package/dist/agent/evidence-ledger.d.ts +7 -12
- package/dist/agent/index.d.ts +2 -2
- package/dist/agent/{emit-results-tool.d.ts → results.d.ts} +5 -40
- package/dist/agent/search-documents.d.ts +14 -19
- package/dist/agent/self-config.d.ts +1 -1
- package/dist/cli/commands/report.d.ts +1 -1
- package/dist/cli/commands/tui.d.ts +3 -2
- package/dist/cli/index.js +1466 -984
- package/dist/cloud/authorize.d.ts +12 -0
- package/dist/cloud/callback-server.d.ts +33 -0
- package/dist/cloud/callback.d.ts +23 -0
- package/dist/cloud/config.d.ts +23 -0
- package/dist/cloud/errors.d.ts +46 -0
- package/dist/cloud/index.d.ts +12 -0
- package/dist/cloud/models.d.ts +32 -0
- package/dist/cloud/oauth-login.d.ts +22 -0
- package/dist/cloud/pkce.d.ts +20 -0
- package/dist/cloud/provider.d.ts +40 -0
- package/dist/cloud/token.d.ts +22 -0
- package/dist/core.d.ts +2 -5
- package/dist/core.js +1341 -918
- package/dist/index.d.ts +2 -1
- package/dist/index.js +1463 -928
- package/dist/mcp/index.js +1404 -925
- package/dist/minsync/client.d.ts +1 -1
- package/dist/minsync/installer.d.ts +1 -1
- package/dist/observability/run-log.d.ts +1 -1
- package/dist/retrieval/index.d.ts +1 -1
- package/dist/retrieval/types.d.ts +0 -23
- package/package.json +1 -1
- package/skills/autorag-lite-setup/SKILL.md +1 -1
- package/skills/autorag-setup/SKILL.md +2 -2
- package/dist/agent/fast-answer-tool.d.ts +0 -56
package/README.md
CHANGED
|
@@ -72,13 +72,13 @@ AutoRAG Agent orchestrates five integrated subsystems:
|
|
|
72
72
|
- **Datasource Skills:** Federated retrieval across every configured external datasource.
|
|
73
73
|
3. **Result Merger & Scope Narrowing:** Cross-method deduplication, score normalization, and ordinary query-scope narrowing.
|
|
74
74
|
4. **Direct Evidence Reading (`bash`):** The agent directly opens and inspects promising files with `cat`, `grep`, or `find` to verify facts against ground truth.
|
|
75
|
-
5. **Curation:**
|
|
75
|
+
5. **Curation:** The final answer is the model's plain reply; the harness resolves its inline `[eN]` citations into numbered, source-backed results; cited evidence that Jev judges to genuinely support the answer is remembered as reference context for later searches ([docs/retrieval-memory.md](docs/retrieval-memory.md)).
|
|
76
76
|
|
|
77
77
|
### Pi host boundary
|
|
78
78
|
|
|
79
79
|
AutoRAG uses `@earendil-works/pi-coding-agent` as the runtime host for model sessions. Pi owns provider credentials and OAuth storage, model-runtime dispatch, session JSONL persistence/resume, extension loading, lifecycle events, and the built-in `read`/`bash`/`edit`/`write`/`grep`/`find`/`ls` tools. AutoRAG registers only its domain tools and keeps orchestration outside the host: datasource orchestration, MinSync/Jikji preparation, memory hints, fast-to-verification two-phase search, structured result emission, and remote-session filtering.
|
|
80
80
|
|
|
81
|
-
The CLI TUI (`autorag tui`) is Pi's interactive mode hosted on the AutoRAG librarian: there is no separate AutoRAG renderer. Pi owns the terminal UI and its native in-session commands — `/login`/`/logout`, `/model`, `/resume`, `/new`, `/tree`, `/compact`, and `/settings` — so provider sign-in and model selection work on first launch even when no AutoRAG model is configured. AutoRAG registers its retrieval tools and
|
|
81
|
+
The CLI TUI (`autorag tui`) is Pi's interactive mode hosted on the AutoRAG librarian: there is no separate AutoRAG renderer. Pi owns the terminal UI and its native in-session commands — `/login`/`/logout`, `/model`, `/resume`, `/new`, `/tree`, `/compact`, and `/settings` — so provider sign-in and model selection work on first launch even when no AutoRAG model is configured. AutoRAG registers its retrieval tools and runs the two-phase search in the same Pi session, where the model's progress notes, first answer, and final answer are ordinary assistant messages, so resume and two-phase search keep working together.
|
|
82
82
|
|
|
83
83
|
Update notices are AutoRAG's, not Pi's: `autorag tui` suppresses Pi's "run `pi update`" banner (Pi is a bundled host, not a separate install) and instead checks the published `@autorag/librarian` release, showing a one-line notice in the session when a newer version exists. Set `AUTORAG_NO_UPDATE_CHECK=1` to skip the lookup; `AUTORAG_UPDATE_CHECK_URL` overrides the registry endpoint.
|
|
84
84
|
|
|
@@ -359,6 +359,60 @@ for (const result of response.results) {
|
|
|
359
359
|
|
|
360
360
|
---
|
|
361
361
|
|
|
362
|
+
## Hosted AutoRAG Plan (Dazzi)
|
|
363
|
+
|
|
364
|
+
AutoRAG can run on the hosted **AutoRAG** plan as a first-class model provider named
|
|
365
|
+
`autorag`. The provider speaks the server's OpenAI Responses API surface (`<root>/v1`),
|
|
366
|
+
so streaming, tool calls, and usage are handled by Pi's built-in `openai-responses`
|
|
367
|
+
implementation, and the model catalog is discovered live from the server — no model id
|
|
368
|
+
is baked into AutoRAG.
|
|
369
|
+
|
|
370
|
+
Sign in from the interactive TUI:
|
|
371
|
+
|
|
372
|
+
```text
|
|
373
|
+
autorag tui
|
|
374
|
+
/login # choose "AutoRAG"; the browser completes a PKCE login
|
|
375
|
+
```
|
|
376
|
+
|
|
377
|
+
Headless (no browser) with a long-lived API key:
|
|
378
|
+
|
|
379
|
+
```bash
|
|
380
|
+
export AUTORAG_API_KEY=dz_...
|
|
381
|
+
```
|
|
382
|
+
|
|
383
|
+
On a fresh agent home, pin the model with an explicit endpoint so it resolves
|
|
384
|
+
offline without a catalog snapshot — this is the headless configuration:
|
|
385
|
+
|
|
386
|
+
```json
|
|
387
|
+
{
|
|
388
|
+
"model": {
|
|
389
|
+
"provider": "autorag",
|
|
390
|
+
"id": "anthropic/claude-haiku-5.5",
|
|
391
|
+
"baseUrl": "https://api.dazziapp.com/v1",
|
|
392
|
+
"api": "openai-responses",
|
|
393
|
+
"apiKeyEnv": "AUTORAG_API_KEY"
|
|
394
|
+
}
|
|
395
|
+
}
|
|
396
|
+
```
|
|
397
|
+
|
|
398
|
+
The `provider`/`id`-only shorthand (`{ "provider": "autorag", "id": "anthropic/claude-haiku-5.5" }`)
|
|
399
|
+
only resolves once a TUI `/login` or a model refresh has persisted the catalog
|
|
400
|
+
snapshot; with no snapshot there is nothing to restore offline, and resolution
|
|
401
|
+
fails with `Unknown configured model: autorag/<id>`.
|
|
402
|
+
|
|
403
|
+
| Variable | Default | Meaning |
|
|
404
|
+
|---|---|---|
|
|
405
|
+
| `AUTORAG_BASE_URL` | `https://api.dazziapp.com` | Server **root** (not the `/v1` path); the API base is `<root>/v1`. |
|
|
406
|
+
| `AUTORAG_API_KEY` | – | Headless API key. A stored `/login` credential takes precedence. |
|
|
407
|
+
|
|
408
|
+
The catalog is persisted after a `/login` (or any model refresh), so the
|
|
409
|
+
`provider`/`id` shorthand `model: { provider: "autorag", id }`
|
|
410
|
+
keeps resolving offline. When you are not signed in the provider still registers — it appears under
|
|
411
|
+
`/login`, contributes no models, and makes no network call. See
|
|
412
|
+
[Hosted AutoRAG provider](docs/cloud-provider.md) for the full flow.
|
|
413
|
+
|
|
414
|
+
---
|
|
415
|
+
|
|
362
416
|
## CLI Command Reference
|
|
363
417
|
|
|
364
418
|
| Command | Description |
|
|
@@ -386,6 +440,7 @@ Deep dive into AutoRAG Agent's architecture, security, and integration guides:
|
|
|
386
440
|
|
|
387
441
|
- **[MinSync Setup & Embedding QA](docs/minsync-setup.md):** Automatic binary installation, CDC chunking, and EmbeddingGemma verification.
|
|
388
442
|
- **[Local Embedding Runtime & Gateway](docs/embedding-runtime.md):** AutoRAG-owned local gateway, model prefetching, and zero-egress semantic search. Model cards: [`qwen3-embedding-0.6b`](docs/model-cards/qwen3-embedding-0.6b.md), [`embeddinggemma-300m`](docs/model-cards/embeddinggemma-300m.md).
|
|
443
|
+
- **[Hosted AutoRAG Provider](docs/cloud-provider.md):** The `autorag` (Dazzi) model provider — PKCE `/login`, `AUTORAG_API_KEY`, `AUTORAG_BASE_URL`, live model discovery, and offline catalog snapshots.
|
|
389
444
|
- **[Datasource Skills Reference](docs/datasource-skills.md):** Full configuration contracts, connection aliases, and connector options.
|
|
390
445
|
- **[Jev Decisions](docs/jev-decisions.md):** Jev query pipeline, **on by default** via OpenRouter: it routes each question to a direct answer (intrinsic knowledge) or local search (everything needing more information; web search is left to the agent after the fast answer), splits multi-part questions into up to five parallel searches (`openrouter/qwen/qwen3.7-flash`), picks which registered datasources to search before the fast answer, and ends the run after the fast answer when that answer is complete. Also covers the `jev` judgment tool, backends (OpenRouter, TypeSafe, Vercel AI Gateway), and how to opt out.
|
|
391
446
|
- **[Manual QA & Datasource Test Harnesses](docs/manual-qa-datasources.md):** Real-world testing guides for Discord, KakaoTalk, Slack, Notion, and email.
|
package/dist/agent/agent.d.ts
CHANGED
|
@@ -17,10 +17,9 @@ import { type DefaultParserRegistryOptions } from "../parser/index.js";
|
|
|
17
17
|
import { RetrievalEngine } from "../retrieval/engine.js";
|
|
18
18
|
import { RetrievalMethodRegistry } from "../retrieval/registry.js";
|
|
19
19
|
import type { DatasourceCatalogEntry } from "../retrieval/selection.js";
|
|
20
|
-
import type {
|
|
20
|
+
import type { RetrievalDiagnostic, RetrievalOptions, RetrievalResult } from "../retrieval/types.js";
|
|
21
21
|
import { type DatasourceAgentSkill } from "./datasource-skill.js";
|
|
22
22
|
import { type ScanDuplicateDocumentsDetails } from "./dupey-tool.js";
|
|
23
|
-
import { type AutoRAGFastAnswerDetails } from "./fast-answer-tool.js";
|
|
24
23
|
import { type JevToolOptions } from "./jev-extension.js";
|
|
25
24
|
import { type JikjiFindProviderResult } from "./jikji-find-tool.js";
|
|
26
25
|
import { type AutoRAGPiInteractiveRuntime, type AutoRAGPiSession } from "./pi-session.js";
|
|
@@ -255,10 +254,10 @@ export interface AutoRAGAgentOptions {
|
|
|
255
254
|
memoryEmbedder?: Embedder | false;
|
|
256
255
|
/**
|
|
257
256
|
* Agent self-configuration. When set (and Jev is on), Jev gains a `config`
|
|
258
|
-
* branch for questions about AutoRAG's own settings: the turn skips
|
|
259
|
-
*
|
|
260
|
-
* `configPath` with the pi tools, and reports
|
|
261
|
-
*
|
|
257
|
+
* branch for questions about AutoRAG's own settings: the turn skips the
|
|
258
|
+
* fast answer, receives the full `autorag-setup` skill, edits
|
|
259
|
+
* `configPath` with the pi tools, and reports what it changed as its final
|
|
260
|
+
* message. Always omitted for remote P2P sessions.
|
|
262
261
|
*/
|
|
263
262
|
selfConfig?: SelfConfigOptions;
|
|
264
263
|
/**
|
|
@@ -356,13 +355,10 @@ export declare class AutoRAGAgent {
|
|
|
356
355
|
private readonly runLogger;
|
|
357
356
|
private lastQuery;
|
|
358
357
|
private lastSessionId;
|
|
359
|
-
private readonly sessions;
|
|
360
358
|
private activeRun;
|
|
361
|
-
private resultCapture;
|
|
362
|
-
private interactiveFastAnswerCallback;
|
|
363
359
|
private modelNativeSearchAuth;
|
|
364
360
|
private retrievalTrace;
|
|
365
|
-
/** Evidence every retrieval tool returned this run;
|
|
361
|
+
/** Evidence every retrieval tool returned this run; the answer's inline `[eN]` citations resolve against it. */
|
|
366
362
|
private readonly evidenceLedger;
|
|
367
363
|
private preliminaryCallback;
|
|
368
364
|
/** Per-phase thinking levels of the two-phase search. */
|
|
@@ -474,6 +470,15 @@ export declare class AutoRAGAgent {
|
|
|
474
470
|
private currentSystemPromptConfig;
|
|
475
471
|
subscribe(listener: Parameters<Agent["subscribe"]>[0]): () => void;
|
|
476
472
|
createPiInteractiveRuntime(): Promise<AutoRAGPiInteractiveRuntime>;
|
|
473
|
+
/** Resolve the model's inline `[eN]` citations against this run's evidence and derive the structured results. */
|
|
474
|
+
private deriveAnswer;
|
|
475
|
+
/**
|
|
476
|
+
* Drives one TUI query through the two-phase search. The model's own
|
|
477
|
+
* assistant messages (progress notes before a tool call, the fast answer,
|
|
478
|
+
* the final answer) are already recorded and rendered by the bound pi
|
|
479
|
+
* session, so nothing is re-sent here: a second copy would show every answer
|
|
480
|
+
* twice and put it in the model's context twice.
|
|
481
|
+
*/
|
|
477
482
|
private runInteractivePiQuery;
|
|
478
483
|
abort(): void;
|
|
479
484
|
/**
|
|
@@ -486,13 +491,14 @@ export declare class AutoRAGAgent {
|
|
|
486
491
|
}): void;
|
|
487
492
|
stopAutoRefresh(): void;
|
|
488
493
|
private runAutoRefreshTick;
|
|
489
|
-
getResultRegistry(sessionId?: string): ReadonlyMap<number, CuratedResult>;
|
|
490
494
|
searchDocuments(query: string, options?: RetrievalOptions): Promise<SearchDocumentsResponse>;
|
|
491
495
|
/**
|
|
492
496
|
* Stream bounded progress updates while retaining the stable
|
|
493
|
-
* {@link searchDocuments} promise API. Progress is
|
|
494
|
-
*
|
|
495
|
-
*
|
|
497
|
+
* {@link searchDocuments} promise API. Progress is the text an assistant
|
|
498
|
+
* message wrote before calling a tool (`stopReason: "toolUse"`); a message
|
|
499
|
+
* that ends its turn is an answer, delivered as `preliminary` or
|
|
500
|
+
* `complete`, never as progress. Callers can render it in a TUI, CLI, or
|
|
501
|
+
* parent agent and can stop by calling {@link abort}.
|
|
496
502
|
*/
|
|
497
503
|
searchDocumentsStream(query: string, options?: RetrievalOptions): AsyncGenerator<SearchDocumentsStreamEvent, void, void>;
|
|
498
504
|
/**
|
|
@@ -529,7 +535,7 @@ export declare class AutoRAGAgent {
|
|
|
529
535
|
/** Path-opaque component diagnostics for the search response. */
|
|
530
536
|
private collectComponentDiagnostics;
|
|
531
537
|
/**
|
|
532
|
-
* Jev check run after
|
|
538
|
+
* Jev check run after the fast answer: does the answer need correction,
|
|
533
539
|
* clarification, or further research? "No" ends the run with the fast
|
|
534
540
|
* answer as the final answer. Without Jev, or when the check fails, the run
|
|
535
541
|
* always continues into verification.
|
|
@@ -574,14 +580,14 @@ export declare class AutoRAGAgent {
|
|
|
574
580
|
private rerankPrefetchPool;
|
|
575
581
|
/**
|
|
576
582
|
* Fast-phase prompt for two-phase searches. The baseline retrieval context
|
|
577
|
-
* is already gathered, so the model answers immediately with thinking off
|
|
578
|
-
*
|
|
583
|
+
* is already gathered, so the model answers immediately with thinking off,
|
|
584
|
+
* as plain text, without any further tool calls.
|
|
579
585
|
*/
|
|
580
586
|
buildFastAnswerPrompt(query: string, options: RetrievalOptions, baseline: string): string;
|
|
581
587
|
/**
|
|
582
588
|
* Prompt for a question Jev routed to a direct answer: general knowledge or
|
|
583
589
|
* small talk. No retrieval ran and no verification phase follows, so the
|
|
584
|
-
*
|
|
590
|
+
* reply is the final answer.
|
|
585
591
|
*/
|
|
586
592
|
buildDirectAnswerPrompt(query: string): string;
|
|
587
593
|
/** Discovery-tool hint naming only the registered discovery tools; empty when none. */
|
|
@@ -595,10 +601,10 @@ export declare class AutoRAGAgent {
|
|
|
595
601
|
/**
|
|
596
602
|
* Verification-phase prompt for two-phase searches. The fast answer, when a
|
|
597
603
|
* consumer already received it, is embedded verbatim so the model can diff
|
|
598
|
-
* against it; the model then verifies with thinking on and
|
|
599
|
-
*
|
|
604
|
+
* against it; the model then verifies with thinking on and ends with the
|
|
605
|
+
* final answer as a plain reply, returning only the delta.
|
|
600
606
|
*/
|
|
601
|
-
buildRefinementPrompt(query: string, options: RetrievalOptions, fastAnswer:
|
|
607
|
+
buildRefinementPrompt(query: string, options: RetrievalOptions, fastAnswer: string | undefined, fastAnswerDelivered: boolean): string;
|
|
602
608
|
buildSearchPrompt(query: string, options: RetrievalOptions, initialRetrievalContext?: string): string;
|
|
603
609
|
refresh(force?: boolean, opts?: AutoRAGRefreshOptions): Promise<AutoRAGRefreshResult>;
|
|
604
610
|
/**
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
import { type EvidenceLedger } from "./evidence-ledger.js";
|
|
2
|
+
import type { AutoRAGResultsDetails } from "./results.js";
|
|
3
|
+
import type { SearchDocumentDiagnostic } from "./search-documents.js";
|
|
4
|
+
export interface DerivedAnswer {
|
|
5
|
+
readonly details: AutoRAGResultsDetails;
|
|
6
|
+
readonly diagnostics: readonly SearchDocumentDiagnostic[];
|
|
7
|
+
}
|
|
8
|
+
/**
|
|
9
|
+
* Turn the model's plain answer into the structured results the rest of the
|
|
10
|
+
* harness consumes. The model cites with the evidence ids the retrieval tools
|
|
11
|
+
* printed (`[e3]`, `[e3, e7]`) or `[file:<abs path>]` for a file it opened
|
|
12
|
+
* itself. Each id is resolved against the run's evidence ledger, so a source,
|
|
13
|
+
* method, and chunk always come from recorded evidence, never from model text.
|
|
14
|
+
*
|
|
15
|
+
* Results are one per cited source, numbered by first appearance, and every
|
|
16
|
+
* marker in the returned answer is rewritten to its result number (`[1]`).
|
|
17
|
+
* A marker that matches no evidence, and any numeric marker the model wrote on
|
|
18
|
+
* its own, is dropped and reported: it can never point at the wrong result.
|
|
19
|
+
*/
|
|
20
|
+
export declare function deriveResultsFromAnswer(text: string, ledger: EvidenceLedger, options: {
|
|
21
|
+
readonly allowLocalFiles: boolean;
|
|
22
|
+
}): DerivedAnswer;
|
|
@@ -1,19 +1,19 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* The single file-path exception to the answer "no file paths" rule
|
|
3
|
-
* (issue #1790). Every prompt
|
|
4
|
-
*
|
|
5
|
-
*
|
|
6
|
-
* otherwise still stands.
|
|
3
|
+
* (issue #1790). Every prompt that shapes the answer uses this exact wording so
|
|
4
|
+
* a caller that can render images receives a markdown image embed whose path is
|
|
5
|
+
* a real evidence source, while the no-path rule otherwise still stands.
|
|
7
6
|
*/
|
|
8
|
-
export declare const ANSWER_IMAGE_EMBED_RULE = "**Images**: when a retrieved result is itself an image file (png, jpg, jpeg, gif, webp, svg, bmp, avif, heic, tiff) and showing it answers the question, embed it in
|
|
7
|
+
export declare const ANSWER_IMAGE_EMBED_RULE = "**Images**: when a retrieved result is itself an image file (png, jpg, jpeg, gif, webp, svg, bmp, avif, heic, tiff) and showing it answers the question, embed it in the answer as a markdown image on its own line: ``, followed by its citation, e.g. ` [e2]`. Use only the real source path of retrieved evidence. Never invent a path, use relative paths, or embed remote URLs. Apart from these image embeds, the no-file-paths rule stands.";
|
|
9
8
|
/**
|
|
10
9
|
* Delta (verification) answers must not re-embed an image the first answer
|
|
11
10
|
* already showed.
|
|
12
11
|
*/
|
|
13
12
|
export declare const ANSWER_IMAGE_DELTA_RULE = "An image already embedded in the first answer must not be embedded again.";
|
|
14
13
|
/**
|
|
15
|
-
*
|
|
16
|
-
*
|
|
17
|
-
*
|
|
14
|
+
* Inline evidence citations. The model writes the id the retrieval tools print
|
|
15
|
+
* next to each result; the harness resolves it to the recorded source and
|
|
16
|
+
* renumbers the markers `[1]`, `[2]`, ... in order of first appearance, so the
|
|
17
|
+
* caller only ever sees numbered citations backed by recorded evidence.
|
|
18
18
|
*/
|
|
19
|
-
export declare const ANSWER_CITATION_RULE = "**
|
|
19
|
+
export declare const ANSWER_CITATION_RULE = "**Citations**: cite supporting evidence inline with the evidence id shown next to each retrieved result, in square brackets (e.g. [e3], or [e3][e7] for two). Copy ids exactly; never invent one. For a local file you opened yourself with `bash`, cite `[file:<absolute path>]`. Cite the evidence that backs a sentence at the end of that sentence. Do not cite numbers, paths, or anything else.";
|
|
@@ -1,7 +1,39 @@
|
|
|
1
|
-
/**
|
|
2
|
-
|
|
1
|
+
/**
|
|
2
|
+
* Bracketed citations in an answer. Two grammars share one scanner:
|
|
3
|
+
*
|
|
4
|
+
* - **Final form** `[n]`: a number that must resolve to a `results[].number`
|
|
5
|
+
* of the same response (issue #1788).
|
|
6
|
+
* - **Model form** `[e3]`, `[e3, e7]`, `[file:/abs/path]`: the evidence ids the
|
|
7
|
+
* retrieval tools print next to each result, or a local file the model opened
|
|
8
|
+
* itself. The harness resolves them against the run's evidence ledger and
|
|
9
|
+
* rewrites them into the final form.
|
|
10
|
+
*
|
|
11
|
+
* A markdown image target `(<...>)` is skipped verbatim so a bracketed token
|
|
12
|
+
* inside a real file path is never mistaken for a citation; `[n](...)` is a
|
|
13
|
+
* markdown link, not a citation.
|
|
14
|
+
*
|
|
15
|
+
* `answer` is model- or caller-supplied, so this is a single linear scan rather
|
|
16
|
+
* than a regex: alternations like `\(<[^>]*>\)` / `[ \t]*\[` backtrack
|
|
17
|
+
* quadratically on inputs such as repeated `(<` or long whitespace runs.
|
|
18
|
+
*/
|
|
19
|
+
export type AnswerMarker = {
|
|
20
|
+
readonly kind: "number";
|
|
21
|
+
readonly start: number;
|
|
22
|
+
readonly end: number;
|
|
3
23
|
readonly number: number;
|
|
4
|
-
}
|
|
24
|
+
} | {
|
|
25
|
+
readonly kind: "evidence";
|
|
26
|
+
readonly start: number;
|
|
27
|
+
readonly end: number;
|
|
28
|
+
readonly ids: readonly string[];
|
|
29
|
+
} | {
|
|
30
|
+
readonly kind: "file";
|
|
31
|
+
readonly start: number;
|
|
32
|
+
readonly end: number;
|
|
33
|
+
readonly path: string;
|
|
34
|
+
};
|
|
35
|
+
/** Every citation marker in `answer`, in order. `isFile` decides which `[file:...]` paths are real. */
|
|
36
|
+
export declare function answerMarkers(answer: string, isFile?: (path: string) => boolean): AnswerMarker[];
|
|
5
37
|
/** Remove citation markers whose number has no matching result. */
|
|
6
38
|
export declare function stripUnresolvedCitations(answer: string, results: readonly {
|
|
7
39
|
readonly number: number;
|
|
@@ -10,34 +42,14 @@ export declare function stripUnresolvedCitations(answer: string, results: readon
|
|
|
10
42
|
readonly unresolved: readonly number[];
|
|
11
43
|
};
|
|
12
44
|
export declare function formatCitationList(numbers: readonly number[]): string;
|
|
13
|
-
/**
|
|
14
|
-
* Throw a corrective error when two results share a number. A repeated number
|
|
15
|
-
* collapses into one registry entry and one feedback id, so the other result's
|
|
16
|
-
* evidence would be lost. Emit tools call this at tool time so the model sees
|
|
17
|
-
* the error and re-emits.
|
|
18
|
-
*/
|
|
19
|
-
export declare function assertUniqueResultNumbers(label: string, results: readonly {
|
|
20
|
-
readonly number: number;
|
|
21
|
-
}[]): void;
|
|
22
45
|
/**
|
|
23
46
|
* Throw a corrective error unless `results` and `mapping` carry the same
|
|
24
47
|
* numbers, exactly one entry each. Duplicates are rejected even when both
|
|
25
48
|
* sides repeat them: a repeated number collapses into a single registry entry,
|
|
26
49
|
* so the other entry's evidence would be lost.
|
|
27
|
-
* Emit tools call this at tool time (issue #1807) so the model sees the error
|
|
28
|
-
* and re-emits; checking only after the run ended failed the whole search and
|
|
29
|
-
* discarded an already-delivered answer.
|
|
30
50
|
*/
|
|
31
51
|
export declare function assertResultsMappingOneToOne(label: string, results: readonly {
|
|
32
52
|
readonly number: number;
|
|
33
53
|
}[], mapping: readonly {
|
|
34
54
|
readonly number: number;
|
|
35
55
|
}[]): void;
|
|
36
|
-
/**
|
|
37
|
-
* Throw a corrective error when `answer` cites numbers absent from `results`.
|
|
38
|
-
* Emit tools surface the message to the model as a tool error so it re-emits
|
|
39
|
-
* with one consistent numbering.
|
|
40
|
-
*/
|
|
41
|
-
export declare function assertCitationsResolve(label: string, answer: string, results: readonly {
|
|
42
|
-
readonly number: number;
|
|
43
|
-
}[]): void;
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import type { RetrievalResult } from "../retrieval/types.js";
|
|
2
|
-
import type { AutoRAGEvidenceRef } from "./
|
|
2
|
+
import type { AutoRAGEvidenceRef } from "./results.js";
|
|
3
3
|
export interface LedgerEvidenceInput {
|
|
4
4
|
readonly method: string;
|
|
5
5
|
readonly source: string;
|
|
@@ -13,16 +13,11 @@ export interface LedgerEvidenceInput {
|
|
|
13
13
|
readonly evidenceType?: string;
|
|
14
14
|
readonly evidenceLocation?: string;
|
|
15
15
|
}
|
|
16
|
-
export interface
|
|
17
|
-
/** Tool name used to prefix corrective errors. */
|
|
18
|
-
readonly label: string;
|
|
19
|
-
/** Result number the refs belong to; named in corrective errors. */
|
|
20
|
-
readonly number: number;
|
|
21
|
-
/** Excerpt recorded for a local file the model opened itself (no search returned it). */
|
|
22
|
-
readonly fallbackContent: string;
|
|
16
|
+
export interface LookupEvidenceOptions {
|
|
23
17
|
/** False in remote sessions: only evidence a tool returned this run may be cited. */
|
|
24
18
|
readonly allowLocalFiles: boolean;
|
|
25
19
|
}
|
|
20
|
+
export declare function isRegularFile(path: string): boolean;
|
|
26
21
|
/**
|
|
27
22
|
* Per-run record of every piece of evidence a retrieval tool returned.
|
|
28
23
|
*
|
|
@@ -47,13 +42,13 @@ export declare class EvidenceLedger {
|
|
|
47
42
|
/** Record one retrieval result as returned by `tool`; backend metadata is carried over. */
|
|
48
43
|
registerResult(tool: string, result: RetrievalResult): string;
|
|
49
44
|
/**
|
|
50
|
-
* Resolve
|
|
45
|
+
* Resolve one cited reference to recorded evidence. A ref is an evidence id
|
|
51
46
|
* (`e3` or `[e3]`), the exact source a tool returned (a URL, a datasource
|
|
52
47
|
* id, a path), or the absolute path of a real local file the model opened
|
|
53
|
-
* itself.
|
|
48
|
+
* itself. Returns an empty list when nothing matches: the caller drops the
|
|
49
|
+
* citation instead of failing the run.
|
|
54
50
|
*/
|
|
55
|
-
|
|
51
|
+
lookup(raw: string, options: LookupEvidenceOptions): AutoRAGEvidenceRef[];
|
|
56
52
|
/** Forget every recorded entry. Ids are never reissued, so a previous run's id stays unresolvable. */
|
|
57
53
|
clear(): void;
|
|
58
|
-
private unresolvedMessage;
|
|
59
54
|
}
|
package/dist/agent/index.d.ts
CHANGED
|
@@ -2,11 +2,11 @@ export type { AutoRAGAgentOptions, AutoRefreshOptions, PeerQueryOptions } from "
|
|
|
2
2
|
export { AutoRAGAgent } from "./agent.js";
|
|
3
3
|
export { buildDatasourceSkillsPrompt, createLoadDatasourceSkillTool, type DatasourceSkillProvider, datasourceSkillLocation, formatDatasourceSkillInvocation, LOAD_DATASOURCE_SKILL_TOOL_NAME, type LoadDatasourceSkillDetails, toDatasourceAgentSkill, } from "./datasource-skill.js";
|
|
4
4
|
export { createScanDuplicateDocumentsTool, SCAN_DUPLICATE_DOCUMENTS_TOOL_NAME, type ScanDuplicateDocumentsDetails, type ScanDuplicateDocumentsProvider, } from "./dupey-tool.js";
|
|
5
|
-
export type { AutoRAGEmittedResult, AutoRAGMappingEntry, AutoRAGResultsDetails, } from "./emit-results-tool.js";
|
|
6
|
-
export { createEmitResultsTool, EMIT_AUTORAG_RESULTS_TOOL_NAME, emitResultsSchema, reportSchema, } from "./emit-results-tool.js";
|
|
7
5
|
export { createJevExtension, JEV_TOOL_NAME, type JevBackendName, type JevToolOptions, } from "./jev-extension.js";
|
|
8
6
|
export { createJikjiFindTool, JIKJI_FIND_TOOL_NAME, type JikjiFindDetails, type JikjiFindPerRootPolicy, type JikjiFindProvider, type JikjiFindProviderResult, type MergedJikjiPolicy, } from "./jikji-find-tool.js";
|
|
9
7
|
export { createQueryPeerAgentTool, QUERY_PEER_AGENT_TOOL_NAME, type QueryPeerAgentDetails, type QueryPeerAgentResult, type QueryPeerAgentToolOptions, } from "./query-peer-tool.js";
|
|
8
|
+
export type { AutoRAGEmittedResult, AutoRAGMappingEntry, AutoRAGResultsDetails, } from "./results.js";
|
|
9
|
+
export { reportSchema } from "./results.js";
|
|
10
10
|
export { createSearchAllDocumentsTool, SEARCH_ALL_DOCUMENTS_TOOL_NAME, type SearchAllDocumentsDetails, type SearchAllDocumentsProvider, type SearchAllDocumentsResult, } from "./search-all-tool.js";
|
|
11
11
|
export type { SearchDocumentDiagnostic, SearchDocumentDiagnosticCode, SearchDocumentDiagnosticSeverity, SearchDocumentEvidence, SearchDocumentResult, SearchDocumentsResponse, SearchDocumentsStreamEvent, SearchDocumentWarning, } from "./search-documents.js";
|
|
12
12
|
export { createSearchMinSyncDocumentsTool, SEARCH_MINSYNC_DOCUMENTS_TOOL_NAME, type SearchMinSyncDocumentsDetails, } from "./search-minsync-tool.js";
|
|
@@ -1,33 +1,11 @@
|
|
|
1
|
-
import type { AgentTool } from "@earendil-works/pi-agent-core";
|
|
2
1
|
import { Type } from "typebox";
|
|
3
|
-
import type { EvidenceLedger } from "./evidence-ledger.js";
|
|
4
|
-
export declare const EMIT_AUTORAG_RESULTS_TOOL_NAME = "emit_autorag_results";
|
|
5
|
-
/**
|
|
6
|
-
* Model-facing schema of `emit_autorag_results`. The model cites evidence by
|
|
7
|
-
* the short ids retrieval tools print next to each result (`refs`); the
|
|
8
|
-
* harness resolves them to the recorded source, method, and chunk, so the
|
|
9
|
-
* model never retypes a path or a chunk.
|
|
10
|
-
*/
|
|
11
|
-
export declare const emitResultsSchema: Type.TObject<{
|
|
12
|
-
answer: Type.TString;
|
|
13
|
-
results: Type.TArray<Type.TObject<{
|
|
14
|
-
number: Type.TInteger;
|
|
15
|
-
title: Type.TString;
|
|
16
|
-
summary: Type.TString;
|
|
17
|
-
evidence: Type.TArray<Type.TObject<{
|
|
18
|
-
excerpt: Type.TString;
|
|
19
|
-
lineNumber: Type.TOptional<Type.TInteger>;
|
|
20
|
-
}>>;
|
|
21
|
-
confidence: Type.TNumber;
|
|
22
|
-
refs: Type.TArray<Type.TString>;
|
|
23
|
-
}>>;
|
|
24
|
-
warnings: Type.TOptional<Type.TArray<Type.TString>>;
|
|
25
|
-
}>;
|
|
26
2
|
/**
|
|
27
3
|
* Report schema with the explicit number -> source mapping. This is the
|
|
28
4
|
* persisted/typed shape and the input contract of `autorag report` and the MCP
|
|
29
5
|
* report tool, where an external agent curated the evidence and no harness
|
|
30
|
-
* ledger exists.
|
|
6
|
+
* ledger exists. The librarian itself never produces this shape from the
|
|
7
|
+
* model: it writes a plain answer and the harness derives the results from the
|
|
8
|
+
* evidence ids the answer cites.
|
|
31
9
|
*/
|
|
32
10
|
export declare const reportSchema: Type.TObject<{
|
|
33
11
|
answer: Type.TString;
|
|
@@ -75,7 +53,8 @@ export interface AutoRAGEmittedResult {
|
|
|
75
53
|
readonly title: string;
|
|
76
54
|
readonly summary: string;
|
|
77
55
|
readonly evidence: readonly AutoRAGEmittedEvidence[];
|
|
78
|
-
|
|
56
|
+
/** Absent for results the harness derived from cited evidence: nothing measured a confidence. */
|
|
57
|
+
readonly confidence?: number;
|
|
79
58
|
}
|
|
80
59
|
export interface AutoRAGEvidenceRef {
|
|
81
60
|
readonly method: string;
|
|
@@ -107,17 +86,3 @@ export interface AutoRAGResultsDetails {
|
|
|
107
86
|
readonly mapping: readonly AutoRAGMappingEntry[];
|
|
108
87
|
readonly warnings: readonly string[];
|
|
109
88
|
}
|
|
110
|
-
export interface EmitResultsToolOptions {
|
|
111
|
-
/** Evidence the run's retrieval tools returned; `refs` resolve against it. */
|
|
112
|
-
readonly ledger: EvidenceLedger;
|
|
113
|
-
/** False in remote sessions: only evidence a tool returned this run may be cited. */
|
|
114
|
-
readonly allowLocalFiles: boolean;
|
|
115
|
-
}
|
|
116
|
-
/**
|
|
117
|
-
* Builds the terminating structured-result tool. The model calls this exactly
|
|
118
|
-
* once as its final action; the typed `details` plus `terminate: true` end the
|
|
119
|
-
* Pi Agent run and hand the curated results back through `capture` — no
|
|
120
|
-
* assistant-text parsing involved. The number -> source mapping is built here
|
|
121
|
-
* from the harness-held evidence the model cited by id, never from model text.
|
|
122
|
-
*/
|
|
123
|
-
export declare function createEmitResultsTool(capture: (details: AutoRAGResultsDetails) => void, options: EmitResultsToolOptions): AgentTool<typeof emitResultsSchema, AutoRAGResultsDetails>;
|
|
@@ -1,7 +1,6 @@
|
|
|
1
|
-
import { type RetrievalMemory } from "../memory/memory.js";
|
|
2
|
-
import type {
|
|
3
|
-
import type { AutoRAGResultsDetails } from "./
|
|
4
|
-
import type { AutoRAGFastAnswerDetails } from "./fast-answer-tool.js";
|
|
1
|
+
import { type RetrievalMemory, type SessionEvidenceRef } from "../memory/memory.js";
|
|
2
|
+
import type { RetrievalResult } from "../retrieval/types.js";
|
|
3
|
+
import type { AutoRAGMappingEntry, AutoRAGResultsDetails } from "./results.js";
|
|
5
4
|
export type SearchDocumentWarning = "empty-query";
|
|
6
5
|
export type SearchDocumentDiagnosticSeverity = "info" | "warning" | "error";
|
|
7
6
|
/**
|
|
@@ -10,7 +9,7 @@ export type SearchDocumentDiagnosticSeverity = "info" | "warning" | "error";
|
|
|
10
9
|
* (#21/#22) does not require another source-breaking type change; codes not yet
|
|
11
10
|
* emitted are still valid members.
|
|
12
11
|
*/
|
|
13
|
-
export type SearchDocumentDiagnosticCode = "empty-query" | "no-verified-results" | "unknown-warning" | "caller-tool-dropped" | "minsync-staging-excluded" | "minsync-sync-failed" | "embedder-unavailable" | "embedding-identity-mismatch" | "parser-skipped" | "parser-failed" | "source-vanished" | "source-unreadable" | "parser-warning" | "duplicate-excluded" | "unsupported-file" | "stale-index" | "deleted-mirror" | "pdf-extract-thin" | "retrieval-method-failed" | "rerank-failed" | "jikji-unavailable" | "jikji-prepare-failed" | "jikji-find-failed" | "everything-index-failed" | "fsearch-index-failed" | "fsearch-binary-missing" | "refresh-failed" | "refresh-interrupted" | "watch-failed" | "watch-limited" | "unknown-datasource-skill" | "
|
|
12
|
+
export type SearchDocumentDiagnosticCode = "empty-query" | "no-verified-results" | "unknown-warning" | "caller-tool-dropped" | "minsync-staging-excluded" | "minsync-sync-failed" | "embedder-unavailable" | "embedding-identity-mismatch" | "parser-skipped" | "parser-failed" | "source-vanished" | "source-unreadable" | "parser-warning" | "duplicate-excluded" | "unsupported-file" | "stale-index" | "deleted-mirror" | "pdf-extract-thin" | "retrieval-method-failed" | "rerank-failed" | "jikji-unavailable" | "jikji-prepare-failed" | "jikji-find-failed" | "everything-index-failed" | "fsearch-index-failed" | "fsearch-binary-missing" | "refresh-failed" | "refresh-interrupted" | "watch-failed" | "watch-limited" | "unknown-datasource-skill" | "no-final-answer" | "model-request-failed" | "search-timeout" | "query-routed" | "query-route-fallback" | "query-decomposition-failed" | "follow-up-skipped" | "follow-up-check-fallback" | "datasources-selected" | "datasource-selection-fallback" | "citation-without-result" | "evidence-judged" | "evidence-judgment-fallback" | "memory-search-fallback" | "self-config-unavailable" | "self-config-rolled-back";
|
|
14
13
|
export interface SearchDocumentDiagnostic {
|
|
15
14
|
readonly code: SearchDocumentDiagnosticCode;
|
|
16
15
|
readonly severity: SearchDocumentDiagnosticSeverity;
|
|
@@ -27,7 +26,8 @@ export interface SearchDocumentResult {
|
|
|
27
26
|
readonly title: string;
|
|
28
27
|
readonly summary: string;
|
|
29
28
|
readonly evidence: readonly SearchDocumentEvidence[];
|
|
30
|
-
|
|
29
|
+
/** Absent when nothing measured one: derived results carry no model-reported confidence. */
|
|
30
|
+
readonly confidence?: number;
|
|
31
31
|
readonly source?: string;
|
|
32
32
|
}
|
|
33
33
|
/** One retrieval candidate captured in a run's retrieval trace. */
|
|
@@ -38,7 +38,7 @@ export interface SearchDocumentRetrievalTraceResult {
|
|
|
38
38
|
}
|
|
39
39
|
/**
|
|
40
40
|
* What one retrieval tool execution found during a search run. Attached to the
|
|
41
|
-
* degraded fallback response when the agent never
|
|
41
|
+
* degraded fallback response when the agent never wrote a final answer,
|
|
42
42
|
* so an upstream agent can still inspect the raw candidates.
|
|
43
43
|
*/
|
|
44
44
|
export interface SearchDocumentRetrievalTraceEntry {
|
|
@@ -84,20 +84,15 @@ export type SearchDocumentsStreamEvent = {
|
|
|
84
84
|
readonly type: "complete";
|
|
85
85
|
readonly response: SearchDocumentsResponse;
|
|
86
86
|
};
|
|
87
|
-
type SearchSession = {
|
|
88
|
-
query: string;
|
|
89
|
-
registry: Map<number, CuratedResult>;
|
|
90
|
-
};
|
|
91
|
-
type SearchSessions = Map<string, SearchSession>;
|
|
92
87
|
/**
|
|
93
88
|
* Build the preliminary (fast-phase) search response. Unlike
|
|
94
|
-
* {@link recordStructuredResultsSession} this NEVER touches memory
|
|
95
|
-
*
|
|
89
|
+
* {@link recordStructuredResultsSession} this NEVER touches memory — the final
|
|
90
|
+
* response owns that.
|
|
96
91
|
*/
|
|
97
|
-
export declare function createPreliminarySearchDocumentsResponse(sessionId: string, query: string, details:
|
|
98
|
-
export declare function createEmptySearchDocumentsResponse(sessionId: string, query: string,
|
|
99
|
-
export declare function
|
|
100
|
-
|
|
92
|
+
export declare function createPreliminarySearchDocumentsResponse(sessionId: string, query: string, details: AutoRAGResultsDetails, diagnostics?: readonly SearchDocumentDiagnostic[]): SearchDocumentsResponse;
|
|
93
|
+
export declare function createEmptySearchDocumentsResponse(sessionId: string, query: string, diagnostics?: readonly SearchDocumentDiagnostic[]): SearchDocumentsResponse;
|
|
94
|
+
export declare function normalizeEntryEvidenceRefs(entry: AutoRAGMappingEntry): SessionEvidenceRef[];
|
|
95
|
+
export declare function recordStructuredResultsSession(sessionId: string, query: string, details: AutoRAGResultsDetails, memory: RetrievalMemory, componentDiagnostics?: readonly SearchDocumentDiagnostic[], options?: {
|
|
96
|
+
/** Skip the retrieval-memory write. */
|
|
101
97
|
readonly isolateMemory?: boolean;
|
|
102
98
|
}): SearchDocumentsResponse;
|
|
103
|
-
export {};
|
|
@@ -35,6 +35,6 @@ export declare function rollbackIfBroken(options: SelfConfigOptions, before: str
|
|
|
35
35
|
/**
|
|
36
36
|
* The turn prompt for the `config` branch: the whole setup skill, where this
|
|
37
37
|
* run's config lives, the safety rules for editing it, and the user's request.
|
|
38
|
-
* The turn ends with
|
|
38
|
+
* The turn ends with the model's plain final message; there is no fast answer.
|
|
39
39
|
*/
|
|
40
40
|
export declare function buildSelfConfigPrompt(input: SelfConfigPromptInput): string;
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { type AutoRAGResultsDetails } from "../../agent/
|
|
1
|
+
import { type AutoRAGResultsDetails } from "../../agent/results.js";
|
|
2
2
|
import type { CommandContext } from "./types.js";
|
|
3
3
|
/** Validate a curated report payload against the `autorag report` contract (explicit number -> source mapping). */
|
|
4
4
|
export declare function validateReport(value: unknown): AutoRAGResultsDetails;
|
|
@@ -5,8 +5,9 @@ import type { CommandContext } from "./types.js";
|
|
|
5
5
|
* Pi owns the terminal UI, `read`/`bash`/`edit`/`write`/`grep`/`find`/`ls`
|
|
6
6
|
* tools, provider credentials/OAuth, model selection, session persistence and
|
|
7
7
|
* resume/fork/compaction, settings, and extension loading. AutoRAG registers
|
|
8
|
-
* its domain tools and
|
|
9
|
-
*
|
|
8
|
+
* its domain tools and runs the two-phase search in the same session through
|
|
9
|
+
* the agent's interactive runtime; the model's progress notes, first answer, and
|
|
10
|
+
* final answer are ordinary assistant messages there. The native
|
|
10
11
|
* `/login`, `/model`, `/resume`, `/new`, `/tree`, `/compact`, and `/settings`
|
|
11
12
|
* commands are therefore always available.
|
|
12
13
|
*/
|