@autorag/librarian 2.10.5 → 2.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (106) hide show
  1. package/NOTICE +14 -9
  2. package/README.md +65 -25
  3. package/dist/agent/agent.d.ts +51 -49
  4. package/dist/agent/answer-guidelines.d.ts +22 -1
  5. package/dist/agent/context-token-guard.d.ts +11 -0
  6. package/dist/agent/discord-remote.d.ts +30 -0
  7. package/dist/agent/embedding-controls.d.ts +22 -0
  8. package/dist/agent/index.d.ts +1 -2
  9. package/dist/agent/mcp-client.d.ts +27 -0
  10. package/dist/agent/pi-session.d.ts +8 -1
  11. package/dist/agent/query-decomposition.d.ts +1 -1
  12. package/dist/agent/search-documents.d.ts +2 -2
  13. package/dist/agent/search-progress.d.ts +96 -0
  14. package/dist/agent/system-prompt.d.ts +1 -1
  15. package/dist/agent/welcome-header.d.ts +35 -0
  16. package/dist/cli/commands/models.d.ts +0 -14
  17. package/dist/cli/commands/search.d.ts +3 -4
  18. package/dist/cli/commands/setup.d.ts +2 -0
  19. package/dist/cli/config.d.ts +7 -19
  20. package/dist/cli/index.d.ts +1 -1
  21. package/dist/cli/index.js +16364 -18337
  22. package/dist/cli/setup.d.ts +22 -36
  23. package/dist/cloud/config.d.ts +3 -3
  24. package/dist/core.js +7372 -9206
  25. package/dist/datasource/embedding-env.d.ts +29 -0
  26. package/dist/datasource/errors.d.ts +1 -1
  27. package/dist/datasource/index.d.ts +2 -1
  28. package/dist/datasource/sidecar/config.d.ts +3 -0
  29. package/dist/datasource/sidecar/index.d.ts +1 -1
  30. package/dist/datasource/skills/discrawl/client.d.ts +1 -0
  31. package/dist/datasource/skills/discrawl/embedding.d.ts +17 -0
  32. package/dist/datasource/skills/discrawl/types.d.ts +11 -3
  33. package/dist/datasource/skills/factory.d.ts +17 -1
  34. package/dist/datasource/skills/github-gist/semantic.d.ts +5 -5
  35. package/dist/datasource/skills/github-gist/skill.d.ts +2 -3
  36. package/dist/datasource/skills/hister/client.d.ts +83 -0
  37. package/dist/datasource/skills/hister/embedding.d.ts +64 -0
  38. package/dist/datasource/skills/hister/index.d.ts +2 -0
  39. package/dist/datasource/skills/hister/setup.d.ts +87 -0
  40. package/dist/datasource/skills/hister/skill.d.ts +63 -0
  41. package/dist/datasource/skills/lazykatok/types.d.ts +9 -1
  42. package/dist/datasource/skills/mailcrawl/types.d.ts +9 -1
  43. package/dist/datasource/skills/obsidian/index.d.ts +4 -6
  44. package/dist/datasource/skills/obsidian/methods.d.ts +13 -17
  45. package/dist/datasource/skills/obsidian/paths.d.ts +24 -24
  46. package/dist/datasource/skills/obsidian/skill.d.ts +10 -15
  47. package/dist/datasource/skills/obsidian/staging.d.ts +50 -0
  48. package/dist/datasource/skills/obsidian/vault-index.d.ts +44 -0
  49. package/dist/embedding/daemon.d.ts +57 -0
  50. package/dist/embedding/embedder.d.ts +59 -0
  51. package/dist/embedding/http-embedder.d.ts +28 -0
  52. package/dist/embedding/in-process.d.ts +19 -0
  53. package/dist/embedding/index.d.ts +28 -0
  54. package/dist/embedding/managed.d.ts +44 -0
  55. package/dist/embedding/prefixes.d.ts +18 -0
  56. package/dist/embedding/settings.d.ts +9 -0
  57. package/dist/index.js +7465 -9298
  58. package/dist/mcp/index.js +7460 -9294
  59. package/dist/memory/context.d.ts +1 -1
  60. package/dist/memory/similar-queries.d.ts +1 -1
  61. package/dist/minsync/binary.d.ts +21 -0
  62. package/dist/minsync/client.d.ts +11 -10
  63. package/dist/minsync/index.d.ts +1 -1
  64. package/dist/minsync/installer.d.ts +1 -1
  65. package/dist/minsync/method.d.ts +3 -11
  66. package/dist/minsync/process.d.ts +1 -0
  67. package/dist/minsync/types.d.ts +0 -2
  68. package/dist/minsync/workspace-lock.d.ts +7 -0
  69. package/dist/process/terminate-tree.d.ts +4 -5
  70. package/dist/retrieval/rerank.d.ts +10 -3
  71. package/licenses/hister-AGPL-3.0-notice.txt +47 -0
  72. package/package.json +4 -4
  73. package/skills/autorag/SKILL.md +9 -11
  74. package/skills/autorag-doctor/SKILL.md +50 -29
  75. package/skills/autorag-lite-setup/SKILL.md +31 -13
  76. package/skills/autorag-setup/SKILL.md +99 -26
  77. package/dist/agent/peer-target-tool.d.ts +0 -19
  78. package/dist/agent/query-peer-tool.d.ts +0 -51
  79. package/dist/cli/commands/gateway.d.ts +0 -7
  80. package/dist/cli/commands/p2p-policy.d.ts +0 -2
  81. package/dist/cli/commands/p2p.d.ts +0 -7
  82. package/dist/cli/commands/serve.d.ts +0 -19
  83. package/dist/datasource/skills/obsidian/client.d.ts +0 -15
  84. package/dist/datasource/skills/obsidian/types.d.ts +0 -92
  85. package/dist/embedding-runtime/cache.d.ts +0 -30
  86. package/dist/embedding-runtime/gateway-embedder.d.ts +0 -41
  87. package/dist/embedding-runtime/gateway.d.ts +0 -27
  88. package/dist/embedding-runtime/index.d.ts +0 -120
  89. package/dist/embedding-runtime/manifest.d.ts +0 -20
  90. package/dist/embedding-runtime/supervisor.d.ts +0 -74
  91. package/dist/embedding-runtime/types.d.ts +0 -70
  92. package/dist/p2p/approval-store.d.ts +0 -32
  93. package/dist/p2p/egress-gate.d.ts +0 -22
  94. package/dist/p2p/file-sharing.d.ts +0 -33
  95. package/dist/p2p/injection-classifier.d.ts +0 -33
  96. package/dist/p2p/injection-gate.d.ts +0 -37
  97. package/dist/p2p/pii-gate.d.ts +0 -47
  98. package/dist/p2p/policy-filter.d.ts +0 -15
  99. package/dist/p2p/policy.d.ts +0 -77
  100. package/dist/p2p/simplex-query-store.d.ts +0 -44
  101. package/dist/p2p/simplex-server.d.ts +0 -115
  102. package/dist/p2p/simplex-transport.d.ts +0 -62
  103. package/dist/p2p/wire.d.ts +0 -39
  104. package/licenses/embedding-assets.json +0 -85
  105. package/licenses/llama.cpp-MIT.txt +0 -21
  106. package/licenses/qwen3-embedding-notice.txt +0 -13
package/NOTICE CHANGED
@@ -24,12 +24,12 @@ License: MIT
24
24
  @modelcontextprotocol/server@2.3.1
25
25
  License: Apache-2.0
26
26
 
27
- @openrouter/sdk@1.4.25
28
- License: Apache-2.0
29
-
30
27
  chardet@2.2.0
31
28
  License: MIT
32
29
 
30
+ discord.js@14.27.0
31
+ License: Apache-2.0
32
+
33
33
  fast-xml-parser@5.11.2
34
34
  License: MIT
35
35
 
@@ -51,6 +51,9 @@ License: MIT
51
51
  pdfjs-dist@6.4.299
52
52
  License: Apache-2.0
53
53
 
54
+ pi-discord-remote@0.2.4
55
+ License: MIT
56
+
54
57
  puppeteer-core@25.13.0
55
58
  License: Apache-2.0
56
59
 
@@ -66,12 +69,6 @@ License: Apache-2.0
66
69
  typebox@1.3.37
67
70
  License: MIT
68
71
 
69
- Embedding runtime and model notices:
70
- licenses/llama.cpp-MIT.txt
71
- licenses/qwen3-embedding-notice.txt
72
- licenses/apache-2.0.txt
73
- licenses/embedding-assets.json
74
-
75
72
  Bundled Windows binaries (vendor/everything, extracted only on Windows):
76
73
  Everything 1.4.1.1032 portable (voidtools, David Carpenter)
77
74
  License: MIT (closed source); includes PCRE (BSD-3-Clause)
@@ -80,3 +77,11 @@ licenses/everything-MIT-and-PCRE-BSD.txt
80
77
  ES 1.1.0.38 command-line interface (voidtools)
81
78
  License: MIT
82
79
  licenses/es-MIT.txt
80
+
81
+ Separate programs AutoRAG talks to (not bundled, not distributed, not linked):
82
+ Hister (https://github.com/asciimoo/hister, asciimoo and Hister contributors)
83
+ License: AGPL-3.0-or-later
84
+ AutoRAG ships no Hister code. Users install the unmodified Hister release themselves and AutoRAG only sends
85
+ search requests to its public HTTP API (GET /search, GET /api/config), as a separate program over a network
86
+ socket. AutoRAG does not modify Hister and never hides that Hister is the backend of the `hister` datasource.
87
+ licenses/hister-AGPL-3.0-notice.txt
package/README.md CHANGED
@@ -18,7 +18,7 @@
18
18
 
19
19
  > [!IMPORTANT]
20
20
  > **Looking for the original AutoRAG (RAG AutoML / pipeline optimization tool)?**
21
- > This repository now hosts **AutoRAG 2.0**, a complete reimagining of AutoRAG as a self-evolving librarian agent. The original Python-based AutoRAG — the RAG AutoML tool for automatically finding an optimal RAG pipeline for your data — now lives in the [`legacy/`](legacy/) directory of this repository.
21
+ > This repository now hosts **AutoRAG 2.0**, a complete reimagining of AutoRAG as a personal librarian agent for your office work. The original Python-based AutoRAG — the RAG AutoML tool for automatically finding an optimal RAG pipeline for your data — now lives in the [`legacy/`](legacy/) directory of this repository.
22
22
  >
23
23
  > **The legacy AutoRAG is NOT abandoned.** It continues to be maintained (bug fixes, dependency updates, and PyPI releases via `pip install AutoRAG`) in maintenance mode. Existing users can keep using it exactly as before — see the [legacy README](legacy/README.md) for its documentation, and file issues in this repository as usual. New feature development is focused on AutoRAG Agent (2.0).
24
24
 
@@ -131,9 +131,10 @@ does, and searches them right alongside your files:
131
131
 
132
132
  ### Private by design
133
133
 
134
- - **Embedded on your machine.** Document understanding runs on the local
135
- [`qwen3-embedding-0.6b`](docs/model-cards/qwen3-embedding-0.6b.md) model.
136
- File contents are never sent to a remote embedder.
134
+ - **Embedded on your machine.** Semantic indexing runs on MinSync's local
135
+ Qwen3-Embedding-0.6B engine (1024 dimensions), which MinSync provisions and
136
+ runs; AutoRAG owns no embedding model. File contents are never sent to a
137
+ remote embedder unless you explicitly configure a remote `minSync.embedder`.
137
138
  - **Keys stay encrypted.** Sign-in tokens and API keys live in the system's
138
139
  encrypted store, never in a config file.
139
140
  - **Bring your own model.** Use the [hosted Dazzi Models plan](#hosted-dazzi-models-plan),
@@ -153,7 +154,7 @@ The app is one way in. The same librarian runs everywhere else:
153
154
  |---|---|---|
154
155
  | **Desktop app (Dazzi)** | Browsing and asking in one window | [NomaDamas/AutoRAG-app](https://github.com/NomaDamas/AutoRAG-app) (Fair Source, FSL-1.1-MIT) |
155
156
  | **Terminal UI** | Interactive, steerable search sessions | [`autorag tui`](#2-search-from-cli) |
156
- | **Messenger** | Asking the librarian from a chat app | [Telegram](#messenger-connections) |
157
+ | **Messenger** | Asking the librarian from a chat app | [Telegram, Discord](#messenger-connections) |
157
158
  | **CLI** | Scripts, indexing, health checks | [CLI Command Reference](#cli-command-reference) |
158
159
  | **MCP server** | Feeding retrieval to Claude Code, Codex, and other agents | [AutoRAG Lite MCP Server](#autorag-lite-mcp-server) |
159
160
  | **TypeScript API** | Embedding the librarian in your own app | [Programmatic usage](#3-programmatic-usage-typescript-api) |
@@ -165,7 +166,7 @@ The app is one way in. The same librarian runs everywhere else:
165
166
 
166
167
  Search tools dump file paths and matching lines — forcing you to open files, read context, and synthesize answers yourself.
167
168
 
168
- **AutoRAG Agent is a self-evolving librarian agent.** Built on the [Pi](https://github.com/earendil-works/pi-mono) agent framework, a single configured model searches across multiple retrieval methods, opens source documents directly via `bash` to verify ground truth, and curates answers into clean, numbered knowledge units:
169
+ **AutoRAG Agent is a personal librarian agent that helps with your office work.** Built on the [Pi](https://github.com/earendil-works/pi-mono) agent framework, a single configured model searches your files, mail, chats, and notes in place, opens source documents directly via `bash` to verify ground truth, and curates answers into clean, numbered knowledge units. Because it works for you rather than only on your index, it can also act on your files when you ask, such as organizing, renaming, moving, or editing them with a record of every change:
169
170
 
170
171
  ```text
171
172
  You ask: "What changes were made to the cloud infrastructure contract in Q3?"
@@ -186,9 +187,9 @@ AutoRAG Agent:
186
187
 
187
188
  Three principles drive every design decision in AutoRAG Agent:
188
189
 
189
- 1. **Never migrate your data to search it.** Traditional RAG systems force you to upload, ETL, and duplicate your files into a centralized vector database. AutoRAG Agent federates your data **in place**, querying CLI-native stores (`lazykatok`, `discrawl`, `slacrawl`, `mailcrawl`, `rclone`, `qmd`) where your data already lives. Results retain opaque, source-native identities (`/kakao/...`, `/slack/...`) that preserve local access control and privacy. *(See our [Competitive Landscape Study](docs/competitive-landscape-2026-09.md) on why in-place federation is the durable differentiator).*
190
+ 1. **Never migrate your data to search it.** Traditional RAG systems force you to upload, ETL, and duplicate your files into a centralized vector database. AutoRAG Agent federates your data **in place**, querying CLI-native stores (`lazykatok`, `discrawl`, `slacrawl`, `mailcrawl`, `rclone`) where your data already lives. Results retain opaque, source-native identities (`/kakao/...`, `/slack/...`) that preserve local access control and privacy. *(See our [Competitive Landscape Study](docs/competitive-landscape-2026-09.md) on why in-place federation is the durable differentiator).*
190
191
 
191
- 2. **Just works — no RAG degree required.** No pipeline tuning, no vector-DB maintenance, and no remote embedding keys required. AutoRAG Agent automatically manages [MinSync](docs/minsync-setup.md) for incremental Change Data Capture (CDC) chunking and provides a local [embedding gateway](docs/embedding-runtime.md) out of the box with zero external telemetry. A [remote rerank model](docs/rerank.md) (the hosted Dazzi Models reranker when signed in, else OpenRouter `voyageai/rerank-3-lite`) is an opt-in extra.
192
+ 2. **Just works — no RAG degree required.** No pipeline tuning, no vector-DB maintenance, and no remote embedding keys required. AutoRAG Agent automatically manages [MinSync](docs/minsync-setup.md) for incremental Change Data Capture (CDC) chunking and provides a local [embedding backend](docs/minsync-setup.md#embedding-backends) out of the box with zero external telemetry. A [remote rerank model](docs/rerank.md) (the hosted Dazzi Models reranker when signed in, else OpenRouter `voyageai/rerank-3-lite`) is an opt-in extra.
192
193
 
193
194
  3. **Fast by design.** Rather than coordinating slow multi-agent hierarchies, a single configured model owns the entire retrieval, direct-read, and curation loop. Local CDC chunks (BM25, vector, and hybrid modes) deliver rapid, low-latency turnaround across multi-turn research queries.
194
195
 
@@ -201,7 +202,7 @@ AutoRAG Agent orchestrates five integrated subsystems:
201
202
  1. **Retrieval Memory (`check_memory`):** After each answer, cited evidence Jev judges to genuinely support it is stored in `~/.autorag/memory.json`. Later searches receive this conversation's evidence, similar past questions, and long-term insights as **advisory reference** — it never reorders results or overrides current evidence. See [docs/retrieval-memory.md](docs/retrieval-memory.md).
202
203
  2. **Pluggable Multi-Method Retrieval:**
203
204
  - **BM25 Lexical Search:** Fast keyword ranking via MinSync over parsed markdown mirrors. AutoRAG always uses MinSync's `multilingual` tokenizer, which keeps each original word and adds Korean morphemes (via the [Kiwi](https://github.com/bab2min/Kiwi) library), so both `연구목표` and `인증서를` → `인증서` match; without Kiwi AutoRAG warns and tokenizes on whitespace. See [docs/minsync-setup.md](docs/minsync-setup.md#bm25-tokenizer).
204
- - **Semantic Vector Search:** Dense vector retrieval over CDC chunks via the built-in embedding gateway.
205
+ - **Semantic Vector Search:** Dense vector retrieval over CDC chunks via the shared MinSync embedder.
205
206
  - **Hybrid Search:** Combines BM25 and vector ranking via Reciprocal Rank Fusion (RRF).
206
207
  - **Jikji Find-First Discovery:** Local CLI-backed fast discovery answer packs.
207
208
  - **Everything File-Name Search (Windows):** The bundled [voidtools Everything](https://www.voidtools.com/) indexes file and folder names under your search roots so the agent finds files by name, extension, path, size, or date instantly (`everything_search`).
@@ -217,8 +218,12 @@ AutoRAG uses `@earendil-works/pi-coding-agent` as the runtime host for model ses
217
218
 
218
219
  The CLI TUI (`autorag tui`) is Pi's interactive mode hosted on the AutoRAG librarian: there is no separate AutoRAG renderer. Pi owns the terminal UI and its native in-session commands — `/login`/`/logout`, `/model`, `/resume`, `/new`, `/tree`, `/compact`, and `/settings` — so provider sign-in and model selection work on first launch even when no AutoRAG model is configured. AutoRAG registers its retrieval tools and runs the two-phase search in the same Pi session, where the model's progress notes, first answer, and final answer are ordinary assistant messages, so resume and two-phase search keep working together.
219
220
 
221
+ A fresh session — at launch and after `/new` (or `/reload`) — opens on AutoRAG's own welcome screen: the AutoRAG logo, the version, and three rotating tips, centered and scaled down to a smaller logo or a plain wordmark on narrow terminals. Pi's own logo, key-hint list, and loaded skills/prompts/extensions listing are hidden by default (`quietStartup: "header"`). A `quietStartup` saved in Pi's global or project `settings.json` wins: `true` removes the banner, `false` brings Pi's resource listing back.
222
+
220
223
  Typing while a search runs works like steering one of Pi's own turns. **Enter** steers the running search: AutoRAG treats the text as a correction of a search that is going wrong, delivers it before the search's next model request (also between phases, where no turn is streaming), cuts in-flight retrieval short so the correction lands promptly, and always runs the verification phase so the answer reflects the correction. **Alt+Enter** queues a new question that runs after the current search answers. **Esc** stops the search instead of letting it continue into its next phase. Pi's registered `/` commands still run during a search; any other line starting with `/` (a path, for example) is a steer like any other text. A query typed after a search answers is a new search in the same session, as before.
221
224
 
225
+ From the moment a question is submitted, a live progress widget sits above the editor: a spinner, the elapsed time, and one line per step the search is running (routing, splitting the question, each retrieval source with its hit count and time, ranking, the first answer, verification). A slow source shows its own running timer, a failing source shows its error text verbatim, and the widget disappears when the answer is final or **Esc** stops the search. While the model streams, Pi's own `Working` loader takes over and the widget keeps only the header and steps.
226
+
222
227
  Update notices are AutoRAG's, not Pi's: `autorag tui` suppresses Pi's "run `pi update`" banner (Pi is a bundled host, not a separate install) and instead checks the published `@autorag/librarian` release, showing a one-line notice in the session when a newer version exists. Set `AUTORAG_NO_UPDATE_CHECK=1` to skip the lookup; `AUTORAG_UPDATE_CHECK_URL` overrides the registry endpoint.
223
228
 
224
229
  ---
@@ -229,12 +234,15 @@ Update notices are AutoRAG's, not Pi's: `autorag tui` suppresses Pi's "run `pi u
229
234
  |---|---|---|
230
235
  | **Telegram** | Supported | [`autorag tui --telegram`](docs/telegram.md) |
231
236
  | **WhatsApp** | Supported (unofficial client) | [`autorag tui --whatsapp`](docs/whatsapp.md) |
232
- | Discord, Slack, others | Not supported yet | |
237
+ | **Discord** | Supported (no steering yet) | [`autorag tui --discord`](docs/discord.md) |
238
+ | Slack, others | Not supported yet | |
233
239
 
234
240
  Telegram runs through the [`@llblab/pi-telegram`](https://github.com/llblab/pi-telegram) bridge inside a running `autorag tui` session: questions you send the bot get the same routing and two-phase search as the terminal. Setup, commands, and keeping it running are in [docs/telegram.md](docs/telegram.md).
235
241
 
236
242
  WhatsApp runs through the pinned [`whatsapp-pi`](https://www.npmjs.com/package/whatsapp-pi) extension the same way; see [docs/whatsapp.md](docs/whatsapp.md).
237
243
 
244
+ Discord runs through the [`pi-discord-remote`](https://pi.dev/packages/pi-discord-remote) extension and creates one channel per TUI session; the fast answer and the verified final answer are posted as they land. Setup, permissions, and limits are in [docs/discord.md](docs/discord.md).
245
+
238
246
  ## Installation & Setup
239
247
 
240
248
  AutoRAG Agent is published as `@autorag/librarian` (requires Node.js ≥ 24 or Bun):
@@ -467,13 +475,11 @@ mirrored are kept and not re-parsed on the next run. `onProgress` reports `parse
467
475
  | `autorag health` | Check model provider authentication, token validity, and API reachability |
468
476
  | `autorag models list` | List chat models the pi runtime can resolve (built-ins, `models.json`, custom/extension providers) with provider auth status; never prints credential values |
469
477
  | `autorag update-check` | Compare the running `autorag` against the published npm version (also runs on `autorag tui` launch) |
470
- | `autorag tui` | Open Pi's interactive librarian TUI (`/login`, `/model`, `/resume`, …); `--telegram` / `--whatsapp` add the Telegram / WhatsApp remote-control bridges |
478
+ | `autorag tui` | Open Pi's interactive librarian TUI (`/login`, `/model`, `/resume`, …); `--telegram` / `--whatsapp` / `--discord` add the Telegram / WhatsApp / Discord remote-control bridges |
471
479
  | `autorag duplicates [DIR]` | Read-only scan for exact and near-duplicate document families with `dupey` |
472
480
  | `autorag lite ...` | CLI bootstrap, indexing repair, terminal maintenance, and fallback interface; agents use MCP for normal Lite operation |
473
481
  | `autorag memory inspect` | Render a read-only, path-opaque snapshot of retrieval memory (judged evidence, curated results, evidence chunks, insights); model-free |
474
482
  | `autorag evidence <session>` | Inspect persisted evidence behind numbered results; MCP clients normally use `autorag.evidence`, and the CLI stays available for terminal maintenance |
475
- | `autorag serve` | Start the P2P query server over SimpleX (opt-in) |
476
- | `autorag p2p ...` | Manage peer trust, query approvals, and sharing policies |
477
483
 
478
484
  ---
479
485
 
@@ -499,6 +505,36 @@ Core Lite tools include `autorag.status`, `autorag.search`,
499
505
  integrated datasources expose additional scoped search tools. Read-only MCP
500
506
  mode omits mutating tools such as refresh and report.
501
507
 
508
+ ## Using external MCP servers and agent skills
509
+
510
+ AutoRAG is also an MCP *client*. Servers listed in the pi agent directory's `mcp.json`
511
+ (`~/.pi/agent/mcp.json`, or `<piAgentDir>/mcp.json` when the agent is given one) are connected
512
+ when a TUI session or a one-shot `autorag search` starts, and their tools reach the model next to
513
+ AutoRAG's own tools. The file uses the format shared by other MCP clients:
514
+
515
+ ```json
516
+ {
517
+ "mcpServers": {
518
+ "filesystem": { "command": "npx", "args": ["-y", "@modelcontextprotocol/server-filesystem", "."] },
519
+ "docs": { "url": "https://example.com/mcp", "headers": { "Authorization": "Bearer ${DOCS_TOKEN}" } }
520
+ }
521
+ }
522
+ ```
523
+
524
+ - **Exposure.** By default a server's tools are reached through the `codemode` tool (the model
525
+ writes a short script that calls them); `"exposure": "direct"` declares them to the model
526
+ like built-in tools, `"deferred"` loads them on demand through `tool_search`, and `"hidden"`
527
+ turns a server off. `description` is shown to the model so it knows when to use the server.
528
+ - **Only the user-level file is read.** A project `.pi/mcp.json` is ignored on purpose: it would
529
+ let any folder AutoRAG starts in choose which processes run.
530
+ - **Failures are reported verbatim.** An invalid entry or a server that fails to start is
531
+ reported at session start with pi's own message; the other servers keep working.
532
+ - **Inside the TUI**, `/mcp` lists servers, reconnects them, signs in to OAuth servers, and
533
+ changes exposure. Pi's `/skill:<name>` commands and `<available_skills>` list also work for
534
+ any agent skill installed under the agent directory.
535
+ - MCP results are data from an external server. AutoRAG tells the model never to follow
536
+ instructions found in them.
537
+
502
538
  ---
503
539
 
504
540
  ## ⚡ AutoRAG Lite: Model-Free Retrieval Engine
@@ -582,7 +618,8 @@ Do not copy or retain the removed `autorag-lite-search` skill.
582
618
  autorag search "your question" --json
583
619
  ```
584
620
  4. **Safety Guidelines for Agents:**
585
- - **Never delete, move, or modify original user documents.**
621
+ - **Setup, diagnostics, indexing, and refresh never modify your documents.** An agent that installs, configures, or repairs AutoRAG (Claude Code, Codex, and similar) must not move, rename, edit, or delete user documents as part of that work. That rule is about those procedures; it is not a limit on the librarian itself, which may organize, create, or modify files in local sessions when the user asks (see `Local File Organization` in the system prompt).
622
+ - **Never permanently delete.** Deletion goes to the OS trash, and every change is listed as old path → new path so it can be undone.
586
623
  - AutoRAG writes index artifacts into `<workspace>/.autorag/` and Jikji's central store (`<OS data dir>/jikji/`, override with `JIKJI_DATA_DIR`). Jikji also drops a `.jikji_agent_map.md` guide into each indexed root.
587
624
  - Never print or leak API keys, tokens, or credential values into stdout or logs.
588
625
 
@@ -603,13 +640,14 @@ AutoRAG Agent connects to external tools and communication platforms using dedic
603
640
  | **Notion** | `notion` | [`notcrawl`](https://github.com/openclaw/notcrawl) CLI | Local page/database/block archive + FTS5 | Lexical FTS5 |
604
641
  | **Email Archives** | `mailcrawl` | [`mailcrawl`](https://github.com/NomaDamas/mailcrawl) CLI | Local Gmail, IMAP, and Maildir storage | BM25, Semantic, Hybrid |
605
642
  | **Local Mail Export**| `mail-export` | Built-in `.mbox` / `.eml` parser | Local filesystem mailboxes | Lexical |
606
- | **Obsidian Vaults** | `obsidian` | [`qmd`](https://github.com/tobi/qmd) CLI | Direct markdown vault indexing | BM25, Semantic |
643
+ | **Obsidian Vaults** | `obsidian` | Indexed with MinSync in its own workspace (no external CLI) | Markdown notes, folder-scopable | Lexical (BM25), Semantic, Scoped |
607
644
  | **GitHub** | `github` | GitHub REST API | In-memory fetched Issues and Pull Requests | Lexical, Scoped |
608
645
  | **GitHub Gists** | `github-gist` | GitHub REST API | Incremental local index of own-account public and permitted secret Gist content/metadata; token not stored | Lexical (BM25), Local Semantic, Scoped |
609
646
  | **Cloud Drives** | `cloud-drive` | [`rclone`](https://rclone.org) CLI | Google Drive (Tier-1), OneDrive, Dropbox, etc. | Incremental Mirror + BM25 |
610
647
  | **Photos & Shots** | `clawgallery` | `clawgallery` CLI | Local screenshot and photo store | Hybrid OCR/Visual Search |
611
648
  | **RSS / News** | `rss` | Native HTTP Poller | RSS 2.0 & Atom feeds (24h deduplication) | Lexical |
612
649
  | **macOS Spotlight** | `spotlight` | Native `mdfind` CLI | macOS system metadata and content index | System Native |
650
+ | **Browser History** | `hister` | [Hister](https://github.com/asciimoo/hister) server (AGPL-3.0-or-later, separate program, HTTP API) | Pages visited in Chrome/Chromium/Firefox, captured by Hister's own extension and indexed by Hister (web pages only; its local-file indexing is never used) | Lexical full-text; optional Semantic via Hister's own embedding endpoint |
613
651
 
614
652
  For configuration syntax and connector details, see [docs/datasource-skills.md](docs/datasource-skills.md). Non-interactive `sync`/`index` steps get a 30-minute per-connector budget (`connector.indexTimeoutMs`) because first-run imports routinely take minutes; interactive search keeps its 60-second default.
615
653
 
@@ -685,14 +723,14 @@ the defaults.
685
723
 
686
724
  Deep dive into AutoRAG Agent's architecture, security, and integration guides:
687
725
 
688
- - **[MinSync Setup & Embedding QA](docs/minsync-setup.md):** Automatic binary installation, CDC chunking, and local embedding verification.
689
- - **[Local Embedding Runtime & Gateway](docs/embedding-runtime.md):** AutoRAG-owned local gateway, model prefetching, and zero-egress semantic search. Model card: [`qwen3-embedding-0.6b`](docs/model-cards/qwen3-embedding-0.6b.md).
726
+ - **[MinSync Setup & Embedding QA](docs/minsync-setup.md):** Automatic binary installation, CDC chunking, embedding backends (remote, `minsync serve` daemon, in-process addon), and local verification.
690
727
  - **[Hosted Dazzi Models Provider](docs/cloud-provider.md):** The hosted `dazzi` model provider — PKCE `/login`, `AUTORAG_BASE_URL`, live model discovery, and offline catalog snapshots.
728
+ - **[Discord Remote Control](docs/discord.md):** Ask the librarian from a Discord channel through a running `autorag tui --discord` session: bot setup, permissions, and limits.
691
729
  - **[Telegram Remote Control](docs/telegram.md):** Ask the librarian from a Telegram chat through a running `autorag tui --telegram` session: setup, commands, and keeping it alive.
692
730
  - **[Datasource Skills Reference](docs/datasource-skills.md):** Full configuration contracts, connection aliases, and connector options.
731
+ - **[Apple Calendar & Reminders via EventKit MCP](docs/eventkit-mcp.md):** macOS-only: read Calendar events and Reminders (Google/iCloud/Exchange accounts already in Calendar.app) by running `mcp-server-apple-events` beside `autorag-mcp`. Live, read-only; not indexed.
693
732
  - **[Jev Decisions](docs/jev-decisions.md):** Jev query pipeline, **on by default** via Dazzi Models when signed in (OpenRouter otherwise): it routes each question to a direct answer (intrinsic knowledge) or local search (everything needing more information; web search is left to the agent after the fast answer), splits multi-part questions into up to five parallel searches (`dazzi/openai/gpt-6-luna` when signed in, else `openrouter/qwen/qwen3.7-flash`), picks which registered datasources to search before the fast answer, and ends the run after the fast answer when that answer is complete. Also covers the `jev` judgment tool, backends (OpenRouter, TypeSafe, Vercel AI Gateway), and how to opt out.
694
733
  - **[Manual QA & Datasource Test Harnesses](docs/manual-qa-datasources.md):** Real-world testing guides for Discord, KakaoTalk, Slack, Notion, and email.
695
- - **[P2P SimpleX Sharing & Path Standard](docs/p2p-path-standard.md):** Decentralized peer query sharing with SimpleX, PII redaction, and approval queues.
696
734
  - **[Supply Chain Security & License Audits](docs/supply-chain.md):** Software bill of materials (SBOM) and dependency gate policies.
697
735
  - **[Competitive Landscape Study](docs/competitive-landscape-2026-09.md):** In-depth analysis of why in-place federation outperforms centralized RAG.
698
736
  - **[Legacy AutoRAG (AutoML)](legacy/README.md):** Documentation for the legacy Python AutoML pipeline optimizer (`pip install AutoRAG`).
@@ -714,26 +752,27 @@ AutoRAG Agent is an open-source project built by the community. We welcome contr
714
752
  AutoRAG Agent stands on the shoulders of fantastic open-source projects:
715
753
 
716
754
  - **[Pi Framework](https://github.com/earendil-works/pi-mono)** by [@earendil-works](https://github.com/earendil-works) — The foundational agent framework powering AutoRAG's reasoning loop.
717
- - **[MinSync](https://github.com/Marker-Inc-Korea/minsync)** — Ultra-fast incremental Change Data Capture (CDC) chunking and local BM25/vector indexing.
755
+ - **[MinSync](https://github.com/Marker-Inc-Korea/minsync)** — Ultra-fast incremental Change Data Capture (CDC) chunking, local BM25/vector indexing, and the Qwen3-Embedding-0.6B embedding engine.
718
756
  - **[Jikji](https://github.com/NomaDamas/jikji)** by [NomaDamas](https://github.com/NomaDamas) — High-performance find-first local document discovery.
719
757
  - **[Everything](https://www.voidtools.com/)** and **[ES](https://github.com/voidtools/ES)** by voidtools (David Carpenter) — Instant Windows file-name indexing and its command-line interface, bundled under the MIT License.
720
758
  - **[FSearch](https://github.com/cboxdoerfer/fsearch)** and **[fsearch-cli](https://github.com/NomaDamas/fsearch-mac)** — Everything-style file-name search for macOS/Linux (GPL-2.0-or-later). Never bundled or linked: users install fsearch-cli separately and AutoRAG spawns it as a separate process (mere aggregation per the FSF GPL FAQ).
759
+ - **[Hister](https://github.com/asciimoo/hister)** by [asciimoo](https://github.com/asciimoo) and contributors — Self-hosted browser web-history search (AGPL-3.0-or-later). Never bundled, linked, copied, or modified: users run Hister as a separate program and AutoRAG only calls its public HTTP search API (mere aggregation). Hister is always named as the backend of the `hister` datasource.
721
760
  - **[dupey](https://github.com/NomaDamas/dupey)** by [NomaDamas](https://github.com/NomaDamas) — Fast duplicate and near-duplicate document family detection.
722
- - **Federated CLI Authors:** External datasource tools [`lazykatok`](https://github.com/changeroa/lazykatok), [`discrawl`](https://github.com/openclaw/discrawl), [`mailcrawl`](https://github.com/NomaDamas/mailcrawl), [`wacrawl`](https://github.com/openclaw/wacrawl), [`telecrawl`](https://github.com/openclaw/telecrawl), [`slacrawl`](https://github.com/openclaw/slacrawl), [`notcrawl`](https://github.com/openclaw/notcrawl), [`qmd`](https://github.com/tobi/qmd), and [`rclone`](https://rclone.org).
761
+ - **Federated CLI Authors:** External datasource tools [`lazykatok`](https://github.com/changeroa/lazykatok), [`discrawl`](https://github.com/openclaw/discrawl), [`mailcrawl`](https://github.com/NomaDamas/mailcrawl), [`wacrawl`](https://github.com/openclaw/wacrawl), [`telecrawl`](https://github.com/openclaw/telecrawl), [`slacrawl`](https://github.com/openclaw/slacrawl), [`notcrawl`](https://github.com/openclaw/notcrawl), and [`rclone`](https://rclone.org).
723
762
  - **[kordoc](https://github.com/chrisryugj/kordoc)** by [chrisryugj](https://github.com/chrisryugj) — HWP/HWPX/HWPML, PDF, DOCX and XLSX parsing with nested-table fidelity, used as AutoRAG's default document parser.
724
763
 
725
764
  ---
726
765
 
727
766
  ## Troubleshooting
728
767
 
729
- When search returns nothing, a datasource disappears from results, refresh hangs, or the gateway will not start, run the **`autorag-doctor`** agent skill (`skills/autorag-doctor/SKILL.md`). Point your coding agent at it and say *"check AutoRAG"* — it walks the full diagnose-and-repair procedure and ends with a per-datasource status table showing what is indexed and what actually returns hits.
768
+ When search returns nothing, a datasource disappears from results, refresh hangs, or the embedder is unavailable, run the **`autorag-doctor`** agent skill (`skills/autorag-doctor/SKILL.md`). Point your coding agent at it and say *"check AutoRAG"* — it walks the full diagnose-and-repair procedure and ends with a per-datasource status table showing what is indexed and what actually returns hits.
730
769
 
731
770
  The first three commands cover most of it:
732
771
 
733
772
  ```bash
734
773
  autorag status --json # freshness, per-component state, diagnostics
735
774
  autorag health --json # model resolution + one live completion probe
736
- autorag gateway status --format json # on-demand embedding runtime
775
+ autorag setup # resolves the embedding backend and reports it
737
776
  ```
738
777
 
739
778
  Indexing is not the same thing as searchability, so always confirm retrieval
@@ -754,8 +793,8 @@ Common failures and their fix:
754
793
  | Symptom | Diagnostic code | Fix |
755
794
  |---|---|---|
756
795
  | Results are missing recent files | `stale-index` | `autorag refresh --method parsed,minsync --json` |
757
- | Semantic search returns nothing after changing the embedder | `embedding-identity-mismatch` | `autorag index rebuild --method minsync` |
758
- | Gateway will not start, a previous run was killed | `lock-conflict` | `autorag gateway stop`, then retry |
796
+ | Semantic search returns nothing after changing the embedder, or after moving off the old embedding gateway | `embedding-identity-mismatch` | `autorag index rebuild --method minsync` |
797
+ | No remote endpoint answers, no `minsync serve` daemon could start, and the in-process addon is missing or failed | `embedder-unavailable` | run `minsync serve` by hand to see its own error, install MinSync, install the optional `@nomadamas/minsync-native` addon, or configure a remote `minSync.embedder` |
759
798
  | A datasource is healthy in its own CLI but absent from results | — | confirm it is configured under `datasources` and connected through its native CLI |
760
799
  | A datasource errors during refresh | `datasource-index-failed` | run that CLI's own `doctor` |
761
800
  | MinSync binary missing (search and refresh fail with `MinSync is required ...`) | — | `cargo install minsync`, or leave `minSync.autoInstall` on and re-run refresh |
@@ -764,7 +803,7 @@ Common failures and their fix:
764
803
  | Windows file-name search fails during refresh | `everything-index-failed` | read the ES exit code and stderr in the message, then `autorag refresh --method everything --json` |
765
804
  | Every search does a full local search and verification, even for small talk | `query-route-fallback` | sign in to Dazzi Models with `/login`, or set `OPENROUTER_API_KEY` (Jev routing and decomposition are on by default) |
766
805
 
767
- Native datasource stores stay owned by their CLIs — AutoRAG never rebuilds them. Fix a broken archive with `lazykatok doctor`, `discrawl --json metadata`, `slacrawl --json doctor`, `wacrawl --json doctor`, `telecrawl --json doctor`, `notcrawl doctor`, `qmd status`, or `mailcrawl doctor`, then re-run `autorag refresh --method datasources --json`.
806
+ Native datasource stores stay owned by their CLIs — AutoRAG never rebuilds them. Fix a broken archive with `lazykatok doctor`, `discrawl --json metadata`, `slacrawl --json doctor`, `wacrawl --json doctor`, `telecrawl --json doctor`, `notcrawl doctor`, or `mailcrawl doctor`, then re-run `autorag refresh --method datasources --json`.
768
807
 
769
808
  ---
770
809
 
@@ -775,3 +814,4 @@ Native datasource stores stay owned by their CLIs — AutoRAG never rebuilds the
775
814
  - Production third-party notices and licenses are documented in [`NOTICE`](NOTICE).
776
815
  - The bundled Windows binaries keep their own licenses: Everything (MIT, plus PCRE BSD-3-Clause) in [`licenses/everything-MIT-and-PCRE-BSD.txt`](licenses/everything-MIT-and-PCRE-BSD.txt) and ES (MIT) in [`licenses/es-MIT.txt`](licenses/es-MIT.txt). Everything's source code is not public; AutoRAG redistributes the unmodified voidtools portable ZIPs.
777
816
  - FSearch/fsearch-cli (GPL-2.0-or-later) are **not** distributed with AutoRAG: no code is copied, linked, or bundled. Users install fsearch-cli themselves and AutoRAG communicates with it only through its command-line interface and daemon socket — separate programs, per the [FSF mere-aggregation FAQ](https://www.gnu.org/licenses/gpl-faq.html#MereAggregation). Source: https://github.com/NomaDamas/fsearch-mac.
817
+ - Hister (AGPL-3.0-or-later) is **not** distributed with AutoRAG: no Hister code is copied, linked, bundled, or modified. Users install an unmodified Hister release themselves and AutoRAG talks to that separate process only over its public HTTP API. Notice: [`licenses/hister-AGPL-3.0-notice.txt`](licenses/hister-AGPL-3.0-notice.txt). If you run Hister as a service for other people, AGPL section 13 applies to you as its operator.
@@ -3,17 +3,13 @@ import type { Api, ImageContent, Model } from "@earendil-works/pi-ai";
3
3
  import { type DatasourceRetryOptions } from "../datasource/retry.js";
4
4
  import type { DatasourceIndexResult, DatasourceSkill } from "../datasource/types.js";
5
5
  import { type DupeyCliOptions } from "../dupey/index.js";
6
- import { type Embedder } from "../embedding-runtime/gateway-embedder.js";
6
+ import { type Embedder, type ManagedEmbedder } from "../embedding/index.js";
7
7
  import { type EverythingClientOptions, type EverythingSearchRequest, type EverythingSearchResult } from "../everything/index.js";
8
8
  import { type FSearchClientOptions, type FSearchFailureReason, type FSearchSearchRequest, type FSearchSearchResult } from "../fsearch/index.js";
9
9
  import { type JikjiFailureReason, type JikjiOptions, type JikjiPrepareOptions } from "../jikji/index.js";
10
10
  import { type LanguageTag } from "../language.js";
11
11
  import { type MinSyncSyncResult, type MinSyncVectorMethodOptions } from "../minsync/index.js";
12
12
  import { type ParsedMirrorSyncOptions, type ParsedMirrorSyncResult } from "../mirror/sync.js";
13
- import type { PolicyResolver } from "../p2p/policy-filter.js";
14
- import type { SimplexQueryState } from "../p2p/simplex-query-store.js";
15
- import { type SimplexTransport } from "../p2p/simplex-transport.js";
16
- import type { PeerQueryResponse } from "../p2p/wire.js";
17
13
  import { type DefaultParserRegistryOptions } from "../parser/index.js";
18
14
  import { RetrievalEngine } from "../retrieval/engine.js";
19
15
  import { RetrievalMethodRegistry } from "../retrieval/registry.js";
@@ -193,43 +189,20 @@ export interface AutoRAGWatchRefreshOptions {
193
189
  readonly watcherFactory?: WatcherFactory;
194
190
  }
195
191
  export type AutoRAGWatchRefreshHandle = WatchRefreshHandle;
196
- declare module "../retrieval/types.js" {
197
- interface RetrievalOptions {
198
- /** Server-only P2P policy identity; never included in model tool schemas. */
199
- peerFingerprint?: string;
200
- /** Server-only P2P policy resolver. */
201
- resolvePolicy?: PolicyResolver;
202
- /** Server-only run registry populated from observed retrieval output. */
203
- observedSources?: Set<string>;
204
- /** Server-only datasource source globs derived from the loaded policy. */
205
- policyDatasourceScopes?: readonly string[];
206
- }
207
- }
208
- export type RemoteSessionRejectionCode = "injection-detected" | "outbound-leak-detected";
209
- export declare class RemoteSessionRejectedError extends Error {
210
- readonly code: RemoteSessionRejectionCode;
211
- constructor(code: RemoteSessionRejectionCode);
212
- }
213
192
  /** Thinking level applied per search phase. "off" requests no reasoning. */
214
193
  export type AutoRAGThinkingLevel = "off" | "minimal" | "low" | "medium" | "high" | "xhigh" | "max";
215
194
  /**
216
195
  * Per-phase thinking for AutoRAG's two-phase search: the fast phase delivers
217
196
  * an immediate first answer with {@link AutoRAGThinkingOptions.fast} thinking
218
- * (default "off"), then the verification phase re-checks and finalizes with
197
+ * (default "minimal"), then the verification phase re-checks and finalizes with
219
198
  * {@link AutoRAGThinkingOptions.final} thinking (default "high").
220
199
  */
221
200
  export interface AutoRAGThinkingOptions {
222
- /** Thinking level for the immediate first answer. Default "off". */
201
+ /** Thinking level for the immediate first answer. Default "minimal": some hosted models reject "off". */
223
202
  readonly fast?: AutoRAGThinkingLevel;
224
203
  /** Thinking level for the verification/finalization phase. Default "high". */
225
204
  readonly final?: AutoRAGThinkingLevel;
226
205
  }
227
- export interface PeerQueryOptions {
228
- readonly port?: number;
229
- readonly simplexDbPrefix?: string;
230
- readonly openTransport?: () => Promise<SimplexTransport>;
231
- readonly onResponse?: (state: SimplexQueryState, response: PeerQueryResponse) => void;
232
- }
233
206
  export interface AutoRAGAgentOptions {
234
207
  model?: Model<Api>;
235
208
  apiKey?: string;
@@ -269,7 +242,6 @@ export interface AutoRAGAgentOptions {
269
242
  * challenges. A self-hosted
270
243
  * SEARXNG_ENDPOINT is the only env-gated option. `false` disables both
271
244
  * tools. The `fetch` sub-option tunes or disables `web_fetch` alone.
272
- * Web tools are always omitted for remote P2P sessions.
273
245
  */
274
246
  webSearch?: (WebSearchToolOptions & {
275
247
  fetch?: WebFetchToolOptions | false;
@@ -289,17 +261,22 @@ export interface AutoRAGAgentOptions {
289
261
  * probabilities out). When set, the two-phase search asks Jev before the
290
262
  * fast answer whether the question is answerable from intrinsic knowledge
291
263
  * (direct answer) or needs local search, and whether to decompose it; after the fast answer, whether
292
- * verification is needed. It also exposes the `jev` tool. Always omitted for
293
- * remote P2P sessions.
264
+ * verification is needed. It also exposes the `jev` tool.
294
265
  *
295
266
  * The CLI config enables this by default on OpenRouter (`buildAgentOptions`
296
267
  * fills it in). On this programmatic option, absent or `false` keeps Jev
297
268
  * off, so library callers never make paid network calls they did not ask for.
298
269
  */
299
270
  jev?: JevToolOptions | false;
271
+ /**
272
+ * The embedder shared by memory and the datasources. Omitted, it is built from `minSync.embedder`: a
273
+ * configured remote endpoint, else MinSync's `minsync serve` daemon, else MinSync's in-process engine.
274
+ * `refresh` brings it up (starting the daemon when none runs); a search only reads from it.
275
+ */
276
+ embedder?: ManagedEmbedder;
300
277
  /**
301
278
  * Embedder used to find similar past questions in memory (semantic half of
302
- * the hybrid search). Defaults to the local embedding gateway; `false` keeps
279
+ * the hybrid search). Defaults to the shared {@link AutoRAGAgentOptions.embedder}; `false` keeps
303
280
  * memory search keyword-only.
304
281
  */
305
282
  memoryEmbedder?: Embedder | false;
@@ -308,7 +285,7 @@ export interface AutoRAGAgentOptions {
308
285
  * branch for questions about AutoRAG's own settings: the turn skips the
309
286
  * fast answer, receives the full `autorag-setup` skill, edits
310
287
  * `configPath` with the pi tools, and reports what it changed as its final
311
- * message. Always omitted for remote P2P sessions.
288
+ * message.
312
289
  */
313
290
  selfConfig?: SelfConfigOptions;
314
291
  /**
@@ -347,12 +324,13 @@ export interface AutoRAGAgentOptions {
347
324
  */
348
325
  toolCallBurstGuard?: ToolBurstGuardOptions | false;
349
326
  /**
350
- * Outbound SimpleX queries to trusted peer AutoRAG agents. Remote sessions
351
- * never receive this tool.
327
+ * Control the interactive (TUI) session from Discord through the
328
+ * `pi-discord-remote` extension: a session channel is created on launch, and
329
+ * allowed users' messages run as questions; the final answer is posted back. Needs `~/.pi/agent/pi-discord-remote/config.json` (or
330
+ * `DISCORD_TOKEN` + `DISCORD_GUILD_ID`).
331
+ * Default false.
352
332
  */
353
- peerQuery?: PeerQueryOptions | false;
354
- /** Restrict the agent to retrieval and result-emission tools for remote runs. */
355
- remoteSession?: boolean;
333
+ discordRemote?: boolean;
356
334
  /** Per-phase thinking levels for the two-phase (fast → verification) search. */
357
335
  thinking?: AutoRAGThinkingOptions;
358
336
  /** pi agent directory for auth/models/extensions. Defaults to ~/.pi/agent. */
@@ -367,6 +345,8 @@ export interface AutoRAGAgentOptions {
367
345
  * a launch.
368
346
  */
369
347
  updateNotice?: () => Promise<string | undefined>;
348
+ /** Interactive only: show the AutoRAG welcome banner (with this version) instead of pi's startup header. */
349
+ welcomeVersion?: string;
370
350
  /** Extension entry files (installed pi packages) loaded into the interactive session. */
371
351
  piExtensionPaths?: readonly string[];
372
352
  /** Flag values for the flags those extensions register. */
@@ -401,7 +381,7 @@ export interface AutoRAGSearchSession {
401
381
  readonly piSession?: AutoRAGPiSession["session"];
402
382
  prompt(text: string): Promise<void>;
403
383
  abort(): Promise<void> | void;
404
- dispose(): void;
384
+ dispose(): void | Promise<void>;
405
385
  }
406
386
  export type AutoRAGJikjiPrepareResult = {
407
387
  readonly ok: true;
@@ -436,6 +416,8 @@ export declare class AutoRAGAgent {
436
416
  * state for `searchDocuments()` / `searchDocumentsStream()` callers.
437
417
  */
438
418
  private readonly interactive;
419
+ /** What the running search is doing, shown as a live widget in the TUI. */
420
+ private readonly searchProgress;
439
421
  /** Aborted by {@link abort} and by the run's timeout: gates publishing and the run's own requests. */
440
422
  private activeRunAbort;
441
423
  private modelNativeSearchAuth;
@@ -467,6 +449,8 @@ export declare class AutoRAGAgent {
467
449
  private readonly reranker;
468
450
  private readonly rerankTopN;
469
451
  private readonly minSyncMethod;
452
+ private readonly minSyncEmbedderConfig;
453
+ private readonly minSyncBinaryPath;
470
454
  private readonly jikjiClient;
471
455
  private readonly everythingClient;
472
456
  private readonly fsearchClient;
@@ -495,11 +479,13 @@ export declare class AutoRAGAgent {
495
479
  private readonly piSessionDir;
496
480
  private readonly persistPiSessions;
497
481
  private readonly updateNotice;
482
+ private readonly welcomeVersion;
498
483
  private readonly piExtensionPaths;
499
484
  private readonly piExtensionFlags;
500
485
  private readonly telegram;
501
486
  private boundPiRuntime;
502
487
  /** Embeds past questions for similar-question search; undefined keeps memory search keyword-only. */
488
+ private readonly embedder;
503
489
  private readonly memoryEmbedder;
504
490
  private readonly memoryVectorPath;
505
491
  /** Every retrieval result of the in-flight run with the query/method that surfaced it. */
@@ -510,8 +496,7 @@ export declare class AutoRAGAgent {
510
496
  private readonly conversationId;
511
497
  /** The memory context loaded for the in-flight run; cleared whenever a run starts. */
512
498
  private memoryContextCache;
513
- /** True when this agent was constructed for an untrusted remote peer. */
514
- readonly remoteSession: boolean;
499
+ private readonly discordRemote;
515
500
  private activeRetrievalOptions;
516
501
  private readonly toolBurstGuard;
517
502
  private readonly limits;
@@ -523,7 +508,7 @@ export declare class AutoRAGAgent {
523
508
  * this conversation, up to 25 similar past questions, and matching
524
509
  * long-term insights. Computed once per run (the datasource check and every
525
510
  * model turn share it) so the similar-question search and its embedder call
526
- * are not repeated and a fallback is reported once. Remote peers never read it.
511
+ * are not repeated and a fallback is reported once.
527
512
  */
528
513
  private loadMemoryContextFor;
529
514
  private withMemoryContext;
@@ -550,7 +535,7 @@ export declare class AutoRAGAgent {
550
535
  * After the final answer: map every cited evidence back to the search query
551
536
  * and method that found it, ask Jev (one batched call, evidence judged in
552
537
  * parallel) whether it really supports the answer sentence it backs, and
553
- * keep only what clears the threshold. Remote peers and runs without Jev
538
+ * keep only what clears the threshold. Runs without Jev
554
539
  * store nothing; a failed judgment stores nothing and says why.
555
540
  */
556
541
  private recordJudgedEvidence;
@@ -561,6 +546,12 @@ export declare class AutoRAGAgent {
561
546
  private currentSystemPromptConfig;
562
547
  subscribe(listener: Parameters<Agent["subscribe"]>[0]): () => void;
563
548
  createPiInteractiveRuntime(): Promise<AutoRAGPiInteractiveRuntime>;
549
+ /**
550
+ * Discord messages enter the same machinery as typed input: a new search when
551
+ * idle, a steer of the running one otherwise (queued as a follow-up question
552
+ * once the search can no longer take it).
553
+ */
554
+ private discordRemoteExtension;
564
555
  /**
565
556
  * The session's custom tools, with retrieval and web reads interruptible by a
566
557
  * steer: stopping them loses no side effect. bash and the file tools are pi
@@ -716,14 +707,20 @@ export declare class AutoRAGAgent {
716
707
  * `rerank.topN`. With decomposition the pool merges every search query's
717
708
  * hits (interleaved), capped at the same per-source sizes as a single query
718
709
  * so the rerank request does not grow with the query count. Returns
719
- * `undefined` when reranking is disabled/unavailable or the pool is empty,
720
- * so the caller keeps the unranked sections; a rerank failure never blocks
721
- * the fast answer.
710
+ * `undefined` when reranking is disabled or the pool is empty. A configured
711
+ * reranker that is unavailable, throws, or returns nothing for a non-empty
712
+ * pool throws, so the caller can report it and fall back.
722
713
  */
723
714
  private rerankPrefetchPool;
715
+ /**
716
+ * Surface a rerank failure the way a model error is surfaced: a diagnostic on
717
+ * the response and, in the TUI, a visible notice in the transcript. The run
718
+ * continues on the retrieval-order fallback.
719
+ */
720
+ private reportRerankFailure;
724
721
  /**
725
722
  * Fast-phase prompt for two-phase searches. The baseline retrieval context
726
- * is already gathered, so the model answers immediately with thinking off,
723
+ * is already gathered, so the model answers immediately with minimal thinking,
727
724
  * as plain text, without any further tool calls.
728
725
  */
729
726
  buildFastAnswerPrompt(query: string, options: RetrievalOptions, baseline: string): string;
@@ -756,6 +753,11 @@ export declare class AutoRAGAgent {
756
753
  buildRefinementPrompt(query: string, options: RetrievalOptions, fastAnswer: string | undefined, fastAnswerDelivered: boolean, steers?: readonly string[]): string;
757
754
  buildSearchPrompt(query: string, options: RetrievalOptions, initialRetrievalContext?: string): string;
758
755
  refresh(force?: boolean, opts?: AutoRAGRefreshOptions): Promise<AutoRAGRefreshResult>;
756
+ /**
757
+ * One warning when nothing can embed, so semantic search is not silently lexical-only. The text names the
758
+ * way out; AutoRAG does not start the daemon itself. Shown by `status`, `lite status`, and the MCP status tool.
759
+ */
760
+ private embeddingDiagnostics;
759
761
  /**
760
762
  * Path-opaque snapshot of corpus freshness and the last refresh outcome. Runs
761
763
  * a cheap parse-free staleness scan (stat only); never parses in this path.
@@ -10,6 +10,27 @@ export declare const ANSWER_IMAGE_EMBED_RULE = "**Images**: when a retrieved res
10
10
  * already showed.
11
11
  */
12
12
  export declare const ANSWER_IMAGE_DELTA_RULE = "An image already embedded in the first answer must not be embedded again.";
13
+ /**
14
+ * Answer readability rules. Every surface that tells the model how to write the
15
+ * answer (system prompt, fast/complete/delta/search phase prompts, report
16
+ * schema) carries the same wording, so the answer's shape does not depend on
17
+ * which phase produced it. Each rule yields to an explicit user request for a
18
+ * different format.
19
+ */
20
+ /**
21
+ * Conclusion first. Only complete answers get a lead line: a delta answer holds
22
+ * only corrections and new findings, and a lead conclusion would restate the
23
+ * first answer. The lead is outside the 5-bullet cap so the cap stays intact.
24
+ */
25
+ export declare const ANSWER_LEAD_RULE = "**Lead with the conclusion**: unless the user explicitly requests a different format, start the answer with one line that states the conclusion in **bold**, followed by the supporting bullet points. The lead line is not counted in the 5-bullet cap. When the whole answer is a single fact, the lead line alone is enough and no bullets are needed.";
26
+ /** One claim per bullet; flat list. */
27
+ export declare const ANSWER_BULLET_RULE = "**One claim per bullet**: each bullet makes one claim in one or two sentences. Never nest bullets; split a bullet that carries two claims into two bullets.";
28
+ /** Bold the values a reader scans for; tabulate real comparisons. */
29
+ export declare const ANSWER_EMPHASIS_RULE = "**Emphasis**: put the values a reader scans for (dates, amounts, names, statuses, deadlines) in **bold**, and nothing else. When comparing three or more items across the same attributes, use a markdown table instead of bullets.";
30
+ /** Answer language follows the question, not the sources. */
31
+ export declare const ANSWER_LANGUAGE_RULE = "**Answer language**: write the answer in the language of the user's question (a Korean question gets a Korean answer) even when the sources are in another language; if the user asks for a specific language, use that one. Keep proper nouns, identifiers, code, and terms quoted from sources in their original form.";
32
+ /** Uncertainty goes last so it never buries the conclusion. */
33
+ export declare const ANSWER_UNCERTAINTY_RULE = "**Uncertainty placement**: when the answer is incomplete or uncertain, state that in one short sentence on the last line of the answer, never before or ahead of the conclusion. Do not hedge the conclusion itself.";
13
34
  /**
14
35
  * Inline evidence citations. The model writes the id the retrieval tools print
15
36
  * next to each result; the harness resolves it to the recorded source and
@@ -27,7 +48,7 @@ export declare const ANSWER_CITATION_RULE = "**Citations**: cite supporting evid
27
48
  * concise by stating the created file's location and briefly summarizing what
28
49
  * was generated (citing only the underlying source evidence).
29
50
  */
30
- export declare const ANSWER_DELIVERABLE_RULE = "**Deliverables and artifacts**: if the user asks you to draft, write, generate, or prepare a deliverable or artifact (a reply email, letter, legal opinion, summary document, standard-format case citation block, checklist, report, or workflow document \u2014 including requests phrased as \"how should I reply...\" to a specific message), produce the finished, ready-to-use artifact as a file (using `write`, `edit`, or `bash` in local sessions, without overwriting existing source files) in a file format, extension, and structure aligned with the context and domain (e.g. `.eml` or `.md`/`.txt` with headers for an email reply; `.md`, `.txt`, `.docx`, `.hwp`, or `.pdf` matching workspace conventions for a legal opinion, insertable case-citation block, or summary document; `.csv`, `.xlsx`, or `.json` for structured tabular data), containing the complete unabridged artifact (including required headers, precedent passages, pinpoint paragraphs, and standard-format citations) inside the file instead of merely describing how to produce it. Do NOT dump the full artifact text into your reply: unless the user explicitly requests otherwise, keep your reply concise within the at-most-5-bullet direct-answer format, clearly stating the created file's exact path/location and briefly summarizing what was generated with inline citations to the underlying source evidence (`[eN]` or `[file:<source path>]`); never cite the newly created deliverable file itself as `[file:...]`.";
51
+ export declare const ANSWER_DELIVERABLE_RULE = "**Deliverables and artifacts**: if the user asks you to draft, write, generate, or prepare a deliverable or artifact (a reply email, letter, legal opinion, summary document, standard-format case citation block, checklist, report, or workflow document \u2014 including requests phrased as \"how should I reply...\" to a specific message), produce the finished, ready-to-use artifact as a file (using `write`, `edit`, or `bash` in local sessions) in a file format, extension, and structure aligned with the context and domain (e.g. `.eml` or `.md`/`.txt` with headers for an email reply; `.md`, `.txt`, `.docx`, `.hwp`, or `.pdf` matching workspace conventions for a legal opinion, insertable case-citation block, or summary document; `.csv`, `.xlsx`, or `.json` for structured tabular data), containing the complete unabridged artifact (including required headers, precedent passages, pinpoint paragraphs, and standard-format citations) inside the file instead of merely describing how to produce it. Do NOT dump the full artifact text into your reply: unless the user explicitly requests otherwise, keep your reply concise within the at-most-5-bullet direct-answer format, clearly stating the created file's exact path/location and briefly summarizing what was generated with inline citations to the underlying source evidence (`[eN]` or `[file:<source path>]`); never cite the newly created deliverable file itself as `[file:...]`.";
31
52
  /**
32
53
  * Verification-phase deliverable rule: create the context-aligned deliverable
33
54
  * file (in local sessions) and report its location and concise summary if the