@autorag/librarian 2.10.4 → 2.10.5

package/NOTICE

@@ -18,6 +18,9 @@ License: MIT
 @earendil-works/pi-tui@1.2.0
 License: MIT
 
+@llblab/pi-telegram@0.57.2
+License: MIT
+
 @modelcontextprotocol/server@2.3.1
 License: Apache-2.0
 

package/README.md

@@ -153,6 +153,7 @@ The app is one way in. The same librarian runs everywhere else:
 |---|---|---|
 | **Desktop app (Dazzi)** | Browsing and asking in one window | [NomaDamas/AutoRAG-app](https://github.com/NomaDamas/AutoRAG-app) (Fair Source, FSL-1.1-MIT) |
 | **Terminal UI** | Interactive, steerable search sessions | [`autorag tui`](#2-search-from-cli) |
+| **Messenger** | Asking the librarian from a chat app | [Telegram](#messenger-connections) |
 | **CLI** | Scripts, indexing, health checks | [CLI Command Reference](#cli-command-reference) |
 | **MCP server** | Feeding retrieval to Claude Code, Codex, and other agents | [AutoRAG Lite MCP Server](#autorag-lite-mcp-server) |
 | **TypeScript API** | Embedding the librarian in your own app | [Programmatic usage](#3-programmatic-usage-typescript-api) |
@@ -199,7 +200,7 @@ AutoRAG Agent orchestrates five integrated subsystems:
 
 1. **Retrieval Memory (`check_memory`):** After each answer, cited evidence Jev judges to genuinely support it is stored in `~/.autorag/memory.json`. Later searches receive this conversation's evidence, similar past questions, and long-term insights as **advisory reference** — it never reorders results or overrides current evidence. See [docs/retrieval-memory.md](docs/retrieval-memory.md).
 2. **Pluggable Multi-Method Retrieval:**
-   - **BM25 Lexical Search:** Fast keyword ranking via MinSync over parsed markdown mirrors.
+   - **BM25 Lexical Search:** Fast keyword ranking via MinSync over parsed markdown mirrors. AutoRAG always uses MinSync's `multilingual` tokenizer, which keeps each original word and adds Korean morphemes (via the [Kiwi](https://github.com/bab2min/Kiwi) library), so both `연구목표` and `인증서를` → `인증서` match; without Kiwi AutoRAG warns and tokenizes on whitespace. See [docs/minsync-setup.md](docs/minsync-setup.md#bm25-tokenizer).
    - **Semantic Vector Search:** Dense vector retrieval over CDC chunks via the built-in embedding gateway.
    - **Hybrid Search:** Combines BM25 and vector ranking via Reciprocal Rank Fusion (RRF).
    - **Jikji Find-First Discovery:** Local CLI-backed fast discovery answer packs.
@@ -222,6 +223,18 @@ Update notices are AutoRAG's, not Pi's: `autorag tui` suppresses Pi's "run `pi u
 
 ---
 
+## Messenger Connections
+
+| Messenger | Status | Start here |
+|---|---|---|
+| **Telegram** | Supported | [`autorag tui --telegram`](docs/telegram.md) |
+| **WhatsApp** | Supported (unofficial client) | [`autorag tui --whatsapp`](docs/whatsapp.md) |
+| Discord, Slack, others | Not supported yet | |
+
+Telegram runs through the [`@llblab/pi-telegram`](https://github.com/llblab/pi-telegram) bridge inside a running `autorag tui` session: questions you send the bot get the same routing and two-phase search as the terminal. Setup, commands, and keeping it running are in [docs/telegram.md](docs/telegram.md).
+
+WhatsApp runs through the pinned [`whatsapp-pi`](https://www.npmjs.com/package/whatsapp-pi) extension the same way; see [docs/whatsapp.md](docs/whatsapp.md).
+
 ## Installation & Setup
 
 AutoRAG Agent is published as `@autorag/librarian` (requires Node.js ≥ 24 or Bun):
@@ -454,7 +467,7 @@ mirrored are kept and not re-parsed on the next run. `onProgress` reports `parse
 | `autorag health` | Check model provider authentication, token validity, and API reachability |
 | `autorag models list` | List chat models the pi runtime can resolve (built-ins, `models.json`, custom/extension providers) with provider auth status; never prints credential values |
 | `autorag update-check` | Compare the running `autorag` against the published npm version (also runs on `autorag tui` launch) |
-| `autorag tui` | Open Pi's interactive librarian TUI (`/login`, `/model`, `/resume`, …) |
+| `autorag tui` | Open Pi's interactive librarian TUI (`/login`, `/model`, `/resume`, …); `--telegram` / `--whatsapp` add the Telegram / WhatsApp remote-control bridges |
 | `autorag duplicates [DIR]` | Read-only scan for exact and near-duplicate document families with `dupey` |
 | `autorag lite ...` | CLI bootstrap, indexing repair, terminal maintenance, and fallback interface; agents use MCP for normal Lite operation |
 | `autorag memory inspect` | Render a read-only, path-opaque snapshot of retrieval memory (judged evidence, curated results, evidence chunks, insights); model-free |
@@ -675,6 +688,7 @@ Deep dive into AutoRAG Agent's architecture, security, and integration guides:
 - **[MinSync Setup & Embedding QA](docs/minsync-setup.md):** Automatic binary installation, CDC chunking, and local embedding verification.
 - **[Local Embedding Runtime & Gateway](docs/embedding-runtime.md):** AutoRAG-owned local gateway, model prefetching, and zero-egress semantic search. Model card: [`qwen3-embedding-0.6b`](docs/model-cards/qwen3-embedding-0.6b.md).
 - **[Hosted Dazzi Models Provider](docs/cloud-provider.md):** The hosted `dazzi` model provider — PKCE `/login`, `AUTORAG_BASE_URL`, live model discovery, and offline catalog snapshots.
+- **[Telegram Remote Control](docs/telegram.md):** Ask the librarian from a Telegram chat through a running `autorag tui --telegram` session: setup, commands, and keeping it alive.
 - **[Datasource Skills Reference](docs/datasource-skills.md):** Full configuration contracts, connection aliases, and connector options.
 - **[Jev Decisions](docs/jev-decisions.md):** Jev query pipeline, **on by default** via Dazzi Models when signed in (OpenRouter otherwise): it routes each question to a direct answer (intrinsic knowledge) or local search (everything needing more information; web search is left to the agent after the fast answer), splits multi-part questions into up to five parallel searches (`dazzi/openai/gpt-6-luna` when signed in, else `openrouter/qwen/qwen3.7-flash`), picks which registered datasources to search before the fast answer, and ends the run after the fast answer when that answer is complete. Also covers the `jev` judgment tool, backends (OpenRouter, TypeSafe, Vercel AI Gateway), and how to opt out.
 - **[Manual QA & Datasource Test Harnesses](docs/manual-qa-datasources.md):** Real-world testing guides for Discord, KakaoTalk, Slack, Notion, and email.

package/dist/agent/agent.d.ts

@@ -263,9 +263,10 @@ export interface AutoRAGAgentOptions {
      * provider-chain web module. Default enabled and credential-free: the
      * chain leads with providers that need no user-issued key — model-native
      * search reusing the agent's own model credentials (Gemini grounding,
-     * Anthropic/OpenAI/xAI web_search), the anonymous Perplexity ask
-     * endpoint, and Parallel's keyless MCP — then scraped engines with
-     * headless-browser escalation for bot challenges. A self-hosted
+     * Anthropic/OpenAI/xAI web_search), Parallel's keyless MCP, and the
+     * DuckDuckGo HTML form — then the anonymous Perplexity ask endpoint and
+     * the other scraped engines with headless-browser escalation for bot
+     * challenges. A self-hosted
      * SEARXNG_ENDPOINT is the only env-gated option. `false` disables both
      * tools. The `fetch` sub-option tunes or disables `web_fetch` alone.
      * Web tools are always omitted for remote P2P sessions.
@@ -366,6 +367,17 @@ export interface AutoRAGAgentOptions {
      * a launch.
      */
     updateNotice?: () => Promise<string | undefined>;
+    /** Extension entry files (installed pi packages) loaded into the interactive session. */
+    piExtensionPaths?: readonly string[];
+    /** Flag values for the flags those extensions register. */
+    piExtensionFlags?: Readonly<Record<string, string | boolean>>;
+    /**
+     * Load the `@llblab/pi-telegram` bridge into the interactive session so a
+     * paired Telegram chat can start, steer, and read answers from it. The
+     * bridge adds `/telegram-setup` and `/telegram-connect`; its bot token and
+     * owner pairing live in pi's `telegram.json`, never in AutoRAG config.
+     */
+    telegram?: boolean;
 }
 /** Post-merge reranking options. Mirrors the CLI `RerankConfig` (secrets via env). */
 export interface RerankAgentOptions {
@@ -483,6 +495,9 @@ export declare class AutoRAGAgent {
     private readonly piSessionDir;
     private readonly persistPiSessions;
     private readonly updateNotice;
+    private readonly piExtensionPaths;
+    private readonly piExtensionFlags;
+    private readonly telegram;
     private boundPiRuntime;
     /** Embeds past questions for similar-question search; undefined keeps memory search keyword-only. */
     private readonly memoryEmbedder;

package/dist/agent/pi-session.d.ts

@@ -25,6 +25,10 @@ export interface AutoRAGPiSessionOptions {
      * extension. Each factory registers its tools, commands, and events with pi.
      */
     readonly extensionFactories?: readonly ExtensionFactory[];
+    /** Extension entry files (e.g. an installed pi package's extension) loaded into the interactive session. */
+    readonly extensionPaths?: readonly string[];
+    /** CLI-style flag values for the flags those extensions register (`--name` / `--name value`). */
+    readonly extensionFlags?: Readonly<Record<string, string | boolean>>;
     /**
      * Tool names registered by {@link extensionFactories}. pi's `tools`
      * allow-list keeps only named tools active, so every extension tool that
@@ -86,6 +90,23 @@ export interface AutoRAGPiSession {
 }
 /** Adapts a pi-agent-core {@link AgentTool} to pi's extension {@link ToolDefinition}. */
 export declare function toToolDefinition(tool: AgentTool): ToolDefinition;
+/**
+ * Prefix `@llblab/pi-telegram` puts on every prompt it injects from a Telegram
+ * chat. The bridge hands them to pi through `pi.sendUserMessage`, so pi tags
+ * them `source: "extension"`, the same source AutoRAG's own phase prompts use.
+ * The prefix is what tells a person typing on a phone from AutoRAG talking to
+ * itself.
+ */
+export declare const TELEGRAM_INPUT_PREFIX = "[telegram]";
+/** `whatsapp-pi` frames every inbound chat message as `Message from <name> (<number>)`. */
+export declare const WHATSAPP_INPUT_PREFIX = "Message from ";
+/**
+ * The user's own words from a bridged Telegram prompt: the bridge's `[telegram]`
+ * prefix and its `[time]` line are envelope, not the question, so a Telegram
+ * question reaches the librarian exactly as the same text typed in the TUI.
+ * Attachment blocks stay: they are part of what the user sent.
+ */
+export declare function stripTelegramEnvelope(text: string): string;
 /** Creates a pi session with AutoRAG's tools and prompt. */
 export declare function createAutoRAGPiSession(options: AutoRAGPiSessionOptions): Promise<AutoRAGPiSession>;
 export interface AutoRAGPiInteractiveRuntime {

package/dist/agent/search-documents.d.ts

@@ -9,7 +9,7 @@ export type SearchDocumentDiagnosticSeverity = "info" | "warning" | "error";
  * (#21/#22) does not require another source-breaking type change; codes not yet
  * emitted are still valid members.
  */
-export type SearchDocumentDiagnosticCode = "empty-query" | "no-verified-results" | "unknown-warning" | "caller-tool-dropped" | "minsync-staging-excluded" | "minsync-file-skipped" | "minsync-sync-failed" | "minsync-lock-held" | "embedder-unavailable" | "embedding-identity-mismatch" | "parser-skipped" | "parser-failed" | "parser-crashed" | "source-vanished" | "source-unreadable" | "source-read-failed" | "mirror-write-failed" | "parser-warning" | "duplicate-excluded" | "unsupported-file" | "stale-index" | "deleted-mirror" | "search-root-unavailable" | "mass-delete-suppressed" | "walk-truncated" | "directory-unreadable" | "pdf-extract-thin" | "retrieval-method-failed" | "rerank-failed" | "jikji-unavailable" | "jikji-prepare-failed" | "jikji-find-failed" | "everything-index-failed" | "fsearch-index-failed" | "fsearch-binary-missing" | "refresh-failed" | "refresh-interrupted" | "watch-failed" | "watch-limited" | "unknown-datasource-skill" | "no-final-answer" | "model-request-failed" | "tool-call-burst" | "query-routed" | "query-route-fallback" | "query-decomposition-failed" | "follow-up-skipped" | "follow-up-check-fallback" | "datasources-selected" | "datasource-selection-fallback" | "citation-without-result" | "evidence-judged" | "evidence-judgment-fallback" | "memory-search-fallback" | "self-config-unavailable" | "self-config-rolled-back";
+export type SearchDocumentDiagnosticCode = "empty-query" | "no-verified-results" | "unknown-warning" | "caller-tool-dropped" | "minsync-staging-excluded" | "minsync-file-skipped" | "minsync-sync-failed" | "minsync-lock-held" | "embedder-unavailable" | "embedding-identity-mismatch" | "lexical-tokenizer-unavailable" | "parser-skipped" | "parser-failed" | "parser-crashed" | "source-vanished" | "source-unreadable" | "source-read-failed" | "mirror-write-failed" | "parser-warning" | "duplicate-excluded" | "unsupported-file" | "stale-index" | "deleted-mirror" | "search-root-unavailable" | "mass-delete-suppressed" | "walk-truncated" | "directory-unreadable" | "pdf-extract-thin" | "retrieval-method-failed" | "rerank-failed" | "jikji-unavailable" | "jikji-prepare-failed" | "jikji-find-failed" | "everything-index-failed" | "fsearch-index-failed" | "fsearch-binary-missing" | "refresh-failed" | "refresh-interrupted" | "watch-failed" | "watch-limited" | "unknown-datasource-skill" | "no-final-answer" | "model-request-failed" | "tool-call-burst" | "query-routed" | "query-route-fallback" | "query-decomposition-failed" | "follow-up-skipped" | "follow-up-check-fallback" | "datasources-selected" | "datasource-selection-fallback" | "citation-without-result" | "evidence-judged" | "evidence-judgment-fallback" | "memory-search-fallback" | "self-config-unavailable" | "self-config-rolled-back";
 export interface SearchDocumentDiagnostic {
     readonly code: SearchDocumentDiagnosticCode;
     readonly severity: SearchDocumentDiagnosticSeverity;

package/dist/agent/web-search-tool.d.ts

@@ -3,8 +3,8 @@
  *
  * LLM-facing wrapper around the oh-my-pi-style provider chain in
  * `src/web/search/`: credential-free end to end (model-native search on the
- * agent's own model credential, the anonymous Perplexity ask endpoint,
- * Parallel's keyless MCP, then the scraped engines), where quota/auth/bot
+ * agent's own model credential, Parallel's keyless MCP, DuckDuckGo, then
+ * the anonymous Perplexity ask endpoint and the other scraped engines), where quota/auth/bot
  * failures automatically fall back down the chain. The model only supplies
  * `query` plus optional hints; provider selection and credentials are never
  * exposed through tool arguments.