pi-better-web-search 0.1.11 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +154 -0
- package/README.md +269 -66
- package/SECURITY.md +31 -4
- package/assets/architecture.svg +43 -21
- package/assets/search-index.png +0 -0
- package/docs/README.zh-CN.md +224 -50
- package/docs/architecture.md +151 -43
- package/docs/providers.md +52 -9
- package/docs/publishing.md +9 -8
- package/docs/search-apis.md +250 -0
- package/docs/search-skills.md +265 -0
- package/docs/testing.md +86 -11
- package/extensions/index.ts +9 -0
- package/lib/api-command.ts +84 -0
- package/lib/atomic-file.ts +32 -0
- package/lib/attempt-timeout.ts +82 -0
- package/lib/auth.ts +9 -0
- package/lib/cache.ts +24 -16
- package/lib/candidates.ts +137 -112
- package/lib/credentials.ts +98 -0
- package/lib/dedicated-config.ts +61 -0
- package/lib/format.ts +14 -0
- package/lib/health.ts +24 -5
- package/lib/login-command.ts +87 -0
- package/lib/preferences.ts +62 -0
- package/lib/provider-command.ts +66 -0
- package/lib/providers/anthropic.ts +7 -4
- package/lib/providers/dedicated-http.ts +80 -0
- package/lib/providers/deepseek.ts +77 -10
- package/lib/providers/exa.ts +66 -0
- package/lib/providers/google.ts +4 -3
- package/lib/providers/http.ts +99 -47
- package/lib/providers/index.ts +9 -0
- package/lib/providers/keenable.ts +61 -0
- package/lib/providers/ollama.ts +13 -6
- package/lib/providers/openai.ts +6 -6
- package/lib/providers/perplexity.ts +20 -0
- package/lib/providers/sse.ts +93 -54
- package/lib/providers/types.ts +6 -2
- package/lib/redact.ts +15 -0
- package/lib/render.ts +41 -0
- package/lib/routes.ts +28 -0
- package/lib/schema.ts +26 -10
- package/lib/search.ts +124 -64
- package/lib/secret-input.ts +63 -0
- package/package.json +8 -2
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,159 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 1.0.0 — Search is an agent superpower
|
|
4
|
+
|
|
5
|
+
### The flagship search upgrade
|
|
6
|
+
|
|
7
|
+
- Bring **Keenable, Exa, and Perplexity**, the top three independent providers
|
|
8
|
+
in the Artificial Analysis Search Index snapshot, plus **DeepSeek native
|
|
9
|
+
Search/Fetch**, directly into pi without requiring a conversation-model change.
|
|
10
|
+
- Lead the English/Chinese READMEs with the search advantage: **33 model-only →
|
|
11
|
+
83 with Keenable Pro (+50 points)**. Show why agents should find existing
|
|
12
|
+
libraries, official docs, and proven fixes before spending tokens reinventing
|
|
13
|
+
them. Add a sourced Search Index chart and redraw the architecture for four
|
|
14
|
+
standalone APIs, native fallback, terminal login, and the 96k output guard.
|
|
15
|
+
|
|
16
|
+
### Regression fixes
|
|
17
|
+
|
|
18
|
+
- Stop OpenAI/xAI streams on completed/done/incomplete terminal events and
|
|
19
|
+
Anthropic streams on `message_stop`, cancelling connections that remain open
|
|
20
|
+
instead of timing out and discarding usable search results.
|
|
21
|
+
- Redact native-provider transport, SSE and JSON errors using a snapshot of sent
|
|
22
|
+
credential headers (including custom API tokens and Bearer/Basic auth). Keep
|
|
23
|
+
status/code metadata for fallback, scrub structured error bodies, and suppress
|
|
24
|
+
JSON parser snippets that could expose a partial credential.
|
|
25
|
+
- Require Anthropic server-tool evidence to name `web_search`; unrelated tools
|
|
26
|
+
no longer produce false search success. Skip non-object stream frames safely.
|
|
27
|
+
- Atomically replace settings and the success cache after writing/syncing a
|
|
28
|
+
same-directory temporary file. Partial-write and rename failures preserve the
|
|
29
|
+
previous file; configured file symlinks continue to update their targets.
|
|
30
|
+
- Recheck Keenable public-access consent before each adapter request, including
|
|
31
|
+
combined calls and URL batches. Key loss/logout no longer downgrades an
|
|
32
|
+
in-flight operation to public endpoints; preserve completed search/pages and
|
|
33
|
+
resolve auth-dependent endpoint paths from the same key snapshot as headers.
|
|
34
|
+
- Recognize aborts by signal, structured identity or canonical legacy messages,
|
|
35
|
+
not arbitrary text containing `abort`. Malformed page responses no longer
|
|
36
|
+
discard completed URLs, and dedicated transport AbortErrors remain redacted
|
|
37
|
+
even if credentials rotate while the request is in flight.
|
|
38
|
+
- Cancel stalled response bodies and responses arriving after an attempt timeout,
|
|
39
|
+
even when a custom transport ignores the signal. Ignore late adapter updates
|
|
40
|
+
after timeout/completion and prevent disposed deadlines from being restarted.
|
|
41
|
+
- Bound individual SSE frames to 4,194,304 UTF-16 code units, including unfinished
|
|
42
|
+
lines and multiline data, while accepting larger streams of bounded frames. Scan
|
|
43
|
+
fragmented lines incrementally instead of repeatedly rescanning their prefix.
|
|
44
|
+
- Share literal/JSON-escaped credential redaction across dedicated API errors,
|
|
45
|
+
native DeepSeek errors and Exa per-URL errors. Also redact HTTP status text and
|
|
46
|
+
structured error codes using the credentials actually sent.
|
|
47
|
+
- Keep routing priorities strict: **manual preference → Keenable → Exa →
|
|
48
|
+
Perplexity → standalone DeepSeek → native automatic routing**. Manual choices
|
|
49
|
+
get a first probe even after failure or Retry-After; cooldowns reorder only
|
|
50
|
+
remaining automatic model routes. Standalone DeepSeek failures do not suppress
|
|
51
|
+
native DeepSeek candidates using separate authentication.
|
|
52
|
+
- Route standalone DeepSeek through shared dedicated HTTP validation/backoff:
|
|
53
|
+
honor credential-specific `Retry-After` across Search/Fetch, reject malformed
|
|
54
|
+
header keys before HTTP, and leave native model/proxy auth unchanged.
|
|
55
|
+
- Redact Exa per-URL errors using the key actually sent even if login rotates it
|
|
56
|
+
in flight; honor canonical-URL error statuses that use `url` instead of `id`.
|
|
57
|
+
- Release bounded HTTP error streams without awaiting an underlying cancellation
|
|
58
|
+
promise that may never settle, so errors can reach fallback immediately.
|
|
59
|
+
- Isolate the live DeepSeek test's settings/cache and disable standalone API
|
|
60
|
+
discovery so other saved/environment search keys cannot trigger unrelated
|
|
61
|
+
provider calls or override the intended native DeepSeek fallback check.
|
|
62
|
+
|
|
63
|
+
### Implementation details
|
|
64
|
+
|
|
65
|
+
- Explicitly provide no skills: remove vendor skill copies and their lockfile
|
|
66
|
+
from the checkout, retain only developer reference notes, and reject skill
|
|
67
|
+
directories/entrypoints in package verification. Keep normal tool descriptions,
|
|
68
|
+
schemas and API behavior unchanged; no automatic vendor skill context is added.
|
|
69
|
+
- Refine both description variants to address insufficient or conflicting
|
|
70
|
+
evidence and cite the source URLs actually used; make fetch conditional on
|
|
71
|
+
need in the URL-capable version, without adding fetch to search-only guidance.
|
|
72
|
+
- Improve capability-aware tool/field descriptions with natural-language
|
|
73
|
+
query guidance, source verification/citation, and optional evidence-driven
|
|
74
|
+
search refinement. Both variants start with the provider-neutral
|
|
75
|
+
`Search the live web and return the most relevant URLs and content.`; only
|
|
76
|
+
URL-capable variants instruct fetching pages. Keep `query` / `urls`, existing validation, adapter requests
|
|
77
|
+
and system prompts unchanged. Document skill wording, provider-specific length
|
|
78
|
+
limits and agent-driven chains without adding a fixed search loop.
|
|
79
|
+
- Add `/pi-better-web-search login`: select DeepSeek, Exa, Keenable or Perplexity
|
|
80
|
+
and enter an API key in a masked local terminal dialog. Persist keys separately
|
|
81
|
+
with restricted permissions, cross-process locking and atomic replacement;
|
|
82
|
+
refresh tool exposure immediately without environment setup or a chat model.
|
|
83
|
+
Saved keys override environment keys; preserve manual routing and fallback.
|
|
84
|
+
- Add safe presence-only `status` and confirmed `logout` (disables standalone
|
|
85
|
+
access without silently switching Keenable to public mode). Refuse RPC/print
|
|
86
|
+
secret entry and key arguments; preserve cancellation and existing credentials
|
|
87
|
+
on save failures. Credential files are local plaintext, not encrypted.
|
|
88
|
+
- Add standalone Exa Search/Fetch with `EXA_API_KEY` and `/web-search-api exa`.
|
|
89
|
+
Search sends the approved `numResults: 10` and `contents.highlights: true`;
|
|
90
|
+
known URLs use `POST /contents` with top-level `text: true`. Inspect per-URL
|
|
91
|
+
statuses even on HTTP 200, preserve successful raw documents, and retry only
|
|
92
|
+
unresolved URLs through existing fallback. Keep safe error redaction,
|
|
93
|
+
cancellation, Retry-After, API priority and the `query` / `urls` contract.
|
|
94
|
+
Document Exa/Keenable skill differences; objective and other advanced options
|
|
95
|
+
remain off; no Exa-specific parameter or system-prompt change is introduced.
|
|
96
|
+
- Add standalone Perplexity Search (`POST /search`, `search_type: web`, never
|
|
97
|
+
fast, no model/effort, no Fetch) and Keenable pro Search/live GET Fetch.
|
|
98
|
+
Preserve raw result objects and use existing timeout, cooldown, fallback and
|
|
99
|
+
partial-success handling. Configured APIs precede automatic model routing;
|
|
100
|
+
explicit per-phase provider preferences still win.
|
|
101
|
+
- Add `/web-search-api [deepseek|exa|keenable|perplexity]` to enable/disable APIs,
|
|
102
|
+
view safe key setup, and opt into Keenable public access with privacy consent.
|
|
103
|
+
This menu saves toggles only; masked login stores credentials separately. Honor Retry-After
|
|
104
|
+
across dedicated search/fetch calls, and add an opt-in status/shape-only smoke
|
|
105
|
+
test: `npm run test:live:search-apis -- all`. No new runtime dependency or MCP.
|
|
106
|
+
- Preserve successful search and URL-fetch data independently. Missing fetch
|
|
107
|
+
capability or fetch failures no longer replace successful search output with
|
|
108
|
+
a whole-call error; failed search (including query-length rejection) no longer
|
|
109
|
+
suppresses requested fetch work. Return partial-success JSON with `search.error`
|
|
110
|
+
and/or per-URL errors, and mark the whole call failed only if no operation
|
|
111
|
+
succeeded. Keep partial search failures visible in collapsed terminal results.
|
|
112
|
+
- Add `/web-search-provider [search|fetch]` terminal pickers with independent,
|
|
113
|
+
persistent search/fetch preferences and an Auto reset. Choices precede
|
|
114
|
+
automatic hints without disabling fallback, per-URL retries, or health
|
|
115
|
+
cooldowns; success-cache updates never overwrite explicit preferences.
|
|
116
|
+
|
|
117
|
+
- Clarify that providing `query` triggers a separate search even with `urls`;
|
|
118
|
+
show URL-only omission guidance only when URL support is exposed. Reject
|
|
119
|
+
additional input arguments in both schemas and at runtime, require non-empty
|
|
120
|
+
URL arrays when supplied, and constrain URL strings to 1–8192 Unicode code
|
|
121
|
+
points consistently with native DeepSeek validation.
|
|
122
|
+
- Strictly limit search and URL analysis candidates to OpenAI `luna`, Claude
|
|
123
|
+
`haiku`, and other brands' `flash` models. Select the newest numeric version
|
|
124
|
+
within the eligible brand/tier, including gateways; current-model and cache
|
|
125
|
+
hints now select a provider instead of pinning an older or larger model.
|
|
126
|
+
Never fall back to an excluded tier, and hide the tool without an eligible
|
|
127
|
+
model or configured standalone API.
|
|
128
|
+
- Compare dotted/dashed Claude minor versions numerically and ignore date
|
|
129
|
+
suffixes independently of the provider API family.
|
|
130
|
+
- Distinguish Search, Fetch, and combined operations in the terminal. Display
|
|
131
|
+
the actual DeepSeek/Ollama standalone API endpoint instead of an inference
|
|
132
|
+
model name; add `details.routes` and progress `details.activeRoute` metadata.
|
|
133
|
+
Raw provider output is unchanged and available in the expanded result.
|
|
134
|
+
- Preserve current-provider priority even when the conversation model's API is
|
|
135
|
+
unsupported, and derive discovery/cache fingerprints from one registry snapshot.
|
|
136
|
+
- Preserve completed DeepSeek/Ollama URL results after a later global HTTP error;
|
|
137
|
+
only unresolved URLs proceed to fallback. Show failed URL counts and truncation
|
|
138
|
+
warnings in collapsed results.
|
|
139
|
+
- Validate timeout overrides against Node's integer timer range and observe
|
|
140
|
+
adapter rejections even when the signal is already aborted.
|
|
141
|
+
- Handle SSE CR/LF/CRLF boundaries, data-less event resets, `[DONE]`, and pending
|
|
142
|
+
read cancellation without waiting for the provider connection to close.
|
|
143
|
+
|
|
144
|
+
### Regression coverage and validation
|
|
145
|
+
|
|
146
|
+
- Add 62 regression tests across response/deadline lifecycle, late adapter
|
|
147
|
+
updates, bounded/fragmented SSE frames, credential redaction and rotation,
|
|
148
|
+
atomic settings/cache writes, runtime Keenable consent, abort classification,
|
|
149
|
+
native terminal events and search-evidence validation.
|
|
150
|
+
- Add four platform-independent simulated symlink tests for settings/cache target
|
|
151
|
+
updates and dangling-link refusal. Keep the real filesystem checks separately.
|
|
152
|
+
- Verify on Node 24: 386 tests total, 384 passed, no failures; two real file-symlink
|
|
153
|
+
tests skipped because Windows returns `EPERM` when creating file symlinks.
|
|
154
|
+
- Type checking and package verification pass. Provider requests are mocked;
|
|
155
|
+
these checks do not call billable live APIs.
|
|
156
|
+
|
|
3
157
|
## 0.1.11
|
|
4
158
|
|
|
5
159
|
- Increase the default timeout to 120 seconds per search and URL fetch attempt
|
package/README.md
CHANGED
|
@@ -3,9 +3,9 @@
|
|
|
3
3
|
**One tool. Every provider. Raw results.**
|
|
4
4
|
|
|
5
5
|
A [pi](https://pi.dev) extension that adds exactly one model-callable tool:
|
|
6
|
-
`web_search`.
|
|
7
|
-
|
|
8
|
-
|
|
6
|
+
`web_search`. Search with **Keenable, Exa, Perplexity, or DeepSeek**, fetch URLs
|
|
7
|
+
when supported, and automatically fall back across authenticated native
|
|
8
|
+
providers. Keep your conversation model; get provider results as-is.
|
|
9
9
|
|
|
10
10
|
[](https://github.com/zhexusun10/pi-better-web-search/actions/workflows/ci.yml)
|
|
11
11
|
[](https://www.npmjs.com/package/pi-better-web-search)
|
|
@@ -13,23 +13,80 @@ and returns provider data as-is.
|
|
|
13
13
|
[](https://pi.dev)
|
|
14
14
|
[](https://nodejs.org)
|
|
15
15
|
|
|
16
|
-
[简体中文](docs/README.zh-CN.md) · [
|
|
16
|
+
[简体中文](docs/README.zh-CN.md) · [Search APIs](docs/search-apis.md) · [Architecture](docs/architecture.md) · [Providers](docs/providers.md) · [Testing](docs/testing.md)
|
|
17
17
|
|
|
18
|
-
|
|
18
|
+
## How it works
|
|
19
19
|
|
|
20
|
-
|
|
21
|
-
|
|
20
|
+

|
|
21
|
+
|
|
22
|
+
Your agent calls `web_search` with a query, URLs, or both. The router honors
|
|
23
|
+
per-phase provider preferences, tries configured search APIs, and falls back to
|
|
24
|
+
eligible native providers. Search and fetch keep their successful results
|
|
25
|
+
independently; raw provider data goes straight back to the agent.
|
|
26
|
+
|
|
27
|
+
## Search is an agent superpower
|
|
28
|
+
|
|
29
|
+
**33 without search. 83 with Keenable Pro. A 50-point leap.**
|
|
30
|
+
|
|
31
|
+
The Artificial Analysis Search Index makes the difference visible: the dashed
|
|
32
|
+
**model-only baseline is 33**, while **Keenable Pro reaches 83**, with Exa and
|
|
33
|
+
Perplexity at **80**. All three independent providers score ahead of the
|
|
34
|
+
integrated **OpenAI Web Search result at 74** in this snapshot.
|
|
35
|
+
|
|
36
|
+

|
|
37
|
+
|
|
38
|
+
*Source: [Artificial Analysis Search Index](https://artificialanalysis.ai/agents/search-api),
|
|
39
|
+
15 Nov 2026 screenshot. Scores combine DeepSearchQA F1, BrowseComp accuracy and
|
|
40
|
+
AA-Omniscience accuracy. [Methodology](https://artificialanalysis.ai/methodology/search-api).*
|
|
41
|
+
|
|
42
|
+
For a coding agent, the web is working memory at internet scale: official docs,
|
|
43
|
+
existing libraries, battle-tested implementations, migration guides, and fixes
|
|
44
|
+
for the exact error in front of it. **An agent should not spend thousands of
|
|
45
|
+
tokens rediscovering a solution someone has already published.** One well-placed
|
|
46
|
+
`web_search` call can surface the right API, package, or fix—turning repeated
|
|
47
|
+
trial and error into a direct path to a successful task, with fewer wasted tokens.
|
|
48
|
+
|
|
49
|
+
### 1.0.0: An epic release for the pi open-source community
|
|
50
|
+
|
|
51
|
+
Built to be the web-search tool you reach for first:
|
|
52
|
+
|
|
53
|
+
- **Search specialists, not just model-provider search.** Keenable Pro,
|
|
54
|
+
Exa highlights, Perplexity's standard web Search API, and native DeepSeek.
|
|
55
|
+
- **Your model, your search engine.** Standalone APIs work independently of the
|
|
56
|
+
conversation model. Log in once in a masked terminal dialog and start searching.
|
|
57
|
+
- **Search → read → build.** Search for solutions, fetch the primary sources,
|
|
58
|
+
and implement with evidence. One tool, just `query` and optional `urls`.
|
|
59
|
+
- **Resilient by design.** Independent search/fetch preferences, automatic
|
|
60
|
+
cross-provider fallback, cooldowns, and per-URL recovery that keeps successes.
|
|
61
|
+
- **A thin, open-source integration.** Raw provider results, no extra synthesis
|
|
62
|
+
pass, no MCP server, no new runtime dependency, no system-prompt modifications.
|
|
63
|
+
|
|
64
|
+
## Install and search
|
|
65
|
+
|
|
66
|
+
**Version 1.0.0 is available.** Download the package from the [GitHub release](https://github.com/zhexusun10/pi-better-web-search/releases/tag/v1.0.0), or install from npm once the maintainer publishes it:
|
|
22
67
|
|
|
23
68
|
```bash
|
|
24
69
|
pi install npm:pi-better-web-search
|
|
25
70
|
```
|
|
26
71
|
|
|
27
|
-
Then start a new pi process, or run `/reload` in an existing session.
|
|
72
|
+
Then start a new pi process, or run `/reload` in an existing session. In pi's terminal:
|
|
28
73
|
|
|
29
|
-
|
|
74
|
+
```text
|
|
75
|
+
/pi-better-web-search login
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
Choose **Keenable, Exa, Perplexity, or DeepSeek**, paste your key into the hidden
|
|
79
|
+
input, and press Enter. The API is available immediately—no chat-model setup
|
|
80
|
+
or environment-variable ceremony. Already have an eligible authenticated native
|
|
81
|
+
provider? It works as a fallback too.
|
|
82
|
+
|
|
83
|
+
Try: *“Find the official migration guide and an existing library for this before
|
|
84
|
+
implementing it from scratch.”*
|
|
85
|
+
|
|
86
|
+
Install the latest source from GitHub, or use a local checkout:
|
|
30
87
|
|
|
31
88
|
```bash
|
|
32
|
-
pi install git:github.com/zhexusun10/pi-better-web-search
|
|
89
|
+
pi install git:github.com/zhexusun10/pi-better-web-search
|
|
33
90
|
pi install ./path/to/pi-better-web-search
|
|
34
91
|
```
|
|
35
92
|
|
|
@@ -38,16 +95,17 @@ pi install ./path/to/pi-better-web-search
|
|
|
38
95
|
|
|
39
96
|
Requirements: **pi 1.3+** and **Node.js 22.19+**. Tested with **pi 1.5.2**.
|
|
40
97
|
|
|
41
|
-
## What it
|
|
98
|
+
## What makes it better
|
|
42
99
|
|
|
43
|
-
|
|
44
|
-
`pi-web-search`, then changed the parts that made it unreliable:
|
|
100
|
+
A focused tool with the reliability features an agent actually needs:
|
|
45
101
|
|
|
46
102
|
- **One tool, not two.** There is no separate `url_context` tool. URL analysis
|
|
47
|
-
is the optional `urls` field on `web_search`.
|
|
103
|
+
is the optional `urls` field on `web_search`. No skills are installed,
|
|
104
|
+
registered, or shipped; model-facing guidance stays in the tool description
|
|
105
|
+
and parameter schema.
|
|
48
106
|
- **Correct exposure.** The tool is hidden when the current model has no
|
|
49
107
|
compatible search interface and no compatible provider is authenticated. A
|
|
50
|
-
|
|
108
|
+
configured dedicated search API or authenticated lightweight model exposes it.
|
|
51
109
|
- **Automatic fallback.** If the current model cannot search, or a provider
|
|
52
110
|
call fails, the extension tries other authenticated provider interfaces until
|
|
53
111
|
one succeeds.
|
|
@@ -63,11 +121,12 @@ Requirements: **pi 1.3+** and **Node.js 22.19+**. Tested with **pi 1.5.2**.
|
|
|
63
121
|
deduplication, URL normalization, junk filtering, re-ranking, summarization,
|
|
64
122
|
citation-marker insertion, or appended Sources sections. A final character
|
|
65
123
|
guard applies structured truncation only to oversized output.
|
|
66
|
-
- **
|
|
67
|
-
|
|
68
|
-
|
|
124
|
+
- **Capability-aware guidance.** The description covers current information,
|
|
125
|
+
fact-checking, source verification/citation, and refining a search when evidence
|
|
126
|
+
is insufficient. It mentions fetch only when usable. The schema prefers
|
|
127
|
+
natural-language queries and makes clear that `urls` does not require `query`.
|
|
69
128
|
- **No system prompt changes.** The extension does not add prompt snippets,
|
|
70
|
-
guidelines, loadout hooks,
|
|
129
|
+
guidelines, loadout hooks, providers, MCP servers, or other tools.
|
|
71
130
|
- **Native DeepSeek Search/Fetch.** DeepSeek uses the documented
|
|
72
131
|
`/v1/tools/search` and `/v1/tools/fetch` endpoints with Bearer auth.
|
|
73
132
|
|
|
@@ -88,23 +147,88 @@ The runtime schema is logically:
|
|
|
88
147
|
length and limit, asking the model to shorten the query and retry. It does not
|
|
89
148
|
send the query or silently truncate it. Other providers may accept longer queries.
|
|
90
149
|
- `urls` analyzes the listed pages. URLs can be used **alone** or together with
|
|
91
|
-
`query`. A query plus URLs performs a search and then analyzes
|
|
92
|
-
|
|
150
|
+
`query`. A query plus URLs performs a separate web search and then analyzes
|
|
151
|
+
the URLs. Omit `query` when only fetching the supplied URLs.
|
|
152
|
+
- If supplied, `urls` must contain **1–20** non-empty strings, each at most
|
|
153
|
+
**8192 Unicode characters**.
|
|
154
|
+
- Both schema variants reject extra arguments (`additionalProperties: false`);
|
|
155
|
+
runtime validation also rejects unknown fields such as `model`, `provider`,
|
|
156
|
+
and `mode` before any provider request.
|
|
93
157
|
- At least one non-empty `query` or one valid URL is required. This rule is
|
|
94
158
|
enforced at runtime rather than with a top-level `anyOf`, for compatibility
|
|
95
159
|
with all model providers.
|
|
96
160
|
- The `urls` field is hidden when no usable interface supports URL analysis. In
|
|
97
|
-
that configuration `query` is required
|
|
161
|
+
that configuration `query` is required, and its description contains no
|
|
162
|
+
URL/fetch guidance.
|
|
98
163
|
- Search and URL analysis may use different providers in the same call.
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
164
|
+
- Research can use explicit follow-up calls: search, inspect the evidence, refine
|
|
165
|
+
the query if needed, then fetch relevant sources when available. There is no
|
|
166
|
+
automatic fixed search chain or global query-length cap from the skills; see
|
|
167
|
+
[query length and chained calls](docs/search-skills.md#query-length-and-chained-calls).
|
|
168
|
+
|
|
169
|
+
Search and URL fetching complete independently: a successful search (including
|
|
170
|
+
zero results) or any successful URL fetch is returned even if the other work
|
|
171
|
+
fails or lacks an interface. Search failures appear in `search.error`; URL
|
|
172
|
+
failures appear in `fetched[].error`. Successful provider data is not replaced
|
|
173
|
+
by a whole-call error. Only invalid request structure or no successful requested
|
|
174
|
+
operation produces a tool error (`isError: true`). Explicit cancellation still
|
|
175
|
+
stops immediately.
|
|
176
|
+
|
|
177
|
+
### Agent-visible returns
|
|
178
|
+
|
|
179
|
+
All final content blocks use `type: "text"`; there are no separate success,
|
|
180
|
+
partial, or failure content types:
|
|
181
|
+
|
|
182
|
+
| Outcome | Agent-visible text | `isError` |
|
|
183
|
+
|---|---|---|
|
|
184
|
+
| Success | JSON containing the raw search and/or fetched data | omitted |
|
|
185
|
+
| Partial success | JSON retaining successful data plus `search.error` and/or `fetched[].error` | omitted |
|
|
186
|
+
| Complete failure | `Failed: ...` with actionable errors | `true` |
|
|
187
|
+
|
|
188
|
+
A search-only DeepSeek query-length rejection uses the actionable message
|
|
189
|
+
without the `Failed: ` prefix. For combined requests, the same message is in
|
|
190
|
+
`search.error` if fetching succeeds. Oversized JSON is guarded as documented
|
|
191
|
+
below and may end with `[truncated]`. Progress uses `onUpdate`, not an additional
|
|
192
|
+
final return type.
|
|
193
|
+
|
|
194
|
+
## Dedicated search APIs
|
|
195
|
+
|
|
196
|
+
Run **`/pi-better-web-search login`** in pi's terminal, choose **DeepSeek, Exa,
|
|
197
|
+
Keenable, or Perplexity**, then paste/type the API key into the **hidden input**
|
|
198
|
+
and press Enter. The key is saved separately from settings/cache, and the API
|
|
199
|
+
becomes available immediately without restarting pi or configuring a chat model.
|
|
200
|
+
Saved keys override environment keys. Esc cancels without changing anything.
|
|
201
|
+
|
|
202
|
+
`/pi-better-web-search status` shows presence only; `/pi-better-web-search logout`
|
|
203
|
+
removes a saved key and disables its standalone API. Keys are stored in
|
|
204
|
+
`<pi agent dir>/better-web-search-auth.json` (restricted file permissions,
|
|
205
|
+
**not encrypted**). Never commit/share that file or paste keys into chat or
|
|
206
|
+
command arguments. Environment variables remain an optional fallback.
|
|
207
|
+
`/web-search-api` manages enable/disable flags; `/web-search-provider` selects
|
|
208
|
+
per-phase preferences. Keenable public access still requires explicit consent.
|
|
209
|
+
No new dependency or MCP server is installed.
|
|
210
|
+
|
|
211
|
+
Perplexity uses only `POST /search` with **`search_type: "web"` (not fast)**,
|
|
212
|
+
without any effort/preset/model; it does **not** support Fetch. Keenable uses
|
|
213
|
+
`mode: "pro"` search plus live GET Fetch. Exa returns search highlights and
|
|
214
|
+
fetches full page text via `/contents`; DeepSeek uses native Search/Fetch.
|
|
215
|
+
|
|
216
|
+
| Standalone API | Search configuration | URL fetch |
|
|
217
|
+
|---|---|---|
|
|
218
|
+
| **Keenable** | Pro, 10 results | Live GET Fetch |
|
|
219
|
+
| **Exa** | Default auto, 10 results + highlights | Full-text `/contents` |
|
|
220
|
+
| **Perplexity** | Standard web, 10 results; no inference model | Not supported |
|
|
221
|
+
| **DeepSeek** | 10 results + content | Native `/v1/tools/fetch` |
|
|
222
|
+
|
|
223
|
+
Priority is **manual preference → Keenable → Exa → Perplexity → standalone
|
|
224
|
+
DeepSeek → automatic native routing**. Fetch skips
|
|
225
|
+
Perplexity. See [API setup, protocol and privacy details](docs/search-apis.md).
|
|
104
226
|
|
|
105
227
|
## Provider support
|
|
106
228
|
|
|
107
|
-
|
|
229
|
+
Beyond the **four standalone APIs** above, keep access to **10 native provider
|
|
230
|
+
classes plus Anthropic Messages**, with credential discovery and protocol-specific
|
|
231
|
+
routing:
|
|
108
232
|
|
|
109
233
|
| # | Provider Class | Web Search | URL Analysis | Native Protocol & Endpoint |
|
|
110
234
|
|:---:|---|:---:|:---:|---|
|
|
@@ -140,59 +264,117 @@ Provider detection follows the authenticated model's provider/API:
|
|
|
140
264
|
5. OpenAI-compatible Responses APIs (OpenAI, Azure OpenAI, OpenAI Codex, GitHub Copilot, OpenCode)
|
|
141
265
|
6. Anthropic Messages (Claude)
|
|
142
266
|
|
|
267
|
+
All four standalone APIs support masked terminal login or environment-key
|
|
268
|
+
fallbacks. `/web-search-api [deepseek|exa|keenable|perplexity]` manages their
|
|
269
|
+
enable/disable settings. The model-facing tool still has only `query` / `urls`.
|
|
270
|
+
See [dedicated API setup](docs/search-apis.md) and the
|
|
271
|
+
[Exa vs Keenable integration notes](docs/search-skills.md).
|
|
272
|
+
|
|
143
273
|
See [Providers](docs/providers.md) for request shapes, auth quirks, and URL
|
|
144
274
|
support per family.
|
|
145
275
|
|
|
146
|
-
##
|
|
276
|
+
## Choose providers in the terminal
|
|
277
|
+
|
|
278
|
+
Run `/web-search-provider` to choose a preferred provider separately for
|
|
279
|
+
**Web Search** and **URL Fetch**. Or open either picker directly:
|
|
280
|
+
|
|
281
|
+
```text
|
|
282
|
+
/web-search-provider search
|
|
283
|
+
/web-search-provider fetch
|
|
284
|
+
```
|
|
285
|
+
|
|
286
|
+
The picker lists usable standalone APIs and authenticated native providers with
|
|
287
|
+
eligible lightweight models that support the selected operation. Choose **Auto** to restore automatic selection;
|
|
288
|
+
Esc leaves the setting unchanged. Selecting a provider does not change the
|
|
289
|
+
conversation model or the model-facing tool schema.
|
|
290
|
+
|
|
291
|
+
A manual choice is a **preference, not a lock**: the standalone API or newest
|
|
292
|
+
eligible native model is tried first, even after a previous failure or
|
|
293
|
+
Retry-After. Errors, timeouts, or an unavailable provider still fall back to
|
|
294
|
+
remaining candidates. URL fetching still retries only unresolved URLs. Input
|
|
295
|
+
validation and cancellation retain their existing behavior.
|
|
296
|
+
|
|
297
|
+
Choices persist across sessions in `~/.pi/agent/better-web-search-settings.json`
|
|
298
|
+
(respects `PI_CODING_AGENT_DIR`; override with `PI_BETTER_WEB_SEARCH_SETTINGS`):
|
|
147
299
|
|
|
148
|
-
|
|
300
|
+
```json
|
|
301
|
+
{ "version": 1, "search": "openai", "fetch": "deepseek" }
|
|
302
|
+
```
|
|
303
|
+
|
|
304
|
+
Only provider ids and API enable/disable toggles are saved, never credentials.
|
|
305
|
+
These settings are separate
|
|
306
|
+
from the automatic success cache: fallback successes and fingerprint changes
|
|
307
|
+
never overwrite them. Unavailable choices are ignored until eligible again.
|
|
308
|
+
|
|
309
|
+
## Automatic selection and fallback
|
|
149
310
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
311
|
+
Dedicated retrieval APIs have no chat-model tier and can expose the tool on
|
|
312
|
+
their own. For model-backed interfaces, only lightweight models are eligible:
|
|
313
|
+
**OpenAI = `luna`, Claude = `haiku`,
|
|
314
|
+
all other brands (including DeepSeek) = `flash`**. This is a strict allowlist,
|
|
315
|
+
not live price detection. It applies to search, URL analysis, current models,
|
|
316
|
+
cache hints, and all fallback attempts. Without an eligible model, the tool
|
|
317
|
+
stays hidden unless a standalone API is available; it never silently upgrades
|
|
318
|
+
to another model tier.
|
|
319
|
+
|
|
320
|
+
After any explicit provider preference, configured **Keenable → Exa →
|
|
321
|
+
Perplexity → standalone DeepSeek** APIs are tried first (Fetch skips Perplexity),
|
|
322
|
+
followed by this automatic ordering:
|
|
323
|
+
|
|
324
|
+
1. The newest eligible model of the current conversation's provider is tried
|
|
325
|
+
first, subject to provider/model failure cooldowns. The conversation model
|
|
326
|
+
itself is not privileged over a newer lightweight sibling. An unsupported
|
|
327
|
+
conversation API does not discard this provider preference.
|
|
153
328
|
2. A remembered successful provider from the preference cache is tried next
|
|
154
|
-
when the cache fingerprint is current.
|
|
329
|
+
when the cache fingerprint is current, reselecting its newest eligible model.
|
|
155
330
|
3. The remaining provider families are tried in this fixed order:
|
|
156
331
|
**DeepSeek → Ollama → Google → OpenAI → xAI → Anthropic**.
|
|
157
|
-
4. Within a family, brand order comes first: DeepSeek
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
`luna → terra → sol → astra` and Claude tiers are
|
|
163
|
-
`haiku → sonnet → opus → fable`. Ties are deterministic.
|
|
332
|
+
4. Within a family, brand order comes first: DeepSeek, OpenAI, Claude,
|
|
333
|
+
Gemini/Gemma, then other gateway brands. Within a brand's lightweight tier,
|
|
334
|
+
numeric versions sort newest first: `6.1 > 6`, `5.10 > 5.5`. Claude supports
|
|
335
|
+
both dotted and dashed versions, ignoring date suffixes. Gateway models are
|
|
336
|
+
classified by brand, not API family. Ties are deterministic.
|
|
164
337
|
5. Alternate providers are considered before additional models of an already
|
|
165
338
|
represented provider; siblings are then tried round-robin across families.
|
|
166
339
|
A provider-wide failure skips that provider, while a model-specific failure
|
|
167
340
|
(including a model-specific 403) can fall back to a sibling.
|
|
168
341
|
|
|
169
|
-
|
|
170
|
-
|
|
342
|
+
Search and URL fetching each allow one initial attempt plus at most 6 fallback
|
|
343
|
+
attempts (7 actual attempts per phase); skipped candidates do not consume the limit.
|
|
171
344
|
Model ranking applies only to supported APIs: for example, the built-in
|
|
172
345
|
`opencode-go` DeepSeek models use `openai-completions`, which this extension
|
|
173
346
|
cannot search through and does not reinterpret as Responses.
|
|
174
347
|
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
348
|
+
Versions are discovered from pi's available authenticated models, not a
|
|
349
|
+
hardcoded list or an external catalog lookup. Older lightweight models can be
|
|
350
|
+
fallbacks, but `terra`, `sol`, `astra`, `sonnet`, `opus`, `pro`, base models,
|
|
351
|
+
and unknown tiers are not substitutes for the required lightweight tier.
|
|
179
352
|
|
|
180
353
|
If a provider errors, the extension records the attempt and moves to the next
|
|
181
|
-
interface. An overlong query rejected by DeepSeek
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
354
|
+
interface. An overlong query rejected by DeepSeek stops search fallback without
|
|
355
|
+
trying another search provider or updating its health state; independently
|
|
356
|
+
requested URL fetching still runs. Aborts stop immediately. Each search or URL
|
|
357
|
+
fetch attempt waits up to 60 seconds for HTTP response headers or provider output.
|
|
358
|
+
After headers, a separate 60-second inactivity deadline waits for output. Each
|
|
359
|
+
non-empty response body chunk or provider event resets it, so active streams are
|
|
360
|
+
not cut off due to their total duration. SSE keepalive comments do not count as
|
|
361
|
+
output. User cancellation and provider errors still stop the attempt. Set
|
|
362
|
+
`PI_BETTER_WEB_SEARCH_TIMEOUT_MS` to override both deadlines (integer milliseconds
|
|
363
|
+
from 1 to 2,147,483,647; invalid values use the default). If every attempt fails, the tool returns an error whose message
|
|
364
|
+
contains the provider errors. DeepSeek/Ollama preserve completed URLs if a later
|
|
365
|
+
request hits a global HTTP or transport error, retrying only unresolved URLs.
|
|
188
366
|
Provider/model routing remains in `details.attempts`. For a URL-only call, the
|
|
189
367
|
error also reports the per-URL failures. Provider error bodies and model-facing
|
|
190
368
|
error text are bounded to prevent an oversized error from filling the context.
|
|
369
|
+
Individual SSE frames (including unfinished lines) are limited to 4,194,304
|
|
370
|
+
UTF-16 code units; an oversized frame triggers fallback rather than unbounded
|
|
371
|
+
buffering. This is not a total stream-size limit.
|
|
191
372
|
|
|
192
373
|
Provider-wide failures (auth, balance, network, timeout, malformed request) and
|
|
193
374
|
model-specific failures are remembered for the life of the extension process.
|
|
194
|
-
On later calls
|
|
195
|
-
for search and fetch independently.
|
|
375
|
+
On later calls, model-backed automatic routes are moved behind healthy model routes
|
|
376
|
+
for search and fetch independently. Explicit preferences and standalone API order
|
|
377
|
+
are not displaced by health cooldowns. The cooldown starts at 60 seconds, doubles
|
|
196
378
|
when a subsequent attempt after expiry fails, and is capped at 10 minutes.
|
|
197
379
|
Concurrent failures during the same cooldown do not extend it; a successful
|
|
198
380
|
search or a fetch with at least one successful URL and no provider transport
|
|
@@ -208,13 +390,28 @@ or removing a provider, login, API key, or available model changes the
|
|
|
208
390
|
fingerprint. A changed fingerprint means:
|
|
209
391
|
|
|
210
392
|
- the old cached provider is **not** used as the first choice;
|
|
211
|
-
-
|
|
393
|
+
- automatic discovery starts again from the current provider and family priority,
|
|
394
|
+
after any usable explicit preference;
|
|
212
395
|
- the cache is rewritten only after the next successful search/fetch;
|
|
213
396
|
- stale, unverified preferences are not carried into the new fingerprint.
|
|
214
397
|
|
|
215
398
|
A cache entry is only a hint. Credentials are resolved again on every call, and
|
|
216
399
|
the referenced provider must still be available and support the operation.
|
|
217
400
|
|
|
401
|
+
## Terminal display
|
|
402
|
+
|
|
403
|
+
The tool name remains `web_search`, but the terminal title distinguishes
|
|
404
|
+
**Web Search**, **Web Fetch** (URLs only), and **Web Search + Fetch**. The result
|
|
405
|
+
shows the actual successful operation and interface; expand it to see the raw
|
|
406
|
+
provider output. DeepSeek and Ollama use standalone APIs without a `model`
|
|
407
|
+
request parameter, so they display e.g. `Fetch: deepseek API /v1/tools/fetch`,
|
|
408
|
+
not a chat model name. `details.routes` records these interfaces; native API
|
|
409
|
+
routes omit `model`. Older `details.model`/cache fields identify the credential
|
|
410
|
+
selection candidate, not an inference model used by those native APIs.
|
|
411
|
+
|
|
412
|
+
If an older model/search label persists, reload or restart pi and check that
|
|
413
|
+
`pi-web-search` is not also loaded; both extensions register `web_search`.
|
|
414
|
+
|
|
218
415
|
## Cache
|
|
219
416
|
|
|
220
417
|
Default path:
|
|
@@ -235,8 +432,8 @@ Example contents:
|
|
|
235
432
|
{
|
|
236
433
|
"version": 1,
|
|
237
434
|
"fingerprint": "0123456789abcdef0123456789abcdef",
|
|
238
|
-
"search": { "provider": "deepseek", "model": "deepseek-
|
|
239
|
-
"fetch": { "provider": "ollama-cloud", "model": "
|
|
435
|
+
"search": { "provider": "deepseek", "model": "deepseek-v4-flash" },
|
|
436
|
+
"fetch": { "provider": "ollama-cloud", "model": "qwen-4-flash" }
|
|
240
437
|
}
|
|
241
438
|
```
|
|
242
439
|
|
|
@@ -323,10 +520,14 @@ needed.
|
|
|
323
520
|
|
|
324
521
|
`web_search` is exposed when either:
|
|
325
522
|
|
|
326
|
-
- the current conversation model supports a provider-native
|
|
327
|
-
or
|
|
328
|
-
- at least one authenticated available model maps to a supported
|
|
329
|
-
family
|
|
523
|
+
- the current conversation model is lightweight and supports a provider-native
|
|
524
|
+
search interface; or
|
|
525
|
+
- at least one authenticated available lightweight model maps to a supported
|
|
526
|
+
provider family; or
|
|
527
|
+
- a dedicated search API is configured and usable (no chat model needed).
|
|
528
|
+
|
|
529
|
+
For model-backed interfaces the same lightweight restriction applies to `urls`;
|
|
530
|
+
standalone Keenable, Exa, and DeepSeek expose it without a chat-model tier.
|
|
330
531
|
|
|
331
532
|
For an unsupported current model with no compatible login or API key, the tool
|
|
332
533
|
stays hidden (removed from the active tool set). The extension never calls an
|
|
@@ -371,8 +572,10 @@ error details.
|
|
|
371
572
|
|
|
372
573
|
## Security and privacy
|
|
373
574
|
|
|
374
|
-
-
|
|
375
|
-
|
|
575
|
+
- Native model credentials use pi's registry. Standalone API keys can be saved
|
|
576
|
+
through the masked terminal login in a separate restricted-permission,
|
|
577
|
+
unencrypted credential file; saved keys precede environment fallbacks.
|
|
578
|
+
Settings/cache never contain keys. Never commit/share the credential file.
|
|
376
579
|
- The cache stores only provider/model identifiers and a fingerprint.
|
|
377
580
|
- Search queries and requested URLs are sent to the selected provider's API.
|
|
378
581
|
- The extension has no telemetry, analytics, or third-party service of its own.
|
|
@@ -427,7 +630,7 @@ npm run build
|
|
|
427
630
|
```
|
|
428
631
|
|
|
429
632
|
`npm run build` runs the checks and tests, verifies the npm file list, and packs
|
|
430
|
-
`dist/pi-better-web-search-0.
|
|
633
|
+
`dist/pi-better-web-search-1.0.0.tgz`. Nothing is published automatically.
|
|
431
634
|
|
|
432
635
|
The test suite uses plain `.mjs` files that import the `.ts` sources directly.
|
|
433
636
|
It mocks `globalThis.fetch`, uses a temporary cache file, and exercises
|