pi-better-web-search 0.1.11 → 1.0.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (46) hide show
  1. package/CHANGELOG.md +202 -0
  2. package/README.md +289 -68
  3. package/SECURITY.md +31 -4
  4. package/assets/architecture.svg +46 -21
  5. package/assets/search-index.png +0 -0
  6. package/docs/README.zh-CN.md +242 -51
  7. package/docs/architecture.md +155 -43
  8. package/docs/providers.md +57 -9
  9. package/docs/publishing.md +9 -8
  10. package/docs/search-apis.md +259 -0
  11. package/docs/search-skills.md +265 -0
  12. package/docs/testing.md +93 -12
  13. package/extensions/index.ts +9 -0
  14. package/lib/api-command.ts +84 -0
  15. package/lib/atomic-file.ts +32 -0
  16. package/lib/attempt-timeout.ts +82 -0
  17. package/lib/auth.ts +9 -0
  18. package/lib/cache.ts +24 -16
  19. package/lib/candidates.ts +182 -112
  20. package/lib/credentials.ts +98 -0
  21. package/lib/dedicated-config.ts +61 -0
  22. package/lib/format.ts +14 -0
  23. package/lib/health.ts +24 -5
  24. package/lib/login-command.ts +98 -0
  25. package/lib/preferences.ts +62 -0
  26. package/lib/provider-command.ts +66 -0
  27. package/lib/providers/anthropic.ts +7 -4
  28. package/lib/providers/dedicated-http.ts +80 -0
  29. package/lib/providers/deepseek.ts +77 -10
  30. package/lib/providers/exa.ts +66 -0
  31. package/lib/providers/google.ts +4 -3
  32. package/lib/providers/http.ts +99 -47
  33. package/lib/providers/index.ts +9 -0
  34. package/lib/providers/keenable.ts +61 -0
  35. package/lib/providers/ollama.ts +13 -6
  36. package/lib/providers/openai.ts +6 -6
  37. package/lib/providers/perplexity.ts +20 -0
  38. package/lib/providers/sse.ts +93 -54
  39. package/lib/providers/types.ts +6 -2
  40. package/lib/redact.ts +15 -0
  41. package/lib/render.ts +41 -0
  42. package/lib/routes.ts +28 -0
  43. package/lib/schema.ts +26 -10
  44. package/lib/search.ts +124 -64
  45. package/lib/secret-input.ts +63 -0
  46. package/package.json +8 -2
package/CHANGELOG.md CHANGED
@@ -1,5 +1,207 @@
1
1
  # Changelog
2
2
 
3
+ ## 1.0.1 — Your pi login, your search tools
4
+
5
+ ### Fixes
6
+
7
+ - Discover official DeepSeek Search/Fetch through pi's provider-level `/login`
8
+ and `models.json` authentication even when the available chat catalog is
9
+ empty, filtered, stale, or has no flash model. Reuse request-time registry
10
+ credentials and configured native/proxy endpoints without copying keys or
11
+ invoking an inference model. Expose Search and Fetch in both provider pickers
12
+ and enable the URL-capable tool schema without a second extension login.
13
+ - Keep the pi-authenticated DeepSeek route separate from extension-saved and
14
+ environment standalone credentials. Preserve configured API priority, manual
15
+ preferences, fallback, and one menu entry per provider. Login/logout changes
16
+ invalidate discovery fingerprints; request-time auth and captured-header
17
+ redaction protect stale routes and in-flight credential rotation. Third-party
18
+ gateways merely hosting DeepSeek chat models are not treated as its native API.
19
+ - Show all terminal command entrypoints in `/pi-better-web-search` and
20
+ `/pi-better-web-search help`, including `/web-search-provider search`,
21
+ `/web-search-provider fetch`, and `/web-search-api`. Empty/help requests show
22
+ informational guidance instead of a warning; invalid arguments still warn
23
+ without echoing potentially sensitive values. Keep hidden-dialog-only key entry.
24
+
25
+ ### Documentation and artwork
26
+
27
+ - Restore the original white-background, black/gray wireframe architecture style
28
+ and concise README introduction. Lead both English and Chinese READMEs with
29
+ what the plugin does; retain the updated standalone APIs and native fallback
30
+ diagram, and remove its duplicate from the later routing section.
31
+ - Use the maintainer's original Artificial Analysis screenshot unchanged as
32
+ `assets/search-index.png`, replacing the redrawn SVG chart. Update related
33
+ documentation links and package verification so the screenshot ships in npm.
34
+ - Prominently mark **Pi 1.5.2 verified** in both READMEs and the architecture
35
+ graphic. Define verification as type checking and automated tests against pi
36
+ 1.5.2 host dependencies, with Node.js 22/24 CI, not live testing every provider.
37
+ - Document command discovery, independent search/fetch preferences, direct reuse
38
+ of pi's DeepSeek credentials, the API-only model-tier exception, and the latest
39
+ 1.0.1 package and publishing commands in English and Chinese.
40
+
41
+ ### Tests
42
+
43
+ - Add real pi 1.5.2 registry integration with isolated credential/model stores
44
+ for pi-login and `models.json` auth, plus regression tests for catalog gaps,
45
+ stale snapshots, legacy registries, tool exposure, provider selection,
46
+ proxy/auth overrides, logout, credential rotation, priority, and fallback.
47
+ - Add full-help discovery tests across TUI/RPC/print contexts and expand invalid
48
+ argument checks to ensure help opens no dialogs, changes no state, and never
49
+ echoes credentials.
50
+
51
+ ## 1.0.0 — Search is an agent superpower
52
+
53
+ ### The flagship search upgrade
54
+
55
+ - Bring **Keenable, Exa, and Perplexity**, the top three independent providers
56
+ in the Artificial Analysis Search Index snapshot, plus **DeepSeek native
57
+ Search/Fetch**, directly into pi without requiring a conversation-model change.
58
+ - Lead the English/Chinese READMEs with the search advantage: **33 model-only →
59
+ 83 with Keenable Pro (+50 points)**. Show why agents should find existing
60
+ libraries, official docs, and proven fixes before spending tokens reinventing
61
+ them. Add a sourced Search Index chart and redraw the architecture for four
62
+ standalone APIs, native fallback, terminal login, and the 96k output guard.
63
+
64
+ ### Regression fixes
65
+
66
+ - Stop OpenAI/xAI streams on completed/done/incomplete terminal events and
67
+ Anthropic streams on `message_stop`, cancelling connections that remain open
68
+ instead of timing out and discarding usable search results.
69
+ - Redact native-provider transport, SSE and JSON errors using a snapshot of sent
70
+ credential headers (including custom API tokens and Bearer/Basic auth). Keep
71
+ status/code metadata for fallback, scrub structured error bodies, and suppress
72
+ JSON parser snippets that could expose a partial credential.
73
+ - Require Anthropic server-tool evidence to name `web_search`; unrelated tools
74
+ no longer produce false search success. Skip non-object stream frames safely.
75
+ - Atomically replace settings and the success cache after writing/syncing a
76
+ same-directory temporary file. Partial-write and rename failures preserve the
77
+ previous file; configured file symlinks continue to update their targets.
78
+ - Recheck Keenable public-access consent before each adapter request, including
79
+ combined calls and URL batches. Key loss/logout no longer downgrades an
80
+ in-flight operation to public endpoints; preserve completed search/pages and
81
+ resolve auth-dependent endpoint paths from the same key snapshot as headers.
82
+ - Recognize aborts by signal, structured identity or canonical legacy messages,
83
+ not arbitrary text containing `abort`. Malformed page responses no longer
84
+ discard completed URLs, and dedicated transport AbortErrors remain redacted
85
+ even if credentials rotate while the request is in flight.
86
+ - Cancel stalled response bodies and responses arriving after an attempt timeout,
87
+ even when a custom transport ignores the signal. Ignore late adapter updates
88
+ after timeout/completion and prevent disposed deadlines from being restarted.
89
+ - Bound individual SSE frames to 4,194,304 UTF-16 code units, including unfinished
90
+ lines and multiline data, while accepting larger streams of bounded frames. Scan
91
+ fragmented lines incrementally instead of repeatedly rescanning their prefix.
92
+ - Share literal/JSON-escaped credential redaction across dedicated API errors,
93
+ native DeepSeek errors and Exa per-URL errors. Also redact HTTP status text and
94
+ structured error codes using the credentials actually sent.
95
+ - Keep routing priorities strict: **manual preference → Keenable → Exa →
96
+ Perplexity → standalone DeepSeek → native automatic routing**. Manual choices
97
+ get a first probe even after failure or Retry-After; cooldowns reorder only
98
+ remaining automatic model routes. Standalone DeepSeek failures do not suppress
99
+ native DeepSeek candidates using separate authentication.
100
+ - Route standalone DeepSeek through shared dedicated HTTP validation/backoff:
101
+ honor credential-specific `Retry-After` across Search/Fetch, reject malformed
102
+ header keys before HTTP, and leave native model/proxy auth unchanged.
103
+ - Redact Exa per-URL errors using the key actually sent even if login rotates it
104
+ in flight; honor canonical-URL error statuses that use `url` instead of `id`.
105
+ - Release bounded HTTP error streams without awaiting an underlying cancellation
106
+ promise that may never settle, so errors can reach fallback immediately.
107
+ - Isolate the live DeepSeek test's settings/cache and disable standalone API
108
+ discovery so other saved/environment search keys cannot trigger unrelated
109
+ provider calls or override the intended native DeepSeek fallback check.
110
+
111
+ ### Implementation details
112
+
113
+ - Explicitly provide no skills: remove vendor skill copies and their lockfile
114
+ from the checkout, retain only developer reference notes, and reject skill
115
+ directories/entrypoints in package verification. Keep normal tool descriptions,
116
+ schemas and API behavior unchanged; no automatic vendor skill context is added.
117
+ - Refine both description variants to address insufficient or conflicting
118
+ evidence and cite the source URLs actually used; make fetch conditional on
119
+ need in the URL-capable version, without adding fetch to search-only guidance.
120
+ - Improve capability-aware tool/field descriptions with natural-language
121
+ query guidance, source verification/citation, and optional evidence-driven
122
+ search refinement. Both variants start with the provider-neutral
123
+ `Search the live web and return the most relevant URLs and content.`; only
124
+ URL-capable variants instruct fetching pages. Keep `query` / `urls`, existing validation, adapter requests
125
+ and system prompts unchanged. Document skill wording, provider-specific length
126
+ limits and agent-driven chains without adding a fixed search loop.
127
+ - Add `/pi-better-web-search login`: select DeepSeek, Exa, Keenable or Perplexity
128
+ and enter an API key in a masked local terminal dialog. Persist keys separately
129
+ with restricted permissions, cross-process locking and atomic replacement;
130
+ refresh tool exposure immediately without environment setup or a chat model.
131
+ Saved keys override environment keys; preserve manual routing and fallback.
132
+ - Add safe presence-only `status` and confirmed `logout` (disables standalone
133
+ access without silently switching Keenable to public mode). Refuse RPC/print
134
+ secret entry and key arguments; preserve cancellation and existing credentials
135
+ on save failures. Credential files are local plaintext, not encrypted.
136
+ - Add standalone Exa Search/Fetch with `EXA_API_KEY` and `/web-search-api exa`.
137
+ Search sends the approved `numResults: 10` and `contents.highlights: true`;
138
+ known URLs use `POST /contents` with top-level `text: true`. Inspect per-URL
139
+ statuses even on HTTP 200, preserve successful raw documents, and retry only
140
+ unresolved URLs through existing fallback. Keep safe error redaction,
141
+ cancellation, Retry-After, API priority and the `query` / `urls` contract.
142
+ Document Exa/Keenable skill differences; objective and other advanced options
143
+ remain off; no Exa-specific parameter or system-prompt change is introduced.
144
+ - Add standalone Perplexity Search (`POST /search`, `search_type: web`, never
145
+ fast, no model/effort, no Fetch) and Keenable pro Search/live GET Fetch.
146
+ Preserve raw result objects and use existing timeout, cooldown, fallback and
147
+ partial-success handling. Configured APIs precede automatic model routing;
148
+ explicit per-phase provider preferences still win.
149
+ - Add `/web-search-api [deepseek|exa|keenable|perplexity]` to enable/disable APIs,
150
+ view safe key setup, and opt into Keenable public access with privacy consent.
151
+ This menu saves toggles only; masked login stores credentials separately. Honor Retry-After
152
+ across dedicated search/fetch calls, and add an opt-in status/shape-only smoke
153
+ test: `npm run test:live:search-apis -- all`. No new runtime dependency or MCP.
154
+ - Preserve successful search and URL-fetch data independently. Missing fetch
155
+ capability or fetch failures no longer replace successful search output with
156
+ a whole-call error; failed search (including query-length rejection) no longer
157
+ suppresses requested fetch work. Return partial-success JSON with `search.error`
158
+ and/or per-URL errors, and mark the whole call failed only if no operation
159
+ succeeded. Keep partial search failures visible in collapsed terminal results.
160
+ - Add `/web-search-provider [search|fetch]` terminal pickers with independent,
161
+ persistent search/fetch preferences and an Auto reset. Choices precede
162
+ automatic hints without disabling fallback, per-URL retries, or health
163
+ cooldowns; success-cache updates never overwrite explicit preferences.
164
+
165
+ - Clarify that providing `query` triggers a separate search even with `urls`;
166
+ show URL-only omission guidance only when URL support is exposed. Reject
167
+ additional input arguments in both schemas and at runtime, require non-empty
168
+ URL arrays when supplied, and constrain URL strings to 1–8192 Unicode code
169
+ points consistently with native DeepSeek validation.
170
+ - Strictly limit search and URL analysis candidates to OpenAI `luna`, Claude
171
+ `haiku`, and other brands' `flash` models. Select the newest numeric version
172
+ within the eligible brand/tier, including gateways; current-model and cache
173
+ hints now select a provider instead of pinning an older or larger model.
174
+ Never fall back to an excluded tier, and hide the tool without an eligible
175
+ model or configured standalone API.
176
+ - Compare dotted/dashed Claude minor versions numerically and ignore date
177
+ suffixes independently of the provider API family.
178
+ - Distinguish Search, Fetch, and combined operations in the terminal. Display
179
+ the actual DeepSeek/Ollama standalone API endpoint instead of an inference
180
+ model name; add `details.routes` and progress `details.activeRoute` metadata.
181
+ Raw provider output is unchanged and available in the expanded result.
182
+ - Preserve current-provider priority even when the conversation model's API is
183
+ unsupported, and derive discovery/cache fingerprints from one registry snapshot.
184
+ - Preserve completed DeepSeek/Ollama URL results after a later global HTTP error;
185
+ only unresolved URLs proceed to fallback. Show failed URL counts and truncation
186
+ warnings in collapsed results.
187
+ - Validate timeout overrides against Node's integer timer range and observe
188
+ adapter rejections even when the signal is already aborted.
189
+ - Handle SSE CR/LF/CRLF boundaries, data-less event resets, `[DONE]`, and pending
190
+ read cancellation without waiting for the provider connection to close.
191
+
192
+ ### Regression coverage and validation
193
+
194
+ - Add 62 regression tests across response/deadline lifecycle, late adapter
195
+ updates, bounded/fragmented SSE frames, credential redaction and rotation,
196
+ atomic settings/cache writes, runtime Keenable consent, abort classification,
197
+ native terminal events and search-evidence validation.
198
+ - Add four platform-independent simulated symlink tests for settings/cache target
199
+ updates and dangling-link refusal. Keep the real filesystem checks separately.
200
+ - Verify on Node 24: 386 tests total, 384 passed, no failures; two real file-symlink
201
+ tests skipped because Windows returns `EPERM` when creating file symlinks.
202
+ - Type checking and package verification pass. Provider requests are mocked;
203
+ these checks do not call billable live APIs.
204
+
3
205
  ## 0.1.11
4
206
 
5
207
  - Increase the default timeout to 120 seconds per search and URL fetch attempt