@fffangchenghao/dsh-web-search 0.19.3 → 0.20.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (60) hide show
  1. package/CHANGELOG.md +173 -1
  2. package/README.md +214 -34
  3. package/README.zh-CN.md +187 -35
  4. package/SECURITY.md +42 -7
  5. package/client.js +248 -10
  6. package/docs/store-evidence.md +10 -5
  7. package/lib/advanced-search.d.ts +107 -0
  8. package/lib/advanced-search.js +255 -0
  9. package/lib/bridge.d.ts +34 -0
  10. package/lib/bridge.js +305 -0
  11. package/lib/cache.d.ts +11 -4
  12. package/lib/cache.js +9 -5
  13. package/lib/config.d.ts +19 -3
  14. package/lib/config.js +24 -0
  15. package/lib/domain-tool.d.ts +5 -0
  16. package/lib/domain-tool.js +45 -6
  17. package/lib/engine-test.d.ts +116 -0
  18. package/lib/engine-test.js +256 -0
  19. package/lib/engines.d.ts +11 -1
  20. package/lib/engines.js +118 -19
  21. package/lib/errors.d.ts +16 -2
  22. package/lib/errors.js +40 -1
  23. package/lib/index.d.ts +24 -2
  24. package/lib/index.js +309 -21
  25. package/lib/inject.d.ts +68 -0
  26. package/lib/inject.js +1 -0
  27. package/lib/markdown.js +18 -1
  28. package/lib/mirrors.d.ts +23 -0
  29. package/lib/mirrors.js +69 -0
  30. package/lib/official-feed.d.ts +114 -0
  31. package/lib/official-feed.js +288 -0
  32. package/lib/official-support.d.ts +74 -0
  33. package/lib/official-support.js +134 -0
  34. package/lib/platform-tool.d.ts +77 -0
  35. package/lib/platform-tool.js +173 -0
  36. package/lib/platforms.d.ts +178 -0
  37. package/lib/platforms.js +1035 -0
  38. package/lib/query-router.d.ts +29 -0
  39. package/lib/query-router.js +58 -0
  40. package/lib/read-page.js +2 -1
  41. package/lib/research-cache.d.ts +115 -0
  42. package/lib/research-cache.js +275 -0
  43. package/lib/research.d.ts +45 -4
  44. package/lib/research.js +505 -22
  45. package/lib/search-many.d.ts +7 -1
  46. package/lib/search-many.js +7 -2
  47. package/lib/system-prompt.d.ts +27 -0
  48. package/lib/system-prompt.js +50 -0
  49. package/lib/tool-suite.d.ts +97 -0
  50. package/lib/tool-suite.js +224 -0
  51. package/lib/transport.d.ts +33 -2
  52. package/lib/transport.js +42 -9
  53. package/lib/types.d.ts +20 -0
  54. package/lib/types.js +9 -1
  55. package/lib/untrusted.d.ts +33 -0
  56. package/lib/untrusted.js +42 -0
  57. package/lib/verticals.d.ts +3 -1
  58. package/lib/verticals.js +26 -7
  59. package/package.json +5 -3
  60. package/skills/dsh-web-search/SKILL.md +21 -6
package/CHANGELOG.md CHANGED
@@ -1,5 +1,177 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.20.1
4
+
5
+ ### Added
6
+ - **A manual update button for deployments without `dsh-autoupdater`.** When
7
+ the Settings page is local and npm reports a newer version, it shows
8
+ **Install update**. The button posts to a loopback-only
9
+ `/api/dsh-web-search/update` bridge, which calls the Host's built-in
10
+ `pluginManager.installBundle` with exactly
11
+ `@fffangchenghao/dsh-web-search@latest`; it never accepts a package name from
12
+ the browser. A `link:`/`file:` checkout is refused rather than replaced, a
13
+ deployment without a plugin manager reports that honestly, and a successful
14
+ install says that a restart is required. This is manual, not a scheduler:
15
+ periodic checks still belong to `dsh-autoupdater`.
16
+ - **The npm version comparison is now directional.** A local build newer than
17
+ the registry is no longer labelled "update available"; the button appears
18
+ only when the registry version is actually newer.
19
+ - **Update-bridge tests.** 8 new cases cover the fixed package spec, link/file
20
+ refusal, manager absence, `ctx.get` fallback, method/loopback/origin guards,
21
+ malformed bodies and manager failure results.
22
+
23
+ **Test suite: 664 tests across 50 files** (663 pass, 1 skipped).
24
+
25
+ ## 0.20.0
26
+
27
+ ### Added
28
+ - **One `web_tools` action surface.** The eight optional capabilities are now
29
+ one model-facing tool with `action` set to `search_many`, `advanced_search`,
30
+ `domain_search`, `platform_search`, `read_page`, `research`, `cache_search`
31
+ or `engine_test`. This cuts the always-visible optional surface from eight
32
+ names to one while keeping each handler's validation and rendering; the old
33
+ names remain available with `toolSurface: legacy`. The real DSH registry
34
+ accepts the unified schema (see `scripts/verify-tool-registration.mjs`).
35
+ - **Auto-update compatibility.** The package is documented and installed as an
36
+ npm bundle so the profile-level `dsh-autoupdater` manages it; `link:`/`file:`
37
+ checkouts are intentionally excluded by that updater. The Settings card shows
38
+ the latest npm version and the running build; the plugin never hot-swaps its
39
+ own loaded code. The desktop profile is now installed from npm (`0.20.0`), so the profile
40
+ updater no longer classifies it as a local checkout.
41
+ - **`advanced_search` uses the official Chrome Releases feed for a windowed
42
+ Chrome security/release query.** The feed carries the real post date that a
43
+ search snippet often loses; the result note states how many posts fell inside
44
+ the window and whether any title or summary mentions zero-day. A feed miss
45
+ falls back to the previous engine path.
46
+ - **`search_many` routes explicit package-registry queries.** A query that names
47
+ npm, PyPI or crates.io is answered from that existing domain source (one exact
48
+ lookup per extracted package name) and falls back to ordinary web results if
49
+ the registry returns nothing.
50
+ - **`platform_search`: community and video sources.** A new model tool queries
51
+ Bilibili, V2EX, Reddit, YouTube, Zhihu and Juejin. It uses each site's keyless
52
+ public endpoint where one exists and falls back to `site:<domain> <query>`
53
+ through the normal search path. Fallback rows are filtered to the platform
54
+ host (and aliases such as `b23.tv`, `youtu.be`, `redd.it`), so a generic page
55
+ cannot become a platform result; one blocked platform does not empty the
56
+ batch. Zhihu has no usable keyless API and is therefore fallback-only.
57
+ - **`cache_search`: a second retrieval over `research` bodies.** `research` now
58
+ fetches up to 200,000 characters per page and stores the full body in an
59
+ in-process cache through an optional `onPage` sink. `cache_search` searches
60
+ those bodies (24 pages, 400,000 characters each, 7-day TTL) and returns the
61
+ matching passages with the same URLs, without another network request. The
62
+ first excerpt is query-aware too: instead of only the opening 6,000
63
+ characters, it scores paragraph windows against the query and keeps the best
64
+ windows plus opening context.
65
+ - **Official-source query expansion is focused, not generic.** When the first
66
+ result set lacks a known issuer, `research` spends an unused query slot on a
67
+ focused `site:` query for the EU AI Act, Chinese CAC, Chrome Releases, Python,
68
+ Redis or WCAG, filters the rows to that issuer's domain, and never promotes a
69
+ landing/home page on authority alone. W3C WCAG pages that return 403 are
70
+ retried through `w3c.github.io`.
71
+ - **`advanced_search` drops a redundant relative-time phrase.** When the caller
72
+ passes an explicit window, a `过去 7 天内` / `last 7 days` fragment is removed
73
+ from the engine query but kept in the returned `query`; the structured window
74
+ already carries that constraint.
75
+ - **`domain_search` accepts `timeRange`.** `off`/`day`/`week`/`month`/`year`
76
+ filters parsed publication dates for that call; undated rows survive only
77
+ when no dated row qualifies, with a `note` explaining the outcome. arXiv
78
+ switches to `submittedDate` order and a quoted phrase when a window is set,
79
+ so "last month" returns recent papers rather than old relevance matches.
80
+ - **Chrome Stable filtering and security summary.** A Stable-only Chrome query now removes Beta, Dev, Early Stable and Extended Stable posts from the official feed; its snippets preview the version, fix count and listed Critical/High/Medium/Low CVE counts when the post states them. An empty Stable result falls back to the ordinary engine path.
81
+
82
+ - **Bilibili rows carry play counts, a topic gate, recency/popularity controls and an alternate endpoint.** Rows show `UP主`, play count and duration when the API returns them; adjacent topics are dropped; queries that explicitly ask for recency or high play apply a bounded date filter and play-count sort; and `/search/all/v2` is tried when `/search/type` is blocked with 412. Juejin rows also surface `点赞`/`阅读`, sort by likes for a `高赞` query, and Zhihu fallback drops a broad unrelated question when at least two specific answers remain.
83
+
84
+ - **PostgreSQL version/EOL questions use the official support table.** A PostgreSQL query that asks about the current version, support or EOL now prepends the project's official versioning row (current major/minor, first release and final release/EOL), while ordinary PostgreSQL how-to queries keep the normal engine path.
85
+ - **A clause/date AI Act question gets the official Article 113 page.** `research` adds the Article 113 page directly for a question that asks when a GPAI obligation applies or which article governs it, so a Chinese-language question does not depend on the English page winning a search ranking. The page is fetched through the normal ladder and its operative date windows are excerpted.
86
+ - **Legal article excerpts target the operative clause.** Research excerpts now keep standalone article numbers and pick the late clause that combines an operative verb with a date, so an EU AI Act Article 113 pack shows `2 August 2025` instead of only the table-of-contents links.
87
+ - **Longer, date-aware research excerpts.** `research` now keeps up to 1,800 characters per page and preserves the final window when a regulation query asks when a rule takes effect, so the effective-date article is not crowded out by definitions.
88
+
89
+ - **Ecosystem comparison refreshed.** The README now carries a 2026-10-25 snapshot with GitHub stars, last-month npm downloads, test evidence and package size for the main DSH competitors; stale claims about tool counts, store status and test leadership were removed.
90
+
91
+ **Test suite: 656 tests across 49 files** (655 pass, 1 skipped). The new
92
+ unified-tool, platform, cache, mirror and landing-page tests are included.
93
+ - **`engine_test`: one probe per engine, with a reason.** A new model tool tests
94
+ one engine or all eight, bypasses the cache and never changes cooldown state,
95
+ and returns `ok`, latency, result count, and a failure class (`quota`, `auth`,
96
+ `bot-wall`, `transient`, `invalid-response`, `unknown`). The Settings page's
97
+ **Test engines** button calls the same runner through a loopback-only
98
+ `POST /api/dsh-web-search/engine-test` bridge, so a person can see which
99
+ engine is slow or blocked without reading a failure log.
100
+ - **`fallbackOn` and a result `Note`.** A primary failure now carries a
101
+ structured `FailureKind`; the fallback tier runs only for the classes enabled
102
+ in `fallbackOn`. The default keeps every class enabled, so the old
103
+ always-fall-back behaviour is unchanged. When policy skips or uses the
104
+ fallback, the result says so in a short `note` (also rendered by
105
+ `search_many`).
106
+ - **`blockedDomains`.** A comma-separated blocklist applied to merged and cached
107
+ results; a subdomain matches its parent. Empty by default, so it cannot change
108
+ existing searches.
109
+ - **`safeSearch`.** `off` (default, engine default), `moderate` or `strict`.
110
+ Forwarded as `adlt` to Bing and `kp` to DuckDuckGo; other engines are left
111
+ untouched.
112
+ - **`advanced_search`: a per-call date window.** One query can ask for `day`,
113
+ `week`, `month` or `year` without changing the global Settings value; `off`
114
+ disables the window for that call and omitting it keeps the setting. It shares
115
+ the engine, cache, cooldown and failure path with `web_search`, and falls back
116
+ to `web_search_advanced` if the name is taken.
117
+ - **An explicit untrusted-content boundary.** Every plugin-owned tool wraps its
118
+ web-derived render in `<untrusted-web-content>...</untrusted-web-content>`,
119
+ strips forged boundary tags from the payload, and a short `systemPrompt`
120
+ section tells the model those contents are data, not instructions. The host's
121
+ own `web_search`/`web_fetch` notice is left in place.
122
+
123
+ ### Changed
124
+ - **`research` now prefers authoritative sources on equal relevance.** A new
125
+ authority signal recognises government/academic TLDs, standards bodies and
126
+ well-known official documentation hosts, and uses documentation/legal/release
127
+ paths as a small boost. A credible official page that matches the query leads
128
+ the pack even when a generic page repeats more query words; relevance still
129
+ gates out official pages that match none of it. The signal is structural, not
130
+ a query-specific allowlist. Three regression tests.
131
+ - The retry-policy code now records HTTP `status` on failures and classifies
132
+ them, but cooldown timing and retryability are unchanged.
133
+ - Cache keys include a non-default `safeSearch`, a non-default `fallbackOn`
134
+ and a non-empty `blockedDomains`; the default paths keep their old keys, and
135
+ removing a block cannot reuse a filtered entry.
136
+
137
+ ### Fixed
138
+ - **`research` rejects bot challenges and login redirects.** A challenge page can
139
+ be well over the 160-character floor (Anubis/Cloudflare shells, for example),
140
+ and a redirect can still carry a readable login URL. Both are now reported as
141
+ failed pages with a reason, instead of being quoted as evidence. Two
142
+ regression tests.
143
+ - **npm publish dates are publish dates.** `domain_search` read the npm search
144
+ index's `updated` field, which moves when the index refreshes; it now reads
145
+ `package.date`, the version's actual publish time, and keeps an undated
146
+ package undated instead of inventing a date. One regression test.
147
+ - **`advanced_search` says when a window is not proven.** If a caller asks for
148
+ `day`/`week`/`month`/`year` and some results have no publication date, the
149
+ result carries a note naming the count, instead of letting undated rows look
150
+ like they passed the filter. One regression test.
151
+ - **Domain rows are readable and carry the metadata the API already returns.**
152
+ Stack Exchange entities (`-&gt;`, `&#39;`) are decoded; Hacker News rows show
153
+ points and comment counts; GitHub rows show the SPDX license and stars; PyPI
154
+ rows show `Requires-Python`. These are presentation-only additions on the
155
+ existing `Source` shape.
156
+ - **`research` rejects shells and boilerplate instead of citing them.** A fetch
157
+ that comes back as a raw HTML document, or as fewer than 160 characters of
158
+ text, is now reported as a failed page with a reason rather than quoted as
159
+ evidence. Two regression tests.
160
+ - **Long RFC-style pages keep the document, not one appendix.** The reading-root
161
+ heuristics could pick a code-heavy ABNF appendix (RFC 9110 returned only the
162
+ collected ABNF) or one dense section of a long document with no
163
+ `<article>`/`<main>`. A code-dominated container is now skipped, and a single
164
+ section is trusted only when it holds at least half the page's text. Two
165
+ regression tests; live RFC 9110 now converts to 603,506 characters of
166
+ document text instead of an ABNF-only excerpt.
167
+
168
+ ### Verification
169
+ - 579 tests / 41 files (578 pass, 1 skip); `npm run typecheck` clean.
170
+ - Replay bench unchanged: 20 queries P@8 `0.4375`, R@8 `0.7917`, MRR@8
171
+ `0.8071`; long corpus `0.4500 / 1.0000 / 0.7667`.
172
+ - No publish yet: the version is bumped for the prepared package, not pushed to
173
+ npm.
174
+
3
175
  ## 0.19.3
4
176
 
5
177
  ### Fixed
@@ -143,7 +315,7 @@
143
315
  and a success clears its strikes. A throttled or rejected key retries the same
144
316
  request on the next key once, in both search and page extraction. Pools are
145
317
  cached per Settings value, so editing the field starts clean.
146
- - **Settings page polish.** A sticky jump bar scrolls to any of the five groups,
318
+ - **Settings page polish.** A sticky jump bar scrolls to any of the six groups,
147
319
  and a line under the title compares the installed version with npm's latest
148
320
  (hidden when the registry is unreachable rather than guessed).
149
321
  - **Portable skill.** `skills/dsh-web-search/SKILL.md` states when to search,
package/README.md CHANGED
@@ -9,7 +9,7 @@
9
9
 
10
10
  <p align="center">
11
11
  <a href="https://www.npmjs.com/package/@fffangchenghao/dsh-web-search"><img src="https://img.shields.io/npm/v/@fffangchenghao/dsh-web-search.svg" alt="npm version"></a>
12
- <img src="https://img.shields.io/badge/tests-500%20passing-brightgreen" alt="tests">
12
+ <img src="https://img.shields.io/badge/tests-664%20passing-brightgreen" alt="tests">
13
13
  <img src="https://img.shields.io/badge/engines-8%2C%20zero%20API%20key-brightgreen" alt="engines">
14
14
  <img src="https://img.shields.io/badge/node-%3E%3D22.18-blue" alt="node">
15
15
  <img src="https://img.shields.io/badge/dsh-%3E%3D0.2.1--rc.2%20%3C0.3.0-blueviolet" alt="dsh">
@@ -19,9 +19,9 @@
19
19
  ## Why this one
20
20
 
21
21
  - **No API key, eight engines.** AnySearch, CN Bing, DuckDuckGo HTML, www Bing, DuckDuckGo Lite and 360 Search are scraped directly; Exa joins the primary tier through its keyless MCP endpoint, and Firecrawl's search API is the measured fallback. The plugin works out of the box.
22
- - **The ranking is measured, not asserted.** Cross-engine reciprocal-rank fusion with authority and freshness boosts, a 20-query offline benchmark, and an 81-configuration parameter sweep — the only search plugin here that publishes both its metric and its confidence intervals. See [How the ranking works](docs/rrf-ranking.md).
22
+ - **The ranking is measured, not asserted.** Cross-engine reciprocal-rank fusion with authority and freshness boosts, a 20-query offline benchmark, and an 81-configuration parameter sweep — one of the few search plugins here that publishes its metric, per-query output and confidence intervals. See [How the ranking works](docs/rrf-ranking.md).
23
23
  - **Fetching admits failure.** Binary documents and empty 200 responses are rejected by name (`SEARCH_UNSUPPORTED`, `SEARCH_EMPTY`) instead of being returned as noise; the direct → tunnel → render → extract ladder shares one 25 s budget and stops when there is no time left. The budget is per request — there is a regression test for exactly that.
24
- - **501 tests across 34 files**, a committed `lib/`, declared `engines.dsh` / `engines.node`, and a [SECURITY.md](SECURITY.md) that lists every process, host and file the plugin touches.
24
+ - **664 tests across 50 files**, a committed `lib/`, declared `engines.dsh` / `engines.node`, and a [SECURITY.md](SECURITY.md) that lists every process, host and file the plugin touches.
25
25
  - **Bilingual documentation.** This README and [中文说明](README.zh-CN.md) are maintained side by side.
26
26
 
27
27
  ![Ranking baseline and parameter sweep](https://raw.githubusercontent.com/fangchenghao/dsh-web-search/main/docs/images/bench.png)
@@ -30,24 +30,51 @@ The benchmark is offline and deterministic: 20 recorded queries, precision/recal
30
30
 
31
31
  ## Where it stands
32
32
 
33
- The web-search corner of the DSH plugin ecosystem is crowded, and most rivals are bigger than this one. An honest snapshot (2026-10-23; check the linked projects for current numbers):
34
-
35
- | | this plugin | [dsh-free-search](https://github.com/DDDMUC/dsh-free-search) | [modsearch](https://github.com/liustack/modsearch) | [anysearch-dsh](https://github.com/anysearch-team/anysearch-dsh) |
36
- |---|---|---|---|---|
37
- | Key-free engines | 8 | ~30 | 7 channels (+X) | 1 official API |
38
- | Dedicated vertical sources | 34 (8 on by default) | 41 | — | — |
39
- | Model-facing tools | 6 | ~10 | 3 | 5 |
40
- | Test suite | 34 files / 501 cases | none | 29 files | 6 files |
41
- | Ranking benchmark | 20 queries + 81-config sweep | — | — | — |
42
- | Store status | source contract satisfied, awaiting review | unlisted | no entry | blocked (review-required) |
43
-
44
- If you need maximum source breadth, use `dsh-free-search`; if you need a cross-harness search bridge, use `modsearch`; if you want the official AnySearch path only, use `anysearch-dsh`. This plugin is the one that treats ranking and failure semantics as the product, and measures them.
33
+ The web-search corner of the DSH plugin ecosystem is crowded. A fresh snapshot
34
+ (2026-10-25; stars and npm downloads are adoption signals, not quality scores):
35
+
36
+ | | this plugin | [dsh-free-search](https://github.com/DDDMUC/dsh-free-search) | [dsh-web-search-pro](https://github.com/anweat/dsh-web-search-pro) | [modsearch](https://github.com/liustack/modsearch) | [anysearch-dsh](https://github.com/anysearch-team/anysearch-dsh) |
37
+ |---|---|---|---|---|---|
38
+ | Key-free path | 8 engines, all key-free | ~30-engine free/paid chain | 5 defaults (seam, Exa, DDG, Bing, Jina) | Firecrawl + Keenable keyless defaults | 1 anonymous AnySearch API |
39
+ | Dedicated vertical sources | 34 (8 on by default) | 41 | platform registry, not a vertical catalog | web + X, no vertical catalog | vertical API behind AnySearch |
40
+ | Model-facing tools | 1 (`web_tools`, 8 actions) | ~10 | 2 (`web_index` / `web_call`, 20 actions) | native search/fetch + skill | 5 |
41
+ | Tests | 50 files / 664 cases, locally verified | no test script in `package.json` | 774 unit + 94 bench, self-reported | Vitest configured; count not published | Vitest configured; count not published |
42
+ | GitHub stars | 0 | 620 | 77 | 836 | 663 |
43
+ | npm downloads, last month | 1,087 | 52,436 | 6,420 | 43,151 | 9,251 |
44
+ | npm unpacked size | 1.1 MB | 0.70 MB | 3.05 MB | 0.51 MB | 0.27 MB |
45
+
46
+ This is a capability snapshot, not a league table with one winner. The honest
47
+ read is:
48
+
49
+ - **Feature depth:** upper-middle. `dsh-free-search` has more engines and
50
+ verticals; `dsh-web-search-pro` has a deeper evidence/ranking pipeline and far
51
+ more platform backends; `modsearch` is the stronger cross-host bridge.
52
+ - **Engineering and verification:** near the top. 664 local tests, a replayable
53
+ ranking benchmark, declared security boundaries and one runtime dependency
54
+ put it alongside the best-maintained entries, while remaining much smaller
55
+ than the Playwright-heavy option.
56
+ - **Key-free + official-source correctness:** its strongest niche. The official
57
+ Chrome, PostgreSQL and AI Act paths and the 34-source vertical layer are not
58
+ common in the other plugins.
59
+ - **Adoption and maturity:** bottom third of this sample. The public npm release
60
+ is 0.20.1, matching this tree, but the GitHub repository is four days old
61
+ with no stars and little independent adoption.
62
+ - **Tool-surface economy:** now competitive. 0.20.x exposes one `web_tools`
63
+ entry with eight actions, close to `dsh-web-search-pro`'s two tools with 20
64
+ actions; the legacy eight-name surface remains opt-in for old prompts.
65
+
66
+ If you need maximum install base and engine breadth, start with
67
+ `dsh-free-search` or `modsearch`. If you need the deepest evidence/scoring
68
+ pipeline and browser-backed platforms, look at `dsh-web-search-pro`. This plugin
69
+ is the compact, fully key-free option for search plus vertical lookup, research
70
+ packs and official-source correction. It is not the most popular, and it should
71
+ not claim to be the largest.
45
72
 
46
73
  > The latest version and install command live on [npm](https://www.npmjs.com/package/@fffangchenghao/dsh-web-search); no version number is hard-coded here, so the docs cannot drift from the release. What follows starts with what this means for you; the technical detail is collected in the appendix.
47
74
 
48
75
  ## In one sentence
49
76
 
50
- With this plugin installed, you can ask something like "how do I configure the version released last week?" and the model can call `web_search` against public search engines by itself, then work from a ranked list of titles, links and snippets. When a snippet is not enough, it can call `web_fetch` on one of those links and get the page back as clean Markdown instead of a wall of HTML. When one question needs several separate searches, it can call `search_many` and get each query's results separately. When the question is about an npm package, a GitHub repository, a paper or a Q&A thread, it can call `domain_search` instead and ask those dedicated sources directly. And when a page is too long to read in one pass, it can call `read_page` and continue from where the last chunk stopped.
77
+ With this plugin installed, you can ask something like "how do I configure the version released last week?" and the model can call `web_search` against public search engines by itself, then work from a ranked list of titles, links and snippets. When a snippet is not enough, it can call `web_fetch` on one of those links and get the page back as clean Markdown instead of a wall of HTML. When one question needs several separate searches, it can call `search_many` and get each query's results separately. When the question is about an npm package, a GitHub repository, a paper or a Q&A thread, it can call `domain_search` instead and ask those dedicated sources directly. When it needs a forum, community or video site, it can call `platform_search` for Bilibili, V2EX, Reddit, YouTube, Zhihu or Juejin. A PostgreSQL version/EOL question also gets the project's official support-table row prepended, rather than trusting a stale press kit. And when a page is too long to read in one pass, it can call `read_page` and continue from where the last chunk stopped.
51
78
 
52
79
  Three things change for you:
53
80
 
@@ -55,7 +82,7 @@ Three things change for you:
55
82
  - When a question is about a package, a repository or a paper, the model can ask the matching source directly instead of hoping a web search understands.
56
83
  - You do not need a search API key, and you do not pay for search separately.
57
84
 
58
- Under the hood it is one sentence too: the plugin registers two providers with the harness's `web` service — `bing` for search and `http-proxy` for fetching pages — and adds four more tools: `domain_search` for packages, repositories, papers, Q&A and reference entries, `search_many` for several searches in one call, `read_page` for walking a long page in chunks, and `research` for turning a few searches into one cited evidence pack. These names come up again below.
85
+ Under the hood it is one sentence too: the plugin registers two providers with the harness's `web` service — `bing` for search and `http-proxy` for fetching pages — and one model-facing `web_tools` action tool: `action: "domain_search"` for packages, repositories, papers, Q&A and reference entries; `search_many` for several searches in one call; `advanced_search` for one query with a per-call date window; `read_page` for walking a long page in chunks; `research` for turning a few searches into one cited evidence pack; `platform_search` for community and video sites; `cache_search` for searching the full texts research already fetched; and `engine_test` for checking which engine is healthy. These are eight capabilities of one plugin, behind one tool registration — not eight separate plugins or eight always-visible tools.
59
86
 
60
87
  ## The problem it solves
61
88
 
@@ -152,7 +179,7 @@ Analogy: `web_search` is asking passers-by where the hardware shop is and gettin
152
179
 
153
180
  They team up well: use `domain_search` to find the package or paper, then `web_fetch` to open its homepage or README and read the detail.
154
181
 
155
- Three more helpers sit beside them: **`search_many`**, **`read_page`** and **`research`**. When one question needs several distinct searches — one per candidate library, say — `search_many` runs one to five queries in a single call and keeps each query's results apart, so one failed query cannot empty the batch; when a page is too long for one fetch, `read_page` continues from where the previous chunk stopped; and when snippets are not enough but the model still has to decide which page to trust, `research` searches up to three queries, opens the leading pages and returns one evidence pack with a citation under every excerpt.
182
+ Four more helpers sit beside them: **`search_many`**, **`advanced_search`**, **`read_page`** and **`research`**. When one question needs several distinct searches — one per candidate library, say — `search_many` runs one to five queries in a single call and keeps each query's results apart, so one failed query cannot empty the batch; when only one question needs a date window, `advanced_search` applies it for that call alone; when a page is too long for one fetch, `read_page` continues from where the previous chunk stopped; and when snippets are not enough but the model still has to decide which page to trust, `research` searches up to three queries, opens the leading pages and returns one evidence pack with a citation under every excerpt.
156
183
 
157
184
  ### Reading a long page in chunks (read_page): when one pass is not enough
158
185
 
@@ -250,7 +277,7 @@ Each line is the plain version first, the technical note second. The technical n
250
277
 
251
278
  7. **Repeating a question does not repeat the work.**
252
279
  The same query within ten minutes is answered from the previous run, almost instantly.
253
- *Technical: TTL + LRU cache, `cacheTtlMs: 600000` (ten minutes) and `cacheEntries: 100`; the key is `lead|wait|limit|sorted engine ids|time window|normalised query`, where the window only appears when it is not `off`, so an unfiltered search keeps its old cache behaviour.*
280
+ *Technical: TTL + LRU cache, `cacheTtlMs: 600000` (ten minutes) and `cacheEntries: 100`; the key is `lead|wait|limit|sorted engine ids|time window|policy fields|normalised query`, where the window only appears when it is not `off` and the policy fields only when they are non-default, so an ordinary search keeps its old cache behaviour.*
254
281
 
255
282
  8. **A throttled engine takes a rest.**
256
283
  If one engine serves a verification page, it is skipped for a few minutes while the rest keep working.
@@ -277,8 +304,8 @@ Each line is the plain version first, the technical note second. The technical n
277
304
  *Technical: proxy sources in order — the `proxy` setting, then `DSH_WEB_SEARCH_PROXY` / `HTTPS_PROXY` / `ALL_PROXY` / `HTTP_PROXY`, then macOS `scutil --proxy`; HTTP proxies only (`CONNECT`), not `socks5://`.*
278
305
 
279
306
  14. **Every setting is editable in the UI.**
280
- Twenty-six controls in five groups; a save takes effect on the next search, apart from `cachePersist` and `cacheFile`, which decide the cache wiring at mount.
281
- *Technical: all twenty-six fields carry `.volatile()`, so the Host commits them in place instead of remounting the plugin; `cacheTtlMs` and `cacheEntries` are re-read per use, `fetchFormat` and `fetchMaxLength` are resolved per request, `renderJs` and `renderTimeoutMs` per fetch; `proxy` and `timeoutMs` are taken at mount, and so is the cache wiring that `cachePersist` and `cacheFile` decide.*
307
+ Thirty controls in six groups; a save takes effect on the next search, apart from `cachePersist` and `cacheFile` (cache wiring at mount) and `toolSurface` (the tool registry is built at mount).
308
+ *Technical: all thirty fields carry `.volatile()`, so the Host commits them in place instead of remounting the plugin; `cacheTtlMs` and `cacheEntries` are re-read per use, `fetchFormat` and `fetchMaxLength` are resolved per request, `renderJs` and `renderTimeoutMs` per fetch; `proxy` and `timeoutMs` are taken at mount, and so is the cache wiring that `cachePersist` and `cacheFile` decide. `toolSurface` is read at mount when the registry is built, so a change needs a reload even though the value is committed live.*
282
309
 
283
310
  15. **Failures are explained, not silently empty.**
284
311
  When nothing succeeds, the error lists which engines were tried and why each one failed.
@@ -290,15 +317,15 @@ Each line is the plain version first, the technical note second. The technical n
290
317
 
291
318
  17. **Packages, repositories, papers and Q&A are one hop away.**
292
319
  For "is this npm package still maintained?" or "is there a paper on RAG?", the model can call `domain_search` against 34 dedicated sources instead of hoping a web search finds them.
293
- *Technical: 34 keyless sources, eight on by default — `github`, `npm`, `pypi`, `arxiv`, `wikipedia`, `hn` (Hacker News), `stackexchange`, `crates` (crates.io); the other 26 are reachable through `sources`. Arguments `query` (required), `sources` (optional array, default = the default eight), `limit` (optional, default 5, capped at 20). Each source has its own deadline (8000 ms by default), one failure never affects the others, and the failed names come back in `failures`. Measured with `react` at limit 3: github 3, npm 3, pypi 1, arxiv 3, wikipedia 3, hn 3, stackexchange 3, crates 3. Known limits: PyPI has no public search API, so it matches the **exact package name** taken from the first word of the query; Wikipedia searches one language only (`zh` for CJK, otherwise `en`). A host without `ctx.tools` does not register the tool and the rest of the plugin works as usual.*
320
+ *Technical: 34 keyless sources, eight on by default — `github`, `npm`, `pypi`, `arxiv`, `wikipedia`, `hn` (Hacker News), `stackexchange`, `crates` (crates.io); the other 26 are reachable through `sources`. Arguments `query` (required), `sources` (optional array, default = the default eight), `limit` (optional, default 5, capped at 20), and `timeRange` (optional `off`/`day`/`week`/`month`/`year`; undated rows survive only when no dated row qualifies, with a `note`). With a window, the arXiv adapter asks for `submittedDate` order and quotes the phrase before the local date filter. Each source has its own deadline (8000 ms by default), one failure never affects the others, and the failed names come back in `failures`. Measured with `react` at limit 3: github 3, npm 3, pypi 1, arxiv 3, wikipedia 3, hn 3, stackexchange 3, crates 3. Known limits: PyPI has no public search API, so it matches the **exact package name** taken from the first word of the query; Wikipedia searches one language only (`zh` for CJK, otherwise `en`). A host without `ctx.tools` does not register the tool and the rest of the plugin works as usual.*
294
321
 
295
322
  18. **You can look only at recently published results.**
296
323
  Choose "last 24 hours / 7 days / 30 days / 1 year" in Settings and only results inside that window stay — useful for news and releases.
297
- *Technical: `timeRange` takes `off` (default) / `day` / `week` / `month` / `year` — 24 hours, 7 days, 30 days, 1 year. Publication dates are parsed from each engine's results (Bing's snippet date and `news_dt`, 360's `发贴时间`, DuckDuckGo's timestamp and snippet, AnySearch's `date:` trailer). DuckDuckGo receives the window as `df=d/w/m/y`; Bing's and 360's parameters were measured to have no effect (the page shows the option selected while the results stay the same), so those two and AnySearch filter locally. With a window on, only **dated** results inside it are kept; only when not one dated result falls inside does the undated list come back, so the answer is never empty for that reason alone.*
324
+ *Technical: `timeRange` takes `off` (default) / `day` / `week` / `month` / `year` — 24 hours, 7 days, 30 days, 1 year. Publication dates are parsed from each engine's results (Bing's snippet date and `news_dt`, 360's `发贴时间`, DuckDuckGo's timestamp and snippet, AnySearch's `date:` trailer). DuckDuckGo receives the window as `df=d/w/m/y`; Bing's and 360's parameters were measured to have no effect (the page shows the option selected while the results stay the same), so those two and AnySearch filter locally. With a window on, only **dated** results inside it are kept; only when not one dated result falls inside does the undated list come back, so the answer is never empty for that reason alone. This Settings window is global; `advanced_search` can override it for one model call (see below).*
298
325
 
299
326
  19. **Several questions, one call.**
300
327
  For "compare these three libraries", the model can hand `search_many` one to five queries and get each query's results back separately; up to three run at once, and one failed query does not take the others down.
301
- *Technical: `queries` (1–5, required), `limit` (default 5, 1–20), `withSites` (default false; true attaches each result's hostname). `SEARCH_MANY_CONCURRENCY = 3`. Each query goes through the same engine layer, cache and cooldown tracker as `web_search`; a thrown search becomes that query's `failures` entry, never the batch's. Use `search_many` for several questions at once, `web_search` for one, `domain_search` for packages, papers or Q&A. The tool's deadline is 30 s and it is concurrency-safe. It registers as `web_search_many` if the name is taken, and is not registered at all when the host has no `ctx.tools`.*
328
+ *Technical: `queries` (1–5, required), `limit` (default 5, 1–20), `withSites` (default false; true attaches each result's hostname). `SEARCH_MANY_CONCURRENCY = 3`. Each query goes through the same engine layer, cache and cooldown tracker as `web_search`; a thrown search becomes that query's `failures` entry, never the batch's. A query that explicitly names npm, PyPI or crates.io is routed to that package registry; it falls back to the ordinary path if the registry returns nothing. Use `search_many` for several questions at once, `web_search` for one, `domain_search` for packages, papers or Q&A. The tool's deadline is 30 s and it is concurrency-safe. It registers as `web_search_many` if the name is taken, and is not registered at all when the host has no `ctx.tools`.*
302
329
 
303
330
  20. **The cache survives a restart.**
304
331
  Ask the same question again after restarting DSH within ten minutes and the previous result is still there; nothing is sent to the engines.
@@ -309,11 +336,135 @@ Each line is the plain version first, the technical note second. The technical n
309
336
  *Technical: `bench/` holds 20 queries (10 Chinese, 10 English), five in each of docs / release / package / news, each annotated with 1–3 expected domains. `node scripts/bench.mjs` replays recorded fixtures offline, never touches the network, and prints precision@8, recall@8, MRR@8 plus each engine's contribution; `--record` re-records live (1.5 s between queries) to refresh the fixtures; `--json` gives machine-readable output for version comparisons, and past results live in `bench/results/`. Baseline for 0.15.0 (tier-aware replay, so the recorded fallback engines only count when production would run them): P@8 `0.4375`, R@8 `0.7917`, MRR@8 `0.8071`; Chinese `0.4000 / 0.8167 / 0.7976`, English `0.4750 / 0.7667 / 0.8167`. A separate five-query long-Chinese diagnostic (`bench/queries-long.json`, recorded with `--queries ... --fixtures ...`) scores `0.4500 / 1.0000 / 0.7667`, with every query hitting an expected domain; it is kept separate so the 20-query baseline stays comparable. The controlled experiments and known gaps are in the appendix.*
310
337
 
311
338
  22. **Research, in one call.**
312
- For "what do the official docs and two other sources actually say about X", `research` runs up to three searches, opens the leading pages through the same fetch ladder, and returns one evidence pack in which every excerpt sits under its URL. A page that will not load is reported by URL and reason instead of aborting the pack.
313
- *Technical: `query` (required), `queries` (up to three extra), `maxSources` (1–5, default 3); one 45 s budget for the whole call, three page fetches in flight, and a page fetch is not started with less than 2.5 s left. The tool registers as `web_research` if another plugin owns `research`. Its fetch side uses its own `http-proxy-research` provider so it cannot disturb the host `web_fetch` renderer cache. The search side is the same `web_search` path, cache and cooling included.*
339
+ For "what do the official docs and two other sources actually say about X", `research` runs up to three searches, opens the leading pages through the same fetch ladder, and returns one evidence pack in which every excerpt sits under its URL. A page that will not load is reported by URL and reason instead of aborting the pack; bot challenges and login redirects are rejected rather than quoted as evidence.
340
+ *Technical: `query` (required), `queries` (up to three extra), `maxSources` (1–5, default 3); one 45 s budget for the whole call, three page fetches in flight, and a page fetch is not started with less than 2.5 s left. A credible government, academic, standards-body or well-known official documentation page that matches the query leads the pack; a landing/home page is never promoted by authority alone, and relevance still excludes unrelated official pages. Up to the remaining query slots (three in total) can be spent on a focused official `site:` query (EU/europa, Chinese CAC, Chrome Releases, Python, Redis, W3C), with results filtered to the issuer's domain. The fetch side reads up to 200,000 characters so the excerpt selector can find a fact deep in the page; it uses its own `http-proxy-research` provider and does not disturb the host `web_fetch` renderer cache. W3C WCAG pages that return 403 are retried once through `w3c.github.io`; the body comes from that official mirror and the citation names the canonical W3C page. The search side is the same `web_search` path, cache and cooling included. The tool registers as `web_research` if another plugin owns `research`.*
341
+
342
+ ### Community and video platforms
343
+
344
+ `platform_search` is the forum/video counterpart to `domain_search`. It asks
345
+ Bilibili, V2EX, Reddit, YouTube, Zhihu and Juejin directly through keyless
346
+ public endpoints where one exists, and falls back to a `site:<domain>` search
347
+ when the endpoint is blocked or has changed. One platform failing does not
348
+ empty the answer; the result names the platform, the URL, the snippet and a
349
+ date when one is available. Fallback rows are filtered to the platform's own
350
+ domain, so a generic page cannot masquerade as a platform result. Bilibili retries `/search/all/v2` when `/search/type` answers 412, applies recency and play-count sorting only when the query asks for them, and Juejin rows expose like/read counts when the API provides them.
351
+
352
+ *Technical: `query` (required), `platforms` (optional subset of `bilibili`,
353
+ `v2ex`, `reddit`, `youtube`, `zhihu`, `juejin`; omitted = all six), `limit`
354
+ (1–10, default 5). Direct requests use a browser UA and the configured proxy;
355
+ falls back to `site:<domain> <query>` through the normal search path. Each
356
+ platform has an 8 s direct deadline, the tool has a 40 s budget, results are
357
+ de-duplicated and interleaved by rank. It registers as
358
+ `web_platform_search` if the name is taken. Zhihu has no usable keyless search
359
+ endpoint, so it is fallback-only; if the fallback is blocked as well, it is
360
+ reported as a platform failure rather than silently omitted.*
361
+
362
+ ### A second retrieval over research bodies
363
+
364
+ `research` fetches full pages but only puts selected passages into the evidence
365
+ pack. `cache_search` searches those full texts again, in-process, without a
366
+ second network request. This is useful when a first answer needs an exact
367
+ number, quote or section: ask `cache_search` for the detail and it returns the
368
+ matching passages with the same source URLs. The cache holds at most 24 bodies,
369
+ 400,000 characters each, expires after seven days and is cleared when the plugin
370
+ unloads; it never writes page bodies to disk.
371
+
372
+ The first retrieval is also query-aware: the excerpt is not simply the first
373
+ 6,000 characters. Research splits the fetched body into windows, scores them
374
+ against the query terms and keeps the best windows plus enough opening context
375
+ to make the excerpt intelligible. This is deterministic and makes no model
376
+ call.
377
+
378
+ *Technical: `cache_search` takes `query` and optional `limit` (1–10, default 3);
379
+ it returns `matches` with `url`, `title`, `snippet`, `score` and `fetchedAt`.
380
+ The cache is in-memory only; a restart is a clean slate. `research` fetches up
381
+ to 200,000 characters per page for this second pass and stores the full body
382
+ via an optional `onPage` sink.*
383
+
384
+ 23. **You can test the engines without guessing.**
385
+ A model tool and a Settings button both run one tiny query through each engine and report which answered, how fast, and why the rest did not. It is diagnostics only: no cache entry, no cooldown, no fallback.
386
+ *Technical: `engine_test` takes optional `engine` and `query`, tests one engine or all eight (three in flight), and returns per-engine `ok`, `latencyMs`, `count`, `failureKind`, `code`, `status` and `reason`. The Settings button posts to a loopback-only `POST /api/dsh-web-search/engine-test` bridge, which shares the same runner. The endpoint refuses non-POST, non-loopback and cross-origin callers; it is registered only when the host exposes an injectable `webServer`, so a headless deployment simply has no button.*
387
+
388
+ 24. **A failure carries a class, and the fallback is a policy.**
389
+ "Blocked", "out of quota" and "temporarily slow" used to produce the same fallback; now each primary failure is classified, and you can decide which classes may switch to a backup engine. When a fallback is skipped or used, the result says so.
390
+ *Technical: `FailureKind` is `quota` / `auth` / `bot-wall` / `transient` / `invalid-response` / `unknown`, derived from the existing `SearchError` code plus HTTP status and message clues. `fallbackOn` defaults to all six, preserving the old always-fall-back behaviour; an empty list means never. `SearchOutcome.failureDetails` exposes the class, code and status, and `note` explains a skipped or used fallback (also rendered by `search_many`). Cooldown timing is unchanged.*
391
+
392
+ 25. **A site can be blocked, and safe search can be forwarded.**
393
+ Put a domain on the blocklist and it disappears from merged and cached results; choose a safe-search level and Bing and DuckDuckGo receive it.
394
+ *Technical: `blockedDomains` is a comma-separated suffix list (a subdomain matches its parent), applied after fusion and to cache hits; empty by default. `safeSearch` is `off` (leave the engine's own default), `moderate` or `strict`, forwarded as Bing `adlt` and DuckDuckGo `kp`; other engines are untouched. A non-empty blocklist and a non-default safe-search level join the cache key, so removing a block cannot reuse a filtered entry, while the default path keeps its old keys.*
395
+
396
+ 26. **A date window is available per call, not only as a setting.**
397
+ When only one question needs fresh results — "what changed this week?" — the model can call `advanced_search` and ask for `day`, `week`, `month` or `year` without changing the global setting. Omitting the window keeps the Settings value; `off` disables it for that one query.
398
+ *Technical: `advanced_search` takes `query` (required), `maxResults` (default 5, capped at 20 and by the deployment's `limit`), and `timeRange` (`off` / `day` / `week` / `month` / `year`; omitted = Settings). It shares the engine, cache, cooldown and failure path with `web_search`, so an `off` call can reuse an ordinary cache entry and a `week` call gets its own key. With an explicit window it removes a redundant `过去 7 天内` / `last 7 days` phrase from the engine query (the structured window already carries it) while returning the original query in the result. For a windowed Chrome security/release query, it reads the official Chrome Releases feed and returns the feed's real publication dates plus a note saying whether any title or summary mentions zero-day; a Stable-only query filters out Beta/Dev/Early Stable/Extended Stable posts, and each post snippet previews the version, fix count and listed CVE severity distribution when the post states them. A feed miss or an empty Stable filter falls back to the ordinary engine path. It registers as `web_search_advanced` if the name is taken, and is not registered at all when the host has no `ctx.tools`.*
399
+
400
+ 27. **Web text is explicitly marked as untrusted.**
401
+ Every custom tool now wraps its web-derived body in `<untrusted-web-content>...</untrusted-web-content>`, strips forged copies of the markers from the payload, and a short system-prompt section tells the model that the contents are data, not instructions.
402
+ *Technical: the wrapper is applied to `domain_search`, `search_many`, `advanced_search`, `read_page`, `research` and `engine_test` renders. The boundary tags are removed from page text (including whitespace/attribute variants) before wrapping, so a page cannot close the boundary early. The host's own `web_search`/`web_fetch` external-content notice is left in place; the plugin does not duplicate it or mutate `Source` objects that DSH also renders in result cards. The prompt section is installed through an optional `systemPrompt` injection and is absent on hosts without that service.*
314
403
 
315
404
  > **What 0.13.0 fixes (a real defect)**: from some networks 360 (`so`) returns a verification page — a 302 to the qcaptcha "unexpected access" page — while the HTTP status is still 200. The old code read that as "zero results" and swallowed it: the engine effectively disappeared and its cooldown never fired. Such pages now fail as `SEARCH_BLOCKED`, and 360 has been demoted to the **fallback tier** (tried only when the primary engines are empty), so it no longer slows every search. Measured: querying that engine alone now returns the failure `so: bot verification page` instead of zero results.
316
405
 
406
+ ## Tool surface: one entry point, eight actions
407
+
408
+ 0.20.x changes the model-facing surface. The eight capabilities above are now
409
+ actions on one `web_tools` tool instead of eight separate registrations. A call
410
+ looks like:
411
+
412
+ ```json
413
+ { "action": "domain_search", "query": "react", "sources": ["github"] }
414
+ ```
415
+
416
+ The host's own `web_search` and `web_fetch` stay as they were. This keeps the
417
+ common path unchanged while cutting the optional tool surface from eight names
418
+ to one.
419
+
420
+ Older prompts that call `domain_search`, `search_many`, `research` and the rest
421
+ directly can opt back into the old names in the plugin config:
422
+
423
+ ```yaml
424
+ - id: web-search-bing
425
+ name: '@fffangchenghao/dsh-web-search'
426
+ config:
427
+ toolSurface: legacy
428
+ ```
429
+
430
+ `unified` is the default and is recommended. Changing the mode requires a
431
+ plugin reload because the registry is built at mount time. The sections below
432
+ still use the action names as labels; read them as `web_tools` actions unless
433
+ you explicitly chose `legacy`.
434
+
435
+ ## Automatic updates
436
+
437
+ The plugin is a DSH bundle and does **not** replace its own running code: Node
438
+ has already loaded `lib/index.js`, and swapping files underneath it is not a
439
+ safe update mechanism. Automatic updates belong to the profile-level updater.
440
+
441
+ Use an npm install, not a `link:`/local checkout:
442
+
443
+ ```sh
444
+ dsh plugin --profile web add @fffangchenghao/dsh-web-search@latest
445
+ ```
446
+
447
+ Then keep a profile updater such as `dsh-autoupdater` (or
448
+ `dsh-self-upgrade`) enabled. The updater checks the npm dist-tag,
449
+ updates in an idle window, and asks for a restart on desktop. `link:` and
450
+ `file:` installs are deliberately skipped by such updaters because replacing a
451
+ local checkout with the registry package would destroy the checkout.
452
+
453
+ The Settings card always shows the latest npm version. If it is newer and the
454
+ page is local, it also shows **Install update**. That button delegates to the
455
+ Host's built-in `pluginManager` (the same service `dsh plugin` uses), installs
456
+ the exact npm package and tells you to restart. It works **without**
457
+ `dsh-autoupdater`; what it does not do is run periodically. The remote web
458
+ surface hides the button, and a `link:`/`file:` checkout is refused before it
459
+ can be replaced. If the deployment has no plugin manager, the button reports
460
+ that instead of pretending to update.
461
+
462
+ Automatic updates trade review for freshness: the updater installs the
463
+ registry package without a human reading the diff. If that trade is wrong for
464
+ your deployment, pin an exact version and leave the profile updater off. The
465
+ plugin never spawns `pnpm` itself; the manual button only asks the Host to do
466
+ it.
467
+
317
468
  ## Installing and enabling it
318
469
 
319
470
  ### What you need
@@ -323,6 +474,17 @@ Each line is the plain version first, the technical note second. The technical n
323
474
  - Ordinary outbound network access. A tunnel is not required — you just get fewer engines without one.
324
475
  - Optional: to read pages whose body is built by JavaScript, install Playwright yourself (item 12 above); not installing it changes nothing else.
325
476
 
477
+ ### Production install (auto-updatable)
478
+
479
+ For a normal user install, add the published package to the profile. Do not use
480
+ a `link:` if you want the profile updater to manage it:
481
+
482
+ ```sh
483
+ dsh plugin --profile web add @fffangchenghao/dsh-web-search@latest
484
+ ```
485
+
486
+ The rest of this section is the source-checkout development path.
487
+
326
488
  ### Step 1: build
327
489
 
328
490
  `main` points at `lib/index.js`, and `lib/` is **committed** — DSH STORE scans the fixed Git tree, so the runtime artifacts have to live in the repository — which means a fresh checkout runs as-is. Rebuild only after changing `src/`:
@@ -409,7 +571,7 @@ Values saved from the Settings page apply immediately; only `cachePersist` and `
409
571
 
410
572
  Any one of these:
411
573
 
412
- 1. **Look at Settings**: Settings → "Web search (Bing)" should show five groups of controls, 26 in all. That means the plugin loaded and the Host accepted its schema.
574
+ 1. **Look at Settings**: Settings → "Web search (Bing)" should show six groups of controls, 30 in all. That means the plugin loaded and the Host accepted its schema.
413
575
  2. **Look at the log**: no warning such as `1 entry did not activate`.
414
576
  3. **Just try it**: ask something that obviously needs the network, such as "search for a DSH plugin tutorial and give me the links". Watch whether `web_search` is called and whether the links are real.
415
577
 
@@ -417,17 +579,18 @@ If the Settings card appears but search still fails, start with [docs/troublesho
417
579
 
418
580
  ## Using the Settings page
419
581
 
420
- Open Settings → "Web search (Bing)". Twenty-six controls, arranged in five groups. A sticky jump bar at the top scrolls to any group, and a line under the title compares the installed version with the latest one on npm (it stays hidden when the registry cannot be reached, so it never guesses). Each group owns one kind of decision:
582
+ Open Settings → "Web search (Bing)". Thirty controls, arranged in six groups, plus an **Engine health** check. A sticky jump bar at the top scrolls to any group, and a line under the title compares the installed version with the latest one on npm (it stays hidden when the registry cannot be reached, so it never guesses). Each group owns one kind of decision:
421
583
 
422
584
  | Group | Controls | What it decides |
423
585
  | --- | --- | --- |
424
- | Engines and transport | 6 | Which engines run, how long to wait, whether to use a proxy, AnySearch credentials and vertical domain |
586
+ | Engines and transport | 9 | Which engines run, how long to wait, whether to use a proxy, AnySearch credentials and vertical domain, blocked sites, safe search and fallback policy |
425
587
  | Results and ranking | 7 | How many results, how far back to look, how they merge and sort, how much official docs and fresh content count |
426
588
  | Dedicated search | 3 | Which sources `domain_search` queries, results per source, deadline per source |
427
589
  | Fetching | 4 | What `web_fetch` returns — Markdown or plain text, how many characters, whether a browser may render a shell, and how long that render may take |
428
590
  | Cache and cooldowns | 6 | How long results are remembered, how many, whether the cache is saved to disk, and how long a throttled engine rests |
591
+ | Model tools | 1 | Whether the eight optional capabilities appear as one `web_tools` action tool or the legacy eight names. Requires a reload |
429
592
 
430
- ### Engines and transport (6)
593
+ ### Engines and transport (9)
431
594
 
432
595
  | Control | Key | Default | Plain meaning |
433
596
  | --- | --- | --- | --- |
@@ -437,13 +600,16 @@ Open Settings → "Web search (Bing)". Twenty-six controls, arranged in five gro
437
600
  | Proxy (relay) | `proxy` | empty (detect) | e.g. `http://127.0.0.1:7890`; empty reads the environment, then the system proxy |
438
601
  | AnySearch key | `anysearchApiKey` | empty (anonymous) | Optional. Sent only to `api.anysearch.com`; raises limits and unlocks page extraction. Several keys may be comma-separated and are rotated, with a throttled or rejected one resting automatically |
439
602
  | Vertical domain | `anysearchDomain` | empty (general) | Typical values: `finance` / `academic` / `security` / `legal` / `code`; a dotted value such as `finance.us_stock` selects a sub-domain |
603
+ | Blocked sites | `blockedDomains` | empty (nothing) | Domains that must never appear, comma-separated; a subdomain matches its parent. Applied after fusion and on a cache hit, so a blocked row cannot come back from the cache |
604
+ | Safe search | `safeSearch` | `off` | `off` keeps each engine's default; `moderate` or `strict` is forwarded to Bing (`adlt`) and DuckDuckGo (`kp`). Other engines ignore it |
605
+ | Fallback on | `fallbackOn` | all six classes | Failure classes that may switch to a fallback engine: `quota`, `auth`, `bot-wall`, `transient`, `invalid-response`, `unknown`. Empty means never fall back; a skipped fallback is explained in the result's `note` |
440
606
 
441
607
  ### Results and ranking (7)
442
608
 
443
609
  | Control | Key | Default | Plain meaning |
444
610
  | --- | --- | --- | --- |
445
611
  | Results per search | `limit` | `8` | How many results at most, 1–20; a smaller caller budget still wins |
446
- | Date window | `timeRange` | `off` | `off` disables it; `day` 24 hours, `week` 7 days, `month` 30 days, `year` 1 year. A window keeps only results inside it |
612
+ | Date window | `timeRange` | `off` | `off` disables it; `day` 24 hours, `week` 7 days, `month` 30 days, `year` 1 year. A window keeps only results inside it. `advanced_search` can override it for one query |
447
613
  | Result blending | `rrfConstant` | `60` | Larger flattens the curve; smaller favours each engine's own top results |
448
614
  | Trusted sites | `authorityDomains` | empty (built-in list) | Extra domains to boost, on top of the built-in official-docs list |
449
615
  | Trusted-site boost | `authorityBoost` | `0.25` | Multiplies an authoritative result by `1 + this`, 0–10 |
@@ -478,9 +644,15 @@ Open Settings → "Web search (Bing)". Twenty-six controls, arranged in five gro
478
644
  | Cooldown (ms) | `baseCooldownMs` | `300000` | First rest after throttling, five minutes |
479
645
  | Max cooldown (ms) | `maxCooldownMs` | `1800000` | Ceiling for the doubling backoff, thirty minutes |
480
646
 
647
+ ### Model tools (1)
648
+
649
+ | Control | Key | Default | Plain meaning |
650
+ | --- | --- | --- | --- |
651
+ | Tool surface | `toolSurface` | `unified` | `unified` registers one `web_tools` with eight actions; `legacy` keeps the old eight standalone names for existing prompts. Changing it builds a different registry, so reload the plugin after saving |
652
+
481
653
  ### My advice
482
654
 
483
- **For normal use: leave all twenty-six alone. The defaults are the tuned values.**
655
+ **For normal use: leave all thirty alone. The defaults are the tuned values.**
484
656
 
485
657
  Only these situations are worth changing:
486
658
 
@@ -859,10 +1031,15 @@ Resolution order: the `proxy` setting → the environment (`DSH_WEB_SEARCH_PROXY
859
1031
  - **The benchmark is only 20 queries**: news recall is 0.367 and `zh-linux-612` has zero hits; a difference of 1–4 hits cannot separate versions. It shows direction, not statistical significance.
860
1032
  - **`search_many` is capped at five queries and three in flight.**
861
1033
  - **Two processes sharing one cache file write last-writer-wins**: neither can corrupt the file, but one process's newer snapshot can be replaced by another's older one.
862
- - **A time window can only filter results that carry a date**: dates are parsed from result pages, so an engine that prints none leaves that row without an age; when nothing dated falls inside the window the undated list comes back. Bing's and 360's time parameters were measured to have no effect; only DuckDuckGo's `df=` is a real pushdown.
1034
+ - **A time window can only filter results that carry a date**: dates are parsed from result pages, so an engine that prints none leaves that row without an age; when nothing dated falls inside the window the undated list comes back, with a `Note` saying the window is not proven for those rows. Bing's and 360's time parameters were measured to have no effect; only DuckDuckGo's `df=` is a real pushdown.
1035
+ - **Domain search is discovery, not a complete record**: GitHub search omits release tags, Wikipedia search omits revision timestamps, Hacker News search omits the top comment, and Stack Overflow search omits accepted-answer text. Follow up with `web_fetch` or `read_page` when the exact record matters.
863
1036
  - **PyPI matches exact package names only**: it has no public search API, so the plugin uses the first word of the query; a typo yields an empty answer.
1037
+ - **Community platforms are best-effort.** Bilibili, V2EX, Reddit, YouTube and Juejin change or block their public endpoints; the tool falls back to a domain-scoped web search and filters the rows to the platform host. Zhihu has no usable keyless API in this environment, so it is fallback-only and can still return nothing.
1038
+ - **`cache_search` is in-process.** Research bodies live for at most seven days, 24 pages and 400,000 characters each, and disappear on restart; they are never written to the search-result cache.
1039
+ - **Official-site hints are bounded.** `research` adds a focused `site:` query only for a small set of issuers (EU, CAC, Chrome Releases, Python, Redis, W3C); other official bodies rely on the ordinary ranking.
1040
+ - **A window cannot invent dates.** When all rows are undated, `advanced_search` / `domain_search` say so in the note; they do not label an undated row as in-window.
864
1041
  - **Wikipedia searches one language only**: `zh` for CJK queries, `en` otherwise.
865
- - **`domain_search`, `search_many` and `read_page` need a host tool registry**: without `ctx.tools` they are not registered, and `web_search` is unaffected.
1042
+ - **The plugin-owned tools need a host tool registry**: without `ctx.tools`, `domain_search`, `search_many`, `advanced_search`, `read_page`, `research`, `cache_search`, `platform_search` and `engine_test` are not registered, and `web_search` is unaffected.
866
1043
  - **Anonymous AnySearch caps at 10 results per call.**
867
1044
  - **HTTP proxies only** — no `socks5://`.
868
1045
  - **`limit` is capped at 20, and so is `domainLimit`.**
@@ -873,6 +1050,9 @@ Resolution order: the `proxy` setting → the environment (`DSH_WEB_SEARCH_PROXY
873
1050
  | --- | --- |
874
1051
  | Result budget `limit` | `8` (1–20) |
875
1052
  | Time window `timeRange` | `off` (no limit; `day` / `week` / `month` / `year`) |
1053
+ | `domain_search` window | omitted (uses no local filter); `off` / `day` / `week` / `month` / `year` |
1054
+ | `platform_search` limit | `5` (1–10) |
1055
+ | Research body cache | 24 pages, 400,000 characters, 7 days, in memory |
876
1056
  | Results per source `domainLimit` | `5` (1–20) |
877
1057
  | Per-source deadline `domainTimeoutMs` | `8000` ms |
878
1058
  | Cache TTL `cacheTtlMs` | `600000` ms = 600 s |