@fffangchenghao/dsh-web-search 0.24.0 → 0.24.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,225 @@
1
+ # Troubleshooting
2
+
3
+ Symptom first, cause second, fix third. Every command below is meant to be run
4
+ from the repository root.
5
+
6
+ - [Architecture](architecture.md)
7
+ - [Engines](engines.md)
8
+ - [README](../README.md) · [中文说明](../README.zh-CN.md)
9
+
10
+ ## `web_search` fails with "no search engine returned results"
11
+
12
+ That message is the provider's own, and the interesting part is in the
13
+ parentheses: one `id: reason` entry per engine that was tried. Read it before
14
+ changing anything.
15
+
16
+ ```
17
+ no search engine returned results for "…" (anysearch: HTTP 429; cn.bing: page does not match the query (possibly rate limited); …)
18
+ ```
19
+
20
+ | Reason in the list | Meaning | What to do |
21
+ | --- | --- | --- |
22
+ | `no results` | the engine answered 2xx and parsed to zero sources | markup may have changed; check that engine's parser against a fresh capture |
23
+ | `page does not match the query (possibly rate limited)` | the echoed-query check failed: a block/trending page for someone else's query | let the cooldown pass, or search a different query |
24
+ | `HTTP 403` / `HTTP 429` | the engine decided this client is a robot or is throttling it | cooldown applies; slow down |
25
+ | `still running at deadline` | the soft deadline released the tier first | `wait: all`, a larger `timeoutMs`, or accept it |
26
+ | `aborted once enough results had arrived` | the tier had enough and stopped waiting | not an error |
27
+ | an `ECONNRESET`/`ENOTFOUND` style message | transport failure | check the tunnel, see below |
28
+
29
+ If *every* reason mentions the tunnel or a timeout, and the `global` engines are
30
+ missing from the list entirely, the run had no proxy: `viaProxy` engines
31
+ (`ddg-html`, `www.bing`, `ddg-lite`) are filtered out rather than failing slowly.
32
+ See [proxy not detected](#proxy-is-not-detected).
33
+
34
+ To narrow a failure down, restrict the run to one engine and watch the reason
35
+ change:
36
+
37
+ ```yaml
38
+ - id: web-search-bing
39
+ name: dsh-web-search-bing
40
+ config:
41
+ hosts: [anysearch]
42
+ ```
43
+
44
+ ## The provider is not selected
45
+
46
+ `web_search` still behaves like the built-in provider → `web.searchProvider` is
47
+ not `bing`. The provider id is fixed (`PROVIDER_ID = 'bing'`) and `available()`
48
+ returns `true` unconditionally, so selection is purely configuration — there is
49
+ no credential or health gate to blame.
50
+
51
+ ```yaml
52
+ - id: web
53
+ name: "@deepseek-ai/dsh-web"
54
+ config:
55
+ searchProvider: bing
56
+ fetchProvider: http
57
+ ```
58
+
59
+ Check that the plugin is actually mounted and registers itself. This snippet
60
+ loads the built entry, calls `apply` with a stub context and prints the provider
61
+ id — no network access:
62
+
63
+ ```sh
64
+ NODE=~/.dsh/dsh-runtimes/dsh-primary-runtime/dependencies/node/bin/node
65
+ $NODE -e "
66
+ import('file://$HOME/.dsh/plugins/dsh-web-search-bing/lib/index.js').then((m) => {
67
+ let p; m.apply({ web: { registerSearchProvider: (x) => { p = x } } }, m.Config({}));
68
+ console.log(p.id, p.available());
69
+ });"
70
+ ```
71
+
72
+ Expected output: `bing true`. If the import fails with `ERR_MODULE_NOT_FOUND`,
73
+ `lib/` has not been built — see [below](#lib-is-missing).
74
+
75
+ ## `lib/` is missing
76
+
77
+ `package.json` points `main` and `types` at `lib/`. The compiled `lib/` is
78
+ **committed** (DSH STORE scans the fixed Git tree), so a fresh checkout already
79
+ contains it. After changing `src/`, rebuild it:
80
+
81
+ ```sh
82
+ npm run build
83
+ ```
84
+
85
+ `prepublishOnly` runs the same build for a published package.
86
+
87
+ ## Code changes have no effect
88
+
89
+ Two separate reasons, usually both:
90
+
91
+ 1. ESM modules are evaluated once per process, so a running DSH keeps the code it
92
+ loaded. Restart the host.
93
+ 2. The host loads `lib/`, not `src/`. Rebuild before restarting.
94
+
95
+ ```sh
96
+ npm run build
97
+ ```
98
+
99
+ ## Proxy is not detected
100
+
101
+ Resolution order is: explicit `proxy` config → environment → the OS system
102
+ proxy (macOS `scutil`, Windows Internet Settings) → direct. The decision is
103
+ cached for 60 seconds, so a change can take up to a minute (or a restart) to be
104
+ honoured.
105
+
106
+ 1. Is a proxy configured at all?
107
+
108
+ macOS/Linux:
109
+
110
+ ```sh
111
+ env | grep -i proxy
112
+ ```
113
+
114
+ Windows PowerShell:
115
+
116
+ ```powershell
117
+ Get-ChildItem Env: | Where-Object Name -match 'proxy'
118
+ ```
119
+
120
+ `DSH_WEB_SEARCH_PROXY`, `HTTPS_PROXY`, `ALL_PROXY` and `HTTP_PROXY` are read
121
+ in both casings. A value counts only if it parses as an **`http://`** proxy;
122
+ `socks5://` and `https://` values are skipped (the tunnel is a plain
123
+ `CONNECT`, so an `https://` proxy would be dialled in cleartext).
124
+ `socks5://` users must point at the client's HTTP/mixed port instead.
125
+
126
+ 2. Is the system proxy set, and is that port actually listening?
127
+
128
+ macOS:
129
+
130
+ ```sh
131
+ scutil --proxy
132
+ nc -z -G 1 127.0.0.1 7890 && echo reachable || echo refused
133
+ ```
134
+
135
+ Windows PowerShell:
136
+
137
+ ```powershell
138
+ Get-ItemProperty 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings' |
139
+ Select-Object ProxyEnable, ProxyServer
140
+ Test-NetConnection 127.0.0.1 -Port 7890
141
+ ```
142
+
143
+ The system proxy is only adopted when the port accepts a TCP connection,
144
+ within 400 ms. A stale `ProxyEnable = 1` / `HTTPSEnable : 1` with nothing
145
+ listening is ignored on purpose, so the search degrades to a direct request
146
+ instead of stalling.
147
+
148
+ 3. Is `NO_PROXY` in the way?
149
+
150
+ A loopback-only list (`localhost,127.0.0.1`, a common shell default) is
151
+ skipped: it cannot cover a public search host and must not be read as "no
152
+ proxy". A wildcard (`NO_PROXY=*`) is honoured and disables proxying.
153
+
154
+ 4. Fix it explicitly instead of relying on detection:
155
+
156
+ ```yaml
157
+ - id: web-search-bing
158
+ name: dsh-web-search-bing
159
+ config:
160
+ proxy: "http://127.0.0.1:7890"
161
+ ```
162
+
163
+ Under TUN/enhanced mode the kernel takes over TCP transparently; this module is
164
+ not involved and the behaviour matches a direct request, so a working TUN setup
165
+ needs no proxy configuration at all.
166
+
167
+ ## An engine is always skipped with "cooling down"
168
+
169
+ The failure list says both how long and why:
170
+ `cooling down for another 2m 10s after SEARCH_THROTTLED`. Durations are
171
+ `Retry-After` when the service sent one, otherwise `baseCooldownMs` doubled per
172
+ consecutive failure, capped at `maxCooldownMs` (5 min → 30 min by default).
173
+
174
+ - `SEARCH_THROTTLED`, `SEARCH_BLOCKED`, `SEARCH_TIMEOUT`, `SEARCH_NETWORK` cool
175
+ the engine down.
176
+ - `SEARCH_PARSE` and `SEARCH_ABORTED` never do: markup drift and caller
177
+ cancellation cannot be fixed by waiting.
178
+
179
+ Check what the service actually said (a 429 with `Retry-After` explains a long
180
+ rest):
181
+
182
+ ```sh
183
+ curl -sSI --max-time 10 'https://www.so.com/s?q=test' | head -20
184
+ ```
185
+
186
+ A cooldown never empties the engine set: if every engine is resting, the run
187
+ uses them anyway. To clear the state early, restart the host — the cache and the
188
+ health tracker are per mount, so a restart starts from a clean slate.
189
+
190
+ ## Results do not change after editing the config
191
+
192
+ The result cache answers identical logical requests for `cacheTtlMs` (default
193
+ 10 minutes). The key includes the lead region, the `wait` policy, the budget, the
194
+ participating engine ids and the normalised query, so most config changes do
195
+ produce a new key — but a query repeat within the TTL is served from memory. Set
196
+ `cacheTtlMs: 0` while experimenting.
197
+
198
+ ## Results have no `publishedAt`
199
+
200
+ Not a bug. No engine currently supplies a date: the scraped pages do not expose
201
+ one in a stable field, and AnySearch's rows carry `title`, `url`, `snippet` and
202
+ `content` but no date. Sources therefore omit `publishedAt`, and the freshness
203
+ boost in the ranker stays inactive until an engine starts providing it.
204
+
205
+ ## Google never shows up
206
+
207
+ Expected, and not configurable. Google is unreachable from this network: a
208
+ tunnelled `www.google.com/search` returns a JavaScript shell (94 KB, 1–3 links),
209
+ and headless Chrome is redirected to `google.com.hk` with a reCAPTCHA page.
210
+ `gbv=1`, a consent cookie and a mobile user agent were all tried. See
211
+ [engines.md](engines.md#what-could-not-be-made-to-work).
212
+
213
+ ## Typecheck fails after an edit
214
+
215
+ Node strips types instead of transforming them, so the source must remain
216
+ erasable:
217
+
218
+ - no `enum`, no `namespace`, no constructor parameter properties;
219
+ - relative imports carry the `.ts` extension (`from './types.ts'`);
220
+ - type-only imports use `import type`.
221
+
222
+ ```sh
223
+ npm run typecheck # tsc --noEmit over src/**/*.ts
224
+ npm run check # typecheck + the whole test suite
225
+ ```
package/locales/en.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "meta": {
3
- "title": "Web Search (Bing)",
4
- "description": "Edits the key-free Bing and AnySearch provider: engine allow-list, wait policy, proxy, result budget, ranking, cache and cooldown settings."
3
+ "title": "Web Search",
4
+ "description": "Key-free search, page fetch and research: eight engines fused with RRF, 34 dedicated sources, fetch ladder, proxy, cache and cooldown settings."
5
5
  }
6
6
  }
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "meta": {
3
- "title": "网页搜索(Bing)",
4
- "description": "编辑免密钥的 Bing 与 AnySearch 搜索提供方:引擎白名单、等待策略、代理、结果上限、排序、缓存与冷却设置。"
3
+ "title": "网页搜索",
4
+ "description": "免密钥的搜索、抓取与研究:8 个引擎 RRF 融合、34 个专用源、抓取梯队、代理、缓存与冷却设置。"
5
5
  }
6
6
  }
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@fffangchenghao/dsh-web-search",
3
- "version": "0.24.0",
4
- "description": "DSH bundle: key-free web search, page fetch and one web_tools action tool for verticals, platforms and cited research; eight engines fused with RRF, 34 vertical sources, arXiv research lookups and readable papers, official release facts, a replayable benchmark, an optional Firecrawl key and a dependency-free MCP server.",
3
+ "version": "0.24.2",
4
+ "description": "DSH bundle: key-free web search, page fetch and research for DeepSeek Harness; eight engines fused with RRF, 34 vertical sources, arXiv, Windows/macOS system-proxy support and a dependency-free MCP server.",
5
5
  "type": "module",
6
6
  "main": "lib/index.js",
7
7
  "types": "lib/index.d.ts",
@@ -33,7 +33,13 @@
33
33
  "docs/cross-agent.md",
34
34
  "docs/comparison.md",
35
35
  "docs/images/web-tools-demo.svg",
36
- "docs/latency-and-sources.md"
36
+ "docs/latency-and-sources.md",
37
+ "docs/plugin-settings-form.md",
38
+ "docs/troubleshooting.md",
39
+ "docs/architecture.md",
40
+ "docs/engines.md",
41
+ "docs/images/bench.png",
42
+ "docs/images/bench.svg"
37
43
  ],
38
44
  "scripts": {
39
45
  "build": "tsc -p tsconfig.json",
@@ -53,7 +59,11 @@
53
59
  "anysearch",
54
60
  "research",
55
61
  "web-tools",
56
- "mcp"
62
+ "mcp",
63
+ "windows",
64
+ "keyless",
65
+ "arxiv",
66
+ "page-fetch"
57
67
  ],
58
68
  "license": "MIT",
59
69
  "dsh": {