@fffangchenghao/dsh-web-search 0.24.0 → 0.24.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +48 -0
- package/README.md +24 -16
- package/README.zh-CN.md +22 -15
- package/client.js +5 -5
- package/docs/architecture.md +181 -0
- package/docs/comparison.md +2 -0
- package/docs/engines.md +203 -0
- package/docs/images/bench.png +0 -0
- package/docs/images/bench.svg +84 -0
- package/docs/latency-and-sources.md +2 -0
- package/docs/plugin-settings-form.md +160 -0
- package/docs/rrf-ranking.md +1 -1
- package/docs/store-evidence.md +1 -1
- package/docs/troubleshooting.md +225 -0
- package/locales/en.json +2 -2
- package/locales/zh-CN.json +2 -2
- package/package.json +14 -4
|
@@ -0,0 +1,225 @@
|
|
|
1
|
+
# Troubleshooting
|
|
2
|
+
|
|
3
|
+
Symptom first, cause second, fix third. Every command below is meant to be run
|
|
4
|
+
from the repository root.
|
|
5
|
+
|
|
6
|
+
- [Architecture](architecture.md)
|
|
7
|
+
- [Engines](engines.md)
|
|
8
|
+
- [README](../README.md) · [中文说明](../README.zh-CN.md)
|
|
9
|
+
|
|
10
|
+
## `web_search` fails with "no search engine returned results"
|
|
11
|
+
|
|
12
|
+
That message is the provider's own, and the interesting part is in the
|
|
13
|
+
parentheses: one `id: reason` entry per engine that was tried. Read it before
|
|
14
|
+
changing anything.
|
|
15
|
+
|
|
16
|
+
```
|
|
17
|
+
no search engine returned results for "…" (anysearch: HTTP 429; cn.bing: page does not match the query (possibly rate limited); …)
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
| Reason in the list | Meaning | What to do |
|
|
21
|
+
| --- | --- | --- |
|
|
22
|
+
| `no results` | the engine answered 2xx and parsed to zero sources | markup may have changed; check that engine's parser against a fresh capture |
|
|
23
|
+
| `page does not match the query (possibly rate limited)` | the echoed-query check failed: a block/trending page for someone else's query | let the cooldown pass, or search a different query |
|
|
24
|
+
| `HTTP 403` / `HTTP 429` | the engine decided this client is a robot or is throttling it | cooldown applies; slow down |
|
|
25
|
+
| `still running at deadline` | the soft deadline released the tier first | `wait: all`, a larger `timeoutMs`, or accept it |
|
|
26
|
+
| `aborted once enough results had arrived` | the tier had enough and stopped waiting | not an error |
|
|
27
|
+
| an `ECONNRESET`/`ENOTFOUND` style message | transport failure | check the tunnel, see below |
|
|
28
|
+
|
|
29
|
+
If *every* reason mentions the tunnel or a timeout, and the `global` engines are
|
|
30
|
+
missing from the list entirely, the run had no proxy: `viaProxy` engines
|
|
31
|
+
(`ddg-html`, `www.bing`, `ddg-lite`) are filtered out rather than failing slowly.
|
|
32
|
+
See [proxy not detected](#proxy-is-not-detected).
|
|
33
|
+
|
|
34
|
+
To narrow a failure down, restrict the run to one engine and watch the reason
|
|
35
|
+
change:
|
|
36
|
+
|
|
37
|
+
```yaml
|
|
38
|
+
- id: web-search-bing
|
|
39
|
+
name: dsh-web-search-bing
|
|
40
|
+
config:
|
|
41
|
+
hosts: [anysearch]
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
## The provider is not selected
|
|
45
|
+
|
|
46
|
+
`web_search` still behaves like the built-in provider → `web.searchProvider` is
|
|
47
|
+
not `bing`. The provider id is fixed (`PROVIDER_ID = 'bing'`) and `available()`
|
|
48
|
+
returns `true` unconditionally, so selection is purely configuration — there is
|
|
49
|
+
no credential or health gate to blame.
|
|
50
|
+
|
|
51
|
+
```yaml
|
|
52
|
+
- id: web
|
|
53
|
+
name: "@deepseek-ai/dsh-web"
|
|
54
|
+
config:
|
|
55
|
+
searchProvider: bing
|
|
56
|
+
fetchProvider: http
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Check that the plugin is actually mounted and registers itself. This snippet
|
|
60
|
+
loads the built entry, calls `apply` with a stub context and prints the provider
|
|
61
|
+
id — no network access:
|
|
62
|
+
|
|
63
|
+
```sh
|
|
64
|
+
NODE=~/.dsh/dsh-runtimes/dsh-primary-runtime/dependencies/node/bin/node
|
|
65
|
+
$NODE -e "
|
|
66
|
+
import('file://$HOME/.dsh/plugins/dsh-web-search-bing/lib/index.js').then((m) => {
|
|
67
|
+
let p; m.apply({ web: { registerSearchProvider: (x) => { p = x } } }, m.Config({}));
|
|
68
|
+
console.log(p.id, p.available());
|
|
69
|
+
});"
|
|
70
|
+
```
|
|
71
|
+
|
|
72
|
+
Expected output: `bing true`. If the import fails with `ERR_MODULE_NOT_FOUND`,
|
|
73
|
+
`lib/` has not been built — see [below](#lib-is-missing).
|
|
74
|
+
|
|
75
|
+
## `lib/` is missing
|
|
76
|
+
|
|
77
|
+
`package.json` points `main` and `types` at `lib/`. The compiled `lib/` is
|
|
78
|
+
**committed** (DSH STORE scans the fixed Git tree), so a fresh checkout already
|
|
79
|
+
contains it. After changing `src/`, rebuild it:
|
|
80
|
+
|
|
81
|
+
```sh
|
|
82
|
+
npm run build
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
`prepublishOnly` runs the same build for a published package.
|
|
86
|
+
|
|
87
|
+
## Code changes have no effect
|
|
88
|
+
|
|
89
|
+
Two separate reasons, usually both:
|
|
90
|
+
|
|
91
|
+
1. ESM modules are evaluated once per process, so a running DSH keeps the code it
|
|
92
|
+
loaded. Restart the host.
|
|
93
|
+
2. The host loads `lib/`, not `src/`. Rebuild before restarting.
|
|
94
|
+
|
|
95
|
+
```sh
|
|
96
|
+
npm run build
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
## Proxy is not detected
|
|
100
|
+
|
|
101
|
+
Resolution order is: explicit `proxy` config → environment → the OS system
|
|
102
|
+
proxy (macOS `scutil`, Windows Internet Settings) → direct. The decision is
|
|
103
|
+
cached for 60 seconds, so a change can take up to a minute (or a restart) to be
|
|
104
|
+
honoured.
|
|
105
|
+
|
|
106
|
+
1. Is a proxy configured at all?
|
|
107
|
+
|
|
108
|
+
macOS/Linux:
|
|
109
|
+
|
|
110
|
+
```sh
|
|
111
|
+
env | grep -i proxy
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
Windows PowerShell:
|
|
115
|
+
|
|
116
|
+
```powershell
|
|
117
|
+
Get-ChildItem Env: | Where-Object Name -match 'proxy'
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
`DSH_WEB_SEARCH_PROXY`, `HTTPS_PROXY`, `ALL_PROXY` and `HTTP_PROXY` are read
|
|
121
|
+
in both casings. A value counts only if it parses as an **`http://`** proxy;
|
|
122
|
+
`socks5://` and `https://` values are skipped (the tunnel is a plain
|
|
123
|
+
`CONNECT`, so an `https://` proxy would be dialled in cleartext).
|
|
124
|
+
`socks5://` users must point at the client's HTTP/mixed port instead.
|
|
125
|
+
|
|
126
|
+
2. Is the system proxy set, and is that port actually listening?
|
|
127
|
+
|
|
128
|
+
macOS:
|
|
129
|
+
|
|
130
|
+
```sh
|
|
131
|
+
scutil --proxy
|
|
132
|
+
nc -z -G 1 127.0.0.1 7890 && echo reachable || echo refused
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
Windows PowerShell:
|
|
136
|
+
|
|
137
|
+
```powershell
|
|
138
|
+
Get-ItemProperty 'HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings' |
|
|
139
|
+
Select-Object ProxyEnable, ProxyServer
|
|
140
|
+
Test-NetConnection 127.0.0.1 -Port 7890
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
The system proxy is only adopted when the port accepts a TCP connection,
|
|
144
|
+
within 400 ms. A stale `ProxyEnable = 1` / `HTTPSEnable : 1` with nothing
|
|
145
|
+
listening is ignored on purpose, so the search degrades to a direct request
|
|
146
|
+
instead of stalling.
|
|
147
|
+
|
|
148
|
+
3. Is `NO_PROXY` in the way?
|
|
149
|
+
|
|
150
|
+
A loopback-only list (`localhost,127.0.0.1`, a common shell default) is
|
|
151
|
+
skipped: it cannot cover a public search host and must not be read as "no
|
|
152
|
+
proxy". A wildcard (`NO_PROXY=*`) is honoured and disables proxying.
|
|
153
|
+
|
|
154
|
+
4. Fix it explicitly instead of relying on detection:
|
|
155
|
+
|
|
156
|
+
```yaml
|
|
157
|
+
- id: web-search-bing
|
|
158
|
+
name: dsh-web-search-bing
|
|
159
|
+
config:
|
|
160
|
+
proxy: "http://127.0.0.1:7890"
|
|
161
|
+
```
|
|
162
|
+
|
|
163
|
+
Under TUN/enhanced mode the kernel takes over TCP transparently; this module is
|
|
164
|
+
not involved and the behaviour matches a direct request, so a working TUN setup
|
|
165
|
+
needs no proxy configuration at all.
|
|
166
|
+
|
|
167
|
+
## An engine is always skipped with "cooling down"
|
|
168
|
+
|
|
169
|
+
The failure list says both how long and why:
|
|
170
|
+
`cooling down for another 2m 10s after SEARCH_THROTTLED`. Durations are
|
|
171
|
+
`Retry-After` when the service sent one, otherwise `baseCooldownMs` doubled per
|
|
172
|
+
consecutive failure, capped at `maxCooldownMs` (5 min → 30 min by default).
|
|
173
|
+
|
|
174
|
+
- `SEARCH_THROTTLED`, `SEARCH_BLOCKED`, `SEARCH_TIMEOUT`, `SEARCH_NETWORK` cool
|
|
175
|
+
the engine down.
|
|
176
|
+
- `SEARCH_PARSE` and `SEARCH_ABORTED` never do: markup drift and caller
|
|
177
|
+
cancellation cannot be fixed by waiting.
|
|
178
|
+
|
|
179
|
+
Check what the service actually said (a 429 with `Retry-After` explains a long
|
|
180
|
+
rest):
|
|
181
|
+
|
|
182
|
+
```sh
|
|
183
|
+
curl -sSI --max-time 10 'https://www.so.com/s?q=test' | head -20
|
|
184
|
+
```
|
|
185
|
+
|
|
186
|
+
A cooldown never empties the engine set: if every engine is resting, the run
|
|
187
|
+
uses them anyway. To clear the state early, restart the host — the cache and the
|
|
188
|
+
health tracker are per mount, so a restart starts from a clean slate.
|
|
189
|
+
|
|
190
|
+
## Results do not change after editing the config
|
|
191
|
+
|
|
192
|
+
The result cache answers identical logical requests for `cacheTtlMs` (default
|
|
193
|
+
10 minutes). The key includes the lead region, the `wait` policy, the budget, the
|
|
194
|
+
participating engine ids and the normalised query, so most config changes do
|
|
195
|
+
produce a new key — but a query repeat within the TTL is served from memory. Set
|
|
196
|
+
`cacheTtlMs: 0` while experimenting.
|
|
197
|
+
|
|
198
|
+
## Results have no `publishedAt`
|
|
199
|
+
|
|
200
|
+
Not a bug. No engine currently supplies a date: the scraped pages do not expose
|
|
201
|
+
one in a stable field, and AnySearch's rows carry `title`, `url`, `snippet` and
|
|
202
|
+
`content` but no date. Sources therefore omit `publishedAt`, and the freshness
|
|
203
|
+
boost in the ranker stays inactive until an engine starts providing it.
|
|
204
|
+
|
|
205
|
+
## Google never shows up
|
|
206
|
+
|
|
207
|
+
Expected, and not configurable. Google is unreachable from this network: a
|
|
208
|
+
tunnelled `www.google.com/search` returns a JavaScript shell (94 KB, 1–3 links),
|
|
209
|
+
and headless Chrome is redirected to `google.com.hk` with a reCAPTCHA page.
|
|
210
|
+
`gbv=1`, a consent cookie and a mobile user agent were all tried. See
|
|
211
|
+
[engines.md](engines.md#what-could-not-be-made-to-work).
|
|
212
|
+
|
|
213
|
+
## Typecheck fails after an edit
|
|
214
|
+
|
|
215
|
+
Node strips types instead of transforming them, so the source must remain
|
|
216
|
+
erasable:
|
|
217
|
+
|
|
218
|
+
- no `enum`, no `namespace`, no constructor parameter properties;
|
|
219
|
+
- relative imports carry the `.ts` extension (`from './types.ts'`);
|
|
220
|
+
- type-only imports use `import type`.
|
|
221
|
+
|
|
222
|
+
```sh
|
|
223
|
+
npm run typecheck # tsc --noEmit over src/**/*.ts
|
|
224
|
+
npm run check # typecheck + the whole test suite
|
|
225
|
+
```
|
package/locales/en.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"meta": {
|
|
3
|
-
"title": "Web Search
|
|
4
|
-
"description": "
|
|
3
|
+
"title": "Web Search",
|
|
4
|
+
"description": "Key-free search, page fetch and research: eight engines fused with RRF, 34 dedicated sources, fetch ladder, proxy, cache and cooldown settings."
|
|
5
5
|
}
|
|
6
6
|
}
|
package/locales/zh-CN.json
CHANGED
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@fffangchenghao/dsh-web-search",
|
|
3
|
-
"version": "0.24.
|
|
4
|
-
"description": "DSH bundle: key-free web search, page fetch and
|
|
3
|
+
"version": "0.24.2",
|
|
4
|
+
"description": "DSH bundle: key-free web search, page fetch and research for DeepSeek Harness; eight engines fused with RRF, 34 vertical sources, arXiv, Windows/macOS system-proxy support and a dependency-free MCP server.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "lib/index.js",
|
|
7
7
|
"types": "lib/index.d.ts",
|
|
@@ -33,7 +33,13 @@
|
|
|
33
33
|
"docs/cross-agent.md",
|
|
34
34
|
"docs/comparison.md",
|
|
35
35
|
"docs/images/web-tools-demo.svg",
|
|
36
|
-
"docs/latency-and-sources.md"
|
|
36
|
+
"docs/latency-and-sources.md",
|
|
37
|
+
"docs/plugin-settings-form.md",
|
|
38
|
+
"docs/troubleshooting.md",
|
|
39
|
+
"docs/architecture.md",
|
|
40
|
+
"docs/engines.md",
|
|
41
|
+
"docs/images/bench.png",
|
|
42
|
+
"docs/images/bench.svg"
|
|
37
43
|
],
|
|
38
44
|
"scripts": {
|
|
39
45
|
"build": "tsc -p tsconfig.json",
|
|
@@ -53,7 +59,11 @@
|
|
|
53
59
|
"anysearch",
|
|
54
60
|
"research",
|
|
55
61
|
"web-tools",
|
|
56
|
-
"mcp"
|
|
62
|
+
"mcp",
|
|
63
|
+
"windows",
|
|
64
|
+
"keyless",
|
|
65
|
+
"arxiv",
|
|
66
|
+
"page-fetch"
|
|
57
67
|
],
|
|
58
68
|
"license": "MIT",
|
|
59
69
|
"dsh": {
|