@fffangchenghao/dsh-web-search 0.20.0 → 0.21.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +56 -4
- package/README.md +67 -20
- package/README.zh-CN.md +36 -19
- package/SECURITY.md +5 -1
- package/client.js +119 -3
- package/docs/comparison.md +130 -0
- package/docs/cross-agent.md +80 -0
- package/docs/images/web-tools-demo.svg +59 -0
- package/docs/quickstart.md +85 -0
- package/docs/store-evidence.md +1 -1
- package/lib/bridge.d.ts +18 -0
- package/lib/bridge.js +145 -0
- package/lib/engines.js +9 -8
- package/lib/health.d.ts +20 -1
- package/lib/health.js +38 -1
- package/lib/index.js +6 -2
- package/lib/inject.d.ts +25 -0
- package/lib/transport.d.ts +10 -6
- package/lib/transport.js +30 -14
- package/mcp/server.mjs +324 -0
- package/package.json +15 -5
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
# 30-query keyless comparison
|
|
2
|
+
|
|
3
|
+
This is a small, reproducible comparison, not a claim that one search plugin is
|
|
4
|
+
"best". It measures what can be measured without private relevance labels:
|
|
5
|
+
whether a provider answers at all, how long it takes, how many distinct domains
|
|
6
|
+
it returns, and whether a page on a known official domain appears in the top 8.
|
|
7
|
+
The failures are part of the result.
|
|
8
|
+
|
|
9
|
+
## Method
|
|
10
|
+
|
|
11
|
+
- **Corpus:** 30 queries in `bench/compare30/queries.json` — 13 release queries
|
|
12
|
+
(8 English, 5 Chinese), 12 documentation queries, 3 package/repository
|
|
13
|
+
queries, 1 arXiv query and 1 Stack Overflow query. Every query names one
|
|
14
|
+
expected official host.
|
|
15
|
+
- **Budget:** top 8 results for each provider, one run per query, same machine
|
|
16
|
+
and same network route (a local HTTP proxy at `127.0.0.1:7890` for providers
|
|
17
|
+
that needed it).
|
|
18
|
+
- **Plugin arm:** `search()` with the current RRF ranker, no cache, 15 s
|
|
19
|
+
timeout. It fuses eight engines and then truncates to 8.
|
|
20
|
+
- **Firecrawl arm:** keyless `POST https://api.firecrawl.dev/v1/search`; each
|
|
21
|
+
request tries direct first, then the tunnel.
|
|
22
|
+
- **Exa arm:** the anonymous `web_search_exa` MCP endpoint with
|
|
23
|
+
`Accept: application/json, text/event-stream`; direct first, then tunnel.
|
|
24
|
+
- **No API keys** were used. All raw responses are in
|
|
25
|
+
[`bench/compare30/results-2026-10-26-sweep1.json`](https://github.com/fangchenghao/dsh-web-search/blob/main/bench/compare30/results-2026-10-26-sweep1.json).
|
|
26
|
+
|
|
27
|
+
Metrics: `P@8`, `R@8` and `MRR@8` are **not** reported here. They need
|
|
28
|
+
human-labelled relevance; inventing labels would make the table look stronger
|
|
29
|
+
than the evidence. The 20-query offline benchmark in
|
|
30
|
+
[rrf-ranking.md](rrf-ranking.md) publishes those metrics separately.
|
|
31
|
+
|
|
32
|
+
## Clean sweep (first run, all providers answered)
|
|
33
|
+
|
|
34
|
+
| Provider | Answered | Official-domain hit | Mean unique domains | Mean ms | p50 ms | p95 ms |
|
|
35
|
+
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
|
36
|
+
| `dsh-web-search` | 30/30 | 28/30 | 5.83 | 3008 | 2791 | 5188 |
|
|
37
|
+
| Firecrawl keyless | 30/30 | 29/30 | 5.43 | 2396 | 2219 | 3958 |
|
|
38
|
+
| Exa anonymous | 30/30 | 29/30 | 3.50 | 1001 | 964 | 1238 |
|
|
39
|
+
|
|
40
|
+
Raw official-domain counts use the corpus as first written. The single shared
|
|
41
|
+
Firecrawl/Exa "miss" and one of the plugin's two misses are the same fixture
|
|
42
|
+
error: Claude Code documentation moved from `docs.anthropic.com` to
|
|
43
|
+
`code.claude.com`; all three providers returned the new host. With that expected
|
|
44
|
+
host corrected, the counts would be **29/30, 30/30 and 30/30**. The remaining
|
|
45
|
+
real plugin miss is the Chinese Rust 1.90 query, which returned the Rust home
|
|
46
|
+
page rather than the release blog.
|
|
47
|
+
|
|
48
|
+
By category (first sweep; answered / official / mean ms):
|
|
49
|
+
|
|
50
|
+
| Category | n | dsh-web-search | Firecrawl | Exa |
|
|
51
|
+
| --- | ---: | --- | --- | --- |
|
|
52
|
+
| Release | 13 | 13 / 12 / 3123 | 13 / 13 / 2557 | 13 / 13 / 1031 |
|
|
53
|
+
| Docs | 12 | 12 / 11 / 2610 | 12 / 11 / 2383 | 12 / 11 / 1006 |
|
|
54
|
+
| Package | 3 | 3 / 3 / 4474 | 3 / 3 / 2105 | 3 / 3 / 916 |
|
|
55
|
+
| Research | 1 | 1 / 1 / 1977 | 1 / 1 / 1712 | 1 / 1 / 863 |
|
|
56
|
+
| Q&A | 1 | 1 / 1 / 2937 | 1 / 1 / 2002 | 1 / 1 / 941 |
|
|
57
|
+
|
|
58
|
+
## What the numbers say
|
|
59
|
+
|
|
60
|
+
- **Exa is the fastest by a wide margin.** About 1.0 s mean versus 3.0 s for
|
|
61
|
+
this plugin, because Exa is one hosted index while this plugin fans out to
|
|
62
|
+
eight engines and waits for fusion. If latency is the only criterion, use
|
|
63
|
+
Exa; that is the honest result.
|
|
64
|
+
- **This plugin is competitive on availability and domain diversity, not on
|
|
65
|
+
speed.** It answered all 30 and returned the most distinct domains on
|
|
66
|
+
average (5.83), but its p95 was 5.2 s. The p95 comes from the 6 s `auto`
|
|
67
|
+
soft deadline when AnySearch is slow.
|
|
68
|
+
- **Firecrawl's clean-sweep quality signal is good, but its keyless quota is
|
|
69
|
+
fragile.** It answered 30/30 with 5.43 distinct domains in the first sweep,
|
|
70
|
+
then 16/30 and 0/30 in two immediate repeats, with HTTP 429 on the tunnel
|
|
71
|
+
after a direct 403. That is a distribution/reliability limitation, not a
|
|
72
|
+
ranking difference. The repeat files are
|
|
73
|
+
[`sweep2`](https://github.com/fangchenghao/dsh-web-search/blob/main/bench/compare30/results-2026-10-26-sweep2.json)
|
|
74
|
+
and
|
|
75
|
+
[`sweep3`](https://github.com/fangchenghao/dsh-web-search/blob/main/bench/compare30/results-2026-10-26-sweep3.json).
|
|
76
|
+
- **No provider "wins" relevance here.** One official-domain miss does not mean
|
|
77
|
+
the answer is wrong, and an official domain is not always the best result.
|
|
78
|
+
The corpus is small and imbalanced: 13 release queries, 12 docs, one research
|
|
79
|
+
query and one Q&A query.
|
|
80
|
+
|
|
81
|
+
## Recorded failures
|
|
82
|
+
|
|
83
|
+
### This plugin
|
|
84
|
+
|
|
85
|
+
- `anysearch: still running at deadline` on the five Chinese release queries.
|
|
86
|
+
The search still returned 8 rows from the other engines; the failure is
|
|
87
|
+
recorded, not hidden. It is the main source of the 5–6 s tail.
|
|
88
|
+
- The Chinese Rust 1.90 query returned `rust-lang.org/zh-CN/` and other general
|
|
89
|
+
pages instead of `blog.rust-lang.org`/`releases.rs`; both competitors found
|
|
90
|
+
the release page.
|
|
91
|
+
- Firecrawl's direct 403 is not charged to the user's search latency when
|
|
92
|
+
Firecrawl is not used (it is a fallback tier), but when it is used the first
|
|
93
|
+
request pays for the 403 before the tunnel retry.
|
|
94
|
+
|
|
95
|
+
### Firecrawl keyless
|
|
96
|
+
|
|
97
|
+
- Direct request: HTTP 403 for every query from this network.
|
|
98
|
+
- Tunnel request: HTTP 200 in the first sweep; HTTP 429 for 14/30 queries in
|
|
99
|
+
sweep 2 and 30/30 in sweep 3, after repeated sweeps from the same IP. This is
|
|
100
|
+
exactly the kind of source the plugin's route memory and health panel are for,
|
|
101
|
+
but a keyless public endpoint can still exhaust its quota.
|
|
102
|
+
|
|
103
|
+
### Exa anonymous
|
|
104
|
+
|
|
105
|
+
- No failures in this run. Direct requests returned HTTP 200 once the `Accept`
|
|
106
|
+
header included both `application/json` and `text/event-stream`; without it,
|
|
107
|
+
the endpoint answers `406 Not Acceptable`.
|
|
108
|
+
- Exa's result snippets are shorter, and its domain diversity is lower. That is
|
|
109
|
+
a tradeoff, not a failure.
|
|
110
|
+
|
|
111
|
+
## Reproduce
|
|
112
|
+
|
|
113
|
+
```sh
|
|
114
|
+
DSH_WEB_SEARCH_PROXY=http://127.0.0.1:7890 \
|
|
115
|
+
COMPARE_SLEEP_MS=1500 \
|
|
116
|
+
node bench/compare30.mjs --out /tmp/compare30.json
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
The script writes a JSON file next to a Markdown report. The three raw files in
|
|
120
|
+
this repository were produced by the same command; sweeps 2 and 3 were run
|
|
121
|
+
immediately after sweep 1 to test repeatability, which is why Firecrawl's quota
|
|
122
|
+
failed. A single sweep is a snapshot, not a permanent ranking.
|
|
123
|
+
|
|
124
|
+
## Bottom line
|
|
125
|
+
|
|
126
|
+
Use Exa when latency dominates. Use Firecrawl only with a quota/health strategy;
|
|
127
|
+
its keyless endpoint is not a dependable primary. Use this plugin when you want
|
|
128
|
+
no key, multiple independent engines, vertical sources and one tool surface, and
|
|
129
|
+
can accept roughly 2–6 s per fused search. It is not the fastest, and this page
|
|
130
|
+
does not claim it is the most relevant.
|
|
@@ -0,0 +1,80 @@
|
|
|
1
|
+
# Cross-agent use: MCP server and portable skill
|
|
2
|
+
|
|
3
|
+
The DSH plugin is the primary product. The same code also ships a small MCP
|
|
4
|
+
server so Claude Code, Codex and other MCP clients can use the keyless search
|
|
5
|
+
without a DSH runtime.
|
|
6
|
+
|
|
7
|
+
## MCP server
|
|
8
|
+
|
|
9
|
+
The server is a single dependency-free Node process. It speaks MCP over stdio
|
|
10
|
+
and imports the same compiled `lib/` modules the DSH plugin uses.
|
|
11
|
+
|
|
12
|
+
```sh
|
|
13
|
+
npx -y @fffangchenghao/dsh-web-search dsh-web-search-mcp
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Claude Code:
|
|
17
|
+
|
|
18
|
+
```sh
|
|
19
|
+
claude mcp add dsh-web-search -- npx -y @fffangchenghao/dsh-web-search dsh-web-search-mcp
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
Codex (`~/.codex/config.toml`):
|
|
23
|
+
|
|
24
|
+
```toml
|
|
25
|
+
[mcp_servers.dsh-web-search]
|
|
26
|
+
command = "npx"
|
|
27
|
+
args = ["-y", "@fffangchenghao/dsh-web-search", "dsh-web-search-mcp"]
|
|
28
|
+
startup_timeout_sec = 30
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
Generic configuration:
|
|
32
|
+
|
|
33
|
+
```json
|
|
34
|
+
{
|
|
35
|
+
"mcpServers": {
|
|
36
|
+
"dsh-web-search": {
|
|
37
|
+
"command": "npx",
|
|
38
|
+
"args": ["-y", "@fffangchenghao/dsh-web-search", "dsh-web-search-mcp"]
|
|
39
|
+
}
|
|
40
|
+
}
|
|
41
|
+
}
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
### Tools
|
|
45
|
+
|
|
46
|
+
| MCP tool | What it returns |
|
|
47
|
+
| --- | --- |
|
|
48
|
+
| `search` | Fused web results with titles, URLs and snippets |
|
|
49
|
+
| `fetch` | One page as Markdown or text through the direct/tunnel ladder |
|
|
50
|
+
| `research` | Up to three searches plus a cited evidence pack |
|
|
51
|
+
| `domain_search` | Packages, papers, repositories, Q&A and dataset rows |
|
|
52
|
+
|
|
53
|
+
All tool output is wrapped in the plugin's untrusted-web-content boundary. Treat
|
|
54
|
+
it as data: a page can contain instructions, but they are not commands.
|
|
55
|
+
|
|
56
|
+
### Environment
|
|
57
|
+
|
|
58
|
+
| Variable | Meaning |
|
|
59
|
+
| --- | --- |
|
|
60
|
+
| `DSH_WEB_SEARCH_PROXY` / `HTTPS_PROXY` / `HTTP_PROXY` | Optional tunnel used when a direct request fails |
|
|
61
|
+
| `ANYSEARCH_API_KEY` | Optional AnySearch key; the keyless path works without it |
|
|
62
|
+
| `DSH_WEB_SEARCH_MCP_LIMIT` | Default result limit (1–20, default 8) |
|
|
63
|
+
|
|
64
|
+
The MCP server never reads DSH profile files and never installs anything. It
|
|
65
|
+
uses ordinary network access, subject to the host's network policy.
|
|
66
|
+
|
|
67
|
+
## Portable skill
|
|
68
|
+
|
|
69
|
+
`skills/dsh-web-search/SKILL.md` is a plain Markdown skill. Copy the folder into
|
|
70
|
+
a harness that loads `SKILL.md` files if you want the same tool-selection
|
|
71
|
+
guidance:
|
|
72
|
+
|
|
73
|
+
```sh
|
|
74
|
+
mkdir -p ~/.claude/skills
|
|
75
|
+
cp -R skills/dsh-web-search ~/.claude/skills/
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
The skill tells the model when to search, when to open a page, when to use
|
|
79
|
+
`research`, and how to cite. The MCP server is what supplies the actual tools
|
|
80
|
+
outside DSH; the skill is optional guidance, not a runtime dependency.
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" width="1200" height="760" viewBox="0 0 1200 760" role="img" aria-label="web_tools demo: one tool, eight actions">
|
|
2
|
+
<defs>
|
|
3
|
+
<linearGradient id="bg" x1="0" y1="0" x2="1" y2="1">
|
|
4
|
+
<stop offset="0" stop-color="#0b1220"/><stop offset="1" stop-color="#111b2e"/>
|
|
5
|
+
</linearGradient>
|
|
6
|
+
<linearGradient id="accent" x1="0" y1="0" x2="1" y2="0">
|
|
7
|
+
<stop offset="0" stop-color="#8b5cf6"/><stop offset="1" stop-color="#22d3ee"/>
|
|
8
|
+
</linearGradient>
|
|
9
|
+
<filter id="shadow" x="-20%" y="-20%" width="140%" height="140%">
|
|
10
|
+
<feDropShadow dx="0" dy="18" stdDeviation="24" flood-color="#000" flood-opacity=".45"/>
|
|
11
|
+
</filter>
|
|
12
|
+
<style>
|
|
13
|
+
.mono{font-family:ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}
|
|
14
|
+
.sans{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,sans-serif}
|
|
15
|
+
.chip{fill:#18243a;stroke:#2b3d59;stroke-width:1}
|
|
16
|
+
.chipOn{fill:#122b3a;stroke:#22d3ee;stroke-width:1.5}
|
|
17
|
+
.t{fill:#e5edf7}.m{fill:#9fb0c7}.g{fill:#7ee2b8}.p{fill:#c4b5fd}
|
|
18
|
+
</style>
|
|
19
|
+
</defs>
|
|
20
|
+
<rect width="1200" height="760" fill="url(#bg)"/>
|
|
21
|
+
<circle cx="1050" cy="90" r="180" fill="#8b5cf6" opacity=".08"/>
|
|
22
|
+
<circle cx="120" cy="700" r="220" fill="#22d3ee" opacity=".06"/>
|
|
23
|
+
<text x="70" y="78" class="sans t" font-size="34" font-weight="700">One tool. Eight actions. Zero API keys.</text>
|
|
24
|
+
<text x="70" y="112" class="sans m" font-size="17">web_tools keeps the model-facing surface small while every capability stays reachable.</text>
|
|
25
|
+
<g filter="url(#shadow)">
|
|
26
|
+
<rect x="70" y="150" width="1060" height="430" rx="18" fill="#0d1728" stroke="#26364f" stroke-width="1.5"/>
|
|
27
|
+
<rect x="70" y="150" width="1060" height="46" rx="18" fill="#132137"/>
|
|
28
|
+
<rect x="70" y="178" width="1060" height="18" fill="#132137"/>
|
|
29
|
+
<circle cx="98" cy="173" r="6" fill="#ff5f57"/><circle cx="120" cy="173" r="6" fill="#febc2e"/><circle cx="142" cy="173" r="6" fill="#28c840"/>
|
|
30
|
+
<text x="170" y="179" class="mono m" font-size="13">deepseek-harness — web_tools</text>
|
|
31
|
+
<text x="100" y="238" class="mono g" font-size="16">$</text>
|
|
32
|
+
<text x="122" y="238" class="mono t" font-size="16">dsh plugin --profile web add @fffangchenghao/dsh-web-search@latest</text>
|
|
33
|
+
<text x="100" y="274" class="mono g" font-size="16">$</text>
|
|
34
|
+
<text x="122" y="274" class="mono t" font-size="16">web_tools action=research query="MCP stdio transport spec"</text>
|
|
35
|
+
<text x="100" y="316" class="mono p" font-size="14">→ search: 4 engines answered in 1.8s</text>
|
|
36
|
+
<text x="100" y="342" class="mono p" font-size="14">→ fetch: 3 pages, 2 cited sections</text>
|
|
37
|
+
<text x="100" y="368" class="mono p" font-size="14">→ evidence pack: 3 sources, failures listed by URL</text>
|
|
38
|
+
<text x="100" y="420" class="mono m" font-size="13">actions</text>
|
|
39
|
+
<g class="sans" font-size="14">
|
|
40
|
+
<rect class="chip" x="100" y="438" width="140" height="40" rx="9"/><text x="118" y="463" class="t">search_many</text>
|
|
41
|
+
<rect class="chip" x="252" y="438" width="168" height="40" rx="9"/><text x="270" y="463" class="t">advanced_search</text>
|
|
42
|
+
<rect class="chip" x="432" y="438" width="150" height="40" rx="9"/><text x="450" y="463" class="t">domain_search</text>
|
|
43
|
+
<rect class="chip" x="594" y="438" width="160" height="40" rx="9"/><text x="612" y="463" class="t">platform_search</text>
|
|
44
|
+
<rect class="chip" x="766" y="438" width="110" height="40" rx="9"/><text x="784" y="463" class="t">read_page</text>
|
|
45
|
+
<rect class="chipOn" x="100" y="490" width="120" height="40" rx="9"/><text x="118" y="515" class="t">research</text>
|
|
46
|
+
<rect class="chip" x="232" y="490" width="150" height="40" rx="9"/><text x="250" y="515" class="t">cache_search</text>
|
|
47
|
+
<rect class="chip" x="394" y="490" width="140" height="40" rx="9"/><text x="412" y="515" class="t">engine_test</text>
|
|
48
|
+
<text x="560" y="515" class="m" font-size="13">one schema · one registration · selected action runs</text>
|
|
49
|
+
</g>
|
|
50
|
+
</g>
|
|
51
|
+
<g class="sans">
|
|
52
|
+
<rect x="70" y="620" width="330" height="76" rx="14" fill="#101d31" stroke="#26364f"/>
|
|
53
|
+
<text x="92" y="651" class="m" font-size="13">Engines</text><text x="92" y="677" class="t" font-size="20" font-weight="700">8 key-free paths</text>
|
|
54
|
+
<rect x="420" y="620" width="330" height="76" rx="14" fill="#101d31" stroke="#26364f"/>
|
|
55
|
+
<text x="442" y="651" class="m" font-size="13">Tests</text><text x="442" y="677" class="t" font-size="20" font-weight="700">668 local cases</text>
|
|
56
|
+
<rect x="770" y="620" width="360" height="76" rx="14" fill="#101d31" stroke="#26364f"/>
|
|
57
|
+
<text x="792" y="651" class="m" font-size="13">Cross-agent</text><text x="792" y="677" class="t" font-size="20" font-weight="700">MCP stdio server</text>
|
|
58
|
+
</g>
|
|
59
|
+
</svg>
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Quickstart: keyless web search for DSH and other agents
|
|
2
|
+
|
|
3
|
+
This page gets you from zero to a working `web_tools` call in about two
|
|
4
|
+
minutes. It is the English quickstart; the full reference is in
|
|
5
|
+
[README.md](../README.md).
|
|
6
|
+
|
|
7
|
+
## 1. Install in DeepSeek Harness
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
dsh plugin --profile defaults add @fffangchenghao/dsh-web-search@latest
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
Use your profile name instead of `defaults` if you run a named profile. Restart
|
|
14
|
+
DeepSeek Harness once after installation. No API key is required.
|
|
15
|
+
|
|
16
|
+
Verify in **Settings → Plugins → Web Search**:
|
|
17
|
+
|
|
18
|
+
- the engine health section can run a one-probe-per-engine test;
|
|
19
|
+
- the latest npm version line appears;
|
|
20
|
+
- if a newer npm release exists, a local **Install update** button appears.
|
|
21
|
+
|
|
22
|
+
## 2. Use it
|
|
23
|
+
|
|
24
|
+
The model gets one optional tool with eight actions:
|
|
25
|
+
|
|
26
|
+
```json
|
|
27
|
+
{
|
|
28
|
+
"name": "web_tools",
|
|
29
|
+
"arguments": {
|
|
30
|
+
"action": "search_many",
|
|
31
|
+
"queries": ["Node.js 24 release notes", "Node.js 24 breaking changes"]
|
|
32
|
+
}
|
|
33
|
+
}
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Common calls:
|
|
37
|
+
|
|
38
|
+
| Goal | Action |
|
|
39
|
+
| --- | --- |
|
|
40
|
+
| One ordinary search | `advanced_search` or `search_many` |
|
|
41
|
+
| Package / repo / paper / Q&A | `domain_search` |
|
|
42
|
+
| Bilibili / V2EX / Reddit / YouTube / Zhihu / Juejin | `platform_search` |
|
|
43
|
+
| Read a long page in chunks | `read_page` |
|
|
44
|
+
| Turn searches into a cited pack | `research` |
|
|
45
|
+
| Search full texts already fetched | `cache_search` |
|
|
46
|
+
| Check which engines answer | `engine_test` |
|
|
47
|
+
|
|
48
|
+
With the plugin's own skill installed, the model sees when to use each action.
|
|
49
|
+
The tool descriptions also carry the boundaries, so the skill is optional.
|
|
50
|
+
|
|
51
|
+
## 3. Use from Claude Code, Codex or another MCP client
|
|
52
|
+
|
|
53
|
+
The package ships a dependency-free stdio MCP server:
|
|
54
|
+
|
|
55
|
+
```sh
|
|
56
|
+
npx -y @fffangchenghao/dsh-web-search dsh-web-search-mcp
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Claude Code:
|
|
60
|
+
|
|
61
|
+
```sh
|
|
62
|
+
claude mcp add dsh-web-search -- npx -y @fffangchenghao/dsh-web-search dsh-web-search-mcp
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Codex (`~/.codex/config.toml`):
|
|
66
|
+
|
|
67
|
+
```toml
|
|
68
|
+
[mcp_servers.dsh-web-search]
|
|
69
|
+
command = "npx"
|
|
70
|
+
args = ["-y", "@fffangchenghao/dsh-web-search", "dsh-web-search-mcp"]
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
The server exposes `search`, `fetch`, `research` and `domain_search`. It needs
|
|
74
|
+
no DSH context and no API key. See [cross-agent.md](cross-agent.md) for details.
|
|
75
|
+
|
|
76
|
+
## 4. What to read next
|
|
77
|
+
|
|
78
|
+
- [Cross-agent MCP setup](cross-agent.md)
|
|
79
|
+
- [How ranking works](rrf-ranking.md)
|
|
80
|
+
- [30-query comparison method and results](comparison.md) — availability,
|
|
81
|
+
latency and official-source presence against keyless Firecrawl and anonymous
|
|
82
|
+
Exa, with failures listed rather than hidden.
|
|
83
|
+
- [Troubleshooting](troubleshooting.md)
|
|
84
|
+
|
|
85
|
+

|
package/docs/store-evidence.md
CHANGED
|
@@ -23,7 +23,7 @@ Only `0.2.1-rc.2` is declared, because that is the version actually exercised.
|
|
|
23
23
|
## Environment
|
|
24
24
|
|
|
25
25
|
- macOS (arm64), DeepSeek Harness **0.2.1-rc.2** (`dsh --version`), Node 22.18+.
|
|
26
|
-
- Runtime artifacts: `lib/` is committed to this repository (
|
|
26
|
+
- Runtime artifacts: `lib/` is committed to this repository (78 files, 912 KiB),
|
|
27
27
|
so the fixed commit's tree contains `lib/index.js` and `lib/index.d.ts`.
|
|
28
28
|
- Disposable profile for the destructive steps: a fresh `DSH_HOME` created with
|
|
29
29
|
`mktemp -d`; the real `~/.dsh` was never written to. Bundled pnpm 11.28.5.
|
package/lib/bridge.d.ts
CHANGED
|
@@ -2,11 +2,29 @@ import type { EngineTestRunner } from './engine-test.ts';
|
|
|
2
2
|
import type { InjectableHost, ServiceContext } from './inject.ts';
|
|
3
3
|
/** Route prefix the client fetches; kept stable for the shipped settings page. */
|
|
4
4
|
export declare const ENGINE_TEST_BRIDGE_PATH = "/api/dsh-web-search/engine-test";
|
|
5
|
+
/** Settings-page update button; the only package it may install is this one. */
|
|
6
|
+
export declare const UPDATE_BRIDGE_PATH = "/api/dsh-web-search/update";
|
|
7
|
+
/** Exact package this bridge is allowed to install. */
|
|
8
|
+
export declare const OWN_PACKAGE_NAME = "@fffangchenghao/dsh-web-search";
|
|
5
9
|
/** The injected services this bridge needs; `webServer` may be absent. */
|
|
6
10
|
export type BridgeContext = ServiceContext;
|
|
11
|
+
/** A context that can inject the web server and may look up the plugin manager. */
|
|
12
|
+
export type UpdateContext = InjectableHost & Pick<InjectableHost, 'get'>;
|
|
7
13
|
/** A context that may support `inject`; test hosts often do not. */
|
|
8
14
|
export type InjectableContext = InjectableHost;
|
|
9
15
|
export type { WebServerLike } from './inject.ts';
|
|
16
|
+
/**
|
|
17
|
+
* The Settings page's update button.
|
|
18
|
+
*
|
|
19
|
+
* It deliberately does not run a package manager itself: the host's own
|
|
20
|
+
* `pluginManager` owns the profile lock and the pnpm invocation. The bridge
|
|
21
|
+
* only accepts this package, refuses a local link/file install, and reports
|
|
22
|
+
* that a restart is required. The model tool path is untouched.
|
|
23
|
+
*
|
|
24
|
+
* @param ctx - injectable plugin context.
|
|
25
|
+
* @returns a disposer, or `undefined` when the host has no injectable server.
|
|
26
|
+
*/
|
|
27
|
+
export declare function registerUpdateBridge(ctx: UpdateContext): (() => void) | undefined;
|
|
10
28
|
/**
|
|
11
29
|
* Register the engine-test endpoint.
|
|
12
30
|
* @param ctx - injectable plugin context.
|
package/lib/bridge.js
CHANGED
|
@@ -2,6 +2,10 @@ import { runEngineTestValue } from "./engine-test.js";
|
|
|
2
2
|
import { HOSTS } from "./engines.js";
|
|
3
3
|
/** Route prefix the client fetches; kept stable for the shipped settings page. */
|
|
4
4
|
export const ENGINE_TEST_BRIDGE_PATH = '/api/dsh-web-search/engine-test';
|
|
5
|
+
/** Settings-page update button; the only package it may install is this one. */
|
|
6
|
+
export const UPDATE_BRIDGE_PATH = '/api/dsh-web-search/update';
|
|
7
|
+
/** Exact package this bridge is allowed to install. */
|
|
8
|
+
export const OWN_PACKAGE_NAME = '@fffangchenghao/dsh-web-search';
|
|
5
9
|
/** Largest request body the bridge will read. */
|
|
6
10
|
const MAX_BODY_BYTES = 16 * 1024;
|
|
7
11
|
/** Whether an address is loopback. */
|
|
@@ -95,6 +99,147 @@ function parseBody(body) {
|
|
|
95
99
|
...(query === undefined ? {} : { query: query.trim() }),
|
|
96
100
|
};
|
|
97
101
|
}
|
|
102
|
+
/** A local checkout must never be replaced by the registry package. */
|
|
103
|
+
function isLocalSource(source) {
|
|
104
|
+
return typeof source === 'string' && /(?:^|@)(?:link|file|workspace):/iu.test(source);
|
|
105
|
+
}
|
|
106
|
+
/** Read the plugin manager from the injected context or the service registry. */
|
|
107
|
+
function managerOf(ctx, services) {
|
|
108
|
+
const injected = services.pluginManager;
|
|
109
|
+
if (injected !== undefined)
|
|
110
|
+
return injected;
|
|
111
|
+
if (typeof ctx.get !== 'function')
|
|
112
|
+
return undefined;
|
|
113
|
+
try {
|
|
114
|
+
const candidate = ctx.get('pluginManager');
|
|
115
|
+
if (typeof candidate !== 'object' || candidate === null)
|
|
116
|
+
return undefined;
|
|
117
|
+
const manager = candidate;
|
|
118
|
+
return typeof manager.listBundles === 'function' && typeof manager.installBundle === 'function'
|
|
119
|
+
? manager
|
|
120
|
+
: undefined;
|
|
121
|
+
}
|
|
122
|
+
catch {
|
|
123
|
+
return undefined;
|
|
124
|
+
}
|
|
125
|
+
}
|
|
126
|
+
/** Human-readable failure from a manager result. */
|
|
127
|
+
function failureText(error) {
|
|
128
|
+
if (error === undefined)
|
|
129
|
+
return 'the plugin manager rejected the update';
|
|
130
|
+
return error.diagnostic ?? error.code ?? 'the plugin manager rejected the update';
|
|
131
|
+
}
|
|
132
|
+
/**
|
|
133
|
+
* The Settings page's update button.
|
|
134
|
+
*
|
|
135
|
+
* It deliberately does not run a package manager itself: the host's own
|
|
136
|
+
* `pluginManager` owns the profile lock and the pnpm invocation. The bridge
|
|
137
|
+
* only accepts this package, refuses a local link/file install, and reports
|
|
138
|
+
* that a restart is required. The model tool path is untouched.
|
|
139
|
+
*
|
|
140
|
+
* @param ctx - injectable plugin context.
|
|
141
|
+
* @returns a disposer, or `undefined` when the host has no injectable server.
|
|
142
|
+
*/
|
|
143
|
+
export function registerUpdateBridge(ctx) {
|
|
144
|
+
if (typeof ctx.inject !== 'function')
|
|
145
|
+
return undefined;
|
|
146
|
+
try {
|
|
147
|
+
ctx.inject(['webServer'], (services) => {
|
|
148
|
+
const webServer = services?.webServer;
|
|
149
|
+
if (webServer === undefined || typeof webServer.register !== 'function')
|
|
150
|
+
return;
|
|
151
|
+
const mount = () => {
|
|
152
|
+
const off = webServer.register({
|
|
153
|
+
kind: 'exact',
|
|
154
|
+
path: UPDATE_BRIDGE_PATH,
|
|
155
|
+
async handler(request, response) {
|
|
156
|
+
if (request.method !== 'POST') {
|
|
157
|
+
writeJson(response, 405, { ok: false, code: 'method-not-allowed', message: 'POST only' });
|
|
158
|
+
return;
|
|
159
|
+
}
|
|
160
|
+
if (!isAllowed(request)) {
|
|
161
|
+
writeJson(response, 403, { ok: false, code: 'forbidden', message: 'loopback settings only' });
|
|
162
|
+
return;
|
|
163
|
+
}
|
|
164
|
+
const body = await readJson(request);
|
|
165
|
+
if (body === undefined || typeof body !== 'object' || body === null || Array.isArray(body)) {
|
|
166
|
+
writeJson(response, 400, { ok: false, code: 'invalid-json', message: 'body must be a JSON object' });
|
|
167
|
+
return;
|
|
168
|
+
}
|
|
169
|
+
const manager = managerOf(ctx, services);
|
|
170
|
+
if (manager === undefined) {
|
|
171
|
+
writeJson(response, 503, {
|
|
172
|
+
ok: false,
|
|
173
|
+
code: 'manager-unavailable',
|
|
174
|
+
message: 'this deployment has no plugin manager; use the profile package manager instead',
|
|
175
|
+
});
|
|
176
|
+
return;
|
|
177
|
+
}
|
|
178
|
+
try {
|
|
179
|
+
const bundles = await manager.listBundles();
|
|
180
|
+
const own = bundles.find((bundle) => bundle.name === OWN_PACKAGE_NAME && bundle.installed !== false);
|
|
181
|
+
if (own === undefined) {
|
|
182
|
+
writeJson(response, 409, {
|
|
183
|
+
ok: false,
|
|
184
|
+
code: 'not-installed',
|
|
185
|
+
message: 'this package is not an installed profile bundle',
|
|
186
|
+
});
|
|
187
|
+
return;
|
|
188
|
+
}
|
|
189
|
+
if (isLocalSource(own.source)) {
|
|
190
|
+
writeJson(response, 409, {
|
|
191
|
+
ok: false,
|
|
192
|
+
code: 'local-install',
|
|
193
|
+
message: 'link:/file: installs are not replaced by the button; install from npm first',
|
|
194
|
+
});
|
|
195
|
+
return;
|
|
196
|
+
}
|
|
197
|
+
const result = await manager.installBundle(`${OWN_PACKAGE_NAME}@latest`, { saveExact: true });
|
|
198
|
+
if (result.application === 'failed' || result.application === 'cancelled') {
|
|
199
|
+
writeJson(response, 502, {
|
|
200
|
+
ok: false,
|
|
201
|
+
code: result.application,
|
|
202
|
+
message: failureText(result.error),
|
|
203
|
+
});
|
|
204
|
+
return;
|
|
205
|
+
}
|
|
206
|
+
writeJson(response, 200, {
|
|
207
|
+
ok: true,
|
|
208
|
+
changed: result.changed,
|
|
209
|
+
application: result.application,
|
|
210
|
+
version: result.version ?? null,
|
|
211
|
+
restartRequired: result.application === 'restart-required',
|
|
212
|
+
message: result.changed
|
|
213
|
+
? result.application === 'restart-required'
|
|
214
|
+
? 'update installed; restart DeepSeek Harness to load it'
|
|
215
|
+
: 'update installed'
|
|
216
|
+
: 'already at the latest version',
|
|
217
|
+
});
|
|
218
|
+
}
|
|
219
|
+
catch (error) {
|
|
220
|
+
writeJson(response, 502, {
|
|
221
|
+
ok: false,
|
|
222
|
+
code: 'install-failed',
|
|
223
|
+
message: error instanceof Error ? error.message : String(error),
|
|
224
|
+
});
|
|
225
|
+
}
|
|
226
|
+
},
|
|
227
|
+
});
|
|
228
|
+
return typeof off === 'function' ? off : undefined;
|
|
229
|
+
};
|
|
230
|
+
if (typeof services.effect === 'function')
|
|
231
|
+
services.effect(mount);
|
|
232
|
+
else
|
|
233
|
+
mount();
|
|
234
|
+
});
|
|
235
|
+
return () => {
|
|
236
|
+
// The injected disposer owns the route; nothing else to release here.
|
|
237
|
+
};
|
|
238
|
+
}
|
|
239
|
+
catch {
|
|
240
|
+
return undefined;
|
|
241
|
+
}
|
|
242
|
+
}
|
|
98
243
|
/**
|
|
99
244
|
* Register the engine-test endpoint.
|
|
100
245
|
* @param ctx - injectable plugin context.
|
package/lib/engines.js
CHANGED
|
@@ -79,10 +79,11 @@ export const HOSTS = [
|
|
|
79
79
|
tier: 'fallback',
|
|
80
80
|
region: 'both',
|
|
81
81
|
weight: 1,
|
|
82
|
-
// Direct first
|
|
83
|
-
//
|
|
84
|
-
//
|
|
85
|
-
//
|
|
82
|
+
// Direct first, then the tunnel. This endpoint is IP-sensitive: measured
|
|
83
|
+
// here the direct route answered in one environment and returned 403 in
|
|
84
|
+
// another while the same query succeeded through the tunnel. The health
|
|
85
|
+
// tracker remembers which route worked, so after the first proxy success a
|
|
86
|
+
// run tries the tunnel first instead of paying for the same 403 again.
|
|
86
87
|
method: 'POST',
|
|
87
88
|
skipEcho: true,
|
|
88
89
|
build: () => 'https://api.firecrawl.dev/v1/search',
|
|
@@ -266,9 +267,9 @@ function allowsFallback(details, fallbackOn) {
|
|
|
266
267
|
* @param domain - AnySearch vertical domain, if configured.
|
|
267
268
|
* @returns the last attempt's outcome.
|
|
268
269
|
*/
|
|
269
|
-
async function searchHostWithKeys(host, query, signal, timeoutMs, proxy, timeRange, pool, domain, safeSearch) {
|
|
270
|
+
async function searchHostWithKeys(host, query, signal, timeoutMs, proxy, timeRange, pool, domain, safeSearch, health) {
|
|
270
271
|
if (host.id !== 'anysearch' || pool.size === 0) {
|
|
271
|
-
return searchHost(host, query, signal, timeoutMs, proxy, timeRange, safeSearch);
|
|
272
|
+
return searchHost(host, query, signal, timeoutMs, proxy, timeRange, safeSearch, health);
|
|
272
273
|
}
|
|
273
274
|
let key = pool.pick();
|
|
274
275
|
let last;
|
|
@@ -277,7 +278,7 @@ async function searchHostWithKeys(host, query, signal, timeoutMs, proxy, timeRan
|
|
|
277
278
|
anysearchApiKey: key,
|
|
278
279
|
...(domain === undefined ? {} : { anysearchDomain: domain }),
|
|
279
280
|
});
|
|
280
|
-
const outcome = await searchHost(configured, query, signal, timeoutMs, proxy, timeRange, safeSearch);
|
|
281
|
+
const outcome = await searchHost(configured, query, signal, timeoutMs, proxy, timeRange, safeSearch, health);
|
|
281
282
|
if (outcome.ok) {
|
|
282
283
|
pool.reward(key);
|
|
283
284
|
return outcome;
|
|
@@ -339,7 +340,7 @@ async function runTier(hosts, query, signal, timeoutMs, failures, failureDetails
|
|
|
339
340
|
releasedEarly = early;
|
|
340
341
|
released();
|
|
341
342
|
};
|
|
342
|
-
const running = hosts.map((host, index) => searchHostWithKeys(host, query, controller.signal, timeoutMs, proxy, limits.timeRange, pool, anysearchDomain, safeSearch).then((outcome) => {
|
|
343
|
+
const running = hosts.map((host, index) => searchHostWithKeys(host, query, controller.signal, timeoutMs, proxy, limits.timeRange, pool, anysearchDomain, safeSearch, health).then((outcome) => {
|
|
343
344
|
if (outcome.ok) {
|
|
344
345
|
rawCounts[index] = outcome.sources.length;
|
|
345
346
|
outcomes[index] = { host, sources: filterRelevant(outcome.sources, query) };
|
package/lib/health.d.ts
CHANGED
|
@@ -33,6 +33,8 @@ export declare class EngineHealth {
|
|
|
33
33
|
private readonly maxCooldownMs;
|
|
34
34
|
/** Current state per engine id. */
|
|
35
35
|
private readonly records;
|
|
36
|
+
/** Engines whose direct route recently failed while the proxy worked. */
|
|
37
|
+
private readonly routePreferences;
|
|
36
38
|
/**
|
|
37
39
|
* @param options - base cooldown and ceiling, both optional.
|
|
38
40
|
*/
|
|
@@ -57,6 +59,23 @@ export declare class EngineHealth {
|
|
|
57
59
|
* @param id - engine id.
|
|
58
60
|
*/
|
|
59
61
|
noteSuccess(id: string): void;
|
|
62
|
+
/**
|
|
63
|
+
* Remember which route worked, so a blocked direct path is not paid for on
|
|
64
|
+
* every request. Only a proxy success creates a preference: a direct success
|
|
65
|
+
* clears it, because the direct path is normally faster and should win again
|
|
66
|
+
* as soon as it recovers.
|
|
67
|
+
*
|
|
68
|
+
* @param id - engine id.
|
|
69
|
+
* @param route - the route that produced the answer.
|
|
70
|
+
* @param now - shared timestamp; defaults to `Date.now`.
|
|
71
|
+
*/
|
|
72
|
+
noteRouteSuccess(id: string, route: 'direct' | 'proxy', now?: number): void;
|
|
73
|
+
/**
|
|
74
|
+
* @param id - engine id.
|
|
75
|
+
* @param now - shared timestamp; defaults to `Date.now`.
|
|
76
|
+
* @returns whether the engine should try its tunnel before its direct route.
|
|
77
|
+
*/
|
|
78
|
+
preferProxy(id: string, now?: number): boolean;
|
|
60
79
|
/**
|
|
61
80
|
* @param id - engine id.
|
|
62
81
|
* @param now - shared timestamp; defaults to `Date.now`.
|
|
@@ -81,6 +100,6 @@ export declare class EngineHealth {
|
|
|
81
100
|
* currently resting; usable engines are omitted.
|
|
82
101
|
*/
|
|
83
102
|
snapshot(now?: number): Record<string, number>;
|
|
84
|
-
/** Forget every engine, including failure streaks. */
|
|
103
|
+
/** Forget every engine, including failure streaks and route preferences. */
|
|
85
104
|
clear(): void;
|
|
86
105
|
}
|