dsh-all-search 0.1.2 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +16 -10
- package/README.zh.md +9 -5
- package/dist/index.d.ts +21 -2
- package/dist/index.js +110 -12
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,18 +1,22 @@
|
|
|
1
1
|
<p align="center">
|
|
2
|
-
<img src="assets/readme/hero.svg" alt="dsh-all-search
|
|
2
|
+
<img src="assets/readme/hero.svg" alt="dsh-all-search AnySearch web search for DeepSeek Harness" width="100%">
|
|
3
3
|
</p>
|
|
4
4
|
|
|
5
5
|
# dsh-all-search
|
|
6
6
|
|
|
7
7
|
Adds an **AnySearch** web-search provider to DeepSeek Harness, registered into `ctx.web`. AnySearch is a single MCP gateway that aggregates exa / tavily / firecrawl / context7 behind **one API key**.
|
|
8
8
|
|
|
9
|
+
When `api_key` is set, queries go to that gateway. When it is absent, the same `anysearch` provider stays available and searches via **Firecrawl** `POST /v1/search` with no `Authorization` header (keyless). Set optional `firecrawl_api_key` to send `Authorization: Bearer` and use your own quota. NAT/CI runners that share an egress IP share that keyless daily credit pool.
|
|
10
|
+
|
|
11
|
+
With `firecrawl_api_key` and an AnySearch key, developer-intent queries (repo / issue / PR / commit / skill) are answered by the **Firecrawl Developer Index** first -- a semantic artifact index over READMEs, issues, PRs, OpenAPI specs and skills -- falling back to AnySearch when it fails or returns nothing.
|
|
12
|
+
|
|
9
13
|
> Port of [pi-all-search](https://github.com/RealAlexandreAI/pi-all-search).
|
|
10
14
|
|
|
11
15
|
[English](README.md) · [中文](README.zh.md)
|
|
12
16
|
|
|
13
17
|
## Why
|
|
14
18
|
|
|
15
|
-
dsh ships Exa / Perplexity / DeepSeek search. This plugin adds AnySearch: one key, many backends, no per-backend credentials.
|
|
19
|
+
dsh ships Exa / Perplexity / DeepSeek search. This plugin adds AnySearch: one key, many backends, no per-backend credentials. Without an AnySearch key, Firecrawl still works so the provider is never a no-op.
|
|
16
20
|
|
|
17
21
|
## Quick start
|
|
18
22
|
|
|
@@ -20,33 +24,35 @@ dsh ships Exa / Perplexity / DeepSeek search. This plugin adds AnySearch: one ke
|
|
|
20
24
|
dsh plugin --profile web add dsh-all-search
|
|
21
25
|
```
|
|
22
26
|
|
|
23
|
-
The provider registers as `anysearch` on `ctx.web`
|
|
27
|
+
The provider registers as `anysearch` on `ctx.web` -- the built-in `web_search` tool picks it up alongside the stock providers.
|
|
24
28
|
|
|
25
29
|
```yaml
|
|
26
30
|
- id: all-search
|
|
27
31
|
name: dsh-all-search
|
|
28
32
|
config:
|
|
29
|
-
api_key: <your anysearch key>
|
|
33
|
+
api_key: <your anysearch key> # optional
|
|
30
34
|
```
|
|
31
35
|
|
|
32
36
|
| key | required | meaning |
|
|
33
37
|
|---|---|---|
|
|
34
|
-
| `api_key` |
|
|
35
|
-
| `base_url` |
|
|
38
|
+
| `api_key` | - | AnySearch key. When set, queries use the AnySearch gateway. |
|
|
39
|
+
| `base_url` | - | MCP endpoint override |
|
|
40
|
+
| `firecrawl_api_key` | - | optional Firecrawl key (Bearer quota). Also enables the Developer Index branch for repo/issue/PR/skill queries when AnySearch is configured |
|
|
36
41
|
|
|
37
|
-
Without
|
|
42
|
+
Without an AnySearch key the provider still reports `available() = true` and searches Firecrawl keyless. Hosts that share an egress IP (NAT, CI) share those daily credits; set `firecrawl_api_key` for a private quota.
|
|
38
43
|
|
|
39
44
|
## Privacy
|
|
40
45
|
|
|
41
|
-
-
|
|
42
|
-
-
|
|
46
|
+
- Keys live only in your config file -- never logged.
|
|
47
|
+
- With `api_key`, only your query and result count go to the AnySearch gateway.
|
|
48
|
+
- Without `api_key`, the query goes to Firecrawl `/v1/search`.
|
|
43
49
|
|
|
44
50
|
## Development
|
|
45
51
|
|
|
46
52
|
```bash
|
|
47
53
|
npm install
|
|
48
54
|
npm run typecheck
|
|
49
|
-
npm test # result parsing / maxResults / HTTP errors
|
|
55
|
+
npm test # result parsing / maxResults / HTTP errors / keyless Firecrawl routing
|
|
50
56
|
npm run build
|
|
51
57
|
```
|
|
52
58
|
|
package/README.zh.md
CHANGED
|
@@ -6,6 +6,8 @@
|
|
|
6
6
|
|
|
7
7
|
给 DeepSeek Harness 加一个 **AnySearch** 搜索 provider,注册进 `ctx.web`。AnySearch 是单 MCP 网关,一把 API key 聚合 exa / tavily / firecrawl / context7 等多家搜索。
|
|
8
8
|
|
|
9
|
+
配置了 api_key 时查询走 AnySearch 网关。没有 AnySearch key 时,同一个 anysearch provider 仍可用,改走 Firecrawl POST /v1/search,不带 Authorization(keyless)。若设置可选的 firecrawl_api_key,则发送 Authorization: Bearer 并使用你自己的配额。共享出口 IP 的 NAT/CI 会共用这份无 key 日额度。
|
|
10
|
+
|
|
9
11
|
> 由 [pi-all-search](https://github.com/RealAlexandreAI/pi-all-search) 移植。
|
|
10
12
|
|
|
11
13
|
[English](README.md) · [中文](README.zh.md)
|
|
@@ -26,27 +28,29 @@ provider 以 `anysearch` 注册到 `ctx.web`,内置的 `web_search` 工具会自
|
|
|
26
28
|
- id: all-search
|
|
27
29
|
name: dsh-all-search
|
|
28
30
|
config:
|
|
29
|
-
api_key: <你的 anysearch key>
|
|
31
|
+
api_key: <你的 anysearch key> # 可选
|
|
30
32
|
```
|
|
31
33
|
|
|
32
34
|
| 键 | 必填 | 说明 |
|
|
33
35
|
|---|---|---|
|
|
34
|
-
| `api_key` |
|
|
36
|
+
| `api_key` | – | AnySearch key。配置后查询走 AnySearch 网关。 |
|
|
35
37
|
| `base_url` | – | MCP 端点覆盖 |
|
|
38
|
+
| `firecrawl_api_key` | – | 可选 Firecrawl key(Bearer 配额)。配置 AnySearch 时还启用 Developer Index 分支 |
|
|
36
39
|
|
|
37
|
-
|
|
40
|
+
没有 AnySearch key 时 provider 仍 available() = true,走 Firecrawl keyless。共享出口 IP 的 NAT/CI 会共用日额度;需要独立配额时设置 firecrawl_api_key。
|
|
38
41
|
|
|
39
42
|
## 隐私
|
|
40
43
|
|
|
41
44
|
- key 只存在于你的配置文件——不写日志
|
|
42
|
-
-
|
|
45
|
+
- 有 api_key 时只向 AnySearch 网关发送查询词和结果数量
|
|
46
|
+
- 没有 api_key 时查询发往 Firecrawl /v1/search
|
|
43
47
|
|
|
44
48
|
## 开发
|
|
45
49
|
|
|
46
50
|
```bash
|
|
47
51
|
npm install
|
|
48
52
|
npm run typecheck
|
|
49
|
-
npm test # 结果解析 / maxResults / HTTP 错误
|
|
53
|
+
npm test # 结果解析 / maxResults / HTTP 错误 / keyless Firecrawl 路由
|
|
50
54
|
npm run build
|
|
51
55
|
```
|
|
52
56
|
|
package/dist/index.d.ts
CHANGED
|
@@ -4,12 +4,16 @@ import type { WebSearchProvider, WebSearchRequest, WebSearchResult } from '@deep
|
|
|
4
4
|
export declare const name = "all-search";
|
|
5
5
|
export declare const inject: string[];
|
|
6
6
|
export interface Config {
|
|
7
|
-
/** AnySearch API key. */
|
|
8
|
-
api_key
|
|
7
|
+
/** AnySearch API key. When set, queries use the AnySearch gateway. */
|
|
8
|
+
api_key?: string;
|
|
9
9
|
/** AnySearch MCP endpoint override. */
|
|
10
10
|
base_url?: string;
|
|
11
|
+
/** Firecrawl API key — Bearer quota for /v1/search; also enables Developer Index when AnySearch is configured. */
|
|
12
|
+
firecrawl_api_key?: string;
|
|
11
13
|
}
|
|
12
14
|
export declare const Config: z<Config>;
|
|
15
|
+
export declare const FIRECRAWL_SEARCH_URL = "https://api.firecrawl.dev/v1/search";
|
|
16
|
+
export declare function isDeveloperQuery(query: string): boolean;
|
|
13
17
|
export declare class AnysearchProvider implements WebSearchProvider {
|
|
14
18
|
private readonly apiKey;
|
|
15
19
|
private readonly baseUrl;
|
|
@@ -20,4 +24,19 @@ export declare class AnysearchProvider implements WebSearchProvider {
|
|
|
20
24
|
search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult>;
|
|
21
25
|
private parseResults;
|
|
22
26
|
}
|
|
27
|
+
/** Firecrawl general web search. Works keyless; Bearer only when an API key is provided. */
|
|
28
|
+
export declare class FirecrawlSearchProvider implements WebSearchProvider {
|
|
29
|
+
private readonly apiKey?;
|
|
30
|
+
readonly id = "firecrawl";
|
|
31
|
+
constructor(apiKey?: string);
|
|
32
|
+
available(): boolean;
|
|
33
|
+
search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult>;
|
|
34
|
+
}
|
|
35
|
+
export declare class FirecrawlDevProvider implements WebSearchProvider {
|
|
36
|
+
private readonly apiKey;
|
|
37
|
+
readonly id = "firecrawl-dev";
|
|
38
|
+
constructor(apiKey: string);
|
|
39
|
+
available(): boolean;
|
|
40
|
+
search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult>;
|
|
41
|
+
}
|
|
23
42
|
export declare function apply(ctx: Context, config: Config): void;
|
package/dist/index.js
CHANGED
|
@@ -1,21 +1,31 @@
|
|
|
1
1
|
// dsh-all-search — DeepSeek Harness (Cordis) plugin.
|
|
2
2
|
//
|
|
3
|
-
// Registers
|
|
4
|
-
// a
|
|
5
|
-
//
|
|
3
|
+
// Registers a single WebSearchProvider (id anysearch) with ctx.web. AnySearch
|
|
4
|
+
// is a MCP gateway that aggregates exa / tavily / firecrawl / context7 behind
|
|
5
|
+
// one API key. When that key is set, queries stay on the gateway (developer-
|
|
6
|
+
// intent queries may try the Firecrawl Developer Index first). When the
|
|
7
|
+
// AnySearch key is absent, the same wrapper stays available and searches via
|
|
8
|
+
// Firecrawl POST /v1/search — no Authorization header unless firecrawl_api_key
|
|
9
|
+
// is set (then Bearer, own quota).
|
|
6
10
|
//
|
|
7
|
-
//
|
|
8
|
-
//
|
|
11
|
+
// One registerSearchProvider call: the dsh-web seam errors on multiple usable
|
|
12
|
+
// providers (WEB_PROVIDER_AMBIGUOUS) unless one is configured explicitly.
|
|
9
13
|
//
|
|
10
|
-
// Privacy:
|
|
14
|
+
// Privacy: API keys live in the plugin config only; they are never logged.
|
|
11
15
|
import z from '@deepseek-ai/schemastery';
|
|
12
16
|
export const name = 'all-search';
|
|
13
17
|
export const inject = ['web'];
|
|
14
18
|
export const Config = z.object({
|
|
15
|
-
api_key: z.string().
|
|
19
|
+
api_key: z.string().description('AnySearch API key (optional — without it, Firecrawl keyless search is used)'),
|
|
16
20
|
base_url: z.string().description('AnySearch MCP endpoint override (default https://api.anysearch.com/mcp)'),
|
|
21
|
+
firecrawl_api_key: z.string().description('Firecrawl API key (optional) — Bearer for Firecrawl quota; also enables Developer Index when AnySearch is configured'),
|
|
17
22
|
});
|
|
18
23
|
const DEFAULT_BASE_URL = 'https://api.anysearch.com/mcp';
|
|
24
|
+
export const FIRECRAWL_SEARCH_URL = 'https://api.firecrawl.dev/v1/search';
|
|
25
|
+
const DEVELOPER_QUERY_RE = /\b(repo|repos|repository|repositories|github|issue|issues|pull request|pull requests|commit|commits|branch|merge|openapi|readme|skill|skills|library|framework)\b/i;
|
|
26
|
+
export function isDeveloperQuery(query) {
|
|
27
|
+
return DEVELOPER_QUERY_RE.test(query);
|
|
28
|
+
}
|
|
19
29
|
export class AnysearchProvider {
|
|
20
30
|
apiKey;
|
|
21
31
|
baseUrl;
|
|
@@ -72,15 +82,103 @@ export class AnysearchProvider {
|
|
|
72
82
|
return results;
|
|
73
83
|
}
|
|
74
84
|
}
|
|
85
|
+
/** Firecrawl general web search. Works keyless; Bearer only when an API key is provided. */
|
|
86
|
+
export class FirecrawlSearchProvider {
|
|
87
|
+
apiKey;
|
|
88
|
+
id = 'firecrawl';
|
|
89
|
+
constructor(apiKey) {
|
|
90
|
+
this.apiKey = apiKey;
|
|
91
|
+
}
|
|
92
|
+
available() {
|
|
93
|
+
return true;
|
|
94
|
+
}
|
|
95
|
+
async search(request, signal) {
|
|
96
|
+
const headers = { 'Content-Type': 'application/json' };
|
|
97
|
+
if (this.apiKey)
|
|
98
|
+
headers.Authorization = `Bearer ${this.apiKey}`;
|
|
99
|
+
const resp = await fetch(FIRECRAWL_SEARCH_URL, {
|
|
100
|
+
method: 'POST',
|
|
101
|
+
headers,
|
|
102
|
+
signal,
|
|
103
|
+
body: JSON.stringify({ query: request.query, limit: request.maxResults ?? 5 }),
|
|
104
|
+
});
|
|
105
|
+
if (!resp.ok)
|
|
106
|
+
throw new Error(`Firecrawl HTTP ${resp.status}`);
|
|
107
|
+
const data = (await resp.json());
|
|
108
|
+
const sources = (data.data ?? []).map((r) => ({
|
|
109
|
+
title: r.title ?? '',
|
|
110
|
+
url: r.url ?? '',
|
|
111
|
+
snippet: r.description ?? '',
|
|
112
|
+
}));
|
|
113
|
+
const truncated = request.maxResults !== undefined && sources.length > request.maxResults;
|
|
114
|
+
return {
|
|
115
|
+
sources: truncated ? sources.slice(0, request.maxResults) : sources,
|
|
116
|
+
truncated,
|
|
117
|
+
};
|
|
118
|
+
}
|
|
119
|
+
}
|
|
120
|
+
export class FirecrawlDevProvider {
|
|
121
|
+
apiKey;
|
|
122
|
+
id = 'firecrawl-dev';
|
|
123
|
+
constructor(apiKey) {
|
|
124
|
+
this.apiKey = apiKey;
|
|
125
|
+
}
|
|
126
|
+
available() {
|
|
127
|
+
return Boolean(this.apiKey);
|
|
128
|
+
}
|
|
129
|
+
async search(request, signal) {
|
|
130
|
+
const resp = await fetch('https://api.firecrawl.dev/v2/search/developer', {
|
|
131
|
+
method: 'POST',
|
|
132
|
+
headers: {
|
|
133
|
+
'Content-Type': 'application/json',
|
|
134
|
+
Authorization: `Bearer ${this.apiKey}`,
|
|
135
|
+
},
|
|
136
|
+
signal,
|
|
137
|
+
body: JSON.stringify({ query: request.query, limit: request.maxResults ?? 5 }),
|
|
138
|
+
});
|
|
139
|
+
if (!resp.ok)
|
|
140
|
+
throw new Error(`Firecrawl Developer Index HTTP ${resp.status}`);
|
|
141
|
+
const data = (await resp.json());
|
|
142
|
+
const sources = (data.data ?? []).map((r) => ({
|
|
143
|
+
title: r.title ?? r.id ?? '',
|
|
144
|
+
url: r.url ?? '',
|
|
145
|
+
snippet: r.description ??
|
|
146
|
+
(Array.isArray(r.passages) ? r.passages.slice(0, 3).join(' ') : '') ??
|
|
147
|
+
'',
|
|
148
|
+
}));
|
|
149
|
+
const truncated = request.maxResults !== undefined && sources.length > request.maxResults;
|
|
150
|
+
return {
|
|
151
|
+
sources: truncated ? sources.slice(0, request.maxResults) : sources,
|
|
152
|
+
truncated,
|
|
153
|
+
};
|
|
154
|
+
}
|
|
155
|
+
}
|
|
75
156
|
export function apply(ctx, config) {
|
|
76
|
-
|
|
157
|
+
const anysearch = new AnysearchProvider(config.api_key ?? '', config.base_url ?? DEFAULT_BASE_URL);
|
|
158
|
+
const firecrawl = new FirecrawlSearchProvider(config.firecrawl_api_key);
|
|
159
|
+
const devIndex = config.firecrawl_api_key ? new FirecrawlDevProvider(config.firecrawl_api_key) : null;
|
|
160
|
+
// Single provider to avoid dsh-web's WEB_PROVIDER_AMBIGUOUS on multiple
|
|
161
|
+
// usable providers. AnySearch key → gateway (Dev Index first for developer
|
|
162
|
+
// queries). No AnySearch key → Firecrawl /v1/search (keyless unless a
|
|
163
|
+
// firecrawl_api_key is set).
|
|
77
164
|
const wrapper = {
|
|
78
165
|
id: 'anysearch',
|
|
79
|
-
available: () => Boolean(config.api_key),
|
|
166
|
+
available: () => Boolean(config.api_key) || firecrawl.available(),
|
|
80
167
|
async search(request, signal) {
|
|
81
|
-
if (
|
|
82
|
-
|
|
83
|
-
|
|
168
|
+
if (config.api_key) {
|
|
169
|
+
if (devIndex && isDeveloperQuery(request.query)) {
|
|
170
|
+
try {
|
|
171
|
+
const dev = await devIndex.search(request, signal);
|
|
172
|
+
if (dev.sources.length > 0)
|
|
173
|
+
return dev;
|
|
174
|
+
}
|
|
175
|
+
catch {
|
|
176
|
+
// fall through to AnySearch
|
|
177
|
+
}
|
|
178
|
+
}
|
|
179
|
+
return anysearch.search(request, signal);
|
|
180
|
+
}
|
|
181
|
+
return firecrawl.search(request, signal);
|
|
84
182
|
},
|
|
85
183
|
};
|
|
86
184
|
ctx.web.registerSearchProvider(wrapper);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "dsh-all-search",
|
|
3
|
-
"version": "0.1
|
|
3
|
+
"version": "0.2.1",
|
|
4
4
|
"description": "DeepSeek Harness plugin: AnySearch-backed web search provider (aggregates exa/tavily/firecrawl via one MCP gateway) registered into ctx.web.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|