dsh-all-search 0.2.0 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,12 +1,14 @@
1
1
  <p align="center">
2
- <img src="assets/readme/hero.svg" alt="dsh-all-search — AnySearch web search for DeepSeek Harness" width="100%">
2
+ <img src="assets/readme/hero.svg" alt="dsh-all-search AnySearch web search for DeepSeek Harness" width="100%">
3
3
  </p>
4
4
 
5
5
  # dsh-all-search
6
6
 
7
7
  Adds an **AnySearch** web-search provider to DeepSeek Harness, registered into `ctx.web`. AnySearch is a single MCP gateway that aggregates exa / tavily / firecrawl / context7 behind **one API key**.
8
8
 
9
- With an optional `firecrawl_api_key`, developer-intent queries (repo / issue / PR / commit / skill) are answered by the **Firecrawl Developer Index** first — a semantic artifact index over READMEs, issues, PRs, OpenAPI specs and skills — falling back to AnySearch when it fails or returns nothing.
9
+ When `api_key` is set, queries go to that gateway. When it is absent, the same `anysearch` provider stays available and searches via **Firecrawl** `POST /v1/search` with no `Authorization` header (keyless). Set optional `firecrawl_api_key` to send `Authorization: Bearer` and use your own quota. NAT/CI runners that share an egress IP share that keyless daily credit pool.
10
+
11
+ With `firecrawl_api_key` and an AnySearch key, developer-intent queries (repo / issue / PR / commit / skill) are answered by the **Firecrawl Developer Index** first -- a semantic artifact index over READMEs, issues, PRs, OpenAPI specs and skills -- falling back to AnySearch when it fails or returns nothing.
10
12
 
11
13
  > Port of [pi-all-search](https://github.com/RealAlexandreAI/pi-all-search).
12
14
 
@@ -14,7 +16,7 @@ With an optional `firecrawl_api_key`, developer-intent queries (repo / issue / P
14
16
 
15
17
  ## Why
16
18
 
17
- dsh ships Exa / Perplexity / DeepSeek search. This plugin adds AnySearch: one key, many backends, no per-backend credentials.
19
+ dsh ships Exa / Perplexity / DeepSeek search. This plugin adds AnySearch: one key, many backends, no per-backend credentials. Without an AnySearch key, Firecrawl still works so the provider is never a no-op.
18
20
 
19
21
  ## Quick start
20
22
 
@@ -22,34 +24,35 @@ dsh ships Exa / Perplexity / DeepSeek search. This plugin adds AnySearch: one ke
22
24
  dsh plugin --profile web add dsh-all-search
23
25
  ```
24
26
 
25
- The provider registers as `anysearch` on `ctx.web` — the built-in `web_search` tool picks it up alongside the stock providers.
27
+ The provider registers as `anysearch` on `ctx.web` -- the built-in `web_search` tool picks it up alongside the stock providers.
26
28
 
27
29
  ```yaml
28
30
  - id: all-search
29
31
  name: dsh-all-search
30
32
  config:
31
- api_key: <your anysearch key>
33
+ api_key: <your anysearch key> # optional
32
34
  ```
33
35
 
34
36
  | key | required | meaning |
35
37
  |---|---|---|
36
- | `api_key` | ✅ | your AnySearch key |
37
- | `base_url` | – | MCP endpoint override |
38
- | `firecrawl_api_key` | – | enables the Firecrawl Developer Index branch for repo/issue/PR/skill queries |
38
+ | `api_key` | - | AnySearch key. When set, queries use the AnySearch gateway. |
39
+ | `base_url` | - | MCP endpoint override |
40
+ | `firecrawl_api_key` | - | optional Firecrawl key (Bearer quota). Also enables the Developer Index branch for repo/issue/PR/skill queries when AnySearch is configured |
39
41
 
40
- Without a key the provider reports `available() = false` and the seam skips it.
42
+ Without an AnySearch key the provider still reports `available() = true` and searches Firecrawl keyless. Hosts that share an egress IP (NAT, CI) share those daily credits; set `firecrawl_api_key` for a private quota.
41
43
 
42
44
  ## Privacy
43
45
 
44
- - The key lives only in your config file — never logged.
45
- - Only your query and result count go to the AnySearch gateway.
46
+ - Keys live only in your config file -- never logged.
47
+ - With `api_key`, only your query and result count go to the AnySearch gateway.
48
+ - Without `api_key`, the query goes to Firecrawl `/v1/search`.
46
49
 
47
50
  ## Development
48
51
 
49
52
  ```bash
50
53
  npm install
51
54
  npm run typecheck
52
- npm test # result parsing / maxResults / HTTP errors
55
+ npm test # result parsing / maxResults / HTTP errors / keyless Firecrawl routing
53
56
  npm run build
54
57
  ```
55
58
 
package/README.zh.md CHANGED
@@ -6,6 +6,8 @@
6
6
 
7
7
  给 DeepSeek Harness 加一个 **AnySearch** 搜索 provider,注册进 `ctx.web`。AnySearch 是单 MCP 网关,一把 API key 聚合 exa / tavily / firecrawl / context7 等多家搜索。
8
8
 
9
+ 配置了 api_key 时查询走 AnySearch 网关。没有 AnySearch key 时,同一个 anysearch provider 仍可用,改走 Firecrawl POST /v1/search,不带 Authorization(keyless)。若设置可选的 firecrawl_api_key,则发送 Authorization: Bearer 并使用你自己的配额。共享出口 IP 的 NAT/CI 会共用这份无 key 日额度。
10
+
9
11
  > 由 [pi-all-search](https://github.com/RealAlexandreAI/pi-all-search) 移植。
10
12
 
11
13
  [English](README.md) · [中文](README.zh.md)
@@ -26,27 +28,29 @@ provider 以 `anysearch` 注册到 `ctx.web`,内置的 `web_search` 工具会自
26
28
  - id: all-search
27
29
  name: dsh-all-search
28
30
  config:
29
- api_key: <你的 anysearch key>
31
+ api_key: <你的 anysearch key> # 可选
30
32
  ```
31
33
 
32
34
  | 键 | 必填 | 说明 |
33
35
  |---|---|---|
34
- | `api_key` | ✅ | 你的 AnySearch key |
36
+ | `api_key` | – | AnySearch key。配置后查询走 AnySearch 网关。 |
35
37
  | `base_url` | – | MCP 端点覆盖 |
38
+ | `firecrawl_api_key` | – | 可选 Firecrawl key(Bearer 配额)。配置 AnySearch 时还启用 Developer Index 分支 |
36
39
 
37
- \* 两个 key 键填其一。没有 key 时 provider `available() = false`,seam 自动跳过。
40
+ 没有 AnySearch key 时 provider 仍 available() = true,走 Firecrawl keyless。共享出口 IP 的 NAT/CI 会共用日额度;需要独立配额时设置 firecrawl_api_key。
38
41
 
39
42
  ## 隐私
40
43
 
41
44
  - key 只存在于你的配置文件——不写日志
42
- - 只向 AnySearch 网关发送查询词和结果数量
45
+ - 有 api_key 时只向 AnySearch 网关发送查询词和结果数量
46
+ - 没有 api_key 时查询发往 Firecrawl /v1/search
43
47
 
44
48
  ## 开发
45
49
 
46
50
  ```bash
47
51
  npm install
48
52
  npm run typecheck
49
- npm test # 结果解析 / maxResults / HTTP 错误
53
+ npm test # 结果解析 / maxResults / HTTP 错误 / keyless Firecrawl 路由
50
54
  npm run build
51
55
  ```
52
56
 
package/dist/index.d.ts CHANGED
@@ -4,14 +4,15 @@ import type { WebSearchProvider, WebSearchRequest, WebSearchResult } from '@deep
4
4
  export declare const name = "all-search";
5
5
  export declare const inject: string[];
6
6
  export interface Config {
7
- /** AnySearch API key. */
8
- api_key: string;
7
+ /** AnySearch API key. When set, queries use the AnySearch gateway. */
8
+ api_key?: string;
9
9
  /** AnySearch MCP endpoint override. */
10
10
  base_url?: string;
11
- /** Firecrawl API key — enables the Developer Index branch for developer-intent queries. */
11
+ /** Firecrawl API key — Bearer quota for /v1/search; also enables Developer Index when AnySearch is configured. */
12
12
  firecrawl_api_key?: string;
13
13
  }
14
14
  export declare const Config: z<Config>;
15
+ export declare const FIRECRAWL_SEARCH_URL = "https://api.firecrawl.dev/v1/search";
15
16
  export declare function isDeveloperQuery(query: string): boolean;
16
17
  export declare class AnysearchProvider implements WebSearchProvider {
17
18
  private readonly apiKey;
@@ -23,6 +24,14 @@ export declare class AnysearchProvider implements WebSearchProvider {
23
24
  search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult>;
24
25
  private parseResults;
25
26
  }
27
+ /** Firecrawl general web search. Works keyless; Bearer only when an API key is provided. */
28
+ export declare class FirecrawlSearchProvider implements WebSearchProvider {
29
+ private readonly apiKey?;
30
+ readonly id = "firecrawl";
31
+ constructor(apiKey?: string);
32
+ available(): boolean;
33
+ search(request: WebSearchRequest, signal?: AbortSignal): Promise<WebSearchResult>;
34
+ }
26
35
  export declare class FirecrawlDevProvider implements WebSearchProvider {
27
36
  private readonly apiKey;
28
37
  readonly id = "firecrawl-dev";
package/dist/index.js CHANGED
@@ -1,29 +1,27 @@
1
1
  // dsh-all-search — DeepSeek Harness (Cordis) plugin.
2
2
  //
3
- // Registers an AnySearch-backed WebSearchProvider with ctx.web. AnySearch is
4
- // a single MCP gateway that aggregates exa / tavily / firecrawl / context7
5
- // behind one API key, so one provider covers many backends.
3
+ // Registers a single WebSearchProvider (id anysearch) with ctx.web. AnySearch
4
+ // is a MCP gateway that aggregates exa / tavily / firecrawl / context7 behind
5
+ // one API key. When that key is set, queries stay on the gateway (developer-
6
+ // intent queries may try the Firecrawl Developer Index first). When the
7
+ // AnySearch key is absent, the same wrapper stays available and searches via
8
+ // Firecrawl POST /v1/search — no Authorization header unless firecrawl_api_key
9
+ // is set (then Bearer, own quota).
6
10
  //
7
- // Ported from pi-all-search (providers/anysearch.ts) with the pi UI/typebox
8
- // surface replaced by the dsh WebSearchProvider seam.
9
- //
10
- // Optionally, with `firecrawl_api_key` set, developer-intent queries
11
- // (repo / issue / PR / github / commit / skill) are answered by the
12
- // Firecrawl Developer Index first (semantic artifact index: READMEs, issues,
13
- // PRs, OpenAPI specs, skills), falling back to AnySearch when it fails or
14
- // returns nothing. Kept inside one provider because the dsh-web seam errors
15
- // on multiple usable providers unless one is configured explicitly.
11
+ // One registerSearchProvider call: the dsh-web seam errors on multiple usable
12
+ // providers (WEB_PROVIDER_AMBIGUOUS) unless one is configured explicitly.
16
13
  //
17
14
  // Privacy: API keys live in the plugin config only; they are never logged.
18
15
  import z from '@deepseek-ai/schemastery';
19
16
  export const name = 'all-search';
20
17
  export const inject = ['web'];
21
18
  export const Config = z.object({
22
- api_key: z.string().required().description('AnySearch API key'),
19
+ api_key: z.string().description('AnySearch API key (optional — without it, Firecrawl keyless search is used)'),
23
20
  base_url: z.string().description('AnySearch MCP endpoint override (default https://api.anysearch.com/mcp)'),
24
- firecrawl_api_key: z.string().description('Firecrawl API key (optional) — enables Developer Index for repo/issue/PR/skill queries'),
21
+ firecrawl_api_key: z.string().description('Firecrawl API key (optional) — Bearer for Firecrawl quota; also enables Developer Index when AnySearch is configured'),
25
22
  });
26
23
  const DEFAULT_BASE_URL = 'https://api.anysearch.com/mcp';
24
+ export const FIRECRAWL_SEARCH_URL = 'https://api.firecrawl.dev/v1/search';
27
25
  const DEVELOPER_QUERY_RE = /\b(repo|repos|repository|repositories|github|issue|issues|pull request|pull requests|commit|commits|branch|merge|openapi|readme|skill|skills|library|framework)\b/i;
28
26
  export function isDeveloperQuery(query) {
29
27
  return DEVELOPER_QUERY_RE.test(query);
@@ -84,6 +82,41 @@ export class AnysearchProvider {
84
82
  return results;
85
83
  }
86
84
  }
85
+ /** Firecrawl general web search. Works keyless; Bearer only when an API key is provided. */
86
+ export class FirecrawlSearchProvider {
87
+ apiKey;
88
+ id = 'firecrawl';
89
+ constructor(apiKey) {
90
+ this.apiKey = apiKey;
91
+ }
92
+ available() {
93
+ return true;
94
+ }
95
+ async search(request, signal) {
96
+ const headers = { 'Content-Type': 'application/json' };
97
+ if (this.apiKey)
98
+ headers.Authorization = `Bearer ${this.apiKey}`;
99
+ const resp = await fetch(FIRECRAWL_SEARCH_URL, {
100
+ method: 'POST',
101
+ headers,
102
+ signal,
103
+ body: JSON.stringify({ query: request.query, limit: request.maxResults ?? 5 }),
104
+ });
105
+ if (!resp.ok)
106
+ throw new Error(`Firecrawl HTTP ${resp.status}`);
107
+ const data = (await resp.json());
108
+ const sources = (data.data ?? []).map((r) => ({
109
+ title: r.title ?? '',
110
+ url: r.url ?? '',
111
+ snippet: r.description ?? '',
112
+ }));
113
+ const truncated = request.maxResults !== undefined && sources.length > request.maxResults;
114
+ return {
115
+ sources: truncated ? sources.slice(0, request.maxResults) : sources,
116
+ truncated,
117
+ };
118
+ }
119
+ }
87
120
  export class FirecrawlDevProvider {
88
121
  apiKey;
89
122
  id = 'firecrawl-dev';
@@ -121,27 +154,31 @@ export class FirecrawlDevProvider {
121
154
  }
122
155
  }
123
156
  export function apply(ctx, config) {
124
- const anysearch = new AnysearchProvider(config.api_key, config.base_url ?? DEFAULT_BASE_URL);
157
+ const anysearch = new AnysearchProvider(config.api_key ?? '', config.base_url ?? DEFAULT_BASE_URL);
158
+ const firecrawl = new FirecrawlSearchProvider(config.firecrawl_api_key);
125
159
  const devIndex = config.firecrawl_api_key ? new FirecrawlDevProvider(config.firecrawl_api_key) : null;
126
160
  // Single provider to avoid dsh-web's WEB_PROVIDER_AMBIGUOUS on multiple
127
- // usable providers. Developer-intent queries try the Developer Index first.
161
+ // usable providers. AnySearch key → gateway (Dev Index first for developer
162
+ // queries). No AnySearch key → Firecrawl /v1/search (keyless unless a
163
+ // firecrawl_api_key is set).
128
164
  const wrapper = {
129
165
  id: 'anysearch',
130
- available: () => Boolean(config.api_key),
166
+ available: () => Boolean(config.api_key) || firecrawl.available(),
131
167
  async search(request, signal) {
132
- if (!config.api_key)
133
- throw new Error('AnySearch API key not configured (set api_key in the plugin config)');
134
- if (devIndex && isDeveloperQuery(request.query)) {
135
- try {
136
- const dev = await devIndex.search(request, signal);
137
- if (dev.sources.length > 0)
138
- return dev;
139
- }
140
- catch {
141
- // fall through to AnySearch
168
+ if (config.api_key) {
169
+ if (devIndex && isDeveloperQuery(request.query)) {
170
+ try {
171
+ const dev = await devIndex.search(request, signal);
172
+ if (dev.sources.length > 0)
173
+ return dev;
174
+ }
175
+ catch {
176
+ // fall through to AnySearch
177
+ }
142
178
  }
179
+ return anysearch.search(request, signal);
143
180
  }
144
- return anysearch.search(request, signal);
181
+ return firecrawl.search(request, signal);
145
182
  },
146
183
  };
147
184
  ctx.web.registerSearchProvider(wrapper);
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dsh-all-search",
3
- "version": "0.2.0",
3
+ "version": "0.2.1",
4
4
  "description": "DeepSeek Harness plugin: AnySearch-backed web search provider (aggregates exa/tavily/firecrawl via one MCP gateway) registered into ctx.web.",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",