@fffangchenghao/dsh-web-search 0.13.6 → 0.13.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,38 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.13.7
4
+
5
+ ### Fixed
6
+ - **A blank query burned seven seconds and six requests.** `search('')` and
7
+ `search(' ')` reached every engine, each failed on its own, and the caller
8
+ got an empty list. Validation now rejects it in under a millisecond with
9
+ `SEARCH_INVALID`.
10
+ - **A PDF arrived as binary noise.** `kindOf()` treated every non-HTML response
11
+ as text, so `application/pdf` came back as 20 000 characters of PDF stream and
12
+ looked like a successful fetch. Binary content types now fail with
13
+ `SEARCH_UNSUPPORTED` and name the type.
14
+ - **A JavaScript shell was a silent success.** `https://github.com/` and
15
+ `https://excalidraw.com/` returned HTTP 200 with zero readable characters, so
16
+ the caller saw an empty page with no explanation. A 200 with no text now
17
+ fails with `SEARCH_EMPTY` and states why, including "no renderer installed".
18
+ Non-2xx responses keep their old behaviour: the status line plus the body.
19
+ - **A layout `<main>` could throw away the article.** Extraction trusted any
20
+ non-empty landmark, so a page whose `<main>` held a breadcrumb produced four
21
+ characters while the prose sat in a sibling container. A landmark now has to
22
+ keep a real share of the page's text, and the body is the last resort.
23
+ - **The fetch ladder stacked deadlines.** Direct (5 s) + tunnel (15 s) + render
24
+ + extraction meant a dead host could hold a caller for half a minute. The
25
+ whole ladder now shares a 25-second budget and later rungs get only the
26
+ remainder.
27
+
28
+ ### Added
29
+ - `scripts/sweep.mjs` sweeps the ranking knobs over the recorded corpus and
30
+ prints a paired-bootstrap interval against the defaults. `bench/README.md`
31
+ records the result: the best configuration beats the defaults by +0.044 MRR
32
+ with an interval of [−0.004, +0.123], so **the defaults are unchanged**. The
33
+ same page documents live `wait`-policy timings (`auto` keeps the most distinct
34
+ domains).
35
+
3
36
  ## 0.13.6
4
37
 
5
38
  ### Fixed
package/README.md CHANGED
@@ -478,9 +478,10 @@ Yes — to the engines taking part in that search, which is what searching means
478
478
  Four usual causes:
479
479
 
480
480
  1. **The site refuses automated access**: a 403 or a verification page. With Playwright installed and content still on the page, `renderJs: auto` tries the local browser first; otherwise, if direct and tunnel both fail, only an AnySearch key enables server-side extraction.
481
- 2. **The content is rendered by JavaScript**: what comes back is an empty shell. With Playwright installed it is rendered automatically; without it, you see the shell.
481
+ 2. **The content is rendered by JavaScript**: what comes back is an empty shell. With Playwright installed it is rendered automatically; without it, you see the shell. **If stripping the shell leaves no readable text at all, the fetch fails with `SEARCH_EMPTY` and says why — including "no renderer installed" — instead of returning an empty page as a success.**
482
482
  3. **It needs a login or a subscription**: fetching carries no session of yours.
483
- 4. **Direct requests time out**: 5 seconds direct, then the tunnel; if neither works and no key is configured you get an error, and with a key there is one more rung, server-side extraction. The message names each rung's failure.
483
+ 4. **Direct requests time out**: 5 seconds direct, then the tunnel; if neither works and no key is configured you get an error, and with a key there is one more rung, server-side extraction. The whole ladder shares a 25-second budget, so later rungs get only what is left. The message names each rung's failure.
484
+ 5. **Binary documents**: PDFs, images and archives are refused explicitly (`SEARCH_UNSUPPORTED`) with the content type, rather than handing you a byte stream as prose. To read a PDF, extract its text with another tool first.
484
485
 
485
486
  360's captcha belongs to the first cause, but it used to be especially quiet: from some networks the request 302s to the qcaptcha "unexpected access" page while the HTTP status is still 200. The plugin now records that as `SEARCH_BLOCKED`; querying that engine alone shows `so: bot verification page` instead of zero results, and 360 has been demoted to the fallback tier.
486
487
 
@@ -511,7 +512,7 @@ If the page is short enough for `web_fetch` in one pass, you do not need `read_p
511
512
  First decide whether it is a shell page — HTML with a title and a loading script, whose body appears only after JavaScript runs. Then, in order:
512
513
 
513
514
  1. **If you installed Playwright yourself** (`npm i -g playwright && npx playwright install chromium`): leave `renderJs` on `auto` and reload the plugin, then retry. The plugin recognises the shell and renders it in headless Chromium; `always` forces a render on every fetch, and `off` disables rendering.
514
- 2. **Without Playwright**: nothing is installed automatically and nothing errors; the result says why rendering was skipped and returns the shell. Try another source (a print view, a static mirror, the site's API), or raise `fetchMaxLength` — but that only helps when the text was actually fetched and cut at the tail; it cannot create a body.
515
+ 2. **Without Playwright**: nothing is installed automatically; if the fetch ends up with **no readable text at all**, it fails with `SEARCH_EMPTY` and names the reason — including that no renderer is installed — rather than returning an empty shell as a success. Try another source (a print view, a static mirror, the site's API), or raise `fetchMaxLength` — but that only helps when the text was actually fetched and cut at the tail; it cannot create a body.
515
516
  3. **If the page needs a login or a subscription**: rendering cannot rescue it either; fetching carries no session of yours.
516
517
 
517
518
  ### Does the plugin secretly install a browser?
package/README.zh-CN.md CHANGED
@@ -478,9 +478,10 @@ pnpm install
478
478
  常见原因有四类:
479
479
 
480
480
  1. **站点拒绝自动访问**:回 403 或验证页。自己装了 Playwright 且页面还有内容时,`renderJs: auto` 会先试本机浏览器;否则直连、隧道都过不去时,只有配了 AnySearch key 才会尝试服务端抽取。
481
- 2. **正文靠 JavaScript 渲染**:抓回来的是空壳 HTML,看不到内容。装了 Playwright 后会自动渲染;没装就只能看到壳。
481
+ 2. **正文靠 JavaScript 渲染**:抓回来的是空壳 HTML,看不到内容。装了 Playwright 后会自动渲染;没装就只能看到壳。**如果去壳之后一个字都不剩,插件会报错(`SEARCH_EMPTY`)并写明「没有可用渲染器」之类的判定,而不是把空壳当成成功结果返回**。
482
482
  3. **要登录或付费**:抓取没有你的登录态。
483
- 4. **直连超时**:会先在 5 秒内试直连,失败转隧道;两个都不通、又没配 key 时报错,配了 key 还会再试服务端抽取。报错里会写清楚每一级失败的原因。
483
+ 4. **直连超时**:会先在 5 秒内试直连,失败转隧道;两个都不通、又没配 key 时报错,配了 key 还会再试服务端抽取。整条梯队总共只有 25 秒预算,后面的档位只能花剩下的时间。报错里会写清楚每一级失败的原因。
484
+ 5. **二进制文档**:PDF、图片、压缩包会明确拒绝(`SEARCH_UNSUPPORTED`)并告出内容类型,不会把字节流当成正文塞给你。需要读 PDF 的话,先用别的工具取文本。
484
485
 
485
486
  360 的验证码页属于第 1 类,但它以前特别隐蔽:从部分网络访问会 302 到 qcaptcha 的「访问异常页面」,HTTP 状态却是 200。插件现在会把它记成 `SEARCH_BLOCKED`,单独查该引擎会看到 `so: bot verification page`,而不是 0 条;360 也因此降为兜底档。
486
487
 
@@ -511,7 +512,7 @@ pnpm install
511
512
  先判断它是不是「壳页面」——HTML 里只有标题和加载脚本,正文要等 JavaScript 跑完才出现。然后按顺序试:
512
513
 
513
514
  1. **自己装了 Playwright**(`npm i -g playwright && npx playwright install chromium`):把 `renderJs` 留在 `auto`,重载插件后再抓。插件会识别壳页面,开一个无头 Chromium 把正文渲染出来;`always` 可以强制每次都渲染,`off` 则完全禁用。
514
- 2. **没装 Playwright**:插件不会自动装,也不会报错;结果里会写明跳过渲染的原因,并返回原来的壳。这时换一个来源(打印版、静态镜像、站点 API),或者把 `fetchMaxLength` 调大——但后者只对「其实抓到了、只是被截断」的情况有用,变不出正文。
515
+ 2. **没装 Playwright**:插件不会自动装;如果抓回来**完全没有可读正文**,它会明确报错(`SEARCH_EMPTY`),信息里说明是否因为没有渲染器,而不是返回一个空壳当成功。这时换一个来源(打印版、静态镜像、站点 API),或者把 `fetchMaxLength` 调大——但后者只对「其实抓到了、只是被截断」的情况有用,变不出正文。
515
516
  3. **页面本身要登录或付费**:渲染也救不回来,抓取没有你的登录态。
516
517
 
517
518
  ### 插件会不会偷偷装浏览器?
package/lib/engines.js CHANGED
@@ -265,6 +265,11 @@ export function resolveHost(host, options) {
265
265
  return { ...host, body, ...(headers === undefined ? {} : { headers }) };
266
266
  }
267
267
  export async function search(query, signal, options) {
268
+ // A blank query used to reach every engine, burn seven seconds and come back
269
+ // with six identical "empty query" failures; refuse it before any request.
270
+ if (typeof query !== 'string' || query.trim().length === 0) {
271
+ throw new SearchError('the search query is empty', 'SEARCH_INVALID', false);
272
+ }
268
273
  const limit = Math.max(1, Math.min(options.limit, 20));
269
274
  const timeoutMs = options.timeoutMs ?? 15000;
270
275
  const wait = options.wait ?? 'auto';
package/lib/fetch.d.ts CHANGED
@@ -5,10 +5,12 @@ export interface FetchOutcome {
5
5
  readonly url: string;
6
6
  readonly statusCode: number;
7
7
  readonly body: {
8
- readonly kind: 'html' | 'text';
8
+ readonly kind: 'html' | 'text' | 'binary';
9
9
  readonly content: string;
10
10
  };
11
11
  readonly truncated: boolean;
12
+ /** The response's mime type, kept so a rejection can name what it refused. */
13
+ readonly contentType?: string;
12
14
  /**
13
15
  * How the optional render rung decided, when this provider produced the
14
16
  * result. The Host ignores the extra field; it is here so a caller (and a
@@ -31,6 +33,8 @@ export interface FetchProviderOptions {
31
33
  id?: string;
32
34
  /** Per-attempt deadline in milliseconds. */
33
35
  timeoutMs?: number;
36
+ /** Budget for the whole ladder; later rungs get only what is left. */
37
+ overallMs?: number;
34
38
  /** Explicit tunnel URL, overriding detection. Empty means detect. */
35
39
  proxy?: string;
36
40
  /** `false` disables the tunnel rung entirely. */
package/lib/fetch.js CHANGED
@@ -44,15 +44,33 @@ const DEFAULT_MAX_BYTES = 1_048_576;
44
44
  const DIRECT_DEADLINE_MS = 5_000;
45
45
  /** Default per-attempt deadline. */
46
46
  const DEFAULT_TIMEOUT_MS = 15_000;
47
+ /**
48
+ * Budget for the whole ladder, however many rungs it tries.
49
+ *
50
+ * Each rung has its own deadline, so without a ceiling a blocked host can spend
51
+ * five seconds on a direct attempt and fifteen on each later rung — long enough
52
+ * that a caller waits half a minute for a page that will never arrive.
53
+ */
54
+ const DEFAULT_OVERALL_MS = 25_000;
55
+ /** Below this much remaining budget a rung is not worth starting. */
56
+ const MIN_RUNG_MS = 750;
57
+ /** Content types whose bytes are documents, not text this plugin can read. */
58
+ const BINARY_TYPE = /^\s*(application\/(pdf|zip|gzip|x-tar|vnd\.|msword|octet-stream|protobuf|x-protobuf)|image\/|audio\/|video\/|font\/)/i;
47
59
  /** Pick the body kind from the content type, defaulting to text. */
48
60
  function kindOf(contentType) {
49
61
  if (contentType === null)
50
62
  return 'text';
51
63
  const value = contentType.toLowerCase();
64
+ if (BINARY_TYPE.test(value))
65
+ return 'binary';
52
66
  if (value.includes('html') || value.includes('xml'))
53
67
  return 'html';
54
68
  return 'text';
55
69
  }
70
+ /** The mime type without parameters, for a message a reader can act on. */
71
+ function mimeOf(contentType) {
72
+ return (contentType ?? '').split(';')[0]?.trim() ?? '';
73
+ }
56
74
  /**
57
75
  * Cut a body at the byte budget without splitting a surrogate pair.
58
76
  * @param text - the decoded body.
@@ -94,7 +112,13 @@ async function directFetch(url, timeoutMs, signal, userAgent, maxBytes) {
94
112
  const contentType = response.headers.get('content-type');
95
113
  const text = await response.text();
96
114
  const { content, truncated } = clamp(text, maxBytes);
97
- return { url: response.url === '' ? url : response.url, statusCode: response.status, body: { kind: kindOf(contentType), content }, truncated };
115
+ return {
116
+ url: response.url === '' ? url : response.url,
117
+ statusCode: response.status,
118
+ body: { kind: kindOf(contentType), content },
119
+ contentType: mimeOf(contentType),
120
+ truncated,
121
+ };
98
122
  }
99
123
  /** Tunnelled request used by the second rung. */
100
124
  async function tunnelFetch(url, proxy, timeoutMs, signal, userAgent, maxBytes) {
@@ -105,10 +129,12 @@ async function tunnelFetch(url, proxy, timeoutMs, signal, userAgent, maxBytes) {
105
129
  ...(signal === undefined ? {} : { signal }),
106
130
  });
107
131
  const { content, truncated } = clamp(response.text, maxBytes);
132
+ const contentType = response.headers?.['content-type'] ?? null;
108
133
  return {
109
134
  url,
110
135
  statusCode: response.status,
111
- body: { kind: kindOf(response.headers?.['content-type'] ?? null), content },
136
+ body: { kind: kindOf(contentType), content },
137
+ contentType: mimeOf(contentType),
112
138
  truncated,
113
139
  };
114
140
  }
@@ -146,6 +172,10 @@ export function createFetchProvider(options = {}, overrides = {}) {
146
172
  ...overrides,
147
173
  };
148
174
  const timeoutMs = options.timeoutMs ?? DEFAULT_TIMEOUT_MS;
175
+ const overallMs = options.overallMs ?? DEFAULT_OVERALL_MS;
176
+ const startedAt = Date.now();
177
+ /** What is left of the whole-ladder budget. */
178
+ const remaining = () => Math.max(0, overallMs - (Date.now() - startedAt));
149
179
  const maxBytes = options.maxBytes ?? DEFAULT_MAX_BYTES;
150
180
  const userAgent = options.userAgent ?? USER_AGENT;
151
181
  const apiKey = (options.anysearchApiKey ?? '').trim();
@@ -196,6 +226,10 @@ export function createFetchProvider(options = {}, overrides = {}) {
196
226
  const finalize = (outcome, render) => {
197
227
  const format = liveFormat();
198
228
  const maxLength = liveMaxLength();
229
+ if (outcome.body.kind === 'binary') {
230
+ const mime = outcome.contentType === undefined || outcome.contentType === '' ? 'binary content' : outcome.contentType;
231
+ throw new SearchError(`cannot read ${mime} from ${outcome.url}: this plugin converts HTML and text to Markdown, and a binary document would arrive as noise`, 'SEARCH_UNSUPPORTED', false);
232
+ }
199
233
  if (outcome.body.kind === 'html') {
200
234
  const rendered = renderFetchText(outcome.body.content, { format, maxLength, baseUrl: outcome.url });
201
235
  return {
@@ -213,6 +247,26 @@ export function createFetchProvider(options = {}, overrides = {}) {
213
247
  render,
214
248
  };
215
249
  };
250
+ /**
251
+ * Reject an outcome a caller would otherwise read as a successful page.
252
+ *
253
+ * A 4xx/5xx stays a normal result on purpose: the Host renders the status line
254
+ * with the body, and a 403 page often carries the only useful explanation. An
255
+ * empty 200 is different — it reads as success while saying nothing.
256
+ */
257
+ const assertUsable = (outcome, failures) => {
258
+ if (outcome.statusCode < 400 && outcome.body.content.trim() === '') {
259
+ // The render rung's reason describes the raw bytes, not the extraction
260
+ // result, so never lead with it: "7387 readable characters" next to "empty
261
+ // page" reads like a contradiction.
262
+ const detail = failures.length === 0 ? '' : `; ${failures.join('; ')}`;
263
+ const render = outcome.render?.reason === undefined || outcome.render.reason === ''
264
+ ? ''
265
+ : `; renderer: ${outcome.render.reason}`;
266
+ throw new SearchError(`no readable text from ${outcome.url} (HTTP ${outcome.statusCode}): the bytes carried no prose once scripts and markup were removed${render}${detail}`, 'SEARCH_EMPTY', true);
267
+ }
268
+ return outcome;
269
+ };
216
270
  return {
217
271
  id: options.id ?? 'http-proxy',
218
272
  available: () => true,
@@ -318,7 +372,10 @@ export function createFetchProvider(options = {}, overrides = {}) {
318
372
  // that cannot work would leave the rung that can work with nothing.
319
373
  let direct;
320
374
  try {
321
- direct = await deps.direct(target, Math.min(timeoutMs, DIRECT_DEADLINE_MS), signal, userAgent, maxBytes);
375
+ const directBudget = Math.min(timeoutMs, DIRECT_DEADLINE_MS, remaining());
376
+ if (directBudget < MIN_RUNG_MS)
377
+ throw new Error('no time left in the fetch budget');
378
+ direct = await deps.direct(target, directBudget, signal, userAgent, maxBytes);
322
379
  }
323
380
  catch (error) {
324
381
  if (cancelled())
@@ -328,7 +385,7 @@ export function createFetchProvider(options = {}, overrides = {}) {
328
385
  if (direct !== undefined) {
329
386
  const rendered = await settle(direct);
330
387
  if (rendered !== undefined)
331
- return finalize(rendered, reportFor());
388
+ return assertUsable(finalize(rendered, reportFor()), failures);
332
389
  }
333
390
  let proxy;
334
391
  if (options.useProxy !== false) {
@@ -342,7 +399,10 @@ export function createFetchProvider(options = {}, overrides = {}) {
342
399
  if (proxy !== undefined && proxy.length > 0) {
343
400
  let tunneled;
344
401
  try {
345
- tunneled = await deps.tunnel(target, proxy, timeoutMs, signal, userAgent, maxBytes);
402
+ const tunnelBudget = Math.min(timeoutMs, remaining());
403
+ if (tunnelBudget < MIN_RUNG_MS)
404
+ throw new Error('no time left in the fetch budget');
405
+ tunneled = await deps.tunnel(target, proxy, tunnelBudget, signal, userAgent, maxBytes);
346
406
  }
347
407
  catch (error) {
348
408
  if (cancelled())
@@ -352,7 +412,7 @@ export function createFetchProvider(options = {}, overrides = {}) {
352
412
  if (tunneled !== undefined) {
353
413
  const rendered = await settle(tunneled);
354
414
  if (rendered !== undefined)
355
- return finalize(rendered, reportFor());
415
+ return assertUsable(finalize(rendered, reportFor()), failures);
356
416
  }
357
417
  }
358
418
  // `always` is also a standalone rescue: a transport that returns 403 or
@@ -362,18 +422,21 @@ export function createFetchProvider(options = {}, overrides = {}) {
362
422
  const step = await renderStep(undefined);
363
423
  lastReport = step.report;
364
424
  if (step.done && step.outcome !== undefined)
365
- return finalize(step.outcome, step.report);
425
+ return assertUsable(finalize(step.outcome, step.report), failures);
366
426
  if (!step.done)
367
427
  failures.push(`render: ${step.report.reason}`);
368
428
  }
369
429
  if (apiKey.length > 0) {
370
430
  try {
371
- const extracted = await deps.extract(target, apiKey, timeoutMs, signal);
431
+ const extractBudget = Math.min(timeoutMs, remaining());
432
+ if (extractBudget < MIN_RUNG_MS)
433
+ throw new Error('no time left in the fetch budget');
434
+ const extracted = await deps.extract(target, apiKey, extractBudget, signal);
372
435
  // Say where the body came from when the browser was tried first.
373
436
  const report = lastReport !== undefined && lastReport.attempted && !lastReport.applied
374
437
  ? makeReport(lastReport.mode, true, false, `${lastReport.reason}; the result came from AnySearch extraction`)
375
438
  : reportFor();
376
- return finalize(extracted, report);
439
+ return assertUsable(finalize(extracted, report), failures);
377
440
  }
378
441
  catch (error) {
379
442
  if (cancelled())
@@ -387,7 +450,7 @@ export function createFetchProvider(options = {}, overrides = {}) {
387
450
  const report = lastReport !== undefined && lastReport.attempted && !lastReport.applied
388
451
  ? makeReport(lastReport.mode, true, false, `${lastReport.reason}; the earlier rung's result was kept`)
389
452
  : reportFor();
390
- return finalize(fallback, report);
453
+ return assertUsable(finalize(fallback, report), failures);
391
454
  }
392
455
  throw new SearchError(`cannot fetch ${target} (${failures.join('; ')})`, 'SEARCH_NETWORK', true);
393
456
  },
package/lib/markdown.js CHANGED
@@ -437,12 +437,25 @@ function pickMain(root) {
437
437
  articleCharacters = characters;
438
438
  }
439
439
  });
440
- if (article !== undefined && articleCharacters > 0)
441
- return article;
442
440
  const main = findElement(body, (node) => node.tag === 'main' || (node.attrs['role'] ?? '').toLowerCase() === 'main');
443
- if (main !== undefined && characterCount(main) > 0)
444
- return main;
445
- return densestContainer(body);
441
+ const densest = densestContainer(body);
442
+ // A `<main>` or `<article>` that holds almost nothing is a layout shell, not
443
+ // the reading root: github.com returns half a megabyte of HTML whose `main`
444
+ // is a sidebar, which used to convert to zero characters and look like a
445
+ // network failure. Only trust a landmark that keeps a real share of the text.
446
+ const bodyCharacters = characterCount(body);
447
+ const floor = Math.min(2_000, Math.max(120, Math.round(bodyCharacters * 0.25)));
448
+ const landmarks = [article, main];
449
+ for (const landmark of landmarks) {
450
+ if (landmark !== undefined && characterCount(landmark) >= floor)
451
+ return landmark;
452
+ }
453
+ if (characterCount(densest) >= floor)
454
+ return densest;
455
+ // Nothing reaches the floor: return whichever of the three holds the most.
456
+ const ranked = [article, main, densest].filter((node) => node !== undefined);
457
+ ranked.sort((left, right) => characterCount(right) - characterCount(left));
458
+ return ranked[0] ?? body;
446
459
  }
447
460
  /** Replace a code fragment with a placeholder so clean-up cannot touch it. */
448
461
  function stashCode(ctx, text) {
@@ -795,13 +808,24 @@ export function htmlToMarkdown(html, options = {}) {
795
808
  const body = findElement(root, (node) => node.tag === 'body') ?? root;
796
809
  if (characterCount(body) === 0)
797
810
  return '';
798
- const main = pickMain(root);
799
- const ctx = {
800
- format: options.format ?? 'markdown',
801
- baseUrl: options.baseUrl,
802
- stash: [],
811
+ const renderFrom = (node) => {
812
+ const ctx = {
813
+ format: options.format ?? 'markdown',
814
+ baseUrl: options.baseUrl,
815
+ stash: [],
816
+ };
817
+ return restoreCode(cleanup(renderElement(node, ctx)), ctx).trim();
803
818
  };
804
- return restoreCode(cleanup(renderElement(main, ctx)), ctx).trim();
819
+ const main = pickMain(root);
820
+ const chosen = renderFrom(main);
821
+ // Last guard: if the landmark produced (almost) nothing while the page holds
822
+ // real prose, read the body instead of returning an empty document.
823
+ if (chosen.length < 120 && characterCount(body) >= 400) {
824
+ const whole = renderFrom(body);
825
+ if (whole.length > chosen.length)
826
+ return whole;
827
+ }
828
+ return chosen;
805
829
  }
806
830
  /**
807
831
  * Apply the output character budget.
package/lib/render.d.ts CHANGED
@@ -43,7 +43,7 @@ export declare const SHELL_MARKER_READABLE_LIMIT = 1200;
43
43
  export interface ShellInput {
44
44
  readonly statusCode: number;
45
45
  readonly body: {
46
- readonly kind: 'html' | 'text';
46
+ readonly kind: 'html' | 'text' | 'binary';
47
47
  readonly content: string;
48
48
  };
49
49
  }
package/lib/types.d.ts CHANGED
@@ -142,4 +142,12 @@ export interface HealthOptions {
142
142
  maxCooldownMs?: number;
143
143
  }
144
144
  /** Business codes a failure can carry. */
145
- export type SearchErrorCode = 'SEARCH_THROTTLED' | 'SEARCH_BLOCKED' | 'SEARCH_TIMEOUT' | 'SEARCH_NETWORK' | 'SEARCH_PARSE' | 'SEARCH_NETWORK_BLOCKED_BY_POLICY' | 'SEARCH_ABORTED';
145
+ export type SearchErrorCode = 'SEARCH_THROTTLED' | 'SEARCH_BLOCKED' | 'SEARCH_TIMEOUT' | 'SEARCH_NETWORK' | 'SEARCH_PARSE' | 'SEARCH_NETWORK_BLOCKED_BY_POLICY' | 'SEARCH_ABORTED'
146
+ /** The caller sent nonsense (an empty query, a non-HTTP URL). */
147
+ | 'SEARCH_INVALID'
148
+ /** The page answered, but nothing readable came back. */
149
+ | 'SEARCH_EMPTY'
150
+ /** The page answered with a 4xx/5xx status. */
151
+ | 'SEARCH_HTTP_STATUS'
152
+ /** The page is a binary document this plugin cannot convert to text. */
153
+ | 'SEARCH_UNSUPPORTED';
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@fffangchenghao/dsh-web-search",
3
- "version": "0.13.6",
3
+ "version": "0.13.7",
4
4
  "description": "DSH bundle: a key-free web search provider. Leads with the anonymous AnySearch API and falls back to public result pages (DuckDuckGo, Bing, 360), merged by reciprocal-rank fusion with a bilingual region policy, an engine cooldown and a TTL cache.",
5
5
  "type": "module",
6
6
  "main": "lib/index.js",