@fffangchenghao/dsh-web-search 0.13.8 → 0.13.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,20 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.13.9
4
+
5
+ ### Fixed
6
+ - **`SECURITY.md` wrongly claimed the plugin spawns nothing.** It does: on macOS
7
+ the system-proxy probe runs `scutil --proxy`. The page now lists that single
8
+ process, the renderer's headless browser, and the `shell` return field that
9
+ the store's scanner had flagged as a shell-mode call site.
10
+ - **The proxy probe had no deadline.** A wedged `scutil` could hold a search
11
+ open; it is now capped at 2 s and the child is killed on expiry.
12
+ - **The store's fixed-source scanner read an internal field name as a
13
+ capability.** `RenderReport.shell` (a boolean: "this page needs rendering") is
14
+ now `shellPage`, so `shell: true` no longer matches the scanner's shell-mode
15
+ pattern. Behaviour is unchanged; the store's capability list should now
16
+ contain only real signals.
17
+
3
18
  ## 0.13.8
4
19
 
5
20
  ### Fixed
package/SECURITY.md CHANGED
@@ -1,8 +1,8 @@
1
1
  # Security and permissions
2
2
 
3
- This page states what the plugin can reach, what it writes, and where it fails.
4
- It is written for a reviewer deciding whether to install it, and for anyone
5
- auditing the fixed commit that DSH STORE scans.
3
+ This page states what the plugin can reach, what it runs, what it writes, and
4
+ where it fails. It is written for a reviewer deciding whether to install it, and
5
+ for anyone auditing the fixed commit that DSH STORE scans.
6
6
 
7
7
  ## Dependencies
8
8
 
@@ -21,7 +21,7 @@ There are no lifecycle scripts (`preinstall`, `install`, `postinstall`,
21
21
 
22
22
  ## Network
23
23
 
24
- Outbound HTTPS only. The default search set is:
24
+ Outbound TCP/HTTPS only, plus one local system query described below.
25
25
 
26
26
  | host | purpose |
27
27
  |---|---|
@@ -29,16 +29,37 @@ Outbound HTTPS only. The default search set is:
29
29
  | `html.duckduckgo.com` | key-free HTML search |
30
30
  | `www.so.com` | key-free HTML search, fallback tier, usually bot-walled |
31
31
  | `api.anysearch.com` | optional: search and page extraction, **only when an AnySearch key is configured** |
32
+ | your configured proxy | HTTP `CONNECT` tunnel for the hosts above |
32
33
 
33
34
  `web_fetch` and `read_page` reach **any** HTTP(S) URL the model asks for; that is
34
35
  their purpose. With `useProxy` enabled they speak to the configured proxy
35
- (default `127.0.0.1:7890`, auto-detected) using HTTP `CONNECT`. The plugin never
36
- contacts an analytics, telemetry, or update endpoint.
36
+ (default `127.0.0.1:7890`, auto-detected) using HTTP `CONNECT`; a 400 ms TCP
37
+ connect first checks that the proxy port is listening so a stale setting cannot
38
+ stall a search. The plugin never contacts an analytics, telemetry, or update
39
+ endpoint.
37
40
 
38
41
  What leaves the machine: the query text goes to each enabled search engine; a
39
42
  URL goes to AnySearch extraction only when a key is configured and the earlier
40
43
  rungs failed. Nothing else is transmitted.
41
44
 
45
+ ## Processes
46
+
47
+ The plugin spawns exactly one helper process:
48
+
49
+ - `/usr/sbin/scutil --proxy` — macOS only, when `useProxy` is on, no explicit
50
+ proxy is configured, and the system proxy has to be discovered. The argv is
51
+ fixed (no shell, no interpolation, no model- or user-controlled arguments),
52
+ stdout is parsed locally with fixed regexes, stderr is discarded, and the
53
+ probe is capped at 2 s before the child is killed. It reads nothing but the
54
+ system proxy settings.
55
+
56
+ The optional renderer is a **library**, not a command: when the user installed
57
+ `playwright` or `playwright-core` and enabled `renderJs`, the plugin imports it
58
+ dynamically and calls `chromium.launch({ headless: true })`, which starts a
59
+ headless browser process. That browser is the only third-party code the plugin
60
+ executes, and only under those two user choices. No shell command is ever
61
+ constructed from fetched content or model output.
62
+
42
63
  ## Credentials
43
64
 
44
65
  The only credential is an optional AnySearch key. It lives where you put it —
@@ -57,13 +78,6 @@ credentials, and it writes no credential anywhere.
57
78
  The cache holds queries and result rows only — no page bodies, no headers, no
58
79
  cookies.
59
80
 
60
- ## Commands
61
-
62
- The plugin executes **no** shell command and spawns **no** process. The optional
63
- renderer launches a headless browser through the Playwright library the user
64
- installed; that is the only way it runs third-party code, and only when the user
65
- both installed Playwright and enabled `renderJs`.
66
-
67
81
  ## Failure bounds
68
82
 
69
83
  | operation | bound |
@@ -74,6 +88,7 @@ both installed Playwright and enabled `renderJs`.
74
88
  | response body | 2 MiB read cap before decoding, 20 000 characters returned by default |
75
89
  | `read_page` | 2 000 000 characters per page, 8-page LRU, chunked reads |
76
90
  | cache | TTL and entry count are bounded and both configurable, `0` disables |
91
+ | system proxy probe | 2 s, then the child is killed |
77
92
 
78
93
  Failures are reported as structured codes (`SEARCH_THROTTLED`, `SEARCH_BLOCKED`,
79
94
  `SEARCH_TIMEOUT`, `SEARCH_NETWORK`, `SEARCH_PARSE`, `SEARCH_EMPTY`,
package/lib/fetch.js CHANGED
@@ -310,7 +310,7 @@ export function createFetchProvider(options = {}, overrides = {}) {
310
310
  }
311
311
  if (candidate !== undefined && mode === 'auto') {
312
312
  const verdict = detectShellPage(candidate);
313
- if (!verdict.shell) {
313
+ if (!verdict.shellPage) {
314
314
  return { done: true, outcome: candidate, report: makeReport(mode, false, false, verdict.reason) };
315
315
  }
316
316
  }
package/lib/proxy.js CHANGED
@@ -189,12 +189,26 @@ export async function proxyFromSystem() {
189
189
  return undefined;
190
190
  const output = await new Promise((resolve) => {
191
191
  let text = '';
192
- const child = spawn('scutil', ['--proxy']);
192
+ let settled = false;
193
+ const finish = (value) => {
194
+ if (settled)
195
+ return;
196
+ settled = true;
197
+ clearTimeout(timer);
198
+ resolve(value);
199
+ };
200
+ const child = spawn('scutil', ['--proxy'], { stdio: ['ignore', 'pipe', 'ignore'] });
201
+ // `scutil` answers in milliseconds; a wedged system call must not hold a
202
+ // search open, so the probe is capped and the child is killed on expiry.
203
+ const timer = setTimeout(() => {
204
+ child.kill();
205
+ finish('');
206
+ }, 2_000);
193
207
  child.stdout.on('data', (chunk) => {
194
208
  text += chunk.toString();
195
209
  });
196
- child.on('error', () => resolve(''));
197
- child.on('close', () => resolve(text));
210
+ child.on('error', () => finish(''));
211
+ child.on('close', () => finish(text));
198
212
  });
199
213
  if (!/HTTPSEnable\s*:\s*1|HTTPEnable\s*:\s*1/.test(output))
200
214
  return undefined;
package/lib/render.d.ts CHANGED
@@ -50,7 +50,7 @@ export interface ShellInput {
50
50
  /** A shell decision plus the numbers and reason behind it. */
51
51
  export interface ShellVerdict {
52
52
  /** Whether a browser render is worth attempting. */
53
- readonly shell: boolean;
53
+ readonly shellPage: boolean;
54
54
  /** Characters of visible text, tags and script bodies removed. */
55
55
  readonly readableChars: number;
56
56
  /** Short, content-free explanation, safe to surface in a report. */
package/lib/render.js CHANGED
@@ -94,27 +94,27 @@ export function readableTextLength(html) {
94
94
  */
95
95
  export function detectShellPage(page) {
96
96
  if (page.body.kind !== 'html') {
97
- return { shell: false, readableChars: 0, reason: 'the body is not HTML' };
97
+ return { shellPage: false, readableChars: 0, reason: 'the body is not HTML' };
98
98
  }
99
99
  const readable = readableTextLength(page.body.content);
100
100
  const status = page.statusCode;
101
101
  const renderableStatus = (status >= 200 && status < 300) || status === 403 || status === 429;
102
102
  if (!renderableStatus) {
103
- return { shell: false, readableChars: readable, reason: `HTTP ${status} is an error response, not a page shell` };
103
+ return { shellPage: false, readableChars: readable, reason: `HTTP ${status} is an error response, not a page shell` };
104
104
  }
105
105
  if (readable === 0) {
106
- return { shell: true, readableChars: 0, reason: 'the HTML carries no readable text' };
106
+ return { shellPage: true, readableChars: 0, reason: 'the HTML carries no readable text' };
107
107
  }
108
108
  if (status === 403 || status === 429) {
109
109
  return {
110
- shell: true,
110
+ shellPage: true,
111
111
  readableChars: readable,
112
112
  reason: `HTTP ${status} still answered with ${readable} readable characters`,
113
113
  };
114
114
  }
115
115
  if (readable < SHELL_READABLE_LIMIT) {
116
116
  return {
117
- shell: true,
117
+ shellPage: true,
118
118
  readableChars: readable,
119
119
  reason: `only ${readable} readable characters, below the ${SHELL_READABLE_LIMIT}-character shell limit`,
120
120
  };
@@ -124,13 +124,13 @@ export function detectShellPage(page) {
124
124
  const marker = SHELL_MARKERS.find((candidate) => haystack.includes(candidate));
125
125
  if (marker !== undefined) {
126
126
  return {
127
- shell: true,
127
+ shellPage: true,
128
128
  readableChars: readable,
129
129
  reason: `${readable} readable characters with the client-render marker ${marker}`,
130
130
  };
131
131
  }
132
132
  }
133
- return { shell: false, readableChars: readable, reason: `${readable} readable characters look like real content` };
133
+ return { shellPage: false, readableChars: readable, reason: `${readable} readable characters look like real content` };
134
134
  }
135
135
  /**
136
136
  * Scrub a diagnostic string before it leaves this module.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@fffangchenghao/dsh-web-search",
3
- "version": "0.13.8",
3
+ "version": "0.13.9",
4
4
  "description": "DSH bundle: a key-free web search provider. Leads with the anonymous AnySearch API and falls back to public result pages (DuckDuckGo, Bing, 360), merged by reciprocal-rank fusion with a bilingual region policy, an engine cooldown and a TTL cache.",
5
5
  "type": "module",
6
6
  "main": "lib/index.js",