deepseek-web-retrieval 0.9.0 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/docs/agents/pi.md CHANGED
@@ -8,7 +8,9 @@ start. The same build also ships an MCP server over stdio as the fallback path
8
8
  ([below](#the-mcp-fallback-path)); the extension is the recommended one:
9
9
 
10
10
  - The tools are declared to the model like built-ins (`exposure: "direct"` is Pi's own
11
- default for extension tools — the MCP path's codemode trap does not exist here).
11
+ default for extension tools — the MCP path defaults its tools to
12
+ `exposure: "codemode"`, which hides them from the model unless the entry opts into
13
+ `direct`; that trap does not exist here).
12
14
  - Results bypass Pi's MCP layer entirely: no 20 KB middle truncation and no 60 s
13
15
  per-request timeout; the adapter's own output cap is the only limit.
14
16
  - The key is read from the Pi process environment — no `env` forwarding, no `${VAR}`
@@ -19,7 +21,8 @@ When loaded, it adds:
19
21
  - A `deepseek_web_search` tool backed by DeepSeek Search
20
22
  - A `deepseek_fetch` tool backed by DeepSeek Fetch
21
23
 
22
- Pi 1.3.0 ships no built-in web tool, so the registered tools are the only web path. For
24
+ Pi 1.3.0 ships no built-in web tool, so the registered tools are the only web path (see
25
+ [Control tiers](#control-tiers)). For
23
26
  shared search/fetch limits, retries, and privacy behavior, see
24
27
  [General behavior](../../README.md#general-behavior).
25
28
  For configuration layers, the full knob mapping, and defaults, see the
@@ -29,7 +32,7 @@ For configuration layers, the full knob mapping, and defaults, see the
29
32
 
30
33
  - Pi with host SDKs in the `^1.3.0` peer range (`>=1.3.0 <2.0.0`); the tested
31
34
  version is recorded in [`docs/hosts.jsonc`](../hosts.jsonc)
32
- - Node.js 20+ and npm when using the installer CLI or MCP fallback
35
+ - Node.js 20 or newer and npm when using the installer CLI or MCP fallback
33
36
  - A DeepSeek account with access to `/v1/tools/search` and `/v1/tools/fetch`
34
37
  - A funded DeepSeek account; Search and Fetch are billed per successful request
35
38
  - `DEEPSEEK_API_KEY` set in the shell that launches Pi
@@ -55,7 +58,8 @@ source at the next session start (the first launch fetches the package from npm;
55
58
  to install the [MCP fallback path](#the-mcp-fallback-path) instead. `uninstall` removes
56
59
  recorded additions and restores unchanged configuration values from their snapshots.
57
60
 
58
- For upgrades from 0.4.0, select `--path mcp` explicitly to retain the old path, or use
61
+ For upgrades from a 0.4.x or earlier install (MCP was Pi's default path before 0.5.0),
62
+ select `--path mcp` explicitly to retain the old path, or use
59
63
  the extension default to migrate; follow the shared
60
64
  [upgrade instructions](../integration.md#upgrading-existing-installs).
61
65
 
@@ -86,6 +90,27 @@ pi install /absolute/path/to/deepseek-web-retrieval
86
90
  The extension entry imports only host-supplied modules (`@earendil-works/pi-coding-agent`,
87
91
  `@earendil-works/pi-ai`) and the package's own `src/core/`, so a plain clone works.
88
92
 
93
+ ## Control tiers
94
+
95
+ Pi 1.3.0 ships no built-in web tool, so the registered tools are the only web path at
96
+ every tier — forced is de-facto rather than exclusion:
97
+
98
+ | Capability | Optional | Preferred | Forced |
99
+ | --- | --- | --- | --- |
100
+ | Search | Tools registered — already the only web search | Plus an AGENTS.md rules block steering the model to the search tool | De-facto: no built-in competitor; web-capable MCP servers in `mcp.json` are disabled |
101
+ | Fetch | Tools registered — already the only web fetch | Plus the rules block | De-facto: same |
102
+
103
+ The installer maps each tier to these artifacts. The rules block lands in
104
+ `~/.pi/agent/AGENTS.md` (global) or `./AGENTS.md` (project), naming the path's tool
105
+ ids (`deepseek_web_search` for the extension, `mcp__deepseek__deepseek_web_search` for
106
+ the MCP fallback). Under a forced tier, web-capable MCP servers detected in
107
+ `~/.pi/agent/mcp.json` or `.pi/mcp.json` get `enabled: false` (each prior value
108
+ snapshotted for uninstall); a web-capable `packages` or `extensions` source in
109
+ settings.json cannot be excluded — Pi has no per-package kill switch — so `init`
110
+ refuses the tier instead of promising routing it cannot deliver. The `off` tier is the
111
+ `deepseekWebRetrieval` settings namespace described under
112
+ [Configuration](#configuration).
113
+
89
114
  ## Authentication
90
115
 
91
116
  `DEEPSEEK_API_KEY` is the only source: the extension runs inside the Pi process and reads
@@ -122,7 +147,8 @@ opens with `Searched for: <effective query>` (plus any degradation applied to fi
122
147
  The runtime config surface is the same `DEEPSEEK_*` environment the MCP server reads
123
148
  (identical defaults and clamping), set in the shell that launches Pi, plus the
124
149
  `deepseekWebRetrieval` settings namespace for the off switches (written by the installer;
125
- it outranks the environment and is preserved by Pi's own settings rewrites).
150
+ it outranks the environment and is preserved by Pi's own settings rewrites). Invalid
151
+ `DEEPSEEK_*` values are dropped at startup and the shared default applies.
126
152
 
127
153
  The shared config file (`~/.config/deepseek-web-retrieval/config.json`, camelCase keys —
128
154
  on Windows, `%APPDATA%\deepseek-web-retrieval\config.json`)
@@ -162,7 +188,7 @@ one vocabulary pair, one pipeline. `—` marks host-integration entries the file
162
188
  | `DEEPSEEK_FETCH_TIMEOUT_MS` | `fetchTimeoutMs` | `30000` | Fetch timeout, 1,000–120,000 ms. No host-side cap applies on the extension path. |
163
189
  | `DEEPSEEK_FETCH_ALLOWED_DOMAINS` | `fetchAllowedDomains` | unset | If set, only these hosts and subdomains may be fetched. |
164
190
  | `DEEPSEEK_FETCH_BLOCKED_DOMAINS` | `fetchBlockedDomains` | unset | Always block these hosts and subdomains. Blocked wins over allowed. |
165
- | `DEEPSEEK_FETCH_BLOCK_PRIVATE_HOSTS` | `fetchBlockPrivateHosts` | `true` | Reject localhost, private IPs, `.local`, `.internal`, and similar hosts. Set `false`/`0` to disable. |
191
+ | `DEEPSEEK_FETCH_BLOCK_PRIVATE_HOSTS` | `fetchBlockPrivateHosts` | `true` | Reject localhost, private IPs, `.local`, `.internal`, and similar hosts. Set `false`/`0`/`no`/`off` to disable. |
166
192
 
167
193
  ### Credentials and retries
168
194
 
@@ -172,8 +198,6 @@ one vocabulary pair, one pipeline. `—` marks host-integration entries the file
172
198
  | `DEEPSEEK_MAX_RETRIES` | `maxRetries` | `2` | Retries for 408, 425, 429, and 5xx responses, 0–5. |
173
199
  | `DEEPSEEK_RETRY_BASE_DELAY_MS` | `retryBaseDelayMs` | `250` | Base delay for exponential backoff. `Retry-After` is honored up to 10 seconds. |
174
200
 
175
- Invalid values fall back to the shared default.
176
-
177
201
  ## Troubleshooting
178
202
 
179
203
  | Error or symptom | What to do |
@@ -197,9 +221,9 @@ Invalid values fall back to the shared default.
197
221
  | Runtime | Inside the Pi process (Node, host-provided) |
198
222
  | DeepSeek API | `/v1/tools/search`, `/v1/tools/fetch` |
199
223
 
200
- Tested with Pi 1.3.0 on Linux. The extension API is version-locked research surface: the
201
- `ToolDefinition` shape and the settings `packages` key are verified at the locked version
202
- (see [the per-host reference](../research/control-matrix.md#pi)).
224
+ Tested with Pi 1.3.0 on Linux. The extension API surface this package relies on — the
225
+ `ToolDefinition` shape and the settings `packages` key — is verified at the locked
226
+ version (see [the per-host reference](../research/control-matrix.md#pi)).
203
227
 
204
228
  ## Uninstall
205
229
 
@@ -13,7 +13,7 @@ guides: [Codex](agents/codex.md), [Claude Code](agents/claude-code.md),
13
13
 
14
14
  Every knob resolves through one chain, low → high, identically on every host:
15
15
 
16
- ```
16
+ ```text
17
17
  built-in defaults < shared config file < DEEPSEEK_* environment < host-native config
18
18
  ```
19
19
 
@@ -26,7 +26,7 @@ variable arrives, not a fifth layer).
26
26
  ## The shared config file
27
27
 
28
28
  `$XDG_CONFIG_HOME/deepseek-web-retrieval/config.json` when `XDG_CONFIG_HOME` is
29
- set (this wins on every platform, Windows included); otherwise
29
+ set to a non-empty value (this wins on every platform, Windows included); otherwise
30
30
  `~/.config/deepseek-web-retrieval/config.json` on Linux and macOS and
31
31
  `%APPDATA%\deepseek-web-retrieval\config.json` on Windows. camelCase keys:
32
32
 
@@ -49,7 +49,7 @@ environment, or OpenCode's stored credential).
49
49
  | Knob class | Shared file | Examples |
50
50
  | --- | --- | --- |
51
51
  | API behavior — the user↔DeepSeek relationship | yes — that is the point of the file | `count`, `blockedDomains`, `language`, `maxRetries` |
52
- | Host-sensitive — interacts with host output caps and deadlines | carries the safe defaults; a host layer is the recommended override point | `searchTimeoutMs`, `fetchTimeoutMs`, `fetchMaxChars`, `maxSummaryChars` |
52
+ | Host-sensitive — interacts with host output caps and deadlines | carries the safe defaults; a host layer is the recommended override point | `searchTimeoutMs`, `fetchTimeoutMs`, `fetchMaxChars`, `maxSummaryChars`, `searchContentMaxChars`, `searchContentTimeoutMs` |
53
53
  | Host integration — the host's tool landscape | rejected — warned about and ignored | `searchEnabled`, `setDefaultWebsearch`, `apiKeySource` |
54
54
 
55
55
  Keep host-independent intent in the shared file; prefer a host layer when a
@@ -58,8 +58,8 @@ host-sensitive knob would cross its host's cap or deadline.
58
58
  ## The knob vocabulary
59
59
 
60
60
  One knob, two names: the camelCase key for the shared file and OpenCode's
61
- options, the `DEEPSEEK_*` variable for the environment. The table generates
62
- from the same config vocabulary the adapters resolve through, so it cannot
61
+ options, the `DEEPSEEK_*` variable for the environment. The table is
62
+ generated from the same config vocabulary the adapters resolve through, so it cannot
63
63
  drift from the code (`bun run docs:matrix` regenerates it; CI checks it).
64
64
 
65
65
  <!-- generated: knob-table — edit src/core/config-summary.ts or env.ts, then run `bun run docs:matrix` -->
@@ -76,6 +76,8 @@ drift from the code (`bun run docs:matrix` regenerates it; CI checks it).
76
76
  | `language` | `DEEPSEEK_SEARCH_LANGUAGE` | unset |
77
77
  | `searchLocalization` | `DEEPSEEK_SEARCH_LOCALIZATION` | unset |
78
78
  | `maxSummaryChars` | `DEEPSEEK_SEARCH_MAX_SUMMARY_CHARS` | 1200 |
79
+ | `searchContentMaxChars` | `DEEPSEEK_SEARCH_CONTENT_MAX_CHARS` | 1500 |
80
+ | `searchContentTimeoutMs` | `DEEPSEEK_SEARCH_CONTENT_TIMEOUT_MS` | 60000 |
79
81
  | `searchTimeoutMs` | `DEEPSEEK_SEARCH_TIMEOUT_MS` | 30000 |
80
82
  | `fetchMaxChars` | `DEEPSEEK_FETCH_MAX_CHARS` | 16000 |
81
83
  | `fetchAllowedDomains` | `DEEPSEEK_FETCH_ALLOWED_DOMAINS` | unset |
@@ -97,8 +99,9 @@ drift from the code (`bun run docs:matrix` regenerates it; CI checks it).
97
99
  `fetchBlockPrivateHosts`, `searchEnabled`, and `fetchEnabled` are on by
98
100
  default; the environment can only turn them off (`false`, `0`, `no`, or
99
101
  `off`). The last two have no file key — they are host-integration keys — so
100
- they exist only as `DEEPSEEK_ENABLE_SEARCH` and `DEEPSEEK_ENABLE_FETCH`. A
101
- value like `=true` changes nothing.
102
+ they exist only as `DEEPSEEK_ENABLE_SEARCH` and `DEEPSEEK_ENABLE_FETCH`.
103
+ Setting any of them to `true` explicitly changes nothing: the default is
104
+ already on.
102
105
  - **Invalid environment values are dropped** at startup and the default
103
106
  applies. `npx deepseek-web-retrieval config` reports every dropped or
104
107
  unknown `DEEPSEEK_*` variable as a warning and exits 1, alongside file
@@ -106,10 +109,17 @@ drift from the code (`bun run docs:matrix` regenerates it; CI checks it).
106
109
  - **`searchLocalization` must be a valid IPv4 or IPv6 address**; anything else
107
110
  is dropped at option resolution. It is deployment-level configuration, never
108
111
  a per-call tool parameter.
112
+ - **The content envelope is deployment-level too.** `searchContentMaxChars` and
113
+ `searchContentTimeoutMs` apply only when a call asks for inline page content
114
+ (`need_content: true`, a per-call tool parameter on the MCP server and the Pi
115
+ extension; OpenCode's provider input has no channel for it, so search stays
116
+ summary-only there). They are never tool parameters — the model cannot see
117
+ which host caps its output. `searchContentMaxChars: 0` drops page bodies
118
+ entirely and the summary digest renders.
109
119
  - **`baseUrl` routes the retrieval API** (`/v1/tools/search` and
110
120
  `/v1/tools/fetch` are composed from it). It is infrastructure routing for
111
- proxies, gateways, and mock upstreams — not a provider switch, so nothing
112
- falls back to another search service. Only an http(s) URL survives
121
+ proxies, gateways, and mock upstreams — not a provider switch, and the package
122
+ never falls back to another search service. Only an http(s) URL survives
113
123
  resolution (trailing slashes are stripped); anything else drops back to
114
124
  `https://api.deepseek.com`.
115
125
  - **Domains** in the environment are comma-separated; entries that are not
@@ -46,7 +46,7 @@ shared base strengthens rather than weakens them (one policy instead of three dr
46
46
 
47
47
  Precedence, low → high, identical semantics in every adapter:
48
48
 
49
- ```
49
+ ```text
50
50
  built-in defaults < shared config file < DEEPSEEK_* environment < host-native config
51
51
  ```
52
52
 
@@ -133,9 +133,13 @@ A bad shared value now fails everywhere at once, so effective configuration must
133
133
 
134
134
  ## What this design does not change
135
135
 
136
- - `need_content` stays hardcoded `false` ([search-inline-content.md](../research/search-inline-content.md)).
137
- - `ori_question` remains per-call-only (`original_question` input / Pi session fallback); it
138
- is not configurable.
136
+ - `need_content` is no longer hardcoded `false` — it became a per-call tool parameter
137
+ with a deployment-owned envelope
138
+ ([search-content-design.md](search-content-design.md),
139
+ [ADR-0003](../adr/0003-need-content-per-call-boolean.md)).
140
+ - `ori_question` remains per-call-only — supplied through the tool input named
141
+ `original_question` (with the Pi session fallback) and sent to the API as `ori_question`;
142
+ it is not configurable.
139
143
  - Auth surfaces, capability tiers, and the installer's plan/confirm flow.
140
144
 
141
145
  ## Testing
@@ -154,7 +158,8 @@ existing layers changes. Documentation updates ship with the code: README gains
154
158
  "Configuration" section with the precedence table and the two-vocabulary table; the host
155
159
  guides' option tables gain a shared-file column; the control-matrix note's Pi sentence
156
160
  ("There is no per-extension options channel: the runtime config surface is the
157
- `DEEPSEEK_*` environment") is updated to name the file as part of that surface.
161
+ `DEEPSEEK_*` environment") is updated to name the file as part of that surface. All of
162
+ these updates are shipped and in place.
158
163
 
159
164
  **Amendment, 2026-10-23** — two changes approved after this design was confirmed alter
160
165
  the effective behavior of *untouched* configurations, superseding the "no existing
@@ -43,8 +43,9 @@ already exists, and the installer uses three phases:
43
43
  1. **Detect.** Read the host config, the rules files, and the registered MCP servers.
44
44
  Report the conflict surface: existing rules steering toward another web tool; other
45
45
  web-capable MCP servers already on the field; array-valued keys the plan must merge
46
- into rather than replace (`disabledTools`, `permissions.deny`, permissions
47
- `exclude`, `disabled_tools`); and higher layers (Claude Code managed policy — its
46
+ into rather than replace (Cline's `disabledTools`, Claude Code's `permissions.deny`,
47
+ permissions `exclude` arrays, Codex's and Devin's `disabled_tools`); and higher
48
+ layers (Claude Code managed policy — its
48
49
  existence is checked by the claude-code module; Devin enterprise team settings) that
49
50
  would outrank what the installer writes.
50
51
  2. **Merge or snapshot.** Array keys merge with existing entries, recording exactly
@@ -69,22 +70,24 @@ already exists, and the installer uses three phases:
69
70
  conditions; it does not resolve them.
70
71
  - **Forced is conditional**: the exclusion kill switches remove the built-in
71
72
  competitor, not third-party web MCP servers — those must be excluded too (the
72
- per-host keys cover them), or routing is not actually forced. Concretely: Pi's
73
- per-server `enabled` key lets the plan disable detected web-capable competitors
74
- (each prior value snapshotted for uninstall); Claude Code excludes them the same way
75
- via server-level `mcp__<server>` deny rules appended to `permissions.deny`; Codex has
76
- no per-server kill switch,
77
- so a detected web-capable competitor makes the plan refuse the tier instead of
78
- promising non-exclusive routing; OpenCode's forced is replacement, which no
79
- third-party server overrides. Detection classifies a server web-capable from its
80
- name plus its command/args/url — a heuristic the plan notes never hide. Forced also
81
- overrides
82
- explicit user choices (OpenCode's `websearch.provider`), and its deny rules affect
83
- workflows beyond this project (a Claude Code deny cannot be carved out by allow
84
- rules), so the plan must state the blast radius before the user approves it.
73
+ per-host keys cover them), or routing is not actually forced.
74
+ - Per-host exclusion of detected web-capable MCP servers: Pi's per-server
75
+ `enabled` key lets the plan disable them (each prior value snapshotted for
76
+ uninstall); Claude Code excludes them the same way via server-level
77
+ `mcp__<server>` deny rules appended to `permissions.deny`; Codex has no
78
+ per-server kill switch, so a detected web-capable competitor makes the plan
79
+ refuse the tier instead of promising non-exclusive routing; OpenCode's forced
80
+ is replacement, which no third-party server overrides.
81
+ - Detection classifies a server web-capable from its name plus its
82
+ command/args/url — a heuristic the plan always discloses.
83
+ - Blast radius: forced also overrides explicit user choices (OpenCode's
84
+ `websearch.provider`), and its deny rules affect workflows beyond this
85
+ project (a Claude Code deny cannot be carved out by allow rules), so the plan
86
+ must state the blast radius before the user approves it.
85
87
  - **Forced is not universally available**: where hosts.jsonc records `forced: "none"`
86
88
  (Cursor's search today), the wizard does not offer the tier — the tier-level analogue
87
- of the scope rule: honor or report, never offer what cannot be delivered. Some kill
89
+ of the scope rule below (honor or report, never silently downgrade: never offer what
90
+ cannot be delivered). Some kill
88
91
  switches are session-scoped (Copilot fetch's `--deny-tool`/`--excluded-tools` flags);
89
92
  an installer cannot persist them, so the plan reports that the exclusion evaporates on
90
93
  restart and offers what persistence exists.
@@ -158,9 +161,33 @@ the artifacts below name the path where they differ.
158
161
  | Optional | Host config registering the integration: Pi's extension path appends an `npm:<package>` source to `packages` in settings.json; the MCP paths register the server (Pi's MCP entry needs `"exposure": "direct"`; the extension path needs nothing — extension tools are `direct` by Pi's own default) |
159
162
  | Preferred | The above plus a rules file for the host (file names per host in the [per-host reference](../research/control-matrix.md#per-host-reference)); the rules name the path's tool ids (`deepseek_web_search` for the extension, `mcp__deepseek__deepseek_web_search` for MCP) |
160
163
  | Forced (replacement) | OpenCode plugin options (`setDefaultWebsearch` / `fetchOverrideWebfetch`) |
161
- | Forced (exclusion) | The built-in competitor's kill switch (per-host keys in the per-host reference), plus the exclusion of any other web-capable MCP servers the detect phase found — Pi sets their `enabled` key to `false`; Claude Code appends server-level `mcp__<server>` deny rules to `permissions.deny` (merging with the user's own entries); Codex, lacking a per-server kill switch, refuses the tier instead; Pi's extension path additionally refuses when a web-capable `packages`/`extensions` source is listed in settings.json (no per-package kill switch exists) |
164
+ | Forced (exclusion) | The built-in competitor's kill switch (per-host keys in the per-host reference), plus exclusion of any other web-capable MCP servers the detect phase found — host by host below |
162
165
  | Forced (de-facto) | Nothing beyond the Optional artifacts — no competitor exists |
163
- | Off | Codex writes the capability's raw tool name into the server entry's `disabled_tools` (host-native: the tool leaves the model's context; the user's own entries in that array are kept); Claude Code appends the capability's full tool id (`mcp__deepseek__*`) to `permissions.deny`, the same host-native removal; the other MCP entry writes the capability's false-only server-side switch instead (`DEEPSEEK_ENABLE_SEARCH` / `DEEPSEEK_ENABLE_FETCH`); Pi's extension path writes the same switches into the `deepseekWebRetrieval` settings namespace the extension reads at load; both capabilities off refuses to install |
166
+ | Off | A host-native removal of the capability's tool — host by host below |
167
+
168
+ Forced (exclusion), host by host:
169
+
170
+ - **Pi** sets each detected web-capable server's `enabled` key to `false`.
171
+ - **Claude Code** appends server-level `mcp__<server>` deny rules to
172
+ `permissions.deny`, merging with the user's own entries.
173
+ - **Codex**, lacking a per-server kill switch, refuses the tier instead.
174
+ - **Pi's extension path** additionally refuses when a web-capable
175
+ `packages`/`extensions` source is listed in settings.json (no per-package kill
176
+ switch exists).
177
+
178
+ Off, host by host:
179
+
180
+ - **Codex** writes the capability's raw tool name into the server entry's
181
+ `disabled_tools` (host-native: the tool leaves the model's context; the user's
182
+ own entries in that array are kept).
183
+ - **Claude Code** appends the capability's full tool id (`mcp__deepseek__*`) to
184
+ `permissions.deny` — the same host-native removal.
185
+ - **The remaining MCP entry** (Pi's fallback) writes the capability's false-only
186
+ server-side switch instead (`DEEPSEEK_ENABLE_SEARCH` / `DEEPSEEK_ENABLE_FETCH`).
187
+ - **Pi's extension path** writes the same switches into the
188
+ `deepseekWebRetrieval` settings namespace the extension reads at load.
189
+
190
+ Both capabilities off refuses to install.
164
191
 
165
192
  OpenCode additionally appends `permissions` allow entries for exactly the tools it
166
193
  registers (`websearch`, `deepseek_fetch`, and `webfetch` when fetch is forced), merged
@@ -256,8 +283,13 @@ edits append arrays. Preview generation is read-only and does not reserve locks.
256
283
 
257
284
  Operation locks are directories ending in `.deepseek-web-retrieval.lock`, with
258
285
  owner PID and start time inside. Ordinary success or failure releases them. A
259
- killed process may leave a lock: after confirming no installer is running, inspect
260
- config, install record, and persistent ownership claims, restore any inconsistent
261
- state, then remove the reported lock directory and generate a new plan. Locks are
286
+ killed process may leave a lock; to recover from one:
287
+
288
+ 1. Confirm no installer is currently running.
289
+ 2. Inspect the config, install record, and persistent ownership claims.
290
+ 3. Restore any inconsistent state.
291
+ 4. Remove the reported lock directory and generate a new plan.
292
+
293
+ Locks are
262
294
  never stolen based only on age or PID. Persistent `.owner.json` claims remain a
263
295
  separate safeguard against two installations owning one shared config.
@@ -0,0 +1,133 @@
1
+ # Search content design
2
+
3
+ Design note for `need_content` — DeepSeek's inline-content search flag — across the two
4
+ model-facing adapters (the MCP server and the Pi extension). It records the parameter
5
+ split (who decides what), the envelope arithmetic that sets the defaults, the rendering
6
+ contract, and the per-adapter composition, so the design survives between conversations.
7
+ The facts it builds on are in [search-inline-content.md](../research/search-inline-content.md)
8
+ (baseline 2026-10-23); the decision record is
9
+ [ADR-0003](../adr/0003-need-content-per-call-boolean.md). This note was verified against
10
+ the code on 2026-10-24.
11
+
12
+ ## Problem
13
+
14
+ The wire parameter is a boolean the repo hardcoded `false`: search results carried a
15
+ summary digest (`summary` falling back to `snippet`) and never a page body. An agent
16
+ needing bodies had to guess which URLs to fetch from the digests — a wrong guess is a
17
+ silent miss. Turning the flag on naively is not safe either: DeepSeek documents **no**
18
+ size bound for the returned `content`, host output caps run as tight as Pi's 20 KB
19
+ MCP-path truncation (middle truncation — whole results vanish), and the 30 s default
20
+ search timeout does not budget for the server-side page crawl the flag triggers.
21
+
22
+ ## The parameter split
23
+
24
+ Two axes, two owners:
25
+
26
+ - **The agent owns the boolean.** `need_content` is a per-call, optional tool parameter
27
+ on every surface where this package defines the search tool's inputs (the MCP server
28
+ and the Pi extension), defaulting to `false`. It rides `SearchRequest.needContent`
29
+ (the `oriQuestion` pattern), not the resolved options — it changes nothing in option
30
+ state, only what that one call sends. Task type is the biggest determinant of whether
31
+ content helps, and only the model knows the task; the repo's other per-call parameters
32
+ (`count`, fetch `maxChars`) are the precedent: always exposed, never mode-gated.
33
+ - **The deployment owns the envelope.** Two deployment-level knobs — never tool
34
+ parameters — apply whenever content flows: `searchContentMaxChars` (per-result body
35
+ budget) and `searchContentTimeoutMs` (the timeout a content call runs on, replacing
36
+ `searchTimeoutMs` for that call; the per-call `timeoutMs` cap still wins if shorter).
37
+ The model cannot see which host caps its output, so it cannot own a budget.
38
+
39
+ ## The envelope
40
+
41
+ Defaults from the host-cap arithmetic in the research note:
42
+
43
+ - `searchContentMaxChars` defaults to **1,500**. At the default `count=10`, bodies plus
44
+ the ~1k chars of per-result metadata land near 16 KB — under the tightest published
45
+ cap (Pi's 20 KB MCP-path truncation) for Latin-script pages. CJK-heavy responses
46
+ measure 3 bytes per character toward that cap, so a CJK-focused deployment on a
47
+ 20 KB host should lower the budget or the count (at 1,500 chars/result, count ≈ 13 is
48
+ the 20 KB ceiling even for Latin text). The native Pi extension bypasses the MCP
49
+ layer entirely — neither its 20 KB truncation nor its 60 s timeout applies — so that
50
+ deployment can afford a larger budget.
51
+ - `searchContentTimeoutMs` defaults to **60,000** — between the 30 s search default the
52
+ digest path keeps and the 300 s ceiling Codex allows, because the API must fetch
53
+ every result page server-side before responding and documents no latency statement.
54
+
55
+ The total a response can carry is budget × count plus metadata. That product is the
56
+ deployment's arithmetic to run when tuning either knob; the core does not narrow any
57
+ parameter's range at runtime (see Decisions).
58
+
59
+ ## Per-adapter composition
60
+
61
+ - **MCP server**: `need_content` in the zod input schema, forwarded as
62
+ `needContent: need_content === true`.
63
+ - **Pi extension**: the same parameter in the TypeBox schema (`SEARCH_PARAMETERS` stays
64
+ a module-level constant on both surfaces — the schema is unconditional), forwarded the
65
+ same way.
66
+ - **OpenCode**: no change. The host fixes the `websearch` provider input to `query`
67
+ (the upstream gap that already blocks per-call `count`), so the provider stays
68
+ summary-only; the envelope knobs resolve through the shared pipeline and are dormant
69
+ there. Documented next to the count gap in
70
+ [opencode-v2.md](../agents/opencode-v2.md).
71
+ - **Installer**: no artifacts change. The knob defaults are global and the deviation
72
+ lines on every adapter already report non-default values through the config
73
+ vocabulary.
74
+
75
+ ## Processing and rendering
76
+
77
+ `processResults` is mode-agnostic: a present, non-empty `content` **replaces** the
78
+ digest (the summary is a proxy for the body it compresses), clamped to
79
+ `searchContentMaxChars` with an ellipsis marker when shortened; an empty, whitespace,
80
+ or missing body — the API documents per-result absence, and `need_content: false`
81
+ responses carry an empty string — falls back to the digest path. `searchContentMaxChars:
82
+ 0` treats the body as absent. Each result carries `contentKind: "page" | "digest"` so
83
+ the renderer knows the source: page bodies render as Markdown indented into the list
84
+ item (heading and list structure survive); digests stay compacted to one line, as
85
+ summaries always were. The `structuredContent` mirror carries the same `content`
86
+ string on hosts that read it.
87
+
88
+ ## What this design does not change
89
+
90
+ - Billing behavior: Search bills per successful request; `need_content` is not a price
91
+ modifier, so a content call costs the same as a digest call (and can replace fetch
92
+ calls at the same per-request price).
93
+ - The fetch tool, its `maxChars` parameter, and the two-tool architecture — a content
94
+ call replaces *some* fetches, not the deep-read path.
95
+ - `maxSummaryChars`, thresholds, dedup, query composition, and degradation.
96
+ - `MAX_RESPONSE_BYTES` (5 MiB) — still the outer guard, still erroring rather than
97
+ truncating; the per-result budget keeps responses far below it.
98
+
99
+ ## Testing
100
+
101
+ Wire assertions on both model-facing adapters (`need_content: true` reaching the wire;
102
+ `false` by default), client tests for the flag and the timeout selection
103
+ (`contentTimeoutMs` used only when the call asks for content), `processResults` tests
104
+ for the body-over-digest precedence, the budget clamp and marker, the empty-body
105
+ fallback, and the zero-budget drop, and format tests for the two rendering shapes.
106
+
107
+ ## Rollout
108
+
109
+ Shipped behind default-`false` behavior: with no configuration and no parameter use,
110
+ every surface behaves exactly as before (the wire still says `need_content: false`).
111
+ Deployments opt in per call by the model, per host by the two envelope knobs. No
112
+ installer or shared-file migration; the knobs join the config vocabulary and the
113
+ generated knob table.
114
+
115
+ ## Decisions (confirmed 2026-10-24)
116
+
117
+ 1. The boolean is a per-call tool parameter, always exposed on the MCP server and the
118
+ Pi extension; no config mode gates it.
119
+ 2. The envelope is deployment-only: `searchContentMaxChars` (1,500) and
120
+ `searchContentTimeoutMs` (60,000), never tool parameters.
121
+ 3. A page body replaces the digest; empty bodies fall back to it.
122
+ 4. `need_content` does not control other parameters — neither their availability nor
123
+ their value ranges. Effects may be conditional (the envelope knobs apply only when
124
+ content flows); domains are static. No surveyed vendor couples a content flag to
125
+ another parameter's range ([search-inline-content.md](../research/search-inline-content.md)),
126
+ and a silent runtime narrowing (say, of `count`) would be a decision the model
127
+ cannot see. The budget × count arithmetic is the deployment's to run.
128
+ 5. OpenCode stays summary-only until the upstream provider input widens.
129
+ 6. The descriptions state facts — mechanics, latency, price, magnitude, the fetch
130
+ alternative — and no usage policy. When content is worth it is the model's per-call
131
+ judgment (or a user's rules file per deployment), not the tool author's; if usage
132
+ data later shows the parameter systematically ignored or overused, that is the point
133
+ to add minimal steering, decided by evidence.
@@ -7,7 +7,8 @@ use it depends on that agent's extension points.
7
7
  CLI/terminal agents only; IDE and editor extension surfaces are out of scope. Versions are
8
8
  the latest stable CLI release checked at that baseline unless noted otherwise. `No` means no
9
9
  documented native provider or override was found in the checked version; several hosts still
10
- ship built-in web tools. MCP tools can add equivalent tools, but do not replace built-ins.
10
+ ship built-in web tools. MCP (Model Context Protocol) tools can add equivalent tools, but
11
+ do not replace built-ins.
11
12
 
12
13
  ## Mechanisms
13
14
 
@@ -20,8 +21,12 @@ ship built-in web tools. MCP tools can add equivalent tools, but do not replace
20
21
  | MCP server | Yes | Yes | Host MCP client | Shipped (`src/mcp/`; Pi, Codex, and Claude Code guides) |
21
22
  | CLI/HTTP wrapper | Yes | Yes | Host can call external tools | Not shipped |
22
23
 
23
- MCP is a protocol, not an agent. It is one possible transport for hosts that support MCP
24
- tools. `Custom tool plugin` means adding tools; it does not replace a host's built-in tools.
24
+ Why the mechanism distinction matters: some agents expose a native websearch provider,
25
+ some only let you add tools, and some only support MCP. A "yes" for one mechanism does
26
+ not mean the others are supported — the adapter code and docs need to match the
27
+ mechanism the host actually provides. MCP is a protocol, not an agent: one possible
28
+ transport for hosts that support MCP tools, never a host itself. `Custom tool plugin`
29
+ means adding tools; it does not replace a host's built-ins.
25
30
 
26
31
  ## Host support
27
32
 
@@ -96,12 +101,13 @@ For an installation managed by `init`:
96
101
  needed for uninstall. A global install can still claim shared configuration from a
97
102
  particular project, so keep the original working directory for upgrades.
98
103
  3. Refresh the package in the host and start or reload a session as described in its
99
- guide. For Pi's extension path, use `pi update --extensions`. To pin the host's
100
- package to the exact version being run, pass `--pin` to init (it writes
101
- `deepseek-web-retrieval@X.Y.Z` into the registration); upgrading a pinned install
102
- means re-running init with the newer CLI, since updating the CLI alone does not
103
- move an independently pinned host package. `--package` still overrides the ref
104
- entirely.
104
+ guide (for Pi's extension path, `pi update --extensions`).
105
+
106
+ Two init flags change what gets registered. `--pin` pins the host's package to the exact
107
+ version being run (`deepseek-web-retrieval@X.Y.Z` in the registration); upgrading a
108
+ pinned install therefore means re-running init with the newer CLI, since updating the
109
+ CLI alone does not move an independently pinned host package. `--package` overrides the
110
+ ref entirely.
105
111
 
106
112
  When upgrading from 0.4.0, Pi's default changes from MCP to the native extension.
107
113
  Pass `--path mcp` explicitly to retain MCP, or `--path extension` to migrate; the plan
@@ -109,12 +115,15 @@ shows the old path's removals and retains the original uninstall snapshots. Open
109
115
  native plugin and Codex's MCP path remain the same. Standalone MCP installs no longer
110
116
  require OpenCode or Pi SDKs; native adapters use the SDKs provided by their hosts.
111
117
 
112
- Keep install records and ownership sidecars: deleting them loses recovery information
118
+ Keep install records and ownership sidecars (the `.owner.json` claim files the installer
119
+ writes beside shared config files): deleting them loses recovery information
113
120
  and can cause a later init to refuse an existing registration. If an operation was
114
121
  interrupted, follow the [recovery guidance](design/installer-design.md#persistence-and-concurrent-operations)
115
122
  before retrying. For manual installations, update the package reference or checkout and
116
123
  follow the host guide rather than adopting the config through init.
117
124
 
125
+ ### Breaking changes for library consumers
126
+
118
127
  Code importing the core's removed `formatFetchContent` helper must migrate to
119
128
  `formatFetchedContent`, which returns both `content` and `truncated`.
120
129
 
@@ -178,12 +187,6 @@ the README Usage section only after the adapter ships. To request evaluation for
178
187
  host, [open an issue](https://github.com/gwokhou/deepseek-web-retrieval/issues) with links
179
188
  to its plugin, tool, or MCP documentation.
180
189
 
181
- ## Why the distinction matters
182
-
183
- Some agents expose a native websearch provider, some only let you add tools, and some only
184
- support MCP. A "yes" for one mechanism does not mean the others are supported. The adapter
185
- code and docs need to match the mechanism the host actually provides.
186
-
187
190
  ## Adding an adapter
188
191
 
189
192
  - Register the host and its version lock in [hosts.jsonc](hosts.jsonc), then regenerate