deepseek-web-retrieval 0.5.1 → 0.7.0

package/CHANGELOG.md

@@ -1,6 +1,26 @@
 # Changelog
 
-All notable changes to this project are documented here, with one sentence per item.
+All notable changes to this project are documented here, one sentence per item.
+
+## 0.7.0
+
+- Layer a shared config file (`$XDG_CONFIG_HOME/deepseek-web-retrieval/config.json`, camelCase keys) under the environment on every host, with the precedence `built-in defaults < shared file < DEEPSEEK_* environment < host-native config`; unknown and host-integration keys are warned about and ignored, and the API key stays out of the file.
+- Add a `config` command that prints every knob with its effective value and source (default, file, or environment — a count-derived `maxResults` is labeled `count`), replays shared-file problems, and verifies the key, config, and network end to end with `config --search "<query>"`.
+- Forward every `DEEPSEEK_*` option variable — not just the API key — in the Codex installer's `env_vars` entry, so shell-set knobs reach the stdio server without hand-editing `config.toml`.
+- Surface DeepSeek's `rerank_score`, `authority`, and `site_name` on search results in both text and structured output, accept an optional `original_question` on the MCP and Pi search tools (sent as `ori_question`), and add the `searchLocalization` deployment knob.
+- Couple `maxResults` to `count` when left unset — configuring only `count` was a silent no-op — and retry transient failures (408/425/429/5xx) twice by default instead of failing the billed call on the first attempt.
+- Resolve the shared config directory the Windows way on win32 (`%APPDATA%`, with an explicitly set `XDG_CONFIG_HOME` still winning), for the shared config file and the installer's global records alike.
+- Add `init --pin` to register the exact running CLI version (`deepseek-web-retrieval@<version>`) instead of the unpinned name, with package-identity comparisons that still match a pinned ref to its lineage.
+- `config` now warns and exits 1 when a `DEEPSEEK_*` variable is set but unusable (unknown name, non-numeric value, no-op switch value) instead of silently showing the default.
+
+## 0.6.0
+
+- Return distinct CLI exit codes — 2 for invalid command lines and 3 when a failed apply could not roll back — and document the codes in the usage text.
+- Add a `version` subcommand, record the installer version in install records, and show it in `list` output.
+- Route any command-line argument to the installer before the MCP server loads, so installer commands never depend on the server's dependency graph and a mistyped command no longer starts a server.
+- Reject duplicate flags instead of silently keeping the last value, and print usage to stderr on a bare invocation.
+- Wait up to five minutes for npm's post-upload processing before finalizing automated releases, with retry guidance if processing takes longer.
+- Clarify OpenCode and Pi native integration and Codex MCP support across setup, upgrade, security, release, and issue-reporting documentation.
 
 ## 0.5.1
 

package/README.md

@@ -1,50 +1,104 @@
 # DeepSeek Web Retrieval
 
-**Bringing [DeepSeek's in-house Search and Fetch APIs](https://api-docs.deepseek.com/guides/search) to coding agents.**
+**DeepSeek Search and Fetch for OpenCode, Pi, and Codex.**
 
-An adapter layer for coding agents. The core is agent-agnostic; each adapter exposes the
-same retrieval behavior to a specific agent. OpenCode v2 has a native plugin adapter, Pi a
-native extension adapter, and an MCP server covers every other MCP-capable host. Guides for
-other coding agents are in progress.
+Search the web and fetch pages as Markdown through
+[DeepSeek's APIs](https://api-docs.deepseek.com/guides/search), using a **native OpenCode
+plugin**, a **native Pi extension**, or **MCP for Codex**. All three share the same retrieval
+core, with results formatted for each host.
 
-This is an unofficial community project. It is not affiliated with OpenCode or DeepSeek.
+This is an unofficial community project, not affiliated with DeepSeek or the supported
+host projects.
 
 ## Features
 
-- **One core, every host** — a native OpenCode v2 plugin, a native Pi extension, plus a
-  stdio MCP server for everything else; identical retrieval behavior on each
-- **Tiered routing** — optional, preferred, forced, or off, per capability;
-  forced refuses when exclusivity is impossible
-- **Plan-then-apply installer** — every write previewed as old → new before
-  touching disk; arrays merge instead of replacing; uninstall reverses
-  exactly what was applied
-- **Trap-aware detection** — catches what the agents' own docs don't warn
-  about: Codex's silent `[tools] web_search` no-op, Pi's codemode hiding MCP
-  tools; conflicting rules and web-capable servers surface in the plan, and
-  `init` notes a Codex release that drifts from the tested lock
-- **Git-safe installs** — the key never enters config files
-  (`${DEEPSEEK_API_KEY}` indirection or `env_vars`), so a project-scope
-  install commits clean
-- **Fails loudly, never falls back** — no silent provider switches;
-  localhost and private URLs blocked before they leave the machine
+- **One package, three hosts** — a native OpenCode v2 plugin, a native Pi extension,
+  and a stdio MCP server for Codex, all sharing one retrieval core
+- **Routing control per tool** — search and fetch are set independently to optional,
+  preferred, forced, or off; the tier table in Usage explains each
+- **Review before anything is written** — `init` prints the full plan and writes only
+  with `--apply`; uninstall restores from snapshots while preserving edits you made
+  afterwards
+- **Knows the host's web landscape** — detects conflicting web tools and rules,
+  configures Pi's MCP fallback for direct tool exposure, and reports Codex version
+  drift
+- **Your key never enters a config file** — installs reference the environment
+  (`${DEEPSEEK_API_KEY}` indirection or `env_vars`), so a project-scope install
+  commits clean
+- **Fails loudly, never switches silently** — no fallback to another provider, and
+  localhost/private URLs are blocked before a request leaves the machine
 
 ## Usage
 
-Shipped adapters:
+Choose your host:
 
-- [OpenCode v2 setup and usage](docs/agents/opencode-v2.md) — native plugin API
-- [Pi setup and usage](docs/agents/pi.md) — native extension API (MCP fallback)
-- [Codex setup and usage](docs/agents/codex.md) — MCP server over stdio
+| Host | Default integration | Setup |
+| --- | --- | --- |
+| OpenCode v2 | Native plugin: search provider and Fetch tools | [OpenCode guide](docs/agents/opencode-v2.md) |
+| Pi | Native extension: Search and Fetch tools; MCP also available | [Pi guide](docs/agents/pi.md) |
+| Codex | MCP server over stdio, with readable text results | [Codex guide](docs/agents/codex.md) |
 
-The MCP server works with any host that supports MCP tools over stdio. The
-`deepseek-web-retrieval init` CLI installs it with plan-then-apply semantics — search and
-fetch tiers chosen per capability, project or global scope, snapshots for rollback. For
-how coding agents integrate—native websearch providers, native fetch tools, MCP, or
-wrappers—see the
-[integration model](docs/integration.md) with its version-locked host support matrix, the
-[control-level research](docs/research/control-matrix.md), and the
-[MCP compatibility research](docs/research/mcp-compatibility.md). A host is listed as supported
-only after an adapter and guide exist.
+The installer CLI requires Node.js 20+ and npm. Retrieval requires a DeepSeek API key
+with access to Search and Fetch and sufficient account balance. Set `DEEPSEEK_API_KEY`
+in the environment that launches your host; OpenCode can also use its stored DeepSeek key.
+
+Run the command for your host to preview a configuration:
+
+```bash
+# OpenCode: select DeepSeek as the search provider and prefer its Fetch tool.
+npx deepseek-web-retrieval init --host opencode --scope project --search forced --fetch preferred
+
+# Pi: use the native extension and prefer both tools.
+npx deepseek-web-retrieval init --host pi --scope project --search preferred --fetch preferred
+
+# Codex: register the MCP server and prefer both tools.
+npx deepseek-web-retrieval init --host codex --scope global --search preferred --fetch preferred
+```
+
+`--search` and `--fetch` each pick a tier — how strongly the host steers toward the
+DeepSeek tool, chosen independently per capability:
+
+| Tier | Effect |
+| --- | --- |
+| `optional` | The tool is declared; the model picks freely among its web tools |
+| `preferred` | Rules files steer the model toward the DeepSeek tool — probable, not guaranteed |
+| `forced` | Built-in competitors are disabled or replaced so routing is deterministic; the installer refuses rather than pretend when a host cannot guarantee it |
+| `off` | The capability is not installed |
+
+A preview run writes nothing. It prints the complete plan — every file it would touch
+and every old value → new value:
+
+```console
+$ npx deepseek-web-retrieval init --host codex --scope global --search preferred --fetch preferred
+Install plan: codex (global scope)
+Tiers: search=preferred, fetch=preferred
+
+Files:
+  ~/.codex/config.toml
+    set mcp_servers.deepseek = {"command":"npx","args":["-y","deepseek-web-retrieval"],"env_vars":["DEEPSEEK_API_KEY","DEEPSEEK_... (was (absent))
+  ~/.codex/AGENTS.md
+    upsert marked rules block
+
+Notes:
+  - Codex stdio children do not inherit the parent environment, so the entry forwards DEEPSEEK_API_KEY and every DEEPSEEK_* option variable explicitly via env_vars; shell-set knobs reach the server without hand-editing config.toml (the shared config file works without any forwarding).
+  - Rules block upserted in ~/.codex/AGENTS.md; Codex reads ~/.codex/AGENTS.md plus every AGENTS.md from the project root to the cwd.
+  - Shared config ownership is recorded in ~/.codex/.config.toml.deepseek-web-retrieval.codex.owner.json; uninstall releases it. Keep this local metadata with the config.
+
+Plan only — re-run with --apply to write these changes.
+```
+
+Repeat your chosen command with `--apply` to write the plan. The host guides explain
+activation steps and per-host tier limits. Pi defaults to the extension path; add
+`--path mcp` to use its MCP fallback. Add `--pin` to register the exact version of the
+CLI you are running (`deepseek-web-retrieval@<version>`) instead of the unpinned name,
+so hosts keep resolving that release rather than every latest publish — reproducible
+setups trade away auto-updates; re-run `init --pin` from a newer CLI to move the pin.
+
+Other stdio MCP hosts may use the server, but the supported setup guides and installer
+currently cover the three hosts above. See the [integration model](docs/integration.md)
+for tested versions and the [control tiers](docs/research/control-matrix.md) for routing
+limits. For an existing installation, follow the
+[upgrade instructions](docs/integration.md#upgrading-existing-installs).
 
 For shared search, fetch, retry, and privacy behavior, see below.
 
@@ -60,13 +114,19 @@ binary files, per-request billing) are documented there.
 - Requests summaries only (`need_content: false`, `summary` falling back to `snippet`);
   full page content never enters the results.
 - Returns at most ten results by default (matching the API's `count` default and the
-  industry norm for AI-native search APIs), deduplicating URLs. Optional
-  `rerank_score` and `authority` thresholds are off by default — the default
-  returns exactly what DeepSeek ranked, including results without a score;
-  when a positive threshold is set, results below it (or without a score to
-  compare) are dropped.
+  industry norm for AI-native search APIs), deduplicating URLs.
+- Optional `rerank_score` and `authority` thresholds are off by default — the default
+  returns exactly what DeepSeek ranked, including results without a score. A positive
+  `rerank_score` threshold drops results below it and results without a score; a positive
+  `authority` threshold drops known authorities below it, while results with an unknown
+  authority stay unless `allowUnknownAuthority` is set to false.
 - Builds `blockedDomains`, `includeDomains`, and `language` into query operators and
   rejects the final query locally when it exceeds the API's 128-character limit.
+- On the paths where this package defines the search tool's inputs (the MCP server and
+  the Pi extension), the tool accepts an optional `original_question` — the user's
+  question in their own words — forwarded as DeepSeek's `ori_question`, trimmed and
+  truncated to the API's 128 characters; a blank input on the Pi extension falls back
+  to the most recent user message in the session.
 - An empty result set is normal and is not an error.
 
 ### Fetch
@@ -83,9 +143,10 @@ binary files, per-request billing) are documented there.
 
 ### Timeouts and retries
 
-Request timeouts include reading the response body. Retryable responses are 408, 425, 429,
-and 5xx; `Retry-After` is honored up to 10 seconds. Other failures, including 401, timeouts,
-and network errors, fail without retrying.
+Request timeouts include reading the response body. Transient failures are retried twice
+by default (0–5, `maxRetries`); retryable responses are 408, 425, 429, and 5xx, and
+`Retry-After` is honored up to 10 seconds. Other failures, including 401, timeouts, and
+network errors, fail without retrying.
 
 ### No automatic fallback
 
@@ -99,6 +160,66 @@ Queries and URLs are sent to `https://api.deepseek.com`; the API key is sent onl
 Bearer token to DeepSeek and to nothing else. Retrieval is billed per successful request —
 see the official docs for pricing and rate limits before enabling a team deployment.
 
+## Configuration
+
+Every knob resolves through one chain, low → high, identically on every host:
+`built-in defaults < shared config file < DEEPSEEK_* environment < host-native config`
+— a higher layer always wins, so nothing already configured loses effect.
+
+The shared file — `$XDG_CONFIG_HOME/deepseek-web-retrieval/config.json`, default
+`~/.config/deepseek-web-retrieval/config.json` on Linux and macOS and
+`%APPDATA%\deepseek-web-retrieval\config.json` on Windows (an explicitly set
+`XDG_CONFIG_HOME` still wins there) — works on every host and uses the
+camelCase option keys:
+
+```json
+{
+  "count": 20,
+  "blockedDomains": ["pinterest.com"],
+  "language": "zh",
+  "searchLocalization": "203.0.113.7"
+}
+```
+
+Unknown keys and host-integration keys are warned about on stderr and ignored. The
+API key does not go in the file — it keeps its existing channels (`DEEPSEEK_API_KEY`
+in the environment, or OpenCode's stored credential).
+
+Knobs fall into three classes; the shared file carries the first two:
+
+| Knob class | Shared file | Examples |
+| --- | --- | --- |
+| API behavior — the user↔DeepSeek relationship | yes — that is the point of the file | `count`, `blockedDomains`, `language`, `maxRetries` |
+| Host-sensitive — interacts with host output caps and deadlines | carries the safe defaults; a host layer is the recommended override point | `searchTimeoutMs`, `fetchTimeoutMs`, `fetchMaxChars`, `maxSummaryChars` |
+| Host integration — the host's tool landscape | rejected — warned about and ignored | `searchEnabled`, `setDefaultWebsearch`, `apiKeySource` |
+
+The same knob has two names — the camelCase key for the shared file and OpenCode's
+options, the `DEEPSEEK_*` variable for the environment:
+
+| Shared file / OpenCode options | Environment |
+| --- | --- |
+| `count` | `DEEPSEEK_SEARCH_COUNT` |
+| `blockedDomains` | `DEEPSEEK_SEARCH_BLOCKED_DOMAINS` |
+| `language` | `DEEPSEEK_SEARCH_LANGUAGE` |
+| `searchLocalization` | `DEEPSEEK_SEARCH_LOCALIZATION` |
+| `fetchMaxChars` | `DEEPSEEK_FETCH_MAX_CHARS` |
+| `maxRetries` | `DEEPSEEK_MAX_RETRIES` |
+
+The full per-knob mapping is documented side by side in each guide's option table;
+per-host mechanics — how the environment reaches a stdio server, each host's output
+caps and deadlines — live there too: [Codex](docs/agents/codex.md),
+[Pi](docs/agents/pi.md), [OpenCode](docs/agents/opencode-v2.md). Keep
+host-independent intent in the shared file; prefer a host layer when a
+host-sensitive knob would cross its host's cap or deadline.
+
+To see what actually resolves, run `npx deepseek-web-retrieval config`: every knob
+printed with its value and its source (`default`, `file`, or `env`), plus any problems
+in the file or in `DEEPSEEK_*` variables (unknown names, unusable values) replayed as
+visible warnings — it exits 1 when there are problems, so it doubles as a dotfile CI
+check. `npx deepseek-web-retrieval config --search "<query>"`
+runs one billed search through the effective configuration to verify the key, config,
+and network end to end.
+
 ## Development
 
 ```bash
@@ -107,9 +228,10 @@ bun run check
 ```
 
 Use `bun run lint:fix` to apply Biome's safe lint fixes and formatting. See
-[CONTRIBUTING.md](CONTRIBUTING.md) for the development checks and e2e scenarios.
+[CONTRIBUTING.md](CONTRIBUTING.md) for development checks, e2e scenarios, and
+[automated tag releases](CONTRIBUTING.md#automated-releases).
 
-The test suite mocks the DeepSeek HTTP API and the OpenCode plugin host, and drives the
+The test suite mocks the DeepSeek HTTP API and the OpenCode and Pi host APIs, and drives the
 MCP server with SDK clients pinned to each supported protocol revision. Real API calls
 require a funded DeepSeek account. Shared retrieval logic lives in `src/core/`; adapters
 live in `src/<agent>/` (the cross-host MCP server lives in `src/mcp/`). Adapter docs live