dsh-exa-search 0.0.0-stage → 1.0.0

package/CHANGELOG.md

@@ -0,0 +1,43 @@
+# Changelog
+
+## 1.0.0
+
+First public release: Exa-backed `web_search` / `web_fetch`, ten additional tools, and three access modes.
+
+### Tools
+
+- `exa_search` — domain / category / date / text / subpage filters, `additionalQueries`, and the same arguments as Exa's advanced search.
+- `exa_deep_search` — Deep Search with a synthesized answer; a `text` output schema is sent by default so Exa actually produces one.
+- `exa_fetch` — up to five URLs, `format: text | highlights | summary`, optional subpages, three-way concurrency, and per-URL failure isolation.
+- `exa_find_similar`, `exa_answer` — REST-only helpers for expanding a known URL and citing an answer.
+- `exa_research` — Exa Agent with `run` / `cancel` / `stop`, resumable by `runId`.
+- `exa_monitor` — list / get / runs / create / update / trigger / delete, webhook events, manual-only monitors (`manual: true` or `period: "none"`).
+- `exa_webset`, `exa_batch` — registered only when the account/team is entitled (probed with free GETs).
+- `exa_status` — active mode, per-mode tool counts, upstream, entitlements, spend estimate.
+- Per-group tool switches in Settings, persisted across restarts.
+
+### Access modes
+
+- **Keyless** via Exa's public MCP endpoint (advanced search opted in; no `agent_run`).
+- **API key** via the full REST surface (search, contents, find similar, answer, agent, monitors, websets, batch).
+- **Exa account** via OAuth 2.1 + PKCE with dynamic client registration; the exposed MCP tools follow what the account actually offers.
+- Settings shows the live mode × tool matrix and can run a minimal "Test connection" search.
+
+### Reliability
+
+- Hard timeouts on every request, retries with backoff for 429/5xx, `Retry-After` support, and cancellation that interrupts immediately.
+- One forced token refresh on a 401; a revoked refresh token drops the grant and falls back to a saved key or keyless mode.
+- Credential records use the required `<scope>/<id>` shape, are resolved through an injection (late activation is safe), and are validated before every write.
+- A dead refresh token, an invalid saved key, or an unentitled team feature can no longer leave web search silently broken.
+- Process-level spend estimate from Exa's `costDollars`, plus a redacted diagnostics endpoint and "Copy diagnostics" button.
+
+### Privacy
+
+- API key and OAuth tokens live in the Harness credential service, never in the package.
+- Diagnostics exclude email, key fragments, tokens and file paths.
+- The published tarball contains no machine paths, credentials or account identifiers.
+
+### Notes
+
+- Early local builds stored credentials as `exa-api-key` / `exa-oauth`, which the credential service rejects at load time and can prevent Harness from starting. If you ran one, run `node scripts/repair-credentials.mjs --dry-run` and then `node scripts/repair-credentials.mjs` before starting Harness.
+- Tests (`node --test`), the CI workflow and the English README are in the repository; they are not part of the npm tarball.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 dsh-exa-search contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.en.md

@@ -0,0 +1,103 @@
+# Exa Search for DeepSeek Harness
+
+Exa-backed `web_search` / `web_fetch` for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness), with three access modes and per-mode tool registration.
+
+- The stock `web_search` / `web_fetch` tools keep their schemas but run on Exa.
+- Ten additional `exa_*` tools are registered **only when the active mode can call them**.
+- **Settings → Exa Search** switches modes, shows which tools each mode exposes, and can test the connection.
+
+## Tool catalog
+
+| Group | Tool | What it does | Keyless | API key | Exa account |
+| --- | --- | --- | --- | --- | --- |
+| Search | `web_search` | Harness web search, served by Exa | ✓ | ✓ | ✓ |
+| Search | `exa_search` | Domain / category / date / text / subpage filters | ✓ | ✓ | ✓ |
+| Search | `exa_deep_search` | Deep Search: multi-step retrieval with a cited answer | — | ✓ | — |
+| Contents | `web_fetch` | Harness page fetch, served by Exa | ✓ | ✓ | ✓ |
+| Contents | `exa_fetch` | Up to 5 pages, optional highlights / summary / subpages, 3-way concurrency | ✓* | ✓ | ✓* |
+| Contents | `exa_find_similar` | Expand a known-good URL into similar pages | — | ✓ | — |
+| Answer | `exa_answer` | `/answer`: cited answer to a question | — | ✓ | — |
+| Research | `exa_research` | Exa Agent: research, list building, structured output; resume / cancel / stop | — | ✓ | per account |
+| Monitors | `exa_monitor` | Recurring or manual searches; list / create / update / trigger / delete | — | ✓ | — |
+| Websets | `exa_webset` | Verified datasets (Pro plan required) | — | ✓* | — |
+| Batch | `exa_batch` | Submit many search / agent requests (team feature) | — | ✓* | — |
+| Status | `exa_status` | Active mode, tool counts, spend estimate | ✓ | ✓ | ✓ |
+
+\* `exa_fetch` highlights/summary/subpages need an API key. Websets needs a Pro plan; Batch must be enabled for the team. The plugin probes both with free GETs and simply does not register tools the team cannot use.
+
+## Access modes
+
+| Mode | Credential | Upstream | Notes |
+| --- | --- | --- | --- |
+| `keyless` | none | `mcp.exa.ai/mcp` | Free, rate-limited per IP. Advanced search is opted in through `?tools=` (Exa allows it unauthenticated); `agent_run` is not available. |
+| `apiKey` | saved key or `EXA_API_KEY` | `api.exa.ai` | Full REST surface: search, contents, find similar, answer, agent, monitors, websets, batch. |
+| `oauth` | browser sign-in | `mcp.exa.ai/mcp` with a bearer token | MCP tools follow the account; `agent_run` appears when the account exposes it. |
+
+The matrix is live, not marketing: the OAuth research row is resolved from the server's real `tools/list` after sign-in.
+
+## Implementation notes
+
+- **MCP tool list**: keyless sends `?tools=web_search_exa,web_fetch_exa,web_search_advanced_exa`; OAuth adds `agent_run` and retries without it if the connection is rejected.
+- **Rate limits**: keyless answers with `_meta.ai.exa/rateLimited` or a plain text notice; both are translated into a clear error instead of being returned as results.
+- **Deep Search** only synthesizes when `outputSchema` is present. The tool sends a `{ "type": "text" }` schema by default and returns structured JSON if you pass your own.
+- **Websets** lives on its own mount point: `api.exa.ai/websets/v0/websets…`.
+- **Credentials** are stored under `exa-search/api-key` / `exa-search/oauth` (the service requires `<scope>/<id>`), resolved through an injection, and writes are validated before touching the vault.
+- **Request policy**: hard timeouts (search 30s, MCP 60s, Agent 240s), retries with backoff for 429/5xx and `Retry-After`, and one forced token refresh on a 401.
+- **Spend estimate**: accumulated from Exa's `costDollars`; Agent runs are counted once at a terminal status. OAuth MCP search responses carry no cost field, so the estimate covers REST calls and Agent runs.
+
+## Install
+
+```bash
+dsh plugin add dsh-exa-search
+```
+
+Or install this directory from the plugin manager. A local directory install links the folder; moving or deleting it breaks the plugin.
+
+Installing pins the `web` row to the `exa` providers (`cordis.patch.yml`). That pin is required: with two usable search providers, `ctx.web` fails closed with `WEB_PROVIDER_AMBIGUOUS` instead of guessing.
+
+## Configuration
+
+| Key | Default | Meaning |
+| --- | --- | --- |
+| `mode` | `keyless` | First-run mode; the settings page overrides it. |
+| `apiKeyEnv` | `EXA_API_KEY` | Environment fallback when no key is saved. |
+| `numResults` | `8` | Default number of sources. |
+| `searchType` | `auto` | REST search effort: `auto` / `lite` / `fast`. |
+| `maxCharacters` | `12000` | Per-page fetch cap. |
+| `mcpTools` | `null` | Overrides the MCP `tools` list (array or comma string). |
+| `toolGroups` | `null` | Initial per-group tool switches, e.g. `{ "answer": false }`. |
+| `locale` | `auto` | Message language: `auto` / `en` / `zh`. |
+
+## Data
+
+- Mode and OAuth client registration: `<DSH_HOME>/exa-search/state.json`.
+- API key and OAuth token: the Harness credential service, under `exa-search/api-key` / `exa-search/oauth`. Only when the vault is unavailable do they fall back to `state.json` (mode 0600).
+- Nothing is written into the install directory, and no state ships with the package.
+
+### Repair after a failed start
+
+If Harness refuses to start with `credential key "..." must be "<scope>/<id>"` (written by a pre-release local build), quit the app and run:
+
+```bash
+node scripts/repair-credentials.mjs --dry-run
+node scripts/repair-credentials.mjs
+```
+
+The script only renames those two record keys, backs the file up first, and refuses to guess if both forms exist.
+
+## Routes
+
+The settings page uses same-origin routes instead of generated remotes:
+
+- `GET /exa-plugin/state` — mode, credentials (masked), upstream tools, matrix, counts;
+- `GET /exa-plugin/diagnostics` — redacted snapshot for bug reports;
+- `POST /exa-plugin/mode` / `api-key` / `login` / `logout` / `tools` / `test` — mutations, cross-origin requests rejected;
+- `GET /exa-plugin/oauth/callback` — OAuth callback.
+
+## Not covered
+
+Team management, webhooks/events, imports and Websets enrichments are account plumbing or plan-gated; they are not exposed as model tools.
+
+## Uninstall
+
+`dsh plugin remove dsh-exa-search`. The `web` row falls back to the shipped default.

package/README.md

@@ -1,3 +1,151 @@
-# Temporary Holding Version
+# Exa Search for DeepSeek Harness
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+> **English** — full README in [README.en.md](./README.en.md). Short version: Exa-backed `web_search` / `web_fetch` with three access modes (keyless, API key, OAuth) and per-mode tool registration; install `dsh-exa-search`, then open **Settings → Exa Search**.
+
+把 Harness 的联网搜索交给 [Exa](https://exa.ai)。安装后：
+
+- `web_search`、`web_fetch` 的 schema 不变，实际请求全部走 Exa；
+- 另外注册 10 个 Exa 工具，**按接入方式注册**：只有当前方式真能调用的工具才会出现在模型面前；
+- 「设置 → Exa 搜索」里可以切换接入方式，并按功能分组看到全部工具在三种方式下的可用性。
+
+## 工具清单（按功能分组）
+
+| 分组 | 工具 | 作用 | 免登录 | API Key | Exa 账号 |
+| --- | --- | --- | --- | --- | --- |
+| 检索 | `web_search` | Harness 原生搜索，已改走 Exa | ✓ | ✓ | ✓ |
+| 检索 | `exa_search` | 域名 / 分类 / 日期 / 文本 / 子页过滤 | ✓ | ✓ | ✓ |
+| 检索 | `exa_deep_search` | Deep Search：多步检索 + 带引用的综合答案 | — | ✓ | — |
+| 内容 | `web_fetch` | Harness 原生抓取，已改走 Exa | ✓ | ✓ | ✓ |
+| 内容 | `exa_fetch` | 抓取最多 5 个网页；可选取 highlights / summary / 子页（后三者需 API Key），并发 3 | ✓ | ✓ | ✓ |
+| 内容 | `exa_find_similar` | 由一篇已知好文扩展相似页面 | — | ✓ | — |
+| 问答 | `exa_answer` | `/answer`：带引用的直接问答 | — | ✓ | — |
+| 研究 | `exa_research` | Exa Agent：研究、建名单、结构化输出（续等 / 取消 / 停止） | — | ✓ | 登录后按账号 |
+| 监控 | `exa_monitor` | Monitors：定时或手动搜索，列表 / 新建 / 修改 / 触发 / 删除 | — | ✓ | — |
+| 数据集 | `exa_webset` | Websets：构建、读取经过验证的数据集 | — | ✓* | — |
+| 批量 | `exa_batch` | Batch：批量提交 `/search` 或 `/agent/runs` | — | ✓* | — |
+| 状态 | `exa_status` | 报告当前方式、可用工具与上游工具表 | ✓ | ✓ | ✓ |
+
+合计 12 个，其中 10 个由插件注册、2 个是 Harness 原生的改道。
+
+\* Websets 需要 Pro 计划，Batch 需要团队单独开通。启用 API Key 时插件会各发一次免费探测；团队没有的功能不会注册成工具，设置页注明「当前团队未开通」，而不是让模型调用后拿到 401/403。
+
+## 旧版 MCP 工具的去向
+
+Exa MCP 仓库的 `toolRegistry.ts` 里还有 8 个已弃用条目（默认关闭）。它们的能力已被现代工具覆盖，所以不重复注册：
+
+| 旧工具 | 现在用 |
+| --- | --- |
+| `get_code_context_exa` | `exa_search`，`category=github` |
+| `company_research_exa` | `exa_search`，`category=company` |
+| `people_search_exa` | `exa_search`，`category=people` |
+| `linkedin_search_exa` | `exa_search`，`category=linkedin profile` |
+| `deep_search_exa` | `exa_deep_search` |
+| `crawling_exa` | `exa_fetch` |
+| `deep_researcher_start` | `exa_research` |
+| `deep_researcher_check` | `exa_research`，传 `runId` |
+
+设置页底部也列出了这张对照表，避免“看起来少了一个工具”。
+
+## 三种接入方式
+
+| 方式 | 凭据 | 上游 | 能覆盖到哪 |
+| --- | --- | --- | --- |
+| 免登录 `keyless` | 不需要 | `https://mcp.exa.ai/mcp?tools=...` | 基础搜索、高级搜索、抓取。Exa 明确说明 `web_search_advanced_exa` 不需要认证，所以用 `tools` 参数把它打开；`agent_run` 需要认证，不能放进免登录连接（否则整个连接 401）。 |
+| API Key `apiKey` | 设置页填入，或 `EXA_API_KEY` | `https://api.exa.ai` | 全部：Search、Contents、Find Similar、Answer、Agent、Monitors、Websets。 |
+| Exa 账号 `oauth` | 浏览器授权 | `https://mcp.exa.ai/mcp?tools=...` | MCP 工具随账号开放；`agent_run` 在登录后探测，能探测到才注册 `exa_research`。 |
+
+工具矩阵里 `·` 表示“登录后才确定”，不是写死的宣传语：`exa_research` 在 OAuth 下是否出现，取决于登录后 `tools/list` 的真实返回。
+
+## 实现要点
+
+**MCP 的 tools 参数**。Exa 的托管 MCP 默认只给 `web_search_exa`、`web_fetch_exa`；`agent_run` 默认只对有凭据的连接开放。插件按模式拼 URL：
+
+- 免登录：`?tools=web_search_exa,web_fetch_exa,web_search_advanced_exa`
+- 账号：`?tools=...,agent_run`（探测失败自动去掉 `agent_run` 重试）
+
+**限流识别**。免登录命中限额时 Exa 返回的是 `_meta.ai.exa/rateLimited` 或一段“free MCP rate limit”文本，而不是 JSON-RPC error。插件会把它翻译成明确错误，提示去配置 Key 或登录，而不是把限流文案当成搜索结果。
+
+**Agent 续等**。`exa_research` 最多等 `timeoutSeconds`（默认 120，上限 600），超时返回 `status: "running"` 和 `runId`；再次调用并传入 `runId` 继续等。REST 走 `POST /agent/runs` + 轮询，OAuth 走 MCP 的 `agent_run`。
+
+**分类差异**。REST 接受 `research paper` / `linkedin profile` 这类分类名，MCP 高级搜索是严格枚举；插件把前者映射成 `publication` / `people`，映射不了的分类会被丢弃而不是原样发送。
+
+**Deep Search 必须带 outputSchema**。Exa 只在请求里有 `outputSchema` 时才做综合——不带的话 `type: deep` 也只返回 `results`，没有 `output`。插件默认补一个 `{ type: "text" }` schema 拿散文答案，调用方传自己的对象 schema 就返回结构化 JSON。
+
+**Websets 有自己的挂载点**。OpenAPI 里 `/v0/websets` 的 `servers` 是 `https://api.exa.ai/websets`，真实地址是 `/websets/v0/websets...`；少这个前缀会一直是 404。
+
+**功能开通探测**。Batch 未开通返回 403、Websets 未开通返回 401（Pro 计划）；两者都是免费 GET，插件在切换/启动时探测，没有开通就不注册对应工具。
+
+**抓取格式**。`exa_fetch` 的 `format` 支持 `text` / `highlights` / `summary`，API Key 模式下分别请求 Contents API 的 `text`、`highlights`、`summary` 对象；免登录/账号的 MCP 抓取只返回正文，请求其它格式会得到明确错误。多 URL 用 3 路并发，单个失败不影响其余页面（结果里带 `error`）。
+
+**监控更新**。`exa_monitor action=update` 支持改名、暂停/恢复（`status: active | paused`）、改查询和改周期；`manual: true` 或 `period: "none"` 创建不排期的监控，只能手动 `trigger`。
+
+**凭据读写**。凭据服务通过 `ctx.inject` 解析，即使它比插件晚激活也不会退回明文；写入前会校验记录键必须是 `<scope>/<id>`，避免出现“能写入、下次启动才炸”的坏记录。刷新令牌被吊销时（`invalid_grant`）会丢弃授权并回退到 API Key 或免登录，而不是让联网一直 401。
+
+**请求策略**。所有出站请求都带硬超时（搜索 30s、内容 30s、MCP 工具 60s、Agent 240s），429/5xx/网络失败按指数退避重试 2 次并尊重 `Retry-After`；取消信号会立刻中断等待。
+
+**OAuth 令牌失效**。MCP 返回 401 时不会直接把错误抛给用户：先强制刷新一次令牌并用新令牌重放，仍失败才报“需要重新登录”。
+
+**取消 Agent**。`exa_research` 的 `action: cancel | stop` 配合 `runId` 可中止运行（`cancel` 立即取消，`stop` 保留已收集结果），仅 API Key 模式可用——OAuth 的令牌只能走 MCP，没有取消入口。
+
+**费用估算**。Exa 在 REST 计费响应里返回 `costDollars.total`，插件按进程累计并在 `exa_status` 与设置页显示；Agent 运行只在到达终态时记一次，轮询不会重复计数。注意 OAuth 走 MCP 时搜索响应不带费用字段，所以账号模式的估算只覆盖 Agent 运行；进程重启后从 0 开始。
+
+## 安装
+
+在 Harness 的插件管理里安装本目录，或从 npm 安装发布后的包（`dsh-exa-search`）。
+以本地目录安装时，profile 里保存的是链接而不是拷贝——移动或删除源目录会让插件失效。
+
+安装后 `web` 行被 pin 到 `exa` provider（见 `cordis.patch.yml`）——这是必须的：同时存在两个可用搜索 provider 时，`ctx.web` 会以 `WEB_PROVIDER_AMBIGUOUS` 拒绝选择，而不是猜。
+
+## 配置
+
+`cordis.patch.yml` 里的行配置：
+
+| 键 | 默认 | 说明 |
+| --- | --- | --- |
+| `mode` | `keyless` | 首次运行的默认接入方式；设置页会覆盖它。 |
+| `apiKeyEnv` | `EXA_API_KEY` | 没有保存 Key 时读取的环境变量。 |
+| `numResults` | `8` | 默认返回来源数。 |
+| `searchType` | `auto` | 检索强度：`auto` / `lite` / `fast`。 |
+| `maxCharacters` | `12000` | 单页抓取字符上限。 |
+| `locale` | `auto` | 消息语言：`auto` 跟随系统，或强制 `en` / `zh`。 |
+| `mcpTools` | `null` | 覆盖 MCP 的 `tools` 列表（数组或逗号分隔字符串）。 |
+| `toolGroups` | `null` | 分组开关的初始值，例如 `{ "answer": false, "webset": false }`；设置页的改动存进 state 并覆盖它。 |
+
+设置页每个分组右侧有一个开关，关掉后该组的 `exa_*` 工具立刻从模型工具表里移除；`web_search` / `web_fetch` 的改道不受影响。设置页顶部还有「测试连接」按钮，会用当前方式跑一次最小搜索并返回耗时与示例结果（消耗一次搜索）。
+
+## 数据落在哪
+
+- 接入方式、OAuth 客户端注册信息：`<DSH_HOME>/exa-search/state.json`。
+  Host 进程不保证有 `DSH_PROFILE_DIR`，所以这份状态按用户存放，而不是按 profile。
+- API Key 与 OAuth 令牌：Harness 凭据服务（`ctx.credentials`），记录键为 `exa-search/api-key`、`exa-search/oauth`；
+  只有凭据服务不可用时才退回上面的 `state.json`（文件权限 0600）。
+- 插件不会把任何凭据写进安装目录，也不会随包分发状态文件。
+
+> **如果你装过 1.0.0 之前的本地开发版**：旧版把凭据写成 `exa-api-key` / `exa-oauth`，而 Harness 的凭据服务只接受 `<scope>/<id>`，会在下次启动时解析失败并让整个应用起不来。关闭应用后运行下面的修复脚本，或手动改名/删除这两条记录。
+
+## 启动崩溃后的修复
+
+如果 Harness 报 `credential key "..." must be "<scope>/<id>"` 而无法启动，先完全退出应用，再运行：
+
+```bash
+node scripts/repair-credentials.mjs --dry-run   # 预演，只打印会改什么
+node scripts/repair-credentials.mjs             # 备份并把两个旧键改成 exa-search/*
+```
+
+脚本只动这两条记录键，写入前在同目录留一份 `.bak-<时间戳>`；如果新旧键同时存在会拒绝猜测并退出。默认路径是 `<DSH_HOME>/.credentials.yaml`，可用 `DSH_HOME` 或 `DSH_CREDENTIALS` 覆盖，也可以 `npm run repair-credentials`。
+
+## 路由
+
+设置页通过同源路由读写 Host，避免为纯 JS 插件生成 Remote 命名空间：
+
+- `GET  /exa-plugin/state` — 当前方式、凭据状态、上游工具表、工具矩阵、各方式计数；
+- `POST /exa-plugin/mode` / `api-key` / `login` / `logout` — 变更操作，拒绝跨源请求；
+- `GET  /exa-plugin/oauth/callback` — OAuth 回调。
+
+## 没覆盖的部分
+
+Team Management、Webhooks/Events、Imports 属于账号与事件投递的底层接口，不注册为模型工具；检索、内容、问答、研究、监控、数据集、批量这条链路已经完整。
+
+## 卸载
+
+`dsh plugin remove dsh-exa-search`。卸载后 `web` 行恢复由下层 bundle 决定。