bookmark-research 0.5.0-beta.1 → 0.5.0-beta.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.codex-plugin/plugin.json +4 -2
- package/README.md +29 -22
- package/README.zh.md +29 -22
- package/docs/cli-launcher-plan.md +1 -1
- package/docs/details.en.md +6 -3
- package/docs/details.md +6 -4
- package/docs/installation.en.md +1 -1
- package/docs/installation.md +2 -2
- package/docs/knowledge-storage-plan.md +43 -0
- package/docs/release-notes-0.5.0-beta.2.md +23 -0
- package/docs/release-notes-0.5.0-beta.3.md +28 -0
- package/package.json +1 -1
- package/scripts/export_bundle.py +4 -0
- package/scripts/install.py +46 -2
- package/scripts/launcher.py +20 -1
- package/skills/bookmark-research/SKILL.md +5 -1
- package/skills/bookmark-research/references/cli.md +3 -1
- package/skills/bookmark-research/references/deep-research.md +3 -2
- package/skills/bookmark-research/references/settings-and-archive.md +12 -5
- package/skills/bookmark-research/references/wiki-and-evaluation.md +4 -0
- package/skills/bookmark-research/references/zh/cli.md +3 -1
- package/skills/bookmark-research/references/zh/deep-research.md +3 -2
- package/skills/bookmark-research/references/zh/settings-and-archive.md +12 -5
- package/skills/bookmark-research/references/zh/skill-guide.md +5 -1
- package/skills/bookmark-research/references/zh/wiki-and-evaluation.md +4 -0
- package/src/archive.py +86 -10
- package/src/cli.py +9 -1
- package/src/jina.py +1 -1
- package/src/mcp_server.py +25 -4
- package/src/onboarding.py +33 -1
- package/src/raw_library.py +388 -0
- package/src/remote_mcp.py +2 -2
- package/src/research.py +239 -19
- package/src/research_locations.py +38 -0
- package/src/settings.py +12 -2
- package/src/wiki.py +61 -1
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "bookmark-research",
|
|
3
|
-
"version": "0.5.0-beta.
|
|
3
|
+
"version": "0.5.0-beta.3",
|
|
4
4
|
"description": "Check bookmark URLs or Canvas packages, compare sources with host-led research, and retain verifiable reports.",
|
|
5
5
|
"author": {
|
|
6
6
|
"name": "kwenxu"
|
|
@@ -39,7 +39,9 @@
|
|
|
39
39
|
"PARALLEL_API_KEY",
|
|
40
40
|
"TAVILY_API_KEY",
|
|
41
41
|
"JINA_API_KEY",
|
|
42
|
-
"OPENAI_API_KEY"
|
|
42
|
+
"OPENAI_API_KEY",
|
|
43
|
+
"SystemRoot",
|
|
44
|
+
"windir"
|
|
43
45
|
]
|
|
44
46
|
}
|
|
45
47
|
}
|
package/README.md
CHANGED
|
@@ -39,6 +39,8 @@ For a permanent `bookmark-research` command: `npm install -g bookmark-research`.
|
|
|
39
39
|
| Pi | `npx bookmark-research install pi --non-interactive` | `pi install` |
|
|
40
40
|
| DSH | `npx bookmark-research install dsh --profile web --non-interactive` | `dsh plugin add` |
|
|
41
41
|
|
|
42
|
+
**Windows:** the installer writes the detected Python path into each client's MCP config, because Windows often has no working `python3`. Install Codex with `bookmark-research install codex` rather than adding the repository to Codex directly. `bookmark-research status` shows the interpreter in use.
|
|
43
|
+
|
|
42
44
|
Several at once: `npx bookmark-research install claude dsh --profile web --non-interactive`. Project scope: `--scope project --project /path/to/project` (Claude Code, Pi). All options: [installation guide](docs/installation.en.md).
|
|
43
45
|
|
|
44
46
|
**Ask your agent to install it.** Paste this into Codex, Claude Code, Pi or DSH:
|
|
@@ -64,7 +66,7 @@ No reinstall is needed. Settings and keys apply to every installed client; start
|
|
|
64
66
|
| --- | --- |
|
|
65
67
|
| Menu: configure, check, install, update | `bookmark-research` |
|
|
66
68
|
| Preferences and API keys | `bookmark-research setup` |
|
|
67
|
-
| Installed clients, preferences, key status | `bookmark-research status` |
|
|
69
|
+
| Installed clients, preferences, storage locations, recent research, key status | `bookmark-research status` |
|
|
68
70
|
| Script preferences | `bookmark-research config show` · `bookmark-research config set --input prefs.json` |
|
|
69
71
|
| Update or verify all clients | `bookmark-research update` · `bookmark-research verify` |
|
|
70
72
|
|
|
@@ -80,24 +82,42 @@ Use `npx bookmark-research …` if the command is not installed globally.
|
|
|
80
82
|
| `JINA_API_KEY` | Jina search | Jina Reader still reads pages anonymously |
|
|
81
83
|
| `OPENAI_API_KEY` | Optional OpenAI Deep Research runs | Feature off; host-led research still works |
|
|
82
84
|
|
|
83
|
-
|
|
85
|
+
Setup also asks where research reports and evidence go: a central folder (default `<data dir>/research`, path changeable) or beside the bookmark file or folder (falls back to the central folder when that is not possible). This affects new research only; existing tasks stay where they are. It then asks whether finished research should update the Wiki: suggest pages and ask first (default), automatic, or off.
|
|
84
86
|
|
|
85
|
-
|
|
87
|
+
Preferences (research depth, answer language, providers, archiving, output location, Wiki follow-up) can also be changed by asking the agent in a client session; it uses the `update_settings` tool.
|
|
88
|
+
|
|
89
|
+
## Where data is stored
|
|
86
90
|
|
|
87
|
-
|
|
91
|
+
Nothing is written into your bookmark files or the plugin. The data directory is `$BOOKMARK_RESEARCH_DATA_DIR`, or `${XDG_DATA_HOME:-~/.local/share}/bookmark-research`.
|
|
88
92
|
|
|
89
|
-
|
|
|
93
|
+
| Location | Contents |
|
|
90
94
|
| --- | --- |
|
|
91
|
-
|
|
|
95
|
+
| `research/` (or `<input>.bookmark-research/` beside the input) | One folder per research task: report, sources, evidence, and `work/` for the agent's intermediate files |
|
|
96
|
+
| `research-locations.json` | Where every task folder is, so tasks stay listed after the location setting changes |
|
|
97
|
+
| `knowledge/` | Saved page reads: `sources/` per fetch, `pages/` with each distinct text stored once |
|
|
98
|
+
| `wiki/` | Reviewed knowledge pages and revisions; `index.md` catalog and `log.md` timeline |
|
|
99
|
+
| `index.sqlite3`, `index.sqlite3.sources/` | Bookmark search index and managed snapshots of imported packages |
|
|
100
|
+
| `raw-library.sqlite3` | Rebuildable full-text index for `search_archive` |
|
|
101
|
+
| `${XDG_CONFIG_HOME:-~/.config}/bookmark-research/` | `settings.json` and `credentials.json` (private, owner-only) |
|
|
102
|
+
|
|
103
|
+
## What's inside
|
|
104
|
+
|
|
105
|
+
A bundle of one Skill, one local MCP server, and the remote MCPs and services that server connects to. The client registers only the local MCP.
|
|
92
106
|
|
|
93
|
-
|
|
107
|
+
| Layer | Component | Runs as | Setup |
|
|
108
|
+
| --- | --- | --- | --- |
|
|
109
|
+
| Skill | [`bookmark-research`](skills/bookmark-research/SKILL.md) and 11 method references, each with a [Chinese copy](skills/bookmark-research/references/zh/skill-guide.md) | Loaded by the client | None |
|
|
110
|
+
| Local MCP | `bookmark-research`, 37 tools (below) | Python process started by the client | Installed for you |
|
|
111
|
+
| Remote MCPs, called by the local MCP | [Exa](https://exa.ai) (search, page reading), [Parallel](https://parallel.ai) (search, fallback reading), [Tavily](https://tavily.com) (search fallback) | HTTPS, no local process | Optional API keys |
|
|
112
|
+
| HTTP service, called by the local MCP | [Jina Reader](https://jina.ai/reader) (fallback reading; search with a key) | HTTPS | Optional `JINA_API_KEY` |
|
|
113
|
+
| Optional client MCPs | Exa Agent, Parallel Task MCP, Tavily Research | Added to your client by you | `bookmark-research setup` shows the steps |
|
|
94
114
|
|
|
95
|
-
|
|
115
|
+
Local MCP tools:
|
|
96
116
|
|
|
97
117
|
| Group | Tools |
|
|
98
118
|
| --- | --- |
|
|
99
119
|
| Bookmarks | `sync_package`, `index_status`, `source_history`, `search_bookmarks`, `get_context` |
|
|
100
|
-
| Web | `search_web`, `fetch_web`, `search_providers` |
|
|
120
|
+
| Web | `search_web`, `fetch_web`, `search_archive`, `search_providers` |
|
|
101
121
|
| Research | `research_readiness`, `research_start`, `research_status`, `research_search`, `research_fetch`, `research_source`, `research_record`, `research_inventory`, `research_coverage`, `research_import_evidence`, `research_finish`, `research_route` |
|
|
102
122
|
| Research services (optional) | `research_services`, `research_service_prepare`, `_start`, `_status`, `_result`, `_cancel`, `_attach`, `_import` |
|
|
103
123
|
| Settings | `get_settings`, `update_settings` |
|
|
@@ -105,19 +125,6 @@ One local stdio server, `bookmark-research` (`python3 src/cli.py serve`), with 3
|
|
|
105
125
|
|
|
106
126
|
Tool descriptions: [details](docs/details.en.md#mcp-tools).
|
|
107
127
|
|
|
108
|
-
### Web services
|
|
109
|
-
|
|
110
|
-
Called by the plugin's own MCP server; you do not add their MCPs separately.
|
|
111
|
-
|
|
112
|
-
| Service | Role |
|
|
113
|
-
| --- | --- |
|
|
114
|
-
| [Exa](https://exa.ai) | Primary search; first page reader |
|
|
115
|
-
| [Parallel](https://parallel.ai) | Primary search; fallback page reader |
|
|
116
|
-
| [Tavily](https://tavily.com) | Search fallback |
|
|
117
|
-
| [Jina Reader](https://jina.ai/reader) | Fallback page reader; search with a key |
|
|
118
|
-
|
|
119
|
-
Optional research MCPs you may add to your client yourself: Exa Agent, Parallel Task and Tavily Research. `bookmark-research setup` shows the steps for each client.
|
|
120
|
-
|
|
121
128
|
## More
|
|
122
129
|
|
|
123
130
|
- [Installation guide](docs/installation.en.md): all install options, updates, ZIP downloads
|
package/README.zh.md
CHANGED
|
@@ -39,6 +39,8 @@ npx bookmark-research install
|
|
|
39
39
|
| Pi | `npx bookmark-research install pi --non-interactive` | `pi install` |
|
|
40
40
|
| DSH | `npx bookmark-research install dsh --profile web --non-interactive` | `dsh plugin add` |
|
|
41
41
|
|
|
42
|
+
**Windows:** Windows 上常常没有能用的 `python3`,所以安装器会把检测到的 Python 路径写进各宿主的 MCP 配置。Codex 请用 `bookmark-research install codex` 安装,不要直接把仓库加进 Codex。`bookmark-research status` 可以查看实际使用的 Python。
|
|
43
|
+
|
|
42
44
|
一次装多个:`npx bookmark-research install claude dsh --profile web --non-interactive`。装到指定项目:`--scope project --project /path/to/project`(Claude Code、Pi)。全部参数见[安装说明](docs/installation.md)。
|
|
43
45
|
|
|
44
46
|
**让 agent 帮你装。** 把下面这段粘贴给 Codex、Claude Code、Pi 或 DSH:
|
|
@@ -64,7 +66,7 @@ npx bookmark-research install
|
|
|
64
66
|
| --- | --- |
|
|
65
67
|
| 主菜单:配置、检查、安装、更新 | `bookmark-research` |
|
|
66
68
|
| 修改偏好和 API Key | `bookmark-research setup` |
|
|
67
|
-
|
|
|
69
|
+
| 查看已装宿主、偏好、保存位置、最近研究、Key 状态 | `bookmark-research status` |
|
|
68
70
|
| 用脚本改偏好 | `bookmark-research config show` · `bookmark-research config set --input prefs.json` |
|
|
69
71
|
| 更新或验证所有宿主 | `bookmark-research update` · `bookmark-research verify` |
|
|
70
72
|
|
|
@@ -80,42 +82,47 @@ npx bookmark-research install
|
|
|
80
82
|
| `JINA_API_KEY` | Jina 搜索 | Jina Reader 仍可匿名读取网页 |
|
|
81
83
|
| `OPENAI_API_KEY` | 可选的 OpenAI Deep Research 任务 | 功能关闭;宿主自己的研究不受影响 |
|
|
82
84
|
|
|
83
|
-
|
|
85
|
+
setup 还会问研究报告和证据放在哪里:统一目录(默认 `<数据目录>/research`,路径可改),或放在书签文件/文件夹旁边(放不了时退回统一目录)。只影响新的研究,已有任务留在原处。接着问研究结束后是否整理进 Wiki:给出建议并先问你(默认)、自动、关闭。
|
|
84
86
|
|
|
85
|
-
|
|
87
|
+
研究深度、答复语言、服务选择、归档、结果位置、Wiki 整理这些偏好,也可以直接在宿主会话里让 agent 改,它会调用 `update_settings` 工具。
|
|
88
|
+
|
|
89
|
+
## 数据保存在哪里
|
|
86
90
|
|
|
87
|
-
|
|
91
|
+
不会写入你的书签文件或插件目录。数据目录为 `$BOOKMARK_RESEARCH_DATA_DIR`,未设置时为 `${XDG_DATA_HOME:-~/.local/share}/bookmark-research`。
|
|
88
92
|
|
|
89
|
-
|
|
|
93
|
+
| 位置 | 内容 |
|
|
90
94
|
| --- | --- |
|
|
91
|
-
|
|
|
95
|
+
| `research/`(或输入旁边的 `<输入名>.bookmark-research/`) | 每个研究任务一个文件夹:报告、来源、证据,以及存放 agent 中间文件的 `work/` |
|
|
96
|
+
| `research-locations.json` | 所有任务文件夹的位置,改了位置设置后旧任务仍能列出 |
|
|
97
|
+
| `knowledge/` | 保存的网页读取:`sources/` 按每次读取保存,`pages/` 中相同正文只存一份 |
|
|
98
|
+
| `wiki/` | 审核后的知识页面与历史版本;`index.md` 目录、`log.md` 时间线 |
|
|
99
|
+
| `index.sqlite3`、`index.sqlite3.sources/` | 书签检索索引,以及导入数据包的托管快照 |
|
|
100
|
+
| `raw-library.sqlite3` | `search_archive` 使用的全文索引,可重建 |
|
|
101
|
+
| `${XDG_CONFIG_HOME:-~/.config}/bookmark-research/` | `settings.json` 和 `credentials.json`(仅当前用户可读) |
|
|
102
|
+
|
|
103
|
+
## 插件包含什么
|
|
104
|
+
|
|
105
|
+
由 1 个 Skill、1 个本地 MCP 服务,以及这个服务内部连接的远程 MCP 和网页服务组成。宿主里只需要登记这 1 个本地 MCP。
|
|
92
106
|
|
|
93
|
-
|
|
107
|
+
| 层 | 组成 | 运行方式 | 需要配置 |
|
|
108
|
+
| --- | --- | --- | --- |
|
|
109
|
+
| Skill | [`bookmark-research`](skills/bookmark-research/SKILL.md) 及 11 份方法参考,每份都有[中文版](skills/bookmark-research/references/zh/skill-guide.md) | 由宿主加载 | 不需要 |
|
|
110
|
+
| 本地 MCP | `bookmark-research`,37 个工具(见下表) | 宿主启动的 Python 进程 | 安装器自动完成 |
|
|
111
|
+
| 本地 MCP 调用的远程 MCP | [Exa](https://exa.ai)(搜索、网页读取)、[Parallel](https://parallel.ai)(搜索、备用读取)、[Tavily](https://tavily.com)(备用搜索) | HTTPS,不启动本地进程 | 可选 API Key |
|
|
112
|
+
| 本地 MCP 调用的 HTTP 服务 | [Jina Reader](https://jina.ai/reader)(备用读取;有 Key 时可搜索) | HTTPS | 可选 `JINA_API_KEY` |
|
|
113
|
+
| 可选的宿主 MCP | Exa Agent、Parallel Task MCP、Tavily Research | 需自行加到宿主里 | `bookmark-research setup` 会给出步骤 |
|
|
94
114
|
|
|
95
|
-
|
|
115
|
+
本地 MCP 的工具:
|
|
96
116
|
|
|
97
117
|
| 分类 | 工具 |
|
|
98
118
|
| --- | --- |
|
|
99
119
|
| 书签 | `sync_package`、`index_status`、`source_history`、`search_bookmarks`、`get_context` |
|
|
100
|
-
| 网页 | `search_web`、`fetch_web`、`search_providers` |
|
|
120
|
+
| 网页 | `search_web`、`fetch_web`、`search_archive`、`search_providers` |
|
|
101
121
|
| 研究 | `research_readiness`、`research_start`、`research_status`、`research_search`、`research_fetch`、`research_source`、`research_record`、`research_inventory`、`research_coverage`、`research_import_evidence`、`research_finish`、`research_route` |
|
|
102
122
|
| 专业研究服务(可选) | `research_services`、`research_service_prepare`、`_start`、`_status`、`_result`、`_cancel`、`_attach`、`_import` |
|
|
103
123
|
| 设置 | `get_settings`、`update_settings` |
|
|
104
124
|
| Wiki 与评测 | `wiki_write`、`wiki_get`、`wiki_list`、`wiki_search`、`wiki_lint`、`evaluate_research` |
|
|
105
125
|
|
|
106
|
-
### 网页服务
|
|
107
|
-
|
|
108
|
-
由插件自带的 MCP 服务直接调用,不需要另外添加它们的 MCP。
|
|
109
|
-
|
|
110
|
-
| 服务 | 作用 |
|
|
111
|
-
| --- | --- |
|
|
112
|
-
| [Exa](https://exa.ai) | 主要搜索;首选网页读取 |
|
|
113
|
-
| [Parallel](https://parallel.ai) | 主要搜索;备用网页读取 |
|
|
114
|
-
| [Tavily](https://tavily.com) | 备用搜索 |
|
|
115
|
-
| [Jina Reader](https://jina.ai/reader) | 备用网页读取;有 Key 时可搜索 |
|
|
116
|
-
|
|
117
|
-
另有可选的研究 MCP 可以自己加到宿主里:Exa Agent、Parallel Task、Tavily Research。`bookmark-research setup` 会给出各宿主的接入步骤。
|
|
118
|
-
|
|
119
126
|
## 更多
|
|
120
127
|
|
|
121
128
|
- [安装说明](docs/installation.md):全部安装方式、更新、ZIP 下载
|
|
@@ -55,4 +55,4 @@ bookmark-research doctor 检查 Python 与 SQLite FTS5
|
|
|
55
55
|
|
|
56
56
|
- 已实现:`bin/bookmark-research.js`、根目录 `package.json`(含 `bin` 与 `pi.skills`)、`scripts/launcher.py` 主菜单与子命令、Windows 按键/控制台/安装锁/Python 路径适配、`.github/workflows/test.yml` 三系统矩阵、README 与安装文档。
|
|
57
57
|
- 已验证(macOS):全量 450 个测试通过;`npm pack` 后全局安装的 `bookmark-research` 真实完成 Claude Code 安装、`status`、`verify`(36 个 MCP 工具);主菜单在伪终端中可用。
|
|
58
|
-
-
|
|
58
|
+
- 已处理(0.5.0-beta.2):Windows 上 Codex 改为登记固定了检测到的 Python 路径的副本,清单转发 `SystemRoot`/`windir`。仍待真机验证:Windows 上的四宿主实际安装、Pi。
|
package/docs/details.en.md
CHANGED
|
@@ -159,7 +159,9 @@ Run `python3 src/cli.py setup` to revisit the terminal wizard. Before each new w
|
|
|
159
159
|
|
|
160
160
|
The default configuration file is `~/.config/bookmark-research/settings.json`, created on the first update. Page archives default to `~/.local/share/bookmark-research/knowledge/`; XDG and plugin data-directory environment variables are respected. You can choose your own absolute paths. These files live outside the plugin and canvas package, so plugin updates do not replace them.
|
|
161
161
|
|
|
162
|
-
|
|
162
|
+
Research task folders go to `~/.local/share/bookmark-research/research/` by default (`output.mode: central`, path changeable) or, with `beside_input`, next to a single bookmark file or folder as `<name>.bookmark-research/`; URL lists, several sources, Git repositories, canvas packages and unwritable locations fall back to the central folder with a stated reason. Changes apply to new tasks only; `research-locations.json` keeps every task listed. `wiki.after_research` (`suggest` by default, `auto`, `off`) decides whether finished research proposes or writes Wiki updates. Each Wiki write also refreshes `wiki/index.md` and appends `wiki/log.md`.
|
|
163
|
+
|
|
164
|
+
With archiving enabled, received fetch responses create a snapshot directory: `response.json` stores the actual MCP response; `manifest.json` records URLs, provider, fetch time, response and text hashes, and status; successfully recognized text is stored once per distinct body in the archive's `pages/<SHA-256>.md`. `search_archive` searches that text and all research evidence without network access. Transport failures without a response return a classified error and no `result` or invented archive. Publication dates and authors are recorded separately. Conflicting responses for the same URL are retained and marked. Excerpts and unknown completeness are identified, and archive errors are returned with the fetch result. Later reads add snapshots without overwriting earlier ones.
|
|
163
165
|
|
|
164
166
|
```sh
|
|
165
167
|
python3 src/cli.py config show
|
|
@@ -185,6 +187,7 @@ The [shared Skill](../skills/bookmark-research/SKILL.md) guides the agent's work
|
|
|
185
187
|
| `search_providers` | List configured web providers |
|
|
186
188
|
| `search_web` | Search through Exa / Parallel / Tavily / keyed Jina and merge results per target |
|
|
187
189
|
| `fetch_web` | Read known URLs and optionally archive responses and page text |
|
|
190
|
+
| `search_archive` | Literal full-text search over saved page text in the knowledge archive and all research evidence, merged by content hash |
|
|
188
191
|
| `get_settings` | Read current settings and storage paths |
|
|
189
192
|
| `update_settings` | Update persistent settings |
|
|
190
193
|
| `research_readiness` | Check credentials, selected services and observed host tools; return setup/login guidance |
|
|
@@ -261,8 +264,8 @@ From this plugin directory **in a full repository checkout**, developers can run
|
|
|
261
264
|
python3 -m unittest discover -s tests -v
|
|
262
265
|
python3 scripts/verify_fixture.py --output /tmp/bookmark-research-verification
|
|
263
266
|
python3 scripts/verify_quality.py
|
|
264
|
-
python3 scripts/build_zip.py --output dist/bookmark-research-0.5.0-beta.
|
|
265
|
-
python3 scripts/build_zip.py --include-tests --output dist/bookmark-research-test-pack-0.5.0-beta.
|
|
267
|
+
python3 scripts/build_zip.py --output dist/bookmark-research-0.5.0-beta.3.zip
|
|
268
|
+
python3 scripts/build_zip.py --include-tests --output dist/bookmark-research-test-pack-0.5.0-beta.3.zip
|
|
266
269
|
```
|
|
267
270
|
|
|
268
271
|
The fixture verifier checks independent bookmark counts, copy-card semantics, scoped queries, unchanged source hashes, incremental sync, and a two-round research lifecycle with a failed page and resumed operation. To additionally validate your own package, append `--package /absolute/path/to/package`; this stays offline and writes derived files only to the new output directory.
|
package/docs/details.md
CHANGED
|
@@ -51,11 +51,13 @@ JSON/`.canvas` 是源数据,SQLite 是可重建的本地查询索引,无
|
|
|
51
51
|
|
|
52
52
|
- `index.sqlite3`:来源登记、书签和画布关系索引。
|
|
53
53
|
- `index.sqlite3.sources/`:原始协议文件和关系清单的版本快照。
|
|
54
|
-
- `knowledge
|
|
55
|
-
- `research
|
|
56
|
-
- `
|
|
54
|
+
- `knowledge/`:普通网页响应、正文(`pages/` 中相同正文只存一份)和来源记录。
|
|
55
|
+
- `research/`:完整输入清单、任务、证据、覆盖与报告;每个任务的 `work/` 存放 agent 中间文件。设置 `output.mode: beside_input` 时,新任务放在单个书签文件/文件夹旁边的 `<名称>.bookmark-research/`,URL 列表、多个来源、Git 仓库、画布包或没有写权限时退回统一目录并说明原因。改设置只影响新任务。
|
|
56
|
+
- `research-locations.json`:所有任务文件夹的位置登记。
|
|
57
|
+
- `wiki/`:主题/实体页和不可变修订;每次写入更新 `index.md`(目录)并追加 `log.md`(时间线)。
|
|
58
|
+
- `raw-library.sqlite3`:`search_archive` 的全文索引,覆盖 `knowledge/` 与所有研究证据,可重建。
|
|
57
59
|
|
|
58
|
-
配置默认为 `~/.config/bookmark-research/settings.json`。支持 XDG 和显式路径覆盖,多宿主可共享数据。索引更新不会自动抓网页、运行 LLM 或改写 Wiki;来源变化时提示复核,旧研究与证据保留。
|
|
60
|
+
配置默认为 `~/.config/bookmark-research/settings.json`。`wiki.after_research` 决定研究结束后的 Wiki 整理:`suggest`(默认,先建议再问)、`auto`、`off`。支持 XDG 和显式路径覆盖,多宿主可共享数据。索引更新不会自动抓网页、运行 LLM 或改写 Wiki;来源变化时提示复核,旧研究与证据保留。
|
|
59
61
|
|
|
60
62
|
`research_record` 支持单条 `entry` 或每批至多 50 条的 `entries`,按顺序校验后一次保存,返回精简编号;稳定的 `batch_id` 可避免重试产生重复记录。依赖新 claim ID 的记录在读取返回值后另批提交。`resume` 和 `external_run` 仍需单条调用,详见 [证据记录](../skills/bookmark-research/references/zh/deep-research.md)。
|
|
61
63
|
|
package/docs/installation.en.md
CHANGED
|
@@ -52,7 +52,7 @@ The Codex branch manages `bookmark-research@bookmark-research`. If already insta
|
|
|
52
52
|
|
|
53
53
|
## Guided setup and readiness
|
|
54
54
|
|
|
55
|
-
After installation, the wizard offers research depth, answer language, primary/fallback search providers, page readers, archive location, check frequency and optional professional APIs. Enter keeps displayed preferences. API-key input is masked; each key is checked immediately and can be re-entered if the check fails. Keys are saved separately in `credentials.json` next to settings, with mode `0600`; this local file is not encrypted. `BOOKMARK_RESEARCH_CREDENTIALS` overrides its path, and environment variables override saved keys. No keys enter preferences, packages, receipts or readiness output. A ChatGPT/Codex login is separate from an OpenAI API key.
|
|
55
|
+
After installation, the wizard offers research depth, answer language, primary/fallback search providers, page readers, archive location, where research reports and evidence go (central folder, a custom central path, or beside the bookmark file/folder with fallback to the central folder; new tasks only), whether finished research updates the Wiki (suggest and ask first, automatic, off), check frequency and optional professional APIs. Enter keeps displayed preferences. API-key input is masked; each key is checked immediately and can be re-entered if the check fails. Keys are saved separately in `credentials.json` next to settings, with mode `0600`; this local file is not encrypted. `BOOKMARK_RESEARCH_CREDENTIALS` overrides its path, and environment variables override saved keys. No keys enter preferences, packages, receipts or readiness output. A ChatGPT/Codex login is separate from an OpenAI API key.
|
|
56
56
|
|
|
57
57
|
Rerun setup at any time with `python3 <installed path>/src/cli.py setup`; the exact command (`getting_started.setup_command` in the JSON result) is printed after installation. From a checkout:
|
|
58
58
|
|
package/docs/installation.md
CHANGED
|
@@ -69,7 +69,7 @@ Codex 分支管理 `bookmark-research@bookmark-research`。已通过 `personal`
|
|
|
69
69
|
|
|
70
70
|
## 向导与就绪检查
|
|
71
71
|
|
|
72
|
-
|
|
72
|
+
安装成功后可依次设置研究深度、答复语言、主要/备用搜索服务、正文读取服务、归档目录、研究报告和证据的位置(统一目录、自定义统一目录,或放在书签文件/文件夹旁边、放不了时退回统一目录;只影响新任务)、研究结束后是否整理进 Wiki(建议并先询问、自动、关闭)、检查频率和可选专业 API。回车保留显示的偏好。API Key 掩码输入并立即检查,失败可重新输入;密钥单独保存为配置旁权限 `0600` 的 `credentials.json`,是本地明文文件;可用 `BOOKMARK_RESEARCH_CREDENTIALS` 指定位置。环境变量优先于保存的密钥,密钥不进入偏好 JSON、安装包、安装记录或就绪检查输出。ChatGPT/Codex 登录与 OpenAI API Key 分开。
|
|
73
73
|
|
|
74
74
|
以后随时运行 `python3 <安装路径>/src/cli.py setup` 重开向导,完整命令(JSON 结果中的 `getting_started.setup_command`)会在安装结束时打印;在源码中可运行:
|
|
75
75
|
|
|
@@ -306,7 +306,7 @@ Git marketplace 的 `update` 保持已注册的仓库与 ref,刷新其快照
|
|
|
306
306
|
|
|
307
307
|
```sh
|
|
308
308
|
python3 -m unittest discover -s tests
|
|
309
|
-
python3 scripts/build_zip.py --output dist/bookmark-research-0.5.0-beta.
|
|
309
|
+
python3 scripts/build_zip.py --output dist/bookmark-research-0.5.0-beta.3.zip
|
|
310
310
|
```
|
|
311
311
|
|
|
312
312
|
ZIP 包含原生插件、marketplace、共享运行时、Skill、四宿主源文件及组装器、安装入口、导出器、文档和许可;不包含测试、数据库、页面归档、用户配置或书签数据包。加 `--include-tests` 可另建测试包,包含合成 fixture、宿主 JS 检查、`verify_fixture.py` 和 `verify_quality.py`,可解压后再次导出宿主包。
|
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
# 知识库统一与研究结果位置计划
|
|
2
|
+
|
|
3
|
+
日期:2026-09-29。依据两次讨论:研究结果放在哪里、按 LLM Wiki 统一知识库。
|
|
4
|
+
|
|
5
|
+
## 决策
|
|
6
|
+
|
|
7
|
+
| 问题 | 决定 |
|
|
8
|
+
| --- | --- |
|
|
9
|
+
| 研究结果默认位置 | 统一目录 `<数据目录>/research`,路径可改 |
|
|
10
|
+
| 放在输入旁边 | 可选 `output.mode = beside_input`:单文件 `a.json` → `a.bookmark-research/`,文件夹或 ZIP `pkg` → `pkg.bookmark-research/`,一律同级,不进原文件夹 |
|
|
11
|
+
| 放不了时 | 退回统一目录并说明原因:没有输入文件(URL 列表)、多个来源、位置落在 git 仓库里、位置在画布包里、没有写权限 |
|
|
12
|
+
| 改设置 | 只影响新任务,旧任务不迁移;位置登记表 `research-locations.json` 保证新旧任务都能列出,不存在的标为 missing |
|
|
13
|
+
| agent 中间文件 | 只放 `<任务文件夹>/work/`,`research_start` 返回该路径 |
|
|
14
|
+
| 研究结束后整理进 Wiki | 默认 `wiki.after_research = suggest`:给出建议,经用户确认再写;可设 `auto` 或 `off` |
|
|
15
|
+
| Wiki 目录与日志 | 每次写入后自动生成 `wiki/index.md`(目录)和追加 `wiki/log.md`(时间线);`index.json` 仍是准确数据 |
|
|
16
|
+
| 原始资料检索 | 新增 `search_archive`:SQLite FTS5 覆盖 `knowledge/` 归档和所有已登记任务的证据原文 |
|
|
17
|
+
|
|
18
|
+
## 对"原始资料只存一处"的调整
|
|
19
|
+
|
|
20
|
+
讨论时提出研究任务的证据改存进 `knowledge/`、任务只记引用。实施前发现它和"放在输入旁边"冲突:任务文件夹会依赖统一目录里的文件,搬走或放在输入旁边时报告链接和哈希校验会失效;冻结证据也是现有设计"任务文件夹可整体移动"的前提。
|
|
21
|
+
|
|
22
|
+
因此改为:
|
|
23
|
+
|
|
24
|
+
- 物理上,研究证据仍随任务保存(冻结、可移动),`knowledge/` 内对完全相同的正文去重(同一内容只存一份页面文件)。
|
|
25
|
+
- 逻辑上,`search_archive` 用一个全文索引覆盖两处,按内容哈希合并同一正文,结果列出所有出现位置。查原始资料只需一个入口。
|
|
26
|
+
|
|
27
|
+
## 分工
|
|
28
|
+
|
|
29
|
+
| 部分 | 文件 |
|
|
30
|
+
| --- | --- |
|
|
31
|
+
| 结果位置、登记表、`work/`、结束后的 Wiki 建议 | `src/research.py`、`tests/test_output_location.py` |
|
|
32
|
+
| 原始资料去重与 `search_archive`(MCP + CLI) | `src/archive.py`、`src/raw_library.py`、`src/mcp_server.py`、`src/cli.py`、`tests/test_raw_library.py` |
|
|
33
|
+
| Wiki `index.md`/`log.md`、向导问题、`status`、Skill 与 README | `src/wiki.py`、`src/onboarding.py`、`scripts/launcher.py`、Skill(中英)、README(中英)、`docs/details*.md` |
|
|
34
|
+
|
|
35
|
+
已先完成:`output` 与 `wiki.after_research` 配置和校验、`update_settings` 结构、`src/research_locations.py`。
|
|
36
|
+
|
|
37
|
+
## 验收
|
|
38
|
+
|
|
39
|
+
本机全量测试通过;CI 三系统通过;真实走一遍:统一目录研究、放在输入旁边研究、git 仓库内回退、改目录后旧任务仍可列出、`search_archive` 查到归档与证据、Wiki 写入后 `index.md`/`log.md` 更新。
|
|
40
|
+
|
|
41
|
+
## 完成情况(2026-09-29)
|
|
42
|
+
|
|
43
|
+
已实现上述全部内容。本机全量 482 个测试通过;隔离数据目录端到端验证:统一目录、放在输入旁边(文件夹、ZIP、单文件)、git 仓库内回退(仓库无新增文件)、URL 列表与多来源回退、单次指定目录、改目录后 9 个任务仍全部列出且删除的任务标为 missing、`search_archive` 中文查到研究证据、Wiki 写入后生成 `index.md` 并追加 `log.md`。
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Bookmark Research 0.5.0 beta 2
|
|
2
|
+
|
|
3
|
+
**Beta.** Please report problems on [GitHub Issues](https://github.com/Browser-bookmark-hub/Bookmark-Research/issues).
|
|
4
|
+
|
|
5
|
+
## Install or update
|
|
6
|
+
|
|
7
|
+
```sh
|
|
8
|
+
npx bookmark-research@latest install # interactive; macOS, Linux, Windows
|
|
9
|
+
bookmark-research update # if already installed
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
## Changes
|
|
13
|
+
|
|
14
|
+
- **Codex on Windows**: the installer now registers a copy whose MCP command is the Python it detected, instead of `python3`, which is often missing or a Microsoft Store stub on Windows. `update` refreshes that copy from the original source. Install Codex on Windows with `bookmark-research install codex`.
|
|
15
|
+
- **Windows environment**: the MCP server receives `SystemRoot` and `windir`, which older Codex builds stripped and Python networking needs.
|
|
16
|
+
- **`bookmark-research status`** shows the detected Python and the MCP command written into client configs.
|
|
17
|
+
- **README** lists the bundle by layer: one Skill, one local MCP server (36 tools), the remote MCPs it calls (Exa, Parallel, Tavily), Jina Reader, and optional client MCPs.
|
|
18
|
+
|
|
19
|
+
## Validation
|
|
20
|
+
|
|
21
|
+
452 regression tests pass in GitHub Actions on Ubuntu, macOS and Windows with Python 3.9 and 3.12, plus an npm pack and install smoke test. Codex on native Windows is covered by simulated tests, not yet by a real Windows machine. Pi has not been tested on a real machine.
|
|
22
|
+
|
|
23
|
+
Assets: plugin ZIP, developer test pack and `SHA256SUMS`.
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Bookmark Research 0.5.0 beta 3
|
|
2
|
+
|
|
3
|
+
**Beta.** Please report problems on [GitHub Issues](https://github.com/Browser-bookmark-hub/Bookmark-Research/issues).
|
|
4
|
+
|
|
5
|
+
## Install or update
|
|
6
|
+
|
|
7
|
+
```sh
|
|
8
|
+
npx bookmark-research@latest install # interactive; macOS, Linux, Windows
|
|
9
|
+
bookmark-research update # if already installed
|
|
10
|
+
bookmark-research setup # choose where research goes and the Wiki follow-up
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
## Changes
|
|
14
|
+
|
|
15
|
+
- **Research output location** (`output.mode`): `central` (default, path configurable) or `beside_input`, which puts task folders next to the bookmark file or folder (`card.bookmark-research/`, never inside the original). Falls back to central for URL lists, multiple sources, git work trees, canvas packages and unwritable locations, and reports why. `research_start` also accepts a one-off `output_directory`.
|
|
16
|
+
- **No migration**: changing the setting affects new tasks only. A location registry keeps every task listed; deleted folders show as `missing`.
|
|
17
|
+
- **`work/` folder** per task for the agent's intermediate files, returned by `research_start`.
|
|
18
|
+
- **`search_archive`** (new MCP tool and `search-archive` CLI): literal full-text search over archived pages and all research evidence, merged by content hash; Chinese supported. The knowledge archive now stores identical page bodies once.
|
|
19
|
+
- **Wiki catalog**: each write regenerates `wiki/index.md` and appends to `wiki/log.md`.
|
|
20
|
+
- **Wiki follow-up** (`wiki.after_research`): after `research_finish`, suggest pages and ask first (default), write automatically, or off.
|
|
21
|
+
- **Setup and status**: two new setup questions; `status` shows storage locations and recent tasks. README adds a "Where data is stored" table.
|
|
22
|
+
- **Cross-platform**: evidence paths are stored with `/`, so task folders move between Windows, macOS and Linux.
|
|
23
|
+
|
|
24
|
+
## Validation
|
|
25
|
+
|
|
26
|
+
482 regression tests pass in GitHub Actions on Ubuntu, macOS and Windows with Python 3.9 and 3.12. An isolated end-to-end run checked central, beside-input (folder, ZIP, single file), every fallback, explicit directories, listing after a directory change, Chinese `search_archive` and the Wiki catalog. Pi and real Windows client installs remain untested.
|
|
27
|
+
|
|
28
|
+
Assets: plugin ZIP, developer test pack and `SHA256SUMS`.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "bookmark-research",
|
|
3
|
-
"version": "0.5.0-beta.
|
|
3
|
+
"version": "0.5.0-beta.3",
|
|
4
4
|
"description": "Check bookmark URLs or Canvas packages, compare sources with host-led research, and retain verifiable reports.",
|
|
5
5
|
"license": "GPL-3.0-only",
|
|
6
6
|
"repository": {"type": "git", "url": "git+https://github.com/Browser-bookmark-hub/Bookmark-Research.git"},
|
package/scripts/export_bundle.py
CHANGED
|
@@ -369,6 +369,10 @@ Reference: https://github.com/deepseek-ai/deepseek-harness/blob/master/packages/
|
|
|
369
369
|
def _write_adapter(stage, format_name, output, manifest, python="python3"):
|
|
370
370
|
metadata = {"name": NAME, "version": manifest["version"], "description": manifest["description"]}
|
|
371
371
|
if format_name == "codex":
|
|
372
|
+
if python != "python3":
|
|
373
|
+
# Codex registers this manifest verbatim; pin the detected interpreter.
|
|
374
|
+
manifest = json.loads(json.dumps(manifest))
|
|
375
|
+
manifest["mcpServers"][NAME]["command"] = python
|
|
372
376
|
_write_json(stage, ".codex-plugin/plugin.json", manifest)
|
|
373
377
|
_write_json(stage, ".agents/plugins/marketplace.json", {
|
|
374
378
|
"name": NAME, "interface": {"displayName": "Bookmark Research"},
|
package/scripts/install.py
CHANGED
|
@@ -168,7 +168,10 @@ def _runtime_check(root, timeout=30):
|
|
|
168
168
|
if health.get("fts5") is not True:
|
|
169
169
|
raise RuntimeError("Installed runtime requires SQLite FTS5")
|
|
170
170
|
server = manifest.get("mcpServers", {}).get(NAME, {})
|
|
171
|
-
|
|
171
|
+
command = server.get("command")
|
|
172
|
+
# Windows installs pin the detected interpreter; the repository manifest keeps python3.
|
|
173
|
+
pinned = isinstance(command, str) and Path(command).is_absolute() and Path(command).is_file()
|
|
174
|
+
if (command != "python3" and not pinned) or server.get("args") != ["src/cli.py", "serve"] or server.get("cwd") != ".":
|
|
172
175
|
raise ValueError("Unexpected native MCP launch configuration; verify it with Codex directly")
|
|
173
176
|
requests = [
|
|
174
177
|
{"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {
|
|
@@ -298,6 +301,40 @@ def _print_getting_started(guide):
|
|
|
298
301
|
print("\n".join(lines), file=sys.stderr)
|
|
299
302
|
|
|
300
303
|
|
|
304
|
+
def _pin_interpreter():
|
|
305
|
+
"""Windows has no reliable python3 command, so Codex gets this interpreter's absolute path."""
|
|
306
|
+
return os.name == "nt"
|
|
307
|
+
|
|
308
|
+
|
|
309
|
+
def _pinned_home():
|
|
310
|
+
data = Path(os.environ.get("XDG_DATA_HOME", str(Path.home() / ".local/share"))).expanduser()
|
|
311
|
+
root = os.environ.get("BOOKMARK_RESEARCH_INSTALL_DIR") or str(data / "bookmark-research/installations")
|
|
312
|
+
return Path(root).expanduser().resolve() / "codex-pinned"
|
|
313
|
+
|
|
314
|
+
|
|
315
|
+
def _pinned_source(origin, timeout, python=None):
|
|
316
|
+
"""Export a Codex marketplace whose MCP command is the detected interpreter."""
|
|
317
|
+
from export_bundle import export_bundle
|
|
318
|
+
from host_install import _checkout
|
|
319
|
+
base = _pinned_home()
|
|
320
|
+
base.mkdir(parents=True, exist_ok=True)
|
|
321
|
+
bundle, stage = base / "bundle", base / "stage"
|
|
322
|
+
shutil.rmtree(stage, ignore_errors=True)
|
|
323
|
+
with _checkout(origin, timeout) as (root, _):
|
|
324
|
+
export_bundle("codex", stage, root, python or sys.executable)
|
|
325
|
+
shutil.rmtree(bundle, ignore_errors=True)
|
|
326
|
+
stage.rename(bundle)
|
|
327
|
+
(base / "origin.json").write_text(json.dumps(origin, ensure_ascii=False), encoding="utf-8")
|
|
328
|
+
return _source(str(bundle))
|
|
329
|
+
|
|
330
|
+
|
|
331
|
+
def _pinned_origin(requested):
|
|
332
|
+
base = _pinned_home()
|
|
333
|
+
if requested["sourceType"] != "local" or Path(requested["source"]) != base / "bundle":
|
|
334
|
+
return None
|
|
335
|
+
return json.loads((base / "origin.json").read_text(encoding="utf-8"))
|
|
336
|
+
|
|
337
|
+
|
|
301
338
|
def manage(action, cli, source=None, ref=None, dry_run=False, installed_path=None, language="auto"):
|
|
302
339
|
language = resolve_language(language)
|
|
303
340
|
if action == "verify":
|
|
@@ -305,6 +342,8 @@ def manage(action, cli, source=None, ref=None, dry_run=False, installed_path=Non
|
|
|
305
342
|
current = _marketplace(cli)
|
|
306
343
|
if action == "install":
|
|
307
344
|
requested = _source(source if source is not None else SOURCE_ROOT, ref)
|
|
345
|
+
if _pin_interpreter() and not dry_run:
|
|
346
|
+
requested = _pinned_source({key: requested[key] for key in ("sourceType", "source", "ref") if key in requested}, cli.timeout)
|
|
308
347
|
if current:
|
|
309
348
|
_check_source(current, requested)
|
|
310
349
|
if ref:
|
|
@@ -318,7 +357,12 @@ def manage(action, cli, source=None, ref=None, dry_run=False, installed_path=Non
|
|
|
318
357
|
if not current or not _installed(cli):
|
|
319
358
|
raise ValueError("No existing bookmark-research installation; run install first")
|
|
320
359
|
requested = _configured_source(current)
|
|
321
|
-
|
|
360
|
+
origin = _pinned_origin(requested)
|
|
361
|
+
if origin and not dry_run:
|
|
362
|
+
# Refresh from the original source, then reinstall the pinned copy.
|
|
363
|
+
requested = _pinned_source(origin, cli.timeout)
|
|
364
|
+
commands = []
|
|
365
|
+
elif requested["sourceType"] == "git":
|
|
322
366
|
commands = [["plugin", "marketplace", "upgrade", NAME]]
|
|
323
367
|
else:
|
|
324
368
|
# Validate the retained local source before touching the installed cache.
|
package/scripts/launcher.py
CHANGED
|
@@ -17,7 +17,7 @@ USAGE = """usage: bookmark-research [--lang auto|en|zh] [COMMAND] [ARGS...]
|
|
|
17
17
|
update [HOST...] Update installed hosts (all when omitted)
|
|
18
18
|
verify [HOST...] Verify installed hosts (all when omitted)
|
|
19
19
|
setup Preferences, services, hidden API-key entry and checks
|
|
20
|
-
status Installed hosts, preferences and key status as JSON
|
|
20
|
+
status Installed hosts, preferences, storage locations, recent tasks and key status as JSON
|
|
21
21
|
config show|set View or change preferences (set reads --input FILE or -)
|
|
22
22
|
doctor Check Python and SQLite FTS5
|
|
23
23
|
|
|
@@ -58,6 +58,19 @@ def installed_targets(codex="codex"):
|
|
|
58
58
|
return rows
|
|
59
59
|
|
|
60
60
|
|
|
61
|
+
def recent_research(limit=5):
|
|
62
|
+
"""Newest registered research task folders; moved or deleted folders report exists=false."""
|
|
63
|
+
import research_locations
|
|
64
|
+
try:
|
|
65
|
+
tasks = research_locations.load()
|
|
66
|
+
except (OSError, ValueError) as error:
|
|
67
|
+
return {"error": str(error)[:500], "tasks": []}
|
|
68
|
+
rows = sorted(tasks.items(), key=lambda row: (str(row[1].get("created_at", "")), row[0]), reverse=True)[:limit]
|
|
69
|
+
return {"registry": str(research_locations.registry_path()), "total": len(tasks),
|
|
70
|
+
"tasks": [{"id": key, "path": value.get("path"), "created_at": value.get("created_at"),
|
|
71
|
+
"exists": bool(value.get("path")) and Path(value["path"]).is_dir()} for key, value in rows]}
|
|
72
|
+
|
|
73
|
+
|
|
61
74
|
def status():
|
|
62
75
|
from credentials import Credentials
|
|
63
76
|
from settings import Settings
|
|
@@ -67,7 +80,13 @@ def status():
|
|
|
67
80
|
"preferences": {"depth": current["research"]["depth"], "language": current["research"]["response_language"],
|
|
68
81
|
"search": current["search"]["providers"], "fallback": current["search"]["fallback_providers"],
|
|
69
82
|
"readers": Settings.fetch_providers(current), "archive": current["archive"]["enabled"]},
|
|
83
|
+
"output": {"mode": current["output"]["mode"], "directory": current["output"]["directory"]},
|
|
84
|
+
"wiki": {"directory": current["wiki"]["directory"], "after_research": current["wiki"]["after_research"]},
|
|
85
|
+
"recent_research": recent_research(),
|
|
70
86
|
"credentials": {row["name"]: row["source"] or "missing" for row in Credentials(settings).describe()["credentials"]},
|
|
87
|
+
# The interpreter detected for this run; Windows installs write it into client MCP configs.
|
|
88
|
+
"python": {"executable": sys.executable, "version": sys.version.split()[0],
|
|
89
|
+
"mcp_command": sys.executable if os.name == "nt" else "python3"},
|
|
71
90
|
"settings_path": str(settings.path)}
|
|
72
91
|
|
|
73
92
|
|
|
@@ -38,6 +38,8 @@ python3 <plugin-root>/src/cli.py fetch-web 'https://example.com/page' --timeout
|
|
|
38
38
|
|
|
39
39
|
`fetch_web.pages` contains one selected extract per URL; each attempt retains status and archive paths. Read the relevant text and cite the original URL. Complete provider envelopes remain in the archive; `raw:true` (CLI `--raw`) returns them when needed. Check archive errors and completeness markers; a provider extract is not proof of a complete or live origin page.
|
|
40
40
|
|
|
41
|
+
Before reading a page again, `search_archive` (CLI `search-archive`) finds literal text already saved in the knowledge archive and every registered research task's evidence; identical bodies are merged and list each occurrence with its retrieval time. Reuse a hit when its retrieval time suits the question; otherwise fetch again. It is not web search.
|
|
42
|
+
|
|
41
43
|
## Search, assess, follow up
|
|
42
44
|
|
|
43
45
|
The current host model owns the first pass. Read known URLs directly; use `search_web` to discover sources, then read the relevant pages. Ordinary comparisons can answer directly. Create persistent research records when the user needs retained progress or a report.
|
|
@@ -65,6 +67,8 @@ Read the [deep workflow](references/deep-research.md) when this mode is needed.
|
|
|
65
67
|
- Canvas packages: use indexed `source_ids`; whole-package scope is the default. Only an explicit subset uses `scope_mode:subset` and selected inventory IDs or bookmark references.
|
|
66
68
|
- A public question without an original bookmark list may omit both inputs. URLs mentioned only in the brief are not tracked input.
|
|
67
69
|
|
|
70
|
+
`research_start` returns `output` (`placement`, `path`, `fallback_reason`) and `work_directory`. Tasks go to the central folder, or with `output.mode:beside_input` next to the single bookmark file/folder (`<name>.bookmark-research/`). When that is impossible (URL list, several sources, Git repository, canvas package, no write permission), the task uses the central folder and `fallback_reason` says why; tell the user. Put all of your own intermediate or scratch files only inside `work_directory`, never elsewhere in the data directory or beside the user's files.
|
|
71
|
+
|
|
68
72
|
Follow every `research_inventory.next_offset`. Keep original `u-...` inventory IDs separate from saved `sN` evidence IDs. Read original text, review it, record quoted claims and each inventory judgment. Batch ready `research_record.entries` (up to 50) with a stable `batch_id`; use returned IDs in dependent batches. Check coverage differences and continue until complete or budget-limited. Failure notes, bulk exclusions, search snippets and external report citations cannot establish full original-source review.
|
|
69
73
|
|
|
70
74
|
The parent owns synthesis. When delegation is available and permitted, give independent readers/verifiers the scope, questions, output language, relevant instructions and actual MCP/CLI access. Read [host workflows](references/host-workflows.md) for Codex, Claude, Pi or DSH execution details; do not assume child context inheritance. Without delegation, conduct and disclose same-agent review.
|
|
@@ -75,6 +79,6 @@ Professional research is optional support for a concrete subproblem or an explic
|
|
|
75
79
|
|
|
76
80
|
Answer quick tasks directly. Before finishing a report, review the actual short answer against evidence and counterevidence; condensing supported claims must preserve their conditions and limits. Use `research_finish` for reports and source/coverage artifacts: distinguish accounted inputs, usable text, substantive review and answered questions. Use `completed` only when coverage, supported answers and conflict checks pass; otherwise retain an `incomplete` report and gaps. Resume an incomplete archive with a `resume` record. Saved state is not a background agent.
|
|
77
81
|
|
|
78
|
-
|
|
82
|
+
After `research_finish`, follow its `wiki_follow_up`: `suggest` → present `candidates` with the eligible reviewed claims and ask before `wiki_write`; `auto` → write or update pages citing only `eligible_claim_ids`, then say which pages changed; `off` → skip unless the user asks. Also create Wiki pages when the user asks to retain conclusions in a Wiki or knowledge base; read [Wiki and evaluation](references/wiki-and-evaluation.md) for publication, lint and actual labeled quality comparisons. The Wiki's `index.md` (catalog) and `log.md` (timeline) can be read or grepped. Do not infer semantic correctness from quote/hash checks.
|
|
79
83
|
|
|
80
84
|
Use `get_settings` for configuration questions and `update_settings` for lasting preferences; temporary choices use per-call options. [Settings and archives](references/settings-and-archive.md) describes storage. All archives, indexes and authored outputs stay outside the original package.
|
|
@@ -11,6 +11,7 @@ Commands use Python 3.9+ and the standard library; stdout is JSON. Replace `<roo
|
|
|
11
11
|
| `index_status` / `sync_package` | `status` / `sync` |
|
|
12
12
|
| `search_bookmarks` / `get_context` | `search` / `context` |
|
|
13
13
|
| `fetch_web` / `search_web` | `fetch-web` / `search-web` |
|
|
14
|
+
| `search_archive` | `search-archive` |
|
|
14
15
|
| `research_start` / `research_fetch` | `research start` / `research fetch` |
|
|
15
16
|
|
|
16
17
|
```sh
|
|
@@ -42,6 +43,7 @@ python3 <root>/src/cli.py providers --probe
|
|
|
42
43
|
python3 <root>/src/cli.py search-web --target 'Example Company A official pricing' --target 'Example Company B official pricing' --limit 5
|
|
43
44
|
python3 <root>/src/cli.py fetch-web https://example.com/company-a https://example.com/company-b
|
|
44
45
|
python3 <root>/src/cli.py fetch-web https://example.com/company-a --no-archive --max-characters 30000
|
|
46
|
+
python3 <root>/src/cli.py search-archive 'exact phrase or terms' --url example.com --limit 10
|
|
45
47
|
python3 <root>/src/cli.py config show
|
|
46
48
|
python3 <root>/src/cli.py config set --search-provider exa --archive true
|
|
47
49
|
python3 <root>/src/cli.py config set --archive-dir /absolute/path/to/knowledge
|
|
@@ -51,7 +53,7 @@ python3 <root>/src/cli.py config set --archive-dir /absolute/path/to/knowledge
|
|
|
51
53
|
|
|
52
54
|
`setup` guides preferences, hidden key entry and selected service checks. `--non-interactive --input FILE` is the agent path; `--skip-checks` prevents network checks. `readiness` uses saved cached/always/manual policy; `--refresh` forces checks, `--offline` prevents network, and `--test-retrieval` opts into sample search/read quota. Repeat `--provider` to restrict retrieval checks, `--service openai|parallel` to select a professional service, and `--tool` for actually observed host tools. `--host` accepts `codex`, `claude_code`, `pi`, `dsh`, `unknown`. Before a new web question, run readiness once and follow its remediation; no professional job is started. See the settings reference for status meanings and private credential storage.
|
|
53
55
|
|
|
54
|
-
`fetch-web` returns one selected text per URL in `pages`, with all attempt statuses and archive paths. `--raw` returns the full provider envelopes. Its `--timeout` applies per provider HTTP request; discovery and fallback can take longer overall. `research fetch` accepts only its documented MCP JSON fields and does not accept `timeout`.
|
|
56
|
+
`search-archive` searches text already saved locally (knowledge archive and research evidence) without network access; `--url` filters by URL substring. `fetch-web` returns one selected text per URL in `pages`, with all attempt statuses and archive paths. `--raw` returns the full provider envelopes. Its `--timeout` applies per provider HTTP request; discovery and fallback can take longer overall. `research fetch` accepts only its documented MCP JSON fields and does not accept `timeout`.
|
|
55
57
|
|
|
56
58
|
`providers` describes configuration only; `--probe` checks online handshake and tool discovery. Anonymous Exa/Parallel access depends on current provider limits. Set `EXA_API_KEY` / `PARALLEL_API_KEY` before starting the process when needed. Optional `--provider tavily` uses Bearer with `TAVILY_API_KEY` or an explicit keyless header without it. A listed tool does not prove execution authorization. Never place keys in manifests or reports.
|
|
57
59
|
|