dsh-moli 1.0.4 → 1.0.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +132 -151
- package/README.zh.md +108 -126
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -3,38 +3,72 @@
|
|
|
3
3
|
[](https://www.npmjs.com/package/dsh-moli)
|
|
4
4
|
[](LICENSE)
|
|
5
5
|
|
|
6
|
-
|
|
7
|
-
browser for AI agents
|
|
6
|
+
**Two tools that let a DeepSeek Harness agent drive a real headless browser.** They wrap
|
|
7
|
+
[Moli](https://github.com/lexmount/moli) — a Rust headless browser built for AI agents — so an agent can
|
|
8
|
+
read pages that only exist after JavaScript runs, and can produce pictures of them.
|
|
8
9
|
|
|
9
10
|
English | [中文](README.zh.md)
|
|
10
11
|
|
|
11
|
-
##
|
|
12
|
+
## What it does, and when you want it
|
|
12
13
|
|
|
13
|
-
|
|
14
|
+
A plain HTTP fetch returns the HTML the server sent. That is enough for articles, docs and APIs, and
|
|
15
|
+
DSH already covers it with `web_fetch`. It is not enough for anything a browser has to build:
|
|
16
|
+
|
|
17
|
+
- **Single-page apps** — a client-rendered news site, dashboard or docs site. The response is an empty
|
|
18
|
+
shell; the content appears only after scripts run.
|
|
19
|
+
- **Logins and consent walls** — a page that needs a session or cookie before it shows anything.
|
|
20
|
+
- **Pictures** — a screenshot to see what a page looks like, or a PDF to keep.
|
|
21
|
+
- **One value from deep inside a page** — a price, a count, a title, read with a selector instead of
|
|
22
|
+
pulling the whole document.
|
|
23
|
+
|
|
24
|
+
For those jobs this plugin gives the agent two tools:
|
|
25
|
+
|
|
26
|
+
| Tool | What it does |
|
|
27
|
+
|---|---|
|
|
28
|
+
| `moli_fetch` | Renders a URL and returns Markdown, HTML, a semantic tree, or a JSON envelope carrying the HTTP status; evaluates a JavaScript expression; writes a PNG screenshot or a PDF |
|
|
29
|
+
| `moli_probe` | Reports whether the engine is installed, which version, and whether HTTPS works — and installs Moli when it is missing |
|
|
30
|
+
|
|
31
|
+
The practical question is whether the agent needs the page as a **browser** would see it. If a plain
|
|
32
|
+
fetch already returns the text you want, `web_fetch` is lighter and faster.
|
|
33
|
+
|
|
34
|
+
```
|
|
35
|
+
agent ──moli_fetch──▶ dsh-moli plugin ──spawns──▶ moli.exe
|
|
36
|
+
▲ │
|
|
37
|
+
│ renders the page,
|
|
38
|
+
│ runs its JavaScript
|
|
39
|
+
│ │
|
|
40
|
+
└──markdown / json / png / pdf◀┘
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
## Install
|
|
44
|
+
|
|
45
|
+
Two routes. Both install the same plugin; pick by what your network reaches.
|
|
46
|
+
|
|
47
|
+
**Route 1 — from npm (recommended).** Resolves through the npm registry, so it works where GitHub is
|
|
48
|
+
slow or blocked, and DSH's install dialog can fall back to a regional mirror:
|
|
14
49
|
|
|
15
50
|
```powershell
|
|
16
51
|
dsh plugin --profile web add dsh-moli
|
|
17
52
|
```
|
|
18
53
|
|
|
19
|
-
|
|
20
|
-
is slow or unreachable, and DSH's install dialog can fall back to a regional mirror. Installing
|
|
21
|
-
straight from the repository works too, and needs GitHub to be reachable:
|
|
54
|
+
**Route 2 — straight from the repository.** Tracks the source directly; needs GitHub to be reachable:
|
|
22
55
|
|
|
23
56
|
```powershell
|
|
24
57
|
dsh plugin --profile web add github:27painter/dsh-moli
|
|
25
58
|
```
|
|
26
59
|
|
|
27
60
|
> [!IMPORTANT]
|
|
28
|
-
> **
|
|
29
|
-
>
|
|
30
|
-
> plugin
|
|
61
|
+
> **This package is the plugin only — it does not include the Moli engine.** It is 26 kB of
|
|
62
|
+
> JavaScript; `moli.exe` is a separate ~101 MB Rust binary with its own release process. Installing the
|
|
63
|
+
> plugin does not install Moli, and `moli_fetch` reports that the engine is missing until you install
|
|
64
|
+
> it.
|
|
31
65
|
>
|
|
32
|
-
> **There is no npm package for
|
|
33
|
-
>
|
|
34
|
-
>
|
|
35
|
-
>
|
|
66
|
+
> **There is no npm package for the engine.** The registry names `moli` and `moli-cli` belong to
|
|
67
|
+
> unrelated projects, and `@lexmount/moli` does not exist. Moli ships as prebuilt binaries on
|
|
68
|
+
> [GitHub Releases](https://github.com/lexmount/moli/releases/latest) — do not install the engine by
|
|
69
|
+
> guessing a package name.
|
|
36
70
|
|
|
37
|
-
Either
|
|
71
|
+
Either route ends the same way. Restart DSH, then check the engine and fetch a page:
|
|
38
72
|
|
|
39
73
|
```
|
|
40
74
|
moli_probe
|
|
@@ -43,15 +77,15 @@ moli_fetch url=https://example.com dump=markdown
|
|
|
43
77
|
|
|
44
78
|
### Getting the Moli engine
|
|
45
79
|
|
|
46
|
-
`moli_probe` reports whether
|
|
80
|
+
`moli_probe` reports whether the engine is present. If it is not, pick one of:
|
|
47
81
|
|
|
48
|
-
**1. Let the plugin install it** — several download routes
|
|
82
|
+
**1. Let the plugin install it** — tries several download routes and reports a SHA256 of what it ran:
|
|
49
83
|
|
|
50
84
|
```
|
|
51
85
|
moli_probe action=install
|
|
52
86
|
```
|
|
53
87
|
|
|
54
|
-
**2. Official one-line installer** (Moli's own documented
|
|
88
|
+
**2. Official one-line installer** (Moli's own documented commands):
|
|
55
89
|
|
|
56
90
|
```powershell
|
|
57
91
|
# Windows
|
|
@@ -63,168 +97,110 @@ powershell -ExecutionPolicy ByPass -c "irm https://github.com/lexmount/moli/rele
|
|
|
63
97
|
curl --proto '=https' --tlsv1.2 -fsSL https://github.com/lexmount/moli/releases/latest/download/moli-installer.sh | sh
|
|
64
98
|
```
|
|
65
99
|
|
|
66
|
-
**3. Direct binary
|
|
100
|
+
**3. Direct binary,** if you would rather not run an installer:
|
|
67
101
|
|
|
68
102
|
```powershell
|
|
69
|
-
# pick the asset
|
|
103
|
+
# pick the asset matching your architecture from the releases page
|
|
70
104
|
curl.exe -L -o moli.zip https://github.com/lexmount/moli/releases/latest/download/moli-x86_64-pc-windows-msvc.zip
|
|
71
105
|
Expand-Archive moli.zip -DestinationPath "$env:LOCALAPPDATA\Moli" -Force
|
|
72
106
|
```
|
|
73
107
|
|
|
74
108
|
Those `releases/latest/download/...` URLs always resolve to the newest release, so they stay valid
|
|
75
|
-
without pinning a version.
|
|
76
|
-
got against GitHub's published digest, and [Manual install of Moli](#manual-install-of-moli) for the
|
|
77
|
-
same commands in context.
|
|
109
|
+
without pinning a version.
|
|
78
110
|
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
| Tool | Purpose |
|
|
82
|
-
|---|---|
|
|
83
|
-
| `moli_fetch` | Fetch a URL and return rendered HTML, Markdown, the semantic tree, or a JSON envelope (`status`, `final_url`); evaluate a JavaScript expression; write a PNG screenshot or a PDF |
|
|
84
|
-
| `moli_probe` | Report the engine path and version and whether HTTPS works; with `action: "install"`, download and install Moli first |
|
|
85
|
-
|
|
86
|
-
`moli_fetch` parameters: `url` (required), `dump`, `writeTo`, `eval`, `wait`, `waitSelector`,
|
|
87
|
-
`delayMs`, `timeoutMs`, `maxChars`, `headers`, `userAgent`, `layout`, `scrollbars`, `noJs`,
|
|
88
|
-
`noCss`, `resources`, `withFrames`, `insecureTls`, `profileDir`, `cookieFile`.
|
|
111
|
+
After any of them, point the plugin at the result if it landed somewhere unusual:
|
|
89
112
|
|
|
90
|
-
```
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
moli_fetch url=https://example.com eval="document.title" # one computed value
|
|
94
|
-
moli_fetch url=https://example.com dump=screenshot writeTo=/abs/path/a.png
|
|
95
|
-
moli_fetch url=https://example.com dump=pdf writeTo=/abs/path/a.pdf
|
|
113
|
+
```yaml
|
|
114
|
+
config:
|
|
115
|
+
exePath: 'C:\path\to\moli.exe'
|
|
96
116
|
```
|
|
97
117
|
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
Moli ships official agent skills in its repository — [`moli-webfetch`](https://github.com/lexmount/moli/tree/main/skills/moli-webfetch),
|
|
101
|
-
[`moli-websearch`](https://github.com/lexmount/moli/tree/main/skills/moli-websearch), and
|
|
102
|
-
[`moli-cdp-server`](https://github.com/lexmount/moli/tree/main/skills/moli-cdp-server) — and their
|
|
103
|
-
README's quick start is to have an agent install them. If your agent loads skills, the simplest path
|
|
104
|
-
is to point it at those; they are maintained upstream and cover web search and a CDP server too.
|
|
118
|
+
#### Verifying the download
|
|
105
119
|
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
- **Always-on tool schemas instead of an on-demand skill.** The tools are registered at Host startup,
|
|
109
|
-
so the model sees them every turn with no discovery step.
|
|
110
|
-
- **DSH-specific failure handling.** The three traps below (sandbox pipe capture, UTF-16 mangling of
|
|
111
|
-
binary output, bare-specifier resolution for a `link:`ed plugin) plus sandbox/revocation/proxy
|
|
112
|
-
classification are solved here, because they are DSH-on-Windows problems.
|
|
113
|
-
- **Smaller surface.** Only `fetch` is wrapped; the official skills also cover web search and the CDP
|
|
114
|
-
server, which this plugin does not.
|
|
115
|
-
|
|
116
|
-
Both can coexist: install the skills if you want upstream coverage, keep this plugin if you want the
|
|
117
|
-
capabilities always visible to the model.
|
|
118
|
-
|
|
119
|
-
## Installation
|
|
120
|
-
|
|
121
|
-
The plugin is installed into a DSH profile. Pick the route that matches your machine.
|
|
122
|
-
|
|
123
|
-
### Route A — Moli is already installed
|
|
124
|
-
|
|
125
|
-
Nothing else to fetch. The plugin finds `moli.exe` on its own and both tools work immediately.
|
|
120
|
+
Moli's README documents no checksum or signature, and the installer publishes no checksum file. What
|
|
121
|
+
does exist is GitHub's own per-asset `sha256` digest, readable from the release API:
|
|
126
122
|
|
|
127
123
|
```powershell
|
|
128
|
-
|
|
124
|
+
(irm https://api.github.com/repos/lexmount/moli/releases/latest).assets | Select-Object name, digest
|
|
129
125
|
```
|
|
130
126
|
|
|
131
|
-
|
|
132
|
-
unreachable for you:
|
|
127
|
+
For `v1.1.17` that reports, among others:
|
|
133
128
|
|
|
134
|
-
```
|
|
135
|
-
|
|
129
|
+
```
|
|
130
|
+
moli-installer.ps1 2,020 sha256:e002f7a6c174d78ae6ed7289aca9f5d3a62ab0b1f6e588606a80c0cb5d37964d
|
|
131
|
+
moli-x86_64-pc-windows-msvc.zip 43.8 MB sha256:5247309837b6eaf6e73bfa18f2a8d8d90473b9e2fe5cfcaaef57d7eee9b074c4
|
|
136
132
|
```
|
|
137
133
|
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
### Route B — Moli is not installed yet
|
|
141
|
-
|
|
142
|
-
Install the plugin the same way, restart, then let it fetch Moli for you:
|
|
134
|
+
Compare it against the file you downloaded:
|
|
143
135
|
|
|
144
136
|
```powershell
|
|
145
|
-
|
|
146
|
-
# restart DSH, then call:
|
|
147
|
-
moli_probe action=install
|
|
137
|
+
(Get-FileHash .\moli-installer.ps1 -Algorithm SHA256).Hash.ToLower()
|
|
148
138
|
```
|
|
149
139
|
|
|
150
|
-
The
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
[Verifying the download](#verifying-the-download). Installation only ever runs when a call asks for
|
|
154
|
-
it; it is never a side effect of using `moli_fetch`.
|
|
155
|
-
|
|
156
|
-
### Installing into the desktop profile
|
|
157
|
-
|
|
158
|
-
`dsh plugin --profile desktop …` is refused — the desktop profile belongs to the Electron app. Either
|
|
159
|
-
edit its `package.json` directly (add a `link:` dependency **and** the bundle entry, then run
|
|
160
|
-
`pnpm install`), or use the in-app plugin page (sidebar → **Plugins** → **Add plugin**), which goes
|
|
161
|
-
through the same pnpm path.
|
|
162
|
-
|
|
163
|
-
A profile plugin needs both halves: a dependency edge and a `dsh.profile.bundles` entry. One without
|
|
164
|
-
the other leaves the plugin invisible to the loader, and an unresolvable bundle is **skipped
|
|
165
|
-
silently** apart from one stderr line (`skipping profile bundle …`). If the tools never appear, that
|
|
166
|
-
line is the first thing to look for.
|
|
140
|
+
The plugin follows the same discipline: `moli_probe action=install` reports the SHA256 of the bytes it
|
|
141
|
+
ran, and `expectedSha256` turns that report into an enforced gate. It does not invent a trusted
|
|
142
|
+
default, because a digest only means something when compared against the release page's own value.
|
|
167
143
|
|
|
168
|
-
|
|
144
|
+
This is a checked property rather than a claim: downloading `v1.1.17`'s installer through the plugin's
|
|
145
|
+
own `downloadInstaller` reproduced `e002f7a6…964d` exactly — the digest GitHub publishes — and
|
|
146
|
+
re-hashing the file from disk matched as well.
|
|
169
147
|
|
|
170
|
-
|
|
148
|
+
### Installing into a desktop profile
|
|
171
149
|
|
|
172
|
-
|
|
150
|
+
`dsh plugin --profile desktop …` is refused — the desktop profile belongs to the Electron app. Either
|
|
151
|
+
edit its `package.json` directly, or use the in-app plugin page (sidebar → **Plugins** → **Add
|
|
152
|
+
plugin**), which installs through the same pnpm path.
|
|
173
153
|
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
154
|
+
A profile plugin needs both halves: a dependency edge (`dependencies`) and a
|
|
155
|
+
`dsh.profile.bundles` entry. One without the other leaves the plugin invisible to the loader. From a
|
|
156
|
+
local checkout the four steps are:
|
|
177
157
|
|
|
178
|
-
|
|
158
|
+
1. Put this directory anywhere, e.g. `D:\dsh_plugins\dsh-moli`.
|
|
159
|
+
2. Add both entries to the profile's `package.json`:
|
|
160
|
+
- `dependencies`: `"dsh-moli": "link:D:/dsh_plugins/dsh-moli"`
|
|
161
|
+
- `dsh.profile.bundles`: `"dsh-moli"`
|
|
162
|
+
3. Run `pnpm install` in the profile directory (creates `node_modules/dsh-moli`).
|
|
163
|
+
4. Restart DSH.
|
|
179
164
|
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
```
|
|
165
|
+
An unresolvable bundle is **skipped silently** apart from one stderr line
|
|
166
|
+
(`skipping profile bundle …`). If the tools never appear, that line is the first thing to look for.
|
|
183
167
|
|
|
184
|
-
|
|
168
|
+
## Usage
|
|
185
169
|
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
Expand-Archive moli.zip -DestinationPath "$env:LOCALAPPDATA\Moli" -Force
|
|
190
|
-
```
|
|
191
|
-
|
|
192
|
-
After any of these, point the plugin at the result if it landed somewhere unusual:
|
|
170
|
+
`moli_fetch` parameters: `url` (required), `dump`, `writeTo`, `eval`, `wait`, `waitSelector`,
|
|
171
|
+
`delayMs`, `timeoutMs`, `maxChars`, `headers`, `userAgent`, `layout`, `scrollbars`, `noJs`,
|
|
172
|
+
`noCss`, `resources`, `withFrames`, `insecureTls`, `profileDir`, `cookieFile`.
|
|
193
173
|
|
|
194
|
-
```yaml
|
|
195
|
-
config:
|
|
196
|
-
exePath: 'C:\path\to\moli.exe'
|
|
197
174
|
```
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
202
|
-
|
|
203
|
-
|
|
204
|
-
```powershell
|
|
205
|
-
(irm https://api.github.com/repos/lexmount/moli/releases/latest).assets | Select-Object name, digest
|
|
175
|
+
moli_fetch url=https://example.com dump=markdown # token-efficient text
|
|
176
|
+
moli_fetch url=https://example.com dump=semantic_tree_text # nodes with backend ids
|
|
177
|
+
moli_fetch url=https://example.com eval="document.title" # one computed value
|
|
178
|
+
moli_fetch url=https://example.com dump=screenshot writeTo=/abs/path/a.png
|
|
179
|
+
moli_fetch url=https://example.com dump=pdf writeTo=/abs/path/a.pdf
|
|
206
180
|
```
|
|
207
181
|
|
|
208
|
-
|
|
182
|
+
Screenshot and PDF dumps need `writeTo`; the bytes are written verbatim rather than printed.
|
|
209
183
|
|
|
210
|
-
|
|
211
|
-
moli-installer.ps1 2,020 sha256:e002f7a6c174d78ae6ed7289aca9f5d3a62ab0b1f6e588606a80c0cb5d37964d
|
|
212
|
-
moli-x86_64-pc-windows-msvc.zip 43.8 MB sha256:5247309837b6eaf6e73bfa18f2a8d8d90473b9e2fe5cfcaaef57d7eee9b074c4
|
|
213
|
-
```
|
|
184
|
+
## Relationship to Moli's own agent skills
|
|
214
185
|
|
|
215
|
-
|
|
186
|
+
Moli ships official agent skills in its repository — [`moli-webfetch`](https://github.com/lexmount/moli/tree/main/skills/moli-webfetch),
|
|
187
|
+
[`moli-websearch`](https://github.com/lexmount/moli/tree/main/skills/moli-websearch), and
|
|
188
|
+
[`moli-cdp-server`](https://github.com/lexmount/moli/tree/main/skills/moli-cdp-server) — and their
|
|
189
|
+
README's quick start is to have an agent install them. If your agent loads skills, the simplest path
|
|
190
|
+
is to point it at those; they are maintained upstream and cover web search and a CDP server too.
|
|
216
191
|
|
|
217
|
-
|
|
218
|
-
(Get-FileHash .\moli-installer.ps1 -Algorithm SHA256).Hash.ToLower()
|
|
219
|
-
```
|
|
192
|
+
This plugin exists because it is a different trade-off, not a replacement:
|
|
220
193
|
|
|
221
|
-
|
|
222
|
-
|
|
223
|
-
|
|
194
|
+
- **Always-on tool schemas instead of an on-demand skill.** The tools are registered at Host startup,
|
|
195
|
+
so the model sees them every turn with no discovery step.
|
|
196
|
+
- **DSH-specific failure handling.** The three traps below (sandbox pipe capture, UTF-16 mangling of
|
|
197
|
+
binary output, bare-specifier resolution for a `link:`ed plugin) plus sandbox/revocation/proxy
|
|
198
|
+
classification are solved here, because they are DSH-on-Windows problems.
|
|
199
|
+
- **Smaller surface.** Only `fetch` is wrapped; the official skills also cover web search and the CDP
|
|
200
|
+
server, which this plugin does not.
|
|
224
201
|
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
re-hashing the file from disk matched too.
|
|
202
|
+
Both can coexist: install the skills if you want upstream coverage, keep this plugin if you want the
|
|
203
|
+
capabilities always visible to the model.
|
|
228
204
|
|
|
229
205
|
## Three design points
|
|
230
206
|
|
|
@@ -252,7 +228,7 @@ Override any field under the plugin's `config` entry in the profile patch layer.
|
|
|
252
228
|
defaultTimeoutMs: 30000 # per-fetch readiness deadline
|
|
253
229
|
maxChars: 200000 # inline text truncation
|
|
254
230
|
proxy: 'http://127.0.0.1:7890' # forward proxy for moli and downloads
|
|
255
|
-
insecureTls: true # see TLS notes below
|
|
231
|
+
insecureTls: true # see the TLS notes below
|
|
256
232
|
```
|
|
257
233
|
|
|
258
234
|
Without `exePath`, the plugin probes `exePath` → `MOLI_BIN` → `%LOCALAPPDATA%\Moli\bin\moli.exe` →
|
|
@@ -291,20 +267,25 @@ installation under a versioned subdirectory is still found.
|
|
|
291
267
|
|
|
292
268
|
## Verification
|
|
293
269
|
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
|
|
297
|
-
|
|
270
|
+
Two suites live in [`verify/`](verify/), and both are runnable against a working tree:
|
|
271
|
+
|
|
272
|
+
- `schema-verify.mjs` checks the tool definitions with the registry's own validator, including
|
|
273
|
+
controls that assert the two invalid schema shapes still fail.
|
|
274
|
+
- `install-verify.mjs` drives the install pipeline against a loopback HTTP server: route ordering,
|
|
275
|
+
the SHA256 of served bytes, refusal to execute on a checksum mismatch, actual execution when the
|
|
276
|
+
digest matches, and the all-routes-failed diagnostic.
|
|
277
|
+
|
|
278
|
+
Beyond those, `lib/moli.js` and the tool layer were exercised end to end against Moli 1.1.16 on
|
|
279
|
+
Windows: registration, probe, Markdown and JSON dumps, `eval` (including a tight-timeout regression
|
|
280
|
+
guard for the `-t` argument position), PNG and PDF artifacts with valid signatures, truncation, every
|
|
281
|
+
rejection path (binary dump without `writeTo`, unknown `dump`, unknown `wait` stage, missing
|
|
298
282
|
destination directory), failure classification (unreachable host, HTTP status, sandbox TLS block,
|
|
299
283
|
revocation block), and the concurrency claims. The blocked-host classifier is covered by pure
|
|
300
284
|
function tests using stderr text captured from real runs.
|
|
301
285
|
|
|
302
|
-
Confirmed at runtime
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
The install pipeline is covered against a loopback HTTP server, which proves the mechanics without
|
|
306
|
-
depending on GitHub: route ordering, the SHA256 of served bytes, refusal to execute on a checksum
|
|
307
|
-
mismatch, actual execution when the digest matches, and the all-routes-failed diagnostic.
|
|
286
|
+
Confirmed at runtime too: a JavaScript-rendered page (GitHub search) returned 14,560 characters of
|
|
287
|
+
Markdown containing 30 real repository matches when fetched through a local proxy. A throwaway
|
|
288
|
+
profile with the plugin installed composes its config with no `JsonSchemaError` and no skipped bundle.
|
|
308
289
|
|
|
309
290
|
## License
|
|
310
291
|
|
package/README.zh.md
CHANGED
|
@@ -1,30 +1,63 @@
|
|
|
1
1
|
# dsh-moli
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**两个工具,让 DeepSeek Harness 的 agent 能驱动真正的无头浏览器。** 它们封装
|
|
4
|
+
[Moli](https://github.com/lexmount/moli)——一个专为 AI agent 打造的 Rust 无头浏览器——所以 agent 能看到
|
|
5
|
+
那些"必须跑完 JavaScript 才存在"的页面内容,也能把它们拍成图。
|
|
4
6
|
|
|
5
7
|
[English](README.md) | 中文
|
|
6
8
|
|
|
7
|
-
##
|
|
9
|
+
## 它做什么,什么时候需要它
|
|
8
10
|
|
|
9
|
-
|
|
11
|
+
普通的 HTTP 抓取拿回的是服务器发来的 HTML。对文章、文档、API 来说这足够了,DSH 的 `web_fetch` 已经覆盖
|
|
12
|
+
这种场景。但它拿不到任何**需要浏览器构建出来**的内容:
|
|
13
|
+
|
|
14
|
+
- **单页应用(SPA)**——客户端渲染的新闻站、仪表盘、文档站。响应只是一个空壳,内容要等脚本跑完才出现。
|
|
15
|
+
- **登录与同意墙**——必须先有会话或 cookie 才肯显示内容的页面。
|
|
16
|
+
- **图**——想看一眼页面长什么样(截图),或想存档(PDF)。
|
|
17
|
+
- **页面深处的一个值**——价格、计数、标题,用选择器直接取,而不是把整篇文档拉回来。
|
|
18
|
+
|
|
19
|
+
针对这些场景,本插件给 agent 两个工具:
|
|
20
|
+
|
|
21
|
+
| 工具 | 作用 |
|
|
22
|
+
|---|---|
|
|
23
|
+
| `moli_fetch` | 渲染 URL 并返回 Markdown / HTML / 语义树 / 带 HTTP 状态的 JSON;执行 JS 表达式取值;写出 PNG 截图或 PDF |
|
|
24
|
+
| `moli_probe` | 报告引擎是否安装、什么版本、HTTPS 是否可用;没装时还能替你装上 |
|
|
25
|
+
|
|
26
|
+
判断标准很简单:**agent 需要的是"浏览器眼里的页面"吗?** 如果普通抓取已经能拿到你要的文字,`web_fetch`
|
|
27
|
+
更轻更快。
|
|
28
|
+
|
|
29
|
+
```
|
|
30
|
+
agent ──moli_fetch──▶ dsh-moli 插件 ──启动──▶ moli.exe
|
|
31
|
+
▲ │
|
|
32
|
+
│ 渲染页面、
|
|
33
|
+
│ 执行其中 JavaScript
|
|
34
|
+
│ │
|
|
35
|
+
└──markdown / json / png / pdf◀┘
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
## 安装
|
|
39
|
+
|
|
40
|
+
两条路,装的是同一个插件,按你的网络情况选。
|
|
41
|
+
|
|
42
|
+
**路线 1 —— 从 npm 装(推荐)。** 经 npm registry 解析,所以在 GitHub 慢或不通的网络里照样能装,DSH 的
|
|
43
|
+
安装对话框还能回退到国内镜像:
|
|
10
44
|
|
|
11
45
|
```powershell
|
|
12
46
|
dsh plugin --profile web add dsh-moli
|
|
13
47
|
```
|
|
14
48
|
|
|
15
|
-
|
|
16
|
-
回退到国内镜像。直接从仓库装也可以,但需要 GitHub 可达:
|
|
49
|
+
**路线 2 —— 直接从仓库装。** 直接跟踪源码,需要 GitHub 可达:
|
|
17
50
|
|
|
18
51
|
```powershell
|
|
19
52
|
dsh plugin --profile web add github:27painter/dsh-moli
|
|
20
53
|
```
|
|
21
54
|
|
|
22
55
|
> [!IMPORTANT]
|
|
23
|
-
>
|
|
24
|
-
>
|
|
56
|
+
> **这个包只有插件,不含 Moli 引擎。** 它是 26 kB 的 JavaScript;`moli.exe` 是另一个约 101 MB 的 Rust
|
|
57
|
+
> 二进制、有自己的发布流程。装插件**不会**顺带装上 Moli,在你装引擎之前 `moli_fetch` 会一直报"引擎缺失"。
|
|
25
58
|
>
|
|
26
|
-
>
|
|
27
|
-
>
|
|
59
|
+
> **引擎没有 npm 包。** registry 上的 `moli` 和 `moli-cli` 属于无关项目,`@lexmount/moli` 并不存在。
|
|
60
|
+
> Moli 以预编译二进制形式发布在
|
|
28
61
|
> [GitHub Releases](https://github.com/lexmount/moli/releases/latest)——**不要靠猜包名去装引擎**。
|
|
29
62
|
|
|
30
63
|
两种方式都要重启 DSH,然后自检并抓一个页面:
|
|
@@ -36,9 +69,9 @@ moli_fetch url=https://example.com dump=markdown
|
|
|
36
69
|
|
|
37
70
|
### 获取 Moli 引擎
|
|
38
71
|
|
|
39
|
-
`moli_probe`
|
|
72
|
+
`moli_probe` 会报告引擎是否在场。如果不在,三种方式任选:
|
|
40
73
|
|
|
41
|
-
**1.
|
|
74
|
+
**1. 让插件替你装**——多路径下载,并报告实际执行字节的 SHA256:
|
|
42
75
|
|
|
43
76
|
```
|
|
44
77
|
moli_probe action=install
|
|
@@ -64,152 +97,94 @@ curl.exe -L -o moli.zip https://github.com/lexmount/moli/releases/latest/downloa
|
|
|
64
97
|
Expand-Archive moli.zip -DestinationPath "$env:LOCALAPPDATA\Moli" -Force
|
|
65
98
|
```
|
|
66
99
|
|
|
67
|
-
这些 `releases/latest/download/...`
|
|
68
|
-
见[校验下载](#校验下载);同样的命令放在上下文里,见[手动安装 Moli](#手动安装-moli)。
|
|
69
|
-
|
|
70
|
-
## 与 Moli 官方 agent skill 的关系
|
|
71
|
-
|
|
72
|
-
Moli 仓库自带官方 agent skill:[`moli-webfetch`](https://github.com/lexmount/moli/tree/main/skills/moli-webfetch)、
|
|
73
|
-
[`moli-websearch`](https://github.com/lexmount/moli/tree/main/skills/moli-websearch)、
|
|
74
|
-
[`moli-cdp-server`](https://github.com/lexmount/moli/tree/main/skills/moli-cdp-server),其 README 的快速开始
|
|
75
|
-
就是让 agent 去安装它们。如果你的 agent 能加载 skill,最省事的路就是直接用官方那套——上游维护,而且额外覆盖网页
|
|
76
|
-
搜索与 CDP 服务。
|
|
77
|
-
|
|
78
|
-
本插件的存在是因为它是另一种取舍,不是替代品:
|
|
100
|
+
这些 `releases/latest/download/...` 链接始终指向最新发布,所以不用钉版本号也不会失效。
|
|
79
101
|
|
|
80
|
-
|
|
81
|
-
- **针对 DSH 的失败处理。** 下面那三个坑(沙箱管道捕获、二进制被 UTF-16 破坏、`link:` 插件的裸包名解析)
|
|
82
|
-
以及沙箱/吊销/代理三类失败分类,都在这里解决了——因为它们本质是"DSH on Windows"的问题。
|
|
83
|
-
- **面更窄。** 只封装了 `fetch`;官方 skill 还覆盖网页搜索和 CDP 服务,本插件没有。
|
|
84
|
-
|
|
85
|
-
两者可以共存:想要上游覆盖就装 skill,想要能力始终对模型可见就留这个插件。
|
|
86
|
-
|
|
87
|
-
## 提供的能力
|
|
88
|
-
|
|
89
|
-
| 工具 | 作用 |
|
|
90
|
-
|---|---|
|
|
91
|
-
| `moli_fetch` | 抓取 URL 并返回渲染后的 HTML / Markdown / 语义树 / JSON(含 `status` 与 `final_url`);执行 JS 表达式取值;写出 PNG 截图或 PDF |
|
|
92
|
-
| `moli_probe` | 报告引擎路径与版本,以及调用进程能否走 HTTPS |
|
|
93
|
-
|
|
94
|
-
`moli_fetch` 参数:`url`(必填)、`dump`、`writeTo`、`eval`、`wait`、`waitSelector`、`delayMs`、
|
|
95
|
-
`timeoutMs`、`maxChars`、`headers`、`userAgent`、`layout`、`scrollbars`、`noJs`、`noCss`、
|
|
96
|
-
`resources`、`withFrames`、`insecureTls`、`profileDir`、`cookieFile`。
|
|
102
|
+
装完若位置不常规,把插件指过去:
|
|
97
103
|
|
|
104
|
+
```yaml
|
|
105
|
+
config:
|
|
106
|
+
exePath: 'C:\path\to\moli.exe'
|
|
98
107
|
```
|
|
99
|
-
moli_fetch url=https://example.com dump=markdown # 省 token 的正文
|
|
100
|
-
moli_fetch url=https://example.com dump=semantic_tree_text # 带节点 id 的结构
|
|
101
|
-
moli_fetch url=https://example.com eval="document.title" # 取一个值
|
|
102
|
-
moli_fetch url=https://example.com dump=screenshot writeTo=/绝对路径/a.png
|
|
103
|
-
moli_fetch url=https://example.com dump=pdf writeTo=/绝对路径/a.pdf
|
|
104
|
-
```
|
|
105
|
-
|
|
106
|
-
## 安装
|
|
107
|
-
|
|
108
|
-
插件装进某个 DSH profile。按你的机器情况选一条路。
|
|
109
108
|
|
|
110
|
-
|
|
109
|
+
#### 校验下载
|
|
111
110
|
|
|
112
|
-
|
|
111
|
+
Moli 的 README 没有记录任何校验和或签名,安装器也不发布校验和文件。**存在的是 GitHub 自己给每个资产的
|
|
112
|
+
`sha256` 摘要**,可以从 release API 读到:
|
|
113
113
|
|
|
114
114
|
```powershell
|
|
115
|
-
|
|
115
|
+
(irm https://api.github.com/repos/lexmount/moli/releases/latest).assets | Select-Object name, digest
|
|
116
116
|
```
|
|
117
117
|
|
|
118
|
-
|
|
118
|
+
`v1.1.17` 下这个命令会给出(节选):
|
|
119
119
|
|
|
120
|
-
```
|
|
121
|
-
|
|
120
|
+
```
|
|
121
|
+
moli-installer.ps1 2,020 sha256:e002f7a6c174d78ae6ed7289aca9f5d3a62ab0b1f6e588606a80c0cb5d37964d
|
|
122
|
+
moli-x86_64-pc-windows-msvc.zip 43.8 MB sha256:5247309837b6eaf6e73bfa18f2a8d8d90473b9e2fe5cfcaaef57d7eee9b074c4
|
|
122
123
|
```
|
|
123
124
|
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
### 路线 B —— 还没装 Moli
|
|
127
|
-
|
|
128
|
-
同样方式装插件、重启,然后让它替你把 Moli 拉下来:
|
|
125
|
+
与你下载的文件比对:
|
|
129
126
|
|
|
130
127
|
```powershell
|
|
131
|
-
|
|
132
|
-
# 重启 DSH,然后调用:
|
|
133
|
-
moli_probe action=install
|
|
128
|
+
(Get-FileHash .\moli-installer.ps1 -Algorithm SHA256).Hash.ToLower()
|
|
134
129
|
```
|
|
135
130
|
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
131
|
+
插件遵循同样的纪律:`moli_probe action=install` 会报告它实际执行的那份字节的 SHA256,而
|
|
132
|
+
`expectedSha256` 能把这份报告变成强制校验门。它**不会**编造一个可信默认值——因为摘要只有在与发布页自己
|
|
133
|
+
的值比对时才有意义。
|
|
134
|
+
|
|
135
|
+
这是**验证过的性质,不是声明**:用插件自己的 `downloadInstaller` 下载 `v1.1.17` 的安装器,复现出的正是
|
|
136
|
+
GitHub 公布的那个摘要 `e002f7a6…964d`,从磁盘重新哈希也对得上。
|
|
139
137
|
|
|
140
138
|
### 装进 desktop profile
|
|
141
139
|
|
|
142
140
|
`dsh plugin --profile desktop …` 会被拒绝——desktop profile 归 Electron 应用所有。直接改它的
|
|
143
|
-
`package.json
|
|
144
|
-
(侧边栏 →**插件**→**添加插件**),那条路径同样走 pnpm。
|
|
141
|
+
`package.json`,或用应用内插件页(侧边栏 →**插件**→**添加插件**),那条路径同样走 pnpm。
|
|
145
142
|
|
|
146
|
-
profile
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
从本地目录安装时,完整的四步是:
|
|
143
|
+
profile 插件需要两半都齐:依赖边(`dependencies`)+ `dsh.profile.bundles` 条目。缺一个 loader 就看不到
|
|
144
|
+
它。从本地目录安装的完整四步是:
|
|
150
145
|
|
|
151
146
|
1. 把本目录放到任意位置,例如 `D:\dsh_plugins\dsh-moli`。
|
|
152
|
-
2. 在该 profile 的 `package.json`
|
|
147
|
+
2. 在该 profile 的 `package.json` 中加入两项:
|
|
153
148
|
- `dependencies`: `"dsh-moli": "link:D:/dsh_plugins/dsh-moli"`
|
|
154
149
|
- `dsh.profile.bundles`: `"dsh-moli"`
|
|
155
150
|
3. 在 profile 目录执行 `pnpm install`(生成 `node_modules/dsh-moli`)。
|
|
156
151
|
4. 重启 DSH。
|
|
157
152
|
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
自动路径都失败时(受限网络可能把 release 主机整个挡掉),自己装。下面是 Moli 官方文档里的命令。
|
|
153
|
+
无法解析的 bundle 会被**静默跳过**,只在 stderr 留一行 `skipping profile bundle …`。工具迟迟不出现时先找
|
|
154
|
+
这行。
|
|
161
155
|
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
```powershell
|
|
165
|
-
powershell -ExecutionPolicy ByPass -c "irm https://github.com/lexmount/moli/releases/latest/download/moli-installer.ps1 | iex"
|
|
166
|
-
```
|
|
167
|
-
|
|
168
|
-
Linux / macOS:
|
|
169
|
-
|
|
170
|
-
```sh
|
|
171
|
-
curl --proto '=https' --tlsv1.2 -fsSL https://github.com/lexmount/moli/releases/latest/download/moli-installer.sh | sh
|
|
172
|
-
```
|
|
156
|
+
## 用法
|
|
173
157
|
|
|
174
|
-
|
|
158
|
+
`moli_fetch` 参数:`url`(必填)、`dump`、`writeTo`、`eval`、`wait`、`waitSelector`、`delayMs`、
|
|
159
|
+
`timeoutMs`、`maxChars`、`headers`、`userAgent`、`layout`、`scrollbars`、`noJs`、`noCss`、
|
|
160
|
+
`resources`、`withFrames`、`insecureTls`、`profileDir`、`cookieFile`。
|
|
175
161
|
|
|
176
|
-
```powershell
|
|
177
|
-
# 从 releases 页挑与你的架构匹配的资产
|
|
178
|
-
curl.exe -L -o moli.zip https://github.com/lexmount/moli/releases/latest/download/moli-x86_64-pc-windows-msvc.zip
|
|
179
|
-
Expand-Archive moli.zip -DestinationPath "$env:LOCALAPPDATA\Moli" -Force
|
|
180
162
|
```
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
exePath: 'C:\path\to\moli.exe'
|
|
163
|
+
moli_fetch url=https://example.com dump=markdown # 省 token 的正文
|
|
164
|
+
moli_fetch url=https://example.com dump=semantic_tree_text # 带节点 id 的结构
|
|
165
|
+
moli_fetch url=https://example.com eval="document.title" # 取一个值
|
|
166
|
+
moli_fetch url=https://example.com dump=screenshot writeTo=/绝对路径/a.png
|
|
167
|
+
moli_fetch url=https://example.com dump=pdf writeTo=/绝对路径/a.pdf
|
|
187
168
|
```
|
|
188
169
|
|
|
189
|
-
|
|
170
|
+
截图与 PDF 必须给 `writeTo`;字节是原样落盘的,不会打印到输出里。
|
|
190
171
|
|
|
191
|
-
Moli
|
|
192
|
-
`sha256` 摘要**,可以从 release API 读到:
|
|
193
|
-
|
|
194
|
-
```powershell
|
|
195
|
-
(irm https://api.github.com/repos/lexmount/moli/releases/latest).assets | Select-Object name, digest
|
|
196
|
-
```
|
|
197
|
-
|
|
198
|
-
`v1.1.17` 下这个命令会给出(节选):
|
|
172
|
+
## 与 Moli 官方 agent skill 的关系
|
|
199
173
|
|
|
200
|
-
|
|
201
|
-
moli-
|
|
202
|
-
moli-
|
|
203
|
-
|
|
174
|
+
Moli 仓库自带官方 agent skill:[`moli-webfetch`](https://github.com/lexmount/moli/tree/main/skills/moli-webfetch)、
|
|
175
|
+
[`moli-websearch`](https://github.com/lexmount/moli/tree/main/skills/moli-websearch)、
|
|
176
|
+
[`moli-cdp-server`](https://github.com/lexmount/moli/tree/main/skills/moli-cdp-server),其 README 的快速开始
|
|
177
|
+
就是让 agent 去安装它们。如果你的 agent 能加载 skill,最省事的路就是直接用官方那套——上游维护,而且额外覆盖
|
|
178
|
+
网页搜索与 CDP 服务。
|
|
204
179
|
|
|
205
|
-
|
|
180
|
+
本插件的存在是因为它是另一种取舍,不是替代品:
|
|
206
181
|
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
182
|
+
- **常驻工具 schema,而非按需 skill。** 工具在 Host 启动时注册,模型每回合都能看到,不需要发现步骤。
|
|
183
|
+
- **针对 DSH 的失败处理。** 下面那三个坑(沙箱管道捕获、二进制被 UTF-16 破坏、`link:` 插件的裸包名解析)
|
|
184
|
+
以及沙箱/吊销/代理三类失败分类,都在这里解决了——因为它们本质是"DSH on Windows"的问题。
|
|
185
|
+
- **面更窄。** 只封装了 `fetch`;官方 skill 还覆盖网页搜索和 CDP 服务,本插件没有。
|
|
210
186
|
|
|
211
|
-
|
|
212
|
-
`expectedSha256` 能强制校验你提供的摘要。它**不会**编造一个可信默认值——因为上游根本没有校验和可比。
|
|
187
|
+
两者可以共存:想要上游覆盖就装 skill,想要能力始终对模型可见就留这个插件。
|
|
213
188
|
|
|
214
189
|
## 三个设计要点
|
|
215
190
|
|
|
@@ -246,7 +221,7 @@ moli-x86_64-pc-windows-msvc.zip 43.8 MB sha256:5247309837b6eaf6e73bfa18f2a8d8d
|
|
|
246
221
|
|
|
247
222
|
- **面向 Windows。** 开发与验证都在 Windows 上;可执行文件探测、文件描述符绕坑、Schannel 说明都
|
|
248
223
|
与 Windows 相关。移植到 POSIX 没有障碍,但未经验证。
|
|
249
|
-
- **不包含 Moli 本体。**
|
|
224
|
+
- **不包含 Moli 本体。** 需自行安装,或让 `moli_probe action=install` 替你拉取。
|
|
250
225
|
- **只封装了 `fetch`。** `moli serve`(CDP/WebDriver 服务器)与 `moli import`(会话导入)尚未做成工具。
|
|
251
226
|
- **不支持单次取消。** 引擎是同步执行的,被中断的回合无法中途终止抓取;已在进行的调用会跑完并
|
|
252
227
|
写出产物。
|
|
@@ -263,17 +238,24 @@ moli-x86_64-pc-windows-msvc.zip 43.8 MB sha256:5247309837b6eaf6e73bfa18f2a8d8d
|
|
|
263
238
|
- **部分网络下主机可达性会波动。** 开发机上观察到的现象:同一时刻,基于 curl 的客户端(moli)访问
|
|
264
239
|
`github.com` 超时,而 `git` over HTTPS 与系统浏览器几秒内就能通;`api.github.com` 正常而
|
|
265
240
|
`github.com` 不通。所以超时可能是暂时性的——重试,或设置 `proxy`。正因裸超时含义不唯一,它被
|
|
266
|
-
上报为 `possiblyUnreachable`
|
|
241
|
+
上报为 `possiblyUnreachable` 而不当作定论。安装动作的多路径回退与上面的手动命令正是为这种情况准备的。
|
|
267
242
|
|
|
268
243
|
## 验证情况
|
|
269
244
|
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
245
|
+
[`verify/`](verify/) 下有两个可直接对工作区运行的测试:
|
|
246
|
+
|
|
247
|
+
- `schema-verify.mjs` 用注册表自己的校验器检查工具定义,并带有断言"两种非法 schema 形状仍会被拒"的对照组。
|
|
248
|
+
- `install-verify.mjs` 用本地回环 HTTP 服务器驱动安装链路:路由顺序、被提供字节的 SHA256、校验不匹配时
|
|
249
|
+
拒绝执行、匹配时真正执行、以及全部路径失败的诊断。
|
|
250
|
+
|
|
251
|
+
此外,`lib/moli.js` 与工具层在 Windows 上对 Moli 1.1.16 做了端到端验证:注册、probe、Markdown 与 JSON
|
|
252
|
+
dump、`eval`(含针对 `-t` 参数位置的紧超时回归防护)、PNG 与 PDF 产物签名、截断、全部拒绝路径(二元 dump
|
|
253
|
+
缺 `writeTo`、非法 `dump`、非法 `wait`、目标目录不存在)、失败分类(不可达主机、HTTP 状态、沙箱 TLS 阻断、
|
|
254
|
+
吊销阻断)以及并发声明。被阻断主机的分类器用真实运行捕获的 stderr 文本做纯函数测试。
|
|
274
255
|
|
|
275
|
-
运行时也确认过:走本地代理抓取 JavaScript 渲染页面(GitHub 搜索)返回 14,560 字符 Markdown
|
|
276
|
-
|
|
256
|
+
运行时也确认过:走本地代理抓取 JavaScript 渲染页面(GitHub 搜索)返回 14,560 字符 Markdown,其中包含
|
|
257
|
+
30 处真实结果匹配。另有一个装了本插件的一次性 profile,组合其配置时既无 `JsonSchemaError` 也无被跳过的
|
|
258
|
+
bundle。
|
|
277
259
|
|
|
278
260
|
## 许可
|
|
279
261
|
|
package/package.json
CHANGED