dsh-moli 1.1.12 → 1.1.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +92 -111
- package/README.zh.md +81 -97
- package/index.js +30 -299
- package/lib/moli.js +185 -7
- package/package.json +1 -1
- package/lib/actions.js +0 -460
- package/lib/session.js +0 -632
package/README.md
CHANGED
|
@@ -209,9 +209,10 @@ An unresolvable bundle is **skipped silently** apart from one stderr line
|
|
|
209
209
|
|
|
210
210
|
## Usage
|
|
211
211
|
|
|
212
|
-
`moli_fetch` parameters: `url` (required), `dump`, `writeTo`, `eval`, `
|
|
213
|
-
`
|
|
214
|
-
`
|
|
212
|
+
`moli_fetch` parameters: `url` (required), `dump`, `writeTo`, `eval`, `scroll`, `wait`,
|
|
213
|
+
`waitSelector`, `waitScript`, `waitResponseUrl`, `waitResponseJson`, `delayMs`, `timeoutMs`,
|
|
214
|
+
`maxChars`, `headers`, `userAgent`, `layout`, `scrollbars`, `noJs`, `noCss`, `resources`,
|
|
215
|
+
`withFrames`, `insecureTls`, `failOnHttpError`, `profileDir`, `cookieFile`, `proxy`.
|
|
215
216
|
|
|
216
217
|
```
|
|
217
218
|
moli_fetch url=https://example.com dump=markdown # token-efficient text
|
|
@@ -219,105 +220,67 @@ moli_fetch url=https://example.com dump=semantic_tree_text # nodes with backe
|
|
|
219
220
|
moli_fetch url=https://example.com eval="document.title" # one computed value
|
|
220
221
|
moli_fetch url=https://example.com dump=screenshot writeTo=/abs/path/a.png
|
|
221
222
|
moli_fetch url=https://example.com dump=pdf writeTo=/abs/path/a.pdf
|
|
223
|
+
moli_fetch url=https://example.com dump=markdown scroll=800 # trigger lazy loading first
|
|
222
224
|
```
|
|
223
225
|
|
|
224
226
|
Screenshot and PDF dumps need `writeTo`; the bytes are written verbatim rather than printed.
|
|
225
227
|
|
|
226
|
-
### `
|
|
228
|
+
### `scroll` — reach what only appears on a scroll
|
|
227
229
|
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
`
|
|
231
|
-
`
|
|
232
|
-
stop it.
|
|
230
|
+
The engine serializes the whole document, so a plain fetch already contains everything the DOM holds.
|
|
231
|
+
What it cannot contain is content that does not exist yet: a list that loads on the first `scroll`
|
|
232
|
+
event, an `IntersectionObserver` that fires on entry, an image swapped in when it nears the viewport.
|
|
233
|
+
`scroll` runs the scroll before the capture, so those handlers fire first.
|
|
233
234
|
|
|
234
235
|
```
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
moli_session action=type selector="#q" text="moli" submit=true
|
|
239
|
-
moli_session action=scroll dy=1200
|
|
240
|
-
moli_session action=waitFor selector=".results"
|
|
241
|
-
moli_session action=eval expression="document.title"
|
|
242
|
-
moli_session action=screenshot writeTo=/abs/path/a.png
|
|
243
|
-
moli_session action=status # is it running, and on what page
|
|
244
|
-
moli_session action=close # stop it and free the port
|
|
236
|
+
moli_fetch url=https://example.com/feed dump=markdown scroll=600
|
|
237
|
+
moli_fetch url=https://example.com/feed dump=markdown scroll=bottom
|
|
238
|
+
moli_fetch url=https://example.com/feed dump=markdown scroll=-400 # back up
|
|
245
239
|
```
|
|
246
240
|
|
|
247
|
-
|
|
248
|
-
|
|
241
|
+
Measured on a page whose `scroll` handler fills a slot: without `scroll` the dump says
|
|
242
|
+
`STILL-EMPTY`, with `scroll=1000` it says `LAZY-CONTENT-1000`. It also moves the screenshot, so a
|
|
243
|
+
scrolled `dump=screenshot` shows that part of the page.
|
|
249
244
|
|
|
250
|
-
|
|
245
|
+
It runs as part of the engine's own pre-capture evaluation and therefore composes with `eval`: the
|
|
246
|
+
scroll happens first, and the expression still returns its own value.
|
|
251
247
|
|
|
252
|
-
`
|
|
253
|
-
because switching engines when one click fails would make the timing and the behaviour depend on which
|
|
254
|
-
operation happened to fail, and you could not tell which browser answered.
|
|
248
|
+
### `failOnHttpError` — stop reading an error page as content
|
|
255
249
|
|
|
256
|
-
|
|
257
|
-
|
|
258
|
-
|
|
259
|
-
sessionBrowserPath: '' # optional path to chrome.exe / msedge.exe / brave
|
|
260
|
-
sessionPort: 9222
|
|
261
|
-
```
|
|
262
|
-
|
|
263
|
-
**Why the choice exists.** Moli's element geometry is document-absolute and does not follow scrolling,
|
|
264
|
-
and its hit test is bounded by the viewport height. Measured on Moli 1.1.16 and Chrome 155, same page,
|
|
265
|
-
same element:
|
|
266
|
-
|
|
267
|
-
| | `rect.y` at scrollY 0 / 400 / 800 | click a control below the first viewport |
|
|
268
|
-
|---|---|---|
|
|
269
|
-
| **moli** | 1008 / 1008 / 1008 — never moves | **impossible**; synthetic input cannot reach it |
|
|
270
|
-
| **chromium** | 1008 / 608 / 208 — follows the scroll | **works**; the browser scrolls to it |
|
|
271
|
-
|
|
272
|
-
So moli is the good default — it is the engine this plugin wraps, it is lighter, and it is already
|
|
273
|
-
installed — and a real Chromium is what you switch to when a task needs an element further down a long
|
|
274
|
-
page. When moli refuses, it says so, names the element, and points at both routes:
|
|
250
|
+
By default a 404 is returned like any other page, so a failed fetch looks successful. Setting
|
|
251
|
+
`failOnHttpError` turns a final status of 400 or above into `ok: false` with the status named — and
|
|
252
|
+
**keeps the body**, because an error page is often the thing you need to read.
|
|
275
253
|
|
|
276
254
|
```
|
|
277
|
-
|
|
278
|
-
→ "#far-down" is below the viewport (y=2548) and this engine cannot send input there
|
|
279
|
-
|
|
280
|
-
# either click from inside the page (reaches it, but skips hover and focus behaviour)
|
|
281
|
-
moli_session action=eval expression="document.querySelector('#far-down').click()"
|
|
282
|
-
|
|
283
|
-
# or use a real browser for that session
|
|
284
|
-
config: sessionBrowser: chromium
|
|
255
|
+
moli_fetch url=https://example.com/nope dump=markdown failOnHttpError=true
|
|
285
256
|
```
|
|
286
257
|
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
Pass `cdpEndpoint` to attach to a CDP port instead of starting anything. This is the only way to inherit
|
|
293
|
-
your own cookies and logins: start your browser with `--remote-debugging-port=9222`, then
|
|
294
|
-
|
|
295
|
-
```
|
|
296
|
-
moli_session action=open url=https://example.com cdpEndpoint=http://127.0.0.1:9222
|
|
297
|
-
```
|
|
258
|
+
| | default | `failOnHttpError: true` |
|
|
259
|
+
|---|---|---|
|
|
260
|
+
| a 404 | `ok: true`, the error page as content | `ok: false`, `status: 404`, body still returned |
|
|
261
|
+
| a screenshot of a 404 | written as a normal image | still written, and reported as a failure |
|
|
298
262
|
|
|
299
|
-
|
|
300
|
-
|
|
263
|
+
It is off by default so existing calls keep their behaviour. A JSON dump already reported the status
|
|
264
|
+
on its own; markdown and HTML dumps did not, and that is the gap this closes.
|
|
301
265
|
|
|
302
|
-
###
|
|
266
|
+
### Waiting for what the page actually fetches
|
|
303
267
|
|
|
304
|
-
|
|
268
|
+
`wait: "networkidle"` plus `delayMs` is a guess about how long a list takes to arrive. These wait for
|
|
269
|
+
the request itself:
|
|
305
270
|
|
|
306
271
|
```
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
moli_session action=cdp method=DOM.setFileInputFiles params={"files":["/abs/a.png"],"nodeId":42}
|
|
310
|
-
```
|
|
272
|
+
# the response that carries the data
|
|
273
|
+
moli_fetch url=https://example.com/feed dump=markdown waitResponseUrl="/api/feed"
|
|
311
274
|
|
|
312
|
-
|
|
313
|
-
|
|
275
|
+
# the payload, when the URL is not distinctive
|
|
276
|
+
moli_fetch url=https://example.com/feed dump=markdown waitResponseJson="data.list.0.id=42"
|
|
314
277
|
|
|
315
|
-
|
|
316
|
-
|
|
278
|
+
# the page's own idea of readiness
|
|
279
|
+
moli_fetch url=https://example.com/app dump=markdown waitScript="window.appReady === true"
|
|
280
|
+
```
|
|
317
281
|
|
|
318
|
-
`
|
|
319
|
-
|
|
320
|
-
alone, because it is not ours to stop.
|
|
282
|
+
`waitResponseJson` takes `PATH=VALUE`, where `PATH` addresses the field and `VALUE` is compared
|
|
283
|
+
literally. Use these when a fixed delay either wastes time or reads the page too early.
|
|
321
284
|
|
|
322
285
|
## Relationship to Moli's own agent skills
|
|
323
286
|
|
|
@@ -406,26 +369,12 @@ Override any field under the plugin's `config` entry in the profile patch layer.
|
|
|
406
369
|
maxChars: 200000 # inline text truncation
|
|
407
370
|
proxy: 'http://127.0.0.1:7890' # forward proxy for moli, downloads, and the installer's engine download
|
|
408
371
|
insecureTls: true # see the TLS notes below
|
|
409
|
-
sessionBrowser: moli # or chromium, for the session tool
|
|
410
|
-
sessionBrowserPath: '' # explicit chrome.exe / msedge.exe / brave
|
|
411
|
-
sessionPort: 9222 # CDP port for moli_session
|
|
412
|
-
blockPrivateNetworks: false # refuse private addresses (see the note below)
|
|
413
|
-
blockCidrs: [] # extra CIDRs to refuse
|
|
414
372
|
```
|
|
415
373
|
|
|
416
374
|
The plugin-level `proxy` above is the default; `moli_fetch` also takes a `proxy` argument, so a single
|
|
417
375
|
call can route through a different proxy without changing the profile. A loopback URL always bypasses
|
|
418
376
|
whichever proxy is in effect.
|
|
419
377
|
|
|
420
|
-
`blockPrivateNetworks` is **off by default, matching the engine**: `moli fetch http://10.0.0.1/` reaches
|
|
421
|
-
a private address, so the plugin does not silently narrow what the moli you installed already does.
|
|
422
|
-
Turn it on, or list `blockCidrs`, when the agent is following links on pages you do not control and
|
|
423
|
-
you want the engine to refuse private destinations. Two caveats worth knowing before you rely on it:
|
|
424
|
-
moli documents the policy as applying to **directly connected** targets, so a configured `proxy`
|
|
425
|
-
routes around it, and only `blockCidrs` (which the proxy must enforce) covers that path. The sharpest
|
|
426
|
-
target this protects is the cloud metadata address `169.254.169.254`, which hands out instance
|
|
427
|
-
credentials on a VM.
|
|
428
|
-
|
|
429
378
|
Without `exePath`, the plugin probes `exePath` → `MOLI_BIN` → `%LOCALAPPDATA%\Moli\bin\moli.exe` →
|
|
430
379
|
`%USERPROFILE%\.local\bin\moli.exe`, then scans inside `%LOCALAPPDATA%\Moli` for a `moli.exe`, so an
|
|
431
380
|
installation under a versioned subdirectory is still found.
|
|
@@ -488,25 +437,27 @@ Four suites live in [`verify/`](verify/), and all are runnable against a working
|
|
|
488
437
|
`dump` discards its value, an expression that throws is classified as such instead of being blamed
|
|
489
438
|
on the installation, truncation keeps the tail as well as the head, `proxy` is a per-call parameter,
|
|
490
439
|
and `moli_probe` states which process its `httpsOk` describes.
|
|
491
|
-
- `
|
|
492
|
-
|
|
493
|
-
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
440
|
+
- `scroll-verify.mjs` checks the expression builder without an engine (pixels, negatives, `top`,
|
|
441
|
+
`bottom`, nonsense rejected, the caller's `eval` value preserved) and then, when an engine is
|
|
442
|
+
present, fetches a page whose `scroll` handler fills a slot and asserts the content appears with
|
|
443
|
+
`scroll` and the placeholder appears without it — a property of the output, not of an internal call.
|
|
444
|
+
- `http-verify.mjs` asserts the parameter-to-flag mapping directly, because a parameter that never
|
|
445
|
+
reaches the engine is indistinguishable from a feature that does not work. It then fetches a 404 and
|
|
446
|
+
checks that `failOnHttpError` reports it while keeping the body, that a healthy page is unaffected,
|
|
447
|
+
and that a screenshot of an error page is still written as a valid PNG rather than decoded as text.
|
|
448
|
+
The end-to-end half needs an HTTP error from a real server: loopback is unreachable from inside this
|
|
449
|
+
sandbox — `curl.exe` times out on `127.0.0.1` while the Node host reaches it — so it uses a path
|
|
450
|
+
under `example.com` and skips when the network is away.
|
|
451
|
+
|
|
452
|
+
`.github/workflows/verify.yml` runs `schema-verify`, `empty-verify` and `scroll-verify` on Linux,
|
|
453
|
+
Windows and macOS (they need no engine and no network) and attempts the engine-dependent suites on
|
|
454
|
+
Linux and Windows, reporting a missing engine as a skip rather than a failure.
|
|
455
|
+
|
|
456
|
+
`report-verify.mjs` is deliberately **not** in CI. Three of its checks fetch `example.com` and one
|
|
457
|
+
fetches a specific forum thread, so it needs the open internet — and on a host whose traffic goes
|
|
458
|
+
through a proxy, that proxy too. Useful where a network is available, but a suite that goes red
|
|
459
|
+
because a website changed is worse than no suite, so it stays a local check. If you run it and see
|
|
460
|
+
failures in sections 4–6 while sections 1–3 pass, suspect the network before the code.
|
|
510
461
|
|
|
511
462
|
Beyond those, `lib/moli.js` and the tool layer were exercised end to end against Moli 1.1.16 on
|
|
512
463
|
Windows: registration, probe, Markdown and JSON dumps, `eval` (including a tight-timeout regression
|
|
@@ -520,6 +471,21 @@ Confirmed at runtime too: a JavaScript-rendered page (GitHub search) returned 14
|
|
|
520
471
|
Markdown containing 30 real repository matches when fetched through a local proxy. A throwaway
|
|
521
472
|
profile with the plugin installed composes its config with no `JsonSchemaError` and no skipped bundle.
|
|
522
473
|
|
|
474
|
+
## Version note: 1.1.11 and 1.1.12 are deprecated
|
|
475
|
+
|
|
476
|
+
**Use 1.1.13 or later.** Two releases between them carried an interactive-browser experiment —
|
|
477
|
+
`moli_session`, a second Chromium backend, a raw-CDP escape hatch, per-call CDP endpoints, persistent
|
|
478
|
+
browser profiles — roughly 2,300 lines of machinery reaching well past what this plugin is for. It has
|
|
479
|
+
been removed and the code is back to the 1.1.10 shape plus one focused addition.
|
|
480
|
+
|
|
481
|
+
The useful finding from that work is kept in [`/lib/moli.js`](lib/moli.js) and appears above as the
|
|
482
|
+
`scroll` parameter. Everything else it needed was already in `moli fetch`'s own `--eval`, so the
|
|
483
|
+
experiment bought nothing that a parameter could not.
|
|
484
|
+
|
|
485
|
+
On the versions themselves: npm allows a version to be unpublished only briefly and never
|
|
486
|
+
individually, so 1.1.11 and 1.1.12 remain installable but are marked deprecated and point here.
|
|
487
|
+
`npm install dsh-moli` resolves to 1.1.13.
|
|
488
|
+
|
|
523
489
|
## Releasing
|
|
524
490
|
|
|
525
491
|
Publishing goes through GitHub Actions' OIDC identity, so **no npm token exists to expire**:
|
|
@@ -546,6 +512,21 @@ which reads as a missing package and is actually an authorisation failure. Run
|
|
|
546
512
|
`node scripts/preflight-publish.mjs` before any local publish: it reports whether the credential can
|
|
547
513
|
authenticate and names that misleading 404 when it cannot.
|
|
548
514
|
|
|
515
|
+
### Deprecating a version: do it on the website
|
|
516
|
+
|
|
517
|
+
Publishing is scripted; deprecating deliberately is not. Use the package's page on npmjs.com —
|
|
518
|
+
**package → Settings → Deprecate** — and name the versions and the reason.
|
|
519
|
+
|
|
520
|
+
`npm deprecate dsh-moli@1.1.11 "…"` produces the same misleading failure as above when the local
|
|
521
|
+
credential is dead, and there is a second reason not to reach for a token here. npm restricted
|
|
522
|
+
2FA-bypass granular tokens from package-management operations on 31 July 2026, so the kind of token
|
|
523
|
+
that used to make `deprecate` scriptable is being withdrawn, while `publish` has a supported
|
|
524
|
+
token-free path through OIDC. A release process that needs a token only for the rare act of
|
|
525
|
+
deprecating is a process that will be broken the next time it is needed, which is exactly what
|
|
526
|
+
happened with 1.1.11 and 1.1.12.
|
|
527
|
+
|
|
528
|
+
The 1.0.0 and 1.0.1 deprecations were done this way, from the website.
|
|
529
|
+
|
|
549
530
|
## License
|
|
550
531
|
|
|
551
532
|
MIT. See [LICENSE](LICENSE).
|
package/README.zh.md
CHANGED
|
@@ -195,9 +195,10 @@ profile 插件需要两半都齐:依赖边(`dependencies`)+ `dsh.profile.b
|
|
|
195
195
|
|
|
196
196
|
## 用法
|
|
197
197
|
|
|
198
|
-
`moli_fetch` 参数:`url`(必填)、`dump`、`writeTo`、`eval`、`wait`、`waitSelector
|
|
199
|
-
`
|
|
200
|
-
`resources`、`withFrames`、`insecureTls
|
|
198
|
+
`moli_fetch` 参数:`url`(必填)、`dump`、`writeTo`、`eval`、`scroll`、`wait`、`waitSelector`、
|
|
199
|
+
`waitScript`、`waitResponseUrl`、`waitResponseJson`、`delayMs`、`timeoutMs`、`maxChars`、`headers`、
|
|
200
|
+
`userAgent`、`layout`、`scrollbars`、`noJs`、`noCss`、`resources`、`withFrames`、`insecureTls`、
|
|
201
|
+
`failOnHttpError`、`profileDir`、`cookieFile`、`proxy`。
|
|
201
202
|
|
|
202
203
|
```
|
|
203
204
|
moli_fetch url=https://example.com dump=markdown # 省 token 的正文
|
|
@@ -205,99 +206,64 @@ moli_fetch url=https://example.com dump=semantic_tree_text # 带节点 id 的
|
|
|
205
206
|
moli_fetch url=https://example.com eval="document.title" # 取一个值
|
|
206
207
|
moli_fetch url=https://example.com dump=screenshot writeTo=/绝对路径/a.png
|
|
207
208
|
moli_fetch url=https://example.com dump=pdf writeTo=/绝对路径/a.pdf
|
|
209
|
+
moli_fetch url=https://example.com dump=markdown scroll=800 # 先触发懒加载再抓取
|
|
208
210
|
```
|
|
209
211
|
|
|
210
212
|
截图与 PDF 必须给 `writeTo`;字节是原样落盘的,不会打印到输出里。
|
|
211
213
|
|
|
212
|
-
### `
|
|
214
|
+
### `scroll` —— 拿到"只有滚动后才出现"的内容
|
|
213
215
|
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
216
|
+
引擎会序列化整个文档,所以普通抓取**已经包含 DOM 里的一切**。它拿不到的是**尚未存在的内容**:首次
|
|
217
|
+
`scroll` 事件才加载的列表、进入视口才触发的 `IntersectionObserver`、靠近视口才换上的图片。`scroll` 会在
|
|
218
|
+
抓取前置入一次滚动,让这些处理器先跑起来。
|
|
217
219
|
|
|
218
220
|
```
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
moli_session action=type selector="#q" text="moli" submit=true
|
|
223
|
-
moli_session action=scroll dy=1200
|
|
224
|
-
moli_session action=waitFor selector=".results"
|
|
225
|
-
moli_session action=eval expression="document.title"
|
|
226
|
-
moli_session action=screenshot writeTo=/绝对路径/a.png
|
|
227
|
-
moli_session action=status # 是否在运行、当前在哪一页
|
|
228
|
-
moli_session action=close # 停止并释放端口
|
|
221
|
+
moli_fetch url=https://example.com/feed dump=markdown scroll=600
|
|
222
|
+
moli_fetch url=https://example.com/feed dump=markdown scroll=bottom
|
|
223
|
+
moli_fetch url=https://example.com/feed dump=markdown scroll=-400 # 往回滚
|
|
229
224
|
```
|
|
230
225
|
|
|
231
|
-
|
|
232
|
-
|
|
226
|
+
在一个"滚动处理器填充某个槽位"的页面上实测:不带 `scroll` 时 dump 是 `STILL-EMPTY`,带
|
|
227
|
+
`scroll=1000` 时是 `LAZY-CONTENT-1000`。它同样会改变截图,所以带滚动的 `dump=screenshot` 拍到的是页面的
|
|
228
|
+
那一段。
|
|
233
229
|
|
|
234
|
-
|
|
230
|
+
它作为引擎自身"抓取前求值"的一部分执行,因此与 `eval` 可组合:**先滚动,表达式再读取页面,并且仍然返回
|
|
231
|
+
它自己的值**。
|
|
235
232
|
|
|
236
|
-
`
|
|
237
|
-
行为取决于哪一步恰好失败,而且你无法知道最后是哪个浏览器在回答。
|
|
233
|
+
### `failOnHttpError` —— 别再把错误页当正文读
|
|
238
234
|
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
sessionBrowser: moli # moli(默认)或 chromium
|
|
242
|
-
sessionBrowserPath: '' # 可选,指定 chrome.exe / msedge.exe / brave
|
|
243
|
-
sessionPort: 9222
|
|
244
|
-
```
|
|
245
|
-
|
|
246
|
-
**为什么需要这个选择。** moli 的元素几何是**文档绝对坐标且不随滚动更新**,命中测试又受视口高度限制。在
|
|
247
|
-
Moli 1.1.16 与 Chrome 155 上,同一页面、同一元素实测:
|
|
248
|
-
|
|
249
|
-
| | `rect.y` 在 scrollY 0 / 400 / 800 | 点击首个视口以下的控件 |
|
|
250
|
-
|---|---|---|
|
|
251
|
-
| **moli** | 1008 / 1008 / 1008 —— 纹丝不动 | **做不到**,合成输入触达不了 |
|
|
252
|
-
| **chromium** | 1008 / 608 / 208 —— 跟随滚动 | **可以**,浏览器会滚动到它 |
|
|
253
|
-
|
|
254
|
-
所以 moli 是合适的默认值——它是本插件包装的引擎、更轻、而且你已经装了;而当任务需要长页面下方的元素时,
|
|
255
|
-
就换成真实 Chromium。moli 拒绝时会把原因、元素和两条出路都讲清楚:
|
|
235
|
+
默认情况下 404 和普通页面一样被返回,于是**抓取失败看起来是成功的**。加上 `failOnHttpError` 后,最终状态码
|
|
236
|
+
400 及以上会变成 `ok: false` 并报出状态码——**而且保留响应体**,因为错误页往往正是你需要读的东西。
|
|
256
237
|
|
|
257
238
|
```
|
|
258
|
-
|
|
259
|
-
→ "#far-down" is below the viewport (y=2548) and this engine cannot send input there
|
|
260
|
-
|
|
261
|
-
# 要么在页面内部点击(能触达,但会跳过 hover 与 focus 行为)
|
|
262
|
-
moli_session action=eval expression="document.querySelector('#far-down').click()"
|
|
263
|
-
|
|
264
|
-
# 要么这个会话改用真实浏览器
|
|
265
|
-
config: sessionBrowser: chromium
|
|
239
|
+
moli_fetch url=https://example.com/nope dump=markdown failOnHttpError=true
|
|
266
240
|
```
|
|
267
241
|
|
|
268
|
-
|
|
269
|
-
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
传 `cdpEndpoint` 即可连接到已有的 CDP 端口,什么都不启动。这是**继承你自己 cookie 与登录态的唯一途径**:
|
|
274
|
-
用 `--remote-debugging-port=9222` 启动你的浏览器,然后
|
|
275
|
-
|
|
276
|
-
```
|
|
277
|
-
moli_session action=open url=https://example.com cdpEndpoint=http://127.0.0.1:9222
|
|
278
|
-
```
|
|
242
|
+
| | 默认 | `failOnHttpError: true` |
|
|
243
|
+
|---|---|---|
|
|
244
|
+
| 一个 404 | `ok: true`,错误页作为正文 | `ok: false`、`status: 404`,响应体仍然返回 |
|
|
245
|
+
| 404 的截图 | 当普通图片写出 | 仍然写出,但报告为失败 |
|
|
279
246
|
|
|
280
|
-
|
|
281
|
-
|
|
247
|
+
默认关闭,因此现有调用行为不变。JSON dump 本来就能自己报出状态码;markdown 与 HTML 不能,这个开关补的正是
|
|
248
|
+
这个缺口。
|
|
282
249
|
|
|
283
|
-
###
|
|
250
|
+
### 等待页面真正取回的东西
|
|
284
251
|
|
|
285
|
-
|
|
252
|
+
`wait: "networkidle"` 加 `delayMs` 是在**猜**列表要多久才到。下面这几个等的是请求本身:
|
|
286
253
|
|
|
287
254
|
```
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
moli_session action=cdp method=DOM.setFileInputFiles params={"files":["/绝对/a.png"],"nodeId":42}
|
|
291
|
-
```
|
|
255
|
+
# 等待承载数据的那次响应
|
|
256
|
+
moli_fetch url=https://example.com/feed dump=markdown waitResponseUrl="/api/feed"
|
|
292
257
|
|
|
293
|
-
|
|
294
|
-
|
|
258
|
+
# URL 不够有特征时,按载荷内容等
|
|
259
|
+
moli_fetch url=https://example.com/feed dump=markdown waitResponseJson="data.list.0.id=42"
|
|
295
260
|
|
|
296
|
-
|
|
297
|
-
|
|
261
|
+
# 让页面用自己的方式表示"就绪"
|
|
262
|
+
moli_fetch url=https://example.com/app dump=markdown waitScript="window.appReady === true"
|
|
263
|
+
```
|
|
298
264
|
|
|
299
|
-
`
|
|
300
|
-
|
|
265
|
+
`waitResponseJson` 接收 `PATH=VALUE`:`PATH` 定位字段,`VALUE` 按字面比较。当固定延时不是浪费时间就是读得太
|
|
266
|
+
早时,用这几个。
|
|
301
267
|
|
|
302
268
|
## 与 Moli 官方 agent skill 的关系
|
|
303
269
|
|
|
@@ -374,22 +340,11 @@ cookie 驱动的查询参数。那次实测同时覆盖了代理下载路由与
|
|
|
374
340
|
maxChars: 200000 # 内联文本截断阈值
|
|
375
341
|
proxy: 'http://127.0.0.1:7890' # 转给本地正向代理(moli、下载、安装器内部引擎下载都走它)
|
|
376
342
|
insecureTls: true # 见下方 TLS 说明
|
|
377
|
-
sessionBrowser: moli # 或 chromium,供 session 工具使用
|
|
378
|
-
sessionBrowserPath: '' # 显式指定 chrome.exe / msedge.exe / brave
|
|
379
|
-
sessionPort: 9222 # moli_session 的 CDP 端口
|
|
380
|
-
blockPrivateNetworks: false # 拒绝私网地址(见下方说明)
|
|
381
|
-
blockCidrs: [] # 额外拒绝的 CIDR
|
|
382
343
|
```
|
|
383
344
|
|
|
384
345
|
上面配置里的 `proxy` 是默认值;`moli_fetch` 也能接收 `proxy` 参数,因此单次调用可以走另一个代理而不必改
|
|
385
346
|
profile。无论生效的是哪个代理,loopback 地址一律绕过它。
|
|
386
347
|
|
|
387
|
-
`blockPrivateNetworks` **默认关闭,与引擎一致**:`moli fetch http://10.0.0.1/` 本来就能访问私网地址,插件
|
|
388
|
-
不会悄悄收窄你已安装的 moli 的能力。当 agent 会在你无法控制的页面上跟着链接走、而你希望引擎拒绝私网目标时,
|
|
389
|
-
再打开它或配置 `blockCidrs`。依赖它之前有两点需要知道:moli 文档写明该策略只作用于**直连**目标,因此配置了
|
|
390
|
-
`proxy` 就会绕过它,只有 `blockCidrs`(由代理执行)能覆盖那条路径。这个开关保护的最锐利目标是云元数据地址
|
|
391
|
-
`169.254.169.254`——在虚拟机上它会交出实例凭据。
|
|
392
|
-
|
|
393
348
|
未设置 `exePath` 时探测顺序:`exePath` → 环境变量 `MOLI_BIN` → `%LOCALAPPDATA%\Moli\bin\moli.exe`
|
|
394
349
|
→ `%USERPROFILE%\.local\bin\moli.exe`,随后还会在 `%LOCALAPPDATA%\Moli` 内递归查找 `moli.exe`,
|
|
395
350
|
所以装在带版本号的子目录里也能找到。
|
|
@@ -436,20 +391,23 @@ profile。无论生效的是哪个代理,loopback 地址一律绕过它。
|
|
|
436
391
|
- `report-verify.mjs` 守住一份使用体验报告里的发现:`eval` 与 `dump` 同用会丢弃表达式值(已在参数描述里
|
|
437
392
|
写明)、表达式抛异常时被正确归类而不是甩锅给"安装有问题"、截断同时保留头尾、`proxy` 是可按调用传入的参数、
|
|
438
393
|
以及 `moli_probe` 会说明它的 `httpsOk` 描述的是哪个进程。
|
|
439
|
-
- `
|
|
440
|
-
|
|
441
|
-
|
|
442
|
-
|
|
443
|
-
|
|
444
|
-
|
|
445
|
-
|
|
446
|
-
|
|
447
|
-
|
|
448
|
-
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
394
|
+
- `scroll-verify.mjs` 在无引擎时校验表达式构造(像素值、负数、`top`、`bottom`、无效输入被拒、调用方的 `eval`
|
|
395
|
+
值被保留),有引擎时抓取一个"滚动处理器填充槽位"的页面,断言带 `scroll` 时内容出现、不带时只有占位符——
|
|
396
|
+
断言的是**输出**的性质,而不是内部调用。
|
|
397
|
+
- `http-verify.mjs` 直接断言"参数 → 引擎选项"的映射,因为**参数没送到引擎和功能不生效,从外面看是一样的**。
|
|
398
|
+
随后抓一个 404,验证 `failOnHttpError` 会报告失败同时保留响应体、健康页面不受影响、以及错误页的截图仍然被
|
|
399
|
+
写为合法 PNG 而不是被当文本解码。端到端部分需要一个真实服务器的 HTTP 错误:**本沙箱内无法访问 loopback**
|
|
400
|
+
——`curl.exe` 连 `127.0.0.1` 会超时,而 Node 宿主进程可以——所以它用 `example.com` 下的一个路径,网络不可用
|
|
401
|
+
时跳过。
|
|
402
|
+
|
|
403
|
+
`.github/workflows/verify.yml` 在 Linux、Windows、macOS 上跑 `schema-verify`、`empty-verify` 与
|
|
404
|
+
`scroll-verify`(无需引擎、无需网络),在 Linux 与 Windows 上尝试依赖引擎的几套;若 runner 上没有引擎,
|
|
405
|
+
则报告为跳过而不是失败。
|
|
406
|
+
|
|
407
|
+
`report-verify.mjs` **故意不放进 CI**:其中三项会抓 `example.com`,还有一项抓一个具体的论坛帖子,因此它
|
|
408
|
+
需要能上外网——如果这台机器的流量走代理,还需要那个代理可用。有网络时它是有用的,但**一个因为网站改版就变
|
|
409
|
+
红的测试套件,比没有套件更糟**,所以它只作为本机检查保留。如果你跑它时发现第 1–3 节通过而第 4–6 节失败,
|
|
410
|
+
**先怀疑网络,再怀疑代码**。
|
|
453
411
|
|
|
454
412
|
此外,`lib/moli.js` 与工具层在 Windows 上对 Moli 1.1.16 做了端到端验证:注册、probe、Markdown 与 JSON
|
|
455
413
|
dump、`eval`(含针对 `-t` 参数位置的紧超时回归防护)、PNG 与 PDF 产物签名、截断、全部拒绝路径(二元 dump
|
|
@@ -460,6 +418,19 @@ dump、`eval`(含针对 `-t` 参数位置的紧超时回归防护)、PNG 与
|
|
|
460
418
|
30 处真实结果匹配。另有一个装了本插件的一次性 profile,组合其配置时既无 `JsonSchemaError` 也无被跳过的
|
|
461
419
|
bundle。
|
|
462
420
|
|
|
421
|
+
## 版本说明:1.1.11 与 1.1.12 已废弃
|
|
422
|
+
|
|
423
|
+
**请使用 1.1.13 或更高版本。** 这两个版本之间夹着一次"交互式浏览器"实验——`moli_session`、第二个
|
|
424
|
+
Chromium 后端、原始 CDP 逃生舱、按调用的 CDP 端点、持久化浏览器 profile——约 2300 行机制,远远超出了这个
|
|
425
|
+
插件的定位。它已被移除,代码回到 1.1.10 的形态,只保留一处聚焦的新增。
|
|
426
|
+
|
|
427
|
+
那次工作里唯一有价值的发现保留在 [`/lib/moli.js`](lib/moli.js),并以 `scroll` 参数的形式出现在上文。**它
|
|
428
|
+
本来需要的其他一切,`moli fetch` 自带的 `--eval` 早就有了**——所以那次实验没有换来任何"一个参数换不来的
|
|
429
|
+
东西"。
|
|
430
|
+
|
|
431
|
+
关于这两个版本本身:npm 只允许在很短的时间窗口内撤回发布,且无法单独撤回某一个版本,因此 1.1.11 与
|
|
432
|
+
1.1.12 仍然可安装,但已被标记为废弃并指向这里。`npm install dsh-moli` 会安装到 1.1.13。
|
|
433
|
+
|
|
463
434
|
## 发布
|
|
464
435
|
|
|
465
436
|
发布走 GitHub Actions 的 OIDC 身份,**本机不存任何 npm token,因此没有会失效的凭据**:
|
|
@@ -484,6 +455,19 @@ E404 Not Found - PUT https://registry.npmjs.org/dsh-moli
|
|
|
484
455
|
**这看起来像"包不存在",实际是认证失败。** 任何本地发布之前先跑
|
|
485
456
|
`node scripts/preflight-publish.mjs`:它会告诉你凭据能否认证,不能时明确点出这个误导性的 404。
|
|
486
457
|
|
|
458
|
+
### 废弃某个版本:去网站操作,别用命令行
|
|
459
|
+
|
|
460
|
+
发布是脚本化的,**废弃则刻意不做成脚本**。用 npmjs.com 上的包页面——**package → Settings →
|
|
461
|
+
Deprecate**——选版本、写理由。
|
|
462
|
+
|
|
463
|
+
`npm deprecate dsh-moli@1.1.11 "…"` 在本机凭据失效时会给出上面那个同样误导的失败,而且这里还有第二个
|
|
464
|
+
理由不要依赖 token:npm 在 **2026 年 7 月 31 日**起禁止 2FA-bypass 类型的 granular token 执行**包管理**
|
|
465
|
+
操作,也就是说"以前能让 `deprecate` 脚本化"的那类凭据正在被取消;而 `publish` 有受支持的、免 token 的
|
|
466
|
+
OIDC 路径。**一个只在"偶尔废弃版本"时才需要 token 的流程,注定在下次需要它时是坏的**——1.1.11 与 1.1.12
|
|
467
|
+
遇到的正是这种情况。
|
|
468
|
+
|
|
469
|
+
1.0.0 与 1.0.1 的废弃就是通过网站完成的。
|
|
470
|
+
|
|
487
471
|
## 许可
|
|
488
472
|
|
|
489
473
|
MIT,见 [LICENSE](LICENSE)。
|