dsh-moli 1.1.12 → 1.1.14

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -209,9 +209,10 @@ An unresolvable bundle is **skipped silently** apart from one stderr line
209
209
 
210
210
  ## Usage
211
211
 
212
- `moli_fetch` parameters: `url` (required), `dump`, `writeTo`, `eval`, `wait`, `waitSelector`,
213
- `delayMs`, `timeoutMs`, `maxChars`, `headers`, `userAgent`, `layout`, `scrollbars`, `noJs`,
214
- `noCss`, `resources`, `withFrames`, `insecureTls`, `profileDir`, `cookieFile`.
212
+ `moli_fetch` parameters: `url` (required), `dump`, `writeTo`, `eval`, `scroll`, `wait`,
213
+ `waitSelector`, `waitScript`, `waitResponseUrl`, `waitResponseJson`, `delayMs`, `timeoutMs`,
214
+ `maxChars`, `headers`, `userAgent`, `layout`, `scrollbars`, `noJs`, `noCss`, `resources`,
215
+ `withFrames`, `insecureTls`, `failOnHttpError`, `profileDir`, `cookieFile`, `proxy`.
215
216
 
216
217
  ```
217
218
  moli_fetch url=https://example.com dump=markdown # token-efficient text
@@ -219,105 +220,67 @@ moli_fetch url=https://example.com dump=semantic_tree_text # nodes with backe
219
220
  moli_fetch url=https://example.com eval="document.title" # one computed value
220
221
  moli_fetch url=https://example.com dump=screenshot writeTo=/abs/path/a.png
221
222
  moli_fetch url=https://example.com dump=pdf writeTo=/abs/path/a.pdf
223
+ moli_fetch url=https://example.com dump=markdown scroll=800 # trigger lazy loading first
222
224
  ```
223
225
 
224
226
  Screenshot and PDF dumps need `writeTo`; the bytes are written verbatim rather than printed.
225
227
 
226
- ### `moli_session` — an interactive browser
228
+ ### `scroll` — reach what only appears on a scroll
227
229
 
228
- `moli_fetch` loads a page and prints it. That covers reading, and nothing else: it cannot dismiss a
229
- consent wall, fill a search box, scroll a lazy list, or wait for a client-side route to change.
230
- `moli_session` drives a persistent browser instead, over the Chrome DevTools Protocol endpoint that
231
- `moli serve` publishes. State — cookies, navigation, typed input — survives between calls until you
232
- stop it.
230
+ The engine serializes the whole document, so a plain fetch already contains everything the DOM holds.
231
+ What it cannot contain is content that does not exist yet: a list that loads on the first `scroll`
232
+ event, an `IntersectionObserver` that fires on entry, an image swapped in when it nears the viewport.
233
+ `scroll` runs the scroll before the capture, so those handlers fire first.
233
234
 
234
235
  ```
235
- moli_session action=open url=https://example.com # start the browser and navigate
236
- moli_session action=snapshot # the rendered page as text
237
- moli_session action=click selector="#accept-cookies"
238
- moli_session action=type selector="#q" text="moli" submit=true
239
- moli_session action=scroll dy=1200
240
- moli_session action=waitFor selector=".results"
241
- moli_session action=eval expression="document.title"
242
- moli_session action=screenshot writeTo=/abs/path/a.png
243
- moli_session action=status # is it running, and on what page
244
- moli_session action=close # stop it and free the port
236
+ moli_fetch url=https://example.com/feed dump=markdown scroll=600
237
+ moli_fetch url=https://example.com/feed dump=markdown scroll=bottom
238
+ moli_fetch url=https://example.com/feed dump=markdown scroll=-400 # back up
245
239
  ```
246
240
 
247
- Elements are addressed by **CSS selector**, never by pixel coordinates: the tool measures the element
248
- and dispatches real mouse and key events, so page handlers run exactly as they do for a person.
241
+ Measured on a page whose `scroll` handler fills a slot: without `scroll` the dump says
242
+ `STILL-EMPTY`, with `scroll=1000` it says `LAZY-CONTENT-1000`. It also moves the screenshot, so a
243
+ scrolled `dump=screenshot` shows that part of the page.
249
244
 
250
- ### Choosing a browser: `moli` or `chromium`
245
+ It runs as part of the engine's own pre-capture evaluation and therefore composes with `eval`: the
246
+ scroll happens first, and the expression still returns its own value.
251
247
 
252
- `moli_session` can drive either engine, and you choose which — there is **no automatic fallback**,
253
- because switching engines when one click fails would make the timing and the behaviour depend on which
254
- operation happened to fail, and you could not tell which browser answered.
248
+ ### `failOnHttpError` — stop reading an error page as content
255
249
 
256
- ```yaml
257
- config:
258
- sessionBrowser: moli # moli (default) or chromium
259
- sessionBrowserPath: '' # optional path to chrome.exe / msedge.exe / brave
260
- sessionPort: 9222
261
- ```
262
-
263
- **Why the choice exists.** Moli's element geometry is document-absolute and does not follow scrolling,
264
- and its hit test is bounded by the viewport height. Measured on Moli 1.1.16 and Chrome 155, same page,
265
- same element:
266
-
267
- | | `rect.y` at scrollY 0 / 400 / 800 | click a control below the first viewport |
268
- |---|---|---|
269
- | **moli** | 1008 / 1008 / 1008 — never moves | **impossible**; synthetic input cannot reach it |
270
- | **chromium** | 1008 / 608 / 208 — follows the scroll | **works**; the browser scrolls to it |
271
-
272
- So moli is the good default — it is the engine this plugin wraps, it is lighter, and it is already
273
- installed — and a real Chromium is what you switch to when a task needs an element further down a long
274
- page. When moli refuses, it says so, names the element, and points at both routes:
250
+ By default a 404 is returned like any other page, so a failed fetch looks successful. Setting
251
+ `failOnHttpError` turns a final status of 400 or above into `ok: false` with the status named — and
252
+ **keeps the body**, because an error page is often the thing you need to read.
275
253
 
276
254
  ```
277
- moli_session action=click selector="#far-down"
278
- → "#far-down" is below the viewport (y=2548) and this engine cannot send input there
279
-
280
- # either click from inside the page (reaches it, but skips hover and focus behaviour)
281
- moli_session action=eval expression="document.querySelector('#far-down').click()"
282
-
283
- # or use a real browser for that session
284
- config: sessionBrowser: chromium
255
+ moli_fetch url=https://example.com/nope dump=markdown failOnHttpError=true
285
256
  ```
286
257
 
287
- With `chromium` the plugin starts the browser it finds (Chrome, Edge or Brave, or `sessionBrowserPath`)
288
- in `--headless=new` mode with its own profile, so it does not touch your normal browsing profile.
289
-
290
- ### Driving a browser you already have open
291
-
292
- Pass `cdpEndpoint` to attach to a CDP port instead of starting anything. This is the only way to inherit
293
- your own cookies and logins: start your browser with `--remote-debugging-port=9222`, then
294
-
295
- ```
296
- moli_session action=open url=https://example.com cdpEndpoint=http://127.0.0.1:9222
297
- ```
258
+ | | default | `failOnHttpError: true` |
259
+ |---|---|---|
260
+ | a 404 | `ok: true`, the error page as content | `ok: false`, `status: 404`, body still returned |
261
+ | a screenshot of a 404 | written as a normal image | still written, and reported as a failure |
298
262
 
299
- The endpoint applies to that call only. A browser this plugin did not start is **never stopped by
300
- `close`**, and `status` reports `external: true` so you can tell.
263
+ It is off by default so existing calls keep their behaviour. A JSON dump already reported the status
264
+ on its own; markdown and HTML dumps did not, and that is the gap this closes.
301
265
 
302
- ### `cdp` — the raw protocol escape hatch
266
+ ### Waiting for what the page actually fetches
303
267
 
304
- Everything the named actions do is an abstraction chosen in advance. `action=cdp` is not:
268
+ `wait: "networkidle"` plus `delayMs` is a guess about how long a list takes to arrive. These wait for
269
+ the request itself:
305
270
 
306
271
  ```
307
- moli_session action=cdp method=Browser.getVersion
308
- moli_session action=cdp method=Network.setExtraHTTPHeaders params={"headers":{"X-Thing":"1"}}
309
- moli_session action=cdp method=DOM.setFileInputFiles params={"files":["/abs/a.png"],"nodeId":42}
310
- ```
272
+ # the response that carries the data
273
+ moli_fetch url=https://example.com/feed dump=markdown waitResponseUrl="/api/feed"
311
274
 
312
- It reaches what the wrapper does not: other targets, cross-origin frames, file inputs, extra headers,
313
- network interception. `params` may be an object or a JSON string.
275
+ # the payload, when the URL is not distinctive
276
+ moli_fetch url=https://example.com/feed dump=markdown waitResponseJson="data.list.0.id=42"
314
277
 
315
- The session needs **Node 21 or newer** for the built-in `WebSocket`; on Node 20 the tool answers with
316
- that reason instead of failing obscurely. `moli_fetch` and `moli_probe` still work on 20.12.
278
+ # the page's own idea of readiness
279
+ moli_fetch url=https://example.com/app dump=markdown waitScript="window.appReady === true"
280
+ ```
317
281
 
318
- `moli_session` also takes `port` (default 9222). If that port is already served by a `moli serve` you
319
- started yourself, the tool connects to it rather than failing to bind — and `close` then leaves it
320
- alone, because it is not ours to stop.
282
+ `waitResponseJson` takes `PATH=VALUE`, where `PATH` addresses the field and `VALUE` is compared
283
+ literally. Use these when a fixed delay either wastes time or reads the page too early.
321
284
 
322
285
  ## Relationship to Moli's own agent skills
323
286
 
@@ -406,26 +369,12 @@ Override any field under the plugin's `config` entry in the profile patch layer.
406
369
  maxChars: 200000 # inline text truncation
407
370
  proxy: 'http://127.0.0.1:7890' # forward proxy for moli, downloads, and the installer's engine download
408
371
  insecureTls: true # see the TLS notes below
409
- sessionBrowser: moli # or chromium, for the session tool
410
- sessionBrowserPath: '' # explicit chrome.exe / msedge.exe / brave
411
- sessionPort: 9222 # CDP port for moli_session
412
- blockPrivateNetworks: false # refuse private addresses (see the note below)
413
- blockCidrs: [] # extra CIDRs to refuse
414
372
  ```
415
373
 
416
374
  The plugin-level `proxy` above is the default; `moli_fetch` also takes a `proxy` argument, so a single
417
375
  call can route through a different proxy without changing the profile. A loopback URL always bypasses
418
376
  whichever proxy is in effect.
419
377
 
420
- `blockPrivateNetworks` is **off by default, matching the engine**: `moli fetch http://10.0.0.1/` reaches
421
- a private address, so the plugin does not silently narrow what the moli you installed already does.
422
- Turn it on, or list `blockCidrs`, when the agent is following links on pages you do not control and
423
- you want the engine to refuse private destinations. Two caveats worth knowing before you rely on it:
424
- moli documents the policy as applying to **directly connected** targets, so a configured `proxy`
425
- routes around it, and only `blockCidrs` (which the proxy must enforce) covers that path. The sharpest
426
- target this protects is the cloud metadata address `169.254.169.254`, which hands out instance
427
- credentials on a VM.
428
-
429
378
  Without `exePath`, the plugin probes `exePath` → `MOLI_BIN` → `%LOCALAPPDATA%\Moli\bin\moli.exe` →
430
379
  `%USERPROFILE%\.local\bin\moli.exe`, then scans inside `%LOCALAPPDATA%\Moli` for a `moli.exe`, so an
431
380
  installation under a versioned subdirectory is still found.
@@ -488,25 +437,27 @@ Four suites live in [`verify/`](verify/), and all are runnable against a working
488
437
  `dump` discards its value, an expression that throws is classified as such instead of being blamed
489
438
  on the installation, truncation keeps the tail as well as the head, `proxy` is a per-call parameter,
490
439
  and `moli_probe` states which process its `httpsOk` describes.
491
- - `session-verify.mjs` drives `moli_session` end to end against a `data:` URL fixture, so it needs no
492
- network and no site stays still underneath it: lifecycle and port release, snapshot, click, type
493
- with and without clearing, press, scroll, `waitFor` in both directions, eval including a throwing
494
- expression, screenshot byte signature, navigation by clicking a link, restart after close, and the
495
- viewport limit described above — which is asserted as a limit, with the `eval` route that works
496
- around it asserted next to it. It skips when no engine is installed.
497
- - `session-backends.mjs` is the comparison that justifies the second backend: the same page, the same
498
- control below the viewport, asserted to be **refused on moli and clicked on Chromium**, plus the
499
- `cdp` action on both. The Chromium half skips when the machine has no Chromium-family browser.
500
- - `session-attach-verify.mjs` covers external endpoints and ownership: attaching to a browser this
501
- plugin did not start, identifying it correctly, refusing to stop it on `close`, and — the trap that
502
- a behavioural probe got wrong — classifying an external **moli** as moli even though it answers
503
- `/json/version` with `"Browser":"Chrome/145.0.0.0"`. It also asserts that naming a per-call endpoint
504
- does not latch it, which an earlier version did: one call against a dead endpoint left the session
505
- pointing at port 9 for the rest of the process.
506
-
507
- `.github/workflows/verify.yml` runs the first two on Linux, Windows and macOS (they need no engine and
508
- no network) and attempts the engine-dependent suites on Linux and Windows, reporting a missing engine
509
- as a skip rather than a failure.
440
+ - `scroll-verify.mjs` checks the expression builder without an engine (pixels, negatives, `top`,
441
+ `bottom`, nonsense rejected, the caller's `eval` value preserved) and then, when an engine is
442
+ present, fetches a page whose `scroll` handler fills a slot and asserts the content appears with
443
+ `scroll` and the placeholder appears without it — a property of the output, not of an internal call.
444
+ - `http-verify.mjs` asserts the parameter-to-flag mapping directly, because a parameter that never
445
+ reaches the engine is indistinguishable from a feature that does not work. It then fetches a 404 and
446
+ checks that `failOnHttpError` reports it while keeping the body, that a healthy page is unaffected,
447
+ and that a screenshot of an error page is still written as a valid PNG rather than decoded as text.
448
+ The end-to-end half needs an HTTP error from a real server: loopback is unreachable from inside this
449
+ sandbox — `curl.exe` times out on `127.0.0.1` while the Node host reaches it — so it uses a path
450
+ under `example.com` and skips when the network is away.
451
+
452
+ `.github/workflows/verify.yml` runs `schema-verify`, `empty-verify` and `scroll-verify` on Linux,
453
+ Windows and macOS (they need no engine and no network) and attempts the engine-dependent suites on
454
+ Linux and Windows, reporting a missing engine as a skip rather than a failure.
455
+
456
+ `report-verify.mjs` is deliberately **not** in CI. Three of its checks fetch `example.com` and one
457
+ fetches a specific forum thread, so it needs the open internet — and on a host whose traffic goes
458
+ through a proxy, that proxy too. Useful where a network is available, but a suite that goes red
459
+ because a website changed is worse than no suite, so it stays a local check. If you run it and see
460
+ failures in sections 4–6 while sections 1–3 pass, suspect the network before the code.
510
461
 
511
462
  Beyond those, `lib/moli.js` and the tool layer were exercised end to end against Moli 1.1.16 on
512
463
  Windows: registration, probe, Markdown and JSON dumps, `eval` (including a tight-timeout regression
@@ -520,6 +471,21 @@ Confirmed at runtime too: a JavaScript-rendered page (GitHub search) returned 14
520
471
  Markdown containing 30 real repository matches when fetched through a local proxy. A throwaway
521
472
  profile with the plugin installed composes its config with no `JsonSchemaError` and no skipped bundle.
522
473
 
474
+ ## Version note: 1.1.11 and 1.1.12 are deprecated
475
+
476
+ **Use 1.1.13 or later.** Two releases between them carried an interactive-browser experiment —
477
+ `moli_session`, a second Chromium backend, a raw-CDP escape hatch, per-call CDP endpoints, persistent
478
+ browser profiles — roughly 2,300 lines of machinery reaching well past what this plugin is for. It has
479
+ been removed and the code is back to the 1.1.10 shape plus one focused addition.
480
+
481
+ The useful finding from that work is kept in [`/lib/moli.js`](lib/moli.js) and appears above as the
482
+ `scroll` parameter. Everything else it needed was already in `moli fetch`'s own `--eval`, so the
483
+ experiment bought nothing that a parameter could not.
484
+
485
+ On the versions themselves: npm allows a version to be unpublished only briefly and never
486
+ individually, so 1.1.11 and 1.1.12 remain installable but are marked deprecated and point here.
487
+ `npm install dsh-moli` resolves to 1.1.13.
488
+
523
489
  ## Releasing
524
490
 
525
491
  Publishing goes through GitHub Actions' OIDC identity, so **no npm token exists to expire**:
@@ -546,6 +512,21 @@ which reads as a missing package and is actually an authorisation failure. Run
546
512
  `node scripts/preflight-publish.mjs` before any local publish: it reports whether the credential can
547
513
  authenticate and names that misleading 404 when it cannot.
548
514
 
515
+ ### Deprecating a version: do it on the website
516
+
517
+ Publishing is scripted; deprecating deliberately is not. Use the package's page on npmjs.com —
518
+ **package → Settings → Deprecate** — and name the versions and the reason.
519
+
520
+ `npm deprecate dsh-moli@1.1.11 "…"` produces the same misleading failure as above when the local
521
+ credential is dead, and there is a second reason not to reach for a token here. npm restricted
522
+ 2FA-bypass granular tokens from package-management operations on 31 July 2026, so the kind of token
523
+ that used to make `deprecate` scriptable is being withdrawn, while `publish` has a supported
524
+ token-free path through OIDC. A release process that needs a token only for the rare act of
525
+ deprecating is a process that will be broken the next time it is needed, which is exactly what
526
+ happened with 1.1.11 and 1.1.12.
527
+
528
+ The 1.0.0 and 1.0.1 deprecations were done this way, from the website.
529
+
549
530
  ## License
550
531
 
551
532
  MIT. See [LICENSE](LICENSE).
package/README.zh.md CHANGED
@@ -195,9 +195,10 @@ profile 插件需要两半都齐:依赖边(`dependencies`)+ `dsh.profile.b
195
195
 
196
196
  ## 用法
197
197
 
198
- `moli_fetch` 参数:`url`(必填)、`dump`、`writeTo`、`eval`、`wait`、`waitSelector`、`delayMs`、
199
- `timeoutMs`、`maxChars`、`headers`、`userAgent`、`layout`、`scrollbars`、`noJs`、`noCss`、
200
- `resources`、`withFrames`、`insecureTls`、`profileDir`、`cookieFile`。
198
+ `moli_fetch` 参数:`url`(必填)、`dump`、`writeTo`、`eval`、`scroll`、`wait`、`waitSelector`、
199
+ `waitScript`、`waitResponseUrl`、`waitResponseJson`、`delayMs`、`timeoutMs`、`maxChars`、`headers`、
200
+ `userAgent`、`layout`、`scrollbars`、`noJs`、`noCss`、`resources`、`withFrames`、`insecureTls`、
201
+ `failOnHttpError`、`profileDir`、`cookieFile`、`proxy`。
201
202
 
202
203
  ```
203
204
  moli_fetch url=https://example.com dump=markdown # 省 token 的正文
@@ -205,99 +206,64 @@ moli_fetch url=https://example.com dump=semantic_tree_text # 带节点 id 的
205
206
  moli_fetch url=https://example.com eval="document.title" # 取一个值
206
207
  moli_fetch url=https://example.com dump=screenshot writeTo=/绝对路径/a.png
207
208
  moli_fetch url=https://example.com dump=pdf writeTo=/绝对路径/a.pdf
209
+ moli_fetch url=https://example.com dump=markdown scroll=800 # 先触发懒加载再抓取
208
210
  ```
209
211
 
210
212
  截图与 PDF 必须给 `writeTo`;字节是原样落盘的,不会打印到输出里。
211
213
 
212
- ### `moli_session` —— 可交互的浏览器
214
+ ### `scroll` —— 拿到"只有滚动后才出现"的内容
213
215
 
214
- `moli_fetch` 是"加载一页、打印出来",只能读。它点不掉同意墙、填不了搜索框、滚不动懒加载列表,也等不到
215
- 客户端路由切换。`moli_session` 驱动的是一个**常驻浏览器**,走 `moli serve` 暴露的 Chrome DevTools
216
- Protocol。状态(cookie、导航位置、已输入的内容)在多次调用之间保持,直到你显式停止。
216
+ 引擎会序列化整个文档,所以普通抓取**已经包含 DOM 里的一切**。它拿不到的是**尚未存在的内容**:首次
217
+ `scroll` 事件才加载的列表、进入视口才触发的 `IntersectionObserver`、靠近视口才换上的图片。`scroll` 会在
218
+ 抓取前置入一次滚动,让这些处理器先跑起来。
217
219
 
218
220
  ```
219
- moli_session action=open url=https://example.com # 启动浏览器并导航
220
- moli_session action=snapshot # 渲染后的页面文本
221
- moli_session action=click selector="#accept-cookies"
222
- moli_session action=type selector="#q" text="moli" submit=true
223
- moli_session action=scroll dy=1200
224
- moli_session action=waitFor selector=".results"
225
- moli_session action=eval expression="document.title"
226
- moli_session action=screenshot writeTo=/绝对路径/a.png
227
- moli_session action=status # 是否在运行、当前在哪一页
228
- moli_session action=close # 停止并释放端口
221
+ moli_fetch url=https://example.com/feed dump=markdown scroll=600
222
+ moli_fetch url=https://example.com/feed dump=markdown scroll=bottom
223
+ moli_fetch url=https://example.com/feed dump=markdown scroll=-400 # 往回滚
229
224
  ```
230
225
 
231
- 元素一律用 **CSS 选择器**定位,不接受像素坐标:工具自己测量元素并派发真实的鼠标与键盘事件,所以页面的
232
- 处理器会像人操作时一样执行。
226
+ 在一个"滚动处理器填充某个槽位"的页面上实测:不带 `scroll` 时 dump 是 `STILL-EMPTY`,带
227
+ `scroll=1000` 时是 `LAZY-CONTENT-1000`。它同样会改变截图,所以带滚动的 `dump=screenshot` 拍到的是页面的
228
+ 那一段。
233
229
 
234
- ### 选哪个浏览器:`moli` 还是 `chromium`
230
+ 它作为引擎自身"抓取前求值"的一部分执行,因此与 `eval` 可组合:**先滚动,表达式再读取页面,并且仍然返回
231
+ 它自己的值**。
235
232
 
236
- `moli_session` 可以驱动两种引擎,由你选择——**没有自动降级**。因为"某个点击失败就换引擎"会让工具的耗时和
237
- 行为取决于哪一步恰好失败,而且你无法知道最后是哪个浏览器在回答。
233
+ ### `failOnHttpError` —— 别再把错误页当正文读
238
234
 
239
- ```yaml
240
- config:
241
- sessionBrowser: moli # moli(默认)或 chromium
242
- sessionBrowserPath: '' # 可选,指定 chrome.exe / msedge.exe / brave
243
- sessionPort: 9222
244
- ```
245
-
246
- **为什么需要这个选择。** moli 的元素几何是**文档绝对坐标且不随滚动更新**,命中测试又受视口高度限制。在
247
- Moli 1.1.16 与 Chrome 155 上,同一页面、同一元素实测:
248
-
249
- | | `rect.y` 在 scrollY 0 / 400 / 800 | 点击首个视口以下的控件 |
250
- |---|---|---|
251
- | **moli** | 1008 / 1008 / 1008 —— 纹丝不动 | **做不到**,合成输入触达不了 |
252
- | **chromium** | 1008 / 608 / 208 —— 跟随滚动 | **可以**,浏览器会滚动到它 |
253
-
254
- 所以 moli 是合适的默认值——它是本插件包装的引擎、更轻、而且你已经装了;而当任务需要长页面下方的元素时,
255
- 就换成真实 Chromium。moli 拒绝时会把原因、元素和两条出路都讲清楚:
235
+ 默认情况下 404 和普通页面一样被返回,于是**抓取失败看起来是成功的**。加上 `failOnHttpError` 后,最终状态码
236
+ 400 及以上会变成 `ok: false` 并报出状态码——**而且保留响应体**,因为错误页往往正是你需要读的东西。
256
237
 
257
238
  ```
258
- moli_session action=click selector="#far-down"
259
- → "#far-down" is below the viewport (y=2548) and this engine cannot send input there
260
-
261
- # 要么在页面内部点击(能触达,但会跳过 hover 与 focus 行为)
262
- moli_session action=eval expression="document.querySelector('#far-down').click()"
263
-
264
- # 要么这个会话改用真实浏览器
265
- config: sessionBrowser: chromium
239
+ moli_fetch url=https://example.com/nope dump=markdown failOnHttpError=true
266
240
  ```
267
241
 
268
- 选 `chromium` 时,插件会在 `--headless=new` 模式下用**独立 profile** 启动它找到的浏览器(Chrome、Edge 或
269
- Brave,也可用 `sessionBrowserPath` 指定),不会碰你日常使用的浏览器配置。
270
-
271
- ### 驱动你自己已经开着的浏览器
272
-
273
- 传 `cdpEndpoint` 即可连接到已有的 CDP 端口,什么都不启动。这是**继承你自己 cookie 与登录态的唯一途径**:
274
- 用 `--remote-debugging-port=9222` 启动你的浏览器,然后
275
-
276
- ```
277
- moli_session action=open url=https://example.com cdpEndpoint=http://127.0.0.1:9222
278
- ```
242
+ | | 默认 | `failOnHttpError: true` |
243
+ |---|---|---|
244
+ | 一个 404 | `ok: true`,错误页作为正文 | `ok: false`、`status: 404`,响应体仍然返回 |
245
+ | 404 的截图 | 当普通图片写出 | 仍然写出,但报告为失败 |
279
246
 
280
- 该端点**只对当次调用生效**。不是本插件启动的浏览器**永远不会被 `close` 关掉**,并且 `status` 会报
281
- `external: true` 让你能分辨。
247
+ 默认关闭,因此现有调用行为不变。JSON dump 本来就能自己报出状态码;markdown 与 HTML 不能,这个开关补的正是
248
+ 这个缺口。
282
249
 
283
- ### `cdp` —— 原始协议逃生舱
250
+ ### 等待页面真正取回的东西
284
251
 
285
- 具名动作全都是"预先替你决定好的抽象",而 `action=cdp` 不是:
252
+ `wait: "networkidle"` 加 `delayMs` 是在**猜**列表要多久才到。下面这几个等的是请求本身:
286
253
 
287
254
  ```
288
- moli_session action=cdp method=Browser.getVersion
289
- moli_session action=cdp method=Network.setExtraHTTPHeaders params={"headers":{"X-Thing":"1"}}
290
- moli_session action=cdp method=DOM.setFileInputFiles params={"files":["/绝对/a.png"],"nodeId":42}
291
- ```
255
+ # 等待承载数据的那次响应
256
+ moli_fetch url=https://example.com/feed dump=markdown waitResponseUrl="/api/feed"
292
257
 
293
- 它能做到包装层做不到的事:切换 target、跨域 iframe、文件上传、额外的请求头、网络拦截。`params` 可以是对象
294
- 或 JSON 字符串。
258
+ # URL 不够有特征时,按载荷内容等
259
+ moli_fetch url=https://example.com/feed dump=markdown waitResponseJson="data.list.0.id=42"
295
260
 
296
- 本工具需要 **Node 21 或更新版本**(内置 `WebSocket`);在 Node 20 上会直接说明原因,而不是给出费解的失败。
297
- `moli_fetch` 与 `moli_probe` 仍支持 20.12。
261
+ # 让页面用自己的方式表示"就绪"
262
+ moli_fetch url=https://example.com/app dump=markdown waitScript="window.appReady === true"
263
+ ```
298
264
 
299
- `moli_session` 也接受 `port`(默认 9222)。如果该端口已经由你自己启动的 `moli serve` 占用,工具会连接它
300
- 而不是绑定失败——此时 `close` **不会**停掉它,因为那不是我们的进程。
265
+ `waitResponseJson` 接收 `PATH=VALUE`:`PATH` 定位字段,`VALUE` 按字面比较。当固定延时不是浪费时间就是读得太
266
+ 早时,用这几个。
301
267
 
302
268
  ## 与 Moli 官方 agent skill 的关系
303
269
 
@@ -374,22 +340,11 @@ cookie 驱动的查询参数。那次实测同时覆盖了代理下载路由与
374
340
  maxChars: 200000 # 内联文本截断阈值
375
341
  proxy: 'http://127.0.0.1:7890' # 转给本地正向代理(moli、下载、安装器内部引擎下载都走它)
376
342
  insecureTls: true # 见下方 TLS 说明
377
- sessionBrowser: moli # 或 chromium,供 session 工具使用
378
- sessionBrowserPath: '' # 显式指定 chrome.exe / msedge.exe / brave
379
- sessionPort: 9222 # moli_session 的 CDP 端口
380
- blockPrivateNetworks: false # 拒绝私网地址(见下方说明)
381
- blockCidrs: [] # 额外拒绝的 CIDR
382
343
  ```
383
344
 
384
345
  上面配置里的 `proxy` 是默认值;`moli_fetch` 也能接收 `proxy` 参数,因此单次调用可以走另一个代理而不必改
385
346
  profile。无论生效的是哪个代理,loopback 地址一律绕过它。
386
347
 
387
- `blockPrivateNetworks` **默认关闭,与引擎一致**:`moli fetch http://10.0.0.1/` 本来就能访问私网地址,插件
388
- 不会悄悄收窄你已安装的 moli 的能力。当 agent 会在你无法控制的页面上跟着链接走、而你希望引擎拒绝私网目标时,
389
- 再打开它或配置 `blockCidrs`。依赖它之前有两点需要知道:moli 文档写明该策略只作用于**直连**目标,因此配置了
390
- `proxy` 就会绕过它,只有 `blockCidrs`(由代理执行)能覆盖那条路径。这个开关保护的最锐利目标是云元数据地址
391
- `169.254.169.254`——在虚拟机上它会交出实例凭据。
392
-
393
348
  未设置 `exePath` 时探测顺序:`exePath` → 环境变量 `MOLI_BIN` → `%LOCALAPPDATA%\Moli\bin\moli.exe`
394
349
  → `%USERPROFILE%\.local\bin\moli.exe`,随后还会在 `%LOCALAPPDATA%\Moli` 内递归查找 `moli.exe`,
395
350
  所以装在带版本号的子目录里也能找到。
@@ -436,20 +391,23 @@ profile。无论生效的是哪个代理,loopback 地址一律绕过它。
436
391
  - `report-verify.mjs` 守住一份使用体验报告里的发现:`eval` 与 `dump` 同用会丢弃表达式值(已在参数描述里
437
392
  写明)、表达式抛异常时被正确归类而不是甩锅给"安装有问题"、截断同时保留头尾、`proxy` 是可按调用传入的参数、
438
393
  以及 `moli_probe` 会说明它的 `httpsOk` 描述的是哪个进程。
439
- - `session-verify.mjs` 用 `data:` URL 夹具端到端驱动 `moli_session`,因此不依赖网络、也不会因为站点变化而
440
- 失效:生命周期与端口释放、snapshot、click、带与不带清空的 type、press、scroll、两个方向的 `waitFor`、
441
- 含抛异常的 eval、截图字节签名、点击链接导航、关闭后重启,以及上面那条视口限制——限制被当作**限制**断言,
442
- 紧随其后断言真正能绕过它的 `eval` 路径。未安装引擎时跳过。
443
- - `session-backends.mjs` 是**证明第二个后端存在必要性**的对照:同一页面、同一个视口以下的控件,断言它
444
- **在 moli 上被拒绝、在 Chromium 上被点中**,并在两者上都验证 `cdp` 动作。机器上没有 Chromium 系浏览器时
445
- 跳过 Chromium 那半。
446
- - `session-attach-verify.mjs` 覆盖外部端点与归属权:连接不是本插件启动的浏览器、正确识别它、`close` 时拒绝
447
- 停掉它;以及那个**行为探测会搞错的陷阱**——外部端点是 **moli** 时仍判定为 moli,尽管它在 `/json/version`
448
- 里报的是 `"Browser":"Chrome/145.0.0.0"`。它还断言按调用传入的端点**不会粘住**——早期版本会:一次对死端点
449
- 的调用会让整个进程后续都指向 9 端口。
450
-
451
- `.github/workflows/verify.yml` 在 Linux、Windows、macOS 上跑前两个(无需引擎、无需网络),在 Linux 与 Windows
452
- 上尝试依赖引擎的几套;若 runner 上没有引擎,则报告为跳过而不是失败。
394
+ - `scroll-verify.mjs` 在无引擎时校验表达式构造(像素值、负数、`top`、`bottom`、无效输入被拒、调用方的 `eval`
395
+ 值被保留),有引擎时抓取一个"滚动处理器填充槽位"的页面,断言带 `scroll` 时内容出现、不带时只有占位符——
396
+ 断言的是**输出**的性质,而不是内部调用。
397
+ - `http-verify.mjs` 直接断言"参数 → 引擎选项"的映射,因为**参数没送到引擎和功能不生效,从外面看是一样的**。
398
+ 随后抓一个 404,验证 `failOnHttpError` 会报告失败同时保留响应体、健康页面不受影响、以及错误页的截图仍然被
399
+ 写为合法 PNG 而不是被当文本解码。端到端部分需要一个真实服务器的 HTTP 错误:**本沙箱内无法访问 loopback**
400
+ ——`curl.exe` 连 `127.0.0.1` 会超时,而 Node 宿主进程可以——所以它用 `example.com` 下的一个路径,网络不可用
401
+ 时跳过。
402
+
403
+ `.github/workflows/verify.yml` 在 Linux、Windows、macOS 上跑 `schema-verify`、`empty-verify` 与
404
+ `scroll-verify`(无需引擎、无需网络),在 Linux 与 Windows 上尝试依赖引擎的几套;若 runner 上没有引擎,
405
+ 则报告为跳过而不是失败。
406
+
407
+ `report-verify.mjs` **故意不放进 CI**:其中三项会抓 `example.com`,还有一项抓一个具体的论坛帖子,因此它
408
+ 需要能上外网——如果这台机器的流量走代理,还需要那个代理可用。有网络时它是有用的,但**一个因为网站改版就变
409
+ 红的测试套件,比没有套件更糟**,所以它只作为本机检查保留。如果你跑它时发现第 1–3 节通过而第 4–6 节失败,
410
+ **先怀疑网络,再怀疑代码**。
453
411
 
454
412
  此外,`lib/moli.js` 与工具层在 Windows 上对 Moli 1.1.16 做了端到端验证:注册、probe、Markdown 与 JSON
455
413
  dump、`eval`(含针对 `-t` 参数位置的紧超时回归防护)、PNG 与 PDF 产物签名、截断、全部拒绝路径(二元 dump
@@ -460,6 +418,19 @@ dump、`eval`(含针对 `-t` 参数位置的紧超时回归防护)、PNG 与
460
418
  30 处真实结果匹配。另有一个装了本插件的一次性 profile,组合其配置时既无 `JsonSchemaError` 也无被跳过的
461
419
  bundle。
462
420
 
421
+ ## 版本说明:1.1.11 与 1.1.12 已废弃
422
+
423
+ **请使用 1.1.13 或更高版本。** 这两个版本之间夹着一次"交互式浏览器"实验——`moli_session`、第二个
424
+ Chromium 后端、原始 CDP 逃生舱、按调用的 CDP 端点、持久化浏览器 profile——约 2300 行机制,远远超出了这个
425
+ 插件的定位。它已被移除,代码回到 1.1.10 的形态,只保留一处聚焦的新增。
426
+
427
+ 那次工作里唯一有价值的发现保留在 [`/lib/moli.js`](lib/moli.js),并以 `scroll` 参数的形式出现在上文。**它
428
+ 本来需要的其他一切,`moli fetch` 自带的 `--eval` 早就有了**——所以那次实验没有换来任何"一个参数换不来的
429
+ 东西"。
430
+
431
+ 关于这两个版本本身:npm 只允许在很短的时间窗口内撤回发布,且无法单独撤回某一个版本,因此 1.1.11 与
432
+ 1.1.12 仍然可安装,但已被标记为废弃并指向这里。`npm install dsh-moli` 会安装到 1.1.13。
433
+
463
434
  ## 发布
464
435
 
465
436
  发布走 GitHub Actions 的 OIDC 身份,**本机不存任何 npm token,因此没有会失效的凭据**:
@@ -484,6 +455,19 @@ E404 Not Found - PUT https://registry.npmjs.org/dsh-moli
484
455
  **这看起来像"包不存在",实际是认证失败。** 任何本地发布之前先跑
485
456
  `node scripts/preflight-publish.mjs`:它会告诉你凭据能否认证,不能时明确点出这个误导性的 404。
486
457
 
458
+ ### 废弃某个版本:去网站操作,别用命令行
459
+
460
+ 发布是脚本化的,**废弃则刻意不做成脚本**。用 npmjs.com 上的包页面——**package → Settings →
461
+ Deprecate**——选版本、写理由。
462
+
463
+ `npm deprecate dsh-moli@1.1.11 "…"` 在本机凭据失效时会给出上面那个同样误导的失败,而且这里还有第二个
464
+ 理由不要依赖 token:npm 在 **2026 年 7 月 31 日**起禁止 2FA-bypass 类型的 granular token 执行**包管理**
465
+ 操作,也就是说"以前能让 `deprecate` 脚本化"的那类凭据正在被取消;而 `publish` 有受支持的、免 token 的
466
+ OIDC 路径。**一个只在"偶尔废弃版本"时才需要 token 的流程,注定在下次需要它时是坏的**——1.1.11 与 1.1.12
467
+ 遇到的正是这种情况。
468
+
469
+ 1.0.0 与 1.0.1 的废弃就是通过网站完成的。
470
+
487
471
  ## 许可
488
472
 
489
473
  MIT,见 [LICENSE](LICENSE)。