dsh-moli 1.1.1 → 1.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +30 -6
- package/README.zh.md +22 -6
- package/index.js +8 -2
- package/lib/install.js +529 -92
- package/lib/moli.js +47 -3
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -119,7 +119,9 @@ moli_fetch url=https://example.com dump=markdown
|
|
|
119
119
|
|
|
120
120
|
`moli_probe` reports whether the engine is present. If it is not, pick one of:
|
|
121
121
|
|
|
122
|
-
**1. Let the plugin install it** — tries several download routes and reports a SHA256 of what it ran
|
|
122
|
+
**1. Let the plugin install it** — tries several download routes and reports a SHA256 of what it ran.
|
|
123
|
+
The whole run is asynchronous (the host keeps serving), and a configured `proxy` is passed to both
|
|
124
|
+
the download routes and the installer's own engine download:
|
|
123
125
|
|
|
124
126
|
```
|
|
125
127
|
moli_probe action=install
|
|
@@ -266,7 +268,7 @@ Works on **Windows, Linux and macOS** (x64 / arm64).
|
|
|
266
268
|
| Platform | Status |
|
|
267
269
|
|---|---|
|
|
268
270
|
| **Windows** | Verified by the maintainer |
|
|
269
|
-
| **Linux** | Verified by the contributor
|
|
271
|
+
| **Linux** | Verified by the contributor, not the maintainer: Ubuntu 24.04 LTS, Node v24.21.0, Moli 1.1.17 — including the async install path, proxy injection, and a `strace`-recorded end-to-end fetch |
|
|
270
272
|
| **macOS** | Implemented, not yet verified on a real host |
|
|
271
273
|
|
|
272
274
|
- **Windows** — the engine is located at the official installer paths (`%LOCALAPPDATA%\Moli\bin\moli.exe`,
|
|
@@ -276,6 +278,19 @@ Works on **Windows, Linux and macOS** (x64 / arm64).
|
|
|
276
278
|
`~/.local/share/moli/bin/moli`, `~/.local/share/moli/moli`, `~/.moli/bin/moli`) and on `$PATH`.
|
|
277
279
|
`moli_probe action=install` runs the official `moli-installer.sh` via `sh`.
|
|
278
280
|
|
|
281
|
+
**Installing never blocks the DSH host.** Both the installer download and the execution of the
|
|
282
|
+
official installer spawn asynchronously with a timeout that escalates from SIGTERM to SIGKILL, and
|
|
283
|
+
an interrupted turn aborts the child immediately — the host keeps serving while Moli's installer
|
|
284
|
+
does its work. A `strace -e trace=connect` run against a real install recorded every connection
|
|
285
|
+
going through the configured proxy port and none to the upstream host, and a fixture installer
|
|
286
|
+
confirmed the proxy arrives in the child's environment.
|
|
287
|
+
|
|
288
|
+
The configured `proxy` feeds **both** phases: the plugin's own download routes (`curl --proxy`, and
|
|
289
|
+
`-Proxy` on PowerShell's `Invoke-WebRequest`), and the official installer's internal engine download
|
|
290
|
+
— on Windows through a `DefaultWebProxy` prelude (PowerShell's `Invoke-WebRequest` ignores proxy
|
|
291
|
+
environment variables), on POSIX through the standard `HTTPS_PROXY`/`HTTP_PROXY`/`ALL_PROXY`
|
|
292
|
+
variables the installer's `curl` reads. A proxy carrying credentials sets them explicitly.
|
|
293
|
+
|
|
279
294
|
On Linux, `moli_probe` discovered `~/.local/bin/moli` unaided, and `moli_fetch` rendered a
|
|
280
295
|
region-gated storefront end to end — through a local forward proxy, with a Netscape `cookieFile` —
|
|
281
296
|
returning a 500+ KB page whose final URL carried the cookie-driven query parameters. That run
|
|
@@ -293,7 +308,7 @@ Override any field under the plugin's `config` entry in the profile patch layer.
|
|
|
293
308
|
exePath: 'C:\Users\me\AppData\Local\Moli\bin\moli.exe' # default: auto-detect
|
|
294
309
|
defaultTimeoutMs: 30000 # per-fetch readiness deadline
|
|
295
310
|
maxChars: 200000 # inline text truncation
|
|
296
|
-
proxy: 'http://127.0.0.1:7890' # forward proxy for moli and
|
|
311
|
+
proxy: 'http://127.0.0.1:7890' # forward proxy for moli, downloads, and the installer's engine download
|
|
297
312
|
insecureTls: true # see the TLS notes below
|
|
298
313
|
```
|
|
299
314
|
|
|
@@ -312,8 +327,10 @@ installation under a versioned subdirectory is still found.
|
|
|
312
327
|
- **Moli itself is not bundled.** Install it separately or let `moli_probe action=install` fetch it.
|
|
313
328
|
- **Only `fetch` is wrapped.** `moli serve` (CDP/WebDriver server) and `moli import` (session
|
|
314
329
|
import) are not exposed as tools yet.
|
|
315
|
-
- **No per-call cancellation.** The engine runs synchronously, so an
|
|
316
|
-
|
|
330
|
+
- **No per-call cancellation for fetches.** The engine runs synchronously for a fetch, so an
|
|
331
|
+
interrupted turn cannot abort it mid-flight; a call already in progress finishes and its artifact
|
|
332
|
+
is written. `moli_probe action=install` is the exception — it is asynchronous and aborts the child
|
|
333
|
+
immediately on an interrupted turn.
|
|
317
334
|
- **HTTP error status is only visible for `dump: json`.** That is the only dump format carrying the
|
|
318
335
|
status field. With `markdown` or `html` an error page is returned as content.
|
|
319
336
|
- **`writeTo` calls are serialized.** They report as unsafe to parallelize, so two artifact writes
|
|
@@ -336,13 +353,20 @@ installation under a versioned subdirectory is still found.
|
|
|
336
353
|
|
|
337
354
|
## Verification
|
|
338
355
|
|
|
339
|
-
|
|
356
|
+
Four suites live in [`verify/`](verify/), and all are runnable against a working tree:
|
|
340
357
|
|
|
341
358
|
- `schema-verify.mjs` checks the tool definitions with the registry's own validator, including
|
|
342
359
|
controls that assert the two invalid schema shapes still fail.
|
|
343
360
|
- `install-verify.mjs` drives the install pipeline against a loopback HTTP server: route ordering,
|
|
344
361
|
the SHA256 of served bytes, refusal to execute on a checksum mismatch, actual execution when the
|
|
345
362
|
digest matches, and the all-routes-failed diagnostic.
|
|
363
|
+
- `linux-verify.mjs` (POSIX hosts) asserts the proxy-helper functions, then against a real install
|
|
364
|
+
proves the two guarantees of the async rewrite: the installer child actually receives the proxy
|
|
365
|
+
through its environment (`HTTPS_PROXY` captured by a fixture installer), and the host event loop
|
|
366
|
+
stays live while a slow installer runs — the acceptance criterion for the async path.
|
|
367
|
+
- `win-verify.mjs` (Windows hosts) runs the same pure assertions, then executes the `DefaultWebProxy`
|
|
368
|
+
prelude in a real `powershell.exe` and inspects the proxy object and credentials it set, exercises
|
|
369
|
+
the Windows download routes, and runs `installMoli` end to end.
|
|
346
370
|
|
|
347
371
|
Beyond those, `lib/moli.js` and the tool layer were exercised end to end against Moli 1.1.16 on
|
|
348
372
|
Windows: registration, probe, Markdown and JSON dumps, `eval` (including a tight-timeout regression
|
package/README.zh.md
CHANGED
|
@@ -110,7 +110,8 @@ moli_fetch url=https://example.com dump=markdown
|
|
|
110
110
|
|
|
111
111
|
`moli_probe` 会报告引擎是否在场。如果不在,三种方式任选:
|
|
112
112
|
|
|
113
|
-
**1. 让插件替你装**——多路径下载,并报告实际执行字节的 SHA256
|
|
113
|
+
**1. 让插件替你装**——多路径下载,并报告实际执行字节的 SHA256。整个过程异步(主机照常服务),配置的
|
|
114
|
+
`proxy` 会同时传给下载路由与安装器内部的引擎下载:
|
|
114
115
|
|
|
115
116
|
```
|
|
116
117
|
moli_probe action=install
|
|
@@ -247,7 +248,7 @@ Moli 仓库自带官方 agent skill:[`moli-webfetch`](https://github.com/lexmo
|
|
|
247
248
|
| 平台 | 状态 |
|
|
248
249
|
|---|---|
|
|
249
250
|
| **Windows** | 由维护者验证 |
|
|
250
|
-
| **Linux** |
|
|
251
|
+
| **Linux** | **由贡献者验证,非维护者**:Ubuntu 24.04 LTS、Node v24.21.0、Moli 1.1.17——含异步安装路径、代理注入、以及 strace 记录的全链路抓取 |
|
|
251
252
|
| **macOS** | 已实现,尚未在真机验证 |
|
|
252
253
|
|
|
253
254
|
- **Windows** —— 引擎在官方安装位置查找(`%LOCALAPPDATA%\Moli\bin\moli.exe` 与 `%USERPROFILE%\.local\bin\moli.exe`),
|
|
@@ -255,6 +256,16 @@ Moli 仓库自带官方 agent skill:[`moli-webfetch`](https://github.com/lexmo
|
|
|
255
256
|
- **Linux / macOS** —— 引擎在标准位置(`~/.local/bin/moli`、`~/.local/share/moli/bin/moli`、
|
|
256
257
|
`~/.local/share/moli/moli`、`~/.moli/bin/moli`)与 `$PATH` 中查找;`moli_probe action=install` 通过 `sh` 执行官方 `moli-installer.sh`。
|
|
257
258
|
|
|
259
|
+
**安装绝不阻塞 DSH 主机。** 安装器的下载与执行都以异步 spawn 进行,超时先 SIGTERM 再 SIGKILL,
|
|
260
|
+
被中断的回合会立刻终止子进程——Moli 安装器干活期间主机照常服务。用 `strace -e trace=connect`
|
|
261
|
+
对真实安装抓包,记录的每次连接都走配置的代理端口、没有任何直连上游主机;fixture 安装器也确认
|
|
262
|
+
代理进入了子进程的环境。
|
|
263
|
+
|
|
264
|
+
配置的 `proxy` **同时喂两个阶段**:插件自己的下载路由(`curl --proxy`,以及 PowerShell `Invoke-WebRequest`
|
|
265
|
+
的 `-Proxy`),以及官方安装器内部的引擎下载——Windows 上通过 `DefaultWebProxy` prelude(PowerShell
|
|
266
|
+
的 `Invoke-WebRequest` 不读代理环境变量),POSIX 上通过安装器 `curl` 会读取的标准 `HTTPS_PROXY`/
|
|
267
|
+
`HTTP_PROXY`/`ALL_PROXY` 变量。带凭据的代理会显式设置凭据。
|
|
268
|
+
|
|
258
269
|
在 Linux 上,`moli_probe` 无需干预就找到了 `~/.local/bin/moli`;`moli_fetch` 端到端渲染了一个地区限制的
|
|
259
270
|
电商站——经由本地正向代理、并带上 Netscape 格式的 `cookieFile`——返回 500+ KB 的页面,最终 URL 带上了
|
|
260
271
|
cookie 驱动的查询参数。那次实测同时覆盖了代理下载路由与引擎探测路径。
|
|
@@ -271,7 +282,7 @@ cookie 驱动的查询参数。那次实测同时覆盖了代理下载路由与
|
|
|
271
282
|
exePath: 'C:\Users\me\AppData\Local\Moli\bin\moli.exe' # 默认自动探测
|
|
272
283
|
defaultTimeoutMs: 30000 # 单次抓取就绪超时
|
|
273
284
|
maxChars: 200000 # 内联文本截断阈值
|
|
274
|
-
proxy: 'http://127.0.0.1:7890' #
|
|
285
|
+
proxy: 'http://127.0.0.1:7890' # 转给本地正向代理(moli、下载、安装器内部引擎下载都走它)
|
|
275
286
|
insecureTls: true # 见下方 TLS 说明
|
|
276
287
|
```
|
|
277
288
|
|
|
@@ -287,8 +298,8 @@ cookie 驱动的查询参数。那次实测同时覆盖了代理下载路由与
|
|
|
287
298
|
[issue](https://github.com/27painter/dsh-moli/issues)**——那是这些路径能被打通的唯一途径。
|
|
288
299
|
- **不包含 Moli 本体。** 需自行安装,或让 `moli_probe action=install` 替你拉取。
|
|
289
300
|
- **只封装了 `fetch`。** `moli serve`(CDP/WebDriver 服务器)与 `moli import`(会话导入)尚未做成工具。
|
|
290
|
-
-
|
|
291
|
-
|
|
301
|
+
- **抓取不支持单次取消。** 引擎对抓取是同步执行的,被中断的回合无法中途终止;已在进行的调用会跑完并
|
|
302
|
+
写出产物。`moli_probe action=install` 是例外——它是异步的,被中断的回合会立刻终止子进程。
|
|
292
303
|
- **HTTP 错误码只在 `dump: json` 下可见。** 那是唯一携带状态字段的格式;用 `markdown` 或 `html`
|
|
293
304
|
时错误页会作为内容返回。
|
|
294
305
|
- **`writeTo` 调用被串行化。** 它们声明为不可并行,因此两次产物写入不会交错。
|
|
@@ -306,11 +317,16 @@ cookie 驱动的查询参数。那次实测同时覆盖了代理下载路由与
|
|
|
306
317
|
|
|
307
318
|
## 验证情况
|
|
308
319
|
|
|
309
|
-
[`verify/`](verify/)
|
|
320
|
+
[`verify/`](verify/) 下有四个可直接对工作区运行的测试:
|
|
310
321
|
|
|
311
322
|
- `schema-verify.mjs` 用注册表自己的校验器检查工具定义,并带有断言"两种非法 schema 形状仍会被拒"的对照组。
|
|
312
323
|
- `install-verify.mjs` 用本地回环 HTTP 服务器驱动安装链路:路由顺序、被提供字节的 SHA256、校验不匹配时
|
|
313
324
|
拒绝执行、匹配时真正执行、以及全部路径失败的诊断。
|
|
325
|
+
- `linux-verify.mjs`(POSIX 主机)断言代理辅助函数,然后对真实安装验证异步改造的两个保证:安装器子进程
|
|
326
|
+
确实通过环境收到代理(fixture 安装器捕获 `HTTPS_PROXY`),以及慢安装器运行期间主机事件循环保持活跃——
|
|
327
|
+
这正是异步路径的验收标准。
|
|
328
|
+
- `win-verify.mjs`(Windows 主机)跑同样的纯函数断言,然后在真实 `powershell.exe` 里执行 `DefaultWebProxy`
|
|
329
|
+
prelude 并检查它设置的代理对象与凭据,练习 Windows 下载路由,并端到端跑 `installMoli`。
|
|
314
330
|
|
|
315
331
|
此外,`lib/moli.js` 与工具层在 Windows 上对 Moli 1.1.16 做了端到端验证:注册、probe、Markdown 与 JSON
|
|
316
332
|
dump、`eval`(含针对 `-t` 参数位置的紧超时回归防护)、PNG 与 PDF 产物签名、截断、全部拒绝路径(二元 dump
|
package/index.js
CHANGED
|
@@ -110,6 +110,7 @@ const FETCH_OUTPUT_SCHEMA = {
|
|
|
110
110
|
path: { type: 'string' },
|
|
111
111
|
chars: { type: 'integer' },
|
|
112
112
|
bytes: { type: 'integer' },
|
|
113
|
+
empty: { type: 'boolean' },
|
|
113
114
|
truncated: { type: 'boolean' },
|
|
114
115
|
error: { type: 'string' },
|
|
115
116
|
hint: { type: 'string' },
|
|
@@ -300,7 +301,7 @@ export function apply(ctx, config) {
|
|
|
300
301
|
output: { schema: PROBE_OUTPUT_SCHEMA, render: renderProbe },
|
|
301
302
|
// Installation downloads and executes code, so it must never overlap itself.
|
|
302
303
|
isConcurrencySafe: args => args.action !== 'install',
|
|
303
|
-
async execute(args) {
|
|
304
|
+
async execute(args, exec) {
|
|
304
305
|
// An explicit install request is honoured even when an engine already
|
|
305
306
|
// exists, so the action can also repair or upgrade an installation. Anything
|
|
306
307
|
// else only inspects, because installing downloads and runs code.
|
|
@@ -309,14 +310,19 @@ export function apply(ctx, config) {
|
|
|
309
310
|
exePath: options.exePath,
|
|
310
311
|
expectedSha256: args.expectedSha256,
|
|
311
312
|
timeoutMs: options.installTimeoutMs ?? 180000,
|
|
313
|
+
proxy: options.proxy,
|
|
312
314
|
locate: findMoli,
|
|
313
315
|
candidates: moliCandidates(options.exePath),
|
|
316
|
+
signal: exec?.signal,
|
|
314
317
|
});
|
|
315
318
|
if (outcome.ok === true) {
|
|
316
319
|
cachedExe = outcome.exe;
|
|
317
320
|
ctx.logger?.info?.(`dsh-moli: installed ${String(outcome.version)} at ${String(outcome.exe)}`);
|
|
318
321
|
}
|
|
319
|
-
|
|
322
|
+
// The output schema declares `installed` as required, but installMoli's
|
|
323
|
+
// outcome carries `ok` instead. Surface both so the registry validator
|
|
324
|
+
// accepts the result and renderProbe still sees the shape it expects.
|
|
325
|
+
return { ...outcome, installed: outcome.ok === true };
|
|
320
326
|
}
|
|
321
327
|
|
|
322
328
|
const exe = engine();
|
package/lib/install.js
CHANGED
|
@@ -19,15 +19,28 @@
|
|
|
19
19
|
* - Installation is never a side effect. It runs only when a tool call asks for
|
|
20
20
|
* it, because it downloads and executes third-party code.
|
|
21
21
|
*
|
|
22
|
+
* Every long-running step runs through {@link runAsync}: a child process is
|
|
23
|
+
* spawned without blocking the event loop, the caller awaits its `close` event,
|
|
24
|
+
* and a timeout or abort signal kills it. The DSH host keeps serving requests
|
|
25
|
+
* while an install is in flight — including the official installer's own
|
|
26
|
+
* engine download, which can hang for a long time when the release host is
|
|
27
|
+
* unreachable without a proxy.
|
|
28
|
+
*
|
|
29
|
+
* Proxy handling is centralized at the head of this module (see the "Unified
|
|
30
|
+
* forward-proxy behaviour" section) and the platform decides how the official
|
|
31
|
+
* installer is fed that proxy: POSIX gets proxy environment variables (curl
|
|
32
|
+
* reads them), Windows gets a DefaultWebProxy prelude (Invoke-WebRequest reads
|
|
33
|
+
* that instead).
|
|
34
|
+
*
|
|
22
35
|
* @module dsh-moli/lib/install
|
|
23
36
|
*/
|
|
24
|
-
import { spawnSync } from 'node:child_process';
|
|
37
|
+
import { spawn, spawnSync } from 'node:child_process';
|
|
25
38
|
import { createHash } from 'node:crypto';
|
|
26
39
|
import { closeSync, existsSync, mkdtempSync, openSync, readFileSync, rmSync, statSync } from 'node:fs';
|
|
27
40
|
import { createConnection } from 'node:net';
|
|
28
41
|
import { tmpdir } from 'node:os';
|
|
29
42
|
import { join } from 'node:path';
|
|
30
|
-
import { findMoli
|
|
43
|
+
import { findMoli } from './moli.js';
|
|
31
44
|
|
|
32
45
|
/** Official installer for the latest release. */
|
|
33
46
|
export const INSTALLER_URL = process.platform === 'win32'
|
|
@@ -45,6 +58,137 @@ export const MANUAL_INSTALL = {
|
|
|
45
58
|
/** PowerShell prelude that makes older hosts negotiate TLS 1.2. */
|
|
46
59
|
const TLS_PRELUDE = '[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12;';
|
|
47
60
|
|
|
61
|
+
/**
|
|
62
|
+
* Unified forward-proxy behaviour for this pipeline.
|
|
63
|
+
*
|
|
64
|
+
* Every proxy decision — parsing/validation, liveness probing, curl argv, the
|
|
65
|
+
* PowerShell DefaultWebProxy prelude, and the child-process environment for the
|
|
66
|
+
* official installer — lives here, so the download routes and (later) the
|
|
67
|
+
* installer execution always agree on one proxy configuration instead of
|
|
68
|
+
* re-deriving it in each spot.
|
|
69
|
+
*
|
|
70
|
+
* The platforms disagree about how a proxy is honoured, and that fact is
|
|
71
|
+
* encoded here:
|
|
72
|
+
*
|
|
73
|
+
* - Linux / macOS — `curl` (used inside moli-installer.sh) reads the standard
|
|
74
|
+
* `HTTPS_PROXY` / `HTTP_PROXY` / `ALL_PROXY` environment variables, so
|
|
75
|
+
* injecting them into the installer's child process is enough.
|
|
76
|
+
* - Windows — `Invoke-WebRequest` (used inside moli-installer.ps1) does NOT
|
|
77
|
+
* read those variables; it honours the WinINET system proxy, i.e. .NET's
|
|
78
|
+
* `DefaultWebProxy`, so the installer must be preceded by a prelude that
|
|
79
|
+
* sets that object.
|
|
80
|
+
*/
|
|
81
|
+
|
|
82
|
+
/** Resolve a proxy URL into a structured configuration; undefined when unusable. */
|
|
83
|
+
function resolveProxy(input) {
|
|
84
|
+
if (typeof input !== 'string' || input.length === 0) return undefined;
|
|
85
|
+
let parsed;
|
|
86
|
+
try {
|
|
87
|
+
parsed = new URL(input);
|
|
88
|
+
} catch {
|
|
89
|
+
return undefined;
|
|
90
|
+
}
|
|
91
|
+
if (parsed.protocol !== 'http:' && parsed.protocol !== 'https:') return undefined;
|
|
92
|
+
const host = parsed.hostname === '' ? undefined : parsed.hostname;
|
|
93
|
+
// A proxy URL without an explicit port uses the scheme default (http 80,
|
|
94
|
+
// https 443); a bare `http://proxy` would otherwise parse as port 0 and be
|
|
95
|
+
// silently discarded while proxyListens probes port 80 for the same string.
|
|
96
|
+
const port = parsed.port === ''
|
|
97
|
+
? (parsed.protocol === 'https:' ? 443 : 80)
|
|
98
|
+
: Number(parsed.port);
|
|
99
|
+
if (host === undefined || !Number.isInteger(port) || port <= 0 || port > 65535) return undefined;
|
|
100
|
+
return {
|
|
101
|
+
url: input,
|
|
102
|
+
scheme: parsed.protocol.slice(0, -1),
|
|
103
|
+
host,
|
|
104
|
+
port,
|
|
105
|
+
auth: parsed.username !== '' || parsed.password !== ''
|
|
106
|
+
? { user: decodeURIComponent(parsed.username), pass: decodeURIComponent(parsed.password) }
|
|
107
|
+
: undefined,
|
|
108
|
+
};
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
/** Whether a URL targets loopback and must never be handed to a proxy. */
|
|
112
|
+
function isLoopbackUrl(url) {
|
|
113
|
+
return /^https?:\/\/(127\.0\.0\.1|localhost|\[::1\])/iu.test(url);
|
|
114
|
+
}
|
|
115
|
+
|
|
116
|
+
/** curl argv that routes a download through the proxy. */
|
|
117
|
+
function curlProxyArgs(proxy) {
|
|
118
|
+
const config = resolveProxy(proxy);
|
|
119
|
+
return config === undefined ? [] : ['--proxy', config.url];
|
|
120
|
+
}
|
|
121
|
+
|
|
122
|
+
/** curl argv that keeps a direct route off the proxy (loopback only). */
|
|
123
|
+
function curlBypassArgs(url) {
|
|
124
|
+
return isLoopbackUrl(url) ? ['--noproxy', '*'] : [];
|
|
125
|
+
}
|
|
126
|
+
|
|
127
|
+
/**
|
|
128
|
+
* PowerShell prelude that clears the default web proxy for a loopback download.
|
|
129
|
+
*
|
|
130
|
+
* PowerShell 5.1's `Invoke-WebRequest` has no `-NoProxy`, and the system proxy
|
|
131
|
+
* is read from the registry, so the only portable way to bypass it is to clear
|
|
132
|
+
* the default proxy object for the process.
|
|
133
|
+
*
|
|
134
|
+
* @param loopback - whether the target URL is loopback.
|
|
135
|
+
* @returns the PowerShell statement, or an empty string when not needed.
|
|
136
|
+
*/
|
|
137
|
+
function powershellProxyBypass(loopback) {
|
|
138
|
+
return loopback
|
|
139
|
+
? '[System.Net.WebRequest]::DefaultWebProxy = New-Object System.Net.WebProxy;'
|
|
140
|
+
: '';
|
|
141
|
+
}
|
|
142
|
+
|
|
143
|
+
/**
|
|
144
|
+
* PowerShell prelude that makes `Invoke-WebRequest` route through a proxy.
|
|
145
|
+
*
|
|
146
|
+
* `IWR` ignores `HTTPS_PROXY`/`HTTP_PROXY` environment variables; it honours
|
|
147
|
+
* the WinINET system proxy, i.e. .NET's `DefaultWebProxy`. Setting that object
|
|
148
|
+
* before running moli-installer.ps1 is what actually makes the Windows
|
|
149
|
+
* installer's internal engine download honour the configured proxy. When the
|
|
150
|
+
* proxy URL carries credentials, they are assigned explicitly as well: the
|
|
151
|
+
* `WebProxy` constructor does parse userinfo in the address, but an explicit
|
|
152
|
+
* `NetworkCredential` is the unambiguous form.
|
|
153
|
+
*
|
|
154
|
+
* @param proxy - proxy URL.
|
|
155
|
+
* @returns the PowerShell statement, or an empty string when absent.
|
|
156
|
+
*/
|
|
157
|
+
export function powershellProxyPrelude(proxy) {
|
|
158
|
+
const config = resolveProxy(proxy);
|
|
159
|
+
if (config === undefined) return '';
|
|
160
|
+
const esc = value => value.replace(/'/gu, "''");
|
|
161
|
+
let prelude = `$__dshMoliProxy = New-Object System.Net.WebProxy('${esc(config.url)}');`;
|
|
162
|
+
if (config.auth !== undefined) {
|
|
163
|
+
prelude += `$__dshMoliProxy.Credentials = New-Object System.Net.NetworkCredential('${esc(config.auth.user)}', '${esc(config.auth.pass)}');`;
|
|
164
|
+
}
|
|
165
|
+
prelude += `[System.Net.WebRequest]::DefaultWebProxy = $__dshMoliProxy;`;
|
|
166
|
+
return prelude;
|
|
167
|
+
}
|
|
168
|
+
|
|
169
|
+
/**
|
|
170
|
+
* Environment variables that make curl (inside moli-installer.sh) route
|
|
171
|
+
* through the proxy.
|
|
172
|
+
*
|
|
173
|
+
* Both cases are set because the tools disagree about which form they read:
|
|
174
|
+
* Windows builds of curl recognise the upper-case names, POSIX tools the
|
|
175
|
+
* lower-case ones.
|
|
176
|
+
*
|
|
177
|
+
* @param proxy - proxy URL.
|
|
178
|
+
* @returns an object to spread into the child's `env`; empty when absent.
|
|
179
|
+
*/
|
|
180
|
+
export function proxyEnv(proxy) {
|
|
181
|
+
const config = resolveProxy(proxy); if (config === undefined) return {};
|
|
182
|
+
return {
|
|
183
|
+
HTTPS_PROXY: config.url,
|
|
184
|
+
HTTP_PROXY: config.url,
|
|
185
|
+
https_proxy: config.url,
|
|
186
|
+
http_proxy: config.url,
|
|
187
|
+
ALL_PROXY: config.url,
|
|
188
|
+
all_proxy: config.url,
|
|
189
|
+
};
|
|
190
|
+
}
|
|
191
|
+
|
|
48
192
|
/**
|
|
49
193
|
* Check whether a forward proxy is actually accepting connections.
|
|
50
194
|
*
|
|
@@ -59,18 +203,10 @@ const TLS_PRELUDE = '[Net.ServicePointManager]::SecurityProtocol = [Net.Security
|
|
|
59
203
|
* @returns whether something is listening.
|
|
60
204
|
*/
|
|
61
205
|
export function proxyListens(proxy, timeoutMs = 700) {
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
try {
|
|
65
|
-
const parsed = new URL(proxy);
|
|
66
|
-
host = parsed.hostname === '' ? host : parsed.hostname;
|
|
67
|
-
port = Number(parsed.port);
|
|
68
|
-
} catch {
|
|
69
|
-
return false;
|
|
70
|
-
}
|
|
71
|
-
if (!Number.isInteger(port) || port <= 0) return false;
|
|
206
|
+
const config = resolveProxy(proxy);
|
|
207
|
+
if (config === undefined) return Promise.resolve(false);
|
|
72
208
|
return new Promise(resolve => {
|
|
73
|
-
const socket = createConnection({ host, port });
|
|
209
|
+
const socket = createConnection({ host: config.host, port: config.port });
|
|
74
210
|
const settle = value => {
|
|
75
211
|
socket.removeAllListeners();
|
|
76
212
|
socket.destroy();
|
|
@@ -93,25 +229,230 @@ export function sha256File(path) {
|
|
|
93
229
|
return createHash('sha256').update(readFileSync(path)).digest('hex');
|
|
94
230
|
}
|
|
95
231
|
|
|
232
|
+
/**
|
|
233
|
+
* Normalize stderr text into one readable line.
|
|
234
|
+
*
|
|
235
|
+
* Keep the leading meaningful lines and drop control bytes so the message
|
|
236
|
+
* survives a JSON round trip.
|
|
237
|
+
*
|
|
238
|
+
* @param text - raw stderr text.
|
|
239
|
+
* @returns the condensed message.
|
|
240
|
+
*/
|
|
241
|
+
function condenseStderr(text) {
|
|
242
|
+
return String(text)
|
|
243
|
+
.replace(/\u001b\[[0-9;]*m/gu, '')
|
|
244
|
+
.replace(/\r/gu, '')
|
|
245
|
+
.split('\n')
|
|
246
|
+
.map(line => line.trim())
|
|
247
|
+
.filter(line => line.length > 0)
|
|
248
|
+
.slice(0, 6)
|
|
249
|
+
.join(' | ');
|
|
250
|
+
}
|
|
251
|
+
|
|
252
|
+
/**
|
|
253
|
+
* Kill a process and everything it spawned.
|
|
254
|
+
*
|
|
255
|
+
* The process that actually does the work — the installer's internal engine
|
|
256
|
+
* download — is a grandchild: POSIX spawns `sh`, which runs `moli-installer.sh`,
|
|
257
|
+
* which runs `curl`; Windows spawns `powershell.exe -Command "... & script.ps1"`
|
|
258
|
+
* and the `Invoke-WebRequest` inside the script does the download. Killing only
|
|
259
|
+
* the direct child would leak that download: the plugin would report a timeout
|
|
260
|
+
* or abort while the installer kept pulling the ~100 MB engine in the temp dir.
|
|
261
|
+
*
|
|
262
|
+
* - POSIX: spawn with `detached: true` makes the child a process-group leader,
|
|
263
|
+
* so a negative pid signal reaches the whole group.
|
|
264
|
+
* - Windows: there is no process group to signal, so `taskkill /T` walks the
|
|
265
|
+
* tree instead.
|
|
266
|
+
*
|
|
267
|
+
* @param child - the direct child process.
|
|
268
|
+
* @param signal - SIGTERM or SIGKILL (POSIX only; Windows always force-kills).
|
|
269
|
+
*/
|
|
270
|
+
function killTree(child, signal) {
|
|
271
|
+
if (child === undefined || child.pid === undefined) return;
|
|
272
|
+
if (process.platform === 'win32') {
|
|
273
|
+
try {
|
|
274
|
+
spawnSync('taskkill', ['/pid', String(child.pid), '/T', '/F'], { stdio: 'ignore', windowsHide: true, timeout: 10000 });
|
|
275
|
+
} catch { /* best effort */ }
|
|
276
|
+
return;
|
|
277
|
+
}
|
|
278
|
+
try { process.kill(-child.pid, signal); } catch { /* best effort */ }
|
|
279
|
+
// Fall back to the direct child in case the group kill did not reach it.
|
|
280
|
+
try { child.kill(signal); } catch { /* best effort */ }
|
|
281
|
+
}
|
|
282
|
+
|
|
283
|
+
/**
|
|
284
|
+
* Run one command asynchronously, never blocking the event loop.
|
|
285
|
+
*
|
|
286
|
+
* stdout and stderr go to real files (the plugin's existing sandbox discipline:
|
|
287
|
+
* a child under DSH's Windows sandbox cannot open inherited pipes, so pipe
|
|
288
|
+
* capture is unreliable), and the caller awaits the `close` event. A timeout
|
|
289
|
+
* sends SIGTERM then SIGKILL to the whole process group; an abort signal kills
|
|
290
|
+
* immediately. The caller's event loop keeps serving while the child runs —
|
|
291
|
+
* including the official installer's internal engine download, which can hang
|
|
292
|
+
* without a proxy.
|
|
293
|
+
*
|
|
294
|
+
* @param options - command, argv, env, timeout, optional stdout file, abort signal.
|
|
295
|
+
* @returns the settled outcome; never throws for a non-zero exit.
|
|
296
|
+
*/
|
|
297
|
+
function runAsync(options) {
|
|
298
|
+
const dir = mkdtempSync(join(tmpdir(), 'dsh-moli-run-'));
|
|
299
|
+
const outPath = options.stdoutFile ?? join(dir, 'stdout.bin');
|
|
300
|
+
const errPath = join(dir, 'stderr.txt');
|
|
301
|
+
let outFd;
|
|
302
|
+
let errFd;
|
|
303
|
+
let timer;
|
|
304
|
+
let child;
|
|
305
|
+
let settled = false;
|
|
306
|
+
|
|
307
|
+
const settle = value => {
|
|
308
|
+
if (settled) return;
|
|
309
|
+
settled = true;
|
|
310
|
+
if (timer !== undefined) clearTimeout(timer);
|
|
311
|
+
if (child !== undefined) child.removeAllListeners();
|
|
312
|
+
if (outFd !== undefined) closeSync(outFd);
|
|
313
|
+
if (errFd !== undefined) closeSync(errFd);
|
|
314
|
+
outFd = undefined;
|
|
315
|
+
errFd = undefined;
|
|
316
|
+
const stdout = existsSync(outPath) ? readFileSync(outPath) : Buffer.alloc(0);
|
|
317
|
+
const stderrText = existsSync(errPath) ? readFileSync(errPath, 'utf8') : '';
|
|
318
|
+
rmSync(dir, { recursive: true, force: true });
|
|
319
|
+
resolve({ status: value.status, spawnError: value.spawnError, stdout, stderr: stderrText });
|
|
320
|
+
};
|
|
321
|
+
|
|
322
|
+
// Promise executor cannot be async; the resolve is captured via closure.
|
|
323
|
+
let resolve;
|
|
324
|
+
const promise = new Promise(r => { resolve = r; });
|
|
325
|
+
|
|
326
|
+
try {
|
|
327
|
+
outFd = openSync(outPath, 'w');
|
|
328
|
+
errFd = openSync(errPath, 'w');
|
|
329
|
+
} catch (error) {
|
|
330
|
+
rmSync(dir, { recursive: true, force: true });
|
|
331
|
+
resolve({ status: null, stdout: Buffer.alloc(0), stderr: '', spawnError: `open failed: ${error.message}` });
|
|
332
|
+
return promise;
|
|
333
|
+
}
|
|
334
|
+
|
|
335
|
+
try {
|
|
336
|
+
child = spawn(options.command, options.args, {
|
|
337
|
+
stdio: ['ignore', outFd, errFd],
|
|
338
|
+
env: options.env ?? process.env,
|
|
339
|
+
windowsHide: true,
|
|
340
|
+
// POSIX: the child becomes a process-group leader so killTree can reach
|
|
341
|
+
// the whole tree (sh → moli-installer.sh → curl) with one negative-pid
|
|
342
|
+
// signal. Windows must NOT detach: a detached console child's stdio
|
|
343
|
+
// handles become invalid and powershell.exe exits 0 without running its
|
|
344
|
+
// command, so killTree force-kills there via taskkill /T /F, which needs
|
|
345
|
+
// no detach at all.
|
|
346
|
+
detached: process.platform !== 'win32',
|
|
347
|
+
});
|
|
348
|
+
} catch (error) {
|
|
349
|
+
settle({ status: null, stdout: Buffer.alloc(0), stderr: '', spawnError: `${error.code ?? 'spawn failed'}: ${error.message}` });
|
|
350
|
+
return promise;
|
|
351
|
+
}
|
|
352
|
+
|
|
353
|
+
child.once('error', error => settle({
|
|
354
|
+
status: null,
|
|
355
|
+
stdout: Buffer.alloc(0),
|
|
356
|
+
stderr: '',
|
|
357
|
+
spawnError: `${error.code ?? 'spawn failed'}: ${error.message}`,
|
|
358
|
+
}));
|
|
359
|
+
child.once('close', code => settle({
|
|
360
|
+
status: code ?? 0,
|
|
361
|
+
stdout: Buffer.alloc(0), // populated by settle from the file
|
|
362
|
+
stderr: '',
|
|
363
|
+
}));
|
|
364
|
+
|
|
365
|
+
const timeoutMs = options.timeoutMs;
|
|
366
|
+
if (timeoutMs !== undefined) {
|
|
367
|
+
timer = setTimeout(() => {
|
|
368
|
+
if (settled || child === undefined) return;
|
|
369
|
+
killTree(child, 'SIGTERM');
|
|
370
|
+
// Give the tree a short grace period, then force it.
|
|
371
|
+
const killer = setTimeout(() => {
|
|
372
|
+
if (!settled && child !== undefined) {
|
|
373
|
+
killTree(child, 'SIGKILL');
|
|
374
|
+
}
|
|
375
|
+
}, 2000);
|
|
376
|
+
killer.unref?.();
|
|
377
|
+
}, timeoutMs);
|
|
378
|
+
}
|
|
379
|
+
|
|
380
|
+
if (options.signal !== undefined && options.signal !== null) {
|
|
381
|
+
if (options.signal.aborted === true) {
|
|
382
|
+
killTree(child, 'SIGKILL');
|
|
383
|
+
} else {
|
|
384
|
+
options.signal.addEventListener('abort', () => {
|
|
385
|
+
if (!settled && child !== undefined) {
|
|
386
|
+
killTree(child, 'SIGKILL');
|
|
387
|
+
}
|
|
388
|
+
}, { once: true });
|
|
389
|
+
}
|
|
390
|
+
}
|
|
391
|
+
|
|
392
|
+
return promise;
|
|
393
|
+
}
|
|
394
|
+
|
|
395
|
+
/**
|
|
396
|
+
* Run one command synchronously, blocking until it exits.
|
|
397
|
+
*
|
|
398
|
+
* This is the Windows path's runner: Windows is deliberately not
|
|
399
|
+
* asynchronized (its installer has bounded `Invoke-WebRequest` timeouts and
|
|
400
|
+
* the platform was not part of the Linux hang investigation), so the whole
|
|
401
|
+
* install stays synchronous there. stdout and stderr still go to real files —
|
|
402
|
+
* under DSH's Windows sandbox a child cannot open inherited pipes — and the
|
|
403
|
+
* caller gets the file contents read back.
|
|
404
|
+
*
|
|
405
|
+
* @param options - command, argv, env, timeout, optional stdout file.
|
|
406
|
+
* @returns the settled outcome; never throws for a non-zero exit.
|
|
407
|
+
*/
|
|
408
|
+
function runSync(options) {
|
|
409
|
+
const dir = mkdtempSync(join(tmpdir(), 'dsh-moli-run-'));
|
|
410
|
+
const outPath = options.stdoutFile ?? join(dir, 'stdout.bin');
|
|
411
|
+
const errPath = join(dir, 'stderr.txt');
|
|
412
|
+
let outFd;
|
|
413
|
+
let errFd;
|
|
414
|
+
try {
|
|
415
|
+
outFd = openSync(outPath, 'w');
|
|
416
|
+
errFd = openSync(errPath, 'w');
|
|
417
|
+
const result = spawnSync(options.command, options.args, {
|
|
418
|
+
stdio: ['ignore', outFd, errFd],
|
|
419
|
+
env: options.env ?? process.env,
|
|
420
|
+
timeout: options.timeoutMs,
|
|
421
|
+
windowsHide: true,
|
|
422
|
+
});
|
|
423
|
+
const stdout = existsSync(outPath) ? readFileSync(outPath) : Buffer.alloc(0);
|
|
424
|
+
const stderrText = existsSync(errPath) ? readFileSync(errPath, 'utf8') : '';
|
|
425
|
+
const failed = result.error !== undefined && result.error !== null;
|
|
426
|
+
return {
|
|
427
|
+
status: failed ? null : (result.status ?? 0),
|
|
428
|
+
stdout,
|
|
429
|
+
stderr: condenseStderr(stderrText),
|
|
430
|
+
spawnError: failed ? `${result.error.code ?? 'spawn failed'}: ${result.error.message}` : undefined,
|
|
431
|
+
};
|
|
432
|
+
} finally {
|
|
433
|
+
if (outFd !== undefined) closeSync(outFd);
|
|
434
|
+
if (errFd !== undefined) closeSync(errFd);
|
|
435
|
+
rmSync(dir, { recursive: true, force: true });
|
|
436
|
+
}
|
|
437
|
+
}
|
|
438
|
+
|
|
96
439
|
/**
|
|
97
440
|
* Run one download attempt, treating a zero-byte file as failure.
|
|
98
441
|
*
|
|
99
442
|
* @param options - argv, environment, and the target path to check afterwards.
|
|
100
443
|
* @returns the attempt outcome.
|
|
101
444
|
*/
|
|
102
|
-
function attempt(options) {
|
|
103
|
-
const
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
445
|
+
async function attempt(options) {
|
|
446
|
+
const outcome = await runAsync({
|
|
447
|
+
command: options.command,
|
|
448
|
+
args: options.args,
|
|
449
|
+
env: options.env,
|
|
450
|
+
timeoutMs: options.timeoutMs,
|
|
108
451
|
});
|
|
109
|
-
const failed = result.error !== undefined && result.error !== null;
|
|
110
|
-
const exit = failed ? null : (result.status ?? 0);
|
|
111
452
|
const bytes = existsSync(options.target) ? statSync(options.target).size : 0;
|
|
453
|
+
const failed = outcome.spawnError !== undefined;
|
|
454
|
+
const exit = failed ? null : outcome.status;
|
|
112
455
|
const ok = !failed && exit === 0 && bytes > 0;
|
|
113
|
-
// Keep the tail of stderr: a bare exit code cannot be acted on.
|
|
114
|
-
const stderr = typeof result.stderr === 'string' ? result.stderr.trim().split('\n').slice(-2).join(' ').slice(0, 240) : '';
|
|
115
456
|
return {
|
|
116
457
|
method: options.method,
|
|
117
458
|
ok,
|
|
@@ -120,10 +461,44 @@ function attempt(options) {
|
|
|
120
461
|
error: ok
|
|
121
462
|
? undefined
|
|
122
463
|
: failed
|
|
123
|
-
?
|
|
464
|
+
? outcome.spawnError
|
|
124
465
|
: exit === 0
|
|
125
466
|
? 'the download produced an empty file'
|
|
126
|
-
: `exit ${String(exit)}${stderr === '' ? '' : `: ${stderr}`}`,
|
|
467
|
+
: `exit ${String(exit)}${outcome.stderr === '' ? '' : `: ${outcome.stderr}`}`,
|
|
468
|
+
};
|
|
469
|
+
}
|
|
470
|
+
|
|
471
|
+
/**
|
|
472
|
+
* Run a stdout-producing download route (the moli self-bootstrap) to the target.
|
|
473
|
+
*
|
|
474
|
+
* @param route - the route descriptor with `command` and `args`.
|
|
475
|
+
* @param target - the file the child's stdout is written to.
|
|
476
|
+
* @param timeoutMs - download deadline.
|
|
477
|
+
* @returns the attempt outcome.
|
|
478
|
+
*/
|
|
479
|
+
async function attemptStdout(route, target, timeoutMs) {
|
|
480
|
+
const outcome = await runAsync({
|
|
481
|
+
command: route.command,
|
|
482
|
+
args: route.args,
|
|
483
|
+
env: { ...process.env, MOLI_DL_TARGET: target },
|
|
484
|
+
timeoutMs,
|
|
485
|
+
stdoutFile: target,
|
|
486
|
+
});
|
|
487
|
+
const bytes = existsSync(target) ? statSync(target).size : 0;
|
|
488
|
+
const failed = outcome.spawnError !== undefined;
|
|
489
|
+
const ok = !failed && outcome.status === 0 && bytes > 0;
|
|
490
|
+
return {
|
|
491
|
+
method: route.method,
|
|
492
|
+
ok,
|
|
493
|
+
exit: failed ? null : outcome.status,
|
|
494
|
+
bytes,
|
|
495
|
+
error: ok
|
|
496
|
+
? undefined
|
|
497
|
+
: failed
|
|
498
|
+
? outcome.spawnError
|
|
499
|
+
: bytes === 0
|
|
500
|
+
? 'empty output'
|
|
501
|
+
: `exit ${String(outcome.status)}`,
|
|
127
502
|
};
|
|
128
503
|
}
|
|
129
504
|
|
|
@@ -136,56 +511,54 @@ function attempt(options) {
|
|
|
136
511
|
export async function downloadInstaller(options) {
|
|
137
512
|
const target = options.target;
|
|
138
513
|
const url = options.installerUrl ?? INSTALLER_URL;
|
|
139
|
-
const configured = typeof options.proxy === 'string' && options.proxy.length > 0 ? options.proxy : undefined;
|
|
140
514
|
const timeoutMs = options.timeoutMs ?? 60000;
|
|
141
515
|
const powershell = options.powershell ?? (process.platform === 'win32' ? 'powershell.exe' : 'powershell');
|
|
516
|
+
const onWindows = process.platform === 'win32';
|
|
517
|
+
|
|
518
|
+
// Head: one proxy decision, then every route below agrees on it.
|
|
519
|
+
const proxyConfig = resolveProxy(options.proxy);
|
|
520
|
+
const proxyAlive = proxyConfig === undefined ? false : await proxyListens(proxyConfig.url);
|
|
521
|
+
const proxy = proxyConfig === undefined ? undefined : proxyConfig.url;
|
|
142
522
|
|
|
143
523
|
// Never hand a loopback URL to a proxy — with a stale system proxy setting the
|
|
144
524
|
// request would go to a dead port and wait out the connect timeout. Windows
|
|
145
525
|
// PowerShell 5.1 has no `-NoProxy` on Invoke-WebRequest, and the system proxy is
|
|
146
526
|
// read from the registry, so the only portable way to bypass it is to clear the
|
|
147
527
|
// default proxy object for the process.
|
|
148
|
-
const
|
|
149
|
-
const bypass = loopback ? ' [System.Net.WebRequest]::DefaultWebProxy = New-Object System.Net.WebProxy;' : '';
|
|
150
|
-
const curlBypass = loopback ? ['--noproxy', '*'] : [];
|
|
528
|
+
const bypass = powershellProxyBypass(isLoopbackUrl(url));
|
|
151
529
|
|
|
152
530
|
const iwr = extra => [
|
|
153
531
|
'-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Bypass', '-Command',
|
|
154
532
|
`${TLS_PRELUDE}${bypass} Invoke-WebRequest -UseBasicParsing ${extra} -Uri '${url}' -OutFile $env:MOLI_DL_TARGET -TimeoutSec ${String(Math.ceil(timeoutMs / 1000))}`,
|
|
155
533
|
];
|
|
156
534
|
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
method: 'Invoke-WebRequest (direct)',
|
|
161
|
-
command: powershell,
|
|
162
|
-
args: iwr(''),
|
|
163
|
-
},
|
|
164
|
-
]
|
|
165
|
-
: [];
|
|
166
|
-
|
|
167
|
-
// Only attempt proxy routes when something is actually listening there.
|
|
168
|
-
let proxyAlive = false;
|
|
169
|
-
if (configured !== undefined) {
|
|
170
|
-
proxyAlive = await proxyListens(configured);
|
|
171
|
-
}
|
|
172
|
-
if (process.platform === 'win32' && configured !== undefined && proxyAlive) {
|
|
535
|
+
// Route order is platform-aware, in order of least surprise.
|
|
536
|
+
const routes = [];
|
|
537
|
+
if (onWindows) {
|
|
173
538
|
routes.push({
|
|
174
|
-
method:
|
|
539
|
+
method: 'Invoke-WebRequest (direct)',
|
|
175
540
|
command: powershell,
|
|
176
|
-
args: iwr(
|
|
541
|
+
args: iwr(''),
|
|
177
542
|
});
|
|
543
|
+
// Only attempt proxy routes when something is actually listening there.
|
|
544
|
+
if (proxy !== undefined && proxyAlive) {
|
|
545
|
+
routes.push({
|
|
546
|
+
method: `Invoke-WebRequest (proxy ${proxy})`,
|
|
547
|
+
command: powershell,
|
|
548
|
+
args: iwr(`-Proxy '${proxy}'`),
|
|
549
|
+
});
|
|
550
|
+
}
|
|
178
551
|
}
|
|
179
552
|
routes.push({
|
|
180
553
|
method: 'curl (direct)',
|
|
181
|
-
command:
|
|
182
|
-
args: ['-sSL', ...
|
|
554
|
+
command: onWindows ? 'curl.exe' : 'curl',
|
|
555
|
+
args: ['-sSL', ...curlBypassArgs(url), '--max-time', String(Math.ceil(timeoutMs / 1000)), '-o', target, url],
|
|
183
556
|
});
|
|
184
|
-
if (
|
|
557
|
+
if (proxy !== undefined && proxyAlive) {
|
|
185
558
|
routes.push({
|
|
186
|
-
method: `curl (proxy ${
|
|
187
|
-
command:
|
|
188
|
-
args: ['-sSL', ...
|
|
559
|
+
method: `curl (proxy ${proxy})`,
|
|
560
|
+
command: onWindows ? 'curl.exe' : 'curl',
|
|
561
|
+
args: ['-sSL', ...curlBypassArgs(url), '--max-time', String(Math.ceil(timeoutMs / 1000)), ...curlProxyArgs(proxy), '-o', target, url],
|
|
189
562
|
});
|
|
190
563
|
}
|
|
191
564
|
// Moli can bootstrap itself: when it is present but broken, or when reinstalling,
|
|
@@ -195,40 +568,28 @@ export async function downloadInstaller(options) {
|
|
|
195
568
|
routes.push({
|
|
196
569
|
method: 'moli fetch (self-bootstrap)',
|
|
197
570
|
command: exe,
|
|
198
|
-
args: ['fetch', url, '--insecure-disable-tls-host-verification', ...(
|
|
199
|
-
stdoutToFile:
|
|
571
|
+
args: ['fetch', url, '--insecure-disable-tls-host-verification', ...(proxy === undefined || !proxyAlive ? [] : ['--http-proxy', proxy])],
|
|
572
|
+
stdoutToFile: true,
|
|
200
573
|
});
|
|
201
574
|
}
|
|
202
575
|
|
|
203
576
|
const log = [];
|
|
204
577
|
for (const route of routes) {
|
|
205
578
|
if (existsSync(target)) rmSync(target, { force: true });
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
const result = spawnSync(route.command, route.args, { stdio: ['ignore', fd, 'ignore'], timeout: timeoutMs, windowsHide: true });
|
|
221
|
-
closeSync(fd);
|
|
222
|
-
const bytes = existsSync(target) ? statSync(target).size : 0;
|
|
223
|
-
const failed = result.error !== undefined && result.error !== null;
|
|
224
|
-
outcome = {
|
|
225
|
-
method: route.method,
|
|
226
|
-
ok: !failed && (result.status ?? 0) === 0 && bytes > 0,
|
|
227
|
-
exit: failed ? null : (result.status ?? 0),
|
|
228
|
-
bytes,
|
|
229
|
-
error: failed ? `${result.error.code ?? 'spawn failed'}` : bytes === 0 ? 'empty output' : undefined,
|
|
230
|
-
};
|
|
231
|
-
}
|
|
579
|
+
// Every route runs asynchronously on every platform: the child process is
|
|
580
|
+
// spawned without blocking the DSH host's event loop, and the per-route
|
|
581
|
+
// timeout keeps a hung download from stalling the host. Windows and POSIX
|
|
582
|
+
// behave identically here.
|
|
583
|
+
const outcome = route.stdoutToFile === undefined
|
|
584
|
+
? await attempt({
|
|
585
|
+
method: route.method,
|
|
586
|
+
command: route.command,
|
|
587
|
+
args: route.args,
|
|
588
|
+
target,
|
|
589
|
+
timeoutMs,
|
|
590
|
+
env: { ...process.env, MOLI_DL_TARGET: target },
|
|
591
|
+
})
|
|
592
|
+
: await attemptStdout(route, target, timeoutMs);
|
|
232
593
|
log.push({ method: outcome.method, ok: outcome.ok, bytes: outcome.bytes, error: outcome.error });
|
|
233
594
|
if (outcome.ok) {
|
|
234
595
|
return { ok: true, path: target, bytes: outcome.bytes, sha256: sha256File(target), method: outcome.method, attempts: log };
|
|
@@ -237,10 +598,43 @@ export async function downloadInstaller(options) {
|
|
|
237
598
|
return { ok: false, attempts: log, error: `every download route failed (${String(log.length)} tried)` };
|
|
238
599
|
}
|
|
239
600
|
|
|
601
|
+
/**
|
|
602
|
+
* Read the engine's version banner asynchronously.
|
|
603
|
+
*
|
|
604
|
+
* @param exe - absolute path to the moli binary.
|
|
605
|
+
* @param options - timeout and abort signal.
|
|
606
|
+
* @returns the version string, or undefined when the call failed.
|
|
607
|
+
*/
|
|
608
|
+
async function moliVersionAsync(exe, options = {}) {
|
|
609
|
+
const outcome = await runAsync({
|
|
610
|
+
command: exe,
|
|
611
|
+
args: ['--version'],
|
|
612
|
+
timeoutMs: options.timeoutMs ?? 15000,
|
|
613
|
+
signal: options.signal,
|
|
614
|
+
});
|
|
615
|
+
if (outcome.status !== 0 || outcome.spawnError !== undefined) return undefined;
|
|
616
|
+
const text = outcome.stdout.toString('utf8').trim();
|
|
617
|
+
return text === '' ? undefined : text;
|
|
618
|
+
}
|
|
619
|
+
|
|
620
|
+
/**
|
|
621
|
+
* Synchronous twin of {@link moliVersionAsync}, used on Windows.
|
|
622
|
+
*
|
|
623
|
+
* @param exe - absolute path to the moli binary.
|
|
624
|
+
* @param timeoutMs - version-call deadline.
|
|
625
|
+
* @returns the version string, or undefined when the call failed.
|
|
626
|
+
*/
|
|
627
|
+
function moliVersionSync(exe, timeoutMs = 15000) {
|
|
628
|
+
const outcome = runSync({ command: exe, args: ['--version'], timeoutMs });
|
|
629
|
+
if (outcome.status !== 0 || outcome.spawnError !== undefined) return undefined;
|
|
630
|
+
const text = outcome.stdout.toString('utf8').trim();
|
|
631
|
+
return text === '' ? undefined : text;
|
|
632
|
+
}
|
|
633
|
+
|
|
240
634
|
/**
|
|
241
635
|
* Install Moli and confirm it answers.
|
|
242
636
|
*
|
|
243
|
-
* @param options - expected checksum, timeouts, proxy, and injection points for tests.
|
|
637
|
+
* @param options - expected checksum, timeouts, proxy, abort signal, and injection points for tests.
|
|
244
638
|
* @returns the settled outcome value.
|
|
245
639
|
*/
|
|
246
640
|
export async function installMoli(options = {}) {
|
|
@@ -248,10 +642,15 @@ export async function installMoli(options = {}) {
|
|
|
248
642
|
const workDir = mkdtempSync(join(tmpdir(), 'dsh-moli-install-'));
|
|
249
643
|
const target = join(workDir, process.platform === 'win32' ? 'moli-installer.ps1' : 'moli-installer.sh');
|
|
250
644
|
try {
|
|
645
|
+
// Head: one proxy decision, shared by the download below and the run after it.
|
|
646
|
+
const proxyConfig = resolveProxy(options.proxy);
|
|
647
|
+
const proxyAlive = proxyConfig === undefined ? false : await proxyListens(proxyConfig.url);
|
|
648
|
+
const proxy = proxyConfig === undefined ? undefined : proxyConfig.url;
|
|
649
|
+
|
|
251
650
|
const download = await downloadInstaller({
|
|
252
651
|
target,
|
|
253
652
|
installerUrl: options.installerUrl,
|
|
254
|
-
proxy
|
|
653
|
+
proxy,
|
|
255
654
|
timeoutMs: options.downloadTimeoutMs ?? 60000,
|
|
256
655
|
locate: options.locate ?? findMoli,
|
|
257
656
|
exePath: options.exePath,
|
|
@@ -283,23 +682,61 @@ export async function installMoli(options = {}) {
|
|
|
283
682
|
}
|
|
284
683
|
}
|
|
285
684
|
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
685
|
+
// Run the official installer, routing its internal engine download through
|
|
686
|
+
// the proxy the way each platform honours it:
|
|
687
|
+
// - POSIX (Linux/macOS): moli-installer.sh uses curl, which reads the proxy
|
|
688
|
+
// environment variables, so injecting them is enough.
|
|
689
|
+
// - Windows: moli-installer.ps1 uses Invoke-WebRequest, which ignores env
|
|
690
|
+
// variables and honours .NET's DefaultWebProxy, so the script must be
|
|
691
|
+
// preceded by a prelude that sets that object.
|
|
692
|
+
const env = { ...process.env, ...(proxyAlive ? proxyEnv(proxy) : {}) };
|
|
693
|
+
let args;
|
|
694
|
+
if (process.platform === 'win32') {
|
|
695
|
+
const prelude = `${TLS_PRELUDE}${proxyAlive ? powershellProxyPrelude(proxy) : ''}`;
|
|
696
|
+
const script = `& '${target.replace(/'/gu, "''")}'`;
|
|
697
|
+
args = ['-NoProfile', '-NonInteractive', '-ExecutionPolicy', 'Bypass', '-Command', `${prelude} ${script}`];
|
|
698
|
+
} else {
|
|
699
|
+
args = [target];
|
|
700
|
+
}
|
|
701
|
+
let run;
|
|
702
|
+
if (process.platform === 'win32') {
|
|
703
|
+
// Windows runs asynchronously too: spawn the installer without blocking
|
|
704
|
+
// the DSH host's event loop. The outer timeout sends SIGTERM then
|
|
705
|
+
// SIGKILL, and an abort signal kills it immediately, so the host keeps
|
|
706
|
+
// serving requests while PowerShell's Invoke-WebRequest does its work.
|
|
707
|
+
run = await runAsync({
|
|
708
|
+
command: options.powershell ?? 'powershell.exe',
|
|
709
|
+
args,
|
|
710
|
+
env,
|
|
711
|
+
timeoutMs,
|
|
712
|
+
signal: options.signal,
|
|
713
|
+
});
|
|
714
|
+
} else {
|
|
715
|
+
// Linux/macOS keeps its existing asynchronous behaviour.
|
|
716
|
+
run = await runAsync({
|
|
717
|
+
command: 'sh',
|
|
718
|
+
args,
|
|
719
|
+
env,
|
|
720
|
+
timeoutMs,
|
|
721
|
+
signal: options.signal,
|
|
722
|
+
});
|
|
723
|
+
}
|
|
724
|
+
const runFailed = run.spawnError !== undefined;
|
|
725
|
+
const runStatus = runFailed ? null : run.status;
|
|
293
726
|
|
|
294
727
|
const locate = options.locate ?? findMoli;
|
|
295
728
|
const exe = locate(options.exePath);
|
|
296
|
-
const version = exe === undefined
|
|
729
|
+
const version = exe === undefined
|
|
730
|
+
? undefined
|
|
731
|
+
: process.platform === 'win32'
|
|
732
|
+
? moliVersionSync(exe)
|
|
733
|
+
: await moliVersionAsync(exe, { signal: options.signal });
|
|
297
734
|
|
|
298
735
|
if (version === undefined) {
|
|
299
736
|
return {
|
|
300
737
|
ok: false,
|
|
301
738
|
error: runFailed
|
|
302
|
-
? `running the installer failed: ${run.
|
|
739
|
+
? `running the installer failed: ${run.spawnError}`
|
|
303
740
|
: `the installer ${runStatus === 0 ? 'reported success' : `exited ${String(runStatus)}`}, but no working moli binary was found afterwards`,
|
|
304
741
|
installerSha256: download.sha256,
|
|
305
742
|
installerBytes: download.bytes,
|
package/lib/moli.js
CHANGED
|
@@ -314,7 +314,45 @@ export function runFetch(options) {
|
|
|
314
314
|
const bytes = outcome.stdout.length;
|
|
315
315
|
const stderr = outcome.spawnError === undefined ? outcome.stderr : `${outcome.spawnError} | ${outcome.stderr}`;
|
|
316
316
|
|
|
317
|
-
|
|
317
|
+
// Empty stdout on a zero exit is not a failure: a page can legitimately have
|
|
318
|
+
// no content in the requested format, which is exactly what a client-rendered
|
|
319
|
+
// page yields under `noJs` with a text dump. moli is right to print nothing
|
|
320
|
+
// for it, and the old code answered with "moli exited with status 0 and no
|
|
321
|
+
// output" plus a hint to re-check the install, sending the caller after a host
|
|
322
|
+
// that was never broken.
|
|
323
|
+
//
|
|
324
|
+
// Both decisions live in this one branch on purpose. Splitting them -- empty
|
|
325
|
+
// stdout treated as success here and stderr checked further down -- lets the
|
|
326
|
+
// case below (silent stdout, a complaint on stderr, exit 0) fall through to
|
|
327
|
+
// the normal success path with the complaint silently dropped.
|
|
328
|
+
if (outcome.status === 0 && bytes === 0) {
|
|
329
|
+
if (stderr !== '') {
|
|
330
|
+
return {
|
|
331
|
+
ok: false,
|
|
332
|
+
url: args.url,
|
|
333
|
+
error: stderr,
|
|
334
|
+
sandboxTlsBlock: isSandboxTlsFailure(stderr),
|
|
335
|
+
unreachable: false,
|
|
336
|
+
possiblyUnreachable: false,
|
|
337
|
+
hint: 'The engine exited cleanly but produced no output and explained why on stderr; the text above is its own message.',
|
|
338
|
+
};
|
|
339
|
+
}
|
|
340
|
+
return {
|
|
341
|
+
ok: true,
|
|
342
|
+
url: args.url,
|
|
343
|
+
kind: 'text',
|
|
344
|
+
dump: args.dump ?? 'html',
|
|
345
|
+
content: '',
|
|
346
|
+
chars: 0,
|
|
347
|
+
bytes: 0,
|
|
348
|
+
empty: true,
|
|
349
|
+
note: 'moli succeeded and produced no output, so the page had no content in this format. '
|
|
350
|
+
+ 'A page whose content is built by JavaScript yields nothing when noJs is set; '
|
|
351
|
+
+ 'the same URL renders normally without it.',
|
|
352
|
+
};
|
|
353
|
+
}
|
|
354
|
+
|
|
355
|
+
if (outcome.status !== 0) {
|
|
318
356
|
const sandboxTls = isSandboxTlsFailure(stderr);
|
|
319
357
|
// A blocked host fails at connect time ("Failed to connect" / RST) or, when
|
|
320
358
|
// the socket never opens, as a readiness timeout with nothing to show. A
|
|
@@ -459,7 +497,11 @@ export function probe(exe, probeUrl, insecureTls = false) {
|
|
|
459
497
|
if (insecureTls) probeArgs.push('--insecure-disable-tls-host-verification');
|
|
460
498
|
const outcome = fetchOnce({ exe, args: probeArgs, timeoutMs: 20000 });
|
|
461
499
|
const stderr = outcome.spawnError === undefined ? outcome.stderr : `${outcome.spawnError} | ${outcome.stderr}`;
|
|
462
|
-
|
|
500
|
+
// A zero exit means moli reached the host and fetched it. Requiring non-empty
|
|
501
|
+
// stdout as well would report httpsOk:false for a page that simply has no
|
|
502
|
+
// text — a probe URL is not guaranteed to carry content, and the field is
|
|
503
|
+
// documented as "can reach HTTPS", which is what the exit status proves.
|
|
504
|
+
const ok = outcome.status === 0;
|
|
463
505
|
const revocation = /CRYPT_E_REVOCATION_OFFLINE|revocation/iu.test(stderr);
|
|
464
506
|
return {
|
|
465
507
|
exe,
|
|
@@ -467,7 +509,9 @@ export function probe(exe, probeUrl, insecureTls = false) {
|
|
|
467
509
|
httpsOk: ok,
|
|
468
510
|
probeUrl,
|
|
469
511
|
detail: ok
|
|
470
|
-
?
|
|
512
|
+
? (outcome.stdout.length > 0
|
|
513
|
+
? 'HTTPS fetch succeeded from the DSH host process.'
|
|
514
|
+
: 'HTTPS fetch succeeded from the DSH host process (the probe URL returned no text).')
|
|
471
515
|
: revocation && !insecureTls
|
|
472
516
|
? `${stderr} | This is a certificate-revocation check failing (offline CRL/OCSP responder); retrying with the no-verify flag normally clears it.`
|
|
473
517
|
: stderr,
|
package/package.json
CHANGED