dsh-web-fetch-playwright 0.2.11 → 0.2.12

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -111,7 +111,7 @@ Some strict sites serve a Cloudflare interstitial before the real page. A real b
111
111
 
112
112
  With the wait on (default):
113
113
 
114
- 1. **Detection** — a response carrying `cf-mitigated: challenge` (the documented signal for every challenge page type), or a 403/503 HTML document from a `server: cloudflare` edge, or the localized interstitial itself (title family like "Just a moment…", `请稍候…`, "Минутку…", plus structural markers: `/cdn-cgi/challenge-platform/` scripts, `#challenge-*` elements, `cf-chl-widget-` frames, `window._cf_chl_opt`). The content-level markers are the *fallback tier* and only run on challenge-compatible responses — 403/429/503 or a Cloudflare edge (`server: cloudflare` / `cf-ray`) — because interstitials never ship as a plain 200, so an ordinary article that merely quotes challenge text can never be mistaken for one. Cloudflare Bot Management's passive JavaScript-Detections telemetry (`/cdn-cgi/challenge-platform/scripts/jsd/`, injected into every *normal* page of a protected zone — e.g. openrouter.ai) is neutralized before the prefix scan, so protected 200 pages with real content never misclassify. A hard block ("Sorry, you have been blocked") is classified separately and fails immediately.
114
+ 1. **Detection** — a response carrying `cf-mitigated: challenge` (the documented signal for every challenge page type), or a 403/503 HTML document from a `server: cloudflare` edge, or the localized interstitial itself (title family like "Just a moment…", `请稍候…`, "Минутку…", plus structural markers: the challenge-platform runtime's `<script src>`, `#challenge-*` elements, `cf-chl-widget-` frames, `window._cf_chl_opt`). The content-level markers are the *fallback tier* and only run on challenge-compatible responses — 403/429/503 or a Cloudflare edge (`server: cloudflare` / `cf-ray`) — because interstitials never ship as a plain 200, so an ordinary article that merely quotes challenge text can never be mistaken for one. Two refinements keep that promise for real pages: the challenge-platform script check reads real `<script src>` attributes only (prose, and escaped code samples that quote a path, are ignored), and it matches the runtime mount `/cdn-cgi/challenge-platform/h/[b|g]/orchestrate/` *positively*. Everything else Cloudflare injects under that prefix — Bot Management's JavaScript Detections (`scripts/jsd/`, `h/[b|g]/scripts/jsd/<hash>/`, `h/[b|g]/jsd/r/`), and Precursor (`scripts/precursor/`) — is passive instrumentation present on *every normal page* of a protected zone (e.g. openrouter.ai, nodeseek.com), so it is never read as a challenge. A hard block ("Sorry, you have been blocked") is classified separately and fails immediately.
115
115
  2. **Bounded wait, same tab and context** — the fetch polls the live DOM (500ms interval) for the challenge to disappear while the browser runs its own verification; the *last* main-frame navigation response is tracked so the reloaded real document's status and headers are the ones reported. SPA-style clears (content swapped without any navigation) are caught by the same DOM probe.
116
116
  3. **Bounded retry** — when a window runs out, the same tab re-navigates once (default; `challengeRetries`) with whatever clearance cookies the context already holds.
117
117
  4. **Clear failure** — `WEB_FETCH_CHALLENGE` (a provider-specific code the web seam's open-string `code` allows) naming the site, the budget spent, and the last challenge status.
package/README.zh-CN.md CHANGED
@@ -111,7 +111,7 @@ google-chrome --remote-debugging-port=9222 --user-data-dir="$HOME/.config/chrome
111
111
 
112
112
  开启等待(默认)后的流程:
113
113
 
114
- 1. **识别** —— 响应带 `cf-mitigated: challenge`(Cloudflare 官方文档注明所有挑战页类型都带此头),或 403/503 且 `server: cloudflare` 的 HTML 文档,或本地化的中间页本身("Just a moment…" / "请稍候…" / "Минутку…" 等 title 家族,以及结构性标记:`/cdn-cgi/challenge-platform/` 脚本、`#challenge-*` 元素、`cf-chl-widget-` 框架、`window._cf_chl_opt`)。内容级标记只是**兜底层**,且仅对"挑战兼容"的响应(403/429/503 或来自 Cloudflare 边缘——`server: cloudflare` / `cf-ray`)运行——因为中间页从不会以普通 200 返回,所以正文里引用了挑战文案的普通文章绝不可能被误判。Cloudflare Bot Management 的被动 JavaScript-Detections 遥测(`/cdn-cgi/challenge-platform/scripts/jsd/`,会注入受保护站点的每一个**正常**页面——如 openrouter.ai)在前缀扫描前被显式中和,因此带真实内容的 200 正常页绝不会被误判。硬封锁页("Sorry, you have been blocked")单独分类并立即失败——等待无法解除。
114
+ 1. **识别** —— 响应带 `cf-mitigated: challenge`(Cloudflare 官方文档注明所有挑战页类型都带此头),或 403/503 且 `server: cloudflare` 的 HTML 文档,或本地化的中间页本身("Just a moment…" / "请稍候…" / "Минутку…" 等 title 家族,以及结构性标记:挑战运行时的 `<script src>`、`#challenge-*` 元素、`cf-chl-widget-` 框架、`window._cf_chl_opt`)。内容级标记只是**兜底层**,且仅对"挑战兼容"的响应(403/429/503 或来自 Cloudflare 边缘——`server: cloudflare` / `cf-ray`)运行——因为中间页从不会以普通 200 返回,所以正文里引用了挑战文案的普通文章绝不可能被误判。两处收紧让这个承诺真正成立:挑战平台脚本只读真实的 `<script src>` 属性(正文、以及转义后的代码示例里引用的路径一律忽略),并且**正面**匹配运行时挂载点 `/cdn-cgi/challenge-platform/h/[b|g]/orchestrate/`。该前缀下 Cloudflare 注入的其余脚本——Bot Management 的 JavaScript Detections(`scripts/jsd/`、`h/[b|g]/scripts/jsd/<hash>/`、`h/[b|g]/jsd/r/`)与 Precursor(`scripts/precursor/`)——都是会注入受保护站点**每一个正常页面**的被动探测脚本(如 openrouter.ai、nodeseek.com),永远不算挑战信号。硬封锁页("Sorry, you have been blocked")单独分类并立即失败——等待无法解除。
115
115
  2. **同标签页、同上下文的有界等待** —— 每 500ms 轮询活 DOM,等浏览器跑完自己的验证;同时跟踪**最后一次主 frame 导航响应**,所以重载进来的真实文档的状态码和响应头才是最终上报的。SPA 式清除(无导航、纯内容替换)由同一个 DOM 探测捕获。
116
116
  3. **有界重试** —— 窗口耗尽后,同一标签页默认再导航一次(`challengeRetries`),上下文里已有的通关 cookie 继续生效。
117
117
  4. **明确失败** —— 返回独立的 `WEB_FETCH_CHALLENGE` 错误码(web seam 的 `code` 是开放字符串,允许 provider 专属码),消息中写明站点、等待预算与最后一次挑战响应的状态。
package/lib/client.js CHANGED
@@ -412,12 +412,12 @@ window.__ModuleLoader__.load({
412
412
  document.head.appendChild(tag);
413
413
  }
414
414
  var PluginCard_module_css_default = {
415
+ "footer": "UM8-Rq_footer",
415
416
  "readOnly": "UM8-Rq_readOnly",
416
417
  "failed": "UM8-Rq_failed",
417
418
  "discard": "UM8-Rq_discard",
418
- "save": "UM8-Rq_save",
419
- "footer": "UM8-Rq_footer",
420
- "form": "UM8-Rq_form"
419
+ "form": "UM8-Rq_form",
420
+ "save": "UM8-Rq_save"
421
421
  };
422
422
  //#endregion
423
423
  //#region src/client/PluginCard.tsx
@@ -477,26 +477,26 @@ window.__ModuleLoader__.load({
477
477
  document.head.appendChild(tag);
478
478
  }
479
479
  var fields_module_css_default = {
480
- "radioLabel": "CFSaVG_radioLabel",
480
+ "head": "CFSaVG_head",
481
+ "radioPick": "CFSaVG_radioPick",
482
+ "badge": "CFSaVG_badge",
481
483
  "radioHint": "CFSaVG_radioHint",
482
- "input": "CFSaVG_input",
483
- "radioText": "CFSaVG_radioText",
484
- "invalid": "CFSaVG_invalid",
485
- "radioGroup": "CFSaVG_radioGroup",
486
484
  "radioContent": "CFSaVG_radioContent",
487
- "label": "CFSaVG_label",
488
485
  "field": "CFSaVG_field",
489
- "hint": "CFSaVG_hint",
486
+ "label": "CFSaVG_label",
490
487
  "fieldEmbedded": "CFSaVG_fieldEmbedded",
491
- "badges": "CFSaVG_badges",
488
+ "checkboxRow": "CFSaVG_checkboxRow",
489
+ "hintIndented": "CFSaVG_hintIndented",
490
+ "radioGroup": "CFSaVG_radioGroup",
492
491
  "reset": "CFSaVG_reset",
493
- "head": "CFSaVG_head",
492
+ "radioText": "CFSaVG_radioText",
493
+ "radioLabel": "CFSaVG_radioLabel",
494
494
  "radioOption": "CFSaVG_radioOption",
495
- "inputInvalid": "CFSaVG_inputInvalid",
496
- "hintIndented": "CFSaVG_hintIndented",
497
- "badge": "CFSaVG_badge",
498
- "radioPick": "CFSaVG_radioPick",
499
- "checkboxRow": "CFSaVG_checkboxRow"
495
+ "hint": "CFSaVG_hint",
496
+ "badges": "CFSaVG_badges",
497
+ "invalid": "CFSaVG_invalid",
498
+ "input": "CFSaVG_input",
499
+ "inputInvalid": "CFSaVG_inputInvalid"
500
500
  };
501
501
  //#endregion
502
502
  //#region src/client/fields.tsx
package/lib/index.js CHANGED
@@ -160,10 +160,12 @@ function normalizeCdpEndpoint(input) {
160
160
  * (`/cdn-cgi/challenge-platform/` scripts, `#challenge-*` elements,
161
161
  * `cf-chl-widget-` Turnstile frames, `window._cf_chl_opt`). Markers are
162
162
  * deliberately structural: a blog post that merely MENTIONS Cloudflare
163
- * must not classify as a challenge. Bot Management's passive JSD
164
- * telemetry (`/cdn-cgi/challenge-platform/scripts/jsd/`), which
165
- * Cloudflare injects into every NORMAL page of a protected zone, is
166
- * explicitly neutralized — it is not a challenge signal.
163
+ * must not classify as a challenge, so the challenge-platform script
164
+ * check reads `<script src>` attributes only (never prose or escaped
165
+ * code samples) and identifies the runtime POSITIVELY by its
166
+ * `/orchestrate/` mount — the JSD/Precursor instrumentation Cloudflare
167
+ * injects into every NORMAL page of a protected zone, on several moving
168
+ * mounts, is not a challenge signal.
167
169
  *
168
170
  * The security stance (issue #2): this module only DETECTS and classifies.
169
171
  * Solving, spoofing, CAPTCHA answering, and cookie lifting live outside the
@@ -231,29 +233,86 @@ function escapeRegExp(text) {
231
233
  */
232
234
  const CHALLENGE_TITLE_RE = new RegExp(`^(?:${CHALLENGE_TITLE_CORES.map(escapeRegExp).join("|")})[\\s.!??…。!]*$`, "i");
233
235
  /**
234
- * Cloudflare Bot Management's passive JavaScript-Detections (JSD)
235
- * telemetry directory. Cloudflare injects this into EVERY normal page of
236
- * a protected zone — as a direct `<script src>` or as the URL strung
237
- * inside the hidden-1x1-iframe bootstrap's inline script text — with the
238
- * page itself a plain 200 of real content. The path therefore proves
239
- * nothing about a challenge and must be neutralized before the
240
- * challenge-platform prefix check: real interstitials load orchestrate
241
- * scripts (`/cdn-cgi/challenge-platform/h/[b|g]/orchestrate/...`), which
242
- * never live under `scripts/jsd/`.
236
+ * Cloudflare's challenge-platform namespace holds two very different things,
237
+ * and telling them apart is the whole game:
238
+ *
239
+ * - the challenge RUNTIME — `/cdn-cgi/challenge-platform/h/[b|g]/orchestrate/…`
240
+ * — which real interstitials load to run the verification;
241
+ * - PASSIVE INSTRUMENTATION — everything else Cloudflare injects into NORMAL
242
+ * pages of a protected zone, as a direct `<script src>` or as the URL
243
+ * strung inside the hidden-1x1-iframe bootstrap's inline script text. The
244
+ * page itself is a plain 200 of real content.
245
+ *
246
+ * Observed instrumentation layouts (a thread that lists all of them:
247
+ * `community.cloudflare.com/t/3-javascripts-are-added-unable-to-disable`):
248
+ *
249
+ * - `/cdn-cgi/challenge-platform/scripts/jsd/…` — JavaScript Detections;
250
+ * - `/cdn-cgi/challenge-platform/scripts/precursor/…` — Precursor, the
251
+ * continuous behavioral-validation engine;
252
+ * - `/cdn-cgi/challenge-platform/h/[b|g]/scripts/jsd/<hash>/…` and
253
+ * `/cdn-cgi/challenge-platform/h/[b|g]/jsd/r/…` — the same detections
254
+ * served from a hash-scoped runtime mount, with no `scripts/` segment.
255
+ *
256
+ * Enumerating instrumentation names (`jsd`, `precursor`) or carving out a
257
+ * `scripts/` namespace is a standing false-positive subscription: Cloudflare
258
+ * keeps ADDING packages and keeps moving them onto new mounts (openrouter.ai
259
+ * broke on JSD, nodeseek.com broke on Precursor, and the hash-scoped variant
260
+ * defeats a `scripts/`-only carve-out). So the runtime is identified
261
+ * POSITIVELY instead: `orchestrate` is the challenge runtime's own mount and
262
+ * never appears in instrumentation. Everything else under the prefix is
263
+ * treated as passive — the directive that a protected page must stay
264
+ * readable.
265
+ *
266
+ * Losing an unknown future runtime mount costs little here: the response
267
+ * header, the localized title family, and the challenge DOM elements remain
268
+ * independent signals, and the content-level script check is only their
269
+ * fallback.
270
+ */
271
+ const CHALLENGE_PLATFORM_PREFIX = "/cdn-cgi/challenge-platform/";
272
+ /** The path segment that identifies the challenge runtime (not telemetry). */
273
+ const CHALLENGE_RUNTIME_SEGMENT = "/orchestrate/";
274
+ /**
275
+ * A `<script src>` value in document HTML: double-quoted, single-quoted, or
276
+ * bare. Only real `src` attributes are read, so prose and escaped code
277
+ * samples that merely quote a path can never match (badcase:
278
+ * `community.cloudflare.com/t/3-javascripts-are-added-unable-to-disable`
279
+ * — a thread ABOUT these injected scripts, which lists
280
+ * `/cdn-cgi/challenge-platform/scripts/jsd/main.js` in its body text and was
281
+ * misclassified as a challenge by the old whole-document substring scan).
282
+ */
283
+ const SCRIPT_SRC_RE = /<script\b[^>]*?\bsrc\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s>]+))/gi;
284
+ /**
285
+ * Extract every `<script src>` value from a document's HTML, lowercased.
286
+ *
287
+ * @param html - the document HTML as `page.content()` returns it.
288
+ * @returns the script source URLs, in document order.
289
+ */
290
+ function scriptSrcsOf(html) {
291
+ const srcs = [];
292
+ SCRIPT_SRC_RE.lastIndex = 0;
293
+ for (let match = SCRIPT_SRC_RE.exec(html); match !== null; match = SCRIPT_SRC_RE.exec(html)) srcs.push((match[1] ?? match[2] ?? match[3] ?? "").toLowerCase());
294
+ return srcs;
295
+ }
296
+ /**
297
+ * Whether a script src belongs to the challenge RUNTIME rather than to the
298
+ * passive instrumentation Cloudflare injects into normal pages.
299
+ *
300
+ * @param src - a lowercased script source URL.
301
+ * @returns true when the script is the challenge runtime.
243
302
  */
244
- const TELEMETRY_JSD_DIR = "/cdn-cgi/challenge-platform/scripts/jsd/";
303
+ function isChallengeRuntimeScript(src) {
304
+ return src.includes(CHALLENGE_PLATFORM_PREFIX) && src.includes(CHALLENGE_RUNTIME_SEGMENT);
305
+ }
245
306
  /**
246
307
  * Structural markers a challenge document carries in its HTML. Kept in
247
308
  * attribute/assignment form (`id="…"`, `window._cf_chl_opt`) rather than bare
248
309
  * substrings so an article that merely QUOTES these strings in prose or code
249
310
  * samples does not classify — the marker must look like the real element.
250
- * The bare `/cdn-cgi/challenge-platform/` prefix is checked only after
251
- * {@link TELEMETRY_JSD_DIR} occurrences are stripped (see
252
- * {@link classifyChallengeHtml}); `scripts/jsd/main.js` in any spelling is
253
- * passive telemetry and deliberately absent from this list.
311
+ * The challenge-platform prefix is deliberately absent: script evidence goes
312
+ * through {@link scriptSrcsOf} + {@link isChallengeRuntimeScript}, which
313
+ * distinguish a real `src` from prose and runtime from instrumentation.
254
314
  */
255
315
  const CHALLENGE_HTML_MARKERS = [
256
- "/cdn-cgi/challenge-platform/",
257
316
  "id=\"cf-chl-widget-",
258
317
  "window._cf_chl_opt",
259
318
  "id=\"challenge-form\"",
@@ -334,8 +393,8 @@ function classifyChallengeHtml(html) {
334
393
  const lowered = html.toLowerCase();
335
394
  if (lowered.includes(BLOCKED_BODY_PHRASE) && BLOCKED_BODY_MARKERS.some((marker) => lowered.includes(marker))) return "blocked";
336
395
  if (CHALLENGE_TITLE_RE.test(title)) return "challenge";
337
- const withoutTelemetry = lowered.split(TELEMETRY_JSD_DIR).join("");
338
- if (CHALLENGE_HTML_MARKERS.some((marker) => withoutTelemetry.includes(marker))) return "challenge";
396
+ if (scriptSrcsOf(html).some(isChallengeRuntimeScript)) return "challenge";
397
+ if (CHALLENGE_HTML_MARKERS.some((marker) => lowered.includes(marker))) return "challenge";
339
398
  if (lowered.includes("class=\"ray-id\"") && lowered.includes("class=\"footer-inner\"")) return "challenge";
340
399
  return "none";
341
400
  }
@@ -345,19 +404,21 @@ function classifyChallengeHtml(html) {
345
404
  * what makes SPA-style clears visible — the document swaps its content
346
405
  * without any navigation, so only a DOM read can see the change. The title
347
406
  * regex is baked in at build time from the same single source the Node-side
348
- * check uses, so the two can never drift apart. Script srcs under
349
- * `/cdn-cgi/challenge-platform/scripts/jsd/` are Bot Management's passive
350
- * JSD telemetry — present on every normal page of a protected zone — and are
351
- * excluded just like in {@link classifyChallengeHtml}.
407
+ * check uses, so the two can never drift apart. Script evidence follows the
408
+ * same rule as {@link classifyChallengeHtml}: only the challenge runtime
409
+ * counts, and the instrumentation Cloudflare injects into normal pages of a
410
+ * protected zone is ignored.
352
411
  */
353
412
  const CHALLENGE_DOM_PROBE = `(() => {
354
413
  const re = new RegExp(${JSON.stringify(CHALLENGE_TITLE_RE.source)}, ${JSON.stringify(CHALLENGE_TITLE_RE.flags)})
414
+ const platform = ${JSON.stringify(CHALLENGE_PLATFORM_PREFIX)}
415
+ const runtime = ${JSON.stringify(CHALLENGE_RUNTIME_SEGMENT)}
355
416
  if (re.test(document.title.trim())) return true
356
417
  if (document.querySelector('#challenge-form, #challenge-running, #challenge-stage, #challenge-error-text')) return true
357
418
  if (document.querySelector('[id^="cf-chl-widget-"]')) return true
358
419
  for (const script of Array.from(document.scripts)) {
359
- const src = script.getAttribute('src') || ''
360
- if (src.includes('/cdn-cgi/challenge-platform/') && !src.includes('/cdn-cgi/challenge-platform/scripts/jsd/')) return true
420
+ const src = (script.getAttribute('src') || '').toLowerCase()
421
+ if (src.includes(platform) && src.includes(runtime)) return true
361
422
  }
362
423
  if (document.querySelector('.footer .footer-inner .ray-id')) return true
363
424
  return false
@@ -16,10 +16,12 @@
16
16
  * (`/cdn-cgi/challenge-platform/` scripts, `#challenge-*` elements,
17
17
  * `cf-chl-widget-` Turnstile frames, `window._cf_chl_opt`). Markers are
18
18
  * deliberately structural: a blog post that merely MENTIONS Cloudflare
19
- * must not classify as a challenge. Bot Management's passive JSD
20
- * telemetry (`/cdn-cgi/challenge-platform/scripts/jsd/`), which
21
- * Cloudflare injects into every NORMAL page of a protected zone, is
22
- * explicitly neutralized — it is not a challenge signal.
19
+ * must not classify as a challenge, so the challenge-platform script
20
+ * check reads `<script src>` attributes only (never prose or escaped
21
+ * code samples) and identifies the runtime POSITIVELY by its
22
+ * `/orchestrate/` mount — the JSD/Precursor instrumentation Cloudflare
23
+ * injects into every NORMAL page of a protected zone, on several moving
24
+ * mounts, is not a challenge signal.
23
25
  *
24
26
  * The security stance (issue #2): this module only DETECTS and classifies.
25
27
  * Solving, spoofing, CAPTCHA answering, and cookie lifting live outside the
@@ -81,9 +83,9 @@ export declare function classifyChallengeHtml(html: string): ChallengeVerdict;
81
83
  * what makes SPA-style clears visible — the document swaps its content
82
84
  * without any navigation, so only a DOM read can see the change. The title
83
85
  * regex is baked in at build time from the same single source the Node-side
84
- * check uses, so the two can never drift apart. Script srcs under
85
- * `/cdn-cgi/challenge-platform/scripts/jsd/` are Bot Management's passive
86
- * JSD telemetry — present on every normal page of a protected zone — and are
87
- * excluded just like in {@link classifyChallengeHtml}.
86
+ * check uses, so the two can never drift apart. Script evidence follows the
87
+ * same rule as {@link classifyChallengeHtml}: only the challenge runtime
88
+ * counts, and the instrumentation Cloudflare injects into normal pages of a
89
+ * protected zone is ignored.
88
90
  */
89
91
  export declare const CHALLENGE_DOM_PROBE: string;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dsh-web-fetch-playwright",
3
- "version": "0.2.11",
3
+ "version": "0.2.12",
4
4
  "description": "Playwright/CDP web-fetch provider for DeepSeek Harness: renders pages in a real browser, denoises them (Readability + DOMPurify), and returns markdown.",
5
5
  "type": "module",
6
6
  "license": "MIT",
package/src/challenge.ts CHANGED
@@ -16,10 +16,12 @@
16
16
  * (`/cdn-cgi/challenge-platform/` scripts, `#challenge-*` elements,
17
17
  * `cf-chl-widget-` Turnstile frames, `window._cf_chl_opt`). Markers are
18
18
  * deliberately structural: a blog post that merely MENTIONS Cloudflare
19
- * must not classify as a challenge. Bot Management's passive JSD
20
- * telemetry (`/cdn-cgi/challenge-platform/scripts/jsd/`), which
21
- * Cloudflare injects into every NORMAL page of a protected zone, is
22
- * explicitly neutralized — it is not a challenge signal.
19
+ * must not classify as a challenge, so the challenge-platform script
20
+ * check reads `<script src>` attributes only (never prose or escaped
21
+ * code samples) and identifies the runtime POSITIVELY by its
22
+ * `/orchestrate/` mount — the JSD/Precursor instrumentation Cloudflare
23
+ * injects into every NORMAL page of a protected zone, on several moving
24
+ * mounts, is not a challenge signal.
23
25
  *
24
26
  * The security stance (issue #2): this module only DETECTS and classifies.
25
27
  * Solving, spoofing, CAPTCHA answering, and cookie lifting live outside the
@@ -89,30 +91,93 @@ export const CHALLENGE_TITLE_RE = new RegExp(
89
91
  )
90
92
 
91
93
  /**
92
- * Cloudflare Bot Management's passive JavaScript-Detections (JSD)
93
- * telemetry directory. Cloudflare injects this into EVERY normal page of
94
- * a protected zone — as a direct `<script src>` or as the URL strung
95
- * inside the hidden-1x1-iframe bootstrap's inline script text — with the
96
- * page itself a plain 200 of real content. The path therefore proves
97
- * nothing about a challenge and must be neutralized before the
98
- * challenge-platform prefix check: real interstitials load orchestrate
99
- * scripts (`/cdn-cgi/challenge-platform/h/[b|g]/orchestrate/...`), which
100
- * never live under `scripts/jsd/`.
94
+ * Cloudflare's challenge-platform namespace holds two very different things,
95
+ * and telling them apart is the whole game:
96
+ *
97
+ * - the challenge RUNTIME — `/cdn-cgi/challenge-platform/h/[b|g]/orchestrate/…`
98
+ * — which real interstitials load to run the verification;
99
+ * - PASSIVE INSTRUMENTATION — everything else Cloudflare injects into NORMAL
100
+ * pages of a protected zone, as a direct `<script src>` or as the URL
101
+ * strung inside the hidden-1x1-iframe bootstrap's inline script text. The
102
+ * page itself is a plain 200 of real content.
103
+ *
104
+ * Observed instrumentation layouts (a thread that lists all of them:
105
+ * `community.cloudflare.com/t/3-javascripts-are-added-unable-to-disable`):
106
+ *
107
+ * - `/cdn-cgi/challenge-platform/scripts/jsd/…` — JavaScript Detections;
108
+ * - `/cdn-cgi/challenge-platform/scripts/precursor/…` — Precursor, the
109
+ * continuous behavioral-validation engine;
110
+ * - `/cdn-cgi/challenge-platform/h/[b|g]/scripts/jsd/<hash>/…` and
111
+ * `/cdn-cgi/challenge-platform/h/[b|g]/jsd/r/…` — the same detections
112
+ * served from a hash-scoped runtime mount, with no `scripts/` segment.
113
+ *
114
+ * Enumerating instrumentation names (`jsd`, `precursor`) or carving out a
115
+ * `scripts/` namespace is a standing false-positive subscription: Cloudflare
116
+ * keeps ADDING packages and keeps moving them onto new mounts (openrouter.ai
117
+ * broke on JSD, nodeseek.com broke on Precursor, and the hash-scoped variant
118
+ * defeats a `scripts/`-only carve-out). So the runtime is identified
119
+ * POSITIVELY instead: `orchestrate` is the challenge runtime's own mount and
120
+ * never appears in instrumentation. Everything else under the prefix is
121
+ * treated as passive — the directive that a protected page must stay
122
+ * readable.
123
+ *
124
+ * Losing an unknown future runtime mount costs little here: the response
125
+ * header, the localized title family, and the challenge DOM elements remain
126
+ * independent signals, and the content-level script check is only their
127
+ * fallback.
101
128
  */
102
- const TELEMETRY_JSD_DIR = '/cdn-cgi/challenge-platform/scripts/jsd/'
129
+ const CHALLENGE_PLATFORM_PREFIX = '/cdn-cgi/challenge-platform/'
130
+
131
+ /** The path segment that identifies the challenge runtime (not telemetry). */
132
+ const CHALLENGE_RUNTIME_SEGMENT = '/orchestrate/'
133
+
134
+ /**
135
+ * A `<script src>` value in document HTML: double-quoted, single-quoted, or
136
+ * bare. Only real `src` attributes are read, so prose and escaped code
137
+ * samples that merely quote a path can never match (badcase:
138
+ * `community.cloudflare.com/t/3-javascripts-are-added-unable-to-disable`
139
+ * — a thread ABOUT these injected scripts, which lists
140
+ * `/cdn-cgi/challenge-platform/scripts/jsd/main.js` in its body text and was
141
+ * misclassified as a challenge by the old whole-document substring scan).
142
+ */
143
+ const SCRIPT_SRC_RE = /<script\b[^>]*?\bsrc\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s>]+))/gi
144
+
145
+ /**
146
+ * Extract every `<script src>` value from a document's HTML, lowercased.
147
+ *
148
+ * @param html - the document HTML as `page.content()` returns it.
149
+ * @returns the script source URLs, in document order.
150
+ */
151
+ function scriptSrcsOf(html: string): string[] {
152
+ const srcs: string[] = []
153
+ SCRIPT_SRC_RE.lastIndex = 0
154
+ for (let match = SCRIPT_SRC_RE.exec(html); match !== null; match = SCRIPT_SRC_RE.exec(html)) {
155
+ srcs.push((match[1] ?? match[2] ?? match[3] ?? '').toLowerCase())
156
+ }
157
+ return srcs
158
+ }
159
+
160
+ /**
161
+ * Whether a script src belongs to the challenge RUNTIME rather than to the
162
+ * passive instrumentation Cloudflare injects into normal pages.
163
+ *
164
+ * @param src - a lowercased script source URL.
165
+ * @returns true when the script is the challenge runtime.
166
+ */
167
+ function isChallengeRuntimeScript(src: string): boolean {
168
+ return src.includes(CHALLENGE_PLATFORM_PREFIX) && src.includes(CHALLENGE_RUNTIME_SEGMENT)
169
+ }
103
170
 
104
171
  /**
105
172
  * Structural markers a challenge document carries in its HTML. Kept in
106
173
  * attribute/assignment form (`id="…"`, `window._cf_chl_opt`) rather than bare
107
174
  * substrings so an article that merely QUOTES these strings in prose or code
108
175
  * samples does not classify — the marker must look like the real element.
109
- * The bare `/cdn-cgi/challenge-platform/` prefix is checked only after
110
- * {@link TELEMETRY_JSD_DIR} occurrences are stripped (see
111
- * {@link classifyChallengeHtml}); `scripts/jsd/main.js` in any spelling is
112
- * passive telemetry and deliberately absent from this list.
176
+ * The challenge-platform prefix is deliberately absent: script evidence goes
177
+ * through {@link scriptSrcsOf} + {@link isChallengeRuntimeScript}, which
178
+ * distinguish a real `src` from prose and runtime from instrumentation.
113
179
  */
114
180
  const CHALLENGE_HTML_MARKERS = [
115
- '/cdn-cgi/challenge-platform/',
116
181
  'id="cf-chl-widget-',
117
182
  'window._cf_chl_opt',
118
183
  'id="challenge-form"',
@@ -206,14 +271,15 @@ export function classifyChallengeHtml(html: string): ChallengeVerdict {
206
271
  const lowered = html.toLowerCase()
207
272
  if (lowered.includes(BLOCKED_BODY_PHRASE) && BLOCKED_BODY_MARKERS.some(marker => lowered.includes(marker))) return 'blocked'
208
273
  if (CHALLENGE_TITLE_RE.test(title)) return 'challenge'
209
- // Strip the passive JSD telemetry directory before the marker scan: its
210
- // URL embeds the `/cdn-cgi/challenge-platform/` prefix, so an unstripped
211
- // scan misreads every Bot-Management-protected normal page (a plain 200
212
- // with real content — e.g. openrouter.ai) as an interstitial, the wait
213
- // can then never confirm a clear, and the fetch dies with
214
- // WEB_FETCH_CHALLENGE despite the page having loaded fine.
215
- const withoutTelemetry = lowered.split(TELEMETRY_JSD_DIR).join('')
216
- if (CHALLENGE_HTML_MARKERS.some(marker => withoutTelemetry.includes(marker))) return 'challenge'
274
+ // Script evidence BEFORE the text markers: only a real `<script src>`
275
+ // counts, and only when it is the challenge runtime rather than passive
276
+ // instrumentation. Both halves matter — scanning prose false-positived on
277
+ // an article quoting these paths, and reading the prefix as a challenge
278
+ // false-positived on every protected normal page (openrouter.ai's JSD,
279
+ // nodeseek.com's Precursor). Either way the wait can never confirm a
280
+ // "clear", and a page that loaded fine dies with WEB_FETCH_CHALLENGE.
281
+ if (scriptSrcsOf(html).some(isChallengeRuntimeScript)) return 'challenge'
282
+ if (CHALLENGE_HTML_MARKERS.some(marker => lowered.includes(marker))) return 'challenge'
217
283
  // Crawlee's footer marker, as a two-class combo so prose quoting one class
218
284
  // alone stays clean.
219
285
  if (lowered.includes('class="ray-id"') && lowered.includes('class="footer-inner"')) return 'challenge'
@@ -226,19 +292,21 @@ export function classifyChallengeHtml(html: string): ChallengeVerdict {
226
292
  * what makes SPA-style clears visible — the document swaps its content
227
293
  * without any navigation, so only a DOM read can see the change. The title
228
294
  * regex is baked in at build time from the same single source the Node-side
229
- * check uses, so the two can never drift apart. Script srcs under
230
- * `/cdn-cgi/challenge-platform/scripts/jsd/` are Bot Management's passive
231
- * JSD telemetry — present on every normal page of a protected zone — and are
232
- * excluded just like in {@link classifyChallengeHtml}.
295
+ * check uses, so the two can never drift apart. Script evidence follows the
296
+ * same rule as {@link classifyChallengeHtml}: only the challenge runtime
297
+ * counts, and the instrumentation Cloudflare injects into normal pages of a
298
+ * protected zone is ignored.
233
299
  */
234
300
  export const CHALLENGE_DOM_PROBE = `(() => {
235
301
  const re = new RegExp(${JSON.stringify(CHALLENGE_TITLE_RE.source)}, ${JSON.stringify(CHALLENGE_TITLE_RE.flags)})
302
+ const platform = ${JSON.stringify(CHALLENGE_PLATFORM_PREFIX)}
303
+ const runtime = ${JSON.stringify(CHALLENGE_RUNTIME_SEGMENT)}
236
304
  if (re.test(document.title.trim())) return true
237
305
  if (document.querySelector('#challenge-form, #challenge-running, #challenge-stage, #challenge-error-text')) return true
238
306
  if (document.querySelector('[id^="cf-chl-widget-"]')) return true
239
307
  for (const script of Array.from(document.scripts)) {
240
- const src = script.getAttribute('src') || ''
241
- if (src.includes('/cdn-cgi/challenge-platform/') && !src.includes('/cdn-cgi/challenge-platform/scripts/jsd/')) return true
308
+ const src = (script.getAttribute('src') || '').toLowerCase()
309
+ if (src.includes(platform) && src.includes(runtime)) return true
242
310
  }
243
311
  if (document.querySelector('.footer .footer-inner .ray-id')) return true
244
312
  return false