pi-smart-fetch 0.1.8 → 0.1.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +85 -7
- package/dist/index.js +386 -9
- package/dist/index.js.map +1 -1
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -1,6 +1,10 @@
|
|
|
1
1
|
# pi-smart-fetch
|
|
2
2
|
|
|
3
|
-
`pi-smart-fetch` adds
|
|
3
|
+
`pi-smart-fetch` adds smarter web fetching tools to pi.dev.
|
|
4
|
+
|
|
5
|
+
It registers:
|
|
6
|
+
- `web_fetch`
|
|
7
|
+
- `batch_web_fetch`
|
|
4
8
|
|
|
5
9
|
## Features
|
|
6
10
|
|
|
@@ -10,8 +14,10 @@ Compared with naive Node.js `fetch()`, this package gives you:
|
|
|
10
14
|
- **better success on bot-defended pages** where plain server-side requests are blocked, challenged, or degraded
|
|
11
15
|
- **useful metadata** like title, author, published date, site, and language when available
|
|
12
16
|
- **multiple output formats**: `markdown`, `html`, `text`, or `json`
|
|
13
|
-
- **
|
|
17
|
+
- **single and batch tools**: `web_fetch` for one URL, `batch_web_fetch` for many
|
|
14
18
|
- **pi-specific behavior** including an optional `verbose` flag and defaults from pi settings
|
|
19
|
+
- **bounded batch fan-out** with a configurable default concurrency of `8`
|
|
20
|
+
- **a richer pi TUI for batch mode** with per-item rows, truncated URLs, statuses, and small progress bars
|
|
15
21
|
- **lower overhead than browser automation** when you do not need JS execution, login, scrolling, or clicks
|
|
16
22
|
- **clear limits**: it does not execute JavaScript or solve interactive anti-bot flows
|
|
17
23
|
|
|
@@ -33,14 +39,31 @@ pi install agent-smart-fetch/packages/pi-smart-fetch
|
|
|
33
39
|
## Use cases
|
|
34
40
|
|
|
35
41
|
Use `web_fetch` when you want to:
|
|
36
|
-
- fetch
|
|
42
|
+
- fetch one article, doc page, or blog post with a browser-like network fingerprint
|
|
37
43
|
- analyze readable content instead of raw HTML
|
|
38
44
|
- reduce agent token waste on noisy page chrome
|
|
39
45
|
- get author/title/published metadata when available
|
|
40
46
|
- work around pages that reject ordinary server-side fetches
|
|
41
47
|
|
|
48
|
+
Use `batch_web_fetch` when you want to:
|
|
49
|
+
- fetch multiple URLs in one tool call
|
|
50
|
+
- preserve a clear mapping between each input URL and its result
|
|
51
|
+
- let pi show per-item progress while the batch runs
|
|
52
|
+
- collect a mix of successes and failures without losing per-item errors
|
|
53
|
+
|
|
54
|
+
## Tool synopsis
|
|
55
|
+
|
|
56
|
+
```text
|
|
57
|
+
web_fetch(url, browser?, os?, headers?, maxChars?, format?, removeImages?, includeReplies?, proxy?, verbose?)
|
|
58
|
+
batch_web_fetch(requests, verbose?)
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
For `batch_web_fetch`, `requests` is an array of objects, and **each item accepts the same parameters as `web_fetch` except `verbose`**.
|
|
62
|
+
|
|
42
63
|
## Output behavior
|
|
43
64
|
|
|
65
|
+
### `web_fetch`
|
|
66
|
+
|
|
44
67
|
By default, the tool returns a compact response containing non-empty:
|
|
45
68
|
- URL
|
|
46
69
|
- Title
|
|
@@ -54,9 +77,24 @@ Set `verbose: true` to include fuller metadata such as:
|
|
|
54
77
|
- word count
|
|
55
78
|
- browser profile info
|
|
56
79
|
|
|
80
|
+
### `batch_web_fetch`
|
|
81
|
+
|
|
82
|
+
Batch output:
|
|
83
|
+
- starts with a batch summary (`Requests`, `Succeeded`, `Failed`, `Concurrency`)
|
|
84
|
+
- keeps results in input order
|
|
85
|
+
- labels each item with its ordinal and URL
|
|
86
|
+
- includes full content for successful items
|
|
87
|
+
- includes a bot-friendly `Error:` line for failed items
|
|
88
|
+
|
|
89
|
+
In the pi TUI, batch mode also streams per-item progress rows showing:
|
|
90
|
+
- a small spinner/check/error glyph
|
|
91
|
+
- a truncated URL
|
|
92
|
+
- a one-word status (`queued`, `fetching`, `extracting`, `done`, `error`)
|
|
93
|
+
- a small progress bar
|
|
94
|
+
|
|
57
95
|
## Example tool outputs
|
|
58
96
|
|
|
59
|
-
### Compact output (default)
|
|
97
|
+
### Compact `web_fetch` output (default)
|
|
60
98
|
|
|
61
99
|
```text
|
|
62
100
|
> URL: https://example.com/blog/some-article
|
|
@@ -70,7 +108,7 @@ This is the cleaned readable content extracted from the page.
|
|
|
70
108
|
It omits most navigation, footer, and unrelated chrome.
|
|
71
109
|
```
|
|
72
110
|
|
|
73
|
-
### Verbose output (`verbose: true`)
|
|
111
|
+
### Verbose `web_fetch` output (`verbose: true`)
|
|
74
112
|
|
|
75
113
|
```text
|
|
76
114
|
> URL: https://example.com/blog/some-article
|
|
@@ -88,6 +126,30 @@ This is the cleaned readable content extracted from the page.
|
|
|
88
126
|
It includes the same body content, but with a richer metadata header.
|
|
89
127
|
```
|
|
90
128
|
|
|
129
|
+
### `batch_web_fetch` output
|
|
130
|
+
|
|
131
|
+
```text
|
|
132
|
+
> Requests: 2
|
|
133
|
+
> Succeeded: 1
|
|
134
|
+
> Failed: 1
|
|
135
|
+
> Concurrency: 8
|
|
136
|
+
|
|
137
|
+
## [1/2] https://example.com/blog/some-article
|
|
138
|
+
> URL: https://example.com/blog/some-article
|
|
139
|
+
> Title: Some Article
|
|
140
|
+
> Author: Jane Doe
|
|
141
|
+
> Published: 2026-03-12
|
|
142
|
+
|
|
143
|
+
# Some Article
|
|
144
|
+
|
|
145
|
+
This is the cleaned readable content extracted from the page.
|
|
146
|
+
|
|
147
|
+
## [2/2] https://blocked.example/post
|
|
148
|
+
> URL: https://blocked.example/post
|
|
149
|
+
> Status: error
|
|
150
|
+
> Error: HTTP 403 Forbidden for https://blocked.example/post
|
|
151
|
+
```
|
|
152
|
+
|
|
91
153
|
### Error output
|
|
92
154
|
|
|
93
155
|
```text
|
|
@@ -96,6 +158,8 @@ Error: Invalid URL: not-a-url
|
|
|
96
158
|
|
|
97
159
|
## Parameters
|
|
98
160
|
|
|
161
|
+
### `web_fetch`
|
|
162
|
+
|
|
99
163
|
| Parameter | Type | Default | Description |
|
|
100
164
|
|-------------------|-------------------------------|-----------------|------------------------------------------------------------------------------|
|
|
101
165
|
| `url` | string | required | URL to fetch |
|
|
@@ -109,6 +173,13 @@ Error: Invalid URL: not-a-url
|
|
|
109
173
|
| `proxy` | string | none | Proxy URL |
|
|
110
174
|
| `verbose` | boolean | `false` | Include the full metadata header. Can default from `smartFetchVerboseByDefault` |
|
|
111
175
|
|
|
176
|
+
### `batch_web_fetch`
|
|
177
|
+
|
|
178
|
+
| Parameter | Type | Default | Description |
|
|
179
|
+
|-------------|---------------------|-----------|-------------|
|
|
180
|
+
| `requests` | array of objects | required | Array of fetch requests. Each item accepts the same parameters as `web_fetch` except `verbose` |
|
|
181
|
+
| `verbose` | boolean | `false` | Include the full metadata header for each successful result |
|
|
182
|
+
|
|
112
183
|
## pi settings
|
|
113
184
|
|
|
114
185
|
Optional custom settings in `~/.pi/agent/settings.json` or `.pi/settings.json`:
|
|
@@ -121,7 +192,8 @@ Optional custom settings in `~/.pi/agent/settings.json` or `.pi/settings.json`:
|
|
|
121
192
|
"smartFetchDefaultBrowser": "chrome_145",
|
|
122
193
|
"smartFetchDefaultOs": "windows",
|
|
123
194
|
"smartFetchDefaultRemoveImages": false,
|
|
124
|
-
"smartFetchDefaultIncludeReplies": "extractors"
|
|
195
|
+
"smartFetchDefaultIncludeReplies": "extractors",
|
|
196
|
+
"smartFetchDefaultBatchConcurrency": 8
|
|
125
197
|
}
|
|
126
198
|
```
|
|
127
199
|
|
|
@@ -133,11 +205,17 @@ Behavior:
|
|
|
133
205
|
- `smartFetchDefaultOs` sets the default OS fingerprint profile
|
|
134
206
|
- `smartFetchDefaultRemoveImages` sets the default for image stripping
|
|
135
207
|
- `smartFetchDefaultIncludeReplies` sets the default replies/comments behavior
|
|
208
|
+
- `smartFetchDefaultBatchConcurrency` sets the default bounded concurrency for `batch_web_fetch`
|
|
136
209
|
- project `.pi/settings.json` overrides global `~/.pi/agent/settings.json`
|
|
137
210
|
|
|
211
|
+
Legacy aliases still supported:
|
|
212
|
+
- `webFetchVerboseByDefault`
|
|
213
|
+
- `webFetchDefaultMaxChars`
|
|
214
|
+
- `webFetchDefaultBatchConcurrency`
|
|
215
|
+
|
|
138
216
|
## When not to use it
|
|
139
217
|
|
|
140
|
-
Do not use
|
|
218
|
+
Do not use these tools when:
|
|
141
219
|
- the site requires JS rendering
|
|
142
220
|
- you need login/session flows
|
|
143
221
|
- you need to click, scroll, or submit forms
|
package/dist/index.js
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
import { getAgentDir } from '@mariozechner/pi-coding-agent';
|
|
2
|
+
import { Text } from '@mariozechner/pi-tui';
|
|
2
3
|
import { Type } from '@sinclair/typebox';
|
|
3
4
|
import { Defuddle } from 'defuddle/node';
|
|
4
5
|
import { getProfiles, fetch } from 'wreq-js';
|
|
@@ -13,6 +14,7 @@ var DEFAULT_BROWSER = "chrome_145";
|
|
|
13
14
|
var DEFAULT_OS = "windows";
|
|
14
15
|
var DEFAULT_MAX_CHARS = 5e4;
|
|
15
16
|
var DEFAULT_TIMEOUT_MS = 15e3;
|
|
17
|
+
var DEFAULT_BATCH_CONCURRENCY = 8;
|
|
16
18
|
var DEFAULT_INCLUDE_REPLIES = "extractors";
|
|
17
19
|
var DEFAULT_ACCEPT_HEADER = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
|
|
18
20
|
var DEFAULT_JSON_ACCEPT_HEADER = "application/json,text/json,application/ld+json;q=0.9,text/plain;q=0.8,*/*;q=0.7";
|
|
@@ -79,6 +81,37 @@ function buildFetchResponseText(result, options = {}) {
|
|
|
79
81
|
|
|
80
82
|
${result.content}` : result.content;
|
|
81
83
|
}
|
|
84
|
+
function buildBatchItemHeading(item, total) {
|
|
85
|
+
const ordinal = item.index + 1;
|
|
86
|
+
const url = item.result?.finalUrl ?? item.request.url;
|
|
87
|
+
return `## [${ordinal}/${total}] ${url}`;
|
|
88
|
+
}
|
|
89
|
+
function buildBatchItemText(item, total, options = {}) {
|
|
90
|
+
const heading = buildBatchItemHeading(item, total);
|
|
91
|
+
if (item.status === "error") {
|
|
92
|
+
const errorHeader = buildHeader([
|
|
93
|
+
["URL", item.request.url],
|
|
94
|
+
["Status", "error"],
|
|
95
|
+
["Error", item.error ?? "Unknown error"]
|
|
96
|
+
]);
|
|
97
|
+
return `${heading}
|
|
98
|
+
${errorHeader}`;
|
|
99
|
+
}
|
|
100
|
+
return `${heading}
|
|
101
|
+
${buildFetchResponseText(item.result, options)}`;
|
|
102
|
+
}
|
|
103
|
+
function buildBatchFetchResponseText(result, options = {}) {
|
|
104
|
+
const summary = buildHeader([
|
|
105
|
+
["Requests", result.total],
|
|
106
|
+
["Succeeded", result.succeeded],
|
|
107
|
+
["Failed", result.failed],
|
|
108
|
+
["Concurrency", result.batchConcurrency]
|
|
109
|
+
]);
|
|
110
|
+
const items = result.items.map(
|
|
111
|
+
(item) => buildBatchItemText(item, result.total, options)
|
|
112
|
+
);
|
|
113
|
+
return [summary, ...items].filter(Boolean).join("\n\n");
|
|
114
|
+
}
|
|
82
115
|
function estimateWordCount(content) {
|
|
83
116
|
const words = content.trim().match(/\S+/g);
|
|
84
117
|
return words?.length ?? 0;
|
|
@@ -163,7 +196,7 @@ function shouldStripReplies(site) {
|
|
|
163
196
|
return site === "Hacker News" || site.startsWith("r/") || site.startsWith("GitHub - ");
|
|
164
197
|
}
|
|
165
198
|
function createDefuddleFetch(dependencies = runtimeDependencies) {
|
|
166
|
-
return async function defuddleFetch2(opts) {
|
|
199
|
+
return async function defuddleFetch2(opts, hooks = {}) {
|
|
167
200
|
const browser = opts.browser ?? DEFAULT_BROWSER;
|
|
168
201
|
const os = opts.os ?? DEFAULT_OS;
|
|
169
202
|
const format = opts.format ?? "markdown";
|
|
@@ -196,6 +229,7 @@ function createDefuddleFetch(dependencies = runtimeDependencies) {
|
|
|
196
229
|
if (opts.proxy) {
|
|
197
230
|
fetchOptions.proxy = opts.proxy;
|
|
198
231
|
}
|
|
232
|
+
hooks.onStatusChange?.("fetching");
|
|
199
233
|
const response = await dependencies.fetch(opts.url, fetchOptions);
|
|
200
234
|
if (!response.ok) {
|
|
201
235
|
return {
|
|
@@ -210,7 +244,7 @@ function createDefuddleFetch(dependencies = runtimeDependencies) {
|
|
|
210
244
|
if (!jsonResponse) {
|
|
211
245
|
return { error: `Not a JSON response (content-type: ${contentType})` };
|
|
212
246
|
}
|
|
213
|
-
|
|
247
|
+
const result2 = buildJsonResult(
|
|
214
248
|
opts,
|
|
215
249
|
finalUrl,
|
|
216
250
|
rawBody,
|
|
@@ -219,9 +253,13 @@ function createDefuddleFetch(dependencies = runtimeDependencies) {
|
|
|
219
253
|
browser,
|
|
220
254
|
os
|
|
221
255
|
);
|
|
256
|
+
if (!isError(result2)) {
|
|
257
|
+
hooks.onStatusChange?.("done");
|
|
258
|
+
}
|
|
259
|
+
return result2;
|
|
222
260
|
}
|
|
223
261
|
if (jsonResponse) {
|
|
224
|
-
|
|
262
|
+
const result2 = buildJsonResult(
|
|
225
263
|
opts,
|
|
226
264
|
finalUrl,
|
|
227
265
|
rawBody,
|
|
@@ -230,10 +268,15 @@ function createDefuddleFetch(dependencies = runtimeDependencies) {
|
|
|
230
268
|
browser,
|
|
231
269
|
os
|
|
232
270
|
);
|
|
271
|
+
if (!isError(result2)) {
|
|
272
|
+
hooks.onStatusChange?.("done");
|
|
273
|
+
}
|
|
274
|
+
return result2;
|
|
233
275
|
}
|
|
234
276
|
if (!HTML_CONTENT_TYPES.some((value) => contentType.includes(value))) {
|
|
235
277
|
return { error: `Not an HTML page (content-type: ${contentType})` };
|
|
236
278
|
}
|
|
279
|
+
hooks.onStatusChange?.("extracting");
|
|
237
280
|
const document = parseLinkedomHTML(rawBody, finalUrl);
|
|
238
281
|
const extracted = await dependencies.defuddle(document, finalUrl, {
|
|
239
282
|
markdown: format !== "html",
|
|
@@ -257,7 +300,7 @@ function createDefuddleFetch(dependencies = runtimeDependencies) {
|
|
|
257
300
|
}
|
|
258
301
|
}
|
|
259
302
|
const normalizedContent = format === "text" ? markdownToText(extractedContent) : extractedContent;
|
|
260
|
-
|
|
303
|
+
const result = {
|
|
261
304
|
url: opts.url,
|
|
262
305
|
finalUrl,
|
|
263
306
|
title: extracted.title ?? "",
|
|
@@ -270,12 +313,20 @@ function createDefuddleFetch(dependencies = runtimeDependencies) {
|
|
|
270
313
|
browser,
|
|
271
314
|
os
|
|
272
315
|
};
|
|
316
|
+
hooks.onStatusChange?.("done");
|
|
317
|
+
return result;
|
|
273
318
|
};
|
|
274
319
|
}
|
|
275
320
|
var defuddleFetch = createDefuddleFetch();
|
|
276
321
|
function isError(result) {
|
|
277
322
|
return "error" in result;
|
|
278
323
|
}
|
|
324
|
+
function resolveBatchConcurrency(value) {
|
|
325
|
+
if (!value || !Number.isFinite(value)) {
|
|
326
|
+
return DEFAULT_BATCH_CONCURRENCY;
|
|
327
|
+
}
|
|
328
|
+
return Math.max(1, Math.floor(value));
|
|
329
|
+
}
|
|
279
330
|
function resolveFetchToolDefaults(config = {}) {
|
|
280
331
|
return {
|
|
281
332
|
maxChars: config.maxChars ?? DEFAULT_MAX_CHARS,
|
|
@@ -283,7 +334,8 @@ function resolveFetchToolDefaults(config = {}) {
|
|
|
283
334
|
browser: config.browser ?? DEFAULT_BROWSER,
|
|
284
335
|
os: config.os ?? DEFAULT_OS,
|
|
285
336
|
removeImages: config.removeImages ?? false,
|
|
286
|
-
includeReplies: config.includeReplies ?? DEFAULT_INCLUDE_REPLIES
|
|
337
|
+
includeReplies: config.includeReplies ?? DEFAULT_INCLUDE_REPLIES,
|
|
338
|
+
batchConcurrency: resolveBatchConcurrency(config.batchConcurrency)
|
|
287
339
|
};
|
|
288
340
|
}
|
|
289
341
|
function createBaseFetchToolParameterProperties(defaults) {
|
|
@@ -339,8 +391,21 @@ function createBaseFetchToolParameterProperties(defaults) {
|
|
|
339
391
|
)
|
|
340
392
|
};
|
|
341
393
|
}
|
|
342
|
-
|
|
343
|
-
return
|
|
394
|
+
function createBatchFetchToolParameterProperties(defaults) {
|
|
395
|
+
return {
|
|
396
|
+
requests: Type.Array(
|
|
397
|
+
Type.Object(createBaseFetchToolParameterProperties(defaults), {
|
|
398
|
+
additionalProperties: false
|
|
399
|
+
}),
|
|
400
|
+
{
|
|
401
|
+
minItems: 1,
|
|
402
|
+
description: "Array of fetch requests. Each item accepts the same parameters as the single-item fetch tool."
|
|
403
|
+
}
|
|
404
|
+
)
|
|
405
|
+
};
|
|
406
|
+
}
|
|
407
|
+
function buildFetchOptionsFromParams(params, defaults) {
|
|
408
|
+
return {
|
|
344
409
|
url: params.url,
|
|
345
410
|
browser: params.browser ?? defaults.browser,
|
|
346
411
|
os: params.os ?? defaults.os,
|
|
@@ -351,7 +416,132 @@ async function executeFetchToolCall(params, defaults) {
|
|
|
351
416
|
includeReplies: params.includeReplies ?? defaults.includeReplies,
|
|
352
417
|
proxy: params.proxy,
|
|
353
418
|
timeoutMs: defaults.timeoutMs
|
|
354
|
-
}
|
|
419
|
+
};
|
|
420
|
+
}
|
|
421
|
+
async function executeFetchToolCall(params, defaults, hooks = {}) {
|
|
422
|
+
return defuddleFetch(buildFetchOptionsFromParams(params, defaults), hooks);
|
|
423
|
+
}
|
|
424
|
+
var PROGRESS_BY_STATUS = {
|
|
425
|
+
queued: 0,
|
|
426
|
+
fetching: 0.35,
|
|
427
|
+
extracting: 0.75,
|
|
428
|
+
done: 1,
|
|
429
|
+
error: 1
|
|
430
|
+
};
|
|
431
|
+
function createInitialProgressItems(requests) {
|
|
432
|
+
return requests.map((request, index) => ({
|
|
433
|
+
index,
|
|
434
|
+
url: typeof request.url === "string" ? request.url : String(request.url ?? ""),
|
|
435
|
+
status: "queued",
|
|
436
|
+
progress: PROGRESS_BY_STATUS.queued
|
|
437
|
+
}));
|
|
438
|
+
}
|
|
439
|
+
function buildProgressSnapshot(items, batchConcurrency) {
|
|
440
|
+
let completed = 0;
|
|
441
|
+
let succeeded = 0;
|
|
442
|
+
let failed = 0;
|
|
443
|
+
for (const item of items) {
|
|
444
|
+
if (item.status === "done" || item.status === "error") {
|
|
445
|
+
completed += 1;
|
|
446
|
+
}
|
|
447
|
+
if (item.status === "done") {
|
|
448
|
+
succeeded += 1;
|
|
449
|
+
}
|
|
450
|
+
if (item.status === "error") {
|
|
451
|
+
failed += 1;
|
|
452
|
+
}
|
|
453
|
+
}
|
|
454
|
+
return {
|
|
455
|
+
items: items.map((item) => ({ ...item })),
|
|
456
|
+
total: items.length,
|
|
457
|
+
completed,
|
|
458
|
+
succeeded,
|
|
459
|
+
failed,
|
|
460
|
+
batchConcurrency
|
|
461
|
+
};
|
|
462
|
+
}
|
|
463
|
+
async function executeBatchFetchToolCall(params, defaults, options = {}) {
|
|
464
|
+
const requests = (params.requests ?? []).map((request) => request ?? {});
|
|
465
|
+
const batchConcurrency = resolveBatchConcurrency(
|
|
466
|
+
options.batchConcurrency ?? defaults.batchConcurrency
|
|
467
|
+
);
|
|
468
|
+
const progressItems = createInitialProgressItems(requests);
|
|
469
|
+
const results = new Array(requests.length);
|
|
470
|
+
const emitProgress = () => {
|
|
471
|
+
options.onProgress?.(
|
|
472
|
+
buildProgressSnapshot(progressItems, batchConcurrency)
|
|
473
|
+
);
|
|
474
|
+
};
|
|
475
|
+
const updateProgress = (index, status, error) => {
|
|
476
|
+
progressItems[index] = {
|
|
477
|
+
...progressItems[index],
|
|
478
|
+
status,
|
|
479
|
+
progress: PROGRESS_BY_STATUS[status],
|
|
480
|
+
...error ? { error } : {}
|
|
481
|
+
};
|
|
482
|
+
emitProgress();
|
|
483
|
+
};
|
|
484
|
+
emitProgress();
|
|
485
|
+
let nextIndex = 0;
|
|
486
|
+
const worker = async () => {
|
|
487
|
+
while (true) {
|
|
488
|
+
const index = nextIndex;
|
|
489
|
+
nextIndex += 1;
|
|
490
|
+
if (index >= requests.length) {
|
|
491
|
+
return;
|
|
492
|
+
}
|
|
493
|
+
const request = requests[index] ?? {};
|
|
494
|
+
const normalizedRequest = buildFetchOptionsFromParams(request, defaults);
|
|
495
|
+
try {
|
|
496
|
+
const executeItem = options.executeItem ?? executeFetchToolCall;
|
|
497
|
+
const result = await executeItem(request, defaults, {
|
|
498
|
+
onStatusChange(status) {
|
|
499
|
+
if (status === "done") return;
|
|
500
|
+
updateProgress(index, status);
|
|
501
|
+
}
|
|
502
|
+
});
|
|
503
|
+
if (isError(result)) {
|
|
504
|
+
results[index] = {
|
|
505
|
+
index,
|
|
506
|
+
request: normalizedRequest,
|
|
507
|
+
status: "error",
|
|
508
|
+
progress: PROGRESS_BY_STATUS.error,
|
|
509
|
+
error: result.error
|
|
510
|
+
};
|
|
511
|
+
updateProgress(index, "error", result.error);
|
|
512
|
+
continue;
|
|
513
|
+
}
|
|
514
|
+
results[index] = {
|
|
515
|
+
index,
|
|
516
|
+
request: normalizedRequest,
|
|
517
|
+
status: "done",
|
|
518
|
+
progress: PROGRESS_BY_STATUS.done,
|
|
519
|
+
result
|
|
520
|
+
};
|
|
521
|
+
updateProgress(index, "done");
|
|
522
|
+
} catch (error) {
|
|
523
|
+
const message = error instanceof Error ? error.message : String(error);
|
|
524
|
+
results[index] = {
|
|
525
|
+
index,
|
|
526
|
+
request: normalizedRequest,
|
|
527
|
+
status: "error",
|
|
528
|
+
progress: PROGRESS_BY_STATUS.error,
|
|
529
|
+
error: message
|
|
530
|
+
};
|
|
531
|
+
updateProgress(index, "error", message);
|
|
532
|
+
}
|
|
533
|
+
}
|
|
534
|
+
};
|
|
535
|
+
const workerCount = requests.length === 0 ? 0 : Math.min(batchConcurrency, requests.length);
|
|
536
|
+
await Promise.all(Array.from({ length: workerCount }, async () => worker()));
|
|
537
|
+
const finalSnapshot = buildProgressSnapshot(progressItems, batchConcurrency);
|
|
538
|
+
return {
|
|
539
|
+
items: results,
|
|
540
|
+
total: finalSnapshot.total,
|
|
541
|
+
succeeded: finalSnapshot.succeeded,
|
|
542
|
+
failed: finalSnapshot.failed,
|
|
543
|
+
batchConcurrency
|
|
544
|
+
};
|
|
355
545
|
}
|
|
356
546
|
var VALID_OS_VALUES = /* @__PURE__ */ new Set([
|
|
357
547
|
"windows",
|
|
@@ -428,6 +618,10 @@ function normalizePiSmartFetchSettings(input) {
|
|
|
428
618
|
]),
|
|
429
619
|
smartFetchDefaultIncludeReplies: readIncludeReplies(source, [
|
|
430
620
|
"smartFetchDefaultIncludeReplies"
|
|
621
|
+
]),
|
|
622
|
+
smartFetchDefaultBatchConcurrency: readPositiveNumber(source, [
|
|
623
|
+
"smartFetchDefaultBatchConcurrency",
|
|
624
|
+
"webFetchDefaultBatchConcurrency"
|
|
431
625
|
])
|
|
432
626
|
};
|
|
433
627
|
}
|
|
@@ -441,7 +635,8 @@ function resolvePiSmartFetchSettings(globalSettings, projectSettings) {
|
|
|
441
635
|
browser: project.smartFetchDefaultBrowser ?? global.smartFetchDefaultBrowser,
|
|
442
636
|
os: project.smartFetchDefaultOs ?? global.smartFetchDefaultOs,
|
|
443
637
|
removeImages: project.smartFetchDefaultRemoveImages ?? global.smartFetchDefaultRemoveImages,
|
|
444
|
-
includeReplies: project.smartFetchDefaultIncludeReplies ?? global.smartFetchDefaultIncludeReplies
|
|
638
|
+
includeReplies: project.smartFetchDefaultIncludeReplies ?? global.smartFetchDefaultIncludeReplies,
|
|
639
|
+
batchConcurrency: project.smartFetchDefaultBatchConcurrency ?? global.smartFetchDefaultBatchConcurrency
|
|
445
640
|
};
|
|
446
641
|
}
|
|
447
642
|
async function readSettingsFile(path) {
|
|
@@ -468,6 +663,105 @@ var toolDescription = [
|
|
|
468
663
|
"Supports the same fetch parameters as the OpenClaw tool, plus an optional verbose flag.",
|
|
469
664
|
"Does NOT execute JavaScript \u2014 use a browser automation tool for JS-heavy pages."
|
|
470
665
|
].join(" ");
|
|
666
|
+
var batchToolDescription = [
|
|
667
|
+
"Fetch multiple URLs with browser-grade TLS fingerprinting and readable extraction.",
|
|
668
|
+
"Each request accepts the same parameters as web_fetch and fans out with bounded concurrency.",
|
|
669
|
+
"Streams per-item progress in the pi TUI with truncated URLs, statuses, and small progress bars.",
|
|
670
|
+
"Does NOT execute JavaScript \u2014 use a browser automation tool for JS-heavy pages."
|
|
671
|
+
].join(" ");
|
|
672
|
+
var SPINNER_FRAMES = ["\u280B", "\u2819", "\u2839", "\u2838", "\u283C", "\u2834", "\u2826", "\u2827", "\u2807", "\u280F"];
|
|
673
|
+
function truncateMiddle(value, width) {
|
|
674
|
+
if (width <= 0) return "";
|
|
675
|
+
if (value.length <= width) return value.padEnd(width, " ");
|
|
676
|
+
if (width === 1) return "\u2026";
|
|
677
|
+
const left = Math.ceil((width - 1) / 2);
|
|
678
|
+
const right = Math.floor((width - 1) / 2);
|
|
679
|
+
return `${value.slice(0, left)}\u2026${value.slice(value.length - right)}`;
|
|
680
|
+
}
|
|
681
|
+
function pad(value, width) {
|
|
682
|
+
return value.length >= width ? value.slice(0, width) : value.padEnd(width, " ");
|
|
683
|
+
}
|
|
684
|
+
function renderProgressBar(item, width, theme) {
|
|
685
|
+
const innerWidth = Math.max(4, width - 2);
|
|
686
|
+
const filled = Math.max(
|
|
687
|
+
0,
|
|
688
|
+
Math.min(innerWidth, Math.round(item.progress * innerWidth))
|
|
689
|
+
);
|
|
690
|
+
const empty = Math.max(0, innerWidth - filled);
|
|
691
|
+
const barColor = item.status === "error" ? "error" : item.status === "done" ? "success" : "accent";
|
|
692
|
+
return [
|
|
693
|
+
theme.fg("muted", "["),
|
|
694
|
+
theme.fg(barColor, "\u2588".repeat(filled)),
|
|
695
|
+
theme.fg("dim", "\u2591".repeat(empty)),
|
|
696
|
+
theme.fg("muted", "]")
|
|
697
|
+
].join("");
|
|
698
|
+
}
|
|
699
|
+
function renderStatusGlyph(item, spinnerIndex, theme) {
|
|
700
|
+
switch (item.status) {
|
|
701
|
+
case "done":
|
|
702
|
+
return theme.fg("success", "\u2713");
|
|
703
|
+
case "error":
|
|
704
|
+
return theme.fg("error", "\u2717");
|
|
705
|
+
case "queued":
|
|
706
|
+
return theme.fg("muted", "\u25CB");
|
|
707
|
+
default:
|
|
708
|
+
return theme.fg(
|
|
709
|
+
"accent",
|
|
710
|
+
SPINNER_FRAMES[spinnerIndex % SPINNER_FRAMES.length] ?? "\u280B"
|
|
711
|
+
);
|
|
712
|
+
}
|
|
713
|
+
}
|
|
714
|
+
function renderBatchProgressText(snapshot, width, expanded, theme) {
|
|
715
|
+
const summary = [
|
|
716
|
+
theme.fg("toolTitle", theme.bold("batch_web_fetch ")),
|
|
717
|
+
theme.fg(
|
|
718
|
+
"muted",
|
|
719
|
+
`${snapshot.completed}/${snapshot.total} done \xB7 ok ${snapshot.succeeded} \xB7 err ${snapshot.failed} \xB7 concurrency ${snapshot.batchConcurrency}`
|
|
720
|
+
)
|
|
721
|
+
].join("");
|
|
722
|
+
const availableRowWidth = Math.max(24, Math.floor(width * 0.8));
|
|
723
|
+
const statusWidth = 10;
|
|
724
|
+
const progressWidth = Math.max(
|
|
725
|
+
10,
|
|
726
|
+
Math.min(18, Math.floor(availableRowWidth * 0.25))
|
|
727
|
+
);
|
|
728
|
+
const glyphWidth = 2;
|
|
729
|
+
const urlWidth = Math.max(
|
|
730
|
+
12,
|
|
731
|
+
availableRowWidth - glyphWidth - statusWidth - progressWidth - 3
|
|
732
|
+
);
|
|
733
|
+
const rows = snapshot.items.map((item, index) => {
|
|
734
|
+
const glyph = renderStatusGlyph(item, snapshot.completed + index, theme);
|
|
735
|
+
const url = theme.fg("accent", truncateMiddle(item.url, urlWidth));
|
|
736
|
+
const statusColor = item.status === "error" ? "error" : item.status === "done" ? "success" : item.status === "queued" ? "muted" : "warning";
|
|
737
|
+
const status = theme.fg(statusColor, pad(item.status, statusWidth));
|
|
738
|
+
const bar = renderProgressBar(item, progressWidth, theme);
|
|
739
|
+
const baseRow = `${glyph} ${url} ${status} ${bar}`;
|
|
740
|
+
if (!expanded || !item.error) {
|
|
741
|
+
return baseRow;
|
|
742
|
+
}
|
|
743
|
+
return `${baseRow}
|
|
744
|
+
${theme.fg("error", `error: ${item.error}`)}`;
|
|
745
|
+
});
|
|
746
|
+
return [summary, ...rows].join("\n");
|
|
747
|
+
}
|
|
748
|
+
function createResponsiveBatchComponent(details, expanded, theme) {
|
|
749
|
+
const text = new Text("", 0, 0);
|
|
750
|
+
return {
|
|
751
|
+
render(width) {
|
|
752
|
+
const snapshot = details.batchProgress;
|
|
753
|
+
if (!snapshot) {
|
|
754
|
+
text.setText(theme.fg("muted", "No batch progress available."));
|
|
755
|
+
return text.render(width);
|
|
756
|
+
}
|
|
757
|
+
text.setText(renderBatchProgressText(snapshot, width, expanded, theme));
|
|
758
|
+
return text.render(width);
|
|
759
|
+
},
|
|
760
|
+
invalidate() {
|
|
761
|
+
text.invalidate();
|
|
762
|
+
}
|
|
763
|
+
};
|
|
764
|
+
}
|
|
471
765
|
function piSmartFetchExtension(pi) {
|
|
472
766
|
const defaults = resolveFetchToolDefaults();
|
|
473
767
|
pi.registerTool({
|
|
@@ -502,6 +796,89 @@ function piSmartFetchExtension(pi) {
|
|
|
502
796
|
};
|
|
503
797
|
}
|
|
504
798
|
});
|
|
799
|
+
pi.registerTool({
|
|
800
|
+
name: "batch_web_fetch",
|
|
801
|
+
label: "batch_web_fetch",
|
|
802
|
+
description: batchToolDescription,
|
|
803
|
+
promptSnippet: "batch_web_fetch(requests, verbose?): fetch multiple URLs concurrently with per-item progress in the pi TUI",
|
|
804
|
+
parameters: Type.Object({
|
|
805
|
+
...createBatchFetchToolParameterProperties(defaults),
|
|
806
|
+
verbose: Type.Optional(
|
|
807
|
+
Type.Boolean({
|
|
808
|
+
description: "Include the full metadata header for each successful result. Default: false, or smartFetchVerboseByDefault from pi settings."
|
|
809
|
+
})
|
|
810
|
+
)
|
|
811
|
+
}),
|
|
812
|
+
async execute(_toolCallId, params, _signal, onUpdate, ctx) {
|
|
813
|
+
const settings = await loadPiSmartFetchSettings(ctx.cwd, getAgentDir());
|
|
814
|
+
const runtimeDefaults = resolveFetchToolDefaults(settings);
|
|
815
|
+
const verbose = params.verbose ?? settings.verboseByDefault;
|
|
816
|
+
const batchResult = await executeBatchFetchToolCall(
|
|
817
|
+
params,
|
|
818
|
+
runtimeDefaults,
|
|
819
|
+
{
|
|
820
|
+
batchConcurrency: runtimeDefaults.batchConcurrency,
|
|
821
|
+
onProgress(snapshot) {
|
|
822
|
+
onUpdate?.({
|
|
823
|
+
content: [
|
|
824
|
+
{
|
|
825
|
+
type: "text",
|
|
826
|
+
text: `Started batch fetch for ${snapshot.total} URLs (${snapshot.completed}/${snapshot.total} complete).`
|
|
827
|
+
}
|
|
828
|
+
],
|
|
829
|
+
details: {
|
|
830
|
+
verbose,
|
|
831
|
+
started: true,
|
|
832
|
+
batchProgress: snapshot
|
|
833
|
+
}
|
|
834
|
+
});
|
|
835
|
+
}
|
|
836
|
+
}
|
|
837
|
+
);
|
|
838
|
+
const finalProgress = {
|
|
839
|
+
items: batchResult.items.map((item) => ({
|
|
840
|
+
index: item.index,
|
|
841
|
+
url: item.request.url,
|
|
842
|
+
status: item.status,
|
|
843
|
+
progress: item.progress,
|
|
844
|
+
...item.error ? { error: item.error } : {}
|
|
845
|
+
})),
|
|
846
|
+
total: batchResult.total,
|
|
847
|
+
completed: batchResult.total,
|
|
848
|
+
succeeded: batchResult.succeeded,
|
|
849
|
+
failed: batchResult.failed,
|
|
850
|
+
batchConcurrency: batchResult.batchConcurrency
|
|
851
|
+
};
|
|
852
|
+
return {
|
|
853
|
+
content: [
|
|
854
|
+
{
|
|
855
|
+
type: "text",
|
|
856
|
+
text: buildBatchFetchResponseText(batchResult, { verbose })
|
|
857
|
+
}
|
|
858
|
+
],
|
|
859
|
+
details: {
|
|
860
|
+
verbose,
|
|
861
|
+
started: true,
|
|
862
|
+
batchProgress: finalProgress,
|
|
863
|
+
batchResult
|
|
864
|
+
}
|
|
865
|
+
};
|
|
866
|
+
},
|
|
867
|
+
renderCall(args, theme) {
|
|
868
|
+
const batchArgs = args;
|
|
869
|
+
const requestCount = Array.isArray(batchArgs.requests) ? batchArgs.requests.length : 0;
|
|
870
|
+
let text = theme.fg("toolTitle", theme.bold("batch_web_fetch "));
|
|
871
|
+
text += theme.fg("muted", `${requestCount} urls`);
|
|
872
|
+
return new Text(text, 0, 0);
|
|
873
|
+
},
|
|
874
|
+
renderResult(result, { expanded }, theme) {
|
|
875
|
+
return createResponsiveBatchComponent(
|
|
876
|
+
result.details ?? {},
|
|
877
|
+
expanded,
|
|
878
|
+
theme
|
|
879
|
+
);
|
|
880
|
+
}
|
|
881
|
+
});
|
|
505
882
|
}
|
|
506
883
|
|
|
507
884
|
export { piSmartFetchExtension as default };
|
package/dist/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"sources":["../../core/src/constants.ts","../../core/src/dependencies.ts","../../core/src/dom.ts","../../core/src/format.ts","../../core/src/extract.ts","../../core/src/tool.ts","../src/settings.ts","../src/index.ts"],"names":["wreqFetch","defuddleFetch","Type","getAgentDir"],"mappings":";;;;;;;;;;;AAEO,IAAM,eAAA,GAAkB,YAAA;AACxB,IAAM,UAAA,GAA4B,SAAA;AAClC,IAAM,iBAAA,GAAoB,GAAA;AAC1B,IAAM,kBAAA,GAAqB,IAAA;AAC3B,IAAM,uBAAA,GAA0B,YAAA;AAChC,IAAM,qBAAA,GACX,iEAAA;AACK,IAAM,0BAAA,GACX,iFAAA;AACK,IAAM,8BAAA,GAAiC,gBAAA;ACPvC,IAAM,mBAAA,GAAyC;AAAA,EACpD,KAAA,EAAOA,KAAA;AAAA,EACP,QAAA,EAAU,QAAA;AAAA,EACV;AACF,CAAA;ACLO,SAAS,iBAAA,CAAkB,MAAc,GAAA,EAAwB;AACtE,EAAA,MAAM,EAAE,QAAA,EAAS,GAAI,SAAA,CAAU,IAAI,CAAA;AACnC,EAAA,MAAM,GAAA,GAAM,QAAA;AACZ,EAAA,MAAM,cAAc,GAAA,CAAI,WAAA;AASxB,EAAA,IAAI,CAAE,IAAkC,WAAA,EAAa;AACnD,IAAC,GAAA,CAAkC,cACjC,EAAC;AAAA,EACL;AAEA,EAAA,IAAI,WAAA,IAAe,CAAC,WAAA,CAAY,gBAAA,EAAkB;AAChD,IAAA,WAAA,CAAY,oBAAoB,OAAO;AAAA,MACrC,OAAA,EAAS;AAAA,KACX,CAAA,CAAA;AAAA,EACF;AAEA,EAAA,IAAI,GAAA,EAAK;AACP,IAAC,IAAyB,GAAA,GAAM,GAAA;AAAA,EAClC;AAEA,EAAA,OAAO,QAAA;AACT;;;AC7BA,SAAS,YACP,KAAA,EACA;AACA,EAAA,OAAO,KAAA,CACJ,MAAA,CAAO,CAAC,GAAG,KAAK,CAAA,KAAM,KAAA,KAAU,MAAA,IAAa,KAAA,KAAU,EAAE,CAAA,CACzD,IAAI,CAAC,CAAC,KAAA,EAAO,KAAK,CAAA,KAAM,CAAA,EAAA,EAAK,KAAK,CAAA,EAAA,EAAK,KAAK,CAAA,CAAE,CAAA,CAC9C,IAAA,CAAK,IAAI,CAAA;AACd;AAEO,SAAS,eAAe,QAAA,EAA0B;AACvD,EAAA,OAAO,QAAA,CACJ,OAAA,CAAQ,cAAA,EAAgB,EAAE,CAAA,CAC1B,OAAA,CAAQ,kBAAA,EAAoB,IAAI,CAAA,CAChC,OAAA,CAAQ,cAAA,EAAgB,IAAI,EAC5B,OAAA,CAAQ,wBAAA,EAA0B,IAAI,CAAA,CACtC,OAAA,CAAQ,uBAAA,EAAyB,EAAE,CAAA,CACnC,QAAQ,SAAA,EAAW,EAAE,CAAA,CACrB,OAAA,CAAQ,aAAA,EAAe,SAAI,CAAA,CAC3B,OAAA,CAAQ,cAAc,IAAI,CAAA;AAC/B;AAEO,SAAS,eAAA,CAAgB,SAAiB,QAAA,EAA0B;AACzE,EAAA,IAAI,OAAA,CAAQ,MAAA,IAAU,QAAA,EAAU,OAAO,OAAA;AACvC,EAAA,OAAO,CAAA,EAAG,OAAA,CAAQ,KAAA,CAAM,CAAA,EAAG,QAAQ,CAAC;;AAAA,eAAA,CAAA;AACtC;AAEO,SAAS,2BAA2B,MAAA,EAA6B;AACtE,EAAA,OAAO,WAAA,CAAY;AAAA,IACjB,CAAC,KAAA,EAAO,MAAA,CAAO,QAAQ,CAAA;AAAA,IACvB,CAAC,OAAA,EAAS,MAAA,CAAO,KAAK,CAAA;AAAA,IACtB,CAAC,QAAA,EAAU,MAAA,CAAO,MAAM,CAAA;AAAA,IACxB,CAAC,WAAA,EAAa,MAAA,CAAO,SAAS;AAAA,GAC/B,CAAA;AACH;AAEO,SAAS,oBAAoB,MAAA,EAA6B;AAC/D,EAAA,OAAO,WAAA,CAAY;AAAA,IACjB,CAAC,KAAA,EAAO,MAAA,CAAO,QAAQ,CAAA;AAAA,IACvB,CAAC,OAAA,EAAS,MAAA,CAAO,KAAK,CAAA;AAAA,IACtB,CAAC,QAAA,EAAU,MAAA,CAAO,MAAM,CAAA;AAAA,IACxB,CAAC,WAAA,EAAa,MAAA,CAAO,SAAS,CAAA;AAAA,IAC9B,CAAC,MAAA,EAAQ,MAAA,CAAO,IAAI,CAAA;AAAA,IACpB,CAAC,UAAA,EAAY,MAAA,CAAO,QAAQ,CAAA;AAAA,IAC5B,CAAC,OAAA,EAAS,MAAA,CAAO,SAAS,CAAA;AAAA,IAC1B,CAAC,WAAW,CAAA,EAAG,MAAA,CAAO,OAAO,CAAA,CAAA,EAAI,MAAA,CAAO,EAAE,CAAA,CAAE;AAAA,GAC7C,CAAA;AACH;AAEO,SAAS,sBAAA,CACd,MAAA,EACA,OAAA,GAAiC,EAAC,EAC1B;AACR,EAAA,MAAM,SAAS,OAAA,CAAQ,OAAA,GACnB,oBAAoB,MAAM,CAAA,GAC1B,2BAA2B,MAAM,CAAA;AAErC,EAAA,OAAO,MAAA,GAAS,GAAG,MAAM;;AAAA,EAAO,MAAA,CAAO,OAAO,CAAA,CAAA,GAAK,MAAA,CAAO,OAAA;AAC5D;AAEO,SAAS,kBAAkB,OAAA,EAAyB;AACzD,EAAA,MAAM,KAAA,GAAQ,OAAA,CAAQ,IAAA,EAAK,CAAE,MAAM,MAAM,CAAA;AACzC,EAAA,OAAO,OAAO,MAAA,IAAU,CAAA;AAC1B;AAEO,SAAS,WAAW,KAAA,EAAuB;AAChD,EAAA,OAAO,MACJ,OAAA,CAAQ,IAAA,EAAM,OAAO,CAAA,CACrB,OAAA,CAAQ,MAAM,MAAM,CAAA,CACpB,QAAQ,IAAA,EAAM,MAAM,EACpB,OAAA,CAAQ,IAAA,EAAM,QAAQ,CAAA,CACtB,OAAA,CAAQ,MAAM,OAAO,CAAA;AAC1B;AAEO,SAAS,mBACd,GAAA,EACoC;AACpC,EAAA,IAAI;AACF,IAAA,OAAO;AAAA,MACL,SAAA,EAAW,KAAK,SAAA,CAAU,IAAA,CAAK,MAAM,GAAG,CAAA,EAAG,MAAM,CAAC;AAAA,KACpD;AAAA,EACF,CAAA,CAAA,MAAQ;AACN,IAAA,OAAO,EAAE,OAAO,uBAAA,EAAwB;AAAA,EAC1C;AACF;AAEO,SAAS,iBAAA,CACd,eACA,MAAA,EACQ;AACR,EAAA,QAAQ,MAAA;AAAQ,IACd,KAAK,MAAA;AAAA,IACL,KAAK,MAAA;AACH,MAAA,OAAO,aAAA;AAAA,IACT,KAAK,MAAA;AACH,MAAA,OAAO,CAAA,iCAAA,EAAoC,UAAA,CAAW,aAAa,CAAC,CAAA,aAAA,CAAA;AAAA,IACtE;AACE,MAAA,OAAO,CAAA;AAAA,EAAe,aAAa;AAAA,MAAA,CAAA;AAAA;AAEzC;AAEO,SAAS,sBAAA,CACd,SACA,MAAA,EACQ;AACR,EAAA,IAAI,WAAW,MAAA,EAAQ;AACrB,IAAA,OAAO,OAAA,CACJ,OAAA,CAAQ,iDAAA,EAAmD,EAAE,EAC7D,OAAA,EAAQ;AAAA,EACb;AAEA,EAAA,OAAO,OAAA,CAAQ,OAAA,CAAQ,gCAAA,EAAkC,EAAE,EAAE,OAAA,EAAQ;AACvE;;;ACjEA,IAAM,kBAAA,GAAqB;AAAA,EACzB,WAAA;AAAA,EACA,uBAAA;AAAA,EACA,YAAA;AAAA,EACA;AACF,CAAA;AAEA,SAAS,oBAAoB,MAAA,EAA8B;AACzD,EAAA,OAAO,MAAA,KAAW,SAAS,0BAAA,GAA6B,qBAAA;AAC1D;AAEA,SAAS,kBAAkB,WAAA,EAA8B;AACvD,EAAA,MAAM,UAAA,GAAa,WAAA,CAAY,KAAA,CAAM,GAAG,CAAA,CAAE,CAAC,CAAA,EAAG,IAAA,EAAK,CAAE,WAAA,EAAY,IAAK,EAAA;AACtE,EAAA,OACE,eAAe,kBAAA,IACf,UAAA,KAAe,WAAA,IACf,UAAA,CAAW,SAAS,OAAO,CAAA;AAE/B;AAEA,SAAS,iBAAiB,IAAA,EAAuB;AAC/C,EAAA,MAAM,OAAA,GAAU,KAAK,IAAA,EAAK;AAC1B,EAAA,OAAO,QAAQ,UAAA,CAAW,GAAG,CAAA,IAAK,OAAA,CAAQ,WAAW,GAAG,CAAA;AAC1D;AAEA,SAAS,cAAA,CAAe,aAAqB,IAAA,EAAuB;AAClE,EAAA,OAAO,iBAAA,CAAkB,WAAW,CAAA,IAAK,gBAAA,CAAiB,IAAI,CAAA;AAChE;AAEA,SAAS,gBACP,IAAA,EACA,QAAA,EACA,SACA,MAAA,EACA,QAAA,EACA,SACA,EAAA,EAC0B;AAC1B,EAAA,MAAM,UAAA,GAAa,mBAAmB,OAAO,CAAA;AAE7C,EAAA,IAAI,WAAW,UAAA,EAAY;AACzB,IAAA,OAAO,UAAA;AAAA,EACT;AAEA,EAAA,MAAM,OAAA,GAAU,eAAA;AAAA,IACd,iBAAA,CAAkB,UAAA,CAAW,SAAA,EAAW,MAAM,CAAA;AAAA,IAC9C;AAAA,GACF;AAEA,EAAA,OAAO;AAAA,IACL,KAAK,IAAA,CAAK,GAAA;AAAA,IACV,QAAA;AAAA,IACA,KAAA,EAAO,EAAA;AAAA,IACP,MAAA,EAAQ,EAAA;AAAA,IACR,SAAA,EAAW,EAAA;AAAA,IACX,IAAA,EAAM,IAAI,GAAA,CAAI,QAAQ,CAAA,CAAE,QAAA;AAAA,IACxB,QAAA,EAAU,EAAA;AAAA,IACV,SAAA,EAAW,iBAAA,CAAkB,UAAA,CAAW,SAAS,CAAA;AAAA,IACjD,OAAA;AAAA,IACA,OAAA;AAAA,IACA;AAAA,GACF;AACF;AAEA,SAAS,mBAAmB,IAAA,EAAuB;AACjD,EAAA,OACE,IAAA,KAAS,iBACT,IAAA,CAAK,UAAA,CAAW,IAAI,CAAA,IACpB,IAAA,CAAK,WAAW,WAAW,CAAA;AAE/B;AAMO,SAAS,mBAAA,CACd,eAAkC,mBAAA,EAClC;AACA,EAAA,OAAO,eAAeC,eACpB,IAAA,EACmC;AACnC,IAAA,MAAM,OAAA,GAAU,KAAK,OAAA,IAAW,eAAA;AAChC,IAAA,MAAM,EAAA,GAAK,KAAK,EAAA,IAAM,UAAA;AACtB,IAAA,MAAM,MAAA,GAAuB,KAAK,MAAA,IAAU,UAAA;AAC5C,IAAA,MAAM,QAAA,GAAW,KAAK,QAAA,IAAY,iBAAA;AAClC,IAAA,MAAM,YAAA,GAAe,KAAK,YAAA,IAAgB,KAAA;AAC1C,IAAA,MAAM,cAAA,GAAiB,KAAK,cAAA,IAAkB,uBAAA;AAC9C,IAAA,MAAM,SAAA,GAAY,KAAK,SAAA,IAAa,kBAAA;AAEpC,IAAA,IAAI,MAAA;AACJ,IAAA,IAAI;AACF,MAAA,MAAA,GAAS,IAAI,GAAA,CAAI,IAAA,CAAK,GAAG,CAAA;AAAA,IAC3B,CAAA,CAAA,MAAQ;AACN,MAAA,OAAO,EAAE,KAAA,EAAO,CAAA,aAAA,EAAgB,IAAA,CAAK,GAAG,CAAA,CAAA,EAAG;AAAA,IAC7C;AAEA,IAAA,IAAI,CAAC,CAAC,OAAA,EAAS,QAAQ,EAAE,QAAA,CAAS,MAAA,CAAO,QAAQ,CAAA,EAAG;AAClD,MAAA,OAAO;AAAA,QACL,KAAA,EAAO,CAAA,oCAAA,EAAuC,MAAA,CAAO,QAAQ,CAAA;AAAA,OAC/D;AAAA,IACF;AAEA,IAAA,MAAM,YAAA,GAAwC;AAAA,MAC5C,OAAA;AAAA,MACA,EAAA;AAAA,MACA,OAAA,EAAS;AAAA,QACP,MAAA,EAAQ,oBAAoB,MAAM,CAAA;AAAA,QAClC,iBAAA,EAAmB,8BAAA;AAAA,QACnB,GAAG,IAAA,CAAK;AAAA,OACV;AAAA,MACA,QAAA,EAAU,QAAA;AAAA,MACV,OAAA,EAAS;AAAA,KACX;AAEA,IAAA,IAAI,KAAK,KAAA,EAAO;AACd,MAAA,YAAA,CAAa,QAAQ,IAAA,CAAK,KAAA;AAAA,IAC5B;AAEA,IAAA,MAAM,WAAW,MAAM,YAAA,CAAa,KAAA,CAAM,IAAA,CAAK,KAAK,YAAY,CAAA;AAEhE,IAAA,IAAI,CAAC,SAAS,EAAA,EAAI;AAChB,MAAA,OAAO;AAAA,QACL,KAAA,EAAO,QAAQ,QAAA,CAAS,MAAM,IAAI,QAAA,CAAS,UAAU,CAAA,KAAA,EAAQ,IAAA,CAAK,GAAG,CAAA;AAAA,OACvE;AAAA,IACF;AAEA,IAAA,MAAM,QAAA,GAAW,QAAA,CAAS,GAAA,IAAO,IAAA,CAAK,GAAA;AACtC,IAAA,MAAM,WAAA,GAAc,QAAA,CAAS,OAAA,CAAQ,GAAA,CAAI,cAAc,CAAA,IAAK,EAAA;AAC5D,IAAA,MAAM,OAAA,GAAU,MAAM,QAAA,CAAS,IAAA,EAAK;AACpC,IAAA,MAAM,YAAA,GAAe,cAAA,CAAe,WAAA,EAAa,OAAO,CAAA;AAExD,IAAA,IAAI,WAAW,MAAA,EAAQ;AACrB,MAAA,IAAI,CAAC,YAAA,EAAc;AACjB,QAAA,OAAO,EAAE,KAAA,EAAO,CAAA,mCAAA,EAAsC,WAAW,CAAA,CAAA,CAAA,EAAI;AAAA,MACvE;AAEA,MAAA,OAAO,eAAA;AAAA,QACL,IAAA;AAAA,QACA,QAAA;AAAA,QACA,OAAA;AAAA,QACA,MAAA;AAAA,QACA,QAAA;AAAA,QACA,OAAA;AAAA,QACA;AAAA,OACF;AAAA,IACF;AAEA,IAAA,IAAI,YAAA,EAAc;AAChB,MAAA,OAAO,eAAA;AAAA,QACL,IAAA;AAAA,QACA,QAAA;AAAA,QACA,OAAA;AAAA,QACA,MAAA;AAAA,QACA,QAAA;AAAA,QACA,OAAA;AAAA,QACA;AAAA,OACF;AAAA,IACF;AAEA,IAAA,IAAI,CAAC,mBAAmB,IAAA,CAAK,CAAC,UAAU,WAAA,CAAY,QAAA,CAAS,KAAK,CAAC,CAAA,EAAG;AACpE,MAAA,OAAO,EAAE,KAAA,EAAO,CAAA,gCAAA,EAAmC,WAAW,CAAA,CAAA,CAAA,EAAI;AAAA,IACpE;AAEA,IAAA,MAAM,QAAA,GAAW,iBAAA,CAAkB,OAAA,EAAS,QAAQ,CAAA;AACpD,IAAA,MAAM,SAAA,GAAY,MAAM,YAAA,CAAa,QAAA,CAAS,UAAU,QAAA,EAAU;AAAA,MAChE,UAAU,MAAA,KAAW,MAAA;AAAA,MACrB,YAAA;AAAA,MACA;AAAA,KACD,CAAA;AAED,IAAA,IAAI,CAAC,SAAA,CAAU,OAAA,IAAW,SAAA,CAAU,cAAc,CAAA,EAAG;AACnD,MAAA,OAAO;AAAA,QACL,KAAA,EAAO,CAAA,0BAAA,EAA6B,IAAA,CAAK,GAAG,CAAA,sCAAA;AAAA,OAC9C;AAAA,IACF;AAEA,IAAA,IAAI,mBAAmB,SAAA,CAAU,OAAA;AACjC,IAAA,IAAI,YAAY,SAAA,CAAU,SAAA;AAE1B,IAAA,IAAI,mBAAmB,KAAA,IAAS,kBAAA,CAAmB,SAAA,CAAU,IAAA,IAAQ,EAAE,CAAA,EAAG;AACxE,MAAA,MAAM,eAAA,GAAkB,sBAAA,CAAuB,gBAAA,EAAkB,MAAM,CAAA;AACvE,MAAA,IAAI,oBAAoB,gBAAA,EAAkB;AACxC,QAAA,gBAAA,GAAmB,eAAA;AACnB,QAAA,SAAA,GAAY,iBAAA;AAAA,UACV,MAAA,KAAW,MAAA,GACP,cAAA,CAAe,gBAAgB,CAAA,GAC/B;AAAA,SACN;AAAA,MACF;AAAA,IACF;AAEA,IAAA,MAAM,iBAAA,GACJ,MAAA,KAAW,MAAA,GAAS,cAAA,CAAe,gBAAgB,CAAA,GAAI,gBAAA;AAEzD,IAAA,OAAO;AAAA,MACL,KAAK,IAAA,CAAK,GAAA;AAAA,MACV,QAAA;AAAA,MACA,KAAA,EAAO,UAAU,KAAA,IAAS,EAAA;AAAA,MAC1B,MAAA,EAAQ,UAAU,MAAA,IAAU,EAAA;AAAA,MAC5B,SAAA,EAAW,UAAU,SAAA,IAAa,EAAA;AAAA,MAClC,IAAA,EAAM,UAAU,IAAA,IAAQ,EAAA;AAAA,MACxB,QAAA,EAAU,UAAU,QAAA,IAAY,EAAA;AAAA,MAChC,SAAA;AAAA,MACA,OAAA,EAAS,eAAA,CAAgB,iBAAA,EAAmB,QAAQ,CAAA;AAAA,MACpD,OAAA;AAAA,MACA;AAAA,KACF;AAAA,EACF,CAAA;AACF;AAEO,IAAM,gBAAgB,mBAAA,EAAoB;AAG1C,SAAS,QACd,MAAA,EACsB;AACtB,EAAA,OAAO,OAAA,IAAW,MAAA;AACpB;AC1PO,SAAS,wBAAA,CACd,MAAA,GAA0B,EAAC,EACR;AACnB,EAAA,OAAO;AAAA,IACL,QAAA,EAAU,OAAO,QAAA,IAAY,iBAAA;AAAA,IAC7B,SAAA,EAAW,OAAO,SAAA,IAAa,kBAAA;AAAA,IAC/B,OAAA,EAAS,OAAO,OAAA,IAAW,eAAA;AAAA,IAC3B,EAAA,EAAI,OAAO,EAAA,IAAM,UAAA;AAAA,IACjB,YAAA,EAAc,OAAO,YAAA,IAAgB,KAAA;AAAA,IACrC,cAAA,EAAgB,OAAO,cAAA,IAAkB;AAAA,GAC3C;AACF;AAEO,SAAS,uCACd,QAAA,EACyB;AACzB,EAAA,OAAO;AAAA,IACL,KAAK,IAAA,CAAK,MAAA,CAAO,EAAE,WAAA,EAAa,kCAAkC,CAAA;AAAA,IAClE,SAAS,IAAA,CAAK,QAAA;AAAA,MACZ,KAAK,MAAA,CAAO;AAAA,QACV,WAAA,EAAa,CAAA,kDAAA,EAAqD,QAAA,CAAS,OAAO,CAAA,oEAAA;AAAA,OACnF;AAAA,KACH;AAAA,IACA,IAAI,IAAA,CAAK,QAAA;AAAA,MACP,KAAK,MAAA,CAAO;AAAA,QACV,WAAA,EAAa,CAAA,yCAAA,EAA4C,QAAA,CAAS,EAAE,CAAA,+CAAA;AAAA,OACrE;AAAA,KACH;AAAA,IACA,SAAS,IAAA,CAAK,QAAA;AAAA,MACZ,KAAK,MAAA,CAAO,IAAA,CAAK,QAAO,EAAG,IAAA,CAAK,QAAO,EAAG;AAAA,QACxC,WAAA,EACE;AAAA,OACH;AAAA,KACH;AAAA,IACA,UAAU,IAAA,CAAK,QAAA;AAAA,MACb,KAAK,MAAA,CAAO;AAAA,QACV,WAAA,EAAa,CAAA,uCAAA,EAA0C,QAAA,CAAS,QAAQ,CAAA;AAAA,OACzE;AAAA,KACH;AAAA,IACA,QAAQ,IAAA,CAAK,QAAA;AAAA,MACX,IAAA,CAAK,KAAA;AAAA,QACH;AAAA,UACE,IAAA,CAAK,QAAQ,UAAU,CAAA;AAAA,UACvB,IAAA,CAAK,QAAQ,MAAM,CAAA;AAAA,UACnB,IAAA,CAAK,QAAQ,MAAM,CAAA;AAAA,UACnB,IAAA,CAAK,QAAQ,MAAM;AAAA,SACrB;AAAA,QACA;AAAA,UACE,WAAA,EACE;AAAA;AACJ;AACF,KACF;AAAA,IACA,cAAc,IAAA,CAAK,QAAA;AAAA,MACjB,KAAK,OAAA,CAAQ;AAAA,QACX,WAAA,EAAa;AAAA,OACd;AAAA,KACH;AAAA,IACA,gBAAgB,IAAA,CAAK,QAAA;AAAA,MACnB,IAAA,CAAK,KAAA,CAAM,CAAC,IAAA,CAAK,OAAA,IAAW,IAAA,CAAK,OAAA,CAAQ,YAAY,CAAC,CAAA,EAAG;AAAA,QACvD,WAAA,EACE;AAAA,OACH;AAAA,KACH;AAAA,IACA,OAAO,IAAA,CAAK,QAAA;AAAA,MACV,KAAK,MAAA,CAAO;AAAA,QACV,WAAA,EACE;AAAA,OACH;AAAA;AACH,GACF;AACF;AAEA,eAAsB,oBAAA,CACpB,QACA,QAAA,EACmC;AACnC,EAAA,OAAO,aAAA,CAAc;AAAA,IACnB,KAAK,MAAA,CAAO,GAAA;AAAA,IACZ,OAAA,EAAU,MAAA,CAAO,OAAA,IAAsB,QAAA,CAAS,OAAA;AAAA,IAChD,EAAA,EAAK,MAAA,CAAO,EAAA,IAAiB,QAAA,CAAS,EAAA;AAAA,IACtC,SAAS,MAAA,CAAO,OAAA;AAAA,IAChB,QAAA,EAAW,MAAA,CAAO,QAAA,IAAuB,QAAA,CAAS,QAAA;AAAA,IAClD,MAAA,EACG,OAAO,MAAA,IAAoD,UAAA;AAAA,IAC9D,YAAA,EAAe,MAAA,CAAO,YAAA,IAA4B,QAAA,CAAS,YAAA;AAAA,IAC3D,cAAA,EACG,MAAA,CAAO,cAAA,IACR,QAAA,CAAS,cAAA;AAAA,IACX,OAAO,MAAA,CAAO,KAAA;AAAA,IACd,WAAW,QAAA,CAAS;AAAA,GACrB,CAAA;AACH;ACnGA,IAAM,eAAA,uBAAsB,GAAA,CAAmB;AAAA,EAC7C,SAAA;AAAA,EACA,OAAA;AAAA,EACA,OAAA;AAAA,EACA,SAAA;AAAA,EACA;AACF,CAAC,CAAA;AAgBD,SAAS,WAAA,CACP,QACA,IAAA,EACqB;AACrB,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,IAAI,OAAO,MAAA,CAAO,GAAG,CAAA,KAAM,SAAA,EAAW;AACpC,MAAA,OAAO,OAAO,GAAG,CAAA;AAAA,IACnB;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AAEA,SAAS,kBAAA,CACP,QACA,IAAA,EACoB;AACpB,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,MAAM,KAAA,GAAQ,OAAO,GAAG,CAAA;AACxB,IAAA,IAAI,OAAO,UAAU,QAAA,IAAY,MAAA,CAAO,SAAS,KAAK,CAAA,IAAK,QAAQ,CAAA,EAAG;AACpE,MAAA,OAAO,KAAA;AAAA,IACT;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AAEA,SAAS,kBAAA,CACP,QACA,IAAA,EACoB;AACpB,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,MAAM,KAAA,GAAQ,OAAO,GAAG,CAAA;AACxB,IAAA,IAAI,OAAO,KAAA,KAAU,QAAA,IAAY,KAAA,CAAM,IAAA,OAAW,EAAA,EAAI;AACpD,MAAA,OAAO,KAAA;AAAA,IACT;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AAEA,SAAS,MAAA,CACP,QACA,IAAA,EAC2B;AAC3B,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,MAAM,KAAA,GAAQ,OAAO,GAAG,CAAA;AACxB,IAAA,IACE,OAAO,KAAA,KAAU,QAAA,IACjB,eAAA,CAAgB,GAAA,CAAI,KAAsB,CAAA,EAC1C;AACA,MAAA,OAAO,KAAA;AAAA,IACT;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AAEA,SAAS,kBAAA,CACP,QACA,IAAA,EACkC;AAClC,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,MAAM,KAAA,GAAQ,OAAO,GAAG,CAAA;AACxB,IAAA,IAAI,OAAO,KAAA,KAAU,SAAA,IAAa,KAAA,KAAU,YAAA,EAAc;AACxD,MAAA,OAAO,KAAA;AAAA,IACT;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AAEA,SAAS,8BAA8B,KAAA,EAAsC;AAC3E,EAAA,IAAI,CAAC,KAAA,IAAS,OAAO,KAAA,KAAU,QAAA,SAAiB,EAAC;AAEjD,EAAA,MAAM,MAAA,GAAS,KAAA;AAEf,EAAA,OAAO;AAAA,IACL,0BAAA,EAA4B,YAAY,MAAA,EAAQ;AAAA,MAC9C,4BAAA;AAAA,MACA;AAAA,KACD,CAAA;AAAA,IACD,yBAAA,EAA2B,mBAAmB,MAAA,EAAQ;AAAA,MACpD,2BAAA;AAAA,MACA;AAAA,KACD,CAAA;AAAA,IACD,0BAAA,EAA4B,mBAAmB,MAAA,EAAQ;AAAA,MACrD;AAAA,KACD,CAAA;AAAA,IACD,wBAAA,EAA0B,mBAAmB,MAAA,EAAQ;AAAA,MACnD;AAAA,KACD,CAAA;AAAA,IACD,mBAAA,EAAqB,MAAA,CAAO,MAAA,EAAQ,CAAC,qBAAqB,CAAC,CAAA;AAAA,IAC3D,6BAAA,EAA+B,YAAY,MAAA,EAAQ;AAAA,MACjD;AAAA,KACD,CAAA;AAAA,IACD,+BAAA,EAAiC,mBAAmB,MAAA,EAAQ;AAAA,MAC1D;AAAA,KACD;AAAA,GACH;AACF;AAEO,SAAS,2BAAA,CACd,gBACA,eAAA,EAC8B;AAC9B,EAAA,MAAM,MAAA,GAAS,8BAA8B,cAAc,CAAA;AAC3D,EAAA,MAAM,OAAA,GAAU,8BAA8B,eAAe,CAAA;AAE7D,EAAA,OAAO;AAAA,IACL,gBAAA,EACE,OAAA,CAAQ,0BAAA,IACR,MAAA,CAAO,0BAAA,IACP,KAAA;AAAA,IACF,QAAA,EACE,OAAA,CAAQ,yBAAA,IAA6B,MAAA,CAAO,yBAAA;AAAA,IAC9C,SAAA,EACE,OAAA,CAAQ,0BAAA,IAA8B,MAAA,CAAO,0BAAA;AAAA,IAC/C,OAAA,EACE,OAAA,CAAQ,wBAAA,IAA4B,MAAA,CAAO,wBAAA;AAAA,IAC7C,EAAA,EAAI,OAAA,CAAQ,mBAAA,IAAuB,MAAA,CAAO,mBAAA;AAAA,IAC1C,YAAA,EACE,OAAA,CAAQ,6BAAA,IACR,MAAA,CAAO,6BAAA;AAAA,IACT,cAAA,EACE,OAAA,CAAQ,+BAAA,IACR,MAAA,CAAO;AAAA,GACX;AACF;AAEA,eAAe,iBAAiB,IAAA,EAAgC;AAC9D,EAAA,IAAI;AACF,IAAA,OAAO,KAAK,KAAA,CAAM,MAAM,QAAA,CAAS,IAAA,EAAM,OAAO,CAAC,CAAA;AAAA,EACjD,CAAA,CAAA,MAAQ;AACN,IAAA,OAAO,EAAC;AAAA,EACV;AACF;AAEA,eAAsB,wBAAA,CACpB,GAAA,EACA,QAAA,GAAW,WAAA,EAAY,EACgB;AACvC,EAAA,MAAM,iBAAiB,MAAM,gBAAA;AAAA,IAC3B,IAAA,CAAK,UAAU,eAAe;AAAA,GAChC;AACA,EAAA,MAAM,kBAAkB,MAAM,gBAAA;AAAA,IAC5B,IAAA,CAAK,GAAA,EAAK,KAAA,EAAO,eAAe;AAAA,GAClC;AAEA,EAAA,OAAO,2BAAA,CAA4B,gBAAgB,eAAe,CAAA;AACpE;;;AC1KA,IAAM,eAAA,GAAkB;AAAA,EACtB,wFAAA;AAAA,EACA,4FAAA;AAAA,EACA,yFAAA;AAAA,EACA;AACF,CAAA,CAAE,KAAK,GAAG,CAAA;AAEK,SAAR,sBAAuC,EAAA,EAAkB;AAC9D,EAAA,MAAM,WAAW,wBAAA,EAAyB;AAE1C,EAAA,EAAA,CAAG,YAAA,CAAa;AAAA,IACd,IAAA,EAAM,WAAA;AAAA,IACN,KAAA,EAAO,WAAA;AAAA,IACP,WAAA,EAAa,eAAA;AAAA,IACb,aAAA,EACE,iKAAA;AAAA,IACF,UAAA,EAAYC,KAAK,MAAA,CAAO;AAAA,MACtB,GAAG,uCAAuC,QAAQ,CAAA;AAAA,MAClD,SAASA,IAAAA,CAAK,QAAA;AAAA,QACZA,KAAK,OAAA,CAAQ;AAAA,UACX,WAAA,EACE;AAAA,SACH;AAAA;AACH,KACD,CAAA;AAAA,IAED,MAAM,OAAA,CAAQ,WAAA,EAAa,MAAA,EAAQ,OAAA,EAAS,WAAW,GAAA,EAAK;AAC1D,MAAA,MAAM,WAAW,MAAM,wBAAA,CAAyB,GAAA,CAAI,GAAA,EAAKC,aAAa,CAAA;AACtE,MAAA,MAAM,eAAA,GAAkB,yBAAyB,QAAQ,CAAA;AACzD,MAAA,MAAM,OAAA,GACH,MAAA,CAAO,OAAA,IAAmC,QAAA,CAAS,gBAAA;AACtD,MAAA,MAAM,MAAA,GAAS,MAAM,oBAAA,CAAqB,MAAA,EAAQ,eAAe,CAAA;AAEjE,MAAA,IAAI,OAAA,CAAQ,MAAM,CAAA,EAAG;AACnB,QAAA,OAAO;AAAA,UACL,OAAA,EAAS,CAAC,EAAE,IAAA,EAAM,MAAA,EAAQ,MAAM,CAAA,OAAA,EAAU,MAAA,CAAO,KAAK,CAAA,CAAA,EAAI,CAAA;AAAA,UAC1D,OAAA,EAAS,EAAE,KAAA,EAAO,IAAA,EAAM,OAAA;AAAQ,SAClC;AAAA,MACF;AAEA,MAAA,OAAO;AAAA,QACL,OAAA,EAAS;AAAA,UACP,EAAE,MAAM,MAAA,EAAQ,IAAA,EAAM,uBAAuB,MAAA,EAAQ,EAAE,OAAA,EAAS,CAAA;AAAE,SACpE;AAAA,QACA,OAAA,EAAS,EAAE,OAAA,EAAS,QAAA,EAAU,gBAAgB,QAAA;AAAS,OACzD;AAAA,IACF;AAAA,GACD,CAAA;AACH","file":"index.js","sourcesContent":["import type { FingerprintOs } from \"./types\";\n\nexport const DEFAULT_BROWSER = \"chrome_145\";\nexport const DEFAULT_OS: FingerprintOs = \"windows\";\nexport const DEFAULT_MAX_CHARS = 50_000;\nexport const DEFAULT_TIMEOUT_MS = 15_000;\nexport const DEFAULT_INCLUDE_REPLIES = \"extractors\" as const;\nexport const DEFAULT_ACCEPT_HEADER =\n \"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\";\nexport const DEFAULT_JSON_ACCEPT_HEADER =\n \"application/json,text/json,application/ld+json;q=0.9,text/plain;q=0.8,*/*;q=0.7\";\nexport const DEFAULT_ACCEPT_LANGUAGE_HEADER = \"en-US,en;q=0.9\";\n","import { Defuddle } from \"defuddle/node\";\nimport { getProfiles, fetch as wreqFetch } from \"wreq-js\";\nimport type { FetchDependencies } from \"./types\";\n\nexport const runtimeDependencies: FetchDependencies = {\n fetch: wreqFetch,\n defuddle: Defuddle,\n getProfiles,\n};\n","import { parseHTML } from \"linkedom\";\n\n/** Apply linkedom polyfills that Defuddle expects (getComputedStyle, styleSheets). */\nexport function parseLinkedomHTML(html: string, url?: string): Document {\n const { document } = parseHTML(html);\n const doc = document as Document & Record<string, unknown>;\n const defaultView = doc.defaultView as\n | (Window & {\n getComputedStyle?: (\n elt: Element,\n pseudoElt?: string | null,\n ) => CSSStyleDeclaration;\n })\n | undefined;\n\n if (!(doc as { styleSheets?: unknown }).styleSheets) {\n (doc as { styleSheets?: unknown }).styleSheets =\n [] as unknown as StyleSheetList;\n }\n\n if (defaultView && !defaultView.getComputedStyle) {\n defaultView.getComputedStyle = (() => ({\n display: \"\",\n })) as unknown as typeof defaultView.getComputedStyle;\n }\n\n if (url) {\n (doc as { URL?: string }).URL = url;\n }\n\n return document;\n}\n","import type { FetchError, FetchResult, OutputFormat } from \"./types\";\n\nfunction buildHeader(\n parts: Array<[label: string, value: string | number | undefined]>,\n) {\n return parts\n .filter(([, value]) => value !== undefined && value !== \"\")\n .map(([label, value]) => `> ${label}: ${value}`)\n .join(\"\\n\");\n}\n\nexport function markdownToText(markdown: string): string {\n return markdown\n .replace(/^#{1,6}\\s+/gm, \"\")\n .replace(/\\*\\*([^*]+)\\*\\*/g, \"$1\")\n .replace(/\\*([^*]+)\\*/g, \"$1\")\n .replace(/\\[([^\\]]+)\\]\\([^)]+\\)/g, \"$1\")\n .replace(/!\\[[^\\]]*\\]\\([^)]+\\)/g, \"\")\n .replace(/^>\\s+/gm, \"\")\n .replace(/^[-*+]\\s+/gm, \"• \")\n .replace(/`([^`]+)`/g, \"$1\");\n}\n\nexport function truncateContent(content: string, maxChars: number): string {\n if (content.length <= maxChars) return content;\n return `${content.slice(0, maxChars)}\\n\\n[... truncated]`;\n}\n\nexport function buildCompactMetadataHeader(result: FetchResult): string {\n return buildHeader([\n [\"URL\", result.finalUrl],\n [\"Title\", result.title],\n [\"Author\", result.author],\n [\"Published\", result.published],\n ]);\n}\n\nexport function buildMetadataHeader(result: FetchResult): string {\n return buildHeader([\n [\"URL\", result.finalUrl],\n [\"Title\", result.title],\n [\"Author\", result.author],\n [\"Published\", result.published],\n [\"Site\", result.site],\n [\"Language\", result.language],\n [\"Words\", result.wordCount],\n [\"Browser\", `${result.browser}/${result.os}`],\n ]);\n}\n\nexport function buildFetchResponseText(\n result: FetchResult,\n options: { verbose?: boolean } = {},\n): string {\n const header = options.verbose\n ? buildMetadataHeader(result)\n : buildCompactMetadataHeader(result);\n\n return header ? `${header}\\n\\n${result.content}` : result.content;\n}\n\nexport function estimateWordCount(content: string): number {\n const words = content.trim().match(/\\S+/g);\n return words?.length ?? 0;\n}\n\nexport function escapeHtml(value: string): string {\n return value\n .replace(/&/g, \"&\")\n .replace(/</g, \"<\")\n .replace(/>/g, \">\")\n .replace(/\"/g, \""\")\n .replace(/'/g, \"'\");\n}\n\nexport function parseAndFormatJson(\n raw: string,\n): { formatted: string } | FetchError {\n try {\n return {\n formatted: JSON.stringify(JSON.parse(raw), null, 2),\n };\n } catch {\n return { error: \"Invalid JSON response\" };\n }\n}\n\nexport function renderJsonContent(\n formattedJson: string,\n format: OutputFormat,\n): string {\n switch (format) {\n case \"json\":\n case \"text\":\n return formattedJson;\n case \"html\":\n return `<pre><code class=\"language-json\">${escapeHtml(formattedJson)}</code></pre>`;\n default:\n return `\\`\\`\\`json\\n${formattedJson}\\n\\`\\`\\``;\n }\n}\n\nexport function stripExtractorComments(\n content: string,\n format: OutputFormat,\n): string {\n if (format === \"html\") {\n return content\n .replace(/\\s*<hr>\\s*<div class=\"[^\"]* comments\">[\\s\\S]*$/i, \"\")\n .trimEnd();\n }\n\n return content.replace(/\\n---\\n+## Comments\\n[\\s\\S]*$/i, \"\").trimEnd();\n}\n","/**\n * Core extraction pipeline: fetch with TLS fingerprinting → parse → Defuddle extract.\n * Separated from the plugin entry so it can be tested independently.\n */\n\nimport {\n DEFAULT_ACCEPT_HEADER,\n DEFAULT_ACCEPT_LANGUAGE_HEADER,\n DEFAULT_BROWSER,\n DEFAULT_INCLUDE_REPLIES,\n DEFAULT_JSON_ACCEPT_HEADER,\n DEFAULT_MAX_CHARS,\n DEFAULT_OS,\n DEFAULT_TIMEOUT_MS,\n} from \"./constants\";\nimport { runtimeDependencies } from \"./dependencies\";\nimport { parseLinkedomHTML } from \"./dom\";\nimport {\n estimateWordCount,\n markdownToText,\n parseAndFormatJson,\n renderJsonContent,\n stripExtractorComments,\n truncateContent,\n} from \"./format\";\nimport { getLatestChromeProfile as getLatestChromeProfileFrom } from \"./profiles\";\nimport type {\n FetchDependencies,\n FetchError,\n FetchOptions,\n FetchResult,\n OutputFormat,\n} from \"./types\";\n\nexport {\n DEFAULT_BROWSER,\n DEFAULT_INCLUDE_REPLIES,\n DEFAULT_MAX_CHARS,\n DEFAULT_OS,\n DEFAULT_TIMEOUT_MS,\n} from \"./constants\";\nexport type {\n FetchError,\n FetchOptions,\n FetchResult,\n OutputFormat,\n} from \"./types\";\n\nconst HTML_CONTENT_TYPES = [\n \"text/html\",\n \"application/xhtml+xml\",\n \"text/plain\",\n \"text/markdown\",\n];\n\nfunction resolveAcceptHeader(format: OutputFormat): string {\n return format === \"json\" ? DEFAULT_JSON_ACCEPT_HEADER : DEFAULT_ACCEPT_HEADER;\n}\n\nfunction isJsonContentType(contentType: string): boolean {\n const normalized = contentType.split(\";\")[0]?.trim().toLowerCase() ?? \"\";\n return (\n normalized === \"application/json\" ||\n normalized === \"text/json\" ||\n normalized.endsWith(\"+json\")\n );\n}\n\nfunction isLikelyJsonBody(body: string): boolean {\n const trimmed = body.trim();\n return trimmed.startsWith(\"{\") || trimmed.startsWith(\"[\");\n}\n\nfunction isJsonResponse(contentType: string, body: string): boolean {\n return isJsonContentType(contentType) || isLikelyJsonBody(body);\n}\n\nfunction buildJsonResult(\n opts: FetchOptions,\n finalUrl: string,\n rawBody: string,\n format: OutputFormat,\n maxChars: number,\n browser: string,\n os: string,\n): FetchResult | FetchError {\n const parsedJson = parseAndFormatJson(rawBody);\n\n if (\"error\" in parsedJson) {\n return parsedJson;\n }\n\n const content = truncateContent(\n renderJsonContent(parsedJson.formatted, format),\n maxChars,\n );\n\n return {\n url: opts.url,\n finalUrl,\n title: \"\",\n author: \"\",\n published: \"\",\n site: new URL(finalUrl).hostname,\n language: \"\",\n wordCount: estimateWordCount(parsedJson.formatted),\n content,\n browser,\n os,\n };\n}\n\nfunction shouldStripReplies(site: string): boolean {\n return (\n site === \"Hacker News\" ||\n site.startsWith(\"r/\") ||\n site.startsWith(\"GitHub - \")\n );\n}\n\nexport function getLatestChromeProfile(): string {\n return getLatestChromeProfileFrom(runtimeDependencies.getProfiles);\n}\n\nexport function createDefuddleFetch(\n dependencies: FetchDependencies = runtimeDependencies,\n) {\n return async function defuddleFetch(\n opts: FetchOptions,\n ): Promise<FetchResult | FetchError> {\n const browser = opts.browser ?? DEFAULT_BROWSER;\n const os = opts.os ?? DEFAULT_OS;\n const format: OutputFormat = opts.format ?? \"markdown\";\n const maxChars = opts.maxChars ?? DEFAULT_MAX_CHARS;\n const removeImages = opts.removeImages ?? false;\n const includeReplies = opts.includeReplies ?? DEFAULT_INCLUDE_REPLIES;\n const timeoutMs = opts.timeoutMs ?? DEFAULT_TIMEOUT_MS;\n\n let parsed: URL;\n try {\n parsed = new URL(opts.url);\n } catch {\n return { error: `Invalid URL: ${opts.url}` };\n }\n\n if (![\"http:\", \"https:\"].includes(parsed.protocol)) {\n return {\n error: `Only http/https URLs supported, got ${parsed.protocol}`,\n };\n }\n\n const fetchOptions: Record<string, unknown> = {\n browser,\n os,\n headers: {\n Accept: resolveAcceptHeader(format),\n \"Accept-Language\": DEFAULT_ACCEPT_LANGUAGE_HEADER,\n ...opts.headers,\n },\n redirect: \"follow\",\n timeout: timeoutMs,\n };\n\n if (opts.proxy) {\n fetchOptions.proxy = opts.proxy;\n }\n\n const response = await dependencies.fetch(opts.url, fetchOptions);\n\n if (!response.ok) {\n return {\n error: `HTTP ${response.status} ${response.statusText} for ${opts.url}`,\n };\n }\n\n const finalUrl = response.url ?? opts.url;\n const contentType = response.headers.get(\"content-type\") ?? \"\";\n const rawBody = await response.text();\n const jsonResponse = isJsonResponse(contentType, rawBody);\n\n if (format === \"json\") {\n if (!jsonResponse) {\n return { error: `Not a JSON response (content-type: ${contentType})` };\n }\n\n return buildJsonResult(\n opts,\n finalUrl,\n rawBody,\n format,\n maxChars,\n browser,\n os,\n );\n }\n\n if (jsonResponse) {\n return buildJsonResult(\n opts,\n finalUrl,\n rawBody,\n format,\n maxChars,\n browser,\n os,\n );\n }\n\n if (!HTML_CONTENT_TYPES.some((value) => contentType.includes(value))) {\n return { error: `Not an HTML page (content-type: ${contentType})` };\n }\n\n const document = parseLinkedomHTML(rawBody, finalUrl);\n const extracted = await dependencies.defuddle(document, finalUrl, {\n markdown: format !== \"html\",\n removeImages,\n includeReplies,\n });\n\n if (!extracted.content || extracted.wordCount === 0) {\n return {\n error: `No content extracted from ${opts.url}. May need JS rendering or is blocked.`,\n };\n }\n\n let extractedContent = extracted.content;\n let wordCount = extracted.wordCount;\n\n if (includeReplies === false && shouldStripReplies(extracted.site ?? \"\")) {\n const strippedContent = stripExtractorComments(extractedContent, format);\n if (strippedContent !== extractedContent) {\n extractedContent = strippedContent;\n wordCount = estimateWordCount(\n format === \"text\"\n ? markdownToText(extractedContent)\n : extractedContent,\n );\n }\n }\n\n const normalizedContent =\n format === \"text\" ? markdownToText(extractedContent) : extractedContent;\n\n return {\n url: opts.url,\n finalUrl,\n title: extracted.title ?? \"\",\n author: extracted.author ?? \"\",\n published: extracted.published ?? \"\",\n site: extracted.site ?? \"\",\n language: extracted.language ?? \"\",\n wordCount,\n content: truncateContent(normalizedContent, maxChars),\n browser,\n os,\n };\n };\n}\n\nexport const defuddleFetch = createDefuddleFetch();\n\n/** Type guard: check if result is an error. */\nexport function isError(\n result: FetchResult | FetchError,\n): result is FetchError {\n return \"error\" in result;\n}\n","import { type TSchema, Type } from \"@sinclair/typebox\";\nimport {\n DEFAULT_BROWSER,\n DEFAULT_INCLUDE_REPLIES,\n DEFAULT_MAX_CHARS,\n DEFAULT_OS,\n DEFAULT_TIMEOUT_MS,\n} from \"./constants\";\nimport { defuddleFetch } from \"./extract\";\nimport type {\n FetchError,\n FetchResult,\n FetchToolConfig,\n FetchToolDefaults,\n} from \"./types\";\n\nexport function resolveFetchToolDefaults(\n config: FetchToolConfig = {},\n): FetchToolDefaults {\n return {\n maxChars: config.maxChars ?? DEFAULT_MAX_CHARS,\n timeoutMs: config.timeoutMs ?? DEFAULT_TIMEOUT_MS,\n browser: config.browser ?? DEFAULT_BROWSER,\n os: config.os ?? DEFAULT_OS,\n removeImages: config.removeImages ?? false,\n includeReplies: config.includeReplies ?? DEFAULT_INCLUDE_REPLIES,\n };\n}\n\nexport function createBaseFetchToolParameterProperties(\n defaults: FetchToolDefaults,\n): Record<string, TSchema> {\n return {\n url: Type.String({ description: \"URL to fetch (http/https only)\" }),\n browser: Type.Optional(\n Type.String({\n description: `Browser profile for TLS fingerprinting. Default: \"${defaults.browser}\". Examples: chrome_145, firefox_147, safari_26, edge_145, opera_127`,\n }),\n ),\n os: Type.Optional(\n Type.String({\n description: `OS profile for fingerprinting. Default: \"${defaults.os}\". Options: windows, macos, linux, android, ios`,\n }),\n ),\n headers: Type.Optional(\n Type.Record(Type.String(), Type.String(), {\n description:\n \"Custom HTTP headers to send. By default, Accept and Accept-Language are set automatically.\",\n }),\n ),\n maxChars: Type.Optional(\n Type.Number({\n description: `Maximum characters to return. Default: ${defaults.maxChars}`,\n }),\n ),\n format: Type.Optional(\n Type.Union(\n [\n Type.Literal(\"markdown\"),\n Type.Literal(\"html\"),\n Type.Literal(\"text\"),\n Type.Literal(\"json\"),\n ],\n {\n description:\n 'Output format. \"markdown\" (default), \"html\" (cleaned HTML), \"text\" (plain text, no formatting), or \"json\" (pretty-printed JSON)',\n },\n ),\n ),\n removeImages: Type.Optional(\n Type.Boolean({\n description: \"Strip image references from output. Default: false\",\n }),\n ),\n includeReplies: Type.Optional(\n Type.Union([Type.Boolean(), Type.Literal(\"extractors\")], {\n description:\n \"Include replies/comments: 'extractors' for site-specific only (default), true for all, false for none\",\n }),\n ),\n proxy: Type.Optional(\n Type.String({\n description:\n \"Proxy URL (http://user:pass@host:port or socks5://host:port)\",\n }),\n ),\n };\n}\n\nexport async function executeFetchToolCall(\n params: Record<string, unknown>,\n defaults: FetchToolDefaults,\n): Promise<FetchResult | FetchError> {\n return defuddleFetch({\n url: params.url as string,\n browser: (params.browser as string) ?? defaults.browser,\n os: (params.os as string) ?? defaults.os,\n headers: params.headers as Record<string, string> | undefined,\n maxChars: (params.maxChars as number) ?? defaults.maxChars,\n format:\n (params.format as \"markdown\" | \"html\" | \"text\" | \"json\") ?? \"markdown\",\n removeImages: (params.removeImages as boolean) ?? defaults.removeImages,\n includeReplies:\n (params.includeReplies as boolean | \"extractors\") ??\n defaults.includeReplies,\n proxy: params.proxy as string | undefined,\n timeoutMs: defaults.timeoutMs,\n });\n}\n","import { readFile } from \"node:fs/promises\";\nimport { join } from \"node:path\";\nimport { getAgentDir } from \"@mariozechner/pi-coding-agent\";\nimport type {\n FetchToolConfig,\n FingerprintOs,\n IncludeRepliesOption,\n} from \"smart-fetch-core\";\n\nconst VALID_OS_VALUES = new Set<FingerprintOs>([\n \"windows\",\n \"macos\",\n \"linux\",\n \"android\",\n \"ios\",\n]);\n\ninterface PiSmartFetchSettings {\n smartFetchVerboseByDefault?: boolean;\n smartFetchDefaultMaxChars?: number;\n smartFetchDefaultTimeoutMs?: number;\n smartFetchDefaultBrowser?: string;\n smartFetchDefaultOs?: FingerprintOs;\n smartFetchDefaultRemoveImages?: boolean;\n smartFetchDefaultIncludeReplies?: IncludeRepliesOption;\n}\n\nexport interface ResolvedPiSmartFetchSettings extends FetchToolConfig {\n verboseByDefault: boolean;\n}\n\nfunction readBoolean(\n source: Record<string, unknown>,\n keys: string[],\n): boolean | undefined {\n for (const key of keys) {\n if (typeof source[key] === \"boolean\") {\n return source[key] as boolean;\n }\n }\n\n return undefined;\n}\n\nfunction readPositiveNumber(\n source: Record<string, unknown>,\n keys: string[],\n): number | undefined {\n for (const key of keys) {\n const value = source[key];\n if (typeof value === \"number\" && Number.isFinite(value) && value > 0) {\n return value;\n }\n }\n\n return undefined;\n}\n\nfunction readNonEmptyString(\n source: Record<string, unknown>,\n keys: string[],\n): string | undefined {\n for (const key of keys) {\n const value = source[key];\n if (typeof value === \"string\" && value.trim() !== \"\") {\n return value;\n }\n }\n\n return undefined;\n}\n\nfunction readOs(\n source: Record<string, unknown>,\n keys: string[],\n): FingerprintOs | undefined {\n for (const key of keys) {\n const value = source[key];\n if (\n typeof value === \"string\" &&\n VALID_OS_VALUES.has(value as FingerprintOs)\n ) {\n return value as FingerprintOs;\n }\n }\n\n return undefined;\n}\n\nfunction readIncludeReplies(\n source: Record<string, unknown>,\n keys: string[],\n): IncludeRepliesOption | undefined {\n for (const key of keys) {\n const value = source[key];\n if (typeof value === \"boolean\" || value === \"extractors\") {\n return value;\n }\n }\n\n return undefined;\n}\n\nfunction normalizePiSmartFetchSettings(input: unknown): PiSmartFetchSettings {\n if (!input || typeof input !== \"object\") return {};\n\n const source = input as Record<string, unknown>;\n\n return {\n smartFetchVerboseByDefault: readBoolean(source, [\n \"smartFetchVerboseByDefault\",\n \"webFetchVerboseByDefault\",\n ]),\n smartFetchDefaultMaxChars: readPositiveNumber(source, [\n \"smartFetchDefaultMaxChars\",\n \"webFetchDefaultMaxChars\",\n ]),\n smartFetchDefaultTimeoutMs: readPositiveNumber(source, [\n \"smartFetchDefaultTimeoutMs\",\n ]),\n smartFetchDefaultBrowser: readNonEmptyString(source, [\n \"smartFetchDefaultBrowser\",\n ]),\n smartFetchDefaultOs: readOs(source, [\"smartFetchDefaultOs\"]),\n smartFetchDefaultRemoveImages: readBoolean(source, [\n \"smartFetchDefaultRemoveImages\",\n ]),\n smartFetchDefaultIncludeReplies: readIncludeReplies(source, [\n \"smartFetchDefaultIncludeReplies\",\n ]),\n };\n}\n\nexport function resolvePiSmartFetchSettings(\n globalSettings: unknown,\n projectSettings: unknown,\n): ResolvedPiSmartFetchSettings {\n const global = normalizePiSmartFetchSettings(globalSettings);\n const project = normalizePiSmartFetchSettings(projectSettings);\n\n return {\n verboseByDefault:\n project.smartFetchVerboseByDefault ??\n global.smartFetchVerboseByDefault ??\n false,\n maxChars:\n project.smartFetchDefaultMaxChars ?? global.smartFetchDefaultMaxChars,\n timeoutMs:\n project.smartFetchDefaultTimeoutMs ?? global.smartFetchDefaultTimeoutMs,\n browser:\n project.smartFetchDefaultBrowser ?? global.smartFetchDefaultBrowser,\n os: project.smartFetchDefaultOs ?? global.smartFetchDefaultOs,\n removeImages:\n project.smartFetchDefaultRemoveImages ??\n global.smartFetchDefaultRemoveImages,\n includeReplies:\n project.smartFetchDefaultIncludeReplies ??\n global.smartFetchDefaultIncludeReplies,\n };\n}\n\nasync function readSettingsFile(path: string): Promise<unknown> {\n try {\n return JSON.parse(await readFile(path, \"utf-8\"));\n } catch {\n return {};\n }\n}\n\nexport async function loadPiSmartFetchSettings(\n cwd: string,\n agentDir = getAgentDir(),\n): Promise<ResolvedPiSmartFetchSettings> {\n const globalSettings = await readSettingsFile(\n join(agentDir, \"settings.json\"),\n );\n const projectSettings = await readSettingsFile(\n join(cwd, \".pi\", \"settings.json\"),\n );\n\n return resolvePiSmartFetchSettings(globalSettings, projectSettings);\n}\n","import { type ExtensionAPI, getAgentDir } from \"@mariozechner/pi-coding-agent\";\nimport { Type } from \"@sinclair/typebox\";\nimport {\n buildFetchResponseText,\n createBaseFetchToolParameterProperties,\n executeFetchToolCall,\n isError,\n resolveFetchToolDefaults,\n} from \"smart-fetch-core\";\nimport { loadPiSmartFetchSettings } from \"./settings\";\n\nconst toolDescription = [\n \"Fetch a URL with browser-grade TLS fingerprinting and extract clean, readable content.\",\n \"Uses wreq-js for browser-like TLS/HTTP2 impersonation and Defuddle for article extraction.\",\n \"Supports the same fetch parameters as the OpenClaw tool, plus an optional verbose flag.\",\n \"Does NOT execute JavaScript — use a browser automation tool for JS-heavy pages.\",\n].join(\" \");\n\nexport default function piSmartFetchExtension(pi: ExtensionAPI) {\n const defaults = resolveFetchToolDefaults();\n\n pi.registerTool({\n name: \"web_fetch\",\n label: \"web_fetch\",\n description: toolDescription,\n promptSnippet:\n \"web_fetch(url, browser?, os?, headers?, maxChars?, format?, removeImages?, includeReplies?, proxy?, verbose?): fetch browser-fingerprinted readable web content\",\n parameters: Type.Object({\n ...createBaseFetchToolParameterProperties(defaults),\n verbose: Type.Optional(\n Type.Boolean({\n description:\n \"Include the full metadata header (site, language, word count, browser fingerprint info). Default: false, or smartFetchVerboseByDefault from pi settings.\",\n }),\n ),\n }),\n\n async execute(_toolCallId, params, _signal, _onUpdate, ctx) {\n const settings = await loadPiSmartFetchSettings(ctx.cwd, getAgentDir());\n const runtimeDefaults = resolveFetchToolDefaults(settings);\n const verbose =\n (params.verbose as boolean | undefined) ?? settings.verboseByDefault;\n const result = await executeFetchToolCall(params, runtimeDefaults);\n\n if (isError(result)) {\n return {\n content: [{ type: \"text\", text: `Error: ${result.error}` }],\n details: { error: true, verbose },\n };\n }\n\n return {\n content: [\n { type: \"text\", text: buildFetchResponseText(result, { verbose }) },\n ],\n details: { verbose, maxChars: runtimeDefaults.maxChars },\n };\n },\n });\n}\n"]}
|
|
1
|
+
{"version":3,"sources":["../../core/src/constants.ts","../../core/src/dependencies.ts","../../core/src/dom.ts","../../core/src/format.ts","../../core/src/extract.ts","../../core/src/tool.ts","../src/settings.ts","../src/index.ts"],"names":["wreqFetch","defuddleFetch","result","Type","getAgentDir"],"mappings":";;;;;;;;;;;;AAEO,IAAM,eAAA,GAAkB,YAAA;AACxB,IAAM,UAAA,GAA4B,SAAA;AAClC,IAAM,iBAAA,GAAoB,GAAA;AAC1B,IAAM,kBAAA,GAAqB,IAAA;AAC3B,IAAM,yBAAA,GAA4B,CAAA;AAClC,IAAM,uBAAA,GAA0B,YAAA;AAChC,IAAM,qBAAA,GACX,iEAAA;AACK,IAAM,0BAAA,GACX,iFAAA;AACK,IAAM,8BAAA,GAAiC,gBAAA;ACRvC,IAAM,mBAAA,GAAyC;AAAA,EACpD,KAAA,EAAOA,KAAA;AAAA,EACP,QAAA,EAAU,QAAA;AAAA,EACV;AACF,CAAA;ACLO,SAAS,iBAAA,CAAkB,MAAc,GAAA,EAAwB;AACtE,EAAA,MAAM,EAAE,QAAA,EAAS,GAAI,SAAA,CAAU,IAAI,CAAA;AACnC,EAAA,MAAM,GAAA,GAAM,QAAA;AACZ,EAAA,MAAM,cAAc,GAAA,CAAI,WAAA;AASxB,EAAA,IAAI,CAAE,IAAkC,WAAA,EAAa;AACnD,IAAC,GAAA,CAAkC,cACjC,EAAC;AAAA,EACL;AAEA,EAAA,IAAI,WAAA,IAAe,CAAC,WAAA,CAAY,gBAAA,EAAkB;AAChD,IAAA,WAAA,CAAY,oBAAoB,OAAO;AAAA,MACrC,OAAA,EAAS;AAAA,KACX,CAAA,CAAA;AAAA,EACF;AAEA,EAAA,IAAI,GAAA,EAAK;AACP,IAAC,IAAyB,GAAA,GAAM,GAAA;AAAA,EAClC;AAEA,EAAA,OAAO,QAAA;AACT;;;ACvBA,SAAS,YACP,KAAA,EACA;AACA,EAAA,OAAO,KAAA,CACJ,MAAA,CAAO,CAAC,GAAG,KAAK,CAAA,KAAM,KAAA,KAAU,MAAA,IAAa,KAAA,KAAU,EAAE,CAAA,CACzD,IAAI,CAAC,CAAC,KAAA,EAAO,KAAK,CAAA,KAAM,CAAA,EAAA,EAAK,KAAK,CAAA,EAAA,EAAK,KAAK,CAAA,CAAE,CAAA,CAC9C,IAAA,CAAK,IAAI,CAAA;AACd;AAEO,SAAS,eAAe,QAAA,EAA0B;AACvD,EAAA,OAAO,QAAA,CACJ,OAAA,CAAQ,cAAA,EAAgB,EAAE,CAAA,CAC1B,OAAA,CAAQ,kBAAA,EAAoB,IAAI,CAAA,CAChC,OAAA,CAAQ,cAAA,EAAgB,IAAI,EAC5B,OAAA,CAAQ,wBAAA,EAA0B,IAAI,CAAA,CACtC,OAAA,CAAQ,uBAAA,EAAyB,EAAE,CAAA,CACnC,QAAQ,SAAA,EAAW,EAAE,CAAA,CACrB,OAAA,CAAQ,aAAA,EAAe,SAAI,CAAA,CAC3B,OAAA,CAAQ,cAAc,IAAI,CAAA;AAC/B;AAEO,SAAS,eAAA,CAAgB,SAAiB,QAAA,EAA0B;AACzE,EAAA,IAAI,OAAA,CAAQ,MAAA,IAAU,QAAA,EAAU,OAAO,OAAA;AACvC,EAAA,OAAO,CAAA,EAAG,OAAA,CAAQ,KAAA,CAAM,CAAA,EAAG,QAAQ,CAAC;;AAAA,eAAA,CAAA;AACtC;AAEO,SAAS,2BAA2B,MAAA,EAA6B;AACtE,EAAA,OAAO,WAAA,CAAY;AAAA,IACjB,CAAC,KAAA,EAAO,MAAA,CAAO,QAAQ,CAAA;AAAA,IACvB,CAAC,OAAA,EAAS,MAAA,CAAO,KAAK,CAAA;AAAA,IACtB,CAAC,QAAA,EAAU,MAAA,CAAO,MAAM,CAAA;AAAA,IACxB,CAAC,WAAA,EAAa,MAAA,CAAO,SAAS;AAAA,GAC/B,CAAA;AACH;AAEO,SAAS,oBAAoB,MAAA,EAA6B;AAC/D,EAAA,OAAO,WAAA,CAAY;AAAA,IACjB,CAAC,KAAA,EAAO,MAAA,CAAO,QAAQ,CAAA;AAAA,IACvB,CAAC,OAAA,EAAS,MAAA,CAAO,KAAK,CAAA;AAAA,IACtB,CAAC,QAAA,EAAU,MAAA,CAAO,MAAM,CAAA;AAAA,IACxB,CAAC,WAAA,EAAa,MAAA,CAAO,SAAS,CAAA;AAAA,IAC9B,CAAC,MAAA,EAAQ,MAAA,CAAO,IAAI,CAAA;AAAA,IACpB,CAAC,UAAA,EAAY,MAAA,CAAO,QAAQ,CAAA;AAAA,IAC5B,CAAC,OAAA,EAAS,MAAA,CAAO,SAAS,CAAA;AAAA,IAC1B,CAAC,WAAW,CAAA,EAAG,MAAA,CAAO,OAAO,CAAA,CAAA,EAAI,MAAA,CAAO,EAAE,CAAA,CAAE;AAAA,GAC7C,CAAA;AACH;AAEO,SAAS,sBAAA,CACd,MAAA,EACA,OAAA,GAAiC,EAAC,EAC1B;AACR,EAAA,MAAM,SAAS,OAAA,CAAQ,OAAA,GACnB,oBAAoB,MAAM,CAAA,GAC1B,2BAA2B,MAAM,CAAA;AAErC,EAAA,OAAO,MAAA,GAAS,GAAG,MAAM;;AAAA,EAAO,MAAA,CAAO,OAAO,CAAA,CAAA,GAAK,MAAA,CAAO,OAAA;AAC5D;AAEA,SAAS,qBAAA,CACP,MACA,KAAA,EACQ;AACR,EAAA,MAAM,OAAA,GAAU,KAAK,KAAA,GAAQ,CAAA;AAC7B,EAAA,MAAM,GAAA,GAAM,IAAA,CAAK,MAAA,EAAQ,QAAA,IAAY,KAAK,OAAA,CAAQ,GAAA;AAClD,EAAA,OAAO,CAAA,IAAA,EAAO,OAAO,CAAA,CAAA,EAAI,KAAK,KAAK,GAAG,CAAA,CAAA;AACxC;AAEA,SAAS,kBAAA,CACP,IAAA,EACA,KAAA,EACA,OAAA,GAAiC,EAAC,EAC1B;AACR,EAAA,MAAM,OAAA,GAAU,qBAAA,CAAsB,IAAA,EAAM,KAAK,CAAA;AAEjD,EAAA,IAAI,IAAA,CAAK,WAAW,OAAA,EAAS;AAC3B,IAAA,MAAM,cAAc,WAAA,CAAY;AAAA,MAC9B,CAAC,KAAA,EAAO,IAAA,CAAK,OAAA,CAAQ,GAAG,CAAA;AAAA,MACxB,CAAC,UAAU,OAAO,CAAA;AAAA,MAClB,CAAC,OAAA,EAAS,IAAA,CAAK,KAAA,IAAS,eAAe;AAAA,KACxC,CAAA;AACD,IAAA,OAAO,GAAG,OAAO;AAAA,EAAK,WAAW,CAAA,CAAA;AAAA,EACnC;AAEA,EAAA,OAAO,GAAG,OAAO;AAAA,EAAK,sBAAA,CAAuB,IAAA,CAAK,MAAA,EAAuB,OAAO,CAAC,CAAA,CAAA;AACnF;AAEO,SAAS,2BAAA,CACd,MAAA,EACA,OAAA,GAAiC,EAAC,EAC1B;AACR,EAAA,MAAM,UAAU,WAAA,CAAY;AAAA,IAC1B,CAAC,UAAA,EAAY,MAAA,CAAO,KAAK,CAAA;AAAA,IACzB,CAAC,WAAA,EAAa,MAAA,CAAO,SAAS,CAAA;AAAA,IAC9B,CAAC,QAAA,EAAU,MAAA,CAAO,MAAM,CAAA;AAAA,IACxB,CAAC,aAAA,EAAe,MAAA,CAAO,gBAAgB;AAAA,GACxC,CAAA;AACD,EAAA,MAAM,KAAA,GAAQ,OAAO,KAAA,CAAM,GAAA;AAAA,IAAI,CAAC,IAAA,KAC9B,kBAAA,CAAmB,IAAA,EAAM,MAAA,CAAO,OAAO,OAAO;AAAA,GAChD;AAEA,EAAA,OAAO,CAAC,SAAS,GAAG,KAAK,EAAE,MAAA,CAAO,OAAO,CAAA,CAAE,IAAA,CAAK,MAAM,CAAA;AACxD;AAEO,SAAS,kBAAkB,OAAA,EAAyB;AACzD,EAAA,MAAM,KAAA,GAAQ,OAAA,CAAQ,IAAA,EAAK,CAAE,MAAM,MAAM,CAAA;AACzC,EAAA,OAAO,OAAO,MAAA,IAAU,CAAA;AAC1B;AAEO,SAAS,WAAW,KAAA,EAAuB;AAChD,EAAA,OAAO,MACJ,OAAA,CAAQ,IAAA,EAAM,OAAO,CAAA,CACrB,OAAA,CAAQ,MAAM,MAAM,CAAA,CACpB,QAAQ,IAAA,EAAM,MAAM,EACpB,OAAA,CAAQ,IAAA,EAAM,QAAQ,CAAA,CACtB,OAAA,CAAQ,MAAM,OAAO,CAAA;AAC1B;AAEO,SAAS,mBACd,GAAA,EACoC;AACpC,EAAA,IAAI;AACF,IAAA,OAAO;AAAA,MACL,SAAA,EAAW,KAAK,SAAA,CAAU,IAAA,CAAK,MAAM,GAAG,CAAA,EAAG,MAAM,CAAC;AAAA,KACpD;AAAA,EACF,CAAA,CAAA,MAAQ;AACN,IAAA,OAAO,EAAE,OAAO,uBAAA,EAAwB;AAAA,EAC1C;AACF;AAEO,SAAS,iBAAA,CACd,eACA,MAAA,EACQ;AACR,EAAA,QAAQ,MAAA;AAAQ,IACd,KAAK,MAAA;AAAA,IACL,KAAK,MAAA;AACH,MAAA,OAAO,aAAA;AAAA,IACT,KAAK,MAAA;AACH,MAAA,OAAO,CAAA,iCAAA,EAAoC,UAAA,CAAW,aAAa,CAAC,CAAA,aAAA,CAAA;AAAA,IACtE;AACE,MAAA,OAAO,CAAA;AAAA,EAAe,aAAa;AAAA,MAAA,CAAA;AAAA;AAEzC;AAEO,SAAS,sBAAA,CACd,SACA,MAAA,EACQ;AACR,EAAA,IAAI,WAAW,MAAA,EAAQ;AACrB,IAAA,OAAO,OAAA,CACJ,OAAA,CAAQ,iDAAA,EAAmD,EAAE,EAC7D,OAAA,EAAQ;AAAA,EACb;AAEA,EAAA,OAAO,OAAA,CAAQ,OAAA,CAAQ,gCAAA,EAAkC,EAAE,EAAE,OAAA,EAAQ;AACvE;;;AClHA,IAAM,kBAAA,GAAqB;AAAA,EACzB,WAAA;AAAA,EACA,uBAAA;AAAA,EACA,YAAA;AAAA,EACA;AACF,CAAA;AAEA,SAAS,oBAAoB,MAAA,EAA8B;AACzD,EAAA,OAAO,MAAA,KAAW,SAAS,0BAAA,GAA6B,qBAAA;AAC1D;AAEA,SAAS,kBAAkB,WAAA,EAA8B;AACvD,EAAA,MAAM,UAAA,GAAa,WAAA,CAAY,KAAA,CAAM,GAAG,CAAA,CAAE,CAAC,CAAA,EAAG,IAAA,EAAK,CAAE,WAAA,EAAY,IAAK,EAAA;AACtE,EAAA,OACE,eAAe,kBAAA,IACf,UAAA,KAAe,WAAA,IACf,UAAA,CAAW,SAAS,OAAO,CAAA;AAE/B;AAEA,SAAS,iBAAiB,IAAA,EAAuB;AAC/C,EAAA,MAAM,OAAA,GAAU,KAAK,IAAA,EAAK;AAC1B,EAAA,OAAO,QAAQ,UAAA,CAAW,GAAG,CAAA,IAAK,OAAA,CAAQ,WAAW,GAAG,CAAA;AAC1D;AAEA,SAAS,cAAA,CAAe,aAAqB,IAAA,EAAuB;AAClE,EAAA,OAAO,iBAAA,CAAkB,WAAW,CAAA,IAAK,gBAAA,CAAiB,IAAI,CAAA;AAChE;AAEA,SAAS,gBACP,IAAA,EACA,QAAA,EACA,SACA,MAAA,EACA,QAAA,EACA,SACA,EAAA,EAC0B;AAC1B,EAAA,MAAM,UAAA,GAAa,mBAAmB,OAAO,CAAA;AAE7C,EAAA,IAAI,WAAW,UAAA,EAAY;AACzB,IAAA,OAAO,UAAA;AAAA,EACT;AAEA,EAAA,MAAM,OAAA,GAAU,eAAA;AAAA,IACd,iBAAA,CAAkB,UAAA,CAAW,SAAA,EAAW,MAAM,CAAA;AAAA,IAC9C;AAAA,GACF;AAEA,EAAA,OAAO;AAAA,IACL,KAAK,IAAA,CAAK,GAAA;AAAA,IACV,QAAA;AAAA,IACA,KAAA,EAAO,EAAA;AAAA,IACP,MAAA,EAAQ,EAAA;AAAA,IACR,SAAA,EAAW,EAAA;AAAA,IACX,IAAA,EAAM,IAAI,GAAA,CAAI,QAAQ,CAAA,CAAE,QAAA;AAAA,IACxB,QAAA,EAAU,EAAA;AAAA,IACV,SAAA,EAAW,iBAAA,CAAkB,UAAA,CAAW,SAAS,CAAA;AAAA,IACjD,OAAA;AAAA,IACA,OAAA;AAAA,IACA;AAAA,GACF;AACF;AAEA,SAAS,mBAAmB,IAAA,EAAuB;AACjD,EAAA,OACE,IAAA,KAAS,iBACT,IAAA,CAAK,UAAA,CAAW,IAAI,CAAA,IACpB,IAAA,CAAK,WAAW,WAAW,CAAA;AAE/B;AAMO,SAAS,mBAAA,CACd,eAAkC,mBAAA,EAClC;AACA,EAAA,OAAO,eAAeC,cAAAA,CACpB,IAAA,EACA,KAAA,GAA6B,EAAC,EACK;AACnC,IAAA,MAAM,OAAA,GAAU,KAAK,OAAA,IAAW,eAAA;AAChC,IAAA,MAAM,EAAA,GAAK,KAAK,EAAA,IAAM,UAAA;AACtB,IAAA,MAAM,MAAA,GAAuB,KAAK,MAAA,IAAU,UAAA;AAC5C,IAAA,MAAM,QAAA,GAAW,KAAK,QAAA,IAAY,iBAAA;AAClC,IAAA,MAAM,YAAA,GAAe,KAAK,YAAA,IAAgB,KAAA;AAC1C,IAAA,MAAM,cAAA,GAAiB,KAAK,cAAA,IAAkB,uBAAA;AAC9C,IAAA,MAAM,SAAA,GAAY,KAAK,SAAA,IAAa,kBAAA;AAEpC,IAAA,IAAI,MAAA;AACJ,IAAA,IAAI;AACF,MAAA,MAAA,GAAS,IAAI,GAAA,CAAI,IAAA,CAAK,GAAG,CAAA;AAAA,IAC3B,CAAA,CAAA,MAAQ;AACN,MAAA,OAAO,EAAE,KAAA,EAAO,CAAA,aAAA,EAAgB,IAAA,CAAK,GAAG,CAAA,CAAA,EAAG;AAAA,IAC7C;AAEA,IAAA,IAAI,CAAC,CAAC,OAAA,EAAS,QAAQ,EAAE,QAAA,CAAS,MAAA,CAAO,QAAQ,CAAA,EAAG;AAClD,MAAA,OAAO;AAAA,QACL,KAAA,EAAO,CAAA,oCAAA,EAAuC,MAAA,CAAO,QAAQ,CAAA;AAAA,OAC/D;AAAA,IACF;AAEA,IAAA,MAAM,YAAA,GAAwC;AAAA,MAC5C,OAAA;AAAA,MACA,EAAA;AAAA,MACA,OAAA,EAAS;AAAA,QACP,MAAA,EAAQ,oBAAoB,MAAM,CAAA;AAAA,QAClC,iBAAA,EAAmB,8BAAA;AAAA,QACnB,GAAG,IAAA,CAAK;AAAA,OACV;AAAA,MACA,QAAA,EAAU,QAAA;AAAA,MACV,OAAA,EAAS;AAAA,KACX;AAEA,IAAA,IAAI,KAAK,KAAA,EAAO;AACd,MAAA,YAAA,CAAa,QAAQ,IAAA,CAAK,KAAA;AAAA,IAC5B;AAEA,IAAA,KAAA,CAAM,iBAAiB,UAAU,CAAA;AACjC,IAAA,MAAM,WAAW,MAAM,YAAA,CAAa,KAAA,CAAM,IAAA,CAAK,KAAK,YAAY,CAAA;AAEhE,IAAA,IAAI,CAAC,SAAS,EAAA,EAAI;AAChB,MAAA,OAAO;AAAA,QACL,KAAA,EAAO,QAAQ,QAAA,CAAS,MAAM,IAAI,QAAA,CAAS,UAAU,CAAA,KAAA,EAAQ,IAAA,CAAK,GAAG,CAAA;AAAA,OACvE;AAAA,IACF;AAEA,IAAA,MAAM,QAAA,GAAW,QAAA,CAAS,GAAA,IAAO,IAAA,CAAK,GAAA;AACtC,IAAA,MAAM,WAAA,GAAc,QAAA,CAAS,OAAA,CAAQ,GAAA,CAAI,cAAc,CAAA,IAAK,EAAA;AAC5D,IAAA,MAAM,OAAA,GAAU,MAAM,QAAA,CAAS,IAAA,EAAK;AACpC,IAAA,MAAM,YAAA,GAAe,cAAA,CAAe,WAAA,EAAa,OAAO,CAAA;AAExD,IAAA,IAAI,WAAW,MAAA,EAAQ;AACrB,MAAA,IAAI,CAAC,YAAA,EAAc;AACjB,QAAA,OAAO,EAAE,KAAA,EAAO,CAAA,mCAAA,EAAsC,WAAW,CAAA,CAAA,CAAA,EAAI;AAAA,MACvE;AAEA,MAAA,MAAMC,OAAAA,GAAS,eAAA;AAAA,QACb,IAAA;AAAA,QACA,QAAA;AAAA,QACA,OAAA;AAAA,QACA,MAAA;AAAA,QACA,QAAA;AAAA,QACA,OAAA;AAAA,QACA;AAAA,OACF;AACA,MAAA,IAAI,CAAC,OAAA,CAAQA,OAAM,CAAA,EAAG;AACpB,QAAA,KAAA,CAAM,iBAAiB,MAAM,CAAA;AAAA,MAC/B;AACA,MAAA,OAAOA,OAAAA;AAAA,IACT;AAEA,IAAA,IAAI,YAAA,EAAc;AAChB,MAAA,MAAMA,OAAAA,GAAS,eAAA;AAAA,QACb,IAAA;AAAA,QACA,QAAA;AAAA,QACA,OAAA;AAAA,QACA,MAAA;AAAA,QACA,QAAA;AAAA,QACA,OAAA;AAAA,QACA;AAAA,OACF;AACA,MAAA,IAAI,CAAC,OAAA,CAAQA,OAAM,CAAA,EAAG;AACpB,QAAA,KAAA,CAAM,iBAAiB,MAAM,CAAA;AAAA,MAC/B;AACA,MAAA,OAAOA,OAAAA;AAAA,IACT;AAEA,IAAA,IAAI,CAAC,mBAAmB,IAAA,CAAK,CAAC,UAAU,WAAA,CAAY,QAAA,CAAS,KAAK,CAAC,CAAA,EAAG;AACpE,MAAA,OAAO,EAAE,KAAA,EAAO,CAAA,gCAAA,EAAmC,WAAW,CAAA,CAAA,CAAA,EAAI;AAAA,IACpE;AAEA,IAAA,KAAA,CAAM,iBAAiB,YAAY,CAAA;AACnC,IAAA,MAAM,QAAA,GAAW,iBAAA,CAAkB,OAAA,EAAS,QAAQ,CAAA;AACpD,IAAA,MAAM,SAAA,GAAY,MAAM,YAAA,CAAa,QAAA,CAAS,UAAU,QAAA,EAAU;AAAA,MAChE,UAAU,MAAA,KAAW,MAAA;AAAA,MACrB,YAAA;AAAA,MACA;AAAA,KACD,CAAA;AAED,IAAA,IAAI,CAAC,SAAA,CAAU,OAAA,IAAW,SAAA,CAAU,cAAc,CAAA,EAAG;AACnD,MAAA,OAAO;AAAA,QACL,KAAA,EAAO,CAAA,0BAAA,EAA6B,IAAA,CAAK,GAAG,CAAA,sCAAA;AAAA,OAC9C;AAAA,IACF;AAEA,IAAA,IAAI,mBAAmB,SAAA,CAAU,OAAA;AACjC,IAAA,IAAI,YAAY,SAAA,CAAU,SAAA;AAE1B,IAAA,IAAI,mBAAmB,KAAA,IAAS,kBAAA,CAAmB,SAAA,CAAU,IAAA,IAAQ,EAAE,CAAA,EAAG;AACxE,MAAA,MAAM,eAAA,GAAkB,sBAAA,CAAuB,gBAAA,EAAkB,MAAM,CAAA;AACvE,MAAA,IAAI,oBAAoB,gBAAA,EAAkB;AACxC,QAAA,gBAAA,GAAmB,eAAA;AACnB,QAAA,SAAA,GAAY,iBAAA;AAAA,UACV,MAAA,KAAW,MAAA,GACP,cAAA,CAAe,gBAAgB,CAAA,GAC/B;AAAA,SACN;AAAA,MACF;AAAA,IACF;AAEA,IAAA,MAAM,iBAAA,GACJ,MAAA,KAAW,MAAA,GAAS,cAAA,CAAe,gBAAgB,CAAA,GAAI,gBAAA;AAEzD,IAAA,MAAM,MAAA,GAAS;AAAA,MACb,KAAK,IAAA,CAAK,GAAA;AAAA,MACV,QAAA;AAAA,MACA,KAAA,EAAO,UAAU,KAAA,IAAS,EAAA;AAAA,MAC1B,MAAA,EAAQ,UAAU,MAAA,IAAU,EAAA;AAAA,MAC5B,SAAA,EAAW,UAAU,SAAA,IAAa,EAAA;AAAA,MAClC,IAAA,EAAM,UAAU,IAAA,IAAQ,EAAA;AAAA,MACxB,QAAA,EAAU,UAAU,QAAA,IAAY,EAAA;AAAA,MAChC,SAAA;AAAA,MACA,OAAA,EAAS,eAAA,CAAgB,iBAAA,EAAmB,QAAQ,CAAA;AAAA,MACpD,OAAA;AAAA,MACA;AAAA,KACF;AAEA,IAAA,KAAA,CAAM,iBAAiB,MAAM,CAAA;AAC7B,IAAA,OAAO,MAAA;AAAA,EACT,CAAA;AACF;AAEO,IAAM,gBAAgB,mBAAA,EAAoB;AAG1C,SAAS,QACd,MAAA,EACsB;AACtB,EAAA,OAAO,OAAA,IAAW,MAAA;AACpB;AClQA,SAAS,wBAAwB,KAAA,EAAmC;AAClE,EAAA,IAAI,CAAC,KAAA,IAAS,CAAC,MAAA,CAAO,QAAA,CAAS,KAAK,CAAA,EAAG;AACrC,IAAA,OAAO,yBAAA;AAAA,EACT;AAEA,EAAA,OAAO,KAAK,GAAA,CAAI,CAAA,EAAG,IAAA,CAAK,KAAA,CAAM,KAAK,CAAC,CAAA;AACtC;AAEO,SAAS,wBAAA,CACd,MAAA,GAA0B,EAAC,EACR;AACnB,EAAA,OAAO;AAAA,IACL,QAAA,EAAU,OAAO,QAAA,IAAY,iBAAA;AAAA,IAC7B,SAAA,EAAW,OAAO,SAAA,IAAa,kBAAA;AAAA,IAC/B,OAAA,EAAS,OAAO,OAAA,IAAW,eAAA;AAAA,IAC3B,EAAA,EAAI,OAAO,EAAA,IAAM,UAAA;AAAA,IACjB,YAAA,EAAc,OAAO,YAAA,IAAgB,KAAA;AAAA,IACrC,cAAA,EAAgB,OAAO,cAAA,IAAkB,uBAAA;AAAA,IACzC,gBAAA,EAAkB,uBAAA,CAAwB,MAAA,CAAO,gBAAgB;AAAA,GACnE;AACF;AAEO,SAAS,uCACd,QAAA,EACyB;AACzB,EAAA,OAAO;AAAA,IACL,KAAK,IAAA,CAAK,MAAA,CAAO,EAAE,WAAA,EAAa,kCAAkC,CAAA;AAAA,IAClE,SAAS,IAAA,CAAK,QAAA;AAAA,MACZ,KAAK,MAAA,CAAO;AAAA,QACV,WAAA,EAAa,CAAA,kDAAA,EAAqD,QAAA,CAAS,OAAO,CAAA,oEAAA;AAAA,OACnF;AAAA,KACH;AAAA,IACA,IAAI,IAAA,CAAK,QAAA;AAAA,MACP,KAAK,MAAA,CAAO;AAAA,QACV,WAAA,EAAa,CAAA,yCAAA,EAA4C,QAAA,CAAS,EAAE,CAAA,+CAAA;AAAA,OACrE;AAAA,KACH;AAAA,IACA,SAAS,IAAA,CAAK,QAAA;AAAA,MACZ,KAAK,MAAA,CAAO,IAAA,CAAK,QAAO,EAAG,IAAA,CAAK,QAAO,EAAG;AAAA,QACxC,WAAA,EACE;AAAA,OACH;AAAA,KACH;AAAA,IACA,UAAU,IAAA,CAAK,QAAA;AAAA,MACb,KAAK,MAAA,CAAO;AAAA,QACV,WAAA,EAAa,CAAA,uCAAA,EAA0C,QAAA,CAAS,QAAQ,CAAA;AAAA,OACzE;AAAA,KACH;AAAA,IACA,QAAQ,IAAA,CAAK,QAAA;AAAA,MACX,IAAA,CAAK,KAAA;AAAA,QACH;AAAA,UACE,IAAA,CAAK,QAAQ,UAAU,CAAA;AAAA,UACvB,IAAA,CAAK,QAAQ,MAAM,CAAA;AAAA,UACnB,IAAA,CAAK,QAAQ,MAAM,CAAA;AAAA,UACnB,IAAA,CAAK,QAAQ,MAAM;AAAA,SACrB;AAAA,QACA;AAAA,UACE,WAAA,EACE;AAAA;AACJ;AACF,KACF;AAAA,IACA,cAAc,IAAA,CAAK,QAAA;AAAA,MACjB,KAAK,OAAA,CAAQ;AAAA,QACX,WAAA,EAAa;AAAA,OACd;AAAA,KACH;AAAA,IACA,gBAAgB,IAAA,CAAK,QAAA;AAAA,MACnB,IAAA,CAAK,KAAA,CAAM,CAAC,IAAA,CAAK,OAAA,IAAW,IAAA,CAAK,OAAA,CAAQ,YAAY,CAAC,CAAA,EAAG;AAAA,QACvD,WAAA,EACE;AAAA,OACH;AAAA,KACH;AAAA,IACA,OAAO,IAAA,CAAK,QAAA;AAAA,MACV,KAAK,MAAA,CAAO;AAAA,QACV,WAAA,EACE;AAAA,OACH;AAAA;AACH,GACF;AACF;AAEO,SAAS,wCACd,QAAA,EACyB;AACzB,EAAA,OAAO;AAAA,IACL,UAAU,IAAA,CAAK,KAAA;AAAA,MACb,IAAA,CAAK,MAAA,CAAO,sCAAA,CAAuC,QAAQ,CAAA,EAAG;AAAA,QAC5D,oBAAA,EAAsB;AAAA,OACvB,CAAA;AAAA,MACD;AAAA,QACE,QAAA,EAAU,CAAA;AAAA,QACV,WAAA,EACE;AAAA;AACJ;AACF,GACF;AACF;AAEA,SAAS,2BAAA,CACP,QACA,QAAA,EACc;AACd,EAAA,OAAO;AAAA,IACL,KAAK,MAAA,CAAO,GAAA;AAAA,IACZ,OAAA,EAAU,MAAA,CAAO,OAAA,IAAsB,QAAA,CAAS,OAAA;AAAA,IAChD,EAAA,EAAK,MAAA,CAAO,EAAA,IAAiB,QAAA,CAAS,EAAA;AAAA,IACtC,SAAS,MAAA,CAAO,OAAA;AAAA,IAChB,QAAA,EAAW,MAAA,CAAO,QAAA,IAAuB,QAAA,CAAS,QAAA;AAAA,IAClD,MAAA,EACG,OAAO,MAAA,IAAoD,UAAA;AAAA,IAC9D,YAAA,EAAe,MAAA,CAAO,YAAA,IAA4B,QAAA,CAAS,YAAA;AAAA,IAC3D,cAAA,EACG,MAAA,CAAO,cAAA,IACR,QAAA,CAAS,cAAA;AAAA,IACX,OAAO,MAAA,CAAO,KAAA;AAAA,IACd,WAAW,QAAA,CAAS;AAAA,GACtB;AACF;AAEA,eAAsB,oBAAA,CACpB,MAAA,EACA,QAAA,EACA,KAAA,GAA6B,EAAC,EACK;AACnC,EAAA,OAAO,aAAA,CAAc,2BAAA,CAA4B,MAAA,EAAQ,QAAQ,GAAG,KAAK,CAAA;AAC3E;AAEA,IAAM,kBAAA,GAA2D;AAAA,EAC/D,MAAA,EAAQ,CAAA;AAAA,EACR,QAAA,EAAU,IAAA;AAAA,EACV,UAAA,EAAY,IAAA;AAAA,EACZ,IAAA,EAAM,CAAA;AAAA,EACN,KAAA,EAAO;AACT,CAAA;AAEA,SAAS,2BACP,QAAA,EAC0B;AAC1B,EAAA,OAAO,QAAA,CAAS,GAAA,CAAI,CAAC,OAAA,EAAS,KAAA,MAAW;AAAA,IACvC,KAAA;AAAA,IACA,GAAA,EACE,OAAO,OAAA,CAAQ,GAAA,KAAQ,QAAA,GAAW,QAAQ,GAAA,GAAM,MAAA,CAAO,OAAA,CAAQ,GAAA,IAAO,EAAE,CAAA;AAAA,IAC1E,MAAA,EAAQ,QAAA;AAAA,IACR,UAAU,kBAAA,CAAmB;AAAA,GAC/B,CAAE,CAAA;AACJ;AAEA,SAAS,qBAAA,CACP,OACA,gBAAA,EAC4B;AAC5B,EAAA,IAAI,SAAA,GAAY,CAAA;AAChB,EAAA,IAAI,SAAA,GAAY,CAAA;AAChB,EAAA,IAAI,MAAA,GAAS,CAAA;AAEb,EAAA,KAAA,MAAW,QAAQ,KAAA,EAAO;AACxB,IAAA,IAAI,IAAA,CAAK,MAAA,KAAW,MAAA,IAAU,IAAA,CAAK,WAAW,OAAA,EAAS;AACrD,MAAA,SAAA,IAAa,CAAA;AAAA,IACf;AACA,IAAA,IAAI,IAAA,CAAK,WAAW,MAAA,EAAQ;AAC1B,MAAA,SAAA,IAAa,CAAA;AAAA,IACf;AACA,IAAA,IAAI,IAAA,CAAK,WAAW,OAAA,EAAS;AAC3B,MAAA,MAAA,IAAU,CAAA;AAAA,IACZ;AAAA,EACF;AAEA,EAAA,OAAO;AAAA,IACL,KAAA,EAAO,MAAM,GAAA,CAAI,CAAC,UAAU,EAAE,GAAG,MAAK,CAAE,CAAA;AAAA,IACxC,OAAO,KAAA,CAAM,MAAA;AAAA,IACb,SAAA;AAAA,IACA,SAAA;AAAA,IACA,MAAA;AAAA,IACA;AAAA,GACF;AACF;AAEA,eAAsB,yBAAA,CACpB,MAAA,EACA,QAAA,EACA,OAAA,GAQI,EAAC,EACsB;AAC3B,EAAA,MAAM,QAAA,GAAA,CACH,MAAA,CAAO,QAAA,IAAsD,EAAC,EAC/D,IAAI,CAAC,OAAA,KAAY,OAAA,IAAW,EAAE,CAAA;AAChC,EAAA,MAAM,gBAAA,GAAmB,uBAAA;AAAA,IACvB,OAAA,CAAQ,oBAAoB,QAAA,CAAS;AAAA,GACvC;AACA,EAAA,MAAM,aAAA,GAAgB,2BAA2B,QAAQ,CAAA;AACzD,EAAA,MAAM,OAAA,GAAU,IAAI,KAAA,CAA4B,QAAA,CAAS,MAAM,CAAA;AAE/D,EAAA,MAAM,eAAe,MAAM;AACzB,IAAA,OAAA,CAAQ,UAAA;AAAA,MACN,qBAAA,CAAsB,eAAe,gBAAgB;AAAA,KACvD;AAAA,EACF,CAAA;AAEA,EAAA,MAAM,cAAA,GAAiB,CACrB,KAAA,EACA,MAAA,EACA,KAAA,KACG;AACH,IAAA,aAAA,CAAc,KAAK,CAAA,GAAI;AAAA,MACrB,GAAG,cAAc,KAAK,CAAA;AAAA,MACtB,MAAA;AAAA,MACA,QAAA,EAAU,mBAAmB,MAAM,CAAA;AAAA,MACnC,GAAI,KAAA,GAAQ,EAAE,KAAA,KAAU;AAAC,KAC3B;AACA,IAAA,YAAA,EAAa;AAAA,EACf,CAAA;AAEA,EAAA,YAAA,EAAa;AAEb,EAAA,IAAI,SAAA,GAAY,CAAA;AAEhB,EAAA,MAAM,SAAS,YAAY;AACzB,IAAA,OAAO,IAAA,EAAM;AACX,MAAA,MAAM,KAAA,GAAQ,SAAA;AACd,MAAA,SAAA,IAAa,CAAA;AAEb,MAAA,IAAI,KAAA,IAAS,SAAS,MAAA,EAAQ;AAC5B,QAAA;AAAA,MACF;AAEA,MAAA,MAAM,OAAA,GAAU,QAAA,CAAS,KAAK,CAAA,IAAK,EAAC;AACpC,MAAA,MAAM,iBAAA,GAAoB,2BAAA,CAA4B,OAAA,EAAS,QAAQ,CAAA;AAEvE,MAAA,IAAI;AACF,QAAA,MAAM,WAAA,GAAc,QAAQ,WAAA,IAAe,oBAAA;AAC3C,QAAA,MAAM,MAAA,GAAS,MAAM,WAAA,CAAY,OAAA,EAAS,QAAA,EAAU;AAAA,UAClD,eAAe,MAAA,EAAQ;AACrB,YAAA,IAAI,WAAW,MAAA,EAAQ;AACvB,YAAA,cAAA,CAAe,OAAO,MAAM,CAAA;AAAA,UAC9B;AAAA,SACD,CAAA;AAED,QAAA,IAAI,OAAA,CAAQ,MAAM,CAAA,EAAG;AACnB,UAAA,OAAA,CAAQ,KAAK,CAAA,GAAI;AAAA,YACf,KAAA;AAAA,YACA,OAAA,EAAS,iBAAA;AAAA,YACT,MAAA,EAAQ,OAAA;AAAA,YACR,UAAU,kBAAA,CAAmB,KAAA;AAAA,YAC7B,OAAO,MAAA,CAAO;AAAA,WAChB;AACA,UAAA,cAAA,CAAe,KAAA,EAAO,OAAA,EAAS,MAAA,CAAO,KAAK,CAAA;AAC3C,UAAA;AAAA,QACF;AAEA,QAAA,OAAA,CAAQ,KAAK,CAAA,GAAI;AAAA,UACf,KAAA;AAAA,UACA,OAAA,EAAS,iBAAA;AAAA,UACT,MAAA,EAAQ,MAAA;AAAA,UACR,UAAU,kBAAA,CAAmB,IAAA;AAAA,UAC7B;AAAA,SACF;AACA,QAAA,cAAA,CAAe,OAAO,MAAM,CAAA;AAAA,MAC9B,SAAS,KAAA,EAAO;AACd,QAAA,MAAM,UAAU,KAAA,YAAiB,KAAA,GAAQ,KAAA,CAAM,OAAA,GAAU,OAAO,KAAK,CAAA;AACrE,QAAA,OAAA,CAAQ,KAAK,CAAA,GAAI;AAAA,UACf,KAAA;AAAA,UACA,OAAA,EAAS,iBAAA;AAAA,UACT,MAAA,EAAQ,OAAA;AAAA,UACR,UAAU,kBAAA,CAAmB,KAAA;AAAA,UAC7B,KAAA,EAAO;AAAA,SACT;AACA,QAAA,cAAA,CAAe,KAAA,EAAO,SAAS,OAAO,CAAA;AAAA,MACxC;AAAA,IACF;AAAA,EACF,CAAA;AAEA,EAAA,MAAM,WAAA,GACJ,SAAS,MAAA,KAAW,CAAA,GAAI,IAAI,IAAA,CAAK,GAAA,CAAI,gBAAA,EAAkB,QAAA,CAAS,MAAM,CAAA;AACxE,EAAA,MAAM,OAAA,CAAQ,GAAA,CAAI,KAAA,CAAM,IAAA,CAAK,EAAE,MAAA,EAAQ,WAAA,EAAY,EAAG,YAAY,MAAA,EAAQ,CAAC,CAAA;AAE3E,EAAA,MAAM,aAAA,GAAgB,qBAAA,CAAsB,aAAA,EAAe,gBAAgB,CAAA;AAE3E,EAAA,OAAO;AAAA,IACL,KAAA,EAAO,OAAA;AAAA,IACP,OAAO,aAAA,CAAc,KAAA;AAAA,IACrB,WAAW,aAAA,CAAc,SAAA;AAAA,IACzB,QAAQ,aAAA,CAAc,MAAA;AAAA,IACtB;AAAA,GACF;AACF;ACnTA,IAAM,eAAA,uBAAsB,GAAA,CAAmB;AAAA,EAC7C,SAAA;AAAA,EACA,OAAA;AAAA,EACA,OAAA;AAAA,EACA,SAAA;AAAA,EACA;AACF,CAAC,CAAA;AAiBD,SAAS,WAAA,CACP,QACA,IAAA,EACqB;AACrB,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,IAAI,OAAO,MAAA,CAAO,GAAG,CAAA,KAAM,SAAA,EAAW;AACpC,MAAA,OAAO,OAAO,GAAG,CAAA;AAAA,IACnB;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AAEA,SAAS,kBAAA,CACP,QACA,IAAA,EACoB;AACpB,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,MAAM,KAAA,GAAQ,OAAO,GAAG,CAAA;AACxB,IAAA,IAAI,OAAO,UAAU,QAAA,IAAY,MAAA,CAAO,SAAS,KAAK,CAAA,IAAK,QAAQ,CAAA,EAAG;AACpE,MAAA,OAAO,KAAA;AAAA,IACT;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AAEA,SAAS,kBAAA,CACP,QACA,IAAA,EACoB;AACpB,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,MAAM,KAAA,GAAQ,OAAO,GAAG,CAAA;AACxB,IAAA,IAAI,OAAO,KAAA,KAAU,QAAA,IAAY,KAAA,CAAM,IAAA,OAAW,EAAA,EAAI;AACpD,MAAA,OAAO,KAAA;AAAA,IACT;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AAEA,SAAS,MAAA,CACP,QACA,IAAA,EAC2B;AAC3B,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,MAAM,KAAA,GAAQ,OAAO,GAAG,CAAA;AACxB,IAAA,IACE,OAAO,KAAA,KAAU,QAAA,IACjB,eAAA,CAAgB,GAAA,CAAI,KAAsB,CAAA,EAC1C;AACA,MAAA,OAAO,KAAA;AAAA,IACT;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AAEA,SAAS,kBAAA,CACP,QACA,IAAA,EACkC;AAClC,EAAA,KAAA,MAAW,OAAO,IAAA,EAAM;AACtB,IAAA,MAAM,KAAA,GAAQ,OAAO,GAAG,CAAA;AACxB,IAAA,IAAI,OAAO,KAAA,KAAU,SAAA,IAAa,KAAA,KAAU,YAAA,EAAc;AACxD,MAAA,OAAO,KAAA;AAAA,IACT;AAAA,EACF;AAEA,EAAA,OAAO,MAAA;AACT;AAEA,SAAS,8BAA8B,KAAA,EAAsC;AAC3E,EAAA,IAAI,CAAC,KAAA,IAAS,OAAO,KAAA,KAAU,QAAA,SAAiB,EAAC;AAEjD,EAAA,MAAM,MAAA,GAAS,KAAA;AAEf,EAAA,OAAO;AAAA,IACL,0BAAA,EAA4B,YAAY,MAAA,EAAQ;AAAA,MAC9C,4BAAA;AAAA,MACA;AAAA,KACD,CAAA;AAAA,IACD,yBAAA,EAA2B,mBAAmB,MAAA,EAAQ;AAAA,MACpD,2BAAA;AAAA,MACA;AAAA,KACD,CAAA;AAAA,IACD,0BAAA,EAA4B,mBAAmB,MAAA,EAAQ;AAAA,MACrD;AAAA,KACD,CAAA;AAAA,IACD,wBAAA,EAA0B,mBAAmB,MAAA,EAAQ;AAAA,MACnD;AAAA,KACD,CAAA;AAAA,IACD,mBAAA,EAAqB,MAAA,CAAO,MAAA,EAAQ,CAAC,qBAAqB,CAAC,CAAA;AAAA,IAC3D,6BAAA,EAA+B,YAAY,MAAA,EAAQ;AAAA,MACjD;AAAA,KACD,CAAA;AAAA,IACD,+BAAA,EAAiC,mBAAmB,MAAA,EAAQ;AAAA,MAC1D;AAAA,KACD,CAAA;AAAA,IACD,iCAAA,EAAmC,mBAAmB,MAAA,EAAQ;AAAA,MAC5D,mCAAA;AAAA,MACA;AAAA,KACD;AAAA,GACH;AACF;AAEO,SAAS,2BAAA,CACd,gBACA,eAAA,EAC8B;AAC9B,EAAA,MAAM,MAAA,GAAS,8BAA8B,cAAc,CAAA;AAC3D,EAAA,MAAM,OAAA,GAAU,8BAA8B,eAAe,CAAA;AAE7D,EAAA,OAAO;AAAA,IACL,gBAAA,EACE,OAAA,CAAQ,0BAAA,IACR,MAAA,CAAO,0BAAA,IACP,KAAA;AAAA,IACF,QAAA,EACE,OAAA,CAAQ,yBAAA,IAA6B,MAAA,CAAO,yBAAA;AAAA,IAC9C,SAAA,EACE,OAAA,CAAQ,0BAAA,IAA8B,MAAA,CAAO,0BAAA;AAAA,IAC/C,OAAA,EACE,OAAA,CAAQ,wBAAA,IAA4B,MAAA,CAAO,wBAAA;AAAA,IAC7C,EAAA,EAAI,OAAA,CAAQ,mBAAA,IAAuB,MAAA,CAAO,mBAAA;AAAA,IAC1C,YAAA,EACE,OAAA,CAAQ,6BAAA,IACR,MAAA,CAAO,6BAAA;AAAA,IACT,cAAA,EACE,OAAA,CAAQ,+BAAA,IACR,MAAA,CAAO,+BAAA;AAAA,IACT,gBAAA,EACE,OAAA,CAAQ,iCAAA,IACR,MAAA,CAAO;AAAA,GACX;AACF;AAEA,eAAe,iBAAiB,IAAA,EAAgC;AAC9D,EAAA,IAAI;AACF,IAAA,OAAO,KAAK,KAAA,CAAM,MAAM,QAAA,CAAS,IAAA,EAAM,OAAO,CAAC,CAAA;AAAA,EACjD,CAAA,CAAA,MAAQ;AACN,IAAA,OAAO,EAAC;AAAA,EACV;AACF;AAEA,eAAsB,wBAAA,CACpB,GAAA,EACA,QAAA,GAAW,WAAA,EAAY,EACgB;AACvC,EAAA,MAAM,iBAAiB,MAAM,gBAAA;AAAA,IAC3B,IAAA,CAAK,UAAU,eAAe;AAAA,GAChC;AACA,EAAA,MAAM,kBAAkB,MAAM,gBAAA;AAAA,IAC5B,IAAA,CAAK,GAAA,EAAK,KAAA,EAAO,eAAe;AAAA,GAClC;AAEA,EAAA,OAAO,2BAAA,CAA4B,gBAAgB,eAAe,CAAA;AACpE;;;AC3KA,IAAM,eAAA,GAAkB;AAAA,EACtB,wFAAA;AAAA,EACA,4FAAA;AAAA,EACA,yFAAA;AAAA,EACA;AACF,CAAA,CAAE,KAAK,GAAG,CAAA;AAEV,IAAM,oBAAA,GAAuB;AAAA,EAC3B,oFAAA;AAAA,EACA,8FAAA;AAAA,EACA,iGAAA;AAAA,EACA;AACF,CAAA,CAAE,KAAK,GAAG,CAAA;AAEV,IAAM,cAAA,GAAiB,CAAC,QAAA,EAAK,QAAA,EAAK,QAAA,EAAK,QAAA,EAAK,QAAA,EAAK,QAAA,EAAK,QAAA,EAAK,QAAA,EAAK,QAAA,EAAK,QAAG,CAAA;AASxE,SAAS,cAAA,CAAe,OAAe,KAAA,EAAuB;AAC5D,EAAA,IAAI,KAAA,IAAS,GAAG,OAAO,EAAA;AACvB,EAAA,IAAI,MAAM,MAAA,IAAU,KAAA,SAAc,KAAA,CAAM,MAAA,CAAO,OAAO,GAAG,CAAA;AACzD,EAAA,IAAI,KAAA,KAAU,GAAG,OAAO,QAAA;AAExB,EAAA,MAAM,IAAA,GAAO,IAAA,CAAK,IAAA,CAAA,CAAM,KAAA,GAAQ,KAAK,CAAC,CAAA;AACtC,EAAA,MAAM,KAAA,GAAQ,IAAA,CAAK,KAAA,CAAA,CAAO,KAAA,GAAQ,KAAK,CAAC,CAAA;AACxC,EAAA,OAAO,CAAA,EAAG,KAAA,CAAM,KAAA,CAAM,CAAA,EAAG,IAAI,CAAC,CAAA,MAAA,EAAI,KAAA,CAAM,KAAA,CAAM,KAAA,CAAM,MAAA,GAAS,KAAK,CAAC,CAAA,CAAA;AACrE;AAEA,SAAS,GAAA,CAAI,OAAe,KAAA,EAAuB;AACjD,EAAA,OAAO,KAAA,CAAM,MAAA,IAAU,KAAA,GACnB,KAAA,CAAM,KAAA,CAAM,CAAA,EAAG,KAAK,CAAA,GACpB,KAAA,CAAM,MAAA,CAAO,KAAA,EAAO,GAAG,CAAA;AAC7B;AAEA,SAAS,iBAAA,CACP,IAAA,EACA,KAAA,EACA,KAAA,EAGQ;AACR,EAAA,MAAM,UAAA,GAAa,IAAA,CAAK,GAAA,CAAI,CAAA,EAAG,QAAQ,CAAC,CAAA;AACxC,EAAA,MAAM,SAAS,IAAA,CAAK,GAAA;AAAA,IAClB,CAAA;AAAA,IACA,IAAA,CAAK,IAAI,UAAA,EAAY,IAAA,CAAK,MAAM,IAAA,CAAK,QAAA,GAAW,UAAU,CAAC;AAAA,GAC7D;AACA,EAAA,MAAM,KAAA,GAAQ,IAAA,CAAK,GAAA,CAAI,CAAA,EAAG,aAAa,MAAM,CAAA;AAC7C,EAAA,MAAM,QAAA,GACJ,KAAK,MAAA,KAAW,OAAA,GACZ,UACA,IAAA,CAAK,MAAA,KAAW,SACd,SAAA,GACA,QAAA;AAER,EAAA,OAAO;AAAA,IACL,KAAA,CAAM,EAAA,CAAG,OAAA,EAAS,GAAG,CAAA;AAAA,IACrB,MAAM,EAAA,CAAG,QAAA,EAAU,QAAA,CAAI,MAAA,CAAO,MAAM,CAAC,CAAA;AAAA,IACrC,MAAM,EAAA,CAAG,KAAA,EAAO,QAAA,CAAI,MAAA,CAAO,KAAK,CAAC,CAAA;AAAA,IACjC,KAAA,CAAM,EAAA,CAAG,OAAA,EAAS,GAAG;AAAA,GACvB,CAAE,KAAK,EAAE,CAAA;AACX;AAEA,SAAS,iBAAA,CACP,IAAA,EACA,YAAA,EACA,KAAA,EAGQ;AACR,EAAA,QAAQ,KAAK,MAAA;AAAQ,IACnB,KAAK,MAAA;AACH,MAAA,OAAO,KAAA,CAAM,EAAA,CAAG,SAAA,EAAW,QAAG,CAAA;AAAA,IAChC,KAAK,OAAA;AACH,MAAA,OAAO,KAAA,CAAM,EAAA,CAAG,OAAA,EAAS,QAAG,CAAA;AAAA,IAC9B,KAAK,QAAA;AACH,MAAA,OAAO,KAAA,CAAM,EAAA,CAAG,OAAA,EAAS,QAAG,CAAA;AAAA,IAC9B;AACE,MAAA,OAAO,KAAA,CAAM,EAAA;AAAA,QACX,QAAA;AAAA,QACA,cAAA,CAAe,YAAA,GAAe,cAAA,CAAe,MAAM,CAAA,IAAK;AAAA,OAC1D;AAAA;AAEN;AAEA,SAAS,uBAAA,CACP,QAAA,EACA,KAAA,EACA,QAAA,EACA,KAAA,EAIQ;AACR,EAAA,MAAM,OAAA,GAAU;AAAA,IACd,MAAM,EAAA,CAAG,WAAA,EAAa,KAAA,CAAM,IAAA,CAAK,kBAAkB,CAAC,CAAA;AAAA,IACpD,KAAA,CAAM,EAAA;AAAA,MACJ,OAAA;AAAA,MACA,CAAA,EAAG,QAAA,CAAS,SAAS,CAAA,CAAA,EAAI,SAAS,KAAK,CAAA,cAAA,EAAc,QAAA,CAAS,SAAS,CAAA,UAAA,EAAU,QAAA,CAAS,MAAM,CAAA,kBAAA,EAAkB,SAAS,gBAAgB,CAAA;AAAA;AAC7I,GACF,CAAE,KAAK,EAAE,CAAA;AAET,EAAA,MAAM,iBAAA,GAAoB,KAAK,GAAA,CAAI,EAAA,EAAI,KAAK,KAAA,CAAM,KAAA,GAAQ,GAAG,CAAC,CAAA;AAC9D,EAAA,MAAM,WAAA,GAAc,EAAA;AACpB,EAAA,MAAM,gBAAgB,IAAA,CAAK,GAAA;AAAA,IACzB,EAAA;AAAA,IACA,KAAK,GAAA,CAAI,EAAA,EAAI,KAAK,KAAA,CAAM,iBAAA,GAAoB,IAAI,CAAC;AAAA,GACnD;AACA,EAAA,MAAM,UAAA,GAAa,CAAA;AACnB,EAAA,MAAM,WAAW,IAAA,CAAK,GAAA;AAAA,IACpB,EAAA;AAAA,IACA,iBAAA,GAAoB,UAAA,GAAa,WAAA,GAAc,aAAA,GAAgB;AAAA,GACjE;AAEA,EAAA,MAAM,OAAO,QAAA,CAAS,KAAA,CAAM,GAAA,CAAI,CAAC,MAAM,KAAA,KAAU;AAC/C,IAAA,MAAM,QAAQ,iBAAA,CAAkB,IAAA,EAAM,QAAA,CAAS,SAAA,GAAY,OAAO,KAAK,CAAA;AACvE,IAAA,MAAM,GAAA,GAAM,MAAM,EAAA,CAAG,QAAA,EAAU,eAAe,IAAA,CAAK,GAAA,EAAK,QAAQ,CAAC,CAAA;AACjE,IAAA,MAAM,WAAA,GACJ,IAAA,CAAK,MAAA,KAAW,OAAA,GACZ,OAAA,GACA,IAAA,CAAK,MAAA,KAAW,MAAA,GACd,SAAA,GACA,IAAA,CAAK,MAAA,KAAW,QAAA,GACd,OAAA,GACA,SAAA;AACV,IAAA,MAAM,MAAA,GAAS,MAAM,EAAA,CAAG,WAAA,EAAa,IAAI,IAAA,CAAK,MAAA,EAAQ,WAAW,CAAC,CAAA;AAClE,IAAA,MAAM,GAAA,GAAM,iBAAA,CAAkB,IAAA,EAAM,aAAA,EAAe,KAAK,CAAA;AAExD,IAAA,MAAM,OAAA,GAAU,GAAG,KAAK,CAAA,CAAA,EAAI,GAAG,CAAA,CAAA,EAAI,MAAM,IAAI,GAAG,CAAA,CAAA;AAChD,IAAA,IAAI,CAAC,QAAA,IAAY,CAAC,IAAA,CAAK,KAAA,EAAO;AAC5B,MAAA,OAAO,OAAA;AAAA,IACT;AAEA,IAAA,OAAO,GAAG,OAAO;AAAA,EAAA,EAAO,MAAM,EAAA,CAAG,OAAA,EAAS,UAAU,IAAA,CAAK,KAAK,EAAE,CAAC,CAAA,CAAA;AAAA,EACnE,CAAC,CAAA;AAED,EAAA,OAAO,CAAC,OAAA,EAAS,GAAG,IAAI,CAAA,CAAE,KAAK,IAAI,CAAA;AACrC;AAEA,SAAS,8BAAA,CACP,OAAA,EACA,QAAA,EACA,KAAA,EAIA;AACA,EAAA,MAAM,IAAA,GAAO,IAAI,IAAA,CAAK,EAAA,EAAI,GAAG,CAAC,CAAA;AAE9B,EAAA,OAAO;AAAA,IACL,OAAO,KAAA,EAAe;AACpB,MAAA,MAAM,WAAW,OAAA,CAAQ,aAAA;AACzB,MAAA,IAAI,CAAC,QAAA,EAAU;AACb,QAAA,IAAA,CAAK,OAAA,CAAQ,KAAA,CAAM,EAAA,CAAG,OAAA,EAAS,8BAA8B,CAAC,CAAA;AAC9D,QAAA,OAAO,IAAA,CAAK,OAAO,KAAK,CAAA;AAAA,MAC1B;AAEA,MAAA,IAAA,CAAK,QAAQ,uBAAA,CAAwB,QAAA,EAAU,KAAA,EAAO,QAAA,EAAU,KAAK,CAAC,CAAA;AACtE,MAAA,OAAO,IAAA,CAAK,OAAO,KAAK,CAAA;AAAA,IAC1B,CAAA;AAAA,IACA,UAAA,GAAa;AACX,MAAA,IAAA,CAAK,UAAA,EAAW;AAAA,IAClB;AAAA,GACF;AACF;AAEe,SAAR,sBAAuC,EAAA,EAAkB;AAC9D,EAAA,MAAM,WAAW,wBAAA,EAAyB;AAE1C,EAAA,EAAA,CAAG,YAAA,CAAa;AAAA,IACd,IAAA,EAAM,WAAA;AAAA,IACN,KAAA,EAAO,WAAA;AAAA,IACP,WAAA,EAAa,eAAA;AAAA,IACb,aAAA,EACE,iKAAA;AAAA,IACF,UAAA,EAAYC,KAAK,MAAA,CAAO;AAAA,MACtB,GAAG,uCAAuC,QAAQ,CAAA;AAAA,MAClD,SAASA,IAAAA,CAAK,QAAA;AAAA,QACZA,KAAK,OAAA,CAAQ;AAAA,UACX,WAAA,EACE;AAAA,SACH;AAAA;AACH,KACD,CAAA;AAAA,IAED,MAAM,OAAA,CAAQ,WAAA,EAAa,MAAA,EAAQ,OAAA,EAAS,WAAW,GAAA,EAAK;AAC1D,MAAA,MAAM,WAAW,MAAM,wBAAA,CAAyB,GAAA,CAAI,GAAA,EAAKC,aAAa,CAAA;AACtE,MAAA,MAAM,eAAA,GAAkB,yBAAyB,QAAQ,CAAA;AACzD,MAAA,MAAM,OAAA,GACH,MAAA,CAAO,OAAA,IAAmC,QAAA,CAAS,gBAAA;AACtD,MAAA,MAAM,MAAA,GAAS,MAAM,oBAAA,CAAqB,MAAA,EAAQ,eAAe,CAAA;AAEjE,MAAA,IAAI,OAAA,CAAQ,MAAM,CAAA,EAAG;AACnB,QAAA,OAAO;AAAA,UACL,OAAA,EAAS,CAAC,EAAE,IAAA,EAAM,MAAA,EAAQ,MAAM,CAAA,OAAA,EAAU,MAAA,CAAO,KAAK,CAAA,CAAA,EAAI,CAAA;AAAA,UAC1D,OAAA,EAAS,EAAE,KAAA,EAAO,IAAA,EAAM,OAAA;AAAQ,SAClC;AAAA,MACF;AAEA,MAAA,OAAO;AAAA,QACL,OAAA,EAAS;AAAA,UACP,EAAE,MAAM,MAAA,EAAQ,IAAA,EAAM,uBAAuB,MAAA,EAAQ,EAAE,OAAA,EAAS,CAAA;AAAE,SACpE;AAAA,QACA,OAAA,EAAS,EAAE,OAAA,EAAS,QAAA,EAAU,gBAAgB,QAAA;AAAS,OACzD;AAAA,IACF;AAAA,GACD,CAAA;AAED,EAAA,EAAA,CAAG,YAAA,CAAa;AAAA,IACd,IAAA,EAAM,iBAAA;AAAA,IACN,KAAA,EAAO,iBAAA;AAAA,IACP,WAAA,EAAa,oBAAA;AAAA,IACb,aAAA,EACE,4GAAA;AAAA,IACF,UAAA,EAAYD,KAAK,MAAA,CAAO;AAAA,MACtB,GAAG,wCAAwC,QAAQ,CAAA;AAAA,MACnD,SAASA,IAAAA,CAAK,QAAA;AAAA,QACZA,KAAK,OAAA,CAAQ;AAAA,UACX,WAAA,EACE;AAAA,SACH;AAAA;AACH,KACD,CAAA;AAAA,IAED,MAAM,OAAA,CAAQ,WAAA,EAAa,MAAA,EAAQ,OAAA,EAAS,UAAU,GAAA,EAAK;AACzD,MAAA,MAAM,WAAW,MAAM,wBAAA,CAAyB,GAAA,CAAI,GAAA,EAAKC,aAAa,CAAA;AACtE,MAAA,MAAM,eAAA,GAAkB,yBAAyB,QAAQ,CAAA;AACzD,MAAA,MAAM,OAAA,GACH,MAAA,CAAO,OAAA,IAAmC,QAAA,CAAS,gBAAA;AAEtD,MAAA,MAAM,cAAc,MAAM,yBAAA;AAAA,QACxB,MAAA;AAAA,QACA,eAAA;AAAA,QACA;AAAA,UACE,kBAAkB,eAAA,CAAgB,gBAAA;AAAA,UAClC,WAAW,QAAA,EAAU;AACnB,YAAA,QAAA,GAAW;AAAA,cACT,OAAA,EAAS;AAAA,gBACP;AAAA,kBACE,IAAA,EAAM,MAAA;AAAA,kBACN,IAAA,EAAM,2BAA2B,QAAA,CAAS,KAAK,UAAU,QAAA,CAAS,SAAS,CAAA,CAAA,EAAI,QAAA,CAAS,KAAK,CAAA,WAAA;AAAA;AAC/F,eACF;AAAA,cACA,OAAA,EAAS;AAAA,gBACP,OAAA;AAAA,gBACA,OAAA,EAAS,IAAA;AAAA,gBACT,aAAA,EAAe;AAAA;AACjB,aACD,CAAA;AAAA,UACH;AAAA;AACF,OACF;AAEA,MAAA,MAAM,aAAA,GAA4C;AAAA,QAChD,KAAA,EAAO,WAAA,CAAY,KAAA,CAAM,GAAA,CAAI,CAAC,IAAA,MAAU;AAAA,UACtC,OAAO,IAAA,CAAK,KAAA;AAAA,UACZ,GAAA,EAAK,KAAK,OAAA,CAAQ,GAAA;AAAA,UAClB,QAAQ,IAAA,CAAK,MAAA;AAAA,UACb,UAAU,IAAA,CAAK,QAAA;AAAA,UACf,GAAI,KAAK,KAAA,GAAQ,EAAE,OAAO,IAAA,CAAK,KAAA,KAAU;AAAC,SAC5C,CAAE,CAAA;AAAA,QACF,OAAO,WAAA,CAAY,KAAA;AAAA,QACnB,WAAW,WAAA,CAAY,KAAA;AAAA,QACvB,WAAW,WAAA,CAAY,SAAA;AAAA,QACvB,QAAQ,WAAA,CAAY,MAAA;AAAA,QACpB,kBAAkB,WAAA,CAAY;AAAA,OAChC;AAEA,MAAA,OAAO;AAAA,QACL,OAAA,EAAS;AAAA,UACP;AAAA,YACE,IAAA,EAAM,MAAA;AAAA,YACN,IAAA,EAAM,2BAAA,CAA4B,WAAA,EAAa,EAAE,SAAS;AAAA;AAC5D,SACF;AAAA,QACA,OAAA,EAAS;AAAA,UACP,OAAA;AAAA,UACA,OAAA,EAAS,IAAA;AAAA,UACT,aAAA,EAAe,aAAA;AAAA,UACf;AAAA;AACF,OACF;AAAA,IACF,CAAA;AAAA,IAEA,UAAA,CAAW,MAAM,KAAA,EAAO;AACtB,MAAA,MAAM,SAAA,GAAY,IAAA;AAClB,MAAA,MAAM,YAAA,GAAe,MAAM,OAAA,CAAQ,SAAA,CAAU,QAAQ,CAAA,GACjD,SAAA,CAAU,SAAS,MAAA,GACnB,CAAA;AACJ,MAAA,IAAI,OAAO,KAAA,CAAM,EAAA,CAAG,aAAa,KAAA,CAAM,IAAA,CAAK,kBAAkB,CAAC,CAAA;AAC/D,MAAA,IAAA,IAAQ,KAAA,CAAM,EAAA,CAAG,OAAA,EAAS,CAAA,EAAG,YAAY,CAAA,KAAA,CAAO,CAAA;AAChD,MAAA,OAAO,IAAI,IAAA,CAAK,IAAA,EAAM,CAAA,EAAG,CAAC,CAAA;AAAA,IAC5B,CAAA;AAAA,IAEA,YAAA,CAAa,MAAA,EAAQ,EAAE,QAAA,IAAY,KAAA,EAAO;AACxC,MAAA,OAAO,8BAAA;AAAA,QACJ,MAAA,CAAO,WAA8C,EAAC;AAAA,QACvD,QAAA;AAAA,QACA;AAAA,OACF;AAAA,IACF;AAAA,GACD,CAAA;AACH","file":"index.js","sourcesContent":["import type { FingerprintOs } from \"./types\";\n\nexport const DEFAULT_BROWSER = \"chrome_145\";\nexport const DEFAULT_OS: FingerprintOs = \"windows\";\nexport const DEFAULT_MAX_CHARS = 50_000;\nexport const DEFAULT_TIMEOUT_MS = 15_000;\nexport const DEFAULT_BATCH_CONCURRENCY = 8;\nexport const DEFAULT_INCLUDE_REPLIES = \"extractors\" as const;\nexport const DEFAULT_ACCEPT_HEADER =\n \"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\";\nexport const DEFAULT_JSON_ACCEPT_HEADER =\n \"application/json,text/json,application/ld+json;q=0.9,text/plain;q=0.8,*/*;q=0.7\";\nexport const DEFAULT_ACCEPT_LANGUAGE_HEADER = \"en-US,en;q=0.9\";\n","import { Defuddle } from \"defuddle/node\";\nimport { getProfiles, fetch as wreqFetch } from \"wreq-js\";\nimport type { FetchDependencies } from \"./types\";\n\nexport const runtimeDependencies: FetchDependencies = {\n fetch: wreqFetch,\n defuddle: Defuddle,\n getProfiles,\n};\n","import { parseHTML } from \"linkedom\";\n\n/** Apply linkedom polyfills that Defuddle expects (getComputedStyle, styleSheets). */\nexport function parseLinkedomHTML(html: string, url?: string): Document {\n const { document } = parseHTML(html);\n const doc = document as Document & Record<string, unknown>;\n const defaultView = doc.defaultView as\n | (Window & {\n getComputedStyle?: (\n elt: Element,\n pseudoElt?: string | null,\n ) => CSSStyleDeclaration;\n })\n | undefined;\n\n if (!(doc as { styleSheets?: unknown }).styleSheets) {\n (doc as { styleSheets?: unknown }).styleSheets =\n [] as unknown as StyleSheetList;\n }\n\n if (defaultView && !defaultView.getComputedStyle) {\n defaultView.getComputedStyle = (() => ({\n display: \"\",\n })) as unknown as typeof defaultView.getComputedStyle;\n }\n\n if (url) {\n (doc as { URL?: string }).URL = url;\n }\n\n return document;\n}\n","import type {\n BatchFetchItemResult,\n BatchFetchResult,\n FetchError,\n FetchResult,\n OutputFormat,\n} from \"./types\";\n\nfunction buildHeader(\n parts: Array<[label: string, value: string | number | undefined]>,\n) {\n return parts\n .filter(([, value]) => value !== undefined && value !== \"\")\n .map(([label, value]) => `> ${label}: ${value}`)\n .join(\"\\n\");\n}\n\nexport function markdownToText(markdown: string): string {\n return markdown\n .replace(/^#{1,6}\\s+/gm, \"\")\n .replace(/\\*\\*([^*]+)\\*\\*/g, \"$1\")\n .replace(/\\*([^*]+)\\*/g, \"$1\")\n .replace(/\\[([^\\]]+)\\]\\([^)]+\\)/g, \"$1\")\n .replace(/!\\[[^\\]]*\\]\\([^)]+\\)/g, \"\")\n .replace(/^>\\s+/gm, \"\")\n .replace(/^[-*+]\\s+/gm, \"• \")\n .replace(/`([^`]+)`/g, \"$1\");\n}\n\nexport function truncateContent(content: string, maxChars: number): string {\n if (content.length <= maxChars) return content;\n return `${content.slice(0, maxChars)}\\n\\n[... truncated]`;\n}\n\nexport function buildCompactMetadataHeader(result: FetchResult): string {\n return buildHeader([\n [\"URL\", result.finalUrl],\n [\"Title\", result.title],\n [\"Author\", result.author],\n [\"Published\", result.published],\n ]);\n}\n\nexport function buildMetadataHeader(result: FetchResult): string {\n return buildHeader([\n [\"URL\", result.finalUrl],\n [\"Title\", result.title],\n [\"Author\", result.author],\n [\"Published\", result.published],\n [\"Site\", result.site],\n [\"Language\", result.language],\n [\"Words\", result.wordCount],\n [\"Browser\", `${result.browser}/${result.os}`],\n ]);\n}\n\nexport function buildFetchResponseText(\n result: FetchResult,\n options: { verbose?: boolean } = {},\n): string {\n const header = options.verbose\n ? buildMetadataHeader(result)\n : buildCompactMetadataHeader(result);\n\n return header ? `${header}\\n\\n${result.content}` : result.content;\n}\n\nfunction buildBatchItemHeading(\n item: BatchFetchItemResult,\n total: number,\n): string {\n const ordinal = item.index + 1;\n const url = item.result?.finalUrl ?? item.request.url;\n return `## [${ordinal}/${total}] ${url}`;\n}\n\nfunction buildBatchItemText(\n item: BatchFetchItemResult,\n total: number,\n options: { verbose?: boolean } = {},\n): string {\n const heading = buildBatchItemHeading(item, total);\n\n if (item.status === \"error\") {\n const errorHeader = buildHeader([\n [\"URL\", item.request.url],\n [\"Status\", \"error\"],\n [\"Error\", item.error ?? \"Unknown error\"],\n ]);\n return `${heading}\\n${errorHeader}`;\n }\n\n return `${heading}\\n${buildFetchResponseText(item.result as FetchResult, options)}`;\n}\n\nexport function buildBatchFetchResponseText(\n result: BatchFetchResult,\n options: { verbose?: boolean } = {},\n): string {\n const summary = buildHeader([\n [\"Requests\", result.total],\n [\"Succeeded\", result.succeeded],\n [\"Failed\", result.failed],\n [\"Concurrency\", result.batchConcurrency],\n ]);\n const items = result.items.map((item) =>\n buildBatchItemText(item, result.total, options),\n );\n\n return [summary, ...items].filter(Boolean).join(\"\\n\\n\");\n}\n\nexport function estimateWordCount(content: string): number {\n const words = content.trim().match(/\\S+/g);\n return words?.length ?? 0;\n}\n\nexport function escapeHtml(value: string): string {\n return value\n .replace(/&/g, \"&\")\n .replace(/</g, \"<\")\n .replace(/>/g, \">\")\n .replace(/\"/g, \""\")\n .replace(/'/g, \"'\");\n}\n\nexport function parseAndFormatJson(\n raw: string,\n): { formatted: string } | FetchError {\n try {\n return {\n formatted: JSON.stringify(JSON.parse(raw), null, 2),\n };\n } catch {\n return { error: \"Invalid JSON response\" };\n }\n}\n\nexport function renderJsonContent(\n formattedJson: string,\n format: OutputFormat,\n): string {\n switch (format) {\n case \"json\":\n case \"text\":\n return formattedJson;\n case \"html\":\n return `<pre><code class=\"language-json\">${escapeHtml(formattedJson)}</code></pre>`;\n default:\n return `\\`\\`\\`json\\n${formattedJson}\\n\\`\\`\\``;\n }\n}\n\nexport function stripExtractorComments(\n content: string,\n format: OutputFormat,\n): string {\n if (format === \"html\") {\n return content\n .replace(/\\s*<hr>\\s*<div class=\"[^\"]* comments\">[\\s\\S]*$/i, \"\")\n .trimEnd();\n }\n\n return content.replace(/\\n---\\n+## Comments\\n[\\s\\S]*$/i, \"\").trimEnd();\n}\n","/**\n * Core extraction pipeline: fetch with TLS fingerprinting → parse → Defuddle extract.\n * Separated from the plugin entry so it can be tested independently.\n */\n\nimport {\n DEFAULT_ACCEPT_HEADER,\n DEFAULT_ACCEPT_LANGUAGE_HEADER,\n DEFAULT_BROWSER,\n DEFAULT_INCLUDE_REPLIES,\n DEFAULT_JSON_ACCEPT_HEADER,\n DEFAULT_MAX_CHARS,\n DEFAULT_OS,\n DEFAULT_TIMEOUT_MS,\n} from \"./constants\";\nimport { runtimeDependencies } from \"./dependencies\";\nimport { parseLinkedomHTML } from \"./dom\";\nimport {\n estimateWordCount,\n markdownToText,\n parseAndFormatJson,\n renderJsonContent,\n stripExtractorComments,\n truncateContent,\n} from \"./format\";\nimport { getLatestChromeProfile as getLatestChromeProfileFrom } from \"./profiles\";\nimport type {\n FetchDependencies,\n FetchError,\n FetchExecutionHooks,\n FetchOptions,\n FetchResult,\n OutputFormat,\n} from \"./types\";\n\nexport {\n DEFAULT_BATCH_CONCURRENCY,\n DEFAULT_BROWSER,\n DEFAULT_INCLUDE_REPLIES,\n DEFAULT_MAX_CHARS,\n DEFAULT_OS,\n DEFAULT_TIMEOUT_MS,\n} from \"./constants\";\nexport type {\n FetchError,\n FetchOptions,\n FetchResult,\n OutputFormat,\n} from \"./types\";\n\nconst HTML_CONTENT_TYPES = [\n \"text/html\",\n \"application/xhtml+xml\",\n \"text/plain\",\n \"text/markdown\",\n];\n\nfunction resolveAcceptHeader(format: OutputFormat): string {\n return format === \"json\" ? DEFAULT_JSON_ACCEPT_HEADER : DEFAULT_ACCEPT_HEADER;\n}\n\nfunction isJsonContentType(contentType: string): boolean {\n const normalized = contentType.split(\";\")[0]?.trim().toLowerCase() ?? \"\";\n return (\n normalized === \"application/json\" ||\n normalized === \"text/json\" ||\n normalized.endsWith(\"+json\")\n );\n}\n\nfunction isLikelyJsonBody(body: string): boolean {\n const trimmed = body.trim();\n return trimmed.startsWith(\"{\") || trimmed.startsWith(\"[\");\n}\n\nfunction isJsonResponse(contentType: string, body: string): boolean {\n return isJsonContentType(contentType) || isLikelyJsonBody(body);\n}\n\nfunction buildJsonResult(\n opts: FetchOptions,\n finalUrl: string,\n rawBody: string,\n format: OutputFormat,\n maxChars: number,\n browser: string,\n os: string,\n): FetchResult | FetchError {\n const parsedJson = parseAndFormatJson(rawBody);\n\n if (\"error\" in parsedJson) {\n return parsedJson;\n }\n\n const content = truncateContent(\n renderJsonContent(parsedJson.formatted, format),\n maxChars,\n );\n\n return {\n url: opts.url,\n finalUrl,\n title: \"\",\n author: \"\",\n published: \"\",\n site: new URL(finalUrl).hostname,\n language: \"\",\n wordCount: estimateWordCount(parsedJson.formatted),\n content,\n browser,\n os,\n };\n}\n\nfunction shouldStripReplies(site: string): boolean {\n return (\n site === \"Hacker News\" ||\n site.startsWith(\"r/\") ||\n site.startsWith(\"GitHub - \")\n );\n}\n\nexport function getLatestChromeProfile(): string {\n return getLatestChromeProfileFrom(runtimeDependencies.getProfiles);\n}\n\nexport function createDefuddleFetch(\n dependencies: FetchDependencies = runtimeDependencies,\n) {\n return async function defuddleFetch(\n opts: FetchOptions,\n hooks: FetchExecutionHooks = {},\n ): Promise<FetchResult | FetchError> {\n const browser = opts.browser ?? DEFAULT_BROWSER;\n const os = opts.os ?? DEFAULT_OS;\n const format: OutputFormat = opts.format ?? \"markdown\";\n const maxChars = opts.maxChars ?? DEFAULT_MAX_CHARS;\n const removeImages = opts.removeImages ?? false;\n const includeReplies = opts.includeReplies ?? DEFAULT_INCLUDE_REPLIES;\n const timeoutMs = opts.timeoutMs ?? DEFAULT_TIMEOUT_MS;\n\n let parsed: URL;\n try {\n parsed = new URL(opts.url);\n } catch {\n return { error: `Invalid URL: ${opts.url}` };\n }\n\n if (![\"http:\", \"https:\"].includes(parsed.protocol)) {\n return {\n error: `Only http/https URLs supported, got ${parsed.protocol}`,\n };\n }\n\n const fetchOptions: Record<string, unknown> = {\n browser,\n os,\n headers: {\n Accept: resolveAcceptHeader(format),\n \"Accept-Language\": DEFAULT_ACCEPT_LANGUAGE_HEADER,\n ...opts.headers,\n },\n redirect: \"follow\",\n timeout: timeoutMs,\n };\n\n if (opts.proxy) {\n fetchOptions.proxy = opts.proxy;\n }\n\n hooks.onStatusChange?.(\"fetching\");\n const response = await dependencies.fetch(opts.url, fetchOptions);\n\n if (!response.ok) {\n return {\n error: `HTTP ${response.status} ${response.statusText} for ${opts.url}`,\n };\n }\n\n const finalUrl = response.url ?? opts.url;\n const contentType = response.headers.get(\"content-type\") ?? \"\";\n const rawBody = await response.text();\n const jsonResponse = isJsonResponse(contentType, rawBody);\n\n if (format === \"json\") {\n if (!jsonResponse) {\n return { error: `Not a JSON response (content-type: ${contentType})` };\n }\n\n const result = buildJsonResult(\n opts,\n finalUrl,\n rawBody,\n format,\n maxChars,\n browser,\n os,\n );\n if (!isError(result)) {\n hooks.onStatusChange?.(\"done\");\n }\n return result;\n }\n\n if (jsonResponse) {\n const result = buildJsonResult(\n opts,\n finalUrl,\n rawBody,\n format,\n maxChars,\n browser,\n os,\n );\n if (!isError(result)) {\n hooks.onStatusChange?.(\"done\");\n }\n return result;\n }\n\n if (!HTML_CONTENT_TYPES.some((value) => contentType.includes(value))) {\n return { error: `Not an HTML page (content-type: ${contentType})` };\n }\n\n hooks.onStatusChange?.(\"extracting\");\n const document = parseLinkedomHTML(rawBody, finalUrl);\n const extracted = await dependencies.defuddle(document, finalUrl, {\n markdown: format !== \"html\",\n removeImages,\n includeReplies,\n });\n\n if (!extracted.content || extracted.wordCount === 0) {\n return {\n error: `No content extracted from ${opts.url}. May need JS rendering or is blocked.`,\n };\n }\n\n let extractedContent = extracted.content;\n let wordCount = extracted.wordCount;\n\n if (includeReplies === false && shouldStripReplies(extracted.site ?? \"\")) {\n const strippedContent = stripExtractorComments(extractedContent, format);\n if (strippedContent !== extractedContent) {\n extractedContent = strippedContent;\n wordCount = estimateWordCount(\n format === \"text\"\n ? markdownToText(extractedContent)\n : extractedContent,\n );\n }\n }\n\n const normalizedContent =\n format === \"text\" ? markdownToText(extractedContent) : extractedContent;\n\n const result = {\n url: opts.url,\n finalUrl,\n title: extracted.title ?? \"\",\n author: extracted.author ?? \"\",\n published: extracted.published ?? \"\",\n site: extracted.site ?? \"\",\n language: extracted.language ?? \"\",\n wordCount,\n content: truncateContent(normalizedContent, maxChars),\n browser,\n os,\n };\n\n hooks.onStatusChange?.(\"done\");\n return result;\n };\n}\n\nexport const defuddleFetch = createDefuddleFetch();\n\n/** Type guard: check if result is an error. */\nexport function isError(\n result: FetchResult | FetchError,\n): result is FetchError {\n return \"error\" in result;\n}\n","import { type TSchema, Type } from \"@sinclair/typebox\";\nimport {\n DEFAULT_BATCH_CONCURRENCY,\n DEFAULT_BROWSER,\n DEFAULT_INCLUDE_REPLIES,\n DEFAULT_MAX_CHARS,\n DEFAULT_OS,\n DEFAULT_TIMEOUT_MS,\n} from \"./constants\";\nimport { defuddleFetch, isError } from \"./extract\";\nimport type {\n BatchFetchItemProgress,\n BatchFetchItemResult,\n BatchFetchItemStatus,\n BatchFetchProgressSnapshot,\n BatchFetchResult,\n FetchError,\n FetchExecutionHooks,\n FetchOptions,\n FetchResult,\n FetchToolConfig,\n FetchToolDefaults,\n} from \"./types\";\n\nfunction resolveBatchConcurrency(value: number | undefined): number {\n if (!value || !Number.isFinite(value)) {\n return DEFAULT_BATCH_CONCURRENCY;\n }\n\n return Math.max(1, Math.floor(value));\n}\n\nexport function resolveFetchToolDefaults(\n config: FetchToolConfig = {},\n): FetchToolDefaults {\n return {\n maxChars: config.maxChars ?? DEFAULT_MAX_CHARS,\n timeoutMs: config.timeoutMs ?? DEFAULT_TIMEOUT_MS,\n browser: config.browser ?? DEFAULT_BROWSER,\n os: config.os ?? DEFAULT_OS,\n removeImages: config.removeImages ?? false,\n includeReplies: config.includeReplies ?? DEFAULT_INCLUDE_REPLIES,\n batchConcurrency: resolveBatchConcurrency(config.batchConcurrency),\n };\n}\n\nexport function createBaseFetchToolParameterProperties(\n defaults: FetchToolDefaults,\n): Record<string, TSchema> {\n return {\n url: Type.String({ description: \"URL to fetch (http/https only)\" }),\n browser: Type.Optional(\n Type.String({\n description: `Browser profile for TLS fingerprinting. Default: \"${defaults.browser}\". Examples: chrome_145, firefox_147, safari_26, edge_145, opera_127`,\n }),\n ),\n os: Type.Optional(\n Type.String({\n description: `OS profile for fingerprinting. Default: \"${defaults.os}\". Options: windows, macos, linux, android, ios`,\n }),\n ),\n headers: Type.Optional(\n Type.Record(Type.String(), Type.String(), {\n description:\n \"Custom HTTP headers to send. By default, Accept and Accept-Language are set automatically.\",\n }),\n ),\n maxChars: Type.Optional(\n Type.Number({\n description: `Maximum characters to return. Default: ${defaults.maxChars}`,\n }),\n ),\n format: Type.Optional(\n Type.Union(\n [\n Type.Literal(\"markdown\"),\n Type.Literal(\"html\"),\n Type.Literal(\"text\"),\n Type.Literal(\"json\"),\n ],\n {\n description:\n 'Output format. \"markdown\" (default), \"html\" (cleaned HTML), \"text\" (plain text, no formatting), or \"json\" (pretty-printed JSON)',\n },\n ),\n ),\n removeImages: Type.Optional(\n Type.Boolean({\n description: \"Strip image references from output. Default: false\",\n }),\n ),\n includeReplies: Type.Optional(\n Type.Union([Type.Boolean(), Type.Literal(\"extractors\")], {\n description:\n \"Include replies/comments: 'extractors' for site-specific only (default), true for all, false for none\",\n }),\n ),\n proxy: Type.Optional(\n Type.String({\n description:\n \"Proxy URL (http://user:pass@host:port or socks5://host:port)\",\n }),\n ),\n };\n}\n\nexport function createBatchFetchToolParameterProperties(\n defaults: FetchToolDefaults,\n): Record<string, TSchema> {\n return {\n requests: Type.Array(\n Type.Object(createBaseFetchToolParameterProperties(defaults), {\n additionalProperties: false,\n }),\n {\n minItems: 1,\n description:\n \"Array of fetch requests. Each item accepts the same parameters as the single-item fetch tool.\",\n },\n ),\n };\n}\n\nfunction buildFetchOptionsFromParams(\n params: Record<string, unknown>,\n defaults: FetchToolDefaults,\n): FetchOptions {\n return {\n url: params.url as string,\n browser: (params.browser as string) ?? defaults.browser,\n os: (params.os as string) ?? defaults.os,\n headers: params.headers as Record<string, string> | undefined,\n maxChars: (params.maxChars as number) ?? defaults.maxChars,\n format:\n (params.format as \"markdown\" | \"html\" | \"text\" | \"json\") ?? \"markdown\",\n removeImages: (params.removeImages as boolean) ?? defaults.removeImages,\n includeReplies:\n (params.includeReplies as boolean | \"extractors\") ??\n defaults.includeReplies,\n proxy: params.proxy as string | undefined,\n timeoutMs: defaults.timeoutMs,\n };\n}\n\nexport async function executeFetchToolCall(\n params: Record<string, unknown>,\n defaults: FetchToolDefaults,\n hooks: FetchExecutionHooks = {},\n): Promise<FetchResult | FetchError> {\n return defuddleFetch(buildFetchOptionsFromParams(params, defaults), hooks);\n}\n\nconst PROGRESS_BY_STATUS: Record<BatchFetchItemStatus, number> = {\n queued: 0,\n fetching: 0.35,\n extracting: 0.75,\n done: 1,\n error: 1,\n};\n\nfunction createInitialProgressItems(\n requests: Record<string, unknown>[],\n): BatchFetchItemProgress[] {\n return requests.map((request, index) => ({\n index,\n url:\n typeof request.url === \"string\" ? request.url : String(request.url ?? \"\"),\n status: \"queued\",\n progress: PROGRESS_BY_STATUS.queued,\n }));\n}\n\nfunction buildProgressSnapshot(\n items: BatchFetchItemProgress[],\n batchConcurrency: number,\n): BatchFetchProgressSnapshot {\n let completed = 0;\n let succeeded = 0;\n let failed = 0;\n\n for (const item of items) {\n if (item.status === \"done\" || item.status === \"error\") {\n completed += 1;\n }\n if (item.status === \"done\") {\n succeeded += 1;\n }\n if (item.status === \"error\") {\n failed += 1;\n }\n }\n\n return {\n items: items.map((item) => ({ ...item })),\n total: items.length,\n completed,\n succeeded,\n failed,\n batchConcurrency,\n };\n}\n\nexport async function executeBatchFetchToolCall(\n params: Record<string, unknown>,\n defaults: FetchToolDefaults,\n options: {\n batchConcurrency?: number;\n onProgress?(snapshot: BatchFetchProgressSnapshot): void;\n executeItem?(\n params: Record<string, unknown>,\n defaults: FetchToolDefaults,\n hooks?: FetchExecutionHooks,\n ): Promise<FetchResult | FetchError>;\n } = {},\n): Promise<BatchFetchResult> {\n const requests = (\n (params.requests as Record<string, unknown>[] | undefined) ?? []\n ).map((request) => request ?? {});\n const batchConcurrency = resolveBatchConcurrency(\n options.batchConcurrency ?? defaults.batchConcurrency,\n );\n const progressItems = createInitialProgressItems(requests);\n const results = new Array<BatchFetchItemResult>(requests.length);\n\n const emitProgress = () => {\n options.onProgress?.(\n buildProgressSnapshot(progressItems, batchConcurrency),\n );\n };\n\n const updateProgress = (\n index: number,\n status: BatchFetchItemStatus,\n error?: string,\n ) => {\n progressItems[index] = {\n ...progressItems[index],\n status,\n progress: PROGRESS_BY_STATUS[status],\n ...(error ? { error } : {}),\n };\n emitProgress();\n };\n\n emitProgress();\n\n let nextIndex = 0;\n\n const worker = async () => {\n while (true) {\n const index = nextIndex;\n nextIndex += 1;\n\n if (index >= requests.length) {\n return;\n }\n\n const request = requests[index] ?? {};\n const normalizedRequest = buildFetchOptionsFromParams(request, defaults);\n\n try {\n const executeItem = options.executeItem ?? executeFetchToolCall;\n const result = await executeItem(request, defaults, {\n onStatusChange(status) {\n if (status === \"done\") return;\n updateProgress(index, status);\n },\n });\n\n if (isError(result)) {\n results[index] = {\n index,\n request: normalizedRequest,\n status: \"error\",\n progress: PROGRESS_BY_STATUS.error,\n error: result.error,\n };\n updateProgress(index, \"error\", result.error);\n continue;\n }\n\n results[index] = {\n index,\n request: normalizedRequest,\n status: \"done\",\n progress: PROGRESS_BY_STATUS.done,\n result,\n };\n updateProgress(index, \"done\");\n } catch (error) {\n const message = error instanceof Error ? error.message : String(error);\n results[index] = {\n index,\n request: normalizedRequest,\n status: \"error\",\n progress: PROGRESS_BY_STATUS.error,\n error: message,\n };\n updateProgress(index, \"error\", message);\n }\n }\n };\n\n const workerCount =\n requests.length === 0 ? 0 : Math.min(batchConcurrency, requests.length);\n await Promise.all(Array.from({ length: workerCount }, async () => worker()));\n\n const finalSnapshot = buildProgressSnapshot(progressItems, batchConcurrency);\n\n return {\n items: results,\n total: finalSnapshot.total,\n succeeded: finalSnapshot.succeeded,\n failed: finalSnapshot.failed,\n batchConcurrency,\n };\n}\n","import { readFile } from \"node:fs/promises\";\nimport { join } from \"node:path\";\nimport { getAgentDir } from \"@mariozechner/pi-coding-agent\";\nimport type {\n FetchToolConfig,\n FingerprintOs,\n IncludeRepliesOption,\n} from \"smart-fetch-core\";\n\nconst VALID_OS_VALUES = new Set<FingerprintOs>([\n \"windows\",\n \"macos\",\n \"linux\",\n \"android\",\n \"ios\",\n]);\n\ninterface PiSmartFetchSettings {\n smartFetchVerboseByDefault?: boolean;\n smartFetchDefaultMaxChars?: number;\n smartFetchDefaultTimeoutMs?: number;\n smartFetchDefaultBrowser?: string;\n smartFetchDefaultOs?: FingerprintOs;\n smartFetchDefaultRemoveImages?: boolean;\n smartFetchDefaultIncludeReplies?: IncludeRepliesOption;\n smartFetchDefaultBatchConcurrency?: number;\n}\n\nexport interface ResolvedPiSmartFetchSettings extends FetchToolConfig {\n verboseByDefault: boolean;\n}\n\nfunction readBoolean(\n source: Record<string, unknown>,\n keys: string[],\n): boolean | undefined {\n for (const key of keys) {\n if (typeof source[key] === \"boolean\") {\n return source[key] as boolean;\n }\n }\n\n return undefined;\n}\n\nfunction readPositiveNumber(\n source: Record<string, unknown>,\n keys: string[],\n): number | undefined {\n for (const key of keys) {\n const value = source[key];\n if (typeof value === \"number\" && Number.isFinite(value) && value > 0) {\n return value;\n }\n }\n\n return undefined;\n}\n\nfunction readNonEmptyString(\n source: Record<string, unknown>,\n keys: string[],\n): string | undefined {\n for (const key of keys) {\n const value = source[key];\n if (typeof value === \"string\" && value.trim() !== \"\") {\n return value;\n }\n }\n\n return undefined;\n}\n\nfunction readOs(\n source: Record<string, unknown>,\n keys: string[],\n): FingerprintOs | undefined {\n for (const key of keys) {\n const value = source[key];\n if (\n typeof value === \"string\" &&\n VALID_OS_VALUES.has(value as FingerprintOs)\n ) {\n return value as FingerprintOs;\n }\n }\n\n return undefined;\n}\n\nfunction readIncludeReplies(\n source: Record<string, unknown>,\n keys: string[],\n): IncludeRepliesOption | undefined {\n for (const key of keys) {\n const value = source[key];\n if (typeof value === \"boolean\" || value === \"extractors\") {\n return value;\n }\n }\n\n return undefined;\n}\n\nfunction normalizePiSmartFetchSettings(input: unknown): PiSmartFetchSettings {\n if (!input || typeof input !== \"object\") return {};\n\n const source = input as Record<string, unknown>;\n\n return {\n smartFetchVerboseByDefault: readBoolean(source, [\n \"smartFetchVerboseByDefault\",\n \"webFetchVerboseByDefault\",\n ]),\n smartFetchDefaultMaxChars: readPositiveNumber(source, [\n \"smartFetchDefaultMaxChars\",\n \"webFetchDefaultMaxChars\",\n ]),\n smartFetchDefaultTimeoutMs: readPositiveNumber(source, [\n \"smartFetchDefaultTimeoutMs\",\n ]),\n smartFetchDefaultBrowser: readNonEmptyString(source, [\n \"smartFetchDefaultBrowser\",\n ]),\n smartFetchDefaultOs: readOs(source, [\"smartFetchDefaultOs\"]),\n smartFetchDefaultRemoveImages: readBoolean(source, [\n \"smartFetchDefaultRemoveImages\",\n ]),\n smartFetchDefaultIncludeReplies: readIncludeReplies(source, [\n \"smartFetchDefaultIncludeReplies\",\n ]),\n smartFetchDefaultBatchConcurrency: readPositiveNumber(source, [\n \"smartFetchDefaultBatchConcurrency\",\n \"webFetchDefaultBatchConcurrency\",\n ]),\n };\n}\n\nexport function resolvePiSmartFetchSettings(\n globalSettings: unknown,\n projectSettings: unknown,\n): ResolvedPiSmartFetchSettings {\n const global = normalizePiSmartFetchSettings(globalSettings);\n const project = normalizePiSmartFetchSettings(projectSettings);\n\n return {\n verboseByDefault:\n project.smartFetchVerboseByDefault ??\n global.smartFetchVerboseByDefault ??\n false,\n maxChars:\n project.smartFetchDefaultMaxChars ?? global.smartFetchDefaultMaxChars,\n timeoutMs:\n project.smartFetchDefaultTimeoutMs ?? global.smartFetchDefaultTimeoutMs,\n browser:\n project.smartFetchDefaultBrowser ?? global.smartFetchDefaultBrowser,\n os: project.smartFetchDefaultOs ?? global.smartFetchDefaultOs,\n removeImages:\n project.smartFetchDefaultRemoveImages ??\n global.smartFetchDefaultRemoveImages,\n includeReplies:\n project.smartFetchDefaultIncludeReplies ??\n global.smartFetchDefaultIncludeReplies,\n batchConcurrency:\n project.smartFetchDefaultBatchConcurrency ??\n global.smartFetchDefaultBatchConcurrency,\n };\n}\n\nasync function readSettingsFile(path: string): Promise<unknown> {\n try {\n return JSON.parse(await readFile(path, \"utf-8\"));\n } catch {\n return {};\n }\n}\n\nexport async function loadPiSmartFetchSettings(\n cwd: string,\n agentDir = getAgentDir(),\n): Promise<ResolvedPiSmartFetchSettings> {\n const globalSettings = await readSettingsFile(\n join(agentDir, \"settings.json\"),\n );\n const projectSettings = await readSettingsFile(\n join(cwd, \".pi\", \"settings.json\"),\n );\n\n return resolvePiSmartFetchSettings(globalSettings, projectSettings);\n}\n","import { type ExtensionAPI, getAgentDir } from \"@mariozechner/pi-coding-agent\";\nimport { Text } from \"@mariozechner/pi-tui\";\nimport { Type } from \"@sinclair/typebox\";\nimport {\n type BatchFetchItemProgress,\n type BatchFetchProgressSnapshot,\n type BatchFetchResult,\n buildBatchFetchResponseText,\n buildFetchResponseText,\n createBaseFetchToolParameterProperties,\n createBatchFetchToolParameterProperties,\n executeBatchFetchToolCall,\n executeFetchToolCall,\n isError,\n resolveFetchToolDefaults,\n} from \"smart-fetch-core\";\nimport { loadPiSmartFetchSettings } from \"./settings\";\n\nconst toolDescription = [\n \"Fetch a URL with browser-grade TLS fingerprinting and extract clean, readable content.\",\n \"Uses wreq-js for browser-like TLS/HTTP2 impersonation and Defuddle for article extraction.\",\n \"Supports the same fetch parameters as the OpenClaw tool, plus an optional verbose flag.\",\n \"Does NOT execute JavaScript — use a browser automation tool for JS-heavy pages.\",\n].join(\" \");\n\nconst batchToolDescription = [\n \"Fetch multiple URLs with browser-grade TLS fingerprinting and readable extraction.\",\n \"Each request accepts the same parameters as web_fetch and fans out with bounded concurrency.\",\n \"Streams per-item progress in the pi TUI with truncated URLs, statuses, and small progress bars.\",\n \"Does NOT execute JavaScript — use a browser automation tool for JS-heavy pages.\",\n].join(\" \");\n\nconst SPINNER_FRAMES = [\"⠋\", \"⠙\", \"⠹\", \"⠸\", \"⠼\", \"⠴\", \"⠦\", \"⠧\", \"⠇\", \"⠏\"];\n\ntype BatchRenderDetails = {\n verbose?: boolean;\n batchProgress?: BatchFetchProgressSnapshot;\n batchResult?: BatchFetchResult;\n started?: boolean;\n};\n\nfunction truncateMiddle(value: string, width: number): string {\n if (width <= 0) return \"\";\n if (value.length <= width) return value.padEnd(width, \" \");\n if (width === 1) return \"…\";\n\n const left = Math.ceil((width - 1) / 2);\n const right = Math.floor((width - 1) / 2);\n return `${value.slice(0, left)}…${value.slice(value.length - right)}`;\n}\n\nfunction pad(value: string, width: number): string {\n return value.length >= width\n ? value.slice(0, width)\n : value.padEnd(width, \" \");\n}\n\nfunction renderProgressBar(\n item: BatchFetchItemProgress,\n width: number,\n theme: {\n fg(color: string, value: string): string;\n },\n): string {\n const innerWidth = Math.max(4, width - 2);\n const filled = Math.max(\n 0,\n Math.min(innerWidth, Math.round(item.progress * innerWidth)),\n );\n const empty = Math.max(0, innerWidth - filled);\n const barColor =\n item.status === \"error\"\n ? \"error\"\n : item.status === \"done\"\n ? \"success\"\n : \"accent\";\n\n return [\n theme.fg(\"muted\", \"[\"),\n theme.fg(barColor, \"█\".repeat(filled)),\n theme.fg(\"dim\", \"░\".repeat(empty)),\n theme.fg(\"muted\", \"]\"),\n ].join(\"\");\n}\n\nfunction renderStatusGlyph(\n item: BatchFetchItemProgress,\n spinnerIndex: number,\n theme: {\n fg(color: string, value: string): string;\n },\n): string {\n switch (item.status) {\n case \"done\":\n return theme.fg(\"success\", \"✓\");\n case \"error\":\n return theme.fg(\"error\", \"✗\");\n case \"queued\":\n return theme.fg(\"muted\", \"○\");\n default:\n return theme.fg(\n \"accent\",\n SPINNER_FRAMES[spinnerIndex % SPINNER_FRAMES.length] ?? \"⠋\",\n );\n }\n}\n\nfunction renderBatchProgressText(\n snapshot: BatchFetchProgressSnapshot,\n width: number,\n expanded: boolean,\n theme: {\n bold(value: string): string;\n fg(color: string, value: string): string;\n },\n): string {\n const summary = [\n theme.fg(\"toolTitle\", theme.bold(\"batch_web_fetch \")),\n theme.fg(\n \"muted\",\n `${snapshot.completed}/${snapshot.total} done · ok ${snapshot.succeeded} · err ${snapshot.failed} · concurrency ${snapshot.batchConcurrency}`,\n ),\n ].join(\"\");\n\n const availableRowWidth = Math.max(24, Math.floor(width * 0.8));\n const statusWidth = 10;\n const progressWidth = Math.max(\n 10,\n Math.min(18, Math.floor(availableRowWidth * 0.25)),\n );\n const glyphWidth = 2;\n const urlWidth = Math.max(\n 12,\n availableRowWidth - glyphWidth - statusWidth - progressWidth - 3,\n );\n\n const rows = snapshot.items.map((item, index) => {\n const glyph = renderStatusGlyph(item, snapshot.completed + index, theme);\n const url = theme.fg(\"accent\", truncateMiddle(item.url, urlWidth));\n const statusColor =\n item.status === \"error\"\n ? \"error\"\n : item.status === \"done\"\n ? \"success\"\n : item.status === \"queued\"\n ? \"muted\"\n : \"warning\";\n const status = theme.fg(statusColor, pad(item.status, statusWidth));\n const bar = renderProgressBar(item, progressWidth, theme);\n\n const baseRow = `${glyph} ${url} ${status} ${bar}`;\n if (!expanded || !item.error) {\n return baseRow;\n }\n\n return `${baseRow}\\n ${theme.fg(\"error\", `error: ${item.error}`)}`;\n });\n\n return [summary, ...rows].join(\"\\n\");\n}\n\nfunction createResponsiveBatchComponent(\n details: BatchRenderDetails,\n expanded: boolean,\n theme: {\n bold(value: string): string;\n fg(color: string, value: string): string;\n },\n) {\n const text = new Text(\"\", 0, 0);\n\n return {\n render(width: number) {\n const snapshot = details.batchProgress;\n if (!snapshot) {\n text.setText(theme.fg(\"muted\", \"No batch progress available.\"));\n return text.render(width);\n }\n\n text.setText(renderBatchProgressText(snapshot, width, expanded, theme));\n return text.render(width);\n },\n invalidate() {\n text.invalidate();\n },\n };\n}\n\nexport default function piSmartFetchExtension(pi: ExtensionAPI) {\n const defaults = resolveFetchToolDefaults();\n\n pi.registerTool({\n name: \"web_fetch\",\n label: \"web_fetch\",\n description: toolDescription,\n promptSnippet:\n \"web_fetch(url, browser?, os?, headers?, maxChars?, format?, removeImages?, includeReplies?, proxy?, verbose?): fetch browser-fingerprinted readable web content\",\n parameters: Type.Object({\n ...createBaseFetchToolParameterProperties(defaults),\n verbose: Type.Optional(\n Type.Boolean({\n description:\n \"Include the full metadata header (site, language, word count, browser fingerprint info). Default: false, or smartFetchVerboseByDefault from pi settings.\",\n }),\n ),\n }),\n\n async execute(_toolCallId, params, _signal, _onUpdate, ctx) {\n const settings = await loadPiSmartFetchSettings(ctx.cwd, getAgentDir());\n const runtimeDefaults = resolveFetchToolDefaults(settings);\n const verbose =\n (params.verbose as boolean | undefined) ?? settings.verboseByDefault;\n const result = await executeFetchToolCall(params, runtimeDefaults);\n\n if (isError(result)) {\n return {\n content: [{ type: \"text\", text: `Error: ${result.error}` }],\n details: { error: true, verbose },\n };\n }\n\n return {\n content: [\n { type: \"text\", text: buildFetchResponseText(result, { verbose }) },\n ],\n details: { verbose, maxChars: runtimeDefaults.maxChars },\n };\n },\n });\n\n pi.registerTool({\n name: \"batch_web_fetch\",\n label: \"batch_web_fetch\",\n description: batchToolDescription,\n promptSnippet:\n \"batch_web_fetch(requests, verbose?): fetch multiple URLs concurrently with per-item progress in the pi TUI\",\n parameters: Type.Object({\n ...createBatchFetchToolParameterProperties(defaults),\n verbose: Type.Optional(\n Type.Boolean({\n description:\n \"Include the full metadata header for each successful result. Default: false, or smartFetchVerboseByDefault from pi settings.\",\n }),\n ),\n }),\n\n async execute(_toolCallId, params, _signal, onUpdate, ctx) {\n const settings = await loadPiSmartFetchSettings(ctx.cwd, getAgentDir());\n const runtimeDefaults = resolveFetchToolDefaults(settings);\n const verbose =\n (params.verbose as boolean | undefined) ?? settings.verboseByDefault;\n\n const batchResult = await executeBatchFetchToolCall(\n params,\n runtimeDefaults,\n {\n batchConcurrency: runtimeDefaults.batchConcurrency,\n onProgress(snapshot) {\n onUpdate?.({\n content: [\n {\n type: \"text\",\n text: `Started batch fetch for ${snapshot.total} URLs (${snapshot.completed}/${snapshot.total} complete).`,\n },\n ],\n details: {\n verbose,\n started: true,\n batchProgress: snapshot,\n } satisfies BatchRenderDetails,\n });\n },\n },\n );\n\n const finalProgress: BatchFetchProgressSnapshot = {\n items: batchResult.items.map((item) => ({\n index: item.index,\n url: item.request.url,\n status: item.status,\n progress: item.progress,\n ...(item.error ? { error: item.error } : {}),\n })),\n total: batchResult.total,\n completed: batchResult.total,\n succeeded: batchResult.succeeded,\n failed: batchResult.failed,\n batchConcurrency: batchResult.batchConcurrency,\n };\n\n return {\n content: [\n {\n type: \"text\",\n text: buildBatchFetchResponseText(batchResult, { verbose }),\n },\n ],\n details: {\n verbose,\n started: true,\n batchProgress: finalProgress,\n batchResult,\n } satisfies BatchRenderDetails,\n };\n },\n\n renderCall(args, theme) {\n const batchArgs = args as { requests?: unknown[] };\n const requestCount = Array.isArray(batchArgs.requests)\n ? batchArgs.requests.length\n : 0;\n let text = theme.fg(\"toolTitle\", theme.bold(\"batch_web_fetch \"));\n text += theme.fg(\"muted\", `${requestCount} urls`);\n return new Text(text, 0, 0);\n },\n\n renderResult(result, { expanded }, theme) {\n return createResponsiveBatchComponent(\n (result.details as BatchRenderDetails | undefined) ?? {},\n expanded,\n theme,\n );\n },\n });\n}\n"]}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "pi-smart-fetch",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.9",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "pi.dev smart fetch extension with browser-grade TLS fingerprinting and Defuddle extraction.",
|
|
6
6
|
"license": "MIT",
|
|
@@ -57,6 +57,7 @@
|
|
|
57
57
|
"pack:dry-run": "bun run build && npm pack --dry-run"
|
|
58
58
|
},
|
|
59
59
|
"dependencies": {
|
|
60
|
+
"@mariozechner/pi-tui": "^0.66.1",
|
|
60
61
|
"@sinclair/typebox": "^0.34.49",
|
|
61
62
|
"defuddle": "^0.14.0",
|
|
62
63
|
"linkedom": "^0.18.12",
|