@isdk/web-fetcher 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (66) hide show
  1. package/README.action.cn.md +469 -0
  2. package/README.action.md +452 -0
  3. package/README.cn.md +147 -0
  4. package/README.engine.cn.md +262 -0
  5. package/README.engine.md +262 -0
  6. package/README.md +147 -0
  7. package/dist/index.d.mts +1603 -0
  8. package/dist/index.d.ts +1603 -0
  9. package/dist/index.js +1 -0
  10. package/dist/index.mjs +1 -0
  11. package/docs/README.md +151 -0
  12. package/docs/_media/LICENSE-MIT +22 -0
  13. package/docs/_media/README.action.md +452 -0
  14. package/docs/_media/README.cn.md +147 -0
  15. package/docs/_media/README.engine.md +262 -0
  16. package/docs/classes/CheerioFetchEngine.md +1447 -0
  17. package/docs/classes/ClickAction.md +533 -0
  18. package/docs/classes/ExtractAction.md +533 -0
  19. package/docs/classes/FetchAction.md +444 -0
  20. package/docs/classes/FetchEngine.md +1230 -0
  21. package/docs/classes/FetchSession.md +111 -0
  22. package/docs/classes/FillAction.md +533 -0
  23. package/docs/classes/GetContentAction.md +533 -0
  24. package/docs/classes/GotoAction.md +537 -0
  25. package/docs/classes/PauseAction.md +533 -0
  26. package/docs/classes/PlaywrightFetchEngine.md +1437 -0
  27. package/docs/classes/SubmitAction.md +533 -0
  28. package/docs/classes/WaitForAction.md +533 -0
  29. package/docs/classes/WebFetcher.md +85 -0
  30. package/docs/enumerations/FetchActionResultStatus.md +40 -0
  31. package/docs/functions/fetchWeb.md +43 -0
  32. package/docs/globals.md +72 -0
  33. package/docs/interfaces/BaseFetchActionProperties.md +83 -0
  34. package/docs/interfaces/BaseFetchCollectorActionProperties.md +145 -0
  35. package/docs/interfaces/BaseFetcherProperties.md +206 -0
  36. package/docs/interfaces/Cookie.md +142 -0
  37. package/docs/interfaces/DispatchedEngineAction.md +60 -0
  38. package/docs/interfaces/ExtractActionProperties.md +113 -0
  39. package/docs/interfaces/FetchActionInContext.md +149 -0
  40. package/docs/interfaces/FetchActionProperties.md +125 -0
  41. package/docs/interfaces/FetchActionResult.md +55 -0
  42. package/docs/interfaces/FetchContext.md +424 -0
  43. package/docs/interfaces/FetchEngineContext.md +328 -0
  44. package/docs/interfaces/FetchMetadata.md +73 -0
  45. package/docs/interfaces/FetchResponse.md +105 -0
  46. package/docs/interfaces/FetchReturnTypeRegistry.md +57 -0
  47. package/docs/interfaces/FetchSite.md +320 -0
  48. package/docs/interfaces/FetcherOptions.md +300 -0
  49. package/docs/interfaces/GotoActionOptions.md +66 -0
  50. package/docs/interfaces/PendingEngineRequest.md +51 -0
  51. package/docs/interfaces/SubmitActionOptions.md +23 -0
  52. package/docs/interfaces/WaitForActionOptions.md +39 -0
  53. package/docs/type-aliases/BaseFetchActionOptions.md +11 -0
  54. package/docs/type-aliases/BaseFetchCollectorOptions.md +11 -0
  55. package/docs/type-aliases/BrowserEngine.md +11 -0
  56. package/docs/type-aliases/FetchActionCapabilities.md +11 -0
  57. package/docs/type-aliases/FetchActionCapabilityMode.md +11 -0
  58. package/docs/type-aliases/FetchActionOptions.md +11 -0
  59. package/docs/type-aliases/FetchEngineAction.md +18 -0
  60. package/docs/type-aliases/FetchEngineType.md +11 -0
  61. package/docs/type-aliases/FetchReturnType.md +11 -0
  62. package/docs/type-aliases/FetchReturnTypeFor.md +17 -0
  63. package/docs/type-aliases/OnFetchPauseCallback.md +23 -0
  64. package/docs/type-aliases/ResourceType.md +11 -0
  65. package/docs/variables/DefaultFetcherProperties.md +11 -0
  66. package/package.json +90 -0
@@ -0,0 +1,1230 @@
1
+ [**@isdk/web-fetcher**](../README.md)
2
+
3
+ ***
4
+
5
+ [@isdk/web-fetcher](../globals.md) / FetchEngine
6
+
7
+ # Abstract Class: FetchEngine\<TContext, TCrawler, TOptions\>
8
+
9
+ Defined in: [packages/web-fetcher/src/engine/base.ts:139](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L139)
10
+
11
+ ## Extended by
12
+
13
+ - [`CheerioFetchEngine`](CheerioFetchEngine.md)
14
+ - [`PlaywrightFetchEngine`](PlaywrightFetchEngine.md)
15
+
16
+ ## Type Parameters
17
+
18
+ ### TContext
19
+
20
+ `TContext` *extends* `CrawlingContext` = `any`
21
+
22
+ ### TCrawler
23
+
24
+ `TCrawler` *extends* `BasicCrawler`\<`TContext`\> = `any`
25
+
26
+ ### TOptions
27
+
28
+ `TOptions` *extends* `BasicCrawlerOptions`\<`TContext`\> = `any`
29
+
30
+ ## Constructors
31
+
32
+ ### Constructor
33
+
34
+ > **new FetchEngine**\<`TContext`, `TCrawler`, `TOptions`\>(): `FetchEngine`\<`TContext`, `TCrawler`, `TOptions`\>
35
+
36
+ #### Returns
37
+
38
+ `FetchEngine`\<`TContext`, `TCrawler`, `TOptions`\>
39
+
40
+ ## Properties
41
+
42
+ ### actionEmitter
43
+
44
+ > `protected` **actionEmitter**: `EventEmitter`
45
+
46
+ Defined in: [packages/web-fetcher/src/engine/base.ts:235](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L235)
47
+
48
+ ***
49
+
50
+ ### blockedTypes
51
+
52
+ > `protected` **blockedTypes**: `Set`\<`string`\>
53
+
54
+ Defined in: [packages/web-fetcher/src/engine/base.ts:239](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L239)
55
+
56
+ ***
57
+
58
+ ### crawler?
59
+
60
+ > `protected` `optional` **crawler**: `TCrawler`
61
+
62
+ Defined in: [packages/web-fetcher/src/engine/base.ts:227](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L227)
63
+
64
+ ***
65
+
66
+ ### ctx?
67
+
68
+ > `protected` `optional` **ctx**: [`FetchEngineContext`](../interfaces/FetchEngineContext.md)
69
+
70
+ Defined in: [packages/web-fetcher/src/engine/base.ts:225](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L225)
71
+
72
+ ***
73
+
74
+ ### hdrs
75
+
76
+ > `protected` **hdrs**: `Record`\<`string`, `string`\> = `{}`
77
+
78
+ Defined in: [packages/web-fetcher/src/engine/base.ts:231](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L231)
79
+
80
+ ***
81
+
82
+ ### isCrawlerReady?
83
+
84
+ > `protected` `optional` **isCrawlerReady**: `boolean`
85
+
86
+ Defined in: [packages/web-fetcher/src/engine/base.ts:228](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L228)
87
+
88
+ ***
89
+
90
+ ### isPageActive
91
+
92
+ > `protected` **isPageActive**: `boolean` = `false`
93
+
94
+ Defined in: [packages/web-fetcher/src/engine/base.ts:236](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L236)
95
+
96
+ ***
97
+
98
+ ### jar
99
+
100
+ > `protected` **jar**: [`Cookie`](../interfaces/Cookie.md)[] = `[]`
101
+
102
+ Defined in: [packages/web-fetcher/src/engine/base.ts:232](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L232)
103
+
104
+ ***
105
+
106
+ ### lastResponse?
107
+
108
+ > `protected` `optional` **lastResponse**: [`FetchResponse`](../interfaces/FetchResponse.md)
109
+
110
+ Defined in: [packages/web-fetcher/src/engine/base.ts:238](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L238)
111
+
112
+ ***
113
+
114
+ ### navigationLock
115
+
116
+ > `protected` **navigationLock**: `PromiseLock`
117
+
118
+ Defined in: [packages/web-fetcher/src/engine/base.ts:237](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L237)
119
+
120
+ ***
121
+
122
+ ### opts?
123
+
124
+ > `protected` `optional` **opts**: [`BaseFetcherProperties`](../interfaces/BaseFetcherProperties.md)
125
+
126
+ Defined in: [packages/web-fetcher/src/engine/base.ts:226](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L226)
127
+
128
+ ***
129
+
130
+ ### pendingRequests
131
+
132
+ > `protected` **pendingRequests**: `Map`\<`string`, [`PendingEngineRequest`](../interfaces/PendingEngineRequest.md)\>
133
+
134
+ Defined in: [packages/web-fetcher/src/engine/base.ts:233](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L233)
135
+
136
+ ***
137
+
138
+ ### requestCounter
139
+
140
+ > `protected` **requestCounter**: `number` = `0`
141
+
142
+ Defined in: [packages/web-fetcher/src/engine/base.ts:234](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L234)
143
+
144
+ ***
145
+
146
+ ### requestQueue?
147
+
148
+ > `protected` `optional` **requestQueue**: `RequestQueue`
149
+
150
+ Defined in: [packages/web-fetcher/src/engine/base.ts:229](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L229)
151
+
152
+ ***
153
+
154
+ ### id
155
+
156
+ > `readonly` `static` **id**: `string`
157
+
158
+ Defined in: [packages/web-fetcher/src/engine/base.ts:215](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L215)
159
+
160
+ Unique identifier for the engine implementation.
161
+
162
+ #### Remarks
163
+
164
+ Must be defined by concrete implementations. Used for registration and lookup in engine registry.
165
+
166
+ ***
167
+
168
+ ### mode
169
+
170
+ > `readonly` `static` **mode**: [`FetchEngineType`](../type-aliases/FetchEngineType.md)
171
+
172
+ Defined in: [packages/web-fetcher/src/engine/base.ts:223](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L223)
173
+
174
+ Execution mode of the engine (`'http'` or `'browser'`).
175
+
176
+ #### Remarks
177
+
178
+ Must be defined by concrete implementations. Indicates whether engine operates at HTTP level or uses full browser.
179
+
180
+ ## Accessors
181
+
182
+ ### context
183
+
184
+ #### Get Signature
185
+
186
+ > **get** **context**(): `undefined` \| [`FetchEngineContext`](../interfaces/FetchEngineContext.md)
187
+
188
+ Defined in: [packages/web-fetcher/src/engine/base.ts:473](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L473)
189
+
190
+ Gets the fetch engine context associated with this instance.
191
+
192
+ ##### Returns
193
+
194
+ `undefined` \| [`FetchEngineContext`](../interfaces/FetchEngineContext.md)
195
+
196
+ ***
197
+
198
+ ### id
199
+
200
+ #### Get Signature
201
+
202
+ > **get** **id**(): `string`
203
+
204
+ Defined in: [packages/web-fetcher/src/engine/base.ts:459](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L459)
205
+
206
+ Gets the unique identifier of this engine implementation.
207
+
208
+ ##### Returns
209
+
210
+ `string`
211
+
212
+ ***
213
+
214
+ ### mode
215
+
216
+ #### Get Signature
217
+
218
+ > **get** **mode**(): [`FetchEngineType`](../type-aliases/FetchEngineType.md)
219
+
220
+ Defined in: [packages/web-fetcher/src/engine/base.ts:466](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L466)
221
+
222
+ Gets the execution mode of this engine (`'http'` or `'browser'`).
223
+
224
+ ##### Returns
225
+
226
+ [`FetchEngineType`](../type-aliases/FetchEngineType.md)
227
+
228
+ ## Methods
229
+
230
+ ### \_cleanup()?
231
+
232
+ > `protected` `optional` **\_cleanup**(): `Promise`\<`void`\>
233
+
234
+ Defined in: [packages/web-fetcher/src/engine/base.ts:241](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L241)
235
+
236
+ #### Returns
237
+
238
+ `Promise`\<`void`\>
239
+
240
+ ***
241
+
242
+ ### \_commonCleanup()
243
+
244
+ > `protected` **\_commonCleanup**(): `Promise`\<`void`\>
245
+
246
+ Defined in: [packages/web-fetcher/src/engine/base.ts:656](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L656)
247
+
248
+ #### Returns
249
+
250
+ `Promise`\<`void`\>
251
+
252
+ ***
253
+
254
+ ### \_createCrawler()
255
+
256
+ > `abstract` `protected` **\_createCrawler**(`options`): `TCrawler`
257
+
258
+ Defined in: [packages/web-fetcher/src/engine/base.ts:296](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L296)
259
+
260
+ **`Internal`**
261
+
262
+ Creates the crawler instance for the specific engine implementation.
263
+
264
+ #### Parameters
265
+
266
+ ##### options
267
+
268
+ `TOptions`
269
+
270
+ The final crawler options.
271
+
272
+ #### Returns
273
+
274
+ `TCrawler`
275
+
276
+ ***
277
+
278
+ ### \_executePendingActions()
279
+
280
+ > `protected` **\_executePendingActions**(`context`): `Promise`\<`void`\>
281
+
282
+ Defined in: [packages/web-fetcher/src/engine/base.ts:572](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L572)
283
+
284
+ **`Internal`**
285
+
286
+ Executes all pending fetch engine actions within the current Crawlee request handler context.
287
+
288
+ **Critical Execution Constraint**: This method **MUST** be awaited within the synchronous execution path
289
+ of Crawlee's [requestHandler](https://crawlee.dev/js/api/basic-crawler) (before any `await` that yields control back to the event loop).
290
+
291
+ ### Why This Constraint Exists
292
+ - Crawlee's page context ([PlaywrightCrawler](https://crawlee.dev/js/api/playwright-crawler)'s `page` or [CheerioCrawler](https://crawlee.dev/js/api/cheerio-crawler)'s `$`)
293
+ is **only valid during the synchronous execution phase** of the request handler
294
+ - After any `await` (e.g., `await page.goto()`), the page context may be destroyed
295
+ due to Crawlee's internal resource management
296
+
297
+ ### How It Works
298
+ 1. Processes all actions queued via [dispatchAction](#dispatchaction) (click, fill, submit, etc.)
299
+ 2. Maintains the page context validity window via [isPageActive](#ispageactive) lifecycle flag
300
+ 3. Automatically cleans up event listeners upon completion
301
+
302
+ Usage see [\_sharedRequestHandler](#_sharedrequesthandler)
303
+
304
+ #### Parameters
305
+
306
+ ##### context
307
+
308
+ `TContext`
309
+
310
+ The active Crawlee crawling context containing the page/$ object
311
+
312
+ #### Returns
313
+
314
+ `Promise`\<`void`\>
315
+
316
+ #### See
317
+
318
+ [\_sharedRequestHandler](#_sharedrequesthandler)
319
+
320
+ #### Throws
321
+
322
+ If called outside valid page context window (`!this.isPageActive`)
323
+ Engine infrastructure method - not for direct consumer use
324
+
325
+ ***
326
+
327
+ ### \_extract()
328
+
329
+ > `protected` **\_extract**(`schema`, `context`): `Promise`\<`any`\>
330
+
331
+ Defined in: [packages/web-fetcher/src/engine/base.ts:246](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L246)
332
+
333
+ #### Parameters
334
+
335
+ ##### schema
336
+
337
+ `ExtractSchema`
338
+
339
+ ##### context
340
+
341
+ `any`
342
+
343
+ #### Returns
344
+
345
+ `Promise`\<`any`\>
346
+
347
+ ***
348
+
349
+ ### \_extractValue()
350
+
351
+ > `abstract` `protected` **\_extractValue**(`schema`, `context`): `Promise`\<`any`\>
352
+
353
+ Defined in: [packages/web-fetcher/src/engine/base.ts:244](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L244)
354
+
355
+ #### Parameters
356
+
357
+ ##### schema
358
+
359
+ `ExtractValueSchema`
360
+
361
+ ##### context
362
+
363
+ `any`
364
+
365
+ #### Returns
366
+
367
+ `Promise`\<`any`\>
368
+
369
+ ***
370
+
371
+ ### \_getSpecificCrawlerOptions()
372
+
373
+ > `abstract` `protected` **\_getSpecificCrawlerOptions**(`ctx`): `Partial`\<`TOptions`\> \| `Promise`\<`Partial`\<`TOptions`\>\>
374
+
375
+ Defined in: [packages/web-fetcher/src/engine/base.ts:303](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L303)
376
+
377
+ **`Internal`**
378
+
379
+ Gets the crawler-specific options from the subclass.
380
+
381
+ #### Parameters
382
+
383
+ ##### ctx
384
+
385
+ [`FetchEngineContext`](../interfaces/FetchEngineContext.md)
386
+
387
+ The fetch engine context.
388
+
389
+ #### Returns
390
+
391
+ `Partial`\<`TOptions`\> \| `Promise`\<`Partial`\<`TOptions`\>\>
392
+
393
+ ***
394
+
395
+ ### \_normalizeSchema()
396
+
397
+ > `protected` **\_normalizeSchema**(`schema`): `ExtractSchema`
398
+
399
+ Defined in: [packages/web-fetcher/src/engine/base.ts:415](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L415)
400
+
401
+ #### Parameters
402
+
403
+ ##### schema
404
+
405
+ `ExtractSchema`
406
+
407
+ #### Returns
408
+
409
+ `ExtractSchema`
410
+
411
+ ***
412
+
413
+ ### \_querySelectorAll()
414
+
415
+ > `abstract` `protected` **\_querySelectorAll**(`context`, `selector`): `Promise`\<`any`[]\>
416
+
417
+ Defined in: [packages/web-fetcher/src/engine/base.ts:243](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L243)
418
+
419
+ #### Parameters
420
+
421
+ ##### context
422
+
423
+ `any`
424
+
425
+ ##### selector
426
+
427
+ `string`
428
+
429
+ #### Returns
430
+
431
+ `Promise`\<`any`[]\>
432
+
433
+ ***
434
+
435
+ ### \_sharedFailedRequestHandler()
436
+
437
+ > `protected` **\_sharedFailedRequestHandler**(`context`, `error?`): `Promise`\<`void`\>
438
+
439
+ Defined in: [packages/web-fetcher/src/engine/base.ts:623](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L623)
440
+
441
+ #### Parameters
442
+
443
+ ##### context
444
+
445
+ `TContext`
446
+
447
+ ##### error?
448
+
449
+ `Error`
450
+
451
+ #### Returns
452
+
453
+ `Promise`\<`void`\>
454
+
455
+ ***
456
+
457
+ ### \_sharedRequestHandler()
458
+
459
+ > `protected` **\_sharedRequestHandler**(`context`): `Promise`\<`void`\>
460
+
461
+ Defined in: [packages/web-fetcher/src/engine/base.ts:595](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L595)
462
+
463
+ #### Parameters
464
+
465
+ ##### context
466
+
467
+ `TContext`
468
+
469
+ #### Returns
470
+
471
+ `Promise`\<`void`\>
472
+
473
+ ***
474
+
475
+ ### blockResources()
476
+
477
+ > **blockResources**(`types`, `overwrite?`): `Promise`\<`number`\>
478
+
479
+ Defined in: [packages/web-fetcher/src/engine/base.ts:700](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L700)
480
+
481
+ Blocks specified resource types from loading.
482
+
483
+ #### Parameters
484
+
485
+ ##### types
486
+
487
+ `string`[]
488
+
489
+ Resource types to block
490
+
491
+ ##### overwrite?
492
+
493
+ `boolean`
494
+
495
+ Whether to replace existing blocked types
496
+
497
+ #### Returns
498
+
499
+ `Promise`\<`number`\>
500
+
501
+ Number of blocked resource types
502
+
503
+ #### Example
504
+
505
+ ```ts
506
+ await engine.blockResources(['image', 'stylesheet']);
507
+ await engine.blockResources(['script'], true); // Replace existing
508
+ ```
509
+
510
+ ***
511
+
512
+ ### buildResponse()
513
+
514
+ > `abstract` `protected` **buildResponse**(`context`): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
515
+
516
+ Defined in: [packages/web-fetcher/src/engine/base.ts:314](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L314)
517
+
518
+ **`Internal`**
519
+
520
+ Abstract method for building standard [FetchResponse] from Crawlee context.
521
+
522
+ #### Parameters
523
+
524
+ ##### context
525
+
526
+ `TContext`
527
+
528
+ Crawlee crawling context
529
+
530
+ #### Returns
531
+
532
+ `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
533
+
534
+ Promise resolving to [FetchResponse] object
535
+
536
+ #### Remarks
537
+
538
+ Converts implementation-specific context (Playwright `page` or Cheerio `$`) to standardized response.
539
+
540
+ ***
541
+
542
+ ### cleanup()
543
+
544
+ > **cleanup**(): `Promise`\<`void`\>
545
+
546
+ Defined in: [packages/web-fetcher/src/engine/base.ts:536](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L536)
547
+
548
+ #### Returns
549
+
550
+ `Promise`\<`void`\>
551
+
552
+ ***
553
+
554
+ ### click()
555
+
556
+ > **click**(`selector`): `Promise`\<`void`\>
557
+
558
+ Defined in: [packages/web-fetcher/src/engine/base.ts:365](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L365)
559
+
560
+ Clicks on element matching selector.
561
+
562
+ #### Parameters
563
+
564
+ ##### selector
565
+
566
+ `string`
567
+
568
+ CSS selector of element to click
569
+
570
+ #### Returns
571
+
572
+ `Promise`\<`void`\>
573
+
574
+ Promise resolving when click is processed
575
+
576
+ #### Throws
577
+
578
+ When no active page context exists
579
+
580
+ ***
581
+
582
+ ### cookies()
583
+
584
+ #### Call Signature
585
+
586
+ > **cookies**(): `Promise`\<[`Cookie`](../interfaces/Cookie.md)[]\>
587
+
588
+ Defined in: [packages/web-fetcher/src/engine/base.ts:811](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L811)
589
+
590
+ Manages cookies for current session with multiple overloads.
591
+
592
+ Gets all cookies.
593
+
594
+ ##### Returns
595
+
596
+ `Promise`\<[`Cookie`](../interfaces/Cookie.md)[]\>
597
+
598
+ Array of cookies
599
+
600
+ Sets cookies for session.
601
+
602
+ ##### Example
603
+
604
+ ```ts
605
+ const cookies = await engine.cookies();
606
+ await engine.cookies([{ name: 'session', value: '123' }]);
607
+ ```
608
+
609
+ #### Call Signature
610
+
611
+ > **cookies**(`cookies`): `Promise`\<`boolean`\>
612
+
613
+ Defined in: [packages/web-fetcher/src/engine/base.ts:812](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L812)
614
+
615
+ Manages cookies for current session with multiple overloads.
616
+
617
+ Gets all cookies.
618
+
619
+ ##### Parameters
620
+
621
+ ###### cookies
622
+
623
+ [`Cookie`](../interfaces/Cookie.md)[]
624
+
625
+ Cookies to set
626
+
627
+ ##### Returns
628
+
629
+ `Promise`\<`boolean`\>
630
+
631
+ Array of cookies
632
+
633
+ Sets cookies for session.
634
+
635
+ ##### Example
636
+
637
+ ```ts
638
+ const cookies = await engine.cookies();
639
+ await engine.cookies([{ name: 'session', value: '123' }]);
640
+ ```
641
+
642
+ ***
643
+
644
+ ### dispatchAction()
645
+
646
+ > `protected` **dispatchAction**\<`T`\>(`action`): `Promise`\<`T`\>
647
+
648
+ Defined in: [packages/web-fetcher/src/engine/base.ts:639](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L639)
649
+
650
+ #### Type Parameters
651
+
652
+ ##### T
653
+
654
+ `T`
655
+
656
+ #### Parameters
657
+
658
+ ##### action
659
+
660
+ [`FetchEngineAction`](../type-aliases/FetchEngineAction.md)
661
+
662
+ #### Returns
663
+
664
+ `Promise`\<`T`\>
665
+
666
+ ***
667
+
668
+ ### dispose()
669
+
670
+ > **dispose**(): `Promise`\<`void`\>
671
+
672
+ Defined in: [packages/web-fetcher/src/engine/base.ts:829](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L829)
673
+
674
+ Disposes of engine, cleaning up all resources.
675
+
676
+ #### Returns
677
+
678
+ `Promise`\<`void`\>
679
+
680
+ Promise resolving when disposal completes
681
+
682
+ ***
683
+
684
+ ### executeAction()
685
+
686
+ > `abstract` `protected` **executeAction**(`context`, `action`): `Promise`\<`any`\>
687
+
688
+ Defined in: [packages/web-fetcher/src/engine/base.ts:326](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L326)
689
+
690
+ **`Internal`**
691
+
692
+ Abstract method for executing action within current page context.
693
+
694
+ #### Parameters
695
+
696
+ ##### context
697
+
698
+ `TContext`
699
+
700
+ Crawlee crawling context
701
+
702
+ ##### action
703
+
704
+ [`FetchEngineAction`](../type-aliases/FetchEngineAction.md)
705
+
706
+ Action to execute
707
+
708
+ #### Returns
709
+
710
+ `Promise`\<`any`\>
711
+
712
+ Promise resolving to action result
713
+
714
+ #### Remarks
715
+
716
+ Handles specific user interactions using underlying technology (Playwright/Cheerio).
717
+
718
+ ***
719
+
720
+ ### extract()
721
+
722
+ > **extract**\<`T`\>(`schema`): `Promise`\<`T`\>
723
+
724
+ Defined in: [packages/web-fetcher/src/engine/base.ts:410](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L410)
725
+
726
+ Extracts structured data from the current page content.
727
+
728
+ #### Type Parameters
729
+
730
+ ##### T
731
+
732
+ `T`
733
+
734
+ #### Parameters
735
+
736
+ ##### schema
737
+
738
+ `ExtractSchema`
739
+
740
+ An object defining the data to extract.
741
+
742
+ #### Returns
743
+
744
+ `Promise`\<`T`\>
745
+
746
+ A promise that resolves to an object with the extracted data.
747
+
748
+ ***
749
+
750
+ ### fill()
751
+
752
+ > **fill**(`selector`, `value`): `Promise`\<`void`\>
753
+
754
+ Defined in: [packages/web-fetcher/src/engine/base.ts:377](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L377)
755
+
756
+ Fills input element with specified value.
757
+
758
+ #### Parameters
759
+
760
+ ##### selector
761
+
762
+ `string`
763
+
764
+ CSS selector of input element
765
+
766
+ ##### value
767
+
768
+ `string`
769
+
770
+ Value to fill
771
+
772
+ #### Returns
773
+
774
+ `Promise`\<`void`\>
775
+
776
+ Promise resolving when fill operation completes
777
+
778
+ #### Throws
779
+
780
+ When no active page context exists
781
+
782
+ ***
783
+
784
+ ### getContent()
785
+
786
+ > **getContent**(): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
787
+
788
+ Defined in: [packages/web-fetcher/src/engine/base.ts:714](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L714)
789
+
790
+ Gets content of current page.
791
+
792
+ #### Returns
793
+
794
+ `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
795
+
796
+ Promise resolving to fetch response
797
+
798
+ #### Throws
799
+
800
+ When no content has been fetched yet
801
+
802
+ ***
803
+
804
+ ### goto()
805
+
806
+ > `abstract` **goto**(`url`, `params?`): `Promise`\<`void` \| [`FetchResponse`](../interfaces/FetchResponse.md)\>
807
+
808
+ Defined in: [packages/web-fetcher/src/engine/base.ts:340](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L340)
809
+
810
+ Navigates to the specified URL.
811
+
812
+ #### Parameters
813
+
814
+ ##### url
815
+
816
+ `string`
817
+
818
+ Target URL
819
+
820
+ ##### params?
821
+
822
+ [`GotoActionOptions`](../interfaces/GotoActionOptions.md)
823
+
824
+ Navigation options
825
+
826
+ #### Returns
827
+
828
+ `Promise`\<`void` \| [`FetchResponse`](../interfaces/FetchResponse.md)\>
829
+
830
+ Promise resolving when navigation completes
831
+
832
+ #### Example
833
+
834
+ ```ts
835
+ await engine.goto('https://example.com');
836
+ ```
837
+
838
+ ***
839
+
840
+ ### headers()
841
+
842
+ #### Call Signature
843
+
844
+ > **headers**(): `Promise`\<`Record`\<`string`, `string`\>\>
845
+
846
+ Defined in: [packages/web-fetcher/src/engine/base.ts:753](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L753)
847
+
848
+ Manages HTTP headers for requests with multiple overloads.
849
+
850
+ Gets all headers.
851
+
852
+ ##### Returns
853
+
854
+ `Promise`\<`Record`\<`string`, `string`\>\>
855
+
856
+ All headers as record
857
+
858
+ Gets specific header value.
859
+
860
+ ##### Example
861
+
862
+ ```ts
863
+ const allHeaders = await engine.headers();
864
+ const userAgent = await engine.headers('user-agent');
865
+ await engine.headers({ 'x-custom': 'value' });
866
+ await engine.headers('auth', 'token');
867
+ ```
868
+
869
+ #### Call Signature
870
+
871
+ > **headers**(`name`): `Promise`\<`string`\>
872
+
873
+ Defined in: [packages/web-fetcher/src/engine/base.ts:754](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L754)
874
+
875
+ Manages HTTP headers for requests with multiple overloads.
876
+
877
+ Gets all headers.
878
+
879
+ ##### Parameters
880
+
881
+ ###### name
882
+
883
+ `string`
884
+
885
+ Header name
886
+
887
+ ##### Returns
888
+
889
+ `Promise`\<`string`\>
890
+
891
+ All headers as record
892
+
893
+ Gets specific header value.
894
+
895
+ ##### Example
896
+
897
+ ```ts
898
+ const allHeaders = await engine.headers();
899
+ const userAgent = await engine.headers('user-agent');
900
+ await engine.headers({ 'x-custom': 'value' });
901
+ await engine.headers('auth', 'token');
902
+ ```
903
+
904
+ #### Call Signature
905
+
906
+ > **headers**(`headers`, `replaced?`): `Promise`\<`boolean`\>
907
+
908
+ Defined in: [packages/web-fetcher/src/engine/base.ts:755](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L755)
909
+
910
+ Manages HTTP headers for requests with multiple overloads.
911
+
912
+ Gets all headers.
913
+
914
+ ##### Parameters
915
+
916
+ ###### headers
917
+
918
+ `Record`\<`string`, `string`\>
919
+
920
+ Headers to set
921
+
922
+ ###### replaced?
923
+
924
+ `boolean`
925
+
926
+ Whether to replace all existing headers
927
+
928
+ ##### Returns
929
+
930
+ `Promise`\<`boolean`\>
931
+
932
+ All headers as record
933
+
934
+ Gets specific header value.
935
+
936
+ ##### Example
937
+
938
+ ```ts
939
+ const allHeaders = await engine.headers();
940
+ const userAgent = await engine.headers('user-agent');
941
+ await engine.headers({ 'x-custom': 'value' });
942
+ await engine.headers('auth', 'token');
943
+ ```
944
+
945
+ #### Call Signature
946
+
947
+ > **headers**(`name`, `value`): `Promise`\<`boolean`\>
948
+
949
+ Defined in: [packages/web-fetcher/src/engine/base.ts:756](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L756)
950
+
951
+ Manages HTTP headers for requests with multiple overloads.
952
+
953
+ Gets all headers.
954
+
955
+ ##### Parameters
956
+
957
+ ###### name
958
+
959
+ `string`
960
+
961
+ Header name
962
+
963
+ ###### value
964
+
965
+ Header value or `null` to remove
966
+
967
+ `null` | `string`
968
+
969
+ ##### Returns
970
+
971
+ `Promise`\<`boolean`\>
972
+
973
+ All headers as record
974
+
975
+ Gets specific header value.
976
+
977
+ ##### Example
978
+
979
+ ```ts
980
+ const allHeaders = await engine.headers();
981
+ const userAgent = await engine.headers('user-agent');
982
+ await engine.headers({ 'x-custom': 'value' });
983
+ await engine.headers('auth', 'token');
984
+ ```
985
+
986
+ ***
987
+
988
+ ### initialize()
989
+
990
+ > **initialize**(`context`, `options?`): `Promise`\<`void`\>
991
+
992
+ Defined in: [packages/web-fetcher/src/engine/base.ts:488](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L488)
993
+
994
+ Initializes the fetch engine with provided context and options.
995
+
996
+ #### Parameters
997
+
998
+ ##### context
999
+
1000
+ [`FetchEngineContext`](../interfaces/FetchEngineContext.md)
1001
+
1002
+ Fetch engine context
1003
+
1004
+ ##### options?
1005
+
1006
+ [`BaseFetcherProperties`](../interfaces/BaseFetcherProperties.md)
1007
+
1008
+ Configuration options
1009
+
1010
+ #### Returns
1011
+
1012
+ `Promise`\<`void`\>
1013
+
1014
+ Promise resolving when initialization completes
1015
+
1016
+ #### Remarks
1017
+
1018
+ Sets up internal state and calls implementation-specific [_initialize](file:///home/riceball/Documents/mywork/public/@isdk/ai-tools/packages/web-fetcher/src/engine/cheerio.ts#L169-L204) method.
1019
+ Automatically called when creating engine via `FetchEngine.create()`.
1020
+
1021
+ ***
1022
+
1023
+ ### pause()
1024
+
1025
+ > **pause**(`message?`): `Promise`\<`void`\>
1026
+
1027
+ Defined in: [packages/web-fetcher/src/engine/base.ts:400](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L400)
1028
+
1029
+ Pauses execution, allowing for manual intervention or inspection.
1030
+
1031
+ #### Parameters
1032
+
1033
+ ##### message?
1034
+
1035
+ `string`
1036
+
1037
+ Optional message to display during pause
1038
+
1039
+ #### Returns
1040
+
1041
+ `Promise`\<`void`\>
1042
+
1043
+ Promise resolving when execution is resumed
1044
+
1045
+ #### Throws
1046
+
1047
+ When no active page context exists
1048
+
1049
+ ***
1050
+
1051
+ ### submit()
1052
+
1053
+ > **submit**(`selector?`, `options?`): `Promise`\<`void`\>
1054
+
1055
+ Defined in: [packages/web-fetcher/src/engine/base.ts:389](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L389)
1056
+
1057
+ Submits a form.
1058
+
1059
+ #### Parameters
1060
+
1061
+ ##### selector?
1062
+
1063
+ `any`
1064
+
1065
+ Optional form/submit button selector
1066
+
1067
+ ##### options?
1068
+
1069
+ [`SubmitActionOptions`](../interfaces/SubmitActionOptions.md)
1070
+
1071
+ Submission options
1072
+
1073
+ #### Returns
1074
+
1075
+ `Promise`\<`void`\>
1076
+
1077
+ Promise resolving when form is submitted
1078
+
1079
+ #### Throws
1080
+
1081
+ When no active page context exists
1082
+
1083
+ ***
1084
+
1085
+ ### waitFor()
1086
+
1087
+ > **waitFor**(`params?`): `Promise`\<`void`\>
1088
+
1089
+ Defined in: [packages/web-fetcher/src/engine/base.ts:354](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L354)
1090
+
1091
+ Waits for specified condition before continuing.
1092
+
1093
+ #### Parameters
1094
+
1095
+ ##### params?
1096
+
1097
+ [`WaitForActionOptions`](../interfaces/WaitForActionOptions.md)
1098
+
1099
+ Wait conditions
1100
+
1101
+ #### Returns
1102
+
1103
+ `Promise`\<`void`\>
1104
+
1105
+ Promise resolving when wait condition is met
1106
+
1107
+ #### Example
1108
+
1109
+ ```ts
1110
+ await engine.waitFor({ ms: 1000 }); // Wait 1 second
1111
+ await engine.waitFor({ selector: '#content' }); // Wait for element
1112
+ ```
1113
+
1114
+ ***
1115
+
1116
+ ### create()
1117
+
1118
+ > `static` **create**(`ctx`, `options?`): `Promise`\<`undefined` \| `AnyFetchEngine`\>
1119
+
1120
+ Defined in: [packages/web-fetcher/src/engine/base.ts:198](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L198)
1121
+
1122
+ Factory method to create and initialize a fetch engine instance.
1123
+
1124
+ #### Parameters
1125
+
1126
+ ##### ctx
1127
+
1128
+ [`FetchEngineContext`](../interfaces/FetchEngineContext.md)
1129
+
1130
+ Fetch engine context
1131
+
1132
+ ##### options?
1133
+
1134
+ [`BaseFetcherProperties`](../interfaces/BaseFetcherProperties.md)
1135
+
1136
+ Configuration options
1137
+
1138
+ #### Returns
1139
+
1140
+ `Promise`\<`undefined` \| `AnyFetchEngine`\>
1141
+
1142
+ Initialized fetch engine instance
1143
+
1144
+ #### Throws
1145
+
1146
+ When no suitable engine implementation is found
1147
+
1148
+ #### Remarks
1149
+
1150
+ Primary entry point for engine creation. Selects appropriate implementation based on `engine` name of the option or context.
1151
+
1152
+ ***
1153
+
1154
+ ### get()
1155
+
1156
+ > `static` **get**(`id`): `undefined` \| `AnyFetchEngineCtor`
1157
+
1158
+ Defined in: [packages/web-fetcher/src/engine/base.ts:171](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L171)
1159
+
1160
+ Retrieves a fetch engine implementation by its unique ID.
1161
+
1162
+ #### Parameters
1163
+
1164
+ ##### id
1165
+
1166
+ `string`
1167
+
1168
+ The ID of the engine to retrieve
1169
+
1170
+ #### Returns
1171
+
1172
+ `undefined` \| `AnyFetchEngineCtor`
1173
+
1174
+ Engine class if found, otherwise `undefined`
1175
+
1176
+ ***
1177
+
1178
+ ### getByMode()
1179
+
1180
+ > `static` **getByMode**(`mode`): `undefined` \| `AnyFetchEngineCtor`
1181
+
1182
+ Defined in: [packages/web-fetcher/src/engine/base.ts:181](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L181)
1183
+
1184
+ Retrieves a fetch engine implementation by execution mode.
1185
+
1186
+ #### Parameters
1187
+
1188
+ ##### mode
1189
+
1190
+ [`FetchEngineType`](../type-aliases/FetchEngineType.md)
1191
+
1192
+ Execution mode (`'http'` or `'browser'`)
1193
+
1194
+ #### Returns
1195
+
1196
+ `undefined` \| `AnyFetchEngineCtor`
1197
+
1198
+ Engine class if found, otherwise `undefined`
1199
+
1200
+ ***
1201
+
1202
+ ### register()
1203
+
1204
+ > `static` **register**(`engineClass`): `void`
1205
+
1206
+ Defined in: [packages/web-fetcher/src/engine/base.ts:158](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/engine/base.ts#L158)
1207
+
1208
+ Registers a fetch engine implementation with the global registry.
1209
+
1210
+ #### Parameters
1211
+
1212
+ ##### engineClass
1213
+
1214
+ `AnyFetchEngineCtor`
1215
+
1216
+ The engine class to register
1217
+
1218
+ #### Returns
1219
+
1220
+ `void`
1221
+
1222
+ #### Throws
1223
+
1224
+ When engine class lacks static `id` or ID is already registered
1225
+
1226
+ #### Example
1227
+
1228
+ ```ts
1229
+ FetchEngine.register(CheerioFetchEngine);
1230
+ ```