@isdk/web-fetcher 0.2.5 → 0.2.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.action.cn.md +6 -4
- package/README.action.md +6 -4
- package/README.cn.md +1 -1
- package/README.engine.cn.md +28 -0
- package/README.engine.md +28 -0
- package/README.md +1 -1
- package/dist/index.d.mts +14 -6
- package/dist/index.d.ts +14 -6
- package/dist/index.js +1 -1
- package/dist/index.mjs +1 -1
- package/docs/README.md +1 -1
- package/docs/_media/README.action.md +6 -4
- package/docs/_media/README.cn.md +1 -1
- package/docs/_media/README.engine.md +28 -0
- package/docs/classes/CheerioFetchEngine.md +83 -71
- package/docs/classes/ClickAction.md +23 -23
- package/docs/classes/ExtractAction.md +23 -23
- package/docs/classes/FetchAction.md +23 -23
- package/docs/classes/FetchEngine.md +75 -67
- package/docs/classes/FetchSession.md +25 -13
- package/docs/classes/FillAction.md +23 -23
- package/docs/classes/GetContentAction.md +23 -23
- package/docs/classes/GotoAction.md +23 -23
- package/docs/classes/PauseAction.md +23 -23
- package/docs/classes/PlaywrightFetchEngine.md +83 -71
- package/docs/classes/SubmitAction.md +23 -23
- package/docs/classes/WaitForAction.md +23 -23
- package/docs/classes/WebFetcher.md +5 -5
- package/docs/enumerations/FetchActionResultStatus.md +4 -4
- package/docs/functions/fetchWeb.md +2 -2
- package/docs/interfaces/BaseFetchActionProperties.md +25 -9
- package/docs/interfaces/BaseFetchCollectorActionProperties.md +37 -13
- package/docs/interfaces/BaseFetcherProperties.md +38 -22
- package/docs/interfaces/DispatchedEngineAction.md +4 -4
- package/docs/interfaces/ExtractActionProperties.md +33 -9
- package/docs/interfaces/FetchActionInContext.md +38 -14
- package/docs/interfaces/FetchActionProperties.md +35 -11
- package/docs/interfaces/FetchActionResult.md +6 -6
- package/docs/interfaces/FetchContext.md +57 -33
- package/docs/interfaces/FetchEngineContext.md +51 -27
- package/docs/interfaces/FetchMetadata.md +5 -5
- package/docs/interfaces/FetchResponse.md +13 -13
- package/docs/interfaces/FetchReturnTypeRegistry.md +7 -7
- package/docs/interfaces/FetchSite.md +49 -25
- package/docs/interfaces/FetcherOptions.md +49 -25
- package/docs/interfaces/GotoActionOptions.md +6 -6
- package/docs/interfaces/PendingEngineRequest.md +3 -3
- package/docs/interfaces/SubmitActionOptions.md +2 -2
- package/docs/interfaces/WaitForActionOptions.md +5 -5
- package/docs/type-aliases/BaseFetchActionOptions.md +2 -2
- package/docs/type-aliases/BaseFetchCollectorOptions.md +2 -2
- package/docs/type-aliases/BrowserEngine.md +1 -1
- package/docs/type-aliases/FetchActionCapabilities.md +1 -1
- package/docs/type-aliases/FetchActionCapabilityMode.md +1 -1
- package/docs/type-aliases/FetchActionOptions.md +2 -2
- package/docs/type-aliases/FetchEngineAction.md +1 -1
- package/docs/type-aliases/FetchEngineType.md +1 -1
- package/docs/type-aliases/FetchReturnType.md +1 -1
- package/docs/type-aliases/FetchReturnTypeFor.md +1 -1
- package/docs/type-aliases/OnFetchPauseCallback.md +1 -1
- package/docs/type-aliases/ResourceType.md +1 -1
- package/docs/variables/DefaultFetcherProperties.md +1 -1
- package/docs/variables/FetcherOptionKeys.md +1 -1
- package/package.json +1 -1
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
# Class: CheerioFetchEngine
|
|
8
8
|
|
|
9
|
-
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:20](https://github.com/isdk/web-fetcher.js/blob/
|
|
9
|
+
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:20](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/cheerio.ts#L20)
|
|
10
10
|
|
|
11
11
|
## Extends
|
|
12
12
|
|
|
@@ -28,11 +28,23 @@ Defined in: [packages/web-fetcher/src/engine/cheerio.ts:20](https://github.com/i
|
|
|
28
28
|
|
|
29
29
|
## Properties
|
|
30
30
|
|
|
31
|
+
### \_initialCookies?
|
|
32
|
+
|
|
33
|
+
> `protected` `optional` **\_initialCookies**: [`Cookie`](../interfaces/Cookie.md)[]
|
|
34
|
+
|
|
35
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:236](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L236)
|
|
36
|
+
|
|
37
|
+
#### Inherited from
|
|
38
|
+
|
|
39
|
+
[`FetchEngine`](FetchEngine.md).[`_initialCookies`](FetchEngine.md#_initialcookies)
|
|
40
|
+
|
|
41
|
+
***
|
|
42
|
+
|
|
31
43
|
### actionEmitter
|
|
32
44
|
|
|
33
45
|
> `protected` **actionEmitter**: `EventEmitter`
|
|
34
46
|
|
|
35
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
47
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:240](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L240)
|
|
36
48
|
|
|
37
49
|
#### Inherited from
|
|
38
50
|
|
|
@@ -44,7 +56,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:236](https://github.com/isd
|
|
|
44
56
|
|
|
45
57
|
> `protected` **blockedTypes**: `Set`\<`string`\>
|
|
46
58
|
|
|
47
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
59
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:244](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L244)
|
|
48
60
|
|
|
49
61
|
#### Inherited from
|
|
50
62
|
|
|
@@ -56,7 +68,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:240](https://github.com/isd
|
|
|
56
68
|
|
|
57
69
|
> `protected` `optional` **crawler**: `CheerioCrawler`
|
|
58
70
|
|
|
59
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
71
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:231](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L231)
|
|
60
72
|
|
|
61
73
|
#### Inherited from
|
|
62
74
|
|
|
@@ -68,7 +80,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:228](https://github.com/isd
|
|
|
68
80
|
|
|
69
81
|
> `protected` `optional` **ctx**: [`FetchEngineContext`](../interfaces/FetchEngineContext.md)
|
|
70
82
|
|
|
71
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
83
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:229](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L229)
|
|
72
84
|
|
|
73
85
|
#### Inherited from
|
|
74
86
|
|
|
@@ -76,11 +88,23 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:226](https://github.com/isd
|
|
|
76
88
|
|
|
77
89
|
***
|
|
78
90
|
|
|
91
|
+
### currentSession?
|
|
92
|
+
|
|
93
|
+
> `protected` `optional` **currentSession**: `Session`
|
|
94
|
+
|
|
95
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:237](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L237)
|
|
96
|
+
|
|
97
|
+
#### Inherited from
|
|
98
|
+
|
|
99
|
+
[`FetchEngine`](FetchEngine.md).[`currentSession`](FetchEngine.md#currentsession)
|
|
100
|
+
|
|
101
|
+
***
|
|
102
|
+
|
|
79
103
|
### hdrs
|
|
80
104
|
|
|
81
105
|
> `protected` **hdrs**: `Record`\<`string`, `string`\> = `{}`
|
|
82
106
|
|
|
83
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
107
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:235](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L235)
|
|
84
108
|
|
|
85
109
|
#### Inherited from
|
|
86
110
|
|
|
@@ -92,7 +116,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:232](https://github.com/isd
|
|
|
92
116
|
|
|
93
117
|
> `protected` `optional` **isCrawlerReady**: `boolean`
|
|
94
118
|
|
|
95
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
119
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:232](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L232)
|
|
96
120
|
|
|
97
121
|
#### Inherited from
|
|
98
122
|
|
|
@@ -104,7 +128,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:229](https://github.com/isd
|
|
|
104
128
|
|
|
105
129
|
> `protected` **isPageActive**: `boolean` = `false`
|
|
106
130
|
|
|
107
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
131
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:241](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L241)
|
|
108
132
|
|
|
109
133
|
#### Inherited from
|
|
110
134
|
|
|
@@ -112,23 +136,11 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:237](https://github.com/isd
|
|
|
112
136
|
|
|
113
137
|
***
|
|
114
138
|
|
|
115
|
-
### jar
|
|
116
|
-
|
|
117
|
-
> `protected` **jar**: [`Cookie`](../interfaces/Cookie.md)[] = `[]`
|
|
118
|
-
|
|
119
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:233](https://github.com/isdk/web-fetcher.js/blob/454450fb9b2c03ba1bee72ac0677a667dd8faece/src/engine/base.ts#L233)
|
|
120
|
-
|
|
121
|
-
#### Inherited from
|
|
122
|
-
|
|
123
|
-
[`FetchEngine`](FetchEngine.md).[`jar`](FetchEngine.md#jar)
|
|
124
|
-
|
|
125
|
-
***
|
|
126
|
-
|
|
127
139
|
### lastResponse?
|
|
128
140
|
|
|
129
141
|
> `protected` `optional` **lastResponse**: [`FetchResponse`](../interfaces/FetchResponse.md)
|
|
130
142
|
|
|
131
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
143
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:243](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L243)
|
|
132
144
|
|
|
133
145
|
#### Inherited from
|
|
134
146
|
|
|
@@ -140,7 +152,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:239](https://github.com/isd
|
|
|
140
152
|
|
|
141
153
|
> `protected` **navigationLock**: `PromiseLock`
|
|
142
154
|
|
|
143
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
155
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:242](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L242)
|
|
144
156
|
|
|
145
157
|
#### Inherited from
|
|
146
158
|
|
|
@@ -152,7 +164,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:238](https://github.com/isd
|
|
|
152
164
|
|
|
153
165
|
> `protected` `optional` **opts**: [`BaseFetcherProperties`](../interfaces/BaseFetcherProperties.md)
|
|
154
166
|
|
|
155
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
167
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:230](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L230)
|
|
156
168
|
|
|
157
169
|
#### Inherited from
|
|
158
170
|
|
|
@@ -164,7 +176,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:227](https://github.com/isd
|
|
|
164
176
|
|
|
165
177
|
> `protected` **pendingRequests**: `Map`\<`string`, [`PendingEngineRequest`](../interfaces/PendingEngineRequest.md)\>
|
|
166
178
|
|
|
167
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
179
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:238](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L238)
|
|
168
180
|
|
|
169
181
|
#### Inherited from
|
|
170
182
|
|
|
@@ -176,7 +188,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:234](https://github.com/isd
|
|
|
176
188
|
|
|
177
189
|
> `protected` **requestCounter**: `number` = `0`
|
|
178
190
|
|
|
179
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
191
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:239](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L239)
|
|
180
192
|
|
|
181
193
|
#### Inherited from
|
|
182
194
|
|
|
@@ -188,7 +200,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:235](https://github.com/isd
|
|
|
188
200
|
|
|
189
201
|
> `protected` `optional` **requestQueue**: `RequestQueue`
|
|
190
202
|
|
|
191
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
203
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:233](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L233)
|
|
192
204
|
|
|
193
205
|
#### Inherited from
|
|
194
206
|
|
|
@@ -200,7 +212,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:230](https://github.com/isd
|
|
|
200
212
|
|
|
201
213
|
> `readonly` `static` **id**: `"cheerio"` = `'cheerio'`
|
|
202
214
|
|
|
203
|
-
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:25](https://github.com/isdk/web-fetcher.js/blob/
|
|
215
|
+
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:25](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/cheerio.ts#L25)
|
|
204
216
|
|
|
205
217
|
Unique identifier for the engine implementation.
|
|
206
218
|
|
|
@@ -218,7 +230,7 @@ Must be defined by concrete implementations. Used for registration and lookup in
|
|
|
218
230
|
|
|
219
231
|
> `readonly` `static` **mode**: `"http"` = `'http'`
|
|
220
232
|
|
|
221
|
-
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:26](https://github.com/isdk/web-fetcher.js/blob/
|
|
233
|
+
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:26](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/cheerio.ts#L26)
|
|
222
234
|
|
|
223
235
|
Execution mode of the engine (`'http'` or `'browser'`).
|
|
224
236
|
|
|
@@ -238,7 +250,7 @@ Must be defined by concrete implementations. Indicates whether engine operates a
|
|
|
238
250
|
|
|
239
251
|
> **get** **context**(): `undefined` \| [`FetchEngineContext`](../interfaces/FetchEngineContext.md)
|
|
240
252
|
|
|
241
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
253
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:496](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L496)
|
|
242
254
|
|
|
243
255
|
Gets the fetch engine context associated with this instance.
|
|
244
256
|
|
|
@@ -258,7 +270,7 @@ Gets the fetch engine context associated with this instance.
|
|
|
258
270
|
|
|
259
271
|
> **get** **id**(): `string`
|
|
260
272
|
|
|
261
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
273
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:471](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L471)
|
|
262
274
|
|
|
263
275
|
Gets the unique identifier of this engine implementation.
|
|
264
276
|
|
|
@@ -278,7 +290,7 @@ Gets the unique identifier of this engine implementation.
|
|
|
278
290
|
|
|
279
291
|
> **get** **mode**(): [`FetchEngineType`](../type-aliases/FetchEngineType.md)
|
|
280
292
|
|
|
281
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
293
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:489](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L489)
|
|
282
294
|
|
|
283
295
|
Gets the execution mode of this engine (`'http'` or `'browser'`).
|
|
284
296
|
|
|
@@ -296,7 +308,7 @@ Gets the execution mode of this engine (`'http'` or `'browser'`).
|
|
|
296
308
|
|
|
297
309
|
> `protected` **\_buildResponse**(`context`): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
298
310
|
|
|
299
|
-
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:28](https://github.com/isdk/web-fetcher.js/blob/
|
|
311
|
+
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:28](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/cheerio.ts#L28)
|
|
300
312
|
|
|
301
313
|
**`Internal`**
|
|
302
314
|
|
|
@@ -330,7 +342,7 @@ Converts implementation-specific context (Playwright `page` or Cheerio `$`) to s
|
|
|
330
342
|
|
|
331
343
|
> `protected` `optional` **\_cleanup**(): `Promise`\<`void`\>
|
|
332
344
|
|
|
333
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
345
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:246](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L246)
|
|
334
346
|
|
|
335
347
|
#### Returns
|
|
336
348
|
|
|
@@ -346,7 +358,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:242](https://github.com/isd
|
|
|
346
358
|
|
|
347
359
|
> `protected` **\_commonCleanup**(): `Promise`\<`void`\>
|
|
348
360
|
|
|
349
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
361
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:710](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L710)
|
|
350
362
|
|
|
351
363
|
#### Returns
|
|
352
364
|
|
|
@@ -362,7 +374,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:675](https://github.com/isd
|
|
|
362
374
|
|
|
363
375
|
> `protected` **\_createCrawler**(`options`): `CheerioCrawler`
|
|
364
376
|
|
|
365
|
-
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:247](https://github.com/isdk/web-fetcher.js/blob/
|
|
377
|
+
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:247](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/cheerio.ts#L247)
|
|
366
378
|
|
|
367
379
|
**`Internal`**
|
|
368
380
|
|
|
@@ -390,7 +402,7 @@ The final crawler options.
|
|
|
390
402
|
|
|
391
403
|
> `protected` **\_executePendingActions**(`context`): `Promise`\<`void`\>
|
|
392
404
|
|
|
393
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
405
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:625](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L625)
|
|
394
406
|
|
|
395
407
|
**`Internal`**
|
|
396
408
|
|
|
@@ -443,7 +455,7 @@ If called outside valid page context window (`!this.isPageActive`)
|
|
|
443
455
|
|
|
444
456
|
> `protected` **\_extract**(`schema`, `context`): `Promise`\<`any`\>
|
|
445
457
|
|
|
446
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
458
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:251](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L251)
|
|
447
459
|
|
|
448
460
|
#### Parameters
|
|
449
461
|
|
|
@@ -469,7 +481,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:247](https://github.com/isd
|
|
|
469
481
|
|
|
470
482
|
> `protected` **\_extractValue**(`schema`, `context`): `Promise`\<`any`\>
|
|
471
483
|
|
|
472
|
-
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:64](https://github.com/isdk/web-fetcher.js/blob/
|
|
484
|
+
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:64](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/cheerio.ts#L64)
|
|
473
485
|
|
|
474
486
|
#### Parameters
|
|
475
487
|
|
|
@@ -497,7 +509,7 @@ Defined in: [packages/web-fetcher/src/engine/cheerio.ts:64](https://github.com/i
|
|
|
497
509
|
|
|
498
510
|
> `protected` **\_getSpecificCrawlerOptions**(`ctx`): `CheerioCrawlerOptions`
|
|
499
511
|
|
|
500
|
-
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:251](https://github.com/isdk/web-fetcher.js/blob/
|
|
512
|
+
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:251](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/cheerio.ts#L251)
|
|
501
513
|
|
|
502
514
|
**`Internal`**
|
|
503
515
|
|
|
@@ -525,7 +537,7 @@ The fetch engine context.
|
|
|
525
537
|
|
|
526
538
|
> `protected` **\_normalizeSchema**(`schema`): `ExtractSchema`
|
|
527
539
|
|
|
528
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
540
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:427](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L427)
|
|
529
541
|
|
|
530
542
|
#### Parameters
|
|
531
543
|
|
|
@@ -547,7 +559,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:423](https://github.com/isd
|
|
|
547
559
|
|
|
548
560
|
> `protected` **\_querySelectorAll**(`context`, `selector`): `Promise`\<`any`[]\>
|
|
549
561
|
|
|
550
|
-
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:59](https://github.com/isdk/web-fetcher.js/blob/
|
|
562
|
+
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:59](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/cheerio.ts#L59)
|
|
551
563
|
|
|
552
564
|
#### Parameters
|
|
553
565
|
|
|
@@ -579,7 +591,7 @@ Defined in: [packages/web-fetcher/src/engine/cheerio.ts:59](https://github.com/i
|
|
|
579
591
|
|
|
580
592
|
> `protected` **\_sharedFailedRequestHandler**(`context`, `error?`): `Promise`\<`void`\>
|
|
581
593
|
|
|
582
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
594
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:677](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L677)
|
|
583
595
|
|
|
584
596
|
#### Parameters
|
|
585
597
|
|
|
@@ -605,7 +617,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:642](https://github.com/isd
|
|
|
605
617
|
|
|
606
618
|
> `protected` **\_sharedRequestHandler**(`context`): `Promise`\<`void`\>
|
|
607
619
|
|
|
608
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
620
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:648](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L648)
|
|
609
621
|
|
|
610
622
|
#### Parameters
|
|
611
623
|
|
|
@@ -627,7 +639,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:614](https://github.com/isd
|
|
|
627
639
|
|
|
628
640
|
> **blockResources**(`types`, `overwrite?`): `Promise`\<`number`\>
|
|
629
641
|
|
|
630
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
642
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:754](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L754)
|
|
631
643
|
|
|
632
644
|
Blocks specified resource types from loading.
|
|
633
645
|
|
|
@@ -668,7 +680,7 @@ await engine.blockResources(['script'], true); // Replace existing
|
|
|
668
680
|
|
|
669
681
|
> `protected` **buildResponse**(`context`): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
670
682
|
|
|
671
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
683
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:320](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L320)
|
|
672
684
|
|
|
673
685
|
#### Parameters
|
|
674
686
|
|
|
@@ -690,7 +702,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:316](https://github.com/isd
|
|
|
690
702
|
|
|
691
703
|
> **cleanup**(): `Promise`\<`void`\>
|
|
692
704
|
|
|
693
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
705
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:589](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L589)
|
|
694
706
|
|
|
695
707
|
#### Returns
|
|
696
708
|
|
|
@@ -706,7 +718,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:555](https://github.com/isd
|
|
|
706
718
|
|
|
707
719
|
> **click**(`selector`): `Promise`\<`void`\>
|
|
708
720
|
|
|
709
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
721
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:377](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L377)
|
|
710
722
|
|
|
711
723
|
Clicks on element matching selector.
|
|
712
724
|
|
|
@@ -740,7 +752,7 @@ When no active page context exists
|
|
|
740
752
|
|
|
741
753
|
> **cookies**(): `Promise`\<[`Cookie`](../interfaces/Cookie.md)[]\>
|
|
742
754
|
|
|
743
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
755
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:865](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L865)
|
|
744
756
|
|
|
745
757
|
Manages cookies for current session with multiple overloads.
|
|
746
758
|
|
|
@@ -769,7 +781,7 @@ await engine.cookies([{ name: 'session', value: '123' }]);
|
|
|
769
781
|
|
|
770
782
|
> **cookies**(`cookies`): `Promise`\<`boolean`\>
|
|
771
783
|
|
|
772
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
784
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:866](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L866)
|
|
773
785
|
|
|
774
786
|
Manages cookies for current session with multiple overloads.
|
|
775
787
|
|
|
@@ -808,7 +820,7 @@ await engine.cookies([{ name: 'session', value: '123' }]);
|
|
|
808
820
|
|
|
809
821
|
> `protected` **dispatchAction**\<`T`\>(`action`): `Promise`\<`T`\>
|
|
810
822
|
|
|
811
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
823
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:693](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L693)
|
|
812
824
|
|
|
813
825
|
#### Type Parameters
|
|
814
826
|
|
|
@@ -836,7 +848,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:658](https://github.com/isd
|
|
|
836
848
|
|
|
837
849
|
> **dispose**(): `Promise`\<`void`\>
|
|
838
850
|
|
|
839
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
851
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:901](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L901)
|
|
840
852
|
|
|
841
853
|
Disposes of engine, cleaning up all resources.
|
|
842
854
|
|
|
@@ -856,7 +868,7 @@ Promise resolving when disposal completes
|
|
|
856
868
|
|
|
857
869
|
> `protected` **executeAction**(`context`, `action`): `Promise`\<`any`\>
|
|
858
870
|
|
|
859
|
-
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:92](https://github.com/isdk/web-fetcher.js/blob/
|
|
871
|
+
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:92](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/cheerio.ts#L92)
|
|
860
872
|
|
|
861
873
|
**`Internal`**
|
|
862
874
|
|
|
@@ -896,7 +908,7 @@ Handles specific user interactions using underlying technology (Playwright/Cheer
|
|
|
896
908
|
|
|
897
909
|
> **extract**\<`T`\>(`schema`): `Promise`\<`T`\>
|
|
898
910
|
|
|
899
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
911
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:422](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L422)
|
|
900
912
|
|
|
901
913
|
Extracts structured data from the current page content.
|
|
902
914
|
|
|
@@ -930,7 +942,7 @@ A promise that resolves to an object with the extracted data.
|
|
|
930
942
|
|
|
931
943
|
> **fill**(`selector`, `value`): `Promise`\<`void`\>
|
|
932
944
|
|
|
933
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
945
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:389](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L389)
|
|
934
946
|
|
|
935
947
|
Fills input element with specified value.
|
|
936
948
|
|
|
@@ -968,7 +980,7 @@ When no active page context exists
|
|
|
968
980
|
|
|
969
981
|
> **getContent**(): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
970
982
|
|
|
971
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
983
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:768](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L768)
|
|
972
984
|
|
|
973
985
|
Gets content of current page.
|
|
974
986
|
|
|
@@ -990,16 +1002,16 @@ When no content has been fetched yet
|
|
|
990
1002
|
|
|
991
1003
|
### getState()
|
|
992
1004
|
|
|
993
|
-
> **getState**(): `Promise`\<\{ `cookies`: [`Cookie`](../interfaces/Cookie.md)[]; \}\>
|
|
1005
|
+
> **getState**(): `Promise`\<\{ `cookies`: [`Cookie`](../interfaces/Cookie.md)[]; `sessionState?`: `any`; \}\>
|
|
994
1006
|
|
|
995
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1007
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:479](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L479)
|
|
996
1008
|
|
|
997
1009
|
Returns the current state of the engine (cookies)
|
|
998
1010
|
that can be used to restore the session later.
|
|
999
1011
|
|
|
1000
1012
|
#### Returns
|
|
1001
1013
|
|
|
1002
|
-
`Promise`\<\{ `cookies`: [`Cookie`](../interfaces/Cookie.md)[]; \}\>
|
|
1014
|
+
`Promise`\<\{ `cookies`: [`Cookie`](../interfaces/Cookie.md)[]; `sessionState?`: `any`; \}\>
|
|
1003
1015
|
|
|
1004
1016
|
#### Inherited from
|
|
1005
1017
|
|
|
@@ -1011,7 +1023,7 @@ that can be used to restore the session later.
|
|
|
1011
1023
|
|
|
1012
1024
|
> **goto**(`url`, `params?`): `Promise`\<`void` \| [`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
1013
1025
|
|
|
1014
|
-
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:
|
|
1026
|
+
Defined in: [packages/web-fetcher/src/engine/cheerio.ts:271](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/cheerio.ts#L271)
|
|
1015
1027
|
|
|
1016
1028
|
Navigates to the specified URL.
|
|
1017
1029
|
|
|
@@ -1053,7 +1065,7 @@ await engine.goto('https://example.com');
|
|
|
1053
1065
|
|
|
1054
1066
|
> **headers**(): `Promise`\<`Record`\<`string`, `string`\>\>
|
|
1055
1067
|
|
|
1056
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1068
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:807](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L807)
|
|
1057
1069
|
|
|
1058
1070
|
Manages HTTP headers for requests with multiple overloads.
|
|
1059
1071
|
|
|
@@ -1084,7 +1096,7 @@ await engine.headers('auth', 'token');
|
|
|
1084
1096
|
|
|
1085
1097
|
> **headers**(`name`): `Promise`\<`string`\>
|
|
1086
1098
|
|
|
1087
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1099
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:808](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L808)
|
|
1088
1100
|
|
|
1089
1101
|
Manages HTTP headers for requests with multiple overloads.
|
|
1090
1102
|
|
|
@@ -1123,7 +1135,7 @@ await engine.headers('auth', 'token');
|
|
|
1123
1135
|
|
|
1124
1136
|
> **headers**(`headers`, `replaced?`): `Promise`\<`boolean`\>
|
|
1125
1137
|
|
|
1126
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1138
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:809](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L809)
|
|
1127
1139
|
|
|
1128
1140
|
Manages HTTP headers for requests with multiple overloads.
|
|
1129
1141
|
|
|
@@ -1168,7 +1180,7 @@ await engine.headers('auth', 'token');
|
|
|
1168
1180
|
|
|
1169
1181
|
> **headers**(`name`, `value`): `Promise`\<`boolean`\>
|
|
1170
1182
|
|
|
1171
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1183
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:810](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L810)
|
|
1172
1184
|
|
|
1173
1185
|
Manages HTTP headers for requests with multiple overloads.
|
|
1174
1186
|
|
|
@@ -1215,7 +1227,7 @@ await engine.headers('auth', 'token');
|
|
|
1215
1227
|
|
|
1216
1228
|
> **initialize**(`context`, `options?`): `Promise`\<`void`\>
|
|
1217
1229
|
|
|
1218
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1230
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:511](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L511)
|
|
1219
1231
|
|
|
1220
1232
|
Initializes the fetch engine with provided context and options.
|
|
1221
1233
|
|
|
@@ -1254,7 +1266,7 @@ Automatically called when creating engine via `FetchEngine.create()`.
|
|
|
1254
1266
|
|
|
1255
1267
|
> **pause**(`message?`): `Promise`\<`void`\>
|
|
1256
1268
|
|
|
1257
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1269
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:412](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L412)
|
|
1258
1270
|
|
|
1259
1271
|
Pauses execution, allowing for manual intervention or inspection.
|
|
1260
1272
|
|
|
@@ -1286,7 +1298,7 @@ When no active page context exists
|
|
|
1286
1298
|
|
|
1287
1299
|
> **submit**(`selector?`, `options?`): `Promise`\<`void`\>
|
|
1288
1300
|
|
|
1289
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1301
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:401](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L401)
|
|
1290
1302
|
|
|
1291
1303
|
Submits a form.
|
|
1292
1304
|
|
|
@@ -1324,7 +1336,7 @@ When no active page context exists
|
|
|
1324
1336
|
|
|
1325
1337
|
> **waitFor**(`params?`): `Promise`\<`void`\>
|
|
1326
1338
|
|
|
1327
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1339
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:366](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L366)
|
|
1328
1340
|
|
|
1329
1341
|
Waits for specified condition before continuing.
|
|
1330
1342
|
|
|
@@ -1359,7 +1371,7 @@ await engine.waitFor({ selector: '#content' }); // Wait for element
|
|
|
1359
1371
|
|
|
1360
1372
|
> `static` **create**(`ctx`, `options?`): `Promise`\<`undefined` \| `AnyFetchEngine`\>
|
|
1361
1373
|
|
|
1362
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1374
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:202](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L202)
|
|
1363
1375
|
|
|
1364
1376
|
Factory method to create and initialize a fetch engine instance.
|
|
1365
1377
|
|
|
@@ -1401,7 +1413,7 @@ Primary entry point for engine creation. Selects appropriate implementation base
|
|
|
1401
1413
|
|
|
1402
1414
|
> `static` **get**(`id`): `undefined` \| `AnyFetchEngineCtor`
|
|
1403
1415
|
|
|
1404
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1416
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:175](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L175)
|
|
1405
1417
|
|
|
1406
1418
|
Retrieves a fetch engine implementation by its unique ID.
|
|
1407
1419
|
|
|
@@ -1429,7 +1441,7 @@ Engine class if found, otherwise `undefined`
|
|
|
1429
1441
|
|
|
1430
1442
|
> `static` **getByMode**(`mode`): `undefined` \| `AnyFetchEngineCtor`
|
|
1431
1443
|
|
|
1432
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1444
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:185](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L185)
|
|
1433
1445
|
|
|
1434
1446
|
Retrieves a fetch engine implementation by execution mode.
|
|
1435
1447
|
|
|
@@ -1457,7 +1469,7 @@ Engine class if found, otherwise `undefined`
|
|
|
1457
1469
|
|
|
1458
1470
|
> `static` **register**(`engineClass`): `void`
|
|
1459
1471
|
|
|
1460
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1472
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:162](https://github.com/isdk/web-fetcher.js/blob/67ac2abdb01356cc0cf363248f4ef8f77c28d78f/src/engine/base.ts#L162)
|
|
1461
1473
|
|
|
1462
1474
|
Registers a fetch engine implementation with the global registry.
|
|
1463
1475
|
|