@isdk/web-fetcher 0.2.12 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.action.cn.md +312 -11
- package/README.action.md +324 -11
- package/README.cn.md +13 -11
- package/README.engine.cn.md +96 -13
- package/README.engine.md +93 -13
- package/README.md +11 -9
- package/dist/index.d.mts +521 -50
- package/dist/index.d.ts +521 -50
- package/dist/index.js +1 -1
- package/dist/index.mjs +1 -1
- package/docs/README.md +11 -9
- package/docs/_media/README.action.md +324 -11
- package/docs/_media/README.cn.md +13 -11
- package/docs/_media/README.engine.md +93 -13
- package/docs/classes/CheerioFetchEngine.md +647 -125
- package/docs/classes/ClickAction.md +33 -33
- package/docs/classes/EvaluateAction.md +559 -0
- package/docs/classes/ExtractAction.md +33 -33
- package/docs/classes/FetchAction.md +35 -33
- package/docs/classes/FetchEngine.md +528 -122
- package/docs/classes/FetchSession.md +38 -16
- package/docs/classes/FillAction.md +33 -33
- package/docs/classes/GetContentAction.md +33 -33
- package/docs/classes/GotoAction.md +33 -33
- package/docs/classes/PauseAction.md +33 -33
- package/docs/classes/PlaywrightFetchEngine.md +570 -122
- package/docs/classes/SubmitAction.md +33 -33
- package/docs/classes/TrimAction.md +533 -0
- package/docs/classes/WaitForAction.md +33 -33
- package/docs/classes/WebFetcher.md +9 -9
- package/docs/enumerations/FetchActionResultStatus.md +4 -4
- package/docs/functions/fetchWeb.md +6 -6
- package/docs/globals.md +6 -0
- package/docs/interfaces/BaseFetchActionProperties.md +12 -12
- package/docs/interfaces/BaseFetchCollectorActionProperties.md +16 -16
- package/docs/interfaces/BaseFetcherProperties.md +28 -28
- package/docs/interfaces/DispatchedEngineAction.md +4 -4
- package/docs/interfaces/EvaluateActionOptions.md +81 -0
- package/docs/interfaces/ExtractActionProperties.md +12 -12
- package/docs/interfaces/FetchActionInContext.md +15 -15
- package/docs/interfaces/FetchActionProperties.md +13 -13
- package/docs/interfaces/FetchActionResult.md +6 -6
- package/docs/interfaces/FetchContext.md +38 -38
- package/docs/interfaces/FetchEngineContext.md +33 -33
- package/docs/interfaces/FetchMetadata.md +5 -5
- package/docs/interfaces/FetchResponse.md +14 -14
- package/docs/interfaces/FetchReturnTypeRegistry.md +8 -8
- package/docs/interfaces/FetchSite.md +31 -31
- package/docs/interfaces/FetcherOptions.md +30 -30
- package/docs/interfaces/GotoActionOptions.md +6 -6
- package/docs/interfaces/PendingEngineRequest.md +3 -3
- package/docs/interfaces/StorageOptions.md +5 -5
- package/docs/interfaces/SubmitActionOptions.md +2 -2
- package/docs/interfaces/TrimActionOptions.md +27 -0
- package/docs/interfaces/WaitForActionOptions.md +5 -5
- package/docs/type-aliases/BaseFetchActionOptions.md +1 -1
- package/docs/type-aliases/BaseFetchCollectorOptions.md +1 -1
- package/docs/type-aliases/BrowserEngine.md +1 -1
- package/docs/type-aliases/FetchActionCapabilities.md +1 -1
- package/docs/type-aliases/FetchActionCapabilityMode.md +1 -1
- package/docs/type-aliases/FetchActionOptions.md +1 -1
- package/docs/type-aliases/FetchEngineAction.md +2 -2
- package/docs/type-aliases/FetchEngineType.md +1 -1
- package/docs/type-aliases/FetchReturnType.md +1 -1
- package/docs/type-aliases/FetchReturnTypeFor.md +1 -1
- package/docs/type-aliases/OnFetchPauseCallback.md +1 -1
- package/docs/type-aliases/ResourceType.md +1 -1
- package/docs/type-aliases/TrimPreset.md +13 -0
- package/docs/variables/DefaultFetcherProperties.md +1 -1
- package/docs/variables/FetcherOptionKeys.md +1 -1
- package/docs/variables/TRIM_PRESETS.md +11 -0
- package/package.json +4 -4
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
# Abstract Class: FetchEngine\<TContext, TCrawler, TOptions\>
|
|
8
8
|
|
|
9
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
9
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:272](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L272)
|
|
10
10
|
|
|
11
11
|
## Extended by
|
|
12
12
|
|
|
@@ -27,6 +27,10 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:172](https://github.com/isd
|
|
|
27
27
|
|
|
28
28
|
`TOptions` *extends* `BasicCrawlerOptions`\<`TContext`\> = `any`
|
|
29
29
|
|
|
30
|
+
## Implements
|
|
31
|
+
|
|
32
|
+
- `IExtractEngine`
|
|
33
|
+
|
|
30
34
|
## Constructors
|
|
31
35
|
|
|
32
36
|
### Constructor
|
|
@@ -43,7 +47,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:172](https://github.com/isd
|
|
|
43
47
|
|
|
44
48
|
> `protected` `optional` **\_initialCookies**: [`Cookie`](../interfaces/Cookie.md)[]
|
|
45
49
|
|
|
46
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
50
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:379](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L379)
|
|
47
51
|
|
|
48
52
|
***
|
|
49
53
|
|
|
@@ -51,7 +55,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:278](https://github.com/isd
|
|
|
51
55
|
|
|
52
56
|
> `protected` **\_initializedSessions**: `Set`\<`string`\>
|
|
53
57
|
|
|
54
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
58
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:380](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L380)
|
|
55
59
|
|
|
56
60
|
***
|
|
57
61
|
|
|
@@ -59,7 +63,23 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:279](https://github.com/isd
|
|
|
59
63
|
|
|
60
64
|
> `protected` **actionEmitter**: `EventEmitter`
|
|
61
65
|
|
|
62
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
66
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:384](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L384)
|
|
67
|
+
|
|
68
|
+
***
|
|
69
|
+
|
|
70
|
+
### actionQueue
|
|
71
|
+
|
|
72
|
+
> `protected` **actionQueue**: [`DispatchedEngineAction`](../interfaces/DispatchedEngineAction.md)[] = `[]`
|
|
73
|
+
|
|
74
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:391](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L391)
|
|
75
|
+
|
|
76
|
+
***
|
|
77
|
+
|
|
78
|
+
### activeContext?
|
|
79
|
+
|
|
80
|
+
> `protected` `optional` **activeContext**: `TContext`
|
|
81
|
+
|
|
82
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:388](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L388)
|
|
63
83
|
|
|
64
84
|
***
|
|
65
85
|
|
|
@@ -67,7 +87,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:283](https://github.com/isd
|
|
|
67
87
|
|
|
68
88
|
> `protected` **blockedTypes**: `Set`\<`string`\>
|
|
69
89
|
|
|
70
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
90
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:393](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L393)
|
|
71
91
|
|
|
72
92
|
***
|
|
73
93
|
|
|
@@ -75,7 +95,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:288](https://github.com/isd
|
|
|
75
95
|
|
|
76
96
|
> `protected` `optional` **config**: `Configuration`
|
|
77
97
|
|
|
78
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
98
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:373](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L373)
|
|
79
99
|
|
|
80
100
|
***
|
|
81
101
|
|
|
@@ -83,7 +103,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:272](https://github.com/isd
|
|
|
83
103
|
|
|
84
104
|
> `protected` `optional` **crawler**: `TCrawler`
|
|
85
105
|
|
|
86
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
106
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:370](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L370)
|
|
87
107
|
|
|
88
108
|
***
|
|
89
109
|
|
|
@@ -91,7 +111,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:269](https://github.com/isd
|
|
|
91
111
|
|
|
92
112
|
> `protected` `optional` **crawlerRunPromise**: `Promise`\<`FinalStatistics`\>
|
|
93
113
|
|
|
94
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
114
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:372](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L372)
|
|
95
115
|
|
|
96
116
|
***
|
|
97
117
|
|
|
@@ -99,7 +119,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:271](https://github.com/isd
|
|
|
99
119
|
|
|
100
120
|
> `protected` `optional` **ctx**: [`FetchEngineContext`](../interfaces/FetchEngineContext.md)
|
|
101
121
|
|
|
102
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
122
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:368](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L368)
|
|
103
123
|
|
|
104
124
|
***
|
|
105
125
|
|
|
@@ -107,7 +127,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:267](https://github.com/isd
|
|
|
107
127
|
|
|
108
128
|
> `protected` `optional` **currentSession**: `Session`
|
|
109
129
|
|
|
110
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
130
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:381](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L381)
|
|
111
131
|
|
|
112
132
|
***
|
|
113
133
|
|
|
@@ -115,7 +135,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:280](https://github.com/isd
|
|
|
115
135
|
|
|
116
136
|
> `protected` **hdrs**: `Record`\<`string`, `string`\> = `{}`
|
|
117
137
|
|
|
118
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
138
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:378](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L378)
|
|
119
139
|
|
|
120
140
|
***
|
|
121
141
|
|
|
@@ -123,7 +143,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:277](https://github.com/isd
|
|
|
123
143
|
|
|
124
144
|
> `protected` `optional` **isCrawlerReady**: `boolean`
|
|
125
145
|
|
|
126
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
146
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:371](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L371)
|
|
127
147
|
|
|
128
148
|
***
|
|
129
149
|
|
|
@@ -131,7 +151,15 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:270](https://github.com/isd
|
|
|
131
151
|
|
|
132
152
|
> `protected` **isEngineDisposed**: `boolean` = `false`
|
|
133
153
|
|
|
134
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
154
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:386](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L386)
|
|
155
|
+
|
|
156
|
+
***
|
|
157
|
+
|
|
158
|
+
### isExecutingAction
|
|
159
|
+
|
|
160
|
+
> `protected` **isExecutingAction**: `boolean` = `false`
|
|
161
|
+
|
|
162
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:389](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L389)
|
|
135
163
|
|
|
136
164
|
***
|
|
137
165
|
|
|
@@ -139,7 +167,15 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:285](https://github.com/isd
|
|
|
139
167
|
|
|
140
168
|
> `protected` **isPageActive**: `boolean` = `false`
|
|
141
169
|
|
|
142
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
170
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:385](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L385)
|
|
171
|
+
|
|
172
|
+
***
|
|
173
|
+
|
|
174
|
+
### isProcessingActionLoop
|
|
175
|
+
|
|
176
|
+
> `protected` **isProcessingActionLoop**: `boolean` = `false`
|
|
177
|
+
|
|
178
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:392](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L392)
|
|
143
179
|
|
|
144
180
|
***
|
|
145
181
|
|
|
@@ -147,7 +183,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:284](https://github.com/isd
|
|
|
147
183
|
|
|
148
184
|
> `protected` `optional` **kvStore**: `KeyValueStore`
|
|
149
185
|
|
|
150
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
186
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:375](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L375)
|
|
151
187
|
|
|
152
188
|
***
|
|
153
189
|
|
|
@@ -155,7 +191,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:274](https://github.com/isd
|
|
|
155
191
|
|
|
156
192
|
> `protected` `optional` **lastResponse**: [`FetchResponse`](../interfaces/FetchResponse.md)
|
|
157
193
|
|
|
158
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
194
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:390](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L390)
|
|
159
195
|
|
|
160
196
|
***
|
|
161
197
|
|
|
@@ -163,7 +199,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:287](https://github.com/isd
|
|
|
163
199
|
|
|
164
200
|
> `protected` **navigationLock**: `PromiseLock`
|
|
165
201
|
|
|
166
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
202
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:387](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L387)
|
|
167
203
|
|
|
168
204
|
***
|
|
169
205
|
|
|
@@ -171,7 +207,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:286](https://github.com/isd
|
|
|
171
207
|
|
|
172
208
|
> `protected` `optional` **opts**: [`BaseFetcherProperties`](../interfaces/BaseFetcherProperties.md)
|
|
173
209
|
|
|
174
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
210
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:369](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L369)
|
|
175
211
|
|
|
176
212
|
***
|
|
177
213
|
|
|
@@ -179,7 +215,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:268](https://github.com/isd
|
|
|
179
215
|
|
|
180
216
|
> `protected` **pendingRequests**: `Map`\<`string`, [`PendingEngineRequest`](../interfaces/PendingEngineRequest.md)\>
|
|
181
217
|
|
|
182
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
218
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:382](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L382)
|
|
183
219
|
|
|
184
220
|
***
|
|
185
221
|
|
|
@@ -187,7 +223,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:281](https://github.com/isd
|
|
|
187
223
|
|
|
188
224
|
> `protected` `optional` **proxyConfiguration**: `ProxyConfiguration`
|
|
189
225
|
|
|
190
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
226
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:376](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L376)
|
|
191
227
|
|
|
192
228
|
***
|
|
193
229
|
|
|
@@ -195,7 +231,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:275](https://github.com/isd
|
|
|
195
231
|
|
|
196
232
|
> `protected` **requestCounter**: `number` = `0`
|
|
197
233
|
|
|
198
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
234
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:383](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L383)
|
|
199
235
|
|
|
200
236
|
***
|
|
201
237
|
|
|
@@ -203,7 +239,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:282](https://github.com/isd
|
|
|
203
239
|
|
|
204
240
|
> `protected` `optional` **requestQueue**: `RequestQueue`
|
|
205
241
|
|
|
206
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
242
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:374](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L374)
|
|
207
243
|
|
|
208
244
|
***
|
|
209
245
|
|
|
@@ -211,7 +247,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:273](https://github.com/isd
|
|
|
211
247
|
|
|
212
248
|
> `readonly` `static` **id**: `string`
|
|
213
249
|
|
|
214
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
250
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:358](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L358)
|
|
215
251
|
|
|
216
252
|
Unique identifier for the engine implementation.
|
|
217
253
|
|
|
@@ -225,7 +261,7 @@ Must be defined by concrete implementations. Used for registration and lookup in
|
|
|
225
261
|
|
|
226
262
|
> `readonly` `static` **mode**: [`FetchEngineType`](../type-aliases/FetchEngineType.md)
|
|
227
263
|
|
|
228
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
264
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:366](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L366)
|
|
229
265
|
|
|
230
266
|
Execution mode of the engine (`'http'` or `'browser'`).
|
|
231
267
|
|
|
@@ -239,15 +275,15 @@ Must be defined by concrete implementations. Indicates whether engine operates a
|
|
|
239
275
|
|
|
240
276
|
#### Get Signature
|
|
241
277
|
|
|
242
|
-
> **get** **context**():
|
|
278
|
+
> **get** **context**(): [`FetchEngineContext`](../interfaces/FetchEngineContext.md) \| `undefined`
|
|
243
279
|
|
|
244
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
280
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:846](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L846)
|
|
245
281
|
|
|
246
282
|
Gets the fetch engine context associated with this instance.
|
|
247
283
|
|
|
248
284
|
##### Returns
|
|
249
285
|
|
|
250
|
-
|
|
286
|
+
[`FetchEngineContext`](../interfaces/FetchEngineContext.md) \| `undefined`
|
|
251
287
|
|
|
252
288
|
***
|
|
253
289
|
|
|
@@ -257,7 +293,7 @@ Gets the fetch engine context associated with this instance.
|
|
|
257
293
|
|
|
258
294
|
> **get** **id**(): `string`
|
|
259
295
|
|
|
260
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
296
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:821](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L821)
|
|
261
297
|
|
|
262
298
|
Gets the unique identifier of this engine implementation.
|
|
263
299
|
|
|
@@ -273,7 +309,7 @@ Gets the unique identifier of this engine implementation.
|
|
|
273
309
|
|
|
274
310
|
> **get** **mode**(): [`FetchEngineType`](../type-aliases/FetchEngineType.md)
|
|
275
311
|
|
|
276
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
312
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:839](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L839)
|
|
277
313
|
|
|
278
314
|
Gets the execution mode of this engine (`'http'` or `'browser'`).
|
|
279
315
|
|
|
@@ -287,7 +323,7 @@ Gets the execution mode of this engine (`'http'` or `'browser'`).
|
|
|
287
323
|
|
|
288
324
|
> `abstract` `protected` **\_buildResponse**(`context`): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
289
325
|
|
|
290
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
326
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:637](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L637)
|
|
291
327
|
|
|
292
328
|
**`Internal`**
|
|
293
329
|
|
|
@@ -317,7 +353,7 @@ Converts implementation-specific context (Playwright `page` or Cheerio `$`) to s
|
|
|
317
353
|
|
|
318
354
|
> `protected` `optional` **\_cleanup**(): `Promise`\<`void`\>
|
|
319
355
|
|
|
320
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
356
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:399](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L399)
|
|
321
357
|
|
|
322
358
|
#### Returns
|
|
323
359
|
|
|
@@ -329,7 +365,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:290](https://github.com/isd
|
|
|
329
365
|
|
|
330
366
|
> `protected` **\_commonCleanup**(): `Promise`\<`void`\>
|
|
331
367
|
|
|
332
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
368
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1237](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1237)
|
|
333
369
|
|
|
334
370
|
#### Returns
|
|
335
371
|
|
|
@@ -337,11 +373,51 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:1239](https://github.com/is
|
|
|
337
373
|
|
|
338
374
|
***
|
|
339
375
|
|
|
376
|
+
### \_contains()
|
|
377
|
+
|
|
378
|
+
> `abstract` **\_contains**(`container`, `element`): `Promise`\<`boolean`\>
|
|
379
|
+
|
|
380
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:511](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L511)
|
|
381
|
+
|
|
382
|
+
**`Internal`**
|
|
383
|
+
|
|
384
|
+
Checks if the `container` scope contains the `element` scope.
|
|
385
|
+
|
|
386
|
+
#### Parameters
|
|
387
|
+
|
|
388
|
+
##### container
|
|
389
|
+
|
|
390
|
+
`any`
|
|
391
|
+
|
|
392
|
+
The potential ancestor element.
|
|
393
|
+
|
|
394
|
+
##### element
|
|
395
|
+
|
|
396
|
+
`any`
|
|
397
|
+
|
|
398
|
+
The potential descendant element.
|
|
399
|
+
|
|
400
|
+
#### Returns
|
|
401
|
+
|
|
402
|
+
`Promise`\<`boolean`\>
|
|
403
|
+
|
|
404
|
+
A promise resolving to `true` if `container` contains `element`.
|
|
405
|
+
|
|
406
|
+
#### See
|
|
407
|
+
|
|
408
|
+
IExtractEngine.\_contains for implementation details.
|
|
409
|
+
|
|
410
|
+
#### Implementation of
|
|
411
|
+
|
|
412
|
+
`IExtractEngine._contains`
|
|
413
|
+
|
|
414
|
+
***
|
|
415
|
+
|
|
340
416
|
### \_createCrawler()
|
|
341
417
|
|
|
342
418
|
> `abstract` `protected` **\_createCrawler**(`options`, `config?`): `TCrawler`
|
|
343
419
|
|
|
344
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
420
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:614](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L614)
|
|
345
421
|
|
|
346
422
|
**`Internal`**
|
|
347
423
|
|
|
@@ -369,7 +445,7 @@ The final crawler options.
|
|
|
369
445
|
|
|
370
446
|
> `protected` **\_executePendingActions**(`context`): `Promise`\<`void`\>
|
|
371
447
|
|
|
372
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
448
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1089](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1089)
|
|
373
449
|
|
|
374
450
|
**`Internal`**
|
|
375
451
|
|
|
@@ -416,9 +492,9 @@ If called outside valid page context window (`!this.isPageActive`)
|
|
|
416
492
|
|
|
417
493
|
### \_extract()
|
|
418
494
|
|
|
419
|
-
> `protected` **\_extract**(`schema`, `
|
|
495
|
+
> `protected` **\_extract**(`schema`, `scope`, `parentStrict?`): `Promise`\<`any`\>
|
|
420
496
|
|
|
421
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
497
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:542](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L542)
|
|
422
498
|
|
|
423
499
|
#### Parameters
|
|
424
500
|
|
|
@@ -426,10 +502,14 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:365](https://github.com/isd
|
|
|
426
502
|
|
|
427
503
|
`ExtractSchema`
|
|
428
504
|
|
|
429
|
-
#####
|
|
505
|
+
##### scope
|
|
430
506
|
|
|
431
507
|
`any`
|
|
432
508
|
|
|
509
|
+
##### parentStrict?
|
|
510
|
+
|
|
511
|
+
`boolean`
|
|
512
|
+
|
|
433
513
|
#### Returns
|
|
434
514
|
|
|
435
515
|
`Promise`\<`any`\>
|
|
@@ -438,9 +518,9 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:365](https://github.com/isd
|
|
|
438
518
|
|
|
439
519
|
### \_extractColumnar()
|
|
440
520
|
|
|
441
|
-
> `protected` **\_extractColumnar**(`schema`, `container`, `opts?`): `Promise`\<`
|
|
521
|
+
> `protected` **\_extractColumnar**(`schema`, `container`, `opts?`): `Promise`\<`any`[] \| `null`\>
|
|
442
522
|
|
|
443
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
523
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:584](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L584)
|
|
444
524
|
|
|
445
525
|
**`Internal`**
|
|
446
526
|
|
|
@@ -468,7 +548,7 @@ Columnar extraction options (strict, inference).
|
|
|
468
548
|
|
|
469
549
|
#### Returns
|
|
470
550
|
|
|
471
|
-
`Promise`\<`
|
|
551
|
+
`Promise`\<`any`[] \| `null`\>
|
|
472
552
|
|
|
473
553
|
An array of extracted items, or null if requirements aren't met.
|
|
474
554
|
|
|
@@ -476,9 +556,9 @@ An array of extracted items, or null if requirements aren't met.
|
|
|
476
556
|
|
|
477
557
|
### \_extractNested()
|
|
478
558
|
|
|
479
|
-
> `protected` **\_extractNested**(`items`, `elements`): `Promise`\<`any`[]\>
|
|
559
|
+
> `protected` **\_extractNested**(`items`, `elements`, `opts?`): `Promise`\<`any`[]\>
|
|
480
560
|
|
|
481
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
561
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:567](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L567)
|
|
482
562
|
|
|
483
563
|
**`Internal`**
|
|
484
564
|
|
|
@@ -498,6 +578,12 @@ The schema for each item.
|
|
|
498
578
|
|
|
499
579
|
The list of item elements.
|
|
500
580
|
|
|
581
|
+
##### opts?
|
|
582
|
+
|
|
583
|
+
###### strict?
|
|
584
|
+
|
|
585
|
+
`boolean`
|
|
586
|
+
|
|
501
587
|
#### Returns
|
|
502
588
|
|
|
503
589
|
`Promise`\<`any`[]\>
|
|
@@ -506,9 +592,9 @@ The list of item elements.
|
|
|
506
592
|
|
|
507
593
|
### \_extractSegmented()
|
|
508
594
|
|
|
509
|
-
> `protected` **\_extractSegmented**(`schema`, `container`, `opts?`): `Promise`\<`
|
|
595
|
+
> `protected` **\_extractSegmented**(`schema`, `container`, `opts?`): `Promise`\<`any`[] \| `null`\>
|
|
510
596
|
|
|
511
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
597
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:601](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L601)
|
|
512
598
|
|
|
513
599
|
**`Internal`**
|
|
514
600
|
|
|
@@ -536,7 +622,7 @@ Segmented extraction options (anchor).
|
|
|
536
622
|
|
|
537
623
|
#### Returns
|
|
538
624
|
|
|
539
|
-
`Promise`\<`
|
|
625
|
+
`Promise`\<`any`[] \| `null`\>
|
|
540
626
|
|
|
541
627
|
An array of extracted items.
|
|
542
628
|
|
|
@@ -544,9 +630,9 @@ An array of extracted items.
|
|
|
544
630
|
|
|
545
631
|
### \_extractValue()
|
|
546
632
|
|
|
547
|
-
> `abstract`
|
|
633
|
+
> `abstract` **\_extractValue**(`schema`, `scope`): `Promise`\<`any`\>
|
|
548
634
|
|
|
549
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
635
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:445](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L445)
|
|
550
636
|
|
|
551
637
|
**`Internal`**
|
|
552
638
|
|
|
@@ -560,23 +646,169 @@ Extracts a primitive value from the element based on schema.
|
|
|
560
646
|
|
|
561
647
|
Value extraction schema.
|
|
562
648
|
|
|
563
|
-
#####
|
|
649
|
+
##### scope
|
|
650
|
+
|
|
651
|
+
`any`
|
|
652
|
+
|
|
653
|
+
The element scope.
|
|
654
|
+
|
|
655
|
+
#### Returns
|
|
656
|
+
|
|
657
|
+
`Promise`\<`any`\>
|
|
658
|
+
|
|
659
|
+
Extracted value.
|
|
660
|
+
|
|
661
|
+
#### See
|
|
662
|
+
|
|
663
|
+
IExtractEngine.\_extractValue for behavior contract.
|
|
664
|
+
|
|
665
|
+
#### Implementation of
|
|
666
|
+
|
|
667
|
+
`IExtractEngine._extractValue`
|
|
668
|
+
|
|
669
|
+
***
|
|
670
|
+
|
|
671
|
+
### \_findClosestAncestor()
|
|
672
|
+
|
|
673
|
+
> `abstract` **\_findClosestAncestor**(`scope`, `candidates`): `Promise`\<`any`\>
|
|
674
|
+
|
|
675
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:497](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L497)
|
|
676
|
+
|
|
677
|
+
**`Internal`**
|
|
678
|
+
|
|
679
|
+
Finds the closest ancestor of `scope` (including itself) that exists in the `candidates` array.
|
|
680
|
+
|
|
681
|
+
#### Parameters
|
|
682
|
+
|
|
683
|
+
##### scope
|
|
684
|
+
|
|
685
|
+
`any`
|
|
686
|
+
|
|
687
|
+
The starting element.
|
|
688
|
+
|
|
689
|
+
##### candidates
|
|
690
|
+
|
|
691
|
+
`any`[]
|
|
692
|
+
|
|
693
|
+
The array of potential ancestor scopes.
|
|
694
|
+
|
|
695
|
+
#### Returns
|
|
696
|
+
|
|
697
|
+
`Promise`\<`any`\>
|
|
698
|
+
|
|
699
|
+
A promise resolving to the matching candidate scope, or `null` if none found.
|
|
700
|
+
|
|
701
|
+
#### See
|
|
702
|
+
|
|
703
|
+
IExtractEngine.\_findClosestAncestor for implementation details.
|
|
704
|
+
|
|
705
|
+
#### Implementation of
|
|
706
|
+
|
|
707
|
+
`IExtractEngine._findClosestAncestor`
|
|
708
|
+
|
|
709
|
+
***
|
|
710
|
+
|
|
711
|
+
### \_findCommonAncestor()
|
|
712
|
+
|
|
713
|
+
> `abstract` **\_findCommonAncestor**(`scope1`, `scope2`): `Promise`\<`any`\>
|
|
714
|
+
|
|
715
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:524](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L524)
|
|
716
|
+
|
|
717
|
+
**`Internal`**
|
|
718
|
+
|
|
719
|
+
Finds the Lowest Common Ancestor (LCA) of two element scopes.
|
|
720
|
+
|
|
721
|
+
#### Parameters
|
|
722
|
+
|
|
723
|
+
##### scope1
|
|
564
724
|
|
|
565
725
|
`any`
|
|
566
726
|
|
|
567
|
-
The element
|
|
727
|
+
The first element scope.
|
|
728
|
+
|
|
729
|
+
##### scope2
|
|
730
|
+
|
|
731
|
+
`any`
|
|
732
|
+
|
|
733
|
+
The second element scope.
|
|
734
|
+
|
|
735
|
+
#### Returns
|
|
736
|
+
|
|
737
|
+
`Promise`\<`any`\>
|
|
738
|
+
|
|
739
|
+
A promise resolving to the LCA element scope, or `null` if none found.
|
|
740
|
+
|
|
741
|
+
#### Implementation of
|
|
742
|
+
|
|
743
|
+
`IExtractEngine._findCommonAncestor`
|
|
744
|
+
|
|
745
|
+
***
|
|
746
|
+
|
|
747
|
+
### \_findContainerChild()
|
|
748
|
+
|
|
749
|
+
> `abstract` **\_findContainerChild**(`element`, `container`): `Promise`\<`any`\>
|
|
750
|
+
|
|
751
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:537](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L537)
|
|
752
|
+
|
|
753
|
+
**`Internal`**
|
|
754
|
+
|
|
755
|
+
Finds the direct child of container that contains element.
|
|
756
|
+
|
|
757
|
+
#### Parameters
|
|
758
|
+
|
|
759
|
+
##### element
|
|
760
|
+
|
|
761
|
+
`any`
|
|
762
|
+
|
|
763
|
+
The descendant element.
|
|
764
|
+
|
|
765
|
+
##### container
|
|
766
|
+
|
|
767
|
+
`any`
|
|
768
|
+
|
|
769
|
+
The container element.
|
|
568
770
|
|
|
569
771
|
#### Returns
|
|
570
772
|
|
|
571
773
|
`Promise`\<`any`\>
|
|
572
774
|
|
|
775
|
+
The child element of container, or null.
|
|
776
|
+
|
|
777
|
+
#### Implementation of
|
|
778
|
+
|
|
779
|
+
`IExtractEngine._findContainerChild`
|
|
780
|
+
|
|
781
|
+
***
|
|
782
|
+
|
|
783
|
+
### \_getInitialElementScope()
|
|
784
|
+
|
|
785
|
+
> `abstract` `protected` **\_getInitialElementScope**(`context`): `any`
|
|
786
|
+
|
|
787
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1010](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1010)
|
|
788
|
+
|
|
789
|
+
**`Internal`**
|
|
790
|
+
|
|
791
|
+
Gets the initial scope for extraction for the specific engine.
|
|
792
|
+
|
|
793
|
+
#### Parameters
|
|
794
|
+
|
|
795
|
+
##### context
|
|
796
|
+
|
|
797
|
+
`TContext`
|
|
798
|
+
|
|
799
|
+
Crawlee crawling context
|
|
800
|
+
|
|
801
|
+
#### Returns
|
|
802
|
+
|
|
803
|
+
`any`
|
|
804
|
+
|
|
573
805
|
***
|
|
574
806
|
|
|
575
807
|
### \_getSpecificCrawlerOptions()
|
|
576
808
|
|
|
577
809
|
> `abstract` `protected` **\_getSpecificCrawlerOptions**(`ctx`): `Partial`\<`TOptions`\> \| `Promise`\<`Partial`\<`TOptions`\>\>
|
|
578
810
|
|
|
579
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
811
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:624](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L624)
|
|
580
812
|
|
|
581
813
|
**`Internal`**
|
|
582
814
|
|
|
@@ -596,59 +828,129 @@ The fetch engine context.
|
|
|
596
828
|
|
|
597
829
|
***
|
|
598
830
|
|
|
599
|
-
### \
|
|
831
|
+
### \_getTrimInfo()
|
|
600
832
|
|
|
601
|
-
> `protected` **\
|
|
833
|
+
> `protected` **\_getTrimInfo**(`options`): `object`
|
|
602
834
|
|
|
603
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
835
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:401](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L401)
|
|
604
836
|
|
|
605
837
|
#### Parameters
|
|
606
838
|
|
|
607
|
-
#####
|
|
839
|
+
##### options
|
|
608
840
|
|
|
609
|
-
`
|
|
841
|
+
[`TrimActionOptions`](../interfaces/TrimActionOptions.md)
|
|
610
842
|
|
|
611
843
|
#### Returns
|
|
612
844
|
|
|
613
|
-
`
|
|
845
|
+
`object`
|
|
846
|
+
|
|
847
|
+
##### removeComments
|
|
848
|
+
|
|
849
|
+
> **removeComments**: `boolean`
|
|
850
|
+
|
|
851
|
+
##### removeHidden
|
|
852
|
+
|
|
853
|
+
> **removeHidden**: `boolean`
|
|
854
|
+
|
|
855
|
+
##### selectors
|
|
856
|
+
|
|
857
|
+
> **selectors**: `string`[] = `allSelectors`
|
|
858
|
+
|
|
859
|
+
***
|
|
860
|
+
|
|
861
|
+
### \_handlePause()
|
|
862
|
+
|
|
863
|
+
> `protected` **\_handlePause**(`action`): `Promise`\<`void`\>
|
|
864
|
+
|
|
865
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1049](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1049)
|
|
866
|
+
|
|
867
|
+
#### Parameters
|
|
868
|
+
|
|
869
|
+
##### action
|
|
870
|
+
|
|
871
|
+
###### message?
|
|
872
|
+
|
|
873
|
+
`string`
|
|
874
|
+
|
|
875
|
+
#### Returns
|
|
876
|
+
|
|
877
|
+
`Promise`\<`void`\>
|
|
614
878
|
|
|
615
879
|
***
|
|
616
880
|
|
|
617
881
|
### \_isSameElement()
|
|
618
882
|
|
|
619
|
-
> `abstract`
|
|
883
|
+
> `abstract` **\_isSameElement**(`scope1`, `scope2`): `Promise`\<`boolean`\>
|
|
620
884
|
|
|
621
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
885
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:469](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L469)
|
|
622
886
|
|
|
623
887
|
**`Internal`**
|
|
624
888
|
|
|
625
|
-
Checks if two elements are the same.
|
|
889
|
+
Checks if two elements are the same identity.
|
|
626
890
|
|
|
627
891
|
#### Parameters
|
|
628
892
|
|
|
629
|
-
#####
|
|
893
|
+
##### scope1
|
|
630
894
|
|
|
631
895
|
`any`
|
|
632
896
|
|
|
633
|
-
First element.
|
|
897
|
+
First element scope.
|
|
634
898
|
|
|
635
|
-
#####
|
|
899
|
+
##### scope2
|
|
636
900
|
|
|
637
901
|
`any`
|
|
638
902
|
|
|
639
|
-
Second element.
|
|
903
|
+
Second element scope.
|
|
640
904
|
|
|
641
905
|
#### Returns
|
|
642
906
|
|
|
643
907
|
`Promise`\<`boolean`\>
|
|
644
908
|
|
|
909
|
+
True if they are the same DOM node.
|
|
910
|
+
|
|
911
|
+
#### Implementation of
|
|
912
|
+
|
|
913
|
+
`IExtractEngine._isSameElement`
|
|
914
|
+
|
|
915
|
+
***
|
|
916
|
+
|
|
917
|
+
### \_logDebug()
|
|
918
|
+
|
|
919
|
+
> **\_logDebug**(`category`, ...`args`): `void`
|
|
920
|
+
|
|
921
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:395](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L395)
|
|
922
|
+
|
|
923
|
+
Logs debug information if debug mode is enabled.
|
|
924
|
+
|
|
925
|
+
#### Parameters
|
|
926
|
+
|
|
927
|
+
##### category
|
|
928
|
+
|
|
929
|
+
`string`
|
|
930
|
+
|
|
931
|
+
The category of the log message.
|
|
932
|
+
|
|
933
|
+
##### args
|
|
934
|
+
|
|
935
|
+
...`any`[]
|
|
936
|
+
|
|
937
|
+
Arguments to log.
|
|
938
|
+
|
|
939
|
+
#### Returns
|
|
940
|
+
|
|
941
|
+
`void`
|
|
942
|
+
|
|
943
|
+
#### Implementation of
|
|
944
|
+
|
|
945
|
+
`IExtractEngine._logDebug`
|
|
946
|
+
|
|
645
947
|
***
|
|
646
948
|
|
|
647
949
|
### \_nextSiblingsUntil()
|
|
648
950
|
|
|
649
|
-
> `abstract`
|
|
951
|
+
> `abstract` **\_nextSiblingsUntil**(`scope`, `untilSelector?`): `Promise`\<`any`[]\>
|
|
650
952
|
|
|
651
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
953
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:483](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L483)
|
|
652
954
|
|
|
653
955
|
**`Internal`**
|
|
654
956
|
|
|
@@ -657,29 +959,35 @@ Used in 'segmented' extraction mode.
|
|
|
657
959
|
|
|
658
960
|
#### Parameters
|
|
659
961
|
|
|
660
|
-
#####
|
|
962
|
+
##### scope
|
|
661
963
|
|
|
662
964
|
`any`
|
|
663
965
|
|
|
664
|
-
The anchor element.
|
|
966
|
+
The anchor element scope.
|
|
665
967
|
|
|
666
968
|
##### untilSelector?
|
|
667
969
|
|
|
668
970
|
`string`
|
|
669
971
|
|
|
670
|
-
Optional selector that marks the end of the segment.
|
|
972
|
+
Optional selector that marks the end of the segment (exclusive).
|
|
671
973
|
|
|
672
974
|
#### Returns
|
|
673
975
|
|
|
674
976
|
`Promise`\<`any`[]\>
|
|
675
977
|
|
|
978
|
+
List of sibling elements between anchor and untilSelector.
|
|
979
|
+
|
|
980
|
+
#### Implementation of
|
|
981
|
+
|
|
982
|
+
`IExtractEngine._nextSiblingsUntil`
|
|
983
|
+
|
|
676
984
|
***
|
|
677
985
|
|
|
678
986
|
### \_normalizeArrayMode()
|
|
679
987
|
|
|
680
988
|
> `protected` **\_normalizeArrayMode**(`mode?`): `any`
|
|
681
989
|
|
|
682
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
990
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:555](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L555)
|
|
683
991
|
|
|
684
992
|
**`Internal`**
|
|
685
993
|
|
|
@@ -699,41 +1007,59 @@ The mode string or options object.
|
|
|
699
1007
|
|
|
700
1008
|
***
|
|
701
1009
|
|
|
702
|
-
### \
|
|
1010
|
+
### \_parentElement()
|
|
703
1011
|
|
|
704
|
-
> `
|
|
1012
|
+
> `abstract` **\_parentElement**(`scope`): `Promise`\<`any`\>
|
|
705
1013
|
|
|
706
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1014
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:457](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L457)
|
|
1015
|
+
|
|
1016
|
+
**`Internal`**
|
|
1017
|
+
|
|
1018
|
+
Gets the parent element of the given element.
|
|
707
1019
|
|
|
708
1020
|
#### Parameters
|
|
709
1021
|
|
|
710
|
-
#####
|
|
1022
|
+
##### scope
|
|
711
1023
|
|
|
712
|
-
`
|
|
1024
|
+
`any`
|
|
1025
|
+
|
|
1026
|
+
The element scope.
|
|
713
1027
|
|
|
714
1028
|
#### Returns
|
|
715
1029
|
|
|
716
|
-
`
|
|
1030
|
+
`Promise`\<`any`\>
|
|
1031
|
+
|
|
1032
|
+
Parent element or null.
|
|
1033
|
+
|
|
1034
|
+
#### Implementation of
|
|
1035
|
+
|
|
1036
|
+
`IExtractEngine._parentElement`
|
|
717
1037
|
|
|
718
1038
|
***
|
|
719
1039
|
|
|
720
|
-
### \
|
|
1040
|
+
### \_processAction()
|
|
721
1041
|
|
|
722
|
-
> `
|
|
1042
|
+
> `protected` **\_processAction**(`context`, `action`): `Promise`\<`any`\>
|
|
723
1043
|
|
|
724
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1044
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1020](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1020)
|
|
725
1045
|
|
|
726
1046
|
**`Internal`**
|
|
727
1047
|
|
|
728
|
-
|
|
1048
|
+
Unified action processor that handles engine-agnostic actions.
|
|
729
1049
|
|
|
730
1050
|
#### Parameters
|
|
731
1051
|
|
|
732
|
-
#####
|
|
1052
|
+
##### context
|
|
733
1053
|
|
|
734
|
-
`
|
|
1054
|
+
`TContext`
|
|
1055
|
+
|
|
1056
|
+
Crawlee crawling context
|
|
1057
|
+
|
|
1058
|
+
##### action
|
|
1059
|
+
|
|
1060
|
+
[`FetchEngineAction`](../type-aliases/FetchEngineAction.md)
|
|
735
1061
|
|
|
736
|
-
|
|
1062
|
+
Action to execute
|
|
737
1063
|
|
|
738
1064
|
#### Returns
|
|
739
1065
|
|
|
@@ -743,21 +1069,21 @@ The element.
|
|
|
743
1069
|
|
|
744
1070
|
### \_querySelectorAll()
|
|
745
1071
|
|
|
746
|
-
> `abstract`
|
|
1072
|
+
> `abstract` **\_querySelectorAll**(`scope`, `selector`): `Promise`\<`any`[]\>
|
|
747
1073
|
|
|
748
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1074
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:431](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L431)
|
|
749
1075
|
|
|
750
1076
|
**`Internal`**
|
|
751
1077
|
|
|
752
|
-
Finds all elements matching the selector within the given
|
|
1078
|
+
Finds all elements matching the selector within the given scope.
|
|
753
1079
|
|
|
754
1080
|
#### Parameters
|
|
755
1081
|
|
|
756
|
-
#####
|
|
1082
|
+
##### scope
|
|
757
1083
|
|
|
758
1084
|
`any`
|
|
759
1085
|
|
|
760
|
-
The
|
|
1086
|
+
The scope to search in (Engine-specific element/node or array of nodes).
|
|
761
1087
|
|
|
762
1088
|
##### selector
|
|
763
1089
|
|
|
@@ -769,13 +1095,23 @@ CSS selector.
|
|
|
769
1095
|
|
|
770
1096
|
`Promise`\<`any`[]\>
|
|
771
1097
|
|
|
1098
|
+
List of matching elements.
|
|
1099
|
+
|
|
1100
|
+
#### See
|
|
1101
|
+
|
|
1102
|
+
IExtractEngine.\_querySelectorAll for behavior contract.
|
|
1103
|
+
|
|
1104
|
+
#### Implementation of
|
|
1105
|
+
|
|
1106
|
+
`IExtractEngine._querySelectorAll`
|
|
1107
|
+
|
|
772
1108
|
***
|
|
773
1109
|
|
|
774
1110
|
### \_sharedFailedRequestHandler()
|
|
775
1111
|
|
|
776
1112
|
> `protected` **\_sharedFailedRequestHandler**(`context`, `error?`): `Promise`\<`void`\>
|
|
777
1113
|
|
|
778
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1114
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1185](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1185)
|
|
779
1115
|
|
|
780
1116
|
#### Parameters
|
|
781
1117
|
|
|
@@ -797,7 +1133,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:1196](https://github.com/is
|
|
|
797
1133
|
|
|
798
1134
|
> `protected` **\_sharedRequestHandler**(`context`): `Promise`\<`void`\>
|
|
799
1135
|
|
|
800
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1136
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1144](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1144)
|
|
801
1137
|
|
|
802
1138
|
#### Parameters
|
|
803
1139
|
|
|
@@ -815,7 +1151,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:1156](https://github.com/is
|
|
|
815
1151
|
|
|
816
1152
|
> **blockResources**(`types`, `overwrite?`): `Promise`\<`number`\>
|
|
817
1153
|
|
|
818
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1154
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1301](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1301)
|
|
819
1155
|
|
|
820
1156
|
Blocks specified resource types from loading.
|
|
821
1157
|
|
|
@@ -852,7 +1188,7 @@ await engine.blockResources(['script'], true); // Replace existing
|
|
|
852
1188
|
|
|
853
1189
|
> `protected` **buildResponse**(`context`): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
854
1190
|
|
|
855
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1191
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:638](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L638)
|
|
856
1192
|
|
|
857
1193
|
#### Parameters
|
|
858
1194
|
|
|
@@ -870,7 +1206,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:723](https://github.com/isd
|
|
|
870
1206
|
|
|
871
1207
|
> **cleanup**(): `Promise`\<`void`\>
|
|
872
1208
|
|
|
873
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1209
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:992](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L992)
|
|
874
1210
|
|
|
875
1211
|
#### Returns
|
|
876
1212
|
|
|
@@ -882,7 +1218,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:1084](https://github.com/is
|
|
|
882
1218
|
|
|
883
1219
|
> **click**(`selector`): `Promise`\<`void`\>
|
|
884
1220
|
|
|
885
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1221
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:732](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L732)
|
|
886
1222
|
|
|
887
1223
|
Clicks on element matching selector.
|
|
888
1224
|
|
|
@@ -912,7 +1248,7 @@ When no active page context exists
|
|
|
912
1248
|
|
|
913
1249
|
> **cookies**(): `Promise`\<[`Cookie`](../interfaces/Cookie.md)[]\>
|
|
914
1250
|
|
|
915
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1251
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1417](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1417)
|
|
916
1252
|
|
|
917
1253
|
Manages cookies for current session with multiple overloads.
|
|
918
1254
|
|
|
@@ -937,7 +1273,7 @@ await engine.cookies([{ name: 'session', value: '123' }]);
|
|
|
937
1273
|
|
|
938
1274
|
> **cookies**(`cookies`): `Promise`\<`boolean`\>
|
|
939
1275
|
|
|
940
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1276
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1418](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1418)
|
|
941
1277
|
|
|
942
1278
|
Manages cookies for current session with multiple overloads.
|
|
943
1279
|
|
|
@@ -972,7 +1308,7 @@ await engine.cookies([{ name: 'session', value: '123' }]);
|
|
|
972
1308
|
|
|
973
1309
|
> `protected` **dispatchAction**\<`T`\>(`action`): `Promise`\<`T`\>
|
|
974
1310
|
|
|
975
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1311
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1208](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1208)
|
|
976
1312
|
|
|
977
1313
|
#### Type Parameters
|
|
978
1314
|
|
|
@@ -996,7 +1332,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:1219](https://github.com/is
|
|
|
996
1332
|
|
|
997
1333
|
> **dispose**(): `Promise`\<`void`\>
|
|
998
1334
|
|
|
999
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1335
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1453](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1453)
|
|
1000
1336
|
|
|
1001
1337
|
Disposes of engine, cleaning up all resources.
|
|
1002
1338
|
|
|
@@ -1008,11 +1344,53 @@ Promise resolving when disposal completes
|
|
|
1008
1344
|
|
|
1009
1345
|
***
|
|
1010
1346
|
|
|
1347
|
+
### evaluate()
|
|
1348
|
+
|
|
1349
|
+
> **evaluate**(`params`): `Promise`\<`any`\>
|
|
1350
|
+
|
|
1351
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:800](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L800)
|
|
1352
|
+
|
|
1353
|
+
Executes a custom function or expression within the current page context.
|
|
1354
|
+
|
|
1355
|
+
#### Parameters
|
|
1356
|
+
|
|
1357
|
+
##### params
|
|
1358
|
+
|
|
1359
|
+
[`EvaluateActionOptions`](../interfaces/EvaluateActionOptions.md)
|
|
1360
|
+
|
|
1361
|
+
Configuration for the execution, including the function and arguments.
|
|
1362
|
+
|
|
1363
|
+
#### Returns
|
|
1364
|
+
|
|
1365
|
+
`Promise`\<`any`\>
|
|
1366
|
+
|
|
1367
|
+
A promise resolving to the result of the execution.
|
|
1368
|
+
|
|
1369
|
+
#### Remarks
|
|
1370
|
+
|
|
1371
|
+
This is a powerful action that allows running custom logic to interact with the DOM,
|
|
1372
|
+
calculate values, or trigger navigations.
|
|
1373
|
+
|
|
1374
|
+
- In **Browser Mode**, it runs in the real browser.
|
|
1375
|
+
- In **HTTP Mode**, it runs in a Node.js sandbox with a mocked DOM.
|
|
1376
|
+
|
|
1377
|
+
The action handles automatic navigation if `window.location` is modified.
|
|
1378
|
+
|
|
1379
|
+
#### Throws
|
|
1380
|
+
|
|
1381
|
+
If no active page context exists or if execution fails.
|
|
1382
|
+
|
|
1383
|
+
#### See
|
|
1384
|
+
|
|
1385
|
+
[EvaluateActionOptions](../interfaces/EvaluateActionOptions.md) for detailed parameter options and examples.
|
|
1386
|
+
|
|
1387
|
+
***
|
|
1388
|
+
|
|
1011
1389
|
### executeAction()
|
|
1012
1390
|
|
|
1013
1391
|
> `abstract` `protected` **executeAction**(`context`, `action`): `Promise`\<`any`\>
|
|
1014
1392
|
|
|
1015
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1393
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:687](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L687)
|
|
1016
1394
|
|
|
1017
1395
|
**`Internal`**
|
|
1018
1396
|
|
|
@@ -1048,7 +1426,7 @@ Handles specific user interactions using underlying technology (Playwright/Cheer
|
|
|
1048
1426
|
|
|
1049
1427
|
> **extract**\<`T`\>(`schema`): `Promise`\<`T`\>
|
|
1050
1428
|
|
|
1051
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1429
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:810](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L810)
|
|
1052
1430
|
|
|
1053
1431
|
Extracts structured data from the current page content.
|
|
1054
1432
|
|
|
@@ -1078,7 +1456,7 @@ A promise that resolves to an object with the extracted data.
|
|
|
1078
1456
|
|
|
1079
1457
|
> **fill**(`selector`, `value`): `Promise`\<`void`\>
|
|
1080
1458
|
|
|
1081
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1459
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:744](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L744)
|
|
1082
1460
|
|
|
1083
1461
|
Fills input element with specified value.
|
|
1084
1462
|
|
|
@@ -1112,7 +1490,7 @@ When no active page context exists
|
|
|
1112
1490
|
|
|
1113
1491
|
> **getContent**(): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
1114
1492
|
|
|
1115
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1493
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1315](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1315)
|
|
1116
1494
|
|
|
1117
1495
|
Gets content of current page.
|
|
1118
1496
|
|
|
@@ -1132,7 +1510,7 @@ When no content has been fetched yet
|
|
|
1132
1510
|
|
|
1133
1511
|
> **getState**(): `Promise`\<\{ `cookies`: [`Cookie`](../interfaces/Cookie.md)[]; `sessionState?`: `any`; \}\>
|
|
1134
1512
|
|
|
1135
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1513
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:829](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L829)
|
|
1136
1514
|
|
|
1137
1515
|
Returns the current state of the engine (cookies)
|
|
1138
1516
|
that can be used to restore the session later.
|
|
@@ -1147,7 +1525,7 @@ that can be used to restore the session later.
|
|
|
1147
1525
|
|
|
1148
1526
|
> `abstract` **goto**(`url`, `params?`): `Promise`\<`void` \| [`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
1149
1527
|
|
|
1150
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1528
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:704](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L704)
|
|
1151
1529
|
|
|
1152
1530
|
Navigates to the specified URL.
|
|
1153
1531
|
|
|
@@ -1185,7 +1563,7 @@ await engine.goto('https://example.com');
|
|
|
1185
1563
|
|
|
1186
1564
|
> **headers**(): `Promise`\<`Record`\<`string`, `string`\>\>
|
|
1187
1565
|
|
|
1188
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1566
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1356](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1356)
|
|
1189
1567
|
|
|
1190
1568
|
Manages HTTP headers for requests with multiple overloads.
|
|
1191
1569
|
|
|
@@ -1212,7 +1590,7 @@ await engine.headers('auth', 'token');
|
|
|
1212
1590
|
|
|
1213
1591
|
> **headers**(`name`): `Promise`\<`string`\>
|
|
1214
1592
|
|
|
1215
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1593
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1357](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1357)
|
|
1216
1594
|
|
|
1217
1595
|
Manages HTTP headers for requests with multiple overloads.
|
|
1218
1596
|
|
|
@@ -1247,7 +1625,7 @@ await engine.headers('auth', 'token');
|
|
|
1247
1625
|
|
|
1248
1626
|
> **headers**(`headers`, `replaced?`): `Promise`\<`boolean`\>
|
|
1249
1627
|
|
|
1250
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1628
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1358](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1358)
|
|
1251
1629
|
|
|
1252
1630
|
Manages HTTP headers for requests with multiple overloads.
|
|
1253
1631
|
|
|
@@ -1288,7 +1666,7 @@ await engine.headers('auth', 'token');
|
|
|
1288
1666
|
|
|
1289
1667
|
> **headers**(`name`, `value`): `Promise`\<`boolean`\>
|
|
1290
1668
|
|
|
1291
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1669
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1362](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1362)
|
|
1292
1670
|
|
|
1293
1671
|
Manages HTTP headers for requests with multiple overloads.
|
|
1294
1672
|
|
|
@@ -1306,7 +1684,7 @@ Header name
|
|
|
1306
1684
|
|
|
1307
1685
|
Header value or `null` to remove
|
|
1308
1686
|
|
|
1309
|
-
`
|
|
1687
|
+
`string` | `null`
|
|
1310
1688
|
|
|
1311
1689
|
##### Returns
|
|
1312
1690
|
|
|
@@ -1331,7 +1709,7 @@ await engine.headers('auth', 'token');
|
|
|
1331
1709
|
|
|
1332
1710
|
> **initialize**(`context`, `options?`): `Promise`\<`void`\>
|
|
1333
1711
|
|
|
1334
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1712
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:861](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L861)
|
|
1335
1713
|
|
|
1336
1714
|
Initializes the fetch engine with provided context and options.
|
|
1337
1715
|
|
|
@@ -1366,7 +1744,7 @@ Automatically called when creating engine via `FetchEngine.create()`.
|
|
|
1366
1744
|
|
|
1367
1745
|
> **pause**(`message?`): `Promise`\<`void`\>
|
|
1368
1746
|
|
|
1369
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1747
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:778](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L778)
|
|
1370
1748
|
|
|
1371
1749
|
Pauses execution, allowing for manual intervention or inspection.
|
|
1372
1750
|
|
|
@@ -1394,7 +1772,7 @@ When no active page context exists
|
|
|
1394
1772
|
|
|
1395
1773
|
> **submit**(`selector?`, `options?`): `Promise`\<`void`\>
|
|
1396
1774
|
|
|
1397
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1775
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:756](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L756)
|
|
1398
1776
|
|
|
1399
1777
|
Submits a form.
|
|
1400
1778
|
|
|
@@ -1424,11 +1802,39 @@ When no active page context exists
|
|
|
1424
1802
|
|
|
1425
1803
|
***
|
|
1426
1804
|
|
|
1805
|
+
### trim()
|
|
1806
|
+
|
|
1807
|
+
> **trim**(`options`): `Promise`\<`void`\>
|
|
1808
|
+
|
|
1809
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:767](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L767)
|
|
1810
|
+
|
|
1811
|
+
Removes elements from the DOM based on selectors and presets.
|
|
1812
|
+
|
|
1813
|
+
#### Parameters
|
|
1814
|
+
|
|
1815
|
+
##### options
|
|
1816
|
+
|
|
1817
|
+
[`TrimActionOptions`](../interfaces/TrimActionOptions.md)
|
|
1818
|
+
|
|
1819
|
+
Trim options specifying selectors and presets
|
|
1820
|
+
|
|
1821
|
+
#### Returns
|
|
1822
|
+
|
|
1823
|
+
`Promise`\<`void`\>
|
|
1824
|
+
|
|
1825
|
+
Promise resolving when trim operation completes
|
|
1826
|
+
|
|
1827
|
+
#### Throws
|
|
1828
|
+
|
|
1829
|
+
When no active page context exists
|
|
1830
|
+
|
|
1831
|
+
***
|
|
1832
|
+
|
|
1427
1833
|
### waitFor()
|
|
1428
1834
|
|
|
1429
1835
|
> **waitFor**(`params?`): `Promise`\<`void`\>
|
|
1430
1836
|
|
|
1431
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1837
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:721](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L721)
|
|
1432
1838
|
|
|
1433
1839
|
Waits for specified condition before continuing.
|
|
1434
1840
|
|
|
@@ -1457,9 +1863,9 @@ await engine.waitFor({ selector: '#content' }); // Wait for element
|
|
|
1457
1863
|
|
|
1458
1864
|
### create()
|
|
1459
1865
|
|
|
1460
|
-
> `static` **create**(`ctx`, `options?`): `Promise`\<`
|
|
1866
|
+
> `static` **create**(`ctx`, `options?`): `Promise`\<`AnyFetchEngine` \| `undefined`\>
|
|
1461
1867
|
|
|
1462
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1868
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:332](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L332)
|
|
1463
1869
|
|
|
1464
1870
|
Factory method to create and initialize a fetch engine instance.
|
|
1465
1871
|
|
|
@@ -1479,7 +1885,7 @@ Configuration options
|
|
|
1479
1885
|
|
|
1480
1886
|
#### Returns
|
|
1481
1887
|
|
|
1482
|
-
`Promise`\<`
|
|
1888
|
+
`Promise`\<`AnyFetchEngine` \| `undefined`\>
|
|
1483
1889
|
|
|
1484
1890
|
Initialized fetch engine instance
|
|
1485
1891
|
|
|
@@ -1495,9 +1901,9 @@ Primary entry point for engine creation. Selects appropriate implementation base
|
|
|
1495
1901
|
|
|
1496
1902
|
### get()
|
|
1497
1903
|
|
|
1498
|
-
> `static` **get**(`id`): `
|
|
1904
|
+
> `static` **get**(`id`): `AnyFetchEngineCtor` \| `undefined`
|
|
1499
1905
|
|
|
1500
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1906
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:305](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L305)
|
|
1501
1907
|
|
|
1502
1908
|
Retrieves a fetch engine implementation by its unique ID.
|
|
1503
1909
|
|
|
@@ -1511,7 +1917,7 @@ The ID of the engine to retrieve
|
|
|
1511
1917
|
|
|
1512
1918
|
#### Returns
|
|
1513
1919
|
|
|
1514
|
-
`
|
|
1920
|
+
`AnyFetchEngineCtor` \| `undefined`
|
|
1515
1921
|
|
|
1516
1922
|
Engine class if found, otherwise `undefined`
|
|
1517
1923
|
|
|
@@ -1519,9 +1925,9 @@ Engine class if found, otherwise `undefined`
|
|
|
1519
1925
|
|
|
1520
1926
|
### getByMode()
|
|
1521
1927
|
|
|
1522
|
-
> `static` **getByMode**(`mode`): `
|
|
1928
|
+
> `static` **getByMode**(`mode`): `AnyFetchEngineCtor` \| `undefined`
|
|
1523
1929
|
|
|
1524
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1930
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:315](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L315)
|
|
1525
1931
|
|
|
1526
1932
|
Retrieves a fetch engine implementation by execution mode.
|
|
1527
1933
|
|
|
@@ -1535,7 +1941,7 @@ Execution mode (`'http'` or `'browser'`)
|
|
|
1535
1941
|
|
|
1536
1942
|
#### Returns
|
|
1537
1943
|
|
|
1538
|
-
`
|
|
1944
|
+
`AnyFetchEngineCtor` \| `undefined`
|
|
1539
1945
|
|
|
1540
1946
|
Engine class if found, otherwise `undefined`
|
|
1541
1947
|
|
|
@@ -1545,7 +1951,7 @@ Engine class if found, otherwise `undefined`
|
|
|
1545
1951
|
|
|
1546
1952
|
> `static` **register**(`engineClass`): `void`
|
|
1547
1953
|
|
|
1548
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1954
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:292](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L292)
|
|
1549
1955
|
|
|
1550
1956
|
Registers a fetch engine implementation with the global registry.
|
|
1551
1957
|
|