@isdk/web-fetcher 0.2.11 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.action.cn.md +381 -19
- package/README.action.md +394 -4
- package/README.cn.md +13 -11
- package/README.engine.cn.md +124 -21
- package/README.engine.md +121 -21
- package/README.md +11 -9
- package/dist/index.d.mts +890 -24
- package/dist/index.d.ts +890 -24
- package/dist/index.js +1 -1
- package/dist/index.mjs +1 -1
- package/docs/README.md +11 -9
- package/docs/_media/README.action.md +394 -4
- package/docs/_media/README.cn.md +13 -11
- package/docs/_media/README.engine.md +121 -21
- package/docs/classes/CheerioFetchEngine.md +996 -187
- package/docs/classes/ClickAction.md +33 -33
- package/docs/classes/EvaluateAction.md +559 -0
- package/docs/classes/ExtractAction.md +33 -33
- package/docs/classes/FetchAction.md +35 -33
- package/docs/classes/FetchEngine.md +740 -85
- package/docs/classes/FetchSession.md +143 -24
- package/docs/classes/FillAction.md +33 -33
- package/docs/classes/GetContentAction.md +33 -33
- package/docs/classes/GotoAction.md +33 -33
- package/docs/classes/PauseAction.md +33 -33
- package/docs/classes/PlaywrightFetchEngine.md +901 -180
- package/docs/classes/SubmitAction.md +33 -33
- package/docs/classes/TrimAction.md +533 -0
- package/docs/classes/WaitForAction.md +33 -33
- package/docs/classes/WebFetcher.md +57 -9
- package/docs/enumerations/FetchActionResultStatus.md +4 -4
- package/docs/functions/fetchWeb.md +6 -6
- package/docs/globals.md +6 -0
- package/docs/interfaces/BaseFetchActionProperties.md +19 -11
- package/docs/interfaces/BaseFetchCollectorActionProperties.md +27 -15
- package/docs/interfaces/BaseFetcherProperties.md +28 -28
- package/docs/interfaces/DispatchedEngineAction.md +4 -4
- package/docs/interfaces/EvaluateActionOptions.md +81 -0
- package/docs/interfaces/ExtractActionProperties.md +23 -11
- package/docs/interfaces/FetchActionInContext.md +32 -15
- package/docs/interfaces/FetchActionProperties.md +24 -12
- package/docs/interfaces/FetchActionResult.md +6 -6
- package/docs/interfaces/FetchContext.md +81 -38
- package/docs/interfaces/FetchEngineContext.md +52 -33
- package/docs/interfaces/FetchMetadata.md +5 -5
- package/docs/interfaces/FetchResponse.md +14 -14
- package/docs/interfaces/FetchReturnTypeRegistry.md +8 -8
- package/docs/interfaces/FetchSite.md +31 -31
- package/docs/interfaces/FetcherOptions.md +30 -30
- package/docs/interfaces/GotoActionOptions.md +6 -6
- package/docs/interfaces/PendingEngineRequest.md +3 -3
- package/docs/interfaces/StorageOptions.md +5 -5
- package/docs/interfaces/SubmitActionOptions.md +2 -2
- package/docs/interfaces/TrimActionOptions.md +27 -0
- package/docs/interfaces/WaitForActionOptions.md +5 -5
- package/docs/type-aliases/BaseFetchActionOptions.md +1 -1
- package/docs/type-aliases/BaseFetchCollectorOptions.md +1 -1
- package/docs/type-aliases/BrowserEngine.md +1 -1
- package/docs/type-aliases/FetchActionCapabilities.md +1 -1
- package/docs/type-aliases/FetchActionCapabilityMode.md +1 -1
- package/docs/type-aliases/FetchActionOptions.md +1 -1
- package/docs/type-aliases/FetchEngineAction.md +2 -2
- package/docs/type-aliases/FetchEngineType.md +1 -1
- package/docs/type-aliases/FetchReturnType.md +1 -1
- package/docs/type-aliases/FetchReturnTypeFor.md +1 -1
- package/docs/type-aliases/OnFetchPauseCallback.md +1 -1
- package/docs/type-aliases/ResourceType.md +1 -1
- package/docs/type-aliases/TrimPreset.md +13 -0
- package/docs/variables/DefaultFetcherProperties.md +1 -1
- package/docs/variables/FetcherOptionKeys.md +1 -1
- package/docs/variables/TRIM_PRESETS.md +11 -0
- package/package.json +10 -7
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
# Abstract Class: FetchEngine\<TContext, TCrawler, TOptions\>
|
|
8
8
|
|
|
9
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
9
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:272](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L272)
|
|
10
10
|
|
|
11
11
|
## Extended by
|
|
12
12
|
|
|
@@ -27,6 +27,10 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:144](https://github.com/isd
|
|
|
27
27
|
|
|
28
28
|
`TOptions` *extends* `BasicCrawlerOptions`\<`TContext`\> = `any`
|
|
29
29
|
|
|
30
|
+
## Implements
|
|
31
|
+
|
|
32
|
+
- `IExtractEngine`
|
|
33
|
+
|
|
30
34
|
## Constructors
|
|
31
35
|
|
|
32
36
|
### Constructor
|
|
@@ -43,7 +47,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:144](https://github.com/isd
|
|
|
43
47
|
|
|
44
48
|
> `protected` `optional` **\_initialCookies**: [`Cookie`](../interfaces/Cookie.md)[]
|
|
45
49
|
|
|
46
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
50
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:379](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L379)
|
|
47
51
|
|
|
48
52
|
***
|
|
49
53
|
|
|
@@ -51,7 +55,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:241](https://github.com/isd
|
|
|
51
55
|
|
|
52
56
|
> `protected` **\_initializedSessions**: `Set`\<`string`\>
|
|
53
57
|
|
|
54
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
58
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:380](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L380)
|
|
55
59
|
|
|
56
60
|
***
|
|
57
61
|
|
|
@@ -59,7 +63,23 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:242](https://github.com/isd
|
|
|
59
63
|
|
|
60
64
|
> `protected` **actionEmitter**: `EventEmitter`
|
|
61
65
|
|
|
62
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
66
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:384](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L384)
|
|
67
|
+
|
|
68
|
+
***
|
|
69
|
+
|
|
70
|
+
### actionQueue
|
|
71
|
+
|
|
72
|
+
> `protected` **actionQueue**: [`DispatchedEngineAction`](../interfaces/DispatchedEngineAction.md)[] = `[]`
|
|
73
|
+
|
|
74
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:391](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L391)
|
|
75
|
+
|
|
76
|
+
***
|
|
77
|
+
|
|
78
|
+
### activeContext?
|
|
79
|
+
|
|
80
|
+
> `protected` `optional` **activeContext**: `TContext`
|
|
81
|
+
|
|
82
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:388](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L388)
|
|
63
83
|
|
|
64
84
|
***
|
|
65
85
|
|
|
@@ -67,7 +87,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:246](https://github.com/isd
|
|
|
67
87
|
|
|
68
88
|
> `protected` **blockedTypes**: `Set`\<`string`\>
|
|
69
89
|
|
|
70
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
90
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:393](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L393)
|
|
71
91
|
|
|
72
92
|
***
|
|
73
93
|
|
|
@@ -75,7 +95,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:251](https://github.com/isd
|
|
|
75
95
|
|
|
76
96
|
> `protected` `optional` **config**: `Configuration`
|
|
77
97
|
|
|
78
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
98
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:373](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L373)
|
|
79
99
|
|
|
80
100
|
***
|
|
81
101
|
|
|
@@ -83,7 +103,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:235](https://github.com/isd
|
|
|
83
103
|
|
|
84
104
|
> `protected` `optional` **crawler**: `TCrawler`
|
|
85
105
|
|
|
86
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
106
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:370](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L370)
|
|
87
107
|
|
|
88
108
|
***
|
|
89
109
|
|
|
@@ -91,7 +111,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:232](https://github.com/isd
|
|
|
91
111
|
|
|
92
112
|
> `protected` `optional` **crawlerRunPromise**: `Promise`\<`FinalStatistics`\>
|
|
93
113
|
|
|
94
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
114
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:372](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L372)
|
|
95
115
|
|
|
96
116
|
***
|
|
97
117
|
|
|
@@ -99,7 +119,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:234](https://github.com/isd
|
|
|
99
119
|
|
|
100
120
|
> `protected` `optional` **ctx**: [`FetchEngineContext`](../interfaces/FetchEngineContext.md)
|
|
101
121
|
|
|
102
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
122
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:368](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L368)
|
|
103
123
|
|
|
104
124
|
***
|
|
105
125
|
|
|
@@ -107,7 +127,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:230](https://github.com/isd
|
|
|
107
127
|
|
|
108
128
|
> `protected` `optional` **currentSession**: `Session`
|
|
109
129
|
|
|
110
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
130
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:381](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L381)
|
|
111
131
|
|
|
112
132
|
***
|
|
113
133
|
|
|
@@ -115,7 +135,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:243](https://github.com/isd
|
|
|
115
135
|
|
|
116
136
|
> `protected` **hdrs**: `Record`\<`string`, `string`\> = `{}`
|
|
117
137
|
|
|
118
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
138
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:378](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L378)
|
|
119
139
|
|
|
120
140
|
***
|
|
121
141
|
|
|
@@ -123,7 +143,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:240](https://github.com/isd
|
|
|
123
143
|
|
|
124
144
|
> `protected` `optional` **isCrawlerReady**: `boolean`
|
|
125
145
|
|
|
126
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
146
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:371](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L371)
|
|
127
147
|
|
|
128
148
|
***
|
|
129
149
|
|
|
@@ -131,7 +151,15 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:233](https://github.com/isd
|
|
|
131
151
|
|
|
132
152
|
> `protected` **isEngineDisposed**: `boolean` = `false`
|
|
133
153
|
|
|
134
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
154
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:386](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L386)
|
|
155
|
+
|
|
156
|
+
***
|
|
157
|
+
|
|
158
|
+
### isExecutingAction
|
|
159
|
+
|
|
160
|
+
> `protected` **isExecutingAction**: `boolean` = `false`
|
|
161
|
+
|
|
162
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:389](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L389)
|
|
135
163
|
|
|
136
164
|
***
|
|
137
165
|
|
|
@@ -139,7 +167,15 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:248](https://github.com/isd
|
|
|
139
167
|
|
|
140
168
|
> `protected` **isPageActive**: `boolean` = `false`
|
|
141
169
|
|
|
142
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
170
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:385](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L385)
|
|
171
|
+
|
|
172
|
+
***
|
|
173
|
+
|
|
174
|
+
### isProcessingActionLoop
|
|
175
|
+
|
|
176
|
+
> `protected` **isProcessingActionLoop**: `boolean` = `false`
|
|
177
|
+
|
|
178
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:392](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L392)
|
|
143
179
|
|
|
144
180
|
***
|
|
145
181
|
|
|
@@ -147,7 +183,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:247](https://github.com/isd
|
|
|
147
183
|
|
|
148
184
|
> `protected` `optional` **kvStore**: `KeyValueStore`
|
|
149
185
|
|
|
150
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
186
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:375](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L375)
|
|
151
187
|
|
|
152
188
|
***
|
|
153
189
|
|
|
@@ -155,7 +191,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:237](https://github.com/isd
|
|
|
155
191
|
|
|
156
192
|
> `protected` `optional` **lastResponse**: [`FetchResponse`](../interfaces/FetchResponse.md)
|
|
157
193
|
|
|
158
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
194
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:390](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L390)
|
|
159
195
|
|
|
160
196
|
***
|
|
161
197
|
|
|
@@ -163,7 +199,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:250](https://github.com/isd
|
|
|
163
199
|
|
|
164
200
|
> `protected` **navigationLock**: `PromiseLock`
|
|
165
201
|
|
|
166
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
202
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:387](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L387)
|
|
167
203
|
|
|
168
204
|
***
|
|
169
205
|
|
|
@@ -171,7 +207,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:249](https://github.com/isd
|
|
|
171
207
|
|
|
172
208
|
> `protected` `optional` **opts**: [`BaseFetcherProperties`](../interfaces/BaseFetcherProperties.md)
|
|
173
209
|
|
|
174
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
210
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:369](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L369)
|
|
175
211
|
|
|
176
212
|
***
|
|
177
213
|
|
|
@@ -179,7 +215,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:231](https://github.com/isd
|
|
|
179
215
|
|
|
180
216
|
> `protected` **pendingRequests**: `Map`\<`string`, [`PendingEngineRequest`](../interfaces/PendingEngineRequest.md)\>
|
|
181
217
|
|
|
182
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
218
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:382](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L382)
|
|
183
219
|
|
|
184
220
|
***
|
|
185
221
|
|
|
@@ -187,7 +223,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:244](https://github.com/isd
|
|
|
187
223
|
|
|
188
224
|
> `protected` `optional` **proxyConfiguration**: `ProxyConfiguration`
|
|
189
225
|
|
|
190
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
226
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:376](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L376)
|
|
191
227
|
|
|
192
228
|
***
|
|
193
229
|
|
|
@@ -195,7 +231,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:238](https://github.com/isd
|
|
|
195
231
|
|
|
196
232
|
> `protected` **requestCounter**: `number` = `0`
|
|
197
233
|
|
|
198
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
234
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:383](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L383)
|
|
199
235
|
|
|
200
236
|
***
|
|
201
237
|
|
|
@@ -203,7 +239,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:245](https://github.com/isd
|
|
|
203
239
|
|
|
204
240
|
> `protected` `optional` **requestQueue**: `RequestQueue`
|
|
205
241
|
|
|
206
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
242
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:374](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L374)
|
|
207
243
|
|
|
208
244
|
***
|
|
209
245
|
|
|
@@ -211,7 +247,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:236](https://github.com/isd
|
|
|
211
247
|
|
|
212
248
|
> `readonly` `static` **id**: `string`
|
|
213
249
|
|
|
214
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
250
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:358](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L358)
|
|
215
251
|
|
|
216
252
|
Unique identifier for the engine implementation.
|
|
217
253
|
|
|
@@ -225,7 +261,7 @@ Must be defined by concrete implementations. Used for registration and lookup in
|
|
|
225
261
|
|
|
226
262
|
> `readonly` `static` **mode**: [`FetchEngineType`](../type-aliases/FetchEngineType.md)
|
|
227
263
|
|
|
228
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
264
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:366](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L366)
|
|
229
265
|
|
|
230
266
|
Execution mode of the engine (`'http'` or `'browser'`).
|
|
231
267
|
|
|
@@ -239,15 +275,15 @@ Must be defined by concrete implementations. Indicates whether engine operates a
|
|
|
239
275
|
|
|
240
276
|
#### Get Signature
|
|
241
277
|
|
|
242
|
-
> **get** **context**():
|
|
278
|
+
> **get** **context**(): [`FetchEngineContext`](../interfaces/FetchEngineContext.md) \| `undefined`
|
|
243
279
|
|
|
244
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
280
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:846](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L846)
|
|
245
281
|
|
|
246
282
|
Gets the fetch engine context associated with this instance.
|
|
247
283
|
|
|
248
284
|
##### Returns
|
|
249
285
|
|
|
250
|
-
|
|
286
|
+
[`FetchEngineContext`](../interfaces/FetchEngineContext.md) \| `undefined`
|
|
251
287
|
|
|
252
288
|
***
|
|
253
289
|
|
|
@@ -257,7 +293,7 @@ Gets the fetch engine context associated with this instance.
|
|
|
257
293
|
|
|
258
294
|
> **get** **id**(): `string`
|
|
259
295
|
|
|
260
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
296
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:821](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L821)
|
|
261
297
|
|
|
262
298
|
Gets the unique identifier of this engine implementation.
|
|
263
299
|
|
|
@@ -273,7 +309,7 @@ Gets the unique identifier of this engine implementation.
|
|
|
273
309
|
|
|
274
310
|
> **get** **mode**(): [`FetchEngineType`](../type-aliases/FetchEngineType.md)
|
|
275
311
|
|
|
276
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
312
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:839](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L839)
|
|
277
313
|
|
|
278
314
|
Gets the execution mode of this engine (`'http'` or `'browser'`).
|
|
279
315
|
|
|
@@ -287,7 +323,7 @@ Gets the execution mode of this engine (`'http'` or `'browser'`).
|
|
|
287
323
|
|
|
288
324
|
> `abstract` `protected` **\_buildResponse**(`context`): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
289
325
|
|
|
290
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
326
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:637](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L637)
|
|
291
327
|
|
|
292
328
|
**`Internal`**
|
|
293
329
|
|
|
@@ -317,7 +353,7 @@ Converts implementation-specific context (Playwright `page` or Cheerio `$`) to s
|
|
|
317
353
|
|
|
318
354
|
> `protected` `optional` **\_cleanup**(): `Promise`\<`void`\>
|
|
319
355
|
|
|
320
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
356
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:399](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L399)
|
|
321
357
|
|
|
322
358
|
#### Returns
|
|
323
359
|
|
|
@@ -329,7 +365,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:253](https://github.com/isd
|
|
|
329
365
|
|
|
330
366
|
> `protected` **\_commonCleanup**(): `Promise`\<`void`\>
|
|
331
367
|
|
|
332
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
368
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1237](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1237)
|
|
333
369
|
|
|
334
370
|
#### Returns
|
|
335
371
|
|
|
@@ -337,11 +373,51 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:781](https://github.com/isd
|
|
|
337
373
|
|
|
338
374
|
***
|
|
339
375
|
|
|
376
|
+
### \_contains()
|
|
377
|
+
|
|
378
|
+
> `abstract` **\_contains**(`container`, `element`): `Promise`\<`boolean`\>
|
|
379
|
+
|
|
380
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:511](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L511)
|
|
381
|
+
|
|
382
|
+
**`Internal`**
|
|
383
|
+
|
|
384
|
+
Checks if the `container` scope contains the `element` scope.
|
|
385
|
+
|
|
386
|
+
#### Parameters
|
|
387
|
+
|
|
388
|
+
##### container
|
|
389
|
+
|
|
390
|
+
`any`
|
|
391
|
+
|
|
392
|
+
The potential ancestor element.
|
|
393
|
+
|
|
394
|
+
##### element
|
|
395
|
+
|
|
396
|
+
`any`
|
|
397
|
+
|
|
398
|
+
The potential descendant element.
|
|
399
|
+
|
|
400
|
+
#### Returns
|
|
401
|
+
|
|
402
|
+
`Promise`\<`boolean`\>
|
|
403
|
+
|
|
404
|
+
A promise resolving to `true` if `container` contains `element`.
|
|
405
|
+
|
|
406
|
+
#### See
|
|
407
|
+
|
|
408
|
+
IExtractEngine.\_contains for implementation details.
|
|
409
|
+
|
|
410
|
+
#### Implementation of
|
|
411
|
+
|
|
412
|
+
`IExtractEngine._contains`
|
|
413
|
+
|
|
414
|
+
***
|
|
415
|
+
|
|
340
416
|
### \_createCrawler()
|
|
341
417
|
|
|
342
418
|
> `abstract` `protected` **\_createCrawler**(`options`, `config?`): `TCrawler`
|
|
343
419
|
|
|
344
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
420
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:614](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L614)
|
|
345
421
|
|
|
346
422
|
**`Internal`**
|
|
347
423
|
|
|
@@ -369,7 +445,7 @@ The final crawler options.
|
|
|
369
445
|
|
|
370
446
|
> `protected` **\_executePendingActions**(`context`): `Promise`\<`void`\>
|
|
371
447
|
|
|
372
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
448
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1089](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1089)
|
|
373
449
|
|
|
374
450
|
**`Internal`**
|
|
375
451
|
|
|
@@ -416,9 +492,9 @@ If called outside valid page context window (`!this.isPageActive`)
|
|
|
416
492
|
|
|
417
493
|
### \_extract()
|
|
418
494
|
|
|
419
|
-
> `protected` **\_extract**(`schema`, `
|
|
495
|
+
> `protected` **\_extract**(`schema`, `scope`, `parentStrict?`): `Promise`\<`any`\>
|
|
420
496
|
|
|
421
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
497
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:542](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L542)
|
|
422
498
|
|
|
423
499
|
#### Parameters
|
|
424
500
|
|
|
@@ -426,21 +502,141 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:258](https://github.com/isd
|
|
|
426
502
|
|
|
427
503
|
`ExtractSchema`
|
|
428
504
|
|
|
429
|
-
#####
|
|
505
|
+
##### scope
|
|
430
506
|
|
|
431
507
|
`any`
|
|
432
508
|
|
|
509
|
+
##### parentStrict?
|
|
510
|
+
|
|
511
|
+
`boolean`
|
|
512
|
+
|
|
433
513
|
#### Returns
|
|
434
514
|
|
|
435
515
|
`Promise`\<`any`\>
|
|
436
516
|
|
|
437
517
|
***
|
|
438
518
|
|
|
519
|
+
### \_extractColumnar()
|
|
520
|
+
|
|
521
|
+
> `protected` **\_extractColumnar**(`schema`, `container`, `opts?`): `Promise`\<`any`[] \| `null`\>
|
|
522
|
+
|
|
523
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:584](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L584)
|
|
524
|
+
|
|
525
|
+
**`Internal`**
|
|
526
|
+
|
|
527
|
+
Performs columnar extraction (Column Alignment Mode).
|
|
528
|
+
|
|
529
|
+
#### Parameters
|
|
530
|
+
|
|
531
|
+
##### schema
|
|
532
|
+
|
|
533
|
+
`ExtractSchema`
|
|
534
|
+
|
|
535
|
+
The schema for a single item (must be an object or implicit object).
|
|
536
|
+
|
|
537
|
+
##### container
|
|
538
|
+
|
|
539
|
+
`any`
|
|
540
|
+
|
|
541
|
+
The container element to search within.
|
|
542
|
+
|
|
543
|
+
##### opts?
|
|
544
|
+
|
|
545
|
+
`ColumnarOptions`
|
|
546
|
+
|
|
547
|
+
Columnar extraction options (strict, inference).
|
|
548
|
+
|
|
549
|
+
#### Returns
|
|
550
|
+
|
|
551
|
+
`Promise`\<`any`[] \| `null`\>
|
|
552
|
+
|
|
553
|
+
An array of extracted items, or null if requirements aren't met.
|
|
554
|
+
|
|
555
|
+
***
|
|
556
|
+
|
|
557
|
+
### \_extractNested()
|
|
558
|
+
|
|
559
|
+
> `protected` **\_extractNested**(`items`, `elements`, `opts?`): `Promise`\<`any`[]\>
|
|
560
|
+
|
|
561
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:567](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L567)
|
|
562
|
+
|
|
563
|
+
**`Internal`**
|
|
564
|
+
|
|
565
|
+
Performs standard nested array extraction.
|
|
566
|
+
|
|
567
|
+
#### Parameters
|
|
568
|
+
|
|
569
|
+
##### items
|
|
570
|
+
|
|
571
|
+
`ExtractSchema`
|
|
572
|
+
|
|
573
|
+
The schema for each item.
|
|
574
|
+
|
|
575
|
+
##### elements
|
|
576
|
+
|
|
577
|
+
`any`[]
|
|
578
|
+
|
|
579
|
+
The list of item elements.
|
|
580
|
+
|
|
581
|
+
##### opts?
|
|
582
|
+
|
|
583
|
+
###### strict?
|
|
584
|
+
|
|
585
|
+
`boolean`
|
|
586
|
+
|
|
587
|
+
#### Returns
|
|
588
|
+
|
|
589
|
+
`Promise`\<`any`[]\>
|
|
590
|
+
|
|
591
|
+
***
|
|
592
|
+
|
|
593
|
+
### \_extractSegmented()
|
|
594
|
+
|
|
595
|
+
> `protected` **\_extractSegmented**(`schema`, `container`, `opts?`): `Promise`\<`any`[] \| `null`\>
|
|
596
|
+
|
|
597
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:601](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L601)
|
|
598
|
+
|
|
599
|
+
**`Internal`**
|
|
600
|
+
|
|
601
|
+
Performs segmented extraction (Anchor-based Scanning).
|
|
602
|
+
|
|
603
|
+
#### Parameters
|
|
604
|
+
|
|
605
|
+
##### schema
|
|
606
|
+
|
|
607
|
+
`ExtractSchema`
|
|
608
|
+
|
|
609
|
+
The schema for a single item (must be an object).
|
|
610
|
+
|
|
611
|
+
##### container
|
|
612
|
+
|
|
613
|
+
`any`
|
|
614
|
+
|
|
615
|
+
The container element to scan.
|
|
616
|
+
|
|
617
|
+
##### opts?
|
|
618
|
+
|
|
619
|
+
`SegmentedOptions`
|
|
620
|
+
|
|
621
|
+
Segmented extraction options (anchor).
|
|
622
|
+
|
|
623
|
+
#### Returns
|
|
624
|
+
|
|
625
|
+
`Promise`\<`any`[] \| `null`\>
|
|
626
|
+
|
|
627
|
+
An array of extracted items.
|
|
628
|
+
|
|
629
|
+
***
|
|
630
|
+
|
|
439
631
|
### \_extractValue()
|
|
440
632
|
|
|
441
|
-
> `abstract`
|
|
633
|
+
> `abstract` **\_extractValue**(`schema`, `scope`): `Promise`\<`any`\>
|
|
442
634
|
|
|
443
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
635
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:445](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L445)
|
|
636
|
+
|
|
637
|
+
**`Internal`**
|
|
638
|
+
|
|
639
|
+
Extracts a primitive value from the element based on schema.
|
|
444
640
|
|
|
445
641
|
#### Parameters
|
|
446
642
|
|
|
@@ -448,21 +644,171 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:256](https://github.com/isd
|
|
|
448
644
|
|
|
449
645
|
`ExtractValueSchema`
|
|
450
646
|
|
|
451
|
-
|
|
647
|
+
Value extraction schema.
|
|
648
|
+
|
|
649
|
+
##### scope
|
|
650
|
+
|
|
651
|
+
`any`
|
|
652
|
+
|
|
653
|
+
The element scope.
|
|
654
|
+
|
|
655
|
+
#### Returns
|
|
656
|
+
|
|
657
|
+
`Promise`\<`any`\>
|
|
658
|
+
|
|
659
|
+
Extracted value.
|
|
660
|
+
|
|
661
|
+
#### See
|
|
662
|
+
|
|
663
|
+
IExtractEngine.\_extractValue for behavior contract.
|
|
664
|
+
|
|
665
|
+
#### Implementation of
|
|
666
|
+
|
|
667
|
+
`IExtractEngine._extractValue`
|
|
668
|
+
|
|
669
|
+
***
|
|
670
|
+
|
|
671
|
+
### \_findClosestAncestor()
|
|
672
|
+
|
|
673
|
+
> `abstract` **\_findClosestAncestor**(`scope`, `candidates`): `Promise`\<`any`\>
|
|
674
|
+
|
|
675
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:497](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L497)
|
|
676
|
+
|
|
677
|
+
**`Internal`**
|
|
678
|
+
|
|
679
|
+
Finds the closest ancestor of `scope` (including itself) that exists in the `candidates` array.
|
|
680
|
+
|
|
681
|
+
#### Parameters
|
|
682
|
+
|
|
683
|
+
##### scope
|
|
452
684
|
|
|
453
685
|
`any`
|
|
454
686
|
|
|
687
|
+
The starting element.
|
|
688
|
+
|
|
689
|
+
##### candidates
|
|
690
|
+
|
|
691
|
+
`any`[]
|
|
692
|
+
|
|
693
|
+
The array of potential ancestor scopes.
|
|
694
|
+
|
|
455
695
|
#### Returns
|
|
456
696
|
|
|
457
697
|
`Promise`\<`any`\>
|
|
458
698
|
|
|
699
|
+
A promise resolving to the matching candidate scope, or `null` if none found.
|
|
700
|
+
|
|
701
|
+
#### See
|
|
702
|
+
|
|
703
|
+
IExtractEngine.\_findClosestAncestor for implementation details.
|
|
704
|
+
|
|
705
|
+
#### Implementation of
|
|
706
|
+
|
|
707
|
+
`IExtractEngine._findClosestAncestor`
|
|
708
|
+
|
|
709
|
+
***
|
|
710
|
+
|
|
711
|
+
### \_findCommonAncestor()
|
|
712
|
+
|
|
713
|
+
> `abstract` **\_findCommonAncestor**(`scope1`, `scope2`): `Promise`\<`any`\>
|
|
714
|
+
|
|
715
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:524](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L524)
|
|
716
|
+
|
|
717
|
+
**`Internal`**
|
|
718
|
+
|
|
719
|
+
Finds the Lowest Common Ancestor (LCA) of two element scopes.
|
|
720
|
+
|
|
721
|
+
#### Parameters
|
|
722
|
+
|
|
723
|
+
##### scope1
|
|
724
|
+
|
|
725
|
+
`any`
|
|
726
|
+
|
|
727
|
+
The first element scope.
|
|
728
|
+
|
|
729
|
+
##### scope2
|
|
730
|
+
|
|
731
|
+
`any`
|
|
732
|
+
|
|
733
|
+
The second element scope.
|
|
734
|
+
|
|
735
|
+
#### Returns
|
|
736
|
+
|
|
737
|
+
`Promise`\<`any`\>
|
|
738
|
+
|
|
739
|
+
A promise resolving to the LCA element scope, or `null` if none found.
|
|
740
|
+
|
|
741
|
+
#### Implementation of
|
|
742
|
+
|
|
743
|
+
`IExtractEngine._findCommonAncestor`
|
|
744
|
+
|
|
745
|
+
***
|
|
746
|
+
|
|
747
|
+
### \_findContainerChild()
|
|
748
|
+
|
|
749
|
+
> `abstract` **\_findContainerChild**(`element`, `container`): `Promise`\<`any`\>
|
|
750
|
+
|
|
751
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:537](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L537)
|
|
752
|
+
|
|
753
|
+
**`Internal`**
|
|
754
|
+
|
|
755
|
+
Finds the direct child of container that contains element.
|
|
756
|
+
|
|
757
|
+
#### Parameters
|
|
758
|
+
|
|
759
|
+
##### element
|
|
760
|
+
|
|
761
|
+
`any`
|
|
762
|
+
|
|
763
|
+
The descendant element.
|
|
764
|
+
|
|
765
|
+
##### container
|
|
766
|
+
|
|
767
|
+
`any`
|
|
768
|
+
|
|
769
|
+
The container element.
|
|
770
|
+
|
|
771
|
+
#### Returns
|
|
772
|
+
|
|
773
|
+
`Promise`\<`any`\>
|
|
774
|
+
|
|
775
|
+
The child element of container, or null.
|
|
776
|
+
|
|
777
|
+
#### Implementation of
|
|
778
|
+
|
|
779
|
+
`IExtractEngine._findContainerChild`
|
|
780
|
+
|
|
781
|
+
***
|
|
782
|
+
|
|
783
|
+
### \_getInitialElementScope()
|
|
784
|
+
|
|
785
|
+
> `abstract` `protected` **\_getInitialElementScope**(`context`): `any`
|
|
786
|
+
|
|
787
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1010](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1010)
|
|
788
|
+
|
|
789
|
+
**`Internal`**
|
|
790
|
+
|
|
791
|
+
Gets the initial scope for extraction for the specific engine.
|
|
792
|
+
|
|
793
|
+
#### Parameters
|
|
794
|
+
|
|
795
|
+
##### context
|
|
796
|
+
|
|
797
|
+
`TContext`
|
|
798
|
+
|
|
799
|
+
Crawlee crawling context
|
|
800
|
+
|
|
801
|
+
#### Returns
|
|
802
|
+
|
|
803
|
+
`any`
|
|
804
|
+
|
|
459
805
|
***
|
|
460
806
|
|
|
461
807
|
### \_getSpecificCrawlerOptions()
|
|
462
808
|
|
|
463
809
|
> `abstract` `protected` **\_getSpecificCrawlerOptions**(`ctx`): `Partial`\<`TOptions`\> \| `Promise`\<`Partial`\<`TOptions`\>\>
|
|
464
810
|
|
|
465
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
811
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:624](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L624)
|
|
466
812
|
|
|
467
813
|
**`Internal`**
|
|
468
814
|
|
|
@@ -482,51 +828,290 @@ The fetch engine context.
|
|
|
482
828
|
|
|
483
829
|
***
|
|
484
830
|
|
|
485
|
-
### \
|
|
831
|
+
### \_getTrimInfo()
|
|
486
832
|
|
|
487
|
-
> `protected` **\
|
|
833
|
+
> `protected` **\_getTrimInfo**(`options`): `object`
|
|
488
834
|
|
|
489
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
835
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:401](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L401)
|
|
490
836
|
|
|
491
837
|
#### Parameters
|
|
492
838
|
|
|
493
|
-
#####
|
|
839
|
+
##### options
|
|
494
840
|
|
|
495
|
-
`
|
|
841
|
+
[`TrimActionOptions`](../interfaces/TrimActionOptions.md)
|
|
496
842
|
|
|
497
843
|
#### Returns
|
|
498
844
|
|
|
499
|
-
`
|
|
845
|
+
`object`
|
|
846
|
+
|
|
847
|
+
##### removeComments
|
|
848
|
+
|
|
849
|
+
> **removeComments**: `boolean`
|
|
850
|
+
|
|
851
|
+
##### removeHidden
|
|
852
|
+
|
|
853
|
+
> **removeHidden**: `boolean`
|
|
854
|
+
|
|
855
|
+
##### selectors
|
|
856
|
+
|
|
857
|
+
> **selectors**: `string`[] = `allSelectors`
|
|
500
858
|
|
|
501
859
|
***
|
|
502
860
|
|
|
503
|
-
### \
|
|
861
|
+
### \_handlePause()
|
|
862
|
+
|
|
863
|
+
> `protected` **\_handlePause**(`action`): `Promise`\<`void`\>
|
|
864
|
+
|
|
865
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1049](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1049)
|
|
866
|
+
|
|
867
|
+
#### Parameters
|
|
868
|
+
|
|
869
|
+
##### action
|
|
870
|
+
|
|
871
|
+
###### message?
|
|
872
|
+
|
|
873
|
+
`string`
|
|
874
|
+
|
|
875
|
+
#### Returns
|
|
876
|
+
|
|
877
|
+
`Promise`\<`void`\>
|
|
878
|
+
|
|
879
|
+
***
|
|
880
|
+
|
|
881
|
+
### \_isSameElement()
|
|
882
|
+
|
|
883
|
+
> `abstract` **\_isSameElement**(`scope1`, `scope2`): `Promise`\<`boolean`\>
|
|
884
|
+
|
|
885
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:469](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L469)
|
|
886
|
+
|
|
887
|
+
**`Internal`**
|
|
888
|
+
|
|
889
|
+
Checks if two elements are the same identity.
|
|
890
|
+
|
|
891
|
+
#### Parameters
|
|
892
|
+
|
|
893
|
+
##### scope1
|
|
894
|
+
|
|
895
|
+
`any`
|
|
896
|
+
|
|
897
|
+
First element scope.
|
|
898
|
+
|
|
899
|
+
##### scope2
|
|
900
|
+
|
|
901
|
+
`any`
|
|
902
|
+
|
|
903
|
+
Second element scope.
|
|
904
|
+
|
|
905
|
+
#### Returns
|
|
906
|
+
|
|
907
|
+
`Promise`\<`boolean`\>
|
|
908
|
+
|
|
909
|
+
True if they are the same DOM node.
|
|
910
|
+
|
|
911
|
+
#### Implementation of
|
|
912
|
+
|
|
913
|
+
`IExtractEngine._isSameElement`
|
|
914
|
+
|
|
915
|
+
***
|
|
916
|
+
|
|
917
|
+
### \_logDebug()
|
|
918
|
+
|
|
919
|
+
> **\_logDebug**(`category`, ...`args`): `void`
|
|
920
|
+
|
|
921
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:395](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L395)
|
|
922
|
+
|
|
923
|
+
Logs debug information if debug mode is enabled.
|
|
924
|
+
|
|
925
|
+
#### Parameters
|
|
926
|
+
|
|
927
|
+
##### category
|
|
928
|
+
|
|
929
|
+
`string`
|
|
930
|
+
|
|
931
|
+
The category of the log message.
|
|
932
|
+
|
|
933
|
+
##### args
|
|
934
|
+
|
|
935
|
+
...`any`[]
|
|
936
|
+
|
|
937
|
+
Arguments to log.
|
|
938
|
+
|
|
939
|
+
#### Returns
|
|
940
|
+
|
|
941
|
+
`void`
|
|
942
|
+
|
|
943
|
+
#### Implementation of
|
|
944
|
+
|
|
945
|
+
`IExtractEngine._logDebug`
|
|
946
|
+
|
|
947
|
+
***
|
|
948
|
+
|
|
949
|
+
### \_nextSiblingsUntil()
|
|
950
|
+
|
|
951
|
+
> `abstract` **\_nextSiblingsUntil**(`scope`, `untilSelector?`): `Promise`\<`any`[]\>
|
|
952
|
+
|
|
953
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:483](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L483)
|
|
954
|
+
|
|
955
|
+
**`Internal`**
|
|
956
|
+
|
|
957
|
+
Gets all subsequent siblings of an element until a sibling matches the selector.
|
|
958
|
+
Used in 'segmented' extraction mode.
|
|
959
|
+
|
|
960
|
+
#### Parameters
|
|
961
|
+
|
|
962
|
+
##### scope
|
|
963
|
+
|
|
964
|
+
`any`
|
|
965
|
+
|
|
966
|
+
The anchor element scope.
|
|
967
|
+
|
|
968
|
+
##### untilSelector?
|
|
969
|
+
|
|
970
|
+
`string`
|
|
971
|
+
|
|
972
|
+
Optional selector that marks the end of the segment (exclusive).
|
|
973
|
+
|
|
974
|
+
#### Returns
|
|
975
|
+
|
|
976
|
+
`Promise`\<`any`[]\>
|
|
977
|
+
|
|
978
|
+
List of sibling elements between anchor and untilSelector.
|
|
979
|
+
|
|
980
|
+
#### Implementation of
|
|
981
|
+
|
|
982
|
+
`IExtractEngine._nextSiblingsUntil`
|
|
983
|
+
|
|
984
|
+
***
|
|
985
|
+
|
|
986
|
+
### \_normalizeArrayMode()
|
|
987
|
+
|
|
988
|
+
> `protected` **\_normalizeArrayMode**(`mode?`): `any`
|
|
989
|
+
|
|
990
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:555](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L555)
|
|
991
|
+
|
|
992
|
+
**`Internal`**
|
|
993
|
+
|
|
994
|
+
Normalizes the array extraction mode into an options object.
|
|
995
|
+
|
|
996
|
+
#### Parameters
|
|
997
|
+
|
|
998
|
+
##### mode?
|
|
504
999
|
|
|
505
|
-
|
|
1000
|
+
`ExtractArrayMode`
|
|
506
1001
|
|
|
507
|
-
|
|
1002
|
+
The mode string or options object.
|
|
1003
|
+
|
|
1004
|
+
#### Returns
|
|
1005
|
+
|
|
1006
|
+
`any`
|
|
1007
|
+
|
|
1008
|
+
***
|
|
1009
|
+
|
|
1010
|
+
### \_parentElement()
|
|
1011
|
+
|
|
1012
|
+
> `abstract` **\_parentElement**(`scope`): `Promise`\<`any`\>
|
|
1013
|
+
|
|
1014
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:457](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L457)
|
|
1015
|
+
|
|
1016
|
+
**`Internal`**
|
|
1017
|
+
|
|
1018
|
+
Gets the parent element of the given element.
|
|
1019
|
+
|
|
1020
|
+
#### Parameters
|
|
1021
|
+
|
|
1022
|
+
##### scope
|
|
1023
|
+
|
|
1024
|
+
`any`
|
|
1025
|
+
|
|
1026
|
+
The element scope.
|
|
1027
|
+
|
|
1028
|
+
#### Returns
|
|
1029
|
+
|
|
1030
|
+
`Promise`\<`any`\>
|
|
1031
|
+
|
|
1032
|
+
Parent element or null.
|
|
1033
|
+
|
|
1034
|
+
#### Implementation of
|
|
1035
|
+
|
|
1036
|
+
`IExtractEngine._parentElement`
|
|
1037
|
+
|
|
1038
|
+
***
|
|
1039
|
+
|
|
1040
|
+
### \_processAction()
|
|
1041
|
+
|
|
1042
|
+
> `protected` **\_processAction**(`context`, `action`): `Promise`\<`any`\>
|
|
1043
|
+
|
|
1044
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1020](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1020)
|
|
1045
|
+
|
|
1046
|
+
**`Internal`**
|
|
1047
|
+
|
|
1048
|
+
Unified action processor that handles engine-agnostic actions.
|
|
508
1049
|
|
|
509
1050
|
#### Parameters
|
|
510
1051
|
|
|
511
1052
|
##### context
|
|
512
1053
|
|
|
1054
|
+
`TContext`
|
|
1055
|
+
|
|
1056
|
+
Crawlee crawling context
|
|
1057
|
+
|
|
1058
|
+
##### action
|
|
1059
|
+
|
|
1060
|
+
[`FetchEngineAction`](../type-aliases/FetchEngineAction.md)
|
|
1061
|
+
|
|
1062
|
+
Action to execute
|
|
1063
|
+
|
|
1064
|
+
#### Returns
|
|
1065
|
+
|
|
1066
|
+
`Promise`\<`any`\>
|
|
1067
|
+
|
|
1068
|
+
***
|
|
1069
|
+
|
|
1070
|
+
### \_querySelectorAll()
|
|
1071
|
+
|
|
1072
|
+
> `abstract` **\_querySelectorAll**(`scope`, `selector`): `Promise`\<`any`[]\>
|
|
1073
|
+
|
|
1074
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:431](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L431)
|
|
1075
|
+
|
|
1076
|
+
**`Internal`**
|
|
1077
|
+
|
|
1078
|
+
Finds all elements matching the selector within the given scope.
|
|
1079
|
+
|
|
1080
|
+
#### Parameters
|
|
1081
|
+
|
|
1082
|
+
##### scope
|
|
1083
|
+
|
|
513
1084
|
`any`
|
|
514
1085
|
|
|
1086
|
+
The scope to search in (Engine-specific element/node or array of nodes).
|
|
1087
|
+
|
|
515
1088
|
##### selector
|
|
516
1089
|
|
|
517
1090
|
`string`
|
|
518
1091
|
|
|
1092
|
+
CSS selector.
|
|
1093
|
+
|
|
519
1094
|
#### Returns
|
|
520
1095
|
|
|
521
1096
|
`Promise`\<`any`[]\>
|
|
522
1097
|
|
|
1098
|
+
List of matching elements.
|
|
1099
|
+
|
|
1100
|
+
#### See
|
|
1101
|
+
|
|
1102
|
+
IExtractEngine.\_querySelectorAll for behavior contract.
|
|
1103
|
+
|
|
1104
|
+
#### Implementation of
|
|
1105
|
+
|
|
1106
|
+
`IExtractEngine._querySelectorAll`
|
|
1107
|
+
|
|
523
1108
|
***
|
|
524
1109
|
|
|
525
1110
|
### \_sharedFailedRequestHandler()
|
|
526
1111
|
|
|
527
1112
|
> `protected` **\_sharedFailedRequestHandler**(`context`, `error?`): `Promise`\<`void`\>
|
|
528
1113
|
|
|
529
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1114
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1185](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1185)
|
|
530
1115
|
|
|
531
1116
|
#### Parameters
|
|
532
1117
|
|
|
@@ -548,7 +1133,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:748](https://github.com/isd
|
|
|
548
1133
|
|
|
549
1134
|
> `protected` **\_sharedRequestHandler**(`context`): `Promise`\<`void`\>
|
|
550
1135
|
|
|
551
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1136
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1144](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1144)
|
|
552
1137
|
|
|
553
1138
|
#### Parameters
|
|
554
1139
|
|
|
@@ -566,7 +1151,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:713](https://github.com/isd
|
|
|
566
1151
|
|
|
567
1152
|
> **blockResources**(`types`, `overwrite?`): `Promise`\<`number`\>
|
|
568
1153
|
|
|
569
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1154
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1301](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1301)
|
|
570
1155
|
|
|
571
1156
|
Blocks specified resource types from loading.
|
|
572
1157
|
|
|
@@ -603,7 +1188,7 @@ await engine.blockResources(['script'], true); // Replace existing
|
|
|
603
1188
|
|
|
604
1189
|
> `protected` **buildResponse**(`context`): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
605
1190
|
|
|
606
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1191
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:638](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L638)
|
|
607
1192
|
|
|
608
1193
|
#### Parameters
|
|
609
1194
|
|
|
@@ -621,7 +1206,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:327](https://github.com/isd
|
|
|
621
1206
|
|
|
622
1207
|
> **cleanup**(): `Promise`\<`void`\>
|
|
623
1208
|
|
|
624
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1209
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:992](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L992)
|
|
625
1210
|
|
|
626
1211
|
#### Returns
|
|
627
1212
|
|
|
@@ -633,7 +1218,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:645](https://github.com/isd
|
|
|
633
1218
|
|
|
634
1219
|
> **click**(`selector`): `Promise`\<`void`\>
|
|
635
1220
|
|
|
636
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1221
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:732](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L732)
|
|
637
1222
|
|
|
638
1223
|
Clicks on element matching selector.
|
|
639
1224
|
|
|
@@ -663,7 +1248,7 @@ When no active page context exists
|
|
|
663
1248
|
|
|
664
1249
|
> **cookies**(): `Promise`\<[`Cookie`](../interfaces/Cookie.md)[]\>
|
|
665
1250
|
|
|
666
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1251
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1417](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1417)
|
|
667
1252
|
|
|
668
1253
|
Manages cookies for current session with multiple overloads.
|
|
669
1254
|
|
|
@@ -688,7 +1273,7 @@ await engine.cookies([{ name: 'session', value: '123' }]);
|
|
|
688
1273
|
|
|
689
1274
|
> **cookies**(`cookies`): `Promise`\<`boolean`\>
|
|
690
1275
|
|
|
691
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1276
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1418](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1418)
|
|
692
1277
|
|
|
693
1278
|
Manages cookies for current session with multiple overloads.
|
|
694
1279
|
|
|
@@ -723,7 +1308,7 @@ await engine.cookies([{ name: 'session', value: '123' }]);
|
|
|
723
1308
|
|
|
724
1309
|
> `protected` **dispatchAction**\<`T`\>(`action`): `Promise`\<`T`\>
|
|
725
1310
|
|
|
726
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1311
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1208](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1208)
|
|
727
1312
|
|
|
728
1313
|
#### Type Parameters
|
|
729
1314
|
|
|
@@ -747,7 +1332,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:764](https://github.com/isd
|
|
|
747
1332
|
|
|
748
1333
|
> **dispose**(): `Promise`\<`void`\>
|
|
749
1334
|
|
|
750
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1335
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1453](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1453)
|
|
751
1336
|
|
|
752
1337
|
Disposes of engine, cleaning up all resources.
|
|
753
1338
|
|
|
@@ -759,11 +1344,53 @@ Promise resolving when disposal completes
|
|
|
759
1344
|
|
|
760
1345
|
***
|
|
761
1346
|
|
|
1347
|
+
### evaluate()
|
|
1348
|
+
|
|
1349
|
+
> **evaluate**(`params`): `Promise`\<`any`\>
|
|
1350
|
+
|
|
1351
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:800](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L800)
|
|
1352
|
+
|
|
1353
|
+
Executes a custom function or expression within the current page context.
|
|
1354
|
+
|
|
1355
|
+
#### Parameters
|
|
1356
|
+
|
|
1357
|
+
##### params
|
|
1358
|
+
|
|
1359
|
+
[`EvaluateActionOptions`](../interfaces/EvaluateActionOptions.md)
|
|
1360
|
+
|
|
1361
|
+
Configuration for the execution, including the function and arguments.
|
|
1362
|
+
|
|
1363
|
+
#### Returns
|
|
1364
|
+
|
|
1365
|
+
`Promise`\<`any`\>
|
|
1366
|
+
|
|
1367
|
+
A promise resolving to the result of the execution.
|
|
1368
|
+
|
|
1369
|
+
#### Remarks
|
|
1370
|
+
|
|
1371
|
+
This is a powerful action that allows running custom logic to interact with the DOM,
|
|
1372
|
+
calculate values, or trigger navigations.
|
|
1373
|
+
|
|
1374
|
+
- In **Browser Mode**, it runs in the real browser.
|
|
1375
|
+
- In **HTTP Mode**, it runs in a Node.js sandbox with a mocked DOM.
|
|
1376
|
+
|
|
1377
|
+
The action handles automatic navigation if `window.location` is modified.
|
|
1378
|
+
|
|
1379
|
+
#### Throws
|
|
1380
|
+
|
|
1381
|
+
If no active page context exists or if execution fails.
|
|
1382
|
+
|
|
1383
|
+
#### See
|
|
1384
|
+
|
|
1385
|
+
[EvaluateActionOptions](../interfaces/EvaluateActionOptions.md) for detailed parameter options and examples.
|
|
1386
|
+
|
|
1387
|
+
***
|
|
1388
|
+
|
|
762
1389
|
### executeAction()
|
|
763
1390
|
|
|
764
1391
|
> `abstract` `protected` **executeAction**(`context`, `action`): `Promise`\<`any`\>
|
|
765
1392
|
|
|
766
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1393
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:687](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L687)
|
|
767
1394
|
|
|
768
1395
|
**`Internal`**
|
|
769
1396
|
|
|
@@ -799,7 +1426,7 @@ Handles specific user interactions using underlying technology (Playwright/Cheer
|
|
|
799
1426
|
|
|
800
1427
|
> **extract**\<`T`\>(`schema`): `Promise`\<`T`\>
|
|
801
1428
|
|
|
802
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1429
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:810](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L810)
|
|
803
1430
|
|
|
804
1431
|
Extracts structured data from the current page content.
|
|
805
1432
|
|
|
@@ -829,7 +1456,7 @@ A promise that resolves to an object with the extracted data.
|
|
|
829
1456
|
|
|
830
1457
|
> **fill**(`selector`, `value`): `Promise`\<`void`\>
|
|
831
1458
|
|
|
832
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1459
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:744](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L744)
|
|
833
1460
|
|
|
834
1461
|
Fills input element with specified value.
|
|
835
1462
|
|
|
@@ -863,7 +1490,7 @@ When no active page context exists
|
|
|
863
1490
|
|
|
864
1491
|
> **getContent**(): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
865
1492
|
|
|
866
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1493
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1315](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1315)
|
|
867
1494
|
|
|
868
1495
|
Gets content of current page.
|
|
869
1496
|
|
|
@@ -883,7 +1510,7 @@ When no content has been fetched yet
|
|
|
883
1510
|
|
|
884
1511
|
> **getState**(): `Promise`\<\{ `cookies`: [`Cookie`](../interfaces/Cookie.md)[]; `sessionState?`: `any`; \}\>
|
|
885
1512
|
|
|
886
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1513
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:829](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L829)
|
|
887
1514
|
|
|
888
1515
|
Returns the current state of the engine (cookies)
|
|
889
1516
|
that can be used to restore the session later.
|
|
@@ -898,7 +1525,7 @@ that can be used to restore the session later.
|
|
|
898
1525
|
|
|
899
1526
|
> `abstract` **goto**(`url`, `params?`): `Promise`\<`void` \| [`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
900
1527
|
|
|
901
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1528
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:704](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L704)
|
|
902
1529
|
|
|
903
1530
|
Navigates to the specified URL.
|
|
904
1531
|
|
|
@@ -936,7 +1563,7 @@ await engine.goto('https://example.com');
|
|
|
936
1563
|
|
|
937
1564
|
> **headers**(): `Promise`\<`Record`\<`string`, `string`\>\>
|
|
938
1565
|
|
|
939
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1566
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1356](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1356)
|
|
940
1567
|
|
|
941
1568
|
Manages HTTP headers for requests with multiple overloads.
|
|
942
1569
|
|
|
@@ -963,7 +1590,7 @@ await engine.headers('auth', 'token');
|
|
|
963
1590
|
|
|
964
1591
|
> **headers**(`name`): `Promise`\<`string`\>
|
|
965
1592
|
|
|
966
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1593
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1357](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1357)
|
|
967
1594
|
|
|
968
1595
|
Manages HTTP headers for requests with multiple overloads.
|
|
969
1596
|
|
|
@@ -998,7 +1625,7 @@ await engine.headers('auth', 'token');
|
|
|
998
1625
|
|
|
999
1626
|
> **headers**(`headers`, `replaced?`): `Promise`\<`boolean`\>
|
|
1000
1627
|
|
|
1001
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1628
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1358](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1358)
|
|
1002
1629
|
|
|
1003
1630
|
Manages HTTP headers for requests with multiple overloads.
|
|
1004
1631
|
|
|
@@ -1039,7 +1666,7 @@ await engine.headers('auth', 'token');
|
|
|
1039
1666
|
|
|
1040
1667
|
> **headers**(`name`, `value`): `Promise`\<`boolean`\>
|
|
1041
1668
|
|
|
1042
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1669
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1362](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L1362)
|
|
1043
1670
|
|
|
1044
1671
|
Manages HTTP headers for requests with multiple overloads.
|
|
1045
1672
|
|
|
@@ -1057,7 +1684,7 @@ Header name
|
|
|
1057
1684
|
|
|
1058
1685
|
Header value or `null` to remove
|
|
1059
1686
|
|
|
1060
|
-
`
|
|
1687
|
+
`string` | `null`
|
|
1061
1688
|
|
|
1062
1689
|
##### Returns
|
|
1063
1690
|
|
|
@@ -1082,7 +1709,7 @@ await engine.headers('auth', 'token');
|
|
|
1082
1709
|
|
|
1083
1710
|
> **initialize**(`context`, `options?`): `Promise`\<`void`\>
|
|
1084
1711
|
|
|
1085
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1712
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:861](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L861)
|
|
1086
1713
|
|
|
1087
1714
|
Initializes the fetch engine with provided context and options.
|
|
1088
1715
|
|
|
@@ -1117,7 +1744,7 @@ Automatically called when creating engine via `FetchEngine.create()`.
|
|
|
1117
1744
|
|
|
1118
1745
|
> **pause**(`message?`): `Promise`\<`void`\>
|
|
1119
1746
|
|
|
1120
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1747
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:778](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L778)
|
|
1121
1748
|
|
|
1122
1749
|
Pauses execution, allowing for manual intervention or inspection.
|
|
1123
1750
|
|
|
@@ -1145,7 +1772,7 @@ When no active page context exists
|
|
|
1145
1772
|
|
|
1146
1773
|
> **submit**(`selector?`, `options?`): `Promise`\<`void`\>
|
|
1147
1774
|
|
|
1148
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1775
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:756](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L756)
|
|
1149
1776
|
|
|
1150
1777
|
Submits a form.
|
|
1151
1778
|
|
|
@@ -1175,11 +1802,39 @@ When no active page context exists
|
|
|
1175
1802
|
|
|
1176
1803
|
***
|
|
1177
1804
|
|
|
1805
|
+
### trim()
|
|
1806
|
+
|
|
1807
|
+
> **trim**(`options`): `Promise`\<`void`\>
|
|
1808
|
+
|
|
1809
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:767](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L767)
|
|
1810
|
+
|
|
1811
|
+
Removes elements from the DOM based on selectors and presets.
|
|
1812
|
+
|
|
1813
|
+
#### Parameters
|
|
1814
|
+
|
|
1815
|
+
##### options
|
|
1816
|
+
|
|
1817
|
+
[`TrimActionOptions`](../interfaces/TrimActionOptions.md)
|
|
1818
|
+
|
|
1819
|
+
Trim options specifying selectors and presets
|
|
1820
|
+
|
|
1821
|
+
#### Returns
|
|
1822
|
+
|
|
1823
|
+
`Promise`\<`void`\>
|
|
1824
|
+
|
|
1825
|
+
Promise resolving when trim operation completes
|
|
1826
|
+
|
|
1827
|
+
#### Throws
|
|
1828
|
+
|
|
1829
|
+
When no active page context exists
|
|
1830
|
+
|
|
1831
|
+
***
|
|
1832
|
+
|
|
1178
1833
|
### waitFor()
|
|
1179
1834
|
|
|
1180
1835
|
> **waitFor**(`params?`): `Promise`\<`void`\>
|
|
1181
1836
|
|
|
1182
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1837
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:721](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L721)
|
|
1183
1838
|
|
|
1184
1839
|
Waits for specified condition before continuing.
|
|
1185
1840
|
|
|
@@ -1208,9 +1863,9 @@ await engine.waitFor({ selector: '#content' }); // Wait for element
|
|
|
1208
1863
|
|
|
1209
1864
|
### create()
|
|
1210
1865
|
|
|
1211
|
-
> `static` **create**(`ctx`, `options?`): `Promise`\<`
|
|
1866
|
+
> `static` **create**(`ctx`, `options?`): `Promise`\<`AnyFetchEngine` \| `undefined`\>
|
|
1212
1867
|
|
|
1213
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1868
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:332](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L332)
|
|
1214
1869
|
|
|
1215
1870
|
Factory method to create and initialize a fetch engine instance.
|
|
1216
1871
|
|
|
@@ -1230,7 +1885,7 @@ Configuration options
|
|
|
1230
1885
|
|
|
1231
1886
|
#### Returns
|
|
1232
1887
|
|
|
1233
|
-
`Promise`\<`
|
|
1888
|
+
`Promise`\<`AnyFetchEngine` \| `undefined`\>
|
|
1234
1889
|
|
|
1235
1890
|
Initialized fetch engine instance
|
|
1236
1891
|
|
|
@@ -1246,9 +1901,9 @@ Primary entry point for engine creation. Selects appropriate implementation base
|
|
|
1246
1901
|
|
|
1247
1902
|
### get()
|
|
1248
1903
|
|
|
1249
|
-
> `static` **get**(`id`): `
|
|
1904
|
+
> `static` **get**(`id`): `AnyFetchEngineCtor` \| `undefined`
|
|
1250
1905
|
|
|
1251
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1906
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:305](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L305)
|
|
1252
1907
|
|
|
1253
1908
|
Retrieves a fetch engine implementation by its unique ID.
|
|
1254
1909
|
|
|
@@ -1262,7 +1917,7 @@ The ID of the engine to retrieve
|
|
|
1262
1917
|
|
|
1263
1918
|
#### Returns
|
|
1264
1919
|
|
|
1265
|
-
`
|
|
1920
|
+
`AnyFetchEngineCtor` \| `undefined`
|
|
1266
1921
|
|
|
1267
1922
|
Engine class if found, otherwise `undefined`
|
|
1268
1923
|
|
|
@@ -1270,9 +1925,9 @@ Engine class if found, otherwise `undefined`
|
|
|
1270
1925
|
|
|
1271
1926
|
### getByMode()
|
|
1272
1927
|
|
|
1273
|
-
> `static` **getByMode**(`mode`): `
|
|
1928
|
+
> `static` **getByMode**(`mode`): `AnyFetchEngineCtor` \| `undefined`
|
|
1274
1929
|
|
|
1275
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1930
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:315](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L315)
|
|
1276
1931
|
|
|
1277
1932
|
Retrieves a fetch engine implementation by execution mode.
|
|
1278
1933
|
|
|
@@ -1286,7 +1941,7 @@ Execution mode (`'http'` or `'browser'`)
|
|
|
1286
1941
|
|
|
1287
1942
|
#### Returns
|
|
1288
1943
|
|
|
1289
|
-
`
|
|
1944
|
+
`AnyFetchEngineCtor` \| `undefined`
|
|
1290
1945
|
|
|
1291
1946
|
Engine class if found, otherwise `undefined`
|
|
1292
1947
|
|
|
@@ -1296,7 +1951,7 @@ Engine class if found, otherwise `undefined`
|
|
|
1296
1951
|
|
|
1297
1952
|
> `static` **register**(`engineClass`): `void`
|
|
1298
1953
|
|
|
1299
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1954
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:292](https://github.com/isdk/web-fetcher.js/blob/1f80c3c783d0455bd4ff73248c57c2c39ab9f7c9/src/engine/base.ts#L292)
|
|
1300
1955
|
|
|
1301
1956
|
Registers a fetch engine implementation with the global registry.
|
|
1302
1957
|
|