@isdk/web-fetcher 0.2.12 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.action.cn.md +197 -155
- package/README.action.extract.cn.md +263 -0
- package/README.action.extract.md +263 -0
- package/README.action.md +202 -147
- package/README.cn.md +25 -15
- package/README.engine.cn.md +118 -14
- package/README.engine.md +115 -14
- package/README.md +19 -10
- package/dist/index.d.mts +667 -50
- package/dist/index.d.ts +667 -50
- package/dist/index.js +1 -1
- package/dist/index.mjs +1 -1
- package/docs/README.md +19 -10
- package/docs/_media/README.action.md +202 -147
- package/docs/_media/README.cn.md +25 -15
- package/docs/_media/README.engine.md +115 -14
- package/docs/classes/CheerioFetchEngine.md +805 -135
- package/docs/classes/ClickAction.md +33 -33
- package/docs/classes/EvaluateAction.md +559 -0
- package/docs/classes/ExtractAction.md +33 -33
- package/docs/classes/FetchAction.md +39 -33
- package/docs/classes/FetchEngine.md +660 -122
- package/docs/classes/FetchSession.md +38 -16
- package/docs/classes/FillAction.md +33 -33
- package/docs/classes/GetContentAction.md +33 -33
- package/docs/classes/GotoAction.md +33 -33
- package/docs/classes/KeyboardPressAction.md +533 -0
- package/docs/classes/KeyboardTypeAction.md +533 -0
- package/docs/classes/MouseClickAction.md +533 -0
- package/docs/classes/MouseMoveAction.md +533 -0
- package/docs/classes/PauseAction.md +33 -33
- package/docs/classes/PlaywrightFetchEngine.md +820 -122
- package/docs/classes/SubmitAction.md +33 -33
- package/docs/classes/TrimAction.md +533 -0
- package/docs/classes/WaitForAction.md +33 -33
- package/docs/classes/WebFetcher.md +9 -9
- package/docs/enumerations/FetchActionResultStatus.md +4 -4
- package/docs/functions/fetchWeb.md +6 -6
- package/docs/globals.md +14 -0
- package/docs/interfaces/BaseFetchActionProperties.md +12 -12
- package/docs/interfaces/BaseFetchCollectorActionProperties.md +16 -16
- package/docs/interfaces/BaseFetcherProperties.md +32 -28
- package/docs/interfaces/Cookie.md +14 -14
- package/docs/interfaces/DispatchedEngineAction.md +4 -4
- package/docs/interfaces/EvaluateActionOptions.md +81 -0
- package/docs/interfaces/ExtractActionProperties.md +12 -12
- package/docs/interfaces/FetchActionInContext.md +15 -15
- package/docs/interfaces/FetchActionProperties.md +13 -13
- package/docs/interfaces/FetchActionResult.md +6 -6
- package/docs/interfaces/FetchContext.md +42 -38
- package/docs/interfaces/FetchEngineContext.md +37 -33
- package/docs/interfaces/FetchMetadata.md +5 -5
- package/docs/interfaces/FetchResponse.md +14 -14
- package/docs/interfaces/FetchReturnTypeRegistry.md +8 -8
- package/docs/interfaces/FetchSite.md +35 -31
- package/docs/interfaces/FetcherOptions.md +34 -30
- package/docs/interfaces/GotoActionOptions.md +14 -6
- package/docs/interfaces/KeyboardPressParams.md +25 -0
- package/docs/interfaces/KeyboardTypeParams.md +25 -0
- package/docs/interfaces/MouseClickParams.md +49 -0
- package/docs/interfaces/MouseMoveParams.md +41 -0
- package/docs/interfaces/PendingEngineRequest.md +3 -3
- package/docs/interfaces/StorageOptions.md +5 -5
- package/docs/interfaces/SubmitActionOptions.md +2 -2
- package/docs/interfaces/TrimActionOptions.md +27 -0
- package/docs/interfaces/WaitForActionOptions.md +5 -5
- package/docs/type-aliases/BaseFetchActionOptions.md +1 -1
- package/docs/type-aliases/BaseFetchCollectorOptions.md +1 -1
- package/docs/type-aliases/BrowserEngine.md +1 -1
- package/docs/type-aliases/FetchActionCapabilities.md +1 -1
- package/docs/type-aliases/FetchActionCapabilityMode.md +1 -1
- package/docs/type-aliases/FetchActionOptions.md +1 -1
- package/docs/type-aliases/FetchEngineAction.md +2 -2
- package/docs/type-aliases/FetchEngineType.md +1 -1
- package/docs/type-aliases/FetchReturnType.md +1 -1
- package/docs/type-aliases/FetchReturnTypeFor.md +1 -1
- package/docs/type-aliases/OnFetchPauseCallback.md +1 -1
- package/docs/type-aliases/ResourceType.md +1 -1
- package/docs/type-aliases/TrimPreset.md +13 -0
- package/docs/variables/DefaultFetcherProperties.md +1 -1
- package/docs/variables/FetcherOptionKeys.md +1 -1
- package/docs/variables/TRIM_PRESETS.md +11 -0
- package/package.json +11 -11
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
# Abstract Class: FetchEngine\<TContext, TCrawler, TOptions\>
|
|
8
8
|
|
|
9
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
9
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:289](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L289)
|
|
10
10
|
|
|
11
11
|
## Extended by
|
|
12
12
|
|
|
@@ -27,6 +27,10 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:172](https://github.com/isd
|
|
|
27
27
|
|
|
28
28
|
`TOptions` *extends* `BasicCrawlerOptions`\<`TContext`\> = `any`
|
|
29
29
|
|
|
30
|
+
## Implements
|
|
31
|
+
|
|
32
|
+
- `IExtractEngine`
|
|
33
|
+
|
|
30
34
|
## Constructors
|
|
31
35
|
|
|
32
36
|
### Constructor
|
|
@@ -43,7 +47,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:172](https://github.com/isd
|
|
|
43
47
|
|
|
44
48
|
> `protected` `optional` **\_initialCookies**: [`Cookie`](../interfaces/Cookie.md)[]
|
|
45
49
|
|
|
46
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
50
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:396](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L396)
|
|
47
51
|
|
|
48
52
|
***
|
|
49
53
|
|
|
@@ -51,7 +55,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:278](https://github.com/isd
|
|
|
51
55
|
|
|
52
56
|
> `protected` **\_initializedSessions**: `Set`\<`string`\>
|
|
53
57
|
|
|
54
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
58
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:397](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L397)
|
|
55
59
|
|
|
56
60
|
***
|
|
57
61
|
|
|
@@ -59,7 +63,23 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:279](https://github.com/isd
|
|
|
59
63
|
|
|
60
64
|
> `protected` **actionEmitter**: `EventEmitter`
|
|
61
65
|
|
|
62
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
66
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:401](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L401)
|
|
67
|
+
|
|
68
|
+
***
|
|
69
|
+
|
|
70
|
+
### actionQueue
|
|
71
|
+
|
|
72
|
+
> `protected` **actionQueue**: [`DispatchedEngineAction`](../interfaces/DispatchedEngineAction.md)[] = `[]`
|
|
73
|
+
|
|
74
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:408](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L408)
|
|
75
|
+
|
|
76
|
+
***
|
|
77
|
+
|
|
78
|
+
### activeContext?
|
|
79
|
+
|
|
80
|
+
> `protected` `optional` **activeContext**: `TContext`
|
|
81
|
+
|
|
82
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:405](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L405)
|
|
63
83
|
|
|
64
84
|
***
|
|
65
85
|
|
|
@@ -67,7 +87,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:283](https://github.com/isd
|
|
|
67
87
|
|
|
68
88
|
> `protected` **blockedTypes**: `Set`\<`string`\>
|
|
69
89
|
|
|
70
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
90
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:410](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L410)
|
|
71
91
|
|
|
72
92
|
***
|
|
73
93
|
|
|
@@ -75,7 +95,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:288](https://github.com/isd
|
|
|
75
95
|
|
|
76
96
|
> `protected` `optional` **config**: `Configuration`
|
|
77
97
|
|
|
78
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
98
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:390](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L390)
|
|
79
99
|
|
|
80
100
|
***
|
|
81
101
|
|
|
@@ -83,7 +103,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:272](https://github.com/isd
|
|
|
83
103
|
|
|
84
104
|
> `protected` `optional` **crawler**: `TCrawler`
|
|
85
105
|
|
|
86
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
106
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:387](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L387)
|
|
87
107
|
|
|
88
108
|
***
|
|
89
109
|
|
|
@@ -91,7 +111,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:269](https://github.com/isd
|
|
|
91
111
|
|
|
92
112
|
> `protected` `optional` **crawlerRunPromise**: `Promise`\<`FinalStatistics`\>
|
|
93
113
|
|
|
94
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
114
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:389](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L389)
|
|
95
115
|
|
|
96
116
|
***
|
|
97
117
|
|
|
@@ -99,7 +119,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:271](https://github.com/isd
|
|
|
99
119
|
|
|
100
120
|
> `protected` `optional` **ctx**: [`FetchEngineContext`](../interfaces/FetchEngineContext.md)
|
|
101
121
|
|
|
102
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
122
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:385](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L385)
|
|
103
123
|
|
|
104
124
|
***
|
|
105
125
|
|
|
@@ -107,7 +127,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:267](https://github.com/isd
|
|
|
107
127
|
|
|
108
128
|
> `protected` `optional` **currentSession**: `Session`
|
|
109
129
|
|
|
110
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
130
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:398](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L398)
|
|
111
131
|
|
|
112
132
|
***
|
|
113
133
|
|
|
@@ -115,7 +135,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:280](https://github.com/isd
|
|
|
115
135
|
|
|
116
136
|
> `protected` **hdrs**: `Record`\<`string`, `string`\> = `{}`
|
|
117
137
|
|
|
118
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
138
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:395](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L395)
|
|
119
139
|
|
|
120
140
|
***
|
|
121
141
|
|
|
@@ -123,7 +143,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:277](https://github.com/isd
|
|
|
123
143
|
|
|
124
144
|
> `protected` `optional` **isCrawlerReady**: `boolean`
|
|
125
145
|
|
|
126
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
146
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:388](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L388)
|
|
127
147
|
|
|
128
148
|
***
|
|
129
149
|
|
|
@@ -131,7 +151,15 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:270](https://github.com/isd
|
|
|
131
151
|
|
|
132
152
|
> `protected` **isEngineDisposed**: `boolean` = `false`
|
|
133
153
|
|
|
134
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
154
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:403](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L403)
|
|
155
|
+
|
|
156
|
+
***
|
|
157
|
+
|
|
158
|
+
### isExecutingAction
|
|
159
|
+
|
|
160
|
+
> `protected` **isExecutingAction**: `boolean` = `false`
|
|
161
|
+
|
|
162
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:406](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L406)
|
|
135
163
|
|
|
136
164
|
***
|
|
137
165
|
|
|
@@ -139,7 +167,15 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:285](https://github.com/isd
|
|
|
139
167
|
|
|
140
168
|
> `protected` **isPageActive**: `boolean` = `false`
|
|
141
169
|
|
|
142
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
170
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:402](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L402)
|
|
171
|
+
|
|
172
|
+
***
|
|
173
|
+
|
|
174
|
+
### isProcessingActionLoop
|
|
175
|
+
|
|
176
|
+
> `protected` **isProcessingActionLoop**: `boolean` = `false`
|
|
177
|
+
|
|
178
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:409](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L409)
|
|
143
179
|
|
|
144
180
|
***
|
|
145
181
|
|
|
@@ -147,7 +183,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:284](https://github.com/isd
|
|
|
147
183
|
|
|
148
184
|
> `protected` `optional` **kvStore**: `KeyValueStore`
|
|
149
185
|
|
|
150
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
186
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:392](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L392)
|
|
151
187
|
|
|
152
188
|
***
|
|
153
189
|
|
|
@@ -155,7 +191,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:274](https://github.com/isd
|
|
|
155
191
|
|
|
156
192
|
> `protected` `optional` **lastResponse**: [`FetchResponse`](../interfaces/FetchResponse.md)
|
|
157
193
|
|
|
158
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
194
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:407](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L407)
|
|
159
195
|
|
|
160
196
|
***
|
|
161
197
|
|
|
@@ -163,7 +199,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:287](https://github.com/isd
|
|
|
163
199
|
|
|
164
200
|
> `protected` **navigationLock**: `PromiseLock`
|
|
165
201
|
|
|
166
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
202
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:404](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L404)
|
|
167
203
|
|
|
168
204
|
***
|
|
169
205
|
|
|
@@ -171,7 +207,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:286](https://github.com/isd
|
|
|
171
207
|
|
|
172
208
|
> `protected` `optional` **opts**: [`BaseFetcherProperties`](../interfaces/BaseFetcherProperties.md)
|
|
173
209
|
|
|
174
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
210
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:386](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L386)
|
|
175
211
|
|
|
176
212
|
***
|
|
177
213
|
|
|
@@ -179,7 +215,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:268](https://github.com/isd
|
|
|
179
215
|
|
|
180
216
|
> `protected` **pendingRequests**: `Map`\<`string`, [`PendingEngineRequest`](../interfaces/PendingEngineRequest.md)\>
|
|
181
217
|
|
|
182
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
218
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:399](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L399)
|
|
183
219
|
|
|
184
220
|
***
|
|
185
221
|
|
|
@@ -187,7 +223,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:281](https://github.com/isd
|
|
|
187
223
|
|
|
188
224
|
> `protected` `optional` **proxyConfiguration**: `ProxyConfiguration`
|
|
189
225
|
|
|
190
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
226
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:393](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L393)
|
|
191
227
|
|
|
192
228
|
***
|
|
193
229
|
|
|
@@ -195,7 +231,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:275](https://github.com/isd
|
|
|
195
231
|
|
|
196
232
|
> `protected` **requestCounter**: `number` = `0`
|
|
197
233
|
|
|
198
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
234
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:400](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L400)
|
|
199
235
|
|
|
200
236
|
***
|
|
201
237
|
|
|
@@ -203,7 +239,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:282](https://github.com/isd
|
|
|
203
239
|
|
|
204
240
|
> `protected` `optional` **requestQueue**: `RequestQueue`
|
|
205
241
|
|
|
206
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
242
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:391](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L391)
|
|
207
243
|
|
|
208
244
|
***
|
|
209
245
|
|
|
@@ -211,7 +247,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:273](https://github.com/isd
|
|
|
211
247
|
|
|
212
248
|
> `readonly` `static` **id**: `string`
|
|
213
249
|
|
|
214
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
250
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:375](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L375)
|
|
215
251
|
|
|
216
252
|
Unique identifier for the engine implementation.
|
|
217
253
|
|
|
@@ -225,7 +261,7 @@ Must be defined by concrete implementations. Used for registration and lookup in
|
|
|
225
261
|
|
|
226
262
|
> `readonly` `static` **mode**: [`FetchEngineType`](../type-aliases/FetchEngineType.md)
|
|
227
263
|
|
|
228
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
264
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:383](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L383)
|
|
229
265
|
|
|
230
266
|
Execution mode of the engine (`'http'` or `'browser'`).
|
|
231
267
|
|
|
@@ -239,15 +275,15 @@ Must be defined by concrete implementations. Indicates whether engine operates a
|
|
|
239
275
|
|
|
240
276
|
#### Get Signature
|
|
241
277
|
|
|
242
|
-
> **get** **context**():
|
|
278
|
+
> **get** **context**(): [`FetchEngineContext`](../interfaces/FetchEngineContext.md) \| `undefined`
|
|
243
279
|
|
|
244
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
280
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:919](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L919)
|
|
245
281
|
|
|
246
282
|
Gets the fetch engine context associated with this instance.
|
|
247
283
|
|
|
248
284
|
##### Returns
|
|
249
285
|
|
|
250
|
-
|
|
286
|
+
[`FetchEngineContext`](../interfaces/FetchEngineContext.md) \| `undefined`
|
|
251
287
|
|
|
252
288
|
***
|
|
253
289
|
|
|
@@ -257,7 +293,7 @@ Gets the fetch engine context associated with this instance.
|
|
|
257
293
|
|
|
258
294
|
> **get** **id**(): `string`
|
|
259
295
|
|
|
260
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
296
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:894](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L894)
|
|
261
297
|
|
|
262
298
|
Gets the unique identifier of this engine implementation.
|
|
263
299
|
|
|
@@ -273,7 +309,7 @@ Gets the unique identifier of this engine implementation.
|
|
|
273
309
|
|
|
274
310
|
> **get** **mode**(): [`FetchEngineType`](../type-aliases/FetchEngineType.md)
|
|
275
311
|
|
|
276
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
312
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:912](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L912)
|
|
277
313
|
|
|
278
314
|
Gets the execution mode of this engine (`'http'` or `'browser'`).
|
|
279
315
|
|
|
@@ -287,7 +323,7 @@ Gets the execution mode of this engine (`'http'` or `'browser'`).
|
|
|
287
323
|
|
|
288
324
|
> `abstract` `protected` **\_buildResponse**(`context`): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
289
325
|
|
|
290
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
326
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:658](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L658)
|
|
291
327
|
|
|
292
328
|
**`Internal`**
|
|
293
329
|
|
|
@@ -317,7 +353,7 @@ Converts implementation-specific context (Playwright `page` or Cheerio `$`) to s
|
|
|
317
353
|
|
|
318
354
|
> `protected` `optional` **\_cleanup**(): `Promise`\<`void`\>
|
|
319
355
|
|
|
320
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
356
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:420](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L420)
|
|
321
357
|
|
|
322
358
|
#### Returns
|
|
323
359
|
|
|
@@ -329,7 +365,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:290](https://github.com/isd
|
|
|
329
365
|
|
|
330
366
|
> `protected` **\_commonCleanup**(): `Promise`\<`void`\>
|
|
331
367
|
|
|
332
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
368
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1321](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1321)
|
|
333
369
|
|
|
334
370
|
#### Returns
|
|
335
371
|
|
|
@@ -337,11 +373,51 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:1239](https://github.com/is
|
|
|
337
373
|
|
|
338
374
|
***
|
|
339
375
|
|
|
376
|
+
### \_contains()
|
|
377
|
+
|
|
378
|
+
> `abstract` **\_contains**(`container`, `element`): `Promise`\<`boolean`\>
|
|
379
|
+
|
|
380
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:532](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L532)
|
|
381
|
+
|
|
382
|
+
**`Internal`**
|
|
383
|
+
|
|
384
|
+
Checks if the `container` scope contains the `element` scope.
|
|
385
|
+
|
|
386
|
+
#### Parameters
|
|
387
|
+
|
|
388
|
+
##### container
|
|
389
|
+
|
|
390
|
+
`any`
|
|
391
|
+
|
|
392
|
+
The potential ancestor element.
|
|
393
|
+
|
|
394
|
+
##### element
|
|
395
|
+
|
|
396
|
+
`any`
|
|
397
|
+
|
|
398
|
+
The potential descendant element.
|
|
399
|
+
|
|
400
|
+
#### Returns
|
|
401
|
+
|
|
402
|
+
`Promise`\<`boolean`\>
|
|
403
|
+
|
|
404
|
+
A promise resolving to `true` if `container` contains `element`.
|
|
405
|
+
|
|
406
|
+
#### See
|
|
407
|
+
|
|
408
|
+
IExtractEngine.\_contains for implementation details.
|
|
409
|
+
|
|
410
|
+
#### Implementation of
|
|
411
|
+
|
|
412
|
+
`IExtractEngine._contains`
|
|
413
|
+
|
|
414
|
+
***
|
|
415
|
+
|
|
340
416
|
### \_createCrawler()
|
|
341
417
|
|
|
342
418
|
> `abstract` `protected` **\_createCrawler**(`options`, `config?`): `TCrawler`
|
|
343
419
|
|
|
344
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
420
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:635](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L635)
|
|
345
421
|
|
|
346
422
|
**`Internal`**
|
|
347
423
|
|
|
@@ -369,7 +445,7 @@ The final crawler options.
|
|
|
369
445
|
|
|
370
446
|
> `protected` **\_executePendingActions**(`context`): `Promise`\<`void`\>
|
|
371
447
|
|
|
372
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
448
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1162](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1162)
|
|
373
449
|
|
|
374
450
|
**`Internal`**
|
|
375
451
|
|
|
@@ -416,9 +492,9 @@ If called outside valid page context window (`!this.isPageActive`)
|
|
|
416
492
|
|
|
417
493
|
### \_extract()
|
|
418
494
|
|
|
419
|
-
> `protected` **\_extract**(`schema`, `
|
|
495
|
+
> `protected` **\_extract**(`schema`, `scope`, `parentStrict?`): `Promise`\<`any`\>
|
|
420
496
|
|
|
421
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
497
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:563](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L563)
|
|
422
498
|
|
|
423
499
|
#### Parameters
|
|
424
500
|
|
|
@@ -426,10 +502,14 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:365](https://github.com/isd
|
|
|
426
502
|
|
|
427
503
|
`ExtractSchema`
|
|
428
504
|
|
|
429
|
-
#####
|
|
505
|
+
##### scope
|
|
430
506
|
|
|
431
507
|
`any`
|
|
432
508
|
|
|
509
|
+
##### parentStrict?
|
|
510
|
+
|
|
511
|
+
`boolean`
|
|
512
|
+
|
|
433
513
|
#### Returns
|
|
434
514
|
|
|
435
515
|
`Promise`\<`any`\>
|
|
@@ -438,9 +518,9 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:365](https://github.com/isd
|
|
|
438
518
|
|
|
439
519
|
### \_extractColumnar()
|
|
440
520
|
|
|
441
|
-
> `protected` **\_extractColumnar**(`schema`, `container`, `opts?`): `Promise`\<`
|
|
521
|
+
> `protected` **\_extractColumnar**(`schema`, `container`, `opts?`): `Promise`\<`any`[] \| `null`\>
|
|
442
522
|
|
|
443
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
523
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:605](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L605)
|
|
444
524
|
|
|
445
525
|
**`Internal`**
|
|
446
526
|
|
|
@@ -468,7 +548,7 @@ Columnar extraction options (strict, inference).
|
|
|
468
548
|
|
|
469
549
|
#### Returns
|
|
470
550
|
|
|
471
|
-
`Promise`\<`
|
|
551
|
+
`Promise`\<`any`[] \| `null`\>
|
|
472
552
|
|
|
473
553
|
An array of extracted items, or null if requirements aren't met.
|
|
474
554
|
|
|
@@ -476,9 +556,9 @@ An array of extracted items, or null if requirements aren't met.
|
|
|
476
556
|
|
|
477
557
|
### \_extractNested()
|
|
478
558
|
|
|
479
|
-
> `protected` **\_extractNested**(`items`, `elements`): `Promise`\<`any`[]\>
|
|
559
|
+
> `protected` **\_extractNested**(`items`, `elements`, `opts?`): `Promise`\<`any`[]\>
|
|
480
560
|
|
|
481
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
561
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:588](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L588)
|
|
482
562
|
|
|
483
563
|
**`Internal`**
|
|
484
564
|
|
|
@@ -498,6 +578,12 @@ The schema for each item.
|
|
|
498
578
|
|
|
499
579
|
The list of item elements.
|
|
500
580
|
|
|
581
|
+
##### opts?
|
|
582
|
+
|
|
583
|
+
###### strict?
|
|
584
|
+
|
|
585
|
+
`boolean`
|
|
586
|
+
|
|
501
587
|
#### Returns
|
|
502
588
|
|
|
503
589
|
`Promise`\<`any`[]\>
|
|
@@ -506,9 +592,9 @@ The list of item elements.
|
|
|
506
592
|
|
|
507
593
|
### \_extractSegmented()
|
|
508
594
|
|
|
509
|
-
> `protected` **\_extractSegmented**(`schema`, `container`, `opts?`): `Promise`\<`
|
|
595
|
+
> `protected` **\_extractSegmented**(`schema`, `container`, `opts?`): `Promise`\<`any`[] \| `null`\>
|
|
510
596
|
|
|
511
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
597
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:622](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L622)
|
|
512
598
|
|
|
513
599
|
**`Internal`**
|
|
514
600
|
|
|
@@ -536,7 +622,7 @@ Segmented extraction options (anchor).
|
|
|
536
622
|
|
|
537
623
|
#### Returns
|
|
538
624
|
|
|
539
|
-
`Promise`\<`
|
|
625
|
+
`Promise`\<`any`[] \| `null`\>
|
|
540
626
|
|
|
541
627
|
An array of extracted items.
|
|
542
628
|
|
|
@@ -544,9 +630,9 @@ An array of extracted items.
|
|
|
544
630
|
|
|
545
631
|
### \_extractValue()
|
|
546
632
|
|
|
547
|
-
> `abstract`
|
|
633
|
+
> `abstract` **\_extractValue**(`schema`, `scope`): `Promise`\<`any`\>
|
|
548
634
|
|
|
549
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
635
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:466](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L466)
|
|
550
636
|
|
|
551
637
|
**`Internal`**
|
|
552
638
|
|
|
@@ -560,23 +646,169 @@ Extracts a primitive value from the element based on schema.
|
|
|
560
646
|
|
|
561
647
|
Value extraction schema.
|
|
562
648
|
|
|
563
|
-
#####
|
|
649
|
+
##### scope
|
|
564
650
|
|
|
565
651
|
`any`
|
|
566
652
|
|
|
567
|
-
The element
|
|
653
|
+
The element scope.
|
|
568
654
|
|
|
569
655
|
#### Returns
|
|
570
656
|
|
|
571
657
|
`Promise`\<`any`\>
|
|
572
658
|
|
|
659
|
+
Extracted value.
|
|
660
|
+
|
|
661
|
+
#### See
|
|
662
|
+
|
|
663
|
+
IExtractEngine.\_extractValue for behavior contract.
|
|
664
|
+
|
|
665
|
+
#### Implementation of
|
|
666
|
+
|
|
667
|
+
`IExtractEngine._extractValue`
|
|
668
|
+
|
|
669
|
+
***
|
|
670
|
+
|
|
671
|
+
### \_findClosestAncestor()
|
|
672
|
+
|
|
673
|
+
> `abstract` **\_findClosestAncestor**(`scope`, `candidates`): `Promise`\<`any`\>
|
|
674
|
+
|
|
675
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:518](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L518)
|
|
676
|
+
|
|
677
|
+
**`Internal`**
|
|
678
|
+
|
|
679
|
+
Finds the closest ancestor of `scope` (including itself) that exists in the `candidates` array.
|
|
680
|
+
|
|
681
|
+
#### Parameters
|
|
682
|
+
|
|
683
|
+
##### scope
|
|
684
|
+
|
|
685
|
+
`any`
|
|
686
|
+
|
|
687
|
+
The starting element.
|
|
688
|
+
|
|
689
|
+
##### candidates
|
|
690
|
+
|
|
691
|
+
`any`[]
|
|
692
|
+
|
|
693
|
+
The array of potential ancestor scopes.
|
|
694
|
+
|
|
695
|
+
#### Returns
|
|
696
|
+
|
|
697
|
+
`Promise`\<`any`\>
|
|
698
|
+
|
|
699
|
+
A promise resolving to the matching candidate scope, or `null` if none found.
|
|
700
|
+
|
|
701
|
+
#### See
|
|
702
|
+
|
|
703
|
+
IExtractEngine.\_findClosestAncestor for implementation details.
|
|
704
|
+
|
|
705
|
+
#### Implementation of
|
|
706
|
+
|
|
707
|
+
`IExtractEngine._findClosestAncestor`
|
|
708
|
+
|
|
709
|
+
***
|
|
710
|
+
|
|
711
|
+
### \_findCommonAncestor()
|
|
712
|
+
|
|
713
|
+
> `abstract` **\_findCommonAncestor**(`scope1`, `scope2`): `Promise`\<`any`\>
|
|
714
|
+
|
|
715
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:545](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L545)
|
|
716
|
+
|
|
717
|
+
**`Internal`**
|
|
718
|
+
|
|
719
|
+
Finds the Lowest Common Ancestor (LCA) of two element scopes.
|
|
720
|
+
|
|
721
|
+
#### Parameters
|
|
722
|
+
|
|
723
|
+
##### scope1
|
|
724
|
+
|
|
725
|
+
`any`
|
|
726
|
+
|
|
727
|
+
The first element scope.
|
|
728
|
+
|
|
729
|
+
##### scope2
|
|
730
|
+
|
|
731
|
+
`any`
|
|
732
|
+
|
|
733
|
+
The second element scope.
|
|
734
|
+
|
|
735
|
+
#### Returns
|
|
736
|
+
|
|
737
|
+
`Promise`\<`any`\>
|
|
738
|
+
|
|
739
|
+
A promise resolving to the LCA element scope, or `null` if none found.
|
|
740
|
+
|
|
741
|
+
#### Implementation of
|
|
742
|
+
|
|
743
|
+
`IExtractEngine._findCommonAncestor`
|
|
744
|
+
|
|
745
|
+
***
|
|
746
|
+
|
|
747
|
+
### \_findContainerChild()
|
|
748
|
+
|
|
749
|
+
> `abstract` **\_findContainerChild**(`element`, `container`): `Promise`\<`any`\>
|
|
750
|
+
|
|
751
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:558](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L558)
|
|
752
|
+
|
|
753
|
+
**`Internal`**
|
|
754
|
+
|
|
755
|
+
Finds the direct child of container that contains element.
|
|
756
|
+
|
|
757
|
+
#### Parameters
|
|
758
|
+
|
|
759
|
+
##### element
|
|
760
|
+
|
|
761
|
+
`any`
|
|
762
|
+
|
|
763
|
+
The descendant element.
|
|
764
|
+
|
|
765
|
+
##### container
|
|
766
|
+
|
|
767
|
+
`any`
|
|
768
|
+
|
|
769
|
+
The container element.
|
|
770
|
+
|
|
771
|
+
#### Returns
|
|
772
|
+
|
|
773
|
+
`Promise`\<`any`\>
|
|
774
|
+
|
|
775
|
+
The child element of container, or null.
|
|
776
|
+
|
|
777
|
+
#### Implementation of
|
|
778
|
+
|
|
779
|
+
`IExtractEngine._findContainerChild`
|
|
780
|
+
|
|
781
|
+
***
|
|
782
|
+
|
|
783
|
+
### \_getInitialElementScope()
|
|
784
|
+
|
|
785
|
+
> `abstract` `protected` **\_getInitialElementScope**(`context`): `any`
|
|
786
|
+
|
|
787
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1083](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1083)
|
|
788
|
+
|
|
789
|
+
**`Internal`**
|
|
790
|
+
|
|
791
|
+
Gets the initial scope for extraction for the specific engine.
|
|
792
|
+
|
|
793
|
+
#### Parameters
|
|
794
|
+
|
|
795
|
+
##### context
|
|
796
|
+
|
|
797
|
+
`TContext`
|
|
798
|
+
|
|
799
|
+
Crawlee crawling context
|
|
800
|
+
|
|
801
|
+
#### Returns
|
|
802
|
+
|
|
803
|
+
`any`
|
|
804
|
+
|
|
573
805
|
***
|
|
574
806
|
|
|
575
807
|
### \_getSpecificCrawlerOptions()
|
|
576
808
|
|
|
577
809
|
> `abstract` `protected` **\_getSpecificCrawlerOptions**(`ctx`): `Partial`\<`TOptions`\> \| `Promise`\<`Partial`\<`TOptions`\>\>
|
|
578
810
|
|
|
579
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
811
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:645](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L645)
|
|
580
812
|
|
|
581
813
|
**`Internal`**
|
|
582
814
|
|
|
@@ -596,59 +828,129 @@ The fetch engine context.
|
|
|
596
828
|
|
|
597
829
|
***
|
|
598
830
|
|
|
599
|
-
### \
|
|
831
|
+
### \_getTrimInfo()
|
|
600
832
|
|
|
601
|
-
> `protected` **\
|
|
833
|
+
> `protected` **\_getTrimInfo**(`options`): `object`
|
|
602
834
|
|
|
603
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
835
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:422](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L422)
|
|
604
836
|
|
|
605
837
|
#### Parameters
|
|
606
838
|
|
|
607
|
-
#####
|
|
839
|
+
##### options
|
|
608
840
|
|
|
609
|
-
`
|
|
841
|
+
[`TrimActionOptions`](../interfaces/TrimActionOptions.md)
|
|
610
842
|
|
|
611
843
|
#### Returns
|
|
612
844
|
|
|
613
|
-
`
|
|
845
|
+
`object`
|
|
846
|
+
|
|
847
|
+
##### removeComments
|
|
848
|
+
|
|
849
|
+
> **removeComments**: `boolean`
|
|
850
|
+
|
|
851
|
+
##### removeHidden
|
|
852
|
+
|
|
853
|
+
> **removeHidden**: `boolean`
|
|
854
|
+
|
|
855
|
+
##### selectors
|
|
856
|
+
|
|
857
|
+
> **selectors**: `string`[] = `allSelectors`
|
|
858
|
+
|
|
859
|
+
***
|
|
860
|
+
|
|
861
|
+
### \_handlePause()
|
|
862
|
+
|
|
863
|
+
> `protected` **\_handlePause**(`action`): `Promise`\<`void`\>
|
|
864
|
+
|
|
865
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1122](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1122)
|
|
866
|
+
|
|
867
|
+
#### Parameters
|
|
868
|
+
|
|
869
|
+
##### action
|
|
870
|
+
|
|
871
|
+
###### message?
|
|
872
|
+
|
|
873
|
+
`string`
|
|
874
|
+
|
|
875
|
+
#### Returns
|
|
876
|
+
|
|
877
|
+
`Promise`\<`void`\>
|
|
614
878
|
|
|
615
879
|
***
|
|
616
880
|
|
|
617
881
|
### \_isSameElement()
|
|
618
882
|
|
|
619
|
-
> `abstract`
|
|
883
|
+
> `abstract` **\_isSameElement**(`scope1`, `scope2`): `Promise`\<`boolean`\>
|
|
620
884
|
|
|
621
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
885
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:490](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L490)
|
|
622
886
|
|
|
623
887
|
**`Internal`**
|
|
624
888
|
|
|
625
|
-
Checks if two elements are the same.
|
|
889
|
+
Checks if two elements are the same identity.
|
|
626
890
|
|
|
627
891
|
#### Parameters
|
|
628
892
|
|
|
629
|
-
#####
|
|
893
|
+
##### scope1
|
|
630
894
|
|
|
631
895
|
`any`
|
|
632
896
|
|
|
633
|
-
First element.
|
|
897
|
+
First element scope.
|
|
634
898
|
|
|
635
|
-
#####
|
|
899
|
+
##### scope2
|
|
636
900
|
|
|
637
901
|
`any`
|
|
638
902
|
|
|
639
|
-
Second element.
|
|
903
|
+
Second element scope.
|
|
640
904
|
|
|
641
905
|
#### Returns
|
|
642
906
|
|
|
643
907
|
`Promise`\<`boolean`\>
|
|
644
908
|
|
|
909
|
+
True if they are the same DOM node.
|
|
910
|
+
|
|
911
|
+
#### Implementation of
|
|
912
|
+
|
|
913
|
+
`IExtractEngine._isSameElement`
|
|
914
|
+
|
|
915
|
+
***
|
|
916
|
+
|
|
917
|
+
### \_logDebug()
|
|
918
|
+
|
|
919
|
+
> **\_logDebug**(`category`, ...`args`): `void`
|
|
920
|
+
|
|
921
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:412](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L412)
|
|
922
|
+
|
|
923
|
+
Logs debug information if debug mode is enabled.
|
|
924
|
+
|
|
925
|
+
#### Parameters
|
|
926
|
+
|
|
927
|
+
##### category
|
|
928
|
+
|
|
929
|
+
`string`
|
|
930
|
+
|
|
931
|
+
The category of the log message.
|
|
932
|
+
|
|
933
|
+
##### args
|
|
934
|
+
|
|
935
|
+
...`any`[]
|
|
936
|
+
|
|
937
|
+
Arguments to log.
|
|
938
|
+
|
|
939
|
+
#### Returns
|
|
940
|
+
|
|
941
|
+
`void`
|
|
942
|
+
|
|
943
|
+
#### Implementation of
|
|
944
|
+
|
|
945
|
+
`IExtractEngine._logDebug`
|
|
946
|
+
|
|
645
947
|
***
|
|
646
948
|
|
|
647
949
|
### \_nextSiblingsUntil()
|
|
648
950
|
|
|
649
|
-
> `abstract`
|
|
951
|
+
> `abstract` **\_nextSiblingsUntil**(`scope`, `untilSelector?`): `Promise`\<`any`[]\>
|
|
650
952
|
|
|
651
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
953
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:504](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L504)
|
|
652
954
|
|
|
653
955
|
**`Internal`**
|
|
654
956
|
|
|
@@ -657,29 +959,35 @@ Used in 'segmented' extraction mode.
|
|
|
657
959
|
|
|
658
960
|
#### Parameters
|
|
659
961
|
|
|
660
|
-
#####
|
|
962
|
+
##### scope
|
|
661
963
|
|
|
662
964
|
`any`
|
|
663
965
|
|
|
664
|
-
The anchor element.
|
|
966
|
+
The anchor element scope.
|
|
665
967
|
|
|
666
968
|
##### untilSelector?
|
|
667
969
|
|
|
668
970
|
`string`
|
|
669
971
|
|
|
670
|
-
Optional selector that marks the end of the segment.
|
|
972
|
+
Optional selector that marks the end of the segment (exclusive).
|
|
671
973
|
|
|
672
974
|
#### Returns
|
|
673
975
|
|
|
674
976
|
`Promise`\<`any`[]\>
|
|
675
977
|
|
|
978
|
+
List of sibling elements between anchor and untilSelector.
|
|
979
|
+
|
|
980
|
+
#### Implementation of
|
|
981
|
+
|
|
982
|
+
`IExtractEngine._nextSiblingsUntil`
|
|
983
|
+
|
|
676
984
|
***
|
|
677
985
|
|
|
678
986
|
### \_normalizeArrayMode()
|
|
679
987
|
|
|
680
988
|
> `protected` **\_normalizeArrayMode**(`mode?`): `any`
|
|
681
989
|
|
|
682
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
990
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:576](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L576)
|
|
683
991
|
|
|
684
992
|
**`Internal`**
|
|
685
993
|
|
|
@@ -699,41 +1007,59 @@ The mode string or options object.
|
|
|
699
1007
|
|
|
700
1008
|
***
|
|
701
1009
|
|
|
702
|
-
### \
|
|
1010
|
+
### \_parentElement()
|
|
1011
|
+
|
|
1012
|
+
> `abstract` **\_parentElement**(`scope`): `Promise`\<`any`\>
|
|
1013
|
+
|
|
1014
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:478](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L478)
|
|
703
1015
|
|
|
704
|
-
|
|
1016
|
+
**`Internal`**
|
|
705
1017
|
|
|
706
|
-
|
|
1018
|
+
Gets the parent element of the given element.
|
|
707
1019
|
|
|
708
1020
|
#### Parameters
|
|
709
1021
|
|
|
710
|
-
#####
|
|
1022
|
+
##### scope
|
|
711
1023
|
|
|
712
|
-
`
|
|
1024
|
+
`any`
|
|
1025
|
+
|
|
1026
|
+
The element scope.
|
|
713
1027
|
|
|
714
1028
|
#### Returns
|
|
715
1029
|
|
|
716
|
-
`
|
|
1030
|
+
`Promise`\<`any`\>
|
|
1031
|
+
|
|
1032
|
+
Parent element or null.
|
|
1033
|
+
|
|
1034
|
+
#### Implementation of
|
|
1035
|
+
|
|
1036
|
+
`IExtractEngine._parentElement`
|
|
717
1037
|
|
|
718
1038
|
***
|
|
719
1039
|
|
|
720
|
-
### \
|
|
1040
|
+
### \_processAction()
|
|
721
1041
|
|
|
722
|
-
> `
|
|
1042
|
+
> `protected` **\_processAction**(`context`, `action`): `Promise`\<`any`\>
|
|
723
1043
|
|
|
724
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1044
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1093](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1093)
|
|
725
1045
|
|
|
726
1046
|
**`Internal`**
|
|
727
1047
|
|
|
728
|
-
|
|
1048
|
+
Unified action processor that handles engine-agnostic actions.
|
|
729
1049
|
|
|
730
1050
|
#### Parameters
|
|
731
1051
|
|
|
732
|
-
#####
|
|
1052
|
+
##### context
|
|
733
1053
|
|
|
734
|
-
`
|
|
1054
|
+
`TContext`
|
|
735
1055
|
|
|
736
|
-
|
|
1056
|
+
Crawlee crawling context
|
|
1057
|
+
|
|
1058
|
+
##### action
|
|
1059
|
+
|
|
1060
|
+
[`FetchEngineAction`](../type-aliases/FetchEngineAction.md)
|
|
1061
|
+
|
|
1062
|
+
Action to execute
|
|
737
1063
|
|
|
738
1064
|
#### Returns
|
|
739
1065
|
|
|
@@ -743,21 +1069,21 @@ The element.
|
|
|
743
1069
|
|
|
744
1070
|
### \_querySelectorAll()
|
|
745
1071
|
|
|
746
|
-
> `abstract`
|
|
1072
|
+
> `abstract` **\_querySelectorAll**(`scope`, `selector`): `Promise`\<`any`[]\>
|
|
747
1073
|
|
|
748
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1074
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:452](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L452)
|
|
749
1075
|
|
|
750
1076
|
**`Internal`**
|
|
751
1077
|
|
|
752
|
-
Finds all elements matching the selector within the given
|
|
1078
|
+
Finds all elements matching the selector within the given scope.
|
|
753
1079
|
|
|
754
1080
|
#### Parameters
|
|
755
1081
|
|
|
756
|
-
#####
|
|
1082
|
+
##### scope
|
|
757
1083
|
|
|
758
1084
|
`any`
|
|
759
1085
|
|
|
760
|
-
The
|
|
1086
|
+
The scope to search in (Engine-specific element/node or array of nodes).
|
|
761
1087
|
|
|
762
1088
|
##### selector
|
|
763
1089
|
|
|
@@ -769,13 +1095,23 @@ CSS selector.
|
|
|
769
1095
|
|
|
770
1096
|
`Promise`\<`any`[]\>
|
|
771
1097
|
|
|
1098
|
+
List of matching elements.
|
|
1099
|
+
|
|
1100
|
+
#### See
|
|
1101
|
+
|
|
1102
|
+
IExtractEngine.\_querySelectorAll for behavior contract.
|
|
1103
|
+
|
|
1104
|
+
#### Implementation of
|
|
1105
|
+
|
|
1106
|
+
`IExtractEngine._querySelectorAll`
|
|
1107
|
+
|
|
772
1108
|
***
|
|
773
1109
|
|
|
774
1110
|
### \_sharedFailedRequestHandler()
|
|
775
1111
|
|
|
776
1112
|
> `protected` **\_sharedFailedRequestHandler**(`context`, `error?`): `Promise`\<`void`\>
|
|
777
1113
|
|
|
778
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1114
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1269](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1269)
|
|
779
1115
|
|
|
780
1116
|
#### Parameters
|
|
781
1117
|
|
|
@@ -797,7 +1133,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:1196](https://github.com/is
|
|
|
797
1133
|
|
|
798
1134
|
> `protected` **\_sharedRequestHandler**(`context`): `Promise`\<`void`\>
|
|
799
1135
|
|
|
800
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1136
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1228](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1228)
|
|
801
1137
|
|
|
802
1138
|
#### Parameters
|
|
803
1139
|
|
|
@@ -815,7 +1151,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:1156](https://github.com/is
|
|
|
815
1151
|
|
|
816
1152
|
> **blockResources**(`types`, `overwrite?`): `Promise`\<`number`\>
|
|
817
1153
|
|
|
818
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1154
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1385](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1385)
|
|
819
1155
|
|
|
820
1156
|
Blocks specified resource types from loading.
|
|
821
1157
|
|
|
@@ -852,7 +1188,7 @@ await engine.blockResources(['script'], true); // Replace existing
|
|
|
852
1188
|
|
|
853
1189
|
> `protected` **buildResponse**(`context`): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
854
1190
|
|
|
855
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1191
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:659](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L659)
|
|
856
1192
|
|
|
857
1193
|
#### Parameters
|
|
858
1194
|
|
|
@@ -870,7 +1206,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:723](https://github.com/isd
|
|
|
870
1206
|
|
|
871
1207
|
> **cleanup**(): `Promise`\<`void`\>
|
|
872
1208
|
|
|
873
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1209
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1065](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1065)
|
|
874
1210
|
|
|
875
1211
|
#### Returns
|
|
876
1212
|
|
|
@@ -882,7 +1218,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:1084](https://github.com/is
|
|
|
882
1218
|
|
|
883
1219
|
> **click**(`selector`): `Promise`\<`void`\>
|
|
884
1220
|
|
|
885
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1221
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:753](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L753)
|
|
886
1222
|
|
|
887
1223
|
Clicks on element matching selector.
|
|
888
1224
|
|
|
@@ -912,7 +1248,7 @@ When no active page context exists
|
|
|
912
1248
|
|
|
913
1249
|
> **cookies**(): `Promise`\<[`Cookie`](../interfaces/Cookie.md)[]\>
|
|
914
1250
|
|
|
915
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1251
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1501](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1501)
|
|
916
1252
|
|
|
917
1253
|
Manages cookies for current session with multiple overloads.
|
|
918
1254
|
|
|
@@ -937,7 +1273,7 @@ await engine.cookies([{ name: 'session', value: '123' }]);
|
|
|
937
1273
|
|
|
938
1274
|
> **cookies**(`cookies`): `Promise`\<`boolean`\>
|
|
939
1275
|
|
|
940
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1276
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1502](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1502)
|
|
941
1277
|
|
|
942
1278
|
Manages cookies for current session with multiple overloads.
|
|
943
1279
|
|
|
@@ -972,7 +1308,7 @@ await engine.cookies([{ name: 'session', value: '123' }]);
|
|
|
972
1308
|
|
|
973
1309
|
> `protected` **dispatchAction**\<`T`\>(`action`): `Promise`\<`T`\>
|
|
974
1310
|
|
|
975
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1311
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1292](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1292)
|
|
976
1312
|
|
|
977
1313
|
#### Type Parameters
|
|
978
1314
|
|
|
@@ -996,7 +1332,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:1219](https://github.com/is
|
|
|
996
1332
|
|
|
997
1333
|
> **dispose**(): `Promise`\<`void`\>
|
|
998
1334
|
|
|
999
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1335
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1537](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1537)
|
|
1000
1336
|
|
|
1001
1337
|
Disposes of engine, cleaning up all resources.
|
|
1002
1338
|
|
|
@@ -1008,11 +1344,53 @@ Promise resolving when disposal completes
|
|
|
1008
1344
|
|
|
1009
1345
|
***
|
|
1010
1346
|
|
|
1347
|
+
### evaluate()
|
|
1348
|
+
|
|
1349
|
+
> **evaluate**(`params`): `Promise`\<`any`\>
|
|
1350
|
+
|
|
1351
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:873](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L873)
|
|
1352
|
+
|
|
1353
|
+
Executes a custom function or expression within the current page context.
|
|
1354
|
+
|
|
1355
|
+
#### Parameters
|
|
1356
|
+
|
|
1357
|
+
##### params
|
|
1358
|
+
|
|
1359
|
+
[`EvaluateActionOptions`](../interfaces/EvaluateActionOptions.md)
|
|
1360
|
+
|
|
1361
|
+
Configuration for the execution, including the function and arguments.
|
|
1362
|
+
|
|
1363
|
+
#### Returns
|
|
1364
|
+
|
|
1365
|
+
`Promise`\<`any`\>
|
|
1366
|
+
|
|
1367
|
+
A promise resolving to the result of the execution.
|
|
1368
|
+
|
|
1369
|
+
#### Remarks
|
|
1370
|
+
|
|
1371
|
+
This is a powerful action that allows running custom logic to interact with the DOM,
|
|
1372
|
+
calculate values, or trigger navigations.
|
|
1373
|
+
|
|
1374
|
+
- In **Browser Mode**, it runs in the real browser.
|
|
1375
|
+
- In **HTTP Mode**, it runs in a Node.js sandbox with a mocked DOM.
|
|
1376
|
+
|
|
1377
|
+
The action handles automatic navigation if `window.location` is modified.
|
|
1378
|
+
|
|
1379
|
+
#### Throws
|
|
1380
|
+
|
|
1381
|
+
If no active page context exists or if execution fails.
|
|
1382
|
+
|
|
1383
|
+
#### See
|
|
1384
|
+
|
|
1385
|
+
[EvaluateActionOptions](../interfaces/EvaluateActionOptions.md) for detailed parameter options and examples.
|
|
1386
|
+
|
|
1387
|
+
***
|
|
1388
|
+
|
|
1011
1389
|
### executeAction()
|
|
1012
1390
|
|
|
1013
1391
|
> `abstract` `protected` **executeAction**(`context`, `action`): `Promise`\<`any`\>
|
|
1014
1392
|
|
|
1015
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1393
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:708](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L708)
|
|
1016
1394
|
|
|
1017
1395
|
**`Internal`**
|
|
1018
1396
|
|
|
@@ -1048,7 +1426,7 @@ Handles specific user interactions using underlying technology (Playwright/Cheer
|
|
|
1048
1426
|
|
|
1049
1427
|
> **extract**\<`T`\>(`schema`): `Promise`\<`T`\>
|
|
1050
1428
|
|
|
1051
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1429
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:883](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L883)
|
|
1052
1430
|
|
|
1053
1431
|
Extracts structured data from the current page content.
|
|
1054
1432
|
|
|
@@ -1078,7 +1456,7 @@ A promise that resolves to an object with the extracted data.
|
|
|
1078
1456
|
|
|
1079
1457
|
> **fill**(`selector`, `value`): `Promise`\<`void`\>
|
|
1080
1458
|
|
|
1081
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1459
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:817](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L817)
|
|
1082
1460
|
|
|
1083
1461
|
Fills input element with specified value.
|
|
1084
1462
|
|
|
@@ -1112,7 +1490,7 @@ When no active page context exists
|
|
|
1112
1490
|
|
|
1113
1491
|
> **getContent**(): `Promise`\<[`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
1114
1492
|
|
|
1115
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1493
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1399](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1399)
|
|
1116
1494
|
|
|
1117
1495
|
Gets content of current page.
|
|
1118
1496
|
|
|
@@ -1132,7 +1510,7 @@ When no content has been fetched yet
|
|
|
1132
1510
|
|
|
1133
1511
|
> **getState**(): `Promise`\<\{ `cookies`: [`Cookie`](../interfaces/Cookie.md)[]; `sessionState?`: `any`; \}\>
|
|
1134
1512
|
|
|
1135
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1513
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:902](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L902)
|
|
1136
1514
|
|
|
1137
1515
|
Returns the current state of the engine (cookies)
|
|
1138
1516
|
that can be used to restore the session later.
|
|
@@ -1147,7 +1525,7 @@ that can be used to restore the session later.
|
|
|
1147
1525
|
|
|
1148
1526
|
> `abstract` **goto**(`url`, `params?`): `Promise`\<`void` \| [`FetchResponse`](../interfaces/FetchResponse.md)\>
|
|
1149
1527
|
|
|
1150
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1528
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:725](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L725)
|
|
1151
1529
|
|
|
1152
1530
|
Navigates to the specified URL.
|
|
1153
1531
|
|
|
@@ -1185,7 +1563,7 @@ await engine.goto('https://example.com');
|
|
|
1185
1563
|
|
|
1186
1564
|
> **headers**(): `Promise`\<`Record`\<`string`, `string`\>\>
|
|
1187
1565
|
|
|
1188
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1566
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1440](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1440)
|
|
1189
1567
|
|
|
1190
1568
|
Manages HTTP headers for requests with multiple overloads.
|
|
1191
1569
|
|
|
@@ -1212,7 +1590,7 @@ await engine.headers('auth', 'token');
|
|
|
1212
1590
|
|
|
1213
1591
|
> **headers**(`name`): `Promise`\<`string`\>
|
|
1214
1592
|
|
|
1215
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1593
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1441](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1441)
|
|
1216
1594
|
|
|
1217
1595
|
Manages HTTP headers for requests with multiple overloads.
|
|
1218
1596
|
|
|
@@ -1247,7 +1625,7 @@ await engine.headers('auth', 'token');
|
|
|
1247
1625
|
|
|
1248
1626
|
> **headers**(`headers`, `replaced?`): `Promise`\<`boolean`\>
|
|
1249
1627
|
|
|
1250
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1628
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1442](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1442)
|
|
1251
1629
|
|
|
1252
1630
|
Manages HTTP headers for requests with multiple overloads.
|
|
1253
1631
|
|
|
@@ -1288,7 +1666,7 @@ await engine.headers('auth', 'token');
|
|
|
1288
1666
|
|
|
1289
1667
|
> **headers**(`name`, `value`): `Promise`\<`boolean`\>
|
|
1290
1668
|
|
|
1291
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1669
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:1446](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L1446)
|
|
1292
1670
|
|
|
1293
1671
|
Manages HTTP headers for requests with multiple overloads.
|
|
1294
1672
|
|
|
@@ -1306,7 +1684,7 @@ Header name
|
|
|
1306
1684
|
|
|
1307
1685
|
Header value or `null` to remove
|
|
1308
1686
|
|
|
1309
|
-
`
|
|
1687
|
+
`string` | `null`
|
|
1310
1688
|
|
|
1311
1689
|
##### Returns
|
|
1312
1690
|
|
|
@@ -1331,7 +1709,7 @@ await engine.headers('auth', 'token');
|
|
|
1331
1709
|
|
|
1332
1710
|
> **initialize**(`context`, `options?`): `Promise`\<`void`\>
|
|
1333
1711
|
|
|
1334
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1712
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:934](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L934)
|
|
1335
1713
|
|
|
1336
1714
|
Initializes the fetch engine with provided context and options.
|
|
1337
1715
|
|
|
@@ -1362,11 +1740,143 @@ Automatically called when creating engine via `FetchEngine.create()`.
|
|
|
1362
1740
|
|
|
1363
1741
|
***
|
|
1364
1742
|
|
|
1743
|
+
### keyboardPress()
|
|
1744
|
+
|
|
1745
|
+
> **keyboardPress**(`key`, `delay?`): `Promise`\<`void`\>
|
|
1746
|
+
|
|
1747
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:802](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L802)
|
|
1748
|
+
|
|
1749
|
+
Presses specified key.
|
|
1750
|
+
|
|
1751
|
+
#### Parameters
|
|
1752
|
+
|
|
1753
|
+
##### key
|
|
1754
|
+
|
|
1755
|
+
`string`
|
|
1756
|
+
|
|
1757
|
+
Key to press
|
|
1758
|
+
|
|
1759
|
+
##### delay?
|
|
1760
|
+
|
|
1761
|
+
`number`
|
|
1762
|
+
|
|
1763
|
+
Delay after key press
|
|
1764
|
+
|
|
1765
|
+
#### Returns
|
|
1766
|
+
|
|
1767
|
+
`Promise`\<`void`\>
|
|
1768
|
+
|
|
1769
|
+
***
|
|
1770
|
+
|
|
1771
|
+
### keyboardType()
|
|
1772
|
+
|
|
1773
|
+
> **keyboardType**(`text`, `delay?`): `Promise`\<`void`\>
|
|
1774
|
+
|
|
1775
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:792](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L792)
|
|
1776
|
+
|
|
1777
|
+
Types text into current focused element.
|
|
1778
|
+
|
|
1779
|
+
#### Parameters
|
|
1780
|
+
|
|
1781
|
+
##### text
|
|
1782
|
+
|
|
1783
|
+
`string`
|
|
1784
|
+
|
|
1785
|
+
Text to type
|
|
1786
|
+
|
|
1787
|
+
##### delay?
|
|
1788
|
+
|
|
1789
|
+
`number`
|
|
1790
|
+
|
|
1791
|
+
Delay between key presses
|
|
1792
|
+
|
|
1793
|
+
#### Returns
|
|
1794
|
+
|
|
1795
|
+
`Promise`\<`void`\>
|
|
1796
|
+
|
|
1797
|
+
***
|
|
1798
|
+
|
|
1799
|
+
### mouseClick()
|
|
1800
|
+
|
|
1801
|
+
> **mouseClick**(`params`): `Promise`\<`void`\>
|
|
1802
|
+
|
|
1803
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:776](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L776)
|
|
1804
|
+
|
|
1805
|
+
Clicks at current position or specified position.
|
|
1806
|
+
|
|
1807
|
+
#### Parameters
|
|
1808
|
+
|
|
1809
|
+
##### params
|
|
1810
|
+
|
|
1811
|
+
Click parameters (x, y, button, clickCount, delay)
|
|
1812
|
+
|
|
1813
|
+
###### button?
|
|
1814
|
+
|
|
1815
|
+
`"left"` \| `"right"` \| `"middle"`
|
|
1816
|
+
|
|
1817
|
+
###### clickCount?
|
|
1818
|
+
|
|
1819
|
+
`number`
|
|
1820
|
+
|
|
1821
|
+
###### delay?
|
|
1822
|
+
|
|
1823
|
+
`number`
|
|
1824
|
+
|
|
1825
|
+
###### x?
|
|
1826
|
+
|
|
1827
|
+
`number`
|
|
1828
|
+
|
|
1829
|
+
###### y?
|
|
1830
|
+
|
|
1831
|
+
`number`
|
|
1832
|
+
|
|
1833
|
+
#### Returns
|
|
1834
|
+
|
|
1835
|
+
`Promise`\<`void`\>
|
|
1836
|
+
|
|
1837
|
+
***
|
|
1838
|
+
|
|
1839
|
+
### mouseMove()
|
|
1840
|
+
|
|
1841
|
+
> **mouseMove**(`params`): `Promise`\<`void`\>
|
|
1842
|
+
|
|
1843
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:762](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L762)
|
|
1844
|
+
|
|
1845
|
+
Moves mouse to specified position or element.
|
|
1846
|
+
|
|
1847
|
+
#### Parameters
|
|
1848
|
+
|
|
1849
|
+
##### params
|
|
1850
|
+
|
|
1851
|
+
Move parameters (x, y, selector, steps)
|
|
1852
|
+
|
|
1853
|
+
###### selector?
|
|
1854
|
+
|
|
1855
|
+
`string`
|
|
1856
|
+
|
|
1857
|
+
###### steps?
|
|
1858
|
+
|
|
1859
|
+
`number`
|
|
1860
|
+
|
|
1861
|
+
###### x?
|
|
1862
|
+
|
|
1863
|
+
`number`
|
|
1864
|
+
|
|
1865
|
+
###### y?
|
|
1866
|
+
|
|
1867
|
+
`number`
|
|
1868
|
+
|
|
1869
|
+
#### Returns
|
|
1870
|
+
|
|
1871
|
+
`Promise`\<`void`\>
|
|
1872
|
+
|
|
1873
|
+
***
|
|
1874
|
+
|
|
1365
1875
|
### pause()
|
|
1366
1876
|
|
|
1367
1877
|
> **pause**(`message?`): `Promise`\<`void`\>
|
|
1368
1878
|
|
|
1369
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1879
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:851](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L851)
|
|
1370
1880
|
|
|
1371
1881
|
Pauses execution, allowing for manual intervention or inspection.
|
|
1372
1882
|
|
|
@@ -1394,7 +1904,7 @@ When no active page context exists
|
|
|
1394
1904
|
|
|
1395
1905
|
> **submit**(`selector?`, `options?`): `Promise`\<`void`\>
|
|
1396
1906
|
|
|
1397
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1907
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:829](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L829)
|
|
1398
1908
|
|
|
1399
1909
|
Submits a form.
|
|
1400
1910
|
|
|
@@ -1424,11 +1934,39 @@ When no active page context exists
|
|
|
1424
1934
|
|
|
1425
1935
|
***
|
|
1426
1936
|
|
|
1937
|
+
### trim()
|
|
1938
|
+
|
|
1939
|
+
> **trim**(`options`): `Promise`\<`void`\>
|
|
1940
|
+
|
|
1941
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:840](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L840)
|
|
1942
|
+
|
|
1943
|
+
Removes elements from the DOM based on selectors and presets.
|
|
1944
|
+
|
|
1945
|
+
#### Parameters
|
|
1946
|
+
|
|
1947
|
+
##### options
|
|
1948
|
+
|
|
1949
|
+
[`TrimActionOptions`](../interfaces/TrimActionOptions.md)
|
|
1950
|
+
|
|
1951
|
+
Trim options specifying selectors and presets
|
|
1952
|
+
|
|
1953
|
+
#### Returns
|
|
1954
|
+
|
|
1955
|
+
`Promise`\<`void`\>
|
|
1956
|
+
|
|
1957
|
+
Promise resolving when trim operation completes
|
|
1958
|
+
|
|
1959
|
+
#### Throws
|
|
1960
|
+
|
|
1961
|
+
When no active page context exists
|
|
1962
|
+
|
|
1963
|
+
***
|
|
1964
|
+
|
|
1427
1965
|
### waitFor()
|
|
1428
1966
|
|
|
1429
1967
|
> **waitFor**(`params?`): `Promise`\<`void`\>
|
|
1430
1968
|
|
|
1431
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
1969
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:742](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L742)
|
|
1432
1970
|
|
|
1433
1971
|
Waits for specified condition before continuing.
|
|
1434
1972
|
|
|
@@ -1457,9 +1995,9 @@ await engine.waitFor({ selector: '#content' }); // Wait for element
|
|
|
1457
1995
|
|
|
1458
1996
|
### create()
|
|
1459
1997
|
|
|
1460
|
-
> `static` **create**(`ctx`, `options?`): `Promise`\<`
|
|
1998
|
+
> `static` **create**(`ctx`, `options?`): `Promise`\<`AnyFetchEngine` \| `undefined`\>
|
|
1461
1999
|
|
|
1462
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
2000
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:349](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L349)
|
|
1463
2001
|
|
|
1464
2002
|
Factory method to create and initialize a fetch engine instance.
|
|
1465
2003
|
|
|
@@ -1479,7 +2017,7 @@ Configuration options
|
|
|
1479
2017
|
|
|
1480
2018
|
#### Returns
|
|
1481
2019
|
|
|
1482
|
-
`Promise`\<`
|
|
2020
|
+
`Promise`\<`AnyFetchEngine` \| `undefined`\>
|
|
1483
2021
|
|
|
1484
2022
|
Initialized fetch engine instance
|
|
1485
2023
|
|
|
@@ -1495,9 +2033,9 @@ Primary entry point for engine creation. Selects appropriate implementation base
|
|
|
1495
2033
|
|
|
1496
2034
|
### get()
|
|
1497
2035
|
|
|
1498
|
-
> `static` **get**(`id`): `
|
|
2036
|
+
> `static` **get**(`id`): `AnyFetchEngineCtor` \| `undefined`
|
|
1499
2037
|
|
|
1500
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
2038
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:322](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L322)
|
|
1501
2039
|
|
|
1502
2040
|
Retrieves a fetch engine implementation by its unique ID.
|
|
1503
2041
|
|
|
@@ -1511,7 +2049,7 @@ The ID of the engine to retrieve
|
|
|
1511
2049
|
|
|
1512
2050
|
#### Returns
|
|
1513
2051
|
|
|
1514
|
-
`
|
|
2052
|
+
`AnyFetchEngineCtor` \| `undefined`
|
|
1515
2053
|
|
|
1516
2054
|
Engine class if found, otherwise `undefined`
|
|
1517
2055
|
|
|
@@ -1519,9 +2057,9 @@ Engine class if found, otherwise `undefined`
|
|
|
1519
2057
|
|
|
1520
2058
|
### getByMode()
|
|
1521
2059
|
|
|
1522
|
-
> `static` **getByMode**(`mode`): `
|
|
2060
|
+
> `static` **getByMode**(`mode`): `AnyFetchEngineCtor` \| `undefined`
|
|
1523
2061
|
|
|
1524
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
2062
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:332](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L332)
|
|
1525
2063
|
|
|
1526
2064
|
Retrieves a fetch engine implementation by execution mode.
|
|
1527
2065
|
|
|
@@ -1535,7 +2073,7 @@ Execution mode (`'http'` or `'browser'`)
|
|
|
1535
2073
|
|
|
1536
2074
|
#### Returns
|
|
1537
2075
|
|
|
1538
|
-
`
|
|
2076
|
+
`AnyFetchEngineCtor` \| `undefined`
|
|
1539
2077
|
|
|
1540
2078
|
Engine class if found, otherwise `undefined`
|
|
1541
2079
|
|
|
@@ -1545,7 +2083,7 @@ Engine class if found, otherwise `undefined`
|
|
|
1545
2083
|
|
|
1546
2084
|
> `static` **register**(`engineClass`): `void`
|
|
1547
2085
|
|
|
1548
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
2086
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:309](https://github.com/isdk/web-fetcher.js/blob/e691a2370f59d15979b47994c8ca14d7b7d2edd3/src/engine/base.ts#L309)
|
|
1549
2087
|
|
|
1550
2088
|
Registers a fetch engine implementation with the global registry.
|
|
1551
2089
|
|