@isdk/web-fetcher 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (66) hide show
  1. package/README.action.cn.md +469 -0
  2. package/README.action.md +452 -0
  3. package/README.cn.md +147 -0
  4. package/README.engine.cn.md +262 -0
  5. package/README.engine.md +262 -0
  6. package/README.md +147 -0
  7. package/dist/index.d.mts +1603 -0
  8. package/dist/index.d.ts +1603 -0
  9. package/dist/index.js +1 -0
  10. package/dist/index.mjs +1 -0
  11. package/docs/README.md +151 -0
  12. package/docs/_media/LICENSE-MIT +22 -0
  13. package/docs/_media/README.action.md +452 -0
  14. package/docs/_media/README.cn.md +147 -0
  15. package/docs/_media/README.engine.md +262 -0
  16. package/docs/classes/CheerioFetchEngine.md +1447 -0
  17. package/docs/classes/ClickAction.md +533 -0
  18. package/docs/classes/ExtractAction.md +533 -0
  19. package/docs/classes/FetchAction.md +444 -0
  20. package/docs/classes/FetchEngine.md +1230 -0
  21. package/docs/classes/FetchSession.md +111 -0
  22. package/docs/classes/FillAction.md +533 -0
  23. package/docs/classes/GetContentAction.md +533 -0
  24. package/docs/classes/GotoAction.md +537 -0
  25. package/docs/classes/PauseAction.md +533 -0
  26. package/docs/classes/PlaywrightFetchEngine.md +1437 -0
  27. package/docs/classes/SubmitAction.md +533 -0
  28. package/docs/classes/WaitForAction.md +533 -0
  29. package/docs/classes/WebFetcher.md +85 -0
  30. package/docs/enumerations/FetchActionResultStatus.md +40 -0
  31. package/docs/functions/fetchWeb.md +43 -0
  32. package/docs/globals.md +72 -0
  33. package/docs/interfaces/BaseFetchActionProperties.md +83 -0
  34. package/docs/interfaces/BaseFetchCollectorActionProperties.md +145 -0
  35. package/docs/interfaces/BaseFetcherProperties.md +206 -0
  36. package/docs/interfaces/Cookie.md +142 -0
  37. package/docs/interfaces/DispatchedEngineAction.md +60 -0
  38. package/docs/interfaces/ExtractActionProperties.md +113 -0
  39. package/docs/interfaces/FetchActionInContext.md +149 -0
  40. package/docs/interfaces/FetchActionProperties.md +125 -0
  41. package/docs/interfaces/FetchActionResult.md +55 -0
  42. package/docs/interfaces/FetchContext.md +424 -0
  43. package/docs/interfaces/FetchEngineContext.md +328 -0
  44. package/docs/interfaces/FetchMetadata.md +73 -0
  45. package/docs/interfaces/FetchResponse.md +105 -0
  46. package/docs/interfaces/FetchReturnTypeRegistry.md +57 -0
  47. package/docs/interfaces/FetchSite.md +320 -0
  48. package/docs/interfaces/FetcherOptions.md +300 -0
  49. package/docs/interfaces/GotoActionOptions.md +66 -0
  50. package/docs/interfaces/PendingEngineRequest.md +51 -0
  51. package/docs/interfaces/SubmitActionOptions.md +23 -0
  52. package/docs/interfaces/WaitForActionOptions.md +39 -0
  53. package/docs/type-aliases/BaseFetchActionOptions.md +11 -0
  54. package/docs/type-aliases/BaseFetchCollectorOptions.md +11 -0
  55. package/docs/type-aliases/BrowserEngine.md +11 -0
  56. package/docs/type-aliases/FetchActionCapabilities.md +11 -0
  57. package/docs/type-aliases/FetchActionCapabilityMode.md +11 -0
  58. package/docs/type-aliases/FetchActionOptions.md +11 -0
  59. package/docs/type-aliases/FetchEngineAction.md +18 -0
  60. package/docs/type-aliases/FetchEngineType.md +11 -0
  61. package/docs/type-aliases/FetchReturnType.md +11 -0
  62. package/docs/type-aliases/FetchReturnTypeFor.md +17 -0
  63. package/docs/type-aliases/OnFetchPauseCallback.md +23 -0
  64. package/docs/type-aliases/ResourceType.md +11 -0
  65. package/docs/variables/DefaultFetcherProperties.md +11 -0
  66. package/package.json +90 -0
@@ -0,0 +1,424 @@
1
+ [**@isdk/web-fetcher**](../README.md)
2
+
3
+ ***
4
+
5
+ [@isdk/web-fetcher](../globals.md) / FetchContext
6
+
7
+ # Interface: FetchContext
8
+
9
+ Defined in: [packages/web-fetcher/src/core/context.ts:35](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L35)
10
+
11
+ ## Extends
12
+
13
+ - [`FetchEngineContext`](FetchEngineContext.md)
14
+
15
+ ## Properties
16
+
17
+ ### antibot?
18
+
19
+ > `optional` **antibot**: `boolean`
20
+
21
+ Defined in: [packages/web-fetcher/src/core/types.ts:37](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L37)
22
+
23
+ #### Inherited from
24
+
25
+ [`FetchEngineContext`](FetchEngineContext.md).[`antibot`](FetchEngineContext.md#antibot)
26
+
27
+ ***
28
+
29
+ ### blockResources?
30
+
31
+ > `optional` **blockResources**: `string`[]
32
+
33
+ Defined in: [packages/web-fetcher/src/core/types.ts:46](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L46)
34
+
35
+ #### Inherited from
36
+
37
+ [`FetchEngineContext`](FetchEngineContext.md).[`blockResources`](FetchEngineContext.md#blockresources)
38
+
39
+ ***
40
+
41
+ ### browser?
42
+
43
+ > `optional` **browser**: `object`
44
+
45
+ Defined in: [packages/web-fetcher/src/core/types.ts:52](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L52)
46
+
47
+ #### engine?
48
+
49
+ > `optional` **engine**: [`BrowserEngine`](../type-aliases/BrowserEngine.md)
50
+
51
+ 浏览器引擎,默认为 playwright
52
+
53
+ - `playwright`: 使用 Playwright 引擎
54
+ - `puppeteer`: 使用 Puppeteer 引擎
55
+
56
+ #### headless?
57
+
58
+ > `optional` **headless**: `boolean`
59
+
60
+ #### waitUntil?
61
+
62
+ > `optional` **waitUntil**: `"load"` \| `"domcontentloaded"` \| `"networkidle"` \| `"commit"`
63
+
64
+ #### Inherited from
65
+
66
+ [`FetchEngineContext`](FetchEngineContext.md).[`browser`](FetchEngineContext.md#browser)
67
+
68
+ ***
69
+
70
+ ### cookies?
71
+
72
+ > `optional` **cookies**: [`Cookie`](Cookie.md)[]
73
+
74
+ Defined in: [packages/web-fetcher/src/core/types.ts:40](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L40)
75
+
76
+ #### Inherited from
77
+
78
+ [`FetchEngineContext`](FetchEngineContext.md).[`cookies`](FetchEngineContext.md#cookies)
79
+
80
+ ***
81
+
82
+ ### currentAction?
83
+
84
+ > `optional` **currentAction**: [`FetchActionInContext`](FetchActionInContext.md)
85
+
86
+ Defined in: [packages/web-fetcher/src/core/context.ts:36](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L36)
87
+
88
+ ***
89
+
90
+ ### delayBetweenRequestsMs?
91
+
92
+ > `optional` **delayBetweenRequestsMs**: `number`
93
+
94
+ Defined in: [packages/web-fetcher/src/core/types.ts:72](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L72)
95
+
96
+ #### Inherited from
97
+
98
+ [`FetchEngineContext`](FetchEngineContext.md).[`delayBetweenRequestsMs`](FetchEngineContext.md#delaybetweenrequestsms)
99
+
100
+ ***
101
+
102
+ ### enableSmart?
103
+
104
+ > `optional` **enableSmart**: `boolean`
105
+
106
+ Defined in: [packages/web-fetcher/src/core/types.ts:35](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L35)
107
+
108
+ #### Inherited from
109
+
110
+ [`FetchEngineContext`](FetchEngineContext.md).[`enableSmart`](FetchEngineContext.md#enablesmart)
111
+
112
+ ***
113
+
114
+ ### engine?
115
+
116
+ > `optional` **engine**: `string`
117
+
118
+ Defined in: [packages/web-fetcher/src/core/types.ts:34](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L34)
119
+
120
+ 抓取模式
121
+
122
+ - `http`: 使用 HTTP 进行抓取
123
+ - `browser`: 使用浏览器进行抓取
124
+ - `auto`: auto 会走“智能探测”选择 http 或 browser, 但是如果没有启用 smart,并且在站点注册表中没有,那么则等价为 http.
125
+
126
+ #### Inherited from
127
+
128
+ [`FetchEngineContext`](FetchEngineContext.md).[`engine`](FetchEngineContext.md#engine)
129
+
130
+ ***
131
+
132
+ ### eventBus
133
+
134
+ > **eventBus**: `EventEmitter`
135
+
136
+ Defined in: [packages/web-fetcher/src/core/context.ts:48](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L48)
137
+
138
+ ***
139
+
140
+ ### finalUrl?
141
+
142
+ > `optional` **finalUrl**: `string`
143
+
144
+ Defined in: [packages/web-fetcher/src/core/context.ts:27](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L27)
145
+
146
+ #### Inherited from
147
+
148
+ [`FetchEngineContext`](FetchEngineContext.md).[`finalUrl`](FetchEngineContext.md#finalurl)
149
+
150
+ ***
151
+
152
+ ### headers?
153
+
154
+ > `optional` **headers**: `Record`\<`string`, `string`\>
155
+
156
+ Defined in: [packages/web-fetcher/src/core/types.ts:39](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L39)
157
+
158
+ #### Inherited from
159
+
160
+ [`FetchEngineContext`](FetchEngineContext.md).[`headers`](FetchEngineContext.md#headers)
161
+
162
+ ***
163
+
164
+ ### http?
165
+
166
+ > `optional` **http**: `object`
167
+
168
+ Defined in: [packages/web-fetcher/src/core/types.ts:64](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L64)
169
+
170
+ #### body?
171
+
172
+ > `optional` **body**: `any`
173
+
174
+ #### method?
175
+
176
+ > `optional` **method**: `"GET"` \| `"POST"` \| `"PUT"` \| `"DELETE"` \| `"PATCH"`
177
+
178
+ #### Inherited from
179
+
180
+ [`FetchEngineContext`](FetchEngineContext.md).[`http`](FetchEngineContext.md#http)
181
+
182
+ ***
183
+
184
+ ### id
185
+
186
+ > **id**: `string`
187
+
188
+ Defined in: [packages/web-fetcher/src/core/context.ts:25](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L25)
189
+
190
+ #### Inherited from
191
+
192
+ [`FetchEngineContext`](FetchEngineContext.md).[`id`](FetchEngineContext.md#id)
193
+
194
+ ***
195
+
196
+ ### ignoreSslErrors?
197
+
198
+ > `optional` **ignoreSslErrors**: `boolean`
199
+
200
+ Defined in: [packages/web-fetcher/src/core/types.ts:50](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L50)
201
+
202
+ #### Inherited from
203
+
204
+ [`FetchEngineContext`](FetchEngineContext.md).[`ignoreSslErrors`](FetchEngineContext.md#ignoresslerrors)
205
+
206
+ ***
207
+
208
+ ### internal
209
+
210
+ > **internal**: `FetchContextInteralState`
211
+
212
+ Defined in: [packages/web-fetcher/src/core/context.ts:46](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L46)
213
+
214
+ #### Overrides
215
+
216
+ [`FetchEngineContext`](FetchEngineContext.md).[`internal`](FetchEngineContext.md#internal)
217
+
218
+ ***
219
+
220
+ ### lastResponse?
221
+
222
+ > `optional` **lastResponse**: [`FetchResponse`](FetchResponse.md)
223
+
224
+ Defined in: [packages/web-fetcher/src/core/context.ts:29](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L29)
225
+
226
+ #### Inherited from
227
+
228
+ [`FetchEngineContext`](FetchEngineContext.md).[`lastResponse`](FetchEngineContext.md#lastresponse)
229
+
230
+ ***
231
+
232
+ ### lastResult?
233
+
234
+ > `optional` **lastResult**: [`FetchActionResult`](FetchActionResult.md)\<[`FetchReturnType`](../type-aliases/FetchReturnType.md)\>
235
+
236
+ Defined in: [packages/web-fetcher/src/core/context.ts:30](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L30)
237
+
238
+ #### Inherited from
239
+
240
+ [`FetchEngineContext`](FetchEngineContext.md).[`lastResult`](FetchEngineContext.md#lastresult)
241
+
242
+ ***
243
+
244
+ ### maxConcurrency?
245
+
246
+ > `optional` **maxConcurrency**: `number`
247
+
248
+ Defined in: [packages/web-fetcher/src/core/types.ts:70](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L70)
249
+
250
+ #### Inherited from
251
+
252
+ [`FetchEngineContext`](FetchEngineContext.md).[`maxConcurrency`](FetchEngineContext.md#maxconcurrency)
253
+
254
+ ***
255
+
256
+ ### maxRequestsPerMinute?
257
+
258
+ > `optional` **maxRequestsPerMinute**: `number`
259
+
260
+ Defined in: [packages/web-fetcher/src/core/types.ts:71](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L71)
261
+
262
+ #### Inherited from
263
+
264
+ [`FetchEngineContext`](FetchEngineContext.md).[`maxRequestsPerMinute`](FetchEngineContext.md#maxrequestsperminute)
265
+
266
+ ***
267
+
268
+ ### outputs
269
+
270
+ > **outputs**: `Record`\<`string`, `any`\>
271
+
272
+ Defined in: [packages/web-fetcher/src/core/context.ts:38](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L38)
273
+
274
+ ***
275
+
276
+ ### proxy?
277
+
278
+ > `optional` **proxy**: `string` \| `string`[]
279
+
280
+ Defined in: [packages/web-fetcher/src/core/types.ts:44](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L44)
281
+
282
+ #### Inherited from
283
+
284
+ [`FetchEngineContext`](FetchEngineContext.md).[`proxy`](FetchEngineContext.md#proxy)
285
+
286
+ ***
287
+
288
+ ### retries?
289
+
290
+ > `optional` **retries**: `number`
291
+
292
+ Defined in: [packages/web-fetcher/src/core/types.ts:73](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L73)
293
+
294
+ #### Inherited from
295
+
296
+ [`FetchEngineContext`](FetchEngineContext.md).[`retries`](FetchEngineContext.md#retries)
297
+
298
+ ***
299
+
300
+ ### reuseCookies?
301
+
302
+ > `optional` **reuseCookies**: `boolean`
303
+
304
+ Defined in: [packages/web-fetcher/src/core/types.ts:41](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L41)
305
+
306
+ #### Inherited from
307
+
308
+ [`FetchEngineContext`](FetchEngineContext.md).[`reuseCookies`](FetchEngineContext.md#reusecookies)
309
+
310
+ ***
311
+
312
+ ### sites?
313
+
314
+ > `optional` **sites**: [`FetchSite`](FetchSite.md)[]
315
+
316
+ Defined in: [packages/web-fetcher/src/core/types.ts:75](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L75)
317
+
318
+ #### Inherited from
319
+
320
+ [`FetchEngineContext`](FetchEngineContext.md).[`sites`](FetchEngineContext.md#sites)
321
+
322
+ ***
323
+
324
+ ### throwHttpErrors?
325
+
326
+ > `optional` **throwHttpErrors**: `boolean`
327
+
328
+ Defined in: [packages/web-fetcher/src/core/types.ts:42](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L42)
329
+
330
+ #### Inherited from
331
+
332
+ [`FetchEngineContext`](FetchEngineContext.md).[`throwHttpErrors`](FetchEngineContext.md#throwhttperrors)
333
+
334
+ ***
335
+
336
+ ### timeoutMs?
337
+
338
+ > `optional` **timeoutMs**: `number`
339
+
340
+ Defined in: [packages/web-fetcher/src/core/types.ts:69](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L69)
341
+
342
+ #### Inherited from
343
+
344
+ [`FetchEngineContext`](FetchEngineContext.md).[`timeoutMs`](FetchEngineContext.md#timeoutms)
345
+
346
+ ***
347
+
348
+ ### url?
349
+
350
+ > `optional` **url**: `string`
351
+
352
+ Defined in: [packages/web-fetcher/src/core/context.ts:26](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L26)
353
+
354
+ #### Inherited from
355
+
356
+ [`FetchEngineContext`](FetchEngineContext.md).[`url`](FetchEngineContext.md#url)
357
+
358
+ ***
359
+
360
+ ### useSiteRegistry?
361
+
362
+ > `optional` **useSiteRegistry**: `boolean`
363
+
364
+ Defined in: [packages/web-fetcher/src/core/types.ts:36](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/types.ts#L36)
365
+
366
+ #### Inherited from
367
+
368
+ [`FetchEngineContext`](FetchEngineContext.md).[`useSiteRegistry`](FetchEngineContext.md#usesiteregistry)
369
+
370
+ ## Methods
371
+
372
+ ### action()
373
+
374
+ > **action**\<`R`\>(`name`, `params?`, `options?`): `Promise`\<[`FetchActionResult`](FetchActionResult.md)\<`R`\>\>
375
+
376
+ Defined in: [packages/web-fetcher/src/core/context.ts:43](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L43)
377
+
378
+ #### Type Parameters
379
+
380
+ ##### R
381
+
382
+ `R` *extends* [`FetchReturnType`](../type-aliases/FetchReturnType.md) = `"any"`
383
+
384
+ #### Parameters
385
+
386
+ ##### name
387
+
388
+ `string`
389
+
390
+ ##### params?
391
+
392
+ `any`
393
+
394
+ ##### options?
395
+
396
+ `Partial`\<`_RequireAtLeastOne`\<[`FetchActionProperties`](FetchActionProperties.md), `"name"` \| `"id"`\>\>
397
+
398
+ #### Returns
399
+
400
+ `Promise`\<[`FetchActionResult`](FetchActionResult.md)\<`R`\>\>
401
+
402
+ ***
403
+
404
+ ### execute()
405
+
406
+ > **execute**\<`R`\>(`actionOptions`): `Promise`\<[`FetchActionResult`](FetchActionResult.md)\<`R`\>\>
407
+
408
+ Defined in: [packages/web-fetcher/src/core/context.ts:41](https://github.com/isdk/web-fetcher.js/blob/ef3a4e527bf3ca75213d55b02c017b9a30a036fd/src/core/context.ts#L41)
409
+
410
+ #### Type Parameters
411
+
412
+ ##### R
413
+
414
+ `R` *extends* [`FetchReturnType`](../type-aliases/FetchReturnType.md) = `"any"`
415
+
416
+ #### Parameters
417
+
418
+ ##### actionOptions
419
+
420
+ `_RequireAtLeastOne`
421
+
422
+ #### Returns
423
+
424
+ `Promise`\<[`FetchActionResult`](FetchActionResult.md)\<`R`\>\>