@isdk/web-fetcher 0.2.10 → 0.2.11
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.cn.md +8 -6
- package/README.md +8 -6
- package/dist/index.d.mts +3 -1
- package/dist/index.d.ts +3 -1
- package/dist/index.js +1 -1
- package/dist/index.mjs +1 -1
- package/docs/README.md +8 -6
- package/docs/_media/README.cn.md +8 -6
- package/docs/classes/CheerioFetchEngine.md +76 -64
- package/docs/classes/ClickAction.md +23 -23
- package/docs/classes/ExtractAction.md +23 -23
- package/docs/classes/FetchAction.md +23 -23
- package/docs/classes/FetchEngine.md +72 -64
- package/docs/classes/FetchSession.md +10 -10
- package/docs/classes/FillAction.md +23 -23
- package/docs/classes/GetContentAction.md +23 -23
- package/docs/classes/GotoAction.md +23 -23
- package/docs/classes/PauseAction.md +23 -23
- package/docs/classes/PlaywrightFetchEngine.md +76 -64
- package/docs/classes/SubmitAction.md +23 -23
- package/docs/classes/WaitForAction.md +23 -23
- package/docs/classes/WebFetcher.md +5 -5
- package/docs/enumerations/FetchActionResultStatus.md +4 -4
- package/docs/functions/fetchWeb.md +2 -2
- package/docs/interfaces/BaseFetchActionProperties.md +11 -11
- package/docs/interfaces/BaseFetchCollectorActionProperties.md +15 -15
- package/docs/interfaces/BaseFetcherProperties.md +34 -26
- package/docs/interfaces/DispatchedEngineAction.md +4 -4
- package/docs/interfaces/ExtractActionProperties.md +11 -11
- package/docs/interfaces/FetchActionInContext.md +15 -15
- package/docs/interfaces/FetchActionProperties.md +12 -12
- package/docs/interfaces/FetchActionResult.md +6 -6
- package/docs/interfaces/FetchContext.md +48 -36
- package/docs/interfaces/FetchEngineContext.md +43 -31
- package/docs/interfaces/FetchMetadata.md +5 -5
- package/docs/interfaces/FetchResponse.md +14 -14
- package/docs/interfaces/FetchReturnTypeRegistry.md +7 -7
- package/docs/interfaces/FetchSite.md +41 -29
- package/docs/interfaces/FetcherOptions.md +40 -28
- package/docs/interfaces/GotoActionOptions.md +6 -6
- package/docs/interfaces/PendingEngineRequest.md +3 -3
- package/docs/interfaces/StorageOptions.md +5 -5
- package/docs/interfaces/SubmitActionOptions.md +2 -2
- package/docs/interfaces/WaitForActionOptions.md +5 -5
- package/docs/type-aliases/BaseFetchActionOptions.md +1 -1
- package/docs/type-aliases/BaseFetchCollectorOptions.md +1 -1
- package/docs/type-aliases/BrowserEngine.md +1 -1
- package/docs/type-aliases/FetchActionCapabilities.md +1 -1
- package/docs/type-aliases/FetchActionCapabilityMode.md +1 -1
- package/docs/type-aliases/FetchActionOptions.md +1 -1
- package/docs/type-aliases/FetchEngineAction.md +1 -1
- package/docs/type-aliases/FetchEngineType.md +1 -1
- package/docs/type-aliases/FetchReturnType.md +1 -1
- package/docs/type-aliases/FetchReturnTypeFor.md +1 -1
- package/docs/type-aliases/OnFetchPauseCallback.md +1 -1
- package/docs/type-aliases/ResourceType.md +1 -1
- package/docs/variables/DefaultFetcherProperties.md +1 -1
- package/docs/variables/FetcherOptionKeys.md +1 -1
- package/package.json +1 -1
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
# Interface: StorageOptions
|
|
8
8
|
|
|
9
|
-
Defined in: [packages/web-fetcher/src/core/types.ts:32](https://github.com/isdk/web-fetcher.js/blob/
|
|
9
|
+
Defined in: [packages/web-fetcher/src/core/types.ts:32](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/types.ts#L32)
|
|
10
10
|
|
|
11
11
|
Storage configuration options for the fetch engine.
|
|
12
12
|
|
|
@@ -20,7 +20,7 @@ Controls how Crawlee's internal storage (RequestQueue, KeyValueStore, SessionPoo
|
|
|
20
20
|
|
|
21
21
|
> `optional` **config**: `Record`\<`string`, `any`\>
|
|
22
22
|
|
|
23
|
-
Defined in: [packages/web-fetcher/src/core/types.ts:54](https://github.com/isdk/web-fetcher.js/blob/
|
|
23
|
+
Defined in: [packages/web-fetcher/src/core/types.ts:54](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/types.ts#L54)
|
|
24
24
|
|
|
25
25
|
Additional Crawlee configuration options.
|
|
26
26
|
Allows fine-grained control over the underlying Crawlee instance.
|
|
@@ -31,7 +31,7 @@ Allows fine-grained control over the underlying Crawlee instance.
|
|
|
31
31
|
|
|
32
32
|
> `optional` **id**: `string`
|
|
33
33
|
|
|
34
|
-
Defined in: [packages/web-fetcher/src/core/types.ts:38](https://github.com/isdk/web-fetcher.js/blob/
|
|
34
|
+
Defined in: [packages/web-fetcher/src/core/types.ts:38](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/types.ts#L38)
|
|
35
35
|
|
|
36
36
|
Custom identifier for the storage.
|
|
37
37
|
If provided, multiple sessions can share the same storage by using the same ID.
|
|
@@ -43,7 +43,7 @@ If not provided, a unique session ID is used (strong isolation).
|
|
|
43
43
|
|
|
44
44
|
> `optional` **persist**: `boolean`
|
|
45
45
|
|
|
46
|
-
Defined in: [packages/web-fetcher/src/core/types.ts:44](https://github.com/isdk/web-fetcher.js/blob/
|
|
46
|
+
Defined in: [packages/web-fetcher/src/core/types.ts:44](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/types.ts#L44)
|
|
47
47
|
|
|
48
48
|
Whether to persist storage to disk.
|
|
49
49
|
If true, uses Crawlee's disk persistence. If false, data might be stored in memory or temporary directory.
|
|
@@ -55,7 +55,7 @@ Corresponds to Crawlee's `persistStorage` configuration.
|
|
|
55
55
|
|
|
56
56
|
> `optional` **purge**: `boolean`
|
|
57
57
|
|
|
58
|
-
Defined in: [packages/web-fetcher/src/core/types.ts:49](https://github.com/isdk/web-fetcher.js/blob/
|
|
58
|
+
Defined in: [packages/web-fetcher/src/core/types.ts:49](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/types.ts#L49)
|
|
59
59
|
|
|
60
60
|
Whether to delete the storage (RequestQueue and KeyValueStore) when the session is closed.
|
|
61
61
|
Defaults to true. Set to false if you want to keep data for future reuse with the same `id`.
|
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
# Interface: SubmitActionOptions
|
|
8
8
|
|
|
9
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
9
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:72](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/engine/base.ts#L72)
|
|
10
10
|
|
|
11
11
|
Options for the [FetchEngine.submit](../classes/FetchEngine.md#submit) action, configuring form submission behavior.
|
|
12
12
|
|
|
@@ -20,4 +20,4 @@ Specifies encoding type for form submissions, particularly relevant for JSON-bas
|
|
|
20
20
|
|
|
21
21
|
> `optional` **enctype**: `"application/x-www-form-urlencoded"` \| `"application/json"` \| `"multipart/form-data"`
|
|
22
22
|
|
|
23
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
23
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:73](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/engine/base.ts#L73)
|
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
# Interface: WaitForActionOptions
|
|
8
8
|
|
|
9
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
9
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:59](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/engine/base.ts#L59)
|
|
10
10
|
|
|
11
11
|
Options for the [FetchEngine.waitFor](../classes/FetchEngine.md#waitfor) action, specifying conditions to wait for before continuing.
|
|
12
12
|
|
|
@@ -20,7 +20,7 @@ Controls timing behavior for interactions, allowing waiting for elements, time i
|
|
|
20
20
|
|
|
21
21
|
> `optional` **failOnTimeout**: `boolean`
|
|
22
22
|
|
|
23
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
23
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:63](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/engine/base.ts#L63)
|
|
24
24
|
|
|
25
25
|
***
|
|
26
26
|
|
|
@@ -28,7 +28,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:62](https://github.com/isdk
|
|
|
28
28
|
|
|
29
29
|
> `optional` **ms**: `number`
|
|
30
30
|
|
|
31
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
31
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:60](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/engine/base.ts#L60)
|
|
32
32
|
|
|
33
33
|
***
|
|
34
34
|
|
|
@@ -36,7 +36,7 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:59](https://github.com/isdk
|
|
|
36
36
|
|
|
37
37
|
> `optional` **networkIdle**: `boolean`
|
|
38
38
|
|
|
39
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
39
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:62](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/engine/base.ts#L62)
|
|
40
40
|
|
|
41
41
|
***
|
|
42
42
|
|
|
@@ -44,4 +44,4 @@ Defined in: [packages/web-fetcher/src/engine/base.ts:61](https://github.com/isdk
|
|
|
44
44
|
|
|
45
45
|
> `optional` **selector**: `string`
|
|
46
46
|
|
|
47
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
47
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:61](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/engine/base.ts#L61)
|
|
@@ -8,4 +8,4 @@
|
|
|
8
8
|
|
|
9
9
|
> **BaseFetchActionOptions** = `RequireAtLeastOne`\<[`BaseFetchActionProperties`](../interfaces/BaseFetchActionProperties.md), `"id"` \| `"name"` \| `"action"`\>
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/action/fetch-action.ts:61](https://github.com/isdk/web-fetcher.js/blob/
|
|
11
|
+
Defined in: [packages/web-fetcher/src/action/fetch-action.ts:61](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/action/fetch-action.ts#L61)
|
|
@@ -8,4 +8,4 @@
|
|
|
8
8
|
|
|
9
9
|
> **BaseFetchCollectorOptions** = `RequireAtLeastOne`\<[`BaseFetchCollectorActionProperties`](../interfaces/BaseFetchCollectorActionProperties.md), `"id"` \| `"name"` \| `"action"`\>
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/action/fetch-action.ts:74](https://github.com/isdk/web-fetcher.js/blob/
|
|
11
|
+
Defined in: [packages/web-fetcher/src/action/fetch-action.ts:74](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/action/fetch-action.ts#L74)
|
|
@@ -8,4 +8,4 @@
|
|
|
8
8
|
|
|
9
9
|
> **BrowserEngine** = `"playwright"` \| `"puppeteer"`
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/core/types.ts:21](https://github.com/isdk/web-fetcher.js/blob/
|
|
11
|
+
Defined in: [packages/web-fetcher/src/core/types.ts:21](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/types.ts#L21)
|
|
@@ -8,4 +8,4 @@
|
|
|
8
8
|
|
|
9
9
|
> **FetchActionCapabilities** = `{ [mode in FetchEngineType]?: FetchActionCapabilityMode }`
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/action/fetch-action.ts:82](https://github.com/isdk/web-fetcher.js/blob/
|
|
11
|
+
Defined in: [packages/web-fetcher/src/action/fetch-action.ts:82](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/action/fetch-action.ts#L82)
|
|
@@ -8,4 +8,4 @@
|
|
|
8
8
|
|
|
9
9
|
> **FetchActionCapabilityMode** = `"native"` \| `"simulate"` \| `"noop"`
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/action/fetch-action.ts:23](https://github.com/isdk/web-fetcher.js/blob/
|
|
11
|
+
Defined in: [packages/web-fetcher/src/action/fetch-action.ts:23](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/action/fetch-action.ts#L23)
|
|
@@ -8,4 +8,4 @@
|
|
|
8
8
|
|
|
9
9
|
> **FetchActionOptions** = `RequireAtLeastOne`\<[`FetchActionProperties`](../interfaces/FetchActionProperties.md), `"id"` \| `"name"` \| `"action"`\>
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/action/fetch-action.ts:80](https://github.com/isdk/web-fetcher.js/blob/
|
|
11
|
+
Defined in: [packages/web-fetcher/src/action/fetch-action.ts:80](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/action/fetch-action.ts#L80)
|
|
@@ -8,7 +8,7 @@
|
|
|
8
8
|
|
|
9
9
|
> **FetchEngineAction** = \{ `selector`: `string`; `type`: `"click"`; \} \| \{ `selector`: `string`; `type`: `"fill"`; `value`: `string`; \} \| \{ `options?`: [`WaitForActionOptions`](../interfaces/WaitForActionOptions.md); `type`: `"waitFor"`; \} \| \{ `options?`: [`SubmitActionOptions`](../interfaces/SubmitActionOptions.md); `selector?`: `any`; `type`: `"submit"`; \} \| \{ `type`: `"getContent"`; \} \| \{ `opts?`: [`GotoActionOptions`](../interfaces/GotoActionOptions.md); `type`: `"navigate"`; `url`: `string`; \} \| \{ `schema`: `ExtractSchema`; `type`: `"extract"`; \} \| \{ `message?`: `string`; `type`: `"pause"`; \} \| \{ `type`: `"dispose"`; \}
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/engine/base.ts:
|
|
11
|
+
Defined in: [packages/web-fetcher/src/engine/base.ts:83](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/engine/base.ts#L83)
|
|
12
12
|
|
|
13
13
|
Union type representing all possible engine actions that can be dispatched.
|
|
14
14
|
|
|
@@ -8,4 +8,4 @@
|
|
|
8
8
|
|
|
9
9
|
> **FetchEngineType** = `"http"` \| `"browser"`
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/core/types.ts:20](https://github.com/isdk/web-fetcher.js/blob/
|
|
11
|
+
Defined in: [packages/web-fetcher/src/core/types.ts:20](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/types.ts#L20)
|
|
@@ -8,4 +8,4 @@
|
|
|
8
8
|
|
|
9
9
|
> **FetchReturnType** = `"response"` \| `"context"` \| `"outputs"` \| `"any"` \| `"none"`
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/core/fetch-return-type.ts:5](https://github.com/isdk/web-fetcher.js/blob/
|
|
11
|
+
Defined in: [packages/web-fetcher/src/core/fetch-return-type.ts:5](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/fetch-return-type.ts#L5)
|
|
@@ -8,7 +8,7 @@
|
|
|
8
8
|
|
|
9
9
|
> **FetchReturnTypeFor**\<`R`\> = `R` *extends* keyof [`FetchReturnTypeRegistry`](../interfaces/FetchReturnTypeRegistry.md) ? [`FetchReturnTypeRegistry`](../interfaces/FetchReturnTypeRegistry.md)\[`R`\] : `never`
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/core/fetch-return-type.ts:22](https://github.com/isdk/web-fetcher.js/blob/
|
|
11
|
+
Defined in: [packages/web-fetcher/src/core/fetch-return-type.ts:22](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/fetch-return-type.ts#L22)
|
|
12
12
|
|
|
13
13
|
## Type Parameters
|
|
14
14
|
|
|
@@ -8,7 +8,7 @@
|
|
|
8
8
|
|
|
9
9
|
> **OnFetchPauseCallback** = (`options`) => `Promise`\<`void`\>
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/core/types.ts:
|
|
11
|
+
Defined in: [packages/web-fetcher/src/core/types.ts:134](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/types.ts#L134)
|
|
12
12
|
|
|
13
13
|
## Parameters
|
|
14
14
|
|
|
@@ -8,4 +8,4 @@
|
|
|
8
8
|
|
|
9
9
|
> **ResourceType** = `"image"` \| `"stylesheet"` \| `"font"` \| `"script"` \| `"media"` \| `string`
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/core/types.ts:24](https://github.com/isdk/web-fetcher.js/blob/
|
|
11
|
+
Defined in: [packages/web-fetcher/src/core/types.ts:24](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/types.ts#L24)
|
|
@@ -8,4 +8,4 @@
|
|
|
8
8
|
|
|
9
9
|
> `const` **DefaultFetcherProperties**: [`BaseFetcherProperties`](../interfaces/BaseFetcherProperties.md)
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/core/types.ts:
|
|
11
|
+
Defined in: [packages/web-fetcher/src/core/types.ts:176](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/types.ts#L176)
|
|
@@ -8,4 +8,4 @@
|
|
|
8
8
|
|
|
9
9
|
> `const` **FetcherOptionKeys**: `string`[]
|
|
10
10
|
|
|
11
|
-
Defined in: [packages/web-fetcher/src/core/types.ts:
|
|
11
|
+
Defined in: [packages/web-fetcher/src/core/types.ts:212](https://github.com/isdk/web-fetcher.js/blob/1f3a81cc4fe8a2cf40fe3f3e8030d14a75c9d27c/src/core/types.ts#L212)
|