@isdk/proxy 0.1.1 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/docs/README.md CHANGED
@@ -19,6 +19,8 @@ In high-concurrency environments—like **API Proxies**, **Web Scrapers**, or **
19
19
  ## Key Features
20
20
 
21
21
  - **🚀 Hybrid Multi-tier Cache**: Extreme speed with L1 (LRU Memory) and persistence with L2 (Content Addressable Disk via `cacache`).
22
+ - **📥 HTTP POST & Method Support**: Full support for caching POST, PUT, and other methods with intelligent request body fingerprinting.
23
+ - **🎯 Precision Filtering**: Fine-grained `cacheRules` to intercept specific paths or query parameters.
22
24
  - **🌊 Streaming Native**: Fully stream-based internal pipeline natively prevents Out-Of-Memory (OOM) issues when proxying large files.
23
25
  - **🧠 Intelligent Meta-Residency**: Metadata (Headers, Status, Policy) stays in memory regardless of body size, ensuring nanosecond cache policy evaluations.
24
26
  - **🔄 Stale-While-Revalidate (SWR)**: Serve stale content instantly while updating the cache silently in the background.
@@ -37,6 +39,8 @@ pnpm add @isdk/proxy
37
39
 
38
40
  The primary way to use `@isdk/proxy` is via the `fetchWithCache` function, which can wrap any HTTP request logic.
39
41
 
42
+ ### Basic Usage (GET)
43
+
40
44
  ```typescript
41
45
  import { SmartCache, createCachedFetch } from '@isdk/proxy';
42
46
 
@@ -46,28 +50,98 @@ const cache = new SmartCache({
46
50
  maxMemorySize: 1024 * 1024 // 1MB threshold
47
51
  });
48
52
 
49
- // 2. Create a pre-configured cached fetcher (automatically tracks concurrent requests)
53
+ // 2. Create a pre-configured cached fetcher
50
54
  const myFetch = createCachedFetch({
51
55
  cache,
52
56
  config: {
53
57
  staleIfError: true,
54
- forceCache: false // Set to true to cache everything (ignore no-store) for offline-first apps
55
58
  },
56
59
  backgroundUpdate: true // Enable SWR
57
60
  });
58
61
 
59
- // 3. Use it anywhere in your app!
60
- const request = new Request('https://api.example.com/data');
61
- const response = await myFetch(request, (req) => fetch(req));
62
+ // 3. Use it!
63
+ const response = await myFetch(new Request('https://api.example.com/data'), (req) => fetch(req));
64
+ console.log(response.headers.get('x-proxy-cache'));
65
+ ```
66
+
67
+ ### Advanced Usage: Caching POST Requests
62
68
 
63
- console.log(response.headers.get('x-proxy-cache')); // "MISS", "HIT", "STALE", or "STALE_IF_ERROR"
64
- const data = await response.json();
69
+ You can cache POST/PUT requests by enabling methods and defining body filters to ignore dynamic fields (like timestamps) in the request body.
70
+
71
+ ```typescript
72
+ const myPostFetch = createCachedFetch({
73
+ cache,
74
+ config: {
75
+ methods: ['GET', 'POST'], // Enable POST caching
76
+ body: {
77
+ exclude: ['timestamp', 'nonce'] // Ignore these fields when generating cache keys
78
+ },
79
+ cacheRules: [
80
+ { method: 'POST', path: '/api/v1/query' } // Only cache specific POST endpoints
81
+ ],
82
+ forceCache: true // Often needed for POST if backend doesn't send Cache-Control
83
+ }
84
+ });
85
+ ```
86
+
87
+ ## Configuration: `SiteCacheConfig`
88
+
89
+ | Field | Type | Description |
90
+ | :--- | :--- | :--- |
91
+ | `methods` | `string[]` | List of allowed HTTP methods. Default: `['GET', 'HEAD']`. |
92
+ | `cacheRules` | `CacheRule[]` | Fine-grained rules. If set, a request must match at least one rule to be cached. |
93
+ | `query` | `KeyFilterConfig` | Filters for URL search parameters (`include`/`exclude`). |
94
+ | `headers` | `KeyFilterConfig` | Filters for request headers. |
95
+ | `cookies` | `KeyFilterConfig` | Filters for cookies. |
96
+ | `body` | `KeyFilterConfig` | Filters for JSON request body fields. |
97
+ | `staleIfError`| `boolean` | Serve stale cache on network failure. |
98
+ | `forceCache` | `boolean` | Ignore `no-store` and force caching (useful for offline support). |
99
+
100
+ ### `CacheRule` Object
101
+
102
+ - `method`: HTTP method to match.
103
+ - `path`: URL pathname matching (supports **RegExp**, **Glob**, **Array**, or **prefix match**).
104
+ - `query`: Key-value pairs. Values can be `string` (exact/Glob match), `true` (must exist), `false` (must not exist), or `RegExp`.
105
+ - `body`: Body content matching (supports **RegExp**, **Glob**, or **Array**).
106
+
107
+ ### Pattern Matching
108
+
109
+ `@isdk/proxy` provides powerful pattern matching for all configurable fields:
110
+
111
+ | Pattern Type | Example | Description |
112
+ | :--- | :--- | :--- |
113
+ | **RegExp** | `/api/v[12]/.*/i` | JavaScript RegExp (in JSON, use string like `"/api/v[12]/.*/i"`) |
114
+ | **Glob** | `/**/*.json` | File path style wildcard matching |
115
+ | **Negation** | `['!/api/private/**', '/api/**']` | Exclude patterns (prefixed with `!`, checked first) |
116
+ | **Array** | `['/api/v1/*', '/api/v2/*']` | Multiple patterns (OR logic, negative takes precedence) |
117
+ | **Boolean** | `true` / `false` | For query params: must/must not exist |
118
+
119
+ **Example with advanced pattern matching:**
120
+
121
+ ```typescript
122
+ const myFetch = createCachedFetch({
123
+ cache,
124
+ config: {
125
+ cacheRules: [
126
+ {
127
+ path: ['/api/v1/items/*', '!/api/v1/items/private/*'], // v1 items, exclude private
128
+ query: {
129
+ format: '/^(json|xml)$/', // Regex for format param
130
+ 'page*': true // Glob: any param starting with 'page' must exist
131
+ },
132
+ body: /\"action\"\s*:\s*\"query\"/ // Regex body match
133
+ }
134
+ ]
135
+ }
136
+ });
65
137
  ```
66
138
 
67
139
  ## Adapters
68
140
 
69
141
  `@isdk/proxy` is designed to be framework-agnostic. While the core library is pure, you can find (or build) adapters for specific environments:
70
142
 
143
+ - **HTTP Caching Proxy Server (Node.js)**: See [@isdk/proxy-server](https://www.npmjs.com/package/@isdk/proxy-server) (separate package) for running a standalone HTTP forward proxy.
144
+ - **Crawlee Adapter**: See [@isdk/proxy-crawlee](https://www.npmjs.com/package/@isdk/proxy-crawlee) (separate package) for integrating with Crawlee web scraping lifecycle.
71
145
  - **MSW Adapter**: See `@isdk/proxy-msw` (separate package) to use this caching engine as an MSW interceptor.
72
146
  - **Axios Adapter**: Easily implemented by converting Axios config to Web `Request`.
73
147
 
@@ -120,9 +194,119 @@ The hybrid multi-tier storage engine.
120
194
  - **`options.maxMemorySize`**: Threshold (in bytes) for offloading bodies to disk (default `1048576`, i.e., 1MB).
121
195
  - **`options.storagePath`**: Disk storage path for the `cacache` engine (defaults to a system temp folder).
122
196
 
197
+ ### Utility Functions
198
+
199
+ Exported from `@isdk/proxy` for advanced usage:
200
+
201
+ #### `isMatch(pattern, value, usePrefix?)`
202
+
203
+ Universal pattern matching function. Supports RegExp, Glob, array patterns (with negation), and string prefix/exact matching.
204
+
205
+ - **`pattern`**: `string | RegExp | (string | RegExp)[]`
206
+ - **`value`**: The string to test against
207
+ - **`usePrefix`**: For plain strings, use prefix match instead of exact match (default: `false`)
208
+ - **Returns**: `boolean`
209
+
210
+ ```typescript
211
+ import { isMatch } from '@isdk/proxy';
212
+
213
+ isMatch('/api/v[12]/.*', '/api/v1/users'); // RegExp
214
+ isMatch('/api/**/*.json', '/api/v1/data.json'); // Glob
215
+ isMatch(['!/private/**', '/api/**'], '/api/data'); // Negation
216
+ ```
217
+
218
+ #### `isGlob(pattern)`
219
+
220
+ Check if a pattern is Glob syntax.
221
+
222
+ - **`pattern`**: `string`
223
+ - **Returns**: `boolean`
224
+
225
+ #### `getSiteConfig(urlString, proxyConfig)`
226
+
227
+ Get the site-specific cache configuration for a given URL.
228
+
229
+ - **`urlString`**: Full URL to match
230
+ - **`proxyConfig`**: `ProxyConfig` object with `sites` and `default` config
231
+ - **Returns**: `SiteCacheConfig`
232
+
233
+ ```typescript
234
+ import { getSiteConfig } from '@isdk/proxy';
235
+
236
+ const config = getSiteConfig('https://api.example.com/data', {
237
+ default: { methods: ['GET'] },
238
+ sites: {
239
+ 'api.example.com': { methods: ['GET', 'POST'], forceCache: true },
240
+ '/internal/': { staleIfError: true } // prefix match
241
+ }
242
+ });
243
+ ```
244
+
245
+ #### `isAllowed(key, config, defaultAllowed?)`
246
+
247
+ Check if a key is allowed to participate in cache key fingerprinting.
248
+
249
+ - **`key`**: The key name to check
250
+ - **`config`**: `KeyFilterConfig` with `include` (whitelist) or `exclude` (blacklist)
251
+ - **`defaultAllowed`**: Optional. Default value when no config or no match
252
+ - **Returns**: `boolean | undefined`
253
+
254
+ **Priority Logic**:
255
+
256
+ 1. `exclude` hit → returns `false` (highest priority)
257
+ 2. `include` exists and hits → returns `true`
258
+ 3. `include` exists but no hit → returns `false`
259
+ 4. No config → uses `defaultAllowed` (returns `undefined` if not provided)
260
+
261
+ ```typescript
262
+ import { isAllowed } from '@isdk/proxy';
263
+
264
+ // No config
265
+ isAllowed('key'); // undefined (falsy)
266
+
267
+ // Whitelist
268
+ isAllowed('id', { include: ['id', 'name'] }); // true
269
+ isAllowed('email', { include: ['id', 'name'] }); // false
270
+
271
+ // Blacklist
272
+ isAllowed('password', { exclude: ['password'] }); // false
273
+ isAllowed('name', { exclude: ['password'] }); // undefined (falsy)
274
+
275
+ // Need defaultAllowed to set default
276
+ isAllowed('name', { exclude: ['password'] }, true); // true
277
+ ```
278
+
279
+ #### `extractData(source, config, defaultAllowed?)`
280
+
281
+ Extract and normalize data from a source object based on filter config. Used for generating cache fingerprints.
282
+
283
+ - **`source`**: Original data object (Query, Headers, Cookies, etc.)
284
+ - **`config`**: `KeyFilterConfig` object
285
+ - **`defaultAllowed`**: Optional. Whether to allow extraction when no config or no match (default `false`)
286
+ - **Returns**: `Record<string, string[]>` normalized data with lowercase keys and sorted array values
287
+
288
+ ```typescript
289
+ import { extractData } from '@isdk/proxy';
290
+
291
+ const headers = { 'Content-Type': 'application/json', 'X-Request-Id': '123' };
292
+
293
+ // No extraction by default
294
+ extractData(headers); // {}
295
+
296
+ // Extract all keys
297
+ extractData(headers, undefined, true); // { 'content-type': ['application/json'], 'x-request-id': ['123'] }
298
+
299
+ // Whitelist
300
+ extractData(headers, { include: ['content-type'] }); // { 'content-type': ['application/json'] }
301
+
302
+ // Blacklist
303
+ extractData(headers, { include: ['*'], exclude: ['x-request-id'] }, true); // { 'content-type': ['application/json'] }
304
+ ```
305
+
123
306
  ### Cache Status Headers
124
307
 
125
308
  Every response processed by `@isdk/proxy` will include an `x-proxy-cache` header indicating its lifecycle:
309
+
126
310
  - `HIT`: Served entirely from L1 or L2 cache.
127
311
  - `MISS`: Bypassed cache and fetched from the origin server.
128
312
  - `STALE`: Served from stale cache while a background update was initiated (SWR).
@@ -6,9 +6,19 @@
6
6
 
7
7
  # Class: SmartCache
8
8
 
9
- Defined in: [core/SmartCache.ts:22](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/SmartCache.ts#L22)
9
+ Defined in: [core/SmartCache.ts:39](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/SmartCache.ts#L39)
10
10
 
11
- 智能混合缓存类 (Hybrid Cache)
11
+ 智能混合缓存类 (Hybrid Multi-tier Cache)
12
+
13
+ 该类实现了 L1 (内存) 和 L2 (磁盘) 的双层混合存储架构,旨在提供高性能且大容量的缓存能力。
14
+
15
+ ### 核心特性:
16
+ - **双层架构**: L1 使用 LRU 内存缓存(基于 `secondary-cache` 的 LRUCache),L2 使用持久化磁盘缓存(基于 `cacache`)。
17
+ - **大小感知存储**: 自动识别响应体大小。小于阈值的文件同时存于内存和磁盘;超过阈值的文件仅存于磁盘,但其元数据仍保留在内存中。
18
+ - **元数据驻留 (Meta-Residency)**: 无论 Body 多大,Headers、Status、Policy 等信息始终优先从内存读取,确保缓存判定性能。
19
+ - **流式支持**: 支持通过 `setStream` 和 `getStream` 直接操作大数据流,防止 OOM。
20
+ - **一致性保障**: 在并发写入时自动清理内存,确保后续读取不会拿到被污染的旧数据。
21
+ - **内存限制**: 通过 `maxTotalMemorySize` 控制 L1 缓存的总内存占用。
12
22
 
13
23
  ## Constructors
14
24
 
@@ -16,7 +26,7 @@ Defined in: [core/SmartCache.ts:22](https://github.com/isdk/proxy.js/blob/bed37f
16
26
 
17
27
  > **new SmartCache**(`options`): `SmartCache`
18
28
 
19
- Defined in: [core/SmartCache.ts:27](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/SmartCache.ts#L27)
29
+ Defined in: [core/SmartCache.ts:44](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/SmartCache.ts#L44)
20
30
 
21
31
  #### Parameters
22
32
 
@@ -34,7 +44,7 @@ Defined in: [core/SmartCache.ts:27](https://github.com/isdk/proxy.js/blob/bed37f
34
44
 
35
45
  > **clear**(): `Promise`\<`void`\>
36
46
 
37
- Defined in: [core/SmartCache.ts:122](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/SmartCache.ts#L122)
47
+ Defined in: [core/SmartCache.ts:193](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/SmartCache.ts#L193)
38
48
 
39
49
  #### Returns
40
50
 
@@ -46,7 +56,7 @@ Defined in: [core/SmartCache.ts:122](https://github.com/isdk/proxy.js/blob/bed37
46
56
 
47
57
  > **delete**(`key`): `Promise`\<`void`\>
48
58
 
49
- Defined in: [core/SmartCache.ts:117](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/SmartCache.ts#L117)
59
+ Defined in: [core/SmartCache.ts:188](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/SmartCache.ts#L188)
50
60
 
51
61
  #### Parameters
52
62
 
@@ -64,10 +74,15 @@ Defined in: [core/SmartCache.ts:117](https://github.com/isdk/proxy.js/blob/bed37
64
74
 
65
75
  > **get**(`key`): `Promise`\<[`CacheEntry`](../interfaces/CacheEntry.md) \| `null`\>
66
76
 
67
- Defined in: [core/SmartCache.ts:41](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/SmartCache.ts#L41)
77
+ Defined in: [core/SmartCache.ts:79](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/SmartCache.ts#L79)
68
78
 
69
79
  获取缓存条目
70
- 如果是小文件,返回带 Buffer 的 Entry;如果是大文件,返回带 ReadStream 的 Entry。
80
+
81
+ 逻辑:
82
+ 1. 首先尝试从 L1 内存获取。
83
+ 2. 如果内存中有 Body,直接返回(Buffer 类型)。
84
+ 3. 如果内存中只有 Meta(大文件),则从 L2 磁盘创建并返回 ReadStream。
85
+ 4. 如果内存完全未命中,从磁盘 L2 检索,并根据大小决定是否回填 L1。
71
86
 
72
87
  #### Parameters
73
88
 
@@ -75,17 +90,25 @@ Defined in: [core/SmartCache.ts:41](https://github.com/isdk/proxy.js/blob/bed37f
75
90
 
76
91
  `string`
77
92
 
93
+ 缓存指纹键
94
+
78
95
  #### Returns
79
96
 
80
97
  `Promise`\<[`CacheEntry`](../interfaces/CacheEntry.md) \| `null`\>
81
98
 
99
+ 完整的缓存条目(带 Buffer 或 Stream 的 Body),未命中返回 null
100
+
82
101
  ***
83
102
 
84
103
  ### getStream()
85
104
 
86
105
  > **getStream**(`key`): `ReadableStream`
87
106
 
88
- Defined in: [core/SmartCache.ts:107](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/SmartCache.ts#L107)
107
+ Defined in: [core/SmartCache.ts:159](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/SmartCache.ts#L159)
108
+
109
+ 获取磁盘读取流
110
+
111
+ 允许直接从 L2 磁盘层以流的形式读取数据,适用于大文件代理。
89
112
 
90
113
  #### Parameters
91
114
 
@@ -93,19 +116,25 @@ Defined in: [core/SmartCache.ts:107](https://github.com/isdk/proxy.js/blob/bed37
93
116
 
94
117
  `string`
95
118
 
119
+ 缓存指纹键
120
+
96
121
  #### Returns
97
122
 
98
123
  `ReadableStream`
99
124
 
125
+ Node.js 可读流
126
+
100
127
  ***
101
128
 
102
129
  ### set()
103
130
 
104
131
  > **set**(`key`, `body`, `metadata`): `Promise`\<`void`\>
105
132
 
106
- Defined in: [core/SmartCache.ts:85](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/SmartCache.ts#L85)
133
+ Defined in: [core/SmartCache.ts:129](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/SmartCache.ts#L129)
107
134
 
108
- 写入缓存
135
+ 写入缓存条目 (原子写入)
136
+
137
+ 适用于已知长度的小型数据块。该操作会同时写入磁盘并回填内存(如果大小未超标)。
109
138
 
110
139
  #### Parameters
111
140
 
@@ -113,14 +142,20 @@ Defined in: [core/SmartCache.ts:85](https://github.com/isdk/proxy.js/blob/bed37f
113
142
 
114
143
  `string`
115
144
 
145
+ 缓存指纹键
146
+
116
147
  ##### body
117
148
 
118
149
  `Buffer`
119
150
 
151
+ 响应体数据 Buffer
152
+
120
153
  ##### metadata
121
154
 
122
155
  `Omit`\<[`CacheMetadata`](../interfaces/CacheMetadata.md), `"size"`\>
123
156
 
157
+ 响应元数据(不含 size,由本方法自动计算)
158
+
124
159
  #### Returns
125
160
 
126
161
  `Promise`\<`void`\>
@@ -131,7 +166,14 @@ Defined in: [core/SmartCache.ts:85](https://github.com/isdk/proxy.js/blob/bed37f
131
166
 
132
167
  > **setStream**(`key`, `metadata`): `WritableStream`
133
168
 
134
- Defined in: [core/SmartCache.ts:111](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/SmartCache.ts#L111)
169
+ Defined in: [core/SmartCache.ts:175](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/SmartCache.ts#L175)
170
+
171
+ 获取磁盘写入流 (流式缓存)
172
+
173
+ 该方法用于支持真正的流式代理。它会执行以下一致性操作:
174
+ 1. 立即清除 L1 内存中的对应键,防止读到旧数据。
175
+ 2. 返回一个可写流,数据将直接流入磁盘。
176
+ 3. **一致性修复**: 在流写入完成(finish)时再次清理内存,防止写入期间的并发读取将旧数据再次回填进内存。
135
177
 
136
178
  #### Parameters
137
179
 
@@ -139,10 +181,16 @@ Defined in: [core/SmartCache.ts:111](https://github.com/isdk/proxy.js/blob/bed37
139
181
 
140
182
  `string`
141
183
 
184
+ 缓存指纹键
185
+
142
186
  ##### metadata
143
187
 
144
188
  `Omit`\<[`CacheMetadata`](../interfaces/CacheMetadata.md), `"size"`\>
145
189
 
190
+ 响应元数据
191
+
146
192
  #### Returns
147
193
 
148
194
  `WritableStream`
195
+
196
+ Node.js 可写流
@@ -8,7 +8,7 @@
8
8
 
9
9
  > **createCachedFetch**(`defaultOptions`): (`request`, `fetcher`, `overrideOptions?`) => `Promise`\<`Response`\>
10
10
 
11
- Defined in: [core/createCachedFetch.ts:17](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/createCachedFetch.ts#L17)
11
+ Defined in: [core/createCachedFetch.ts:17](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/createCachedFetch.ts#L17)
12
12
 
13
13
  缓存请求工厂函数 (针对终端用户的顶层高阶 API)
14
14
 
@@ -8,7 +8,7 @@
8
8
 
9
9
  > **createFetchWithCache**(`activeCacheWrites?`): (`request`, `fetcher`, `options`) => `Promise`\<`Response`\>
10
10
 
11
- Defined in: [core/createFetchWithCache.ts:16](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/createFetchWithCache.ts#L16)
11
+ Defined in: [core/createFetchWithCache.ts:16](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/createFetchWithCache.ts#L16)
12
12
 
13
13
  单一职责高阶函数:专门用于封装和隔离 activeCacheWrites 并发追踪器。
14
14
 
@@ -6,18 +6,23 @@
6
6
 
7
7
  # Function: extractData()
8
8
 
9
- > **extractData**(`source`, `config?`): `Record`\<`string`, `string`[]\>
9
+ > **extractData**(`source`, `config?`, `defaultAllowed?`): `Record`\<`string`, `string`[]\>
10
10
 
11
- Defined in: [utils/extractData.ts:17](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/utils/extractData.ts#L17)
11
+ Defined in: [utils/extractData.ts:40](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/utils/extractData.ts#L40)
12
12
 
13
13
  从源对象中根据过滤配置提取数据并标准化。
14
14
 
15
15
  此函数主要用于生成缓存指纹。它会:
16
- 1. 根据 `config` (include/exclude) 过滤键。
16
+ 1. 根据 `config` (include/exclude) 过滤键,调用 `isAllowed` 判断每个键是否允许。
17
17
  2. 对键进行排序以保证指纹的一致性。
18
18
  3. 将所有键转换为小写。
19
19
  4. 将值统一包装为数组并进行排序,消除数组项顺序差异。
20
20
 
21
+ **关于 `defaultAllowed` 参数**:
22
+ - 只有当没有配置 `include` 和 `exclude` 时,`defaultAllowed` 才会生效。
23
+ - 如果配置了 `include`(即使为空数组),`defaultAllowed` 也不会生效。
24
+ - 详见 `isAllowed` 函数的优先级逻辑。
25
+
21
26
  ## Parameters
22
27
 
23
28
  ### source
@@ -30,10 +35,34 @@ Defined in: [utils/extractData.ts:17](https://github.com/isdk/proxy.js/blob/bed3
30
35
 
31
36
  [`KeyFilterConfig`](../interfaces/KeyFilterConfig.md)
32
37
 
33
- 过滤配置 (白名单或黑名单)
38
+ 过滤配置,支持 `include`(白名单)和 `exclude`(黑名单)
39
+
40
+ ### defaultAllowed?
41
+
42
+ `boolean`
43
+
44
+ 当没有配置时的默认值(默认 `false`,即不提取任何键)
34
45
 
35
46
  ## Returns
36
47
 
37
48
  `Record`\<`string`, `string`[]\>
38
49
 
39
- 标准化后的数据 Map,键为小写,值为字符串数组
50
+ 标准化后的数据 Map,键为小写,值为排序后的字符串数组
51
+
52
+ ## Example
53
+
54
+ ```typescript
55
+ const headers = { 'Content-Type': 'application/json', 'X-Request-Id': '123' };
56
+
57
+ // 默认不提取任何键
58
+ extractData(headers); // {}
59
+
60
+ // 提取所有键
61
+ extractData(headers, undefined, true); // { 'content-type': ['application/json'], 'x-request-id': ['123'] }
62
+
63
+ // 白名单
64
+ extractData(headers, { include: ['content-type'] }); // { 'content-type': ['application/json'] }
65
+
66
+ // 黑名单(需要 include 或 defaultAllowed)
67
+ extractData(headers, { include: ['*'], exclude: ['x-request-id'] }, true); // { 'content-type': ['application/json'] }
68
+ ```
@@ -8,11 +8,13 @@
8
8
 
9
9
  > **fetchWithCache**(`request`, `fetcher`, `options`): `Promise`\<`Response`\>
10
10
 
11
- Defined in: [core/fetchWithCache.ts:244](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/fetchWithCache.ts#L244)
11
+ Defined in: [core/fetchWithCache.ts:370](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/fetchWithCache.ts#L370)
12
12
 
13
13
  核心协调函数 (Fetcher Orchestrator)
14
14
 
15
15
  实现了基于流的混合缓存代理核心逻辑,主要机制包括:
16
+ - **多方法支持与过滤**:支持通过 `allowedMethods` 配置可缓存的方法(如 POST, PUT),并通过 `cacheRules` 进行精细化的路径与参数匹配拦截。
17
+ - **异步 Request Body 处理**:当缓存 POST/PUT 请求时,会自动读取 Body 并计算唯一指纹(支持 JSON 字段过滤)。
16
18
  - **大文件流式处理**:底层完全通过 Streams 实现,代理大文件时自动写入磁盘且防 OOM。
17
19
  - **SWR (Stale-While-Revalidate)**:后台静默更新机制。
18
20
  - **并发防击穿 (Request Coalescing)**:利用 `activeCacheWrites` 将并发请求合并。
@@ -26,14 +28,22 @@ Defined in: [core/fetchWithCache.ts:244](https://github.com/isdk/proxy.js/blob/b
26
28
 
27
29
  `Request`
28
30
 
31
+ 原始 Web 标准 Request 对象
32
+
29
33
  ### fetcher
30
34
 
31
35
  (`req`) => `Promise`\<`Response`\>
32
36
 
37
+ 实际执行网络请求的函数
38
+
33
39
  ### options
34
40
 
35
41
  [`FetchWithCacheOptions`](../interfaces/FetchWithCacheOptions.md)
36
42
 
43
+ 缓存配置选项
44
+
37
45
  ## Returns
38
46
 
39
47
  `Promise`\<`Response`\>
48
+
49
+ 带有缓存标识头和流式 Body 的 Response 对象
@@ -6,11 +6,26 @@
6
6
 
7
7
  # Function: generateCacheKey()
8
8
 
9
- > **generateCacheKey**(`req`, `config`): `string`
9
+ > **generateCacheKey**(`req`, `config`): `Promise`\<`string`\>
10
10
 
11
- Defined in: [core/generateCacheKey.ts:8](https://github.com/isdk/proxy.js/blob/bed37fa43507dcbe5cdfa453876163571399d761/src/core/generateCacheKey.ts#L8)
11
+ Defined in: [core/generateCacheKey.ts:36](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/core/generateCacheKey.ts#L36)
12
12
 
13
- 根据 Request 和配置生成唯一的缓存键
13
+ 根据 Request 对象和站点配置生成唯一的缓存指纹 (异步)
14
+
15
+ 该函数是缓存系统的核心组件,用于将复杂的 HTTP 请求对象转换为唯一的 SHA-256 字符串。
16
+ 它实现了高度可定制的提取逻辑,允许通过配置排除掉请求中不稳定的因素(如时间戳、Nonce 等)。
17
+
18
+ ### 生成指纹包含的要素:
19
+ 1. **Method**: 请求方法(统一转为大写)。
20
+ 2. **Host & Path**: 请求的域名和路径。
21
+ 3. **Query Params**: URL 查询参数,受 `config.query` 过滤影响。
22
+ 4. **Headers**: 请求头信息,受 `config.headers` 过滤影响。默认排除 `cookie` 头。
23
+ 5. **Cookies**: 特别提取的 Cookie 字段,受 `config.cookies` 过滤影响。
24
+ 6. **Request Body**:
25
+ - 对于 `POST`, `PUT`, `PATCH` 请求,会自动尝试读取 Body。
26
+ - **JSON 类型**: 如果 `Content-Type` 包含 `application/json`,则解析为对象并应用 `config.body` 过滤。
27
+ - **非 JSON/流类型**: 回退到对原始 Body 字节流进行 SHA-256 哈希计算。
28
+ - **安全性**: 使用 `req.clone()` 读取 Body,确保不影响后续真实的 Fetch 请求流消费。
14
29
 
15
30
  ## Parameters
16
31
 
@@ -18,10 +33,25 @@ Defined in: [core/generateCacheKey.ts:8](https://github.com/isdk/proxy.js/blob/b
18
33
 
19
34
  `Request`
20
35
 
36
+ 原始 Web 标准 Request 对象。
37
+
21
38
  ### config
22
39
 
23
40
  [`SiteCacheConfig`](../interfaces/SiteCacheConfig.md)
24
41
 
42
+ 站点级缓存配置,决定了哪些字段参与指纹计算。
43
+
25
44
  ## Returns
26
45
 
27
- `string`
46
+ `Promise`\<`string`\>
47
+
48
+ 返回一个 64 位十六进制的 SHA-256 哈希字符串作为缓存键。
49
+
50
+ ## Example
51
+
52
+ ```typescript
53
+ const cacheKey = await generateCacheKey(request, {
54
+ query: { exclude: ['timestamp'] },
55
+ body: { include: ['id', 'action'] }
56
+ });
57
+ ```
@@ -0,0 +1,39 @@
1
+ [**@isdk/proxy**](../README.md)
2
+
3
+ ***
4
+
5
+ [@isdk/proxy](../globals.md) / getSiteConfig
6
+
7
+ # Function: getSiteConfig()
8
+
9
+ > **getSiteConfig**(`urlString`, `proxyConfig`): [`SiteCacheConfig`](../interfaces/SiteCacheConfig.md)
10
+
11
+ Defined in: [utils/getSiteConfig.ts:17](https://github.com/isdk/proxy.js/blob/76fee3a101f98e5bf29599fe7ea02ab06479cf70/src/utils/getSiteConfig.ts#L17)
12
+
13
+ 根据 URL 获取对应的站点缓存配置
14
+
15
+ 匹配逻辑:
16
+ 1. 遍历 sites 中的所有 key。
17
+ 2. 如果 key 是正则或 Glob 格式字符串,则对完整 URL 进行匹配。
18
+ 3. 如果 key 是普通字符串,则作为 URL 前缀进行匹配。
19
+ 4. 返回第一个匹配到的配置;若均未匹配,则返回 defaultConfig。
20
+
21
+ ## Parameters
22
+
23
+ ### urlString
24
+
25
+ `string`
26
+
27
+ 请求的完整 URL
28
+
29
+ ### proxyConfig
30
+
31
+ [`ProxyConfig`](../interfaces/ProxyConfig.md)
32
+
33
+ 全局代理配置
34
+
35
+ ## Returns
36
+
37
+ [`SiteCacheConfig`](../interfaces/SiteCacheConfig.md)
38
+
39
+ 匹配到的站点配置