pi-cursor-sdk 0.1.50 → 0.1.51

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (276) hide show
  1. package/CHANGELOG.md +19 -0
  2. package/README.md +1 -1
  3. package/docs/cursor-live-smoke-checklist.md +4 -4
  4. package/docs/cursor-model-ux-spec.md +5 -5
  5. package/docs/cursor-native-tool-replay.md +2 -2
  6. package/docs/cursor-native-tool-visual-audit.md +1 -1
  7. package/docs/cursor-testing-lessons.md +1 -1
  8. package/package.json +3 -12
  9. package/src/cursor-fallback-models.generated.ts +536 -302
  10. package/src/cursor-provider-errors.ts +29 -0
  11. package/src/cursor-tool-manifest.ts +1 -1
  12. package/node_modules/@connectrpc/connect-node/README.md +0 -141
  13. package/node_modules/@connectrpc/connect-node/dist/cjs/compression.d.ts +0 -15
  14. package/node_modules/@connectrpc/connect-node/dist/cjs/compression.js +0 -89
  15. package/node_modules/@connectrpc/connect-node/dist/cjs/connect-node-adapter.d.ts +0 -42
  16. package/node_modules/@connectrpc/connect-node/dist/cjs/connect-node-adapter.js +0 -61
  17. package/node_modules/@connectrpc/connect-node/dist/cjs/connect-transport.d.ts +0 -100
  18. package/node_modules/@connectrpc/connect-node/dist/cjs/connect-transport.js +0 -25
  19. package/node_modules/@connectrpc/connect-node/dist/cjs/grpc-transport.d.ts +0 -96
  20. package/node_modules/@connectrpc/connect-node/dist/cjs/grpc-transport.js +0 -25
  21. package/node_modules/@connectrpc/connect-node/dist/cjs/grpc-web-transport.d.ts +0 -96
  22. package/node_modules/@connectrpc/connect-node/dist/cjs/grpc-web-transport.js +0 -25
  23. package/node_modules/@connectrpc/connect-node/dist/cjs/http2-session-manager.d.ts +0 -146
  24. package/node_modules/@connectrpc/connect-node/dist/cjs/http2-session-manager.js +0 -536
  25. package/node_modules/@connectrpc/connect-node/dist/cjs/index.d.ts +0 -15
  26. package/node_modules/@connectrpc/connect-node/dist/cjs/index.js +0 -36
  27. package/node_modules/@connectrpc/connect-node/dist/cjs/node-error.d.ts +0 -45
  28. package/node_modules/@connectrpc/connect-node/dist/cjs/node-error.js +0 -154
  29. package/node_modules/@connectrpc/connect-node/dist/cjs/node-headers-polyfill.d.ts +0 -1
  30. package/node_modules/@connectrpc/connect-node/dist/cjs/node-headers-polyfill.js +0 -29
  31. package/node_modules/@connectrpc/connect-node/dist/cjs/node-transport-options.d.ts +0 -89
  32. package/node_modules/@connectrpc/connect-node/dist/cjs/node-transport-options.js +0 -58
  33. package/node_modules/@connectrpc/connect-node/dist/cjs/node-universal-client.d.ts +0 -54
  34. package/node_modules/@connectrpc/connect-node/dist/cjs/node-universal-client.js +0 -326
  35. package/node_modules/@connectrpc/connect-node/dist/cjs/node-universal-handler.d.ts +0 -41
  36. package/node_modules/@connectrpc/connect-node/dist/cjs/node-universal-handler.js +0 -234
  37. package/node_modules/@connectrpc/connect-node/dist/cjs/node-universal-header.d.ts +0 -19
  38. package/node_modules/@connectrpc/connect-node/dist/cjs/node-universal-header.js +0 -97
  39. package/node_modules/@connectrpc/connect-node/dist/cjs/package.json +0 -1
  40. package/node_modules/@connectrpc/connect-node/dist/esm/compression.d.ts +0 -15
  41. package/node_modules/@connectrpc/connect-node/dist/esm/compression.js +0 -86
  42. package/node_modules/@connectrpc/connect-node/dist/esm/connect-node-adapter.d.ts +0 -42
  43. package/node_modules/@connectrpc/connect-node/dist/esm/connect-node-adapter.js +0 -58
  44. package/node_modules/@connectrpc/connect-node/dist/esm/connect-transport.d.ts +0 -100
  45. package/node_modules/@connectrpc/connect-node/dist/esm/connect-transport.js +0 -22
  46. package/node_modules/@connectrpc/connect-node/dist/esm/grpc-transport.d.ts +0 -96
  47. package/node_modules/@connectrpc/connect-node/dist/esm/grpc-transport.js +0 -22
  48. package/node_modules/@connectrpc/connect-node/dist/esm/grpc-web-transport.d.ts +0 -96
  49. package/node_modules/@connectrpc/connect-node/dist/esm/grpc-web-transport.js +0 -22
  50. package/node_modules/@connectrpc/connect-node/dist/esm/http2-session-manager.d.ts +0 -146
  51. package/node_modules/@connectrpc/connect-node/dist/esm/http2-session-manager.js +0 -532
  52. package/node_modules/@connectrpc/connect-node/dist/esm/index.d.ts +0 -15
  53. package/node_modules/@connectrpc/connect-node/dist/esm/index.js +0 -23
  54. package/node_modules/@connectrpc/connect-node/dist/esm/node-error.d.ts +0 -45
  55. package/node_modules/@connectrpc/connect-node/dist/esm/node-error.js +0 -147
  56. package/node_modules/@connectrpc/connect-node/dist/esm/node-headers-polyfill.d.ts +0 -1
  57. package/node_modules/@connectrpc/connect-node/dist/esm/node-headers-polyfill.js +0 -27
  58. package/node_modules/@connectrpc/connect-node/dist/esm/node-transport-options.d.ts +0 -89
  59. package/node_modules/@connectrpc/connect-node/dist/esm/node-transport-options.js +0 -58
  60. package/node_modules/@connectrpc/connect-node/dist/esm/node-universal-client.d.ts +0 -54
  61. package/node_modules/@connectrpc/connect-node/dist/esm/node-universal-client.js +0 -324
  62. package/node_modules/@connectrpc/connect-node/dist/esm/node-universal-handler.d.ts +0 -41
  63. package/node_modules/@connectrpc/connect-node/dist/esm/node-universal-handler.js +0 -230
  64. package/node_modules/@connectrpc/connect-node/dist/esm/node-universal-header.d.ts +0 -19
  65. package/node_modules/@connectrpc/connect-node/dist/esm/node-universal-header.js +0 -93
  66. package/node_modules/@connectrpc/connect-node/package.json +0 -46
  67. package/node_modules/undici/LICENSE +0 -21
  68. package/node_modules/undici/README.md +0 -741
  69. package/node_modules/undici/docs/docs/api/Agent.md +0 -84
  70. package/node_modules/undici/docs/docs/api/BalancedPool.md +0 -99
  71. package/node_modules/undici/docs/docs/api/CacheStorage.md +0 -30
  72. package/node_modules/undici/docs/docs/api/CacheStore.md +0 -164
  73. package/node_modules/undici/docs/docs/api/Client.md +0 -288
  74. package/node_modules/undici/docs/docs/api/ClientStats.md +0 -27
  75. package/node_modules/undici/docs/docs/api/Connector.md +0 -115
  76. package/node_modules/undici/docs/docs/api/ContentType.md +0 -57
  77. package/node_modules/undici/docs/docs/api/Cookies.md +0 -128
  78. package/node_modules/undici/docs/docs/api/Debug.md +0 -62
  79. package/node_modules/undici/docs/docs/api/DiagnosticsChannel.md +0 -315
  80. package/node_modules/undici/docs/docs/api/Dispatcher.md +0 -1392
  81. package/node_modules/undici/docs/docs/api/EnvHttpProxyAgent.md +0 -159
  82. package/node_modules/undici/docs/docs/api/Errors.md +0 -49
  83. package/node_modules/undici/docs/docs/api/EventSource.md +0 -45
  84. package/node_modules/undici/docs/docs/api/Fetch.md +0 -60
  85. package/node_modules/undici/docs/docs/api/GlobalInstallation.md +0 -139
  86. package/node_modules/undici/docs/docs/api/H2CClient.md +0 -263
  87. package/node_modules/undici/docs/docs/api/MockAgent.md +0 -603
  88. package/node_modules/undici/docs/docs/api/MockCallHistory.md +0 -197
  89. package/node_modules/undici/docs/docs/api/MockCallHistoryLog.md +0 -43
  90. package/node_modules/undici/docs/docs/api/MockClient.md +0 -81
  91. package/node_modules/undici/docs/docs/api/MockErrors.md +0 -12
  92. package/node_modules/undici/docs/docs/api/MockPool.md +0 -555
  93. package/node_modules/undici/docs/docs/api/Pool.md +0 -84
  94. package/node_modules/undici/docs/docs/api/PoolStats.md +0 -35
  95. package/node_modules/undici/docs/docs/api/ProxyAgent.md +0 -229
  96. package/node_modules/undici/docs/docs/api/RedirectHandler.md +0 -93
  97. package/node_modules/undici/docs/docs/api/RetryAgent.md +0 -50
  98. package/node_modules/undici/docs/docs/api/RetryHandler.md +0 -118
  99. package/node_modules/undici/docs/docs/api/RoundRobinPool.md +0 -145
  100. package/node_modules/undici/docs/docs/api/SnapshotAgent.md +0 -616
  101. package/node_modules/undici/docs/docs/api/Socks5ProxyAgent.md +0 -275
  102. package/node_modules/undici/docs/docs/api/Util.md +0 -25
  103. package/node_modules/undici/docs/docs/api/WebSocket.md +0 -141
  104. package/node_modules/undici/docs/docs/api/api-lifecycle.md +0 -91
  105. package/node_modules/undici/docs/docs/best-practices/client-certificate.md +0 -64
  106. package/node_modules/undici/docs/docs/best-practices/crawling.md +0 -58
  107. package/node_modules/undici/docs/docs/best-practices/mocking-request.md +0 -190
  108. package/node_modules/undici/docs/docs/best-practices/proxy.md +0 -127
  109. package/node_modules/undici/docs/docs/best-practices/undici-vs-builtin-fetch.md +0 -224
  110. package/node_modules/undici/docs/docs/best-practices/writing-tests.md +0 -63
  111. package/node_modules/undici/index-fetch.js +0 -65
  112. package/node_modules/undici/index.d.ts +0 -3
  113. package/node_modules/undici/index.js +0 -234
  114. package/node_modules/undici/lib/api/abort-signal.js +0 -59
  115. package/node_modules/undici/lib/api/api-connect.js +0 -110
  116. package/node_modules/undici/lib/api/api-pipeline.js +0 -252
  117. package/node_modules/undici/lib/api/api-request.js +0 -214
  118. package/node_modules/undici/lib/api/api-stream.js +0 -209
  119. package/node_modules/undici/lib/api/api-upgrade.js +0 -111
  120. package/node_modules/undici/lib/api/index.js +0 -7
  121. package/node_modules/undici/lib/api/readable.js +0 -580
  122. package/node_modules/undici/lib/cache/memory-cache-store.js +0 -234
  123. package/node_modules/undici/lib/cache/sqlite-cache-store.js +0 -461
  124. package/node_modules/undici/lib/core/connect.js +0 -153
  125. package/node_modules/undici/lib/core/constants.js +0 -143
  126. package/node_modules/undici/lib/core/diagnostics.js +0 -227
  127. package/node_modules/undici/lib/core/errors.js +0 -477
  128. package/node_modules/undici/lib/core/request.js +0 -453
  129. package/node_modules/undici/lib/core/socks5-client.js +0 -422
  130. package/node_modules/undici/lib/core/socks5-utils.js +0 -212
  131. package/node_modules/undici/lib/core/symbols.js +0 -75
  132. package/node_modules/undici/lib/core/tree.js +0 -160
  133. package/node_modules/undici/lib/core/util.js +0 -992
  134. package/node_modules/undici/lib/dispatcher/agent.js +0 -158
  135. package/node_modules/undici/lib/dispatcher/balanced-pool.js +0 -219
  136. package/node_modules/undici/lib/dispatcher/client-h1.js +0 -1722
  137. package/node_modules/undici/lib/dispatcher/client-h2.js +0 -995
  138. package/node_modules/undici/lib/dispatcher/client.js +0 -664
  139. package/node_modules/undici/lib/dispatcher/dispatcher-base.js +0 -184
  140. package/node_modules/undici/lib/dispatcher/dispatcher.js +0 -48
  141. package/node_modules/undici/lib/dispatcher/env-http-proxy-agent.js +0 -146
  142. package/node_modules/undici/lib/dispatcher/fixed-queue.js +0 -135
  143. package/node_modules/undici/lib/dispatcher/h2c-client.js +0 -51
  144. package/node_modules/undici/lib/dispatcher/pool-base.js +0 -214
  145. package/node_modules/undici/lib/dispatcher/pool.js +0 -118
  146. package/node_modules/undici/lib/dispatcher/proxy-agent.js +0 -319
  147. package/node_modules/undici/lib/dispatcher/retry-agent.js +0 -35
  148. package/node_modules/undici/lib/dispatcher/round-robin-pool.js +0 -137
  149. package/node_modules/undici/lib/dispatcher/socks5-proxy-agent.js +0 -260
  150. package/node_modules/undici/lib/encoding/index.js +0 -33
  151. package/node_modules/undici/lib/global.js +0 -59
  152. package/node_modules/undici/lib/handler/cache-handler.js +0 -578
  153. package/node_modules/undici/lib/handler/cache-revalidation-handler.js +0 -124
  154. package/node_modules/undici/lib/handler/decorator-handler.js +0 -67
  155. package/node_modules/undici/lib/handler/deduplication-handler.js +0 -460
  156. package/node_modules/undici/lib/handler/redirect-handler.js +0 -238
  157. package/node_modules/undici/lib/handler/retry-handler.js +0 -394
  158. package/node_modules/undici/lib/handler/unwrap-handler.js +0 -106
  159. package/node_modules/undici/lib/handler/wrap-handler.js +0 -105
  160. package/node_modules/undici/lib/interceptor/cache.js +0 -495
  161. package/node_modules/undici/lib/interceptor/decompress.js +0 -259
  162. package/node_modules/undici/lib/interceptor/deduplicate.js +0 -117
  163. package/node_modules/undici/lib/interceptor/dns.js +0 -571
  164. package/node_modules/undici/lib/interceptor/dump.js +0 -112
  165. package/node_modules/undici/lib/interceptor/redirect.js +0 -21
  166. package/node_modules/undici/lib/interceptor/response-error.js +0 -95
  167. package/node_modules/undici/lib/interceptor/retry.js +0 -19
  168. package/node_modules/undici/lib/llhttp/.gitkeep +0 -0
  169. package/node_modules/undici/lib/llhttp/constants.d.ts +0 -195
  170. package/node_modules/undici/lib/llhttp/constants.js +0 -531
  171. package/node_modules/undici/lib/llhttp/llhttp-wasm.js +0 -15
  172. package/node_modules/undici/lib/llhttp/llhttp_simd-wasm.js +0 -15
  173. package/node_modules/undici/lib/llhttp/utils.d.ts +0 -2
  174. package/node_modules/undici/lib/llhttp/utils.js +0 -12
  175. package/node_modules/undici/lib/mock/mock-agent.js +0 -232
  176. package/node_modules/undici/lib/mock/mock-call-history.js +0 -248
  177. package/node_modules/undici/lib/mock/mock-client.js +0 -68
  178. package/node_modules/undici/lib/mock/mock-errors.js +0 -29
  179. package/node_modules/undici/lib/mock/mock-interceptor.js +0 -209
  180. package/node_modules/undici/lib/mock/mock-pool.js +0 -68
  181. package/node_modules/undici/lib/mock/mock-symbols.js +0 -32
  182. package/node_modules/undici/lib/mock/mock-utils.js +0 -486
  183. package/node_modules/undici/lib/mock/pending-interceptors-formatter.js +0 -43
  184. package/node_modules/undici/lib/mock/snapshot-agent.js +0 -353
  185. package/node_modules/undici/lib/mock/snapshot-recorder.js +0 -588
  186. package/node_modules/undici/lib/mock/snapshot-utils.js +0 -158
  187. package/node_modules/undici/lib/util/cache.js +0 -414
  188. package/node_modules/undici/lib/util/date.js +0 -653
  189. package/node_modules/undici/lib/util/promise.js +0 -28
  190. package/node_modules/undici/lib/util/runtime-features.js +0 -124
  191. package/node_modules/undici/lib/util/stats.js +0 -32
  192. package/node_modules/undici/lib/util/timers.js +0 -425
  193. package/node_modules/undici/lib/web/cache/cache.js +0 -864
  194. package/node_modules/undici/lib/web/cache/cachestorage.js +0 -152
  195. package/node_modules/undici/lib/web/cache/util.js +0 -45
  196. package/node_modules/undici/lib/web/cookies/constants.js +0 -12
  197. package/node_modules/undici/lib/web/cookies/index.js +0 -199
  198. package/node_modules/undici/lib/web/cookies/parse.js +0 -314
  199. package/node_modules/undici/lib/web/cookies/util.js +0 -282
  200. package/node_modules/undici/lib/web/eventsource/eventsource-stream.js +0 -399
  201. package/node_modules/undici/lib/web/eventsource/eventsource.js +0 -501
  202. package/node_modules/undici/lib/web/eventsource/util.js +0 -29
  203. package/node_modules/undici/lib/web/fetch/LICENSE +0 -21
  204. package/node_modules/undici/lib/web/fetch/body.js +0 -509
  205. package/node_modules/undici/lib/web/fetch/constants.js +0 -131
  206. package/node_modules/undici/lib/web/fetch/data-url.js +0 -596
  207. package/node_modules/undici/lib/web/fetch/formdata-parser.js +0 -586
  208. package/node_modules/undici/lib/web/fetch/formdata.js +0 -259
  209. package/node_modules/undici/lib/web/fetch/global.js +0 -40
  210. package/node_modules/undici/lib/web/fetch/headers.js +0 -719
  211. package/node_modules/undici/lib/web/fetch/index.js +0 -2413
  212. package/node_modules/undici/lib/web/fetch/request.js +0 -1115
  213. package/node_modules/undici/lib/web/fetch/response.js +0 -641
  214. package/node_modules/undici/lib/web/fetch/util.js +0 -1522
  215. package/node_modules/undici/lib/web/infra/index.js +0 -229
  216. package/node_modules/undici/lib/web/subresource-integrity/Readme.md +0 -9
  217. package/node_modules/undici/lib/web/subresource-integrity/subresource-integrity.js +0 -307
  218. package/node_modules/undici/lib/web/webidl/index.js +0 -1006
  219. package/node_modules/undici/lib/web/websocket/connection.js +0 -329
  220. package/node_modules/undici/lib/web/websocket/constants.js +0 -126
  221. package/node_modules/undici/lib/web/websocket/events.js +0 -331
  222. package/node_modules/undici/lib/web/websocket/frame.js +0 -133
  223. package/node_modules/undici/lib/web/websocket/permessage-deflate.js +0 -100
  224. package/node_modules/undici/lib/web/websocket/receiver.js +0 -507
  225. package/node_modules/undici/lib/web/websocket/sender.js +0 -109
  226. package/node_modules/undici/lib/web/websocket/stream/websocketerror.js +0 -104
  227. package/node_modules/undici/lib/web/websocket/stream/websocketstream.js +0 -498
  228. package/node_modules/undici/lib/web/websocket/util.js +0 -347
  229. package/node_modules/undici/lib/web/websocket/websocket.js +0 -758
  230. package/node_modules/undici/package.json +0 -152
  231. package/node_modules/undici/scripts/strip-comments.js +0 -10
  232. package/node_modules/undici/types/README.md +0 -6
  233. package/node_modules/undici/types/agent.d.ts +0 -32
  234. package/node_modules/undici/types/api.d.ts +0 -43
  235. package/node_modules/undici/types/balanced-pool.d.ts +0 -30
  236. package/node_modules/undici/types/cache-interceptor.d.ts +0 -179
  237. package/node_modules/undici/types/cache.d.ts +0 -36
  238. package/node_modules/undici/types/client-stats.d.ts +0 -15
  239. package/node_modules/undici/types/client.d.ts +0 -139
  240. package/node_modules/undici/types/connector.d.ts +0 -36
  241. package/node_modules/undici/types/content-type.d.ts +0 -21
  242. package/node_modules/undici/types/cookies.d.ts +0 -30
  243. package/node_modules/undici/types/diagnostics-channel.d.ts +0 -74
  244. package/node_modules/undici/types/dispatcher.d.ts +0 -275
  245. package/node_modules/undici/types/env-http-proxy-agent.d.ts +0 -22
  246. package/node_modules/undici/types/errors.d.ts +0 -177
  247. package/node_modules/undici/types/eventsource.d.ts +0 -66
  248. package/node_modules/undici/types/fetch.d.ts +0 -231
  249. package/node_modules/undici/types/formdata.d.ts +0 -114
  250. package/node_modules/undici/types/global-dispatcher.d.ts +0 -9
  251. package/node_modules/undici/types/global-origin.d.ts +0 -7
  252. package/node_modules/undici/types/h2c-client.d.ts +0 -73
  253. package/node_modules/undici/types/handlers.d.ts +0 -14
  254. package/node_modules/undici/types/header.d.ts +0 -160
  255. package/node_modules/undici/types/index.d.ts +0 -91
  256. package/node_modules/undici/types/interceptors.d.ts +0 -80
  257. package/node_modules/undici/types/mock-agent.d.ts +0 -68
  258. package/node_modules/undici/types/mock-call-history.d.ts +0 -111
  259. package/node_modules/undici/types/mock-client.d.ts +0 -27
  260. package/node_modules/undici/types/mock-errors.d.ts +0 -12
  261. package/node_modules/undici/types/mock-interceptor.d.ts +0 -94
  262. package/node_modules/undici/types/mock-pool.d.ts +0 -27
  263. package/node_modules/undici/types/patch.d.ts +0 -29
  264. package/node_modules/undici/types/pool-stats.d.ts +0 -19
  265. package/node_modules/undici/types/pool.d.ts +0 -41
  266. package/node_modules/undici/types/proxy-agent.d.ts +0 -29
  267. package/node_modules/undici/types/readable.d.ts +0 -68
  268. package/node_modules/undici/types/retry-agent.d.ts +0 -8
  269. package/node_modules/undici/types/retry-handler.d.ts +0 -125
  270. package/node_modules/undici/types/round-robin-pool.d.ts +0 -41
  271. package/node_modules/undici/types/snapshot-agent.d.ts +0 -109
  272. package/node_modules/undici/types/socks5-proxy-agent.d.ts +0 -25
  273. package/node_modules/undici/types/util.d.ts +0 -18
  274. package/node_modules/undici/types/utility.d.ts +0 -7
  275. package/node_modules/undici/types/webidl.d.ts +0 -347
  276. package/node_modules/undici/types/websocket.d.ts +0 -188
@@ -1,275 +0,0 @@
1
- # Class: Socks5ProxyAgent
2
-
3
- Extends: `undici.Dispatcher`
4
-
5
- A SOCKS5 proxy wrapper class that implements the Dispatcher API. It enables HTTP requests to be routed through a SOCKS5 proxy server, providing connection tunneling and authentication support.
6
-
7
- ## `new Socks5ProxyAgent(proxyUrl[, options])`
8
-
9
- Arguments:
10
-
11
- * **proxyUrl** `string | URL` (required) - The SOCKS5 proxy server URL. Must use `socks5://` or `socks://` protocol.
12
- * **options** `Socks5ProxyAgent.Options` (optional) - Additional configuration options.
13
-
14
- Returns: `Socks5ProxyAgent`
15
-
16
- ### Parameter: `Socks5ProxyAgent.Options`
17
-
18
- Extends: [`PoolOptions`](/docs/docs/api/Pool.md#parameter-pooloptions)
19
-
20
- * **headers** `IncomingHttpHeaders` (optional) - Additional headers to send with proxy connections.
21
- * **username** `string` (optional) - SOCKS5 proxy username for authentication. Can also be provided in the proxy URL.
22
- * **password** `string` (optional) - SOCKS5 proxy password for authentication. Can also be provided in the proxy URL.
23
- * **connect** `Function` (optional) - Custom connector function for the proxy connection.
24
- * **proxyTls** `BuildOptions` (optional) - TLS options for the proxy connection (when using SOCKS5 over TLS).
25
- * **requestTls** `BuildOptions` (optional) - TLS options applied to the HTTPS connection to the target server through the SOCKS5 tunnel. Use this to configure `ca`, `cert`, `key`, `rejectUnauthorized`, `servername`, etc. for the target HTTPS endpoint.
26
-
27
- Examples:
28
-
29
- ```js
30
- import { Socks5ProxyAgent } from 'undici'
31
-
32
- const socks5Proxy = new Socks5ProxyAgent('socks5://localhost:1080')
33
- // or with authentication
34
- const socks5ProxyWithAuth = new Socks5ProxyAgent('socks5://user:pass@localhost:1080')
35
- // or with options
36
- const socks5ProxyWithOptions = new Socks5ProxyAgent('socks5://localhost:1080', {
37
- username: 'user',
38
- password: 'pass',
39
- connections: 10
40
- })
41
- ```
42
-
43
- #### Example - Basic SOCKS5 Proxy instantiation
44
-
45
- This will instantiate the Socks5ProxyAgent. It will not do anything until registered as the dispatcher to use with requests.
46
-
47
- ```js
48
- import { Socks5ProxyAgent } from 'undici'
49
-
50
- const socks5Proxy = new Socks5ProxyAgent('socks5://localhost:1080')
51
- ```
52
-
53
- #### Example - Basic SOCKS5 Proxy Request with global dispatcher
54
-
55
- ```js
56
- import { setGlobalDispatcher, request, Socks5ProxyAgent } from 'undici'
57
-
58
- const socks5Proxy = new Socks5ProxyAgent('socks5://localhost:1080')
59
- setGlobalDispatcher(socks5Proxy)
60
-
61
- const { statusCode, body } = await request('http://localhost:3000/foo')
62
-
63
- console.log('response received', statusCode) // response received 200
64
-
65
- for await (const data of body) {
66
- console.log('data', data.toString('utf8')) // data foo
67
- }
68
- ```
69
-
70
- #### Example - Basic SOCKS5 Proxy Request with local dispatcher
71
-
72
- ```js
73
- import { Socks5ProxyAgent, request } from 'undici'
74
-
75
- const socks5Proxy = new Socks5ProxyAgent('socks5://localhost:1080')
76
-
77
- const {
78
- statusCode,
79
- body
80
- } = await request('http://localhost:3000/foo', { dispatcher: socks5Proxy })
81
-
82
- console.log('response received', statusCode) // response received 200
83
-
84
- for await (const data of body) {
85
- console.log('data', data.toString('utf8')) // data foo
86
- }
87
- ```
88
-
89
- #### Example - SOCKS5 Proxy Request with authentication
90
-
91
- ```js
92
- import { setGlobalDispatcher, request, Socks5ProxyAgent } from 'undici'
93
-
94
- // Authentication via URL
95
- const socks5Proxy = new Socks5ProxyAgent('socks5://username:password@localhost:1080')
96
-
97
- // Or authentication via options
98
- // const socks5Proxy = new Socks5ProxyAgent('socks5://localhost:1080', {
99
- // username: 'username',
100
- // password: 'password'
101
- // })
102
-
103
- setGlobalDispatcher(socks5Proxy)
104
-
105
- const { statusCode, body } = await request('http://localhost:3000/foo')
106
-
107
- console.log('response received', statusCode) // response received 200
108
-
109
- for await (const data of body) {
110
- console.log('data', data.toString('utf8')) // data foo
111
- }
112
- ```
113
-
114
- #### Example - SOCKS5 Proxy with HTTPS requests
115
-
116
- SOCKS5 proxy supports both HTTP and HTTPS requests through tunneling:
117
-
118
- ```js
119
- import { Socks5ProxyAgent, request } from 'undici'
120
-
121
- const socks5Proxy = new Socks5ProxyAgent('socks5://localhost:1080')
122
-
123
- const response = await request('https://api.example.com/data', {
124
- dispatcher: socks5Proxy,
125
- method: 'GET'
126
- })
127
-
128
- console.log('Response status:', response.statusCode)
129
- console.log('Response data:', await response.body.json())
130
- ```
131
-
132
- #### Example - SOCKS5 Proxy with Fetch
133
-
134
- ```js
135
- import { Socks5ProxyAgent, fetch } from 'undici'
136
-
137
- const socks5Proxy = new Socks5ProxyAgent('socks5://localhost:1080')
138
-
139
- const response = await fetch('http://localhost:3000/api/users', {
140
- dispatcher: socks5Proxy,
141
- method: 'GET'
142
- })
143
-
144
- console.log('Response status:', response.status)
145
- console.log('Response data:', await response.text())
146
- ```
147
-
148
- #### Example - Connection Pooling
149
-
150
- SOCKS5ProxyWrapper automatically manages connection pooling for better performance:
151
-
152
- ```js
153
- import { Socks5ProxyAgent, request } from 'undici'
154
-
155
- const socks5Proxy = new Socks5ProxyAgent('socks5://localhost:1080', {
156
- connections: 10, // Allow up to 10 concurrent connections
157
- pipelining: 1 // Enable HTTP/1.1 pipelining
158
- })
159
-
160
- // Multiple requests will reuse connections through the SOCKS5 tunnel
161
- const responses = await Promise.all([
162
- request('http://api.example.com/endpoint1', { dispatcher: socks5Proxy }),
163
- request('http://api.example.com/endpoint2', { dispatcher: socks5Proxy }),
164
- request('http://api.example.com/endpoint3', { dispatcher: socks5Proxy })
165
- ])
166
-
167
- console.log('All requests completed through the same SOCKS5 proxy')
168
- ```
169
-
170
- ### `Socks5ProxyAgent.close()`
171
-
172
- Closes the SOCKS5 proxy wrapper and waits for all underlying pools and connections to close before resolving.
173
-
174
- Returns: `Promise<void>`
175
-
176
- #### Example - clean up after tests are complete
177
-
178
- ```js
179
- import { Socks5ProxyAgent, setGlobalDispatcher } from 'undici'
180
-
181
- const socks5Proxy = new Socks5ProxyAgent('socks5://localhost:1080')
182
- setGlobalDispatcher(socks5Proxy)
183
-
184
- // ... make requests
185
-
186
- await socks5Proxy.close()
187
- ```
188
-
189
- ### `Socks5ProxyAgent.destroy([err])`
190
-
191
- Destroys the SOCKS5 proxy wrapper and all underlying connections immediately.
192
-
193
- Arguments:
194
- * **err** `Error` (optional) - The error that caused the destruction.
195
-
196
- Returns: `Promise<void>`
197
-
198
- #### Example - force close all connections
199
-
200
- ```js
201
- import { Socks5ProxyAgent } from 'undici'
202
-
203
- const socks5Proxy = new Socks5ProxyAgent('socks5://localhost:1080')
204
-
205
- // Force close all connections
206
- await socks5Proxy.destroy()
207
- ```
208
-
209
- ### `Socks5ProxyAgent.dispatch(options, handlers)`
210
-
211
- Implements [`Dispatcher.dispatch(options, handlers)`](/docs/docs/api/Dispatcher.md#dispatcherdispatchoptions-handlers).
212
-
213
- ### `Socks5ProxyAgent.request(options[, callback])`
214
-
215
- See [`Dispatcher.request(options [, callback])`](/docs/docs/api/Dispatcher.md#dispatcherrequestoptions-callback).
216
-
217
- ## Debugging
218
-
219
- SOCKS5 proxy connections can be debugged using Node.js diagnostics:
220
-
221
- ```sh
222
- NODE_DEBUG=undici:socks5 node script.js
223
- ```
224
-
225
- This will output detailed information about the SOCKS5 handshake, authentication, and connection establishment.
226
-
227
- ## SOCKS5 Protocol Support
228
-
229
- The Socks5ProxyAgent supports the following SOCKS5 features:
230
-
231
- ### Authentication Methods
232
-
233
- - **No Authentication** (`0x00`) - For public or internal proxies
234
- - **Username/Password** (`0x02`) - RFC 1929 authentication
235
-
236
- ### Address Types
237
-
238
- - **IPv4** (`0x01`) - Standard IPv4 addresses
239
- - **Domain Name** (`0x03`) - Domain names (recommended for flexibility)
240
- - **IPv6** (`0x04`) - IPv6 addresses (full support for standard and compressed notation)
241
-
242
- ### Commands
243
-
244
- - **CONNECT** (`0x01`) - Establish TCP connection (primary use case for HTTP)
245
-
246
- ### Error Handling
247
-
248
- The wrapper handles various SOCKS5 error conditions:
249
-
250
- - Connection refused by proxy
251
- - Authentication failures
252
- - Network unreachable
253
- - Host unreachable
254
- - Unsupported address types or commands
255
-
256
- ## Performance Considerations
257
-
258
- - **Connection Pooling**: Automatically pools connections through the SOCKS5 tunnel for better performance
259
- - **HTTP/1.1 Pipelining**: Supports pipelining when enabled
260
- - **DNS Resolution**: Domain names are resolved by the SOCKS5 proxy, reducing local DNS queries
261
- - **TLS Termination**: HTTPS connections are encrypted end-to-end, with the SOCKS5 proxy only handling the TCP tunnel
262
-
263
- ## Security Notes
264
-
265
- 1. **Authentication**: Credentials are sent to the SOCKS5 proxy in plaintext unless using SOCKS5 over TLS
266
- 2. **DNS Leaks**: All DNS resolution happens on the proxy server, preventing DNS leaks
267
- 3. **End-to-end Encryption**: HTTPS traffic remains encrypted between client and final destination
268
- 4. **Connection Security**: Consider using authenticated proxies and secure networks
269
-
270
- ## Compatibility
271
-
272
- - **Protocol**: SOCKS5 (RFC 1928) with Username/Password Authentication (RFC 1929)
273
- - **Transport**: TCP only (UDP support not implemented)
274
- - **Node.js**: Compatible with all supported Node.js versions
275
- - **HTTP Versions**: Works with HTTP/1.1 and HTTP/2 over the tunnel
@@ -1,25 +0,0 @@
1
- # Util
2
-
3
- Utility API for third-party implementations of the dispatcher API.
4
-
5
- ## `parseHeaders(headers, [obj])`
6
-
7
- Receives a header object and returns the parsed value.
8
-
9
- Arguments:
10
-
11
- - **headers** `(Buffer | string | (Buffer | string)[])[]` (required) - Header object.
12
-
13
- - **obj** `Record<string, string | string[]>` (optional) - Object to specify a proxy object. The parsed value is assigned to this object. But, if **headers** is an object, it is not used.
14
-
15
- Returns: `Record<string, string | string[]>` If **obj** is specified, it is equivalent to **obj**.
16
-
17
- ## `headerNameToString(value)`
18
-
19
- Retrieves a header name and returns its lowercase value.
20
-
21
- Arguments:
22
-
23
- - **value** `string | Buffer` (required) - Header name.
24
-
25
- Returns: `string`
@@ -1,141 +0,0 @@
1
- # Class: WebSocket
2
-
3
- Extends: [`EventTarget`](https://developer.mozilla.org/en-US/docs/Web/API/EventTarget)
4
-
5
- The WebSocket object provides a way to manage a WebSocket connection to a server, allowing bidirectional communication. The API follows the [WebSocket spec](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket) and [RFC 6455](https://datatracker.ietf.org/doc/html/rfc6455).
6
-
7
- ## `new WebSocket(url[, protocol])`
8
-
9
- Arguments:
10
-
11
- * **url** `URL | string`
12
- * **protocol** `string | string[] | WebSocketInit` (optional) - Subprotocol(s) to request the server use, or a [`Dispatcher`](/docs/docs/api/Dispatcher.md).
13
-
14
- ### WebSocketInit
15
-
16
- When passing an object as the second argument, the following options are available:
17
-
18
- * **protocols** `string | string[]` (optional) - Subprotocol(s) to request the server use.
19
- * **dispatcher** `Dispatcher` (optional) - A custom [`Dispatcher`](/docs/docs/api/Dispatcher.md) to use for the connection.
20
- * **headers** `HeadersInit` (optional) - Custom headers to include in the WebSocket handshake request.
21
-
22
- ### Example:
23
-
24
- This example will not work in browsers or other platforms that don't allow passing an object.
25
-
26
- ```js
27
- import { WebSocket, ProxyAgent } from 'undici'
28
-
29
- const proxyAgent = new ProxyAgent('my.proxy.server')
30
-
31
- const ws = new WebSocket('wss://echo.websocket.events', {
32
- dispatcher: proxyAgent,
33
- protocols: ['echo', 'chat']
34
- })
35
- ```
36
-
37
- If you do not need a custom Dispatcher, it's recommended to use the following pattern:
38
-
39
- ```js
40
- import { WebSocket } from 'undici'
41
-
42
- const ws = new WebSocket('wss://echo.websocket.events', ['echo', 'chat'])
43
- ```
44
-
45
- ### Example with HTTP/2:
46
-
47
- > ⚠️ Warning: WebSocket over HTTP/2 is experimental, it is likely to change in the future.
48
-
49
- > 🗒️ Note: WebSocket over HTTP/2 may be enabled by default in a future version,
50
- > this will happen by enabling HTTP/2 connections as the default behavior of Undici's Agent as well the global dispatcher.
51
- > Stay tuned to the changelog for more information.
52
-
53
- This example will not work in browsers or other platforms that don't allow passing an object.
54
-
55
- ```js
56
- import { Agent } from 'undici'
57
-
58
- const agent = new Agent({ allowH2: true })
59
-
60
- const ws = new WebSocket('wss://echo.websocket.events', {
61
- dispatcher: agent,
62
- protocols: ['echo', 'chat']
63
- })
64
- ```
65
-
66
- # Class: WebSocketStream
67
-
68
- > ⚠️ Warning: the WebSocketStream API has not been finalized and is likely to change.
69
-
70
- See [MDN](https://developer.mozilla.org/en-US/docs/Web/API/WebSocketStream) for more information.
71
-
72
- ## `new WebSocketStream(url[, protocol])`
73
-
74
- Arguments:
75
-
76
- * **url** `URL | string`
77
- * **options** `WebSocketStreamOptions` (optional)
78
-
79
- ### WebSocketStream Example
80
-
81
- ```js
82
- const stream = new WebSocketStream('https://echo.websocket.org/')
83
- const { readable, writable } = await stream.opened
84
-
85
- async function read () {
86
- /** @type {ReadableStreamReader} */
87
- const reader = readable.getReader()
88
-
89
- while (true) {
90
- const { done, value } = await reader.read()
91
- if (done) break
92
-
93
- // do something with value
94
- }
95
- }
96
-
97
- async function write () {
98
- /** @type {WritableStreamDefaultWriter} */
99
- const writer = writable.getWriter()
100
- writer.write('Hello, world!')
101
- writer.releaseLock()
102
- }
103
-
104
- read()
105
-
106
- setInterval(() => write(), 5000)
107
-
108
- ```
109
-
110
- ## ping(websocket, payload)
111
- Arguments:
112
-
113
- * **websocket** `WebSocket` - The WebSocket instance to send the ping frame on
114
- * **payload** `Buffer|undefined` (optional) - Optional payload data to include with the ping frame. Must not exceed 125 bytes.
115
-
116
- Sends a ping frame to the WebSocket server. The server must respond with a pong frame containing the same payload data. This can be used for keepalive purposes or to verify that the connection is still active.
117
-
118
- ### Example:
119
-
120
- ```js
121
- import { WebSocket, ping } from 'undici'
122
-
123
- const ws = new WebSocket('wss://echo.websocket.events')
124
-
125
- ws.addEventListener('open', () => {
126
- // Send ping with no payload
127
- ping(ws)
128
-
129
- // Send ping with payload
130
- const payload = Buffer.from('hello')
131
- ping(ws, payload)
132
- })
133
- ```
134
-
135
- **Note**: A ping frame cannot have a payload larger than 125 bytes. The ping will only be sent if the WebSocket connection is in the OPEN state.
136
-
137
- ## Read More
138
-
139
- - [MDN - WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSocket)
140
- - [The WebSocket Specification](https://www.rfc-editor.org/rfc/rfc6455)
141
- - [The WHATWG WebSocket Specification](https://websockets.spec.whatwg.org/)
@@ -1,91 +0,0 @@
1
- # Client Lifecycle
2
-
3
- An Undici [Client](/docs/docs/api/Client.md) can be best described as a state machine. The following list is a summary of the various state transitions the `Client` will go through in its lifecycle. This document also contains detailed breakdowns of each state.
4
-
5
- > This diagram is not a perfect representation of the undici Client. Since the Client class is not actually implemented as a state-machine, actual execution may deviate slightly from what is described below. Consider this as a general resource for understanding the inner workings of the Undici client rather than some kind of formal specification.
6
-
7
- ## State Transition Overview
8
-
9
- * A `Client` begins in the **idle** state with no socket connection and no requests in queue.
10
- * The *connect* event transitions the `Client` to the **pending** state where requests can be queued prior to processing.
11
- * The *close* and *destroy* events transition the `Client` to the **destroyed** state. Since there are no requests in the queue, the *close* event immediately transitions to the **destroyed** state.
12
- * The **pending** state indicates the underlying socket connection has been successfully established and requests are queueing.
13
- * The *process* event transitions the `Client` to the **processing** state where requests are processed.
14
- * If requests are queued, the *close* event transitions to the **processing** state; otherwise, it transitions to the **destroyed** state.
15
- * The *destroy* event transitions to the **destroyed** state.
16
- * The **processing** state initializes to the **processing.running** state.
17
- * If the current request requires draining, the *needDrain* event transitions the `Client` into the **processing.busy** state which will return to the **processing.running** state with the *drainComplete* event.
18
- * After all queued requests are completed, the *keepalive* event transitions the `Client` back to the **pending** state. If no requests are queued during the timeout, the **close** event transitions the `Client` to the **destroyed** state.
19
- * If the *close* event is fired while the `Client` still has queued requests, the `Client` transitions to the **process.closing** state where it will complete all existing requests before firing the *done* event.
20
- * The *done* event gracefully transitions the `Client` to the **destroyed** state.
21
- * At any point in time, the *destroy* event will transition the `Client` from the **processing** state to the **destroyed** state, destroying any queued requests.
22
- * The **destroyed** state is a final state and the `Client` is no longer functional.
23
-
24
- A state diagram representing an Undici Client instance:
25
-
26
- ```mermaid
27
- stateDiagram-v2
28
- [*] --> idle
29
- idle --> pending : connect
30
- idle --> destroyed : destroy/close
31
-
32
- pending --> idle : timeout
33
- pending --> destroyed : destroy
34
-
35
- state close_fork <<fork>>
36
- pending --> close_fork : close
37
- close_fork --> processing
38
- close_fork --> destroyed
39
-
40
- pending --> processing : process
41
-
42
- processing --> pending : keepalive
43
- processing --> destroyed : done
44
- processing --> destroyed : destroy
45
-
46
- destroyed --> [*]
47
-
48
- state processing {
49
- [*] --> running
50
- running --> closing : close
51
- running --> busy : needDrain
52
- busy --> running : drainComplete
53
- running --> [*] : keepalive
54
- closing --> [*] : done
55
- }
56
- ```
57
- ## State details
58
-
59
- ### idle
60
-
61
- The **idle** state is the initial state of a `Client` instance. While an `origin` is required for instantiating a `Client` instance, the underlying socket connection will not be established until a request is queued using [`Client.dispatch()`](/docs/docs/api/Client.md#clientdispatchoptions-handlers). By calling `Client.dispatch()` directly or using one of the multiple implementations ([`Client.connect()`](Client.md#clientconnectoptions-callback), [`Client.pipeline()`](Client.md#clientpipelineoptions-handler), [`Client.request()`](Client.md#clientrequestoptions-callback), [`Client.stream()`](Client.md#clientstreamoptions-factory-callback), and [`Client.upgrade()`](/docs/docs/api/Client.md#clientupgradeoptions-callback)), the `Client` instance will transition from **idle** to [**pending**](/docs/docs/api/Client.md#pending) and then most likely directly to [**processing**](/docs/docs/api/Client.md#processing).
62
-
63
- Calling [`Client.close()`](/docs/docs/api/Client.md#clientclosecallback) or [`Client.destroy()`](Client.md#clientdestroyerror-callback) transitions directly to the [**destroyed**](/docs/docs/api/Client.md#destroyed) state since the `Client` instance will have no queued requests in this state.
64
-
65
- ### pending
66
-
67
- The **pending** state signifies a non-processing `Client`. Upon entering this state, the `Client` establishes a socket connection and emits the [`'connect'`](/docs/docs/api/Client.md#event-connect) event signalling a connection was successfully established with the `origin` provided during `Client` instantiation. The internal queue is initially empty, and requests can start queueing.
68
-
69
- Calling [`Client.close()`](/docs/docs/api/Client.md#clientclosecallback) with queued requests, transitions the `Client` to the [**processing**](/docs/docs/api/Client.md#processing) state. Without queued requests, it transitions to the [**destroyed**](/docs/docs/api/Client.md#destroyed) state.
70
-
71
- Calling [`Client.destroy()`](/docs/docs/api/Client.md#clientdestroyerror-callback) transitions directly to the [**destroyed**](/docs/docs/api/Client.md#destroyed) state regardless of existing requests.
72
-
73
- ### processing
74
-
75
- The **processing** state is a state machine within itself. It initializes to the [**processing.running**](/docs/docs/api/Client.md#running) state. The [`Client.dispatch()`](/docs/docs/api/Client.md#clientdispatchoptions-handlers), [`Client.close()`](Client.md#clientclosecallback), and [`Client.destroy()`](Client.md#clientdestroyerror-callback) can be called at any time while the `Client` is in this state. `Client.dispatch()` will add more requests to the queue while existing requests continue to be processed. `Client.close()` will transition to the [**processing.closing**](/docs/docs/api/Client.md#closing) state. And `Client.destroy()` will transition to [**destroyed**](/docs/docs/api/Client.md#destroyed).
76
-
77
- #### running
78
-
79
- In the **processing.running** sub-state, queued requests are being processed in a FIFO order. If a request body requires draining, the *needDrain* event transitions to the [**processing.busy**](/docs/docs/api/Client.md#busy) sub-state. The *close* event transitions the Client to the [**process.closing**](/docs/docs/api/Client.md#closing) sub-state. If all queued requests are processed and neither [`Client.close()`](/docs/docs/api/Client.md#clientclosecallback) nor [`Client.destroy()`](Client.md#clientdestroyerror-callback) are called, then the [**processing**](/docs/docs/api/Client.md#processing) machine will trigger a *keepalive* event transitioning the `Client` back to the [**pending**](/docs/docs/api/Client.md#pending) state. During this time, the `Client` is waiting for the socket connection to timeout, and once it does, it triggers the *timeout* event and transitions to the [**idle**](/docs/docs/api/Client.md#idle) state.
80
-
81
- #### busy
82
-
83
- This sub-state is only entered when a request body is an instance of [Stream](https://nodejs.org/api/stream.html) and requires draining. The `Client` cannot process additional requests while in this state and must wait until the currently processing request body is completely drained before transitioning back to [**processing.running**](/docs/docs/api/Client.md#running).
84
-
85
- #### closing
86
-
87
- This sub-state is only entered when a `Client` instance has queued requests and the [`Client.close()`](/docs/docs/api/Client.md#clientclosecallback) method is called. In this state, the `Client` instance continues to process requests as usual, with the one exception that no additional requests can be queued. Once all of the queued requests are processed, the `Client` will trigger the *done* event gracefully entering the [**destroyed**](/docs/docs/api/Client.md#destroyed) state without an error.
88
-
89
- ### destroyed
90
-
91
- The **destroyed** state is a final state for the `Client` instance. Once in this state, a `Client` is nonfunctional. Calling any other `Client` methods will result in an `ClientDestroyedError`.
@@ -1,64 +0,0 @@
1
- # Client certificate
2
-
3
- Client certificate authentication can be configured with the `Client`, the required options are passed along through the `connect` option.
4
-
5
- The client certificates must be signed by a trusted CA. The Node.js default is to trust the well-known CAs curated by Mozilla.
6
-
7
- Setting the server option `requestCert: true` tells the server to request the client certificate.
8
-
9
- The server option `rejectUnauthorized: false` allows us to handle any invalid certificate errors in client code. The `authorized` property on the socket of the incoming request will show if the client certificate was valid. The `authorizationError` property will give the reason if the certificate was not valid.
10
-
11
- ### Client Certificate Authentication
12
-
13
- ```js
14
- const { readFileSync } = require('node:fs')
15
- const { join } = require('node:path')
16
- const { createServer } = require('node:https')
17
- const { Client } = require('undici')
18
-
19
- const serverOptions = {
20
- ca: [
21
- readFileSync(join(__dirname, 'client-ca-crt.pem'), 'utf8')
22
- ],
23
- key: readFileSync(join(__dirname, 'server-key.pem'), 'utf8'),
24
- cert: readFileSync(join(__dirname, 'server-crt.pem'), 'utf8'),
25
- requestCert: true,
26
- rejectUnauthorized: false
27
- }
28
-
29
- const server = createServer(serverOptions, (req, res) => {
30
- // true if client cert is valid
31
- if(req.client.authorized === true) {
32
- console.log('valid')
33
- } else {
34
- console.error(req.client.authorizationError)
35
- }
36
- res.end()
37
- })
38
-
39
- server.listen(0, function () {
40
- const tls = {
41
- ca: [
42
- readFileSync(join(__dirname, 'server-ca-crt.pem'), 'utf8')
43
- ],
44
- key: readFileSync(join(__dirname, 'client-key.pem'), 'utf8'),
45
- cert: readFileSync(join(__dirname, 'client-crt.pem'), 'utf8'),
46
- rejectUnauthorized: false,
47
- servername: 'agent1'
48
- }
49
- const client = new Client(`https://localhost:${server.address().port}`, {
50
- connect: tls
51
- })
52
-
53
- client.request({
54
- path: '/',
55
- method: 'GET'
56
- }, (err, { body }) => {
57
- body.on('data', (buf) => {})
58
- body.on('end', () => {
59
- client.close()
60
- server.close()
61
- })
62
- })
63
- })
64
- ```
@@ -1,58 +0,0 @@
1
- # Crawling
2
-
3
- [RFC 9309](https://datatracker.ietf.org/doc/html/rfc9309) defines crawlers as automated clients.
4
-
5
- Some web servers may reject requests that omit the `User-Agent` header or that use common defaults such as `'curl/7.79.1'`.
6
-
7
- In **undici**, the default user agent is `'undici'`. Since undici is integrated into Node.js core as the implementation of `fetch()`, requests made via `fetch()` use `'node'` as the default user agent.
8
-
9
- It is recommended to specify a **custom `User-Agent` header** when implementing crawlers. Providing a descriptive user agent allows servers to correctly identify the client and reduces the likelihood of requests being denied.
10
-
11
- A user agent string should include sufficient detail to identify the crawler and provide contact information. For example:
12
-
13
- ```
14
- AcmeCo Crawler - acme.co - contact@acme.co
15
- ```
16
-
17
- When adding contact details, avoid using personal identifiers such as your own name or a private email address—especially in a professional or employment context. Instead, use a role-based or organizational contact (e.g., crawler-team@company.com) to protect individual privacy while still enabling communication.
18
-
19
- If a crawler behaves unexpectedly—for example, due to misconfiguration or implementation errors—server administrators can use the information in the user agent to contact the operator and coordinate an appropriate resolution.
20
-
21
- The `User-Agent` header can be set on individual requests or applied globally by configuring a custom dispatcher.
22
-
23
- **Example: setting a `User-Agent` per request**
24
-
25
- ```js
26
- import { fetch } from 'undici'
27
-
28
- const headers = {
29
- 'User-Agent': 'AcmeCo Crawler - acme.co - contact@acme.co'
30
- }
31
-
32
- const res = await fetch('https://example.com', { headers })
33
- ```
34
-
35
- ## Best Practices for Crawlers
36
-
37
- When developing a crawler, the following practices are recommended in addition to setting a descriptive `User-Agent` header:
38
-
39
- * **Respect `robots.txt`**
40
- Follow the directives defined in the target site’s `robots.txt` file, including disallowed paths and optional crawl-delay settings (see [W3C guidelines](https://www.w3.org/wiki/Write_Web_Crawler)).
41
-
42
- * **Rate limiting**
43
- Regulate request frequency to avoid imposing excessive load on servers. Introduce delays between requests or limit the number of concurrent requests. The W3C suggests at least one second between requests.
44
-
45
- * **Error handling**
46
- Implement retry logic with exponential backoff for transient failures, and stop requests when persistent errors occur (e.g., HTTP 403 or 429).
47
-
48
- * **Monitoring and logging**
49
- Track request volume, response codes, and error rates to detect misbehavior and address issues proactively.
50
-
51
- * **Contact information**
52
- Always include valid and current contact details in the `User-Agent` string so that administrators can reach the crawler operator if necessary.
53
-
54
- ## References and Further Reading
55
-
56
- * [RFC 9309: The Robots Exclusion Protocol](https://datatracker.ietf.org/doc/html/rfc9309)
57
- * [W3C Wiki: Write Web Crawler](https://www.w3.org/wiki/Write_Web_Crawler)
58
- * [Ethical Web Crawling (WWW 2010 Conference Paper)](https://archives.iw3c2.org/www2010/proceedings/www/p1101.pdf)