es6-crawler-detect 3.3.0 → 4.0.0-beta.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (383) hide show
  1. package/.babelrc +1 -1
  2. package/.mocharc.json +5 -0
  3. package/README.md +1 -0
  4. package/dist/index.d.ts +5 -0
  5. package/dist/index.d.ts.map +1 -0
  6. package/dist/index.js +18 -0
  7. package/dist/index.js.map +1 -0
  8. package/dist/lib/crawler/crawlers.d.ts +5 -0
  9. package/dist/lib/crawler/crawlers.d.ts.map +1 -0
  10. package/dist/lib/crawler/crawlers.js +1400 -0
  11. package/dist/lib/crawler/crawlers.js.map +1 -0
  12. package/dist/lib/crawler/exclusions.d.ts +5 -0
  13. package/dist/lib/crawler/exclusions.d.ts.map +1 -0
  14. package/dist/lib/crawler/exclusions.js +59 -0
  15. package/dist/lib/crawler/exclusions.js.map +1 -0
  16. package/dist/lib/crawler/headers.d.ts +5 -0
  17. package/dist/lib/crawler/headers.d.ts.map +1 -0
  18. package/dist/lib/crawler/headers.js +21 -0
  19. package/dist/lib/crawler/headers.js.map +1 -0
  20. package/dist/lib/crawler/provider.d.ts +4 -0
  21. package/dist/lib/crawler/provider.d.ts.map +1 -0
  22. package/dist/lib/crawler/provider.js +3 -0
  23. package/dist/lib/crawler/provider.js.map +1 -0
  24. package/dist/lib/crawler.d.ts +21 -0
  25. package/dist/lib/crawler.d.ts.map +1 -0
  26. package/dist/lib/crawler.js +98 -0
  27. package/dist/lib/crawler.js.map +1 -0
  28. package/dist/types.d.ts +9 -0
  29. package/dist/types.d.ts.map +1 -0
  30. package/dist/types.js +3 -0
  31. package/dist/types.js.map +1 -0
  32. package/eslint.config.mjs +62 -0
  33. package/example/node/package.json +1 -1
  34. package/example/node/yarn.lock +222 -99
  35. package/package.json +30 -24
  36. package/src/index.ts +18 -0
  37. package/src/lib/crawler/{crawlers.js → crawlers.ts} +1397 -1403
  38. package/src/lib/crawler/{exclusions.js → exclusions.ts} +56 -62
  39. package/src/lib/crawler/{headers.js → headers.ts} +4 -10
  40. package/src/lib/crawler/provider.ts +3 -0
  41. package/src/lib/crawler.ts +128 -0
  42. package/src/types.ts +15 -0
  43. package/test/lib/crawler.spec.ts +116 -0
  44. package/tsconfig.json +28 -0
  45. package/.eslintignore +0 -6
  46. package/.eslintrc.json +0 -30
  47. package/dist/main.bundle.js +0 -1
  48. package/example/node/dist/crawler.min.js +0 -1
  49. package/example/node/node_modules/.bin/mime +0 -15
  50. package/example/node/node_modules/.bin/mime.cmd +0 -7
  51. package/example/node/node_modules/.yarn-integrity +0 -67
  52. package/example/node/node_modules/accepts/HISTORY.md +0 -236
  53. package/example/node/node_modules/accepts/LICENSE +0 -23
  54. package/example/node/node_modules/accepts/README.md +0 -142
  55. package/example/node/node_modules/accepts/index.js +0 -238
  56. package/example/node/node_modules/accepts/package.json +0 -47
  57. package/example/node/node_modules/array-flatten/LICENSE +0 -21
  58. package/example/node/node_modules/array-flatten/README.md +0 -43
  59. package/example/node/node_modules/array-flatten/array-flatten.js +0 -64
  60. package/example/node/node_modules/array-flatten/package.json +0 -39
  61. package/example/node/node_modules/body-parser/HISTORY.md +0 -624
  62. package/example/node/node_modules/body-parser/LICENSE +0 -23
  63. package/example/node/node_modules/body-parser/README.md +0 -456
  64. package/example/node/node_modules/body-parser/index.js +0 -157
  65. package/example/node/node_modules/body-parser/lib/read.js +0 -181
  66. package/example/node/node_modules/body-parser/lib/types/json.js +0 -230
  67. package/example/node/node_modules/body-parser/lib/types/raw.js +0 -101
  68. package/example/node/node_modules/body-parser/lib/types/text.js +0 -121
  69. package/example/node/node_modules/body-parser/lib/types/urlencoded.js +0 -284
  70. package/example/node/node_modules/body-parser/package.json +0 -52
  71. package/example/node/node_modules/bytes/History.md +0 -92
  72. package/example/node/node_modules/bytes/LICENSE +0 -23
  73. package/example/node/node_modules/bytes/Readme.md +0 -152
  74. package/example/node/node_modules/bytes/index.js +0 -166
  75. package/example/node/node_modules/bytes/package.json +0 -42
  76. package/example/node/node_modules/content-disposition/HISTORY.md +0 -60
  77. package/example/node/node_modules/content-disposition/LICENSE +0 -22
  78. package/example/node/node_modules/content-disposition/README.md +0 -142
  79. package/example/node/node_modules/content-disposition/index.js +0 -458
  80. package/example/node/node_modules/content-disposition/package.json +0 -44
  81. package/example/node/node_modules/content-type/HISTORY.md +0 -24
  82. package/example/node/node_modules/content-type/LICENSE +0 -22
  83. package/example/node/node_modules/content-type/README.md +0 -92
  84. package/example/node/node_modules/content-type/index.js +0 -222
  85. package/example/node/node_modules/content-type/package.json +0 -40
  86. package/example/node/node_modules/cookie/HISTORY.md +0 -128
  87. package/example/node/node_modules/cookie/LICENSE +0 -24
  88. package/example/node/node_modules/cookie/README.md +0 -257
  89. package/example/node/node_modules/cookie/index.js +0 -202
  90. package/example/node/node_modules/cookie/package.json +0 -40
  91. package/example/node/node_modules/cookie-signature/History.md +0 -38
  92. package/example/node/node_modules/cookie-signature/Readme.md +0 -42
  93. package/example/node/node_modules/cookie-signature/index.js +0 -51
  94. package/example/node/node_modules/cookie-signature/package.json +0 -18
  95. package/example/node/node_modules/debug/.coveralls.yml +0 -1
  96. package/example/node/node_modules/debug/.eslintrc +0 -11
  97. package/example/node/node_modules/debug/.travis.yml +0 -14
  98. package/example/node/node_modules/debug/CHANGELOG.md +0 -362
  99. package/example/node/node_modules/debug/LICENSE +0 -19
  100. package/example/node/node_modules/debug/Makefile +0 -50
  101. package/example/node/node_modules/debug/README.md +0 -312
  102. package/example/node/node_modules/debug/component.json +0 -19
  103. package/example/node/node_modules/debug/karma.conf.js +0 -70
  104. package/example/node/node_modules/debug/node.js +0 -1
  105. package/example/node/node_modules/debug/package.json +0 -49
  106. package/example/node/node_modules/debug/src/browser.js +0 -185
  107. package/example/node/node_modules/debug/src/debug.js +0 -202
  108. package/example/node/node_modules/debug/src/index.js +0 -10
  109. package/example/node/node_modules/debug/src/inspector-log.js +0 -15
  110. package/example/node/node_modules/debug/src/node.js +0 -248
  111. package/example/node/node_modules/depd/History.md +0 -96
  112. package/example/node/node_modules/depd/LICENSE +0 -22
  113. package/example/node/node_modules/depd/Readme.md +0 -280
  114. package/example/node/node_modules/depd/index.js +0 -522
  115. package/example/node/node_modules/depd/lib/browser/index.js +0 -77
  116. package/example/node/node_modules/depd/lib/compat/callsite-tostring.js +0 -103
  117. package/example/node/node_modules/depd/lib/compat/event-listener-count.js +0 -22
  118. package/example/node/node_modules/depd/lib/compat/index.js +0 -79
  119. package/example/node/node_modules/depd/package.json +0 -41
  120. package/example/node/node_modules/destroy/LICENSE +0 -22
  121. package/example/node/node_modules/destroy/README.md +0 -60
  122. package/example/node/node_modules/destroy/index.js +0 -75
  123. package/example/node/node_modules/destroy/package.json +0 -37
  124. package/example/node/node_modules/ee-first/LICENSE +0 -22
  125. package/example/node/node_modules/ee-first/README.md +0 -80
  126. package/example/node/node_modules/ee-first/index.js +0 -95
  127. package/example/node/node_modules/ee-first/package.json +0 -29
  128. package/example/node/node_modules/encodeurl/HISTORY.md +0 -14
  129. package/example/node/node_modules/encodeurl/LICENSE +0 -22
  130. package/example/node/node_modules/encodeurl/README.md +0 -128
  131. package/example/node/node_modules/encodeurl/index.js +0 -60
  132. package/example/node/node_modules/encodeurl/package.json +0 -40
  133. package/example/node/node_modules/escape-html/LICENSE +0 -24
  134. package/example/node/node_modules/escape-html/Readme.md +0 -43
  135. package/example/node/node_modules/escape-html/index.js +0 -78
  136. package/example/node/node_modules/escape-html/package.json +0 -24
  137. package/example/node/node_modules/etag/HISTORY.md +0 -83
  138. package/example/node/node_modules/etag/LICENSE +0 -22
  139. package/example/node/node_modules/etag/README.md +0 -159
  140. package/example/node/node_modules/etag/index.js +0 -131
  141. package/example/node/node_modules/etag/package.json +0 -47
  142. package/example/node/node_modules/express/History.md +0 -3510
  143. package/example/node/node_modules/express/LICENSE +0 -24
  144. package/example/node/node_modules/express/Readme.md +0 -158
  145. package/example/node/node_modules/express/index.js +0 -11
  146. package/example/node/node_modules/express/lib/application.js +0 -644
  147. package/example/node/node_modules/express/lib/express.js +0 -116
  148. package/example/node/node_modules/express/lib/middleware/init.js +0 -43
  149. package/example/node/node_modules/express/lib/middleware/query.js +0 -47
  150. package/example/node/node_modules/express/lib/request.js +0 -525
  151. package/example/node/node_modules/express/lib/response.js +0 -1147
  152. package/example/node/node_modules/express/lib/router/index.js +0 -668
  153. package/example/node/node_modules/express/lib/router/layer.js +0 -181
  154. package/example/node/node_modules/express/lib/router/route.js +0 -216
  155. package/example/node/node_modules/express/lib/utils.js +0 -302
  156. package/example/node/node_modules/express/lib/view.js +0 -182
  157. package/example/node/node_modules/express/package.json +0 -99
  158. package/example/node/node_modules/finalhandler/HISTORY.md +0 -187
  159. package/example/node/node_modules/finalhandler/LICENSE +0 -22
  160. package/example/node/node_modules/finalhandler/README.md +0 -148
  161. package/example/node/node_modules/finalhandler/index.js +0 -331
  162. package/example/node/node_modules/finalhandler/package.json +0 -45
  163. package/example/node/node_modules/forwarded/HISTORY.md +0 -21
  164. package/example/node/node_modules/forwarded/LICENSE +0 -22
  165. package/example/node/node_modules/forwarded/README.md +0 -57
  166. package/example/node/node_modules/forwarded/index.js +0 -90
  167. package/example/node/node_modules/forwarded/package.json +0 -45
  168. package/example/node/node_modules/fresh/HISTORY.md +0 -70
  169. package/example/node/node_modules/fresh/LICENSE +0 -23
  170. package/example/node/node_modules/fresh/README.md +0 -119
  171. package/example/node/node_modules/fresh/index.js +0 -137
  172. package/example/node/node_modules/fresh/package.json +0 -46
  173. package/example/node/node_modules/http-errors/HISTORY.md +0 -165
  174. package/example/node/node_modules/http-errors/LICENSE +0 -23
  175. package/example/node/node_modules/http-errors/README.md +0 -169
  176. package/example/node/node_modules/http-errors/index.js +0 -299
  177. package/example/node/node_modules/http-errors/package.json +0 -49
  178. package/example/node/node_modules/iconv-lite/Changelog.md +0 -162
  179. package/example/node/node_modules/iconv-lite/LICENSE +0 -21
  180. package/example/node/node_modules/iconv-lite/README.md +0 -156
  181. package/example/node/node_modules/iconv-lite/encodings/dbcs-codec.js +0 -555
  182. package/example/node/node_modules/iconv-lite/encodings/dbcs-data.js +0 -176
  183. package/example/node/node_modules/iconv-lite/encodings/index.js +0 -22
  184. package/example/node/node_modules/iconv-lite/encodings/internal.js +0 -188
  185. package/example/node/node_modules/iconv-lite/encodings/sbcs-codec.js +0 -72
  186. package/example/node/node_modules/iconv-lite/encodings/sbcs-data-generated.js +0 -451
  187. package/example/node/node_modules/iconv-lite/encodings/sbcs-data.js +0 -174
  188. package/example/node/node_modules/iconv-lite/encodings/tables/big5-added.json +0 -122
  189. package/example/node/node_modules/iconv-lite/encodings/tables/cp936.json +0 -264
  190. package/example/node/node_modules/iconv-lite/encodings/tables/cp949.json +0 -273
  191. package/example/node/node_modules/iconv-lite/encodings/tables/cp950.json +0 -177
  192. package/example/node/node_modules/iconv-lite/encodings/tables/eucjp.json +0 -182
  193. package/example/node/node_modules/iconv-lite/encodings/tables/gb18030-ranges.json +0 -1
  194. package/example/node/node_modules/iconv-lite/encodings/tables/gbk-added.json +0 -55
  195. package/example/node/node_modules/iconv-lite/encodings/tables/shiftjis.json +0 -125
  196. package/example/node/node_modules/iconv-lite/encodings/utf16.js +0 -177
  197. package/example/node/node_modules/iconv-lite/encodings/utf7.js +0 -290
  198. package/example/node/node_modules/iconv-lite/lib/bom-handling.js +0 -52
  199. package/example/node/node_modules/iconv-lite/lib/extend-node.js +0 -217
  200. package/example/node/node_modules/iconv-lite/lib/index.d.ts +0 -24
  201. package/example/node/node_modules/iconv-lite/lib/index.js +0 -153
  202. package/example/node/node_modules/iconv-lite/lib/streams.js +0 -121
  203. package/example/node/node_modules/iconv-lite/package.json +0 -46
  204. package/example/node/node_modules/inherits/LICENSE +0 -16
  205. package/example/node/node_modules/inherits/README.md +0 -42
  206. package/example/node/node_modules/inherits/inherits.js +0 -9
  207. package/example/node/node_modules/inherits/inherits_browser.js +0 -27
  208. package/example/node/node_modules/inherits/package.json +0 -29
  209. package/example/node/node_modules/ipaddr.js/LICENSE +0 -19
  210. package/example/node/node_modules/ipaddr.js/README.md +0 -233
  211. package/example/node/node_modules/ipaddr.js/ipaddr.min.js +0 -1
  212. package/example/node/node_modules/ipaddr.js/lib/ipaddr.js +0 -673
  213. package/example/node/node_modules/ipaddr.js/lib/ipaddr.js.d.ts +0 -68
  214. package/example/node/node_modules/ipaddr.js/package.json +0 -35
  215. package/example/node/node_modules/media-typer/HISTORY.md +0 -22
  216. package/example/node/node_modules/media-typer/LICENSE +0 -22
  217. package/example/node/node_modules/media-typer/README.md +0 -81
  218. package/example/node/node_modules/media-typer/index.js +0 -270
  219. package/example/node/node_modules/media-typer/package.json +0 -26
  220. package/example/node/node_modules/merge-descriptors/HISTORY.md +0 -21
  221. package/example/node/node_modules/merge-descriptors/LICENSE +0 -23
  222. package/example/node/node_modules/merge-descriptors/README.md +0 -48
  223. package/example/node/node_modules/merge-descriptors/index.js +0 -60
  224. package/example/node/node_modules/merge-descriptors/package.json +0 -32
  225. package/example/node/node_modules/methods/HISTORY.md +0 -29
  226. package/example/node/node_modules/methods/LICENSE +0 -24
  227. package/example/node/node_modules/methods/README.md +0 -51
  228. package/example/node/node_modules/methods/index.js +0 -69
  229. package/example/node/node_modules/methods/package.json +0 -36
  230. package/example/node/node_modules/mime/CHANGELOG.md +0 -164
  231. package/example/node/node_modules/mime/LICENSE +0 -21
  232. package/example/node/node_modules/mime/README.md +0 -90
  233. package/example/node/node_modules/mime/cli.js +0 -8
  234. package/example/node/node_modules/mime/mime.js +0 -108
  235. package/example/node/node_modules/mime/package.json +0 -44
  236. package/example/node/node_modules/mime/src/build.js +0 -53
  237. package/example/node/node_modules/mime/src/test.js +0 -60
  238. package/example/node/node_modules/mime/types.json +0 -1
  239. package/example/node/node_modules/mime-db/HISTORY.md +0 -499
  240. package/example/node/node_modules/mime-db/LICENSE +0 -22
  241. package/example/node/node_modules/mime-db/README.md +0 -100
  242. package/example/node/node_modules/mime-db/db.json +0 -8471
  243. package/example/node/node_modules/mime-db/index.js +0 -11
  244. package/example/node/node_modules/mime-db/package.json +0 -59
  245. package/example/node/node_modules/mime-types/HISTORY.md +0 -388
  246. package/example/node/node_modules/mime-types/LICENSE +0 -23
  247. package/example/node/node_modules/mime-types/README.md +0 -113
  248. package/example/node/node_modules/mime-types/index.js +0 -188
  249. package/example/node/node_modules/mime-types/package.json +0 -44
  250. package/example/node/node_modules/ms/index.js +0 -152
  251. package/example/node/node_modules/ms/license.md +0 -21
  252. package/example/node/node_modules/ms/package.json +0 -37
  253. package/example/node/node_modules/ms/readme.md +0 -51
  254. package/example/node/node_modules/negotiator/HISTORY.md +0 -103
  255. package/example/node/node_modules/negotiator/LICENSE +0 -24
  256. package/example/node/node_modules/negotiator/README.md +0 -203
  257. package/example/node/node_modules/negotiator/index.js +0 -124
  258. package/example/node/node_modules/negotiator/lib/charset.js +0 -169
  259. package/example/node/node_modules/negotiator/lib/encoding.js +0 -184
  260. package/example/node/node_modules/negotiator/lib/language.js +0 -179
  261. package/example/node/node_modules/negotiator/lib/mediaType.js +0 -294
  262. package/example/node/node_modules/negotiator/package.json +0 -42
  263. package/example/node/node_modules/on-finished/HISTORY.md +0 -88
  264. package/example/node/node_modules/on-finished/LICENSE +0 -23
  265. package/example/node/node_modules/on-finished/README.md +0 -154
  266. package/example/node/node_modules/on-finished/index.js +0 -196
  267. package/example/node/node_modules/on-finished/package.json +0 -31
  268. package/example/node/node_modules/parseurl/HISTORY.md +0 -58
  269. package/example/node/node_modules/parseurl/LICENSE +0 -24
  270. package/example/node/node_modules/parseurl/README.md +0 -133
  271. package/example/node/node_modules/parseurl/index.js +0 -158
  272. package/example/node/node_modules/parseurl/package.json +0 -40
  273. package/example/node/node_modules/path-to-regexp/History.md +0 -36
  274. package/example/node/node_modules/path-to-regexp/LICENSE +0 -21
  275. package/example/node/node_modules/path-to-regexp/Readme.md +0 -35
  276. package/example/node/node_modules/path-to-regexp/index.js +0 -129
  277. package/example/node/node_modules/path-to-regexp/package.json +0 -30
  278. package/example/node/node_modules/proxy-addr/HISTORY.md +0 -161
  279. package/example/node/node_modules/proxy-addr/LICENSE +0 -22
  280. package/example/node/node_modules/proxy-addr/README.md +0 -139
  281. package/example/node/node_modules/proxy-addr/index.js +0 -327
  282. package/example/node/node_modules/proxy-addr/package.json +0 -47
  283. package/example/node/node_modules/qs/.editorconfig +0 -39
  284. package/example/node/node_modules/qs/.eslintignore +0 -2
  285. package/example/node/node_modules/qs/.eslintrc +0 -35
  286. package/example/node/node_modules/qs/.github/FUNDING.yml +0 -12
  287. package/example/node/node_modules/qs/.nycrc +0 -13
  288. package/example/node/node_modules/qs/CHANGELOG.md +0 -359
  289. package/example/node/node_modules/qs/LICENSE.md +0 -29
  290. package/example/node/node_modules/qs/README.md +0 -604
  291. package/example/node/node_modules/qs/dist/qs.js +0 -832
  292. package/example/node/node_modules/qs/lib/formats.js +0 -23
  293. package/example/node/node_modules/qs/lib/index.js +0 -11
  294. package/example/node/node_modules/qs/lib/parse.js +0 -257
  295. package/example/node/node_modules/qs/lib/stringify.js +0 -278
  296. package/example/node/node_modules/qs/lib/utils.js +0 -251
  297. package/example/node/node_modules/qs/package.json +0 -69
  298. package/example/node/node_modules/qs/test/parse.js +0 -772
  299. package/example/node/node_modules/qs/test/stringify.js +0 -793
  300. package/example/node/node_modules/qs/test/utils.js +0 -136
  301. package/example/node/node_modules/range-parser/HISTORY.md +0 -56
  302. package/example/node/node_modules/range-parser/LICENSE +0 -23
  303. package/example/node/node_modules/range-parser/README.md +0 -84
  304. package/example/node/node_modules/range-parser/index.js +0 -162
  305. package/example/node/node_modules/range-parser/package.json +0 -44
  306. package/example/node/node_modules/raw-body/HISTORY.md +0 -284
  307. package/example/node/node_modules/raw-body/LICENSE +0 -22
  308. package/example/node/node_modules/raw-body/README.md +0 -217
  309. package/example/node/node_modules/raw-body/index.d.ts +0 -87
  310. package/example/node/node_modules/raw-body/index.js +0 -286
  311. package/example/node/node_modules/raw-body/package.json +0 -48
  312. package/example/node/node_modules/safe-buffer/LICENSE +0 -21
  313. package/example/node/node_modules/safe-buffer/README.md +0 -584
  314. package/example/node/node_modules/safe-buffer/index.d.ts +0 -187
  315. package/example/node/node_modules/safe-buffer/index.js +0 -65
  316. package/example/node/node_modules/safe-buffer/package.json +0 -51
  317. package/example/node/node_modules/safer-buffer/LICENSE +0 -21
  318. package/example/node/node_modules/safer-buffer/Porting-Buffer.md +0 -268
  319. package/example/node/node_modules/safer-buffer/Readme.md +0 -156
  320. package/example/node/node_modules/safer-buffer/dangerous.js +0 -58
  321. package/example/node/node_modules/safer-buffer/package.json +0 -34
  322. package/example/node/node_modules/safer-buffer/safer.js +0 -77
  323. package/example/node/node_modules/safer-buffer/tests.js +0 -406
  324. package/example/node/node_modules/send/HISTORY.md +0 -506
  325. package/example/node/node_modules/send/LICENSE +0 -23
  326. package/example/node/node_modules/send/README.md +0 -327
  327. package/example/node/node_modules/send/index.js +0 -1133
  328. package/example/node/node_modules/send/node_modules/.bin/mime +0 -15
  329. package/example/node/node_modules/send/node_modules/.bin/mime.cmd +0 -7
  330. package/example/node/node_modules/send/node_modules/ms/index.js +0 -162
  331. package/example/node/node_modules/send/node_modules/ms/license.md +0 -21
  332. package/example/node/node_modules/send/node_modules/ms/package.json +0 -38
  333. package/example/node/node_modules/send/node_modules/ms/readme.md +0 -59
  334. package/example/node/node_modules/send/package.json +0 -61
  335. package/example/node/node_modules/serve-static/HISTORY.md +0 -459
  336. package/example/node/node_modules/serve-static/LICENSE +0 -25
  337. package/example/node/node_modules/serve-static/README.md +0 -257
  338. package/example/node/node_modules/serve-static/index.js +0 -210
  339. package/example/node/node_modules/serve-static/package.json +0 -42
  340. package/example/node/node_modules/setprototypeof/LICENSE +0 -13
  341. package/example/node/node_modules/setprototypeof/README.md +0 -31
  342. package/example/node/node_modules/setprototypeof/index.d.ts +0 -2
  343. package/example/node/node_modules/setprototypeof/index.js +0 -17
  344. package/example/node/node_modules/setprototypeof/package.json +0 -38
  345. package/example/node/node_modules/setprototypeof/test/index.js +0 -24
  346. package/example/node/node_modules/statuses/HISTORY.md +0 -65
  347. package/example/node/node_modules/statuses/LICENSE +0 -23
  348. package/example/node/node_modules/statuses/README.md +0 -127
  349. package/example/node/node_modules/statuses/codes.json +0 -66
  350. package/example/node/node_modules/statuses/index.js +0 -113
  351. package/example/node/node_modules/statuses/package.json +0 -48
  352. package/example/node/node_modules/toidentifier/HISTORY.md +0 -9
  353. package/example/node/node_modules/toidentifier/LICENSE +0 -21
  354. package/example/node/node_modules/toidentifier/README.md +0 -61
  355. package/example/node/node_modules/toidentifier/index.js +0 -32
  356. package/example/node/node_modules/toidentifier/package.json +0 -38
  357. package/example/node/node_modules/type-is/HISTORY.md +0 -259
  358. package/example/node/node_modules/type-is/LICENSE +0 -23
  359. package/example/node/node_modules/type-is/README.md +0 -170
  360. package/example/node/node_modules/type-is/index.js +0 -266
  361. package/example/node/node_modules/type-is/package.json +0 -45
  362. package/example/node/node_modules/unpipe/HISTORY.md +0 -4
  363. package/example/node/node_modules/unpipe/LICENSE +0 -22
  364. package/example/node/node_modules/unpipe/README.md +0 -43
  365. package/example/node/node_modules/unpipe/index.js +0 -69
  366. package/example/node/node_modules/unpipe/package.json +0 -27
  367. package/example/node/node_modules/utils-merge/LICENSE +0 -20
  368. package/example/node/node_modules/utils-merge/README.md +0 -34
  369. package/example/node/node_modules/utils-merge/index.js +0 -23
  370. package/example/node/node_modules/utils-merge/package.json +0 -40
  371. package/example/node/node_modules/vary/HISTORY.md +0 -39
  372. package/example/node/node_modules/vary/LICENSE +0 -22
  373. package/example/node/node_modules/vary/README.md +0 -101
  374. package/example/node/node_modules/vary/index.js +0 -149
  375. package/example/node/node_modules/vary/package.json +0 -43
  376. package/example/node/server.js +0 -38
  377. package/src/index.js +0 -16
  378. package/src/lib/crawler/provider.js +0 -11
  379. package/src/lib/crawler.js +0 -130
  380. package/test/lib/crawler.test.js +0 -94
  381. package/webpack.common.js +0 -20
  382. package/webpack.dev.js +0 -7
  383. package/webpack.prod.js +0 -14
@@ -1,299 +0,0 @@
1
- /*!
2
- * http-errors
3
- * Copyright(c) 2014 Jonathan Ong
4
- * Copyright(c) 2016 Douglas Christopher Wilson
5
- * MIT Licensed
6
- */
7
-
8
- 'use strict'
9
-
10
- /**
11
- * Module dependencies.
12
- * @private
13
- */
14
-
15
- var deprecate = require('depd')('http-errors')
16
- var setPrototypeOf = require('setprototypeof')
17
- var statuses = require('statuses')
18
- var inherits = require('inherits')
19
- var toIdentifier = require('toidentifier')
20
-
21
- /**
22
- * Module exports.
23
- * @public
24
- */
25
-
26
- module.exports = createError
27
- module.exports.HttpError = createHttpErrorConstructor()
28
- module.exports.isHttpError = createIsHttpErrorFunction(module.exports.HttpError)
29
-
30
- // Populate exports for all constructors
31
- populateConstructorExports(module.exports, statuses.codes, module.exports.HttpError)
32
-
33
- /**
34
- * Get the code class of a status code.
35
- * @private
36
- */
37
-
38
- function codeClass (status) {
39
- return Number(String(status).charAt(0) + '00')
40
- }
41
-
42
- /**
43
- * Create a new HTTP Error.
44
- *
45
- * @returns {Error}
46
- * @public
47
- */
48
-
49
- function createError () {
50
- // so much arity going on ~_~
51
- var err
52
- var msg
53
- var status = 500
54
- var props = {}
55
- for (var i = 0; i < arguments.length; i++) {
56
- var arg = arguments[i]
57
- if (arg instanceof Error) {
58
- err = arg
59
- status = err.status || err.statusCode || status
60
- continue
61
- }
62
- switch (typeof arg) {
63
- case 'string':
64
- msg = arg
65
- break
66
- case 'number':
67
- status = arg
68
- if (i !== 0) {
69
- deprecate('non-first-argument status code; replace with createError(' + arg + ', ...)')
70
- }
71
- break
72
- case 'object':
73
- props = arg
74
- break
75
- }
76
- }
77
-
78
- if (typeof status === 'number' && (status < 400 || status >= 600)) {
79
- deprecate('non-error status code; use only 4xx or 5xx status codes')
80
- }
81
-
82
- if (typeof status !== 'number' ||
83
- (!statuses[status] && (status < 400 || status >= 600))) {
84
- status = 500
85
- }
86
-
87
- // constructor
88
- var HttpError = createError[status] || createError[codeClass(status)]
89
-
90
- if (!err) {
91
- // create error
92
- err = HttpError
93
- ? new HttpError(msg)
94
- : new Error(msg || statuses[status])
95
- Error.captureStackTrace(err, createError)
96
- }
97
-
98
- if (!HttpError || !(err instanceof HttpError) || err.status !== status) {
99
- // add properties to generic error
100
- err.expose = status < 500
101
- err.status = err.statusCode = status
102
- }
103
-
104
- for (var key in props) {
105
- if (key !== 'status' && key !== 'statusCode') {
106
- err[key] = props[key]
107
- }
108
- }
109
-
110
- return err
111
- }
112
-
113
- /**
114
- * Create HTTP error abstract base class.
115
- * @private
116
- */
117
-
118
- function createHttpErrorConstructor () {
119
- function HttpError () {
120
- throw new TypeError('cannot construct abstract class')
121
- }
122
-
123
- inherits(HttpError, Error)
124
-
125
- return HttpError
126
- }
127
-
128
- /**
129
- * Create a constructor for a client error.
130
- * @private
131
- */
132
-
133
- function createClientErrorConstructor (HttpError, name, code) {
134
- var className = toClassName(name)
135
-
136
- function ClientError (message) {
137
- // create the error object
138
- var msg = message != null ? message : statuses[code]
139
- var err = new Error(msg)
140
-
141
- // capture a stack trace to the construction point
142
- Error.captureStackTrace(err, ClientError)
143
-
144
- // adjust the [[Prototype]]
145
- setPrototypeOf(err, ClientError.prototype)
146
-
147
- // redefine the error message
148
- Object.defineProperty(err, 'message', {
149
- enumerable: true,
150
- configurable: true,
151
- value: msg,
152
- writable: true
153
- })
154
-
155
- // redefine the error name
156
- Object.defineProperty(err, 'name', {
157
- enumerable: false,
158
- configurable: true,
159
- value: className,
160
- writable: true
161
- })
162
-
163
- return err
164
- }
165
-
166
- inherits(ClientError, HttpError)
167
- nameFunc(ClientError, className)
168
-
169
- ClientError.prototype.status = code
170
- ClientError.prototype.statusCode = code
171
- ClientError.prototype.expose = true
172
-
173
- return ClientError
174
- }
175
-
176
- /**
177
- * Create function to test is a value is a HttpError.
178
- * @private
179
- */
180
-
181
- function createIsHttpErrorFunction (HttpError) {
182
- return function isHttpError (val) {
183
- if (!val || typeof val !== 'object') {
184
- return false
185
- }
186
-
187
- if (val instanceof HttpError) {
188
- return true
189
- }
190
-
191
- return val instanceof Error &&
192
- typeof val.expose === 'boolean' &&
193
- typeof val.statusCode === 'number' && val.status === val.statusCode
194
- }
195
- }
196
-
197
- /**
198
- * Create a constructor for a server error.
199
- * @private
200
- */
201
-
202
- function createServerErrorConstructor (HttpError, name, code) {
203
- var className = toClassName(name)
204
-
205
- function ServerError (message) {
206
- // create the error object
207
- var msg = message != null ? message : statuses[code]
208
- var err = new Error(msg)
209
-
210
- // capture a stack trace to the construction point
211
- Error.captureStackTrace(err, ServerError)
212
-
213
- // adjust the [[Prototype]]
214
- setPrototypeOf(err, ServerError.prototype)
215
-
216
- // redefine the error message
217
- Object.defineProperty(err, 'message', {
218
- enumerable: true,
219
- configurable: true,
220
- value: msg,
221
- writable: true
222
- })
223
-
224
- // redefine the error name
225
- Object.defineProperty(err, 'name', {
226
- enumerable: false,
227
- configurable: true,
228
- value: className,
229
- writable: true
230
- })
231
-
232
- return err
233
- }
234
-
235
- inherits(ServerError, HttpError)
236
- nameFunc(ServerError, className)
237
-
238
- ServerError.prototype.status = code
239
- ServerError.prototype.statusCode = code
240
- ServerError.prototype.expose = false
241
-
242
- return ServerError
243
- }
244
-
245
- /**
246
- * Set the name of a function, if possible.
247
- * @private
248
- */
249
-
250
- function nameFunc (func, name) {
251
- var desc = Object.getOwnPropertyDescriptor(func, 'name')
252
-
253
- if (desc && desc.configurable) {
254
- desc.value = name
255
- Object.defineProperty(func, 'name', desc)
256
- }
257
- }
258
-
259
- /**
260
- * Populate the exports object with constructors for every error class.
261
- * @private
262
- */
263
-
264
- function populateConstructorExports (exports, codes, HttpError) {
265
- codes.forEach(function forEachCode (code) {
266
- var CodeError
267
- var name = toIdentifier(statuses[code])
268
-
269
- switch (codeClass(code)) {
270
- case 400:
271
- CodeError = createClientErrorConstructor(HttpError, name, code)
272
- break
273
- case 500:
274
- CodeError = createServerErrorConstructor(HttpError, name, code)
275
- break
276
- }
277
-
278
- if (CodeError) {
279
- // export the constructor
280
- exports[code] = CodeError
281
- exports[name] = CodeError
282
- }
283
- })
284
-
285
- // backwards-compatibility
286
- exports["I'mateapot"] = deprecate.function(exports.ImATeapot,
287
- '"I\'mateapot"; use "ImATeapot" instead')
288
- }
289
-
290
- /**
291
- * Get a class name from a name identifier.
292
- * @private
293
- */
294
-
295
- function toClassName (name) {
296
- return name.substr(-5) !== 'Error'
297
- ? name + 'Error'
298
- : name
299
- }
@@ -1,49 +0,0 @@
1
- {
2
- "name": "http-errors",
3
- "description": "Create HTTP error objects",
4
- "version": "1.8.1",
5
- "author": "Jonathan Ong <me@jongleberry.com> (http://jongleberry.com)",
6
- "contributors": [
7
- "Alan Plum <me@pluma.io>",
8
- "Douglas Christopher Wilson <doug@somethingdoug.com>"
9
- ],
10
- "license": "MIT",
11
- "repository": "jshttp/http-errors",
12
- "dependencies": {
13
- "depd": "~1.1.2",
14
- "inherits": "2.0.4",
15
- "setprototypeof": "1.2.0",
16
- "statuses": ">= 1.5.0 < 2",
17
- "toidentifier": "1.0.1"
18
- },
19
- "devDependencies": {
20
- "eslint": "7.32.0",
21
- "eslint-config-standard": "14.1.1",
22
- "eslint-plugin-import": "2.25.3",
23
- "eslint-plugin-markdown": "2.2.1",
24
- "eslint-plugin-node": "11.1.0",
25
- "eslint-plugin-promise": "5.1.1",
26
- "eslint-plugin-standard": "4.1.0",
27
- "mocha": "9.1.3",
28
- "nyc": "15.1.0"
29
- },
30
- "engines": {
31
- "node": ">= 0.6"
32
- },
33
- "scripts": {
34
- "lint": "eslint . && node ./scripts/lint-readme-list.js",
35
- "test": "mocha --reporter spec --bail",
36
- "test-ci": "nyc --reporter=lcov --reporter=text npm test",
37
- "test-cov": "nyc --reporter=html --reporter=text npm test"
38
- },
39
- "keywords": [
40
- "http",
41
- "error"
42
- ],
43
- "files": [
44
- "index.js",
45
- "HISTORY.md",
46
- "LICENSE",
47
- "README.md"
48
- ]
49
- }
@@ -1,162 +0,0 @@
1
- # 0.4.24 / 2018-08-22
2
-
3
- * Added MIK encoding (#196, by @Ivan-Kalatchev)
4
-
5
-
6
- # 0.4.23 / 2018-05-07
7
-
8
- * Fix deprecation warning in Node v10 due to the last usage of `new Buffer` (#185, by @felixbuenemann)
9
- * Switched from NodeBuffer to Buffer in typings (#155 by @felixfbecker, #186 by @larssn)
10
-
11
-
12
- # 0.4.22 / 2018-05-05
13
-
14
- * Use older semver style for dependencies to be compatible with Node version 0.10 (#182, by @dougwilson)
15
- * Fix tests to accomodate fixes in Node v10 (#182, by @dougwilson)
16
-
17
-
18
- # 0.4.21 / 2018-04-06
19
-
20
- * Fix encoding canonicalization (#156)
21
- * Fix the paths in the "browser" field in package.json (#174 by @LMLB)
22
- * Removed "contributors" section in package.json - see Git history instead.
23
-
24
-
25
- # 0.4.20 / 2018-04-06
26
-
27
- * Updated `new Buffer()` usages with recommended replacements as it's being deprecated in Node v10 (#176, #178 by @ChALkeR)
28
-
29
-
30
- # 0.4.19 / 2017-09-09
31
-
32
- * Fixed iso8859-1 codec regression in handling untranslatable characters (#162, caused by #147)
33
- * Re-generated windows1255 codec, because it was updated in iconv project
34
- * Fixed grammar in error message when iconv-lite is loaded with encoding other than utf8
35
-
36
-
37
- # 0.4.18 / 2017-06-13
38
-
39
- * Fixed CESU-8 regression in Node v8.
40
-
41
-
42
- # 0.4.17 / 2017-04-22
43
-
44
- * Updated typescript definition file to support Angular 2 AoT mode (#153 by @larssn)
45
-
46
-
47
- # 0.4.16 / 2017-04-22
48
-
49
- * Added support for React Native (#150)
50
- * Changed iso8859-1 encoding to usine internal 'binary' encoding, as it's the same thing (#147 by @mscdex)
51
- * Fixed typo in Readme (#138 by @jiangzhuo)
52
- * Fixed build for Node v6.10+ by making correct version comparison
53
- * Added a warning if iconv-lite is loaded not as utf-8 (see #142)
54
-
55
-
56
- # 0.4.15 / 2016-11-21
57
-
58
- * Fixed typescript type definition (#137)
59
-
60
-
61
- # 0.4.14 / 2016-11-20
62
-
63
- * Preparation for v1.0
64
- * Added Node v6 and latest Node versions to Travis CI test rig
65
- * Deprecated Node v0.8 support
66
- * Typescript typings (@larssn)
67
- * Fix encoding of Euro character in GB 18030 (inspired by @lygstate)
68
- * Add ms prefix to dbcs windows encodings (@rokoroku)
69
-
70
-
71
- # 0.4.13 / 2015-10-01
72
-
73
- * Fix silly mistake in deprecation notice.
74
-
75
-
76
- # 0.4.12 / 2015-09-26
77
-
78
- * Node v4 support:
79
- * Added CESU-8 decoding (#106)
80
- * Added deprecation notice for `extendNodeEncodings`
81
- * Added Travis tests for Node v4 and io.js latest (#105 by @Mithgol)
82
-
83
-
84
- # 0.4.11 / 2015-07-03
85
-
86
- * Added CESU-8 encoding.
87
-
88
-
89
- # 0.4.10 / 2015-05-26
90
-
91
- * Changed UTF-16 endianness heuristic to take into account any ASCII chars, not
92
- just spaces. This should minimize the importance of "default" endianness.
93
-
94
-
95
- # 0.4.9 / 2015-05-24
96
-
97
- * Streamlined BOM handling: strip BOM by default, add BOM when encoding if
98
- addBOM: true. Added docs to Readme.
99
- * UTF16 now uses UTF16-LE by default.
100
- * Fixed minor issue with big5 encoding.
101
- * Added io.js testing on Travis; updated node-iconv version to test against.
102
- Now we just skip testing SBCS encodings that node-iconv doesn't support.
103
- * (internal refactoring) Updated codec interface to use classes.
104
- * Use strict mode in all files.
105
-
106
-
107
- # 0.4.8 / 2015-04-14
108
-
109
- * added alias UNICODE-1-1-UTF-7 for UTF-7 encoding (#94)
110
-
111
-
112
- # 0.4.7 / 2015-02-05
113
-
114
- * stop official support of Node.js v0.8. Should still work, but no guarantees.
115
- reason: Packages needed for testing are hard to get on Travis CI.
116
- * work in environment where Object.prototype is monkey patched with enumerable
117
- props (#89).
118
-
119
-
120
- # 0.4.6 / 2015-01-12
121
-
122
- * fix rare aliases of single-byte encodings (thanks @mscdex)
123
- * double the timeout for dbcs tests to make them less flaky on travis
124
-
125
-
126
- # 0.4.5 / 2014-11-20
127
-
128
- * fix windows-31j and x-sjis encoding support (@nleush)
129
- * minor fix: undefined variable reference when internal error happens
130
-
131
-
132
- # 0.4.4 / 2014-07-16
133
-
134
- * added encodings UTF-7 (RFC2152) and UTF-7-IMAP (RFC3501 Section 5.1.3)
135
- * fixed streaming base64 encoding
136
-
137
-
138
- # 0.4.3 / 2014-06-14
139
-
140
- * added encodings UTF-16BE and UTF-16 with BOM
141
-
142
-
143
- # 0.4.2 / 2014-06-12
144
-
145
- * don't throw exception if `extendNodeEncodings()` is called more than once
146
-
147
-
148
- # 0.4.1 / 2014-06-11
149
-
150
- * codepage 808 added
151
-
152
-
153
- # 0.4.0 / 2014-06-10
154
-
155
- * code is rewritten from scratch
156
- * all widespread encodings are supported
157
- * streaming interface added
158
- * browserify compatibility added
159
- * (optional) extend core primitive encodings to make usage even simpler
160
- * moved from vows to mocha as the testing framework
161
-
162
-
@@ -1,21 +0,0 @@
1
- Copyright (c) 2011 Alexander Shtuchkin
2
-
3
- Permission is hereby granted, free of charge, to any person obtaining
4
- a copy of this software and associated documentation files (the
5
- "Software"), to deal in the Software without restriction, including
6
- without limitation the rights to use, copy, modify, merge, publish,
7
- distribute, sublicense, and/or sell copies of the Software, and to
8
- permit persons to whom the Software is furnished to do so, subject to
9
- the following conditions:
10
-
11
- The above copyright notice and this permission notice shall be
12
- included in all copies or substantial portions of the Software.
13
-
14
- THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
15
- EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
16
- MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
17
- NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
18
- LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
19
- OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
20
- WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
21
-
@@ -1,156 +0,0 @@
1
- ## Pure JS character encoding conversion [![Build Status](https://travis-ci.org/ashtuchkin/iconv-lite.svg?branch=master)](https://travis-ci.org/ashtuchkin/iconv-lite)
2
-
3
- * Doesn't need native code compilation. Works on Windows and in sandboxed environments like [Cloud9](http://c9.io).
4
- * Used in popular projects like [Express.js (body_parser)](https://github.com/expressjs/body-parser),
5
- [Grunt](http://gruntjs.com/), [Nodemailer](http://www.nodemailer.com/), [Yeoman](http://yeoman.io/) and others.
6
- * Faster than [node-iconv](https://github.com/bnoordhuis/node-iconv) (see below for performance comparison).
7
- * Intuitive encode/decode API
8
- * Streaming support for Node v0.10+
9
- * [Deprecated] Can extend Node.js primitives (buffers, streams) to support all iconv-lite encodings.
10
- * In-browser usage via [Browserify](https://github.com/substack/node-browserify) (~180k gzip compressed with Buffer shim included).
11
- * Typescript [type definition file](https://github.com/ashtuchkin/iconv-lite/blob/master/lib/index.d.ts) included.
12
- * React Native is supported (need to explicitly `npm install` two more modules: `buffer` and `stream`).
13
- * License: MIT.
14
-
15
- [![NPM Stats](https://nodei.co/npm/iconv-lite.png?downloads=true&downloadRank=true)](https://npmjs.org/packages/iconv-lite/)
16
-
17
- ## Usage
18
- ### Basic API
19
- ```javascript
20
- var iconv = require('iconv-lite');
21
-
22
- // Convert from an encoded buffer to js string.
23
- str = iconv.decode(Buffer.from([0x68, 0x65, 0x6c, 0x6c, 0x6f]), 'win1251');
24
-
25
- // Convert from js string to an encoded buffer.
26
- buf = iconv.encode("Sample input string", 'win1251');
27
-
28
- // Check if encoding is supported
29
- iconv.encodingExists("us-ascii")
30
- ```
31
-
32
- ### Streaming API (Node v0.10+)
33
- ```javascript
34
-
35
- // Decode stream (from binary stream to js strings)
36
- http.createServer(function(req, res) {
37
- var converterStream = iconv.decodeStream('win1251');
38
- req.pipe(converterStream);
39
-
40
- converterStream.on('data', function(str) {
41
- console.log(str); // Do something with decoded strings, chunk-by-chunk.
42
- });
43
- });
44
-
45
- // Convert encoding streaming example
46
- fs.createReadStream('file-in-win1251.txt')
47
- .pipe(iconv.decodeStream('win1251'))
48
- .pipe(iconv.encodeStream('ucs2'))
49
- .pipe(fs.createWriteStream('file-in-ucs2.txt'));
50
-
51
- // Sugar: all encode/decode streams have .collect(cb) method to accumulate data.
52
- http.createServer(function(req, res) {
53
- req.pipe(iconv.decodeStream('win1251')).collect(function(err, body) {
54
- assert(typeof body == 'string');
55
- console.log(body); // full request body string
56
- });
57
- });
58
- ```
59
-
60
- ### [Deprecated] Extend Node.js own encodings
61
- > NOTE: This doesn't work on latest Node versions. See [details](https://github.com/ashtuchkin/iconv-lite/wiki/Node-v4-compatibility).
62
-
63
- ```javascript
64
- // After this call all Node basic primitives will understand iconv-lite encodings.
65
- iconv.extendNodeEncodings();
66
-
67
- // Examples:
68
- buf = new Buffer(str, 'win1251');
69
- buf.write(str, 'gbk');
70
- str = buf.toString('latin1');
71
- assert(Buffer.isEncoding('iso-8859-15'));
72
- Buffer.byteLength(str, 'us-ascii');
73
-
74
- http.createServer(function(req, res) {
75
- req.setEncoding('big5');
76
- req.collect(function(err, body) {
77
- console.log(body);
78
- });
79
- });
80
-
81
- fs.createReadStream("file.txt", "shift_jis");
82
-
83
- // External modules are also supported (if they use Node primitives, which they probably do).
84
- request = require('request');
85
- request({
86
- url: "http://github.com/",
87
- encoding: "cp932"
88
- });
89
-
90
- // To remove extensions
91
- iconv.undoExtendNodeEncodings();
92
- ```
93
-
94
- ## Supported encodings
95
-
96
- * All node.js native encodings: utf8, ucs2 / utf16-le, ascii, binary, base64, hex.
97
- * Additional unicode encodings: utf16, utf16-be, utf-7, utf-7-imap.
98
- * All widespread singlebyte encodings: Windows 125x family, ISO-8859 family,
99
- IBM/DOS codepages, Macintosh family, KOI8 family, all others supported by iconv library.
100
- Aliases like 'latin1', 'us-ascii' also supported.
101
- * All widespread multibyte encodings: CP932, CP936, CP949, CP950, GB2312, GBK, GB18030, Big5, Shift_JIS, EUC-JP.
102
-
103
- See [all supported encodings on wiki](https://github.com/ashtuchkin/iconv-lite/wiki/Supported-Encodings).
104
-
105
- Most singlebyte encodings are generated automatically from [node-iconv](https://github.com/bnoordhuis/node-iconv). Thank you Ben Noordhuis and libiconv authors!
106
-
107
- Multibyte encodings are generated from [Unicode.org mappings](http://www.unicode.org/Public/MAPPINGS/) and [WHATWG Encoding Standard mappings](http://encoding.spec.whatwg.org/). Thank you, respective authors!
108
-
109
-
110
- ## Encoding/decoding speed
111
-
112
- Comparison with node-iconv module (1000x256kb, on MacBook Pro, Core i5/2.6 GHz, Node v0.12.0).
113
- Note: your results may vary, so please always check on your hardware.
114
-
115
- operation iconv@2.1.4 iconv-lite@0.4.7
116
- ----------------------------------------------------------
117
- encode('win1251') ~96 Mb/s ~320 Mb/s
118
- decode('win1251') ~95 Mb/s ~246 Mb/s
119
-
120
- ## BOM handling
121
-
122
- * Decoding: BOM is stripped by default, unless overridden by passing `stripBOM: false` in options
123
- (f.ex. `iconv.decode(buf, enc, {stripBOM: false})`).
124
- A callback might also be given as a `stripBOM` parameter - it'll be called if BOM character was actually found.
125
- * If you want to detect UTF-8 BOM when decoding other encodings, use [node-autodetect-decoder-stream](https://github.com/danielgindi/node-autodetect-decoder-stream) module.
126
- * Encoding: No BOM added, unless overridden by `addBOM: true` option.
127
-
128
- ## UTF-16 Encodings
129
-
130
- This library supports UTF-16LE, UTF-16BE and UTF-16 encodings. First two are straightforward, but UTF-16 is trying to be
131
- smart about endianness in the following ways:
132
- * Decoding: uses BOM and 'spaces heuristic' to determine input endianness. Default is UTF-16LE, but can be
133
- overridden with `defaultEncoding: 'utf-16be'` option. Strips BOM unless `stripBOM: false`.
134
- * Encoding: uses UTF-16LE and writes BOM by default. Use `addBOM: false` to override.
135
-
136
- ## Other notes
137
-
138
- When decoding, be sure to supply a Buffer to decode() method, otherwise [bad things usually happen](https://github.com/ashtuchkin/iconv-lite/wiki/Use-Buffers-when-decoding).
139
- Untranslatable characters are set to � or ?. No transliteration is currently supported.
140
- Node versions 0.10.31 and 0.11.13 are buggy, don't use them (see #65, #77).
141
-
142
- ## Testing
143
-
144
- ```bash
145
- $ git clone git@github.com:ashtuchkin/iconv-lite.git
146
- $ cd iconv-lite
147
- $ npm install
148
- $ npm test
149
-
150
- $ # To view performance:
151
- $ node test/performance.js
152
-
153
- $ # To view test coverage:
154
- $ npm run coverage
155
- $ open coverage/lcov-report/index.html
156
- ```