browsed 0.2.5 → 0.2.6

Sign up to get free protection for your applications and to get access to all the features.
Files changed (172) hide show
  1. checksums.yaml +4 -4
  2. data/.gitignore +2 -0
  3. data/Gemfile.lock +2 -2
  4. data/README.md +12 -17
  5. data/bin/browse +13 -10
  6. data/lib/browsed.rb +9 -2
  7. data/lib/browsed/browser.rb +82 -0
  8. data/lib/browsed/chrome.rb +21 -12
  9. data/lib/browsed/client.rb +48 -130
  10. data/lib/browsed/constants.rb +7 -1
  11. data/lib/browsed/errors.rb +1 -0
  12. data/lib/browsed/extensions/hash.rb +27 -0
  13. data/lib/browsed/firefox.rb +18 -17
  14. data/lib/browsed/manager.rb +5 -5
  15. data/lib/browsed/{poltergeist.rb → phantomjs.rb} +2 -2
  16. data/lib/browsed/proxies.rb +26 -0
  17. data/lib/browsed/{extensions → proxies}/chrome/proxy_authentication.rb +3 -3
  18. data/lib/browsed/proxies/proxy-chain-server/node_modules/.bin/portastic +1 -0
  19. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/LICENSE +21 -0
  20. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/README.md +57 -0
  21. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/changelog.md +1 -0
  22. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/browser/bluebird.core.js +3824 -0
  23. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/browser/bluebird.core.min.js +31 -0
  24. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/browser/bluebird.js +5666 -0
  25. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/browser/bluebird.min.js +31 -0
  26. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/any.js +21 -0
  27. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/assert.js +55 -0
  28. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/async.js +165 -0
  29. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/bind.js +67 -0
  30. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/bluebird.js +11 -0
  31. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/call_get.js +123 -0
  32. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/cancel.js +129 -0
  33. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/catch_filter.js +42 -0
  34. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/context.js +69 -0
  35. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/debuggability.js +934 -0
  36. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/direct_resolve.js +46 -0
  37. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/each.js +30 -0
  38. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/errors.js +116 -0
  39. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/es5.js +80 -0
  40. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/filter.js +12 -0
  41. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/finally.js +146 -0
  42. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/generators.js +223 -0
  43. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/join.js +168 -0
  44. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/map.js +168 -0
  45. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/method.js +55 -0
  46. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/nodeback.js +51 -0
  47. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/nodeify.js +58 -0
  48. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/promise.js +789 -0
  49. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/promise_array.js +185 -0
  50. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/promisify.js +314 -0
  51. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/props.js +118 -0
  52. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/queue.js +73 -0
  53. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/race.js +49 -0
  54. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/reduce.js +172 -0
  55. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/schedule.js +62 -0
  56. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/settle.js +43 -0
  57. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/some.js +148 -0
  58. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/synchronous_inspection.js +103 -0
  59. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/thenables.js +86 -0
  60. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/timers.js +93 -0
  61. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/using.js +226 -0
  62. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/js/release/util.js +389 -0
  63. data/lib/browsed/proxies/proxy-chain-server/node_modules/bluebird/package.json +102 -0
  64. data/lib/browsed/proxies/proxy-chain-server/node_modules/commander/CHANGELOG.md +408 -0
  65. data/lib/browsed/proxies/proxy-chain-server/node_modules/commander/LICENSE +22 -0
  66. data/lib/browsed/proxies/proxy-chain-server/node_modules/commander/Readme.md +428 -0
  67. data/lib/browsed/proxies/proxy-chain-server/node_modules/commander/index.js +1224 -0
  68. data/lib/browsed/proxies/proxy-chain-server/node_modules/commander/package.json +70 -0
  69. data/lib/browsed/proxies/proxy-chain-server/node_modules/commander/typings/index.d.ts +310 -0
  70. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/.coveralls.yml +1 -0
  71. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/.eslintrc +11 -0
  72. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/.npmignore +9 -0
  73. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/.travis.yml +14 -0
  74. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/CHANGELOG.md +362 -0
  75. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/LICENSE +19 -0
  76. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/Makefile +50 -0
  77. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/README.md +312 -0
  78. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/component.json +19 -0
  79. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/karma.conf.js +70 -0
  80. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/node.js +1 -0
  81. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/package.json +88 -0
  82. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/src/browser.js +185 -0
  83. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/src/debug.js +202 -0
  84. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/src/index.js +10 -0
  85. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/src/inspector-log.js +15 -0
  86. data/lib/browsed/proxies/proxy-chain-server/node_modules/debug/src/node.js +248 -0
  87. data/lib/browsed/proxies/proxy-chain-server/node_modules/ms/index.js +152 -0
  88. data/lib/browsed/proxies/proxy-chain-server/node_modules/ms/license.md +21 -0
  89. data/lib/browsed/proxies/proxy-chain-server/node_modules/ms/package.json +69 -0
  90. data/lib/browsed/proxies/proxy-chain-server/node_modules/ms/readme.md +51 -0
  91. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/LICENSE +13 -0
  92. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/README.md +148 -0
  93. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/bin/portastic +65 -0
  94. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/history.md +7 -0
  95. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/index.js +4 -0
  96. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/lib/monitor.js +59 -0
  97. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/lib/portastic.js +167 -0
  98. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/LICENSE +21 -0
  99. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/README.md +677 -0
  100. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/changelog.md +1730 -0
  101. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/browser/bluebird.js +4892 -0
  102. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/browser/bluebird.min.js +31 -0
  103. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/any.js +21 -0
  104. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/assert.js +55 -0
  105. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/async.js +150 -0
  106. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/bind.js +72 -0
  107. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/bluebird.js +11 -0
  108. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/call_get.js +123 -0
  109. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/cancel.js +48 -0
  110. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/captured_trace.js +493 -0
  111. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/catch_filter.js +66 -0
  112. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/context.js +38 -0
  113. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/debuggability.js +162 -0
  114. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/direct_resolve.js +63 -0
  115. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/each.js +12 -0
  116. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/errors.js +111 -0
  117. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/es5.js +80 -0
  118. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/filter.js +12 -0
  119. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/finally.js +98 -0
  120. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/generators.js +136 -0
  121. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/join.js +107 -0
  122. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/map.js +133 -0
  123. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/method.js +44 -0
  124. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/nodeify.js +59 -0
  125. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/progress.js +76 -0
  126. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/promise.js +759 -0
  127. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/promise_array.js +142 -0
  128. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/promise_resolver.js +123 -0
  129. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/promisify.js +307 -0
  130. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/props.js +79 -0
  131. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/queue.js +90 -0
  132. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/race.js +47 -0
  133. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/reduce.js +148 -0
  134. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/schedule.js +35 -0
  135. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/settle.js +40 -0
  136. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/some.js +125 -0
  137. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/synchronous_inspection.js +94 -0
  138. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/thenables.js +84 -0
  139. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/timers.js +64 -0
  140. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/using.js +213 -0
  141. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/js/main/util.js +321 -0
  142. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/node_modules/bluebird/package.json +103 -0
  143. data/lib/browsed/proxies/proxy-chain-server/node_modules/portastic/package.json +69 -0
  144. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/CHANGELOG.md +109 -0
  145. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/LICENSE +201 -0
  146. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/README.md +278 -0
  147. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/anonymize_proxy.js +100 -0
  148. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/handler_base.js +319 -0
  149. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/handler_custom_response.js +96 -0
  150. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/handler_forward.js +173 -0
  151. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/handler_tunnel_chain.js +132 -0
  152. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/handler_tunnel_direct.js +89 -0
  153. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/handler_tunnel_tcp_chain.js +271 -0
  154. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/index.js +26 -0
  155. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/run_locally.js +66 -0
  156. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/server.js +595 -0
  157. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/tcp_tunnel.js +138 -0
  158. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/testing_tcp_service.js +37 -0
  159. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/build/tools.js +220 -0
  160. data/lib/browsed/proxies/proxy-chain-server/node_modules/proxy-chain/package.json +104 -0
  161. data/lib/browsed/proxies/proxy-chain-server/node_modules/underscore/LICENSE +23 -0
  162. data/lib/browsed/proxies/proxy-chain-server/node_modules/underscore/README.md +28 -0
  163. data/lib/browsed/proxies/proxy-chain-server/node_modules/underscore/package.json +81 -0
  164. data/lib/browsed/proxies/proxy-chain-server/node_modules/underscore/underscore-min.js +5 -0
  165. data/lib/browsed/proxies/proxy-chain-server/node_modules/underscore/underscore-min.js.map +1 -0
  166. data/lib/browsed/proxies/proxy-chain-server/node_modules/underscore/underscore.js +1692 -0
  167. data/lib/browsed/proxies/proxy-chain-server/package-lock.json +63 -0
  168. data/lib/browsed/proxies/proxy-chain-server/package.json +15 -0
  169. data/lib/browsed/proxies/proxy-chain-server/server.js +25 -0
  170. data/lib/browsed/proxies/proxy_chain.rb +36 -0
  171. data/lib/browsed/version.rb +1 -1
  172. metadata +160 -4
@@ -0,0 +1,109 @@
1
+ 0.2.7 / 2018-02-19
2
+ ===================
3
+ - Updated README
4
+
5
+ 0.2.6 / 2018-12-27
6
+ ===================
7
+ - Bugfix: Added `Host` header to `HTTP CONNECT` requests to upstream proxies
8
+
9
+ 0.2.5 / 2018-09-10
10
+ ===================
11
+ - Bugfix: Invalid request headers broke proxy chain connection. Now they will be skipped instead.
12
+
13
+ 0.2.4 / 2018-07-27
14
+ ===================
15
+ - Bugfix: large custom responses were not delivered completely because the socket was closed too early
16
+
17
+ 0.2.3 / 2018-06-21
18
+ ===================
19
+ - Bugfix: 'requestFailed' was emitting `{ request, err }` instead of `{ request, error }`
20
+
21
+ 0.2.2 / 2018-06-19
22
+ ===================
23
+ - BREAKING: The 'requestFailed' event now emits object `{ request, error }` instead of just `error`
24
+
25
+ 0.1.35 / 2018-06-12
26
+ ===================
27
+ - Bugfix: When target URL cannot be parsed instead of crashing, throw RequestError
28
+
29
+ 0.1.34 / 2018-06-08
30
+ ===================
31
+ - Minor improvement: HandlerBase.fail() now supports RequestError
32
+
33
+ 0.1.33 / 2018-06-08
34
+ ===================
35
+ - Renamed `customResponseFunc` to `customResponseFunction` and changed parameters for more clarity
36
+
37
+ 0.1.32 / 2018-06-08
38
+ ===================
39
+ - Added `customResponseFunc` option to `prepareRequestFunction` to support custom response to HTTP requests
40
+
41
+ 0.1.31 / 2018-05-21
42
+ ===================
43
+ - Updated project homepage in package.json
44
+
45
+ 0.1.29 / 2018-04-15
46
+ ===================
47
+ - Fix: anonymizeProxy() now supports upstream proxies with empty password
48
+
49
+ 0.1.28 / 2018-03-27
50
+ ===================
51
+ - Added `createTunnel()` function to create tunnels through HTTP proxies for arbitrary TCP network connections
52
+ (eq. connection to mongodb/sql database through HTTP proxy)
53
+
54
+ 0.1.27 / 2018-03-05
55
+ ===================
56
+ - Better error messages for common network errors
57
+ - Pass headers from target socket in HTTPS tunnel chains
58
+
59
+ 0.1.26 / 2018-02-14
60
+ ===================
61
+ - If connection is denied because of authentication error, optionally "prepareRequestFunction" can provide error message.
62
+
63
+ 0.1.25 / 2018-02-12
64
+ ===================
65
+ - When connection is only through socket, close srcSocket when trgSocket ends
66
+
67
+ 0.1.24 / 2018-02-09
68
+ ===================
69
+ - Fixed incorrect closing of ServerResponse object which caused phantomjs to mark resource requests as errors.
70
+
71
+ 0.1.23 / 2018-02-07
72
+ ===================
73
+ - Fixed missing variable in "Incorrect protocol" error message.
74
+
75
+ 0.1.22 / 2018-02-05
76
+ ===================
77
+ - Renamed project's GitHub organization
78
+
79
+ 0.1.21 / 2018-01-26
80
+ ===================
81
+ - Added Server.getConnectionIds() function
82
+
83
+ 0.1.20 / 2018-01-26
84
+ ===================
85
+ - Fixed "TypeError: The header content contains invalid characters" bug
86
+
87
+ 0.1.19 / 2018-01-25
88
+ ===================
89
+ - fixed uncaught error events, code improved
90
+
91
+ 0.1.18 / 2018-01-25
92
+ ===================
93
+ - fixed a memory leak, improved logging and consolidated code
94
+
95
+ 0.1.17 / 2018-01-23
96
+ ===================
97
+ - added `connectionClosed` event to notify users about closed proxy connections
98
+
99
+ 0.1.16 / 2018-01-09
100
+ ===================
101
+ - added measuring of proxy stats - see `getConnectionStats()` function
102
+
103
+ 0.1.14 / 2017-12-19
104
+ ===================
105
+ - added support for multiple headers with the same name (thx shershennm)
106
+
107
+ 0.0.1 / 2017-11-06
108
+ ===================
109
+ - Project created
@@ -0,0 +1,201 @@
1
+ Apache License
2
+ Version 2.0, January 2004
3
+ http://www.apache.org/licenses/
4
+
5
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
6
+
7
+ 1. Definitions.
8
+
9
+ "License" shall mean the terms and conditions for use, reproduction,
10
+ and distribution as defined by Sections 1 through 9 of this document.
11
+
12
+ "Licensor" shall mean the copyright owner or entity authorized by
13
+ the copyright owner that is granting the License.
14
+
15
+ "Legal Entity" shall mean the union of the acting entity and all
16
+ other entities that control, are controlled by, or are under common
17
+ control with that entity. For the purposes of this definition,
18
+ "control" means (i) the power, direct or indirect, to cause the
19
+ direction or management of such entity, whether by contract or
20
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
21
+ outstanding shares, or (iii) beneficial ownership of such entity.
22
+
23
+ "You" (or "Your") shall mean an individual or Legal Entity
24
+ exercising permissions granted by this License.
25
+
26
+ "Source" form shall mean the preferred form for making modifications,
27
+ including but not limited to software source code, documentation
28
+ source, and configuration files.
29
+
30
+ "Object" form shall mean any form resulting from mechanical
31
+ transformation or translation of a Source form, including but
32
+ not limited to compiled object code, generated documentation,
33
+ and conversions to other media types.
34
+
35
+ "Work" shall mean the work of authorship, whether in Source or
36
+ Object form, made available under the License, as indicated by a
37
+ copyright notice that is included in or attached to the work
38
+ (an example is provided in the Appendix below).
39
+
40
+ "Derivative Works" shall mean any work, whether in Source or Object
41
+ form, that is based on (or derived from) the Work and for which the
42
+ editorial revisions, annotations, elaborations, or other modifications
43
+ represent, as a whole, an original work of authorship. For the purposes
44
+ of this License, Derivative Works shall not include works that remain
45
+ separable from, or merely link (or bind by name) to the interfaces of,
46
+ the Work and Derivative Works thereof.
47
+
48
+ "Contribution" shall mean any work of authorship, including
49
+ the original version of the Work and any modifications or additions
50
+ to that Work or Derivative Works thereof, that is intentionally
51
+ submitted to Licensor for inclusion in the Work by the copyright owner
52
+ or by an individual or Legal Entity authorized to submit on behalf of
53
+ the copyright owner. For the purposes of this definition, "submitted"
54
+ means any form of electronic, verbal, or written communication sent
55
+ to the Licensor or its representatives, including but not limited to
56
+ communication on electronic mailing lists, source code control systems,
57
+ and issue tracking systems that are managed by, or on behalf of, the
58
+ Licensor for the purpose of discussing and improving the Work, but
59
+ excluding communication that is conspicuously marked or otherwise
60
+ designated in writing by the copyright owner as "Not a Contribution."
61
+
62
+ "Contributor" shall mean Licensor and any individual or Legal Entity
63
+ on behalf of whom a Contribution has been received by Licensor and
64
+ subsequently incorporated within the Work.
65
+
66
+ 2. Grant of Copyright License. Subject to the terms and conditions of
67
+ this License, each Contributor hereby grants to You a perpetual,
68
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
69
+ copyright license to reproduce, prepare Derivative Works of,
70
+ publicly display, publicly perform, sublicense, and distribute the
71
+ Work and such Derivative Works in Source or Object form.
72
+
73
+ 3. Grant of Patent License. Subject to the terms and conditions of
74
+ this License, each Contributor hereby grants to You a perpetual,
75
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
76
+ (except as stated in this section) patent license to make, have made,
77
+ use, offer to sell, sell, import, and otherwise transfer the Work,
78
+ where such license applies only to those patent claims licensable
79
+ by such Contributor that are necessarily infringed by their
80
+ Contribution(s) alone or by combination of their Contribution(s)
81
+ with the Work to which such Contribution(s) was submitted. If You
82
+ institute patent litigation against any entity (including a
83
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
84
+ or a Contribution incorporated within the Work constitutes direct
85
+ or contributory patent infringement, then any patent licenses
86
+ granted to You under this License for that Work shall terminate
87
+ as of the date such litigation is filed.
88
+
89
+ 4. Redistribution. You may reproduce and distribute copies of the
90
+ Work or Derivative Works thereof in any medium, with or without
91
+ modifications, and in Source or Object form, provided that You
92
+ meet the following conditions:
93
+
94
+ (a) You must give any other recipients of the Work or
95
+ Derivative Works a copy of this License; and
96
+
97
+ (b) You must cause any modified files to carry prominent notices
98
+ stating that You changed the files; and
99
+
100
+ (c) You must retain, in the Source form of any Derivative Works
101
+ that You distribute, all copyright, patent, trademark, and
102
+ attribution notices from the Source form of the Work,
103
+ excluding those notices that do not pertain to any part of
104
+ the Derivative Works; and
105
+
106
+ (d) If the Work includes a "NOTICE" text file as part of its
107
+ distribution, then any Derivative Works that You distribute must
108
+ include a readable copy of the attribution notices contained
109
+ within such NOTICE file, excluding those notices that do not
110
+ pertain to any part of the Derivative Works, in at least one
111
+ of the following places: within a NOTICE text file distributed
112
+ as part of the Derivative Works; within the Source form or
113
+ documentation, if provided along with the Derivative Works; or,
114
+ within a display generated by the Derivative Works, if and
115
+ wherever such third-party notices normally appear. The contents
116
+ of the NOTICE file are for informational purposes only and
117
+ do not modify the License. You may add Your own attribution
118
+ notices within Derivative Works that You distribute, alongside
119
+ or as an addendum to the NOTICE text from the Work, provided
120
+ that such additional attribution notices cannot be construed
121
+ as modifying the License.
122
+
123
+ You may add Your own copyright statement to Your modifications and
124
+ may provide additional or different license terms and conditions
125
+ for use, reproduction, or distribution of Your modifications, or
126
+ for any such Derivative Works as a whole, provided Your use,
127
+ reproduction, and distribution of the Work otherwise complies with
128
+ the conditions stated in this License.
129
+
130
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
131
+ any Contribution intentionally submitted for inclusion in the Work
132
+ by You to the Licensor shall be under the terms and conditions of
133
+ this License, without any additional terms or conditions.
134
+ Notwithstanding the above, nothing herein shall supersede or modify
135
+ the terms of any separate license agreement you may have executed
136
+ with Licensor regarding such Contributions.
137
+
138
+ 6. Trademarks. This License does not grant permission to use the trade
139
+ names, trademarks, service marks, or product names of the Licensor,
140
+ except as required for reasonable and customary use in describing the
141
+ origin of the Work and reproducing the content of the NOTICE file.
142
+
143
+ 7. Disclaimer of Warranty. Unless required by applicable law or
144
+ agreed to in writing, Licensor provides the Work (and each
145
+ Contributor provides its Contributions) on an "AS IS" BASIS,
146
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
147
+ implied, including, without limitation, any warranties or conditions
148
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
149
+ PARTICULAR PURPOSE. You are solely responsible for determining the
150
+ appropriateness of using or redistributing the Work and assume any
151
+ risks associated with Your exercise of permissions under this License.
152
+
153
+ 8. Limitation of Liability. In no event and under no legal theory,
154
+ whether in tort (including negligence), contract, or otherwise,
155
+ unless required by applicable law (such as deliberate and grossly
156
+ negligent acts) or agreed to in writing, shall any Contributor be
157
+ liable to You for damages, including any direct, indirect, special,
158
+ incidental, or consequential damages of any character arising as a
159
+ result of this License or out of the use or inability to use the
160
+ Work (including but not limited to damages for loss of goodwill,
161
+ work stoppage, computer failure or malfunction, or any and all
162
+ other commercial damages or losses), even if such Contributor
163
+ has been advised of the possibility of such damages.
164
+
165
+ 9. Accepting Warranty or Additional Liability. While redistributing
166
+ the Work or Derivative Works thereof, You may choose to offer,
167
+ and charge a fee for, acceptance of support, warranty, indemnity,
168
+ or other liability obligations and/or rights consistent with this
169
+ License. However, in accepting such obligations, You may act only
170
+ on Your own behalf and on Your sole responsibility, not on behalf
171
+ of any other Contributor, and only if You agree to indemnify,
172
+ defend, and hold each Contributor harmless for any liability
173
+ incurred by, or claims asserted against, such Contributor by reason
174
+ of your accepting any such warranty or additional liability.
175
+
176
+ END OF TERMS AND CONDITIONS
177
+
178
+ APPENDIX: How to apply the Apache License to your work.
179
+
180
+ To apply the Apache License to your work, attach the following
181
+ boilerplate notice, with the fields enclosed by brackets "{}"
182
+ replaced with your own identifying information. (Don't include
183
+ the brackets!) The text should be enclosed in the appropriate
184
+ comment syntax for the file format. We also recommend that a
185
+ file or class name and description of purpose be included on the
186
+ same "printed page" as the copyright notice for easier
187
+ identification within third-party archives.
188
+
189
+ Copyright 2018 Apify Technologies s.r.o.
190
+
191
+ Licensed under the Apache License, Version 2.0 (the "License");
192
+ you may not use this file except in compliance with the License.
193
+ You may obtain a copy of the License at
194
+
195
+ http://www.apache.org/licenses/LICENSE-2.0
196
+
197
+ Unless required by applicable law or agreed to in writing, software
198
+ distributed under the License is distributed on an "AS IS" BASIS,
199
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
200
+ See the License for the specific language governing permissions and
201
+ limitations under the License.
@@ -0,0 +1,278 @@
1
+ # Programmable HTTP proxy server for Node.js
2
+
3
+ [![npm version](https://badge.fury.io/js/proxy-chain.svg)](http://badge.fury.io/js/proxy-chain)
4
+ [![Build Status](https://travis-ci.org/apifytech/proxy-chain.svg)](https://travis-ci.org/apifytech/proxy-chain)
5
+
6
+ Node.js implementation of a proxy server (think Squid) with support for SSL, authentication, upstream proxy chaining,
7
+ custom HTTP responses and measuring traffic statistics.
8
+ The authentication and proxy chaining configuration is defined in code and can be dynamic.
9
+ Note that the proxy server only supports Basic authentication
10
+ (see [Proxy-Authorization](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Proxy-Authorization) for details).
11
+
12
+ For example, this package is useful if you need to use proxies with authentication
13
+ in the headless Chrome web browser, because it doesn't accept proxy URLs such as `http://username:password@proxy.example.com:8080`.
14
+ With this library, you can set up a local proxy server without any password
15
+ that will forward requests to the upstream proxy with password.
16
+ The package is used for this exact purpose by the [Apify web scraping platform](https://www.apify.com).
17
+
18
+ To learn more about the rationale behind this package,
19
+ read [How to make headless Chrome and Puppeteer use a proxy server with authentication](https://medium.com/@jancurn/how-to-make-headless-chrome-and-puppeteer-use-a-proxy-server-with-authentication-249a21a79212).
20
+
21
+
22
+ ## Run a simple HTTP/HTTPS proxy server
23
+
24
+ ```javascript
25
+ const ProxyChain = require('proxy-chain');
26
+
27
+ const server = new ProxyChain.Server({ port: 8000 });
28
+
29
+ server.listen(() => {
30
+ console.log(`Proxy server is listening on port ${8000}`);
31
+ });
32
+ ```
33
+
34
+ ## Run a HTTP/HTTPS proxy server with credentials and upstream proxy
35
+
36
+ ```javascript
37
+ const ProxyChain = require('proxy-chain');
38
+
39
+ const server = new ProxyChain.Server({
40
+ // Port where the server will listen. By default 8000.
41
+ port: 8000,
42
+
43
+ // Enables verbose logging
44
+ verbose: true,
45
+
46
+ // Custom function to authenticate proxy requests and provide the URL to chained upstream proxy.
47
+ // It must return an object (or promise resolving to the object) with the following form:
48
+ // { requestAuthentication: Boolean, upstreamProxyUrl: String }
49
+ // If the function is not defined or is null, the server runs in simple mode.
50
+ // Note that the function takes a single argument with the following properties:
51
+ // * request - An instance of http.IncomingMessage class with information about the client request
52
+ // (which is either HTTP CONNECT for SSL protocol, or other HTTP request)
53
+ // * username - Username parsed from the Proxy-Authorization header. Might be empty string.
54
+ // * password - Password parsed from the Proxy-Authorization header. Might be empty string.
55
+ // * hostname - Hostname of the target server
56
+ // * port - Port of the target server
57
+ // * isHttp - If true, this is a HTTP request, otherwise it's a HTTP CONNECT tunnel for SSL
58
+ // or other protocols
59
+ // * connectionId - Unique ID of the HTTP connection. It can be used to obtain traffic statistics.
60
+ prepareRequestFunction: ({ request, username, password, hostname, port, isHttp, connectionId }) => {
61
+ return {
62
+ // Require clients to authenticate with username 'bob' and password 'TopSecret'
63
+ requestAuthentication: username !== 'bob' || password !== 'TopSecret',
64
+
65
+ // Sets up an upstream HTTP proxy to which all the requests are forwarded.
66
+ // If null, the proxy works in direct mode, i.e. the connection is forwarded directly
67
+ // to the target server.
68
+ upstreamProxyUrl: `http://username:password@proxy.example.com:3128`,
69
+ };
70
+ },
71
+ });
72
+
73
+ server.listen(() => {
74
+ console.log(`Proxy server is listening on port ${server.port}`);
75
+ });
76
+
77
+ // Emitted when HTTP connection is closed
78
+ server.on('connectionClosed', ({ connectionId, stats }) => {
79
+ console.log(`Connection ${connectionId} closed`);
80
+ console.dir(stats);
81
+ });
82
+
83
+ // Emitted when HTTP request fails
84
+ server.on('requestFailed', ({ request, error }) => {
85
+ console.log(`Request ${request.url} failed`);
86
+ console.error(error);
87
+ });
88
+ ```
89
+
90
+ ## Measuring traffic statistics
91
+
92
+ To get traffic statistics for a certain HTTP connection, you can use:
93
+ ```javascript
94
+ const stats = server.getConnectionStats(connectionId);
95
+ console.dir(stats);
96
+ ```
97
+
98
+ The resulting object looks like:
99
+ ```javascript
100
+ {
101
+ srcTxBytes: Number,
102
+ srcRxBytes: Number,
103
+ trgTxBytes: Number,
104
+ trgRxBytes: Number,
105
+ }
106
+ ```
107
+
108
+ ## Custom responses
109
+
110
+ Custom responses allow you to override the response to a HTTP requests to the proxy, without contacting any target host.
111
+ For example, this is useful if you want to provide a HTTP proxy-style interface
112
+ to an external API or respond with some custom page to certain requests.
113
+ Note that this feature is only available for HTTP connections. That's because HTTPS
114
+ connections cannot be intercepted without access to the target host's private key.
115
+
116
+ To provide a custom response, the result of the `prepareRequestFunction` function must
117
+ define the `customResponseFunction` property, which contains a function that generates the custom response.
118
+ The function is passed no parameters and it must return an object (or a promise resolving to an object)
119
+ with the following properties:
120
+
121
+ ```javascript
122
+ {
123
+ // Optional HTTP status code of the response. By default it is 200.
124
+ statusCode: 200,
125
+
126
+ // Optional HTTP headers of the response
127
+ headers: {
128
+ 'X-My-Header': 'bla bla',
129
+ }
130
+
131
+ // Optional string with the body of the HTTP response
132
+ body: 'My custom response',
133
+
134
+ // Optional encoding of the body. If not provided, defaults to 'UTF-8'
135
+ encoding: 'UTF-8',
136
+ }
137
+ ```
138
+
139
+ Here is a simple example:
140
+
141
+ ```javascript
142
+ const ProxyChain = require('proxy-chain');
143
+
144
+ const server = new ProxyChain.Server({
145
+ port: 8000,
146
+ prepareRequestFunction: ({ request, username, password, hostname, port, isHttp }) => {
147
+ return {
148
+ customResponseFunction: () => {
149
+ return {
150
+ statusCode: 200,
151
+ body: `My custom response to ${request.url}`,
152
+ };
153
+ },
154
+ };
155
+ },
156
+ });
157
+
158
+ server.listen(() => {
159
+ console.log(`Proxy server is listening on port ${server.port}`);
160
+ });
161
+ ```
162
+
163
+ ## Closing the server
164
+
165
+ To shut down the proxy server, call the `close([destroyConnections], [callback])` function. For example:
166
+
167
+ ```javascript
168
+ server.close(true, () => {
169
+ console.log('Proxy server was closed.');
170
+ });
171
+ ```
172
+
173
+ The `closeConnections` parameter indicates whether pending proxy connections should be forcibly closed.
174
+ If the `callback` parameter is omitted, the function returns a promise.
175
+
176
+
177
+ ## Helper functions
178
+
179
+ The package also provides several utility functions.
180
+
181
+
182
+ ### `anonymizeProxy(proxyUrl, callback)`
183
+
184
+ Parses and validates a HTTP proxy URL. If the proxy requires authentication,
185
+ then the function starts an open local proxy server that forwards to the proxy.
186
+ The port is chosen randomly.
187
+
188
+ The function takes an optional callback that receives the anonymous proxy URL.
189
+ If no callback is supplied, the function returns a promise that resolves to a String with
190
+ anonymous proxy URL or the original URL if it was already anonymous.
191
+
192
+ The following example shows how you can use a proxy with authentication
193
+ from headless Chrome and [Puppeteer](https://github.com/GoogleChrome/puppeteer).
194
+ For details, read this [blog post](https://blog.apify.com/how-to-make-headless-chrome-and-puppeteer-use-a-proxy-server-with-authentication-249a21a79212).
195
+
196
+ ```javascript
197
+ const puppeteer = require('puppeteer');
198
+ const proxyChain = require('proxy-chain');
199
+
200
+ (async() => {
201
+ const oldProxyUrl = 'http://bob:password123@proxy.example.com:8000';
202
+ const newProxyUrl = await proxyChain.anonymizeProxy(oldProxyUrl);
203
+
204
+ // Prints something like "http://127.0.0.1:45678"
205
+ console.log(newProxyUrl);
206
+
207
+ const browser = await puppeteer.launch({
208
+ args: [`--proxy-server=${newProxyUrl}`],
209
+ });
210
+
211
+ // Do your magic here...
212
+ const page = await browser.newPage();
213
+ await page.goto('https://www.example.com');
214
+ await page.screenshot({ path: 'example.png' });
215
+ await browser.close();
216
+ })();
217
+ ```
218
+
219
+ ### `closeAnonymizedProxy(anonymizedProxyUrl, closeConnections, callback)`
220
+
221
+ Closes anonymous proxy previously started by `anonymizeProxy()`.
222
+ If proxy was not found or was already closed, the function has no effect
223
+ and its result is `false`. Otherwise the result is `true`.
224
+
225
+ The `closeConnections` parameter indicates whether pending proxy connections are forcibly closed.
226
+
227
+ The function takes an optional callback that receives the result Boolean from the function.
228
+ If callback is not provided, the function returns a promise instead.
229
+
230
+ ### `createTunnel(proxyUrl, targetHost, options, callback)`
231
+
232
+ Creates a TCP tunnel to `targetHost` that goes through a HTTP proxy server
233
+ specified by the `proxyUrl` parameter.
234
+
235
+ The result of the function is a local endpoint in a form of `hostname:port`.
236
+ All TCP connections made to the local endpoint will be tunneled through the proxy to the target host and port.
237
+ For example, this is useful if you want to access a certain service from a specific IP address.
238
+
239
+ The tunnel should be eventually closed by calling the `closeTunnel()` function.
240
+
241
+ The `createTunnel()` function accepts an optional Node.js-style callback that receives the path to the local endpoint.
242
+ If no callback is supplied, the function returns a promise that resolves to a String with
243
+ the path to the local endpoint.
244
+
245
+ Example:
246
+
247
+ ```javascript
248
+ const host = await createTunnel('http://bob:pass123@proxy.example.com:8000', 'service.example.com:356');
249
+ // Prints something like "localhost:56836"
250
+ console.log(host);
251
+ ```
252
+
253
+ ### `closeTunnel(tunnelString, closeConnections, callback)`
254
+
255
+ Closes tunnel previously started by `createTunnel()`.
256
+ The result value is `false` if the tunnel was not found or was already closed, otherwise it is `true`.
257
+
258
+ The `closeConnections` parameter indicates whether pending connections are forcibly closed.
259
+
260
+ The function takes an optional callback that receives the result of the function.
261
+ If the callback is not provided, the function returns a promise instead.
262
+
263
+ ### `parseUrl(url)`
264
+
265
+ Calls Node.js's [url.parse](https://nodejs.org/docs/latest/api/url.html#url_url_parse_urlstring_parsequerystring_slashesdenotehost)
266
+ function and extends the resulting object with the following fields: `scheme`, `username` and `password`.
267
+ For example, for `HTTP://bob:pass123@example.com` these values are
268
+ `http`, `bob` and `pass123`, respectively.
269
+
270
+
271
+ ### `redactUrl(url, passwordReplacement)`
272
+
273
+ Takes a URL and hides the password from it. For example:
274
+
275
+ ```javascript
276
+ // Prints 'http://bob:<redacted>@example.com'
277
+ console.log(redactUrl('http://bob:pass123@example.com'));
278
+ ```