ethon-impersonate 0.17.9-arm64-darwin
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +391 -0
- data/LICENSE +20 -0
- data/README.md +116 -0
- data/config/puma.rb +1 -0
- data/ethon-impersonate.gemspec +29 -0
- data/ext/libcurl-impersonate.4.dylib +0 -0
- data/lib/ethon-impersonate.rb +1 -0
- data/lib/ethon_impersonate/curl.rb +90 -0
- data/lib/ethon_impersonate/curls/classes.rb +65 -0
- data/lib/ethon_impersonate/curls/codes.rb +122 -0
- data/lib/ethon_impersonate/curls/constants.rb +81 -0
- data/lib/ethon_impersonate/curls/form_options.rb +37 -0
- data/lib/ethon_impersonate/curls/functions.rb +59 -0
- data/lib/ethon_impersonate/curls/infos.rb +151 -0
- data/lib/ethon_impersonate/curls/messages.rb +19 -0
- data/lib/ethon_impersonate/curls/options.rb +503 -0
- data/lib/ethon_impersonate/curls/settings.rb +13 -0
- data/lib/ethon_impersonate/easy/callbacks.rb +149 -0
- data/lib/ethon_impersonate/easy/debug_info.rb +49 -0
- data/lib/ethon_impersonate/easy/features.rb +31 -0
- data/lib/ethon_impersonate/easy/form.rb +107 -0
- data/lib/ethon_impersonate/easy/header.rb +65 -0
- data/lib/ethon_impersonate/easy/http/actionable.rb +157 -0
- data/lib/ethon_impersonate/easy/http/custom.rb +29 -0
- data/lib/ethon_impersonate/easy/http/delete.rb +25 -0
- data/lib/ethon_impersonate/easy/http/get.rb +24 -0
- data/lib/ethon_impersonate/easy/http/head.rb +24 -0
- data/lib/ethon_impersonate/easy/http/options.rb +24 -0
- data/lib/ethon_impersonate/easy/http/patch.rb +24 -0
- data/lib/ethon_impersonate/easy/http/post.rb +26 -0
- data/lib/ethon_impersonate/easy/http/postable.rb +32 -0
- data/lib/ethon_impersonate/easy/http/put.rb +27 -0
- data/lib/ethon_impersonate/easy/http/putable.rb +25 -0
- data/lib/ethon_impersonate/easy/http.rb +68 -0
- data/lib/ethon_impersonate/easy/informations.rb +118 -0
- data/lib/ethon_impersonate/easy/mirror.rb +38 -0
- data/lib/ethon_impersonate/easy/operations.rb +64 -0
- data/lib/ethon_impersonate/easy/options.rb +50 -0
- data/lib/ethon_impersonate/easy/params.rb +29 -0
- data/lib/ethon_impersonate/easy/queryable.rb +154 -0
- data/lib/ethon_impersonate/easy/response_callbacks.rb +136 -0
- data/lib/ethon_impersonate/easy/util.rb +28 -0
- data/lib/ethon_impersonate/easy.rb +332 -0
- data/lib/ethon_impersonate/errors/ethon_error.rb +9 -0
- data/lib/ethon_impersonate/errors/global_init.rb +13 -0
- data/lib/ethon_impersonate/errors/invalid_option.rb +13 -0
- data/lib/ethon_impersonate/errors/invalid_value.rb +13 -0
- data/lib/ethon_impersonate/errors/multi_add.rb +12 -0
- data/lib/ethon_impersonate/errors/multi_fdset.rb +12 -0
- data/lib/ethon_impersonate/errors/multi_remove.rb +12 -0
- data/lib/ethon_impersonate/errors/multi_timeout.rb +13 -0
- data/lib/ethon_impersonate/errors/select.rb +13 -0
- data/lib/ethon_impersonate/errors.rb +17 -0
- data/lib/ethon_impersonate/impersonate/fingerprints.rb +16 -0
- data/lib/ethon_impersonate/impersonate/settings.rb +85 -0
- data/lib/ethon_impersonate/impersonate/targets.rb +99 -0
- data/lib/ethon_impersonate/impersonate/tls.rb +189 -0
- data/lib/ethon_impersonate/impersonate.rb +10 -0
- data/lib/ethon_impersonate/libc.rb +21 -0
- data/lib/ethon_impersonate/loggable.rb +59 -0
- data/lib/ethon_impersonate/multi/operations.rb +228 -0
- data/lib/ethon_impersonate/multi/options.rb +117 -0
- data/lib/ethon_impersonate/multi/stack.rb +49 -0
- data/lib/ethon_impersonate/multi.rb +126 -0
- data/lib/ethon_impersonate/version.rb +6 -0
- data/lib/ethon_impersonate.rb +28 -0
- metadata +122 -0
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA256:
|
3
|
+
metadata.gz: aa425a488d2af5a54a43710e282583b8d3d14a25f5e83f885a9c77ce2418598e
|
4
|
+
data.tar.gz: ba23385a332a35b4b5c24b623a8c4ae39a622c46d1077763af8959d22c2c05bb
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: c95260a7c32280a4fea21ba520f4efd26f7e4df5c5238aabedcc3cdb95391a5991b4dce5758104b067dbfd8b2c57d87d72dfb65e218abcfb576460507701a64a
|
7
|
+
data.tar.gz: 54c73aa2d644740191a1e4ba8ff625d515d938810e7f1b089bf42faaab5ef414028564ae9854dc04fc6efe5ac47918af98762954fbc165df8d42e3d22ca1b9cf
|
data/CHANGELOG.md
ADDED
@@ -0,0 +1,391 @@
|
|
1
|
+
# Changelog
|
2
|
+
|
3
|
+
## Master
|
4
|
+
|
5
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.17.9...master)
|
6
|
+
|
7
|
+
## 0.17.9
|
8
|
+
|
9
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.17.8...v0.17.9)
|
10
|
+
|
11
|
+
* Bumped [`curl-impersonate`](https://github.com/lexiforest/curl-impersonate) to `v1.0.0`
|
12
|
+
|
13
|
+
## 0.17.8
|
14
|
+
|
15
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.16.0...v0.17.8)
|
16
|
+
|
17
|
+
* Added basic support for [`curl-impersonate`](https://github.com/lexiforest/curl-impersonate) @ `v1.0.0rc1`
|
18
|
+
|
19
|
+
## 0.16.0
|
20
|
+
|
21
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.15.0...v0.16.0)
|
22
|
+
|
23
|
+
* Added `redirect_url` value to available informations and `Easy::Mirror`.
|
24
|
+
([Adrien Rey-Jarthon](https://github.com/jarthod)
|
25
|
+
|
26
|
+
## 0.15.0
|
27
|
+
|
28
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.14.0...v0.15.0)
|
29
|
+
|
30
|
+
## 0.12.0
|
31
|
+
|
32
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.11.0...v0.12.0)
|
33
|
+
|
34
|
+
- Removed deprecated `CURLE_SSL_CACERT` pinned in curl v7.62.0 ([@arku](https://github.com/arku) in [#158](https://github.com/dsojevic/ethon-impersonate/pull/158))
|
35
|
+
|
36
|
+
|
37
|
+
## 0.11.0
|
38
|
+
|
39
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.10.1...v0.11.0)
|
40
|
+
|
41
|
+
## 0.10.1
|
42
|
+
|
43
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.10.0...v0.10.1)
|
44
|
+
|
45
|
+
## 0.10.0
|
46
|
+
|
47
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.9.1...v0.10.0)
|
48
|
+
|
49
|
+
## 0.9.1
|
50
|
+
|
51
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.9.0...v0.9.1)
|
52
|
+
|
53
|
+
## 0.9.0
|
54
|
+
|
55
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.8.1...v0.9.0)
|
56
|
+
|
57
|
+
## 0.8.1
|
58
|
+
|
59
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.8.0...v0.8.1)
|
60
|
+
|
61
|
+
* Support optional escaping of params.
|
62
|
+
([Tasos Laskos](https://github.com/zapotek)
|
63
|
+
* `Easy::Mirror`: Reduced object allocations and method calls during info handling.
|
64
|
+
([Tasos Laskos](https://github.com/zapotek)
|
65
|
+
|
66
|
+
## 0.8.0
|
67
|
+
|
68
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.7.3...v0.7.4)
|
69
|
+
|
70
|
+
* `Easy::Mirror`: Reduced object allocations and method calls during info handling.
|
71
|
+
([Tasos Laskos](https://github.com/zapotek)
|
72
|
+
|
73
|
+
## 0.7.4
|
74
|
+
|
75
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.7.3...v0.7.4)
|
76
|
+
|
77
|
+
* Support different array encodings for params.
|
78
|
+
([Marcello Barnaba](https://github.com/ifad), [\#104](https://github.com/dsojevic/ethon-impersonate/pull/104))
|
79
|
+
* Programtic access to version infos.
|
80
|
+
([Jonas Wagner](https://github.com/jwagner), [\#90](https://github.com/dsojevic/ethon-impersonate/pull/90))
|
81
|
+
|
82
|
+
|
83
|
+
## 0.7.3
|
84
|
+
|
85
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.7.2...v0.7.3)
|
86
|
+
|
87
|
+
* `EthonImpersonate::Curl::FDSet`
|
88
|
+
* Set `:fd_array` size to the current MS Windows `FD_SETSIZE` (2048).
|
89
|
+
([Tasos Laskos](https://github.com/zapotek)
|
90
|
+
|
91
|
+
* Added `redirect_time` value to available informations and `Easy::Mirror`.
|
92
|
+
([Adrien Jarthon](https://github.com/jarthod)
|
93
|
+
|
94
|
+
## 0.7.2
|
95
|
+
|
96
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.7.1...v0.7.2)
|
97
|
+
|
98
|
+
* FFI data-types updated to be more correct.
|
99
|
+
|
100
|
+
## 0.7.1
|
101
|
+
|
102
|
+
* MS Windows determination delegated to `Gem.windows?` for better accuracy.
|
103
|
+
* FFI data-types updated to work on MS Windows.
|
104
|
+
|
105
|
+
## 0.7.0
|
106
|
+
|
107
|
+
Not backwards compatible changes:
|
108
|
+
|
109
|
+
* `mime-types` are no longer a dependency. The gem will be still used if available to determine the mime type of a file which is uploaded. That means you have to have take care of the gem installation yourself.
|
110
|
+
|
111
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.6.3...v0.7.0)
|
112
|
+
|
113
|
+
## 0.6.3
|
114
|
+
|
115
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.6.2...v0.6.3)
|
116
|
+
|
117
|
+
## 0.6.2
|
118
|
+
|
119
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.6.1...v0.6.2)
|
120
|
+
|
121
|
+
## 0.6.1
|
122
|
+
|
123
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.6.0...v0.6.1)
|
124
|
+
|
125
|
+
The changelog entries are coming soon!
|
126
|
+
|
127
|
+
## 0.6.0
|
128
|
+
|
129
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.12...v0.6.0)
|
130
|
+
|
131
|
+
The changelog entries are coming soon!
|
132
|
+
|
133
|
+
Bugfixes:
|
134
|
+
|
135
|
+
* URL-encode nullbytes in parameters instead of escaping them to `\\0`.
|
136
|
+
([Tasos Laskos](https://github.com/zapotek)
|
137
|
+
|
138
|
+
## 0.5.12
|
139
|
+
|
140
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.11...v0.5.12)
|
141
|
+
|
142
|
+
Enhancements:
|
143
|
+
|
144
|
+
* Performance optimizations.
|
145
|
+
([Kyle Oppenheim](https://github.com/koppenheim) and [Richie Vos](https://github.com/richievos), [\#48](https://github.com/dsojevic/ethon-impersonate/pull/48))
|
146
|
+
* Reuse memory pointer.
|
147
|
+
([Richie Vos](https://github.com/richievos), [\#49](https://github.com/dsojevic/ethon-impersonate/pull/49))
|
148
|
+
|
149
|
+
Bugfixes:
|
150
|
+
|
151
|
+
* Fix windows install.
|
152
|
+
([Derik Olsson](https://github.com/derikolsson), [\#47](https://github.com/dsojevic/ethon-impersonate/pull/47))
|
153
|
+
* Handle urls that already contain query params.
|
154
|
+
([Turner King](https://github.com/turnerking ), [\#45](https://github.com/dsojevic/ethon-impersonate/pull/45))
|
155
|
+
|
156
|
+
## 0.5.11
|
157
|
+
|
158
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.10...v0.5.11)
|
159
|
+
|
160
|
+
Enhancements:
|
161
|
+
|
162
|
+
* Add support for postredirs, unrestricted_auth.
|
163
|
+
* Add support for cookie, cookiejar, cookiefile.
|
164
|
+
([erwanlr](https://github.com/erwanlr), [\#46](https://github.com/dsojevic/ethon-impersonate/pull/46))
|
165
|
+
* Relax ffi requirements.
|
166
|
+
([voxik](https://github.com/voxik), [\#40](https://github.com/dsojevic/ethon-impersonate/pull/40))
|
167
|
+
* Various documentation improvements.
|
168
|
+
([Craig Little](https://github.com/craiglittle))
|
169
|
+
|
170
|
+
Bugfixes:
|
171
|
+
|
172
|
+
* Fix the memory leaks.
|
173
|
+
([Richie Vos](https://github.com/richievos), [\#45](https://github.com/dsojevic/ethon-impersonate/pull/45))
|
174
|
+
|
175
|
+
## 0.5.10
|
176
|
+
|
177
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.9...v0.5.10)
|
178
|
+
|
179
|
+
Enhancements:
|
180
|
+
|
181
|
+
* Allow custom requests.
|
182
|
+
([Nathan Sutton](https://github.com/nate), [\#36](https://github.com/dsojevic/ethon-impersonate/pull/36))
|
183
|
+
* Use updated version of FFI.
|
184
|
+
|
185
|
+
Bugfixes:
|
186
|
+
|
187
|
+
* Fix windows install issue.
|
188
|
+
([brainsucker](https://github.com/brainsucker), [\#38](https://github.com/dsojevic/ethon-impersonate/pull/38))
|
189
|
+
|
190
|
+
## 0.5.9
|
191
|
+
|
192
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.8...v0.5.9)
|
193
|
+
|
194
|
+
Enhancements:
|
195
|
+
|
196
|
+
* Allow to set multiple protocols.
|
197
|
+
|
198
|
+
## 0.5.8
|
199
|
+
|
200
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.7...v0.5.8)
|
201
|
+
|
202
|
+
Enhancements:
|
203
|
+
|
204
|
+
* Add support for protocols and redir_protocols(
|
205
|
+
[libcurl SASL buffer overflow vulnerability](http://curl.haxx.se/docs/adv_20130206.html)).
|
206
|
+
* Add max_send_speed_large and max_recv_speed_large([Paul Schuegraf](https://github.com/pschuegr), [\#33](https://github.com/dsojevic/ethon-impersonate/pull/33))
|
207
|
+
|
208
|
+
## 0.5.7
|
209
|
+
|
210
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.6...v0.5.7)
|
211
|
+
|
212
|
+
Enhancements:
|
213
|
+
|
214
|
+
* Use new version of ffi.
|
215
|
+
|
216
|
+
## 0.5.6
|
217
|
+
|
218
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.4...v0.5.6)
|
219
|
+
|
220
|
+
Bugfixes:
|
221
|
+
|
222
|
+
* Easy#reset resets on_complete callbacks.
|
223
|
+
|
224
|
+
## 0.5.4
|
225
|
+
|
226
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.3...v0.5.4)
|
227
|
+
|
228
|
+
Enhancements:
|
229
|
+
|
230
|
+
* Use Libc#getdtablesize to get the FDSet size.
|
231
|
+
* New libcurl option accept_encoding.
|
232
|
+
* Documentation updates.
|
233
|
+
|
234
|
+
## 0.5.3
|
235
|
+
|
236
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.2...v0.5.3)
|
237
|
+
|
238
|
+
Enhancements:
|
239
|
+
|
240
|
+
* Deprecate Easy#prepare. It is no longer necessary.
|
241
|
+
* Unroll metaprogramming for easy and multi options.
|
242
|
+
* More specs.
|
243
|
+
|
244
|
+
Bugfixes:
|
245
|
+
|
246
|
+
* Correct size for FDSets
|
247
|
+
* Add proxytypes to enums.
|
248
|
+
|
249
|
+
## 0.5.2
|
250
|
+
|
251
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.1...v0.5.2)
|
252
|
+
|
253
|
+
Enhancements:
|
254
|
+
|
255
|
+
* New libcurl option keypasswd.
|
256
|
+
|
257
|
+
Bugfixes:
|
258
|
+
|
259
|
+
* Correct request logging when using multi interface.
|
260
|
+
* Remove invalid libcurl option sslcertpasswd.
|
261
|
+
|
262
|
+
## 0.5.1
|
263
|
+
|
264
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.5.0...v0.5.1)
|
265
|
+
|
266
|
+
Bugfixes:
|
267
|
+
|
268
|
+
* Mark Curl.select and Curl.easy_perform as blocking so that the GIL is
|
269
|
+
released by ffi.
|
270
|
+
|
271
|
+
## 0.5.0
|
272
|
+
|
273
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.4.4...v0.5.0)
|
274
|
+
|
275
|
+
Enhancements:
|
276
|
+
|
277
|
+
* New libcurl option proxyuserpwd
|
278
|
+
* Rename response_header to response_headers
|
279
|
+
|
280
|
+
Bugfixes:
|
281
|
+
|
282
|
+
* Mark Curl.select and Curl.easy_perform as blocking so that the GIL is
|
283
|
+
released by ffi.
|
284
|
+
|
285
|
+
## 0.4.4
|
286
|
+
|
287
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.4.3...v0.4.4)
|
288
|
+
|
289
|
+
Enhancements:
|
290
|
+
|
291
|
+
* Prepare multi explicit like easy
|
292
|
+
|
293
|
+
## 0.4.3
|
294
|
+
|
295
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.4.2...v0.4.3)
|
296
|
+
|
297
|
+
Enhancements:
|
298
|
+
|
299
|
+
* Remove deprecated libcurl option put
|
300
|
+
* More documentation
|
301
|
+
* New libcurl option connecttimeout_ms and timeout_ms
|
302
|
+
* Support multi options
|
303
|
+
|
304
|
+
Bugfixes:
|
305
|
+
|
306
|
+
* Handle nil values in query params
|
307
|
+
|
308
|
+
## 0.4.2
|
309
|
+
|
310
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.4.1...v0.4.2)
|
311
|
+
|
312
|
+
Enhancements:
|
313
|
+
|
314
|
+
* New libcurl option forbid_reuse
|
315
|
+
* Use libcurls escape instead of CGI::escape
|
316
|
+
|
317
|
+
## 0.4.1
|
318
|
+
|
319
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.4.0...v0.4.1)
|
320
|
+
|
321
|
+
Bugfixes:
|
322
|
+
|
323
|
+
* Handle nested hash in an array in params correct
|
324
|
+
( [\#201](https://github.com/typhoeus/typhoeus/issues/201) )
|
325
|
+
|
326
|
+
## 0.4.0
|
327
|
+
|
328
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.3.0...v0.4.0)
|
329
|
+
|
330
|
+
Enhancements:
|
331
|
+
|
332
|
+
* ruby 1.8.7 compatible
|
333
|
+
* EthonImpersonate.logger
|
334
|
+
* Deal with string param/body
|
335
|
+
* More documentation
|
336
|
+
|
337
|
+
Bugfixes:
|
338
|
+
|
339
|
+
* Add multi_cleanup to curl
|
340
|
+
|
341
|
+
## 0.3.0
|
342
|
+
|
343
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.2.0...v0.3.0)
|
344
|
+
|
345
|
+
Enhancements:
|
346
|
+
|
347
|
+
* New libcurl option proxyport
|
348
|
+
* Raise invalid value error when providing a wrong key for sslversion or httpauth
|
349
|
+
|
350
|
+
Bugfixes:
|
351
|
+
|
352
|
+
* Libcurl option sslversion is handled correct
|
353
|
+
|
354
|
+
## 0.2.0
|
355
|
+
|
356
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.1.0...v0.2.0)
|
357
|
+
|
358
|
+
Enhancements:
|
359
|
+
|
360
|
+
* GET requests are using custom requests only when there is a request body
|
361
|
+
* Easy#on_complete takes multiple callbacks
|
362
|
+
* raise Errors::GlobalInit when libcurls global_init failed instead of
|
363
|
+
runtime error
|
364
|
+
* raise Errors::InvalidOption if option is invalid
|
365
|
+
|
366
|
+
## 0.1.0
|
367
|
+
|
368
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.0.2...v0.1.0)
|
369
|
+
|
370
|
+
Enhancements:
|
371
|
+
|
372
|
+
* Documentation
|
373
|
+
( [Alex P](https://github.com/ifesdjeen), [\#13](https://github.com/dsojevic/ethon-impersonate/issues/13) )
|
374
|
+
* New libcurl option dns_cache_timeout
|
375
|
+
( [Chris Heald](https://github.com/cheald), [\#192](https://github.com/typhoeus/typhoeus/pull/192) )
|
376
|
+
|
377
|
+
Bugfixes:
|
378
|
+
|
379
|
+
* Libcurl option ssl_verifyhost takes an integer.
|
380
|
+
* Add space between header key and value.
|
381
|
+
|
382
|
+
## 0.0.2
|
383
|
+
|
384
|
+
[Full Changelog](https://github.com/dsojevic/ethon-impersonate/compare/v0.0.1...v0.0.2)
|
385
|
+
|
386
|
+
Bugfixes:
|
387
|
+
|
388
|
+
* Add libcurl.so.4 to ffi_lib in order to load correct lib on Debian.
|
389
|
+
* Escape zero bytes.
|
390
|
+
|
391
|
+
## 0.0.1 Initial version
|
data/LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2012-2016 Hans Hasselberg
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
"Software"), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
17
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
18
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
19
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
20
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,116 @@
|
|
1
|
+
[](https://badge.fury.io/rb/ethon-impersonate)
|
2
|
+
[](https://github.com/dsojevic/ethon-impersonate/actions/workflows/ruby.yml)
|
3
|
+
|
4
|
+
# Ethon Impersonate
|
5
|
+
|
6
|
+
In Greek mythology, Ethon, the son of Typhoeus and Echidna, is a gigantic eagle. So much for the history.
|
7
|
+
In the modern world, Ethon is a very basic libcurl wrapper using ffi.
|
8
|
+
|
9
|
+
## Installation
|
10
|
+
|
11
|
+
With bundler:
|
12
|
+
```bash
|
13
|
+
gem "ethon-impersonate"
|
14
|
+
```
|
15
|
+
|
16
|
+
With rubygems:
|
17
|
+
```bash
|
18
|
+
gem install ethon-impersonate
|
19
|
+
```
|
20
|
+
|
21
|
+
## Usage
|
22
|
+
|
23
|
+
Making the first request is simple:
|
24
|
+
|
25
|
+
```ruby
|
26
|
+
easy = EthonImpersonate::Easy.new(url: "www.example.com")
|
27
|
+
easy.perform
|
28
|
+
#=> :ok
|
29
|
+
```
|
30
|
+
|
31
|
+
You have access to various options, such as following redirects:
|
32
|
+
|
33
|
+
```ruby
|
34
|
+
easy = EthonImpersonate::Easy.new(url: "www.example.com", followlocation: true)
|
35
|
+
easy.perform
|
36
|
+
#=> :ok
|
37
|
+
```
|
38
|
+
|
39
|
+
Once you're done you can inspect the response code and body:
|
40
|
+
|
41
|
+
```ruby
|
42
|
+
easy = EthonImpersonate::Easy.new(url: "www.example.com", followlocation: true)
|
43
|
+
easy.perform
|
44
|
+
easy.response_code
|
45
|
+
#=> 200
|
46
|
+
easy.response_body
|
47
|
+
#=> "<!doctype html><html ..."
|
48
|
+
```
|
49
|
+
|
50
|
+
## Http
|
51
|
+
|
52
|
+
In order to make life easier, there are some helpers for making HTTP requests:
|
53
|
+
|
54
|
+
```ruby
|
55
|
+
easy = EthonImpersonate::Easy.new
|
56
|
+
easy.http_request("www.example.com", :get, { params: {a: 1} })
|
57
|
+
easy.perform
|
58
|
+
#=> :ok
|
59
|
+
```
|
60
|
+
|
61
|
+
```ruby
|
62
|
+
easy = EthonImpersonate::Easy.new
|
63
|
+
easy.http_request("www.example.com", :post, { params: { a: 1 }, body: { b: 2 } })
|
64
|
+
easy.perform
|
65
|
+
#=> :ok
|
66
|
+
```
|
67
|
+
|
68
|
+
This is really handy when making requests since you don't have to care about setting
|
69
|
+
everything up correctly.
|
70
|
+
|
71
|
+
## Http2
|
72
|
+
Standard http2 servers require the client to connect once and create a session (multi) and then add simple requests to the multi handler.
|
73
|
+
The `perform` method then takes all the requests in the multi handler and sends them to the server.
|
74
|
+
|
75
|
+
See the following example
|
76
|
+
```ruby
|
77
|
+
multi = EthonImpersonate::Multi.new
|
78
|
+
easy = EthonImpersonate::Easy.new
|
79
|
+
|
80
|
+
easy.http_request("www.example.com/get", :get, { http_version: :httpv2_0 })
|
81
|
+
|
82
|
+
# Sending a request with http version 2 will send an Upgrade header to the server, which many older servers will not support
|
83
|
+
# See below for more info: https://everything.curl.dev/http/http2
|
84
|
+
# If this is a problem, send the below:
|
85
|
+
easy.http_request("www.example.com/get", :get, { http_version: :httpv2_prior_knowledge })
|
86
|
+
|
87
|
+
# To set the server to use http2 with https and http1 with http, send the following:
|
88
|
+
easy.http_request("www.example.com/get", :get, { http_version: :httpv2_tls })
|
89
|
+
|
90
|
+
multi.add(easy)
|
91
|
+
multi.perform
|
92
|
+
```
|
93
|
+
|
94
|
+
## LICENSE
|
95
|
+
|
96
|
+
(The MIT License)
|
97
|
+
|
98
|
+
Copyright © 2012-2016 [Hans Hasselberg](http://www.hans.io)
|
99
|
+
|
100
|
+
Permission is hereby granted, free of charge, to any person obtaining a
|
101
|
+
copy of this software and associated documentation files (the "Software"),
|
102
|
+
to deal in the Software without restriction, including without
|
103
|
+
limitation the rights to use, copy, modify, merge, publish, distribute,
|
104
|
+
sublicense, and/or sell copies of the Software, and to permit persons
|
105
|
+
to whom the Software is furnished to do so, subject to the following conditions:
|
106
|
+
|
107
|
+
The above copyright notice and this permission notice shall be included
|
108
|
+
in all copies or substantial portions of the Software.
|
109
|
+
|
110
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
|
111
|
+
OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
112
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
|
113
|
+
THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR
|
114
|
+
OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
|
115
|
+
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
|
116
|
+
OTHER DEALINGS IN THE SOFTWARE.
|
data/config/puma.rb
ADDED
@@ -0,0 +1 @@
|
|
1
|
+
ENV['OBJC_DISABLE_INITIALIZE_FORK_SAFETY'] = 'YES'
|
@@ -0,0 +1,29 @@
|
|
1
|
+
# encoding: utf-8
|
2
|
+
# frozen_string_literal: true
|
3
|
+
lib = File.expand_path("../lib/", __FILE__)
|
4
|
+
$:.unshift lib unless $:.include?(lib)
|
5
|
+
|
6
|
+
require "ethon_impersonate/version"
|
7
|
+
|
8
|
+
Gem::Specification.new do |s|
|
9
|
+
s.name = "ethon-impersonate"
|
10
|
+
s.version = EthonImpersonate::VERSION
|
11
|
+
s.platform = Gem::Platform::RUBY
|
12
|
+
s.authors = ["David Sojevic", "Hans Hasselberg"]
|
13
|
+
s.email = ["david@sojevic.com"]
|
14
|
+
s.homepage = "https://github.com/dsojevic/ethon-impersonate"
|
15
|
+
s.summary = "Impersonate browser-like HTTP requests."
|
16
|
+
s.description = "Realistic browser-like HTTP requests using curl-impersonate's impersonation capabilities."
|
17
|
+
|
18
|
+
s.required_ruby_version = ">= 2.7"
|
19
|
+
s.required_rubygems_version = ">= 1.3.6"
|
20
|
+
s.license = "MIT"
|
21
|
+
|
22
|
+
s.add_dependency("ffi", ["~> 1.15"])
|
23
|
+
|
24
|
+
s.files = Dir.chdir(__dir__) do
|
25
|
+
`git ls-files -z`.split("\x0")
|
26
|
+
end.reject { |file| file.start_with?(*%w[. Gemfile Guardfile Rakefile profile spec scripts]) || file.end_with?(".gitignore") }
|
27
|
+
|
28
|
+
s.require_path = "lib"
|
29
|
+
end
|
Binary file
|
@@ -0,0 +1 @@
|
|
1
|
+
require "ethon_impersonate"
|
@@ -0,0 +1,90 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
require 'ethon_impersonate/curls/codes'
|
3
|
+
require 'ethon_impersonate/curls/options'
|
4
|
+
require 'ethon_impersonate/curls/infos'
|
5
|
+
require 'ethon_impersonate/curls/form_options'
|
6
|
+
require 'ethon_impersonate/curls/messages'
|
7
|
+
require 'ethon_impersonate/curls/functions'
|
8
|
+
|
9
|
+
module EthonImpersonate
|
10
|
+
|
11
|
+
# FFI Wrapper module for Curl. Holds constants and required initializers.
|
12
|
+
#
|
13
|
+
# @api private
|
14
|
+
module Curl
|
15
|
+
extend ::FFI::Library
|
16
|
+
extend EthonImpersonate::Curls::Codes
|
17
|
+
extend EthonImpersonate::Curls::Options
|
18
|
+
extend EthonImpersonate::Curls::Infos
|
19
|
+
extend EthonImpersonate::Curls::FormOptions
|
20
|
+
extend EthonImpersonate::Curls::Messages
|
21
|
+
|
22
|
+
# :nodoc:
|
23
|
+
def self.windows?
|
24
|
+
Libc.windows?
|
25
|
+
end
|
26
|
+
|
27
|
+
require 'ethon_impersonate/curls/constants'
|
28
|
+
require 'ethon_impersonate/curls/settings'
|
29
|
+
require 'ethon_impersonate/curls/classes'
|
30
|
+
extend EthonImpersonate::Curls::Functions
|
31
|
+
|
32
|
+
@blocking = true
|
33
|
+
|
34
|
+
@@initialized = false
|
35
|
+
@@curl_mutex = Mutex.new
|
36
|
+
|
37
|
+
class << self
|
38
|
+
# This function sets up the program environment that libcurl needs.
|
39
|
+
# Think of it as an extension of the library loader.
|
40
|
+
#
|
41
|
+
# This function must be called at least once within a program (a program is all the
|
42
|
+
# code that shares a memory space) before the program calls any other function in libcurl.
|
43
|
+
# The environment it sets up is constant for the life of the program and is the same for
|
44
|
+
# every program, so multiple calls have the same effect as one call.
|
45
|
+
#
|
46
|
+
# The flags option is a bit pattern that tells libcurl exactly what features to init,
|
47
|
+
# as described below. Set the desired bits by ORing the values together. In normal
|
48
|
+
# operation, you must specify CURL_GLOBAL_ALL. Don't use any other value unless
|
49
|
+
# you are familiar with it and mean to control internal operations of libcurl.
|
50
|
+
#
|
51
|
+
# This function is not thread safe. You must not call it when any other thread in
|
52
|
+
# the program (i.e. a thread sharing the same memory) is running. This doesn't just
|
53
|
+
# mean no other thread that is using libcurl. Because curl_global_init() calls
|
54
|
+
# functions of other libraries that are similarly thread unsafe, it could conflict with
|
55
|
+
# any other thread that uses these other libraries.
|
56
|
+
#
|
57
|
+
# @raise [ EthonImpersonate::Errors::GlobalInit ] If Curl.global_init fails.
|
58
|
+
def init
|
59
|
+
@@curl_mutex.synchronize {
|
60
|
+
if not @@initialized
|
61
|
+
raise Errors::GlobalInit.new if Curl.global_init(GLOBAL_ALL) != 0
|
62
|
+
@@initialized = true
|
63
|
+
EthonImpersonate.logger.debug("ETHON: Libcurl initialized") if EthonImpersonate.logger
|
64
|
+
end
|
65
|
+
}
|
66
|
+
end
|
67
|
+
|
68
|
+
# This function releases resources acquired by curl_global_init.
|
69
|
+
# You should call curl_global_cleanup once for each call you make to
|
70
|
+
# curl_global_init, after you are done using libcurl.
|
71
|
+
# This function is not thread safe. You must not call it when any other thread in the
|
72
|
+
# program (i.e. a thread sharing the same memory) is running. This doesn't just
|
73
|
+
# mean no other thread that is using libcurl. Because curl_global_cleanup calls functions of other
|
74
|
+
# libraries that are similarly thread unsafe, it could conflict with
|
75
|
+
# any other thread that uses these other libraries.
|
76
|
+
# See the description in libcurl of global environment requirements
|
77
|
+
# for details of how to use this function.
|
78
|
+
def cleanup
|
79
|
+
@@curl_mutex.synchronize {
|
80
|
+
if @@initialized
|
81
|
+
Curl.global_cleanup()
|
82
|
+
@@initialized = false
|
83
|
+
EthonImpersonate.logger.debug("ETHON: Libcurl cleanup") if EthonImpersonate.logger
|
84
|
+
end
|
85
|
+
}
|
86
|
+
end
|
87
|
+
|
88
|
+
end
|
89
|
+
end
|
90
|
+
end
|