ferrum 0.13 → 0.15
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/LICENSE +1 -1
- data/README.md +288 -154
- data/lib/ferrum/browser/command.rb +8 -0
- data/lib/ferrum/browser/options/chrome.rb +17 -5
- data/lib/ferrum/browser/options.rb +38 -25
- data/lib/ferrum/browser/process.rb +44 -17
- data/lib/ferrum/browser.rb +34 -52
- data/lib/ferrum/client/subscriber.rb +76 -0
- data/lib/ferrum/{browser → client}/web_socket.rb +36 -22
- data/lib/ferrum/client.rb +169 -0
- data/lib/ferrum/context.rb +19 -15
- data/lib/ferrum/contexts.rb +46 -12
- data/lib/ferrum/cookies/cookie.rb +57 -0
- data/lib/ferrum/cookies.rb +40 -4
- data/lib/ferrum/downloads.rb +60 -0
- data/lib/ferrum/errors.rb +2 -1
- data/lib/ferrum/frame.rb +1 -0
- data/lib/ferrum/headers.rb +1 -1
- data/lib/ferrum/network/exchange.rb +29 -2
- data/lib/ferrum/network/intercepted_request.rb +8 -17
- data/lib/ferrum/network/request.rb +23 -39
- data/lib/ferrum/network/request_params.rb +57 -0
- data/lib/ferrum/network/response.rb +25 -5
- data/lib/ferrum/network.rb +43 -16
- data/lib/ferrum/node.rb +21 -1
- data/lib/ferrum/page/frames.rb +5 -5
- data/lib/ferrum/page/screenshot.rb +42 -24
- data/lib/ferrum/page.rb +183 -131
- data/lib/ferrum/proxy.rb +1 -1
- data/lib/ferrum/target.rb +25 -5
- data/lib/ferrum/utils/elapsed_time.rb +0 -2
- data/lib/ferrum/utils/event.rb +19 -0
- data/lib/ferrum/utils/platform.rb +4 -0
- data/lib/ferrum/utils/thread.rb +18 -0
- data/lib/ferrum/version.rb +1 -1
- data/lib/ferrum.rb +3 -0
- metadata +14 -114
- data/lib/ferrum/browser/client.rb +0 -102
- data/lib/ferrum/browser/subscriber.rb +0 -36
data/README.md
CHANGED
@@ -25,6 +25,8 @@ going to crawl sites you better use Ferrum or
|
|
25
25
|
* [Vessel](https://github.com/rubycdp/vessel) high-level web crawling framework
|
26
26
|
based on Ferrum and Mechanize.
|
27
27
|
|
28
|
+
The development is done in [![RubyMine](https://resources.jetbrains.com/storage/products/company/brand/logos/RubyMine_icon.svg?width=10px)](https://jb.gg/ruby)
|
29
|
+
provided by [OSS license](https://jb.gg/OpenSourceSupport).
|
28
30
|
|
29
31
|
## Index
|
30
32
|
|
@@ -35,8 +37,8 @@ based on Ferrum and Mechanize.
|
|
35
37
|
* [Navigation](https://github.com/rubycdp/ferrum#navigation)
|
36
38
|
* [Finders](https://github.com/rubycdp/ferrum#finders)
|
37
39
|
* [Screenshots](https://github.com/rubycdp/ferrum#screenshots)
|
38
|
-
* [Cleaning Up](https://github.com/rubycdp/ferrum#cleaning-up)
|
39
40
|
* [Network](https://github.com/rubycdp/ferrum#network)
|
41
|
+
* [Downloads](https://github.com/rubycdp/ferrum#downloads)
|
40
42
|
* [Proxy](https://github.com/rubycdp/ferrum#proxy)
|
41
43
|
* [Mouse](https://github.com/rubycdp/ferrum#mouse)
|
42
44
|
* [Keyboard](https://github.com/rubycdp/ferrum#keyboard)
|
@@ -49,6 +51,7 @@ based on Ferrum and Mechanize.
|
|
49
51
|
* [Animation](https://github.com/rubycdp/ferrum#animation)
|
50
52
|
* [Node](https://github.com/rubycdp/ferrum#node)
|
51
53
|
* [Tracing](https://github.com/rubycdp/ferrum#tracing)
|
54
|
+
* [Clean Up](https://github.com/rubycdp/ferrum#clean-up)
|
52
55
|
* [Thread safety](https://github.com/rubycdp/ferrum#thread-safety)
|
53
56
|
* [Development](https://github.com/rubycdp/ferrum#development)
|
54
57
|
* [Contributing](https://github.com/rubycdp/ferrum#contributing)
|
@@ -61,7 +64,7 @@ There's no official Chrome or Chromium package for Linux don't install it this
|
|
61
64
|
way because it's either outdated or unofficial, both are bad. Download it from
|
62
65
|
official source for [Chrome](https://www.google.com/chrome/) or
|
63
66
|
[Chromium](https://www.chromium.org/getting-involved/download-chromium).
|
64
|
-
Chrome binary should be in the `PATH` or `BROWSER_PATH`
|
67
|
+
Chrome binary should be in the `PATH` or `BROWSER_PATH` and you can pass it as an
|
65
68
|
option to browser instance see `:browser_path` in
|
66
69
|
[Customization](https://github.com/rubycdp/ferrum#customization).
|
67
70
|
|
@@ -83,14 +86,17 @@ browser.screenshot(path: "google.png")
|
|
83
86
|
browser.quit
|
84
87
|
```
|
85
88
|
|
86
|
-
|
89
|
+
When you work with browser instance Ferrum creates and maintains a default page for you, in fact all the methods above
|
90
|
+
are sent to the `page` instance that is created in the `default_context` of the `browser` instance. You can interact
|
91
|
+
with a page created manually and this is preferred:
|
87
92
|
|
88
93
|
```ruby
|
89
94
|
browser = Ferrum::Browser.new
|
90
|
-
browser.
|
91
|
-
|
95
|
+
page = browser.create_page
|
96
|
+
page.go_to("https://google.com")
|
97
|
+
input = page.at_xpath("//input[@name='q']")
|
92
98
|
input.focus.type("Ruby headless driver for Chrome", :Enter)
|
93
|
-
|
99
|
+
page.at_css("a > h3").text # => "rubycdp/ferrum: Ruby Chrome/Chromium driver - GitHub"
|
94
100
|
browser.quit
|
95
101
|
```
|
96
102
|
|
@@ -98,8 +104,9 @@ Evaluate some JavaScript and get full width/height:
|
|
98
104
|
|
99
105
|
```ruby
|
100
106
|
browser = Ferrum::Browser.new
|
101
|
-
browser.
|
102
|
-
|
107
|
+
page = browser.create_page
|
108
|
+
page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
|
109
|
+
width, height = page.evaluate <<~JS
|
103
110
|
[document.documentElement.offsetWidth,
|
104
111
|
document.documentElement.offsetHeight]
|
105
112
|
JS
|
@@ -112,8 +119,9 @@ Do any mouse movements you like:
|
|
112
119
|
```ruby
|
113
120
|
# Trace a 100x100 square
|
114
121
|
browser = Ferrum::Browser.new
|
115
|
-
browser.
|
116
|
-
|
122
|
+
page = browser.create_page
|
123
|
+
page.go_to("https://google.com")
|
124
|
+
page.mouse
|
117
125
|
.move(x: 0, y: 0)
|
118
126
|
.down
|
119
127
|
.move(x: 0, y: 100)
|
@@ -134,6 +142,7 @@ In docker as root you must pass the no-sandbox browser option:
|
|
134
142
|
Ferrum::Browser.new(browser_options: { 'no-sandbox': nil })
|
135
143
|
```
|
136
144
|
|
145
|
+
It has also been reported that the Chrome process repeatedly crashes when running inside a Docker container on an M1 Mac preventing Ferrum from working. Ferrum should work as expected when deployed to a Docker container on a non-M1 Mac.
|
137
146
|
|
138
147
|
## Customization
|
139
148
|
|
@@ -144,8 +153,10 @@ Ferrum::Browser.new(options)
|
|
144
153
|
```
|
145
154
|
|
146
155
|
* options `Hash`
|
147
|
-
* `:headless` (Boolean) - Set browser as headless or not, `true` by default.
|
156
|
+
* `:headless` (String | Boolean) - Set browser as headless or not, `true` by default. You can set `"new"` to support
|
157
|
+
[new headless mode](https://developer.chrome.com/articles/new-headless/).
|
148
158
|
* `:xvfb` (Boolean) - Run browser in a virtual framebuffer, `false` by default.
|
159
|
+
* `:flatten` (Boolean) - Use one websocket connection to the browser and all the pages in flatten mode.
|
149
160
|
* `:window_size` (Array) - The dimensions of the browser window in which to
|
150
161
|
test, expressed as a 2-element array, e.g. [1024, 768]. Default: [1024, 768]
|
151
162
|
* `:extensions` (Array[String | Hash]) - An array of paths to files or JS
|
@@ -176,6 +187,8 @@ Ferrum::Browser.new(options)
|
|
176
187
|
* `:host` (String) - Remote debugging address for headless Chrome.
|
177
188
|
* `:url` (String) - URL for a running instance of Chrome. If this is set, a
|
178
189
|
browser process will not be spawned.
|
190
|
+
* `:ws_url` (String) - Websocket url for a running instance of Chrome. If this is set, a
|
191
|
+
browser process will not be spawned.
|
179
192
|
* `:process_timeout` (Integer) - How long to wait for the Chrome process to
|
180
193
|
respond on startup.
|
181
194
|
* `:ws_max_receive_size` (Integer) - How big messages to accept from Chrome
|
@@ -196,7 +209,7 @@ Navigate page to.
|
|
196
209
|
configuring driver.
|
197
210
|
|
198
211
|
```ruby
|
199
|
-
|
212
|
+
page.go_to("https://github.com/")
|
200
213
|
```
|
201
214
|
|
202
215
|
#### back
|
@@ -204,9 +217,9 @@ browser.go_to("https://github.com/")
|
|
204
217
|
Navigate to the previous page in history.
|
205
218
|
|
206
219
|
```ruby
|
207
|
-
|
208
|
-
|
209
|
-
|
220
|
+
page.go_to("https://github.com/")
|
221
|
+
page.at_xpath("//a").click
|
222
|
+
page.back
|
210
223
|
```
|
211
224
|
|
212
225
|
#### forward
|
@@ -214,10 +227,10 @@ browser.back
|
|
214
227
|
Navigate to the next page in history.
|
215
228
|
|
216
229
|
```ruby
|
217
|
-
|
218
|
-
|
219
|
-
|
220
|
-
|
230
|
+
page.go_to("https://github.com/")
|
231
|
+
page.at_xpath("//a").click
|
232
|
+
page.back
|
233
|
+
page.forward
|
221
234
|
```
|
222
235
|
|
223
236
|
#### refresh
|
@@ -225,8 +238,8 @@ browser.forward
|
|
225
238
|
Reload current page.
|
226
239
|
|
227
240
|
```ruby
|
228
|
-
|
229
|
-
|
241
|
+
page.go_to("https://github.com/")
|
242
|
+
page.refresh
|
230
243
|
```
|
231
244
|
|
232
245
|
#### stop
|
@@ -234,8 +247,8 @@ browser.refresh
|
|
234
247
|
Stop all navigations and loading pending resources on the page
|
235
248
|
|
236
249
|
```ruby
|
237
|
-
|
238
|
-
|
250
|
+
page.go_to("https://github.com/")
|
251
|
+
page.stop
|
239
252
|
```
|
240
253
|
|
241
254
|
#### position = \*\*options
|
@@ -258,6 +271,37 @@ Get the position for the browser window
|
|
258
271
|
browser.position # => [10, 20]
|
259
272
|
```
|
260
273
|
|
274
|
+
#### window_bounds = \*\*options
|
275
|
+
|
276
|
+
Set window bounds
|
277
|
+
|
278
|
+
* options `Hash`
|
279
|
+
* :left `Integer`
|
280
|
+
* :top `Integer`
|
281
|
+
* :width `Integer`
|
282
|
+
* :height `Integer`
|
283
|
+
* :window_state `String`
|
284
|
+
|
285
|
+
```ruby
|
286
|
+
browser.window_bounds = { left: 10, top: 20, width: 1024, height: 768, window_state: "normal" }
|
287
|
+
```
|
288
|
+
|
289
|
+
#### window_bounds : `Hash<String, Integer | String>`
|
290
|
+
|
291
|
+
Get window bounds
|
292
|
+
|
293
|
+
```ruby
|
294
|
+
browser.window_bounds # => { "left": 0, "top": 1286, "width": 10, "height": 10, "windowState": "normal" }
|
295
|
+
```
|
296
|
+
|
297
|
+
#### window_id : `Integer`
|
298
|
+
|
299
|
+
Current window id
|
300
|
+
|
301
|
+
```ruby
|
302
|
+
browser.window_id # => 1
|
303
|
+
```
|
304
|
+
|
261
305
|
## Finders
|
262
306
|
|
263
307
|
#### at_css(selector, \*\*options) : `Node` | `nil`
|
@@ -270,8 +314,8 @@ provided node.
|
|
270
314
|
* :within `Node` | `nil`
|
271
315
|
|
272
316
|
```ruby
|
273
|
-
|
274
|
-
|
317
|
+
page.go_to("https://github.com/")
|
318
|
+
page.at_css("a[aria-label='Issues you created']") # => Node
|
275
319
|
```
|
276
320
|
|
277
321
|
|
@@ -285,8 +329,8 @@ document or provided node.
|
|
285
329
|
* :within `Node` | `nil`
|
286
330
|
|
287
331
|
```ruby
|
288
|
-
|
289
|
-
|
332
|
+
page.go_to("https://github.com/")
|
333
|
+
page.css("a[aria-label='Issues you created']") # => [Node]
|
290
334
|
```
|
291
335
|
|
292
336
|
#### at_xpath(selector, \*\*options) : `Node` | `nil`
|
@@ -298,8 +342,8 @@ Find node by xpath.
|
|
298
342
|
* :within `Node` | `nil`
|
299
343
|
|
300
344
|
```ruby
|
301
|
-
|
302
|
-
|
345
|
+
page.go_to("https://github.com/")
|
346
|
+
page.at_xpath("//a[@aria-label='Issues you created']") # => Node
|
303
347
|
```
|
304
348
|
|
305
349
|
#### xpath(selector, \*\*options) : `Array<Node>` | `[]`
|
@@ -311,8 +355,8 @@ Find nodes by xpath.
|
|
311
355
|
* :within `Node` | `nil`
|
312
356
|
|
313
357
|
```ruby
|
314
|
-
|
315
|
-
|
358
|
+
page.go_to("https://github.com/")
|
359
|
+
page.xpath("//a[@aria-label='Issues you created']") # => [Node]
|
316
360
|
```
|
317
361
|
|
318
362
|
#### current_url : `String`
|
@@ -320,8 +364,8 @@ browser.xpath("//a[@aria-label='Issues you created']") # => [Node]
|
|
320
364
|
Returns current top window location href.
|
321
365
|
|
322
366
|
```ruby
|
323
|
-
|
324
|
-
|
367
|
+
page.go_to("https://google.com/")
|
368
|
+
page.current_url # => "https://www.google.com/"
|
325
369
|
```
|
326
370
|
|
327
371
|
#### current_title : `String`
|
@@ -329,8 +373,8 @@ browser.current_url # => "https://www.google.com/"
|
|
329
373
|
Returns current top window title
|
330
374
|
|
331
375
|
```ruby
|
332
|
-
|
333
|
-
|
376
|
+
page.go_to("https://google.com/")
|
377
|
+
page.current_title # => "Google"
|
334
378
|
```
|
335
379
|
|
336
380
|
#### body : `String`
|
@@ -338,8 +382,8 @@ browser.current_title # => "Google"
|
|
338
382
|
Returns current page's html.
|
339
383
|
|
340
384
|
```ruby
|
341
|
-
|
342
|
-
|
385
|
+
page.go_to("https://google.com/")
|
386
|
+
page.body # => '<html itemscope="" itemtype="http://schema.org/WebPage" lang="ru"><head>...
|
343
387
|
```
|
344
388
|
|
345
389
|
|
@@ -357,20 +401,29 @@ Saves screenshot on a disk or returns it as base64.
|
|
357
401
|
* :format `String` "jpeg" | "png"
|
358
402
|
* :quality `Integer` 0-100 works for jpeg only
|
359
403
|
* :full `Boolean` whether you need full page screenshot or a viewport
|
360
|
-
* :selector `String` css selector for given element
|
404
|
+
* :selector `String` css selector for given element, optional
|
405
|
+
* :area `Hash` area for screenshot, optional
|
406
|
+
* :x `Integer`
|
407
|
+
* :y `Integer`
|
408
|
+
* :width `Integer`
|
409
|
+
* :height `Integer`
|
361
410
|
* :scale `Float` zoom in/out
|
362
411
|
* :background_color `Ferrum::RGBA.new(0, 0, 0, 0.0)` to have specific background color
|
363
412
|
|
364
413
|
```ruby
|
365
|
-
|
414
|
+
page.go_to("https://google.com/")
|
366
415
|
# Save on the disk in PNG
|
367
|
-
|
416
|
+
page.screenshot(path: "google.png") # => 134660
|
368
417
|
# Save on the disk in JPG
|
369
|
-
|
418
|
+
page.screenshot(path: "google.jpg") # => 30902
|
370
419
|
# Save to Base64 the whole page not only viewport and reduce quality
|
371
|
-
|
420
|
+
page.screenshot(full: true, quality: 60, encoding: :base64) # "iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAYAAAC6uhUNAAAAAXNSR0IArs4c6Q...
|
421
|
+
# Save on the disk with the selected element in PNG
|
422
|
+
page.screenshot(path: "google.png", selector: 'textarea') # => 11340
|
423
|
+
# Save to Base64 with an area of the page in PNG
|
424
|
+
page.screenshot(path: "google.png", area: { x: 0, y: 0, width: 400, height: 300 }) # => 54239
|
372
425
|
# Save with specific background color
|
373
|
-
|
426
|
+
page.screenshot(background_color: Ferrum::RGBA.new(0, 0, 0, 0.0))
|
374
427
|
```
|
375
428
|
|
376
429
|
#### pdf(\*\*options) : `String` | `Boolean`
|
@@ -391,9 +444,9 @@ Saves PDF on a disk or returns it as base64.
|
|
391
444
|
* See other [native options](https://chromedevtools.github.io/devtools-protocol/tot/Page#method-printToPDF) you can pass
|
392
445
|
|
393
446
|
```ruby
|
394
|
-
|
447
|
+
page.go_to("https://google.com/")
|
395
448
|
# Save to disk as a PDF
|
396
|
-
|
449
|
+
page.pdf(path: "google.pdf", paper_width: 1.0, paper_height: 1.0) # => true
|
397
450
|
```
|
398
451
|
|
399
452
|
#### mhtml(\*\*options) : `String` | `Integer`
|
@@ -404,33 +457,14 @@ Saves MHTML on a disk or returns it as a string.
|
|
404
457
|
* :path `String` to save a file on the disk.
|
405
458
|
|
406
459
|
```ruby
|
407
|
-
|
408
|
-
|
409
|
-
```
|
410
|
-
|
411
|
-
|
412
|
-
## Cleaning Up
|
413
|
-
|
414
|
-
#### reset
|
415
|
-
|
416
|
-
Closes browser tabs opened by the `Browser` instance.
|
417
|
-
|
418
|
-
```ruby
|
419
|
-
# connect to a long-running Chrome process
|
420
|
-
browser = Ferrum::Browser.new(url: 'http://localhost:9222')
|
421
|
-
|
422
|
-
browser.go_to("https://github.com/")
|
423
|
-
|
424
|
-
# clean up, lest the tab stays there hanging forever
|
425
|
-
browser.reset
|
426
|
-
|
427
|
-
browser.quit
|
460
|
+
page.go_to("https://google.com/")
|
461
|
+
page.mhtml(path: "google.mhtml") # => 87742
|
428
462
|
```
|
429
463
|
|
430
464
|
|
431
465
|
## Network
|
432
466
|
|
433
|
-
`
|
467
|
+
`page.network`
|
434
468
|
|
435
469
|
#### traffic `Array<Network::Exchange>`
|
436
470
|
|
@@ -438,8 +472,8 @@ Returns all information about network traffic as `Network::Exchange` instance
|
|
438
472
|
which in general is a wrapper around `request`, `response` and `error`.
|
439
473
|
|
440
474
|
```ruby
|
441
|
-
|
442
|
-
|
475
|
+
page.go_to("https://github.com/")
|
476
|
+
page.network.traffic # => [#<Ferrum::Network::Exchange, ...]
|
443
477
|
```
|
444
478
|
|
445
479
|
#### request : `Network::Request`
|
@@ -447,8 +481,8 @@ browser.network.traffic # => [#<Ferrum::Network::Exchange, ...]
|
|
447
481
|
Page request of the main frame.
|
448
482
|
|
449
483
|
```ruby
|
450
|
-
|
451
|
-
|
484
|
+
page.go_to("https://github.com/")
|
485
|
+
page.network.request # => #<Ferrum::Network::Request...
|
452
486
|
```
|
453
487
|
|
454
488
|
#### response : `Network::Response`
|
@@ -456,8 +490,8 @@ browser.network.request # => #<Ferrum::Network::Request...
|
|
456
490
|
Page response of the main frame.
|
457
491
|
|
458
492
|
```ruby
|
459
|
-
|
460
|
-
|
493
|
+
page.go_to("https://github.com/")
|
494
|
+
page.network.response # => #<Ferrum::Network::Response...
|
461
495
|
```
|
462
496
|
|
463
497
|
#### status : `Integer`
|
@@ -466,8 +500,8 @@ Contains the status code of the main page response (e.g., 200 for a
|
|
466
500
|
success). This is just a shortcut for `response.status`.
|
467
501
|
|
468
502
|
```ruby
|
469
|
-
|
470
|
-
|
503
|
+
page.go_to("https://github.com/")
|
504
|
+
page.network.status # => 200
|
471
505
|
```
|
472
506
|
|
473
507
|
#### wait_for_idle(\*\*options)
|
@@ -483,22 +517,22 @@ Waits for network idle or raises `Ferrum::TimeoutError` error
|
|
483
517
|
by default
|
484
518
|
|
485
519
|
```ruby
|
486
|
-
|
487
|
-
|
488
|
-
|
520
|
+
page.go_to("https://example.com/")
|
521
|
+
page.at_xpath("//a[text() = 'No UI changes button']").click
|
522
|
+
page.network.wait_for_idle
|
489
523
|
```
|
490
524
|
|
491
525
|
#### clear(type)
|
492
526
|
|
493
|
-
Clear
|
527
|
+
Clear page's cache or collected traffic.
|
494
528
|
|
495
529
|
* type `Symbol` it is either `:traffic` or `:cache`
|
496
530
|
|
497
531
|
```ruby
|
498
|
-
traffic =
|
499
|
-
|
532
|
+
traffic = page.network.traffic # => []
|
533
|
+
page.go_to("https://github.com/")
|
500
534
|
traffic.size # => 51
|
501
|
-
|
535
|
+
page.network.clear(:traffic)
|
502
536
|
traffic.size # => 0
|
503
537
|
```
|
504
538
|
|
@@ -514,8 +548,9 @@ continue them.
|
|
514
548
|
|
515
549
|
```ruby
|
516
550
|
browser = Ferrum::Browser.new
|
517
|
-
browser.
|
518
|
-
|
551
|
+
page = browser.create_page
|
552
|
+
page.network.intercept
|
553
|
+
page.on(:request) do |request|
|
519
554
|
if request.match?(/bla-bla/)
|
520
555
|
request.abort
|
521
556
|
elsif request.match?(/lorem/)
|
@@ -524,7 +559,7 @@ browser.on(:request) do |request|
|
|
524
559
|
request.continue
|
525
560
|
end
|
526
561
|
end
|
527
|
-
|
562
|
+
page.go_to("https://google.com")
|
528
563
|
```
|
529
564
|
|
530
565
|
#### authorize(\*\*options, &block)
|
@@ -539,10 +574,10 @@ If site or proxy uses authorization you can provide credentials using this metho
|
|
539
574
|
care about unwanted requests just call `request.continue`.
|
540
575
|
|
541
576
|
```ruby
|
542
|
-
|
543
|
-
|
544
|
-
puts
|
545
|
-
puts
|
577
|
+
page.network.authorize(user: "login", password: "pass") { |req| req.continue }
|
578
|
+
page.go_to("http://example.com/authenticated")
|
579
|
+
puts page.network.status # => 200
|
580
|
+
puts page.body # => Welcome, authenticated client
|
546
581
|
```
|
547
582
|
|
548
583
|
Since Chrome implements authorize using request interception you must continue or abort authorized requests. If you
|
@@ -551,8 +586,9 @@ block, so this is version doesn't pass block and can work just fine:
|
|
551
586
|
|
552
587
|
```ruby
|
553
588
|
browser = Ferrum::Browser.new
|
554
|
-
browser.
|
555
|
-
|
589
|
+
page = browser.create_page
|
590
|
+
page.network.intercept
|
591
|
+
page.on(:request) do |request|
|
556
592
|
if request.resource_type == "Image"
|
557
593
|
request.abort
|
558
594
|
else
|
@@ -560,9 +596,9 @@ browser.on(:request) do |request|
|
|
560
596
|
end
|
561
597
|
end
|
562
598
|
|
563
|
-
|
599
|
+
page.network.authorize(user: "login", password: "pass", type: :proxy)
|
564
600
|
|
565
|
-
|
601
|
+
page.go_to("https://google.com")
|
566
602
|
```
|
567
603
|
|
568
604
|
You used to call `authorize` method without block, but since it's implemented using request interception there could be
|
@@ -585,8 +621,8 @@ Activates emulation of network conditions.
|
|
585
621
|
bluetooth, ethernet, wifi, wimax, other. `nil` by default
|
586
622
|
|
587
623
|
```ruby
|
588
|
-
|
589
|
-
|
624
|
+
page.network.emulate_network_conditions(connection_type: "cellular2g")
|
625
|
+
page.go_to("https://github.com/")
|
590
626
|
```
|
591
627
|
|
592
628
|
#### offline_mode
|
@@ -594,8 +630,59 @@ browser.go_to("https://github.com/")
|
|
594
630
|
Activates offline mode for a page.
|
595
631
|
|
596
632
|
```ruby
|
597
|
-
|
598
|
-
|
633
|
+
page.network.offline_mode
|
634
|
+
page.go_to("https://github.com/") # => Ferrum::StatusError (Request to https://github.com/ failed(net::ERR_INTERNET_DISCONNECTED))
|
635
|
+
```
|
636
|
+
|
637
|
+
#### cache(disable: `Boolean`)
|
638
|
+
|
639
|
+
Toggles ignoring cache for each request. If true, cache will not be used.
|
640
|
+
|
641
|
+
```ruby
|
642
|
+
page.network.cache(disable: true)
|
643
|
+
```
|
644
|
+
|
645
|
+
|
646
|
+
## Downloads
|
647
|
+
|
648
|
+
`page.downloads`
|
649
|
+
|
650
|
+
#### files `Array<Hash>`
|
651
|
+
|
652
|
+
Returns all information about downloaded files as a `Hash`.
|
653
|
+
|
654
|
+
```ruby
|
655
|
+
page.go_to("http://localhost/attachment.pdf")
|
656
|
+
page.downloads.files # => [{"frameId"=>"E3316DF1B5383D38F8ADF7485005FDE3", "guid"=>"11a68745-98ac-4d54-9b57-9f9016c268b3", "url"=>"http://localhost/attachment.pdf", "suggestedFilename"=>"attachment.pdf", "totalBytes"=>4911, "receivedBytes"=>4911, "state"=>"completed"}]
|
657
|
+
```
|
658
|
+
|
659
|
+
#### wait(timeout)
|
660
|
+
|
661
|
+
Waits until the download is finished.
|
662
|
+
|
663
|
+
```ruby
|
664
|
+
page.go_to("http://localhost/attachment.pdf")
|
665
|
+
page.downloads.wait
|
666
|
+
```
|
667
|
+
|
668
|
+
or
|
669
|
+
|
670
|
+
```ruby
|
671
|
+
page.go_to("http://localhost/page")
|
672
|
+
page.downloads.wait { page.at_css("#download").click }
|
673
|
+
```
|
674
|
+
|
675
|
+
#### set_behavior(\*\*options)
|
676
|
+
|
677
|
+
Sets behavior in case of file to be downloaded.
|
678
|
+
|
679
|
+
* options `Hash`
|
680
|
+
* :save_path `String` absolute path of where to store the file
|
681
|
+
* :behavior `Symbol` `deny | allow | allowAndName | default`, `allow` by default
|
682
|
+
|
683
|
+
```ruby
|
684
|
+
page.go_to("https://example.com/")
|
685
|
+
page.downloads.set_behavior(save_path: "/tmp", behavior: :allow)
|
599
686
|
```
|
600
687
|
|
601
688
|
|
@@ -604,13 +691,13 @@ browser.go_to("https://github.com/") # => Ferrum::StatusError (Request to https:
|
|
604
691
|
You can set a proxy with a `:proxy` option:
|
605
692
|
|
606
693
|
```ruby
|
607
|
-
|
694
|
+
Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", user: "user", password: "pa$$" })
|
608
695
|
```
|
609
696
|
|
610
697
|
`:bypass` can specify semi-colon-separated list of hosts for which proxy shouldn't be used:
|
611
698
|
|
612
699
|
```ruby
|
613
|
-
|
700
|
+
Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", bypass: "*.google.com;*foo.com" })
|
614
701
|
```
|
615
702
|
|
616
703
|
In general passing a proxy option when instantiating a browser results in a browser running with proxy command line
|
@@ -634,7 +721,7 @@ end
|
|
634
721
|
|
635
722
|
### Mouse
|
636
723
|
|
637
|
-
`
|
724
|
+
`page.mouse`
|
638
725
|
|
639
726
|
#### scroll_to(x, y)
|
640
727
|
|
@@ -646,8 +733,8 @@ Scroll page to a given x, y
|
|
646
733
|
displayed in the upper left
|
647
734
|
|
648
735
|
```ruby
|
649
|
-
|
650
|
-
|
736
|
+
page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
|
737
|
+
page.mouse.scroll_to(0, 400)
|
651
738
|
```
|
652
739
|
|
653
740
|
#### click(\*\*options) : `Mouse`
|
@@ -691,7 +778,7 @@ Mouse move to given x and y.
|
|
691
778
|
|
692
779
|
### Keyboard
|
693
780
|
|
694
|
-
|
781
|
+
`page.keyboard`
|
695
782
|
|
696
783
|
#### down(key) : `Keyboard`
|
697
784
|
|
@@ -721,14 +808,14 @@ Returns bitfield for a given keys
|
|
721
808
|
|
722
809
|
## Cookies
|
723
810
|
|
724
|
-
`
|
811
|
+
`page.cookies`
|
725
812
|
|
726
813
|
#### all : `Hash<String, Cookie>`
|
727
814
|
|
728
815
|
Returns cookies hash
|
729
816
|
|
730
817
|
```ruby
|
731
|
-
|
818
|
+
page.cookies.all # => {"NID"=>#<Ferrum::Cookies::Cookie:0x0000558624b37a40 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>}
|
732
819
|
```
|
733
820
|
|
734
821
|
#### [](value) : `Cookie`
|
@@ -738,7 +825,7 @@ Returns cookie
|
|
738
825
|
* value `String`
|
739
826
|
|
740
827
|
```ruby
|
741
|
-
|
828
|
+
page.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>
|
742
829
|
```
|
743
830
|
|
744
831
|
#### set(value) : `Boolean`
|
@@ -754,14 +841,14 @@ Sets a cookie
|
|
754
841
|
* :httponly `Boolean`
|
755
842
|
|
756
843
|
```ruby
|
757
|
-
|
844
|
+
page.cookies.set(name: "stealth", value: "omg", domain: "google.com") # => true
|
758
845
|
```
|
759
846
|
|
760
847
|
* value `Cookie`
|
761
848
|
|
762
849
|
```ruby
|
763
|
-
nid_cookie =
|
764
|
-
|
850
|
+
nid_cookie = page.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88>
|
851
|
+
page.cookies.set(nid_cookie) # => true
|
765
852
|
```
|
766
853
|
|
767
854
|
#### remove(\*\*options) : `Boolean`
|
@@ -774,7 +861,7 @@ Removes given cookie
|
|
774
861
|
* :url `String`
|
775
862
|
|
776
863
|
```ruby
|
777
|
-
|
864
|
+
page.cookies.remove(name: "stealth", domain: "google.com") # => true
|
778
865
|
```
|
779
866
|
|
780
867
|
#### clear : `Boolean`
|
@@ -782,12 +869,12 @@ browser.cookies.remove(name: "stealth", domain: "google.com") # => true
|
|
782
869
|
Removes all cookies for current page
|
783
870
|
|
784
871
|
```ruby
|
785
|
-
|
872
|
+
page.cookies.clear # => true
|
786
873
|
```
|
787
874
|
|
788
875
|
## Headers
|
789
876
|
|
790
|
-
`
|
877
|
+
`page.headers`
|
791
878
|
|
792
879
|
#### get : `Hash`
|
793
880
|
|
@@ -821,7 +908,7 @@ Evaluate and return result for given JS expression
|
|
821
908
|
simple value.
|
822
909
|
|
823
910
|
```ruby
|
824
|
-
|
911
|
+
page.evaluate("[window.scrollX, window.scrollY]")
|
825
912
|
```
|
826
913
|
|
827
914
|
#### evaluate_async(expression, wait_time, \*args)
|
@@ -834,7 +921,7 @@ Evaluate asynchronous expression and return result
|
|
834
921
|
simple value.
|
835
922
|
|
836
923
|
```ruby
|
837
|
-
|
924
|
+
page.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
|
838
925
|
```
|
839
926
|
|
840
927
|
#### execute(expression, \*args)
|
@@ -846,7 +933,7 @@ Execute expression. Doesn't return the result
|
|
846
933
|
simple value.
|
847
934
|
|
848
935
|
```ruby
|
849
|
-
|
936
|
+
page.execute(%(1 + 1)) # => true
|
850
937
|
```
|
851
938
|
|
852
939
|
#### evaluate_on_new_document(expression)
|
@@ -872,7 +959,7 @@ JS
|
|
872
959
|
* :type `String` - `text/javascript` by default
|
873
960
|
|
874
961
|
```ruby
|
875
|
-
|
962
|
+
page.add_script_tag(url: "http://example.com/stylesheet.css") # => true
|
876
963
|
```
|
877
964
|
|
878
965
|
#### add_style_tag(\*\*options) : `Boolean`
|
@@ -883,7 +970,7 @@ browser.add_script_tag(url: "http://example.com/stylesheet.css") # => true
|
|
883
970
|
* :content `String`
|
884
971
|
|
885
972
|
```ruby
|
886
|
-
|
973
|
+
page.add_style_tag(content: "h1 { font-size: 40px; }") # => true
|
887
974
|
|
888
975
|
```
|
889
976
|
#### bypass_csp(\*\*options) : `Boolean`
|
@@ -892,11 +979,39 @@ browser.add_style_tag(content: "h1 { font-size: 40px; }") # => true
|
|
892
979
|
* :enabled `Boolean`, `true` by default
|
893
980
|
|
894
981
|
```ruby
|
895
|
-
|
896
|
-
|
897
|
-
|
898
|
-
|
899
|
-
|
982
|
+
page.bypass_csp # => true
|
983
|
+
page.go_to("https://github.com/ruby-concurrency/concurrent-ruby/blob/master/docs-source/promises.in.md")
|
984
|
+
page.refresh
|
985
|
+
page.add_script_tag(content: "window.__injected = 42")
|
986
|
+
page.evaluate("window.__injected") # => 42
|
987
|
+
```
|
988
|
+
|
989
|
+
|
990
|
+
## Emulation
|
991
|
+
|
992
|
+
#### disable_javascript
|
993
|
+
|
994
|
+
Disables Javascripts from the loaded HTML source.
|
995
|
+
You can still evaluate JavaScript with `evaluate` or `execute`.
|
996
|
+
Returns nothing.
|
997
|
+
|
998
|
+
```ruby
|
999
|
+
page.disable_javascript
|
1000
|
+
```
|
1001
|
+
|
1002
|
+
|
1003
|
+
#### set_viewport
|
1004
|
+
|
1005
|
+
Overrides device screen dimensions and emulates viewport.
|
1006
|
+
|
1007
|
+
* options `Hash`
|
1008
|
+
* :width `Integer`, viewport width. `0` by default
|
1009
|
+
* :height `Integer`, viewport height. `0` by default
|
1010
|
+
* :scale_factor `Float`, device scale factor. `0` by default
|
1011
|
+
* :mobile `Boolean`, whether to emulate mobile device. `false` by default
|
1012
|
+
|
1013
|
+
```ruby
|
1014
|
+
page.set_viewport(width: 1000, height: 600, scale_factor: 3)
|
900
1015
|
```
|
901
1016
|
|
902
1017
|
|
@@ -907,8 +1022,8 @@ browser.evaluate("window.__injected") # => 42
|
|
907
1022
|
Returns all the frames current page have.
|
908
1023
|
|
909
1024
|
```ruby
|
910
|
-
|
911
|
-
|
1025
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1026
|
+
page.frames # =>
|
912
1027
|
# [
|
913
1028
|
# #<Ferrum::Frame @id="C6D104CE454A025FBCF22B98DE612B12" @parent_id=nil @name=nil @state=:stopped_loading @execution_id=1>,
|
914
1029
|
# #<Ferrum::Frame @id="C09C4E4404314AAEAE85928EAC109A93" @parent_id="C6D104CE454A025FBCF22B98DE612B12" @state=:stopped_loading @execution_id=2>,
|
@@ -930,7 +1045,7 @@ Find frame by given options.
|
|
930
1045
|
* :name `String` - Frame's name if there's one
|
931
1046
|
|
932
1047
|
```ruby
|
933
|
-
|
1048
|
+
page.frame_by(id: "C6D104CE454A025FBCF22B98DE612B12")
|
934
1049
|
```
|
935
1050
|
|
936
1051
|
|
@@ -966,8 +1081,8 @@ One of the states frame's in:
|
|
966
1081
|
Returns current frame's location href.
|
967
1082
|
|
968
1083
|
```ruby
|
969
|
-
|
970
|
-
frame =
|
1084
|
+
page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
1085
|
+
frame = page.frames[1]
|
971
1086
|
frame.url # => https://interactive-examples.mdn.mozilla.net/pages/tabbed/iframe.html
|
972
1087
|
```
|
973
1088
|
|
@@ -976,8 +1091,8 @@ frame.url # => https://interactive-examples.mdn.mozilla.net/pages/tabbed/iframe.
|
|
976
1091
|
Returns current frame's title.
|
977
1092
|
|
978
1093
|
```ruby
|
979
|
-
|
980
|
-
frame =
|
1094
|
+
page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
1095
|
+
frame = page.frames[1]
|
981
1096
|
frame.title # => HTML Demo: <iframe>
|
982
1097
|
```
|
983
1098
|
|
@@ -986,8 +1101,8 @@ frame.title # => HTML Demo: <iframe>
|
|
986
1101
|
If current frame is the main frame of the page (top of the tree).
|
987
1102
|
|
988
1103
|
```ruby
|
989
|
-
|
990
|
-
frame =
|
1104
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1105
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
991
1106
|
frame.main? # => false
|
992
1107
|
```
|
993
1108
|
|
@@ -996,8 +1111,8 @@ frame.main? # => false
|
|
996
1111
|
Returns current frame's top window location href.
|
997
1112
|
|
998
1113
|
```ruby
|
999
|
-
|
1000
|
-
frame =
|
1114
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1115
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
1001
1116
|
frame.current_url # => "https://www.w3schools.com/tags/tag_frame.asp"
|
1002
1117
|
```
|
1003
1118
|
|
@@ -1006,8 +1121,8 @@ frame.current_url # => "https://www.w3schools.com/tags/tag_frame.asp"
|
|
1006
1121
|
Returns current frame's top window title.
|
1007
1122
|
|
1008
1123
|
```ruby
|
1009
|
-
|
1010
|
-
frame =
|
1124
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1125
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
1011
1126
|
frame.current_title # => "HTML frame tag"
|
1012
1127
|
```
|
1013
1128
|
|
@@ -1016,8 +1131,8 @@ frame.current_title # => "HTML frame tag"
|
|
1016
1131
|
Returns current frame's html.
|
1017
1132
|
|
1018
1133
|
```ruby
|
1019
|
-
|
1020
|
-
frame =
|
1134
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1135
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
1021
1136
|
frame.body # => "<html><head></head><body></body></html>"
|
1022
1137
|
```
|
1023
1138
|
|
@@ -1026,8 +1141,8 @@ frame.body # => "<html><head></head><body></body></html>"
|
|
1026
1141
|
Returns current frame's doctype.
|
1027
1142
|
|
1028
1143
|
```ruby
|
1029
|
-
|
1030
|
-
|
1144
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1145
|
+
page.main_frame.doctype # => "<!DOCTYPE html>"
|
1031
1146
|
```
|
1032
1147
|
|
1033
1148
|
#### content = html
|
@@ -1037,8 +1152,8 @@ Sets a content of a given frame.
|
|
1037
1152
|
* html `String`
|
1038
1153
|
|
1039
1154
|
```ruby
|
1040
|
-
|
1041
|
-
frame =
|
1155
|
+
page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
1156
|
+
frame = page.frames[1]
|
1042
1157
|
frame.body # <html lang="en"><head><style>body {transition: opacity ease-in 0.2s; }...
|
1043
1158
|
frame.content = "<html><head></head><body><p>lol</p></body></html>"
|
1044
1159
|
frame.body # => <html><head></head><body><p>lol</p></body></html>
|
@@ -1058,15 +1173,14 @@ Accept dialog with given text or default prompt if applicable
|
|
1058
1173
|
Dismiss dialog
|
1059
1174
|
|
1060
1175
|
```ruby
|
1061
|
-
|
1062
|
-
browser.on(:dialog) do |dialog|
|
1176
|
+
page.on(:dialog) do |dialog|
|
1063
1177
|
if dialog.match?(/bla-bla/)
|
1064
1178
|
dialog.accept
|
1065
1179
|
else
|
1066
1180
|
dialog.dismiss
|
1067
1181
|
end
|
1068
1182
|
end
|
1069
|
-
|
1183
|
+
page.go_to("https://google.com")
|
1070
1184
|
```
|
1071
1185
|
|
1072
1186
|
|
@@ -1086,10 +1200,9 @@ Sets playback rate of CSS animations
|
|
1086
1200
|
* value `Integer`
|
1087
1201
|
|
1088
1202
|
```ruby
|
1089
|
-
|
1090
|
-
|
1091
|
-
|
1092
|
-
browser.playback_rate # => 2000
|
1203
|
+
page.playback_rate = 2000
|
1204
|
+
page.go_to("https://google.com")
|
1205
|
+
page.playback_rate # => 2000
|
1093
1206
|
```
|
1094
1207
|
|
1095
1208
|
|
@@ -1103,7 +1216,7 @@ Returns [Frame](https://github.com/rubycdp/ferrum#frame) object for current node
|
|
1103
1216
|
[Finders](https://github.com/rubycdp/ferrum#Finders) for that object:
|
1104
1217
|
|
1105
1218
|
```ruby
|
1106
|
-
frame =
|
1219
|
+
frame = page.at_xpath("//iframe").frame # => Frame
|
1107
1220
|
frame.at_css("//a[text() = 'Log in']") # => Node
|
1108
1221
|
```
|
1109
1222
|
|
@@ -1128,19 +1241,21 @@ frame.at_css("//a[text() = 'Log in']") # => Node
|
|
1128
1241
|
#### evaluate
|
1129
1242
|
#### selected : `Array<Node>`
|
1130
1243
|
#### select
|
1244
|
+
#### scroll_into_view
|
1245
|
+
#### in_viewport?(of: `Node | nil`) : `Boolean`
|
1131
1246
|
|
1132
1247
|
(chainable) Selects options by passed attribute.
|
1133
1248
|
|
1134
1249
|
```ruby
|
1135
|
-
|
1136
|
-
|
1250
|
+
page.at_xpath("//*[select]").select(["1"]) # => Node (select)
|
1251
|
+
page.at_xpath("//*[select]").select(["text"], by: :text) # => Node (select)
|
1137
1252
|
```
|
1138
1253
|
|
1139
1254
|
Accept string, array or strings:
|
1140
1255
|
```ruby
|
1141
|
-
|
1142
|
-
|
1143
|
-
|
1256
|
+
page.at_xpath("//*[select]").select("1")
|
1257
|
+
page.at_xpath("//*[select]").select("1", "2")
|
1258
|
+
page.at_xpath("//*[select]").select(["1", "2"])
|
1144
1259
|
```
|
1145
1260
|
|
1146
1261
|
|
@@ -1171,6 +1286,25 @@ Accepts block, records trace and by default returns trace data from `Tracing.tra
|
|
1171
1286
|
only one trace config can be active at a time per browser.
|
1172
1287
|
|
1173
1288
|
|
1289
|
+
## Clean Up
|
1290
|
+
|
1291
|
+
#### reset
|
1292
|
+
|
1293
|
+
Closes browser tabs opened by the `Browser` instance.
|
1294
|
+
|
1295
|
+
```ruby
|
1296
|
+
# connect to a long-running Chrome process
|
1297
|
+
browser = Ferrum::Browser.new(url: 'http://localhost:9222')
|
1298
|
+
|
1299
|
+
browser.go_to("https://github.com/")
|
1300
|
+
|
1301
|
+
# clean up, lest the tab stays there hanging forever
|
1302
|
+
browser.reset
|
1303
|
+
|
1304
|
+
browser.quit
|
1305
|
+
```
|
1306
|
+
|
1307
|
+
|
1174
1308
|
## Thread safety ##
|
1175
1309
|
|
1176
1310
|
Ferrum is fully thread-safe. You can create one browser or a few as you wish and
|