ferrum 0.13 → 0.15

Sign up to get free protection for your applications and to get access to all the features.
Files changed (40) hide show
  1. checksums.yaml +4 -4
  2. data/LICENSE +1 -1
  3. data/README.md +288 -154
  4. data/lib/ferrum/browser/command.rb +8 -0
  5. data/lib/ferrum/browser/options/chrome.rb +17 -5
  6. data/lib/ferrum/browser/options.rb +38 -25
  7. data/lib/ferrum/browser/process.rb +44 -17
  8. data/lib/ferrum/browser.rb +34 -52
  9. data/lib/ferrum/client/subscriber.rb +76 -0
  10. data/lib/ferrum/{browser → client}/web_socket.rb +36 -22
  11. data/lib/ferrum/client.rb +169 -0
  12. data/lib/ferrum/context.rb +19 -15
  13. data/lib/ferrum/contexts.rb +46 -12
  14. data/lib/ferrum/cookies/cookie.rb +57 -0
  15. data/lib/ferrum/cookies.rb +40 -4
  16. data/lib/ferrum/downloads.rb +60 -0
  17. data/lib/ferrum/errors.rb +2 -1
  18. data/lib/ferrum/frame.rb +1 -0
  19. data/lib/ferrum/headers.rb +1 -1
  20. data/lib/ferrum/network/exchange.rb +29 -2
  21. data/lib/ferrum/network/intercepted_request.rb +8 -17
  22. data/lib/ferrum/network/request.rb +23 -39
  23. data/lib/ferrum/network/request_params.rb +57 -0
  24. data/lib/ferrum/network/response.rb +25 -5
  25. data/lib/ferrum/network.rb +43 -16
  26. data/lib/ferrum/node.rb +21 -1
  27. data/lib/ferrum/page/frames.rb +5 -5
  28. data/lib/ferrum/page/screenshot.rb +42 -24
  29. data/lib/ferrum/page.rb +183 -131
  30. data/lib/ferrum/proxy.rb +1 -1
  31. data/lib/ferrum/target.rb +25 -5
  32. data/lib/ferrum/utils/elapsed_time.rb +0 -2
  33. data/lib/ferrum/utils/event.rb +19 -0
  34. data/lib/ferrum/utils/platform.rb +4 -0
  35. data/lib/ferrum/utils/thread.rb +18 -0
  36. data/lib/ferrum/version.rb +1 -1
  37. data/lib/ferrum.rb +3 -0
  38. metadata +14 -114
  39. data/lib/ferrum/browser/client.rb +0 -102
  40. data/lib/ferrum/browser/subscriber.rb +0 -36
data/README.md CHANGED
@@ -25,6 +25,8 @@ going to crawl sites you better use Ferrum or
25
25
  * [Vessel](https://github.com/rubycdp/vessel) high-level web crawling framework
26
26
  based on Ferrum and Mechanize.
27
27
 
28
+ The development is done in [![RubyMine](https://resources.jetbrains.com/storage/products/company/brand/logos/RubyMine_icon.svg?width=10px)](https://jb.gg/ruby)
29
+ provided by [OSS license](https://jb.gg/OpenSourceSupport).
28
30
 
29
31
  ## Index
30
32
 
@@ -35,8 +37,8 @@ based on Ferrum and Mechanize.
35
37
  * [Navigation](https://github.com/rubycdp/ferrum#navigation)
36
38
  * [Finders](https://github.com/rubycdp/ferrum#finders)
37
39
  * [Screenshots](https://github.com/rubycdp/ferrum#screenshots)
38
- * [Cleaning Up](https://github.com/rubycdp/ferrum#cleaning-up)
39
40
  * [Network](https://github.com/rubycdp/ferrum#network)
41
+ * [Downloads](https://github.com/rubycdp/ferrum#downloads)
40
42
  * [Proxy](https://github.com/rubycdp/ferrum#proxy)
41
43
  * [Mouse](https://github.com/rubycdp/ferrum#mouse)
42
44
  * [Keyboard](https://github.com/rubycdp/ferrum#keyboard)
@@ -49,6 +51,7 @@ based on Ferrum and Mechanize.
49
51
  * [Animation](https://github.com/rubycdp/ferrum#animation)
50
52
  * [Node](https://github.com/rubycdp/ferrum#node)
51
53
  * [Tracing](https://github.com/rubycdp/ferrum#tracing)
54
+ * [Clean Up](https://github.com/rubycdp/ferrum#clean-up)
52
55
  * [Thread safety](https://github.com/rubycdp/ferrum#thread-safety)
53
56
  * [Development](https://github.com/rubycdp/ferrum#development)
54
57
  * [Contributing](https://github.com/rubycdp/ferrum#contributing)
@@ -61,7 +64,7 @@ There's no official Chrome or Chromium package for Linux don't install it this
61
64
  way because it's either outdated or unofficial, both are bad. Download it from
62
65
  official source for [Chrome](https://www.google.com/chrome/) or
63
66
  [Chromium](https://www.chromium.org/getting-involved/download-chromium).
64
- Chrome binary should be in the `PATH` or `BROWSER_PATH` or you can pass it as an
67
+ Chrome binary should be in the `PATH` or `BROWSER_PATH` and you can pass it as an
65
68
  option to browser instance see `:browser_path` in
66
69
  [Customization](https://github.com/rubycdp/ferrum#customization).
67
70
 
@@ -83,14 +86,17 @@ browser.screenshot(path: "google.png")
83
86
  browser.quit
84
87
  ```
85
88
 
86
- Interact with a page:
89
+ When you work with browser instance Ferrum creates and maintains a default page for you, in fact all the methods above
90
+ are sent to the `page` instance that is created in the `default_context` of the `browser` instance. You can interact
91
+ with a page created manually and this is preferred:
87
92
 
88
93
  ```ruby
89
94
  browser = Ferrum::Browser.new
90
- browser.go_to("https://google.com")
91
- input = browser.at_xpath("//input[@name='q']")
95
+ page = browser.create_page
96
+ page.go_to("https://google.com")
97
+ input = page.at_xpath("//input[@name='q']")
92
98
  input.focus.type("Ruby headless driver for Chrome", :Enter)
93
- browser.at_css("a > h3").text # => "rubycdp/ferrum: Ruby Chrome/Chromium driver - GitHub"
99
+ page.at_css("a > h3").text # => "rubycdp/ferrum: Ruby Chrome/Chromium driver - GitHub"
94
100
  browser.quit
95
101
  ```
96
102
 
@@ -98,8 +104,9 @@ Evaluate some JavaScript and get full width/height:
98
104
 
99
105
  ```ruby
100
106
  browser = Ferrum::Browser.new
101
- browser.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
102
- width, height = browser.evaluate <<~JS
107
+ page = browser.create_page
108
+ page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
109
+ width, height = page.evaluate <<~JS
103
110
  [document.documentElement.offsetWidth,
104
111
  document.documentElement.offsetHeight]
105
112
  JS
@@ -112,8 +119,9 @@ Do any mouse movements you like:
112
119
  ```ruby
113
120
  # Trace a 100x100 square
114
121
  browser = Ferrum::Browser.new
115
- browser.go_to("https://google.com")
116
- browser.mouse
122
+ page = browser.create_page
123
+ page.go_to("https://google.com")
124
+ page.mouse
117
125
  .move(x: 0, y: 0)
118
126
  .down
119
127
  .move(x: 0, y: 100)
@@ -134,6 +142,7 @@ In docker as root you must pass the no-sandbox browser option:
134
142
  Ferrum::Browser.new(browser_options: { 'no-sandbox': nil })
135
143
  ```
136
144
 
145
+ It has also been reported that the Chrome process repeatedly crashes when running inside a Docker container on an M1 Mac preventing Ferrum from working. Ferrum should work as expected when deployed to a Docker container on a non-M1 Mac.
137
146
 
138
147
  ## Customization
139
148
 
@@ -144,8 +153,10 @@ Ferrum::Browser.new(options)
144
153
  ```
145
154
 
146
155
  * options `Hash`
147
- * `:headless` (Boolean) - Set browser as headless or not, `true` by default.
156
+ * `:headless` (String | Boolean) - Set browser as headless or not, `true` by default. You can set `"new"` to support
157
+ [new headless mode](https://developer.chrome.com/articles/new-headless/).
148
158
  * `:xvfb` (Boolean) - Run browser in a virtual framebuffer, `false` by default.
159
+ * `:flatten` (Boolean) - Use one websocket connection to the browser and all the pages in flatten mode.
149
160
  * `:window_size` (Array) - The dimensions of the browser window in which to
150
161
  test, expressed as a 2-element array, e.g. [1024, 768]. Default: [1024, 768]
151
162
  * `:extensions` (Array[String | Hash]) - An array of paths to files or JS
@@ -176,6 +187,8 @@ Ferrum::Browser.new(options)
176
187
  * `:host` (String) - Remote debugging address for headless Chrome.
177
188
  * `:url` (String) - URL for a running instance of Chrome. If this is set, a
178
189
  browser process will not be spawned.
190
+ * `:ws_url` (String) - Websocket url for a running instance of Chrome. If this is set, a
191
+ browser process will not be spawned.
179
192
  * `:process_timeout` (Integer) - How long to wait for the Chrome process to
180
193
  respond on startup.
181
194
  * `:ws_max_receive_size` (Integer) - How big messages to accept from Chrome
@@ -196,7 +209,7 @@ Navigate page to.
196
209
  configuring driver.
197
210
 
198
211
  ```ruby
199
- browser.go_to("https://github.com/")
212
+ page.go_to("https://github.com/")
200
213
  ```
201
214
 
202
215
  #### back
@@ -204,9 +217,9 @@ browser.go_to("https://github.com/")
204
217
  Navigate to the previous page in history.
205
218
 
206
219
  ```ruby
207
- browser.go_to("https://github.com/")
208
- browser.at_xpath("//a").click
209
- browser.back
220
+ page.go_to("https://github.com/")
221
+ page.at_xpath("//a").click
222
+ page.back
210
223
  ```
211
224
 
212
225
  #### forward
@@ -214,10 +227,10 @@ browser.back
214
227
  Navigate to the next page in history.
215
228
 
216
229
  ```ruby
217
- browser.go_to("https://github.com/")
218
- browser.at_xpath("//a").click
219
- browser.back
220
- browser.forward
230
+ page.go_to("https://github.com/")
231
+ page.at_xpath("//a").click
232
+ page.back
233
+ page.forward
221
234
  ```
222
235
 
223
236
  #### refresh
@@ -225,8 +238,8 @@ browser.forward
225
238
  Reload current page.
226
239
 
227
240
  ```ruby
228
- browser.go_to("https://github.com/")
229
- browser.refresh
241
+ page.go_to("https://github.com/")
242
+ page.refresh
230
243
  ```
231
244
 
232
245
  #### stop
@@ -234,8 +247,8 @@ browser.refresh
234
247
  Stop all navigations and loading pending resources on the page
235
248
 
236
249
  ```ruby
237
- browser.go_to("https://github.com/")
238
- browser.stop
250
+ page.go_to("https://github.com/")
251
+ page.stop
239
252
  ```
240
253
 
241
254
  #### position = \*\*options
@@ -258,6 +271,37 @@ Get the position for the browser window
258
271
  browser.position # => [10, 20]
259
272
  ```
260
273
 
274
+ #### window_bounds = \*\*options
275
+
276
+ Set window bounds
277
+
278
+ * options `Hash`
279
+ * :left `Integer`
280
+ * :top `Integer`
281
+ * :width `Integer`
282
+ * :height `Integer`
283
+ * :window_state `String`
284
+
285
+ ```ruby
286
+ browser.window_bounds = { left: 10, top: 20, width: 1024, height: 768, window_state: "normal" }
287
+ ```
288
+
289
+ #### window_bounds : `Hash<String, Integer | String>`
290
+
291
+ Get window bounds
292
+
293
+ ```ruby
294
+ browser.window_bounds # => { "left": 0, "top": 1286, "width": 10, "height": 10, "windowState": "normal" }
295
+ ```
296
+
297
+ #### window_id : `Integer`
298
+
299
+ Current window id
300
+
301
+ ```ruby
302
+ browser.window_id # => 1
303
+ ```
304
+
261
305
  ## Finders
262
306
 
263
307
  #### at_css(selector, \*\*options) : `Node` | `nil`
@@ -270,8 +314,8 @@ provided node.
270
314
  * :within `Node` | `nil`
271
315
 
272
316
  ```ruby
273
- browser.go_to("https://github.com/")
274
- browser.at_css("a[aria-label='Issues you created']") # => Node
317
+ page.go_to("https://github.com/")
318
+ page.at_css("a[aria-label='Issues you created']") # => Node
275
319
  ```
276
320
 
277
321
 
@@ -285,8 +329,8 @@ document or provided node.
285
329
  * :within `Node` | `nil`
286
330
 
287
331
  ```ruby
288
- browser.go_to("https://github.com/")
289
- browser.css("a[aria-label='Issues you created']") # => [Node]
332
+ page.go_to("https://github.com/")
333
+ page.css("a[aria-label='Issues you created']") # => [Node]
290
334
  ```
291
335
 
292
336
  #### at_xpath(selector, \*\*options) : `Node` | `nil`
@@ -298,8 +342,8 @@ Find node by xpath.
298
342
  * :within `Node` | `nil`
299
343
 
300
344
  ```ruby
301
- browser.go_to("https://github.com/")
302
- browser.at_xpath("//a[@aria-label='Issues you created']") # => Node
345
+ page.go_to("https://github.com/")
346
+ page.at_xpath("//a[@aria-label='Issues you created']") # => Node
303
347
  ```
304
348
 
305
349
  #### xpath(selector, \*\*options) : `Array<Node>` | `[]`
@@ -311,8 +355,8 @@ Find nodes by xpath.
311
355
  * :within `Node` | `nil`
312
356
 
313
357
  ```ruby
314
- browser.go_to("https://github.com/")
315
- browser.xpath("//a[@aria-label='Issues you created']") # => [Node]
358
+ page.go_to("https://github.com/")
359
+ page.xpath("//a[@aria-label='Issues you created']") # => [Node]
316
360
  ```
317
361
 
318
362
  #### current_url : `String`
@@ -320,8 +364,8 @@ browser.xpath("//a[@aria-label='Issues you created']") # => [Node]
320
364
  Returns current top window location href.
321
365
 
322
366
  ```ruby
323
- browser.go_to("https://google.com/")
324
- browser.current_url # => "https://www.google.com/"
367
+ page.go_to("https://google.com/")
368
+ page.current_url # => "https://www.google.com/"
325
369
  ```
326
370
 
327
371
  #### current_title : `String`
@@ -329,8 +373,8 @@ browser.current_url # => "https://www.google.com/"
329
373
  Returns current top window title
330
374
 
331
375
  ```ruby
332
- browser.go_to("https://google.com/")
333
- browser.current_title # => "Google"
376
+ page.go_to("https://google.com/")
377
+ page.current_title # => "Google"
334
378
  ```
335
379
 
336
380
  #### body : `String`
@@ -338,8 +382,8 @@ browser.current_title # => "Google"
338
382
  Returns current page's html.
339
383
 
340
384
  ```ruby
341
- browser.go_to("https://google.com/")
342
- browser.body # => '<html itemscope="" itemtype="http://schema.org/WebPage" lang="ru"><head>...
385
+ page.go_to("https://google.com/")
386
+ page.body # => '<html itemscope="" itemtype="http://schema.org/WebPage" lang="ru"><head>...
343
387
  ```
344
388
 
345
389
 
@@ -357,20 +401,29 @@ Saves screenshot on a disk or returns it as base64.
357
401
  * :format `String` "jpeg" | "png"
358
402
  * :quality `Integer` 0-100 works for jpeg only
359
403
  * :full `Boolean` whether you need full page screenshot or a viewport
360
- * :selector `String` css selector for given element
404
+ * :selector `String` css selector for given element, optional
405
+ * :area `Hash` area for screenshot, optional
406
+ * :x `Integer`
407
+ * :y `Integer`
408
+ * :width `Integer`
409
+ * :height `Integer`
361
410
  * :scale `Float` zoom in/out
362
411
  * :background_color `Ferrum::RGBA.new(0, 0, 0, 0.0)` to have specific background color
363
412
 
364
413
  ```ruby
365
- browser.go_to("https://google.com/")
414
+ page.go_to("https://google.com/")
366
415
  # Save on the disk in PNG
367
- browser.screenshot(path: "google.png") # => 134660
416
+ page.screenshot(path: "google.png") # => 134660
368
417
  # Save on the disk in JPG
369
- browser.screenshot(path: "google.jpg") # => 30902
418
+ page.screenshot(path: "google.jpg") # => 30902
370
419
  # Save to Base64 the whole page not only viewport and reduce quality
371
- browser.screenshot(full: true, quality: 60) # "iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAYAAAC6uhUNAAAAAXNSR0IArs4c6Q...
420
+ page.screenshot(full: true, quality: 60, encoding: :base64) # "iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAYAAAC6uhUNAAAAAXNSR0IArs4c6Q...
421
+ # Save on the disk with the selected element in PNG
422
+ page.screenshot(path: "google.png", selector: 'textarea') # => 11340
423
+ # Save to Base64 with an area of the page in PNG
424
+ page.screenshot(path: "google.png", area: { x: 0, y: 0, width: 400, height: 300 }) # => 54239
372
425
  # Save with specific background color
373
- browser.screenshot(background_color: Ferrum::RGBA.new(0, 0, 0, 0.0))
426
+ page.screenshot(background_color: Ferrum::RGBA.new(0, 0, 0, 0.0))
374
427
  ```
375
428
 
376
429
  #### pdf(\*\*options) : `String` | `Boolean`
@@ -391,9 +444,9 @@ Saves PDF on a disk or returns it as base64.
391
444
  * See other [native options](https://chromedevtools.github.io/devtools-protocol/tot/Page#method-printToPDF) you can pass
392
445
 
393
446
  ```ruby
394
- browser.go_to("https://google.com/")
447
+ page.go_to("https://google.com/")
395
448
  # Save to disk as a PDF
396
- browser.pdf(path: "google.pdf", paper_width: 1.0, paper_height: 1.0) # => true
449
+ page.pdf(path: "google.pdf", paper_width: 1.0, paper_height: 1.0) # => true
397
450
  ```
398
451
 
399
452
  #### mhtml(\*\*options) : `String` | `Integer`
@@ -404,33 +457,14 @@ Saves MHTML on a disk or returns it as a string.
404
457
  * :path `String` to save a file on the disk.
405
458
 
406
459
  ```ruby
407
- browser.go_to("https://google.com/")
408
- browser.mhtml(path: "google.mhtml") # => 87742
409
- ```
410
-
411
-
412
- ## Cleaning Up
413
-
414
- #### reset
415
-
416
- Closes browser tabs opened by the `Browser` instance.
417
-
418
- ```ruby
419
- # connect to a long-running Chrome process
420
- browser = Ferrum::Browser.new(url: 'http://localhost:9222')
421
-
422
- browser.go_to("https://github.com/")
423
-
424
- # clean up, lest the tab stays there hanging forever
425
- browser.reset
426
-
427
- browser.quit
460
+ page.go_to("https://google.com/")
461
+ page.mhtml(path: "google.mhtml") # => 87742
428
462
  ```
429
463
 
430
464
 
431
465
  ## Network
432
466
 
433
- `browser.network`
467
+ `page.network`
434
468
 
435
469
  #### traffic `Array<Network::Exchange>`
436
470
 
@@ -438,8 +472,8 @@ Returns all information about network traffic as `Network::Exchange` instance
438
472
  which in general is a wrapper around `request`, `response` and `error`.
439
473
 
440
474
  ```ruby
441
- browser.go_to("https://github.com/")
442
- browser.network.traffic # => [#<Ferrum::Network::Exchange, ...]
475
+ page.go_to("https://github.com/")
476
+ page.network.traffic # => [#<Ferrum::Network::Exchange, ...]
443
477
  ```
444
478
 
445
479
  #### request : `Network::Request`
@@ -447,8 +481,8 @@ browser.network.traffic # => [#<Ferrum::Network::Exchange, ...]
447
481
  Page request of the main frame.
448
482
 
449
483
  ```ruby
450
- browser.go_to("https://github.com/")
451
- browser.network.request # => #<Ferrum::Network::Request...
484
+ page.go_to("https://github.com/")
485
+ page.network.request # => #<Ferrum::Network::Request...
452
486
  ```
453
487
 
454
488
  #### response : `Network::Response`
@@ -456,8 +490,8 @@ browser.network.request # => #<Ferrum::Network::Request...
456
490
  Page response of the main frame.
457
491
 
458
492
  ```ruby
459
- browser.go_to("https://github.com/")
460
- browser.network.response # => #<Ferrum::Network::Response...
493
+ page.go_to("https://github.com/")
494
+ page.network.response # => #<Ferrum::Network::Response...
461
495
  ```
462
496
 
463
497
  #### status : `Integer`
@@ -466,8 +500,8 @@ Contains the status code of the main page response (e.g., 200 for a
466
500
  success). This is just a shortcut for `response.status`.
467
501
 
468
502
  ```ruby
469
- browser.go_to("https://github.com/")
470
- browser.network.status # => 200
503
+ page.go_to("https://github.com/")
504
+ page.network.status # => 200
471
505
  ```
472
506
 
473
507
  #### wait_for_idle(\*\*options)
@@ -483,22 +517,22 @@ Waits for network idle or raises `Ferrum::TimeoutError` error
483
517
  by default
484
518
 
485
519
  ```ruby
486
- browser.go_to("https://example.com/")
487
- browser.at_xpath("//a[text() = 'No UI changes button']").click
488
- browser.network.wait_for_idle
520
+ page.go_to("https://example.com/")
521
+ page.at_xpath("//a[text() = 'No UI changes button']").click
522
+ page.network.wait_for_idle
489
523
  ```
490
524
 
491
525
  #### clear(type)
492
526
 
493
- Clear browser's cache or collected traffic.
527
+ Clear page's cache or collected traffic.
494
528
 
495
529
  * type `Symbol` it is either `:traffic` or `:cache`
496
530
 
497
531
  ```ruby
498
- traffic = browser.network.traffic # => []
499
- browser.go_to("https://github.com/")
532
+ traffic = page.network.traffic # => []
533
+ page.go_to("https://github.com/")
500
534
  traffic.size # => 51
501
- browser.network.clear(:traffic)
535
+ page.network.clear(:traffic)
502
536
  traffic.size # => 0
503
537
  ```
504
538
 
@@ -514,8 +548,9 @@ continue them.
514
548
 
515
549
  ```ruby
516
550
  browser = Ferrum::Browser.new
517
- browser.network.intercept
518
- browser.on(:request) do |request|
551
+ page = browser.create_page
552
+ page.network.intercept
553
+ page.on(:request) do |request|
519
554
  if request.match?(/bla-bla/)
520
555
  request.abort
521
556
  elsif request.match?(/lorem/)
@@ -524,7 +559,7 @@ browser.on(:request) do |request|
524
559
  request.continue
525
560
  end
526
561
  end
527
- browser.go_to("https://google.com")
562
+ page.go_to("https://google.com")
528
563
  ```
529
564
 
530
565
  #### authorize(\*\*options, &block)
@@ -539,10 +574,10 @@ If site or proxy uses authorization you can provide credentials using this metho
539
574
  care about unwanted requests just call `request.continue`.
540
575
 
541
576
  ```ruby
542
- browser.network.authorize(user: "login", password: "pass") { |req| req.continue }
543
- browser.go_to("http://example.com/authenticated")
544
- puts browser.network.status # => 200
545
- puts browser.body # => Welcome, authenticated client
577
+ page.network.authorize(user: "login", password: "pass") { |req| req.continue }
578
+ page.go_to("http://example.com/authenticated")
579
+ puts page.network.status # => 200
580
+ puts page.body # => Welcome, authenticated client
546
581
  ```
547
582
 
548
583
  Since Chrome implements authorize using request interception you must continue or abort authorized requests. If you
@@ -551,8 +586,9 @@ block, so this is version doesn't pass block and can work just fine:
551
586
 
552
587
  ```ruby
553
588
  browser = Ferrum::Browser.new
554
- browser.network.intercept
555
- browser.on(:request) do |request|
589
+ page = browser.create_page
590
+ page.network.intercept
591
+ page.on(:request) do |request|
556
592
  if request.resource_type == "Image"
557
593
  request.abort
558
594
  else
@@ -560,9 +596,9 @@ browser.on(:request) do |request|
560
596
  end
561
597
  end
562
598
 
563
- browser.network.authorize(user: "login", password: "pass", type: :proxy)
599
+ page.network.authorize(user: "login", password: "pass", type: :proxy)
564
600
 
565
- browser.go_to("https://google.com")
601
+ page.go_to("https://google.com")
566
602
  ```
567
603
 
568
604
  You used to call `authorize` method without block, but since it's implemented using request interception there could be
@@ -585,8 +621,8 @@ Activates emulation of network conditions.
585
621
  bluetooth, ethernet, wifi, wimax, other. `nil` by default
586
622
 
587
623
  ```ruby
588
- browser.network.emulate_network_conditions(connection_type: "cellular2g")
589
- browser.go_to("https://github.com/")
624
+ page.network.emulate_network_conditions(connection_type: "cellular2g")
625
+ page.go_to("https://github.com/")
590
626
  ```
591
627
 
592
628
  #### offline_mode
@@ -594,8 +630,59 @@ browser.go_to("https://github.com/")
594
630
  Activates offline mode for a page.
595
631
 
596
632
  ```ruby
597
- browser.network.offline_mode
598
- browser.go_to("https://github.com/") # => Ferrum::StatusError (Request to https://github.com/ failed to reach server, check DNS and server status)
633
+ page.network.offline_mode
634
+ page.go_to("https://github.com/") # => Ferrum::StatusError (Request to https://github.com/ failed(net::ERR_INTERNET_DISCONNECTED))
635
+ ```
636
+
637
+ #### cache(disable: `Boolean`)
638
+
639
+ Toggles ignoring cache for each request. If true, cache will not be used.
640
+
641
+ ```ruby
642
+ page.network.cache(disable: true)
643
+ ```
644
+
645
+
646
+ ## Downloads
647
+
648
+ `page.downloads`
649
+
650
+ #### files `Array<Hash>`
651
+
652
+ Returns all information about downloaded files as a `Hash`.
653
+
654
+ ```ruby
655
+ page.go_to("http://localhost/attachment.pdf")
656
+ page.downloads.files # => [{"frameId"=>"E3316DF1B5383D38F8ADF7485005FDE3", "guid"=>"11a68745-98ac-4d54-9b57-9f9016c268b3", "url"=>"http://localhost/attachment.pdf", "suggestedFilename"=>"attachment.pdf", "totalBytes"=>4911, "receivedBytes"=>4911, "state"=>"completed"}]
657
+ ```
658
+
659
+ #### wait(timeout)
660
+
661
+ Waits until the download is finished.
662
+
663
+ ```ruby
664
+ page.go_to("http://localhost/attachment.pdf")
665
+ page.downloads.wait
666
+ ```
667
+
668
+ or
669
+
670
+ ```ruby
671
+ page.go_to("http://localhost/page")
672
+ page.downloads.wait { page.at_css("#download").click }
673
+ ```
674
+
675
+ #### set_behavior(\*\*options)
676
+
677
+ Sets behavior in case of file to be downloaded.
678
+
679
+ * options `Hash`
680
+ * :save_path `String` absolute path of where to store the file
681
+ * :behavior `Symbol` `deny | allow | allowAndName | default`, `allow` by default
682
+
683
+ ```ruby
684
+ page.go_to("https://example.com/")
685
+ page.downloads.set_behavior(save_path: "/tmp", behavior: :allow)
599
686
  ```
600
687
 
601
688
 
@@ -604,13 +691,13 @@ browser.go_to("https://github.com/") # => Ferrum::StatusError (Request to https:
604
691
  You can set a proxy with a `:proxy` option:
605
692
 
606
693
  ```ruby
607
- browser = Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", user: "user", password: "pa$$" })
694
+ Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", user: "user", password: "pa$$" })
608
695
  ```
609
696
 
610
697
  `:bypass` can specify semi-colon-separated list of hosts for which proxy shouldn't be used:
611
698
 
612
699
  ```ruby
613
- browser = Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", bypass: "*.google.com;*foo.com" })
700
+ Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", bypass: "*.google.com;*foo.com" })
614
701
  ```
615
702
 
616
703
  In general passing a proxy option when instantiating a browser results in a browser running with proxy command line
@@ -634,7 +721,7 @@ end
634
721
 
635
722
  ### Mouse
636
723
 
637
- `browser.mouse`
724
+ `page.mouse`
638
725
 
639
726
  #### scroll_to(x, y)
640
727
 
@@ -646,8 +733,8 @@ Scroll page to a given x, y
646
733
  displayed in the upper left
647
734
 
648
735
  ```ruby
649
- browser.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
650
- browser.mouse.scroll_to(0, 400)
736
+ page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
737
+ page.mouse.scroll_to(0, 400)
651
738
  ```
652
739
 
653
740
  #### click(\*\*options) : `Mouse`
@@ -691,7 +778,7 @@ Mouse move to given x and y.
691
778
 
692
779
  ### Keyboard
693
780
 
694
- browser.keyboard
781
+ `page.keyboard`
695
782
 
696
783
  #### down(key) : `Keyboard`
697
784
 
@@ -721,14 +808,14 @@ Returns bitfield for a given keys
721
808
 
722
809
  ## Cookies
723
810
 
724
- `browser.cookies`
811
+ `page.cookies`
725
812
 
726
813
  #### all : `Hash<String, Cookie>`
727
814
 
728
815
  Returns cookies hash
729
816
 
730
817
  ```ruby
731
- browser.cookies.all # => {"NID"=>#<Ferrum::Cookies::Cookie:0x0000558624b37a40 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>}
818
+ page.cookies.all # => {"NID"=>#<Ferrum::Cookies::Cookie:0x0000558624b37a40 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>}
732
819
  ```
733
820
 
734
821
  #### [](value) : `Cookie`
@@ -738,7 +825,7 @@ Returns cookie
738
825
  * value `String`
739
826
 
740
827
  ```ruby
741
- browser.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>
828
+ page.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>
742
829
  ```
743
830
 
744
831
  #### set(value) : `Boolean`
@@ -754,14 +841,14 @@ Sets a cookie
754
841
  * :httponly `Boolean`
755
842
 
756
843
  ```ruby
757
- browser.cookies.set(name: "stealth", value: "omg", domain: "google.com") # => true
844
+ page.cookies.set(name: "stealth", value: "omg", domain: "google.com") # => true
758
845
  ```
759
846
 
760
847
  * value `Cookie`
761
848
 
762
849
  ```ruby
763
- nid_cookie = browser.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88>
764
- browser.cookies.set(nid_cookie) # => true
850
+ nid_cookie = page.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88>
851
+ page.cookies.set(nid_cookie) # => true
765
852
  ```
766
853
 
767
854
  #### remove(\*\*options) : `Boolean`
@@ -774,7 +861,7 @@ Removes given cookie
774
861
  * :url `String`
775
862
 
776
863
  ```ruby
777
- browser.cookies.remove(name: "stealth", domain: "google.com") # => true
864
+ page.cookies.remove(name: "stealth", domain: "google.com") # => true
778
865
  ```
779
866
 
780
867
  #### clear : `Boolean`
@@ -782,12 +869,12 @@ browser.cookies.remove(name: "stealth", domain: "google.com") # => true
782
869
  Removes all cookies for current page
783
870
 
784
871
  ```ruby
785
- browser.cookies.clear # => true
872
+ page.cookies.clear # => true
786
873
  ```
787
874
 
788
875
  ## Headers
789
876
 
790
- `browser.headers`
877
+ `page.headers`
791
878
 
792
879
  #### get : `Hash`
793
880
 
@@ -821,7 +908,7 @@ Evaluate and return result for given JS expression
821
908
  simple value.
822
909
 
823
910
  ```ruby
824
- browser.evaluate("[window.scrollX, window.scrollY]")
911
+ page.evaluate("[window.scrollX, window.scrollY]")
825
912
  ```
826
913
 
827
914
  #### evaluate_async(expression, wait_time, \*args)
@@ -834,7 +921,7 @@ Evaluate asynchronous expression and return result
834
921
  simple value.
835
922
 
836
923
  ```ruby
837
- browser.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
924
+ page.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
838
925
  ```
839
926
 
840
927
  #### execute(expression, \*args)
@@ -846,7 +933,7 @@ Execute expression. Doesn't return the result
846
933
  simple value.
847
934
 
848
935
  ```ruby
849
- browser.execute(%(1 + 1)) # => true
936
+ page.execute(%(1 + 1)) # => true
850
937
  ```
851
938
 
852
939
  #### evaluate_on_new_document(expression)
@@ -872,7 +959,7 @@ JS
872
959
  * :type `String` - `text/javascript` by default
873
960
 
874
961
  ```ruby
875
- browser.add_script_tag(url: "http://example.com/stylesheet.css") # => true
962
+ page.add_script_tag(url: "http://example.com/stylesheet.css") # => true
876
963
  ```
877
964
 
878
965
  #### add_style_tag(\*\*options) : `Boolean`
@@ -883,7 +970,7 @@ browser.add_script_tag(url: "http://example.com/stylesheet.css") # => true
883
970
  * :content `String`
884
971
 
885
972
  ```ruby
886
- browser.add_style_tag(content: "h1 { font-size: 40px; }") # => true
973
+ page.add_style_tag(content: "h1 { font-size: 40px; }") # => true
887
974
 
888
975
  ```
889
976
  #### bypass_csp(\*\*options) : `Boolean`
@@ -892,11 +979,39 @@ browser.add_style_tag(content: "h1 { font-size: 40px; }") # => true
892
979
  * :enabled `Boolean`, `true` by default
893
980
 
894
981
  ```ruby
895
- browser.bypass_csp # => true
896
- browser.go_to("https://github.com/ruby-concurrency/concurrent-ruby/blob/master/docs-source/promises.in.md")
897
- browser.refresh
898
- browser.add_script_tag(content: "window.__injected = 42")
899
- browser.evaluate("window.__injected") # => 42
982
+ page.bypass_csp # => true
983
+ page.go_to("https://github.com/ruby-concurrency/concurrent-ruby/blob/master/docs-source/promises.in.md")
984
+ page.refresh
985
+ page.add_script_tag(content: "window.__injected = 42")
986
+ page.evaluate("window.__injected") # => 42
987
+ ```
988
+
989
+
990
+ ## Emulation
991
+
992
+ #### disable_javascript
993
+
994
+ Disables Javascripts from the loaded HTML source.
995
+ You can still evaluate JavaScript with `evaluate` or `execute`.
996
+ Returns nothing.
997
+
998
+ ```ruby
999
+ page.disable_javascript
1000
+ ```
1001
+
1002
+
1003
+ #### set_viewport
1004
+
1005
+ Overrides device screen dimensions and emulates viewport.
1006
+
1007
+ * options `Hash`
1008
+ * :width `Integer`, viewport width. `0` by default
1009
+ * :height `Integer`, viewport height. `0` by default
1010
+ * :scale_factor `Float`, device scale factor. `0` by default
1011
+ * :mobile `Boolean`, whether to emulate mobile device. `false` by default
1012
+
1013
+ ```ruby
1014
+ page.set_viewport(width: 1000, height: 600, scale_factor: 3)
900
1015
  ```
901
1016
 
902
1017
 
@@ -907,8 +1022,8 @@ browser.evaluate("window.__injected") # => 42
907
1022
  Returns all the frames current page have.
908
1023
 
909
1024
  ```ruby
910
- browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
911
- browser.frames # =>
1025
+ page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
1026
+ page.frames # =>
912
1027
  # [
913
1028
  # #<Ferrum::Frame @id="C6D104CE454A025FBCF22B98DE612B12" @parent_id=nil @name=nil @state=:stopped_loading @execution_id=1>,
914
1029
  # #<Ferrum::Frame @id="C09C4E4404314AAEAE85928EAC109A93" @parent_id="C6D104CE454A025FBCF22B98DE612B12" @state=:stopped_loading @execution_id=2>,
@@ -930,7 +1045,7 @@ Find frame by given options.
930
1045
  * :name `String` - Frame's name if there's one
931
1046
 
932
1047
  ```ruby
933
- browser.frame_by(id: "C6D104CE454A025FBCF22B98DE612B12")
1048
+ page.frame_by(id: "C6D104CE454A025FBCF22B98DE612B12")
934
1049
  ```
935
1050
 
936
1051
 
@@ -966,8 +1081,8 @@ One of the states frame's in:
966
1081
  Returns current frame's location href.
967
1082
 
968
1083
  ```ruby
969
- browser.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
970
- frame = browser.frames[1]
1084
+ page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
1085
+ frame = page.frames[1]
971
1086
  frame.url # => https://interactive-examples.mdn.mozilla.net/pages/tabbed/iframe.html
972
1087
  ```
973
1088
 
@@ -976,8 +1091,8 @@ frame.url # => https://interactive-examples.mdn.mozilla.net/pages/tabbed/iframe.
976
1091
  Returns current frame's title.
977
1092
 
978
1093
  ```ruby
979
- browser.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
980
- frame = browser.frames[1]
1094
+ page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
1095
+ frame = page.frames[1]
981
1096
  frame.title # => HTML Demo: <iframe>
982
1097
  ```
983
1098
 
@@ -986,8 +1101,8 @@ frame.title # => HTML Demo: <iframe>
986
1101
  If current frame is the main frame of the page (top of the tree).
987
1102
 
988
1103
  ```ruby
989
- browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
990
- frame = browser.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
1104
+ page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
1105
+ frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
991
1106
  frame.main? # => false
992
1107
  ```
993
1108
 
@@ -996,8 +1111,8 @@ frame.main? # => false
996
1111
  Returns current frame's top window location href.
997
1112
 
998
1113
  ```ruby
999
- browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
1000
- frame = browser.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
1114
+ page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
1115
+ frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
1001
1116
  frame.current_url # => "https://www.w3schools.com/tags/tag_frame.asp"
1002
1117
  ```
1003
1118
 
@@ -1006,8 +1121,8 @@ frame.current_url # => "https://www.w3schools.com/tags/tag_frame.asp"
1006
1121
  Returns current frame's top window title.
1007
1122
 
1008
1123
  ```ruby
1009
- browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
1010
- frame = browser.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
1124
+ page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
1125
+ frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
1011
1126
  frame.current_title # => "HTML frame tag"
1012
1127
  ```
1013
1128
 
@@ -1016,8 +1131,8 @@ frame.current_title # => "HTML frame tag"
1016
1131
  Returns current frame's html.
1017
1132
 
1018
1133
  ```ruby
1019
- browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
1020
- frame = browser.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
1134
+ page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
1135
+ frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
1021
1136
  frame.body # => "<html><head></head><body></body></html>"
1022
1137
  ```
1023
1138
 
@@ -1026,8 +1141,8 @@ frame.body # => "<html><head></head><body></body></html>"
1026
1141
  Returns current frame's doctype.
1027
1142
 
1028
1143
  ```ruby
1029
- browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
1030
- browser.main_frame.doctype # => "<!DOCTYPE html>"
1144
+ page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
1145
+ page.main_frame.doctype # => "<!DOCTYPE html>"
1031
1146
  ```
1032
1147
 
1033
1148
  #### content = html
@@ -1037,8 +1152,8 @@ Sets a content of a given frame.
1037
1152
  * html `String`
1038
1153
 
1039
1154
  ```ruby
1040
- browser.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
1041
- frame = browser.frames[1]
1155
+ page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
1156
+ frame = page.frames[1]
1042
1157
  frame.body # <html lang="en"><head><style>body {transition: opacity ease-in 0.2s; }...
1043
1158
  frame.content = "<html><head></head><body><p>lol</p></body></html>"
1044
1159
  frame.body # => <html><head></head><body><p>lol</p></body></html>
@@ -1058,15 +1173,14 @@ Accept dialog with given text or default prompt if applicable
1058
1173
  Dismiss dialog
1059
1174
 
1060
1175
  ```ruby
1061
- browser = Ferrum::Browser.new
1062
- browser.on(:dialog) do |dialog|
1176
+ page.on(:dialog) do |dialog|
1063
1177
  if dialog.match?(/bla-bla/)
1064
1178
  dialog.accept
1065
1179
  else
1066
1180
  dialog.dismiss
1067
1181
  end
1068
1182
  end
1069
- browser.go_to("https://google.com")
1183
+ page.go_to("https://google.com")
1070
1184
  ```
1071
1185
 
1072
1186
 
@@ -1086,10 +1200,9 @@ Sets playback rate of CSS animations
1086
1200
  * value `Integer`
1087
1201
 
1088
1202
  ```ruby
1089
- browser = Ferrum::Browser.new
1090
- browser.playback_rate = 2000
1091
- browser.go_to("https://google.com")
1092
- browser.playback_rate # => 2000
1203
+ page.playback_rate = 2000
1204
+ page.go_to("https://google.com")
1205
+ page.playback_rate # => 2000
1093
1206
  ```
1094
1207
 
1095
1208
 
@@ -1103,7 +1216,7 @@ Returns [Frame](https://github.com/rubycdp/ferrum#frame) object for current node
1103
1216
  [Finders](https://github.com/rubycdp/ferrum#Finders) for that object:
1104
1217
 
1105
1218
  ```ruby
1106
- frame = browser.at_xpath("//iframe").frame # => Frame
1219
+ frame = page.at_xpath("//iframe").frame # => Frame
1107
1220
  frame.at_css("//a[text() = 'Log in']") # => Node
1108
1221
  ```
1109
1222
 
@@ -1128,19 +1241,21 @@ frame.at_css("//a[text() = 'Log in']") # => Node
1128
1241
  #### evaluate
1129
1242
  #### selected : `Array<Node>`
1130
1243
  #### select
1244
+ #### scroll_into_view
1245
+ #### in_viewport?(of: `Node | nil`) : `Boolean`
1131
1246
 
1132
1247
  (chainable) Selects options by passed attribute.
1133
1248
 
1134
1249
  ```ruby
1135
- browser.at_xpath("//*[select]").select(["1"]) # => Node (select)
1136
- browser.at_xpath("//*[select]").select(["text"], by: :text) # => Node (select)
1250
+ page.at_xpath("//*[select]").select(["1"]) # => Node (select)
1251
+ page.at_xpath("//*[select]").select(["text"], by: :text) # => Node (select)
1137
1252
  ```
1138
1253
 
1139
1254
  Accept string, array or strings:
1140
1255
  ```ruby
1141
- browser.at_xpath("//*[select]").select("1")
1142
- browser.at_xpath("//*[select]").select("1", "2")
1143
- browser.at_xpath("//*[select]").select(["1", "2"])
1256
+ page.at_xpath("//*[select]").select("1")
1257
+ page.at_xpath("//*[select]").select("1", "2")
1258
+ page.at_xpath("//*[select]").select(["1", "2"])
1144
1259
  ```
1145
1260
 
1146
1261
 
@@ -1171,6 +1286,25 @@ Accepts block, records trace and by default returns trace data from `Tracing.tra
1171
1286
  only one trace config can be active at a time per browser.
1172
1287
 
1173
1288
 
1289
+ ## Clean Up
1290
+
1291
+ #### reset
1292
+
1293
+ Closes browser tabs opened by the `Browser` instance.
1294
+
1295
+ ```ruby
1296
+ # connect to a long-running Chrome process
1297
+ browser = Ferrum::Browser.new(url: 'http://localhost:9222')
1298
+
1299
+ browser.go_to("https://github.com/")
1300
+
1301
+ # clean up, lest the tab stays there hanging forever
1302
+ browser.reset
1303
+
1304
+ browser.quit
1305
+ ```
1306
+
1307
+
1174
1308
  ## Thread safety ##
1175
1309
 
1176
1310
  Ferrum is fully thread-safe. You can create one browser or a few as you wish and