ferrum 0.14 → 0.16
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/LICENSE +1 -1
- data/README.md +309 -158
- data/lib/ferrum/browser/command.rb +4 -0
- data/lib/ferrum/browser/options/chrome.rb +8 -3
- data/lib/ferrum/browser/options.rb +38 -25
- data/lib/ferrum/browser/process.rb +42 -17
- data/lib/ferrum/browser.rb +38 -50
- data/lib/ferrum/client/subscriber.rb +76 -0
- data/lib/ferrum/client/web_socket.rb +126 -0
- data/lib/ferrum/client.rb +171 -0
- data/lib/ferrum/context.rb +19 -15
- data/lib/ferrum/contexts.rb +46 -12
- data/lib/ferrum/cookies.rb +28 -1
- data/lib/ferrum/downloads.rb +60 -0
- data/lib/ferrum/errors.rb +10 -3
- data/lib/ferrum/headers.rb +1 -1
- data/lib/ferrum/network/exchange.rb +10 -1
- data/lib/ferrum/network/intercepted_request.rb +5 -5
- data/lib/ferrum/network/request.rb +9 -0
- data/lib/ferrum/network.rb +36 -24
- data/lib/ferrum/node.rb +11 -0
- data/lib/ferrum/page/frames.rb +7 -9
- data/lib/ferrum/page/screenshot.rb +54 -28
- data/lib/ferrum/page.rb +192 -118
- data/lib/ferrum/proxy.rb +1 -1
- data/lib/ferrum/target.rb +25 -5
- data/lib/ferrum/utils/elapsed_time.rb +0 -2
- data/lib/ferrum/utils/event.rb +19 -0
- data/lib/ferrum/utils/platform.rb +4 -0
- data/lib/ferrum/utils/thread.rb +18 -0
- data/lib/ferrum/version.rb +1 -1
- data/lib/ferrum.rb +3 -0
- metadata +28 -17
- data/lib/ferrum/browser/client.rb +0 -103
- data/lib/ferrum/browser/subscriber.rb +0 -36
- data/lib/ferrum/browser/web_socket.rb +0 -91
data/README.md
CHANGED
@@ -25,7 +25,6 @@ going to crawl sites you better use Ferrum or
|
|
25
25
|
* [Vessel](https://github.com/rubycdp/vessel) high-level web crawling framework
|
26
26
|
based on Ferrum and Mechanize.
|
27
27
|
|
28
|
-
|
29
28
|
## Index
|
30
29
|
|
31
30
|
* [Install](https://github.com/rubycdp/ferrum#install)
|
@@ -35,8 +34,8 @@ based on Ferrum and Mechanize.
|
|
35
34
|
* [Navigation](https://github.com/rubycdp/ferrum#navigation)
|
36
35
|
* [Finders](https://github.com/rubycdp/ferrum#finders)
|
37
36
|
* [Screenshots](https://github.com/rubycdp/ferrum#screenshots)
|
38
|
-
* [Cleaning Up](https://github.com/rubycdp/ferrum#cleaning-up)
|
39
37
|
* [Network](https://github.com/rubycdp/ferrum#network)
|
38
|
+
* [Downloads](https://github.com/rubycdp/ferrum#downloads)
|
40
39
|
* [Proxy](https://github.com/rubycdp/ferrum#proxy)
|
41
40
|
* [Mouse](https://github.com/rubycdp/ferrum#mouse)
|
42
41
|
* [Keyboard](https://github.com/rubycdp/ferrum#keyboard)
|
@@ -49,6 +48,7 @@ based on Ferrum and Mechanize.
|
|
49
48
|
* [Animation](https://github.com/rubycdp/ferrum#animation)
|
50
49
|
* [Node](https://github.com/rubycdp/ferrum#node)
|
51
50
|
* [Tracing](https://github.com/rubycdp/ferrum#tracing)
|
51
|
+
* [Clean Up](https://github.com/rubycdp/ferrum#clean-up)
|
52
52
|
* [Thread safety](https://github.com/rubycdp/ferrum#thread-safety)
|
53
53
|
* [Development](https://github.com/rubycdp/ferrum#development)
|
54
54
|
* [Contributing](https://github.com/rubycdp/ferrum#contributing)
|
@@ -61,7 +61,7 @@ There's no official Chrome or Chromium package for Linux don't install it this
|
|
61
61
|
way because it's either outdated or unofficial, both are bad. Download it from
|
62
62
|
official source for [Chrome](https://www.google.com/chrome/) or
|
63
63
|
[Chromium](https://www.chromium.org/getting-involved/download-chromium).
|
64
|
-
Chrome binary should be in the `PATH` or `BROWSER_PATH`
|
64
|
+
Chrome binary should be in the `PATH` or `BROWSER_PATH` and you can pass it as an
|
65
65
|
option to browser instance see `:browser_path` in
|
66
66
|
[Customization](https://github.com/rubycdp/ferrum#customization).
|
67
67
|
|
@@ -83,14 +83,17 @@ browser.screenshot(path: "google.png")
|
|
83
83
|
browser.quit
|
84
84
|
```
|
85
85
|
|
86
|
-
|
86
|
+
When you work with browser instance Ferrum creates and maintains a default page for you, in fact all the methods above
|
87
|
+
are sent to the `page` instance that is created in the `default_context` of the `browser` instance. You can interact
|
88
|
+
with a page created manually and this is preferred:
|
87
89
|
|
88
90
|
```ruby
|
89
91
|
browser = Ferrum::Browser.new
|
90
|
-
browser.
|
91
|
-
|
92
|
+
page = browser.create_page
|
93
|
+
page.go_to("https://google.com")
|
94
|
+
input = page.at_xpath("//input[@name='q']")
|
92
95
|
input.focus.type("Ruby headless driver for Chrome", :Enter)
|
93
|
-
|
96
|
+
page.at_css("a > h3").text # => "rubycdp/ferrum: Ruby Chrome/Chromium driver - GitHub"
|
94
97
|
browser.quit
|
95
98
|
```
|
96
99
|
|
@@ -98,8 +101,9 @@ Evaluate some JavaScript and get full width/height:
|
|
98
101
|
|
99
102
|
```ruby
|
100
103
|
browser = Ferrum::Browser.new
|
101
|
-
browser.
|
102
|
-
|
104
|
+
page = browser.create_page
|
105
|
+
page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
|
106
|
+
width, height = page.evaluate <<~JS
|
103
107
|
[document.documentElement.offsetWidth,
|
104
108
|
document.documentElement.offsetHeight]
|
105
109
|
JS
|
@@ -112,8 +116,9 @@ Do any mouse movements you like:
|
|
112
116
|
```ruby
|
113
117
|
# Trace a 100x100 square
|
114
118
|
browser = Ferrum::Browser.new
|
115
|
-
browser.
|
116
|
-
|
119
|
+
page = browser.create_page
|
120
|
+
page.go_to("https://google.com")
|
121
|
+
page.mouse
|
117
122
|
.move(x: 0, y: 0)
|
118
123
|
.down
|
119
124
|
.move(x: 0, y: 100)
|
@@ -148,6 +153,7 @@ Ferrum::Browser.new(options)
|
|
148
153
|
* `:headless` (String | Boolean) - Set browser as headless or not, `true` by default. You can set `"new"` to support
|
149
154
|
[new headless mode](https://developer.chrome.com/articles/new-headless/).
|
150
155
|
* `:xvfb` (Boolean) - Run browser in a virtual framebuffer, `false` by default.
|
156
|
+
* `:flatten` (Boolean) - Use one websocket connection to the browser and all the pages in flatten mode.
|
151
157
|
* `:window_size` (Array) - The dimensions of the browser window in which to
|
152
158
|
test, expressed as a 2-element array, e.g. [1024, 768]. Default: [1024, 768]
|
153
159
|
* `:extensions` (Array[String | Hash]) - An array of paths to files or JS
|
@@ -178,6 +184,8 @@ Ferrum::Browser.new(options)
|
|
178
184
|
* `:host` (String) - Remote debugging address for headless Chrome.
|
179
185
|
* `:url` (String) - URL for a running instance of Chrome. If this is set, a
|
180
186
|
browser process will not be spawned.
|
187
|
+
* `:ws_url` (String) - Websocket url for a running instance of Chrome. If this is set, a
|
188
|
+
browser process will not be spawned. It's higher priority than `:url`, setting both doesn't make sense.
|
181
189
|
* `:process_timeout` (Integer) - How long to wait for the Chrome process to
|
182
190
|
respond on startup.
|
183
191
|
* `:ws_max_receive_size` (Integer) - How big messages to accept from Chrome
|
@@ -198,7 +206,7 @@ Navigate page to.
|
|
198
206
|
configuring driver.
|
199
207
|
|
200
208
|
```ruby
|
201
|
-
|
209
|
+
page.go_to("https://github.com/")
|
202
210
|
```
|
203
211
|
|
204
212
|
#### back
|
@@ -206,9 +214,9 @@ browser.go_to("https://github.com/")
|
|
206
214
|
Navigate to the previous page in history.
|
207
215
|
|
208
216
|
```ruby
|
209
|
-
|
210
|
-
|
211
|
-
|
217
|
+
page.go_to("https://github.com/")
|
218
|
+
page.at_xpath("//a").click
|
219
|
+
page.back
|
212
220
|
```
|
213
221
|
|
214
222
|
#### forward
|
@@ -216,10 +224,10 @@ browser.back
|
|
216
224
|
Navigate to the next page in history.
|
217
225
|
|
218
226
|
```ruby
|
219
|
-
|
220
|
-
|
221
|
-
|
222
|
-
|
227
|
+
page.go_to("https://github.com/")
|
228
|
+
page.at_xpath("//a").click
|
229
|
+
page.back
|
230
|
+
page.forward
|
223
231
|
```
|
224
232
|
|
225
233
|
#### refresh
|
@@ -227,8 +235,8 @@ browser.forward
|
|
227
235
|
Reload current page.
|
228
236
|
|
229
237
|
```ruby
|
230
|
-
|
231
|
-
|
238
|
+
page.go_to("https://github.com/")
|
239
|
+
page.refresh
|
232
240
|
```
|
233
241
|
|
234
242
|
#### stop
|
@@ -236,8 +244,8 @@ browser.refresh
|
|
236
244
|
Stop all navigations and loading pending resources on the page
|
237
245
|
|
238
246
|
```ruby
|
239
|
-
|
240
|
-
|
247
|
+
page.go_to("https://github.com/")
|
248
|
+
page.stop
|
241
249
|
```
|
242
250
|
|
243
251
|
#### position = \*\*options
|
@@ -260,6 +268,37 @@ Get the position for the browser window
|
|
260
268
|
browser.position # => [10, 20]
|
261
269
|
```
|
262
270
|
|
271
|
+
#### window_bounds = \*\*options
|
272
|
+
|
273
|
+
Set window bounds
|
274
|
+
|
275
|
+
* options `Hash`
|
276
|
+
* :left `Integer`
|
277
|
+
* :top `Integer`
|
278
|
+
* :width `Integer`
|
279
|
+
* :height `Integer`
|
280
|
+
* :window_state `String`
|
281
|
+
|
282
|
+
```ruby
|
283
|
+
browser.window_bounds = { left: 10, top: 20, width: 1024, height: 768, window_state: "normal" }
|
284
|
+
```
|
285
|
+
|
286
|
+
#### window_bounds : `Hash<String, Integer | String>`
|
287
|
+
|
288
|
+
Get window bounds
|
289
|
+
|
290
|
+
```ruby
|
291
|
+
browser.window_bounds # => { "left": 0, "top": 1286, "width": 10, "height": 10, "windowState": "normal" }
|
292
|
+
```
|
293
|
+
|
294
|
+
#### window_id : `Integer`
|
295
|
+
|
296
|
+
Current window id
|
297
|
+
|
298
|
+
```ruby
|
299
|
+
browser.window_id # => 1
|
300
|
+
```
|
301
|
+
|
263
302
|
## Finders
|
264
303
|
|
265
304
|
#### at_css(selector, \*\*options) : `Node` | `nil`
|
@@ -272,8 +311,8 @@ provided node.
|
|
272
311
|
* :within `Node` | `nil`
|
273
312
|
|
274
313
|
```ruby
|
275
|
-
|
276
|
-
|
314
|
+
page.go_to("https://github.com/")
|
315
|
+
page.at_css("a[aria-label='Issues you created']") # => Node
|
277
316
|
```
|
278
317
|
|
279
318
|
|
@@ -287,8 +326,8 @@ document or provided node.
|
|
287
326
|
* :within `Node` | `nil`
|
288
327
|
|
289
328
|
```ruby
|
290
|
-
|
291
|
-
|
329
|
+
page.go_to("https://github.com/")
|
330
|
+
page.css("a[aria-label='Issues you created']") # => [Node]
|
292
331
|
```
|
293
332
|
|
294
333
|
#### at_xpath(selector, \*\*options) : `Node` | `nil`
|
@@ -300,8 +339,8 @@ Find node by xpath.
|
|
300
339
|
* :within `Node` | `nil`
|
301
340
|
|
302
341
|
```ruby
|
303
|
-
|
304
|
-
|
342
|
+
page.go_to("https://github.com/")
|
343
|
+
page.at_xpath("//a[@aria-label='Issues you created']") # => Node
|
305
344
|
```
|
306
345
|
|
307
346
|
#### xpath(selector, \*\*options) : `Array<Node>` | `[]`
|
@@ -313,8 +352,8 @@ Find nodes by xpath.
|
|
313
352
|
* :within `Node` | `nil`
|
314
353
|
|
315
354
|
```ruby
|
316
|
-
|
317
|
-
|
355
|
+
page.go_to("https://github.com/")
|
356
|
+
page.xpath("//a[@aria-label='Issues you created']") # => [Node]
|
318
357
|
```
|
319
358
|
|
320
359
|
#### current_url : `String`
|
@@ -322,8 +361,8 @@ browser.xpath("//a[@aria-label='Issues you created']") # => [Node]
|
|
322
361
|
Returns current top window location href.
|
323
362
|
|
324
363
|
```ruby
|
325
|
-
|
326
|
-
|
364
|
+
page.go_to("https://google.com/")
|
365
|
+
page.current_url # => "https://www.google.com/"
|
327
366
|
```
|
328
367
|
|
329
368
|
#### current_title : `String`
|
@@ -331,8 +370,8 @@ browser.current_url # => "https://www.google.com/"
|
|
331
370
|
Returns current top window title
|
332
371
|
|
333
372
|
```ruby
|
334
|
-
|
335
|
-
|
373
|
+
page.go_to("https://google.com/")
|
374
|
+
page.current_title # => "Google"
|
336
375
|
```
|
337
376
|
|
338
377
|
#### body : `String`
|
@@ -340,8 +379,8 @@ browser.current_title # => "Google"
|
|
340
379
|
Returns current page's html.
|
341
380
|
|
342
381
|
```ruby
|
343
|
-
|
344
|
-
|
382
|
+
page.go_to("https://google.com/")
|
383
|
+
page.body # => '<html itemscope="" itemtype="http://schema.org/WebPage" lang="ru"><head>...
|
345
384
|
```
|
346
385
|
|
347
386
|
|
@@ -356,23 +395,32 @@ Saves screenshot on a disk or returns it as base64.
|
|
356
395
|
`:binary` automatically
|
357
396
|
* :encoding `Symbol` `:base64` | `:binary` you can set it to return image as
|
358
397
|
Base64
|
359
|
-
* :format `String` "jpeg" | "png"
|
398
|
+
* :format `String` "jpeg" ("jpg") | "png" | "webp"
|
360
399
|
* :quality `Integer` 0-100 works for jpeg only
|
361
400
|
* :full `Boolean` whether you need full page screenshot or a viewport
|
362
|
-
* :selector `String` css selector for given element
|
401
|
+
* :selector `String` css selector for given element, optional
|
402
|
+
* :area `Hash` area for screenshot, optional
|
403
|
+
* :x `Integer`
|
404
|
+
* :y `Integer`
|
405
|
+
* :width `Integer`
|
406
|
+
* :height `Integer`
|
363
407
|
* :scale `Float` zoom in/out
|
364
408
|
* :background_color `Ferrum::RGBA.new(0, 0, 0, 0.0)` to have specific background color
|
365
409
|
|
366
410
|
```ruby
|
367
|
-
|
411
|
+
page.go_to("https://google.com/")
|
368
412
|
# Save on the disk in PNG
|
369
|
-
|
413
|
+
page.screenshot(path: "google.png") # => 134660
|
370
414
|
# Save on the disk in JPG
|
371
|
-
|
415
|
+
page.screenshot(path: "google.jpg") # => 30902
|
372
416
|
# Save to Base64 the whole page not only viewport and reduce quality
|
373
|
-
|
417
|
+
page.screenshot(full: true, quality: 60, encoding: :base64) # "iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAYAAAC6uhUNAAAAAXNSR0IArs4c6Q...
|
418
|
+
# Save on the disk with the selected element in PNG
|
419
|
+
page.screenshot(path: "google.png", selector: 'textarea') # => 11340
|
420
|
+
# Save to Base64 with an area of the page in PNG
|
421
|
+
page.screenshot(path: "google.png", area: { x: 0, y: 0, width: 400, height: 300 }) # => 54239
|
374
422
|
# Save with specific background color
|
375
|
-
|
423
|
+
page.screenshot(background_color: Ferrum::RGBA.new(0, 0, 0, 0.0))
|
376
424
|
```
|
377
425
|
|
378
426
|
#### pdf(\*\*options) : `String` | `Boolean`
|
@@ -393,9 +441,9 @@ Saves PDF on a disk or returns it as base64.
|
|
393
441
|
* See other [native options](https://chromedevtools.github.io/devtools-protocol/tot/Page#method-printToPDF) you can pass
|
394
442
|
|
395
443
|
```ruby
|
396
|
-
|
444
|
+
page.go_to("https://google.com/")
|
397
445
|
# Save to disk as a PDF
|
398
|
-
|
446
|
+
page.pdf(path: "google.pdf", paper_width: 1.0, paper_height: 1.0) # => true
|
399
447
|
```
|
400
448
|
|
401
449
|
#### mhtml(\*\*options) : `String` | `Integer`
|
@@ -406,33 +454,14 @@ Saves MHTML on a disk or returns it as a string.
|
|
406
454
|
* :path `String` to save a file on the disk.
|
407
455
|
|
408
456
|
```ruby
|
409
|
-
|
410
|
-
|
411
|
-
```
|
412
|
-
|
413
|
-
|
414
|
-
## Cleaning Up
|
415
|
-
|
416
|
-
#### reset
|
417
|
-
|
418
|
-
Closes browser tabs opened by the `Browser` instance.
|
419
|
-
|
420
|
-
```ruby
|
421
|
-
# connect to a long-running Chrome process
|
422
|
-
browser = Ferrum::Browser.new(url: 'http://localhost:9222')
|
423
|
-
|
424
|
-
browser.go_to("https://github.com/")
|
425
|
-
|
426
|
-
# clean up, lest the tab stays there hanging forever
|
427
|
-
browser.reset
|
428
|
-
|
429
|
-
browser.quit
|
457
|
+
page.go_to("https://google.com/")
|
458
|
+
page.mhtml(path: "google.mhtml") # => 87742
|
430
459
|
```
|
431
460
|
|
432
461
|
|
433
462
|
## Network
|
434
463
|
|
435
|
-
`
|
464
|
+
`page.network`
|
436
465
|
|
437
466
|
#### traffic `Array<Network::Exchange>`
|
438
467
|
|
@@ -440,8 +469,8 @@ Returns all information about network traffic as `Network::Exchange` instance
|
|
440
469
|
which in general is a wrapper around `request`, `response` and `error`.
|
441
470
|
|
442
471
|
```ruby
|
443
|
-
|
444
|
-
|
472
|
+
page.go_to("https://github.com/")
|
473
|
+
page.network.traffic # => [#<Ferrum::Network::Exchange, ...]
|
445
474
|
```
|
446
475
|
|
447
476
|
#### request : `Network::Request`
|
@@ -449,8 +478,8 @@ browser.network.traffic # => [#<Ferrum::Network::Exchange, ...]
|
|
449
478
|
Page request of the main frame.
|
450
479
|
|
451
480
|
```ruby
|
452
|
-
|
453
|
-
|
481
|
+
page.go_to("https://github.com/")
|
482
|
+
page.network.request # => #<Ferrum::Network::Request...
|
454
483
|
```
|
455
484
|
|
456
485
|
#### response : `Network::Response`
|
@@ -458,8 +487,8 @@ browser.network.request # => #<Ferrum::Network::Request...
|
|
458
487
|
Page response of the main frame.
|
459
488
|
|
460
489
|
```ruby
|
461
|
-
|
462
|
-
|
490
|
+
page.go_to("https://github.com/")
|
491
|
+
page.network.response # => #<Ferrum::Network::Response...
|
463
492
|
```
|
464
493
|
|
465
494
|
#### status : `Integer`
|
@@ -468,13 +497,13 @@ Contains the status code of the main page response (e.g., 200 for a
|
|
468
497
|
success). This is just a shortcut for `response.status`.
|
469
498
|
|
470
499
|
```ruby
|
471
|
-
|
472
|
-
|
500
|
+
page.go_to("https://github.com/")
|
501
|
+
page.network.status # => 200
|
473
502
|
```
|
474
503
|
|
475
|
-
#### wait_for_idle(\*\*options)
|
504
|
+
#### wait_for_idle(\*\*options) : `Boolean`
|
476
505
|
|
477
|
-
Waits for network idle
|
506
|
+
Waits for network idle, returns `true` in case of success and `false` if there are still connections.
|
478
507
|
|
479
508
|
* options `Hash`
|
480
509
|
* :connections `Integer` how many connections are allowed for network to be
|
@@ -485,22 +514,32 @@ Waits for network idle or raises `Ferrum::TimeoutError` error
|
|
485
514
|
by default
|
486
515
|
|
487
516
|
```ruby
|
488
|
-
|
489
|
-
|
490
|
-
|
517
|
+
page.go_to("https://example.com/")
|
518
|
+
page.at_xpath("//a[text() = 'No UI changes button']").click
|
519
|
+
page.network.wait_for_idle # => true
|
520
|
+
```
|
521
|
+
|
522
|
+
#### wait_for_idle!(\*\*options)
|
523
|
+
|
524
|
+
Waits for network idle or raises `Ferrum::TimeoutError` error. Accepts same arguments as `wait_for_idle`.
|
525
|
+
|
526
|
+
```ruby
|
527
|
+
page.go_to("https://example.com/")
|
528
|
+
page.at_xpath("//a[text() = 'No UI changes button']").click
|
529
|
+
page.network.wait_for_idle! # might raise an error
|
491
530
|
```
|
492
531
|
|
493
532
|
#### clear(type)
|
494
533
|
|
495
|
-
Clear
|
534
|
+
Clear page's cache or collected traffic.
|
496
535
|
|
497
536
|
* type `Symbol` it is either `:traffic` or `:cache`
|
498
537
|
|
499
538
|
```ruby
|
500
|
-
traffic =
|
501
|
-
|
539
|
+
traffic = page.network.traffic # => []
|
540
|
+
page.go_to("https://github.com/")
|
502
541
|
traffic.size # => 51
|
503
|
-
|
542
|
+
page.network.clear(:traffic)
|
504
543
|
traffic.size # => 0
|
505
544
|
```
|
506
545
|
|
@@ -516,8 +555,9 @@ continue them.
|
|
516
555
|
|
517
556
|
```ruby
|
518
557
|
browser = Ferrum::Browser.new
|
519
|
-
browser.
|
520
|
-
|
558
|
+
page = browser.create_page
|
559
|
+
page.network.intercept
|
560
|
+
page.on(:request) do |request|
|
521
561
|
if request.match?(/bla-bla/)
|
522
562
|
request.abort
|
523
563
|
elsif request.match?(/lorem/)
|
@@ -526,7 +566,7 @@ browser.on(:request) do |request|
|
|
526
566
|
request.continue
|
527
567
|
end
|
528
568
|
end
|
529
|
-
|
569
|
+
page.go_to("https://google.com")
|
530
570
|
```
|
531
571
|
|
532
572
|
#### authorize(\*\*options, &block)
|
@@ -541,10 +581,10 @@ If site or proxy uses authorization you can provide credentials using this metho
|
|
541
581
|
care about unwanted requests just call `request.continue`.
|
542
582
|
|
543
583
|
```ruby
|
544
|
-
|
545
|
-
|
546
|
-
puts
|
547
|
-
puts
|
584
|
+
page.network.authorize(user: "login", password: "pass") { |req| req.continue }
|
585
|
+
page.go_to("http://example.com/authenticated")
|
586
|
+
puts page.network.status # => 200
|
587
|
+
puts page.body # => Welcome, authenticated client
|
548
588
|
```
|
549
589
|
|
550
590
|
Since Chrome implements authorize using request interception you must continue or abort authorized requests. If you
|
@@ -553,8 +593,9 @@ block, so this is version doesn't pass block and can work just fine:
|
|
553
593
|
|
554
594
|
```ruby
|
555
595
|
browser = Ferrum::Browser.new
|
556
|
-
browser.
|
557
|
-
|
596
|
+
page = browser.create_page
|
597
|
+
page.network.intercept
|
598
|
+
page.on(:request) do |request|
|
558
599
|
if request.resource_type == "Image"
|
559
600
|
request.abort
|
560
601
|
else
|
@@ -562,9 +603,9 @@ browser.on(:request) do |request|
|
|
562
603
|
end
|
563
604
|
end
|
564
605
|
|
565
|
-
|
606
|
+
page.network.authorize(user: "login", password: "pass", type: :proxy)
|
566
607
|
|
567
|
-
|
608
|
+
page.go_to("https://google.com")
|
568
609
|
```
|
569
610
|
|
570
611
|
You used to call `authorize` method without block, but since it's implemented using request interception there could be
|
@@ -587,8 +628,8 @@ Activates emulation of network conditions.
|
|
587
628
|
bluetooth, ethernet, wifi, wimax, other. `nil` by default
|
588
629
|
|
589
630
|
```ruby
|
590
|
-
|
591
|
-
|
631
|
+
page.network.emulate_network_conditions(connection_type: "cellular2g")
|
632
|
+
page.go_to("https://github.com/")
|
592
633
|
```
|
593
634
|
|
594
635
|
#### offline_mode
|
@@ -596,8 +637,8 @@ browser.go_to("https://github.com/")
|
|
596
637
|
Activates offline mode for a page.
|
597
638
|
|
598
639
|
```ruby
|
599
|
-
|
600
|
-
|
640
|
+
page.network.offline_mode
|
641
|
+
page.go_to("https://github.com/") # => Ferrum::StatusError (Request to https://github.com/ failed(net::ERR_INTERNET_DISCONNECTED))
|
601
642
|
```
|
602
643
|
|
603
644
|
#### cache(disable: `Boolean`)
|
@@ -605,21 +646,65 @@ browser.go_to("https://github.com/") # => Ferrum::StatusError (Request to https:
|
|
605
646
|
Toggles ignoring cache for each request. If true, cache will not be used.
|
606
647
|
|
607
648
|
```ruby
|
608
|
-
|
649
|
+
page.network.cache(disable: true)
|
609
650
|
```
|
610
651
|
|
652
|
+
|
653
|
+
## Downloads
|
654
|
+
|
655
|
+
`page.downloads`
|
656
|
+
|
657
|
+
#### files `Array<Hash>`
|
658
|
+
|
659
|
+
Returns all information about downloaded files as a `Hash`.
|
660
|
+
|
661
|
+
```ruby
|
662
|
+
page.go_to("http://localhost/attachment.pdf")
|
663
|
+
page.downloads.files # => [{"frameId"=>"E3316DF1B5383D38F8ADF7485005FDE3", "guid"=>"11a68745-98ac-4d54-9b57-9f9016c268b3", "url"=>"http://localhost/attachment.pdf", "suggestedFilename"=>"attachment.pdf", "totalBytes"=>4911, "receivedBytes"=>4911, "state"=>"completed"}]
|
664
|
+
```
|
665
|
+
|
666
|
+
#### wait(timeout)
|
667
|
+
|
668
|
+
Waits until the download is finished.
|
669
|
+
|
670
|
+
```ruby
|
671
|
+
page.go_to("http://localhost/attachment.pdf")
|
672
|
+
page.downloads.wait
|
673
|
+
```
|
674
|
+
|
675
|
+
or
|
676
|
+
|
677
|
+
```ruby
|
678
|
+
page.go_to("http://localhost/page")
|
679
|
+
page.downloads.wait { page.at_css("#download").click }
|
680
|
+
```
|
681
|
+
|
682
|
+
#### set_behavior(\*\*options)
|
683
|
+
|
684
|
+
Sets behavior in case of file to be downloaded.
|
685
|
+
|
686
|
+
* options `Hash`
|
687
|
+
* :save_path `String` absolute path of where to store the file
|
688
|
+
* :behavior `Symbol` `deny | allow | allowAndName | default`, `allow` by default
|
689
|
+
|
690
|
+
```ruby
|
691
|
+
page.go_to("https://example.com/")
|
692
|
+
page.downloads.set_behavior(save_path: "/tmp", behavior: :allow)
|
693
|
+
```
|
694
|
+
|
695
|
+
|
611
696
|
## Proxy
|
612
697
|
|
613
698
|
You can set a proxy with a `:proxy` option:
|
614
699
|
|
615
700
|
```ruby
|
616
|
-
|
701
|
+
Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", user: "user", password: "pa$$" })
|
617
702
|
```
|
618
703
|
|
619
704
|
`:bypass` can specify semi-colon-separated list of hosts for which proxy shouldn't be used:
|
620
705
|
|
621
706
|
```ruby
|
622
|
-
|
707
|
+
Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", bypass: "*.google.com;*foo.com" })
|
623
708
|
```
|
624
709
|
|
625
710
|
In general passing a proxy option when instantiating a browser results in a browser running with proxy command line
|
@@ -643,7 +728,7 @@ end
|
|
643
728
|
|
644
729
|
### Mouse
|
645
730
|
|
646
|
-
`
|
731
|
+
`page.mouse`
|
647
732
|
|
648
733
|
#### scroll_to(x, y)
|
649
734
|
|
@@ -655,8 +740,8 @@ Scroll page to a given x, y
|
|
655
740
|
displayed in the upper left
|
656
741
|
|
657
742
|
```ruby
|
658
|
-
|
659
|
-
|
743
|
+
page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
|
744
|
+
page.mouse.scroll_to(0, 400)
|
660
745
|
```
|
661
746
|
|
662
747
|
#### click(\*\*options) : `Mouse`
|
@@ -700,7 +785,7 @@ Mouse move to given x and y.
|
|
700
785
|
|
701
786
|
### Keyboard
|
702
787
|
|
703
|
-
|
788
|
+
`page.keyboard`
|
704
789
|
|
705
790
|
#### down(key) : `Keyboard`
|
706
791
|
|
@@ -730,14 +815,14 @@ Returns bitfield for a given keys
|
|
730
815
|
|
731
816
|
## Cookies
|
732
817
|
|
733
|
-
`
|
818
|
+
`page.cookies`
|
734
819
|
|
735
820
|
#### all : `Hash<String, Cookie>`
|
736
821
|
|
737
822
|
Returns cookies hash
|
738
823
|
|
739
824
|
```ruby
|
740
|
-
|
825
|
+
page.cookies.all # => {"NID"=>#<Ferrum::Cookies::Cookie:0x0000558624b37a40 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>}
|
741
826
|
```
|
742
827
|
|
743
828
|
#### [](value) : `Cookie`
|
@@ -747,7 +832,7 @@ Returns cookie
|
|
747
832
|
* value `String`
|
748
833
|
|
749
834
|
```ruby
|
750
|
-
|
835
|
+
page.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>
|
751
836
|
```
|
752
837
|
|
753
838
|
#### set(value) : `Boolean`
|
@@ -763,14 +848,14 @@ Sets a cookie
|
|
763
848
|
* :httponly `Boolean`
|
764
849
|
|
765
850
|
```ruby
|
766
|
-
|
851
|
+
page.cookies.set(name: "stealth", value: "omg", domain: "google.com") # => true
|
767
852
|
```
|
768
853
|
|
769
854
|
* value `Cookie`
|
770
855
|
|
771
856
|
```ruby
|
772
|
-
nid_cookie =
|
773
|
-
|
857
|
+
nid_cookie = page.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88>
|
858
|
+
page.cookies.set(nid_cookie) # => true
|
774
859
|
```
|
775
860
|
|
776
861
|
#### remove(\*\*options) : `Boolean`
|
@@ -783,7 +868,7 @@ Removes given cookie
|
|
783
868
|
* :url `String`
|
784
869
|
|
785
870
|
```ruby
|
786
|
-
|
871
|
+
page.cookies.remove(name: "stealth", domain: "google.com") # => true
|
787
872
|
```
|
788
873
|
|
789
874
|
#### clear : `Boolean`
|
@@ -791,12 +876,31 @@ browser.cookies.remove(name: "stealth", domain: "google.com") # => true
|
|
791
876
|
Removes all cookies for current page
|
792
877
|
|
793
878
|
```ruby
|
794
|
-
|
879
|
+
page.cookies.clear # => true
|
795
880
|
```
|
796
881
|
|
882
|
+
#### store(path) : `Boolean`
|
883
|
+
|
884
|
+
Stores all cookies of current page in a file.
|
885
|
+
|
886
|
+
```ruby
|
887
|
+
# Cookies are saved into cookies.yml
|
888
|
+
page.cookies.store # => 15657
|
889
|
+
```
|
890
|
+
|
891
|
+
#### load(path) : `Boolean`
|
892
|
+
|
893
|
+
Loads all cookies from the file and sets them for current page.
|
894
|
+
|
895
|
+
```ruby
|
896
|
+
# Cookies are loaded from cookies.yml
|
897
|
+
page.cookies.load # => true
|
898
|
+
```
|
899
|
+
|
900
|
+
|
797
901
|
## Headers
|
798
902
|
|
799
|
-
`
|
903
|
+
`page.headers`
|
800
904
|
|
801
905
|
#### get : `Hash`
|
802
906
|
|
@@ -830,7 +934,7 @@ Evaluate and return result for given JS expression
|
|
830
934
|
simple value.
|
831
935
|
|
832
936
|
```ruby
|
833
|
-
|
937
|
+
page.evaluate("[window.scrollX, window.scrollY]")
|
834
938
|
```
|
835
939
|
|
836
940
|
#### evaluate_async(expression, wait_time, \*args)
|
@@ -843,7 +947,7 @@ Evaluate asynchronous expression and return result
|
|
843
947
|
simple value.
|
844
948
|
|
845
949
|
```ruby
|
846
|
-
|
950
|
+
page.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
|
847
951
|
```
|
848
952
|
|
849
953
|
#### execute(expression, \*args)
|
@@ -855,7 +959,7 @@ Execute expression. Doesn't return the result
|
|
855
959
|
simple value.
|
856
960
|
|
857
961
|
```ruby
|
858
|
-
|
962
|
+
page.execute(%(1 + 1)) # => true
|
859
963
|
```
|
860
964
|
|
861
965
|
#### evaluate_on_new_document(expression)
|
@@ -881,7 +985,7 @@ JS
|
|
881
985
|
* :type `String` - `text/javascript` by default
|
882
986
|
|
883
987
|
```ruby
|
884
|
-
|
988
|
+
page.add_script_tag(url: "http://example.com/stylesheet.css") # => true
|
885
989
|
```
|
886
990
|
|
887
991
|
#### add_style_tag(\*\*options) : `Boolean`
|
@@ -892,7 +996,7 @@ browser.add_script_tag(url: "http://example.com/stylesheet.css") # => true
|
|
892
996
|
* :content `String`
|
893
997
|
|
894
998
|
```ruby
|
895
|
-
|
999
|
+
page.add_style_tag(content: "h1 { font-size: 40px; }") # => true
|
896
1000
|
|
897
1001
|
```
|
898
1002
|
#### bypass_csp(\*\*options) : `Boolean`
|
@@ -901,11 +1005,39 @@ browser.add_style_tag(content: "h1 { font-size: 40px; }") # => true
|
|
901
1005
|
* :enabled `Boolean`, `true` by default
|
902
1006
|
|
903
1007
|
```ruby
|
904
|
-
|
905
|
-
|
906
|
-
|
907
|
-
|
908
|
-
|
1008
|
+
page.bypass_csp # => true
|
1009
|
+
page.go_to("https://github.com/ruby-concurrency/concurrent-ruby/blob/master/docs-source/promises.in.md")
|
1010
|
+
page.refresh
|
1011
|
+
page.add_script_tag(content: "window.__injected = 42")
|
1012
|
+
page.evaluate("window.__injected") # => 42
|
1013
|
+
```
|
1014
|
+
|
1015
|
+
|
1016
|
+
## Emulation
|
1017
|
+
|
1018
|
+
#### disable_javascript
|
1019
|
+
|
1020
|
+
Disables Javascripts from the loaded HTML source.
|
1021
|
+
You can still evaluate JavaScript with `evaluate` or `execute`.
|
1022
|
+
Returns nothing.
|
1023
|
+
|
1024
|
+
```ruby
|
1025
|
+
page.disable_javascript
|
1026
|
+
```
|
1027
|
+
|
1028
|
+
|
1029
|
+
#### set_viewport
|
1030
|
+
|
1031
|
+
Overrides device screen dimensions and emulates viewport.
|
1032
|
+
|
1033
|
+
* options `Hash`
|
1034
|
+
* :width `Integer`, viewport width. `0` by default
|
1035
|
+
* :height `Integer`, viewport height. `0` by default
|
1036
|
+
* :scale_factor `Float`, device scale factor. `0` by default
|
1037
|
+
* :mobile `Boolean`, whether to emulate mobile device. `false` by default
|
1038
|
+
|
1039
|
+
```ruby
|
1040
|
+
page.set_viewport(width: 1000, height: 600, scale_factor: 3)
|
909
1041
|
```
|
910
1042
|
|
911
1043
|
|
@@ -916,8 +1048,8 @@ browser.evaluate("window.__injected") # => 42
|
|
916
1048
|
Returns all the frames current page have.
|
917
1049
|
|
918
1050
|
```ruby
|
919
|
-
|
920
|
-
|
1051
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1052
|
+
page.frames # =>
|
921
1053
|
# [
|
922
1054
|
# #<Ferrum::Frame @id="C6D104CE454A025FBCF22B98DE612B12" @parent_id=nil @name=nil @state=:stopped_loading @execution_id=1>,
|
923
1055
|
# #<Ferrum::Frame @id="C09C4E4404314AAEAE85928EAC109A93" @parent_id="C6D104CE454A025FBCF22B98DE612B12" @state=:stopped_loading @execution_id=2>,
|
@@ -939,7 +1071,7 @@ Find frame by given options.
|
|
939
1071
|
* :name `String` - Frame's name if there's one
|
940
1072
|
|
941
1073
|
```ruby
|
942
|
-
|
1074
|
+
page.frame_by(id: "C6D104CE454A025FBCF22B98DE612B12")
|
943
1075
|
```
|
944
1076
|
|
945
1077
|
|
@@ -975,8 +1107,8 @@ One of the states frame's in:
|
|
975
1107
|
Returns current frame's location href.
|
976
1108
|
|
977
1109
|
```ruby
|
978
|
-
|
979
|
-
frame =
|
1110
|
+
page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
1111
|
+
frame = page.frames[1]
|
980
1112
|
frame.url # => https://interactive-examples.mdn.mozilla.net/pages/tabbed/iframe.html
|
981
1113
|
```
|
982
1114
|
|
@@ -985,8 +1117,8 @@ frame.url # => https://interactive-examples.mdn.mozilla.net/pages/tabbed/iframe.
|
|
985
1117
|
Returns current frame's title.
|
986
1118
|
|
987
1119
|
```ruby
|
988
|
-
|
989
|
-
frame =
|
1120
|
+
page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
1121
|
+
frame = page.frames[1]
|
990
1122
|
frame.title # => HTML Demo: <iframe>
|
991
1123
|
```
|
992
1124
|
|
@@ -995,8 +1127,8 @@ frame.title # => HTML Demo: <iframe>
|
|
995
1127
|
If current frame is the main frame of the page (top of the tree).
|
996
1128
|
|
997
1129
|
```ruby
|
998
|
-
|
999
|
-
frame =
|
1130
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1131
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
1000
1132
|
frame.main? # => false
|
1001
1133
|
```
|
1002
1134
|
|
@@ -1005,8 +1137,8 @@ frame.main? # => false
|
|
1005
1137
|
Returns current frame's top window location href.
|
1006
1138
|
|
1007
1139
|
```ruby
|
1008
|
-
|
1009
|
-
frame =
|
1140
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1141
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
1010
1142
|
frame.current_url # => "https://www.w3schools.com/tags/tag_frame.asp"
|
1011
1143
|
```
|
1012
1144
|
|
@@ -1015,8 +1147,8 @@ frame.current_url # => "https://www.w3schools.com/tags/tag_frame.asp"
|
|
1015
1147
|
Returns current frame's top window title.
|
1016
1148
|
|
1017
1149
|
```ruby
|
1018
|
-
|
1019
|
-
frame =
|
1150
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1151
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
1020
1152
|
frame.current_title # => "HTML frame tag"
|
1021
1153
|
```
|
1022
1154
|
|
@@ -1025,8 +1157,8 @@ frame.current_title # => "HTML frame tag"
|
|
1025
1157
|
Returns current frame's html.
|
1026
1158
|
|
1027
1159
|
```ruby
|
1028
|
-
|
1029
|
-
frame =
|
1160
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1161
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
1030
1162
|
frame.body # => "<html><head></head><body></body></html>"
|
1031
1163
|
```
|
1032
1164
|
|
@@ -1035,8 +1167,8 @@ frame.body # => "<html><head></head><body></body></html>"
|
|
1035
1167
|
Returns current frame's doctype.
|
1036
1168
|
|
1037
1169
|
```ruby
|
1038
|
-
|
1039
|
-
|
1170
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1171
|
+
page.main_frame.doctype # => "<!DOCTYPE html>"
|
1040
1172
|
```
|
1041
1173
|
|
1042
1174
|
#### content = html
|
@@ -1046,8 +1178,8 @@ Sets a content of a given frame.
|
|
1046
1178
|
* html `String`
|
1047
1179
|
|
1048
1180
|
```ruby
|
1049
|
-
|
1050
|
-
frame =
|
1181
|
+
page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
1182
|
+
frame = page.frames[1]
|
1051
1183
|
frame.body # <html lang="en"><head><style>body {transition: opacity ease-in 0.2s; }...
|
1052
1184
|
frame.content = "<html><head></head><body><p>lol</p></body></html>"
|
1053
1185
|
frame.body # => <html><head></head><body><p>lol</p></body></html>
|
@@ -1067,15 +1199,14 @@ Accept dialog with given text or default prompt if applicable
|
|
1067
1199
|
Dismiss dialog
|
1068
1200
|
|
1069
1201
|
```ruby
|
1070
|
-
|
1071
|
-
browser.on(:dialog) do |dialog|
|
1202
|
+
page.on(:dialog) do |dialog|
|
1072
1203
|
if dialog.match?(/bla-bla/)
|
1073
1204
|
dialog.accept
|
1074
1205
|
else
|
1075
1206
|
dialog.dismiss
|
1076
1207
|
end
|
1077
1208
|
end
|
1078
|
-
|
1209
|
+
page.go_to("https://google.com")
|
1079
1210
|
```
|
1080
1211
|
|
1081
1212
|
|
@@ -1095,10 +1226,9 @@ Sets playback rate of CSS animations
|
|
1095
1226
|
* value `Integer`
|
1096
1227
|
|
1097
1228
|
```ruby
|
1098
|
-
|
1099
|
-
|
1100
|
-
|
1101
|
-
browser.playback_rate # => 2000
|
1229
|
+
page.playback_rate = 2000
|
1230
|
+
page.go_to("https://google.com")
|
1231
|
+
page.playback_rate # => 2000
|
1102
1232
|
```
|
1103
1233
|
|
1104
1234
|
|
@@ -1112,7 +1242,7 @@ Returns [Frame](https://github.com/rubycdp/ferrum#frame) object for current node
|
|
1112
1242
|
[Finders](https://github.com/rubycdp/ferrum#Finders) for that object:
|
1113
1243
|
|
1114
1244
|
```ruby
|
1115
|
-
frame =
|
1245
|
+
frame = page.at_xpath("//iframe").frame # => Frame
|
1116
1246
|
frame.at_css("//a[text() = 'Log in']") # => Node
|
1117
1247
|
```
|
1118
1248
|
|
@@ -1139,19 +1269,21 @@ frame.at_css("//a[text() = 'Log in']") # => Node
|
|
1139
1269
|
#### select
|
1140
1270
|
#### scroll_into_view
|
1141
1271
|
#### in_viewport?(of: `Node | nil`) : `Boolean`
|
1272
|
+
#### remove
|
1273
|
+
#### exists?
|
1142
1274
|
|
1143
1275
|
(chainable) Selects options by passed attribute.
|
1144
1276
|
|
1145
1277
|
```ruby
|
1146
|
-
|
1147
|
-
|
1278
|
+
page.at_xpath("//*[select]").select(["1"]) # => Node (select)
|
1279
|
+
page.at_xpath("//*[select]").select(["text"], by: :text) # => Node (select)
|
1148
1280
|
```
|
1149
1281
|
|
1150
1282
|
Accept string, array or strings:
|
1151
1283
|
```ruby
|
1152
|
-
|
1153
|
-
|
1154
|
-
|
1284
|
+
page.at_xpath("//*[select]").select("1")
|
1285
|
+
page.at_xpath("//*[select]").select("1", "2")
|
1286
|
+
page.at_xpath("//*[select]").select(["1", "2"])
|
1155
1287
|
```
|
1156
1288
|
|
1157
1289
|
|
@@ -1182,6 +1314,25 @@ Accepts block, records trace and by default returns trace data from `Tracing.tra
|
|
1182
1314
|
only one trace config can be active at a time per browser.
|
1183
1315
|
|
1184
1316
|
|
1317
|
+
## Clean Up
|
1318
|
+
|
1319
|
+
#### reset
|
1320
|
+
|
1321
|
+
Closes browser tabs opened by the `Browser` instance.
|
1322
|
+
|
1323
|
+
```ruby
|
1324
|
+
# connect to a long-running Chrome process
|
1325
|
+
browser = Ferrum::Browser.new(url: 'http://localhost:9222')
|
1326
|
+
|
1327
|
+
browser.go_to("https://github.com/")
|
1328
|
+
|
1329
|
+
# clean up, lest the tab stays there hanging forever
|
1330
|
+
browser.reset
|
1331
|
+
|
1332
|
+
browser.quit
|
1333
|
+
```
|
1334
|
+
|
1335
|
+
|
1185
1336
|
## Thread safety ##
|
1186
1337
|
|
1187
1338
|
Ferrum is fully thread-safe. You can create one browser or a few as you wish and
|