ferrum 0.13 → 0.15
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/LICENSE +1 -1
- data/README.md +288 -154
- data/lib/ferrum/browser/command.rb +8 -0
- data/lib/ferrum/browser/options/chrome.rb +17 -5
- data/lib/ferrum/browser/options.rb +38 -25
- data/lib/ferrum/browser/process.rb +44 -17
- data/lib/ferrum/browser.rb +34 -52
- data/lib/ferrum/client/subscriber.rb +76 -0
- data/lib/ferrum/{browser → client}/web_socket.rb +36 -22
- data/lib/ferrum/client.rb +169 -0
- data/lib/ferrum/context.rb +19 -15
- data/lib/ferrum/contexts.rb +46 -12
- data/lib/ferrum/cookies/cookie.rb +57 -0
- data/lib/ferrum/cookies.rb +40 -4
- data/lib/ferrum/downloads.rb +60 -0
- data/lib/ferrum/errors.rb +2 -1
- data/lib/ferrum/frame.rb +1 -0
- data/lib/ferrum/headers.rb +1 -1
- data/lib/ferrum/network/exchange.rb +29 -2
- data/lib/ferrum/network/intercepted_request.rb +8 -17
- data/lib/ferrum/network/request.rb +23 -39
- data/lib/ferrum/network/request_params.rb +57 -0
- data/lib/ferrum/network/response.rb +25 -5
- data/lib/ferrum/network.rb +43 -16
- data/lib/ferrum/node.rb +21 -1
- data/lib/ferrum/page/frames.rb +5 -5
- data/lib/ferrum/page/screenshot.rb +42 -24
- data/lib/ferrum/page.rb +183 -131
- data/lib/ferrum/proxy.rb +1 -1
- data/lib/ferrum/target.rb +25 -5
- data/lib/ferrum/utils/elapsed_time.rb +0 -2
- data/lib/ferrum/utils/event.rb +19 -0
- data/lib/ferrum/utils/platform.rb +4 -0
- data/lib/ferrum/utils/thread.rb +18 -0
- data/lib/ferrum/version.rb +1 -1
- data/lib/ferrum.rb +3 -0
- metadata +14 -114
- data/lib/ferrum/browser/client.rb +0 -102
- data/lib/ferrum/browser/subscriber.rb +0 -36
data/README.md
CHANGED
@@ -25,6 +25,8 @@ going to crawl sites you better use Ferrum or
|
|
25
25
|
* [Vessel](https://github.com/rubycdp/vessel) high-level web crawling framework
|
26
26
|
based on Ferrum and Mechanize.
|
27
27
|
|
28
|
+
The development is done in [](https://jb.gg/ruby)
|
29
|
+
provided by [OSS license](https://jb.gg/OpenSourceSupport).
|
28
30
|
|
29
31
|
## Index
|
30
32
|
|
@@ -35,8 +37,8 @@ based on Ferrum and Mechanize.
|
|
35
37
|
* [Navigation](https://github.com/rubycdp/ferrum#navigation)
|
36
38
|
* [Finders](https://github.com/rubycdp/ferrum#finders)
|
37
39
|
* [Screenshots](https://github.com/rubycdp/ferrum#screenshots)
|
38
|
-
* [Cleaning Up](https://github.com/rubycdp/ferrum#cleaning-up)
|
39
40
|
* [Network](https://github.com/rubycdp/ferrum#network)
|
41
|
+
* [Downloads](https://github.com/rubycdp/ferrum#downloads)
|
40
42
|
* [Proxy](https://github.com/rubycdp/ferrum#proxy)
|
41
43
|
* [Mouse](https://github.com/rubycdp/ferrum#mouse)
|
42
44
|
* [Keyboard](https://github.com/rubycdp/ferrum#keyboard)
|
@@ -49,6 +51,7 @@ based on Ferrum and Mechanize.
|
|
49
51
|
* [Animation](https://github.com/rubycdp/ferrum#animation)
|
50
52
|
* [Node](https://github.com/rubycdp/ferrum#node)
|
51
53
|
* [Tracing](https://github.com/rubycdp/ferrum#tracing)
|
54
|
+
* [Clean Up](https://github.com/rubycdp/ferrum#clean-up)
|
52
55
|
* [Thread safety](https://github.com/rubycdp/ferrum#thread-safety)
|
53
56
|
* [Development](https://github.com/rubycdp/ferrum#development)
|
54
57
|
* [Contributing](https://github.com/rubycdp/ferrum#contributing)
|
@@ -61,7 +64,7 @@ There's no official Chrome or Chromium package for Linux don't install it this
|
|
61
64
|
way because it's either outdated or unofficial, both are bad. Download it from
|
62
65
|
official source for [Chrome](https://www.google.com/chrome/) or
|
63
66
|
[Chromium](https://www.chromium.org/getting-involved/download-chromium).
|
64
|
-
Chrome binary should be in the `PATH` or `BROWSER_PATH`
|
67
|
+
Chrome binary should be in the `PATH` or `BROWSER_PATH` and you can pass it as an
|
65
68
|
option to browser instance see `:browser_path` in
|
66
69
|
[Customization](https://github.com/rubycdp/ferrum#customization).
|
67
70
|
|
@@ -83,14 +86,17 @@ browser.screenshot(path: "google.png")
|
|
83
86
|
browser.quit
|
84
87
|
```
|
85
88
|
|
86
|
-
|
89
|
+
When you work with browser instance Ferrum creates and maintains a default page for you, in fact all the methods above
|
90
|
+
are sent to the `page` instance that is created in the `default_context` of the `browser` instance. You can interact
|
91
|
+
with a page created manually and this is preferred:
|
87
92
|
|
88
93
|
```ruby
|
89
94
|
browser = Ferrum::Browser.new
|
90
|
-
browser.
|
91
|
-
|
95
|
+
page = browser.create_page
|
96
|
+
page.go_to("https://google.com")
|
97
|
+
input = page.at_xpath("//input[@name='q']")
|
92
98
|
input.focus.type("Ruby headless driver for Chrome", :Enter)
|
93
|
-
|
99
|
+
page.at_css("a > h3").text # => "rubycdp/ferrum: Ruby Chrome/Chromium driver - GitHub"
|
94
100
|
browser.quit
|
95
101
|
```
|
96
102
|
|
@@ -98,8 +104,9 @@ Evaluate some JavaScript and get full width/height:
|
|
98
104
|
|
99
105
|
```ruby
|
100
106
|
browser = Ferrum::Browser.new
|
101
|
-
browser.
|
102
|
-
|
107
|
+
page = browser.create_page
|
108
|
+
page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
|
109
|
+
width, height = page.evaluate <<~JS
|
103
110
|
[document.documentElement.offsetWidth,
|
104
111
|
document.documentElement.offsetHeight]
|
105
112
|
JS
|
@@ -112,8 +119,9 @@ Do any mouse movements you like:
|
|
112
119
|
```ruby
|
113
120
|
# Trace a 100x100 square
|
114
121
|
browser = Ferrum::Browser.new
|
115
|
-
browser.
|
116
|
-
|
122
|
+
page = browser.create_page
|
123
|
+
page.go_to("https://google.com")
|
124
|
+
page.mouse
|
117
125
|
.move(x: 0, y: 0)
|
118
126
|
.down
|
119
127
|
.move(x: 0, y: 100)
|
@@ -134,6 +142,7 @@ In docker as root you must pass the no-sandbox browser option:
|
|
134
142
|
Ferrum::Browser.new(browser_options: { 'no-sandbox': nil })
|
135
143
|
```
|
136
144
|
|
145
|
+
It has also been reported that the Chrome process repeatedly crashes when running inside a Docker container on an M1 Mac preventing Ferrum from working. Ferrum should work as expected when deployed to a Docker container on a non-M1 Mac.
|
137
146
|
|
138
147
|
## Customization
|
139
148
|
|
@@ -144,8 +153,10 @@ Ferrum::Browser.new(options)
|
|
144
153
|
```
|
145
154
|
|
146
155
|
* options `Hash`
|
147
|
-
* `:headless` (Boolean) - Set browser as headless or not, `true` by default.
|
156
|
+
* `:headless` (String | Boolean) - Set browser as headless or not, `true` by default. You can set `"new"` to support
|
157
|
+
[new headless mode](https://developer.chrome.com/articles/new-headless/).
|
148
158
|
* `:xvfb` (Boolean) - Run browser in a virtual framebuffer, `false` by default.
|
159
|
+
* `:flatten` (Boolean) - Use one websocket connection to the browser and all the pages in flatten mode.
|
149
160
|
* `:window_size` (Array) - The dimensions of the browser window in which to
|
150
161
|
test, expressed as a 2-element array, e.g. [1024, 768]. Default: [1024, 768]
|
151
162
|
* `:extensions` (Array[String | Hash]) - An array of paths to files or JS
|
@@ -176,6 +187,8 @@ Ferrum::Browser.new(options)
|
|
176
187
|
* `:host` (String) - Remote debugging address for headless Chrome.
|
177
188
|
* `:url` (String) - URL for a running instance of Chrome. If this is set, a
|
178
189
|
browser process will not be spawned.
|
190
|
+
* `:ws_url` (String) - Websocket url for a running instance of Chrome. If this is set, a
|
191
|
+
browser process will not be spawned.
|
179
192
|
* `:process_timeout` (Integer) - How long to wait for the Chrome process to
|
180
193
|
respond on startup.
|
181
194
|
* `:ws_max_receive_size` (Integer) - How big messages to accept from Chrome
|
@@ -196,7 +209,7 @@ Navigate page to.
|
|
196
209
|
configuring driver.
|
197
210
|
|
198
211
|
```ruby
|
199
|
-
|
212
|
+
page.go_to("https://github.com/")
|
200
213
|
```
|
201
214
|
|
202
215
|
#### back
|
@@ -204,9 +217,9 @@ browser.go_to("https://github.com/")
|
|
204
217
|
Navigate to the previous page in history.
|
205
218
|
|
206
219
|
```ruby
|
207
|
-
|
208
|
-
|
209
|
-
|
220
|
+
page.go_to("https://github.com/")
|
221
|
+
page.at_xpath("//a").click
|
222
|
+
page.back
|
210
223
|
```
|
211
224
|
|
212
225
|
#### forward
|
@@ -214,10 +227,10 @@ browser.back
|
|
214
227
|
Navigate to the next page in history.
|
215
228
|
|
216
229
|
```ruby
|
217
|
-
|
218
|
-
|
219
|
-
|
220
|
-
|
230
|
+
page.go_to("https://github.com/")
|
231
|
+
page.at_xpath("//a").click
|
232
|
+
page.back
|
233
|
+
page.forward
|
221
234
|
```
|
222
235
|
|
223
236
|
#### refresh
|
@@ -225,8 +238,8 @@ browser.forward
|
|
225
238
|
Reload current page.
|
226
239
|
|
227
240
|
```ruby
|
228
|
-
|
229
|
-
|
241
|
+
page.go_to("https://github.com/")
|
242
|
+
page.refresh
|
230
243
|
```
|
231
244
|
|
232
245
|
#### stop
|
@@ -234,8 +247,8 @@ browser.refresh
|
|
234
247
|
Stop all navigations and loading pending resources on the page
|
235
248
|
|
236
249
|
```ruby
|
237
|
-
|
238
|
-
|
250
|
+
page.go_to("https://github.com/")
|
251
|
+
page.stop
|
239
252
|
```
|
240
253
|
|
241
254
|
#### position = \*\*options
|
@@ -258,6 +271,37 @@ Get the position for the browser window
|
|
258
271
|
browser.position # => [10, 20]
|
259
272
|
```
|
260
273
|
|
274
|
+
#### window_bounds = \*\*options
|
275
|
+
|
276
|
+
Set window bounds
|
277
|
+
|
278
|
+
* options `Hash`
|
279
|
+
* :left `Integer`
|
280
|
+
* :top `Integer`
|
281
|
+
* :width `Integer`
|
282
|
+
* :height `Integer`
|
283
|
+
* :window_state `String`
|
284
|
+
|
285
|
+
```ruby
|
286
|
+
browser.window_bounds = { left: 10, top: 20, width: 1024, height: 768, window_state: "normal" }
|
287
|
+
```
|
288
|
+
|
289
|
+
#### window_bounds : `Hash<String, Integer | String>`
|
290
|
+
|
291
|
+
Get window bounds
|
292
|
+
|
293
|
+
```ruby
|
294
|
+
browser.window_bounds # => { "left": 0, "top": 1286, "width": 10, "height": 10, "windowState": "normal" }
|
295
|
+
```
|
296
|
+
|
297
|
+
#### window_id : `Integer`
|
298
|
+
|
299
|
+
Current window id
|
300
|
+
|
301
|
+
```ruby
|
302
|
+
browser.window_id # => 1
|
303
|
+
```
|
304
|
+
|
261
305
|
## Finders
|
262
306
|
|
263
307
|
#### at_css(selector, \*\*options) : `Node` | `nil`
|
@@ -270,8 +314,8 @@ provided node.
|
|
270
314
|
* :within `Node` | `nil`
|
271
315
|
|
272
316
|
```ruby
|
273
|
-
|
274
|
-
|
317
|
+
page.go_to("https://github.com/")
|
318
|
+
page.at_css("a[aria-label='Issues you created']") # => Node
|
275
319
|
```
|
276
320
|
|
277
321
|
|
@@ -285,8 +329,8 @@ document or provided node.
|
|
285
329
|
* :within `Node` | `nil`
|
286
330
|
|
287
331
|
```ruby
|
288
|
-
|
289
|
-
|
332
|
+
page.go_to("https://github.com/")
|
333
|
+
page.css("a[aria-label='Issues you created']") # => [Node]
|
290
334
|
```
|
291
335
|
|
292
336
|
#### at_xpath(selector, \*\*options) : `Node` | `nil`
|
@@ -298,8 +342,8 @@ Find node by xpath.
|
|
298
342
|
* :within `Node` | `nil`
|
299
343
|
|
300
344
|
```ruby
|
301
|
-
|
302
|
-
|
345
|
+
page.go_to("https://github.com/")
|
346
|
+
page.at_xpath("//a[@aria-label='Issues you created']") # => Node
|
303
347
|
```
|
304
348
|
|
305
349
|
#### xpath(selector, \*\*options) : `Array<Node>` | `[]`
|
@@ -311,8 +355,8 @@ Find nodes by xpath.
|
|
311
355
|
* :within `Node` | `nil`
|
312
356
|
|
313
357
|
```ruby
|
314
|
-
|
315
|
-
|
358
|
+
page.go_to("https://github.com/")
|
359
|
+
page.xpath("//a[@aria-label='Issues you created']") # => [Node]
|
316
360
|
```
|
317
361
|
|
318
362
|
#### current_url : `String`
|
@@ -320,8 +364,8 @@ browser.xpath("//a[@aria-label='Issues you created']") # => [Node]
|
|
320
364
|
Returns current top window location href.
|
321
365
|
|
322
366
|
```ruby
|
323
|
-
|
324
|
-
|
367
|
+
page.go_to("https://google.com/")
|
368
|
+
page.current_url # => "https://www.google.com/"
|
325
369
|
```
|
326
370
|
|
327
371
|
#### current_title : `String`
|
@@ -329,8 +373,8 @@ browser.current_url # => "https://www.google.com/"
|
|
329
373
|
Returns current top window title
|
330
374
|
|
331
375
|
```ruby
|
332
|
-
|
333
|
-
|
376
|
+
page.go_to("https://google.com/")
|
377
|
+
page.current_title # => "Google"
|
334
378
|
```
|
335
379
|
|
336
380
|
#### body : `String`
|
@@ -338,8 +382,8 @@ browser.current_title # => "Google"
|
|
338
382
|
Returns current page's html.
|
339
383
|
|
340
384
|
```ruby
|
341
|
-
|
342
|
-
|
385
|
+
page.go_to("https://google.com/")
|
386
|
+
page.body # => '<html itemscope="" itemtype="http://schema.org/WebPage" lang="ru"><head>...
|
343
387
|
```
|
344
388
|
|
345
389
|
|
@@ -357,20 +401,29 @@ Saves screenshot on a disk or returns it as base64.
|
|
357
401
|
* :format `String` "jpeg" | "png"
|
358
402
|
* :quality `Integer` 0-100 works for jpeg only
|
359
403
|
* :full `Boolean` whether you need full page screenshot or a viewport
|
360
|
-
* :selector `String` css selector for given element
|
404
|
+
* :selector `String` css selector for given element, optional
|
405
|
+
* :area `Hash` area for screenshot, optional
|
406
|
+
* :x `Integer`
|
407
|
+
* :y `Integer`
|
408
|
+
* :width `Integer`
|
409
|
+
* :height `Integer`
|
361
410
|
* :scale `Float` zoom in/out
|
362
411
|
* :background_color `Ferrum::RGBA.new(0, 0, 0, 0.0)` to have specific background color
|
363
412
|
|
364
413
|
```ruby
|
365
|
-
|
414
|
+
page.go_to("https://google.com/")
|
366
415
|
# Save on the disk in PNG
|
367
|
-
|
416
|
+
page.screenshot(path: "google.png") # => 134660
|
368
417
|
# Save on the disk in JPG
|
369
|
-
|
418
|
+
page.screenshot(path: "google.jpg") # => 30902
|
370
419
|
# Save to Base64 the whole page not only viewport and reduce quality
|
371
|
-
|
420
|
+
page.screenshot(full: true, quality: 60, encoding: :base64) # "iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAYAAAC6uhUNAAAAAXNSR0IArs4c6Q...
|
421
|
+
# Save on the disk with the selected element in PNG
|
422
|
+
page.screenshot(path: "google.png", selector: 'textarea') # => 11340
|
423
|
+
# Save to Base64 with an area of the page in PNG
|
424
|
+
page.screenshot(path: "google.png", area: { x: 0, y: 0, width: 400, height: 300 }) # => 54239
|
372
425
|
# Save with specific background color
|
373
|
-
|
426
|
+
page.screenshot(background_color: Ferrum::RGBA.new(0, 0, 0, 0.0))
|
374
427
|
```
|
375
428
|
|
376
429
|
#### pdf(\*\*options) : `String` | `Boolean`
|
@@ -391,9 +444,9 @@ Saves PDF on a disk or returns it as base64.
|
|
391
444
|
* See other [native options](https://chromedevtools.github.io/devtools-protocol/tot/Page#method-printToPDF) you can pass
|
392
445
|
|
393
446
|
```ruby
|
394
|
-
|
447
|
+
page.go_to("https://google.com/")
|
395
448
|
# Save to disk as a PDF
|
396
|
-
|
449
|
+
page.pdf(path: "google.pdf", paper_width: 1.0, paper_height: 1.0) # => true
|
397
450
|
```
|
398
451
|
|
399
452
|
#### mhtml(\*\*options) : `String` | `Integer`
|
@@ -404,33 +457,14 @@ Saves MHTML on a disk or returns it as a string.
|
|
404
457
|
* :path `String` to save a file on the disk.
|
405
458
|
|
406
459
|
```ruby
|
407
|
-
|
408
|
-
|
409
|
-
```
|
410
|
-
|
411
|
-
|
412
|
-
## Cleaning Up
|
413
|
-
|
414
|
-
#### reset
|
415
|
-
|
416
|
-
Closes browser tabs opened by the `Browser` instance.
|
417
|
-
|
418
|
-
```ruby
|
419
|
-
# connect to a long-running Chrome process
|
420
|
-
browser = Ferrum::Browser.new(url: 'http://localhost:9222')
|
421
|
-
|
422
|
-
browser.go_to("https://github.com/")
|
423
|
-
|
424
|
-
# clean up, lest the tab stays there hanging forever
|
425
|
-
browser.reset
|
426
|
-
|
427
|
-
browser.quit
|
460
|
+
page.go_to("https://google.com/")
|
461
|
+
page.mhtml(path: "google.mhtml") # => 87742
|
428
462
|
```
|
429
463
|
|
430
464
|
|
431
465
|
## Network
|
432
466
|
|
433
|
-
`
|
467
|
+
`page.network`
|
434
468
|
|
435
469
|
#### traffic `Array<Network::Exchange>`
|
436
470
|
|
@@ -438,8 +472,8 @@ Returns all information about network traffic as `Network::Exchange` instance
|
|
438
472
|
which in general is a wrapper around `request`, `response` and `error`.
|
439
473
|
|
440
474
|
```ruby
|
441
|
-
|
442
|
-
|
475
|
+
page.go_to("https://github.com/")
|
476
|
+
page.network.traffic # => [#<Ferrum::Network::Exchange, ...]
|
443
477
|
```
|
444
478
|
|
445
479
|
#### request : `Network::Request`
|
@@ -447,8 +481,8 @@ browser.network.traffic # => [#<Ferrum::Network::Exchange, ...]
|
|
447
481
|
Page request of the main frame.
|
448
482
|
|
449
483
|
```ruby
|
450
|
-
|
451
|
-
|
484
|
+
page.go_to("https://github.com/")
|
485
|
+
page.network.request # => #<Ferrum::Network::Request...
|
452
486
|
```
|
453
487
|
|
454
488
|
#### response : `Network::Response`
|
@@ -456,8 +490,8 @@ browser.network.request # => #<Ferrum::Network::Request...
|
|
456
490
|
Page response of the main frame.
|
457
491
|
|
458
492
|
```ruby
|
459
|
-
|
460
|
-
|
493
|
+
page.go_to("https://github.com/")
|
494
|
+
page.network.response # => #<Ferrum::Network::Response...
|
461
495
|
```
|
462
496
|
|
463
497
|
#### status : `Integer`
|
@@ -466,8 +500,8 @@ Contains the status code of the main page response (e.g., 200 for a
|
|
466
500
|
success). This is just a shortcut for `response.status`.
|
467
501
|
|
468
502
|
```ruby
|
469
|
-
|
470
|
-
|
503
|
+
page.go_to("https://github.com/")
|
504
|
+
page.network.status # => 200
|
471
505
|
```
|
472
506
|
|
473
507
|
#### wait_for_idle(\*\*options)
|
@@ -483,22 +517,22 @@ Waits for network idle or raises `Ferrum::TimeoutError` error
|
|
483
517
|
by default
|
484
518
|
|
485
519
|
```ruby
|
486
|
-
|
487
|
-
|
488
|
-
|
520
|
+
page.go_to("https://example.com/")
|
521
|
+
page.at_xpath("//a[text() = 'No UI changes button']").click
|
522
|
+
page.network.wait_for_idle
|
489
523
|
```
|
490
524
|
|
491
525
|
#### clear(type)
|
492
526
|
|
493
|
-
Clear
|
527
|
+
Clear page's cache or collected traffic.
|
494
528
|
|
495
529
|
* type `Symbol` it is either `:traffic` or `:cache`
|
496
530
|
|
497
531
|
```ruby
|
498
|
-
traffic =
|
499
|
-
|
532
|
+
traffic = page.network.traffic # => []
|
533
|
+
page.go_to("https://github.com/")
|
500
534
|
traffic.size # => 51
|
501
|
-
|
535
|
+
page.network.clear(:traffic)
|
502
536
|
traffic.size # => 0
|
503
537
|
```
|
504
538
|
|
@@ -514,8 +548,9 @@ continue them.
|
|
514
548
|
|
515
549
|
```ruby
|
516
550
|
browser = Ferrum::Browser.new
|
517
|
-
browser.
|
518
|
-
|
551
|
+
page = browser.create_page
|
552
|
+
page.network.intercept
|
553
|
+
page.on(:request) do |request|
|
519
554
|
if request.match?(/bla-bla/)
|
520
555
|
request.abort
|
521
556
|
elsif request.match?(/lorem/)
|
@@ -524,7 +559,7 @@ browser.on(:request) do |request|
|
|
524
559
|
request.continue
|
525
560
|
end
|
526
561
|
end
|
527
|
-
|
562
|
+
page.go_to("https://google.com")
|
528
563
|
```
|
529
564
|
|
530
565
|
#### authorize(\*\*options, &block)
|
@@ -539,10 +574,10 @@ If site or proxy uses authorization you can provide credentials using this metho
|
|
539
574
|
care about unwanted requests just call `request.continue`.
|
540
575
|
|
541
576
|
```ruby
|
542
|
-
|
543
|
-
|
544
|
-
puts
|
545
|
-
puts
|
577
|
+
page.network.authorize(user: "login", password: "pass") { |req| req.continue }
|
578
|
+
page.go_to("http://example.com/authenticated")
|
579
|
+
puts page.network.status # => 200
|
580
|
+
puts page.body # => Welcome, authenticated client
|
546
581
|
```
|
547
582
|
|
548
583
|
Since Chrome implements authorize using request interception you must continue or abort authorized requests. If you
|
@@ -551,8 +586,9 @@ block, so this is version doesn't pass block and can work just fine:
|
|
551
586
|
|
552
587
|
```ruby
|
553
588
|
browser = Ferrum::Browser.new
|
554
|
-
browser.
|
555
|
-
|
589
|
+
page = browser.create_page
|
590
|
+
page.network.intercept
|
591
|
+
page.on(:request) do |request|
|
556
592
|
if request.resource_type == "Image"
|
557
593
|
request.abort
|
558
594
|
else
|
@@ -560,9 +596,9 @@ browser.on(:request) do |request|
|
|
560
596
|
end
|
561
597
|
end
|
562
598
|
|
563
|
-
|
599
|
+
page.network.authorize(user: "login", password: "pass", type: :proxy)
|
564
600
|
|
565
|
-
|
601
|
+
page.go_to("https://google.com")
|
566
602
|
```
|
567
603
|
|
568
604
|
You used to call `authorize` method without block, but since it's implemented using request interception there could be
|
@@ -585,8 +621,8 @@ Activates emulation of network conditions.
|
|
585
621
|
bluetooth, ethernet, wifi, wimax, other. `nil` by default
|
586
622
|
|
587
623
|
```ruby
|
588
|
-
|
589
|
-
|
624
|
+
page.network.emulate_network_conditions(connection_type: "cellular2g")
|
625
|
+
page.go_to("https://github.com/")
|
590
626
|
```
|
591
627
|
|
592
628
|
#### offline_mode
|
@@ -594,8 +630,59 @@ browser.go_to("https://github.com/")
|
|
594
630
|
Activates offline mode for a page.
|
595
631
|
|
596
632
|
```ruby
|
597
|
-
|
598
|
-
|
633
|
+
page.network.offline_mode
|
634
|
+
page.go_to("https://github.com/") # => Ferrum::StatusError (Request to https://github.com/ failed(net::ERR_INTERNET_DISCONNECTED))
|
635
|
+
```
|
636
|
+
|
637
|
+
#### cache(disable: `Boolean`)
|
638
|
+
|
639
|
+
Toggles ignoring cache for each request. If true, cache will not be used.
|
640
|
+
|
641
|
+
```ruby
|
642
|
+
page.network.cache(disable: true)
|
643
|
+
```
|
644
|
+
|
645
|
+
|
646
|
+
## Downloads
|
647
|
+
|
648
|
+
`page.downloads`
|
649
|
+
|
650
|
+
#### files `Array<Hash>`
|
651
|
+
|
652
|
+
Returns all information about downloaded files as a `Hash`.
|
653
|
+
|
654
|
+
```ruby
|
655
|
+
page.go_to("http://localhost/attachment.pdf")
|
656
|
+
page.downloads.files # => [{"frameId"=>"E3316DF1B5383D38F8ADF7485005FDE3", "guid"=>"11a68745-98ac-4d54-9b57-9f9016c268b3", "url"=>"http://localhost/attachment.pdf", "suggestedFilename"=>"attachment.pdf", "totalBytes"=>4911, "receivedBytes"=>4911, "state"=>"completed"}]
|
657
|
+
```
|
658
|
+
|
659
|
+
#### wait(timeout)
|
660
|
+
|
661
|
+
Waits until the download is finished.
|
662
|
+
|
663
|
+
```ruby
|
664
|
+
page.go_to("http://localhost/attachment.pdf")
|
665
|
+
page.downloads.wait
|
666
|
+
```
|
667
|
+
|
668
|
+
or
|
669
|
+
|
670
|
+
```ruby
|
671
|
+
page.go_to("http://localhost/page")
|
672
|
+
page.downloads.wait { page.at_css("#download").click }
|
673
|
+
```
|
674
|
+
|
675
|
+
#### set_behavior(\*\*options)
|
676
|
+
|
677
|
+
Sets behavior in case of file to be downloaded.
|
678
|
+
|
679
|
+
* options `Hash`
|
680
|
+
* :save_path `String` absolute path of where to store the file
|
681
|
+
* :behavior `Symbol` `deny | allow | allowAndName | default`, `allow` by default
|
682
|
+
|
683
|
+
```ruby
|
684
|
+
page.go_to("https://example.com/")
|
685
|
+
page.downloads.set_behavior(save_path: "/tmp", behavior: :allow)
|
599
686
|
```
|
600
687
|
|
601
688
|
|
@@ -604,13 +691,13 @@ browser.go_to("https://github.com/") # => Ferrum::StatusError (Request to https:
|
|
604
691
|
You can set a proxy with a `:proxy` option:
|
605
692
|
|
606
693
|
```ruby
|
607
|
-
|
694
|
+
Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", user: "user", password: "pa$$" })
|
608
695
|
```
|
609
696
|
|
610
697
|
`:bypass` can specify semi-colon-separated list of hosts for which proxy shouldn't be used:
|
611
698
|
|
612
699
|
```ruby
|
613
|
-
|
700
|
+
Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", bypass: "*.google.com;*foo.com" })
|
614
701
|
```
|
615
702
|
|
616
703
|
In general passing a proxy option when instantiating a browser results in a browser running with proxy command line
|
@@ -634,7 +721,7 @@ end
|
|
634
721
|
|
635
722
|
### Mouse
|
636
723
|
|
637
|
-
`
|
724
|
+
`page.mouse`
|
638
725
|
|
639
726
|
#### scroll_to(x, y)
|
640
727
|
|
@@ -646,8 +733,8 @@ Scroll page to a given x, y
|
|
646
733
|
displayed in the upper left
|
647
734
|
|
648
735
|
```ruby
|
649
|
-
|
650
|
-
|
736
|
+
page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
|
737
|
+
page.mouse.scroll_to(0, 400)
|
651
738
|
```
|
652
739
|
|
653
740
|
#### click(\*\*options) : `Mouse`
|
@@ -691,7 +778,7 @@ Mouse move to given x and y.
|
|
691
778
|
|
692
779
|
### Keyboard
|
693
780
|
|
694
|
-
|
781
|
+
`page.keyboard`
|
695
782
|
|
696
783
|
#### down(key) : `Keyboard`
|
697
784
|
|
@@ -721,14 +808,14 @@ Returns bitfield for a given keys
|
|
721
808
|
|
722
809
|
## Cookies
|
723
810
|
|
724
|
-
`
|
811
|
+
`page.cookies`
|
725
812
|
|
726
813
|
#### all : `Hash<String, Cookie>`
|
727
814
|
|
728
815
|
Returns cookies hash
|
729
816
|
|
730
817
|
```ruby
|
731
|
-
|
818
|
+
page.cookies.all # => {"NID"=>#<Ferrum::Cookies::Cookie:0x0000558624b37a40 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>}
|
732
819
|
```
|
733
820
|
|
734
821
|
#### [](value) : `Cookie`
|
@@ -738,7 +825,7 @@ Returns cookie
|
|
738
825
|
* value `String`
|
739
826
|
|
740
827
|
```ruby
|
741
|
-
|
828
|
+
page.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>
|
742
829
|
```
|
743
830
|
|
744
831
|
#### set(value) : `Boolean`
|
@@ -754,14 +841,14 @@ Sets a cookie
|
|
754
841
|
* :httponly `Boolean`
|
755
842
|
|
756
843
|
```ruby
|
757
|
-
|
844
|
+
page.cookies.set(name: "stealth", value: "omg", domain: "google.com") # => true
|
758
845
|
```
|
759
846
|
|
760
847
|
* value `Cookie`
|
761
848
|
|
762
849
|
```ruby
|
763
|
-
nid_cookie =
|
764
|
-
|
850
|
+
nid_cookie = page.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88>
|
851
|
+
page.cookies.set(nid_cookie) # => true
|
765
852
|
```
|
766
853
|
|
767
854
|
#### remove(\*\*options) : `Boolean`
|
@@ -774,7 +861,7 @@ Removes given cookie
|
|
774
861
|
* :url `String`
|
775
862
|
|
776
863
|
```ruby
|
777
|
-
|
864
|
+
page.cookies.remove(name: "stealth", domain: "google.com") # => true
|
778
865
|
```
|
779
866
|
|
780
867
|
#### clear : `Boolean`
|
@@ -782,12 +869,12 @@ browser.cookies.remove(name: "stealth", domain: "google.com") # => true
|
|
782
869
|
Removes all cookies for current page
|
783
870
|
|
784
871
|
```ruby
|
785
|
-
|
872
|
+
page.cookies.clear # => true
|
786
873
|
```
|
787
874
|
|
788
875
|
## Headers
|
789
876
|
|
790
|
-
`
|
877
|
+
`page.headers`
|
791
878
|
|
792
879
|
#### get : `Hash`
|
793
880
|
|
@@ -821,7 +908,7 @@ Evaluate and return result for given JS expression
|
|
821
908
|
simple value.
|
822
909
|
|
823
910
|
```ruby
|
824
|
-
|
911
|
+
page.evaluate("[window.scrollX, window.scrollY]")
|
825
912
|
```
|
826
913
|
|
827
914
|
#### evaluate_async(expression, wait_time, \*args)
|
@@ -834,7 +921,7 @@ Evaluate asynchronous expression and return result
|
|
834
921
|
simple value.
|
835
922
|
|
836
923
|
```ruby
|
837
|
-
|
924
|
+
page.evaluate_async(%(arguments[0]({foo: "bar"})), 5) # => { "foo" => "bar" }
|
838
925
|
```
|
839
926
|
|
840
927
|
#### execute(expression, \*args)
|
@@ -846,7 +933,7 @@ Execute expression. Doesn't return the result
|
|
846
933
|
simple value.
|
847
934
|
|
848
935
|
```ruby
|
849
|
-
|
936
|
+
page.execute(%(1 + 1)) # => true
|
850
937
|
```
|
851
938
|
|
852
939
|
#### evaluate_on_new_document(expression)
|
@@ -872,7 +959,7 @@ JS
|
|
872
959
|
* :type `String` - `text/javascript` by default
|
873
960
|
|
874
961
|
```ruby
|
875
|
-
|
962
|
+
page.add_script_tag(url: "http://example.com/stylesheet.css") # => true
|
876
963
|
```
|
877
964
|
|
878
965
|
#### add_style_tag(\*\*options) : `Boolean`
|
@@ -883,7 +970,7 @@ browser.add_script_tag(url: "http://example.com/stylesheet.css") # => true
|
|
883
970
|
* :content `String`
|
884
971
|
|
885
972
|
```ruby
|
886
|
-
|
973
|
+
page.add_style_tag(content: "h1 { font-size: 40px; }") # => true
|
887
974
|
|
888
975
|
```
|
889
976
|
#### bypass_csp(\*\*options) : `Boolean`
|
@@ -892,11 +979,39 @@ browser.add_style_tag(content: "h1 { font-size: 40px; }") # => true
|
|
892
979
|
* :enabled `Boolean`, `true` by default
|
893
980
|
|
894
981
|
```ruby
|
895
|
-
|
896
|
-
|
897
|
-
|
898
|
-
|
899
|
-
|
982
|
+
page.bypass_csp # => true
|
983
|
+
page.go_to("https://github.com/ruby-concurrency/concurrent-ruby/blob/master/docs-source/promises.in.md")
|
984
|
+
page.refresh
|
985
|
+
page.add_script_tag(content: "window.__injected = 42")
|
986
|
+
page.evaluate("window.__injected") # => 42
|
987
|
+
```
|
988
|
+
|
989
|
+
|
990
|
+
## Emulation
|
991
|
+
|
992
|
+
#### disable_javascript
|
993
|
+
|
994
|
+
Disables Javascripts from the loaded HTML source.
|
995
|
+
You can still evaluate JavaScript with `evaluate` or `execute`.
|
996
|
+
Returns nothing.
|
997
|
+
|
998
|
+
```ruby
|
999
|
+
page.disable_javascript
|
1000
|
+
```
|
1001
|
+
|
1002
|
+
|
1003
|
+
#### set_viewport
|
1004
|
+
|
1005
|
+
Overrides device screen dimensions and emulates viewport.
|
1006
|
+
|
1007
|
+
* options `Hash`
|
1008
|
+
* :width `Integer`, viewport width. `0` by default
|
1009
|
+
* :height `Integer`, viewport height. `0` by default
|
1010
|
+
* :scale_factor `Float`, device scale factor. `0` by default
|
1011
|
+
* :mobile `Boolean`, whether to emulate mobile device. `false` by default
|
1012
|
+
|
1013
|
+
```ruby
|
1014
|
+
page.set_viewport(width: 1000, height: 600, scale_factor: 3)
|
900
1015
|
```
|
901
1016
|
|
902
1017
|
|
@@ -907,8 +1022,8 @@ browser.evaluate("window.__injected") # => 42
|
|
907
1022
|
Returns all the frames current page have.
|
908
1023
|
|
909
1024
|
```ruby
|
910
|
-
|
911
|
-
|
1025
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1026
|
+
page.frames # =>
|
912
1027
|
# [
|
913
1028
|
# #<Ferrum::Frame @id="C6D104CE454A025FBCF22B98DE612B12" @parent_id=nil @name=nil @state=:stopped_loading @execution_id=1>,
|
914
1029
|
# #<Ferrum::Frame @id="C09C4E4404314AAEAE85928EAC109A93" @parent_id="C6D104CE454A025FBCF22B98DE612B12" @state=:stopped_loading @execution_id=2>,
|
@@ -930,7 +1045,7 @@ Find frame by given options.
|
|
930
1045
|
* :name `String` - Frame's name if there's one
|
931
1046
|
|
932
1047
|
```ruby
|
933
|
-
|
1048
|
+
page.frame_by(id: "C6D104CE454A025FBCF22B98DE612B12")
|
934
1049
|
```
|
935
1050
|
|
936
1051
|
|
@@ -966,8 +1081,8 @@ One of the states frame's in:
|
|
966
1081
|
Returns current frame's location href.
|
967
1082
|
|
968
1083
|
```ruby
|
969
|
-
|
970
|
-
frame =
|
1084
|
+
page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
1085
|
+
frame = page.frames[1]
|
971
1086
|
frame.url # => https://interactive-examples.mdn.mozilla.net/pages/tabbed/iframe.html
|
972
1087
|
```
|
973
1088
|
|
@@ -976,8 +1091,8 @@ frame.url # => https://interactive-examples.mdn.mozilla.net/pages/tabbed/iframe.
|
|
976
1091
|
Returns current frame's title.
|
977
1092
|
|
978
1093
|
```ruby
|
979
|
-
|
980
|
-
frame =
|
1094
|
+
page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
1095
|
+
frame = page.frames[1]
|
981
1096
|
frame.title # => HTML Demo: <iframe>
|
982
1097
|
```
|
983
1098
|
|
@@ -986,8 +1101,8 @@ frame.title # => HTML Demo: <iframe>
|
|
986
1101
|
If current frame is the main frame of the page (top of the tree).
|
987
1102
|
|
988
1103
|
```ruby
|
989
|
-
|
990
|
-
frame =
|
1104
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1105
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
991
1106
|
frame.main? # => false
|
992
1107
|
```
|
993
1108
|
|
@@ -996,8 +1111,8 @@ frame.main? # => false
|
|
996
1111
|
Returns current frame's top window location href.
|
997
1112
|
|
998
1113
|
```ruby
|
999
|
-
|
1000
|
-
frame =
|
1114
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1115
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
1001
1116
|
frame.current_url # => "https://www.w3schools.com/tags/tag_frame.asp"
|
1002
1117
|
```
|
1003
1118
|
|
@@ -1006,8 +1121,8 @@ frame.current_url # => "https://www.w3schools.com/tags/tag_frame.asp"
|
|
1006
1121
|
Returns current frame's top window title.
|
1007
1122
|
|
1008
1123
|
```ruby
|
1009
|
-
|
1010
|
-
frame =
|
1124
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1125
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
1011
1126
|
frame.current_title # => "HTML frame tag"
|
1012
1127
|
```
|
1013
1128
|
|
@@ -1016,8 +1131,8 @@ frame.current_title # => "HTML frame tag"
|
|
1016
1131
|
Returns current frame's html.
|
1017
1132
|
|
1018
1133
|
```ruby
|
1019
|
-
|
1020
|
-
frame =
|
1134
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1135
|
+
frame = page.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
1021
1136
|
frame.body # => "<html><head></head><body></body></html>"
|
1022
1137
|
```
|
1023
1138
|
|
@@ -1026,8 +1141,8 @@ frame.body # => "<html><head></head><body></body></html>"
|
|
1026
1141
|
Returns current frame's doctype.
|
1027
1142
|
|
1028
1143
|
```ruby
|
1029
|
-
|
1030
|
-
|
1144
|
+
page.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
1145
|
+
page.main_frame.doctype # => "<!DOCTYPE html>"
|
1031
1146
|
```
|
1032
1147
|
|
1033
1148
|
#### content = html
|
@@ -1037,8 +1152,8 @@ Sets a content of a given frame.
|
|
1037
1152
|
* html `String`
|
1038
1153
|
|
1039
1154
|
```ruby
|
1040
|
-
|
1041
|
-
frame =
|
1155
|
+
page.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
1156
|
+
frame = page.frames[1]
|
1042
1157
|
frame.body # <html lang="en"><head><style>body {transition: opacity ease-in 0.2s; }...
|
1043
1158
|
frame.content = "<html><head></head><body><p>lol</p></body></html>"
|
1044
1159
|
frame.body # => <html><head></head><body><p>lol</p></body></html>
|
@@ -1058,15 +1173,14 @@ Accept dialog with given text or default prompt if applicable
|
|
1058
1173
|
Dismiss dialog
|
1059
1174
|
|
1060
1175
|
```ruby
|
1061
|
-
|
1062
|
-
browser.on(:dialog) do |dialog|
|
1176
|
+
page.on(:dialog) do |dialog|
|
1063
1177
|
if dialog.match?(/bla-bla/)
|
1064
1178
|
dialog.accept
|
1065
1179
|
else
|
1066
1180
|
dialog.dismiss
|
1067
1181
|
end
|
1068
1182
|
end
|
1069
|
-
|
1183
|
+
page.go_to("https://google.com")
|
1070
1184
|
```
|
1071
1185
|
|
1072
1186
|
|
@@ -1086,10 +1200,9 @@ Sets playback rate of CSS animations
|
|
1086
1200
|
* value `Integer`
|
1087
1201
|
|
1088
1202
|
```ruby
|
1089
|
-
|
1090
|
-
|
1091
|
-
|
1092
|
-
browser.playback_rate # => 2000
|
1203
|
+
page.playback_rate = 2000
|
1204
|
+
page.go_to("https://google.com")
|
1205
|
+
page.playback_rate # => 2000
|
1093
1206
|
```
|
1094
1207
|
|
1095
1208
|
|
@@ -1103,7 +1216,7 @@ Returns [Frame](https://github.com/rubycdp/ferrum#frame) object for current node
|
|
1103
1216
|
[Finders](https://github.com/rubycdp/ferrum#Finders) for that object:
|
1104
1217
|
|
1105
1218
|
```ruby
|
1106
|
-
frame =
|
1219
|
+
frame = page.at_xpath("//iframe").frame # => Frame
|
1107
1220
|
frame.at_css("//a[text() = 'Log in']") # => Node
|
1108
1221
|
```
|
1109
1222
|
|
@@ -1128,19 +1241,21 @@ frame.at_css("//a[text() = 'Log in']") # => Node
|
|
1128
1241
|
#### evaluate
|
1129
1242
|
#### selected : `Array<Node>`
|
1130
1243
|
#### select
|
1244
|
+
#### scroll_into_view
|
1245
|
+
#### in_viewport?(of: `Node | nil`) : `Boolean`
|
1131
1246
|
|
1132
1247
|
(chainable) Selects options by passed attribute.
|
1133
1248
|
|
1134
1249
|
```ruby
|
1135
|
-
|
1136
|
-
|
1250
|
+
page.at_xpath("//*[select]").select(["1"]) # => Node (select)
|
1251
|
+
page.at_xpath("//*[select]").select(["text"], by: :text) # => Node (select)
|
1137
1252
|
```
|
1138
1253
|
|
1139
1254
|
Accept string, array or strings:
|
1140
1255
|
```ruby
|
1141
|
-
|
1142
|
-
|
1143
|
-
|
1256
|
+
page.at_xpath("//*[select]").select("1")
|
1257
|
+
page.at_xpath("//*[select]").select("1", "2")
|
1258
|
+
page.at_xpath("//*[select]").select(["1", "2"])
|
1144
1259
|
```
|
1145
1260
|
|
1146
1261
|
|
@@ -1171,6 +1286,25 @@ Accepts block, records trace and by default returns trace data from `Tracing.tra
|
|
1171
1286
|
only one trace config can be active at a time per browser.
|
1172
1287
|
|
1173
1288
|
|
1289
|
+
## Clean Up
|
1290
|
+
|
1291
|
+
#### reset
|
1292
|
+
|
1293
|
+
Closes browser tabs opened by the `Browser` instance.
|
1294
|
+
|
1295
|
+
```ruby
|
1296
|
+
# connect to a long-running Chrome process
|
1297
|
+
browser = Ferrum::Browser.new(url: 'http://localhost:9222')
|
1298
|
+
|
1299
|
+
browser.go_to("https://github.com/")
|
1300
|
+
|
1301
|
+
# clean up, lest the tab stays there hanging forever
|
1302
|
+
browser.reset
|
1303
|
+
|
1304
|
+
browser.quit
|
1305
|
+
```
|
1306
|
+
|
1307
|
+
|
1174
1308
|
## Thread safety ##
|
1175
1309
|
|
1176
1310
|
Ferrum is fully thread-safe. You can create one browser or a few as you wish and
|