ferrum 0.6.2 → 0.10.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/LICENSE +1 -1
- data/README.md +370 -78
- data/lib/ferrum.rb +38 -4
- data/lib/ferrum/browser.rb +19 -12
- data/lib/ferrum/browser/client.rb +23 -10
- data/lib/ferrum/browser/command.rb +57 -0
- data/lib/ferrum/browser/options/base.rb +46 -0
- data/lib/ferrum/browser/options/chrome.rb +73 -0
- data/lib/ferrum/browser/options/firefox.rb +34 -0
- data/lib/ferrum/browser/process.rb +56 -108
- data/lib/ferrum/browser/subscriber.rb +9 -1
- data/lib/ferrum/browser/web_socket.rb +23 -4
- data/lib/ferrum/browser/xvfb.rb +37 -0
- data/lib/ferrum/context.rb +3 -3
- data/lib/ferrum/cookies.rb +7 -0
- data/lib/ferrum/dialog.rb +2 -2
- data/lib/ferrum/frame.rb +20 -5
- data/lib/ferrum/frame/dom.rb +34 -37
- data/lib/ferrum/frame/runtime.rb +90 -84
- data/lib/ferrum/headers.rb +1 -1
- data/lib/ferrum/keyboard.rb +3 -3
- data/lib/ferrum/mouse.rb +14 -3
- data/lib/ferrum/network.rb +81 -20
- data/lib/ferrum/network/error.rb +8 -15
- data/lib/ferrum/network/exchange.rb +24 -21
- data/lib/ferrum/network/intercepted_request.rb +12 -3
- data/lib/ferrum/network/response.rb +4 -0
- data/lib/ferrum/node.rb +70 -26
- data/lib/ferrum/page.rb +66 -26
- data/lib/ferrum/page/frames.rb +12 -15
- data/lib/ferrum/page/screenshot.rb +64 -12
- data/lib/ferrum/rbga.rb +38 -0
- data/lib/ferrum/version.rb +1 -1
- metadata +13 -7
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 1cea2150084c494e0461e9a63b53ff215e34ed98f2bfe345458ae46c95f0f8f3
|
|
4
|
+
data.tar.gz: 46fad57af780b5a5b6ff9659125642d6b57674db386af34b868dc07d1164e2b6
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 63d6104319d89bfa85897c946c01f6d44efd92de88204b4882ccea879e196dd80bebfd61b06a7566272dd6e22de1c306eb1385d17a6a72df739d94753adb0a41
|
|
7
|
+
data.tar.gz: ef333fd514ea94d674bf4ad51117f736c4d7982ca56981192fe636010e1f34df6c84d889ce2ce40bfeeb47534f1c912a5aec54efdad01e23b7d915ac1be88524
|
data/LICENSE
CHANGED
data/README.md
CHANGED
|
@@ -1,43 +1,83 @@
|
|
|
1
|
-
# Ferrum -
|
|
1
|
+
# Ferrum - high-level API to control Chrome in Ruby
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
<img align="right" width="95" height="95"
|
|
3
|
+
<img align="right"
|
|
4
|
+
width="320" height="241"
|
|
6
5
|
alt="Ferrum logo"
|
|
7
|
-
src="https://raw.githubusercontent.com/
|
|
8
|
-
|
|
9
|
-
As simple as Puppeteer, though even simpler.
|
|
10
|
-
|
|
11
|
-
It is Ruby clean and high-level API to Chrome. Runs headless by default,
|
|
12
|
-
|
|
13
|
-
Chrome/
|
|
6
|
+
src="https://raw.githubusercontent.com/rubycdp/ferrum/master/logo.svg?sanitize=true">
|
|
7
|
+
|
|
8
|
+
#### As simple as Puppeteer, though even simpler.
|
|
9
|
+
|
|
10
|
+
It is Ruby clean and high-level API to Chrome. Runs headless by default, but you
|
|
11
|
+
can configure it to run in a headful mode. All you need is Ruby and
|
|
12
|
+
[Chrome](https://www.google.com/chrome/) or
|
|
13
|
+
[Chromium](https://www.chromium.org/). Ferrum connects to the browser by [CDP
|
|
14
|
+
protocol](https://chromedevtools.github.io/devtools-protocol/) and there's _no_
|
|
15
|
+
Selenium/WebDriver/ChromeDriver dependency. The emphasis was made on a raw CDP
|
|
16
|
+
protocol because Chrome allows you to do so many things that are barely
|
|
17
|
+
supported by WebDriver because it should have consistent design with other
|
|
18
|
+
browsers.
|
|
19
|
+
|
|
20
|
+
* [Cuprite](https://github.com/rubycdp/cuprite) is a pure Ruby driver for
|
|
21
|
+
[Capybara](https://github.com/teamcapybara/capybara) based on Ferrum. If you are
|
|
22
|
+
going to crawl sites you better use Ferrum or
|
|
23
|
+
[Vessel](https://github.com/rubycdp/vessel) because you crawl, not test.
|
|
24
|
+
|
|
25
|
+
* [Vessel](https://github.com/rubycdp/vessel) high-level web crawling framework
|
|
26
|
+
based on Ferrum. It looks like [Scrapy](https://scrapy.org/) except that it uses
|
|
27
|
+
a real browser in order to grab data.
|
|
28
|
+
|
|
29
|
+
Web design by [Evrone](https://evrone.com/), what else
|
|
30
|
+
[we build with Ruby on Rails](https://evrone.com/ruby), what else
|
|
31
|
+
[we do at Evrone](https://evrone.com/cases#case-studies).
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
## Index
|
|
35
|
+
|
|
36
|
+
* [Install](https://github.com/rubycdp/ferrum#install)
|
|
37
|
+
* [Examples](https://github.com/rubycdp/ferrum#examples)
|
|
38
|
+
* [Docker](https://github.com/rubycdp/ferrum#docker)
|
|
39
|
+
* [Customization](https://github.com/rubycdp/ferrum#customization)
|
|
40
|
+
* [Navigation](https://github.com/rubycdp/ferrum#navigation)
|
|
41
|
+
* [Finders](https://github.com/rubycdp/ferrum#finders)
|
|
42
|
+
* [Screenshots](https://github.com/rubycdp/ferrum#screenshots)
|
|
43
|
+
* [Network](https://github.com/rubycdp/ferrum#network)
|
|
44
|
+
* [Mouse](https://github.com/rubycdp/ferrum#mouse)
|
|
45
|
+
* [Keyboard](https://github.com/rubycdp/ferrum#keyboard)
|
|
46
|
+
* [Cookies](https://github.com/rubycdp/ferrum#cookies)
|
|
47
|
+
* [Headers](https://github.com/rubycdp/ferrum#headers)
|
|
48
|
+
* [JavaScript](https://github.com/rubycdp/ferrum#javascript)
|
|
49
|
+
* [Frames](https://github.com/rubycdp/ferrum#frames)
|
|
50
|
+
* [Frame](https://github.com/rubycdp/ferrum#frame)
|
|
51
|
+
* [Dialog](https://github.com/rubycdp/ferrum#dialog)
|
|
52
|
+
* [Thread safety](https://github.com/rubycdp/ferrum#thread-safety)
|
|
53
|
+
* [Development](https://github.com/rubycdp/ferrum#development)
|
|
54
|
+
* [Contributing](https://github.com/rubycdp/ferrum#contributing)
|
|
55
|
+
* [License](https://github.com/rubycdp/ferrum#license)
|
|
14
56
|
|
|
15
|
-
Relation to [Cuprite](https://github.com/machinio/cuprite). Cuprite used to have
|
|
16
|
-
this code inside in one form or another but the thing is you don't need capybara
|
|
17
|
-
if you are going to crawl sites. You crawl, not test. Besides that clean
|
|
18
|
-
lightweight API to browser is what Ruby was missing, so here it comes.
|
|
19
|
-
|
|
20
|
-
If you like this project, please consider to [become a backer](https://www.patreon.com/rferrum) on Patreon.
|
|
21
57
|
|
|
22
58
|
## Install
|
|
23
59
|
|
|
24
60
|
There's no official Chrome or Chromium package for Linux don't install it this
|
|
25
|
-
way because it either
|
|
26
|
-
|
|
61
|
+
way because it's either outdated or unofficial, both are bad. Download it from
|
|
62
|
+
official [source](https://www.chromium.org/getting-involved/download-chromium).
|
|
27
63
|
Chrome binary should be in the `PATH` or `BROWSER_PATH` or you can pass it as an
|
|
28
|
-
option to browser instance `:browser_path
|
|
64
|
+
option to browser instance see `:browser_path` in
|
|
65
|
+
[Customization](https://github.com/rubycdp/ferrum#customization).
|
|
29
66
|
|
|
30
|
-
Add this to your Gemfile
|
|
67
|
+
Add this to your `Gemfile` and run `bundle install`.
|
|
31
68
|
|
|
32
69
|
``` ruby
|
|
33
70
|
gem "ferrum"
|
|
34
71
|
```
|
|
35
72
|
|
|
73
|
+
|
|
74
|
+
## Examples
|
|
75
|
+
|
|
36
76
|
Navigate to a website and save a screenshot:
|
|
37
77
|
|
|
38
78
|
```ruby
|
|
39
79
|
browser = Ferrum::Browser.new
|
|
40
|
-
browser.
|
|
80
|
+
browser.go_to("https://google.com")
|
|
41
81
|
browser.screenshot(path: "google.png")
|
|
42
82
|
browser.quit
|
|
43
83
|
```
|
|
@@ -46,10 +86,10 @@ Interact with a page:
|
|
|
46
86
|
|
|
47
87
|
```ruby
|
|
48
88
|
browser = Ferrum::Browser.new
|
|
49
|
-
browser.
|
|
50
|
-
input = browser.at_xpath("//
|
|
51
|
-
input.focus.type("Ruby headless driver for
|
|
52
|
-
browser.at_css("a > h3").text # => "
|
|
89
|
+
browser.go_to("https://google.com")
|
|
90
|
+
input = browser.at_xpath("//input[@name='q']")
|
|
91
|
+
input.focus.type("Ruby headless driver for Chrome", :Enter)
|
|
92
|
+
browser.at_css("a > h3").text # => "rubycdp/ferrum: Ruby Chrome/Chromium driver - GitHub"
|
|
53
93
|
browser.quit
|
|
54
94
|
```
|
|
55
95
|
|
|
@@ -57,7 +97,7 @@ Evaluate some JavaScript and get full width/height:
|
|
|
57
97
|
|
|
58
98
|
```ruby
|
|
59
99
|
browser = Ferrum::Browser.new
|
|
60
|
-
browser.
|
|
100
|
+
browser.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
|
|
61
101
|
width, height = browser.evaluate <<~JS
|
|
62
102
|
[document.documentElement.offsetWidth,
|
|
63
103
|
document.documentElement.offsetHeight]
|
|
@@ -71,7 +111,7 @@ Do any mouse movements you like:
|
|
|
71
111
|
```ruby
|
|
72
112
|
# Trace a 100x100 square
|
|
73
113
|
browser = Ferrum::Browser.new
|
|
74
|
-
browser.
|
|
114
|
+
browser.go_to("https://google.com")
|
|
75
115
|
browser.mouse
|
|
76
116
|
.move(x: 0, y: 0)
|
|
77
117
|
.down
|
|
@@ -84,7 +124,17 @@ browser.mouse
|
|
|
84
124
|
browser.quit
|
|
85
125
|
```
|
|
86
126
|
|
|
87
|
-
|
|
127
|
+
|
|
128
|
+
## Docker
|
|
129
|
+
|
|
130
|
+
In docker as root you must pass the no-sandbox browser option:
|
|
131
|
+
|
|
132
|
+
```ruby
|
|
133
|
+
Ferrum::Browser.new(browser_options: { 'no-sandbox': nil })
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
|
|
137
|
+
## Customization
|
|
88
138
|
|
|
89
139
|
You can customize options with the following code in your test setup:
|
|
90
140
|
|
|
@@ -93,36 +143,48 @@ Ferrum::Browser.new(options)
|
|
|
93
143
|
```
|
|
94
144
|
|
|
95
145
|
* options `Hash`
|
|
96
|
-
* `:browser_path` (String) - Path to chrome binary, you can also set ENV
|
|
97
|
-
variable as `BROWSER_PATH=some/path/chrome bundle exec rspec`.
|
|
98
146
|
* `:headless` (Boolean) - Set browser as headless or not, `true` by default.
|
|
99
|
-
* `:
|
|
100
|
-
|
|
147
|
+
* `:xvfb` (Boolean) - Run browser in a virtual framebuffer, `false` by default.
|
|
148
|
+
* `:window_size` (Array) - The dimensions of the browser window in which to
|
|
149
|
+
test, expressed as a 2-element array, e.g. [1024, 768]. Default: [1024, 768]
|
|
150
|
+
* `:extensions` (Array[String | Hash]) - An array of paths to files or JS
|
|
151
|
+
source code to be preloaded into the browser e.g.:
|
|
152
|
+
`["/path/to/script.js", { source: "window.secret = 'top'" }]`
|
|
101
153
|
* `:logger` (Object responding to `puts`) - When present, debug output is
|
|
102
154
|
written to this object.
|
|
155
|
+
* `:slowmo` (Integer | Float) - Set a delay in seconds to wait before sending command.
|
|
156
|
+
Usefull companion of headless option, so that you have time to see changes.
|
|
103
157
|
* `:timeout` (Numeric) - The number of seconds we'll wait for a response when
|
|
104
158
|
communicating with browser. Default is 5.
|
|
105
159
|
* `:js_errors` (Boolean) - When true, JavaScript errors get re-raised in Ruby.
|
|
106
|
-
* `:
|
|
107
|
-
|
|
160
|
+
* `:pending_connection_errors` (Boolean) - When main frame is still waiting for slow responses while timeout is
|
|
161
|
+
reached `PendingConnectionsError` is raised. It's better to figure out why you have slow responses and fix or
|
|
162
|
+
block them rather than turn this setting off. Default is true.
|
|
163
|
+
* `:browser_name` (Symbol) - `:chrome` by default, only experimental support
|
|
164
|
+
for `:firefox` for now.
|
|
165
|
+
* `:browser_path` (String) - Path to Chrome binary, you can also set ENV
|
|
166
|
+
variable as `BROWSER_PATH=some/path/chrome bundle exec rspec`.
|
|
108
167
|
* `:browser_options` (Hash) - Additional command line options,
|
|
109
168
|
[see them all](https://peter.sh/experiments/chromium-command-line-switches/)
|
|
110
169
|
e.g. `{ "ignore-certificate-errors" => nil }`
|
|
111
|
-
* `:
|
|
170
|
+
* `:ignore_default_browser_options` (Boolean) - Ferrum has a number of default
|
|
171
|
+
options it passes to the browser, if you set this to `true` then only
|
|
172
|
+
options you put in `:browser_options` will be passed to the browser,
|
|
173
|
+
except required ones of course.
|
|
112
174
|
* `:port` (Integer) - Remote debugging port for headless Chrome
|
|
113
175
|
* `:host` (String) - Remote debugging address for headless Chrome
|
|
114
176
|
* `:url` (String) - URL for a running instance of Chrome. If this is set, a
|
|
115
177
|
browser process will not be spawned.
|
|
116
178
|
* `:process_timeout` (Integer) - How long to wait for the Chrome process to
|
|
117
179
|
respond on startup
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
180
|
+
* `:ws_max_receive_size` (Integer) - How big messages to accept from Chrome
|
|
181
|
+
over the web socket, in bytes. Defaults to 64MB. Incoming messages larger
|
|
182
|
+
than this will cause a `Ferrum::DeadBrowserError`.
|
|
121
183
|
|
|
122
184
|
|
|
123
185
|
## Navigation
|
|
124
186
|
|
|
125
|
-
####
|
|
187
|
+
#### go_to(url) : `String`
|
|
126
188
|
|
|
127
189
|
Navigate page to.
|
|
128
190
|
|
|
@@ -130,7 +192,7 @@ Navigate page to.
|
|
|
130
192
|
configuring driver.
|
|
131
193
|
|
|
132
194
|
```ruby
|
|
133
|
-
browser.
|
|
195
|
+
browser.go_to("https://github.com/")
|
|
134
196
|
```
|
|
135
197
|
|
|
136
198
|
#### back
|
|
@@ -138,7 +200,7 @@ browser.goto("https://github.com/")
|
|
|
138
200
|
Navigate to the previous page in history.
|
|
139
201
|
|
|
140
202
|
```ruby
|
|
141
|
-
browser.
|
|
203
|
+
browser.go_to("https://github.com/")
|
|
142
204
|
browser.at_xpath("//a").click
|
|
143
205
|
browser.back
|
|
144
206
|
```
|
|
@@ -148,7 +210,7 @@ browser.back
|
|
|
148
210
|
Navigate to the next page in history.
|
|
149
211
|
|
|
150
212
|
```ruby
|
|
151
|
-
browser.
|
|
213
|
+
browser.go_to("https://github.com/")
|
|
152
214
|
browser.at_xpath("//a").click
|
|
153
215
|
browser.back
|
|
154
216
|
browser.forward
|
|
@@ -159,10 +221,19 @@ browser.forward
|
|
|
159
221
|
Reload current page.
|
|
160
222
|
|
|
161
223
|
```ruby
|
|
162
|
-
browser.
|
|
224
|
+
browser.go_to("https://github.com/")
|
|
163
225
|
browser.refresh
|
|
164
226
|
```
|
|
165
227
|
|
|
228
|
+
#### stop
|
|
229
|
+
|
|
230
|
+
Stop all navigations and loading pending resources on the page
|
|
231
|
+
|
|
232
|
+
```ruby
|
|
233
|
+
browser.go_to("https://github.com/")
|
|
234
|
+
browser.stop
|
|
235
|
+
```
|
|
236
|
+
|
|
166
237
|
|
|
167
238
|
## Finders
|
|
168
239
|
|
|
@@ -176,7 +247,7 @@ provided node.
|
|
|
176
247
|
* :within `Node` | `nil`
|
|
177
248
|
|
|
178
249
|
```ruby
|
|
179
|
-
browser.
|
|
250
|
+
browser.go_to("https://github.com/")
|
|
180
251
|
browser.at_css("a[aria-label='Issues you created']") # => Node
|
|
181
252
|
```
|
|
182
253
|
|
|
@@ -191,7 +262,7 @@ document or provided node.
|
|
|
191
262
|
* :within `Node` | `nil`
|
|
192
263
|
|
|
193
264
|
```ruby
|
|
194
|
-
browser.
|
|
265
|
+
browser.go_to("https://github.com/")
|
|
195
266
|
browser.css("a[aria-label='Issues you created']") # => [Node]
|
|
196
267
|
```
|
|
197
268
|
|
|
@@ -204,7 +275,7 @@ Find node by xpath.
|
|
|
204
275
|
* :within `Node` | `nil`
|
|
205
276
|
|
|
206
277
|
```ruby
|
|
207
|
-
browser.
|
|
278
|
+
browser.go_to("https://github.com/")
|
|
208
279
|
browser.at_xpath("//a[@aria-label='Issues you created']") # => Node
|
|
209
280
|
```
|
|
210
281
|
|
|
@@ -217,7 +288,7 @@ Find nodes by xpath.
|
|
|
217
288
|
* :within `Node` | `nil`
|
|
218
289
|
|
|
219
290
|
```ruby
|
|
220
|
-
browser.
|
|
291
|
+
browser.go_to("https://github.com/")
|
|
221
292
|
browser.xpath("//a[@aria-label='Issues you created']") # => [Node]
|
|
222
293
|
```
|
|
223
294
|
|
|
@@ -226,7 +297,7 @@ browser.xpath("//a[@aria-label='Issues you created']") # => [Node]
|
|
|
226
297
|
Returns current top window location href.
|
|
227
298
|
|
|
228
299
|
```ruby
|
|
229
|
-
browser.
|
|
300
|
+
browser.go_to("https://google.com/")
|
|
230
301
|
browser.current_url # => "https://www.google.com/"
|
|
231
302
|
```
|
|
232
303
|
|
|
@@ -235,7 +306,7 @@ browser.current_url # => "https://www.google.com/"
|
|
|
235
306
|
Returns current top window title
|
|
236
307
|
|
|
237
308
|
```ruby
|
|
238
|
-
browser.
|
|
309
|
+
browser.go_to("https://google.com/")
|
|
239
310
|
browser.current_title # => "Google"
|
|
240
311
|
```
|
|
241
312
|
|
|
@@ -244,7 +315,7 @@ browser.current_title # => "Google"
|
|
|
244
315
|
Returns current page's html.
|
|
245
316
|
|
|
246
317
|
```ruby
|
|
247
|
-
browser.
|
|
318
|
+
browser.go_to("https://google.com/")
|
|
248
319
|
browser.body # => '<html itemscope="" itemtype="http://schema.org/WebPage" lang="ru"><head>...
|
|
249
320
|
```
|
|
250
321
|
|
|
@@ -265,18 +336,21 @@ Saves screenshot on a disk or returns it as base64.
|
|
|
265
336
|
* :full `Boolean` whether you need full page screenshot or a viewport
|
|
266
337
|
* :selector `String` css selector for given element
|
|
267
338
|
* :scale `Float` zoom in/out
|
|
339
|
+
* :background_color `Ferrum::RGBA.new(0, 0, 0, 0.0)` to have specific background color
|
|
268
340
|
|
|
269
341
|
```ruby
|
|
270
|
-
browser.
|
|
342
|
+
browser.go_to("https://google.com/")
|
|
271
343
|
# Save on the disk in PNG
|
|
272
344
|
browser.screenshot(path: "google.png") # => 134660
|
|
273
345
|
# Save on the disk in JPG
|
|
274
346
|
browser.screenshot(path: "google.jpg") # => 30902
|
|
275
347
|
# Save to Base64 the whole page not only viewport and reduce quality
|
|
276
348
|
browser.screenshot(full: true, quality: 60) # "iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAYAAAC6uhUNAAAAAXNSR0IArs4c6Q...
|
|
349
|
+
# Save with specific background color
|
|
350
|
+
browser.screenshot(background_color: Ferrum::RGBA.new(0, 0, 0, 0.0))
|
|
277
351
|
```
|
|
278
352
|
|
|
279
|
-
#### pdf(\*\*options) : `String` | `
|
|
353
|
+
#### pdf(\*\*options) : `String` | `Boolean`
|
|
280
354
|
|
|
281
355
|
Saves PDF on a disk or returns it as base64.
|
|
282
356
|
|
|
@@ -294,9 +368,21 @@ Saves PDF on a disk or returns it as base64.
|
|
|
294
368
|
* See other [native options](https://chromedevtools.github.io/devtools-protocol/tot/Page#method-printToPDF) you can pass
|
|
295
369
|
|
|
296
370
|
```ruby
|
|
297
|
-
browser.
|
|
371
|
+
browser.go_to("https://google.com/")
|
|
298
372
|
# Save to disk as a PDF
|
|
299
|
-
browser.pdf(path: "google.pdf", paper_width: 1.0, paper_height: 1.0) # =>
|
|
373
|
+
browser.pdf(path: "google.pdf", paper_width: 1.0, paper_height: 1.0) # => true
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
#### mhtml(\*\*options) : `String` | `Integer`
|
|
377
|
+
|
|
378
|
+
Saves MHTML on a disk or returns it as a string.
|
|
379
|
+
|
|
380
|
+
* options `Hash`
|
|
381
|
+
* :path `String` to save a file on the disk.
|
|
382
|
+
|
|
383
|
+
```ruby
|
|
384
|
+
browser.go_to("https://google.com/")
|
|
385
|
+
browser.mhtml(path: "google.mhtml") # => 87742
|
|
300
386
|
```
|
|
301
387
|
|
|
302
388
|
|
|
@@ -310,7 +396,7 @@ Returns all information about network traffic as `Network::Exchange` instance
|
|
|
310
396
|
which in general is a wrapper around `request`, `response` and `error`.
|
|
311
397
|
|
|
312
398
|
```ruby
|
|
313
|
-
browser.
|
|
399
|
+
browser.go_to("https://github.com/")
|
|
314
400
|
browser.network.traffic # => [#<Ferrum::Network::Exchange, ...]
|
|
315
401
|
```
|
|
316
402
|
|
|
@@ -319,7 +405,7 @@ browser.network.traffic # => [#<Ferrum::Network::Exchange, ...]
|
|
|
319
405
|
Page request of the main frame.
|
|
320
406
|
|
|
321
407
|
```ruby
|
|
322
|
-
browser.
|
|
408
|
+
browser.go_to("https://github.com/")
|
|
323
409
|
browser.network.request # => #<Ferrum::Network::Request...
|
|
324
410
|
```
|
|
325
411
|
|
|
@@ -328,7 +414,7 @@ browser.network.request # => #<Ferrum::Network::Request...
|
|
|
328
414
|
Page response of the main frame.
|
|
329
415
|
|
|
330
416
|
```ruby
|
|
331
|
-
browser.
|
|
417
|
+
browser.go_to("https://github.com/")
|
|
332
418
|
browser.network.response # => #<Ferrum::Network::Response...
|
|
333
419
|
```
|
|
334
420
|
|
|
@@ -338,10 +424,28 @@ Contains the status code of the main page response (e.g., 200 for a
|
|
|
338
424
|
success). This is just a shortcut for `response.status`.
|
|
339
425
|
|
|
340
426
|
```ruby
|
|
341
|
-
browser.
|
|
427
|
+
browser.go_to("https://github.com/")
|
|
342
428
|
browser.network.status # => 200
|
|
343
429
|
```
|
|
344
430
|
|
|
431
|
+
#### wait_for_idle(\*\*options)
|
|
432
|
+
|
|
433
|
+
Waits for network idle or raises `Ferrum::TimeoutError` error
|
|
434
|
+
|
|
435
|
+
* options `Hash`
|
|
436
|
+
* :connections `Integer` how many connections are allowed for network to be
|
|
437
|
+
idling, `0` by default
|
|
438
|
+
* :duration `Float` sleep for given amount of time and check again, `0.05` by
|
|
439
|
+
default
|
|
440
|
+
* :timeout `Float` during what time we try to check idle, `browser.timeout`
|
|
441
|
+
by default
|
|
442
|
+
|
|
443
|
+
```ruby
|
|
444
|
+
browser.go_to("https://example.com/")
|
|
445
|
+
browser.at_xpath("//a[text() = 'No UI changes button']").click
|
|
446
|
+
browser.network.wait_for_idle
|
|
447
|
+
```
|
|
448
|
+
|
|
345
449
|
#### clear(type)
|
|
346
450
|
|
|
347
451
|
Clear browser's cache or collected traffic.
|
|
@@ -350,7 +454,7 @@ Clear browser's cache or collected traffic.
|
|
|
350
454
|
|
|
351
455
|
```ruby
|
|
352
456
|
traffic = browser.network.traffic # => []
|
|
353
|
-
browser.
|
|
457
|
+
browser.go_to("https://github.com/")
|
|
354
458
|
traffic.size # => 51
|
|
355
459
|
browser.network.clear(:traffic)
|
|
356
460
|
traffic.size # => 0
|
|
@@ -378,25 +482,52 @@ browser.on(:request) do |request|
|
|
|
378
482
|
request.continue
|
|
379
483
|
end
|
|
380
484
|
end
|
|
381
|
-
browser.
|
|
485
|
+
browser.go_to("https://google.com")
|
|
382
486
|
```
|
|
383
487
|
|
|
384
|
-
#### authorize(\*\*options)
|
|
488
|
+
#### authorize(\*\*options, &block)
|
|
385
489
|
|
|
386
|
-
If site uses authorization you can provide credentials using this method.
|
|
490
|
+
If site or proxy uses authorization you can provide credentials using this method.
|
|
387
491
|
|
|
388
492
|
* options `Hash`
|
|
389
493
|
* :type `Symbol` `:server` | `:proxy` site or proxy authorization
|
|
390
494
|
* :user `String`
|
|
391
495
|
* :password `String`
|
|
496
|
+
* &block accepts authenticated request, which you must subsequently allow or deny, if you don't
|
|
497
|
+
care about unwanted requests just call `request.continue`.
|
|
392
498
|
|
|
393
499
|
```ruby
|
|
394
|
-
browser.network.authorize(user: "login", password: "pass")
|
|
395
|
-
browser.
|
|
500
|
+
browser.network.authorize(user: "login", password: "pass") { |req| req.continue }
|
|
501
|
+
browser.go_to("http://example.com/authenticated")
|
|
396
502
|
puts browser.network.status # => 200
|
|
397
503
|
puts browser.body # => Welcome, authenticated client
|
|
398
504
|
```
|
|
399
505
|
|
|
506
|
+
Since Chrome implements authorize using request interception you must continue or abort authorized requests. If you
|
|
507
|
+
already have code that uses interception you can use `authorize` without block, but if not you are obliged to pass
|
|
508
|
+
block, so this is version doesn't pass block and can work just fine:
|
|
509
|
+
|
|
510
|
+
```ruby
|
|
511
|
+
browser = Ferrum::Browser.new
|
|
512
|
+
browser.network.intercept
|
|
513
|
+
browser.on(:request) do |request|
|
|
514
|
+
if request.resource_type == "Image"
|
|
515
|
+
request.abort
|
|
516
|
+
else
|
|
517
|
+
request.continue
|
|
518
|
+
end
|
|
519
|
+
end
|
|
520
|
+
|
|
521
|
+
browser.network.authorize(user: "login", password: "pass", type: :proxy)
|
|
522
|
+
|
|
523
|
+
browser.go_to("https://google.com")
|
|
524
|
+
|
|
525
|
+
```
|
|
526
|
+
|
|
527
|
+
You used to call `authorize` method without block, but since it's implemented using request interception there could be
|
|
528
|
+
a collision with another part of your code that also uses request interception, so that authorize allows the request
|
|
529
|
+
while your code denies but it's too late. The block is mandatory now.
|
|
530
|
+
|
|
400
531
|
|
|
401
532
|
### Mouse
|
|
402
533
|
|
|
@@ -412,7 +543,7 @@ Scroll page to a given x, y
|
|
|
412
543
|
displayed in the upper left
|
|
413
544
|
|
|
414
545
|
```ruby
|
|
415
|
-
browser.
|
|
546
|
+
browser.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
|
|
416
547
|
browser.mouse.scroll_to(0, 400)
|
|
417
548
|
```
|
|
418
549
|
|
|
@@ -517,6 +648,8 @@ Sets given values as cookie
|
|
|
517
648
|
* :value `String`
|
|
518
649
|
* :domain `String`
|
|
519
650
|
* :expires `Integer`
|
|
651
|
+
* :samesite `String`
|
|
652
|
+
* :httponly `Boolean`
|
|
520
653
|
|
|
521
654
|
```ruby
|
|
522
655
|
browser.cookies.set(name: "stealth", value: "omg", domain: "google.com") # => true
|
|
@@ -628,22 +761,163 @@ browser.add_script_tag(url: "http://example.com/stylesheet.css") # => true
|
|
|
628
761
|
|
|
629
762
|
```ruby
|
|
630
763
|
browser.add_style_tag(content: "h1 { font-size: 40px; }") # => true
|
|
764
|
+
|
|
765
|
+
```
|
|
766
|
+
#### bypass_csp(enabled) : `Boolean`
|
|
767
|
+
|
|
768
|
+
* enabled `Boolean`, `true` by default
|
|
769
|
+
|
|
770
|
+
```ruby
|
|
771
|
+
browser.bypass_csp # => true
|
|
772
|
+
browser.go_to("https://github.com/ruby-concurrency/concurrent-ruby/blob/master/docs-source/promises.in.md")
|
|
773
|
+
browser.refresh
|
|
774
|
+
browser.add_script_tag(content: "window.__injected = 42")
|
|
775
|
+
browser.evaluate("window.__injected") # => 42
|
|
631
776
|
```
|
|
632
777
|
|
|
633
778
|
|
|
634
779
|
## Frames
|
|
635
780
|
|
|
636
|
-
#### frames
|
|
637
|
-
#### main_frame
|
|
638
|
-
#### frame_by
|
|
781
|
+
#### frames : `Array[Frame] | []`
|
|
639
782
|
|
|
640
|
-
|
|
783
|
+
Returns all the frames current page have.
|
|
641
784
|
|
|
642
785
|
```ruby
|
|
643
|
-
browser.
|
|
786
|
+
browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
|
787
|
+
browser.frames # =>
|
|
788
|
+
# [
|
|
789
|
+
# #<Ferrum::Frame @id="C6D104CE454A025FBCF22B98DE612B12" @parent_id=nil @name=nil @state=:stopped_loading @execution_id=1>,
|
|
790
|
+
# #<Ferrum::Frame @id="C09C4E4404314AAEAE85928EAC109A93" @parent_id="C6D104CE454A025FBCF22B98DE612B12" @state=:stopped_loading @execution_id=2>,
|
|
791
|
+
# #<Ferrum::Frame @id="2E9C7F476ED09D87A42F2FEE3C6FBC3C" @parent_id="C6D104CE454A025FBCF22B98DE612B12" @state=:stopped_loading @execution_id=3>,
|
|
792
|
+
# ...
|
|
793
|
+
# ]
|
|
794
|
+
```
|
|
795
|
+
|
|
796
|
+
#### main_frame : `Frame`
|
|
797
|
+
|
|
798
|
+
Returns page's main frame, the top of the tree and the parent of all frames.
|
|
799
|
+
|
|
800
|
+
#### frame_by(\*\*options) : `Frame | nil`
|
|
801
|
+
|
|
802
|
+
Find frame by given options.
|
|
803
|
+
|
|
804
|
+
* options `Hash`
|
|
805
|
+
* :id `String` - Unique frame's id that browser provides
|
|
806
|
+
* :name `String` - Frame's name if there's one
|
|
807
|
+
|
|
808
|
+
```ruby
|
|
809
|
+
browser.frame_by(id: "C6D104CE454A025FBCF22B98DE612B12")
|
|
810
|
+
```
|
|
811
|
+
|
|
812
|
+
|
|
813
|
+
## Frame
|
|
814
|
+
|
|
815
|
+
#### id : `String`
|
|
816
|
+
|
|
817
|
+
Frame's unique id.
|
|
818
|
+
|
|
819
|
+
#### parent_id : `String | nil`
|
|
820
|
+
|
|
821
|
+
Parent frame id if this one is nested in another one.
|
|
822
|
+
|
|
823
|
+
#### execution_id : `Integer`
|
|
824
|
+
|
|
825
|
+
Execution context id which is used by JS, each frame has it's own context in
|
|
826
|
+
which JS evaluates.
|
|
827
|
+
|
|
828
|
+
#### name : `String | nil`
|
|
829
|
+
|
|
830
|
+
If frame was given a name it should be here.
|
|
831
|
+
|
|
832
|
+
#### state : `Symbol | nil`
|
|
833
|
+
|
|
834
|
+
One of the states frame's in:
|
|
835
|
+
|
|
836
|
+
* `:started_loading`
|
|
837
|
+
* `:navigated`
|
|
838
|
+
* `:stopped_loading`
|
|
839
|
+
|
|
840
|
+
#### url : `String`
|
|
841
|
+
|
|
842
|
+
Returns current frame's location href.
|
|
843
|
+
|
|
844
|
+
```ruby
|
|
845
|
+
browser.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
|
846
|
+
frame = browser.frames[1]
|
|
847
|
+
frame.url # => https://interactive-examples.mdn.mozilla.net/pages/tabbed/iframe.html
|
|
848
|
+
```
|
|
849
|
+
|
|
850
|
+
#### title
|
|
851
|
+
|
|
852
|
+
Returns current frame's title.
|
|
853
|
+
|
|
854
|
+
```ruby
|
|
855
|
+
browser.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
|
856
|
+
frame = browser.frames[1]
|
|
857
|
+
frame.title # => HTML Demo: <iframe>
|
|
858
|
+
```
|
|
859
|
+
|
|
860
|
+
#### main? : `Boolean`
|
|
861
|
+
|
|
862
|
+
If current frame is the main frame of the page (top of the tree).
|
|
863
|
+
|
|
864
|
+
```ruby
|
|
865
|
+
browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
|
866
|
+
frame = browser.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
|
867
|
+
frame.main? # => false
|
|
868
|
+
```
|
|
869
|
+
|
|
870
|
+
#### current_url : `String`
|
|
871
|
+
|
|
872
|
+
Returns current frame's top window location href.
|
|
873
|
+
|
|
874
|
+
```ruby
|
|
875
|
+
browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
|
876
|
+
frame = browser.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
|
877
|
+
frame.current_url # => "https://www.w3schools.com/tags/tag_frame.asp"
|
|
878
|
+
```
|
|
879
|
+
|
|
880
|
+
#### current_title : `String`
|
|
881
|
+
|
|
882
|
+
Returns current frame's top window title.
|
|
883
|
+
|
|
884
|
+
```ruby
|
|
885
|
+
browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
|
886
|
+
frame = browser.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
|
887
|
+
frame.current_title # => "HTML frame tag"
|
|
888
|
+
```
|
|
889
|
+
|
|
890
|
+
#### body : `String`
|
|
891
|
+
|
|
892
|
+
Returns current frame's html.
|
|
893
|
+
|
|
894
|
+
```ruby
|
|
895
|
+
browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
|
896
|
+
frame = browser.frame_by(id: "C09C4E4404314AAEAE85928EAC109A93")
|
|
897
|
+
frame.body # => "<html><head></head><body></body></html>"
|
|
898
|
+
```
|
|
899
|
+
|
|
900
|
+
#### doctype
|
|
901
|
+
|
|
902
|
+
Returns current frame's doctype.
|
|
903
|
+
|
|
904
|
+
```ruby
|
|
905
|
+
browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
|
|
906
|
+
browser.main_frame.doctype # => "<!DOCTYPE html>"
|
|
907
|
+
```
|
|
908
|
+
|
|
909
|
+
#### set_content(html)
|
|
910
|
+
|
|
911
|
+
Sets a content of a given frame.
|
|
912
|
+
|
|
913
|
+
* html `String`
|
|
914
|
+
|
|
915
|
+
```ruby
|
|
916
|
+
browser.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
|
|
644
917
|
frame = browser.frames[1]
|
|
645
|
-
|
|
646
|
-
|
|
918
|
+
frame.body # <html lang="en"><head><style>body {transition: opacity ease-in 0.2s; }...
|
|
919
|
+
frame.set_content("<html><head></head><body><p>lol</p></body></html>")
|
|
920
|
+
frame.body # => <html><head></head><body><p>lol</p></body></html>
|
|
647
921
|
```
|
|
648
922
|
|
|
649
923
|
|
|
@@ -668,7 +942,7 @@ browser.on(:dialog) do |dialog|
|
|
|
668
942
|
dialog.dismiss
|
|
669
943
|
end
|
|
670
944
|
end
|
|
671
|
-
browser.
|
|
945
|
+
browser.go_to("https://google.com")
|
|
672
946
|
```
|
|
673
947
|
|
|
674
948
|
|
|
@@ -685,13 +959,13 @@ context = browser.contexts.create
|
|
|
685
959
|
|
|
686
960
|
t1 = Thread.new(context) do |c|
|
|
687
961
|
page = c.create_page
|
|
688
|
-
page.
|
|
962
|
+
page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
|
|
689
963
|
page.screenshot(path: "t1.png")
|
|
690
964
|
end
|
|
691
965
|
|
|
692
966
|
t2 = Thread.new(context) do |c|
|
|
693
967
|
page = c.create_page
|
|
694
|
-
page.
|
|
968
|
+
page.go_to("https://www.google.com/search?q=Ruby+static+typing")
|
|
695
969
|
page.screenshot(path: "t2.png")
|
|
696
970
|
end
|
|
697
971
|
|
|
@@ -710,7 +984,7 @@ browser = Ferrum::Browser.new
|
|
|
710
984
|
t1 = Thread.new(browser) do |b|
|
|
711
985
|
context = b.contexts.create
|
|
712
986
|
page = context.create_page
|
|
713
|
-
page.
|
|
987
|
+
page.go_to("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
|
|
714
988
|
page.screenshot(path: "t1.png")
|
|
715
989
|
context.dispose
|
|
716
990
|
end
|
|
@@ -718,7 +992,7 @@ end
|
|
|
718
992
|
t2 = Thread.new(browser) do |b|
|
|
719
993
|
context = b.contexts.create
|
|
720
994
|
page = context.create_page
|
|
721
|
-
page.
|
|
995
|
+
page.go_to("https://www.google.com/search?q=Ruby+static+typing")
|
|
722
996
|
page.screenshot(path: "t2.png")
|
|
723
997
|
context.dispose
|
|
724
998
|
end
|
|
@@ -728,3 +1002,21 @@ t2.join
|
|
|
728
1002
|
|
|
729
1003
|
browser.quit
|
|
730
1004
|
```
|
|
1005
|
+
|
|
1006
|
+
## Development
|
|
1007
|
+
|
|
1008
|
+
After checking out the repo, run `bundle install` to install dependencies.
|
|
1009
|
+
|
|
1010
|
+
Then, run `bundle exec rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
|
|
1011
|
+
|
|
1012
|
+
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
|
1013
|
+
|
|
1014
|
+
|
|
1015
|
+
## Contributing
|
|
1016
|
+
|
|
1017
|
+
Bug reports and pull requests are welcome on [GitHub](https://github.com/rubycdp/ferrum).
|
|
1018
|
+
|
|
1019
|
+
## License
|
|
1020
|
+
|
|
1021
|
+
The gem is available as open source under the terms of the
|
|
1022
|
+
[MIT License](https://opensource.org/licenses/MIT).
|