ferrum 0.2.1 → 0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 8c6a15f2244fefda208b8b35181892fd1287414a31485acf98aa55226210cdb3
4
- data.tar.gz: 6214c762f7ea1b85558220de3501fa43791f7f2027682b33d3ee0d6c6ba654ae
3
+ metadata.gz: 5c762e928c31b548af3ecfc9e2ca3cd90e7315c34c313387a0bde299bc201e31
4
+ data.tar.gz: c46c348b3631700428723ca110bb4a40e0106cfca9b84ab0f743bf4356f348e9
5
5
  SHA512:
6
- metadata.gz: 41499a98b5c623ce277ea890255bac0ae89c696c0d297b322b60e73594cad6ad1451dff474ff14fbd58cc5dead1f62d3d7962ec81677a89c9bdc5abb52296bf8
7
- data.tar.gz: 280dc77020f401cdcd0bca4f87214fa83d1899413cda6f1decb761c201cece76a7dec48f4c45973e0726dcf9cb1acaa6b37512fc860f8bfeeea8cf42d1b15d62
6
+ metadata.gz: 854dd57cdd66bfd68535d88f087b145e4f700afdb078c3f0776e374d1fd9c919010c32282cef9b5ac4bb1c2cff21ee7307c7d5b3e3e58bc354cf4477206b8cd9
7
+ data.tar.gz: 207738d98111f3d67322ab926f022e184672b73ae8246acb4a01260d12db07d602f10c29b6612ca1830926091444dae51aeaf0896c5e33d8f202fe9c791dc2a7
data/README.md CHANGED
@@ -1,5 +1,11 @@
1
1
  # Ferrum - fearless Ruby Chrome driver
2
2
 
3
+ [![Build Status](https://travis-ci.org/route/ferrum.svg?branch=master)](https://travis-ci.org/route/ferrum)
4
+
5
+ <img align="right" width="95" height="95"
6
+ alt="Ferrum logo"
7
+ src="https://raw.githubusercontent.com/route/ferrum/master/logo.svg?sanitize=true">
8
+
3
9
  As simple as Puppeteer, though even simpler.
4
10
 
5
11
  It is Ruby clean and high-level API to Chrome. Runs headless by default,
@@ -20,7 +26,7 @@ Chrome binary should be in the `PATH` or `BROWSER_PATH` or you can pass it as an
20
26
  option.
21
27
 
22
28
  ``` ruby
23
- gem "ferrum"
29
+ gem install "ferrum"
24
30
  ```
25
31
 
26
32
  Navigate to a website and save a screenshot:
@@ -265,7 +271,6 @@ Saves screenshot on a disk or returns it as base64.
265
271
  * :selector `String` css selector for given element
266
272
  * :scale `Float` zoom in/out
267
273
 
268
-
269
274
  ```ruby
270
275
  browser.goto("https://google.com/")
271
276
  # Save on the disk in PNG
@@ -291,10 +296,9 @@ Saves PDF on a disk or returns it as base64.
291
296
  * :paper_height `Float` set paper height
292
297
  * See other [native options](https://chromedevtools.github.io/devtools-protocol/tot/Page#method-printToPDF) you can pass
293
298
 
294
-
295
299
  ```ruby
296
300
  browser.goto("https://google.com/")
297
- # Save on the disk in PNG
301
+ # Save to disk as a PDF
298
302
  browser.pdf(path: "google.pdf", paper_width: 1.0, paper_height: 1.0) # => 14983
299
303
  ```
300
304
 
@@ -314,7 +318,9 @@ Cleans up collected data.
314
318
  Returns all headers for a given request in `goto` method.
315
319
 
316
320
 
317
- ## Input
321
+ ### Mouse
322
+
323
+ browser.mouse
318
324
 
319
325
  #### scroll_to(x, y)
320
326
 
@@ -325,9 +331,10 @@ Scroll page to a given x, y
325
331
  * y `Integer` the pixel along the vertical axis of the document that you want
326
332
  displayed in the upper left
327
333
 
328
- ### Mouse
329
-
330
- browser.mouse
334
+ ```ruby
335
+ browser.goto("https://www.google.com/search?q=Ruby+headless+driver+for+Capybara")
336
+ browser.mouse.scroll_to(0, 400)
337
+ ```
331
338
 
332
339
  #### click(\*\*options) : `Mouse`
333
340
 
@@ -538,40 +545,64 @@ end
538
545
  ```
539
546
 
540
547
 
541
- ## Modals
548
+ ## Authorization
549
+
550
+ #### authorize(\*\*options)
551
+
552
+ If site uses authorization you can provide credentials using this method.
542
553
 
543
- #### find_modal
544
- #### accept_confirm
545
- #### dismiss_confirm
546
- #### accept_prompt
547
- #### dismiss_prompt
548
- #### reset_modals
554
+ * options `Hash`
555
+ * :type `Symbol` `:server` | `:proxy` site or proxy authorization
556
+ * :user `String`
557
+ * :password `String`
549
558
 
550
559
 
551
- ## Auth
560
+ ## Dialog
552
561
 
553
- #### authorize(user, password)
562
+ #### accept(text)
554
563
 
555
- If site uses authorization you can provide credentials using this method.
564
+ Accept dialog with given text or default prompt if applicable
556
565
 
557
- * user `String`
558
- * passowrd `String`
566
+ * text `String`
559
567
 
560
- #### proxy_authorize(user, password)
568
+ #### dismiss
561
569
 
562
- If you want to use proxy that requires authentication this is the method you need.
570
+ Dismiss dialog
563
571
 
564
- * user `String`
565
- * passowrd `String`
572
+ ```ruby
573
+ browser = Ferrum::Browser.new
574
+ browser.on(:dialog) do |dialog|
575
+ if dialog.match?(/bla-bla/)
576
+ dialog.accept
577
+ else
578
+ dialog.dismiss
579
+ end
580
+ end
581
+ browser.goto("https://google.com")
582
+ ```
566
583
 
567
584
 
568
585
  ## Interception
569
586
 
570
- #### intercept_request
587
+ #### intercept_request(\*\*options)
588
+
589
+ Set request interception for given options. This method is only sets request
590
+ interception, you should use `on` callback to catch it.
591
+
592
+ * options `Hash`
593
+ * :pattern `String` \* by default
594
+ * :resource_type `Symbol` one of the [resource types](https://chromedevtools.github.io/devtools-protocol/tot/Network#type-ResourceType)
595
+
596
+ #### on(event)
597
+
598
+ Set callback for given event.
599
+
600
+ * event `Symbol`
571
601
 
572
602
  ```ruby
573
603
  browser = Ferrum::Browser.new
574
- browser.intercept_request do |request|
604
+ browser.intercept_request
605
+ browser.on(:request_intercepted) do |request|
575
606
  if request.match?(/bla-bla/)
576
607
  request.abort
577
608
  else
@@ -3,13 +3,9 @@
3
3
  require "ferrum/browser"
4
4
  require "ferrum/node"
5
5
 
6
- Thread.abort_on_exception = true
7
- Thread.report_on_exception = true if Thread.respond_to?(:report_on_exception=)
8
-
9
6
  module Ferrum
10
7
  class Error < StandardError; end
11
8
  class NoSuchWindowError < Error; end
12
- class ModalNotFoundError < Error; end
13
9
  class NotImplementedError < Error; end
14
10
 
15
11
  class EmptyTargetsError < Error
@@ -93,6 +89,22 @@ module Ferrum
93
89
  defined?(RUBY_ENGINE) && RUBY_ENGINE == "ruby"
94
90
  end
95
91
 
92
+ def started
93
+ @@started ||= monotonic_time
94
+ end
95
+
96
+ def elapsed_time(start = nil)
97
+ monotonic_time - (start || @@started)
98
+ end
99
+
100
+ def monotonic_time
101
+ Concurrent.monotonic_time
102
+ end
103
+
104
+ def timeout?(start, timeout)
105
+ elapsed_time(start) > timeout
106
+ end
107
+
96
108
  def with_attempts(errors:, max:, wait:)
97
109
  attempts ||= 1
98
110
  yield
@@ -9,29 +9,26 @@ require "ferrum/browser/client"
9
9
 
10
10
  module Ferrum
11
11
  class Browser
12
- TIMEOUT = 5
12
+ DEFAULT_TIMEOUT = ENV.fetch("FERRUM_DEFAULT_TIMEOUT", 5).to_i
13
13
  WINDOW_SIZE = [1024, 768].freeze
14
14
  BASE_URL_SCHEMA = %w[http https].freeze
15
15
 
16
16
  extend Forwardable
17
-
18
- attr_reader :window_size
19
-
20
- delegate on: :@client
21
17
  delegate %i[window_handle window_handles switch_to_window
22
18
  open_new_window close_window within_window page] => :targets
23
19
  delegate %i[goto back forward refresh status
24
20
  at_css at_xpath css xpath current_url title body
25
21
  headers cookies network_traffic clear_network_traffic response_headers
26
- intercept_request on_request_intercepted continue_request abort_request
27
- mouse keyboard scroll_to
22
+ intercept_request continue_request abort_request
23
+ mouse keyboard
28
24
  screenshot pdf
29
25
  evaluate evaluate_on evaluate_async execute
30
26
  frame_url frame_title within_frame
31
- find_modal accept_confirm dismiss_confirm accept_prompt dismiss_prompt reset_modals
32
- authorize proxy_authorize] => :page
27
+ authorize
28
+ on] => :page
33
29
 
34
- attr_reader :process, :logger, :js_errors, :slowmo, :base_url, :options
30
+ attr_reader :client, :process, :logger, :js_errors, :slowmo, :base_url,
31
+ :options, :window_size
35
32
  attr_writer :timeout
36
33
 
37
34
  def initialize(options = nil)
@@ -75,7 +72,7 @@ module Ferrum
75
72
  end
76
73
 
77
74
  def timeout
78
- @timeout || TIMEOUT
75
+ @timeout || DEFAULT_TIMEOUT
79
76
  end
80
77
 
81
78
  def command(*args)
@@ -121,6 +118,7 @@ module Ferrum
121
118
  private
122
119
 
123
120
  def start
121
+ Ferrum.started
124
122
  @process = Process.start(@options)
125
123
  @client = Client.new(self, @process.ws_url, 0, false)
126
124
  end
@@ -16,6 +16,9 @@ module Ferrum
16
16
  @subscriber = Subscriber.new
17
17
 
18
18
  @thread = Thread.new do
19
+ Thread.current.abort_on_exception = true
20
+ Thread.current.report_on_exception = true if Thread.current.respond_to?(:report_on_exception=)
21
+
19
22
  while message = @ws.messages.pop
20
23
  if message.key?("method")
21
24
  @subscriber.async.call(message)
@@ -50,9 +53,7 @@ module Ferrum
50
53
  @ws.close
51
54
  # Give a thread some time to handle a tail of messages
52
55
  @pendings.clear
53
- Timeout.timeout(1) { @thread.join }
54
- rescue Timeout::Error
55
- @thread.kill
56
+ @thread.kill unless @thread.join(1)
56
57
  end
57
58
 
58
59
  private
@@ -10,7 +10,8 @@ module Ferrum
10
10
  class Browser
11
11
  class Process
12
12
  KILL_TIMEOUT = 2
13
- PROCESS_TIMEOUT = 2
13
+ WAIT_KILLED = 0.05
14
+ PROCESS_TIMEOUT = ENV.fetch("FERRUM_PROCESS_TIMEOUT", 2).to_i
14
15
  BROWSER_PATH = ENV["BROWSER_PATH"]
15
16
  BROWSER_HOST = "127.0.0.1"
16
17
  BROWSER_PORT = "0"
@@ -69,10 +70,10 @@ module Ferrum
69
70
  ::Process.kill("KILL", pid)
70
71
  else
71
72
  ::Process.kill("USR1", pid)
72
- start = ::Process.clock_gettime(::Process::CLOCK_MONOTONIC)
73
+ start = Ferrum.monotonic_time
73
74
  while ::Process.wait(pid, ::Process::WNOHANG).nil?
74
- sleep 0.05
75
- next unless (::Process.clock_gettime(::Process::CLOCK_MONOTONIC) - start) > KILL_TIMEOUT
75
+ sleep(WAIT_KILLED)
76
+ next unless Ferrum.timeout?(start, KILL_TIMEOUT)
76
77
  ::Process.kill("KILL", pid)
77
78
  ::Process.wait(pid)
78
79
  break
@@ -142,17 +143,13 @@ module Ferrum
142
143
  read_io, write_io = IO.pipe
143
144
  process_options = { in: File::NULL }
144
145
  process_options[:pgroup] = true unless Ferrum.windows?
145
- if Ferrum.mri?
146
- process_options[:out] = process_options[:err] = write_io
147
- end
146
+ process_options[:out] = process_options[:err] = write_io
148
147
 
149
148
  raise Cliver::Dependency::NotFound.new(NOT_FOUND) unless @path
150
149
 
151
- redirect_stdout(write_io) do
152
- @cmd = [@path] + @options.map { |k, v| v.nil? ? "--#{k}" : "--#{k}=#{v}" }
153
- @pid = ::Process.spawn(*@cmd, process_options)
154
- ObjectSpace.define_finalizer(self, self.class.process_killer(@pid))
155
- end
150
+ @cmd = [@path] + @options.map { |k, v| v.nil? ? "--#{k}" : "--#{k}=#{v}" }
151
+ @pid = ::Process.spawn(*@cmd, process_options)
152
+ ObjectSpace.define_finalizer(self, self.class.process_killer(@pid))
156
153
 
157
154
  parse_ws_url(read_io, @process_timeout)
158
155
  ensure
@@ -173,34 +170,17 @@ module Ferrum
173
170
 
174
171
  private
175
172
 
176
- def redirect_stdout(write_io)
177
- if Ferrum.mri?
178
- yield
179
- else
180
- begin
181
- prev = STDOUT.dup
182
- $stdout = write_io
183
- STDOUT.reopen(write_io)
184
- yield
185
- ensure
186
- STDOUT.reopen(prev)
187
- $stdout = STDOUT
188
- prev.close
189
- end
190
- end
191
- end
192
-
193
173
  def kill
194
174
  self.class.process_killer(@pid).call
195
175
  @pid = nil
196
176
  end
197
177
 
198
- def parse_ws_url(read_io, timeout = PROCESS_TIMEOUT)
178
+ def parse_ws_url(read_io, timeout)
199
179
  output = ""
200
- start = ::Process.clock_gettime(::Process::CLOCK_MONOTONIC)
180
+ start = Ferrum.monotonic_time
201
181
  max_time = start + timeout
202
182
  regexp = /DevTools listening on (ws:\/\/.*)/
203
- while (now = ::Process.clock_gettime(::Process::CLOCK_MONOTONIC)) < max_time
183
+ while (now = Ferrum.monotonic_time) < max_time
204
184
  begin
205
185
  output += read_io.read_nonblock(512)
206
186
  rescue IO::WaitReadable
@@ -24,6 +24,9 @@ module Ferrum
24
24
  @driver.on(:close, &method(:on_close))
25
25
 
26
26
  @thread = Thread.new do
27
+ Thread.current.abort_on_exception = true
28
+ Thread.current.report_on_exception = true if Thread.current.respond_to?(:report_on_exception=)
29
+
27
30
  begin
28
31
  while data = @sock.readpartial(512)
29
32
  @driver.parse(data)
@@ -33,8 +36,6 @@ module Ferrum
33
36
  end
34
37
  end
35
38
 
36
- @thread.priority = 1
37
-
38
39
  @driver.start
39
40
  end
40
41
 
@@ -46,7 +47,7 @@ module Ferrum
46
47
  def on_message(event)
47
48
  data = JSON.parse(event.data)
48
49
  @messages.push(data)
49
- @logger&.puts(" ◀ #{event.data}\n")
50
+ @logger&.puts(" ◀ #{Ferrum.elapsed_time} #{event.data}\n")
50
51
  end
51
52
 
52
53
  def on_close(_event)
@@ -57,7 +58,7 @@ module Ferrum
57
58
  def send_message(data)
58
59
  json = data.to_json
59
60
  @driver.text(json)
60
- @logger&.puts("\n\n▶ #{json}")
61
+ @logger&.puts("\n\n▶ #{Ferrum.elapsed_time} #{json}")
61
62
  end
62
63
 
63
64
  def write(data)
@@ -0,0 +1,28 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Ferrum
4
+ class Dialog
5
+ attr_reader :message, :default_prompt
6
+
7
+ def initialize(page, params)
8
+ @page = page
9
+ @message = params["message"]
10
+ @default_prompt = params["defaultPrompt"]
11
+ end
12
+
13
+ def accept(prompt_text = nil)
14
+ options = { accept: true }
15
+ response = prompt_text || default_prompt
16
+ options.merge!(promptText: response) if response
17
+ @page.command("Page.handleJavaScriptDialog", **options)
18
+ end
19
+
20
+ def dismiss
21
+ @page.command("Page.handleJavaScriptDialog", accept: false)
22
+ end
23
+
24
+ def match?(regexp)
25
+ !!message.match(regexp)
26
+ end
27
+ end
28
+ end
@@ -2,6 +2,7 @@
2
2
 
3
3
  module Ferrum
4
4
  class Mouse
5
+ CLICK_WAIT = ENV.fetch("FERRUM_CLICK_WAIT", 0.1).to_f
5
6
  VALID_BUTTONS = %w[none left middle right back forward].freeze
6
7
 
7
8
  def initialize(page)
@@ -9,12 +10,17 @@ module Ferrum
9
10
  @x = @y = 0
10
11
  end
11
12
 
12
- def click(x:, y:, delay: 0, timeout: 0, **options)
13
+ def scroll_to(top, left)
14
+ tap { @page.execute("window.scrollTo(#{top}, #{left})") }
15
+ end
16
+
17
+ def click(x:, y:, delay: 0, wait: CLICK_WAIT, **options)
13
18
  move(x: x, y: y)
14
19
  down(**options)
15
20
  sleep(delay)
16
- # Potential wait because if network event is triggered then we have to wait until it's over.
17
- up(timeout: timeout, **options)
21
+ # Potential wait because if some network event is triggered then we have
22
+ # to wait until it's over and frame is loaded or failed to load.
23
+ up(wait: wait, **options)
18
24
  self
19
25
  end
20
26
 
@@ -35,11 +41,11 @@ module Ferrum
35
41
 
36
42
  private
37
43
 
38
- def mouse_event(type:, button: :left, count: 1, modifiers: nil, timeout: 0)
44
+ def mouse_event(type:, button: :left, count: 1, modifiers: nil, wait: 0)
39
45
  button = validate_button(button)
40
46
  options = { x: @x, y: @y, type: type, button: button, clickCount: count }
41
47
  options.merge!(modifiers: modifiers) if modifiers
42
- @page.command("Input.dispatchMouseEvent", timeout: timeout, **options)
48
+ @page.command("Input.dispatchMouseEvent", wait: wait, **options)
43
49
  end
44
50
 
45
51
  def validate_button(button)
@@ -19,7 +19,7 @@ module Ferrum::Network
19
19
  end
20
20
 
21
21
  def match?(regexp)
22
- !!(url =~ regexp)
22
+ !!url.match(regexp)
23
23
  end
24
24
 
25
25
  def abort
@@ -30,7 +30,7 @@ module Ferrum
30
30
  # keys: (:alt, (:ctrl | :control), (:meta | :command), :shift)
31
31
  # offset: { :x, :y }
32
32
  def click(mode: :left, keys: [], offset: {})
33
- x, y = page.find_position(self, offset[:x], offset[:y])
33
+ x, y = find_position(offset[:x], offset[:y])
34
34
  modifiers = page.keyboard.modifiers(keys)
35
35
 
36
36
  case mode
@@ -43,7 +43,7 @@ module Ferrum
43
43
  page.mouse.down(modifiers: modifiers, count: 2)
44
44
  page.mouse.up(modifiers: modifiers, count: 2)
45
45
  when :left
46
- page.mouse.click(x: x, y: y, modifiers: modifiers, timeout: 0.05)
46
+ page.mouse.click(x: x, y: y, modifiers: modifiers)
47
47
  end
48
48
 
49
49
  self
@@ -53,10 +53,6 @@ module Ferrum
53
53
  raise NotImplementedError
54
54
  end
55
55
 
56
- def trigger(event)
57
- raise NotImplementedError
58
- end
59
-
60
56
  def select_file(value)
61
57
  page.command("DOM.setFileInputFiles", nodeId: node_id, files: Array(value))
62
58
  end
@@ -81,6 +77,11 @@ module Ferrum
81
77
  evaluate("this.textContent")
82
78
  end
83
79
 
80
+ # FIXME: clear API for text and inner_text
81
+ def inner_text
82
+ evaluate("this.innerText")
83
+ end
84
+
84
85
  def value
85
86
  evaluate("this.value")
86
87
  end
@@ -108,5 +109,36 @@ module Ferrum
108
109
  def inspect
109
110
  %(#<#{self.class} @target_id=#{@target_id.inspect} @node_id=#{@node_id} @description=#{@description.inspect}>)
110
111
  end
112
+
113
+ def find_position(offset_x = nil, offset_y = nil)
114
+ quads = get_content_quads
115
+ offset_x, offset_y = offset_x.to_i, offset_y.to_i
116
+
117
+ if offset_x > 0 || offset_y > 0
118
+ point = quads.first
119
+ [point[:x] + offset_x, point[:y] + offset_y]
120
+ else
121
+ x, y = quads.inject([0, 0]) do |memo, point|
122
+ [memo[0] + point[:x],
123
+ memo[1] + point[:y]]
124
+ end
125
+ [x / 4, y / 4]
126
+ end
127
+ end
128
+
129
+ private
130
+
131
+ def get_content_quads
132
+ result = page.command("DOM.getContentQuads", nodeId: node_id)
133
+ raise "Node is either not visible or not an HTMLElement" if result["quads"].size == 0
134
+
135
+ # FIXME: Case when a few quads returned
136
+ result["quads"].map do |quad|
137
+ [{x: quad[0], y: quad[1]},
138
+ {x: quad[2], y: quad[3]},
139
+ {x: quad[4], y: quad[5]},
140
+ {x: quad[6], y: quad[7]}]
141
+ end.first
142
+ end
111
143
  end
112
144
  end
@@ -4,8 +4,8 @@ require "ferrum/mouse"
4
4
  require "ferrum/keyboard"
5
5
  require "ferrum/headers"
6
6
  require "ferrum/cookies"
7
+ require "ferrum/dialog"
7
8
  require "ferrum/page/dom"
8
- require "ferrum/page/input"
9
9
  require "ferrum/page/runtime"
10
10
  require "ferrum/page/frame"
11
11
  require "ferrum/page/net"
@@ -35,9 +35,23 @@ require "ferrum/network/intercepted_request"
35
35
  # details (DOM.describeNode).
36
36
  module Ferrum
37
37
  class Page
38
- NEW_WINDOW_BUG_SLEEP = 0.3
38
+ NEW_WINDOW_WAIT = ENV.fetch("FERRUM_NEW_WINDOW_WAIT", 0.3).to_f
39
39
 
40
- include Input, DOM, Runtime, Frame, Net, Screenshot
40
+ class Event < Concurrent::Event
41
+ def iteration
42
+ synchronize { @iteration }
43
+ end
44
+
45
+ def reset
46
+ synchronize do
47
+ @iteration += 1
48
+ @set = false if @set
49
+ @iteration
50
+ end
51
+ end
52
+ end
53
+
54
+ include DOM, Runtime, Frame, Net, Screenshot
41
55
 
42
56
  attr_accessor :referrer
43
57
  attr_reader :target_id, :status,
@@ -48,16 +62,14 @@ module Ferrum
48
62
  def initialize(target_id, browser, new_window = false)
49
63
  @target_id, @browser = target_id, browser
50
64
  @network_traffic = []
51
- @event = Concurrent::Event.new.tap(&:set)
65
+ @event = Event.new.tap(&:set)
52
66
 
53
67
  @frames = {}
54
68
  @waiting_frames ||= Set.new
55
69
  @frame_stack = []
56
- @accept_modal = []
57
- @modal_messages = []
58
70
 
59
71
  # Dirty hack because new window doesn't have events at all
60
- sleep(NEW_WINDOW_BUG_SLEEP) if new_window
72
+ sleep(NEW_WINDOW_WAIT) if new_window
61
73
 
62
74
  @session_id = @browser.command("Target.attachToTarget", targetId: @target_id)["sessionId"]
63
75
 
@@ -80,7 +92,7 @@ module Ferrum
80
92
  def goto(url = nil)
81
93
  options = { url: combine_url!(url) }
82
94
  options.merge!(referrer: referrer) if referrer
83
- response = command("Page.navigate", timeout: timeout, **options)
95
+ response = command("Page.navigate", wait: timeout, **options)
84
96
  # https://cs.chromium.org/chromium/src/net/base/net_error_list.h
85
97
  if %w[net::ERR_NAME_NOT_RESOLVED
86
98
  net::ERR_NAME_RESOLUTION_FAILED
@@ -116,7 +128,7 @@ module Ferrum
116
128
  end
117
129
 
118
130
  def refresh
119
- command("Page.reload", timeout: timeout)
131
+ command("Page.reload", wait: timeout)
120
132
  end
121
133
 
122
134
  def network_traffic(type = nil)
@@ -142,58 +154,31 @@ module Ferrum
142
154
  history_navigate(delta: 1)
143
155
  end
144
156
 
145
- def accept_confirm
146
- @accept_modal << true
147
- end
148
-
149
- def dismiss_confirm
150
- @accept_modal << false
151
- end
152
-
153
- def accept_prompt(modal_response)
154
- @accept_modal << true
155
- @modal_response = modal_response
156
- end
157
-
158
- def dismiss_prompt
159
- @accept_modal << false
160
- end
161
-
162
- def find_modal(options)
163
- start_time = ::Process.clock_gettime(::Process::CLOCK_MONOTONIC)
164
- timeout_sec = options.fetch(:wait) { session_wait_time }
165
- expect_text = options[:text]
166
- expect_regexp = expect_text.is_a?(Regexp) ? expect_text : Regexp.escape(expect_text.to_s)
167
- not_found_msg = "Unable to find modal dialog"
168
- not_found_msg += " with #{expect_text}" if expect_text
169
-
170
- begin
171
- modal_text = @modal_messages.shift
172
- raise ModalNotFoundError if modal_text.nil? || (expect_text && !modal_text.match(expect_regexp))
173
- rescue ModalNotFoundError => e
174
- raise e, not_found_msg if (::Process.clock_gettime(::Process::CLOCK_MONOTONIC) - start_time) >= timeout_sec
175
- sleep(0.05)
176
- retry
157
+ def command(method, wait: 0, **params)
158
+ iteration = @event.reset if wait > 0
159
+ result = @client.command(method, params)
160
+ if wait > 0
161
+ @event.wait(wait)
162
+ @event.wait(@browser.timeout) if iteration != @event.iteration
177
163
  end
178
-
179
- modal_text
180
- end
181
-
182
- def reset_modals
183
- @accept_modal = []
184
- @modal_response = nil
185
- @modal_messages = []
164
+ result
186
165
  end
187
166
 
188
- def command(method, timeout: 0, **params)
189
- result = @client.command(method, params)
190
-
191
- if timeout > 0
192
- @event.reset
193
- @event.wait(timeout)
167
+ def on(name, &block)
168
+ case name
169
+ when :dialog
170
+ @client.on("Page.javascriptDialogOpening") do |params, index, total|
171
+ dialog = Dialog.new(self, params)
172
+ block.call(dialog, index, total)
173
+ end
174
+ when :request_intercepted
175
+ @client.on("Network.requestIntercepted") do |params, index, total|
176
+ request = Network::InterceptedRequest.new(self, params)
177
+ block.call(request, index, total)
178
+ end
179
+ else
180
+ @client.on(name, &block)
194
181
  end
195
-
196
- result
197
182
  end
198
183
 
199
184
  private
@@ -209,28 +194,11 @@ module Ferrum
209
194
 
210
195
  if @browser.js_errors
211
196
  @client.on("Runtime.exceptionThrown") do |params|
197
+ # FIXME https://jvns.ca/blog/2015/11/27/why-rubys-timeout-is-dangerous-and-thread-dot-raise-is-terrifying/
212
198
  Thread.main.raise JavaScriptError.new(params.dig("exceptionDetails", "exception"))
213
199
  end
214
200
  end
215
201
 
216
- @client.on("Page.javascriptDialogOpening") do |params|
217
- accept_modal = @accept_modal.last
218
- if accept_modal == true || accept_modal == false
219
- @accept_modal.pop
220
- @modal_messages << params["message"]
221
- options = { accept: accept_modal }
222
- response = @modal_response || params["defaultPrompt"]
223
- options.merge!(promptText: response) if response
224
- command("Page.handleJavaScriptDialog", **options)
225
- else
226
- warn "Modal window has been opened, but you didn't wrap your code into (`accept_prompt` | `dismiss_prompt` | `accept_confirm` | `dismiss_confirm` | `accept_alert`), accepting by default"
227
- options = { accept: true }
228
- response = params["defaultPrompt"]
229
- options.merge!(promptText: response) if response
230
- command("Page.handleJavaScriptDialog", **options)
231
- end
232
- end
233
-
234
202
  @client.on("Page.windowOpen") do
235
203
  @browser.targets.refresh
236
204
  end
@@ -352,7 +320,7 @@ module Ferrum
352
320
 
353
321
  if entry = entries[index + delta]
354
322
  # Potential wait because of network event
355
- command("Page.navigateToHistoryEntry", timeout: 0.05, entryId: entry["id"])
323
+ command("Page.navigateToHistoryEntry", wait: Mouse::CLICK_WAIT, entryId: entry["id"])
356
324
  end
357
325
  end
358
326
 
@@ -3,40 +3,28 @@
3
3
  module Ferrum
4
4
  class Page
5
5
  module Net
6
+ AUTHORIZE_TYPE = %i[server proxy]
6
7
  RESOURCE_TYPES = %w[Document Stylesheet Image Media Font Script TextTrack
7
8
  XHR Fetch EventSource WebSocket Manifest
8
9
  SignedExchange Ping CSPViolationReport Other]
9
10
 
10
- def proxy_authorize(user, password)
11
- @proxy_authorized_ids ||= []
12
-
13
- if user && password
14
- intercept_request do |request, index, total|
15
- if request.auth_challenge?(:proxy)
16
- response = authorized_response(@proxy_authorized_ids,
17
- request.interception_id,
18
- user, password)
19
- @proxy_authorized_ids << request.interception_id
20
- request.continue(authChallengeResponse: response)
21
- elsif index + 1 < total
22
- next # There are other callbacks that can handle this, skip
23
- else
24
- request.continue
25
- end
26
- end
11
+ def authorize(user:, password:, type: :server)
12
+ unless AUTHORIZE_TYPE.include?(type)
13
+ raise ArgumentError, ":type should be in #{AUTHORIZE_TYPE}"
27
14
  end
28
- end
29
15
 
30
- def authorize(user, password)
31
- @authorized_ids ||= []
16
+ @authorized_ids ||= {}
17
+ @authorized_ids[type] ||= []
18
+
19
+ intercept_request
32
20
 
33
- intercept_request do |request, index, total|
34
- if request.auth_challenge?(:server)
35
- response = authorized_response(@authorized_ids,
21
+ on(:request_intercepted) do |request, index, total|
22
+ if request.auth_challenge?(type)
23
+ response = authorized_response(@authorized_ids[type],
36
24
  request.interception_id,
37
25
  user, password)
38
26
 
39
- @authorized_ids << request.interception_id
27
+ @authorized_ids[type] << request.interception_id
40
28
  request.continue(authChallengeResponse: response)
41
29
  elsif index + 1 < total
42
30
  next # There are other callbacks that can handle this, skip
@@ -46,22 +34,13 @@ module Ferrum
46
34
  end
47
35
  end
48
36
 
49
- def intercept_request(pattern: "*", resource_type: nil, &block)
37
+ def intercept_request(pattern: "*", resource_type: nil)
50
38
  pattern = { urlPattern: pattern }
51
39
  if resource_type && RESOURCE_TYPES.include?(resource_type.to_s)
52
40
  pattern[:resourceType] = resource_type
53
41
  end
54
42
 
55
43
  command("Network.setRequestInterception", patterns: [pattern])
56
-
57
- on_request_intercepted(&block) if block_given?
58
- end
59
-
60
- def on_request_intercepted(&block)
61
- @client.on("Network.requestIntercepted") do |params, index, total|
62
- request = Network::InterceptedRequest.new(self, params)
63
- block.call(request, index, total)
64
- end
65
44
  end
66
45
 
67
46
  def continue_request(interception_id, options = nil)
@@ -3,6 +3,9 @@
3
3
  module Ferrum
4
4
  class Page
5
5
  module Runtime
6
+ INTERMITTENT_ATTEMPTS = ENV.fetch("FERRUM_INTERMITTENT_ATTEMPTS", 6).to_i
7
+ INTERMITTENT_SLEEP = ENV.fetch("FERRUM_INTERMITTENT_SLEEP", 0.1).to_f
8
+
6
9
  EXECUTE_OPTIONS = {
7
10
  returnByValue: true,
8
11
  functionDeclaration: %(function() { %s })
@@ -41,12 +44,11 @@ module Ferrum
41
44
  true
42
45
  end
43
46
 
44
- def evaluate_on(node:, expression:, by_value: true, timeout: 0)
47
+ def evaluate_on(node:, expression:, by_value: true, wait: 0)
45
48
  errors = [NodeNotFoundError, NoExecutionContextError]
46
- max = ENV.fetch("FERRUM_INTERMITTENT_ATTEMPTS", 6).to_i
47
- wait = ENV.fetch("FERRUM_INTERMITTENT_SLEEP", 0.1).to_f
49
+ attempts, sleep = INTERMITTENT_ATTEMPTS, INTERMITTENT_SLEEP
48
50
 
49
- Ferrum.with_attempts(errors: errors, max: max, wait: wait) do
51
+ Ferrum.with_attempts(errors: errors, max: attempts, wait: sleep) do
50
52
  response = command("DOM.resolveNode", nodeId: node.node_id)
51
53
  object_id = response.dig("object", "objectId")
52
54
  options = DEFAULT_OPTIONS.merge(objectId: object_id)
@@ -54,8 +56,8 @@ module Ferrum
54
56
  options.merge!(returnByValue: by_value)
55
57
 
56
58
  response = command("Runtime.callFunctionOn",
57
- timeout: timeout,
58
- **options)["result"].tap { |r| handle_error(r) }
59
+ wait: wait, **options)["result"]
60
+ .tap { |r| handle_error(r) }
59
61
 
60
62
  by_value ? response.dig("value") : handle_response(response)
61
63
  end
@@ -65,10 +67,9 @@ module Ferrum
65
67
 
66
68
  def call(*args, expression:, wait_time: nil, handle: true, **options)
67
69
  errors = [NodeNotFoundError, NoExecutionContextError]
68
- max = ENV.fetch("FERRUM_INTERMITTENT_ATTEMPTS", 6).to_i
69
- wait = ENV.fetch("FERRUM_INTERMITTENT_SLEEP", 0.1).to_f
70
+ attempts, sleep = INTERMITTENT_ATTEMPTS, INTERMITTENT_SLEEP
70
71
 
71
- Ferrum.with_attempts(errors: errors, max: max, wait: wait) do
72
+ Ferrum.with_attempts(errors: errors, max: attempts, wait: sleep) do
72
73
  arguments = prepare_args(args)
73
74
  params = DEFAULT_OPTIONS.merge(options)
74
75
  expression = [wait_time, expression] if wait_time
@@ -172,12 +173,24 @@ module Ferrum
172
173
  return false;
173
174
  }
174
175
 
175
- try {
176
- JSON.stringify(this);
176
+ const seen = [];
177
+ function detectCycle(obj) {
178
+ if (typeof obj === 'object') {
179
+ if (seen.indexOf(obj) !== -1) {
180
+ return true;
181
+ }
182
+ seen.push(obj);
183
+ for (let key in obj) {
184
+ if (obj.hasOwnProperty(key) && detectCycle(obj[key])) {
185
+ return true;
186
+ }
187
+ }
188
+ }
189
+
177
190
  return false;
178
- } catch (e) {
179
- return true;
180
191
  }
192
+
193
+ return detectCycle(this);
181
194
  }
182
195
  JS
183
196
  )
@@ -11,7 +11,7 @@ module Ferrum
11
11
  @browser = browser
12
12
  @_default = targets.first["targetId"]
13
13
 
14
- @browser.on("Target.detachedFromTarget") do |params|
14
+ @browser.client.on("Target.detachedFromTarget") do |params|
15
15
  page = remove_page(params["targetId"])
16
16
  page&.close_connection
17
17
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Ferrum
4
- VERSION = "0.2.1"
4
+ VERSION = "0.3"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: ferrum
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.1
4
+ version: '0.3'
5
5
  platform: ruby
6
6
  authors:
7
7
  - Dmitry Vorotilin
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2019-09-05 00:00:00.000000000 Z
11
+ date: 2019-09-12 00:00:00.000000000 Z
12
12
  dependencies:
13
13
  - !ruby/object:Gem::Dependency
14
14
  name: websocket-driver
@@ -129,61 +129,61 @@ dependencies:
129
129
  - !ruby/object:Gem::Version
130
130
  version: '4.1'
131
131
  - !ruby/object:Gem::Dependency
132
- name: byebug
132
+ name: image_size
133
133
  requirement: !ruby/object:Gem::Requirement
134
134
  requirements:
135
135
  - - "~>"
136
136
  - !ruby/object:Gem::Version
137
- version: '10.0'
137
+ version: '2.0'
138
138
  type: :development
139
139
  prerelease: false
140
140
  version_requirements: !ruby/object:Gem::Requirement
141
141
  requirements:
142
142
  - - "~>"
143
143
  - !ruby/object:Gem::Version
144
- version: '10.0'
144
+ version: '2.0'
145
145
  - !ruby/object:Gem::Dependency
146
- name: image_size
146
+ name: pdf-reader
147
147
  requirement: !ruby/object:Gem::Requirement
148
148
  requirements:
149
149
  - - "~>"
150
150
  - !ruby/object:Gem::Version
151
- version: '2.0'
151
+ version: '2.2'
152
152
  type: :development
153
153
  prerelease: false
154
154
  version_requirements: !ruby/object:Gem::Requirement
155
155
  requirements:
156
156
  - - "~>"
157
157
  - !ruby/object:Gem::Version
158
- version: '2.0'
158
+ version: '2.2'
159
159
  - !ruby/object:Gem::Dependency
160
- name: pdf-reader
160
+ name: chunky_png
161
161
  requirement: !ruby/object:Gem::Requirement
162
162
  requirements:
163
163
  - - "~>"
164
164
  - !ruby/object:Gem::Version
165
- version: '2.2'
165
+ version: '1.3'
166
166
  type: :development
167
167
  prerelease: false
168
168
  version_requirements: !ruby/object:Gem::Requirement
169
169
  requirements:
170
170
  - - "~>"
171
171
  - !ruby/object:Gem::Version
172
- version: '2.2'
172
+ version: '1.3'
173
173
  - !ruby/object:Gem::Dependency
174
- name: chunky_png
174
+ name: byebug
175
175
  requirement: !ruby/object:Gem::Requirement
176
176
  requirements:
177
177
  - - "~>"
178
178
  - !ruby/object:Gem::Version
179
- version: '1.3'
179
+ version: '10.0'
180
180
  type: :development
181
181
  prerelease: false
182
182
  version_requirements: !ruby/object:Gem::Requirement
183
183
  requirements:
184
184
  - - "~>"
185
185
  - !ruby/object:Gem::Version
186
- version: '1.3'
186
+ version: '10.0'
187
187
  description: Ferrum allows you to control headless Chrome browser
188
188
  email:
189
189
  - d.vorotilin@gmail.com
@@ -200,6 +200,7 @@ files:
200
200
  - lib/ferrum/browser/subscriber.rb
201
201
  - lib/ferrum/browser/web_socket.rb
202
202
  - lib/ferrum/cookies.rb
203
+ - lib/ferrum/dialog.rb
203
204
  - lib/ferrum/headers.rb
204
205
  - lib/ferrum/keyboard.json
205
206
  - lib/ferrum/keyboard.rb
@@ -212,7 +213,6 @@ files:
212
213
  - lib/ferrum/page.rb
213
214
  - lib/ferrum/page/dom.rb
214
215
  - lib/ferrum/page/frame.rb
215
- - lib/ferrum/page/input.rb
216
216
  - lib/ferrum/page/net.rb
217
217
  - lib/ferrum/page/runtime.rb
218
218
  - lib/ferrum/page/screenshot.rb
@@ -1,44 +0,0 @@
1
- # frozen_string_literal: true
2
-
3
- require "json"
4
-
5
- module Ferrum
6
- class Page
7
- module Input
8
- def scroll_to(top, left)
9
- execute("window.scrollTo(#{top}, #{left})")
10
- end
11
-
12
- def find_position(node, offset_x = nil, offset_y = nil)
13
- quads = get_content_quads(node)
14
- offset_x, offset_y = offset_x.to_i, offset_y.to_i
15
-
16
- if offset_x > 0 || offset_y > 0
17
- point = quads.first
18
- [point[:x] + offset_x, point[:y] + offset_y]
19
- else
20
- x, y = quads.inject([0, 0]) do |memo, point|
21
- [memo[0] + point[:x],
22
- memo[1] + point[:y]]
23
- end
24
- [x / 4, y / 4]
25
- end
26
- end
27
-
28
- private
29
-
30
- def get_content_quads(node)
31
- result = command("DOM.getContentQuads", nodeId: node.node_id)
32
- raise "Node is either not visible or not an HTMLElement" if result["quads"].size == 0
33
-
34
- # FIXME: Case when a few quads returned
35
- result["quads"].map do |quad|
36
- [{x: quad[0], y: quad[1]},
37
- {x: quad[2], y: quad[3]},
38
- {x: quad[4], y: quad[5]},
39
- {x: quad[6], y: quad[7]}]
40
- end.first
41
- end
42
- end
43
- end
44
- end