ferrum 0.11 → 0.12

Sign up to get free protection for your applications and to get access to all the features.
Files changed (45) hide show
  1. checksums.yaml +4 -4
  2. data/LICENSE +1 -1
  3. data/README.md +178 -29
  4. data/lib/ferrum/browser/binary.rb +46 -0
  5. data/lib/ferrum/browser/client.rb +13 -12
  6. data/lib/ferrum/browser/command.rb +7 -8
  7. data/lib/ferrum/browser/options/base.rb +1 -7
  8. data/lib/ferrum/browser/options/chrome.rb +17 -11
  9. data/lib/ferrum/browser/options/firefox.rb +11 -4
  10. data/lib/ferrum/browser/process.rb +41 -35
  11. data/lib/ferrum/browser/subscriber.rb +1 -3
  12. data/lib/ferrum/browser/web_socket.rb +9 -12
  13. data/lib/ferrum/browser/xvfb.rb +4 -8
  14. data/lib/ferrum/browser.rb +44 -12
  15. data/lib/ferrum/context.rb +6 -2
  16. data/lib/ferrum/contexts.rb +10 -8
  17. data/lib/ferrum/cookies.rb +10 -9
  18. data/lib/ferrum/errors.rb +115 -0
  19. data/lib/ferrum/frame/runtime.rb +20 -17
  20. data/lib/ferrum/frame.rb +32 -24
  21. data/lib/ferrum/headers.rb +2 -2
  22. data/lib/ferrum/keyboard.rb +11 -11
  23. data/lib/ferrum/mouse.rb +8 -7
  24. data/lib/ferrum/network/auth_request.rb +7 -2
  25. data/lib/ferrum/network/exchange.rb +14 -10
  26. data/lib/ferrum/network/intercepted_request.rb +10 -8
  27. data/lib/ferrum/network/request.rb +5 -0
  28. data/lib/ferrum/network/response.rb +4 -4
  29. data/lib/ferrum/network.rb +124 -35
  30. data/lib/ferrum/node.rb +69 -23
  31. data/lib/ferrum/page/animation.rb +0 -1
  32. data/lib/ferrum/page/frames.rb +46 -20
  33. data/lib/ferrum/page/screenshot.rb +51 -65
  34. data/lib/ferrum/page/stream.rb +38 -0
  35. data/lib/ferrum/page/tracing.rb +71 -0
  36. data/lib/ferrum/page.rb +81 -36
  37. data/lib/ferrum/proxy.rb +58 -0
  38. data/lib/ferrum/{rbga.rb → rgba.rb} +4 -2
  39. data/lib/ferrum/target.rb +1 -0
  40. data/lib/ferrum/utils/attempt.rb +20 -0
  41. data/lib/ferrum/utils/elapsed_time.rb +27 -0
  42. data/lib/ferrum/utils/platform.rb +28 -0
  43. data/lib/ferrum/version.rb +1 -1
  44. data/lib/ferrum.rb +4 -146
  45. metadata +60 -51
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: c953c12b582e6d81031c8daa598f25d4f8b2dba8e6d736f227b2e8c461a83d41
4
- data.tar.gz: 229fe35bc81a133eff2628e1b3e0e0d31d733da2b277e0dbe70c9699e80e2a15
3
+ metadata.gz: 9e4ca57662283c8c6b917aa3e2f1f06b154cb66ffdd8522756f55e09be84a423
4
+ data.tar.gz: 50d37d615399fe481634154d619d55872ee13b0cace511a1b147da38454247b5
5
5
  SHA512:
6
- metadata.gz: 1f48e9fac5bfbcf9959d855b1b7c7259f97d845e812f3fce6fa59dc6a3e6b5147cced16167c2a3603e964978a9d3b1e4e5b4f2c7b5b686454acaa47d1aadf6d3
7
- data.tar.gz: aa9f267ea80365e636e8b047bc87e6d5fe6761814cbd1b59b3d14bc3350ccdd9e2199718ba57c774abdf5bc089ef000cdbd55d03d7dbd6158cee4c975ee9d9e0
6
+ metadata.gz: 9cc514740ae8041b61a184bdeee0a44ce26b276f36af0358c6a00524f8e13b6db9ddda48acf5781c4122acaa4980579cf06053ffb420c2013e0b44177eab2571
7
+ data.tar.gz: cc00b6d3cb440db600241156354dcd92a6bf67c0402b9e2881f32ea507549d80f69b51011bb246baf9faf95f33e5596933ce569cd9b03f41c7cc066d215b6073
data/LICENSE CHANGED
@@ -1,6 +1,6 @@
1
1
  MIT License
2
2
 
3
- Copyright (c) 2019-2020 Dmitry Vorotilin
3
+ Copyright (c) 2019-2022 Dmitry Vorotilin
4
4
 
5
5
  Permission is hereby granted, free of charge, to any person obtaining a copy
6
6
  of this software and associated documentation files (the "Software"), to deal
data/README.md CHANGED
@@ -3,7 +3,7 @@
3
3
  <img align="right"
4
4
  width="320" height="241"
5
5
  alt="Ferrum logo"
6
- src="https://raw.githubusercontent.com/rubycdp/ferrum/master/logo.svg?sanitize=true">
6
+ src="https://raw.githubusercontent.com/rubycdp/ferrum/main/logo.svg?sanitize=true">
7
7
 
8
8
  #### As simple as Puppeteer, though even simpler.
9
9
 
@@ -23,12 +23,7 @@ going to crawl sites you better use Ferrum or
23
23
  [Vessel](https://github.com/rubycdp/vessel) because you crawl, not test.
24
24
 
25
25
  * [Vessel](https://github.com/rubycdp/vessel) high-level web crawling framework
26
- based on Ferrum. It looks like [Scrapy](https://scrapy.org/) except that it uses
27
- a real browser in order to grab data.
28
-
29
- Web design by [Evrone](https://evrone.com/), what else
30
- [we build with Ruby on Rails](https://evrone.com/ruby), what else
31
- [we do at Evrone](https://evrone.com/cases#case-studies).
26
+ based on Ferrum and Mechanize.
32
27
 
33
28
 
34
29
  ## Index
@@ -40,7 +35,9 @@ Web design by [Evrone](https://evrone.com/), what else
40
35
  * [Navigation](https://github.com/rubycdp/ferrum#navigation)
41
36
  * [Finders](https://github.com/rubycdp/ferrum#finders)
42
37
  * [Screenshots](https://github.com/rubycdp/ferrum#screenshots)
38
+ * [Cleaning Up](https://github.com/rubycdp/ferrum#cleaning-up)
43
39
  * [Network](https://github.com/rubycdp/ferrum#network)
40
+ * [Proxy](https://github.com/rubycdp/ferrum#proxy)
44
41
  * [Mouse](https://github.com/rubycdp/ferrum#mouse)
45
42
  * [Keyboard](https://github.com/rubycdp/ferrum#keyboard)
46
43
  * [Cookies](https://github.com/rubycdp/ferrum#cookies)
@@ -48,9 +45,10 @@ Web design by [Evrone](https://evrone.com/), what else
48
45
  * [JavaScript](https://github.com/rubycdp/ferrum#javascript)
49
46
  * [Frames](https://github.com/rubycdp/ferrum#frames)
50
47
  * [Frame](https://github.com/rubycdp/ferrum#frame)
51
- * [Dialog](https://github.com/rubycdp/ferrum#dialog)
48
+ * [Dialogs](https://github.com/rubycdp/ferrum#dialogs)
52
49
  * [Animation](https://github.com/rubycdp/ferrum#animation)
53
50
  * [Node](https://github.com/rubycdp/ferrum#node)
51
+ * [Tracing](https://github.com/rubycdp/ferrum#tracing)
54
52
  * [Thread safety](https://github.com/rubycdp/ferrum#thread-safety)
55
53
  * [Development](https://github.com/rubycdp/ferrum#development)
56
54
  * [Contributing](https://github.com/rubycdp/ferrum#contributing)
@@ -61,7 +59,8 @@ Web design by [Evrone](https://evrone.com/), what else
61
59
 
62
60
  There's no official Chrome or Chromium package for Linux don't install it this
63
61
  way because it's either outdated or unofficial, both are bad. Download it from
64
- official [source](https://www.chromium.org/getting-involved/download-chromium).
62
+ official source for [Chrome](https://www.google.com/chrome/) or
63
+ [Chromium](https://www.chromium.org/getting-involved/download-chromium).
65
64
  Chrome binary should be in the `PATH` or `BROWSER_PATH` or you can pass it as an
66
65
  option to browser instance see `:browser_path` in
67
66
  [Customization](https://github.com/rubycdp/ferrum#customization).
@@ -173,15 +172,18 @@ Ferrum::Browser.new(options)
173
172
  options it passes to the browser, if you set this to `true` then only
174
173
  options you put in `:browser_options` will be passed to the browser,
175
174
  except required ones of course.
176
- * `:port` (Integer) - Remote debugging port for headless Chrome
177
- * `:host` (String) - Remote debugging address for headless Chrome
175
+ * `:port` (Integer) - Remote debugging port for headless Chrome.
176
+ * `:host` (String) - Remote debugging address for headless Chrome.
178
177
  * `:url` (String) - URL for a running instance of Chrome. If this is set, a
179
178
  browser process will not be spawned.
180
179
  * `:process_timeout` (Integer) - How long to wait for the Chrome process to
181
- respond on startup
180
+ respond on startup.
182
181
  * `:ws_max_receive_size` (Integer) - How big messages to accept from Chrome
183
182
  over the web socket, in bytes. Defaults to 64MB. Incoming messages larger
184
183
  than this will cause a `Ferrum::DeadBrowserError`.
184
+ * `:proxy` (Hash) - Specify proxy settings, [read more](https://github.com/rubycdp/ferrum#proxy)
185
+ * `:save_path` (String) - Path to save attachments with [Content-Disposition](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition) header.
186
+ * `:env` (Hash) - Environment variables you'd like to pass through to the process
185
187
 
186
188
 
187
189
  ## Navigation
@@ -407,9 +409,28 @@ browser.mhtml(path: "google.mhtml") # => 87742
407
409
  ```
408
410
 
409
411
 
412
+ ## Cleaning Up
413
+
414
+ #### reset
415
+
416
+ Closes browser tabs opened by the `Browser` instance.
417
+
418
+ ```ruby
419
+ # connect to a long-running Chrome process
420
+ browser = Ferrum::Browser.new(url: 'http://localhost:9222')
421
+
422
+ browser.go_to("https://github.com/")
423
+
424
+ # clean up, lest the tab stays there hanging forever
425
+ browser.reset
426
+
427
+ browser.quit
428
+ ```
429
+
430
+
410
431
  ## Network
411
432
 
412
- browser.network
433
+ `browser.network`
413
434
 
414
435
  #### traffic `Array<Network::Exchange>`
415
436
 
@@ -542,17 +563,83 @@ end
542
563
  browser.network.authorize(user: "login", password: "pass", type: :proxy)
543
564
 
544
565
  browser.go_to("https://google.com")
545
-
546
566
  ```
547
567
 
548
568
  You used to call `authorize` method without block, but since it's implemented using request interception there could be
549
569
  a collision with another part of your code that also uses request interception, so that authorize allows the request
550
570
  while your code denies but it's too late. The block is mandatory now.
551
571
 
572
+ #### emulate_network_conditions(\*\*options)
573
+
574
+ Activates emulation of network conditions.
575
+
576
+ * options `Hash`
577
+ * :offline `Boolean` emulate internet disconnection, `false` by default
578
+ * :latency `Integer` minimum latency from request sent to response headers received (ms), `0` by
579
+ default
580
+ * :download_throughput `Integer` maximal aggregated download throughput (bytes/sec), `-1`
581
+ by default, disables download throttling
582
+ * :upload_throughput `Integer` maximal aggregated upload throughput (bytes/sec), `-1`
583
+ by default, disables download throttling
584
+ * :connection_type `String` connection type if known, one of: none, cellular2g, cellular3g, cellular4g,
585
+ bluetooth, ethernet, wifi, wimax, other. `nil` by default
586
+
587
+ ```ruby
588
+ browser.network.emulate_network_conditions(connection_type: "cellular2g")
589
+ browser.go_to("https://github.com/")
590
+ ```
591
+
592
+ #### offline_mode
593
+
594
+ Activates offline mode for a page.
595
+
596
+ ```ruby
597
+ browser.network.offline_mode
598
+ browser.go_to("https://github.com/") # => Ferrum::StatusError (Request to https://github.com/ failed to reach server, check DNS and server status)
599
+ ```
600
+
601
+
602
+ ## Proxy
603
+
604
+ You can set a proxy with the `proxy` option.
605
+
606
+ ```ruby
607
+ browser = Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800" })
608
+ browser = Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", user: "user", pasword: "pa$$" })
609
+ ```
610
+
611
+ Chrome Devtools Protocol does not support changing proxies after the browser is launched. If you want to change proxies, you must restart your browser, which may not be convenient. There is a workaround. Ferrum provides a wrapper for a proxy server that can rotate proxies. We can run a proxy in the same process and rotate proxies inside this proxy server:
612
+
613
+ ```ruby
614
+ browser = Ferrum::Browser.new(proxy: { server: true })
615
+
616
+ browser.proxy_server.rotate(host: "x.x.x.x", port: 31337, user: "user", password: "password")
617
+ browser.create_page(new_context: true) do |page|
618
+ page.go_to("https://api.ipify.org?format=json")
619
+ page.body # => "x.x.x.x"
620
+ end
621
+
622
+ browser.proxy_server.rotate(host: "y.y.y.y", port: 31337, user: "user", password: "password")
623
+ browser.create_page(new_context: true) do |page|
624
+ page.go_to("https://api.ipify.org?format=json")
625
+ page.body # => "y.y.y.y"
626
+ end
627
+ ```
628
+
629
+ Make sure to create page in the new context, because Chrome doesn't break the connection with the proxy for `CONNECT`
630
+ requests even if you close the page.
631
+
632
+ You can specify semi-colon-separated list of hosts for which proxy shouldn't be used:
633
+
634
+ ```ruby
635
+ browser = Ferrum::Browser.new(proxy: { host: "x.x.x.x", port: "8800", bypass: "*.google.com;*foo.com" })
636
+ browser = Ferrum::Browser.new(proxy: { server: true, bypass: "*.google.com;*foo.com" })
637
+ ```
638
+
552
639
 
553
640
  ### Mouse
554
641
 
555
- browser.mouse
642
+ `browser.mouse`
556
643
 
557
644
  #### scroll_to(x, y)
558
645
 
@@ -639,13 +726,12 @@ Returns bitfield for a given keys
639
726
 
640
727
  ## Cookies
641
728
 
642
- browser.cookies
729
+ `browser.cookies`
643
730
 
644
731
  #### all : `Hash<String, Cookie>`
645
732
 
646
733
  Returns cookies hash
647
734
 
648
-
649
735
  ```ruby
650
736
  browser.cookies.all # => {"NID"=>#<Ferrum::Cookies::Cookie:0x0000558624b37a40 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>}
651
737
  ```
@@ -660,11 +746,11 @@ Returns cookie
660
746
  browser.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88 @attributes={"name"=>"NID", "value"=>"...", "domain"=>".google.com", "path"=>"/", "expires"=>1583211046.575681, "size"=>178, "httpOnly"=>true, "secure"=>false, "session"=>false}>
661
747
  ```
662
748
 
663
- #### set(\*\*options) : `Boolean`
749
+ #### set(value) : `Boolean`
664
750
 
665
- Sets given values as cookie
751
+ Sets a cookie
666
752
 
667
- * options `Hash`
753
+ * value `Hash`
668
754
  * :name `String`
669
755
  * :value `String`
670
756
  * :domain `String`
@@ -676,6 +762,13 @@ Sets given values as cookie
676
762
  browser.cookies.set(name: "stealth", value: "omg", domain: "google.com") # => true
677
763
  ```
678
764
 
765
+ * value `Cookie`
766
+
767
+ ```ruby
768
+ nid_cookie = browser.cookies["NID"] # => <Ferrum::Cookies::Cookie:0x0000558624b67a88>
769
+ browser.cookies.set(nid_cookie) # => true
770
+ ```
771
+
679
772
  #### remove(\*\*options) : `Boolean`
680
773
 
681
774
  Removes given cookie
@@ -699,7 +792,7 @@ browser.cookies.clear # => true
699
792
 
700
793
  ## Headers
701
794
 
702
- browser.headers
795
+ `browser.headers`
703
796
 
704
797
  #### get : `Hash`
705
798
 
@@ -798,9 +891,10 @@ browser.add_script_tag(url: "http://example.com/stylesheet.css") # => true
798
891
  browser.add_style_tag(content: "h1 { font-size: 40px; }") # => true
799
892
 
800
893
  ```
801
- #### bypass_csp(enabled) : `Boolean`
894
+ #### bypass_csp(\*\*options) : `Boolean`
802
895
 
803
- * enabled `Boolean`, `true` by default
896
+ * options `Hash`
897
+ * :enabled `Boolean`, `true` by default
804
898
 
805
899
  ```ruby
806
900
  browser.bypass_csp # => true
@@ -941,7 +1035,7 @@ browser.go_to("https://www.w3schools.com/tags/tag_frame.asp")
941
1035
  browser.main_frame.doctype # => "<!DOCTYPE html>"
942
1036
  ```
943
1037
 
944
- #### set_content(html)
1038
+ #### content = html
945
1039
 
946
1040
  Sets a content of a given frame.
947
1041
 
@@ -951,12 +1045,12 @@ Sets a content of a given frame.
951
1045
  browser.go_to("https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe")
952
1046
  frame = browser.frames[1]
953
1047
  frame.body # <html lang="en"><head><style>body {transition: opacity ease-in 0.2s; }...
954
- frame.set_content("<html><head></head><body><p>lol</p></body></html>")
1048
+ frame.content = "<html><head></head><body><p>lol</p></body></html>"
955
1049
  frame.body # => <html><head></head><body><p>lol</p></body></html>
956
1050
  ```
957
1051
 
958
1052
 
959
- ## Dialog
1053
+ ## Dialogs
960
1054
 
961
1055
  #### accept(text)
962
1056
 
@@ -1008,7 +1102,16 @@ browser.playback_rate # => 2000
1008
1102
 
1009
1103
  #### node? : `Boolean`
1010
1104
  #### frame_id
1011
- #### frame
1105
+ #### frame : `Frame`
1106
+
1107
+ Returns [Frame](https://github.com/rubycdp/ferrum#frame) object for current node, you can keep using
1108
+ [Finders](https://github.com/rubycdp/ferrum#Finders) for that object:
1109
+
1110
+ ```ruby
1111
+ frame = browser.at_xpath("//iframe").frame # => Frame
1112
+ frame.at_css("//a[text() = 'Log in']") # => Node
1113
+ ```
1114
+
1012
1115
  #### focus
1013
1116
  #### focusable?
1014
1117
  #### moving? : `Boolean`
@@ -1028,6 +1131,49 @@ browser.playback_rate # => 2000
1028
1131
  #### property
1029
1132
  #### attribute
1030
1133
  #### evaluate
1134
+ #### selected : `Array<Node>`
1135
+ #### select
1136
+
1137
+ (chainable) Selects options by passed attribute.
1138
+
1139
+ ```ruby
1140
+ browser.at_xpath("//*[select]").select(["1"]) # => Node (select)
1141
+ browser.at_xpath("//*[select]").select(["text"], by: :text) # => Node (select)
1142
+ ```
1143
+
1144
+ Accept string, array or strings:
1145
+ ```ruby
1146
+ browser.at_xpath("//*[select]").select("1")
1147
+ browser.at_xpath("//*[select]").select("1", "2")
1148
+ browser.at_xpath("//*[select]").select(["1", "2"])
1149
+ ```
1150
+
1151
+
1152
+ ## Tracing
1153
+
1154
+ You can use `tracing.record` to create a trace file which can be opened in Chrome DevTools or
1155
+ [timeline viewer](https://chromedevtools.github.io/timeline-viewer/).
1156
+
1157
+ ```ruby
1158
+ page.tracing.record(path: "trace.json") do
1159
+ page.go_to("https://www.google.com")
1160
+ end
1161
+ ```
1162
+
1163
+ #### tracing.record(\*\*options) : `String`
1164
+
1165
+ Accepts block, records trace and by default returns trace data from `Tracing.tracingComplete` event as output. When
1166
+ `path` is specified returns `true` and stores trace data into file.
1167
+
1168
+ * options `Hash`
1169
+ * :path `String` save data on the disk, `nil` by default
1170
+ * :encoding `Symbol` `:base64` | `:binary` encode output as Base64 or plain text. `:binary` by default
1171
+ * :timeout `Float` wait until file streaming finishes in the specified time or raise error, defaults to `nil`
1172
+ * :screenshots `Boolean` capture screenshots in the trace, `false` by default
1173
+ * :trace_config `Hash<String, Object>` config for
1174
+ [trace](https://chromedevtools.github.io/devtools-protocol/tot/Tracing/#type-TraceConfig), for categories
1175
+ see [getCategories](https://chromedevtools.github.io/devtools-protocol/tot/Tracing/#method-getCategories),
1176
+ only one trace config can be active at a time per browser.
1031
1177
 
1032
1178
 
1033
1179
  ## Thread safety ##
@@ -1091,9 +1237,12 @@ browser.quit
1091
1237
 
1092
1238
  After checking out the repo, run `bundle install` to install dependencies.
1093
1239
 
1094
- Then, run `bundle exec rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
1240
+ Then, run `bundle exec rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will
1241
+ allow you to experiment.
1095
1242
 
1096
- To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
1243
+ To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the
1244
+ version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version,
1245
+ push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
1097
1246
 
1098
1247
 
1099
1248
  ## Contributing
@@ -0,0 +1,46 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Ferrum
4
+ class Browser
5
+ module Binary
6
+ module_function
7
+
8
+ def find(commands)
9
+ enum(commands).first
10
+ end
11
+
12
+ def all(commands)
13
+ enum(commands).force
14
+ end
15
+
16
+ def enum(commands)
17
+ paths, exts = prepare_paths
18
+ cmds = Array(commands).product(paths, exts)
19
+ lazy_find(cmds)
20
+ end
21
+
22
+ def prepare_paths
23
+ exts = (ENV.key?("PATHEXT") ? ENV.fetch("PATHEXT").split(";") : []) << ""
24
+ paths = ENV["PATH"].split(File::PATH_SEPARATOR)
25
+ raise EmptyPathError if paths.empty?
26
+
27
+ [paths, exts]
28
+ end
29
+
30
+ # rubocop:disable Style/CollectionCompact
31
+ def lazy_find(cmds)
32
+ cmds.lazy.map do |cmd, path, ext|
33
+ absolute_path = File.absolute_path(cmd)
34
+ is_absolute_path = absolute_path == cmd
35
+ cmd = File.expand_path("#{cmd}#{ext}", path) unless is_absolute_path
36
+
37
+ next unless File.executable?(cmd)
38
+ next if File.directory?(cmd)
39
+
40
+ cmd
41
+ end.reject(&:nil?) # .compact isn't defined on Enumerator::Lazy
42
+ end
43
+ # rubocop:enable Style/CollectionCompact
44
+ end
45
+ end
46
+ end
@@ -1,6 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
- require "concurrent-ruby"
4
3
  require "ferrum/browser/subscriber"
5
4
  require "ferrum/browser/web_socket"
6
5
 
@@ -14,17 +13,18 @@ module Ferrum
14
13
  @command_id = id_starts_with
15
14
  @pendings = Concurrent::Hash.new
16
15
  @ws = WebSocket.new(ws_url, @browser.ws_max_receive_size, @browser.logger)
17
- @subscriber, @interruptor = Subscriber.build(2)
16
+ @subscriber, @interrupter = Subscriber.build(2)
18
17
 
19
18
  @thread = Thread.new do
20
19
  Thread.current.abort_on_exception = true
21
- if Thread.current.respond_to?(:report_on_exception=)
22
- Thread.current.report_on_exception = true
23
- end
20
+ Thread.current.report_on_exception = true if Thread.current.respond_to?(:report_on_exception=)
21
+
22
+ loop do
23
+ message = @ws.messages.pop
24
+ break unless message
24
25
 
25
- while message = @ws.messages.pop
26
26
  if INTERRUPTIONS.include?(message["method"])
27
- @interruptor.async.call(message)
27
+ @interrupter.async.call(message)
28
28
  elsif message.key?("method")
29
29
  @subscriber.async.call(message)
30
30
  else
@@ -44,6 +44,7 @@ module Ferrum
44
44
 
45
45
  raise DeadBrowserError if data.nil? && @ws.messages.closed?
46
46
  raise TimeoutError unless data
47
+
47
48
  error, response = data.values_at("error", "result")
48
49
  raise_browser_error(error) if error
49
50
  response
@@ -52,14 +53,14 @@ module Ferrum
52
53
  def on(event, &block)
53
54
  case event
54
55
  when *INTERRUPTIONS
55
- @interruptor.on(event, &block)
56
+ @interrupter.on(event, &block)
56
57
  else
57
58
  @subscriber.on(event, &block)
58
59
  end
59
60
  end
60
61
 
61
62
  def subscribed?(event)
62
- [@interruptor, @subscriber].any? { |s| s.subscribed?(event) }
63
+ [@interrupter, @subscriber].any? { |s| s.subscribed?(event) }
63
64
  end
64
65
 
65
66
  def close
@@ -84,16 +85,16 @@ module Ferrum
84
85
  # Node has disappeared while we were trying to get it
85
86
  when "No node with given id found",
86
87
  "Could not find node with given id"
87
- raise NodeNotFoundError.new(error)
88
+ raise NodeNotFoundError, error
88
89
  # Context is lost, page is reloading
89
90
  when "Cannot find context with specified id"
90
- raise NoExecutionContextError.new(error)
91
+ raise NoExecutionContextError, error
91
92
  when "No target with given id found"
92
93
  raise NoSuchPageError
93
94
  when /Could not compute content quads/
94
95
  raise CoordinatesNotFoundError
95
96
  else
96
- raise BrowserError.new(error)
97
+ raise BrowserError, error
97
98
  end
98
99
  end
99
100
  end
@@ -5,7 +5,7 @@ module Ferrum
5
5
  class Command
6
6
  NOT_FOUND = "Could not find an executable for the browser. Try to make " \
7
7
  "it available on the PATH or set environment variable for " \
8
- "example BROWSER_PATH=\"/usr/bin/chrome\"".freeze
8
+ "example BROWSER_PATH=\"/usr/bin/chrome\""
9
9
 
10
10
  # Currently only these browsers support CDP:
11
11
  # https://github.com/cyrus-and/chrome-remote-interface#implementations
@@ -27,9 +27,11 @@ module Ferrum
27
27
  def initialize(defaults, options, user_data_dir)
28
28
  @flags = {}
29
29
  @defaults = defaults
30
- @options, @user_data_dir = options, user_data_dir
31
- @path = options[:browser_path] || ENV["BROWSER_PATH"] || defaults.detect_path
32
- raise Cliver::Dependency::NotFound.new(NOT_FOUND) unless @path
30
+ @options = options
31
+ @user_data_dir = user_data_dir
32
+ @path = options[:browser_path] || ENV.fetch("BROWSER_PATH", nil) || defaults.detect_path
33
+ raise BinaryNotFoundError, NOT_FOUND unless @path
34
+
33
35
  merge_options
34
36
  end
35
37
 
@@ -45,10 +47,7 @@ module Ferrum
45
47
 
46
48
  def merge_options
47
49
  @flags = defaults.merge_required(@flags, options, @user_data_dir)
48
-
49
- unless options[:ignore_default_browser_options]
50
- @flags = defaults.merge_default(@flags, options)
51
- end
50
+ @flags = defaults.merge_default(@flags, options) unless options[:ignore_default_browser_options]
52
51
 
53
52
  @flags.merge!(options.fetch(:browser_options, {}))
54
53
  end
@@ -24,13 +24,7 @@ module Ferrum
24
24
  end
25
25
 
26
26
  def detect_path
27
- if Ferrum.mac?
28
- self.class::MAC_BIN_PATH.find { |n| File.exist?(n) }
29
- else
30
- self.class::LINUX_BIN_PATH.find do |name|
31
- path = Cliver.detect(name) and break(path)
32
- end
33
- end
27
+ Binary.find(self.class::PLATFORM_PATH[Utils::Platform.name])
34
28
  end
35
29
 
36
30
  def merge_required(flags, options, user_data_dir)
@@ -36,19 +36,27 @@ module Ferrum
36
36
  "metrics-recording-only" => nil,
37
37
  "safebrowsing-disable-auto-update" => nil,
38
38
  "password-store" => "basic",
39
- "no-startup-window" => nil,
40
- # Note: --no-sandbox is not needed if you properly setup a user in the container.
39
+ "no-startup-window" => nil
40
+ # NOTE: --no-sandbox is not needed if you properly setup a user in the container.
41
41
  # https://github.com/ebidel/lighthouse-ci/blob/master/builder/Dockerfile#L35-L40
42
42
  # "no-sandbox" => nil,
43
43
  }.freeze
44
44
 
45
45
  MAC_BIN_PATH = [
46
- "/Applications/Chromium.app/Contents/MacOS/Chromium",
47
- "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome"
46
+ "/Applications/Google Chrome.app/Contents/MacOS/Google Chrome",
47
+ "/Applications/Chromium.app/Contents/MacOS/Chromium"
48
48
  ].freeze
49
- LINUX_BIN_PATH = %w[chromium google-chrome-unstable google-chrome-beta
50
- google-chrome chrome chromium-browser
51
- google-chrome-stable].freeze
49
+ LINUX_BIN_PATH = %w[chrome google-chrome google-chrome-stable google-chrome-beta
50
+ chromium chromium-browser google-chrome-unstable].freeze
51
+ WINDOWS_BIN_PATH = [
52
+ "C:/Program Files/Google/Chrome/Application/chrome.exe",
53
+ "C:/Program Files/Google/Chrome Dev/Application/chrome.exe"
54
+ ].freeze
55
+ PLATFORM_PATH = {
56
+ mac: MAC_BIN_PATH,
57
+ windows: WINDOWS_BIN_PATH,
58
+ linux: LINUX_BIN_PATH
59
+ }.freeze
52
60
 
53
61
  def merge_required(flags, options, user_data_dir)
54
62
  port = options.fetch(:port, BROWSER_PORT)
@@ -56,14 +64,12 @@ module Ferrum
56
64
  flags.merge("remote-debugging-port" => port,
57
65
  "remote-debugging-address" => host,
58
66
  # Doesn't work on MacOS, so we need to set it by CDP
59
- "window-size" => options[:window_size].join(","),
67
+ "window-size" => options[:window_size]&.join(","),
60
68
  "user-data-dir" => user_data_dir)
61
69
  end
62
70
 
63
71
  def merge_default(flags, options)
64
- unless options.fetch(:headless, true)
65
- defaults = except("headless", "disable-gpu")
66
- end
72
+ defaults = except("headless", "disable-gpu") unless options.fetch(:headless, true)
67
73
 
68
74
  defaults ||= DEFAULT_OPTIONS
69
75
  defaults.merge(flags)
@@ -5,13 +5,22 @@ module Ferrum
5
5
  module Options
6
6
  class Firefox < Base
7
7
  DEFAULT_OPTIONS = {
8
- "headless" => nil,
8
+ "headless" => nil
9
9
  }.freeze
10
10
 
11
11
  MAC_BIN_PATH = [
12
12
  "/Applications/Firefox.app/Contents/MacOS/firefox-bin"
13
13
  ].freeze
14
14
  LINUX_BIN_PATH = %w[firefox].freeze
15
+ WINDOWS_BIN_PATH = [
16
+ "C:/Program Files/Firefox Developer Edition/firefox.exe",
17
+ "C:/Program Files/Mozilla Firefox/firefox.exe"
18
+ ].freeze
19
+ PLATFORM_PATH = {
20
+ mac: MAC_BIN_PATH,
21
+ windows: WINDOWS_BIN_PATH,
22
+ linux: LINUX_BIN_PATH
23
+ }.freeze
15
24
 
16
25
  def merge_required(flags, options, user_data_dir)
17
26
  port = options.fetch(:port, BROWSER_PORT)
@@ -21,9 +30,7 @@ module Ferrum
21
30
  end
22
31
 
23
32
  def merge_default(flags, options)
24
- unless options.fetch(:headless, true)
25
- defaults = except("headless")
26
- end
33
+ defaults = except("headless") unless options.fetch(:headless, true)
27
34
 
28
35
  defaults ||= DEFAULT_OPTIONS
29
36
  defaults.merge(flags)