selenium_tor 0.1.1 → 0.1.3
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.yamllint +7 -0
- data/CHANGELOG.md +21 -0
- data/README.md +22 -11
- data/lib/firefox_driver_lock_file_patch.rb +18 -0
- data/lib/tor/driver.rb +3 -6
- data/lib/tor/driver_not_yet_connected_to_tor_network.rb +1 -0
- data/lib/tor/options.rb +12 -9
- data/lib/tor/profile.rb +16 -0
- data/lib/tor/tor_prefs.rb +25 -0
- data/lib/tor/version.rb +1 -1
- data/selenium_tor.gemspec +3 -1
- metadata +22 -6
- data/lib/mixin.rb +0 -13
- data/lib/tor/prefs.rb +0 -17
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 387e1f6800b7d9b1f7eb269a0e6d53f699123b59f3daa1b4ae66536037fdc3ff
|
4
|
+
data.tar.gz: a59fa065eef2328abb0832ead2ada221172b5d4d55c34fe0285c02f6075e8d93
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6a5d2aa8ecf3527edadcec9aab17480dfdd0055525322b7c97f0c8f4e6588e392d0e5b10d4eca05c5a1766e22f87b6793109da8965a45ac6bc26216888be50de
|
7
|
+
data.tar.gz: b145dab8c18821750a8be414af2cf44a8046a0393e69907c52dc14cc8dd4f376e626d10d77fe473655604d27810d93ea5539f5a316f78659a6a8abaaf6780f8c
|
data/.yamllint
ADDED
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,26 @@
|
|
1
1
|
## master (unreleased)
|
2
2
|
|
3
|
+
## [0.1.3] - 2023-12-08
|
4
|
+
|
5
|
+
### Bug fixes
|
6
|
+
|
7
|
+
* added selenium-webdriver as runtime dependency
|
8
|
+
* [#2](https://gitlab.com/matzfan/selenium_tor/-/issues/2) fixed, fingerprintjs VisitorId now same as TB's
|
9
|
+
|
10
|
+
### New features
|
11
|
+
|
12
|
+
* Fix circular require issue
|
13
|
+
* Expand fingerprint tests
|
14
|
+
|
15
|
+
## [0.1.2] - 2023-11-20
|
16
|
+
|
17
|
+
* Breaking change: change constant names in Tor namespace - see README
|
18
|
+
|
19
|
+
### New features
|
20
|
+
|
21
|
+
* Added Tor::Profile and lock file patch for Selenium < 4.16 see [Selenium #11576](https://github.com/SeleniumHQ/selenium/issues/11576)
|
22
|
+
* Add tests for NoScript extension
|
23
|
+
|
3
24
|
## [0.1.1] - 2023-11-19
|
4
25
|
|
5
26
|
* Rename gem from selenium-tor, yanked previous versions
|
data/README.md
CHANGED
@@ -8,6 +8,12 @@ A Selenium extension for Tor Browser.
|
|
8
8
|
driver = Selenium::WebDriver.for :tor
|
9
9
|
```
|
10
10
|
|
11
|
+
# \_why?
|
12
|
+
|
13
|
+
I can use Firefox with Selenium and set a SOCKS proxy to use the Tor network, so why the need?
|
14
|
+
|
15
|
+
The above approach will hide your IP, but there is a good chance your browser's unique or near-unique fingerprint may be logged by site owners. Subsequent visits could identify you. A primary aim of this project is to enable Selenium to leverage Tor Browser's unique anonymity characteristics - in particular its resitance to browser fingerprinting. The aim is to ensure Selenium Tor site visits leave an identical fingerprint to the thousands of regular Tor Browser users.
|
16
|
+
|
11
17
|
## Installation
|
12
18
|
|
13
19
|
Install the gem and add to the application's Gemfile by executing:
|
@@ -29,42 +35,47 @@ options.add_argument '--headless' # whatever you need
|
|
29
35
|
driver = Selenium::WebDriver.for :tor, options: options
|
30
36
|
|
31
37
|
driver.get 'https://check.torproject.org'
|
32
|
-
puts driver.title
|
33
|
-
|
38
|
+
puts driver.title # => Congratulations. This browser is configured to use Tor.
|
39
|
+
driver.quit
|
34
40
|
```
|
35
41
|
The driver will not be instantiated until a connection to the Tor network is made. If the network is inaccessible for any reason a `TorNetworkError` will result.
|
36
42
|
|
37
|
-
The `Selenium::WebDriver::Tor` namespace is used
|
43
|
+
The `Selenium::WebDriver::Tor` namespace is used for `Driver`, `Options` and `Profile`. Otherwise Selenium's `Selenium::WebDriver::Firefox` namespace is used.
|
38
44
|
|
39
45
|
Remote functionality is not tested, but will be if Selenium provide a Tor Browser Docker container for Selenium Grid.
|
40
46
|
|
41
|
-
A number of constants are set during driver initialization based upon values found in the Tor Browser Bundle (TBB) root directory
|
47
|
+
A number of constants are set during driver initialization based upon values found in the Tor Browser Bundle (TBB) root directory - see below. These include:
|
42
48
|
```ruby
|
43
49
|
Tor::TBB_DIR # path to the TBB root directory
|
44
50
|
Tor::TBB_BROWSER_DIR # path to the 'Browser' directory in the above
|
45
|
-
Tor::
|
46
|
-
Tor::
|
51
|
+
Tor::TBB_BINARY_PATH # path to the binary
|
52
|
+
Tor::TBB_PROFILE_DIR # path to the default profile directory
|
53
|
+
Tor::TBB_EXTENSIONS_DIR # path to the 'extensions' directory in the above
|
54
|
+
Tor::TBB_VERSION # the version installed, e.g. "13.0.1", note: driver.capabilities.browser_version returns the Firefox version Tor Browser is based on
|
47
55
|
```
|
48
56
|
|
49
57
|
## Dependencies and configuration
|
50
58
|
|
51
|
-
As with Firefox browser,
|
59
|
+
As with Firefox browser, `geckodriver` needs to be installed and in your PATH.
|
52
60
|
|
53
|
-
The gem needs to know the location of the Tor Browser Bundle (TBB). This can be installed from the Tor Project [downloads page](https://www.torproject.org/download/). The download package archive must be extracted and the root TBB directory (
|
61
|
+
The gem needs to know the location of the Tor Browser Bundle (TBB). This can be installed from the Tor Project [downloads page](https://www.torproject.org/download/). The download package archive must be extracted and the root TBB directory (named "tor-browser") placed somewhere on your system. By default it is assumed to be in the current user's HOME directory. An alternative location can be set via the env var `TOR_BROWSER_ROOT_DIR` - e.g. `export TOR_BROWSER_ROOT_DIR=/home/<user>/Downloads`. The Tor Browser binary location is *automatically set* by reference to this directory, so there is no need to do this:
|
54
62
|
```ruby
|
55
63
|
options.binary = '/some/path/to/tor_firefox_binary' # UNNECESSARY
|
56
64
|
```
|
65
|
+
|
66
|
+
The location of the TBB is not expected to change during execution.
|
67
|
+
|
57
68
|
Tor Selenium is tested on **Linux only** right now.
|
58
69
|
|
59
70
|
## Testing
|
60
71
|
|
61
|
-
After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake
|
72
|
+
After checking out the repo, run `bin/setup` to install dependencies. Then, run `bundle exec rake` to run the tests.
|
62
73
|
|
63
74
|
Tests are run in headless mode by default. If you want to see what the browser is doing you can do this:
|
64
75
|
|
65
|
-
$ export RUN_HEADED_TESTS=1 && rake
|
76
|
+
$ export RUN_HEADED_TESTS=1 && bundle exec rake
|
66
77
|
|
67
|
-
If you find tests are failing with `TorNetworkError` and a timeout message, check you have no other Tor Browser processes running. The process to look for is called
|
78
|
+
If you find tests are failing with `TorNetworkError` and a timeout message, check you have no other Tor Browser processes running. The process to look for is called "firefox.real". You may also want to check the Tor network is actually up, it isn't always..
|
68
79
|
|
69
80
|
## Development
|
70
81
|
|
@@ -0,0 +1,18 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Selenium
|
4
|
+
module WebDriver
|
5
|
+
module Firefox
|
6
|
+
# patched class
|
7
|
+
class Profile
|
8
|
+
alias old_delete_lock_files delete_lock_files
|
9
|
+
|
10
|
+
def delete_lock_files(directory)
|
11
|
+
%w[.parentlock parent.lock lock].each do |name| # 'lock' file missing from Selenium < 4.16
|
12
|
+
FileUtils.rm_f File.join(directory, name)
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
data/lib/tor/driver.rb
CHANGED
@@ -2,15 +2,12 @@
|
|
2
2
|
|
3
3
|
require 'open-uri'
|
4
4
|
require_relative 'driver_not_yet_connected_to_tor_network'
|
5
|
-
require_relative '../mixin'
|
6
5
|
|
7
6
|
module Selenium
|
8
7
|
module WebDriver
|
9
8
|
module Tor
|
10
9
|
# tor driver
|
11
|
-
class Driver < DelegateClass(
|
12
|
-
include Mixin
|
13
|
-
|
10
|
+
class Driver < DelegateClass(DriverNotYetConnectedToTorNetwork)
|
14
11
|
class TorNetworkError < StandardError; end
|
15
12
|
|
16
13
|
CONNECTION_STATUS_ID = 'torPreferences-status-tor-connect'
|
@@ -19,7 +16,7 @@ module Selenium
|
|
19
16
|
|
20
17
|
def initialize(...)
|
21
18
|
@instance = DriverNotYetConnectedToTorNetwork.new(...)
|
22
|
-
super
|
19
|
+
super(@instance)
|
23
20
|
wait_for_tor_connection
|
24
21
|
install_addons
|
25
22
|
end
|
@@ -28,7 +25,7 @@ module Selenium
|
|
28
25
|
|
29
26
|
def wait_for_tor_connection
|
30
27
|
@instance.get 'about:preferences#connection'
|
31
|
-
|
28
|
+
Wait.new(timeout: 10).until { @instance.find_element(:id, CONNECTION_STATUS_ID).text == CONNECTED }
|
32
29
|
rescue Selenium::WebDriver::Error::TimeoutError => e
|
33
30
|
@instance.quit # abort initialization
|
34
31
|
raise TorNetworkError, "Cannot connect to Tor network: #{e.message}"
|
data/lib/tor/options.rb
CHANGED
@@ -1,6 +1,6 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
require_relative '
|
3
|
+
require_relative 'tor_prefs'
|
4
4
|
require 'selenium-webdriver'
|
5
5
|
|
6
6
|
module Selenium
|
@@ -11,27 +11,30 @@ module Selenium
|
|
11
11
|
DEFAULT_TBB_DIR = File.join Dir.home, 'tor-browser'
|
12
12
|
Tor::TBB_DIR = ENV.fetch('TOR_BROWSER_ROOT_DIR', nil) || DEFAULT_TBB_DIR
|
13
13
|
Tor::TBB_BROWSER_DIR = File.join Tor::TBB_DIR, 'Browser'
|
14
|
-
Tor::
|
15
|
-
Tor::
|
14
|
+
Tor::TBB_BINARY_PATH = File.join Tor::TBB_BROWSER_DIR, 'firefox'
|
15
|
+
Tor::TBB_PROFILE_DIR = File.join Tor::TBB_BROWSER_DIR, *%w[TorBrowser Data Browser profile.default]
|
16
|
+
Tor::TBB_EXTENSIONS_DIR = File.join Tor::TBB_PROFILE_DIR, 'extensions'
|
17
|
+
Tor::TBB_VERSION = JSON.parse(File.read(File.join(Tor::TBB_BROWSER_DIR, 'tbb_version.json')))['version']
|
16
18
|
|
17
19
|
def initialize(log_level: nil, **opts)
|
18
|
-
super(log_level: log_level, **tor_options.merge(opts))
|
20
|
+
super(log_level: log_level, **tor_options(opts.delete(:prefs)).merge(opts))
|
19
21
|
do_start_tor_browser_script_stuff # stuff the start-tor-browser script in TBB does
|
20
22
|
end
|
21
23
|
|
22
24
|
private
|
23
25
|
|
24
|
-
def tor_options
|
25
|
-
{ binary:
|
26
|
+
def tor_options(prefs)
|
27
|
+
{ binary: TBB_BINARY_PATH, prefs: (prefs || {}).merge(TOR_PREFS) }
|
26
28
|
end
|
27
29
|
|
28
30
|
def do_start_tor_browser_script_stuff
|
31
|
+
ENV['SESSION_MANAGER'] = nil
|
29
32
|
ENV['XAUTHORITY'] = File.join(Dir.home, '.Xauthority') unless ENV.fetch('XAUTHORITY', nil)
|
30
|
-
ENV['
|
31
|
-
ENV['FONTCONFIG_PATH'] = "#{TBB_BROWSER_DIR}/fontconfig" # these two stop font leaks [#1]
|
33
|
+
ENV['FONTCONFIG_PATH'] = File.join Tor::TBB_BROWSER_DIR, 'fontconfig' # supposed to stop font leaks..
|
32
34
|
ENV['FONTCONFIG_FILE'] = 'fonts.conf'
|
35
|
+
FileUtils.rm_rf File.join(Tor::TBB_BROWSER_DIR, *%w[TorBrowser Data fontconfig])
|
33
36
|
ENV['GSETTINGS_BACKEND'] = 'memory'
|
34
|
-
FileUtils.
|
37
|
+
FileUtils.cd Tor::TBB_BROWSER_DIR # crucial - fixes [#2](https://gitlab.com/matzfan/selenium-tor/-/issues/2)
|
35
38
|
end
|
36
39
|
end
|
37
40
|
end
|
data/lib/tor/profile.rb
ADDED
@@ -0,0 +1,16 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
require 'selenium-webdriver'
|
4
|
+
require_relative 'options'
|
5
|
+
require_relative '../firefox_driver_lock_file_patch' unless
|
6
|
+
Gem::Version.new(Selenium::WebDriver::VERSION) > Gem::Version.new('4.15')
|
7
|
+
|
8
|
+
module Selenium
|
9
|
+
module WebDriver
|
10
|
+
module Tor
|
11
|
+
# subclass
|
12
|
+
class Profile < Firefox::Profile
|
13
|
+
end
|
14
|
+
end
|
15
|
+
end
|
16
|
+
end
|
@@ -0,0 +1,25 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
module Selenium
|
4
|
+
module WebDriver
|
5
|
+
module Tor
|
6
|
+
FIRST_CONNECTION_PREFS = {
|
7
|
+
'extensions.torlauncher.prompt_at_startup' => false,
|
8
|
+
'privacy.purge_trackers.date_in_cookie_database' => '0',
|
9
|
+
'torbrowser.settings.bridges.builtin_type' => '',
|
10
|
+
'torbrowser.settings.bridges.enabled' => false,
|
11
|
+
'torbrowser.settings.bridges.source' => -1,
|
12
|
+
'torbrowser.settings.enabled' => true,
|
13
|
+
'torbrowser.settings.firewall.enabled' => false,
|
14
|
+
'torbrowser.settings.proxy.enabled' => false,
|
15
|
+
'torbrowser.settings.quickstart.enabled' => true
|
16
|
+
}.freeze
|
17
|
+
|
18
|
+
OTHER_PREFS = {
|
19
|
+
'intl.language_notification.shown' => true # affects font fingerprint (viewport size)
|
20
|
+
}.freeze
|
21
|
+
|
22
|
+
TOR_PREFS = FIRST_CONNECTION_PREFS.merge OTHER_PREFS
|
23
|
+
end
|
24
|
+
end
|
25
|
+
end
|
data/lib/tor/version.rb
CHANGED
data/selenium_tor.gemspec
CHANGED
@@ -9,7 +9,7 @@ Gem::Specification.new do |spec|
|
|
9
9
|
spec.email = ['matzfan@mailinator.com']
|
10
10
|
|
11
11
|
spec.summary = 'Selenium extension for Tor Browser'
|
12
|
-
spec.description = '
|
12
|
+
spec.description = 'An extension for Selenium::WebDriver that automates Tor Browser'
|
13
13
|
spec.homepage = 'https://gitlab.com/matzfan/selenium_tor'
|
14
14
|
spec.license = 'MIT'
|
15
15
|
spec.required_ruby_version = '>= 2.7.0'
|
@@ -32,4 +32,6 @@ Gem::Specification.new do |spec|
|
|
32
32
|
spec.bindir = 'exe'
|
33
33
|
spec.executables = spec.files.grep(%r{\Aexe/}) { |f| File.basename(f) }
|
34
34
|
spec.require_paths = ['lib']
|
35
|
+
|
36
|
+
spec.add_runtime_dependency 'selenium-webdriver', '4.16.0'
|
35
37
|
end
|
metadata
CHANGED
@@ -1,16 +1,30 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: selenium_tor
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.3
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- MatzFan
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2023-
|
12
|
-
dependencies:
|
13
|
-
|
11
|
+
date: 2023-12-08 00:00:00.000000000 Z
|
12
|
+
dependencies:
|
13
|
+
- !ruby/object:Gem::Dependency
|
14
|
+
name: selenium-webdriver
|
15
|
+
requirement: !ruby/object:Gem::Requirement
|
16
|
+
requirements:
|
17
|
+
- - '='
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: 4.16.0
|
20
|
+
type: :runtime
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - '='
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: 4.16.0
|
27
|
+
description: An extension for Selenium::WebDriver that automates Tor Browser
|
14
28
|
email:
|
15
29
|
- matzfan@mailinator.com
|
16
30
|
executables: []
|
@@ -19,6 +33,7 @@ extra_rdoc_files: []
|
|
19
33
|
files:
|
20
34
|
- ".rubocop.yml"
|
21
35
|
- ".ruby-version"
|
36
|
+
- ".yamllint"
|
22
37
|
- CHANGELOG.md
|
23
38
|
- CODE_OF_CONDUCT.md
|
24
39
|
- Guardfile
|
@@ -26,14 +41,15 @@ files:
|
|
26
41
|
- README.md
|
27
42
|
- Rakefile
|
28
43
|
- lib/driver.rb
|
29
|
-
- lib/
|
44
|
+
- lib/firefox_driver_lock_file_patch.rb
|
30
45
|
- lib/options.rb
|
31
46
|
- lib/selenium_tor.rb
|
32
47
|
- lib/service.rb
|
33
48
|
- lib/tor/driver.rb
|
34
49
|
- lib/tor/driver_not_yet_connected_to_tor_network.rb
|
35
50
|
- lib/tor/options.rb
|
36
|
-
- lib/tor/
|
51
|
+
- lib/tor/profile.rb
|
52
|
+
- lib/tor/tor_prefs.rb
|
37
53
|
- lib/tor/version.rb
|
38
54
|
- selenium_tor.gemspec
|
39
55
|
- sig/tor.rbs
|
data/lib/mixin.rb
DELETED
data/lib/tor/prefs.rb
DELETED
@@ -1,17 +0,0 @@
|
|
1
|
-
# frozen_string_literal: true
|
2
|
-
|
3
|
-
module Selenium
|
4
|
-
module WebDriver
|
5
|
-
module Tor
|
6
|
-
PREFS = { 'extensions.torlauncher.prompt_at_startup' => false,
|
7
|
-
'intl.language_notification.shown' => true,
|
8
|
-
'torbrowser.settings.bridges.builtin_type' => '',
|
9
|
-
'torbrowser.settings.bridges.enabled' => false,
|
10
|
-
'torbrowser.settings.bridges.source' => -1,
|
11
|
-
'torbrowser.settings.enabled' => true,
|
12
|
-
'torbrowser.settings.firewall.enabled' => false,
|
13
|
-
'torbrowser.settings.proxy.enabled' => false,
|
14
|
-
'torbrowser.settings.quickstart.enabled' => true }.freeze
|
15
|
-
end
|
16
|
-
end
|
17
|
-
end
|