chromate-rb 0.0.1.pre → 0.0.3.pre
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.rubocop.yml +2 -0
- data/CHANGELOG.md +72 -3
- data/README.md +33 -6
- data/Rakefile +48 -16
- data/docker_root/Gemfile +4 -0
- data/docker_root/Gemfile.lock +28 -0
- data/docker_root/TestInDocker.gif +0 -0
- data/docker_root/app.rb +87 -0
- data/dockerfiles/Dockerfile +21 -7
- data/dockerfiles/README.md +49 -0
- data/docs/BOT_BROWSER.md +74 -0
- data/docs/README.md +74 -0
- data/docs/browser.md +124 -102
- data/docs/client.md +126 -0
- data/docs/element.md +365 -0
- data/docs/elements/checkbox.md +69 -0
- data/docs/elements/radio.md +57 -0
- data/lib/bot_browser/downloader.rb +64 -0
- data/lib/bot_browser/installer.rb +99 -0
- data/lib/bot_browser.rb +43 -0
- data/lib/chromate/actions/dom.rb +35 -27
- data/lib/chromate/actions/navigate.rb +7 -5
- data/lib/chromate/actions/screenshot.rb +71 -14
- data/lib/chromate/actions/stealth.rb +62 -0
- data/lib/chromate/binary.rb +83 -0
- data/lib/chromate/browser.rb +120 -24
- data/lib/chromate/c_logger.rb +8 -0
- data/lib/chromate/client.rb +65 -26
- data/lib/chromate/configuration.rb +31 -14
- data/lib/chromate/element.rb +119 -16
- data/lib/chromate/elements/checkbox.rb +40 -0
- data/lib/chromate/elements/option.rb +43 -0
- data/lib/chromate/elements/radio.rb +37 -0
- data/lib/chromate/elements/select.rb +50 -6
- data/lib/chromate/elements/tags.rb +29 -0
- data/lib/chromate/exceptions.rb +2 -0
- data/lib/chromate/files/agents.json +11 -0
- data/lib/chromate/files/stealth.js +199 -0
- data/lib/chromate/hardwares/keyboard_controller.rb +45 -0
- data/lib/chromate/hardwares/keyboards/virtual_controller.rb +65 -0
- data/lib/chromate/hardwares/mouse_controller.rb +55 -11
- data/lib/chromate/hardwares/mouses/linux_controller.rb +124 -21
- data/lib/chromate/hardwares/mouses/mac_os_controller.rb +6 -6
- data/lib/chromate/hardwares/mouses/virtual_controller.rb +95 -7
- data/lib/chromate/hardwares/mouses/x11.rb +36 -0
- data/lib/chromate/hardwares.rb +19 -3
- data/lib/chromate/helpers.rb +22 -15
- data/lib/chromate/user_agent.rb +41 -15
- data/lib/chromate/version.rb +1 -1
- data/lib/chromate.rb +2 -0
- data/logo.png +0 -0
- data/results/bot.png +0 -0
- data/results/brotector.png +0 -0
- data/results/cloudflare.png +0 -0
- data/results/headers.png +0 -0
- data/results/pixelscan.png +0 -0
- metadata +45 -2
data/docs/browser.md
CHANGED
@@ -1,163 +1,185 @@
|
|
1
|
-
|
1
|
+
## `Chromate::Browser` Class
|
2
2
|
|
3
|
-
The `Chromate::Browser` class is
|
3
|
+
The `Chromate::Browser` class is responsible for controlling a browser instance using the Chrome DevTools Protocol (CDP). It provides methods for navigation, screenshots, and DOM interactions, as well as handling browser lifecycle (start and stop).
|
4
4
|
|
5
|
-
|
6
|
-
|
7
|
-
To create a new instance of the `Browser` class, you can pass in various options:
|
5
|
+
### Initialization
|
8
6
|
|
9
7
|
```ruby
|
10
|
-
browser = Chromate::Browser.new(
|
11
|
-
headless: true,
|
12
|
-
native_control: true,
|
13
|
-
user_data_dir: '/path/to/user/data',
|
14
|
-
record: false
|
15
|
-
)
|
8
|
+
browser = Chromate::Browser.new(options = {})
|
16
9
|
```
|
17
10
|
|
18
|
-
|
11
|
+
- **Parameters:**
|
12
|
+
- `options` (Hash, optional): Configuration options for the browser instance.
|
13
|
+
- `:chrome_path` (String): Path to the Chrome executable.
|
14
|
+
- `:user_data_dir` (String): Directory for storing user data (default: a temporary directory).
|
15
|
+
- `:headless` (Boolean): Run the browser in headless mode.
|
16
|
+
- `:xfvb` (Boolean): Use Xvfb for headless mode on Linux.
|
17
|
+
- `:native_control` (Boolean): Enable native control for enhanced undetection.
|
18
|
+
- `:record` (Boolean): Enable video recording of the browser session.
|
19
19
|
|
20
|
-
|
21
|
-
- `native_control`: Use native mouse control (default: false).
|
22
|
-
- `user_data_dir`: Directory to store user data (default: a temporary directory).
|
23
|
-
- `record`: Record the browser session (default: false).
|
24
|
-
- `xfvb`: Use xvfb for headless mode (default: false).
|
20
|
+
### Public Methods
|
25
21
|
|
26
|
-
|
22
|
+
#### `#start`
|
27
23
|
|
28
|
-
|
24
|
+
Starts the browser process and initializes the CDP client.
|
29
25
|
|
30
|
-
|
26
|
+
- **Example:**
|
27
|
+
```ruby
|
28
|
+
browser.start
|
29
|
+
```
|
31
30
|
|
31
|
+
#### `#stop`
|
32
32
|
|
33
|
+
Stops the browser process, including any associated Xvfb or video recording processes.
|
33
34
|
|
34
|
-
|
35
|
+
- **Example:**
|
36
|
+
```ruby
|
37
|
+
browser.stop
|
38
|
+
```
|
35
39
|
|
36
|
-
|
37
|
-
browser.start
|
38
|
-
```
|
40
|
+
#### `#native_control?`
|
39
41
|
|
40
|
-
|
42
|
+
Checks if native control is enabled for the browser instance.
|
41
43
|
|
42
|
-
|
44
|
+
- **Returns:**
|
45
|
+
- `Boolean`: `true` if native control is enabled, `false` otherwise.
|
43
46
|
|
47
|
+
- **Example:**
|
48
|
+
```ruby
|
49
|
+
puts "Native control enabled" if browser.native_control?
|
50
|
+
```
|
44
51
|
|
52
|
+
### Navigation Methods (from `Actions::Navigate`)
|
45
53
|
|
46
|
-
|
54
|
+
#### `#navigate_to(url)`
|
47
55
|
|
48
|
-
|
49
|
-
browser.stop
|
50
|
-
```
|
56
|
+
Navigates the browser to the specified URL.
|
51
57
|
|
52
|
-
|
58
|
+
- **Parameters:**
|
59
|
+
- `url` (String): The URL to navigate to.
|
53
60
|
|
54
|
-
|
61
|
+
- **Example:**
|
62
|
+
```ruby
|
63
|
+
browser.navigate_to('https://example.com')
|
64
|
+
```
|
55
65
|
|
56
|
-
|
57
|
-
browser.navigate_to('http://example.com')
|
58
|
-
```
|
66
|
+
#### `#wait_for_page_load`
|
59
67
|
|
60
|
-
|
68
|
+
Waits until the page has fully loaded, including the `DOMContentLoaded` event, `load` event, and `frameStoppedLoading` event.
|
61
69
|
|
62
|
-
|
70
|
+
- **Example:**
|
71
|
+
```ruby
|
72
|
+
browser.wait_for_page_load
|
73
|
+
```
|
63
74
|
|
64
|
-
|
65
|
-
element = browser.find_element('#some-element')
|
66
|
-
```
|
67
|
-
|
68
|
-
### `click_element(selector)`
|
69
|
-
|
70
|
-
Clicks on an element specified by the CSS selector.
|
75
|
+
#### `#refresh`
|
71
76
|
|
72
|
-
|
73
|
-
browser.click_element('#some-element')
|
74
|
-
```
|
77
|
+
Reloads the current page.
|
75
78
|
|
76
|
-
|
79
|
+
- **Example:**
|
80
|
+
```ruby
|
81
|
+
browser.refresh
|
82
|
+
```
|
77
83
|
|
78
|
-
|
84
|
+
#### `#go_back`
|
79
85
|
|
80
|
-
|
81
|
-
browser.hover_element('#some-element')
|
82
|
-
```
|
86
|
+
Navigates back to the previous page in the browser history.
|
83
87
|
|
84
|
-
|
88
|
+
- **Example:**
|
89
|
+
```ruby
|
90
|
+
browser.go_back
|
91
|
+
```
|
85
92
|
|
86
|
-
|
93
|
+
### Screenshot Methods (from `Actions::Screenshot`)
|
87
94
|
|
88
|
-
|
89
|
-
browser.type_text('#input-field', 'Hello, world!')
|
90
|
-
```
|
95
|
+
#### `#screenshot(file_path, options = {})`
|
91
96
|
|
92
|
-
|
97
|
+
Takes a screenshot of the current page and saves it to the specified file.
|
93
98
|
|
94
|
-
|
99
|
+
- **Parameters:**
|
100
|
+
- `file_path` (String, optional): The file path to save the screenshot.
|
101
|
+
- `options` (Hash, optional): Additional options for the screenshot.
|
102
|
+
- - `full_page` (Boolean, optional): Take a full page screenshot
|
95
103
|
|
96
|
-
|
97
|
-
browser.screenshot_to_file('screenshot.png')
|
98
|
-
```
|
104
|
+
It will call `#xvfb_screenshot` private method if `xvfb` mode is `true`
|
99
105
|
|
100
|
-
|
106
|
+
- **Example:**
|
107
|
+
```ruby
|
108
|
+
browser.screenshot('screenshot.png')
|
109
|
+
```
|
101
110
|
|
102
|
-
|
111
|
+
### DOM Methods (from `Actions::Dom`)
|
103
112
|
|
113
|
+
#### `#find_element(selector)`
|
104
114
|
|
115
|
+
Finds a single element on the page using the specified CSS selector. Returns a specialized element class based on the element type:
|
105
116
|
|
106
|
-
|
117
|
+
- **Parameters:**
|
118
|
+
- `selector` (String): The CSS selector to locate the element.
|
107
119
|
|
108
|
-
|
109
|
-
|
110
|
-
|
120
|
+
- **Returns:**
|
121
|
+
- `Chromate::Elements::Select`: For `<select>` elements
|
122
|
+
- `Chromate::Elements::Option`: For `<option>` elements
|
123
|
+
- `Chromate::Elements::Radio`: For radio button inputs (`<input type="radio">`)
|
124
|
+
- `Chromate::Elements::Checkbox`: For checkbox inputs (`<input type="checkbox">`)
|
125
|
+
- `Chromate::Element`: For all other element types
|
111
126
|
|
112
|
-
|
127
|
+
Each specialized element type provides specific methods for interacting with that type of element. For example:
|
113
128
|
|
114
129
|
```ruby
|
115
|
-
|
116
|
-
|
117
|
-
|
118
|
-
browser.start
|
130
|
+
# Working with radio buttons
|
131
|
+
radio = browser.find_element('input[type="radio"]')
|
132
|
+
radio.check if !radio.checked?
|
119
133
|
|
120
|
-
|
121
|
-
browser.find_element('
|
122
|
-
|
134
|
+
# Working with checkboxes
|
135
|
+
checkbox = browser.find_element('input[type="checkbox"]')
|
136
|
+
checkbox.toggle
|
123
137
|
|
124
|
-
|
138
|
+
# Working with select elements
|
139
|
+
select = browser.find_element('select#country')
|
140
|
+
select.select_option('France')
|
125
141
|
```
|
126
142
|
|
127
|
-
|
128
|
-
|
129
|
-
###
|
130
|
-
|
131
|
-
start_video_recording
|
132
|
-
|
133
|
-
|
134
|
-
|
135
|
-
Starts recording the browser session using `ffmpeg`.
|
143
|
+
See the [Element documentation](element.md) for more details about specialized elements.
|
136
144
|
|
137
|
-
|
145
|
+
#### `#evaluate_script(script)`
|
138
146
|
|
139
|
-
|
147
|
+
Executes the specified JavaScript expression on the page.
|
140
148
|
|
149
|
+
- **Parameters:**
|
150
|
+
- `script` (String): The JavaScript code to evaluate.
|
141
151
|
|
152
|
+
- **Returns:**
|
153
|
+
- The result of the JavaScript evaluation.
|
142
154
|
|
143
|
-
|
155
|
+
- **Example:**
|
156
|
+
```ruby
|
157
|
+
result = browser.evaluate_script('document.title')
|
158
|
+
puts "Page title: #{result}"
|
159
|
+
```
|
144
160
|
|
145
|
-
###
|
161
|
+
### Exception Handling
|
146
162
|
|
147
|
-
|
148
|
-
|
149
|
-
|
150
|
-
|
151
|
-
Stops the browser and exits the process.
|
152
|
-
|
153
|
-
###
|
154
|
-
|
155
|
-
config
|
163
|
+
- The browser handles `INT` and `TERM` signals gracefully by stopping the browser process and exiting safely.
|
164
|
+
- The `stop_and_exit` method is used to ensure proper shutdown.
|
156
165
|
|
166
|
+
### Example Usage
|
157
167
|
|
168
|
+
```ruby
|
169
|
+
require 'chromate'
|
158
170
|
|
159
|
-
|
171
|
+
options = {
|
172
|
+
chrome_path: '/usr/bin/google-chrome',
|
173
|
+
headless: true,
|
174
|
+
native_control: true,
|
175
|
+
record: true
|
176
|
+
}
|
160
177
|
|
161
|
-
|
178
|
+
browser = Chromate::Browser.new(options)
|
162
179
|
|
163
|
-
|
180
|
+
browser.start
|
181
|
+
browser.navigate_to('https://example.com')
|
182
|
+
browser.screenshot('example.png')
|
183
|
+
element = browser.find_element('#main-header')
|
184
|
+
puts element.text
|
185
|
+
browser.stop
|
data/docs/client.md
ADDED
@@ -0,0 +1,126 @@
|
|
1
|
+
# Client
|
2
|
+
|
3
|
+
The `Chromate::Client` class is responsible for managing WebSocket connections to Chrome DevTools Protocol (CDP). It handles communication between the Chromate library and the Chrome browser, including message sending, receiving, and event handling.
|
4
|
+
|
5
|
+
### Initialization
|
6
|
+
|
7
|
+
```ruby
|
8
|
+
client = Chromate::Client.new(browser)
|
9
|
+
```
|
10
|
+
|
11
|
+
- **Parameters:**
|
12
|
+
- `browser` (Chromate::Browser): The browser instance to connect to.
|
13
|
+
|
14
|
+
### Public Methods
|
15
|
+
|
16
|
+
#### `#start`
|
17
|
+
|
18
|
+
Establishes the WebSocket connection to Chrome DevTools Protocol and sets up event handlers.
|
19
|
+
|
20
|
+
- **Returns:**
|
21
|
+
- `self`: Returns the client instance for method chaining.
|
22
|
+
|
23
|
+
- **Example:**
|
24
|
+
```ruby
|
25
|
+
client.start
|
26
|
+
```
|
27
|
+
|
28
|
+
#### `#stop`
|
29
|
+
|
30
|
+
Closes the WebSocket connection.
|
31
|
+
|
32
|
+
- **Returns:**
|
33
|
+
- `self`: Returns the client instance for method chaining.
|
34
|
+
|
35
|
+
- **Example:**
|
36
|
+
```ruby
|
37
|
+
client.stop
|
38
|
+
```
|
39
|
+
|
40
|
+
#### `#send_message(method, params = {})`
|
41
|
+
|
42
|
+
Sends a message to Chrome DevTools Protocol and waits for the response.
|
43
|
+
|
44
|
+
- **Parameters:**
|
45
|
+
- `method` (String): The CDP method to call.
|
46
|
+
- `params` (Hash, optional): Parameters for the CDP method.
|
47
|
+
|
48
|
+
- **Returns:**
|
49
|
+
- `Hash`: The response from Chrome DevTools Protocol.
|
50
|
+
|
51
|
+
- **Example:**
|
52
|
+
```ruby
|
53
|
+
result = client.send_message('DOM.getDocument')
|
54
|
+
```
|
55
|
+
|
56
|
+
#### `#reconnect`
|
57
|
+
|
58
|
+
Reestablishes the WebSocket connection if it was lost.
|
59
|
+
|
60
|
+
- **Returns:**
|
61
|
+
- `self`: Returns the client instance for method chaining.
|
62
|
+
|
63
|
+
- **Example:**
|
64
|
+
```ruby
|
65
|
+
client.reconnect
|
66
|
+
```
|
67
|
+
|
68
|
+
#### `#on_message`
|
69
|
+
|
70
|
+
Subscribes to WebSocket messages. Allows different parts of the application to listen for CDP events.
|
71
|
+
|
72
|
+
- **Parameters:**
|
73
|
+
- `&block` (Block): The block to execute when a message is received.
|
74
|
+
|
75
|
+
- **Example:**
|
76
|
+
```ruby
|
77
|
+
client.on_message do |message|
|
78
|
+
puts "Received message: #{message}"
|
79
|
+
end
|
80
|
+
```
|
81
|
+
|
82
|
+
### Class Methods
|
83
|
+
|
84
|
+
#### `.listeners`
|
85
|
+
|
86
|
+
Returns the array of registered message listeners.
|
87
|
+
|
88
|
+
- **Returns:**
|
89
|
+
- `Array<Proc>`: The array of listener blocks.
|
90
|
+
|
91
|
+
### Event Handling
|
92
|
+
|
93
|
+
The client automatically handles several WebSocket events:
|
94
|
+
|
95
|
+
- `:message`: Processes incoming CDP messages and notifies listeners
|
96
|
+
- `:open`: Logs successful connection
|
97
|
+
- `:error`: Logs WebSocket errors
|
98
|
+
- `:close`: Logs connection closure
|
99
|
+
|
100
|
+
### Error Handling
|
101
|
+
|
102
|
+
The client includes automatic reconnection logic when message sending fails:
|
103
|
+
|
104
|
+
- Attempts to reconnect to the WebSocket
|
105
|
+
- Retries the failed message
|
106
|
+
- Logs errors and debug information through `Chromate::CLogger`
|
107
|
+
|
108
|
+
### Example Usage
|
109
|
+
|
110
|
+
```ruby
|
111
|
+
browser = Chromate::Browser.new
|
112
|
+
client = Chromate::Client.new(browser)
|
113
|
+
|
114
|
+
client.start
|
115
|
+
|
116
|
+
# Send a CDP command
|
117
|
+
result = client.send_message('DOM.getDocument')
|
118
|
+
|
119
|
+
# Listen for specific events
|
120
|
+
client.on_message do |msg|
|
121
|
+
puts msg if msg['method'] == 'DOM.documentUpdated'
|
122
|
+
end
|
123
|
+
|
124
|
+
# Clean up
|
125
|
+
client.stop
|
126
|
+
```
|