chromate-rb 0.0.1.pre → 0.0.2.pre

Sign up to get free protection for your applications and to get access to all the features.
Files changed (46) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop.yml +1 -0
  3. data/CHANGELOG.md +54 -3
  4. data/README.md +33 -6
  5. data/Rakefile +48 -16
  6. data/docker_root/Gemfile +4 -0
  7. data/docker_root/Gemfile.lock +28 -0
  8. data/docker_root/TestInDocker.gif +0 -0
  9. data/docker_root/app.rb +92 -0
  10. data/dockerfiles/Dockerfile +21 -7
  11. data/dockerfiles/README.md +49 -0
  12. data/docs/README.md +74 -0
  13. data/docs/browser.md +149 -92
  14. data/docs/element.md +289 -0
  15. data/lib/bot_browser/downloader.rb +52 -0
  16. data/lib/bot_browser/installer.rb +81 -0
  17. data/lib/bot_browser.rb +39 -0
  18. data/lib/chromate/actions/dom.rb +28 -9
  19. data/lib/chromate/actions/navigate.rb +4 -5
  20. data/lib/chromate/actions/screenshot.rb +30 -11
  21. data/lib/chromate/actions/stealth.rb +47 -0
  22. data/lib/chromate/browser.rb +64 -12
  23. data/lib/chromate/c_logger.rb +7 -0
  24. data/lib/chromate/client.rb +40 -18
  25. data/lib/chromate/configuration.rb +31 -14
  26. data/lib/chromate/element.rb +65 -15
  27. data/lib/chromate/elements/select.rb +59 -7
  28. data/lib/chromate/hardwares/keyboard_controller.rb +34 -0
  29. data/lib/chromate/hardwares/keyboards/virtual_controller.rb +65 -0
  30. data/lib/chromate/hardwares/mouse_controller.rb +47 -11
  31. data/lib/chromate/hardwares/mouses/linux_controller.rb +124 -21
  32. data/lib/chromate/hardwares/mouses/mac_os_controller.rb +6 -6
  33. data/lib/chromate/hardwares/mouses/virtual_controller.rb +95 -7
  34. data/lib/chromate/hardwares/mouses/x11.rb +36 -0
  35. data/lib/chromate/hardwares.rb +16 -0
  36. data/lib/chromate/helpers.rb +22 -15
  37. data/lib/chromate/user_agent.rb +39 -15
  38. data/lib/chromate/version.rb +1 -1
  39. data/lib/chromate.rb +2 -0
  40. data/logo.png +0 -0
  41. data/results/bot.png +0 -0
  42. data/results/brotector.png +0 -0
  43. data/results/cloudflare.png +0 -0
  44. data/results/headers.png +0 -0
  45. data/results/pixelscan.png +0 -0
  46. metadata +20 -2
data/docs/browser.md CHANGED
@@ -1,163 +1,220 @@
1
- # Browser
1
+ ## `Chromate::Browser` Class
2
2
 
3
- The `Chromate::Browser` class is the main interface for interacting with the Chrome browser using the Chromate gem. It provides methods for navigating, interacting with elements, taking screenshots, and more.
3
+ The `Chromate::Browser` class is responsible for controlling a browser instance using the Chrome DevTools Protocol (CDP). It provides methods for navigation, screenshots, and DOM interactions, as well as handling browser lifecycle (start and stop).
4
4
 
5
- ## Initialization
6
-
7
- To create a new instance of the `Browser` class, you can pass in various options:
5
+ ### Initialization
8
6
 
9
7
  ```ruby
10
- browser = Chromate::Browser.new(
11
- headless: true,
12
- native_control: true,
13
- user_data_dir: '/path/to/user/data',
14
- record: false
15
- )
8
+ browser = Chromate::Browser.new(options = {})
16
9
  ```
17
10
 
18
- ### Options
11
+ - **Parameters:**
12
+ - `options` (Hash, optional): Configuration options for the browser instance.
13
+ - `:chrome_path` (String): Path to the Chrome executable.
14
+ - `:user_data_dir` (String): Directory for storing user data (default: a temporary directory).
15
+ - `:headless` (Boolean): Run the browser in headless mode.
16
+ - `:xfvb` (Boolean): Use Xvfb for headless mode on Linux.
17
+ - `:native_control` (Boolean): Enable native control for enhanced undetection.
18
+ - `:record` (Boolean): Enable video recording of the browser session.
19
19
 
20
- - `headless`: Run Chrome in headless mode (default: true).
21
- - `native_control`: Use native mouse control (default: false).
22
- - `user_data_dir`: Directory to store user data (default: a temporary directory).
23
- - `record`: Record the browser session (default: false).
24
- - `xfvb`: Use xvfb for headless mode (default: false).
20
+ ### Public Methods
25
21
 
26
- ## Methods
22
+ #### `#start`
27
23
 
28
- ###
24
+ Starts the browser process and initializes the CDP client.
29
25
 
30
- start
26
+ - **Example:**
27
+ ```ruby
28
+ browser.start
29
+ ```
31
30
 
31
+ #### `#stop`
32
32
 
33
+ Stops the browser process, including any associated Xvfb or video recording processes.
33
34
 
34
- Starts the Chrome browser with the specified options.
35
+ - **Example:**
36
+ ```ruby
37
+ browser.stop
38
+ ```
35
39
 
36
- ```ruby
37
- browser.start
38
- ```
40
+ #### `#native_control?`
39
41
 
40
- ###
42
+ Checks if native control is enabled for the browser instance.
41
43
 
42
- stop
44
+ - **Returns:**
45
+ - `Boolean`: `true` if native control is enabled, `false` otherwise.
43
46
 
47
+ - **Example:**
48
+ ```ruby
49
+ puts "Native control enabled" if browser.native_control?
50
+ ```
44
51
 
52
+ ### Navigation Methods (from `Actions::Navigate`)
45
53
 
46
- Stops the Chrome browser and any associated processes.
54
+ #### `#navigate_to(url)`
47
55
 
48
- ```ruby
49
- browser.stop
50
- ```
56
+ Navigates the browser to the specified URL.
51
57
 
52
- ### `navigate_to(url)`
58
+ - **Parameters:**
59
+ - `url` (String): The URL to navigate to.
53
60
 
54
- Navigates to the specified URL and waits for the page to load.
61
+ - **Example:**
62
+ ```ruby
63
+ browser.navigate_to('https://example.com')
64
+ ```
55
65
 
56
- ```ruby
57
- browser.navigate_to('http://example.com')
58
- ```
66
+ #### `#wait_for_page_load`
59
67
 
60
- ### `find_element(selector)`
68
+ Waits until the page has fully loaded, including the `DOMContentLoaded` event, `load` event, and `frameStoppedLoading` event.
61
69
 
62
- Finds an element on the page using the specified CSS selector.
70
+ - **Example:**
71
+ ```ruby
72
+ browser.wait_for_page_load
73
+ ```
63
74
 
64
- ```ruby
65
- element = browser.find_element('#some-element')
66
- ```
75
+ #### `#refresh`
67
76
 
68
- ### `click_element(selector)`
77
+ Reloads the current page.
69
78
 
70
- Clicks on an element specified by the CSS selector.
79
+ - **Example:**
80
+ ```ruby
81
+ browser.refresh
82
+ ```
71
83
 
72
- ```ruby
73
- browser.click_element('#some-element')
74
- ```
84
+ #### `#go_back`
75
85
 
76
- ### `hover_element(selector)`
86
+ Navigates back to the previous page in the browser history.
77
87
 
78
- Hovers over an element specified by the CSS selector.
88
+ - **Example:**
89
+ ```ruby
90
+ browser.go_back
91
+ ```
79
92
 
80
- ```ruby
81
- browser.hover_element('#some-element')
82
- ```
93
+ ### Screenshot Methods (from `Actions::Screenshot`)
83
94
 
84
- ### `type_text(selector, text)`
95
+ #### `#screenshot(file_path, options = {})`
85
96
 
86
- Types text into an element specified by the CSS selector.
97
+ Takes a screenshot of the current page and saves it to the specified file.
87
98
 
88
- ```ruby
89
- browser.type_text('#input-field', 'Hello, world!')
90
- ```
99
+ - **Parameters:**
100
+ - `file_path` (String, optional): The file path to save the screenshot.
101
+ - `options` (Hash, optional): Additional options for the screenshot.
102
+ - - `full_page` (Boolean, optional): Take a full page screenshot
91
103
 
92
- ### `screenshot_to_file(file_path, options = {})`
104
+ It will call `#xvfb_screenshot` private method if `xvfb` mode is `true`
93
105
 
94
- Takes a screenshot of the current page and saves it to the specified file path.
106
+ - **Example:**
107
+ ```ruby
108
+ browser.screenshot('screenshot.png')
109
+ ```
95
110
 
96
- ```ruby
97
- browser.screenshot_to_file('screenshot.png')
98
- ```
111
+ ### DOM Methods (from `Actions::Dom`)
99
112
 
100
- ###
113
+ #### `#find_element(selector)`
101
114
 
102
- native_control?
115
+ Finds a single element on the page using the specified CSS selector.
103
116
 
117
+ - **Parameters:**
118
+ - `selector` (String): The CSS selector to locate the element.
104
119
 
120
+ - **Returns:**
121
+ - `Chromate::Element`: The found element.
105
122
 
106
- Returns whether native control is enabled.
123
+ - **Example:**
124
+ ```ruby
125
+ element = browser.find_element('#my-element')
126
+ puts element.text
127
+ ```
107
128
 
108
- ```ruby
109
- puts browser.native_control? # => true or false
110
- ```
129
+ #### `#click_element(selector)`
111
130
 
112
- ## Example Usage
131
+ Finds an element by selector and clicks it.
113
132
 
114
- ```ruby
115
- require 'chromate'
133
+ - **Parameters:**
134
+ - `selector` (String): The CSS selector of the element to click.
116
135
 
117
- browser = Chromate::Browser.new(headless: true, native_control: true)
118
- browser.start
136
+ - **Example:**
137
+ ```ruby
138
+ browser.click_element('#submit-button')
139
+ ```
119
140
 
120
- browser.navigate_to('http://example.com')
121
- browser.find_element('#some-element').click
122
- browser.screenshot_to_file('screenshot.png')
141
+ #### `#hover_element(selector)`
123
142
 
124
- browser.stop
125
- ```
126
-
127
- ## Private Methods
143
+ Finds an element by selector and hovers over it.
128
144
 
129
- ###
145
+ - **Parameters:**
146
+ - `selector` (String): The CSS selector of the element to hover.
130
147
 
131
- start_video_recording
148
+ - **Example:**
149
+ ```ruby
150
+ browser.hover_element('#hover-target')
151
+ ```
132
152
 
153
+ #### `#type_text(selector, text)`
133
154
 
155
+ Finds an element by selector and types the specified text into it.
134
156
 
135
- Starts recording the browser session using `ffmpeg`.
157
+ - **Parameters:**
158
+ - `selector` (String): The CSS selector of the element.
159
+ - `text` (String): The text to type into the element.
136
160
 
137
- ###
161
+ - **Example:**
162
+ ```ruby
163
+ browser.type_text('#input-field', 'Hello, Chromate!')
164
+ ```
138
165
 
139
- build_args
166
+ #### `#select_option(selector, option)`
140
167
 
168
+ Selects an option from a dropdown element.
141
169
 
170
+ - **Parameters:**
171
+ - `selector` (String): The CSS selector of the dropdown element.
172
+ - `option` (String): The value of the option to select.
142
173
 
143
- Builds the arguments for starting the Chrome process.
174
+ - **Example:**
175
+ ```ruby
176
+ browser.select_option('#dropdown', 'option2')
177
+ ```
144
178
 
145
- ###
179
+ #### `#evaluate_script(script)`
146
180
 
147
- stop_and_exit
181
+ Executes the specified JavaScript expression on the page.
148
182
 
183
+ - **Parameters:**
184
+ - `script` (String): The JavaScript code to evaluate.
149
185
 
186
+ - **Returns:**
187
+ - The result of the JavaScript evaluation.
150
188
 
151
- Stops the browser and exits the process.
189
+ - **Example:**
190
+ ```ruby
191
+ result = browser.evaluate_script('document.title')
192
+ puts "Page title: #{result}"
193
+ ```
152
194
 
153
- ###
195
+ ### Exception Handling
154
196
 
155
- config
197
+ - The browser handles `INT` and `TERM` signals gracefully by stopping the browser process and exiting safely.
198
+ - The `stop_and_exit` method is used to ensure proper shutdown.
156
199
 
200
+ ### Example Usage
157
201
 
202
+ ```ruby
203
+ require 'chromate'
158
204
 
159
- Returns the Chromate configuration.
205
+ options = {
206
+ chrome_path: '/usr/bin/google-chrome',
207
+ headless: true,
208
+ native_control: true,
209
+ record: true
210
+ }
160
211
 
161
- ## Conclusion
212
+ browser = Chromate::Browser.new(options)
162
213
 
163
- The `Chromate::Browser` class provides a powerful interface for automating interactions with the Chrome browser. With support for headless mode, native mouse control, and more, it is a versatile tool for creating undetectable bots with human-like behavior.
214
+ browser.start
215
+ browser.navigate_to('https://example.com')
216
+ browser.screenshot('example.png')
217
+ element = browser.find_element('#main-header')
218
+ puts element.text
219
+ browser.stop
220
+ ```
data/docs/element.md ADDED
@@ -0,0 +1,289 @@
1
+ # Element
2
+
3
+ The `Chromate::Element` class represents a DOM element in a browser controlled via the Chrome DevTools Protocol (CDP). It provides methods to interact with the element, including manipulating its attributes, getting its text content, and simulating user actions.
4
+
5
+ ### Initialization
6
+
7
+ ```ruby
8
+ element = Chromate::Element.new(selector, client, node_id: nil, object_id: nil, root_id: nil)
9
+ ```
10
+
11
+ - **Parameters:**
12
+ - `selector` (String): The CSS selector used to locate the element.
13
+ - `client` (Chromate::Client): An instance of the CDP client.
14
+ - `node_id` (Integer, optional): The node ID of the element.
15
+ - `object_id` (String, optional): The object ID of the element.
16
+ - `root_id` (Integer, optional): The root node ID of the document.
17
+
18
+ ### Public Methods
19
+
20
+ #### `#mouse`
21
+
22
+ Returns the mouse controller for interacting with the element.
23
+
24
+ - **Example:**
25
+ ```ruby
26
+ element.mouse.click
27
+ ```
28
+
29
+ #### `#keyboard`
30
+
31
+ Returns the keyboard controller for interacting with the element.
32
+
33
+ - **Example:**
34
+ ```ruby
35
+ element.keyboard.type('Hello World')
36
+ ```
37
+
38
+ #### `#inspect`
39
+
40
+ Returns a string representation of the element.
41
+
42
+ - **Example:**
43
+ ```ruby
44
+ puts element.inspect
45
+ ```
46
+
47
+ #### `#text`
48
+
49
+ Retrieves the inner text of the element.
50
+
51
+ - **Example:**
52
+ ```ruby
53
+ text = element.text
54
+ puts "Element text: #{text}"
55
+ ```
56
+
57
+ #### `#html`
58
+
59
+ Retrieves the outer HTML of the element.
60
+
61
+ - **Example:**
62
+ ```ruby
63
+ html = element.html
64
+ puts "Element HTML: #{html}"
65
+ ```
66
+
67
+ #### `#attributes`
68
+
69
+ Returns a hash of the element's attributes.
70
+
71
+ - **Example:**
72
+ ```ruby
73
+ attributes = element.attributes
74
+ puts "Element attributes: #{attributes}"
75
+ ```
76
+
77
+ #### `#set_attribute(name, value)`
78
+
79
+ Sets an attribute on the element.
80
+
81
+ - **Parameters:**
82
+ - `name` (String): The name of the attribute.
83
+ - `value` (String): The value to set for the attribute.
84
+
85
+ - **Example:**
86
+ ```ruby
87
+ element.set_attribute('class', 'highlighted')
88
+ ```
89
+
90
+ #### `#bounding_box`
91
+
92
+ Returns a hash with the dimensions of the element's bounding box.
93
+
94
+ - **Example:**
95
+ ```ruby
96
+ box = element.bounding_box
97
+ puts "Bounding box: #{box}"
98
+ ```
99
+
100
+ #### `#x`
101
+
102
+ Returns the x-coordinate of the element's position.
103
+
104
+ - **Example:**
105
+ ```ruby
106
+ x_position = element.x
107
+ puts "X Position: #{x_position}"
108
+ ```
109
+
110
+ #### `#y`
111
+
112
+ Returns the y-coordinate of the element's position.
113
+
114
+ - **Example:**
115
+ ```ruby
116
+ y_position = element.y
117
+ puts "Y Position: #{y_position}"
118
+ ```
119
+
120
+ #### `#width`
121
+
122
+ Returns the width of the element.
123
+
124
+ - **Example:**
125
+ ```ruby
126
+ width = element.width
127
+ puts "Element width: #{width}"
128
+ ```
129
+
130
+ #### `#height`
131
+
132
+ Returns the height of the element.
133
+
134
+ - **Example:**
135
+ ```ruby
136
+ height = element.height
137
+ puts "Element height: #{height}"
138
+ ```
139
+
140
+ #### `#focus`
141
+
142
+ Sets focus on the element.
143
+
144
+ - **Example:**
145
+ ```ruby
146
+ element.focus
147
+ ```
148
+
149
+ #### `#click`
150
+
151
+ Simulates a click on the element.
152
+
153
+ - **Example:**
154
+ ```ruby
155
+ element.click
156
+ ```
157
+
158
+ #### `#hover`
159
+
160
+ Simulates a hover action over the element.
161
+
162
+ - **Example:**
163
+ ```ruby
164
+ element.hover
165
+ ```
166
+
167
+ #### `#type(text)`
168
+
169
+ Types the specified text into the element.
170
+
171
+ - **Parameters:**
172
+ - `text` (String): The text to type.
173
+
174
+ - **Example:**
175
+ ```ruby
176
+ element.type('Hello, Chromate!')
177
+ ```
178
+
179
+ #### `#press_enter`
180
+
181
+ Simulates pressing the Enter key and submits the form if the element is inside one.
182
+
183
+ - **Example:**
184
+ ```ruby
185
+ element.press_enter
186
+ ```
187
+
188
+ #### `#drop_to(element)`
189
+
190
+ Drag current element and drop to target
191
+
192
+ - **Example:**
193
+ ```ruby
194
+ element.drop_to(target_element)
195
+ ```
196
+
197
+ #### `#find_element(selector)`
198
+
199
+ Finds a single child element matching the given selector.
200
+
201
+ - **Parameters:**
202
+ - `selector` (String): The CSS selector to find the element.
203
+
204
+ - **Returns:**
205
+ - `Chromate::Element`: The found element.
206
+
207
+ - **Example:**
208
+ ```ruby
209
+ child_element = element.find_element('.child')
210
+ puts child_element.text
211
+ ```
212
+
213
+ #### `#find_elements(selector, max: 0)`
214
+
215
+ Finds all child elements matching the given selector.
216
+
217
+ - **Parameters:**
218
+ - `selector` (String): The CSS selector to find elements.
219
+ - `max` (Integer, optional): The maximum number of elements to find (0 for no limit).
220
+
221
+ - **Returns:**
222
+ - `Array<Chromate::Element>`: An array of found elements.
223
+
224
+ - **Example:**
225
+ ```ruby
226
+ elements = element.find_elements('.item')
227
+ elements.each { |el| puts el.text }
228
+ ```
229
+
230
+ #### `#shadow_root_id`
231
+
232
+ Returns the shadow root ID of the element if it has one.
233
+
234
+ - **Example:**
235
+ ```ruby
236
+ shadow_id = element.shadow_root_id
237
+ puts "Shadow root ID: #{shadow_id}"
238
+ ```
239
+
240
+ #### `#shadow_root?`
241
+
242
+ Checks if the element has a shadow root.
243
+
244
+ - **Returns:**
245
+ - `Boolean`: `true` if the element has a shadow root, otherwise `false`.
246
+
247
+ - **Example:**
248
+ ```ruby
249
+ if element.shadow_root?
250
+ puts "Element has a shadow root."
251
+ end
252
+ ```
253
+
254
+ #### `#find_shadow_child(selector)`
255
+
256
+ Finds a single child element inside the shadow root using the given selector.
257
+
258
+ - **Parameters:**
259
+ - `selector` (String): The CSS selector to find the shadow child element.
260
+
261
+ - **Returns:**
262
+ - `Chromate::Element` or `nil`: The found element or `nil` if not found.
263
+
264
+ - **Example:**
265
+ ```ruby
266
+ shadow_child = element.find_shadow_child('.shadow-element')
267
+ puts shadow_child.text if shadow_child
268
+ ```
269
+
270
+ #### `#find_shadow_children(selector)`
271
+
272
+ Finds all child elements inside the shadow root using the given selector.
273
+
274
+ - **Parameters:**
275
+ - `selector` (String): The CSS selector to find shadow child elements.
276
+
277
+ - **Returns:**
278
+ - `Array<Chromate::Element>`: An array of found elements.
279
+
280
+ - **Example:**
281
+ ```ruby
282
+ shadow_elements = element.find_shadow_children('.shadow-item')
283
+ shadow_elements.each { |el| puts el.text }
284
+ ```
285
+
286
+ ### Exceptions
287
+
288
+ - `NotFoundError`: Raised when an element cannot be found with the given selector.
289
+ - `InvalidSelectorError`: Raised when the selector cannot resolve to a valid element.
@@ -0,0 +1,52 @@
1
+ # frozen_string_literal: true
2
+
3
+ module BotBrowser
4
+ class Downloader
5
+ class << self
6
+ def download(version = nil)
7
+ version ||= versions.keys.first
8
+ version = version.to_sym
9
+ platform = :mac
10
+ arch = :arm64
11
+ binary_path = download_file(versions[version][platform][arch][:binary], "/tmp/botbrowser_#{version}_#{platform}_#{arch}.dmg")
12
+ profile_path = download_file(versions[version][platform][arch][:profile], "/tmp/botbrowser_#{version}_#{platform}_#{arch}.json")
13
+
14
+ [binary_path, profile_path]
15
+ end
16
+
17
+ def download_file(url, path)
18
+ Chromate::CLogger.log("Downloading #{url} to #{path}")
19
+ `curl -L #{url} >> #{path}`
20
+
21
+ path
22
+ end
23
+
24
+ def versions
25
+ {
26
+ v130: {
27
+ mac: {
28
+ arm64: {
29
+ binary: 'https://github.com/MiddleSchoolStudent/BotBrowser/releases/download/v130/botbrowser_130.0.6723.92_mac_arm64.dmg',
30
+ profile: 'https://raw.githubusercontent.com/MiddleSchoolStudent/BotBrowser/refs/heads/main/profiles/v130/chrome130-macarm.enc'
31
+ }
32
+ },
33
+ linux: {
34
+ x64: {
35
+ binary: 'https://github.com/MiddleSchoolStudent/BotBrowser/releases/download/v130/botbrowser_130.0.6723.117_amd64.deb',
36
+ # no specifi linux profile for moment
37
+ profile: 'https://raw.githubusercontent.com/MiddleSchoolStudent/BotBrowser/refs/heads/main/profiles/v130/chrome130-macarm.enc'
38
+ }
39
+ },
40
+ windows: {
41
+ x64: {
42
+ binary: 'https://github.com/MiddleSchoolStudent/BotBrowser/releases/download/v130/botbrowser_130.0.6723.117_win_x86_64.7z',
43
+ # no specific windows profile for moment
44
+ profile: 'https://raw.githubusercontent.com/MiddleSchoolStudent/BotBrowser/refs/heads/main/profiles/v130/chrome130-macarm.enc'
45
+ }
46
+ }
47
+ }
48
+ }
49
+ end
50
+ end
51
+ end
52
+ end