chromate-rb 0.0.1.pre → 0.0.3.pre

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (58) hide show
  1. checksums.yaml +4 -4
  2. data/.rubocop.yml +2 -0
  3. data/CHANGELOG.md +72 -3
  4. data/README.md +33 -6
  5. data/Rakefile +48 -16
  6. data/docker_root/Gemfile +4 -0
  7. data/docker_root/Gemfile.lock +28 -0
  8. data/docker_root/TestInDocker.gif +0 -0
  9. data/docker_root/app.rb +87 -0
  10. data/dockerfiles/Dockerfile +21 -7
  11. data/dockerfiles/README.md +49 -0
  12. data/docs/BOT_BROWSER.md +74 -0
  13. data/docs/README.md +74 -0
  14. data/docs/browser.md +124 -102
  15. data/docs/client.md +126 -0
  16. data/docs/element.md +365 -0
  17. data/docs/elements/checkbox.md +69 -0
  18. data/docs/elements/radio.md +57 -0
  19. data/lib/bot_browser/downloader.rb +64 -0
  20. data/lib/bot_browser/installer.rb +99 -0
  21. data/lib/bot_browser.rb +43 -0
  22. data/lib/chromate/actions/dom.rb +35 -27
  23. data/lib/chromate/actions/navigate.rb +7 -5
  24. data/lib/chromate/actions/screenshot.rb +71 -14
  25. data/lib/chromate/actions/stealth.rb +62 -0
  26. data/lib/chromate/binary.rb +83 -0
  27. data/lib/chromate/browser.rb +120 -24
  28. data/lib/chromate/c_logger.rb +8 -0
  29. data/lib/chromate/client.rb +65 -26
  30. data/lib/chromate/configuration.rb +31 -14
  31. data/lib/chromate/element.rb +119 -16
  32. data/lib/chromate/elements/checkbox.rb +40 -0
  33. data/lib/chromate/elements/option.rb +43 -0
  34. data/lib/chromate/elements/radio.rb +37 -0
  35. data/lib/chromate/elements/select.rb +50 -6
  36. data/lib/chromate/elements/tags.rb +29 -0
  37. data/lib/chromate/exceptions.rb +2 -0
  38. data/lib/chromate/files/agents.json +11 -0
  39. data/lib/chromate/files/stealth.js +199 -0
  40. data/lib/chromate/hardwares/keyboard_controller.rb +45 -0
  41. data/lib/chromate/hardwares/keyboards/virtual_controller.rb +65 -0
  42. data/lib/chromate/hardwares/mouse_controller.rb +55 -11
  43. data/lib/chromate/hardwares/mouses/linux_controller.rb +124 -21
  44. data/lib/chromate/hardwares/mouses/mac_os_controller.rb +6 -6
  45. data/lib/chromate/hardwares/mouses/virtual_controller.rb +95 -7
  46. data/lib/chromate/hardwares/mouses/x11.rb +36 -0
  47. data/lib/chromate/hardwares.rb +19 -3
  48. data/lib/chromate/helpers.rb +22 -15
  49. data/lib/chromate/user_agent.rb +41 -15
  50. data/lib/chromate/version.rb +1 -1
  51. data/lib/chromate.rb +2 -0
  52. data/logo.png +0 -0
  53. data/results/bot.png +0 -0
  54. data/results/brotector.png +0 -0
  55. data/results/cloudflare.png +0 -0
  56. data/results/headers.png +0 -0
  57. data/results/pixelscan.png +0 -0
  58. metadata +45 -2
data/docs/browser.md CHANGED
@@ -1,163 +1,185 @@
1
- # Browser
1
+ ## `Chromate::Browser` Class
2
2
 
3
- The `Chromate::Browser` class is the main interface for interacting with the Chrome browser using the Chromate gem. It provides methods for navigating, interacting with elements, taking screenshots, and more.
3
+ The `Chromate::Browser` class is responsible for controlling a browser instance using the Chrome DevTools Protocol (CDP). It provides methods for navigation, screenshots, and DOM interactions, as well as handling browser lifecycle (start and stop).
4
4
 
5
- ## Initialization
6
-
7
- To create a new instance of the `Browser` class, you can pass in various options:
5
+ ### Initialization
8
6
 
9
7
  ```ruby
10
- browser = Chromate::Browser.new(
11
- headless: true,
12
- native_control: true,
13
- user_data_dir: '/path/to/user/data',
14
- record: false
15
- )
8
+ browser = Chromate::Browser.new(options = {})
16
9
  ```
17
10
 
18
- ### Options
11
+ - **Parameters:**
12
+ - `options` (Hash, optional): Configuration options for the browser instance.
13
+ - `:chrome_path` (String): Path to the Chrome executable.
14
+ - `:user_data_dir` (String): Directory for storing user data (default: a temporary directory).
15
+ - `:headless` (Boolean): Run the browser in headless mode.
16
+ - `:xfvb` (Boolean): Use Xvfb for headless mode on Linux.
17
+ - `:native_control` (Boolean): Enable native control for enhanced undetection.
18
+ - `:record` (Boolean): Enable video recording of the browser session.
19
19
 
20
- - `headless`: Run Chrome in headless mode (default: true).
21
- - `native_control`: Use native mouse control (default: false).
22
- - `user_data_dir`: Directory to store user data (default: a temporary directory).
23
- - `record`: Record the browser session (default: false).
24
- - `xfvb`: Use xvfb for headless mode (default: false).
20
+ ### Public Methods
25
21
 
26
- ## Methods
22
+ #### `#start`
27
23
 
28
- ###
24
+ Starts the browser process and initializes the CDP client.
29
25
 
30
- start
26
+ - **Example:**
27
+ ```ruby
28
+ browser.start
29
+ ```
31
30
 
31
+ #### `#stop`
32
32
 
33
+ Stops the browser process, including any associated Xvfb or video recording processes.
33
34
 
34
- Starts the Chrome browser with the specified options.
35
+ - **Example:**
36
+ ```ruby
37
+ browser.stop
38
+ ```
35
39
 
36
- ```ruby
37
- browser.start
38
- ```
40
+ #### `#native_control?`
39
41
 
40
- ###
42
+ Checks if native control is enabled for the browser instance.
41
43
 
42
- stop
44
+ - **Returns:**
45
+ - `Boolean`: `true` if native control is enabled, `false` otherwise.
43
46
 
47
+ - **Example:**
48
+ ```ruby
49
+ puts "Native control enabled" if browser.native_control?
50
+ ```
44
51
 
52
+ ### Navigation Methods (from `Actions::Navigate`)
45
53
 
46
- Stops the Chrome browser and any associated processes.
54
+ #### `#navigate_to(url)`
47
55
 
48
- ```ruby
49
- browser.stop
50
- ```
56
+ Navigates the browser to the specified URL.
51
57
 
52
- ### `navigate_to(url)`
58
+ - **Parameters:**
59
+ - `url` (String): The URL to navigate to.
53
60
 
54
- Navigates to the specified URL and waits for the page to load.
61
+ - **Example:**
62
+ ```ruby
63
+ browser.navigate_to('https://example.com')
64
+ ```
55
65
 
56
- ```ruby
57
- browser.navigate_to('http://example.com')
58
- ```
66
+ #### `#wait_for_page_load`
59
67
 
60
- ### `find_element(selector)`
68
+ Waits until the page has fully loaded, including the `DOMContentLoaded` event, `load` event, and `frameStoppedLoading` event.
61
69
 
62
- Finds an element on the page using the specified CSS selector.
70
+ - **Example:**
71
+ ```ruby
72
+ browser.wait_for_page_load
73
+ ```
63
74
 
64
- ```ruby
65
- element = browser.find_element('#some-element')
66
- ```
67
-
68
- ### `click_element(selector)`
69
-
70
- Clicks on an element specified by the CSS selector.
75
+ #### `#refresh`
71
76
 
72
- ```ruby
73
- browser.click_element('#some-element')
74
- ```
77
+ Reloads the current page.
75
78
 
76
- ### `hover_element(selector)`
79
+ - **Example:**
80
+ ```ruby
81
+ browser.refresh
82
+ ```
77
83
 
78
- Hovers over an element specified by the CSS selector.
84
+ #### `#go_back`
79
85
 
80
- ```ruby
81
- browser.hover_element('#some-element')
82
- ```
86
+ Navigates back to the previous page in the browser history.
83
87
 
84
- ### `type_text(selector, text)`
88
+ - **Example:**
89
+ ```ruby
90
+ browser.go_back
91
+ ```
85
92
 
86
- Types text into an element specified by the CSS selector.
93
+ ### Screenshot Methods (from `Actions::Screenshot`)
87
94
 
88
- ```ruby
89
- browser.type_text('#input-field', 'Hello, world!')
90
- ```
95
+ #### `#screenshot(file_path, options = {})`
91
96
 
92
- ### `screenshot_to_file(file_path, options = {})`
97
+ Takes a screenshot of the current page and saves it to the specified file.
93
98
 
94
- Takes a screenshot of the current page and saves it to the specified file path.
99
+ - **Parameters:**
100
+ - `file_path` (String, optional): The file path to save the screenshot.
101
+ - `options` (Hash, optional): Additional options for the screenshot.
102
+ - - `full_page` (Boolean, optional): Take a full page screenshot
95
103
 
96
- ```ruby
97
- browser.screenshot_to_file('screenshot.png')
98
- ```
104
+ It will call `#xvfb_screenshot` private method if `xvfb` mode is `true`
99
105
 
100
- ###
106
+ - **Example:**
107
+ ```ruby
108
+ browser.screenshot('screenshot.png')
109
+ ```
101
110
 
102
- native_control?
111
+ ### DOM Methods (from `Actions::Dom`)
103
112
 
113
+ #### `#find_element(selector)`
104
114
 
115
+ Finds a single element on the page using the specified CSS selector. Returns a specialized element class based on the element type:
105
116
 
106
- Returns whether native control is enabled.
117
+ - **Parameters:**
118
+ - `selector` (String): The CSS selector to locate the element.
107
119
 
108
- ```ruby
109
- puts browser.native_control? # => true or false
110
- ```
120
+ - **Returns:**
121
+ - `Chromate::Elements::Select`: For `<select>` elements
122
+ - `Chromate::Elements::Option`: For `<option>` elements
123
+ - `Chromate::Elements::Radio`: For radio button inputs (`<input type="radio">`)
124
+ - `Chromate::Elements::Checkbox`: For checkbox inputs (`<input type="checkbox">`)
125
+ - `Chromate::Element`: For all other element types
111
126
 
112
- ## Example Usage
127
+ Each specialized element type provides specific methods for interacting with that type of element. For example:
113
128
 
114
129
  ```ruby
115
- require 'chromate'
116
-
117
- browser = Chromate::Browser.new(headless: true, native_control: true)
118
- browser.start
130
+ # Working with radio buttons
131
+ radio = browser.find_element('input[type="radio"]')
132
+ radio.check if !radio.checked?
119
133
 
120
- browser.navigate_to('http://example.com')
121
- browser.find_element('#some-element').click
122
- browser.screenshot_to_file('screenshot.png')
134
+ # Working with checkboxes
135
+ checkbox = browser.find_element('input[type="checkbox"]')
136
+ checkbox.toggle
123
137
 
124
- browser.stop
138
+ # Working with select elements
139
+ select = browser.find_element('select#country')
140
+ select.select_option('France')
125
141
  ```
126
142
 
127
- ## Private Methods
128
-
129
- ###
130
-
131
- start_video_recording
132
-
133
-
134
-
135
- Starts recording the browser session using `ffmpeg`.
143
+ See the [Element documentation](element.md) for more details about specialized elements.
136
144
 
137
- ###
145
+ #### `#evaluate_script(script)`
138
146
 
139
- build_args
147
+ Executes the specified JavaScript expression on the page.
140
148
 
149
+ - **Parameters:**
150
+ - `script` (String): The JavaScript code to evaluate.
141
151
 
152
+ - **Returns:**
153
+ - The result of the JavaScript evaluation.
142
154
 
143
- Builds the arguments for starting the Chrome process.
155
+ - **Example:**
156
+ ```ruby
157
+ result = browser.evaluate_script('document.title')
158
+ puts "Page title: #{result}"
159
+ ```
144
160
 
145
- ###
161
+ ### Exception Handling
146
162
 
147
- stop_and_exit
148
-
149
-
150
-
151
- Stops the browser and exits the process.
152
-
153
- ###
154
-
155
- config
163
+ - The browser handles `INT` and `TERM` signals gracefully by stopping the browser process and exiting safely.
164
+ - The `stop_and_exit` method is used to ensure proper shutdown.
156
165
 
166
+ ### Example Usage
157
167
 
168
+ ```ruby
169
+ require 'chromate'
158
170
 
159
- Returns the Chromate configuration.
171
+ options = {
172
+ chrome_path: '/usr/bin/google-chrome',
173
+ headless: true,
174
+ native_control: true,
175
+ record: true
176
+ }
160
177
 
161
- ## Conclusion
178
+ browser = Chromate::Browser.new(options)
162
179
 
163
- The `Chromate::Browser` class provides a powerful interface for automating interactions with the Chrome browser. With support for headless mode, native mouse control, and more, it is a versatile tool for creating undetectable bots with human-like behavior.
180
+ browser.start
181
+ browser.navigate_to('https://example.com')
182
+ browser.screenshot('example.png')
183
+ element = browser.find_element('#main-header')
184
+ puts element.text
185
+ browser.stop
data/docs/client.md ADDED
@@ -0,0 +1,126 @@
1
+ # Client
2
+
3
+ The `Chromate::Client` class is responsible for managing WebSocket connections to Chrome DevTools Protocol (CDP). It handles communication between the Chromate library and the Chrome browser, including message sending, receiving, and event handling.
4
+
5
+ ### Initialization
6
+
7
+ ```ruby
8
+ client = Chromate::Client.new(browser)
9
+ ```
10
+
11
+ - **Parameters:**
12
+ - `browser` (Chromate::Browser): The browser instance to connect to.
13
+
14
+ ### Public Methods
15
+
16
+ #### `#start`
17
+
18
+ Establishes the WebSocket connection to Chrome DevTools Protocol and sets up event handlers.
19
+
20
+ - **Returns:**
21
+ - `self`: Returns the client instance for method chaining.
22
+
23
+ - **Example:**
24
+ ```ruby
25
+ client.start
26
+ ```
27
+
28
+ #### `#stop`
29
+
30
+ Closes the WebSocket connection.
31
+
32
+ - **Returns:**
33
+ - `self`: Returns the client instance for method chaining.
34
+
35
+ - **Example:**
36
+ ```ruby
37
+ client.stop
38
+ ```
39
+
40
+ #### `#send_message(method, params = {})`
41
+
42
+ Sends a message to Chrome DevTools Protocol and waits for the response.
43
+
44
+ - **Parameters:**
45
+ - `method` (String): The CDP method to call.
46
+ - `params` (Hash, optional): Parameters for the CDP method.
47
+
48
+ - **Returns:**
49
+ - `Hash`: The response from Chrome DevTools Protocol.
50
+
51
+ - **Example:**
52
+ ```ruby
53
+ result = client.send_message('DOM.getDocument')
54
+ ```
55
+
56
+ #### `#reconnect`
57
+
58
+ Reestablishes the WebSocket connection if it was lost.
59
+
60
+ - **Returns:**
61
+ - `self`: Returns the client instance for method chaining.
62
+
63
+ - **Example:**
64
+ ```ruby
65
+ client.reconnect
66
+ ```
67
+
68
+ #### `#on_message`
69
+
70
+ Subscribes to WebSocket messages. Allows different parts of the application to listen for CDP events.
71
+
72
+ - **Parameters:**
73
+ - `&block` (Block): The block to execute when a message is received.
74
+
75
+ - **Example:**
76
+ ```ruby
77
+ client.on_message do |message|
78
+ puts "Received message: #{message}"
79
+ end
80
+ ```
81
+
82
+ ### Class Methods
83
+
84
+ #### `.listeners`
85
+
86
+ Returns the array of registered message listeners.
87
+
88
+ - **Returns:**
89
+ - `Array<Proc>`: The array of listener blocks.
90
+
91
+ ### Event Handling
92
+
93
+ The client automatically handles several WebSocket events:
94
+
95
+ - `:message`: Processes incoming CDP messages and notifies listeners
96
+ - `:open`: Logs successful connection
97
+ - `:error`: Logs WebSocket errors
98
+ - `:close`: Logs connection closure
99
+
100
+ ### Error Handling
101
+
102
+ The client includes automatic reconnection logic when message sending fails:
103
+
104
+ - Attempts to reconnect to the WebSocket
105
+ - Retries the failed message
106
+ - Logs errors and debug information through `Chromate::CLogger`
107
+
108
+ ### Example Usage
109
+
110
+ ```ruby
111
+ browser = Chromate::Browser.new
112
+ client = Chromate::Client.new(browser)
113
+
114
+ client.start
115
+
116
+ # Send a CDP command
117
+ result = client.send_message('DOM.getDocument')
118
+
119
+ # Listen for specific events
120
+ client.on_message do |msg|
121
+ puts msg if msg['method'] == 'DOM.documentUpdated'
122
+ end
123
+
124
+ # Clean up
125
+ client.stop
126
+ ```