ferrum-mcp 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (63) hide show
  1. checksums.yaml +7 -0
  2. data/.env.example +90 -0
  3. data/CHANGELOG.md +229 -0
  4. data/CONTRIBUTING.md +469 -0
  5. data/LICENSE +21 -0
  6. data/README.md +334 -0
  7. data/SECURITY.md +286 -0
  8. data/bin/ferrum-mcp +66 -0
  9. data/bin/lint +10 -0
  10. data/bin/serve +3 -0
  11. data/bin/test +4 -0
  12. data/docs/API_REFERENCE.md +1410 -0
  13. data/docs/CONFIGURATION.md +254 -0
  14. data/docs/DEPLOYMENT.md +846 -0
  15. data/docs/DOCKER.md +836 -0
  16. data/docs/DOCKER_BOTBROWSER.md +455 -0
  17. data/docs/GETTING_STARTED.md +249 -0
  18. data/docs/TROUBLESHOOTING.md +677 -0
  19. data/lib/ferrum_mcp/browser_manager.rb +101 -0
  20. data/lib/ferrum_mcp/cli/command_handler.rb +99 -0
  21. data/lib/ferrum_mcp/cli/server_runner.rb +166 -0
  22. data/lib/ferrum_mcp/configuration.rb +229 -0
  23. data/lib/ferrum_mcp/resource_manager.rb +223 -0
  24. data/lib/ferrum_mcp/server.rb +254 -0
  25. data/lib/ferrum_mcp/session.rb +227 -0
  26. data/lib/ferrum_mcp/session_manager.rb +183 -0
  27. data/lib/ferrum_mcp/tools/accept_cookies_tool.rb +458 -0
  28. data/lib/ferrum_mcp/tools/base_tool.rb +114 -0
  29. data/lib/ferrum_mcp/tools/clear_cookies_tool.rb +66 -0
  30. data/lib/ferrum_mcp/tools/click_tool.rb +218 -0
  31. data/lib/ferrum_mcp/tools/close_session_tool.rb +49 -0
  32. data/lib/ferrum_mcp/tools/create_session_tool.rb +146 -0
  33. data/lib/ferrum_mcp/tools/drag_and_drop_tool.rb +171 -0
  34. data/lib/ferrum_mcp/tools/evaluate_js_tool.rb +46 -0
  35. data/lib/ferrum_mcp/tools/execute_script_tool.rb +48 -0
  36. data/lib/ferrum_mcp/tools/fill_form_tool.rb +78 -0
  37. data/lib/ferrum_mcp/tools/find_by_text_tool.rb +153 -0
  38. data/lib/ferrum_mcp/tools/get_attribute_tool.rb +56 -0
  39. data/lib/ferrum_mcp/tools/get_cookies_tool.rb +70 -0
  40. data/lib/ferrum_mcp/tools/get_html_tool.rb +52 -0
  41. data/lib/ferrum_mcp/tools/get_session_info_tool.rb +40 -0
  42. data/lib/ferrum_mcp/tools/get_text_tool.rb +67 -0
  43. data/lib/ferrum_mcp/tools/get_title_tool.rb +42 -0
  44. data/lib/ferrum_mcp/tools/get_url_tool.rb +39 -0
  45. data/lib/ferrum_mcp/tools/go_back_tool.rb +49 -0
  46. data/lib/ferrum_mcp/tools/go_forward_tool.rb +49 -0
  47. data/lib/ferrum_mcp/tools/hover_tool.rb +76 -0
  48. data/lib/ferrum_mcp/tools/list_sessions_tool.rb +33 -0
  49. data/lib/ferrum_mcp/tools/navigate_tool.rb +59 -0
  50. data/lib/ferrum_mcp/tools/press_key_tool.rb +91 -0
  51. data/lib/ferrum_mcp/tools/query_shadow_dom_tool.rb +225 -0
  52. data/lib/ferrum_mcp/tools/refresh_tool.rb +49 -0
  53. data/lib/ferrum_mcp/tools/screenshot_tool.rb +121 -0
  54. data/lib/ferrum_mcp/tools/session_tool.rb +37 -0
  55. data/lib/ferrum_mcp/tools/set_cookie_tool.rb +77 -0
  56. data/lib/ferrum_mcp/tools/solve_captcha_tool.rb +528 -0
  57. data/lib/ferrum_mcp/transport/http_server.rb +93 -0
  58. data/lib/ferrum_mcp/transport/rate_limiter.rb +79 -0
  59. data/lib/ferrum_mcp/transport/stdio_server.rb +63 -0
  60. data/lib/ferrum_mcp/version.rb +5 -0
  61. data/lib/ferrum_mcp/whisper_service.rb +222 -0
  62. data/lib/ferrum_mcp.rb +35 -0
  63. metadata +248 -0
@@ -0,0 +1,254 @@
1
+ # Configuration Guide
2
+
3
+ This guide covers all configuration options for FerrumMCP.
4
+
5
+ ## Environment Variables
6
+
7
+ All configuration is done via environment variables. You can set them in a `.env` file in the project root or export them in your shell.
8
+
9
+ See `.env.example` for a complete example configuration.
10
+
11
+ ## Server Configuration
12
+
13
+ ### Basic Server Options
14
+
15
+ | Variable | Description | Default |
16
+ |----------|-------------|---------|
17
+ | `MCP_SERVER_HOST` | HTTP server host | `0.0.0.0` |
18
+ | `MCP_SERVER_PORT` | HTTP server port | `3000` |
19
+ | `LOG_LEVEL` | Logging level (debug/info/warn/error) | `info` |
20
+
21
+ ### Browser Defaults
22
+
23
+ | Variable | Description | Default |
24
+ |----------|-------------|---------|
25
+ | `BROWSER_HEADLESS` | Run browser in headless mode | `true` |
26
+ | `BROWSER_TIMEOUT` | Browser timeout in seconds | `60` |
27
+
28
+ ### Session Management
29
+
30
+ | Variable | Description | Default |
31
+ |----------|-------------|---------|
32
+ | `MAX_CONCURRENT_SESSIONS` | Maximum number of concurrent browser sessions | `10` |
33
+
34
+ **Note**: When the session limit is reached, new session creation will fail with an error. Close unused sessions to free up capacity.
35
+
36
+ ### Rate Limiting (HTTP Transport Only)
37
+
38
+ | Variable | Description | Default |
39
+ |----------|-------------|---------|
40
+ | `RATE_LIMIT_ENABLED` | Enable/disable rate limiting | `true` |
41
+ | `RATE_LIMIT_MAX_REQUESTS` | Maximum requests per time window | `100` |
42
+ | `RATE_LIMIT_WINDOW` | Time window in seconds | `60` |
43
+
44
+ **Note**: Rate limiting is applied per client IP address. When exceeded, HTTP 429 (Too Many Requests) is returned with a `Retry-After` header.
45
+
46
+ ## Multi-Browser Configuration
47
+
48
+ FerrumMCP supports multiple browser configurations using structured environment variables.
49
+
50
+ ### Browser Configuration Format
51
+
52
+ ```bash
53
+ BROWSER_<ID>=type:path:name:description
54
+ ```
55
+
56
+ **Parameters:**
57
+ - `<ID>`: Unique identifier (e.g., `CHROME`, `BOTBROWSER`, `EDGE`)
58
+ - `type`: Browser type (chrome, chromium, edge, brave, botbrowser)
59
+ - `path`: Path to browser executable (leave empty for system default)
60
+ - `name`: Human-readable name
61
+ - `description`: Optional description
62
+
63
+ **Examples:**
64
+
65
+ ```bash
66
+ # Use system Chrome
67
+ BROWSER_CHROME=chrome::Google Chrome:Standard browser
68
+
69
+ # Specify custom Chrome path
70
+ BROWSER_CHROME=chrome:/usr/bin/google-chrome:Google Chrome:Standard browser
71
+
72
+ # Microsoft Edge
73
+ BROWSER_EDGE=edge:/usr/bin/microsoft-edge:Microsoft Edge:Edge browser
74
+
75
+ # Brave Browser
76
+ BROWSER_BRAVE=brave:/Applications/Brave Browser.app/Contents/MacOS/Brave Browser:Brave:Privacy-focused browser
77
+
78
+ # BotBrowser (anti-detection)
79
+ BROWSER_BOTBROWSER=botbrowser:/opt/botbrowser/chrome:BotBrowser:Anti-detection browser
80
+ ```
81
+
82
+ ### Supported Browser Types
83
+
84
+ - `chrome` - Google Chrome
85
+ - `chromium` - Chromium
86
+ - `edge` - Microsoft Edge
87
+ - `brave` - Brave Browser
88
+ - `botbrowser` - BotBrowser (requires separate installation)
89
+
90
+ ## User Profile Configuration
91
+
92
+ Chrome user profiles allow you to maintain separate browsing contexts with different extensions, cookies, and settings.
93
+
94
+ ### Profile Configuration Format
95
+
96
+ ```bash
97
+ USER_PROFILE_<ID>=path:name:description
98
+ ```
99
+
100
+ **Examples:**
101
+
102
+ ```bash
103
+ USER_PROFILE_DEV=/home/user/.chrome-dev:Development:Dev profile with extensions
104
+ USER_PROFILE_TEST=/home/user/.chrome-test:Testing:Clean testing profile
105
+ USER_PROFILE_PROD=/home/user/.chrome-prod:Production:Production environment
106
+ ```
107
+
108
+ ## BotBrowser Profile Configuration
109
+
110
+ BotBrowser profiles contain fingerprinting configurations for anti-detection.
111
+
112
+ ### Profile Configuration Format
113
+
114
+ ```bash
115
+ BOT_PROFILE_<ID>=path:name:description
116
+ ```
117
+
118
+ **Examples:**
119
+
120
+ ```bash
121
+ # Encrypted profiles (recommended)
122
+ BOT_PROFILE_US=/profiles/us_chrome.enc:US Chrome:US-based Chrome fingerprint
123
+ BOT_PROFILE_EU=/profiles/eu_firefox.enc:EU Firefox:EU-based Firefox fingerprint
124
+ BOT_PROFILE_MOBILE=/profiles/android.enc:Android:Mobile Android fingerprint
125
+
126
+ # Unencrypted profiles
127
+ BOT_PROFILE_TEST=/profiles/test_profile.json:Test Profile:Testing profile
128
+ ```
129
+
130
+ **Note:** Profiles ending in `.enc` are automatically recognized as encrypted.
131
+
132
+ ## Resource Discovery
133
+
134
+ AI agents can discover available configurations through MCP Resources:
135
+
136
+ - `ferrum://browsers` - List all configured browsers
137
+ - `ferrum://user-profiles` - List all user profiles
138
+ - `ferrum://bot-profiles` - List all BotBrowser profiles
139
+ - `ferrum://capabilities` - Server capabilities
140
+
141
+ This allows AI to dynamically select the appropriate browser/profile for each task.
142
+
143
+ ## Legacy Configuration (Deprecated)
144
+
145
+ For backward compatibility, these variables still work but are deprecated:
146
+
147
+ ```bash
148
+ BROWSER_PATH=/usr/bin/google-chrome # Creates browser with id "default"
149
+ BOTBROWSER_PATH=/opt/botbrowser/chrome # Creates BotBrowser with id "default"
150
+ BOTBROWSER_PROFILE=/profiles/profile.enc # Creates bot profile with id "default"
151
+ ```
152
+
153
+ **Recommendation:** Use the new multi-configuration format for better flexibility.
154
+
155
+ ## Session Configuration
156
+
157
+ ### Session Limits (Coming in v1.1)
158
+
159
+ ```bash
160
+ MAX_CONCURRENT_SESSIONS=10 # Maximum concurrent browser sessions
161
+ SESSION_IDLE_TIMEOUT=1800 # Auto-close after 30 minutes (seconds)
162
+ SESSION_CLEANUP_INTERVAL=300 # Cleanup check every 5 minutes (seconds)
163
+ ```
164
+
165
+ **Note:** Session limits are currently hardcoded but will be configurable in v1.1.
166
+
167
+ ## Whisper Configuration (CAPTCHA Solving)
168
+
169
+ ```bash
170
+ WHISPER_MODEL=base # Model: tiny, base, small, medium
171
+ WHISPER_MODELS_PATH=~/.whisper.cpp/models/ # Model storage location
172
+ ```
173
+
174
+ Models are automatically downloaded on first use.
175
+
176
+ ## Example Complete Configuration
177
+
178
+ ```bash
179
+ # Server
180
+ MCP_SERVER_HOST=0.0.0.0
181
+ MCP_SERVER_PORT=3000
182
+ LOG_LEVEL=info
183
+
184
+ # Browser Defaults
185
+ BROWSER_HEADLESS=true
186
+ BROWSER_TIMEOUT=60
187
+
188
+ # Browsers
189
+ BROWSER_CHROME=chrome::Google Chrome:Standard browser
190
+ BROWSER_BOTBROWSER=botbrowser:/opt/botbrowser/chrome:BotBrowser:Anti-detection
191
+
192
+ # User Profiles
193
+ USER_PROFILE_DEV=/home/user/.chrome-dev:Development:Dev profile
194
+ USER_PROFILE_PROD=/home/user/.chrome-prod:Production:Prod profile
195
+
196
+ # BotBrowser Profiles
197
+ BOT_PROFILE_US=/profiles/us.enc:US Chrome:US fingerprint
198
+ BOT_PROFILE_EU=/profiles/eu.enc:EU Firefox:EU fingerprint
199
+
200
+ # Whisper (CAPTCHA)
201
+ WHISPER_MODEL=base
202
+ ```
203
+
204
+ ## Configuration Validation
205
+
206
+ FerrumMCP validates all configuration at startup:
207
+
208
+ - Browser paths are checked for existence
209
+ - Profile paths are verified
210
+ - Invalid configurations log warnings
211
+ - Fallbacks to defaults when possible
212
+
213
+ Check logs at `logs/ferrum_mcp.log` for validation messages.
214
+
215
+ ## Using Configuration in Sessions
216
+
217
+ When creating sessions, you can reference configured browsers and profiles:
218
+
219
+ ```ruby
220
+ # Use configured browser by ID
221
+ create_session(browser_id: "botbrowser")
222
+
223
+ # Use configured user profile
224
+ create_session(browser_id: "chrome", user_profile_id: "dev")
225
+
226
+ # Use configured bot profile
227
+ create_session(browser_id: "botbrowser", bot_profile_id: "us")
228
+
229
+ # Custom configuration (overrides defaults)
230
+ create_session(
231
+ browser_id: "chrome",
232
+ headless: false,
233
+ timeout: 120,
234
+ browser_options: { '--window-size': '1920,1080' }
235
+ )
236
+ ```
237
+
238
+ ## Docker Configuration
239
+
240
+ When using Docker, pass environment variables with `-e` or `--env-file`:
241
+
242
+ ```bash
243
+ # Individual variables
244
+ docker run -e BROWSER_HEADLESS=false -p 3000:3000 eth3rnit3/ferrum-mcp
245
+
246
+ # From .env file
247
+ docker run --env-file .env -p 3000:3000 eth3rnit3/ferrum-mcp
248
+ ```
249
+
250
+ ## Next Steps
251
+
252
+ - See [Getting Started](GETTING_STARTED.md) for basic setup
253
+ - Read [API Reference](API_REFERENCE.md) for tool documentation
254
+ - Check [BotBrowser Integration](BOTBROWSER_INTEGRATION.md) for anti-detection setup