brave-real-browser-mcp-server 2.29.0 → 2.30.0

package/README.md

@@ -107,6 +107,115 @@ npm run dev
 
 ---
 
+## 🆕 New in v2.28.1 - Advanced Enhancements
+
+### 🌐 Advanced Navigation
+| Option | Description |
+|--------|-------------|
+| `blockResources` | Block images, fonts, CSS for faster loading |
+| `customHeaders` | Set custom HTTP headers |
+| `referrer` | Custom referrer URL |
+| `waitForSelector` | Wait for specific element after load |
+| `waitForContent` | Wait for specific text content |
+| `scrollToBottom` | Auto-scroll for lazy loading |
+| `randomDelay` | Human-like delay (100-500ms) |
+| `bypassCSP` | Bypass Content Security Policy |
+
+**Example:**
+```json
+{
+  "url": "https://example.com",
+  "blockResources": ["image", "font"],
+  "waitForSelector": ".main-content",
+  "scrollToBottom": true,
+  "randomDelay": true
+}
+```
+
+### 🚀 Parallel Scraping (js_scrape)
+| Option | Description |
+|--------|-------------|
+| `urls` | Array of multiple URLs to scrape |
+| `concurrency` | Max concurrent scrapes (1-10) |
+| `continueOnError` | Continue even if some URLs fail |
+| `delayBetween` | Delay between scrapes (ms) |
+
+**Example:**
+```json
+{
+  "urls": ["https://site1.com", "https://site2.com", "https://site3.com"],
+  "concurrency": 3,
+  "extractSelector": "article"
+}
+```
+
+### 📄 Auto-Pagination (link_harvester)
+| Option | Description |
+|--------|-------------|
+| `followPagination` | Auto-follow pagination links |
+| `maxPages` | Maximum pages to scrape (1-20) |
+| `paginationSelector` | Custom next page selector |
+| `delayBetweenPages` | Delay between pages (ms) |
+
+**Example:**
+```json
+{
+  "followPagination": true,
+  "maxPages": 10,
+  "filter": "movie"
+}
+```
+
+### 🔌 API Interceptor (network_recorder)
+| Option | Description |
+|--------|-------------|
+| `interceptMode` | `record`, `intercept`, or `mock` |
+| `blockPatterns` | URL patterns to block |
+| `mockResponses` | Fake responses for URLs |
+| `modifyHeaders` | Modify request headers |
+| `capturePayloads` | Capture POST/PUT bodies |
+
+**Example:**
+```json
+{
+  "interceptMode": "intercept",
+  "blockPatterns": ["ads", "tracking"],
+  "capturePayloads": true
+}
+```
+
+### 🛡️ Anti-Detection Enhancements
+
+| Feature | Description |
+|---------|-------------|
+| **Fingerprint Randomizer** | Canvas, Audio, Hardware randomization |
+| **Human Behavior Simulation** | Natural mouse/scroll patterns |
+| **WebGL Spoofing** | Random GPU from 8 configs (Intel, NVIDIA, AMD) |
+| **Proxy Rotation** | round-robin, random, least-used, on-error strategies |
+| **Rate Limiter** | Per-second/minute limits, domain-specific |
+| **Error Auto-Recovery** | Smart retry with different strategies |
+
+### 🔄 Internal Systems
+
+```javascript
+// Rate Limiter
+initRateLimiter({ requestsPerSecond: 5, requestsPerMinute: 100 });
+setDomainRateLimit('api.example.com', 2); // 2 req/sec for this domain
+
+// Proxy Rotation
+initProxyRotation(['proxy1:8080', 'proxy2:8080'], 'round-robin');
+rotateProxy('error'); // Rotate on error
+
+// Error Recovery
+executeWithRecovery(operation, {
+  toolName: 'navigate',
+  page: pageInstance,
+  restartBrowser: () => browser.restart()
+});
+```
+
+---
+
 ## 🌐 Unified MCP+LSP+SSE Ecosystem
 
 **एक command से सब active:**
@@ -117,7 +226,7 @@ npm run dev
 
 ### Output:
 ```
-🦁 Brave Real Browser - Unified Server v2.22.0
+🦁 Brave Real Browser - Unified Server v2.28.1
 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
 📡 HTTP Server: http://localhost:3000
 
@@ -185,14 +294,14 @@ brave-real-launcher
 ### Navigation
 | Tool | Description |
 |------|-------------|
-| `navigate` | Navigate to URL |
+| `navigate` | Navigate to URL with advanced options (resource blocking, custom headers, auto-scroll) |
 | `wait` | Wait for conditions |
 
 ### Content
 | Tool | Description |
 |------|-------------|
 | `get_content` | Get page HTML/text |
-| `find_element` | Find by selector/text/AI |
+| `find_element` | Find by selector/text/AI with batch mode |
 | `save_content_as_markdown` | Save as markdown file |
 
 ### Interaction
@@ -204,34 +313,32 @@ brave-real-launcher
 | `random_scroll` | Natural scrolling |
 | `solve_captcha` | Solve CAPTCHAs |
 
-### Media Extraction
+### Media & Streaming
+| Tool | Description |
+|------|-------------|
+| `stream_extractor` | Extract streams with multi-quality selector, VidSrc/Filemoon/StreamWish support |
+| `player_api_hook` | Hook into JWPlayer, Video.js, HLS.js, Plyr, Vidstack, DASH.js |
+| `iframe_handler` | Deep iframe scraping with video source extraction |
+
+### Scraping
 | Tool | Description |
 |------|-------------|
-| `media_extractor` | Extract video/audio |
-| `m3u8_parser` | Parse HLS streams |
-| `stream_extractor` | Direct download URLs |
+| `js_scrape` | JavaScript-rendered scraping with parallel URL support |
+| `link_harvester` | Harvest links with auto-pagination |
+| `network_recorder` | Record traffic with API interception |
+| `extract_json` | Extract embedded JSON with AES decryption |
+| `search_regex` | Regex search like regex101.com |
 
 ### Advanced
 | Tool | Description |
 |------|-------------|
-| `search_content` | Search patterns |
-| `extract_json` | Extract embedded JSON |
 | `scrape_meta_tags` | Meta/OG tags |
-| `deep_analysis` | Full page analysis |
-| `network_recorder` | Record traffic |
-| `api_finder` | Discover APIs |
-| `ajax_content_waiter` | Wait for AJAX |
-| `link_harvester` | Harvest links |
-| `batch_element_scraper` | Batch scrape |
-| `extract_schema` | Schema.org data |
-| `element_screenshot` | Screenshot element |
-| `breadcrumb_navigator` | Navigate breadcrumbs |
+| `deep_analysis` | Full page analysis with screenshot |
 | `redirect_tracer` | Trace redirects |
 | `progress_tracker` | Track progress |
 | `cookie_manager` | Manage cookies |
 | `file_downloader` | Download files |
-| `iframe_handler` | Handle iframes |
-| `popup_handler` | Handle popups |
+| `execute_js` | Run custom JavaScript |
 
 ---
 

package/dist/handlers/advanced-tools.js

@@ -5962,3 +5962,321 @@ export async function handlePlayerApiHook(page, args) {
         };
     }
 }
+// Field type detection patterns
+const FIELD_PATTERNS = {
+    email: [/email/i, /e-mail/i, /mail/i],
+    password: [/password/i, /pass/i, /pwd/i],
+    username: [/username/i, /user/i, /login/i, /uname/i],
+    firstName: [/first.?name/i, /fname/i, /given.?name/i],
+    lastName: [/last.?name/i, /lname/i, /surname/i, /family.?name/i],
+    fullName: [/full.?name/i, /name/i],
+    phone: [/phone/i, /tel/i, /mobile/i, /cell/i],
+    address: [/address/i, /street/i, /addr/i],
+    city: [/city/i, /town/i],
+    state: [/state/i, /province/i, /region/i],
+    zip: [/zip/i, /postal/i, /postcode/i],
+    country: [/country/i, /nation/i],
+    cardNumber: [/card.?number/i, /cc.?num/i, /credit.?card/i],
+    cvv: [/cvv/i, /cvc/i, /security.?code/i],
+    expiry: [/expir/i, /exp.?date/i],
+    search: [/search/i, /query/i, /q/i],
+    message: [/message/i, /comment/i, /feedback/i, /note/i],
+    company: [/company/i, /organization/i, /org/i],
+    website: [/website/i, /url/i, /web/i],
+    age: [/age/i, /birth/i, /dob/i],
+};
+// Detect field type from attributes
+function detectFieldType(attributes) {
+    const { name = '', id = '', type = '', placeholder = '', label = '', autocomplete = '' } = attributes;
+    const combined = `${name} ${id} ${type} ${placeholder} ${label} ${autocomplete}`.toLowerCase();
+    for (const [fieldType, patterns] of Object.entries(FIELD_PATTERNS)) {
+        for (const pattern of patterns) {
+            if (pattern.test(combined)) {
+                return fieldType;
+            }
+        }
+    }
+    return 'unknown';
+}
+export async function handleFormAutomator(page, args) {
+    const action = args.action || 'fill';
+    const formSelector = args.formSelector || 'form';
+    const humanLike = args.humanLike !== false;
+    const clearFields = args.clearFields !== false;
+    const delayBetweenFields = args.delayBetweenFields || 500;
+    try {
+        // Get form fields
+        const getFormFields = async () => {
+            return await page.evaluate((selector) => {
+                const form = document.querySelector(selector);
+                if (!form)
+                    return [];
+                const inputs = form.querySelectorAll('input, textarea, select');
+                const fields = [];
+                inputs.forEach((input, index) => {
+                    // Find associated label
+                    let label = '';
+                    if (input.id) {
+                        const labelEl = document.querySelector(`label[for="${input.id}"]`);
+                        if (labelEl)
+                            label = labelEl.textContent?.trim() || '';
+                    }
+                    if (!label) {
+                        const parent = input.closest('label');
+                        if (parent)
+                            label = parent.textContent?.replace(input.value || '', '').trim() || '';
+                    }
+                    fields.push({
+                        index,
+                        tagName: input.tagName.toLowerCase(),
+                        type: input.type || 'text',
+                        name: input.name || '',
+                        id: input.id || '',
+                        placeholder: input.placeholder || '',
+                        label,
+                        autocomplete: input.autocomplete || '',
+                        required: input.required,
+                        value: input.value || '',
+                        visible: input.offsetWidth > 0 && input.offsetHeight > 0,
+                        selector: input.id ? `#${input.id}` :
+                            input.name ? `[name="${input.name}"]` :
+                                `${selector} ${input.tagName.toLowerCase()}:nth-of-type(${index + 1})`
+                    });
+                });
+                return fields;
+            }, formSelector);
+        };
+        // Human-like typing function
+        const humanType = async (selector, text) => {
+            const element = await page.$(selector);
+            if (!element)
+                throw new Error(`Element not found: ${selector}`);
+            // Focus and clear field
+            await element.click();
+            await page.keyboard.down('Control');
+            await page.keyboard.press('a');
+            await page.keyboard.up('Control');
+            await page.keyboard.press('Backspace');
+            const minDelay = 50;
+            const maxDelay = 150;
+            const errorRate = 0.02;
+            for (let i = 0; i < text.length; i++) {
+                const char = text[i];
+                // Random delay between keystrokes
+                const delay = minDelay + Math.random() * (maxDelay - minDelay);
+                await new Promise(r => setTimeout(r, delay));
+                // Simulate occasional typo
+                if (Math.random() < errorRate && i > 0 && i < text.length - 1) {
+                    const wrongChar = String.fromCharCode(char.charCodeAt(0) + (Math.random() > 0.5 ? 1 : -1));
+                    await page.keyboard.type(wrongChar);
+                    await new Promise(r => setTimeout(r, 200));
+                    await page.keyboard.press('Backspace');
+                    await new Promise(r => setTimeout(r, 50 + Math.random() * 50));
+                }
+                await page.keyboard.type(char);
+            }
+        };
+        // Handle different actions
+        switch (action) {
+            case 'getFields': {
+                const fields = await getFormFields();
+                const detectedFields = fields.map(f => ({
+                    ...f,
+                    detectedType: detectFieldType(f)
+                }));
+                return {
+                    success: true,
+                    formFound: fields.length > 0,
+                    totalFields: fields.length,
+                    visibleFields: fields.filter(f => f.visible).length,
+                    fields: detectedFields,
+                    message: `Found ${fields.length} fields in form`
+                };
+            }
+            case 'fillField': {
+                if (!args.fieldSelector || !args.fieldValue) {
+                    throw new Error('fieldSelector and fieldValue are required for fillField action');
+                }
+                if (humanLike) {
+                    await humanType(args.fieldSelector, args.fieldValue);
+                }
+                else {
+                    if (clearFields) {
+                        await page.$eval(args.fieldSelector, (el) => el.value = '');
+                    }
+                    await page.type(args.fieldSelector, args.fieldValue);
+                }
+                return {
+                    success: true,
+                    field: args.fieldSelector,
+                    value: args.fieldValue,
+                    message: `Field filled successfully`
+                };
+            }
+            case 'submit': {
+                const submitSelector = args.submitSelector || 'button[type="submit"], input[type="submit"]';
+                const waitForNav = args.waitForNavigation !== false;
+                const submitButton = await page.$(submitSelector);
+                if (waitForNav) {
+                    const navigationPromise = page.waitForNavigation({ timeout: 30000 }).catch(() => null);
+                    if (submitButton) {
+                        await submitButton.click();
+                    }
+                    else {
+                        await page.$eval(formSelector, (f) => f.submit());
+                    }
+                    await navigationPromise;
+                }
+                else {
+                    if (submitButton) {
+                        await submitButton.click();
+                    }
+                    else {
+                        await page.$eval(formSelector, (f) => f.submit());
+                    }
+                }
+                return {
+                    success: true,
+                    submitted: true,
+                    currentUrl: page.url(),
+                    message: 'Form submitted successfully'
+                };
+            }
+            case 'autoFill': {
+                // Default test data
+                const defaultData = {
+                    email: 'test@example.com',
+                    password: 'SecurePass123!',
+                    username: 'testuser',
+                    firstName: 'John',
+                    lastName: 'Doe',
+                    fullName: 'John Doe',
+                    phone: '+1-555-123-4567',
+                    address: '123 Main Street',
+                    city: 'New York',
+                    state: 'NY',
+                    zip: '10001',
+                    country: 'United States',
+                    company: 'Acme Corp',
+                    website: 'https://example.com',
+                    message: 'This is a test message.',
+                    ...args.data
+                };
+                args.data = defaultData;
+                // Fall through to fill action
+            }
+            case 'fill':
+            default: {
+                const data = args.data || {};
+                const fields = await getFormFields();
+                const filledFields = [];
+                const errors = [];
+                for (const field of fields) {
+                    if (!field.visible)
+                        continue;
+                    // Detect field type
+                    const fieldType = detectFieldType(field);
+                    // Find matching data
+                    let value = null;
+                    if (data[field.name])
+                        value = data[field.name];
+                    else if (data[field.id])
+                        value = data[field.id];
+                    else if (data[fieldType])
+                        value = data[fieldType];
+                    if (!value)
+                        continue;
+                    try {
+                        if (field.tagName === 'select') {
+                            await page.select(field.selector, value);
+                        }
+                        else if (field.type === 'checkbox') {
+                            const isChecked = await page.$eval(field.selector, (el) => el.checked);
+                            if ((value === true || value === 'true' || value === '1') && !isChecked) {
+                                await page.click(field.selector);
+                            }
+                            else if ((value === false || value === 'false' || value === '0') && isChecked) {
+                                await page.click(field.selector);
+                            }
+                        }
+                        else if (field.type === 'radio') {
+                            const radioSelector = `${field.selector}[value="${value}"]`;
+                            await page.click(radioSelector);
+                        }
+                        else {
+                            if (humanLike) {
+                                await humanType(field.selector, String(value));
+                            }
+                            else {
+                                if (clearFields) {
+                                    await page.$eval(field.selector, (el) => el.value = '');
+                                }
+                                await page.type(field.selector, String(value));
+                            }
+                        }
+                        filledFields.push({
+                            field: field.name || field.id || field.selector,
+                            type: fieldType,
+                            success: true
+                        });
+                        // Delay between fields
+                        if (humanLike) {
+                            await new Promise(r => setTimeout(r, delayBetweenFields * (0.5 + Math.random())));
+                        }
+                    }
+                    catch (err) {
+                        errors.push({
+                            field: field.name || field.id || field.selector,
+                            error: err.message
+                        });
+                    }
+                }
+                // Submit if requested
+                let submitted = false;
+                if (args.submitAfter) {
+                    try {
+                        const submitSelector = args.submitSelector || 'button[type="submit"], input[type="submit"]';
+                        const submitButton = await page.$(submitSelector);
+                        if (args.waitForNavigation !== false) {
+                            const navigationPromise = page.waitForNavigation({ timeout: 30000 }).catch(() => null);
+                            if (submitButton) {
+                                await submitButton.click();
+                            }
+                            else {
+                                await page.$eval(formSelector, (f) => f.submit());
+                            }
+                            await navigationPromise;
+                        }
+                        else {
+                            if (submitButton) {
+                                await submitButton.click();
+                            }
+                            else {
+                                await page.$eval(formSelector, (f) => f.submit());
+                            }
+                        }
+                        submitted = true;
+                    }
+                    catch (err) {
+                        errors.push({ field: 'submit', error: err.message });
+                    }
+                }
+                return {
+                    success: errors.length === 0,
+                    filledFields,
+                    errors,
+                    submitted,
+                    totalFields: fields.filter(f => f.visible).length,
+                    currentUrl: page.url(),
+                    message: `Filled ${filledFields.length} fields${submitted ? ' and submitted form' : ''}`
+                };
+            }
+        }
+    }
+    catch (error) {
+        return {
+            success: false,
+            message: `Form automator error: ${error instanceof Error ? error.message : String(error)}`
+        };
+    }
+}

package/dist/index.js

@@ -67,7 +67,9 @@ handleFileDownloader,
 // Enhanced streaming/download tools
 handleIframeHandler, handleStreamExtractor, handleJsScrape, 
 // New JS extraction tools
-handleExecuteJs, handlePlayerApiHook, } from './handlers/advanced-tools.js';
+handleExecuteJs, handlePlayerApiHook, 
+// Form automation
+handleFormAutomator, } from './handlers/advanced-tools.js';
 // State for video recording
 const recorderState = new Map();
 debug('All modules loaded successfully');
@@ -256,6 +258,10 @@ server.setRequestHandler(CallToolRequestSchema, async (request) => {
                 if (!page)
                     throw new Error('Browser not initialized. Call browser_init first.');
                 return { content: [{ type: 'text', text: JSON.stringify(await handlePlayerApiHook(page, args)) }] };
+            case TOOL_NAMES.FORM_AUTOMATOR:
+                if (!page)
+                    throw new Error('Browser not initialized. Call browser_init first.');
+                return { content: [{ type: 'text', text: JSON.stringify(await handleFormAutomator(page, args)) }] };
             default:
                 throw new Error(`Unknown tool: ${name}`);
         }

package/dist/tool-definitions.js

@@ -914,6 +914,71 @@ export const TOOLS = [
             },
         },
     },
+    // Form Automator Tool
+    {
+        name: 'form_automator',
+        description: 'Auto-fill forms with human-like typing, field detection, and smart form submission. Supports auto-detection of field types (email, password, name, phone, address, etc.)',
+        inputSchema: {
+            type: 'object',
+            additionalProperties: false,
+            properties: {
+                action: {
+                    type: 'string',
+                    enum: ['fill', 'autoFill', 'getFields', 'submit', 'fillField'],
+                    description: 'Action to perform: fill (with custom data), autoFill (with test data), getFields (detect fields), submit, fillField (single field)',
+                    default: 'fill'
+                },
+                formSelector: {
+                    type: 'string',
+                    description: 'CSS selector for the form (default: "form")',
+                    default: 'form'
+                },
+                data: {
+                    type: 'object',
+                    description: 'Data to fill in form. Keys can be field names, IDs, or types (email, password, username, firstName, lastName, phone, address, etc.)',
+                    additionalProperties: true
+                },
+                fieldSelector: {
+                    type: 'string',
+                    description: 'CSS selector for specific field (used with fillField action)'
+                },
+                fieldValue: {
+                    type: 'string',
+                    description: 'Value to fill in specific field (used with fillField action)'
+                },
+                humanLike: {
+                    type: 'boolean',
+                    description: 'Use human-like typing with variable delays and occasional typos',
+                    default: true
+                },
+                clearFields: {
+                    type: 'boolean',
+                    description: 'Clear existing field values before filling',
+                    default: true
+                },
+                submitAfter: {
+                    type: 'boolean',
+                    description: 'Submit form after filling',
+                    default: false
+                },
+                submitSelector: {
+                    type: 'string',
+                    description: 'CSS selector for submit button',
+                    default: 'button[type="submit"], input[type="submit"]'
+                },
+                waitForNavigation: {
+                    type: 'boolean',
+                    description: 'Wait for page navigation after submit',
+                    default: true
+                },
+                delayBetweenFields: {
+                    type: 'number',
+                    description: 'Delay in ms between filling fields (for human-like behavior)',
+                    default: 500
+                }
+            }
+        },
+    },
 ];
 // Tool name constants for type safety
 export const TOOL_NAMES = {
@@ -956,6 +1021,8 @@ export const TOOL_NAMES = {
     // New JS extraction tools
     EXECUTE_JS: 'execute_js',
     PLAYER_API_HOOK: 'player_api_hook',
+    // Form automation
+    FORM_AUTOMATOR: 'form_automator',
 };
 // Tool categories for organization
 export const TOOL_CATEGORIES = {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "brave-real-browser-mcp-server",
-  "version": "2.29.0",
+  "version": "2.30.0",
   "description": "🦁 MCP server for Brave Real Browser - NPM Workspaces Monorepo with anti-detection features, SSE streaming, and LSP compatibility",
   "type": "module",
   "main": "dist/index.js",
@@ -50,7 +50,7 @@
   "dependencies": {
     "@modelcontextprotocol/sdk": "latest",
     "@types/turndown": "latest",
-    "brave-real-browser": "^2.9.0",
+    "brave-real-browser": "^2.10.0",
     "puppeteer-core": "^24.35.0",
     "turndown": "latest",
     "vscode-languageserver": "^9.0.1",