@monostate/browsernative-client 1.2.1 → 2.0.0

package/README.md

@@ -1,461 +1,129 @@
 # @monostate/browsernative-client
 
-> **Official JavaScript/TypeScript client for Browser Native API**
+> JavaScript/TypeScript client for the Browser Native web scraping API
 
-[![npm version](https://badge.fury.io/js/%40monostate%2Fbrowsernative-client.svg)](https://badge.fury.io/js/%40monostate%2Fbrowsernative-client)
+[![npm](https://img.shields.io/npm/v/@monostate/browsernative-client.svg)](https://www.npmjs.com/package/@monostate/browsernative-client)
+[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/monostate/browsernative-client/blob/main/LICENSE)
 [![TypeScript](https://img.shields.io/badge/TypeScript-Ready-blue.svg)](https://www.typescriptlang.org/)
-[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](../../LICENSE)
 
-A lightweight, fast, and reliable client for the Browser Native web scraping and content extraction API. Works in browsers, Node.js, Deno, and edge environments.
+Zero-dependency client for scraping, screenshots, and AI analysis. Works in Node.js, browsers, and edge environments.
 
-## 🚀 Quick Start
-
-### Installation
+## Install
 
 ```bash
 npm install @monostate/browsernative-client
-# or
-yarn add @monostate/browsernative-client
-# or  
-pnpm add @monostate/browsernative-client
 ```
 
-### Get Your API Key
-
-1. Sign up at [bnca.monostate.ai](https://bnca.monostate.ai)
-2. Get your API key from the dashboard
-3. Start scraping!
+Get an API key at [bnca.monostate.ai](https://bnca.monostate.ai).
 
-### Basic Usage
+## Usage
 
 ```javascript
 import { BrowserNativeClient } from '@monostate/browsernative-client';
 
 const client = new BrowserNativeClient('your-api-key');
 
-// Scrape any website
+// Scrape
 const result = await client.scrape('https://example.com');
-console.log(result.data.title);
-console.log(result.data.content);
-
-// Quick screenshot capture (optimized for speed)
-const screenshot = await client.quickshot('https://example.com');
-console.log(screenshot.screenshot); // Base64 image
-```
-
-### Quick Functions
-
-```javascript
-import { quickScrape, quickScreenshot, quickShot, quickAnalyze } from '@monostate/browsernative-client';
-
-// One-line scraping
-const content = await quickScrape('https://example.com', 'your-api-key');
-
-// Take a screenshot (with content extraction)
-const screenshot = await quickScreenshot('https://example.com', 'your-api-key');
-
-// Quick screenshot only (fastest option)
-const quickScreenshot = await quickShot('https://example.com', 'your-api-key');
-
-// AI-powered analysis
-const analysis = await quickAnalyze(
-  'https://news.ycombinator.com', 
-  'What are the top 3 trending topics?',
-  'your-api-key'
-);
-```
-
-## 📋 API Reference
-
-### Client Initialization
-
-```javascript
-const client = new BrowserNativeClient(apiKey, options);
-```
-
-**Options:**
-- `baseUrl` (string): API base URL (default: `https://bnca-api.fly.dev`)
-- `timeout` (number): Request timeout in ms (default: `30000`)
-- `retries` (number): Number of retry attempts (default: `2`)
-- `verbose` (boolean): Enable logging (default: `false`)
-
-### Methods
-
-#### `client.scrape(url, options)`
-
-Extract structured content from any webpage.
-
-```javascript
-const result = await client.scrape('https://example.com', {
-  includeScreenshot: true,
-  waitForSelector: '.main-content',
-  extractMetadata: true,
-  userAgent: 'Custom Bot 1.0'
-});
-
-console.log(result.data.title);      // Page title
-console.log(result.data.content);   // Main content
-console.log(result.data.metadata);  // Meta tags, etc.
-console.log(result.screenshot);     // Base64 image (if requested)
-console.log(result.method);         // Scraping method used
-```
-
-**Options:**
-- `includeScreenshot` (boolean): Include page screenshot
-- `waitForSelector` (string): CSS selector to wait for
-- `userAgent` (string): Custom user agent
-- `viewport` (object): `{ width: number, height: number }`
-- `extractMetadata` (boolean): Extract meta tags and structured data
-
-#### `client.screenshot(url, options)`
-
-Take high-quality screenshots of webpages with content extraction.
-
-```javascript
-const result = await client.screenshot('https://example.com', {
-  fullPage: true,
-  format: 'png',
-  viewport: { width: 1920, height: 1080 }
-});
-
-// Use the base64 image
-const img = `data:image/png;base64,${result.screenshot}`;
-```
-
-**Options:**
-- `fullPage` (boolean): Capture full page scroll
-- `format` (string): `'png'`, `'jpeg'`, or `'webp'`
-- `quality` (number): JPEG quality (1-100)
-- All scrape options are also available
-
-#### `client.quickshot(url, options)`
-
-Optimized screenshot capture for maximum speed (no content extraction).
-
-```javascript
-const result = await client.quickshot('https://example.com');
-
-// Returns screenshot immediately
-if (result.success && result.screenshot) {
-  const img = result.screenshot; // Already includes data:image/png;base64,
-}
-```
-
-**Benefits:**
-- 2-3x faster than regular screenshot
-- Optimized for visual capture only
-- Perfect for thumbnails and previews
-
-#### `client.analyze(url, question, options)`
-
-AI-powered content analysis and question answering.
-
-```javascript
-const result = await client.analyze(
-  'https://techcrunch.com',
-  'What are the latest AI developments mentioned?',
-  {
-    language: 'en',
-    style: 'detailed'
-  }
-);
-
-console.log(result.analysis.answer);     // AI response
-console.log(result.analysis.confidence); // Confidence score
-```
-
-**Options:**
-- `language` (string): Response language (`'en'`, `'pt'`, `'es'`, `'fr'`, `'de'`, `'auto'`)
-- `style` (string): Response style (`'concise'`, `'detailed'`, `'technical'`)
-- All scrape options are also available
-
-#### `client.getUsage(days)`
-
-Get your account usage statistics.
-
-```javascript
-const usage = await client.getUsage(30); // Last 30 days
-
-console.log(usage.data.monthlyTotal.totalRequests);
-console.log(usage.data.currentUsage);
-console.log(usage.data.usageLimit);
-```
-
-#### `client.healthCheck()`
-
-Check API status and your account health.
-
-```javascript
-const health = await client.healthCheck();
-
-console.log(health.data.status);   // 'healthy', 'degraded', or 'unhealthy'
-console.log(health.data.services); // Service status breakdown
-```
-
-## 🌐 Framework Integration
-
-### React Hook
-
-```javascript
-// hooks/useBrowserNative.js
-import { useState } from 'react';
-import { BrowserNativeClient } from '@monostate/browsernative-client';
 
-export function useBrowserNative(apiKey) {
-  const [loading, setLoading] = useState(false);
-  const [data, setData] = useState(null);
-  const [error, setError] = useState(null);
-
-  const client = new BrowserNativeClient(apiKey);
-
-  const scrape = async (url, options = {}) => {
-    setLoading(true);
-    setError(null);
-    
-    try {
-      const result = await client.scrape(url, options);
-      if (result.success) {
-        setData(result.data);
-      } else {
-        setError(result.error);
-      }
-    } catch (err) {
-      setError(err.message);
-    } finally {
-      setLoading(false);
-    }
-  };
-
-  return { scrape, loading, data, error };
-}
-```
+// Force a specific scraping method
+const result = await client.scrape('https://example.com', { method: 'lightpanda' });
 
-### Vue Composable
+// Screenshot
+const screenshot = await client.screenshot('https://example.com');
 
-```javascript
-// composables/useBrowserNative.js
-import { ref } from 'vue';
-import { BrowserNativeClient } from '@monostate/browsernative-client';
+// Quick screenshot (faster, no content extraction)
+const quick = await client.quickshot('https://example.com');
 
-export function useBrowserNative(apiKey) {
-  const loading = ref(false);
-  const data = ref(null);
-  const error = ref(null);
-
-  const client = new BrowserNativeClient(apiKey);
-
-  const scrape = async (url, options = {}) => {
-    loading.value = true;
-    error.value = null;
-    
-    try {
-      const result = await client.scrape(url, options);
-      if (result.success) {
-        data.value = result.data;
-      } else {
-        error.value = result.error;
-      }
-    } catch (err) {
-      error.value = err.message;
-    } finally {
-      loading.value = false;
-    }
-  };
-
-  return { scrape, loading, data, error };
-}
+// AI Q&A
+const analysis = await client.analyze('https://example.com', 'What is this site about?');
 ```
 
-### Next.js API Route
+### Convenience functions
 
 ```javascript
-// pages/api/scrape.js
-import { BrowserNativeClient } from '@monostate/browsernative-client';
+import { quickScrape, quickScreenshot, quickShot, quickAnalyze, bulkScrape } from '@monostate/browsernative-client';
 
-const client = new BrowserNativeClient(process.env.BROWSER_NATIVE_API_KEY);
-
-export default async function handler(req, res) {
-  if (req.method !== 'POST') {
-    return res.status(405).json({ error: 'Method not allowed' });
-  }
-
-  try {
-    const { url, question } = req.body;
-    
-    let result;
-    if (question) {
-      result = await client.analyze(url, question);
-    } else {
-      result = await client.scrape(url);
-    }
-    
-    res.status(200).json(result);
-  } catch (error) {
-    res.status(500).json({ error: error.message });
-  }
-}
+const content = await quickScrape('https://example.com', 'your-api-key');
+const shot = await quickShot('https://example.com', 'your-api-key');
+const answer = await quickAnalyze('https://example.com', 'What is this?', 'your-api-key');
 ```
 
-### Express.js
+### Bulk scraping
 
 ```javascript
-import express from 'express';
-import { BrowserNativeClient } from '@monostate/browsernative-client';
+const urls = ['https://site1.com', 'https://site2.com', 'https://site3.com'];
 
-const app = express();
-const client = new BrowserNativeClient(process.env.BROWSER_NATIVE_API_KEY);
-
-app.post('/scrape', async (req, res) => {
-  try {
-    const { url } = req.body;
-    const result = await client.scrape(url);
-    res.json(result);
-  } catch (error) {
-    res.status(500).json({ error: error.message });
-  }
+const { results, stats } = await client.bulkScrape(urls, {
+  concurrency: 5,
+  continueOnError: true,
+  progressCallback: (p) => console.log(`${p.percentage.toFixed(1)}%`),
 });
-```
 
-## 🔒 Environment Variables
-
-Create a `.env` file:
-
-```bash
-BROWSER_NATIVE_API_KEY=your_api_key_here
+console.log(`${stats.successful}/${stats.total} succeeded in ${stats.totalTime}ms`);
 ```
 
-Then use in your code:
+## API Reference
 
-```javascript
-const client = new BrowserNativeClient(process.env.BROWSER_NATIVE_API_KEY);
-```
-
-## 📱 Browser Usage
-
-You can use this client directly in browsers, but be careful with API keys:
-
-```html
-<!DOCTYPE html>
-<html>
-<head>
-    <title>Browser Native Demo</title>
-</head>
-<body>
-    <script type="module">
-        import { BrowserNativeClient } from 'https://cdn.skypack.dev/@monostate/browsernative-client';
-        
-        // ⚠️ Never expose your API key in client-side code!
-        // Use a proxy server or environment variables
-        const client = new BrowserNativeClient(await getApiKeyFromServer());
-        
-        const result = await client.scrape('https://example.com');
-        console.log(result);
-    </script>
-</body>
-</html>
-```
-
-**Security Note**: Never expose your API key in client-side code. Use a backend proxy or server-side rendering.
-
-## ⚡ Performance Tips
-
-1. **Reuse Client Instances**: Create one client instance and reuse it
-2. **Enable Retries**: The client automatically retries failed requests
-3. **Use Appropriate Timeouts**: Adjust timeout based on your use case
-4. **Batch Requests**: Process multiple URLs concurrently
+### Client options
 
 ```javascript
 const client = new BrowserNativeClient(apiKey, {
-  timeout: 45000,  // Longer timeout for complex sites
-  retries: 3,      // More retries for reliability
-  verbose: true    // Enable logging for debugging
+  baseUrl: 'https://bnca-api.fly.dev', // API endpoint
+  timeout: 30000,                       // Request timeout (ms)
+  retries: 2,                           // Retry attempts
+  verbose: false,                       // Debug logging
 });
+```
 
-// Concurrent scraping
-const urls = ['https://site1.com', 'https://site2.com', 'https://site3.com'];
-const results = await Promise.all(
-  urls.map(url => client.scrape(url))
-);
+### Methods
+
+| Method | Description |
+|--------|-------------|
+| `scrape(url, opts?)` | Extract structured content |
+| `screenshot(url, opts?)` | Screenshot with content extraction |
+| `quickshot(url, opts?)` | Fast screenshot only |
+| `analyze(url, question, opts?)` | AI-powered Q&A |
+| `bulkScrape(urls, opts?)` | Scrape multiple URLs concurrently |
+| `getUsage(days?)` | Account usage statistics |
+| `healthCheck()` | API health status |
+
+### Scrape options
+
+```javascript
+{
+  method: 'auto',            // 'auto' | 'direct' | 'lightpanda' | 'puppeteer'
+  includeScreenshot: false,
+  waitForSelector: '.content',
+  userAgent: 'Custom Bot',
+  viewport: { width: 1920, height: 1080 },
+  extractMetadata: true,
+}
 ```
 
-## 🔧 Error Handling
+### Response shape
 
 ```javascript
-try {
-  const result = await client.scrape(url);
-  
-  if (result.success) {
-    console.log('Success:', result.data);
-  } else {
-    console.error('Scraping failed:', result.error);
-  }
-} catch (error) {
-  console.error('Request failed:', error.message);
+{
+  success: true,
+  data: { title, content, metadata, ... },
+  responseTime: 1234,
+  attempt: 1,
 }
 ```
 
-Common error scenarios:
-- **Invalid API Key**: Check your API key and account status
-- **Rate Limited**: Upgrade your plan or reduce request frequency
-- **Timeout**: Increase timeout or try a simpler extraction method
-- **Invalid URL**: Ensure the URL is accessible and properly formatted
-
-## 🚀 TypeScript Support
+## TypeScript
 
-Full TypeScript support with comprehensive type definitions:
+Full type definitions included.
 
 ```typescript
-import { BrowserNativeClient, ScrapeResult, AnalyzeResult } from '@monostate/browsernative-client';
-
-const client: BrowserNativeClient = new BrowserNativeClient('your-api-key');
-
-const result: ScrapeResult = await client.scrape('https://example.com');
-const analysis: AnalyzeResult = await client.analyze(url, question);
+import { BrowserNativeClient, ScrapeResult, BulkScrapeResult } from '@monostate/browsernative-client';
 ```
 
-## 📊 Rate Limits
-
-| Plan | Requests/Month | Rate Limit |
-|------|----------------|------------|
-| **Free** | 1,000 | 10/minute |
-| **Starter** | 10,000 | 60/minute |
-| **Pro** | 100,000 | 300/minute |
-| **Enterprise** | Unlimited | Custom |
-
-## 📋 Changelog
-
-### v1.2.0 (Latest)
-- 🔧 **Timeout Improvements**: Enhanced timeout handling and request reliability
-- 📝 **Documentation Updates**: Comprehensive API documentation and examples
-- 🏷️ **Package Naming**: Proper package name consistency across all imports
-- ⚡ **Performance Optimizations**: Better error handling and response processing
-- 🌐 **Framework Integration**: Improved React, Vue, and Next.js examples
-
-### v1.1.2
-- Bug fixes and stability improvements
-- Enhanced error handling
-
-### v1.1.1
-- Initial TypeScript support
-- Basic API client functionality
-
-## 🤝 Support
-
-- 📧 **Email**: [support@bnca.monostate.ai](mailto:support@bnca.monostate.ai)
-- 📖 **Documentation**: [bnca.monostate.ai/docs](https://bnca.monostate.ai/docs)
-- 🐛 **Issues**: [GitHub Issues](https://github.com/browsernative/client-sdk/issues)
-- 💬 **Discord**: [Join our community](https://discord.gg/browsernative)
-
-## 📄 License
-
-MIT License - see [LICENSE](../../LICENSE) file for details.
-
----
-
-<div align="center">
+## Changelog
 
-**Built with ❤️ for developers who value speed and reliability**
+See [CHANGELOG.md](./CHANGELOG.md).
 
-[🌐 Website](https://bnca.monostate.ai) | [📖 Docs](https://bnca.monostate.ai/docs) | [🚀 Get API Key](https://bnca.monostate.ai/signup)
+## License
 
-</div>
+MIT

package/index.d.ts

@@ -14,6 +14,8 @@ export interface ClientOptions {
 }
 
 export interface ScrapeOptions {
+  /** Force a specific scraping method */
+  method?: 'auto' | 'direct' | 'lightpanda' | 'puppeteer';
   /** Include screenshot in the response */
   includeScreenshot?: boolean;
   /** Wait for specific selector before extracting content */
@@ -29,6 +31,26 @@ export interface ScrapeOptions {
   extractMetadata?: boolean;
 }
 
+export interface BulkScrapeOptions extends ScrapeOptions {
+  /** Number of concurrent requests (default: 5) */
+  concurrency?: number;
+  /** Continue on error (default: true) */
+  continueOnError?: boolean;
+  /** Progress callback */
+  progressCallback?: (progress: { processed: number; total: number; percentage: number }) => void;
+}
+
+export interface BulkScrapeResult {
+  results: Array<ScrapeResult & { url: string }>;
+  stats: {
+    total: number;
+    successful: number;
+    failed: number;
+    totalTime: number;
+    averageTime: number;
+  };
+}
+
 export interface ScreenshotOptions extends ScrapeOptions {
   /** Force full page screenshot */
   fullPage?: boolean;
@@ -74,14 +96,33 @@ export interface ScrapeResult {
   attempt?: number;
 }
 
-export interface AnalyzeResult extends ScrapeResult {
-  /** AI analysis response */
-  analysis?: {
-    answer: string;
-    confidence: number;
-    processingTime: number;
-    language: string;
+export interface AnalyzeResult {
+  /** Whether the request was successful */
+  success: boolean;
+  /** API response data */
+  data?: {
+    /** Whether the inner request was successful */
+    success?: boolean;
+    /** The answer from AI analysis */
+    answer?: string;
+    /** Metadata about the request */
+    metadata?: {
+      url: string;
+      question: string;
+      method: string;
+      timestamp: string;
+      performance: {
+        totalTime: number;
+      };
+      screenshot?: string | null;
+    };
   };
+  /** Error message (if failed) */
+  error?: string;
+  /** Response time in milliseconds */
+  responseTime: number;
+  /** Attempt number that succeeded */
+  attempt?: number;
 }
 
 export interface UsageResult {
@@ -153,6 +194,11 @@ export declare class BrowserNativeClient {
    */
   analyze(url: string, question: string, options?: AnalyzeOptions): Promise<AnalyzeResult>;
 
+  /**
+   * Scrape multiple URLs with concurrency control
+   */
+  bulkScrape(urls: string[], options?: BulkScrapeOptions): Promise<BulkScrapeResult>;
+
   /**
    * Get account usage statistics
    */
@@ -201,4 +247,13 @@ export declare function quickShot(
   options?: ScreenshotOptions
 ): Promise<ScrapeResult>;
 
+/**
+ * Convenience function for bulk scraping multiple URLs
+ */
+export declare function bulkScrape(
+  urls: string[],
+  apiKey: string,
+  options?: BulkScrapeOptions
+): Promise<BulkScrapeResult>;
+
 export default BrowserNativeClient;

package/index.js

@@ -83,6 +83,57 @@ export class BrowserNativeClient {
     return this._makeRequest('/aireply', payload);
   }
 
+  /**
+   * Scrape multiple URLs with concurrency control
+   * @param {string[]} urls - URLs to scrape
+   * @param {object} options - Bulk scraping options
+   * @returns {Promise<object>} Aggregated results
+   */
+  async bulkScrape(urls, options = {}) {
+    const concurrency = options.concurrency || 5;
+    const continueOnError = options.continueOnError !== false;
+    const results = [];
+    const startTime = Date.now();
+    let processed = 0;
+
+    const queue = [...urls];
+    const workers = Array.from({ length: Math.min(concurrency, urls.length) }, async () => {
+      while (queue.length > 0) {
+        const url = queue.shift();
+        if (!url) break;
+        try {
+          const result = await this.scrape(url, options);
+          results.push({ url, ...result });
+        } catch (error) {
+          results.push({ url, success: false, error: error.message });
+          if (!continueOnError) throw error;
+        }
+        processed++;
+        if (options.progressCallback) {
+          options.progressCallback({
+            processed,
+            total: urls.length,
+            percentage: (processed / urls.length) * 100,
+          });
+        }
+      }
+    });
+
+    await Promise.all(workers);
+
+    const successful = results.filter(r => r.success).length;
+    return {
+      results,
+      stats: {
+        total: urls.length,
+        successful,
+        failed: urls.length - successful,
+        totalTime: Date.now() - startTime,
+        averageTime: Math.round((Date.now() - startTime) / urls.length),
+      },
+    };
+  }
+
   /**
    * Get account usage statistics
    * @param {number} days - Number of days to fetch (max 30)
@@ -121,7 +172,7 @@ export class BrowserNativeClient {
           headers: {
             'x-api-key': this.apiKey,
             'Content-Type': 'application/json',
-            'User-Agent': 'Browser Native Client SDK/1.2.1'
+            'User-Agent': 'Browser Native Client SDK/2.0.0'
           }
         };
 
@@ -231,5 +282,17 @@ export async function quickShot(url, apiKey, options = {}) {
   return client.quickshot(url, options);
 }
 
+/**
+ * Convenience function for bulk scraping multiple URLs
+ * @param {string[]} urls - URLs to scrape
+ * @param {string} apiKey - Your API key
+ * @param {object} options - Bulk options (concurrency, continueOnError, progressCallback)
+ * @returns {Promise<object>} Aggregated results
+ */
+export async function bulkScrape(urls, apiKey, options = {}) {
+  const client = new BrowserNativeClient(apiKey, options);
+  return client.bulkScrape(urls, options);
+}
+
 // Default export for CommonJS compatibility
 export default BrowserNativeClient;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@monostate/browsernative-client",
-  "version": "1.2.1",
+  "version": "2.0.0",
   "description": "Browser Native client SDK for web scraping and content extraction API",
   "type": "module",
   "main": "index.js",
@@ -33,7 +33,7 @@
   "license": "MIT",
   "dependencies": {},
   "engines": {
-    "node": ">=16.0.0"
+    "node": ">=18.0.0"
   },
   "repository": {
     "type": "git",

package/LICENSE

@@ -1,21 +0,0 @@
-MIT License
-
-Copyright (c) 2025 BNCA Team
-
-Permission is hereby granted, free of charge, to any person obtaining a copy
-of this software and associated documentation files (the "Software"), to deal
-in the Software without restriction, including without limitation the rights
-to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
-copies of the Software, and to permit persons to whom the Software is
-furnished to do so, subject to the following conditions:
-
-The above copyright notice and this permission notice shall be included in all
-copies or substantial portions of the Software.
-
-THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
-IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
-FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
-AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
-LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
-OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
-SOFTWARE.