@olib-ai/owl-browser-sdk 1.0.0

package/README.md

@@ -0,0 +1,582 @@
+# Owl Browser SDK
+
+AI-first browser automation SDK for Node.js/TypeScript. Built on top of the Owl Browser - a custom Chromium-based browser with built-in AI intelligence.
+
+## Features
+
+- 🧠 **Natural Language Selectors**: Use "search button" instead of complex CSS selectors
+- 🤖 **On-Device LLM**: Built-in Qwen3-1.7B for intelligent page understanding
+- 🌍 **Context Awareness**: Auto-detect location, time, and weather for smart automation
+- ⚡ **Lightning Fast**: Native C++ browser with <1s cold start
+- 🛡️ **Maximum Stealth**: No WebDriver detection, built-in ad blocking
+- 📸 **Full HD Screenshots**: 1920x1080 high-quality captures
+- 🎥 **Video Recording**: Record browser sessions as video
+- 🏠 **Custom Homepage**: Beautiful branded homepage with real-time demographics
+- 🌐 **Dual Mode**: Connect to local browser binary OR remote HTTP server
+
+## Installation
+
+```bash
+npm install owl-browser-sdk
+```
+
+**Prerequisites:**
+- Node.js 18+
+- **Local mode**: The Owl Browser binary must be built (see main README)
+- **HTTP mode**: A running Owl Browser HTTP server
+
+## Connection Modes
+
+The SDK supports two connection modes:
+
+### Local Mode (Default)
+
+Uses the local browser binary via IPC (stdin/stdout). Best for development and single-machine deployments.
+
+```typescript
+import { Browser } from 'owl-browser-sdk';
+
+// Local mode (default)
+const browser = new Browser();
+await browser.launch();
+```
+
+### HTTP Mode
+
+Connects to a remote browser server via REST API. Best for Docker deployments, microservices, and distributed systems.
+
+```typescript
+import { Browser } from 'owl-browser-sdk';
+
+// HTTP mode - connect to remote server
+const browser = new Browser({
+  mode: 'http',
+  http: {
+    baseUrl: 'http://localhost:8080',
+    token: 'your-secret-token',
+    timeout: 30000  // optional, default 30s
+  }
+});
+await browser.launch();
+```
+
+**Setting up the HTTP server:**
+
+```bash
+# Build the HTTP server
+cd build && cmake .. -DCMAKE_BUILD_TYPE=Release && make owl_http_server
+
+# Run the server
+OWL_HTTP_TOKEN=your-secret-token \
+OWL_BROWSER_PATH=/path/to/owl_browser \
+./build/Release/owl_http_server
+```
+
+See [HTTP Server Documentation](../http-server/README.md) for more details.
+
+## Quick Start
+
+```typescript
+import { Browser } from 'owl-browser-sdk';
+
+// Create and launch browser
+const browser = new Browser();
+await browser.launch();
+
+// Create a new page
+const page = await browser.newPage();
+
+// Navigate to a URL
+await page.goto('https://example.com');
+
+// Use natural language selectors!
+await page.click('search button');
+await page.type('email input', 'user@example.com');
+await page.pressKey('Enter');
+
+// Take a screenshot
+const screenshot = await page.screenshot();
+
+// Close browser
+await browser.close();
+```
+
+## API Reference
+
+### Browser
+
+```typescript
+const browser = new Browser(config?: BrowserConfig);
+```
+
+**Methods:**
+- `launch()` - Start the browser process
+- `newPage()` - Create a new browser context (page/tab)
+- `pages()` - Get all active pages
+- `getLLMStatus()` - Check on-device LLM status
+- `listTemplates()` - List available extraction templates
+- `getDemographics()` - Get complete demographics (location, time, weather)
+- `getLocation()` - Get geographic location based on IP
+- `getDateTime()` - Get current date and time
+- `getWeather()` - Get current weather for user's location
+- `getHomepage()` - Get custom browser homepage HTML
+- `close()` - Shutdown browser and cleanup
+- `isRunning()` - Check if browser is running
+
+### BrowserContext (Page)
+
+All page interactions happen through a BrowserContext instance.
+
+#### Navigation
+
+```typescript
+await page.goto('https://example.com');
+await page.reload();
+await page.goBack();
+await page.goForward();
+```
+
+#### Interactions
+
+```typescript
+// Natural language selectors work!
+await page.click('search button');
+await page.type('email input', 'user@example.com');
+await page.pressKey('Enter');
+
+// Or use CSS selectors
+await page.click('#submit-btn');
+await page.type('input[name="email"]', 'user@example.com');
+
+// Or use position coordinates (x,y)
+await page.click('500x300');  // Click at x=500, y=300
+await page.type('640x400', 'text here');  // Type at position
+
+// Highlight elements for debugging
+await page.highlight('login button');
+```
+
+#### Content Extraction
+
+```typescript
+// Extract text
+const text = await page.extractText('body');
+
+// Get HTML (with cleaning)
+const html = await page.getHTML('basic'); // 'minimal' | 'basic' | 'aggressive'
+
+// Get Markdown
+const markdown = await page.getMarkdown({
+  includeLinks: true,
+  includeImages: true,
+  maxLength: 10000
+});
+
+// Extract structured JSON
+const data = await page.extractJSON('google_search');
+const siteType = await page.detectWebsiteType();
+```
+
+#### AI Features
+
+```typescript
+// Ask questions about the page
+const answer = await page.queryPage('What is the main topic of this page?');
+
+// Execute natural language commands
+await page.executeNLA('go to google.com and search for banana');
+```
+
+#### Screenshots & Video
+
+```typescript
+// Screenshot
+const screenshot = await page.screenshot();
+fs.writeFileSync('screenshot.png', screenshot);
+
+// Video recording
+await page.startVideoRecording({ fps: 30 });
+// ... perform actions ...
+const videoPath = await page.stopVideoRecording();
+console.log('Video saved to:', videoPath);
+```
+
+#### Scrolling
+
+```typescript
+await page.scrollBy(0, 500);
+await page.scrollTo(0, 0);
+await page.scrollToTop();
+await page.scrollToBottom();
+await page.scrollToElement('footer');
+```
+
+#### Waiting
+
+```typescript
+// Wait for element
+await page.waitForSelector('submit button', { timeout: 5000 });
+
+// Wait for time
+await page.wait(1000); // 1 second
+```
+
+#### Test Execution
+
+```typescript
+// Load test from Developer Playground JSON export
+const test = JSON.parse(fs.readFileSync('test.json', 'utf-8'));
+const result = await page.runTest(test, {
+  verbose: true,
+  continueOnError: false,
+  screenshotOnError: true
+});
+
+console.log(`Success: ${result.successfulSteps}/${result.totalSteps}`);
+console.log(`Time: ${result.executionTime}ms`);
+
+// Or define test inline
+const inlineTest = {
+  name: "Login Test",
+  steps: [
+    { type: "navigate", url: "https://example.com/login" },
+    { type: "type", selector: "#email", text: "user@example.com" },
+    { type: "click", selector: "500x300" },  // Position-based click
+    { type: "screenshot", filename: "result.png" }
+  ]
+};
+const result = await page.runTest(inlineTest);
+```
+
+#### Page Information
+
+```typescript
+const url = await page.getCurrentURL();
+const title = await page.getTitle();
+const info = await page.getPageInfo(); // { url, title, canGoBack, canGoForward }
+```
+
+#### Viewport
+
+```typescript
+await page.setViewport({ width: 1920, height: 1080 });
+const viewport = await page.getViewport();
+```
+
+## Configuration
+
+### Local Mode Configuration
+
+```typescript
+const browser = new Browser({
+  // Connection mode (default: 'local')
+  mode: 'local',
+
+  // Path to browser binary (auto-detected if not provided)
+  browserPath: '/path/to/owl_browser',
+
+  // Enable headless mode (default: true)
+  headless: true,
+
+  // Enable verbose logging (default: false)
+  verbose: true,
+
+  // Initialization timeout in ms (default: 30000)
+  initTimeout: 30000
+});
+```
+
+### HTTP Mode Configuration
+
+```typescript
+const browser = new Browser({
+  // Use HTTP mode to connect to remote server
+  mode: 'http',
+
+  // HTTP server connection settings (required for HTTP mode)
+  http: {
+    // Base URL of the HTTP server
+    baseUrl: 'http://localhost:8080',
+
+    // Bearer token for authentication (must match server's OWL_HTTP_TOKEN)
+    token: 'your-secret-token',
+
+    // Request timeout in ms (default: 30000)
+    timeout: 30000
+  },
+
+  // Enable verbose logging (default: false)
+  verbose: true
+});
+```
+
+### Configuration from Environment Variables
+
+```typescript
+// HTTP mode with environment variables
+const browser = new Browser({
+  mode: 'http',
+  http: {
+    baseUrl: process.env.OWL_HTTP_URL || 'http://localhost:8080',
+    token: process.env.OWL_HTTP_TOKEN || ''
+  }
+});
+```
+
+## Examples
+
+### Demographics & Context Awareness
+
+```typescript
+import { Browser } from 'owl-browser-sdk';
+
+const browser = new Browser();
+await browser.launch();
+
+// Get location based on IP address
+const location = await browser.getLocation();
+console.log(`Location: ${location.city}, ${location.country}`);
+console.log(`Coordinates: ${location.latitude}, ${location.longitude}`);
+
+// Get current weather
+const weather = await browser.getWeather();
+console.log(`Weather: ${weather.temperature_c}°C, ${weather.condition}`);
+
+// Get date and time
+const datetime = await browser.getDateTime();
+console.log(`Date: ${datetime.date} (${datetime.day_of_week})`);
+console.log(`Time: ${datetime.time} ${datetime.timezone}`);
+
+// Get everything at once
+const demographics = await browser.getDemographics();
+console.log('Complete context:', demographics);
+
+// Get custom homepage
+const homepage = await browser.getHomepage();
+fs.writeFileSync('homepage.html', homepage);
+
+await browser.close();
+```
+
+### Simple Google Search
+
+```typescript
+import { Browser } from 'owl-browser-sdk';
+import fs from 'fs';
+
+async function googleSearch() {
+  const browser = new Browser();
+  await browser.launch();
+
+  const page = await browser.newPage();
+  await page.goto('https://www.google.com');
+
+  // Natural language selectors!
+  await page.type('search box', 'Owl Browser');
+  await page.pressKey('Enter');
+
+  await page.wait(2000);
+  const screenshot = await page.screenshot();
+  fs.writeFileSync('search-results.png', screenshot);
+
+  await browser.close();
+}
+
+googleSearch();
+```
+
+### Web Scraping with AI
+
+```typescript
+import { Browser } from 'owl-browser-sdk';
+
+async function scrapeWithAI() {
+  const browser = new Browser();
+  await browser.launch();
+
+  const page = await browser.newPage();
+  await page.goto('https://news.ycombinator.com');
+
+  // Ask AI about the page
+  const answer = await page.queryPage('What are the top 3 story titles?');
+  console.log(answer);
+
+  // Extract as Markdown
+  const markdown = await page.getMarkdown();
+  console.log(markdown);
+
+  await browser.close();
+}
+
+scrapeWithAI();
+```
+
+### Video Recording
+
+```typescript
+import { Browser } from 'owl-browser-sdk';
+
+async function recordSession() {
+  const browser = new Browser();
+  await browser.launch();
+
+  const page = await browser.newPage();
+
+  // Start recording
+  await page.startVideoRecording({ fps: 30 });
+
+  await page.goto('https://example.com');
+  await page.click('some button');
+  await page.scrollToBottom();
+
+  // Stop and get video path
+  const videoPath = await page.stopVideoRecording();
+  console.log('Video saved to:', videoPath);
+
+  await browser.close();
+}
+
+recordSession();
+```
+
+## TypeScript Support
+
+This SDK is written in TypeScript and includes full type definitions.
+
+```typescript
+import { Browser, BrowserContext, Viewport } from 'owl-browser-sdk';
+
+const browser: Browser = new Browser();
+const page: BrowserContext = await browser.newPage();
+const viewport: Viewport = await page.getViewport();
+```
+
+## Why Olib SDK vs Puppeteer/Playwright?
+
+| Feature | Olib SDK | Puppeteer/Playwright |
+|---------|----------|----------------------|
+| Natural Language Selectors | ✅ Built-in | ❌ Not supported |
+| On-Device LLM | ✅ Qwen3-1.7B | ❌ Not supported |
+| Cold Start | <1s | 2-5s |
+| WebDriver Detection | ✅ None | ⚠️ Detectable |
+| Built-in Ad Blocking | ✅ 72 domains | ❌ Manual setup |
+| Maintenance Overhead | ✅ Low (AI handles changes) | ⚠️ High (selectors break) |
+
+## Advanced Features
+
+### Semantic Element Matching
+
+The browser uses an advanced semantic matcher that understands natural language:
+
+```typescript
+// All of these work!
+await page.click('search button');
+await page.click('search btn');
+await page.click('search');
+await page.type('email input', 'test@example.com');
+await page.type('email field', 'test@example.com');
+await page.type('email box', 'test@example.com');
+```
+
+The matcher uses:
+- Keyword extraction and normalization
+- Role inference (search_input, submit_button, etc.)
+- Multi-source scoring (aria-label, placeholder, text, title)
+- Fuzzy matching with confidence thresholds
+
+### On-Device LLM
+
+Check LLM status:
+
+```typescript
+const status = await browser.getLLMStatus();
+// Returns: 'ready' | 'loading' | 'unavailable'
+```
+
+Query pages with natural language:
+
+```typescript
+const answer = await page.queryPage('What products are shown on this page?');
+const summary = await page.queryPage('Summarize this article in 2 sentences');
+```
+
+## Error Handling
+
+```typescript
+import { Browser } from 'owl-browser-sdk';
+
+async function withErrorHandling() {
+  const browser = new Browser();
+
+  try {
+    await browser.launch();
+    const page = await browser.newPage();
+    await page.goto('https://example.com');
+
+    // Your automation code...
+
+  } catch (error) {
+    console.error('Automation failed:', error);
+  } finally {
+    await browser.close();
+  }
+}
+```
+
+## Performance Tips
+
+1. **Reuse browser instances** - Creating a new browser is expensive
+2. **Use multiple pages** - Instead of multiple browsers, create multiple pages
+3. **Wait strategically** - Use `waitForSelector()` instead of fixed `wait()`
+4. **Close contexts** - Call `page.close()` when done with a page
+
+## Troubleshooting
+
+### Browser binary not found
+
+Make sure you've built the browser first:
+
+```bash
+cd /path/to/owl-browser
+npm run build
+```
+
+Or provide the path explicitly:
+
+```typescript
+const browser = new Browser({
+  browserPath: '/custom/path/to/owl_browser'
+});
+```
+
+### Timeout errors
+
+Increase the timeout:
+
+```typescript
+const browser = new Browser({
+  initTimeout: 60000 // 60 seconds
+});
+```
+
+### Element not found
+
+Try different selector variations:
+
+```typescript
+// Try multiple approaches
+await page.click('search button');
+await page.click('search');
+await page.click('[aria-label="Search"]');
+await page.click('#search-btn');
+```
+
+## License
+
+MIT
+
+## Credits
+
+- Built with Chromium Embedded Framework (CEF)
+- LLM powered by llama.cpp + Qwen3-1.7B
+- Designed for AI automation and developer productivity