btcp-browser-agent 0.1.0 → 0.1.1

package/examples/chrome-extension/tests/README.md

@@ -1,206 +0,0 @@
-# Chrome Extension Scenario Tests
-
-This directory contains demonstration scripts that showcase how AI agents would interact with the BTCP Browser Agent API in real-world scenarios.
-
-## Available Scenarios
-
-### 1. Google Search → GitHub Star (`scenario-google-to-github-star.ts`)
-
-A complete workflow demonstrating AI agent reasoning patterns:
-
-**Workflow:**
-1. Navigate to Google
-2. Search for "btcp-cowork"
-3. Find and click the first GitHub link in results
-4. Navigate to the repository
-5. Attempt to star the repository
-
-**What it demonstrates:**
-- AI reasoning simulation with detailed logging
-- Snapshot-based page understanding
-- Element selection strategies using accessibility tree
-- Error recovery with fallback approaches
-- Multi-step navigation and validation
-- Authentication gate detection
-
-## Running the Scenarios
-
-### Prerequisites
-
-1. **Install Dependencies**
-   ```bash
-   cd examples/chrome-extension
-   npm install
-   ```
-
-2. **Build the Extension**
-   ```bash
-   npm run build
-   ```
-
-3. **Load Extension in Chrome**
-   - Open Chrome and navigate to `chrome://extensions`
-   - Enable "Developer mode" (toggle in top right)
-   - Click "Load unpacked"
-   - Select the `examples/chrome-extension/dist/` directory
-
-4. **Create a Session**
-   - Click the extension icon in Chrome toolbar
-   - Click "Start Session" button
-   - A new tab group will be created
-
-### Running the Demo
-
-With the extension loaded and a session active:
-
-```bash
-cd examples/chrome-extension
-npm run demo:scenario
-```
-
-### Expected Output
-
-The script will output detailed logs showing AI reasoning at each step:
-
-```
-🤔 AI Agent: Starting Google → GitHub → Star workflow demonstration
-
-[Step 1/15] Initialize connection to browser extension
-✨ Success: Client connected
-
-[Step 2/15] Navigate to Google
-🤔 AI Agent: Need to navigate to google.com to start search
-🎯 Action: Navigating to https://www.google.com...
-✅ Navigation complete
-
-[Step 3/15] Take snapshot to analyze page structure
-🤔 AI Agent: Taking snapshot to identify interactive elements
-🎯 Action: Calling snapshot API...
-📊 Verification: Snapshot captured: 156 elements found
-
-[Step 4/15] Locate search input field
-🔍 Analyzing: Looking for search box (role=searchbox or combobox)
-✅ Found: @ref:12 - role='combobox' name='Search'
-
-...
-```
-
-## Understanding the Output
-
-The demo uses emoji prefixes to indicate different types of information:
-
-- 🤔 **Thinking**: AI agent's reasoning process
-- 🔍 **Analyzing**: Element search criteria
-- ✅ **Found**: Successfully located element
-- 🎯 **Action**: Executing a command
-- 📊 **Verification**: Validating results
-- ❌ **Error**: Something went wrong
-- ⚠️ **Warning**: Non-critical issue or fallback triggered
-- ✨ **Success**: Step completed successfully
-
-## Common Issues
-
-### "Could not find search input"
-
-Google's page structure varies by region and personalization. The script includes fallback strategies, but may need adjustment for your specific Google layout.
-
-**Solution**: The script will try multiple approaches automatically. Check the logs to see which strategy worked.
-
-### "No GitHub links found in search results"
-
-This can happen if:
-- Google blocks automated searches
-- Search results don't include GitHub repositories
-- Results are personalized differently
-
-**Solution**: Try running the script again, or manually verify that searching "btcp-cowork" on Google returns GitHub results.
-
-### "Redirected to login page"
-
-GitHub requires authentication to star repositories.
-
-**Solution**: This is expected behavior. The script will detect the login redirect and explain the situation. In a real AI agent, this would trigger an OAuth flow or use stored credentials.
-
-### "Error: connect ECONNREFUSED"
-
-The extension is not running or no session is active.
-
-**Solution**:
-1. Verify the extension is loaded in Chrome
-2. Click the extension icon
-3. Click "Start Session"
-4. Run the script again
-
-## API Quality Insights
-
-This scenario helps evaluate the BTCP API for AI agent use cases:
-
-### Strengths
-✅ **Clear accessibility tree**: Snapshot provides semantic element information
-✅ **Stable references**: `@ref:N` selectors work reliably within a session
-✅ **Simple API**: Navigation and interaction methods are intuitive
-✅ **Error messages**: Informative feedback when operations fail
-
-### Areas for Improvement
-⚠️ **Complex pages**: Some dynamic content needs multiple snapshot strategies
-⚠️ **Authentication**: OAuth flows require additional handling
-⚠️ **Timing**: Some actions need explicit wait times (could use smart waiting)
-⚠️ **Ref invalidation**: References become stale after navigation (expected, but needs handling)
-
-## Writing Your Own Scenarios
-
-To create a new scenario test:
-
-1. **Create a new file** in this directory (e.g., `scenario-form-filling.ts`)
-
-2. **Import the client**:
-   ```typescript
-   import { createClient } from '../../../packages/extension/src/index.js';
-   ```
-
-3. **Structure your workflow**:
-   ```typescript
-   async function main() {
-     const client = createClient();
-
-     // Step 1: Navigate
-     await client.navigate('https://example.com');
-
-     // Step 2: Understand page
-     const snapshot = await client.snapshot({ format: 'tree' });
-
-     // Step 3: Find and interact with elements
-     const button = findElement(snapshot.tree, { role: 'button', name: 'Submit' });
-     await client.click(button);
-
-     // Step 4: Verify outcome
-     const newUrl = await client.getUrl();
-     console.log('Success:', newUrl.includes('success'));
-   }
-
-   main();
-   ```
-
-4. **Add logging** to show AI reasoning at each step
-
-5. **Add to package.json**:
-   ```json
-   "scripts": {
-     "demo:your-scenario": "tsx tests/scenario-your-name.ts"
-   }
-   ```
-
-## Contributing
-
-When adding new scenarios, please:
-- Include detailed AI reasoning logs
-- Handle errors gracefully with fallbacks
-- Document what the scenario tests/demonstrates
-- Update this README with usage instructions
-- Consider edge cases and authentication requirements
-
-## Further Reading
-
-- [BTCP API Documentation](../../CLAUDE.md)
-- [Extension Architecture](../../../packages/extension/README.md)
-- [Core Actions Reference](../../../packages/core/README.md)

package/examples/chrome-extension/tests/scenario-google-to-github-star.ts

@@ -1,380 +0,0 @@
-/**
- * Real-World AI Agent Scenario: Google Search → GitHub Repository → Star
- *
- * This demo script mimics how an AI agent would reason through a complete browser workflow:
- * 1. Navigate to Google
- * 2. Search for "btcp-cowork"
- * 3. Find and click the first GitHub link in results
- * 4. Attempt to star the repository
- *
- * The script demonstrates:
- * - AI reasoning patterns (element selection, verification, error handling)
- * - Snapshot-based navigation (accessibility tree parsing)
- * - Graceful error recovery
- * - Step-by-step validation
- *
- * Prerequisites:
- * - Chrome extension must be loaded and running
- * - Extension must have a session created
- *
- * Run with: npm run demo:scenario
- */
-
-import { createClient } from '../../../packages/extension/src/index.js';
-
-// Utility for delays (simulating think time)
-const sleep = (ms: number) => new Promise(resolve => setTimeout(resolve, ms));
-
-// Console formatting helpers
-const log = {
-  thinking: (msg: string) => console.log(`\n🤔 AI Agent: ${msg}`),
-  analyzing: (msg: string) => console.log(`🔍 Analyzing: ${msg}`),
-  found: (msg: string) => console.log(`✅ Found: ${msg}`),
-  action: (msg: string) => console.log(`🎯 Action: ${msg}`),
-  verify: (msg: string) => console.log(`📊 Verification: ${msg}`),
-  error: (msg: string) => console.log(`❌ Error: ${msg}`),
-  warning: (msg: string) => console.log(`⚠️  Warning: ${msg}`),
-  success: (msg: string) => console.log(`✨ Success: ${msg}`),
-  step: (step: number, total: number, msg: string) => console.log(`\n[Step ${step}/${total}] ${msg}`),
-};
-
-/**
- * Simulates AI reasoning to find an element in the snapshot tree
- * Returns the @ref:N selector for the found element
- */
-function findElement(tree: string, criteria: {
-  role?: string;
-  name?: string;
-  nameContains?: string;
-  type?: string;
-}): string | null {
-  const lines = tree.split('\n');
-
-  for (const line of lines) {
-    // Parse line format: "@ref:N role='...' name='...' ..."
-    const refMatch = line.match(/@ref:(\d+)/);
-    if (!refMatch) continue;
-
-    const ref = `@ref:${refMatch[1]}`;
-
-    // Check all criteria
-    let matches = true;
-
-    if (criteria.role) {
-      const roleMatch = line.match(/role='([^']+)'/);
-      if (!roleMatch || roleMatch[1] !== criteria.role) matches = false;
-    }
-
-    if (criteria.name) {
-      const nameMatch = line.match(/name='([^']+)'/);
-      if (!nameMatch || nameMatch[1] !== criteria.name) matches = false;
-    }
-
-    if (criteria.nameContains) {
-      const nameMatch = line.match(/name='([^']+)'/);
-      if (!nameMatch || !nameMatch[1].toLowerCase().includes(criteria.nameContains.toLowerCase())) {
-        matches = false;
-      }
-    }
-
-    if (criteria.type) {
-      const typeMatch = line.match(/type='([^']+)'/);
-      if (!typeMatch || typeMatch[1] !== criteria.type) matches = false;
-    }
-
-    if (matches) {
-      log.found(`${ref} - ${line.trim()}`);
-      return ref;
-    }
-  }
-
-  return null;
-}
-
-/**
- * Find first link containing specific domain
- */
-function findLinkByDomain(tree: string, domain: string): string | null {
-  const lines = tree.split('\n');
-
-  for (const line of lines) {
-    if (!line.includes("role='link'")) continue;
-
-    const refMatch = line.match(/@ref:(\d+)/);
-    if (!refMatch) continue;
-
-    // Check if line contains the domain (in name or other attributes)
-    if (line.toLowerCase().includes(domain.toLowerCase())) {
-      const ref = `@ref:${refMatch[1]}`;
-      log.found(`${ref} - GitHub link: ${line.trim()}`);
-      return ref;
-    }
-  }
-
-  return null;
-}
-
-async function main() {
-  const TOTAL_STEPS = 15;
-  let currentStep = 0;
-
-  log.thinking('Starting Google → GitHub → Star workflow demonstration');
-  console.log('This script simulates AI agent reasoning patterns for browser automation\n');
-
-  try {
-    // Step 1: Initialize client
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Initialize connection to browser extension');
-    const client = createClient();
-    log.success('Client connected');
-    await sleep(500);
-
-    // Step 2: Navigate to Google
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Navigate to Google');
-    log.thinking('Need to navigate to google.com to start search');
-    log.action('Navigating to https://www.google.com...');
-
-    await client.navigate('https://www.google.com');
-    log.success('Navigation complete');
-    await sleep(1000); // Wait for page to settle
-
-    // Step 3: Take snapshot to understand page structure
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Take snapshot to analyze page structure');
-    log.thinking('Taking snapshot to identify interactive elements');
-    log.action('Calling snapshot API...');
-
-    const snapshot1 = await client.snapshot({ format: 'tree' });
-    log.verify(`Snapshot captured: ${snapshot1.split('\n').length} elements found`);
-    await sleep(500);
-
-    // Step 4: Find search input
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Locate search input field');
-    log.analyzing('Looking for search box (role=searchbox or combobox)');
-
-    let searchInput = findElement(snapshot1, { role: 'combobox' });
-    if (!searchInput) {
-      searchInput = findElement(snapshot1, { role: 'searchbox' });
-    }
-
-    if (!searchInput) {
-      log.error('Could not find search input on Google homepage');
-      log.warning('This might be due to Google\'s dynamic content or region-specific layout');
-      // Try alternative approach: look for input with name containing "search" or "q"
-      log.analyzing('Trying alternative strategy: looking for textbox with search-related name');
-      searchInput = findElement(snapshot1, { role: 'textbox', nameContains: 'search' });
-    }
-
-    if (!searchInput) {
-      throw new Error('Unable to locate search input field');
-    }
-
-    await sleep(500);
-
-    // Step 5: Type search query
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Type search query into input');
-    log.thinking('Need to type "btcp-cowork" into the search box');
-    log.action(`Typing "btcp-cowork" into ${searchInput}...`);
-
-    await client.type(searchInput, 'btcp-cowork');
-    log.success('Query typed successfully');
-    await sleep(500);
-
-    // Step 6: Find and click search button
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Locate and click search button');
-    log.analyzing('Looking for search submit button (role=button, name contains "search")');
-
-    const searchButton = findElement(snapshot1, { role: 'button', nameContains: 'search' });
-
-    if (!searchButton) {
-      log.warning('Search button not found, trying Enter key instead');
-      log.action('Pressing Enter key to submit search...');
-      await client.execute({
-        id: crypto.randomUUID(),
-        action: 'press',
-        key: 'Enter',
-      });
-    } else {
-      log.action(`Clicking search button ${searchButton}...`);
-      await client.click(searchButton);
-    }
-
-    log.success('Search submitted');
-    await sleep(2000); // Wait for search results to load
-
-    // Step 7: Wait for results page to load
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Wait for search results to load');
-    log.thinking('Giving page time to load search results');
-
-    const currentUrl = await client.getUrl();
-    log.verify(`Current URL: ${currentUrl}`);
-
-    if (!currentUrl.includes('google.com/search')) {
-      log.warning('URL does not appear to be a search results page');
-    }
-    await sleep(1000);
-
-    // Step 8: Take snapshot of results page
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Snapshot search results page');
-    log.action('Taking snapshot of search results...');
-
-    const snapshot2 = await client.snapshot({ format: 'tree' });
-    log.verify(`Results snapshot captured: ${snapshot2.split('\n').length} elements found`);
-    await sleep(500);
-
-    // Step 9: Find first GitHub link
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Locate first GitHub link in results');
-    log.analyzing('Searching for links containing "github.com"');
-    log.thinking('AI reasoning: GitHub links are likely to contain "github" in the URL or link text');
-
-    const githubLink = findLinkByDomain(snapshot2, 'github.com');
-
-    if (!githubLink) {
-      log.error('No GitHub links found in search results');
-      log.warning('Possible reasons:');
-      console.log('  - Search results may not include GitHub repositories');
-      console.log('  - Google may have blocked automated searches');
-      console.log('  - Results structure may differ from expected format');
-      throw new Error('Cannot proceed without GitHub link');
-    }
-
-    await sleep(500);
-
-    // Step 10: Click GitHub link
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Navigate to GitHub repository');
-    log.action(`Clicking GitHub link ${githubLink}...`);
-
-    await client.click(githubLink);
-    log.success('Clicked GitHub link, waiting for page load...');
-    await sleep(3000); // Wait for GitHub page to load
-
-    // Step 11: Verify we're on GitHub
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Verify navigation to GitHub');
-
-    const githubUrl = await client.getUrl();
-    log.verify(`Current URL: ${githubUrl}`);
-
-    if (!githubUrl.includes('github.com')) {
-      log.warning('URL does not appear to be github.com');
-    } else {
-      log.success('Successfully navigated to GitHub');
-    }
-
-    await sleep(1000);
-
-    // Step 12: Take snapshot of GitHub repo page
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Snapshot GitHub repository page');
-    log.action('Taking snapshot of repository page...');
-
-    const snapshot3 = await client.snapshot({ format: 'tree' });
-    log.verify(`GitHub page snapshot: ${snapshot3.split('\n').length} elements found`);
-    await sleep(500);
-
-    // Step 13: Find star button
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Locate star button');
-    log.analyzing('Looking for Star/Unstar button (role=button, name contains "star")');
-    log.thinking('AI reasoning: Star button typically has role=button and name includes "star" or "unstar"');
-
-    let starButton = findElement(snapshot3, { role: 'button', nameContains: 'star' });
-
-    if (!starButton) {
-      log.warning('Star button not found in standard format');
-      log.analyzing('Trying alternative: looking for any element with "star" in name');
-
-      // Try to find any element with "star" in it
-      const lines = snapshot3.split('\n');
-      for (const line of lines) {
-        if (line.toLowerCase().includes('star')) {
-          const refMatch = line.match(/@ref:(\d+)/);
-          if (refMatch) {
-            starButton = `@ref:${refMatch[1]}`;
-            log.found(`Possible star element: ${line.trim()}`);
-            break;
-          }
-        }
-      }
-    }
-
-    if (!starButton) {
-      log.error('Could not locate star button');
-      log.warning('Possible reasons:');
-      console.log('  - User may not be logged into GitHub');
-      console.log('  - Page structure may differ from expected format');
-      console.log('  - Repository may have restricted star functionality');
-      throw new Error('Cannot proceed without star button');
-    }
-
-    await sleep(500);
-
-    // Step 14: Click star button
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Attempt to star the repository');
-    log.thinking('Clicking star button - this may require GitHub login');
-    log.action(`Clicking star button ${starButton}...`);
-
-    await client.click(starButton);
-    log.success('Star button clicked');
-    await sleep(2000);
-
-    // Step 15: Verify outcome
-    currentStep++;
-    log.step(currentStep, TOTAL_STEPS, 'Verify final state');
-
-    const finalUrl = await client.getUrl();
-    log.verify(`Final URL: ${finalUrl}`);
-
-    if (finalUrl.includes('login')) {
-      log.warning('Redirected to login page - authentication required to star repositories');
-      log.thinking('AI reasoning: In a real scenario, the agent would need to handle OAuth flow or use stored credentials');
-    } else if (finalUrl.includes('github.com')) {
-      log.success('Still on GitHub page - star action may have succeeded');
-      log.thinking('Taking final snapshot to verify star state...');
-
-      const finalSnapshot = await client.snapshot({ format: 'tree' });
-      const hasUnstar = finalSnapshot.toLowerCase().includes('unstar');
-
-      if (hasUnstar) {
-        log.success('✨ Repository successfully starred! (Found "Unstar" button)');
-      } else {
-        log.verify('Star action completed, but state verification inconclusive');
-      }
-    }
-
-    console.log('\n' + '='.repeat(80));
-    log.success('🎉 Demo workflow completed successfully!');
-    console.log('='.repeat(80));
-
-    console.log('\n📊 Workflow Summary:');
-    console.log(`   • Navigated to Google`);
-    console.log(`   • Searched for "btcp-cowork"`);
-    console.log(`   • Found and clicked GitHub link`);
-    console.log(`   • Attempted to star repository`);
-    console.log(`   • Total steps executed: ${currentStep}/${TOTAL_STEPS}`);
-
-    console.log('\n💡 API Quality Assessment:');
-    console.log('   ✅ Snapshot API provides clear accessibility tree');
-    console.log('   ✅ Element refs (@ref:N) are stable and easy to use');
-    console.log('   ✅ Navigation and interaction methods work reliably');
-    console.log('   ⚠️  Some complex page structures need fallback strategies');
-    console.log('   ⚠️  Authentication flows require additional handling');
-
-  } catch (error) {
-    log.error(`Workflow failed: ${error instanceof Error ? error.message : String(error)}`);
-    console.error(error);
-    process.exit(1);
-  }
-}
-
-// Run the demo
-main();

package/examples/chrome-extension/tsconfig.json

@@ -1,14 +0,0 @@
-{
-  "compilerOptions": {
-    "target": "ES2022",
-    "module": "ESNext",
-    "moduleResolution": "bundler",
-    "strict": true,
-    "esModuleInterop": true,
-    "skipLibCheck": true,
-    "outDir": "dist",
-    "rootDir": "src",
-    "types": ["chrome"]
-  },
-  "include": ["src/**/*"]
-}