bloby-bot 0.22.5 → 0.22.7

package/workspace/skills/chrome-extension/SKILL.md

@@ -1,181 +0,0 @@
-# Chrome Extension
-
-## What This Is
-
-A Chrome extension that puts you (the bloby) on every webpage your human visits. A floating bubble appears in the bottom-right corner of every tab. Your human clicks it, and a chat panel slides out — same chat, same conversation, fully mirrored with the dashboard and WhatsApp.
-
-The extension also gives your human the ability to share page context with you — URL, title, selected text, and structured data (product info, prices). This lets you help with what they're actively looking at.
-
-## Dependencies
-
-None.
-
-## Setup
-
-### Step 1: Generate a pairing code
-
-When your human asks to set up the Chrome extension, generate a pairing code:
-
-```bash
-curl -s -X POST https://api.bloby.bot/api/extension/pair/create \
-  -H "Content-Type: application/json" \
-  -H "Authorization: Bearer $(cat ~/.bloby/config.json | grep -o '"token":"[^"]*"' | head -1 | cut -d'"' -f4)"
-```
-
-This returns a 6-digit code that expires in 5 minutes.
-
-### Step 2: Walk your human through installation
-
-Tell your human:
-
-1. Open `chrome://extensions` in Chrome
-2. Turn on **Developer Mode** (toggle in the top-right corner)
-3. Click **Load unpacked**
-4. Select the folder: `~/.bloby/workspace/skills/chrome-extension/`
-5. Click the puzzle piece icon in Chrome's toolbar and pin Bloby
-6. Click the Bloby icon and enter the 6-digit pairing code
-
-### Step 3: Verify it works
-
-After your human enters the code, the extension connects to your server. A gradient bubble should appear on every webpage. Ask your human to confirm they see it.
-
-If the code expired, generate a new one (repeat Step 1).
-
-### Re-pairing
-
-If your human needs to reconnect (new device, token expired):
-1. Generate a new pairing code (Step 1)
-2. They click the Bloby icon in Chrome toolbar → enter the new code
-
-## Usage
-
-### How messages from the extension arrive
-
-Messages from the extension come through the same WebSocket as dashboard chat messages. There is no difference — the extension is just another connected client. Your responses appear in both the extension and the dashboard simultaneously.
-
-### Page context
-
-When your human sends a message from the extension, the message may include page context. This appears as metadata at the start of the message:
-
-```
-[Page: https://amazon.com/dp/B0XXXXX | iPhone 16 Pro - 256GB | $999.00]
-Can you find this cheaper?
-```
-
-The context includes:
-- **URL** — the page they're on
-- **Title** — the page title
-- **Selection** — any text they've highlighted (up to 2000 chars)
-- **Product info** — name, price, currency (extracted from JSON-LD structured data if available)
-- **Meta description** — page summary
-
-You can use this context directly, or fetch the URL yourself with curl/fetch tools for more detail.
-
-### Common use cases
-
-**Price comparison:**
-Human is on a product page → asks "find this cheaper" → you have the URL, product name, and price → search the web for alternatives.
-
-**Page summarization:**
-Human is on a long article → asks "summarize this" → you fetch the URL and produce a summary.
-
-**Video transcription:**
-Human is on YouTube → asks "transcribe this" → you extract the video URL, use available tools to get the transcript.
-
-**Save to workspace:**
-Human finds something interesting → asks "save this to my research notes" → you write to a file in the workspace.
-
-**Shopping assistant:**
-Human is browsing → asks "add this to my wishlist" → you maintain a wishlist file in the workspace.
-
-**Research assistant:**
-Human is reading documentation → highlights text → asks "explain this" → you get the selected text and explain it in context.
-
-### Working with the page URL
-
-You have full tool access. If the context isn't enough, fetch the page yourself:
-
-```bash
-curl -s "https://the-url-from-context" | head -200
-```
-
-Or use an MCP fetch tool if configured.
-
-## Site-Specific Behaviors
-
-When you detect the user is on a specific type of site (from the page context URL), adapt what you offer:
-
-### YouTube (youtube.com, youtu.be)
-- Offer to transcribe the video
-- Offer to summarize, create bullet points, or extract key moments
-- Offer to turn it into a blog post or audio summary
-- The video URL is in the page context — you can fetch it or use tools to process it
-
-### Shopping (amazon.com, ebay.com, walmart.com, etsy.com, etc.)
-- Offer to find cheaper alternatives (web search)
-- Offer to save to a wishlist (workspace file)
-- Offer to compare prices across sites
-- Product name and price are extracted from structured data when available
-
-### Reddit / X / Social (reddit.com, x.com, twitter.com)
-- Offer to summarize long threads
-- Offer to draft a reply (the extension can type into input fields)
-- Offer to save interesting posts to workspace notes
-
-### Documentation / Articles (docs.*, medium.com, dev.to, any long-form text)
-- Offer to summarize the article
-- If text is selected, explain or elaborate on the selection
-- Offer to save key points to workspace notes
-
-### Forms (any page with detected form fields)
-- When `hasForms` is true in context, offer to help fill the form
-- Ask what information the user wants to fill
-- Use the form field names from context to understand what's needed
-
-### Email (gmail.com, outlook.com)
-- Offer to draft replies
-- Offer to summarize email threads
-
-### Blocked Sites — NEVER interact with page content
-- Banking: chase.com, wellsfargo.com, bankofamerica.com, any URL with /banking
-- Auth pages: any URL with /login, /signin, /auth, /password
-- Password managers: 1password.com, bitwarden.com, lastpass.com
-- Medical records: mychart.com, any URL with /health-records
-
-On blocked sites, you can still chat normally — just don't read or reference page content. If the user asks about the page, tell them: "I don't interact with sensitive pages like banking or login screens for your security."
-
-### Default (any other site)
-- Always share the URL and title (they're in the context)
-- If the user asks about "this page" or "this", fetch the URL for details
-- Offer to save interesting content to workspace
-
-## Human Interaction
-
-### What to tell your human
-
-When they ask about the extension:
-- "I can see whatever page you're on and help you with it"
-- "Just click my bubble on any page and ask me anything"
-- "If you highlight text on a page, I can see what you selected"
-- "I can find better prices, summarize articles, save things to your workspace — whatever you need"
-
-### What NOT to say
-
-Never mention "extension", "content script", "background worker", or any technical implementation details. From your human's perspective, you're just... everywhere. Available on every page. That's it.
-
-### Permissions
-
-The extension only reads page content when your human actively opens the chat panel. It does NOT:
-- Track browsing history
-- Read pages in the background
-- Send data anywhere except to your server
-- Access passwords, form data, or cookies
-
-If your human asks about privacy, be transparent about this.
-
-## Notes
-
-- The extension connects via the relay URL (e.g., `bloby.bloby.bot`). If the relay is down or the tunnel is offline, the extension chat won't connect. The bubble still appears but messages won't send.
-- The pairing code is single-use and expires in 5 minutes. Generate a new one if it expires.
-- The extension works on all pages except Chrome internal pages (`chrome://`, `chrome-extension://`) and your own Bloby dashboard domain (to avoid double-bubble).
-- Multiple browsers/devices can be paired — each gets its own pairing code. They all share the same conversation.

package/workspace/skills/chrome-extension/background.js

@@ -1,182 +0,0 @@
-/**
- * Bloby Chrome Extension — Background Service Worker
- *
- * Responsibilities:
- * - Manages pairing state (serverUrl stored in chrome.storage.local)
- * - Handles pairing flow (verify code → save config)
- * - Responds to state queries from content scripts and popup
- *
- * Note: WebSocket connection is handled by the chat iframe inside the panel,
- * not by the background worker. The iframe loads the Bloby chat app which
- * manages its own WS connection and auth (portal login).
- */
-
-let config = null; // { serverUrl, username, tier }
-
-// ── Config Management ──────────────────────────────────────────────────────
-
-async function loadConfig() {
-  const data = await chrome.storage.local.get(['serverUrl', 'username', 'tier']);
-  if (data.serverUrl) {
-    config = data;
-    console.log(`[bloby-bg] Config loaded: ${config.serverUrl} (${config.username})`);
-    return true;
-  }
-  console.log('[bloby-bg] No config found — not paired');
-  return false;
-}
-
-async function saveConfig(newConfig) {
-  config = newConfig;
-  await chrome.storage.local.set(newConfig);
-  console.log(`[bloby-bg] Config saved: ${config.serverUrl}`);
-}
-
-async function clearConfig() {
-  config = null;
-  await chrome.storage.local.clear();
-  console.log('[bloby-bg] Config cleared — unpaired');
-}
-
-// ── Broadcast to All Tabs ──────────────────────────────────────────────────
-
-async function broadcastToTabs(msg) {
-  try {
-    const tabs = await chrome.tabs.query({});
-    for (const tab of tabs) {
-      if (tab.id) {
-        chrome.tabs.sendMessage(tab.id, msg).catch(() => {});
-      }
-    }
-  } catch { /* ignore */ }
-}
-
-// ── Message Handler (from content scripts + popup) ─────────────────────────
-
-chrome.runtime.onMessage.addListener((msg, sender, sendResponse) => {
-  if (msg.type === 'bloby:get-state') {
-    sendResponse({
-      paired: !!config?.serverUrl,
-      connected: true, // iframe manages its own connection
-      config: config ? { serverUrl: config.serverUrl, username: config.username } : null,
-    });
-    return true;
-  }
-
-  if (msg.type === 'bloby:pair') {
-    handlePair(msg.code).then(sendResponse);
-    return true;
-  }
-
-  // Content script: take screenshot
-  if (msg.type === 'bloby:screenshot') {
-    chrome.tabs.captureVisibleTab(null, { format: 'png' }, (dataUrl) => {
-      sendResponse({ dataUrl: dataUrl || null });
-    });
-    return true; // async
-  }
-
-  if (msg.type === 'bloby:unpair') {
-    (async () => {
-      // Clear frame rules
-      try {
-        const existing = await chrome.declarativeNetRequest.getDynamicRules();
-        await chrome.declarativeNetRequest.updateDynamicRules({
-          removeRuleIds: existing.map((r) => r.id),
-        });
-      } catch {}
-      await clearConfig();
-      broadcastToTabs({ type: 'bloby:unpaired' });
-      sendResponse({ success: true });
-    })();
-    return true;
-  }
-});
-
-// ── Pairing Flow ───────────────────────────────────────────────────────────
-
-async function handlePair(code) {
-  try {
-    console.log(`[bloby-bg] Verifying pairing code: ${code}`);
-
-    const res = await fetch('https://api.bloby.bot/api/extension/pair/verify', {
-      method: 'POST',
-      headers: { 'Content-Type': 'application/json' },
-      body: JSON.stringify({ code }),
-    });
-
-    if (!res.ok) {
-      const data = await res.json().catch(() => ({}));
-      console.log(`[bloby-bg] Pairing failed: ${data.error || res.status}`);
-      return { success: false, error: data.error || 'Invalid code' };
-    }
-
-    const data = await res.json();
-    console.log(`[bloby-bg] Pairing verified: ${data.serverUrl}`);
-
-    await saveConfig({
-      serverUrl: data.serverUrl,
-      username: data.username,
-      tier: data.tier,
-    });
-
-    // Set up header rules to allow iframing the Bloby server
-    await updateFrameRules(data.serverUrl);
-
-    // Notify all tabs to show the bubble
-    broadcastToTabs({ type: 'bloby:paired', serverUrl: data.serverUrl });
-
-    return { success: true, username: data.username, serverUrl: data.serverUrl };
-  } catch (err) {
-    console.error('[bloby-bg] Pairing error:', err);
-    return { success: false, error: 'Network error — check your connection' };
-  }
-}
-
-// ── Frame Header Rules ─────────────────────────────────────────────────────
-// Strip X-Frame-Options and restrictive frame-ancestors from the Bloby server
-// so the chat can be loaded in the extension's panel iframe.
-
-async function updateFrameRules(serverUrl) {
-  const host = serverUrl.replace(/^https?:\/\//, '').replace(/\/$/, '');
-  console.log(`[bloby-bg] Setting frame rules for: *://${host}/*`);
-
-  try {
-    // Remove old rules
-    const existing = await chrome.declarativeNetRequest.getDynamicRules();
-    const removeIds = existing.map((r) => r.id);
-
-    await chrome.declarativeNetRequest.updateDynamicRules({
-      removeRuleIds: removeIds,
-      addRules: [
-        {
-          id: 1,
-          priority: 1,
-          action: {
-            type: 'modifyHeaders',
-            responseHeaders: [
-              { header: 'X-Frame-Options', operation: 'remove' },
-              { header: 'Content-Security-Policy', operation: 'remove' },
-            ],
-          },
-          condition: {
-            urlFilter: `*://${host}/*`,
-            resourceTypes: ['sub_frame'],
-          },
-        },
-      ],
-    });
-    console.log('[bloby-bg] Frame rules set successfully');
-  } catch (err) {
-    console.error('[bloby-bg] Failed to set frame rules:', err);
-  }
-}
-
-// ── Startup ────────────────────────────────────────────────────────────────
-
-(async () => {
-  const hasCfg = await loadConfig();
-  if (hasCfg && config.serverUrl) {
-    await updateFrameRules(config.serverUrl);
-  }
-})();

package/workspace/skills/chrome-extension/content-script.js

@@ -1,309 +0,0 @@
-/**
- * Bloby Chrome Extension — Content Script
- *
- * Injected into every page. Responsibilities:
- * - Renders the Bloby bubble (bottom-right corner)
- * - Opens/closes the chat panel (iframe to extension's panel.html)
- * - Extracts page context (URL, title, selection, structured data)
- * - Handles ExtensionAction commands from the agent (screenshot, fill form, type)
- * - Bridges messages between the panel iframe and the background worker
- */
-
-(function () {
-  // Guard: don't inject on extension pages or Bloby's own domain
-  if (document.getElementById('bloby-ext-bubble')) return;
-  if (location.hostname.endsWith('bloby.bot')) return;
-
-  let panelOpen = false;
-  let paired = false;
-
-  // ── Check if paired ────────────────────────────────────────────────────────
-
-  chrome.runtime.sendMessage({ type: 'bloby:get-state' }, (state) => {
-    if (chrome.runtime.lastError) return;
-    if (state?.paired) {
-      paired = true;
-      createBubble();
-    }
-  });
-
-  // Listen for pairing from background worker
-  chrome.runtime.onMessage.addListener((msg) => {
-    if (msg.type === 'bloby:paired') {
-      console.log('[bloby-ext] Paired! Showing bubble.');
-      paired = true;
-      if (!document.getElementById('bloby-ext-bubble')) createBubble();
-    }
-    if (msg.type === 'bloby:unpaired') {
-      paired = false;
-      removeBubble();
-    }
-  });
-
-  // Listen for pairing changes via storage
-  chrome.storage.onChanged.addListener((changes) => {
-    if (changes.serverUrl) {
-      if (changes.serverUrl.newValue) {
-        paired = true;
-        if (!document.getElementById('bloby-ext-bubble')) createBubble();
-      } else {
-        paired = false;
-        removeBubble();
-      }
-    }
-  });
-
-  // ── Bubble ─────────────────────────────────────────────────────────────────
-
-  function createBubble() {
-    const bubble = document.createElement('div');
-    bubble.id = 'bloby-ext-bubble';
-    bubble.title = 'Chat with Bloby';
-    bubble.addEventListener('click', togglePanel);
-    document.body.appendChild(bubble);
-
-    const dot = document.createElement('div');
-    dot.className = 'bloby-ext-bubble-dot';
-    bubble.appendChild(dot);
-  }
-
-  function removeBubble() {
-    document.getElementById('bloby-ext-bubble')?.remove();
-    document.getElementById('bloby-ext-panel')?.remove();
-    document.getElementById('bloby-ext-backdrop')?.remove();
-    panelOpen = false;
-  }
-
-  // ── Panel ──────────────────────────────────────────────────────────────────
-
-  function togglePanel() {
-    panelOpen ? closePanel() : openPanel();
-  }
-
-  function openPanel() {
-    if (document.getElementById('bloby-ext-panel')) {
-      document.getElementById('bloby-ext-panel').classList.add('bloby-ext-panel-open');
-      panelOpen = true;
-      showBackdrop();
-      return;
-    }
-
-    showBackdrop();
-
-    const panel = document.createElement('div');
-    panel.id = 'bloby-ext-panel';
-
-    const iframe = document.createElement('iframe');
-    iframe.id = 'bloby-ext-iframe';
-    iframe.src = chrome.runtime.getURL('panel/panel.html');
-    iframe.allow = 'microphone';
-    panel.appendChild(iframe);
-
-    document.body.appendChild(panel);
-
-    requestAnimationFrame(() => {
-      panel.classList.add('bloby-ext-panel-open');
-      panelOpen = true;
-    });
-
-    window.addEventListener('message', handlePanelMessage);
-  }
-
-  function closePanel() {
-    const panel = document.getElementById('bloby-ext-panel');
-    if (panel) {
-      panel.classList.remove('bloby-ext-panel-open');
-      panelOpen = false;
-      hideBackdrop();
-    }
-  }
-
-  function showBackdrop() {
-    let backdrop = document.getElementById('bloby-ext-backdrop');
-    if (!backdrop) {
-      backdrop = document.createElement('div');
-      backdrop.id = 'bloby-ext-backdrop';
-      backdrop.addEventListener('click', closePanel);
-      document.body.appendChild(backdrop);
-    }
-    requestAnimationFrame(() => backdrop.classList.add('bloby-ext-backdrop-visible'));
-  }
-
-  function hideBackdrop() {
-    const backdrop = document.getElementById('bloby-ext-backdrop');
-    if (backdrop) backdrop.classList.remove('bloby-ext-backdrop-visible');
-  }
-
-  // ── Message Bridge (panel ↔ content script) ────────────────────────────────
-
-  function handlePanelMessage(event) {
-    const iframe = document.getElementById('bloby-ext-iframe');
-    if (!iframe || event.source !== iframe.contentWindow) return;
-
-    const msg = event.data;
-    if (!msg?.type) return;
-
-    if (msg.type === 'bloby:close') {
-      closePanel();
-      return;
-    }
-
-    // Panel requests page context
-    if (msg.type === 'bloby:get-page-context') {
-      const context = getPageContext();
-      iframe.contentWindow.postMessage({ type: 'bloby:page-context', context }, '*');
-      return;
-    }
-
-    // Agent requests an action on the page
-    if (msg.type === 'bloby:extension-action') {
-      handleExtensionAction(msg.action, msg.data);
-      return;
-    }
-  }
-
-  // ── Page Context Extraction ────────────────────────────────────────────────
-
-  function getPageContext() {
-    const context = {
-      url: location.href,
-      title: document.title,
-      selection: window.getSelection()?.toString()?.slice(0, 2000) || '',
-    };
-
-    // Extract structured data (JSON-LD) — product info, articles, etc.
-    try {
-      const scripts = document.querySelectorAll('script[type="application/ld+json"]');
-      for (const script of scripts) {
-        const data = JSON.parse(script.textContent);
-        // Handle arrays (some sites wrap in array)
-        const items = Array.isArray(data) ? data : [data];
-        for (const item of items) {
-          if (item.name && !context.productName) context.productName = item.name;
-          if (item.offers?.price) context.price = item.offers.price;
-          if (item.offers?.priceCurrency) context.currency = item.offers.priceCurrency;
-          if (item.description && !context.description) context.description = item.description?.slice(0, 500);
-          if (item['@type'] === 'VideoObject' && item.name) context.videoTitle = item.name;
-        }
-      }
-    } catch { /* ignore */ }
-
-    // Meta description fallback
-    if (!context.description) {
-      const meta = document.querySelector('meta[name="description"]');
-      if (meta) context.description = meta.content?.slice(0, 500);
-    }
-
-    // Detect forms on the page
-    const forms = document.querySelectorAll('form');
-    if (forms.length > 0) {
-      context.hasForms = true;
-      context.formCount = forms.length;
-      // Extract visible form field labels/names
-      const fields = [];
-      forms.forEach((form) => {
-        form.querySelectorAll('input, select, textarea').forEach((el) => {
-          const name = el.getAttribute('name') || el.getAttribute('id') || el.getAttribute('placeholder') || '';
-          const type = el.getAttribute('type') || el.tagName.toLowerCase();
-          if (name && type !== 'hidden' && type !== 'submit') {
-            fields.push({ name, type });
-          }
-        });
-      });
-      if (fields.length > 0) context.formFields = fields.slice(0, 20); // cap at 20
-    }
-
-    console.log('[bloby-ext] Page context:', context);
-    return context;
-  }
-
-  // ── Extension Actions (agent → page) ───────────────────────────────────────
-
-  function handleExtensionAction(action, data) {
-    console.log(`[bloby-ext] Action: ${action}`, data);
-    const iframe = document.getElementById('bloby-ext-iframe');
-
-    switch (action) {
-      case 'screenshot': {
-        // Request screenshot via background worker (needs chrome.tabs API)
-        chrome.runtime.sendMessage({ type: 'bloby:screenshot' }, (response) => {
-          if (response?.dataUrl && iframe?.contentWindow) {
-            iframe.contentWindow.postMessage({
-              type: 'bloby:action-result',
-              action: 'screenshot',
-              result: response.dataUrl,
-            }, '*');
-          }
-        });
-        break;
-      }
-
-      case 'read-page': {
-        // Extract main text content from the page
-        const text = document.body.innerText?.slice(0, 10000) || '';
-        if (iframe?.contentWindow) {
-          iframe.contentWindow.postMessage({
-            type: 'bloby:action-result',
-            action: 'read-page',
-            result: text,
-          }, '*');
-        }
-        break;
-      }
-
-      case 'fill-form': {
-        // Fill form fields from agent-provided data
-        if (data?.fields && typeof data.fields === 'object') {
-          for (const [selector, value] of Object.entries(data.fields)) {
-            const el = document.querySelector(selector) ||
-                       document.querySelector(`[name="${selector}"]`) ||
-                       document.querySelector(`[id="${selector}"]`);
-            if (el) {
-              el.value = value;
-              el.dispatchEvent(new Event('input', { bubbles: true }));
-              el.dispatchEvent(new Event('change', { bubbles: true }));
-            }
-          }
-          if (iframe?.contentWindow) {
-            iframe.contentWindow.postMessage({
-              type: 'bloby:action-result',
-              action: 'fill-form',
-              result: 'done',
-            }, '*');
-          }
-        }
-        break;
-      }
-
-      case 'click': {
-        // Click an element by selector
-        if (data?.selector) {
-          const el = document.querySelector(data.selector);
-          if (el) el.click();
-        }
-        break;
-      }
-
-      case 'type': {
-        // Type into the currently focused input or a targeted selector
-        const target = data?.selector
-          ? document.querySelector(data.selector)
-          : document.activeElement;
-        if (target && data?.text) {
-          target.value = data.text;
-          target.dispatchEvent(new Event('input', { bubbles: true }));
-        }
-        break;
-      }
-
-      default:
-        console.warn(`[bloby-ext] Unknown action: ${action}`);
-    }
-  }
-
-  // ── Keyboard shortcut ──────────────────────────────────────────────────────
-
-  document.addEventListener('keydown', (e) => {
-    if (e.key === 'Escape' && panelOpen) closePanel();
-  });
-})();