opencode-browser-annotation-plugin 0.1.3 → 0.2.1

package/README.md

@@ -12,16 +12,16 @@ the code from the selector/DOM context you send.
 
 ```
 Desktop Chrome (extension)
-  └─ pick element + type instruction → list → [Submit]
-       └─ POST http://127.0.0.1:39517/annotations   (via ssh -R when remote)
+  └─ Alt+A → overlay + sidebar → pick element, type instruction, Act/Queue → [Submit]
+       └─ POST http://127.0.0.1:39517/annotations   (via ssh -L when remote)
 OpenCode host (plugin)
   └─ HTTP server on 127.0.0.1 → injects a turn into the active session
        └─ agent receives the instruction + element metadata and acts
 ```
 
 The plugin binds to `127.0.0.1` only. When OpenCode is on a remote host, an
-`ssh -R` reverse tunnel carries the extension's POST to it — no public port, and
-SSH provides the auth and encryption.
+`ssh -L` local forward carries the extension's POST from your desktop to it — no
+public port, and SSH provides the auth and encryption.
 
 ## Install
 
@@ -51,16 +51,19 @@ Until it is on the Chrome Web Store, load it unpacked:
 
 ### 3. Tunnel (when OpenCode is remote)
 
-On your desktop, forward the plugin port to the server (separate window):
+The extension (on your desktop) needs to reach the plugin server (on the host),
+so use a **local forward**. Run on your desktop, in a separate window:
 
 ```bash
-ssh -N -R 39517:127.0.0.1:39517 you@server
+ssh -N -L 39517:127.0.0.1:39517 you@host
 ```
 
-If you also drive the browser from the agent (desktop-drive), forward both:
+If you also drive the browser from the agent (desktop-drive on 9333), that is the
+opposite direction (host → desktop), so it uses a reverse forward. You can run
+both at once:
 
 ```bash
-ssh -N -R 9333:127.0.0.1:9333 -R 39517:127.0.0.1:39517 you@server
+ssh -N -L 39517:127.0.0.1:39517 -R 9333:127.0.0.1:9333 you@host
 ```
 
 If OpenCode runs on the same machine as your browser, no tunnel is needed — the
@@ -70,10 +73,15 @@ extension's **Settings** page.
 ## Use
 
 1. Send at least one message in OpenCode so the plugin knows the active session.
-2. Click the extension icon → **Select element**.
-3. Hover to highlight, click the element, type your instruction (Esc cancels).
-4. Repeat to queue more, then **Submit to agent**.
-5. The agent receives the annotations as a new turn and responds.
+2. Press **Alt+A** on any page to open the annotation overlay (or click the
+   toolbar icon). Press Alt+A again or Esc to close it.
+3. In the sidebar, click **Select element**, hover to highlight, and click the
+   element you want (works inside shadow DOM).
+4. Type an instruction in the card. Choose **Act now** (agent responds
+   immediately) or **Queue** (held for context until the next Act). Repeat for
+   more elements.
+5. Click **Submit to agent**. The sidebar footer shows the connection and active
+   session; a toast confirms how many were sent vs queued.
 
 ## Payload
 
@@ -81,18 +89,23 @@ The extension POSTs to `/annotations`:
 
 ```json
 {
-  "extensionVersion": "0.1.0",
+  "extensionVersion": "0.2.0",
   "annotations": [
     {
+      "mode": "act",
       "instruction": "Make this button larger and blue",
       "page": { "url": "https://example.com/app", "title": "My App" },
       "element": {
         "selector": "button.cta",
         "tag": "BUTTON",
-        "text": "Sign up",
+        "id": "signup",
+        "testId": "signup-cta",
         "role": "button",
         "ariaLabel": "Sign up",
+        "classes": ["cta", "primary"],
+        "text": "Sign up",
         "bounds": { "x": 100, "y": 200, "width": 120, "height": 40 },
+        "inShadow": false,
         "html": "<button class=\"cta\">Sign up</button>"
       }
     }
@@ -100,7 +113,13 @@ The extension POSTs to `/annotations`:
 }
 ```
 
-`GET /status` returns `{ ok, activeSession, host, port }` for a quick health check.
+`mode` is `"act"` (respond now, flushing any queued) or `"queue"` (hold for
+context until the next act). The plugin surfaces the most code-locatable
+identifiers first (testId, id, name, role) and treats the CSS path and bounds as
+weak hints.
+
+`GET /status` returns `{ ok, activeSession, sessionID, sessionTitle, queued, host, port }`
+for a quick health check and to show the target session.
 
 ## Develop
 

package/dist/plugin.js

@@ -3,10 +3,13 @@ import { createServer } from "node:http";
  * opencode-browser-annotation-plugin
  *
  * Runs a loopback HTTP server on the OpenCode host. A browser extension POSTs
- * annotations (a typed instruction plus selected-element metadata) to it, over
- * an `ssh -R` reverse tunnel when the browser is on a separate desktop. On
- * receipt, the plugin injects a new user turn into the most recently active
- * OpenCode session so the agent responds.
+ * annotations (a typed instruction plus selected-element metadata) to it. When
+ * the browser runs on a separate desktop, the extension reaches this server over
+ * an `ssh -L` local forward (desktop -> host).
+ *
+ * On receipt the plugin injects a new user turn into the most recently active
+ * OpenCode session so the agent responds. Annotations may be acted on
+ * immediately ("act") or queued and flushed with a later act ("queue").
  *
  * Scope: text + element metadata only. No screenshots, no image/vision.
  */
@@ -23,52 +26,79 @@ function envPort() {
 function truncate(value, max) {
     return value.length > max ? `${value.slice(0, max)}…` : value;
 }
+/**
+ * Surface the most code-locatable identifiers first: test ids and framework
+ * hooks are what map to source, not pixel bounds.
+ */
 function formatElement(el) {
     if (!el)
-        return "- (no element captured)";
+        return "  (no element captured)";
     const lines = [];
-    if (el.tag)
-        lines.push(`- Tag: ${el.tag}`);
-    if (el.selector)
-        lines.push(`- Selector: ${el.selector}`);
+    const tag = el.tag ? el.tag.toLowerCase() : "element";
+    lines.push(`  Element: <${tag}>`);
+    if (el.testId)
+        lines.push(`  data-testid: ${el.testId}`);
+    if (el.id)
+        lines.push(`  id: ${el.id}`);
+    if (el.name)
+        lines.push(`  name: ${el.name}`);
     if (el.role)
-        lines.push(`- Role: ${el.role}`);
+        lines.push(`  role: ${el.role}`);
     if (el.ariaLabel)
-        lines.push(`- ARIA label: ${el.ariaLabel}`);
+        lines.push(`  aria-label: ${el.ariaLabel}`);
+    if (el.classes && el.classes.length)
+        lines.push(`  classes: ${el.classes.slice(0, 8).join(" ")}`);
+    if (el.href)
+        lines.push(`  href: ${el.href}`);
+    if (el.src)
+        lines.push(`  src: ${el.src}`);
     if (el.text)
-        lines.push(`- Text: ${truncate(el.text.trim(), 300)}`);
+        lines.push(`  text: ${JSON.stringify(truncate(el.text.trim(), 200))}`);
+    if (el.selector)
+        lines.push(`  css path: ${el.selector}`);
+    const context = [];
+    if (el.inShadow)
+        context.push("inside a shadow DOM");
+    if (el.inIframe)
+        context.push(`inside an iframe${el.framePath ? ` (${el.framePath})` : ""}`);
+    if (context.length)
+        lines.push(`  context: ${context.join(", ")}`);
     if (el.bounds) {
         const b = el.bounds;
-        lines.push(`- Bounds: x=${Math.round(b.x)} y=${Math.round(b.y)} w=${Math.round(b.width)} h=${Math.round(b.height)}`);
+        lines.push(`  viewport bounds: ${Math.round(b.width)}×${Math.round(b.height)} at (${Math.round(b.x)}, ${Math.round(b.y)})`);
     }
     if (el.html)
-        lines.push(`- Outer HTML: ${truncate(el.html.trim(), 600)}`);
-    return lines.length ? lines.join("\n") : "- (no element details)";
+        lines.push(`  outer html: ${truncate(el.html.trim(), 500)}`);
+    return lines.join("\n");
+}
+function annotationBlock(a, index, total) {
+    const label = total > 1 ? ` ${index + 1}` : "";
+    const page = a.page ?? {};
+    const instruction = (a.instruction ?? "").trim() || "(no instruction text)";
+    return [
+        `### Annotation${label}`,
+        `Instruction: ${instruction}`,
+        page.url ? `Page: ${page.title ? `${page.title} — ` : ""}${page.url}` : "",
+        "Selected element:",
+        formatElement(a.element),
+    ]
+        .filter(Boolean)
+        .join("\n");
 }
 function buildPrompt(annotations) {
     const header = annotations.length > 1
-        ? `The user submitted ${annotations.length} browser annotations. Address each one.`
-        : "The user submitted a browser annotation.";
-    const blocks = annotations.map((a, i) => {
-        const n = annotations.length > 1 ? ` ${i + 1}` : "";
-        const page = a.page ?? {};
-        const instruction = (a.instruction ?? "").trim() || "(no instruction text)";
-        return [
-            `## Annotation${n}`,
-            `Instruction: ${instruction}`,
-            page.url ? `Page: ${page.title ? `${page.title} — ` : ""}${page.url}` : "",
-            "Selected element:",
-            formatElement(a.element),
-        ]
-            .filter(Boolean)
-            .join("\n");
-    });
+        ? `The user made ${annotations.length} annotations in the browser. Address each one.`
+        : "The user made an annotation in the browser.";
+    const blocks = annotations.map((a, i) => annotationBlock(a, i, annotations.length));
     return [
         header,
         "",
         ...blocks,
         "",
-        "Use the element metadata to locate the relevant code and make the requested change. No screenshot is attached.",
+        "Guidance:",
+        "- Locate the code for each element using the most stable identifier available (data-testid, id, name, role, then unique class or text). Treat the CSS path and viewport bounds as weak hints only.",
+        "- Confirm the element actually exists in this codebase before editing; if you cannot find it, say so instead of guessing.",
+        "- No screenshot is attached; reason from the metadata and the code.",
     ].join("\n");
 }
 async function readJsonBody(req, limitBytes = 2_000_000) {
@@ -112,38 +142,71 @@ export const BrowserAnnotationPlugin = async ({ client, directory }) => {
     const host = envHost();
     const port = envPort();
     let activeSessionID = null;
+    let activeSessionTitle = null;
     let server = null;
+    const queued = [];
     const log = (level, message, extra) => {
         void client.app
             .log({ body: { service: "browser-annotation", level, message, extra } })
             .catch(() => { });
     };
-    async function inject(annotations) {
-        if (!activeSessionID) {
-            return { ok: false, error: "No active OpenCode session yet. Send a message in OpenCode first." };
-        }
+    async function injectPrompt(annotations) {
         try {
-            // promptAsync injects the turn without blocking on the agent's full
-            // response, so the extension gets a prompt acknowledgement.
             await client.session.promptAsync({
                 path: { id: activeSessionID },
                 query: { directory },
                 body: { parts: [{ type: "text", text: buildPrompt(annotations) }] },
             });
-            return { ok: true, sessionID: activeSessionID };
+            return { ok: true };
         }
         catch (error) {
-            const message = error instanceof Error ? error.message : "session.prompt failed";
-            return { ok: false, error: message, sessionID: activeSessionID };
+            return { ok: false, error: error instanceof Error ? error.message : "session.prompt failed" };
         }
     }
+    /**
+     * Queue annotations are held until an "act" annotation arrives (or all-queue
+     * submits nothing yet). An act annotation flushes the queue plus itself.
+     */
+    async function handleSubmit(annotations) {
+        if (!activeSessionID) {
+            return {
+                ok: false,
+                error: "No active OpenCode session yet. Send a message in OpenCode first.",
+                injected: 0,
+                queued: queued.length,
+            };
+        }
+        const toQueue = annotations.filter((a) => a.mode === "queue");
+        const toAct = annotations.filter((a) => a.mode !== "queue");
+        queued.push(...toQueue);
+        if (toAct.length === 0) {
+            return { ok: true, injected: 0, queued: queued.length, sessionID: activeSessionID };
+        }
+        const batch = [...queued, ...toAct];
+        queued.length = 0;
+        const result = await injectPrompt(batch);
+        if (!result.ok) {
+            // Re-queue so nothing is lost.
+            queued.unshift(...batch.filter((a) => a.mode === "queue"));
+            return { ok: false, error: result.error, injected: 0, queued: queued.length, sessionID: activeSessionID };
+        }
+        return { ok: true, injected: batch.length, queued: queued.length, sessionID: activeSessionID };
+    }
     function handle(req, res) {
         if (req.method === "OPTIONS") {
             sendJson(res, 204, {});
             return;
         }
         if (req.method === "GET" && req.url === "/status") {
-            sendJson(res, 200, { ok: true, activeSession: Boolean(activeSessionID), host, port });
+            sendJson(res, 200, {
+                ok: true,
+                activeSession: Boolean(activeSessionID),
+                sessionID: activeSessionID,
+                sessionTitle: activeSessionTitle,
+                queued: queued.length,
+                host,
+                port,
+            });
             return;
         }
         if (req.method === "POST" && req.url === "/annotations") {
@@ -155,13 +218,15 @@ export const BrowserAnnotationPlugin = async ({ client, directory }) => {
                     sendJson(res, 400, { ok: false, error: "No annotations in payload." });
                     return;
                 }
-                const result = await inject(annotations);
+                const result = await handleSubmit(annotations);
                 if (result.ok) {
-                    log("info", `Injected ${annotations.length} annotation(s)`, { sessionID: result.sessionID });
-                    sendJson(res, 200, { ok: true, count: annotations.length, sessionID: result.sessionID });
+                    log("info", `Annotations: injected ${result.injected}, queued ${result.queued}`, {
+                        sessionID: result.sessionID,
+                    });
+                    sendJson(res, 200, result);
                 }
                 else {
-                    log("warn", `Annotation injection failed: ${result.error}`);
+                    log("warn", `Annotation submit failed: ${result.error}`);
                     sendJson(res, 409, result);
                 }
             })
@@ -198,9 +263,17 @@ export const BrowserAnnotationPlugin = async ({ client, directory }) => {
                 activeSessionID = input.sessionID;
         },
         event: async ({ event }) => {
-            if (event.type === "session.deleted") {
-                if (event.properties.info.id === activeSessionID)
+            if (event.type === "session.updated") {
+                const info = event.properties.info;
+                if (info.id === activeSessionID && typeof info.title === "string") {
+                    activeSessionTitle = info.title;
+                }
+            }
+            else if (event.type === "session.deleted") {
+                if (event.properties.info.id === activeSessionID) {
                     activeSessionID = null;
+                    activeSessionTitle = null;
+                }
             }
         },
     };

package/extension/background.js

@@ -1,6 +1,8 @@
-// Background service worker: holds the annotation list and submits the batch to
-// the plugin's loopback HTTP server (reached over an ssh -R tunnel when the
-// OpenCode host is remote).
+// Background service worker.
+// - Toggles the annotation overlay on Alt+A (command) or toolbar click.
+// - Performs all network I/O to the plugin endpoint (status + submit), because
+//   page CSP can block a content script from fetching localhost. Results are
+//   returned to the overlay via message responses.
 
 const DEFAULT_ENDPOINT = "http://127.0.0.1:39517";
 const EXTENSION_VERSION = chrome.runtime.getManifest().version;
@@ -10,47 +12,52 @@ async function getEndpoint() {
   return (endpoint || DEFAULT_ENDPOINT).replace(/\/+$/, "");
 }
 
-async function getAnnotations() {
-  const { annotations } = await chrome.storage.local.get("annotations");
-  return Array.isArray(annotations) ? annotations : [];
+async function toggleOverlay(tab) {
+  if (!tab?.id) return;
+  try {
+    // The overlay content script listens for this and toggles itself. If it is
+    // not injected yet, inject it first, then it opens on load.
+    const [{ result } = {}] = await chrome.scripting.executeScript({
+      target: { tabId: tab.id },
+      func: () => Boolean(window.__ocAnnotationInjected),
+    });
+    if (!result) {
+      await chrome.scripting.executeScript({ target: { tabId: tab.id }, files: ["overlay.js"] });
+    } else {
+      await chrome.tabs.sendMessage(tab.id, { type: "oc-toggle" });
+    }
+  } catch {
+    // e.g. chrome:// pages or the web store cannot be injected.
+  }
 }
 
-async function setAnnotations(list) {
-  await chrome.storage.local.set({ annotations: list });
-}
+chrome.commands.onCommand.addListener(async (command) => {
+  if (command !== "toggle-overlay") return;
+  const [tab] = await chrome.tabs.query({ active: true, currentWindow: true });
+  await toggleOverlay(tab);
+});
+
+chrome.action.onClicked.addListener((tab) => {
+  void toggleOverlay(tab);
+});
 
 chrome.runtime.onMessage.addListener((msg, _sender, sendResponse) => {
   (async () => {
-    if (msg?.type === "oc-add-annotation" && msg.annotation) {
-      const list = await getAnnotations();
-      list.push({ ...msg.annotation, ts: Date.now() });
-      await setAnnotations(list);
-      sendResponse({ ok: true, count: list.length });
-      return;
-    }
-
-    if (msg?.type === "oc-list") {
-      sendResponse({ ok: true, annotations: await getAnnotations() });
-      return;
-    }
-
-    if (msg?.type === "oc-clear") {
-      await setAnnotations([]);
-      sendResponse({ ok: true });
-      return;
-    }
-
-    if (msg?.type === "oc-remove" && typeof msg.index === "number") {
-      const list = await getAnnotations();
-      list.splice(msg.index, 1);
-      await setAnnotations(list);
-      sendResponse({ ok: true, annotations: list });
+    if (msg?.type === "oc-status") {
+      try {
+        const endpoint = await getEndpoint();
+        const res = await fetch(`${endpoint}/status`, { method: "GET" });
+        const data = await res.json().catch(() => ({}));
+        sendResponse({ ok: res.ok, endpoint, data });
+      } catch (error) {
+        sendResponse({ ok: false, error: error?.message || "unreachable" });
+      }
       return;
     }
 
     if (msg?.type === "oc-submit") {
-      const list = await getAnnotations();
-      if (list.length === 0) {
+      const annotations = Array.isArray(msg.annotations) ? msg.annotations : [];
+      if (annotations.length === 0) {
         sendResponse({ ok: false, error: "No annotations to submit." });
         return;
       }
@@ -59,12 +66,11 @@ chrome.runtime.onMessage.addListener((msg, _sender, sendResponse) => {
         const res = await fetch(`${endpoint}/annotations`, {
           method: "POST",
           headers: { "content-type": "application/json" },
-          body: JSON.stringify({ extensionVersion: EXTENSION_VERSION, annotations: list }),
+          body: JSON.stringify({ extensionVersion: EXTENSION_VERSION, annotations }),
         });
         const data = await res.json().catch(() => ({}));
         if (res.ok && data.ok) {
-          await setAnnotations([]);
-          sendResponse({ ok: true, count: data.count });
+          sendResponse({ ok: true, injected: data.injected ?? 0, queued: data.queued ?? 0, sessionID: data.sessionID });
         } else {
           sendResponse({ ok: false, error: data.error || `HTTP ${res.status}` });
         }

package/extension/manifest.json

@@ -1,17 +1,32 @@
 {
   "manifest_version": 3,
   "name": "OpenCode Browser Annotation",
-  "version": "0.1.0",
-  "description": "Select an element, type an instruction, and send it to your OpenCode agent. Text and element metadata only.",
-  "permissions": ["activeTab", "scripting", "storage"],
-  "host_permissions": ["http://127.0.0.1/*", "http://localhost/*"],
-  "action": {
-    "default_popup": "popup.html",
-    "default_title": "OpenCode Annotation"
-  },
+  "version": "0.2.1",
+  "description": "Alt+A to annotate elements and send them to your OpenCode agent. Text and element metadata only.",
+  "permissions": [
+    "activeTab",
+    "scripting",
+    "storage"
+  ],
+  "host_permissions": [
+    "http://127.0.0.1/*",
+    "http://localhost/*"
+  ],
   "background": {
     "service_worker": "background.js",
     "type": "module"
   },
-  "options_page": "options.html"
+  "action": {
+    "default_title": "Toggle OpenCode Annotation (Alt+A)"
+  },
+  "options_page": "options.html",
+  "commands": {
+    "toggle-overlay": {
+      "suggested_key": {
+        "default": "Alt+A",
+        "mac": "Alt+A"
+      },
+      "description": "Toggle the annotation overlay"
+    }
+  }
 }

package/extension/options.html

@@ -3,65 +3,44 @@
   <head>
     <meta charset="utf-8" />
     <style>
+      :root { color-scheme: light; }
       body {
-        font-family: system-ui, sans-serif;
-        max-width: 520px;
-        margin: 24px auto;
-        padding: 0 16px;
-        color: #1a1a1a;
-      }
-      label {
-        display: block;
-        font-weight: 600;
-        margin-bottom: 6px;
+        font: 14px/1.5 system-ui, -apple-system, sans-serif;
+        max-width: 540px; margin: 40px auto; padding: 0 20px; color: #1c1d21;
       }
+      h1 { font-size: 18px; margin: 0 0 4px; }
+      p.sub { color: #6a6d75; margin: 0 0 24px; }
+      label { display: block; font-weight: 650; margin-bottom: 6px; }
       input {
-        width: 100%;
-        font: inherit;
-        padding: 8px;
-        border: 1px solid #ccc;
-        border-radius: 6px;
-        box-sizing: border-box;
+        width: 100%; box-sizing: border-box; font: inherit;
+        padding: 10px 12px; border: 1px solid rgba(0,0,0,.15); border-radius: 10px;
       }
+      input:focus { outline: none; border-color: #2b7cff; box-shadow: 0 0 0 3px rgba(43,124,255,.15); }
       button {
-        font: inherit;
-        margin-top: 12px;
-        padding: 8px 14px;
-        border: 1px solid #2b7cff;
-        border-radius: 6px;
-        background: #2b7cff;
-        color: #fff;
-        cursor: pointer;
-      }
-      p {
-        color: #555;
-        font-size: 13px;
-        line-height: 1.5;
-      }
-      code {
-        background: #f2f2f2;
-        padding: 1px 4px;
-        border-radius: 4px;
-      }
-      #saved {
-        color: #0a0;
-        font-size: 13px;
-        margin-left: 8px;
+        font: inherit; font-weight: 600; margin-top: 16px;
+        padding: 9px 16px; border-radius: 10px; cursor: pointer;
+        color: #fff; background: #2b7cff; border: 1px solid #2b7cff;
       }
+      button:hover { background: #1e6ef0; }
+      .hint { color: #6a6d75; font-size: 13px; line-height: 1.6; margin-top: 18px; }
+      code { background: #f0f1f4; padding: 2px 6px; border-radius: 6px; font: 12.5px ui-monospace, monospace; }
+      #saved { color: #178048; font-size: 13px; margin-left: 10px; }
     </style>
   </head>
   <body>
-    <h1>OpenCode Annotation settings</h1>
+    <h1>OpenCode Annotation</h1>
+    <p class="sub">Send element annotations to your OpenCode agent. Text and metadata only.</p>
+
     <label for="endpoint">Plugin endpoint</label>
     <input id="endpoint" type="text" placeholder="http://127.0.0.1:39517" />
-    <p>
-      The URL of the plugin's HTTP server. When OpenCode runs on a remote host,
-      forward the port with a reverse tunnel and point this at the local end:
-      <br />
-      <code>ssh -N -R 39517:127.0.0.1:39517 you@server</code>
-    </p>
     <button id="save">Save</button>
     <span id="saved"></span>
+
+    <div class="hint">
+      <p><strong>Shortcut:</strong> press <code>Alt+A</code> on any page to toggle the annotation overlay.</p>
+      <p>When OpenCode runs on a remote host, forward the port from your desktop and point this at the local end:</p>
+      <p><code>ssh -N -L 39517:127.0.0.1:39517 you@host</code></p>
+    </div>
     <script src="options.js"></script>
   </body>
 </html>