opencode-browser-annotation-plugin 0.5.0 → 0.6.0

package/README.md

@@ -41,7 +41,23 @@ Configure with environment variables if needed:
 - `OPENCODE_ANNOTATION_PORT` (default `39517`)
 - `OPENCODE_ANNOTATION_HOST` (default `127.0.0.1`)
 
-### 2. Extension (in your desktop Chrome)
+### 2. Skill (on the OpenCode host)
+
+The agent learns how to interpret annotations from a bundled skill, so the
+annotation payload stays lean (no repeated instructions). Copy it into your
+OpenCode skills directory:
+
+```bash
+mkdir -p ~/.config/opencode/skills/browser-annotation
+cp node_modules/opencode-browser-annotation-plugin/skills/browser-annotation/SKILL.md \
+   ~/.config/opencode/skills/browser-annotation/SKILL.md
+```
+
+Restart OpenCode. The agent auto-loads this skill when an annotation arrives
+(the payload identifies itself and references the skill). Verify with
+`opencode debug skill`.
+
+### 3. Extension (in your desktop Chrome)
 
 Until it is on the Chrome Web Store, load it unpacked:
 
@@ -49,7 +65,7 @@ Until it is on the Chrome Web Store, load it unpacked:
 2. Open `chrome://extensions`, enable **Developer mode**, click **Load unpacked**, and select `extension/`.
 3. Use a dedicated Chrome profile — the same debug profile you drive with the agent is a good choice.
 
-### 3. Tunnel (when OpenCode is remote)
+### 4. Tunnel (when OpenCode is remote)
 
 The extension (on your desktop) needs to reach the plugin server (on the host),
 so use a **local forward**. Run on your desktop, in a separate window:
@@ -72,30 +88,48 @@ extension's **Settings** page.
 
 ## Use
 
-1. Send at least one message in OpenCode so the plugin knows the active session.
+1. Send at least one message in OpenCode so the plugin knows the active session
+   (or pick a target session in the sidebar).
 2. Press **Alt+A** on any page, then click an element (highlight follows the
    cursor; works inside shadow DOM). A popup opens next to it.
-3. In the popup, type an instruction and choose **Act now** (agent responds
-   immediately) or **Queue** (held for context until the next Act). Then either:
+3. In the popup, type an instruction, then either:
    - **Send** (or Cmd/Ctrl+Enter) — submit this one right away, no sidebar needed.
    - **Add to list** — stash it in the sidebar to batch with others.
 4. Press **Alt+Shift+A** (or click the toolbar icon) to open the **list sidebar**,
-   review pending annotations, and **Submit** them together. The footer shows the
-   connection status; a toast confirms how many were sent vs queued.
+   review pending annotations (each with a thumbnail), pick the **target session**,
+   and **Submit** them together. The footer shows a steady connection status.
 
 Shortcuts: `Alt+A` picks an element, `Alt+Shift+A` toggles the list. If they do
 nothing after loading an unpacked build, set them at `chrome://extensions/shortcuts`.
 
+## How the agent handles it
+
+When you submit, the plugin injects a short user turn into the chosen session
+that identifies itself as a browser annotation and references the
+`browser-annotation` skill. The agent loads that skill (installed in step 2) to
+interpret the element metadata and locate the code — so the interpretation rules
+live once in the skill, not in every message.
+
+If your agents use a project `AGENTS.md`, you can make routing explicit by adding
+a line such as:
+
+```
+When a turn is a browser annotation (it says so and references the
+`browser-annotation` skill), load that skill and use the element metadata
+(component path, data-testid/id/role, ancestors, nearest region, text) to locate
+and change the corresponding code. Confirm the element exists before editing.
+```
+
 ## Payload
 
 The extension POSTs to `/annotations`:
 
 ```json
 {
-  "extensionVersion": "0.2.0",
+  "extensionVersion": "0.6.0",
+  "sessionID": "ses_...",
   "annotations": [
     {
-      "mode": "act",
       "instruction": "Make this button larger and blue",
       "page": { "url": "https://example.com/app", "title": "My App" },
       "element": {
@@ -107,22 +141,28 @@ The extension POSTs to `/annotations`:
         "ariaLabel": "Sign up",
         "classes": ["cta", "primary"],
         "text": "Sign up",
+        "componentPath": "App > Header > SignupButton",
+        "framework": "react",
+        "landmark": "header[role=banner]",
+        "ancestors": ["div.actions", "header[role=banner]"],
         "bounds": { "x": 100, "y": 200, "width": 120, "height": 40 },
         "inShadow": false,
-        "html": "<button class=\"cta\">Sign up</button>"
+        "html": "<button class=\"cta\" id=\"signup\">"
       }
     }
   ]
 }
 ```
 
-`mode` is `"act"` (respond now, flushing any queued) or `"queue"` (hold for
-context until the next act). The plugin surfaces the most code-locatable
-identifiers first (testId, id, name, role) and treats the CSS path and bounds as
-weak hints.
+The plugin surfaces the most code-locatable identifiers first (component path,
+testId, id, name, role, ancestors, region) and treats the CSS path and bounds as
+weak hints. Build-generated/hashed class names are filtered out, text is
+whitespace-collapsed, and `html` is the element's opening tag with secret-looking
+attribute values redacted. `sessionID` is optional; without it the plugin targets
+the last-active session.
 
-`GET /status` returns `{ ok, activeSession, sessionID, sessionTitle, queued, host, port }`
-for a quick health check and to show the target session.
+`GET /status` returns `{ ok, activeSession, sessionID, sessionTitle, sessions, host, port }`
+for a quick health check and to list/target sessions.
 
 ## Develop
 

package/dist/plugin.js

@@ -74,7 +74,7 @@ function formatElement(el) {
         lines.push(`  viewport bounds: ${Math.round(b.width)}×${Math.round(b.height)} at (${Math.round(b.x)}, ${Math.round(b.y)})`);
     }
     if (el.html)
-        lines.push(`  outer html: ${truncate(el.html.trim(), 500)}`);
+        lines.push(`  opening tag: ${truncate(el.html.trim(), 500)}`);
     return lines.join("\n");
 }
 function annotationBlock(a, index, total) {
@@ -92,21 +92,15 @@ function annotationBlock(a, index, total) {
         .join("\n");
 }
 function buildPrompt(annotations) {
+    // The static interpretation rules live in the `browser-annotation` skill, not
+    // in every payload. The header marks this as a browser annotation and tells
+    // the agent to use that skill, so the rules are stated once (in the skill),
+    // never repeated per message.
     const header = annotations.length > 1
-        ? `The user made ${annotations.length} annotations in the browser. Address each one.`
-        : "The user made an annotation in the browser.";
+        ? `Browser annotations (${annotations.length}) — handle using the \`browser-annotation\` skill. Address each one.`
+        : "Browser annotation — handle using the `browser-annotation` skill.";
     const blocks = annotations.map((a, i) => annotationBlock(a, i, annotations.length));
-    return [
-        header,
-        "",
-        ...blocks,
-        "",
-        "Guidance:",
-        "- Locate the code for each element using the most stable identifier available: framework component path first if given, then data-testid, id, name, role, then the ancestor chain and nearest region to disambiguate, then unique class or text. Treat the CSS path and viewport bounds as weak hints only.",
-        "- Use the ancestors and nearest region to find the right component/file when the element itself is generic (e.g. a bare button that appears many times).",
-        "- Confirm the element actually exists in this codebase before editing; if you cannot find it, say so instead of guessing.",
-        "- No screenshot is attached; reason from the metadata and the code.",
-    ].join("\n");
+    return [header, "", ...blocks].join("\n");
 }
 async function readJsonBody(req, limitBytes = 2_000_000) {
     return await new Promise((resolve, reject) => {

package/extension/manifest.json

@@ -1,7 +1,7 @@
 {
   "manifest_version": 3,
   "name": "OpenCode Browser Annotation",
-  "version": "0.5.0",
+  "version": "0.6.0",
   "description": "Alt+A to annotate an element, Alt+Shift+A for the annotation list. Sends element metadata to your OpenCode agent.",
   "permissions": [
     "activeTab",

package/extension/overlay.js

@@ -139,6 +139,7 @@
   let sessions = []; // [{ id, title, updated }]
   let targetSessionID = null; // user-chosen target; null = auto (last active)
   let autoSessionID = null; // the plugin's last-active session
+  let lastStatusState = null; // "good" | "bad" | "checking" | null; avoids flicker
 
   // A content script keeps running after its extension is reloaded/updated, but
   // its chrome.* calls then throw "Extension context invalidated". Guard every
@@ -203,6 +204,25 @@
     return undefined;
   }
 
+  // Drop build-generated/hashed class names (CSS Modules, styled-components,
+  // Emotion, etc.). They change every build and don't map to source, so they are
+  // noise for locating code. Keep human-authored classes like "heading-element".
+  function isHashedClass(c) {
+    if (!c) return true;
+    if (/module__/i.test(c)) return true; // CSS Modules: Foo-module__Bar__hHXUL
+    if (/__[A-Za-z0-9]{4,}$/.test(c)) return true; // trailing __hash
+    if (/^css-[a-z0-9]{5,}$/i.test(c)) return true; // Emotion: css-1ab2c3
+    if (/^sc-[A-Za-z0-9]{5,}$/.test(c)) return true; // styled-components: sc-bdVaJa
+    if (/^[A-Za-z0-9]{7,}$/.test(c) && /[0-9]/.test(c) && /[A-Z]/.test(c)) return true; // mixed-case+digit opaque hash
+    return false;
+  }
+
+  function cleanClasses(list) {
+    if (!list || !list.length) return undefined;
+    const kept = list.filter((c) => !isHashedClass(c));
+    return kept.length ? kept : undefined;
+  }
+
   function cssPath(el) {
     if (!(el instanceof Element)) return "";
     if (el.id) return `#${CSS.escape(el.id)}`;
@@ -237,7 +257,8 @@
     else if (el.id) s += `#${el.id}`;
     const role = el.getAttribute && el.getAttribute("role");
     if (role) s += `[role=${role}]`;
-    if (el.classList && el.classList.length) s += "." + Array.from(el.classList).slice(0, 3).join(".");
+    const cls = cleanClasses(el.classList ? Array.from(el.classList) : []);
+    if (cls) s += "." + cls.slice(0, 3).join(".");
     return s;
   }
 
@@ -317,9 +338,41 @@
     return undefined;
   }
 
+  // Collapse runs of whitespace to single spaces and trim, so a container's
+  // textContent doesn't arrive as a wall of newlines.
+  function cleanText(s) {
+    const t = (s || "").replace(/\s+/g, " ").trim();
+    return t ? t.slice(0, 300) : undefined;
+  }
+
+  // Names/attributes whose values may be secrets and must never be sent.
+  const SECRET_ATTR = /(token|csrf|authenticity|secret|api[-_]?key|password|session|auth|nonce|signature)/i;
+
+  // For the outer HTML, send only the element's own opening tag (not the whole
+  // subtree), with secret-looking attribute values redacted. This avoids leaking
+  // CSRF tokens / API keys and keeps the payload small for large containers.
+  function safeOpenTag(el) {
+    try {
+      const tag = el.tagName.toLowerCase();
+      const attrs = [];
+      for (const a of Array.from(el.attributes || [])) {
+        let v = a.value;
+        if (SECRET_ATTR.test(a.name) || (a.name === "value" && /password|hidden/i.test(el.getAttribute("type") || ""))) {
+          v = "[redacted]";
+        } else if (v && v.length > 120) {
+          v = v.slice(0, 120) + "…";
+        }
+        attrs.push(v === "" ? a.name : `${a.name}="${v}"`);
+      }
+      const open = `<${tag}${attrs.length ? " " + attrs.join(" ") : ""}>`;
+      return open.slice(0, 600);
+    } catch {
+      return undefined;
+    }
+  }
+
   function elementMeta(el, inShadow) {
     const r = el.getBoundingClientRect();
-    const classes = el.classList ? Array.from(el.classList) : [];
     const fw = frameworkComponents(el);
     const m = {
       selector: cssPath(el),
@@ -329,8 +382,8 @@
       testId: bestTestId(el),
       role: el.getAttribute("role") || undefined,
       ariaLabel: el.getAttribute("aria-label") || undefined,
-      classes: classes.length ? classes : undefined,
-      text: (el.textContent || "").trim().slice(0, 500) || undefined,
+      classes: cleanClasses(el.classList ? Array.from(el.classList) : []),
+      text: cleanText(el.textContent),
       href: el.getAttribute("href") || undefined,
       src: el.getAttribute("src") || undefined,
       bounds: { x: r.left, y: r.top, width: r.width, height: r.height },
@@ -340,7 +393,7 @@
       landmark: nearestLandmark(el),
       componentPath: fw ? fw.path : undefined,
       framework: fw ? fw.framework : undefined,
-      html: (el.outerHTML || "").slice(0, 800),
+      html: safeOpenTag(el),
     };
     for (const k of Object.keys(m)) if (m[k] === undefined) delete m[k];
     if (Array.isArray(m.ancestors) && m.ancestors.length === 0) delete m.ancestors;
@@ -371,18 +424,22 @@
             const pad = 6 * dpr;
             let sx = Math.max(0, rect.left * dpr - pad);
             let sy = Math.max(0, rect.top * dpr - pad);
-            let sw = Math.min(img.width - sx, rect.width * dpr + pad * 2);
-            let sh = Math.min(img.height - sy, rect.height * dpr + pad * 2);
+            const sw = Math.min(img.width - sx, rect.width * dpr + pad * 2);
+            const sh = Math.min(img.height - sy, rect.height * dpr + pad * 2);
             if (sw <= 0 || sh <= 0) return resolve(null);
-            // cap output size for a compact list thumbnail
-            const maxW = 300;
+            // Keep the crop at (up to) full captured resolution so it stays sharp
+            // on the sidebar's HiDPI display. Only downscale if it's very large,
+            // to keep the data URL reasonable.
+            const maxW = 900;
             const scale = Math.min(1, maxW / sw);
-            const cw = Math.round(sw * scale);
-            const ch = Math.round(sh * scale);
+            const cw = Math.max(1, Math.round(sw * scale));
+            const ch = Math.max(1, Math.round(sh * scale));
             const c = document.createElement("canvas");
             c.width = cw;
             c.height = ch;
             const ctx = c.getContext("2d");
+            ctx.imageSmoothingEnabled = true;
+            ctx.imageSmoothingQuality = "high";
             ctx.drawImage(img, sx, sy, sw, sh, 0, 0, cw, ch);
             resolve(c.toDataURL("image/png"));
           } catch {
@@ -620,7 +677,7 @@
         </div>`;
       root.appendChild(sb);
       sb.querySelector("#oc-close").addEventListener("click", closeSidebar);
-      sb.querySelector("#oc-pick").addEventListener("click", () => startPicking());
+      sb.querySelector("#oc-pick").addEventListener("click", () => (picking ? stopPicking() : startPicking()));
       sb.querySelector("#oc-dd-btn").addEventListener("click", (e) => {
         e.stopPropagation();
         sb.querySelector("#oc-dd").classList.toggle("open");
@@ -654,6 +711,7 @@
       clearInterval(statusTimer);
       statusTimer = null;
     }
+    lastStatusState = null; // re-check cleanly on reopen
   }
 
   function toggleSidebar() {
@@ -693,27 +751,33 @@
 
   // ---------- status + submit ----------
 
+  // Apply a status only when it actually changes, so routine polls don't make
+  // the text flicker. Only the very first check shows "Checking…".
+  function setStatus(state, text) {
+    const el = root && root.getElementById("oc-status");
+    if (!el) return;
+    if (lastStatusState === state) return; // stable during polls
+    lastStatusState = state;
+    el.className = `oc-status ${state}`;
+    el.textContent = text;
+  }
+
   function refreshStatus() {
     if (!root) return;
-    const el = root.getElementById("oc-status");
-    if (!el) return;
-    el.className = "oc-status checking";
-    el.textContent = "Checking connection…";
+    if (lastStatusState === null) setStatus("checking", "Checking connection…");
     sendMsg({ type: "oc-status" }, (res) => {
       if (!res) {
-        el.className = "oc-status bad";
-        el.textContent = "Extension error";
+        setStatus("bad", "Extension error");
         return;
       }
       if (res.ok && res.data?.ok) {
+        // Session list updates every poll; the status text stays stable.
         sessions = Array.isArray(res.data.sessions) ? res.data.sessions : [];
         autoSessionID = res.data.sessionID || null;
         renderSessions();
-        el.className = "oc-status good";
-        el.textContent = "Connected";
+        setStatus("good", "Connected");
       } else {
-        el.className = "oc-status bad";
-        el.textContent = "Not connected — check the SSH tunnel";
+        setStatus("bad", "Not connected — check the SSH tunnel");
       }
     });
   }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "opencode-browser-annotation-plugin",
-  "version": "0.5.0",
+  "version": "0.6.0",
   "description": "Select an element in your browser, type an instruction, and send it to your OpenCode agent over a loopback + SSH tunnel. Text and element metadata only (no screenshots).",
   "keywords": [
     "opencode",
@@ -28,6 +28,7 @@
   "files": [
     "dist",
     "extension",
+    "skills",
     "README.md"
   ],
   "scripts": {

package/skills/browser-annotation/SKILL.md

@@ -0,0 +1,60 @@
+---
+name: browser-annotation
+description: >-
+  Interpret and act on browser annotations sent by the
+  opencode-browser-annotation extension. Load this whenever a message says it is
+  a "Browser annotation" or references the `browser-annotation` skill, or when a
+  turn contains a "Selected element:" block with fields like data-testid,
+  ancestors, nearest region, or a component path. It explains how to turn that
+  element metadata into the correct code location and change.
+---
+
+# Browser annotations
+
+Messages from the `opencode-browser-annotation` extension describe a DOM element
+the user selected in their browser plus an instruction. Your job is to map that
+element to the right code and carry out the instruction. The payload is text and
+element metadata only — no screenshot is attached.
+
+## How to locate the code
+
+Use the most stable identifier available, in this order:
+
+1. **Framework component path** (e.g. `App > Dashboard > SignupCard`) — the
+   strongest hint. Find that component's source file first.
+2. **`data-testid` / `id` / `name` / `role`** on the element itself — usually
+   maps directly to source.
+3. **Ancestors (nearest first) + nearest region** — use these to disambiguate a
+   generic element (e.g. a bare `<button>` that appears many times). They tell
+   you which container/feature/file the element belongs to.
+4. **Unique class or text** — human-authored class names and the element's text
+   are good locators. Build-generated/hashed class names are already filtered
+   out; treat any remaining opaque tokens as weak.
+5. **CSS path and viewport bounds** — weak hints only; do not rely on them.
+
+## Rules
+
+- Confirm the element actually exists in this codebase before editing. If you
+  cannot find it, say so instead of guessing or inventing a file.
+- Reason from the metadata and the actual code; there is no image.
+- The `opening tag` field is the element's own opening tag with secret-looking
+  attribute values redacted — do not expect a full HTML subtree, and never try
+  to recover redacted values.
+- If several annotations arrive together, address each one.
+- Treat the page URL as context for which route/screen the user was on.
+
+## Field reference
+
+- `Element` — tag name.
+- `data-testid`, `id`, `name`, `role`, `aria-label` — element identifiers.
+- `classes` — authored class names (hashed/build-generated ones removed).
+- `text` — the element's visible text (whitespace-collapsed, truncated).
+- `href` / `src` — link or media source when present.
+- `<framework> components` — component hierarchy from React/Preact/Vue when
+  detectable.
+- `nearest region` — closest semantic landmark/region ancestor.
+- `ancestors (nearest first)` — parent chain signatures, shadow DOM pierced.
+- `css path` — a positional selector; weak.
+- `context` — notes such as "inside a shadow DOM" or "inside an iframe".
+- `viewport bounds` — on-screen size/position; weak.
+- `opening tag` — redacted opening tag of the element.