opencode-browser-annotation-plugin 0.5.1 → 0.6.1

package/README.md

@@ -41,7 +41,23 @@ Configure with environment variables if needed:
 - `OPENCODE_ANNOTATION_PORT` (default `39517`)
 - `OPENCODE_ANNOTATION_HOST` (default `127.0.0.1`)
 
-### 2. Extension (in your desktop Chrome)
+### 2. Skill (on the OpenCode host)
+
+The agent learns how to interpret annotations from a bundled skill, so the
+annotation payload stays lean (no repeated instructions). Copy it into your
+OpenCode skills directory:
+
+```bash
+mkdir -p ~/.config/opencode/skills/browser-annotation
+cp node_modules/opencode-browser-annotation-plugin/skills/browser-annotation/SKILL.md \
+   ~/.config/opencode/skills/browser-annotation/SKILL.md
+```
+
+Restart OpenCode. The agent auto-loads this skill when an annotation arrives
+(the payload identifies itself and references the skill). Verify with
+`opencode debug skill`.
+
+### 3. Extension (in your desktop Chrome)
 
 Until it is on the Chrome Web Store, load it unpacked:
 
@@ -49,7 +65,7 @@ Until it is on the Chrome Web Store, load it unpacked:
 2. Open `chrome://extensions`, enable **Developer mode**, click **Load unpacked**, and select `extension/`.
 3. Use a dedicated Chrome profile — the same debug profile you drive with the agent is a good choice.
 
-### 3. Tunnel (when OpenCode is remote)
+### 4. Tunnel (when OpenCode is remote)
 
 The extension (on your desktop) needs to reach the plugin server (on the host),
 so use a **local forward**. Run on your desktop, in a separate window:
@@ -72,30 +88,48 @@ extension's **Settings** page.
 
 ## Use
 
-1. Send at least one message in OpenCode so the plugin knows the active session.
+1. Send at least one message in OpenCode so the plugin knows the active session
+   (or pick a target session in the sidebar).
 2. Press **Alt+A** on any page, then click an element (highlight follows the
    cursor; works inside shadow DOM). A popup opens next to it.
-3. In the popup, type an instruction and choose **Act now** (agent responds
-   immediately) or **Queue** (held for context until the next Act). Then either:
+3. In the popup, type an instruction, then either:
    - **Send** (or Cmd/Ctrl+Enter) — submit this one right away, no sidebar needed.
    - **Add to list** — stash it in the sidebar to batch with others.
 4. Press **Alt+Shift+A** (or click the toolbar icon) to open the **list sidebar**,
-   review pending annotations, and **Submit** them together. The footer shows the
-   connection status; a toast confirms how many were sent vs queued.
+   review pending annotations (each with a thumbnail), pick the **target session**,
+   and **Submit** them together. The footer shows a steady connection status.
 
 Shortcuts: `Alt+A` picks an element, `Alt+Shift+A` toggles the list. If they do
 nothing after loading an unpacked build, set them at `chrome://extensions/shortcuts`.
 
+## How the agent handles it
+
+When you submit, the plugin injects a short user turn into the chosen session
+that identifies itself as a browser annotation and references the
+`browser-annotation` skill. The agent loads that skill (installed in step 2) to
+interpret the element metadata and locate the code — so the interpretation rules
+live once in the skill, not in every message.
+
+If your agents use a project `AGENTS.md`, you can make routing explicit by adding
+a line such as:
+
+```
+When a turn is a browser annotation (it says so and references the
+`browser-annotation` skill), load that skill and use the element metadata
+(component path, data-testid/id/role, ancestors, nearest region, text) to locate
+and change the corresponding code. Confirm the element exists before editing.
+```
+
 ## Payload
 
 The extension POSTs to `/annotations`:
 
 ```json
 {
-  "extensionVersion": "0.2.0",
+  "extensionVersion": "0.6.0",
+  "sessionID": "ses_...",
   "annotations": [
     {
-      "mode": "act",
       "instruction": "Make this button larger and blue",
       "page": { "url": "https://example.com/app", "title": "My App" },
       "element": {
@@ -107,22 +141,28 @@ The extension POSTs to `/annotations`:
         "ariaLabel": "Sign up",
         "classes": ["cta", "primary"],
         "text": "Sign up",
+        "componentPath": "App > Header > SignupButton",
+        "framework": "react",
+        "landmark": "header[role=banner]",
+        "ancestors": ["div.actions", "header[role=banner]"],
         "bounds": { "x": 100, "y": 200, "width": 120, "height": 40 },
         "inShadow": false,
-        "html": "<button class=\"cta\">Sign up</button>"
+        "html": "<button class=\"cta\" id=\"signup\">"
       }
     }
   ]
 }
 ```
 
-`mode` is `"act"` (respond now, flushing any queued) or `"queue"` (hold for
-context until the next act). The plugin surfaces the most code-locatable
-identifiers first (testId, id, name, role) and treats the CSS path and bounds as
-weak hints.
+The plugin surfaces the most code-locatable identifiers first (component path,
+testId, id, name, role, ancestors, region) and treats the CSS path and bounds as
+weak hints. Build-generated/hashed class names are filtered out, text is
+whitespace-collapsed, and `html` is the element's opening tag with secret-looking
+attribute values redacted. `sessionID` is optional; without it the plugin targets
+the last-active session.
 
-`GET /status` returns `{ ok, activeSession, sessionID, sessionTitle, queued, host, port }`
-for a quick health check and to show the target session.
+`GET /status` returns `{ ok, activeSession, sessionID, sessionTitle, sessions, host, port }`
+for a quick health check and to list/target sessions.
 
 ## Develop
 

package/dist/plugin.js

@@ -74,7 +74,7 @@ function formatElement(el) {
         lines.push(`  viewport bounds: ${Math.round(b.width)}×${Math.round(b.height)} at (${Math.round(b.x)}, ${Math.round(b.y)})`);
     }
     if (el.html)
-        lines.push(`  outer html: ${truncate(el.html.trim(), 500)}`);
+        lines.push(`  opening tag: ${truncate(el.html.trim(), 500)}`);
     return lines.join("\n");
 }
 function annotationBlock(a, index, total) {
@@ -92,21 +92,15 @@ function annotationBlock(a, index, total) {
         .join("\n");
 }
 function buildPrompt(annotations) {
+    // The static interpretation rules live in the `browser-annotation` skill, not
+    // in every payload. The header marks this as a browser annotation and tells
+    // the agent to use that skill, so the rules are stated once (in the skill),
+    // never repeated per message.
     const header = annotations.length > 1
-        ? `The user made ${annotations.length} annotations in the browser. Address each one.`
-        : "The user made an annotation in the browser.";
+        ? `Browser annotations (${annotations.length}) — handle using the \`browser-annotation\` skill. Address each one.`
+        : "Browser annotation — handle using the `browser-annotation` skill.";
     const blocks = annotations.map((a, i) => annotationBlock(a, i, annotations.length));
-    return [
-        header,
-        "",
-        ...blocks,
-        "",
-        "Guidance:",
-        "- Locate the code for each element using the most stable identifier available: framework component path first if given, then data-testid, id, name, role, then the ancestor chain and nearest region to disambiguate, then unique class or text. Treat the CSS path and viewport bounds as weak hints only.",
-        "- Use the ancestors and nearest region to find the right component/file when the element itself is generic (e.g. a bare button that appears many times).",
-        "- Confirm the element actually exists in this codebase before editing; if you cannot find it, say so instead of guessing.",
-        "- No screenshot is attached; reason from the metadata and the code.",
-    ].join("\n");
+    return [header, "", ...blocks].join("\n");
 }
 async function readJsonBody(req, limitBytes = 2_000_000) {
     return await new Promise((resolve, reject) => {

package/extension/manifest.json

@@ -1,7 +1,7 @@
 {
   "manifest_version": 3,
   "name": "OpenCode Browser Annotation",
-  "version": "0.5.1",
+  "version": "0.6.1",
   "description": "Alt+A to annotate an element, Alt+Shift+A for the annotation list. Sends element metadata to your OpenCode agent.",
   "permissions": [
     "activeTab",

package/extension/overlay.js

@@ -338,6 +338,48 @@
     return undefined;
   }
 
+  // Collapse runs of whitespace to single spaces and trim, so a container's
+  // textContent doesn't arrive as a wall of newlines.
+  function cleanText(s) {
+    const t = (s || "").replace(/\s+/g, " ").trim();
+    return t ? t.slice(0, 300) : undefined;
+  }
+
+  // Names/attributes whose values may be secrets and must never be sent.
+  const SECRET_ATTR = /(token|csrf|authenticity|secret|api[-_]?key|password|session|auth|nonce|signature)/i;
+
+  // For the outer HTML, send only the element's own opening tag (not the whole
+  // subtree), with secret-looking attribute values redacted. This avoids leaking
+  // CSRF tokens / API keys and keeps the payload small for large containers.
+  function safeOpenTag(el) {
+    try {
+      const tag = el.tagName.toLowerCase();
+      const attrs = [];
+      const nameAttr = el.getAttribute("name") || "";
+      const typeAttr = el.getAttribute("type") || "";
+      // Redact the value= of any input whose name/type looks sensitive (CSRF
+      // tokens, passwords, hidden auth fields), regardless of input type.
+      const valueIsSecret = SECRET_ATTR.test(nameAttr) || /password|hidden/i.test(typeAttr);
+      for (const a of Array.from(el.attributes || [])) {
+        // Skip class/style: class is already captured (cleaned) in `classes`,
+        // and inline style is noise. This avoids duplication and the ugly
+        // mid-class truncation.
+        if (a.name === "class" || a.name === "style") continue;
+        let v = a.value;
+        if (SECRET_ATTR.test(a.name) || (a.name === "value" && valueIsSecret)) {
+          v = "[redacted]";
+        } else if (v && v.length > 120) {
+          v = v.slice(0, 120) + "…";
+        }
+        attrs.push(v === "" ? a.name : `${a.name}="${v}"`);
+      }
+      const open = `<${tag}${attrs.length ? " " + attrs.join(" ") : ""}>`;
+      return open;
+    } catch {
+      return undefined;
+    }
+  }
+
   function elementMeta(el, inShadow) {
     const r = el.getBoundingClientRect();
     const fw = frameworkComponents(el);
@@ -350,7 +392,7 @@
       role: el.getAttribute("role") || undefined,
       ariaLabel: el.getAttribute("aria-label") || undefined,
       classes: cleanClasses(el.classList ? Array.from(el.classList) : []),
-      text: (el.textContent || "").trim().slice(0, 500) || undefined,
+      text: cleanText(el.textContent),
       href: el.getAttribute("href") || undefined,
       src: el.getAttribute("src") || undefined,
       bounds: { x: r.left, y: r.top, width: r.width, height: r.height },
@@ -360,8 +402,11 @@
       landmark: nearestLandmark(el),
       componentPath: fw ? fw.path : undefined,
       framework: fw ? fw.framework : undefined,
-      html: (el.outerHTML || "").slice(0, 800),
+      html: safeOpenTag(el),
     };
+    // Drop the opening tag if it carries no attributes beyond the tag name
+    // (redundant with `tag` once class/style are excluded).
+    if (m.html && /^<[a-z0-9-]+>$/i.test(m.html)) delete m.html;
     for (const k of Object.keys(m)) if (m[k] === undefined) delete m[k];
     if (Array.isArray(m.ancestors) && m.ancestors.length === 0) delete m.ancestors;
     return m;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "opencode-browser-annotation-plugin",
-  "version": "0.5.1",
+  "version": "0.6.1",
   "description": "Select an element in your browser, type an instruction, and send it to your OpenCode agent over a loopback + SSH tunnel. Text and element metadata only (no screenshots).",
   "keywords": [
     "opencode",
@@ -28,6 +28,7 @@
   "files": [
     "dist",
     "extension",
+    "skills",
     "README.md"
   ],
   "scripts": {

package/skills/browser-annotation/SKILL.md

@@ -0,0 +1,60 @@
+---
+name: browser-annotation
+description: >-
+  Interpret and act on browser annotations sent by the
+  opencode-browser-annotation extension. Load this whenever a message says it is
+  a "Browser annotation" or references the `browser-annotation` skill, or when a
+  turn contains a "Selected element:" block with fields like data-testid,
+  ancestors, nearest region, or a component path. It explains how to turn that
+  element metadata into the correct code location and change.
+---
+
+# Browser annotations
+
+Messages from the `opencode-browser-annotation` extension describe a DOM element
+the user selected in their browser plus an instruction. Your job is to map that
+element to the right code and carry out the instruction. The payload is text and
+element metadata only — no screenshot is attached.
+
+## How to locate the code
+
+Use the most stable identifier available, in this order:
+
+1. **Framework component path** (e.g. `App > Dashboard > SignupCard`) — the
+   strongest hint. Find that component's source file first.
+2. **`data-testid` / `id` / `name` / `role`** on the element itself — usually
+   maps directly to source.
+3. **Ancestors (nearest first) + nearest region** — use these to disambiguate a
+   generic element (e.g. a bare `<button>` that appears many times). They tell
+   you which container/feature/file the element belongs to.
+4. **Unique class or text** — human-authored class names and the element's text
+   are good locators. Build-generated/hashed class names are already filtered
+   out; treat any remaining opaque tokens as weak.
+5. **CSS path and viewport bounds** — weak hints only; do not rely on them.
+
+## Rules
+
+- Confirm the element actually exists in this codebase before editing. If you
+  cannot find it, say so instead of guessing or inventing a file.
+- Reason from the metadata and the actual code; there is no image.
+- The `opening tag` field is the element's own opening tag with secret-looking
+  attribute values redacted — do not expect a full HTML subtree, and never try
+  to recover redacted values.
+- If several annotations arrive together, address each one.
+- Treat the page URL as context for which route/screen the user was on.
+
+## Field reference
+
+- `Element` — tag name.
+- `data-testid`, `id`, `name`, `role`, `aria-label` — element identifiers.
+- `classes` — authored class names (hashed/build-generated ones removed).
+- `text` — the element's visible text (whitespace-collapsed, truncated).
+- `href` / `src` — link or media source when present.
+- `<framework> components` — component hierarchy from React/Preact/Vue when
+  detectable.
+- `nearest region` — closest semantic landmark/region ancestor.
+- `ancestors (nearest first)` — parent chain signatures, shadow DOM pierced.
+- `css path` — a positional selector; weak.
+- `context` — notes such as "inside a shadow DOM" or "inside an iframe".
+- `viewport bounds` — on-screen size/position; weak.
+- `opening tag` — redacted opening tag of the element.