@pxlarified/browser 0.1.0

package/README.md

@@ -0,0 +1,182 @@
+# OpenBrowser
+
+OpenBrowser is a minimal browser-control skill and CLI. It controls one browser tab that it creates and owns, using a browser extension plus a local, user-scoped native bridge.
+
+The npm package is intended to be published as `@pxlarified/browser` and used with:
+
+```bash
+npx @pxlarified/browser <command>
+```
+
+The OpenBrowser skill lives in `skills/OpenBrowser/` and is not included as an npm-published skill package.
+
+## Status
+
+- Initial browser support: Zen, a Firefox-based browser.
+- Extension artifact for Firefox-based browsers: `.xpi`.
+- Chromium support is intentionally adapter-ready but not implemented yet.
+- The default build creates an unsigned development `.xpi`.
+- Firefox release signing is implemented as a separate `web-ext sign --channel unlisted` step.
+
+## Security model
+
+The CLI does not expose browser control through a public network service. The extension talks to a native messaging host, and the CLI talks to that host through a user-scoped local bridge:
+
+- Unix/macOS: `~/OpenBrowser/bridge/<browser>.sock` with user-only permissions.
+- Windows: a user-local named pipe.
+
+## Session model
+
+Each supported browser may have at most one active OpenBrowser session.
+
+- A session is exactly one browser tab created by OpenBrowser.
+- `open` fails if a session already exists for that browser.
+- `close` closes only the OpenBrowser-owned tab.
+- OpenBrowser never closes, reuses, inspects, navigates, or modifies tabs opened manually by the user.
+- If the user manually closes the owned tab, the session is automatically considered closed.
+
+## Install and build
+
+```bash
+npm install
+npm run build
+```
+
+This builds:
+
+```text
+dist/extensions/firefox/openbrowser-dev.xpi
+```
+
+The Firefox development artifact is unsigned. It is useful for local development and for browser builds/profiles that allow unsigned extensions.
+
+## Install the extension and native bridge
+
+```bash
+npx @pxlarified/browser install zen
+```
+
+The installer:
+
+1. Copies the bundled Firefox `.xpi` into detected Zen profiles as `openbrowser@mizius.com.xpi`.
+2. Installs the user-scoped native messaging manifest.
+3. Installs the native host launcher under `~/OpenBrowser/native-host/`.
+
+If an OpenBrowser extension is already staged in a Zen profile, it is overwritten with the bundled version. Restart Zen if the update does not load immediately.
+
+## CLI commands
+
+`--browser zen` is optional while Zen is the only supported browser.
+
+```bash
+# Installation and session lifecycle
+npx @pxlarified/browser install zen
+npx @pxlarified/browser open <url> --browser zen
+npx @pxlarified/browser close --browser zen
+npx @pxlarified/browser status --browser zen
+
+# Navigation
+npx @pxlarified/browser navigate <url> --browser zen
+npx @pxlarified/browser reload --browser zen
+npx @pxlarified/browser back --browser zen
+npx @pxlarified/browser forward --browser zen
+
+# Page state
+npx @pxlarified/browser state --browser zen
+
+# Screenshots
+npx @pxlarified/browser screenshot --browser zen
+npx @pxlarified/browser screenshot --base64 --browser zen
+
+# Interaction
+npx @pxlarified/browser click <ref> --browser zen
+npx @pxlarified/browser keys <text> --browser zen
+npx @pxlarified/browser press <key> --browser zen
+npx @pxlarified/browser select <ref> <option> --browser zen
+
+# Content inspection
+npx @pxlarified/browser get --html --browser zen
+npx @pxlarified/browser get --html --ref <ref> --browser zen
+
+# Scrolling
+npx @pxlarified/browser scroll up [pixels] --browser zen
+npx @pxlarified/browser scroll down [pixels] --browser zen
+npx @pxlarified/browser scroll --to <ref> --browser zen
+```
+
+Most commands print JSON to stdout. Screenshot commands are special:
+
+- `screenshot` saves a PNG to `~/OpenBrowser/screenshots/<8-char-uuid>.png` and prints only the absolute file path to stdout.
+- `screenshot --base64` prints only the Base64 PNG data to stdout.
+- Diagnostics are written to stderr.
+
+## Page state and references
+
+`state` returns the current URL, title, viewport, and actionable elements. Element references are generated by OpenBrowser and do not depend on raw webpage IDs.
+
+```json
+{"url":"https://example.com","title":"Example Domain","elements":[{"ref":"e_1","role":"link","name":"More information"}]}
+```
+
+References become invalid after navigation or relevant DOM updates. Commands that use invalid references return a clear `STALE_REFERENCE` error. Run `state` again to get fresh references.
+
+## Firefox signing and release pipeline
+
+The development build is unsigned:
+
+```bash
+npm run build:firefox
+```
+
+When Mozilla Add-ons credentials are available, save them in a local uncommitted `.env` file:
+
+```env
+AMO_JWT_ISSUER="..."
+AMO_JWT_SECRET="..."
+```
+
+Then create the signed unlisted release artifact with:
+
+```bash
+npm run release:firefox
+```
+
+You can also override the `.env` values with shell environment variables.
+
+The release script runs the equivalent of:
+
+```bash
+npx web-ext sign \
+  --source-dir ./extensions/ \
+  --channel unlisted \
+  --api-key "$AMO_JWT_ISSUER" \
+  --api-secret "$AMO_JWT_SECRET"
+```
+
+It then bundles the resulting signed XPI at:
+
+```text
+dist/extensions/firefox/openbrowser.xpi
+```
+
+That signed `.xpi` is the artifact that should be included in the npm package for Firefox-based browser installs.
+
+### Signing information you need to provide
+
+Before publishing a signed Firefox release, provide:
+
+1. `AMO_JWT_ISSUER` - the Mozilla Add-ons API key / JWT issuer.
+2. `AMO_JWT_SECRET` - the Mozilla Add-ons API secret / JWT secret.
+3. Confirmation of the final extension ID: `openbrowser@mizius.com`.
+4. Confirmation of the package/version to submit to AMO for unlisted signing.
+
+## Project layout
+
+```text
+bin/OpenBrowser.js              CLI entry point
+src/                            CLI, browser adapters, installer, native host
+extensions/                      Browser extension source
+scripts/build.js                 Development XPI build
+scripts/sign-firefox.js          Unlisted Firefox signing pipeline
+skills/OpenBrowser/SKILL.md      Local agent skill, not npm-published
+```

package/bin/OpenBrowser.js

@@ -0,0 +1,8 @@
+#!/usr/bin/env node
+import { runCli } from "../src/cli.js";
+
+runCli(process.argv.slice(2)).catch((error) => {
+  const message = error && error.message ? error.message : String(error);
+  console.error(message);
+  process.exitCode = 1;
+});

package/extensions/background.js

@@ -0,0 +1,194 @@
+const NATIVE_HOST = "openbrowser";
+const SESSION_KEY = "session";
+let port;
+let reconnectTimer;
+
+connectNativeHost();
+browser.tabs.onRemoved.addListener(handleTabRemoved);
+
+function connectNativeHost() {
+  try {
+    port = browser.runtime.connectNative(NATIVE_HOST);
+    port.onMessage.addListener(handleNativeMessage);
+    port.onDisconnect.addListener(() => {
+      port = null;
+      scheduleReconnect();
+    });
+  } catch (error) {
+    scheduleReconnect();
+  }
+}
+
+function scheduleReconnect() {
+  if (reconnectTimer) return;
+  reconnectTimer = setTimeout(() => {
+    reconnectTimer = null;
+    connectNativeHost();
+  }, 1000);
+}
+
+async function handleNativeMessage(message) {
+  const id = message && message.id;
+  if (!id) return;
+
+  try {
+    const result = await dispatchCommand(message.command, message.args || {});
+    postNative({ replyTo: id, ok: true, result });
+  } catch (error) {
+    postNative({
+      replyTo: id,
+      ok: false,
+      error: {
+        code: error.code || "COMMAND_FAILED",
+        message: error.message || String(error),
+      },
+    });
+  }
+}
+
+function postNative(message) {
+  if (!port) return;
+  port.postMessage(message);
+}
+
+async function dispatchCommand(command, args) {
+  switch (command) {
+    case "open": return openSession(args.url);
+    case "close": return closeSession();
+    case "status": return status();
+    case "navigate": return navigate(args.url);
+    case "reload": return tabAction((tabId) => browser.tabs.reload(tabId));
+    case "back": return contentCommand("historyBack", {});
+    case "forward": return contentCommand("historyForward", {});
+    case "state": return contentCommand("state", {});
+    case "screenshot": return screenshot();
+    case "click": return contentCommand("click", { ref: args.ref });
+    case "keys": return contentCommand("keys", { text: args.text });
+    case "press": return contentCommand("press", { key: args.key });
+    case "select": return contentCommand("select", { ref: args.ref, option: args.option });
+    case "getHtml": return contentCommand("getHtml", { ref: args.ref });
+    case "scroll": return contentCommand("scroll", args);
+    default: throw codedError("UNKNOWN_COMMAND", `Unknown OpenBrowser command: ${command}`);
+  }
+}
+
+async function openSession(url) {
+  const existing = await getLiveSession();
+  if (existing) throw codedError("SESSION_EXISTS", "An OpenBrowser session already exists for this browser. Close it first.");
+
+  const tab = await browser.tabs.create({ url });
+  const session = { tabId: tab.id, createdAt: new Date().toISOString() };
+  await browser.storage.local.set({ [SESSION_KEY]: session });
+  return { ok: true, session, url: tab.url || url };
+}
+
+async function closeSession() {
+  const session = await getStoredSession();
+  if (!session) return { ok: true, open: false };
+
+  try {
+    await browser.tabs.remove(session.tabId);
+  } catch {
+    // The owned tab may already have been manually closed.
+  }
+  await clearSession();
+  return { ok: true, open: false };
+}
+
+async function status() {
+  const session = await getLiveSession();
+  if (!session) return { open: false };
+  const tab = await browser.tabs.get(session.tabId);
+  return {
+    open: true,
+    session,
+    tab: {
+      id: tab.id,
+      url: tab.url,
+      title: tab.title,
+      status: tab.status,
+    },
+  };
+}
+
+async function navigate(url) {
+  const session = await requireLiveSession();
+  await browser.tabs.update(session.tabId, { url });
+  return { ok: true };
+}
+
+async function tabAction(action) {
+  const session = await requireLiveSession();
+  await action(session.tabId);
+  return { ok: true };
+}
+
+async function screenshot() {
+  const session = await requireLiveSession();
+  await browser.tabs.update(session.tabId, { active: true });
+  const dataUrl = await browser.tabs.captureTab(session.tabId, { format: "png" });
+  return { dataUrl };
+}
+
+async function contentCommand(type, payload) {
+  const session = await requireLiveSession();
+  await ensureContentScript(session.tabId);
+  try {
+    return await browser.tabs.sendMessage(session.tabId, { type, ...payload });
+  } catch (error) {
+    throw codedError("CONTENT_UNAVAILABLE", error.message || "OpenBrowser content script is unavailable on this page.");
+  }
+}
+
+async function ensureContentScript(tabId) {
+  try {
+    await browser.tabs.sendMessage(tabId, { type: "ping" });
+    return;
+  } catch {
+    // Inject below.
+  }
+
+  try {
+    await browser.tabs.executeScript(tabId, { file: "content.js", runAt: "document_idle" });
+  } catch (error) {
+    throw codedError("INJECTION_FAILED", `Cannot control this page: ${error.message || error}`);
+  }
+}
+
+async function getStoredSession() {
+  const stored = await browser.storage.local.get(SESSION_KEY);
+  return stored[SESSION_KEY] || null;
+}
+
+async function getLiveSession() {
+  const session = await getStoredSession();
+  if (!session) return null;
+  try {
+    await browser.tabs.get(session.tabId);
+    return session;
+  } catch {
+    await clearSession();
+    return null;
+  }
+}
+
+async function requireLiveSession() {
+  const session = await getLiveSession();
+  if (!session) throw codedError("NO_SESSION", "No active OpenBrowser session exists for this browser.");
+  return session;
+}
+
+async function clearSession() {
+  await browser.storage.local.remove(SESSION_KEY);
+}
+
+async function handleTabRemoved(tabId) {
+  const session = await getStoredSession();
+  if (session && session.tabId === tabId) await clearSession();
+}
+
+function codedError(code, message) {
+  const error = new Error(message);
+  error.code = code;
+  return error;
+}

package/extensions/content.js

@@ -0,0 +1,275 @@
+(function installOpenBrowserContentScript() {
+  if (window.__openbrowserContentInstalled) return;
+  window.__openbrowserContentInstalled = true;
+
+  var refs = new Map();
+  var generation = 0;
+
+  var observer = new MutationObserver(function () {
+    invalidateReferences();
+  });
+
+  observer.observe(document.documentElement || document, {
+    childList: true,
+    subtree: true,
+    attributes: true,
+    characterData: true,
+    attributeFilter: ["href", "disabled", "aria-label", "aria-hidden", "role", "tabindex", "value"],
+  });
+
+  window.addEventListener("pageshow", invalidateReferences, true);
+  window.addEventListener("hashchange", invalidateReferences, true);
+
+  browser.runtime.onMessage.addListener(function (message) {
+    if (!message || !message.type) return undefined;
+
+    try {
+      switch (message.type) {
+        case "ping": return Promise.resolve({ ok: true });
+        case "state": return Promise.resolve(getState());
+        case "click": return Promise.resolve(clickRef(message.ref));
+        case "keys": return Promise.resolve(typeKeys(message.text || ""));
+        case "press": return Promise.resolve(pressKey(message.key));
+        case "select": return Promise.resolve(selectOption(message.ref, message.option));
+        case "getHtml": return Promise.resolve(getHtml(message.ref));
+        case "scroll": return Promise.resolve(scrollPage(message));
+        case "historyBack": history.back(); return Promise.resolve({ ok: true });
+        case "historyForward": history.forward(); return Promise.resolve({ ok: true });
+        default: return Promise.reject(codedError("UNKNOWN_CONTENT_COMMAND", "Unknown content command."));
+      }
+    } catch (error) {
+      return Promise.reject(error);
+    }
+  });
+
+  function invalidateReferences() {
+    generation += 1;
+    refs.clear();
+  }
+
+  function getState() {
+    refs.clear();
+    var elements = collectActionableElements();
+    var stateGeneration = generation;
+    var result = [];
+
+    elements.forEach(function (element, index) {
+      var ref = "e_" + (index + 1);
+      refs.set(ref, { element: element, generation: stateGeneration });
+      result.push({
+        ref: ref,
+        role: roleOf(element),
+        name: accessibleName(element),
+      });
+    });
+
+    return {
+      url: location.href,
+      title: document.title,
+      viewport: {
+        width: window.innerWidth,
+        height: window.innerHeight,
+        scrollX: window.scrollX,
+        scrollY: window.scrollY,
+        devicePixelRatio: window.devicePixelRatio,
+      },
+      elements: result,
+    };
+  }
+
+  function collectActionableElements() {
+    var selector = [
+      "a[href]",
+      "button",
+      "input:not([type=hidden])",
+      "select",
+      "textarea",
+      "summary",
+      "[role=button]",
+      "[role=link]",
+      "[role=menuitem]",
+      "[role=checkbox]",
+      "[role=radio]",
+      "[tabindex]",
+      "[contenteditable=true]",
+    ].join(",");
+
+    return Array.prototype.slice.call(document.querySelectorAll(selector))
+      .filter(function (element) {
+        return isVisible(element) && !isDisabled(element) && accessibleName(element);
+      })
+      .slice(0, 200);
+  }
+
+  function getRef(ref) {
+    var entry = refs.get(ref);
+    if (!entry || entry.generation !== generation || !entry.element.isConnected) {
+      throw codedError("STALE_REFERENCE", "Stale OpenBrowser reference. Run state again and use a fresh ref.");
+    }
+    return entry.element;
+  }
+
+  function clickRef(ref) {
+    var element = getRef(ref);
+    element.scrollIntoView({ block: "center", inline: "center" });
+    element.focus({ preventScroll: true });
+    dispatchMouse(element, "mouseover");
+    dispatchMouse(element, "mousedown");
+    dispatchMouse(element, "mouseup");
+    if (typeof element.click === "function") element.click();
+    else dispatchMouse(element, "click");
+    return { ok: true };
+  }
+
+  function typeKeys(text) {
+    var element = document.activeElement;
+    if (!element || element === document.body) throw codedError("NO_FOCUS", "No focused element is available for text input.");
+
+    if (isTextInput(element)) {
+      insertText(element, text);
+      return { ok: true };
+    }
+
+    if (element.isContentEditable) {
+      document.execCommand("insertText", false, text);
+      element.dispatchEvent(new InputEvent("input", { bubbles: true, inputType: "insertText", data: text }));
+      return { ok: true };
+    }
+
+    throw codedError("NOT_EDITABLE", "The focused element does not accept text input.");
+  }
+
+  function pressKey(key) {
+    var element = document.activeElement || document.body;
+    var options = { key: key, bubbles: true, cancelable: true };
+    element.dispatchEvent(new KeyboardEvent("keydown", options));
+    element.dispatchEvent(new KeyboardEvent("keypress", options));
+
+    if (key === "Enter" && typeof element.click === "function" && /^(BUTTON|A)$/.test(element.tagName)) element.click();
+    if (key === "Tab") focusNextElement();
+
+    element.dispatchEvent(new KeyboardEvent("keyup", options));
+    return { ok: true };
+  }
+
+  function selectOption(ref, option) {
+    var element = getRef(ref);
+    if (element.tagName !== "SELECT") throw codedError("NOT_SELECT", "Referenced element is not a select control.");
+
+    var options = Array.prototype.slice.call(element.options);
+    var match = options.find(function (candidate, index) {
+      return candidate.value === option || candidate.text.trim() === option || String(index) === option;
+    });
+    if (!match) throw codedError("OPTION_NOT_FOUND", "Select option was not found.");
+
+    element.value = match.value;
+    element.dispatchEvent(new Event("input", { bubbles: true }));
+    element.dispatchEvent(new Event("change", { bubbles: true }));
+    return { ok: true };
+  }
+
+  function getHtml(ref) {
+    if (!ref) return { html: document.documentElement.outerHTML };
+    return { html: getRef(ref).outerHTML };
+  }
+
+  function scrollPage(message) {
+    if (message.to) {
+      getRef(message.to).scrollIntoView({ block: "center", inline: "nearest" });
+      return { ok: true, scrollX: window.scrollX, scrollY: window.scrollY };
+    }
+
+    var pixels = Number(message.pixels || 600);
+    if (!Number.isFinite(pixels) || pixels <= 0) pixels = 600;
+    window.scrollBy({ top: message.direction === "up" ? -pixels : pixels, left: 0, behavior: "auto" });
+    return { ok: true, scrollX: window.scrollX, scrollY: window.scrollY };
+  }
+
+  function accessibleName(element) {
+    return (
+      element.getAttribute("aria-label") ||
+      element.getAttribute("title") ||
+      element.getAttribute("alt") ||
+      associatedLabel(element) ||
+      element.value ||
+      element.textContent ||
+      element.getAttribute("placeholder") ||
+      ""
+    ).replace(/\s+/g, " ").trim().slice(0, 160);
+  }
+
+  function associatedLabel(element) {
+    if (element.id) {
+      var label = document.querySelector('label[for="' + cssEscape(element.id) + '"]');
+      if (label) return label.textContent;
+    }
+    var parent = element.closest("label");
+    return parent ? parent.textContent : "";
+  }
+
+  function roleOf(element) {
+    var explicit = element.getAttribute("role");
+    if (explicit) return explicit;
+    var tag = element.tagName.toLowerCase();
+    if (tag === "a") return "link";
+    if (tag === "button") return "button";
+    if (tag === "select") return "select";
+    if (tag === "textarea") return "textbox";
+    if (tag === "summary") return "button";
+    if (tag === "input") {
+      var type = (element.getAttribute("type") || "text").toLowerCase();
+      if (["button", "submit", "reset"].includes(type)) return "button";
+      if (["checkbox", "radio"].includes(type)) return type;
+      return "textbox";
+    }
+    return "control";
+  }
+
+  function isVisible(element) {
+    var style = getComputedStyle(element);
+    if (style.visibility === "hidden" || style.display === "none" || Number(style.opacity) === 0) return false;
+    var rect = element.getBoundingClientRect();
+    return rect.width > 0 && rect.height > 0;
+  }
+
+  function isDisabled(element) {
+    return Boolean(element.disabled || element.getAttribute("aria-disabled") === "true");
+  }
+
+  function dispatchMouse(element, type) {
+    element.dispatchEvent(new MouseEvent(type, { bubbles: true, cancelable: true, view: window }));
+  }
+
+  function isTextInput(element) {
+    return element.tagName === "TEXTAREA" || (element.tagName === "INPUT" && !["button", "checkbox", "file", "radio", "reset", "submit"].includes((element.type || "text").toLowerCase()));
+  }
+
+  function insertText(element, text) {
+    var start = element.selectionStart || 0;
+    var end = element.selectionEnd || start;
+    var value = element.value || "";
+    element.value = value.slice(0, start) + text + value.slice(end);
+    var cursor = start + text.length;
+    element.setSelectionRange(cursor, cursor);
+    element.dispatchEvent(new InputEvent("input", { bubbles: true, inputType: "insertText", data: text }));
+    element.dispatchEvent(new Event("change", { bubbles: true }));
+  }
+
+  function focusNextElement() {
+    var focusable = collectActionableElements();
+    var index = focusable.indexOf(document.activeElement);
+    var next = focusable[(index + 1) % focusable.length];
+    if (next) next.focus();
+  }
+
+  function cssEscape(value) {
+    if (window.CSS && CSS.escape) return CSS.escape(value);
+    return String(value).replace(/"/g, '\\"');
+  }
+
+  function codedError(code, message) {
+    var error = new Error(message);
+    error.code = code;
+    return error;
+  }
+})();

package/extensions/manifest.json

@@ -0,0 +1,26 @@
+{
+  "manifest_version": 2,
+  "name": "OpenBrowser",
+  "version": "0.1.0",
+  "description": "Local user-scoped browser control bridge for the OpenBrowser CLI.",
+  "permissions": [
+    "nativeMessaging",
+    "storage",
+    "tabs",
+    "activeTab",
+    "<all_urls>"
+  ],
+  "background": {
+    "scripts": ["background.js"],
+    "persistent": true
+  },
+  "browser_specific_settings": {
+    "gecko": {
+      "id": "openbrowser@mizius.com",
+      "strict_min_version": "109.0",
+      "data_collection_permissions": {
+        "required": ["none"]
+      }
+    }
+  }
+}