@aion0/forge 0.6.3 → 0.8.1

package/docs/RFC-Browser-Connectors.md

@@ -0,0 +1,509 @@
+# RFC: Browser Connectors
+
+> **Status**: Draft — design only, no implementation.
+> **Owner**: aion0
+> **Date**: 2026-05-16
+> **Related**:
+> - `PRD.md` (Forge product PRD)
+> - `roadmap-multi-agent-workflow.md`
+> - `../temper` (memory service used by enterprise agents)
+
+## Problem
+
+Enterprise agents need access to dozens of internal web systems (JIRA,
+Mantis, Confluence, Teams, internal CRM, monitoring dashboards). Two
+existing approaches both fail:
+
+1. **OAuth / API tokens per system** — IT申请慢、credentials 难管、
+   refresh tokens 复杂、有些遗留系统压根没 API。
+2. **Anthropic Computer Use 视觉路径** — 截图 + vision LLM 推坐标，慢、
+   不稳、token 消耗大、出错难调试。
+
+We propose a **third path**: a Chrome browser extension that runs as the
+user's already-authenticated session, executes semantic actions via
+per-site **connectors** that extract data from the rendered DOM (using
+the extension's Chrome MCP capabilities), and exposes them as MCP tools
+to Forge / any MCP client.
+
+**Connectors do not call REST APIs.** They navigate tabs and parse the
+rendered HTML, inheriting the user's UI session implicitly through
+cookies. Token-based API access is an opt-in escape hatch, not the
+default — see "Execution Model" below for the rationale.
+
+## Goals
+
+- **Zero credential setup** — re-use the user's existing browser sessions.
+- **Reliable per-site integration** — structured tool calls (not vision).
+- **Forge-orchestratable** — agents in Forge pipelines/crafts can call
+  browser connectors like any other MCP tool.
+- **Composable** — connectors are independent packages, easy to publish
+  and version.
+
+## Non-Goals
+
+- Building a general-purpose web automation framework (not Selenium).
+- Replacing Computer Use entirely — vision-based fallback is a separate
+  Tier 3 below.
+- Synchronous chat UI in the browser — that's a separate component
+  layered on top (see "Out of Scope" below).
+
+## High-level Architecture
+
+```
+┌──────────────────────────────────────────────┐
+│ Browser (Chrome / Edge with Chromium)        │
+│ ┌────────────────────────────────────────┐   │
+│ │ Forge Browser Extension (MV3)          │   │
+│ │  • background service worker           │   │
+│ │  • content scripts (per-domain)        │   │
+│ │  • connectors (built-in + downloaded)  │   │
+│ └─────────────────┬──────────────────────┘   │
+└───────────────────┼──────────────────────────┘
+                    │ WebSocket (loopback, paired)
+                    ▼
+┌──────────────────────────────────────────────┐
+│ Forge installation                           │
+│ ┌────────────────────────────────────────┐   │
+│ │ browser-bridge-standalone (NEW)        │   │
+│ │ port 8407                              │   │
+│ │  • WebSocket server → extension        │   │
+│ │  • MCP server (exposes connector tools)│   │
+│ │  • pairing token store                 │   │
+│ │  • health monitor                      │   │
+│ └────────────────────────────────────────┘   │
+└──────────────────────────────────────────────┘
+                    ▲
+                    │ MCP / SSE
+                    │
+      Forge agents / pipelines / crafts
+      (and Claude Desktop, Cursor, etc.)
+```
+
+### Why a bridge process
+
+Chrome MV3 service workers can't keep long-lived TCP servers or expose
+SSE endpoints. The bridge is a small Node process inside Forge that:
+
+1. Holds the persistent MCP server (port 8407)
+2. Maintains the WebSocket connection to the extension
+3. Routes MCP tool calls into extension actions
+4. Tracks connector health + paired devices
+
+The bridge is one of Forge's standalone services (alongside
+terminal/workspace/telegram). Stops when Forge stops; auto-detaches
+in background mode (same SIGHUP fix already in `forge-server.mjs`).
+
+## Connector Spec
+
+A **connector** is a JavaScript module that:
+- Declares which URLs it can handle (`matches`)
+- Exposes a set of MCP tools (`tools`)
+- Each tool is an async function that runs in the extension's content
+  script context
+
+### Manifest
+
+```ts
+import { defineBrowserConnector } from '@forge/browser-connector-sdk';
+
+export default defineBrowserConnector({
+  id: 'mantis',
+  name: 'MantisBT',
+  version: '0.1.0',
+  matches: ['https://mantis.mycompany.com/*'],
+  description: 'Search + create + update Mantis bug tickets',
+
+  // Hint to the orchestrator whether opening a tab is required when
+  // none of the user's tabs match `matches`.
+  requiresActiveTab: true,
+
+  // Optional: health probe — bridge calls this periodically; failure
+  // raises an alert. Should return quickly + use minimal quota.
+  healthCheck: async (ctx) => {
+    const r = await ctx.fetch('/api/rest/users/me');
+    return r.ok;
+  },
+
+  tools: {
+    'mantis.list_my_bugs': {
+      description: 'List bugs assigned to the current user',
+      parameters: {
+        status: { type: 'string', enum: ['open', 'closed', 'all'], default: 'open' },
+        limit: { type: 'number', default: 50 },
+      },
+      handler: async (args, ctx) => {
+        // ctx.fetch attaches cookies automatically (credentials: include)
+        const r = await ctx.fetch(
+          `/api/rest/issues?handler_id=me&status_id=${args.status === 'open' ? '!=resolved' : ''}&page_size=${args.limit}`,
+        );
+        const data = await r.json();
+        return { bugs: data.issues, total: data.total };
+      },
+    },
+    'mantis.add_comment': {
+      description: 'Add a comment to a bug',
+      parameters: {
+        bug_id: { type: 'integer', required: true },
+        text: { type: 'string', required: true },
+      },
+      // destructive: true → bridge requires user confirmation before exec
+      destructive: true,
+      handler: async (args, ctx) => {
+        const r = await ctx.fetch(`/api/rest/issues/${args.bug_id}/notes`, {
+          method: 'POST',
+          headers: { 'Content-Type': 'application/json' },
+          body: JSON.stringify({ text: args.text }),
+        });
+        return { ok: r.ok };
+      },
+    },
+  },
+});
+```
+
+### Packaging in Forge
+
+The `defineBrowserConnector(...)` module above is the **runtime spec** for a
+single connector. It's shipped inside a **Forge plugin** — the same plugin
+system used for `gmail`, `jenkins`, etc. Two rules:
+
+**1. Default 1:1 — one connector per plugin.**
+
+Marketplace UX is cleanest when "install Mantis" = one entry. Each plugin
+owns its `host_permissions`, version, and auth lifecycle independently.
+A connector outage doesn't drag down unrelated plugins.
+
+**2. Escape hatch: 1:N for same-vendor suites with shared auth.**
+
+Atlassian (Jira + Confluence + Bitbucket), Google Workspace (Gmail + Drive +
+Calendar), and Microsoft 365 (Teams + Outlook + SharePoint) share OAuth /
+SSO across products. Forcing them into separate plugins triples the auth
+dance. The plugin manifest therefore allows multiple `connectors:` entries:
+
+```yaml
+# plugin.yaml
+name: atlassian-suite
+category: connector
+mode: hybrid
+auth: oauth2-atlassian       # shared across all connectors below
+connectors:
+  - id: jira
+    host_permissions: ["*://*.atlassian.net/*"]
+    tools: [...]
+  - id: confluence
+    host_permissions: ["*://*.atlassian.net/wiki/*"]
+    tools: [...]
+```
+
+This is the **minority case**. All 5 MVP defaults (gmail / teams / jira /
+github / gitlab) are 1 plugin = 1 connector.
+
+**3. `category: connector` marks the plugin as a connector in marketplace.**
+
+Forge plugins serve different purposes (pipeline nodes, MCP servers, tools,
+connectors). The plugin manifest gets a top-level `category` field so the
+marketplace and extension can filter for connectors specifically:
+
+```yaml
+category: connector   # | pipeline-node | mcp-server | tool | other
+```
+
+Only `category: connector` plugins appear in the extension's "Connectors"
+panel. See the plugin-mode ADR in `Implementation-Plan-Browser-Agent.md`
+for the full plugin schema changes.
+
+**Three-layer concept (don't conflate):**
+
+| Layer | Unit | Example |
+|---|---|---|
+| Plugin | Install unit (marketplace entry) | `mantis`, `atlassian-suite` |
+| Connector | Per-system adapter | `mantis`, `jira`, `confluence` |
+| Capability | A method on a connector | `mantis.list_my_bugs`, `jira.get_issue` |
+
+GitLab exposes issues, MRs, wiki, and CI — that's **one connector with many
+capabilities**, not multiple connectors.
+
+### `ctx` API (passed to handlers)
+
+DOM-centric. Handlers navigate, wait, query, and parse. A `fetch` is
+still provided for the rare cases where an XHR endpoint of the rendered
+app is the cleanest path (still inside the tab's origin, still using
+cookies — but **not** for documented REST APIs that demand tokens).
+
+```ts
+interface ConnectorContext {
+  /** Navigate the active tab to a path within the connector's origin. */
+  navigate(path: string, opts?: { waitFor?: string; timeout?: number }): Promise<void>;
+
+  /** Query the DOM. Returns null if not found. */
+  $(selector: string): Element | null;
+  $$(selector: string): Element[];
+
+  /** Wait for selector with timeout (most pages aren't ready synchronously). */
+  waitFor(selector: string, opts?: { timeout?: number }): Promise<Element>;
+
+  /**
+   * Same-origin fetch with cookies attached. Use sparingly — only for XHR
+   * endpoints the rendered page itself calls (i.e. cookie-authenticated).
+   * Do NOT use for documented REST APIs that require Authorization headers;
+   * those need a Tier 3 escape hatch (user-provided token).
+   */
+  fetch(input: string | URL, init?: RequestInit): Promise<Response>;
+
+  /** Take a screenshot of the current viewport (only for Tier 2 vision fallback). */
+  screenshot(): Promise<string>;  // base64 PNG
+
+  /** Structured logging visible in the bridge dashboard. */
+  log(...args: any[]): void;
+
+  /** Throw this to signal "auth needed" — bridge surfaces a toast + pauses pipeline. */
+  AuthExpiredError: typeof Error;
+}
+```
+
+## Execution Model: DOM Extraction First (Decision 2026-05-16)
+
+**Browser-side connectors extract data from the rendered DOM of the user's
+active tab, via the extension's Chrome MCP / scripting capabilities. They
+do NOT call site REST APIs.**
+
+This is a deliberate design choice, not a fallback.
+
+### Why DOM, not API?
+
+The whole point of routing through a browser extension is to **reuse the
+user's existing logged-in browser session for free**. Many enterprise
+systems make this trade-off-laden:
+
+| System | REST API auth | DOM auth |
+|---|---|---|
+| **MantisBT** | Requires API token (`Authorization` header). Cookies don't work on `/api/rest/*`. | PHPSESSID cookie authenticates rendered pages. ✅ |
+| **GitLab** | Requires PAT (cookies don't work on `/api/v4/*` for browsers due to CSRF). | Session cookie authenticates UI pages. ✅ |
+| **JIRA Server** | Requires basic auth or PAT. | Session cookie OK on rendered pages. ✅ |
+| **Teams / Outlook** | Requires MSAL bearer token. | Session OK on rendered pages. ✅ |
+
+If we used REST APIs, every connector would need the user to mint a token,
+copy it into a settings page, rotate it on expiry, and deal with scope-
+limited tokens that don't grant the same permissions as their UI session.
+That defeats the value proposition of the extension entirely — the user
+might as well configure server-side plugins with PATs.
+
+**DOM extraction inherits the user's UI session, no token management,
+ever.** The cost is paid in robustness against site redesigns — accepted.
+
+### Execution path
+
+```
+LLM in extension → tool call (e.g. mantis.list_my_bugs)
+                  │
+                  ▼
+extension's connector runtime:
+  1. find tab matching connector's host_permissions (or open one)
+  2. navigate to the appropriate URL inside that tab
+  3. use Chrome MCP / scripting / page DOM to extract structured data
+  4. return the data to the LLM
+```
+
+No `fetch('/api/rest/...')`. No `Authorization` headers. Cookies are
+implicit — the tab is already authenticated because the user logged in
+through normal browsing.
+
+### Fallback tiers (in declining preference)
+
+| Tier | Mechanism | When |
+|---|---|---|
+| **1 (primary)** | DOM extraction via Chrome MCP — selectors, structure parsing | Always try first |
+| **2 (fallback)** | Screenshot → vision LLM | Site redesigned / selector broken |
+| **3 (escape)** | Optional REST API + user-provided token | Only if user explicitly enables it in connector settings |
+
+Tier 3 is **off by default**. We don't ship token-based fallbacks because
+the friction of token management is exactly what the extension model
+exists to avoid. A connector author MAY surface an optional `api_token`
+setting if their site simply cannot be scraped reliably; users who want
+to use it accept the trade-off explicitly.
+
+### What this means for connector authors
+
+- Tools execute via DOM, not HTTP. Implementation goes in the extension,
+  not in REST clients.
+- Tool descriptions in the manifest should describe **observable
+  behavior** ("list bugs assigned to me"), not the URL path or API
+  surface used to obtain them.
+- The connector should declare which page(s) it needs open
+  (`host_permissions`) so the extension can route or open tabs accordingly.
+- Selector breakage = expected maintenance burden. Bake stable selectors
+  into the connector code (e.g., `data-testid`, schema.org microdata)
+  and version-pin them.
+
+## Connector Lifecycle
+
+### Discovery
+- Built-in connectors ship in the extension package
+- Remote connectors fetched from a GitHub registry
+  (`forge-browser-connectors` — mirrors the `forge-crafts` model)
+- Extension popup → "Marketplace" → install / update / remove
+
+### Activation
+- When a tab navigates to a URL matching some connector's `matches`,
+  the extension auto-injects that connector's content script
+- Bridge is notified, MCP tools are advertised dynamically
+- "Connector X became available in tab Y" → Forge sees the new tools
+
+### Health monitoring
+- Bridge runs `healthCheck()` on each active connector every 10 min
+- Failures show in `Settings → Browser → Connectors` (red badge)
+- Optional: notify the user via Forge UI / Telegram bot
+
+### Auth expiry
+- Handler throws `AuthExpiredError` on 401 / login-redirect
+- Bridge pauses any in-flight pipeline using that tool
+- Notifies the user ("Please re-login to Mantis to resume")
+- Resumes automatically when the next health check passes
+
+## Connector Examples (DOM Extraction)
+
+### MantisBT
+- **Approach**: navigate the active tab to `my_view_page.php` /
+  `view_all_bug_page.php` / `view.php?id=<n>` and parse the rendered
+  HTML via Chrome MCP. PHP-rendered, structure hasn't changed in years.
+- **Why not REST?** `/api/rest/*` requires a Mantis API token; cookies
+  don't authenticate against it. Defeats the "logged-in session for free"
+  model.
+- **Estimated uptime**: 90%+. Self-hosted instances often theme the UI,
+  but core selectors (`#buglist`, `.column-id`, `.bug-status`) are stable.
+
+### GitLab (browser)
+- **Approach**: navigate to `/-/issues/?assignee_username=me` /
+  `/-/merge_requests/` / `/-/profile` pages and parse the rendered DOM.
+- **Why not REST?** `/api/v4/*` requires a PAT — same problem as Mantis.
+- **Selectors**: GitLab uses `data-testid` and `data-qa-*` attributes
+  heavily — relatively stable across versions.
+
+### Microsoft Teams Web
+- **Approach**: DOM scrape `[data-tid='message-content']` from rendered
+  messages. Microsoft's internal test-id attributes are surprisingly
+  stable across releases.
+- Avoid extracting MSAL tokens — fragile, key format changes 1-2× per
+  year, and surfacing them ourselves is a security smell.
+
+### Confluence / JIRA Server (browser)
+- **Approach**: DOM scrape `#main-content` and structured fields. The
+  rendered UI carries everything an agent needs (title, body, comments,
+  status), and the session cookie keeps working.
+
+### Internal homemade dashboards
+- **Approach**: stable selectors with `data-testid` if available, else
+  semantic class names. Pin selector versions; expect maintenance when
+  the dashboard ships a redesign.
+
+### When DOM extraction genuinely cannot work
+
+Some sites are pure SPAs that render via canvas, virtualize lists
+aggressively, or require many sequential interactions to expose data.
+For these, the connector author may surface an optional `api_token`
+setting and fall back to REST. This is **opt-in per user**, not a
+default code path. See "Tier 3 (escape)" in the execution model above.
+
+## Security
+
+### Trust boundaries
+- Extension runs with `host_permissions` for each connector's `matches`
+- Bridge ↔ extension uses a **per-installation pairing token** (random
+  256-bit, generated by Forge, copied into extension on first run)
+- Connector code is **not arbitrary** — must come from the marketplace,
+  signed by maintainer's GPG (sketched, full design TBD)
+
+### Destructive actions
+- Connector spec marks tools as `destructive: true` (writes, deletes,
+  sends messages)
+- Bridge presents an interactive confirmation to the user before
+  executing them — at least the first time per session per tool
+- Audit log retains request + response for every destructive call
+
+### Data flow
+- All connector responses pass through the bridge → Forge agent context
+- **No connector data is ever sent to external LLMs without explicit
+  agent invocation** (i.e., the connector doesn't auto-train the agent;
+  the agent decides what to do with the result)
+
+### Compliance
+- Some sites' ToS prohibit automated access. For internal company
+  systems and CRMs, this is a non-issue (you have a license). For
+  SaaS like Teams, we operate as "the user" — gray area; document
+  expectations in the extension's first-run consent.
+
+## Open Questions
+
+1. **Pairing token rotation** — should it be revocable per-device?
+   How is it rotated? Web UI?
+2. **Remote Forge** — extension currently assumes loopback bridge.
+   Should it support connecting to a Forge behind Cloudflare tunnel
+   so a phone can drive desktop's browser? Probably not v1.
+3. **Off-screen tab handling** — if a connector needs an active tab
+   but the user closed it, do we auto-reopen + wait, or error out?
+   Default: error, with optional auto-reopen flag.
+4. **Vision tier provider** — Anthropic Computer Use API vs OpenAI
+   gpt-4o-vision vs local model? Probably configurable.
+5. **Connector marketplace governance** — review process for accepting
+   new connectors? Mirrors forge-crafts but stricter (these run inside
+   the user's session).
+6. **Smith folding** — should `@aion0/smith-engine` (pi-coding-agent
+   loop + tempr memory client) be embedded in the extension as the
+   default chat backend? Or stay as a separate optional process?
+   See "Mutual-enhancement boundaries" in the agent strategy doc.
+
+## Implementation Phases
+
+### Phase 0 — Pairing + bridge skeleton (week 1)
+- `lib/browser-bridge-standalone.ts` in Forge, port 8407
+- Pairing UI in Forge Settings → Browser
+- Minimal MV3 extension shell, popup with "Connect to Forge" button
+- WebSocket handshake + ping/pong
+
+### Phase 1 — First connector + MCP wiring (week 2)
+- Connector SDK package (`@forge/browser-connector-sdk`)
+- Single connector: Mantis (assume internal install)
+- MCP tools auto-registered to Forge from bridge
+- End-to-end smoke: Forge agent calls `mantis.list_my_bugs` → result
+
+### Phase 2 — Tier 2 + monitoring (week 3)
+- DOM scraping fallback in Mantis connector
+- Health checks every 10 min, dashboard in Forge
+- Auth expiry detection + user notification
+
+### Phase 3 — Multi-connector + marketplace (week 4-5)
+- 2-3 more connectors (pick highest-value internal systems)
+- `forge-browser-connectors` GitHub registry
+- One-click install from extension marketplace
+- Versioning + update detection
+
+### Phase 4 — Vision fallback + Computer Use bridge (later)
+- Tier 3 vision LLM integration
+- Configurable provider
+- Reserved for known-bad sites only
+
+### Phase 5 — Smith integration / chat UI (separate RFC)
+- Decide whether extension hosts chat UI directly
+- If yes: temper-as-memory + pi-coding-agent loop in extension
+- Probably one quarter out
+
+## Out of Scope (for this RFC)
+
+- **Chat UI in extension popup/sidepanel** — separate design needed;
+  this RFC is purely about connector mechanics.
+- **Mobile** — same design works in principle but each browser's
+  extension API differs; Phase ∞.
+- **Cross-browser** (Safari, Firefox) — eventually yes, but Chrome
+  + Edge (Chromium) cover ~95% of enterprise.
+
+## References
+
+- `RFC-…` (placeholder for future chat-UI / agent-loop RFC)
+- MCP spec: <https://spec.modelcontextprotocol.io>
+- pi-coding-agent: <https://github.com/earendil-works/pi>
+- Anthropic Computer Use: <https://docs.anthropic.com/en/docs/build-with-claude/computer-use>
+- Chrome MV3 extension docs: <https://developer.chrome.com/docs/extensions/mv3/intro>
+
+## Changelog
+
+- **2026-05-16**: initial draft.