@mcinteerj/openclaw-gmail 1.3.1 → 1.4.1

package/README.md

@@ -1,117 +1,145 @@
-# Gmail Channel (Plugin)
+# openclaw-gmail
 
-Connects Moltbot to Gmail via the `gog` CLI.
+Gmail channel plugin for [OpenClaw](https://github.com/openclaw/openclaw). Supports two backends:
+
+- **API** (recommended) — connects directly via the Gmail API using OAuth2. No external CLI needed.
+- **gog** — shells out to the [gog CLI](https://github.com/jay/gog). The original backend, still fully supported.
+
+Both backends coexist — you can run different accounts on different backends.
 
 ## Installation
 
-This is a plugin. To install from source:
+```bash
+openclaw plugins install @mcinteerj/openclaw-gmail
+```
+
+Or from a local clone:
 
 ```bash
-moltbot plugins install ./extensions/gmail
+openclaw plugins install --link /path/to/openclaw-gmail
 ```
 
-## Features
+Requires `openclaw >= 2026.1.0`.
 
-- **Polling-based sync**: Robustly fetches new unread emails from Inbox.
-- **Circuit Breaker**: Handles API failures and rate limiting gracefully.
-- **Rich Text**: Markdown support for outbound emails.
-- **Threading**: Native Gmail thread support with quoted reply context.
-- **Archiving**: Automatically archives threads upon reply.
-- **Email Body Sanitisation**: Automatically cleans incoming email bodies for LLM consumption.
+## Setup
 
-## Email Body Sanitisation
+### Option 1: API backend (recommended)
 
-Incoming emails are automatically sanitised to produce clean, readable text — no configuration needed.
+The API backend connects directly to Gmail — no gog CLI required.
 
-### What It Does
+**If you have gog installed**, the onboarding flow will detect your existing OAuth client credentials from `~/.config/gogcli/credentials.json` and reuse them. You only need a one-time browser authorization to get a new refresh token.
 
-- **HTML-to-text conversion**: Strips tags, removes `<style>` and `<script>` blocks, filters out tracking pixels, and decodes HTML entities.
-- **Footer junk removal**: Strips common noise like unsubscribe links, "Sent from my iPhone", and confidentiality notices.
-- **Whitespace cleanup**: Collapses excessive blank lines and trims leading/trailing whitespace.
-- **Signature stripping**: Removes content below `-- ` signature separators by default.
+**If you don't have gog**, you'll need to create a GCP OAuth client:
 
-### Configurable Signature Stripping
+1. Go to [Google Cloud Console → Credentials](https://console.cloud.google.com/apis/credentials)
+2. Create a project (or use an existing one)
+3. Enable the Gmail API
+4. Create an OAuth 2.0 Client ID (type: **Desktop app**)
+5. Copy the Client ID and Client Secret
 
-Signature stripping is enabled by default. If you need to preserve content after `--` separators (e.g. for emails where dashes appear in the body), you can disable it programmatically:
+Then run `openclaw configure`, select Gmail, and follow the prompts. The flow will:
+- Ask for your email address
+- Prompt for client credentials (or reuse gog's)
+- Open a browser for OAuth consent
+- Store the refresh token in your OpenClaw config
+- Set `backend: "api"` on the account
 
-```ts
-extractTextBody(html, plain, { stripSignature: false })
+**Manual config** (if you prefer to skip the wizard):
+
+```json5
+{
+  "channels": {
+    "openclaw-gmail": {
+      "accounts": {
+        "you@gmail.com": {
+          "email": "you@gmail.com",
+          "backend": "api",
+          "oauth": {
+            "clientId": "your-client-id.apps.googleusercontent.com",
+            "clientSecret": "your-client-secret",
+            "refreshToken": "your-refresh-token"
+          },
+          "allowFrom": ["*"],
+          "pollIntervalMs": 60000
+        }
+      }
+    }
+  }
+}
 ```
 
-No plugin configuration is required — sanitisation runs automatically on every inbound message.
+### Option 2: gog backend
 
-## Reply Behavior
+Install the [gog CLI](https://github.com/jay/gog) (v1.2.0+), authorize it (`gog auth add you@gmail.com`), then run `openclaw configure`. The account will use gog by default (no `backend` field needed).
 
-- **Reply All**: When the bot replies to a thread, it uses "Reply All" to ensure all participants are included.
-- **Quoted Replies**: By default, replies include the full thread history as quoted text (standard Gmail format: "On [date], [author] wrote:"). This can be disabled per-account or globally.
-- **Allowlist Gatekeeping**: The bot only responds to emails from senders on the `allowFrom` list. However, if an allowed user includes others (CC) who are *not* on the allowlist, the bot will still "Reply All", including them in the conversation. This allows authorized users to bring others into the loop.
-- **Outbound Restrictions** (optional): Use `allowOutboundTo` and `threadReplyPolicy` to control who the bot can send emails to. By default, no outbound restrictions are applied (backwards compatible).
+### Upgrading from gog to API
 
-## Configuration
+Existing gog users upgrading the plugin will continue working with no changes — gog remains the default. To migrate an account to the API backend:
+
+1. Run `openclaw configure` → select Gmail
+2. The wizard detects your gog credentials and offers migration
+3. Authorize in the browser (one-time, ~10 seconds)
+4. Done — your account now uses the API directly
+
+Your gog installation is not affected and other accounts can continue using it.
+
+## Features
 
-Add to `moltbot.json`:
+- **Polling-based sync**: Fetches new unread emails from Inbox
+- **Rich text**: Markdown responses are converted to HTML emails via `marked`
+- **Threading**: Native Gmail thread support with quoted reply context
+- **Reply All**: Replies include all thread participants
+- **Archiving**: Automatically archives threads upon reply
+- **Email body sanitization**: Cleans incoming HTML for LLM consumption
+- **Circuit breaker** (gog backend): Handles API failures and rate limiting
+- **MIME construction** (API backend): Builds RFC 2822 messages with proper threading headers
+
+## Configuration
 
 ```json5
 {
   "channels": {
-    "gmail": {
+    "openclaw-gmail": {
       "accounts": {
-        "main": {
-          "email": "user@gmail.com",
+        "you@gmail.com": {
+          "email": "you@gmail.com",
           "allowFrom": ["*"],
-          "includeQuotedReplies": true,  // default: true
-          // Optional: restrict who the bot can send emails TO
-          "allowOutboundTo": ["@mycompany.com", "partner@example.com"],
-          "threadReplyPolicy": "allowlist"  // default: "open"
+          "pollIntervalMs": 60000,
+          "includeQuotedReplies": true,        // default: true
+          "allowOutboundTo": ["@company.com"],  // optional
+          "threadReplyPolicy": "allowlist"      // default: "open"
         }
       },
       "defaults": {
-        "includeQuotedReplies": true  // global default
+        "includeQuotedReplies": true
       }
     }
   }
 }
 ```
 
-### Configuration Options
-
 | Key | Type | Default | Description |
 |-----|------|---------|-------------|
-| `includeQuotedReplies` | boolean | `true` | Include thread history as quoted text in replies. Set to `false` for cleaner, shorter replies. |
-| `allowOutboundTo` | string[] | (falls back to `allowFrom`) | Restrict who the bot can send emails to. Supports exact emails and domain wildcards (e.g., `@company.com`). |
-| `threadReplyPolicy` | string | `"open"` | Controls thread reply behavior: `"open"` (no restrictions), `"allowlist"` (all recipients must be in `allowOutboundTo`), or `"sender-only"` (only original thread sender checked). |
+| `backend` | `"api"` \| `"gog"` | `"gog"` | Which backend to use for this account |
+| `oauth` | object | — | OAuth credentials (required for API backend) |
+| `allowFrom` | string[] | `[]` | Sender allowlist. `["*"]` allows all. |
+| `pollIntervalMs` | number | `60000` | Polling interval in milliseconds |
+| `includeQuotedReplies` | boolean | `true` | Include thread history as quoted text in replies |
+| `allowOutboundTo` | string[] | (falls back to `allowFrom`) | Restrict who the bot can send to. Supports domain wildcards (`@company.com`). |
+| `threadReplyPolicy` | `"open"` \| `"allowlist"` \| `"sender-only"` | `"open"` | Controls reply restrictions |
 
 ### Thread Reply Policies
 
-- **`open`** (default): No outbound restrictions. The bot can reply to any thread it was BCCd into. Backwards compatible.
-- **`allowlist`**: All thread participants (To, CC, From) must be in `allowOutboundTo`. Blocks replies if *any* recipient is not allowed.
-- **`sender-only`**: Only checks if the original thread sender is in `allowOutboundTo`. Useful when you want to reply to threads started by allowed users, even if they CC'd external parties.
+- **`open`** (default): No outbound restrictions. Backwards compatible.
+- **`allowlist`**: All thread participants must be in `allowOutboundTo`.
+- **`sender-only`**: Only checks if the original thread sender is allowed.
 
 ## Development
 
-Run tests:
 ```bash
-npm test
-# or
-./node_modules/.bin/vitest run src/
+npx vitest run
 ```
 
 ## Publishing
 
-This repo includes a GitHub Actions workflow for npm publishing.
-
-### Automatic (on release)
-Create a GitHub release → automatically publishes to npm.
-
-### Manual (workflow dispatch)
-Go to Actions → "Publish to npm" → Run workflow.
-
-**Note:** Manual dispatch will:
-1. Bump the version (patch by default, or specify major/minor)
-2. Commit and push the version bump with a tag
-3. Publish to npm
-
-### Setup
-Add `NPM_TOKEN` secret to the repository:
-1. Generate token at npmjs.com → Access Tokens → Classic Token (Automation)
-2. Add to repo: Settings → Secrets → Actions → New repository secret
+Create a GitHub release or run the "Publish to npm" workflow via Actions.

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@mcinteerj/openclaw-gmail",
-  "version": "1.3.1",
-  "description": "Gmail channel plugin for OpenClaw - uses gog CLI for secure Gmail access",
+  "version": "1.4.1",
+  "description": "Gmail channel plugin for OpenClaw - direct API or gog CLI",
   "type": "module",
   "main": "index.ts",
   "author": "Jake McInteer <mcinteerj@gmail.com>",
@@ -24,11 +24,11 @@
     "channel": {
       "id": "openclaw-gmail",
       "label": "Gmail",
-      "selectionLabel": "Gmail (gog)",
+      "selectionLabel": "Gmail",
       "detailLabel": "Gmail",
       "docsPath": "/channels/gmail",
       "docsLabel": "gmail",
-      "blurb": "Uses gog for secure Gmail access.",
+      "blurb": "Gmail integration via direct API or gog CLI.",
       "systemImage": "envelope",
       "order": 100,
       "showConfigured": true
@@ -41,9 +41,12 @@
     "README.md"
   ],
   "dependencies": {
+    "@googleapis/gmail": "^16.1.1",
     "dompurify": "^3.2.3",
+    "google-auth-library": "^10.5.0",
     "jsdom": "^25.0.1",
     "marked": "^15.0.6",
+    "nodemailer": "^8.0.1",
     "proper-lockfile": "^4.1.2",
     "sanitize-html": "^2.14.0",
     "zod": "^4.3.5"

package/src/accounts.ts

@@ -10,6 +10,12 @@ export interface ResolvedGmailAccount extends ResolvedChannelAccount {
   historyId?: string;
   delegate?: string;
   pollIntervalMs?: number;
+  backend?: "gog" | "api";
+  oauth?: {
+    clientId: string;
+    clientSecret: string;
+    refreshToken: string;
+  };
 }
 
 export function resolveGmailAccount(
@@ -42,6 +48,8 @@ export function resolveGmailAccount(
     delegate: account.delegate,
     allowFrom: account.allowFrom,
     pollIntervalMs: account.pollIntervalMs,
+    backend: account.backend,
+    oauth: account.oauth,
   };
 }
 

package/src/api-client.ts

@@ -0,0 +1,406 @@
+import { gmail as gmailApi, type gmail_v1 } from "@googleapis/gmail";
+import type { OAuth2Client } from "google-auth-library";
+import fs from "node:fs/promises";
+import type { GmailClient } from "./gmail-client.js";
+import type { ThreadResponse, GogRawMessage } from "./quoting.js";
+import type { GogSearchMessage } from "./inbound.js";
+import { buildMimeMessage } from "./mime.js";
+import { parseEmailAddresses } from "./outbound-check.js";
+
+/**
+ * Gmail API client using googleapis library directly.
+ * Implements GmailClient interface for the "api" backend.
+ */
+export class ApiGmailClient implements GmailClient {
+  private gmail: gmail_v1.Gmail;
+
+  constructor(auth: OAuth2Client) {
+    this.gmail = gmailApi({ version: "v1", auth });
+  }
+
+  async send(opts: {
+    account?: string;
+    to?: string;
+    subject: string;
+    textBody: string;
+    htmlBody?: string;
+    threadId?: string;
+    replyToMessageId?: string;
+    replyAll?: boolean;
+  }): Promise<void> {
+    const selfEmail = opts.account || "";
+    let to = opts.to || "";
+    let cc: string | undefined;
+    let inReplyTo: string | undefined;
+    let references: string | undefined;
+
+    // For thread replies, resolve recipients and threading headers
+    if (opts.threadId) {
+      const replyCtx = await this.resolveReplyContext(
+        opts.threadId,
+        selfEmail,
+        opts.replyAll ?? false,
+      );
+      if (replyCtx) {
+        to = replyCtx.to;
+        cc = replyCtx.cc;
+        inReplyTo = replyCtx.inReplyTo;
+        references = replyCtx.references;
+      }
+    }
+
+    if (!to) {
+      throw new Error("Cannot send: no recipient resolved");
+    }
+
+    const mime = await buildMimeMessage({
+      from: selfEmail,
+      to,
+      cc,
+      subject: opts.subject,
+      text: opts.textBody,
+      html: opts.htmlBody,
+      inReplyTo,
+      references,
+    });
+
+    await this.gmail.users.messages.send({
+      userId: "me",
+      requestBody: {
+        raw: mime.toString("base64url"),
+        threadId: opts.threadId,
+      },
+    });
+  }
+
+  /**
+   * Resolve reply recipients and threading headers from the last message in a thread.
+   */
+  private async resolveReplyContext(
+    threadId: string,
+    selfEmail: string,
+    replyAll: boolean,
+  ): Promise<{ to: string; cc?: string; inReplyTo?: string; references?: string } | null> {
+    const thread = await this.getThread(threadId, { full: true });
+    if (!thread || thread.messages.length === 0) return null;
+
+    const lastMsg = thread.messages[thread.messages.length - 1];
+
+    // Resolve To: reply to the sender of the last message
+    const to = lastMsg.from;
+
+    // For reply-all, Cc = everyone from To + Cc minus self and the new To
+    let cc: string | undefined;
+    if (replyAll) {
+      const selfLower = selfEmail.toLowerCase();
+      const toAddresses = parseEmailAddresses(to);
+      const toLower = new Set(toAddresses.map((a) => a.email.toLowerCase()));
+
+      const ccCandidates: string[] = [];
+      for (const field of [lastMsg.to, lastMsg.cc]) {
+        if (!field) continue;
+        const addresses = parseEmailAddresses(field);
+        for (const addr of addresses) {
+          if (addr.email.toLowerCase() !== selfLower && !toLower.has(addr.email.toLowerCase())) {
+            ccCandidates.push(addr.name ? `${addr.name} <${addr.email}>` : addr.email);
+            toLower.add(addr.email.toLowerCase()); // dedupe
+          }
+        }
+      }
+      if (ccCandidates.length > 0) {
+        cc = ccCandidates.join(", ");
+      }
+    }
+
+    // Resolve In-Reply-To and References from the last message
+    // We need the raw Message-ID header — fetch it from the API directly
+    let messageId: string | undefined;
+    try {
+      const res = await this.gmail.users.messages.get({
+        userId: "me",
+        id: lastMsg.id,
+        format: "metadata",
+        metadataHeaders: ["Message-ID", "References"],
+      });
+      const headers = res.data.payload?.headers || [];
+      messageId = headers.find((h) => h.name?.toLowerCase() === "message-id")?.value ?? undefined;
+      const existingRefs = headers.find((h) => h.name?.toLowerCase() === "references")?.value;
+      if (messageId) {
+        return {
+          to,
+          cc,
+          inReplyTo: messageId,
+          references: existingRefs ? `${existingRefs} ${messageId}` : messageId,
+        };
+      }
+    } catch {
+      // Fall through — send without threading headers
+    }
+
+    return { to, cc };
+  }
+
+  async getThread(threadId: string, opts?: { full?: boolean }): Promise<ThreadResponse | null> {
+    try {
+      const res = await this.gmail.users.threads.get({
+        userId: "me",
+        id: threadId,
+        format: opts?.full ? "full" : "metadata",
+      });
+
+      const thread = res.data;
+      if (!thread || !thread.messages) return null;
+
+      return {
+        id: thread.id!,
+        historyId: thread.historyId!,
+        messages: thread.messages.map((msg) => {
+          const raw = mapApiMessage(msg);
+          return parseRawToThreadMessage(raw);
+        }),
+      };
+    } catch (err: any) {
+      if (err?.code === 404) return null;
+      throw err;
+    }
+  }
+
+  async getMessage(messageId: string): Promise<Record<string, unknown> | null> {
+    try {
+      const res = await this.gmail.users.messages.get({
+        userId: "me",
+        id: messageId,
+        format: "full",
+      });
+
+      // Wrap in { message: ... } to match gog output shape consumed by monitor.ts
+      return { message: mapApiMessage(res.data) };
+    } catch (err: any) {
+      if (err?.code === 404) return null;
+      throw err;
+    }
+  }
+
+  async searchMessages(
+    query: string,
+    opts?: { maxResults?: number; includeBody?: boolean },
+  ): Promise<GogSearchMessage[]> {
+    const res = await this.gmail.users.messages.list({
+      userId: "me",
+      q: query,
+      maxResults: opts?.maxResults ?? 50,
+    });
+
+    const ids = res.data.messages || [];
+    if (ids.length === 0) return [];
+
+    // Fetch full message details in parallel (N+1 pattern)
+    const messages = await Promise.all(
+      ids.map(async (m) => {
+        try {
+          const detail = await this.gmail.users.messages.get({
+            userId: "me",
+            id: m.id!,
+            format: "full",
+          });
+          return detail.data;
+        } catch {
+          return null;
+        }
+      }),
+    );
+
+    return messages
+      .filter((m): m is gmail_v1.Schema$Message => m !== null)
+      .map((msg) => {
+        const headers = msg.payload?.headers || [];
+        const getH = (n: string) =>
+          headers.find((h) => h.name?.toLowerCase() === n.toLowerCase())?.value || "";
+
+        const body = extractPlainText(msg.payload ?? {});
+
+        return {
+          id: msg.id!,
+          threadId: msg.threadId!,
+          date: getH("Date"),
+          from: getH("From"),
+          subject: getH("Subject"),
+          body,
+          labels: msg.labelIds || [],
+        };
+      });
+  }
+
+  async searchThreads(
+    query: string,
+    opts?: { maxResults?: number },
+  ): Promise<Record<string, unknown> | null> {
+    const res = await this.gmail.users.threads.list({
+      userId: "me",
+      q: query,
+      maxResults: opts?.maxResults ?? 50,
+    });
+    return res.data as Record<string, unknown>;
+  }
+
+  async modifyLabels(id: string, opts: { add?: string[]; remove?: string[] }): Promise<void> {
+    await this.gmail.users.messages.modify({
+      userId: "me",
+      id,
+      requestBody: {
+        addLabelIds: opts.add,
+        removeLabelIds: opts.remove,
+      },
+    });
+  }
+
+  async modifyThreadLabels(
+    threadId: string,
+    opts: { add?: string[]; remove?: string[] },
+  ): Promise<void> {
+    await this.gmail.users.threads.modify({
+      userId: "me",
+      id: threadId,
+      requestBody: {
+        addLabelIds: opts.add,
+        removeLabelIds: opts.remove,
+      },
+    });
+  }
+
+  async listLabels(): Promise<{ id: string; name: string }[]> {
+    const res = await this.gmail.users.labels.list({ userId: "me" });
+    return (res.data.labels || []).map((l) => ({
+      id: l.id!,
+      name: l.name!,
+    }));
+  }
+
+  async createLabel(name: string): Promise<void> {
+    await this.gmail.users.labels.create({
+      userId: "me",
+      requestBody: { name },
+    });
+  }
+
+  async downloadAttachment(
+    messageId: string,
+    attachmentId: string,
+    outPath: string,
+  ): Promise<void> {
+    const res = await this.gmail.users.messages.attachments.get({
+      userId: "me",
+      messageId,
+      id: attachmentId,
+    });
+    // Gmail API returns base64url-encoded data
+    const buf = Buffer.from(res.data.data!, "base64url");
+    await fs.writeFile(outPath, buf);
+  }
+
+  async getSendAs(): Promise<{ displayName?: string; email: string; isPrimary?: boolean }[]> {
+    const res = await this.gmail.users.settings.sendAs.list({ userId: "me" });
+    return (res.data.sendAs || []).map((s) => ({
+      displayName: s.displayName || undefined,
+      email: s.sendAsEmail!,
+      isPrimary: s.isPrimary || false,
+    }));
+  }
+}
+
+// ── Response mapping helpers ──────────────────────────────────────────
+
+/**
+ * Map Gmail API Schema$Message to GogRawMessage shape.
+ * This keeps all downstream consumers (quoting.ts, inbound.ts, monitor.ts) working
+ * without changes.
+ */
+function mapApiMessage(msg: gmail_v1.Schema$Message): GogRawMessage {
+  return {
+    id: msg.id!,
+    threadId: msg.threadId!,
+    internalDate: msg.internalDate!,
+    labelIds: msg.labelIds || [],
+    payload: mapPayload(msg.payload ?? {}),
+  };
+}
+
+function mapPayload(
+  p: gmail_v1.Schema$MessagePart,
+): GogRawMessage["payload"] {
+  return {
+    headers: (p.headers || []).map((h) => ({
+      name: h.name!,
+      value: h.value!,
+    })),
+    parts: p.parts?.map((part) => ({
+      body: part.body?.data ? { data: part.body.data } : undefined,
+      mimeType: part.mimeType!,
+    })),
+    body: p.body?.data ? { data: p.body.data } : undefined,
+  };
+}
+
+/**
+ * Parse a GogRawMessage into a ThreadMessage (same logic as quoting.ts:parseGogMessage).
+ */
+function parseRawToThreadMessage(raw: GogRawMessage) {
+  const getH = (name: string) =>
+    raw.payload.headers.find((h) => h.name.toLowerCase() === name.toLowerCase())?.value;
+
+  const body = extractPlainFromRaw(raw);
+  const bodyHtml = extractHtmlFromRaw(raw);
+
+  return {
+    id: raw.id,
+    threadId: raw.threadId,
+    date: getH("Date") || new Date(parseInt(raw.internalDate)).toISOString(),
+    from: getH("From") || "",
+    to: getH("To"),
+    cc: getH("Cc"),
+    subject: getH("Subject") || "",
+    body,
+    bodyHtml,
+    labels: raw.labelIds,
+  };
+}
+
+function extractPlainFromRaw(raw: GogRawMessage): string {
+  if (raw.payload.parts) {
+    const plain = raw.payload.parts.find((p) => p.mimeType === "text/plain");
+    if (plain?.body?.data) return Buffer.from(plain.body.data, "base64").toString("utf-8");
+  }
+  if (raw.payload.body?.data) return Buffer.from(raw.payload.body.data, "base64").toString("utf-8");
+  return "";
+}
+
+function extractHtmlFromRaw(raw: GogRawMessage): string {
+  if (raw.payload.parts) {
+    const html = raw.payload.parts.find((p) => p.mimeType === "text/html");
+    if (html?.body?.data) return Buffer.from(html.body.data, "base64").toString("utf-8");
+  }
+  if (raw.payload.body?.data) return Buffer.from(raw.payload.body.data, "base64").toString("utf-8");
+  return "";
+}
+
+/**
+ * Extract plain text body from a Gmail API MessagePart.
+ */
+function extractPlainText(part: gmail_v1.Schema$MessagePart): string {
+  if (part.mimeType === "text/plain" && part.body?.data) {
+    return Buffer.from(part.body.data, "base64").toString("utf-8");
+  }
+  if (part.parts) {
+    // multipart/alternative: prefer text/plain
+    if (part.mimeType === "multipart/alternative") {
+      const plain = part.parts.find((p) => p.mimeType === "text/plain");
+      if (plain) return extractPlainText(plain);
+    }
+    // Recurse into sub-parts
+    for (const sub of part.parts) {
+      const text = extractPlainText(sub);
+      if (text) return text;
+    }
+  }
+  return "";
+}