@framers/agentos-ext-gmail-source 0.4.0 → 0.5.0

package/README.md

@@ -1,8 +1,8 @@
 # Gmail source
 
-This library provides read-only Gmail authorization on the AgentOS redirect grant and sealed-token contracts, text helpers that sanitize and trim mail before it is stored (see Text ingestion below), a bounded mapper that turns one thread response into a sanitized source document (see Thread documents below), and a capped read-only API client that sends every call through the host's quota and grant services (see Read-only API below). It requests the sole Gmail read-only scope, offline access and incremental consent. It exports no send, reply, label-modification or attachment-download tool.
+This library provides read-only Gmail authorization on the AgentOS redirect grant and sealed-token contracts, text helpers that sanitize and trim mail before it is stored (see Text ingestion below), a bounded mapper that turns one thread response into a sanitized source document (see Thread documents below), a capped read-only API client that sends every call through the host's quota and grant services (see Read-only API below), and a connector that binds them to durable account-source synchronization (see Connector and durable synchronization below). It requests the sole Gmail read-only scope, offline access and incremental consent. It exports no send, reply, label-modification or attachment-download tool.
 
-Install it next to `@framers/agentos` 0.13.134 or later and `@framers/agentos-ext-account-sources` 0.7.0 or later. Both are peer dependencies with a floor and no upper bound. Node.js 20.19 or later is required.
+Install it next to `@framers/agentos` 0.13.146 or later, whose HTML loader strips tags in linear time, and `@framers/agentos-ext-account-sources` 0.7.0 or later. Both are peer dependencies with a floor and no upper bound. Node.js 20.19 or later is required.
 
 ```ts
 import { GmailOAuthFlow } from '@framers/agentos-ext-gmail-source';
@@ -129,6 +129,46 @@ const changes = await api.history(profile.historyId, 100, null, signal); // { ki
 
 One provider operation makes one attempt, with one fresh-token retry permitted after 401. Every HTTP dispatch is separately metered. Token refreshes also use the one-unit profile reservation as conservative local accounting; this does not assert a Gmail quota cost for the OAuth endpoint. The durable account-source coordinator owns retry episodes, exponential backoff and exhaustion. The client persists provider waits through the quota service and never sleeps or buys capacity. A quota deferral, or a start that missed the quota's send window, sends nothing and throws `rate_limited` with the time to try again; only a reused attempt id is `invalid_response`. A retry time the client hands back is always after its own clock, even when the quota's database clock lags it; give the client the clock the host's sync store reads (`Date.now` for both unless given). Unknown errors become fixed failures without retaining their messages or causes.
 
+Each operation refreshes the grant unless the client has a `tokenHolder`: a memory-only `{ current: null }` slot that the clients of one sync lease share. A held access token is reused only while `currentHandle` answers the lease it was obtained under (the same connection, generation and lease token) and that lease has not expired by the client's clock; a 401 drops it, and the repeat refreshes as before. A reused token skips the grant operation, so check the sync handle before and after each call, as the connector does. Give each lease a new holder and never persist or log it. `GmailSourceConnector` makes one for each connector, so a sync cycle refreshes the grant once.
+
 A canonical thread 404 means gone. History 404 means the history anchor expired and bounded recovery is required. Domain policy is a fixed failure. Missing or explicitly invalid grants and insufficient scope are passed to the coordinator's fenced grant-loss lifecycle. Client configuration errors and transient network/provider failures are not reported as revoked mailboxes. Oversized successful thread responses are counted skips; oversized control pages fail without a cursor advance. Response streams are capped before decoding and are cancelled on size, deadline or caller cancellation.
 
 The client performs no storage write, purge, logging, model request or attachment fetch. Its successful result is still untrusted source material and needs the host's fresh owner/generation/expiry checks before storage, preparation or delivery. Limits are defensive library policy, not provider maxima; hosts must record any live compatibility failure before changing them.
+
+## Connector and durable synchronization
+
+```ts
+import { runGmailSourceSync, verifyGmailSource } from '@framers/agentos-ext-gmail-source';
+
+const options = {
+  scope, // the connection's tenant, owner and connection ids
+  coordinator: { store, prepare, clock }, // the released AccountSourceSync options without provider and lifecycle
+  revocations, // the host's SourceRevocations on the same tables
+  connector: {
+    api: { client, quota, projectKey: 'project-1', bounds, limits }, // as for GmailApi, without the handle and keys
+    mailbox, // the host's private identity binding: key(handle) and accept(handle, address)
+    bootstrapQuotaKey, // a stable keyed hash for the owner, never an address
+    permission: async (request) => cachePermission, // the evidenced permission for a read, or null
+    sourceDetailUrl: (scope, sourceId) => detailPath(sourceId), // an authenticated relative path, never a Gmail link
+  },
+};
+
+// Once, after the host installed the grant and recorded its generation as grant-ready.
+const verification = await verifyGmailSource(options, generation, signal); // { kind: 'verified' }, or why not
+// Each scheduled or requested sync. The run's connector is disposed when the run ends.
+const result = await runGmailSourceSync(options, 'scheduled', signal);
+```
+
+`GmailSourceConnector(options)` implements the original selected-id account-source interface and exposes the additive `sync` provider. It performs no document commit. Supply the released SQL sync store, revocation service, API configuration, dynamic current-handle getter, private mailbox binding, cache-permission callback, trusted relative detail-path builder and a stable opaque owner probe key. Every network operation uses the existing grant and quota services. The API clients it builds for one lease share one access token in memory, so a sync cycle refreshes the grant once; a 401 or a new lease refreshes it again, and disposal drops it. Never capture the first handle in a getter: source application can refresh the owner ticket.
+
+The mailbox binding is server-only. Both callbacks run under the exact handle's transaction through the released store. `key(handle)` reads the existing stable mailbox key or returns null before initial acceptance. `accept(handle, address)` compares or atomically creates the host's private identity binding and returns its stable key; null refuses the identity. It cannot replace an existing identity on reconnect. Store any display address sealed, never in a source, telemetry field or URL. The initial profile is charged to a stable owner probe key; all content uses the accepted mailbox key. The shared project budget applies to both.
+
+The connector verifies the mailbox before its first content operation even when a resumed coordinator skips `begin`. A fresh `begin` reads a new profile anchor before enumeration. Raw profile identity is passed only to the private binding. The connector retains no profile address. Each cycle can hold one sanitized label snapshot under its independent response expiry. Expired or refused names are omitted without repeated label requests. Cancellation, failure, cycle/generation changes and disposal clear the snapshot. Call `dispose()` in a finally block for direct use.
+
+The original `list`, `read` and `changesSince` methods operate only on selected ids, bounded by the smaller retained and per-lease changed-item allowance and a hard ceiling of 100. Empty ids never enumerate a mailbox. Initial selected changes capture an anchor before reading the selection. Incremental selected changes drain bounded history and then reread only selected changed ids. A partial history, expired anchor, retry, skipped read or stale fence returns no successful cursor. Only a canonical missing thread is reported as removed. An empty selection with a null cursor returns an empty-string sentinel which must never be used with a nonempty selection or persisted as a provider history id.
+
+Use `runGmailSourceSync(options, trigger, signal?)` for durable ingestion. Its options supply scope, coordinator configuration without a provider/lifecycle, the revocation service and connector configuration without a store/current-handle/clock/revocation service. The function constructs the released account-source coordinator, supplies this connector's sync provider and disposes the connector when the run ends. The coordinator alone owns candidate bounds, work persistence, retry episodes, history completion, reconciliation and generation-fenced application. A returned document is always passed through the released store's current policy and expiry checks. Do not persist a facade SourceRead directly and lose the full sync document's cache/provenance envelope.
+
+After a host atomically installs a candidate sealed grant under an authenticated one-use start, transitions that same generation to grant-ready and records pending identity verification, it may call `verifyGmailSource(options, expectedGeneration, signal?)`. This claims a real lease, verifies the profile, binds identity and releases the lease without fetching content or advancing any cursor. Busy means another lease exists, not identity approval. A scheduled worker also verifies identity before content. Failed or cancelled authorization cannot install a grant after its generation was fenced. Hosts must clean up exchanged grants that could not be installed. Never create a synthetic handle to bypass the grant service.
+
+`revoke()` invokes the released immediate fence/purge lifecycle and disposes the connector. Provider confirmation is a separate status from the revocation service; the account-independent queue worker performs that request. Run maintenance and expiry even while polling is paused. All source-derived results retain private ownership and deletion/expiry restrictions. This package exposes no send, reply, attachment download or general URL fetch.

package/dist/GmailSourceConnector.d.ts

@@ -0,0 +1,161 @@
+import { type ResumableAccountSourceConnector, type AccountSourceSyncProvider, type SourceSelection, type SourceItem, type SourceRead, type SourceGone, type SourceChanges, type SourceSyncRequest, type SourceSyncScope, type SourceSyncHandle, type SourceSyncSkipReason, type SourceCachePermission, type SqlAccountSourceSyncStore, type SourceRevocations } from '@framers/agentos-ext-account-sources';
+import { type GmailApiOptions } from './gmail.js';
+/** All callbacks are server-only and use the exact current handle's owner transaction. */
+export interface GmailMailboxBinding {
+    /** Return the stable opaque mailbox key, or null before first profile acceptance. */
+    key(handle: SourceSyncHandle): Promise<string | null>;
+    /** Compare or create the private identity binding; null refuses any content access. */
+    accept(handle: SourceSyncHandle, emailAddress: string): Promise<string | null>;
+}
+/**
+ * What a `GmailSourceConnector` is built from: trusted server configuration and the host's released account-source
+ * services, never values from a browser request.
+ *
+ * @typeParam Prepared - The host's prepared-document type of its sync store and revocation service.
+ */
+export interface GmailSourceConnectorOptions<Prepared = unknown> {
+    /**
+     * The API client's configuration without the values the connector supplies itself: the current handle, the quota
+     * key of the mailbox, the revocation service, the clock and the lease's access-token holder.
+     */
+    readonly api: Omit<GmailApiOptions<Prepared>, 'currentHandle' | 'mailboxKey' | 'revocations' | 'clock' | 'tokenHolder'>;
+    /** The released SQL sync store: its snapshot and its exact-handle transaction. */
+    readonly store: Pick<SqlAccountSourceSyncStore<Prepared>, 'snapshot' | 'withCurrentHandle'>;
+    /** The released revocation service: its grant operation and the fenced disconnect `revoke` uses. */
+    readonly revocations: Pick<SourceRevocations<Prepared>, 'withGrant' | 'disconnect'>;
+    /** Answers the handle that holds this connection's lease now; never a getter of the first handle. */
+    readonly currentHandle: () => SourceSyncHandle;
+    /** The host's private mailbox identity binding. */
+    readonly mailbox: GmailMailboxBinding;
+    /**
+     * A stable opaque quota key for the identity probe before a mailbox is bound: 1 to 512 letters, digits, `:`, `_` or
+     * `-`, derived by the host with a server secret, never an address or a key that rotates with a connection.
+     */
+    readonly bootstrapQuotaKey: string;
+    /** The host's evidenced cache permission for one read; null records a `not_permitted` skip without a request. */
+    readonly permission: (request: SourceSyncRequest) => Promise<SourceCachePermission | null>;
+    /** The host's trusted, authenticated relative source-detail path for one thread; never a Gmail link. */
+    readonly sourceDetailUrl: (scope: SourceSyncScope, sourceId: string) => string;
+    /** Current time as integer UTC epoch milliseconds, the clock the sync store reads; `Date.now` unless given. */
+    readonly clock?: () => number;
+}
+/**
+ * A connector's fixed refusal; its message is `gmail_connector_<reason>`. `configuration` for an invalid option or
+ * clock, `selection` for a malformed or oversized selection or cursor, `history_bound` when selected history did not
+ * finish within its page budget, `busy` for an operation that overlaps another on the same connector, and `skipped`
+ * for a selected read that was deliberately not retained, with the skip reason in `skip`.
+ */
+export declare class GmailConnectorFault extends Error {
+    readonly reason: 'configuration' | 'selection' | 'history_bound' | 'busy' | 'skipped';
+    readonly skip: SourceSyncSkipReason | null;
+    /**
+     * @param reason Why the connector refused.
+     * @param skip The skip reason of a `skipped` refusal, otherwise null.
+     */
+    constructor(reason: 'configuration' | 'selection' | 'history_bound' | 'busy' | 'skipped', skip?: SourceSyncSkipReason | null);
+}
+/**
+ * One serial connector per active lease. Store, quota and grant services own durability.
+ *
+ * Every operation, original or `sync`, runs alone on the connector within at most 90 seconds and one second before
+ * the lease expires. It checks the exact current handle, the connection's generation, barrier, pause flag and policy
+ * before its work, and the handle again after it, so a late result of a superseded lease is refused. The first content
+ * operation of each lease, and every `begin`, reads the profile and passes its address only to the host's binding.
+ * The clients the connector builds for one lease share one access token, so a cycle refreshes the grant once. Call
+ * `dispose()` when the run ends.
+ *
+ * @typeParam Prepared - The host's prepared-document type of its sync store and revocation service.
+ */
+export declare class GmailSourceConnector<Prepared = unknown> implements ResumableAccountSourceConnector {
+    private readonly options;
+    /** The account-source kind. */
+    readonly kind = "gmail";
+    /** The resumable provider the account-source coordinator drives; it commits nothing. */
+    readonly sync: AccountSourceSyncProvider;
+    private disposed;
+    private controller;
+    private verified;
+    private api;
+    private labelContext;
+    private labelPermission;
+    private labelAttempted;
+    private labels;
+    private labelTimer;
+    private readonly tokens;
+    private readonly clock;
+    /**
+     * @param options The API configuration, the host's store, revocation service, current-handle getter, mailbox
+     * binding, owner probe key, permission callback, detail-path builder and optional clock. Construction sends nothing.
+     * @throws {GmailConnectorFault} `configuration` for a malformed owner probe key.
+     */
+    constructor(options: GmailSourceConnectorOptions<Prepared>);
+    private now;
+    private clearLabels;
+    /**
+     * Ends the connector: aborts the operation in flight and drops the label snapshot, the verified identity, the API
+     * clients and the held access token. Every later call throws `cancelled`. Call it in a `finally` block.
+     */
+    dispose(): void;
+    private stored;
+    private current;
+    private fenced;
+    private context;
+    private operation;
+    private makeApi;
+    private needApi;
+    private verifyIdentity;
+    private cycleLabels;
+    private readDocument;
+    private selection;
+    private selected;
+    /**
+     * Reads the selected threads, freshly mapped; an empty selection enumerates nothing.
+     *
+     * @param selection Thread ids, at most the smaller of 100, the retained-document limit and the changed-per-lease
+     * limit.
+     * @param signal Cancels the call.
+     * @returns The read items and the threads Gmail no longer holds.
+     * @throws {GmailConnectorFault} `selection` before any request for a malformed or oversized selection, `skipped`
+     * for a thread that was not retained.
+     * @throws {SourceSyncFault} A fixed failure, including a retry with its time.
+     */
+    list(selection: SourceSelection, signal?: AbortSignal): Promise<{
+        items: SourceItem[];
+        gone: SourceGone[];
+    }>;
+    /**
+     * Reads one thread by its id; the caller's title, size and version are not trusted.
+     *
+     * @param item The thread to read.
+     * @param signal Cancels the call.
+     * @returns The freshly mapped read, or `removed` when Gmail no longer holds the thread.
+     * @throws {GmailConnectorFault} `selection` for a malformed id, `skipped` for a thread that was not retained.
+     * @throws {SourceSyncFault} A fixed failure, including a retry with its time.
+     */
+    read(item: SourceItem, signal?: AbortSignal): Promise<SourceRead | SourceGone>;
+    /**
+     * The selected threads changed since a cursor. With a null cursor it reads the profile's history id first and then
+     * the whole selection, answering that id; with a cursor it drains history within the page budget and rereads only
+     * the selected threads it names. An empty selection answers the given cursor, or the empty-string sentinel for a
+     * null one, without a request; never persist that sentinel or pass it with a nonempty selection.
+     *
+     * @param fromCursor A history id from an earlier call, or null.
+     * @param selection The thread ids, bounded as for `list`.
+     * @param signal Cancels the call.
+     * @returns The changed and removed selected threads and the cursor reached; only a fully successful call answers one.
+     * @throws {GmailConnectorFault} `selection` for a malformed cursor or selection, `history_bound` when history did not
+     * finish within the page budget, `skipped` for a thread that was not retained.
+     * @throws {SourceSyncFault} `history_expired` for an expired cursor, or another fixed failure.
+     */
+    changesSince(fromCursor: string | null, selection: SourceSelection, signal?: AbortSignal): Promise<SourceChanges>;
+    /**
+     * Disposes the connector and runs the released fenced disconnect for the current generation: the local purge and
+     * the queueing of the grant for the revocation worker. Resolving does not mean Google has confirmed the
+     * revocation; the revocation service's `status` reports that.
+     *
+     * @throws {SourceSyncFault} `grant_rejected` when the generation has moved on, `storage_unavailable` while the
+     * local purge is still pending.
+     */
+    revoke(): Promise<void>;
+}
+//# sourceMappingURL=GmailSourceConnector.d.ts.map

package/dist/GmailSourceConnector.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"GmailSourceConnector.d.ts","sourceRoot":"","sources":["../src/GmailSourceConnector.ts"],"names":[],"mappings":"AAUA,OAAO,EAEL,KAAK,+BAA+B,EAAE,KAAK,yBAAyB,EACpE,KAAK,eAAe,EAAE,KAAK,UAAU,EAAE,KAAK,UAAU,EAAE,KAAK,UAAU,EAAE,KAAK,aAAa,EAC3F,KAAK,iBAAiB,EAAE,KAAK,eAAe,EAAE,KAAK,gBAAgB,EACnE,KAAK,oBAAoB,EAAE,KAAK,qBAAqB,EAAE,KAAK,yBAAyB,EACrF,KAAK,iBAAiB,EACvB,MAAM,sCAAsC,CAAC;AAC9C,OAAO,EAAY,KAAK,eAAe,EAA2C,MAAM,YAAY,CAAC;AAErG,0FAA0F;AAC1F,MAAM,WAAW,mBAAmB;IAClC,qFAAqF;IACrF,GAAG,CAAC,MAAM,EAAE,gBAAgB,GAAG,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,CAAC;IACtD,uFAAuF;IACvF,MAAM,CAAC,MAAM,EAAE,gBAAgB,EAAE,YAAY,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,IAAI,CAAC,CAAC;CAChF;AACD;;;;;GAKG;AACH,MAAM,WAAW,2BAA2B,CAAC,QAAQ,GAAG,OAAO;IAC7D;;;OAGG;IACH,QAAQ,CAAC,GAAG,EAAE,IAAI,CAAC,eAAe,CAAC,QAAQ,CAAC,EAAE,eAAe,GAAG,YAAY,GAAG,aAAa,GAAG,OAAO,GAAG,aAAa,CAAC,CAAC;IACxH,kFAAkF;IAClF,QAAQ,CAAC,KAAK,EAAE,IAAI,CAAC,yBAAyB,CAAC,QAAQ,CAAC,EAAE,UAAU,GAAG,mBAAmB,CAAC,CAAC;IAC5F,oGAAoG;IACpG,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC,iBAAiB,CAAC,QAAQ,CAAC,EAAE,WAAW,GAAG,YAAY,CAAC,CAAC;IACpF,qGAAqG;IACrG,QAAQ,CAAC,aAAa,EAAE,MAAM,gBAAgB,CAAC;IAC/C,mDAAmD;IACnD,QAAQ,CAAC,OAAO,EAAE,mBAAmB,CAAC;IACtC;;;OAGG;IACH,QAAQ,CAAC,iBAAiB,EAAE,MAAM,CAAC;IACnC,iHAAiH;IACjH,QAAQ,CAAC,UAAU,EAAE,CAAC,OAAO,EAAE,iBAAiB,KAAK,OAAO,CAAC,qBAAqB,GAAG,IAAI,CAAC,CAAC;IAC3F,wGAAwG;IACxG,QAAQ,CAAC,eAAe,EAAE,CAAC,KAAK,EAAE,eAAe,EAAE,QAAQ,EAAE,MAAM,KAAK,MAAM,CAAC;IAC/E,+GAA+G;IAC/G,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,MAAM,CAAC;CAC/B;AACD;;;;;GAKG;AACH,qBAAa,mBAAoB,SAAQ,KAAK;IAKhC,QAAQ,CAAC,MAAM,EAAE,eAAe,GAAG,WAAW,GAAG,eAAe,GAAG,MAAM,GAAG,SAAS;IAC/F,QAAQ,CAAC,IAAI,EAAE,oBAAoB,GAAG,IAAI;IAL5C;;;OAGG;gBACkB,MAAM,EAAE,eAAe,GAAG,WAAW,GAAG,eAAe,GAAG,MAAM,GAAG,SAAS,EACtF,IAAI,GAAE,oBAAoB,GAAG,IAAW;CAGpD;AAeD;;;;;;;;;;;GAWG;AACH,qBAAa,oBAAoB,CAAC,QAAQ,GAAG,OAAO,CAAE,YAAW,+BAA+B;IAsBlF,OAAO,CAAC,QAAQ,CAAC,OAAO;IArBpC,+BAA+B;IAC/B,QAAQ,CAAC,IAAI,WAAW;IACxB,wFAAwF;IACxF,QAAQ,CAAC,IAAI,EAAE,yBAAyB,CAAC;IACzC,OAAO,CAAC,QAAQ,CAAS;IACzB,OAAO,CAAC,UAAU,CAAgC;IAClD,OAAO,CAAC,QAAQ,CAAuB;IACvC,OAAO,CAAC,GAAG,CAAmC;IAC9C,OAAO,CAAC,YAAY,CAAuB;IAC3C,OAAO,CAAC,eAAe,CAAuB;IAC9C,OAAO,CAAC,cAAc,CAAS;IAC/B,OAAO,CAAC,MAAM,CAA4B;IAC1C,OAAO,CAAC,UAAU,CAA8C;IAEhE,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAuC;IAC9D,OAAO,CAAC,QAAQ,CAAC,KAAK,CAAe;IACrC;;;;OAIG;gBAC0B,OAAO,EAAE,2BAA2B,CAAC,QAAQ,CAAC;IAmB3E,OAAO,CAAC,GAAG;IAGX,OAAO,CAAC,WAAW;IAKnB;;;OAGG;IACH,OAAO,IAAI,IAAI;YAKD,MAAM;IAOpB,OAAO,CAAC,OAAO;YASD,MAAM;YAON,OAAO;YAQP,SAAS;IA0CvB,OAAO,CAAC,OAAO;IAKf,OAAO,CAAC,OAAO;YAMD,cAAc;YAiBd,WAAW;YAyBX,YAAY;IAa1B,OAAO,CAAC,SAAS;IAOjB,OAAO,CAAC,QAAQ;IAMhB;;;;;;;;;;OAUG;IACG,IAAI,CAAC,SAAS,EAAE,eAAe,EAAE,MAAM,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC;QAAE,KAAK,EAAE,UAAU,EAAE,CAAC;QAAC,IAAI,EAAE,UAAU,EAAE,CAAA;KAAE,CAAC;IAWlH;;;;;;;;OAQG;IACG,IAAI,CAAC,IAAI,EAAE,UAAU,EAAE,MAAM,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC,UAAU,GAAG,UAAU,CAAC;IAOpF;;;;;;;;;;;;;OAaG;IACG,YAAY,CAAC,UAAU,EAAE,MAAM,GAAG,IAAI,EAAE,SAAS,EAAE,eAAe,EAAE,MAAM,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC,aAAa,CAAC;IAkCvH;;;;;;;OAOG;IACG,MAAM,IAAI,OAAO,CAAC,IAAI,CAAC;CAM9B"}