@framers/agentos-ext-gmail-source 0.1.0 → 0.3.0

package/README.md

@@ -1,8 +1,8 @@
 # Gmail source
 
-This library provides read-only Gmail authorization on the AgentOS redirect grant and sealed-token contracts. It requests the sole Gmail read-only scope, offline access and incremental consent. It exports no send, reply, label-modification or attachment-download tool.
+This library provides read-only Gmail authorization on the AgentOS redirect grant and sealed-token contracts, text helpers that sanitize and trim mail before it is stored (see Text ingestion below), and a bounded mapper that turns one thread response into a sanitized source document (see Thread documents below). It requests the sole Gmail read-only scope, offline access and incremental consent. It exports no send, reply, label-modification or attachment-download tool.
 
-Install it next to `@framers/agentos` 0.13.134 or later and `@framers/agentos-ext-account-sources` 0.4.0 or later. Both are peer dependencies with a floor and no upper bound. Node.js 20.19 or later is required.
+Install it next to `@framers/agentos` 0.13.134 or later and `@framers/agentos-ext-account-sources` 0.7.0 or later. Both are peer dependencies with a floor and no upper bound. Node.js 20.19 or later is required.
 
 ```ts
 import { GmailOAuthFlow } from '@framers/agentos-ext-gmail-source';
@@ -37,3 +37,60 @@ OAuth responses are capped at 65,536 bytes and must finish within 15 seconds. Pa
 The host supplies informed consent, cache permission, retention policy, account isolation and a truthful availability state. Authorization alone does not establish permission to store or send mailbox content to a model. Do not infer those permissions from this package's successful tests.
 
 Protocol references: https://developers.google.com/identity/protocols/oauth2/web-server and https://developers.google.com/workspace/gmail/api/auth/scopes.
+
+## Text ingestion
+
+```ts
+import { messageText, sanitizeMailHeaders, sanitizeMailMetadata } from '@framers/agentos-ext-gmail-source';
+
+const policy = { maxInputCharacters: 2_097_152, maxLineCharacters: 65_536, maxOutputCharacters: 20_000 };
+
+// Strings the host's MIME reader has already decoded within its byte and structure bounds.
+const headers = sanitizeMailHeaders(decodedHeaders, policy); // { subject, date, from, to, cc }
+const body = messageText(decodedPlainText, policy);
+const filename = sanitizeMailMetadata(decodedFilename, { ...policy, maxOutputCharacters: 200 }).text;
+// body.text is sanitized and trimmed; body.trimmedQuote, body.trimmedSignature and body.truncated report omissions.
+```
+
+`sanitizeMailText(input, policy)` removes address tokens and credential-shaped content without a network call. `sanitizeMailMetadata` applies the same rules and folds whitespace for filenames, label names and display fields. `sanitizeMailHeaders` retains only Subject, Date, From, To and Cc; its Subject output is capped at 200 characters. MIME header decoding and structural validation must happen before these functions receive strings. Bcc is never returned.
+
+`policy` supplies `maxInputCharacters`, `maxLineCharacters` and `maxOutputCharacters` as positive UTF-16 code-unit counts. The input ceiling is 2,097,152 and the line ceiling is 65,536. The output cap cannot exceed the input cap. Oversized input throws `GmailTextFault` with a fixed reason; an overlong line is suppressed. Processing never scans only a truncated prefix for credentials. Repeated percent and common entity decoding is bounded, and unresolved encoded-byte lines are removed. Output truncation avoids splitting a surrogate pair and is marked in the result.
+
+`messageText(input, policy)` sanitizes first, trims recognized reply history and short signatures, then applies the output cap. Explicit forwarded content and ambiguous prose are retained. Its `trimmedQuote`, `trimmedSignature` and `truncated` flags describe omissions. Quote spans must refer to its returned string, without inserted summaries or an ellipsis presented as source text. A host should disclose that attachments are listed and their contents are not read when its mapper implements that boundary.
+
+The functions return text and counts only. They do not store raw values or a reverse-redaction map, log matches, fetch URLs, interpret mail as instructions or invoke a model. Sanitization can remove benign text and cannot identify every possible secret. It does not make personal mail anonymous or authorize model processing, sharing or retention. Hosts must preserve the source's privacy, expiry and deletion rules for both text and derived results.
+
+## Thread documents
+
+```ts
+import { GmailThreadMapper } from '@framers/agentos-ext-gmail-source';
+
+const mapper = new GmailThreadMapper({
+  maxResponseBytes: 2_097_152, maxDecodedBytes: 2_097_152, maxMimeDepth: 20, maxMimeParts: 1_000,
+  maxMessages: 100, maxMessageCharacters: 20_000, maxThreadCharacters: 200_000,
+  maxMetadataCharacters: 200_000, maxInputCharacters: 2_097_152, maxLineCharacters: 65_536,
+  maximumCacheMs: 86_400_000,
+});
+
+// `response` is the parsed threads.get (format=full) body from a transport that capped its bytes.
+const result = await mapper.map(response, {
+  sourceId: response.id,
+  sourceDetailUrl: detailPath, // the host's authenticated source-detail route, never a Gmail link
+  cutoff, windowMs: 30 * 86_400_000, readStartedAt, receivedAt,
+  permission, cacheHeaders, labelNames, labelCache, signal,
+});
+// result.kind is 'document' (text, message spans, metadata, cache expiry) or 'skipped' with a fixed reason.
+```
+
+Construct `GmailThreadMapper(bounds)` with explicit response, decoded-byte, MIME depth/part, retained-message, message/thread text, metadata, sanitizer and cache ceilings. `map(thread, options)` returns the account-source `SourceSyncReadResult` union. The input is the parsed response from a transport that already enforced its byte cap before JSON decoding. The mapper separately checks recognized MIME data and the entire part tree before decoding any selected body. A parsed object alone cannot establish its original wire size.
+
+Supply a fixed selection cutoff, message window, original read start and completion times, actual response cache headers, independently established cache permission and a host-generated relative detail path. The path must resolve authentication and current source generation in the host. It is never a Gmail deep link or a request-derived external URL. A successful OAuth grant does not establish cache or model-processing permission.
+Pass `labelCache` as the separate labels response’s cache decision. Missing, refused or expired label permission keeps ids without names. Permitted names shorten the entire document’s expiry when their allowed lifetime ends first.
+
+Each message is filtered by its internal timestamp and excluded labels. Retained messages are selected newest first with stable id ties, within the explicit cap, and rendered chronologically. Plain text wins within a MIME alternative. Other multipart containers retain their non-attachment text children in MIME order. HTML uses the public AgentOS `HtmlLoader` with an in-memory Buffer. No browser or remote resource is loaded. The resulting text, structured headers, filenames, media types and label names all pass the text sanitizer.
+
+Attachments are listed; their contents are not read. An external body or attachment contributes only sanitized filename, media type and size. Inline attachment bytes and attachment ids are discarded. Bcc, raw payloads, snippets and unknown fields are not returned. The mapper never calls any Gmail endpoint. Unsupported charset, invalid bytes and omitted content have fixed counted reasons. Invalid protocol shape is a failure rather than an empty successful source.
+
+A returned document has matching item/ref versions, a normalized-content hash, message ids and body spans, earliest allowed expiry and explicit omission/truncation metadata. Spans count UTF-16 code units in the returned text and exclude the header prefix. Quote trimming preserves normalized source text; no generated summary or ellipsis is inserted into a quote. Header-only messages can have an empty body span. Render structured metadata as escaped text and treat all mail as untrusted data.
+
+A history-only change does not extend cache life or change the normalized hash. Storage still requires the account-source transactional fence, fresh expiry checks and host allowances. Derived results retain the whole source's personal-use, expiry and deletion restrictions. Never persist the transient MIME types or bypass the account-source store with a mapper result.

package/dist/index.d.ts

@@ -1,8 +1,15 @@
 /**
  * @fileoverview A read-only Gmail source's grant: Google's read-only scope on the AgentOS redirect OAuth flow and
- * sealed-token store, with capped, cancellable OAuth calls and fixed failures.
+ * sealed-token store, with capped, cancellable OAuth calls and fixed failures; its text helpers: bounded
+ * pre-storage sanitization of mail text, allow-listed headers and metadata, and conservative trimming of reply history
+ * and signatures; and its thread mapper: bounded MIME reading of one thread into a sanitized account-source document
+ * with message provenance, attachment listings and an explicit cache decision.
  * @module agentos-ext-gmail-source
  */
 export { GmailOAuthFlow } from './GmailOAuthFlow.js';
 export { GMAIL_READONLY_SCOPE, GmailOAuthFault, type GmailOAuthFailure, type GmailGrantCleanup, type GmailOAuthClient, type GmailOAuthEndpoints, type GmailOAuthOptions, } from './types.js';
+export { GmailTextFault, sanitizeMailText, sanitizeMailMetadata, sanitizeMailHeaders, type GmailTextPolicy, type GmailSanitizedText, type GmailHeaderInput, type GmailSanitizedHeaders, type GmailTextFailure, } from './mailSanitize.js';
+export { messageText, type GmailMessageText } from './messageText.js';
+export { GmailThreadMapper, type GmailMapThreadOptions } from './mapThread.js';
+export { GmailMappingFault, type GmailMappingBounds } from './mime.js';
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAE,cAAc,EAAE,MAAM,qBAAqB,CAAC;AACrD,OAAO,EACL,oBAAoB,EAAE,eAAe,EAAE,KAAK,iBAAiB,EAAE,KAAK,iBAAiB,EACrF,KAAK,gBAAgB,EAAE,KAAK,mBAAmB,EAAE,KAAK,iBAAiB,GACxE,MAAM,YAAY,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AAEH,OAAO,EAAE,cAAc,EAAE,MAAM,qBAAqB,CAAC;AACrD,OAAO,EACL,oBAAoB,EAAE,eAAe,EAAE,KAAK,iBAAiB,EAAE,KAAK,iBAAiB,EACrF,KAAK,gBAAgB,EAAE,KAAK,mBAAmB,EAAE,KAAK,iBAAiB,GACxE,MAAM,YAAY,CAAC;AACpB,OAAO,EACL,cAAc,EAAE,gBAAgB,EAAE,oBAAoB,EAAE,mBAAmB,EAC3E,KAAK,eAAe,EAAE,KAAK,kBAAkB,EAAE,KAAK,gBAAgB,EACpE,KAAK,qBAAqB,EAAE,KAAK,gBAAgB,GAClD,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EAAE,WAAW,EAAE,KAAK,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AACtE,OAAO,EAAE,iBAAiB,EAAE,KAAK,qBAAqB,EAAE,MAAM,gBAAgB,CAAC;AAC/E,OAAO,EAAE,iBAAiB,EAAE,KAAK,kBAAkB,EAAE,MAAM,WAAW,CAAC"}

package/dist/index.js

@@ -1,8 +1,15 @@
 /**
  * @fileoverview A read-only Gmail source's grant: Google's read-only scope on the AgentOS redirect OAuth flow and
- * sealed-token store, with capped, cancellable OAuth calls and fixed failures.
+ * sealed-token store, with capped, cancellable OAuth calls and fixed failures; its text helpers: bounded
+ * pre-storage sanitization of mail text, allow-listed headers and metadata, and conservative trimming of reply history
+ * and signatures; and its thread mapper: bounded MIME reading of one thread into a sanitized account-source document
+ * with message provenance, attachment listings and an explicit cache decision.
  * @module agentos-ext-gmail-source
  */
 export { GmailOAuthFlow } from './GmailOAuthFlow.js';
 export { GMAIL_READONLY_SCOPE, GmailOAuthFault, } from './types.js';
+export { GmailTextFault, sanitizeMailText, sanitizeMailMetadata, sanitizeMailHeaders, } from './mailSanitize.js';
+export { messageText } from './messageText.js';
+export { GmailThreadMapper } from './mapThread.js';
+export { GmailMappingFault } from './mime.js';
 //# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;GAIG;AAEH,OAAO,EAAE,cAAc,EAAE,MAAM,qBAAqB,CAAC;AACrD,OAAO,EACL,oBAAoB,EAAE,eAAe,GAEtC,MAAM,YAAY,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;;;GAOG;AAEH,OAAO,EAAE,cAAc,EAAE,MAAM,qBAAqB,CAAC;AACrD,OAAO,EACL,oBAAoB,EAAE,eAAe,GAEtC,MAAM,YAAY,CAAC;AACpB,OAAO,EACL,cAAc,EAAE,gBAAgB,EAAE,oBAAoB,EAAE,mBAAmB,GAG5E,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EAAE,WAAW,EAAyB,MAAM,kBAAkB,CAAC;AACtE,OAAO,EAAE,iBAAiB,EAA8B,MAAM,gBAAgB,CAAC;AAC/E,OAAO,EAAE,iBAAiB,EAA2B,MAAM,WAAW,CAAC"}

package/dist/mailSanitize.d.ts

@@ -0,0 +1,100 @@
+/**
+ * @fileoverview Deterministic pre-storage sanitization of mail text: address tokens, credential assignments, known
+ * credential shapes, private-key blocks and long mixed tokens are replaced with fixed marks, and a final gate
+ * suppresses any line that still carries a credential cue. Every count is a number; no matched value, offset or
+ * reverse mapping is kept, nothing is logged and no model or network is called.
+ * @module agentos-ext-gmail-source/mailSanitize
+ */
+/**
+ * The bounds of one sanitization call, as positive safe integer UTF-16 code-unit counts. The input ceiling is
+ * 2,097,152, the line ceiling 65,536, and the output cap cannot exceed the input cap.
+ */
+export interface GmailTextPolicy {
+    /** The largest input accepted, before and after decoding; larger input throws `GmailTextFault('input_too_large')`. */
+    readonly maxInputCharacters: number;
+    /** The longest line inspected; a longer line is suppressed whole before any token matching. */
+    readonly maxLineCharacters: number;
+    /** The longest text returned; the complete input is checked before this cap clips the result. */
+    readonly maxOutputCharacters: number;
+}
+/** Sanitized text and content-free counts of what was removed. The counts are not permission to log the text. */
+export interface GmailSanitizedText {
+    /** The sanitized text, clipped to the output cap without splitting a surrogate pair. */
+    readonly text: string;
+    /** How many address tokens became `[address removed]`. */
+    readonly addressRedactions: number;
+    /** How many credential lines, shapes, long mixed tokens and private-key blocks became `[secret removed]`. */
+    readonly secretRedactions: number;
+    /** How many lines became `[line removed by the secret gate]`. */
+    readonly suppressedLines: number;
+    /** Whether the output cap removed text. */
+    readonly truncated: boolean;
+}
+/** One message header after bounded MIME header decoding: its name and its decoded value. */
+export interface GmailHeaderInput {
+    readonly name: string;
+    readonly value: string;
+}
+/** The retained headers, each sanitized; Bcc and every other header are never returned. */
+export interface GmailSanitizedHeaders {
+    /** The first Subject, at most 200 characters. */
+    readonly subject: string;
+    /** The first Date, at most 1,000 characters. */
+    readonly date: string;
+    /** Every From value, joined with commas, at most 4,000 characters. */
+    readonly from: string;
+    /** Every To value, joined with commas, at most 4,000 characters. */
+    readonly to: string;
+    /** Every Cc value, joined with commas, at most 4,000 characters. */
+    readonly cc: string;
+}
+/** Why a text helper refused its input, as a fixed word: an invalid policy or input above its cap. */
+export type GmailTextFailure = 'configuration' | 'input_too_large';
+/** A text helper's fixed refusal; its message is `gmail_text_<reason>` and never carries source text. */
+export declare class GmailTextFault extends Error {
+    readonly reason: GmailTextFailure;
+    /** @param reason Why the helper refused its input. */
+    constructor(reason: GmailTextFailure);
+}
+/**
+ * Pure pre-storage transformation. It never records matched values or invokes a model.
+ *
+ * Decodes repeated percent and character-reference encoding (at most four passes), normalizes Unicode and line breaks,
+ * then works line by line: private-key blocks, credential assignments and authenticated URLs are replaced, credential
+ * shapes and long mixed tokens become `[secret removed]`, address tokens become `[address removed]`, and a line with
+ * unresolved encoded bytes or character references, above the line cap or still carrying a credential cue becomes
+ * `[line removed by the secret gate]`. An assignment with an empty value, a block scalar or an unclosed quote also
+ * suppresses the rest of the text. A replaced assignment and a line the final gate suppresses keep their quote markers
+ * (`>`). The complete input is checked before the output cap clips the result.
+ *
+ * @param input Decoded text: a body or one display field.
+ * @param policy The input, line and output bounds.
+ * @returns The sanitized text and content-free counts.
+ * @throws {GmailTextFault} `configuration` for an invalid policy, `input_too_large` above the input cap.
+ */
+export declare function sanitizeMailText(input: string, policy: GmailTextPolicy): GmailSanitizedText;
+/**
+ * Use for every filename, label name and display field, after MIME decoding. Applies the same rules as
+ * `sanitizeMailText` to the complete value, folds each whitespace run to one space and trims, then applies the
+ * output cap.
+ *
+ * @param input One decoded metadata value.
+ * @param policy The input, line and output bounds.
+ * @returns The sanitized one-line value and content-free counts.
+ * @throws {GmailTextFault} `configuration` for an invalid policy, `input_too_large` above the input cap.
+ */
+export declare function sanitizeMailMetadata(input: string, policy: GmailTextPolicy): GmailSanitizedText;
+/**
+ * Input values must already have passed bounded MIME header decoding. Retains only Subject, Date, From, To and Cc,
+ * matched without regard to case, and sanitizes each as metadata: the first Subject (at most 200 characters) and Date
+ * (at most 1,000), and every From, To and Cc value joined with commas (at most 4,000 each), each also within the
+ * policy's output cap.
+ *
+ * @param headers At most 1,000 decoded headers, whose names and values together fit the input cap.
+ * @param policy The input, line and output bounds.
+ * @returns The five retained fields; a missing header is an empty string.
+ * @throws {GmailTextFault} `configuration` for an invalid policy, `input_too_large` for more than 1,000 headers or
+ * text above the input cap.
+ */
+export declare function sanitizeMailHeaders(headers: readonly GmailHeaderInput[], policy: GmailTextPolicy): GmailSanitizedHeaders;
+//# sourceMappingURL=mailSanitize.d.ts.map

package/dist/mailSanitize.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"mailSanitize.d.ts","sourceRoot":"","sources":["../src/mailSanitize.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AAEH;;;GAGG;AACH,MAAM,WAAW,eAAe;IAC9B,sHAAsH;IACtH,QAAQ,CAAC,kBAAkB,EAAE,MAAM,CAAC;IACpC,+FAA+F;IAC/F,QAAQ,CAAC,iBAAiB,EAAE,MAAM,CAAC;IACnC,iGAAiG;IACjG,QAAQ,CAAC,mBAAmB,EAAE,MAAM,CAAC;CACtC;AACD,iHAAiH;AACjH,MAAM,WAAW,kBAAkB;IACjC,wFAAwF;IACxF,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,0DAA0D;IAC1D,QAAQ,CAAC,iBAAiB,EAAE,MAAM,CAAC;IACnC,6GAA6G;IAC7G,QAAQ,CAAC,gBAAgB,EAAE,MAAM,CAAC;IAClC,iEAAiE;IACjE,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;IACjC,2CAA2C;IAC3C,QAAQ,CAAC,SAAS,EAAE,OAAO,CAAC;CAC7B;AACD,6FAA6F;AAC7F,MAAM,WAAW,gBAAgB;IAAG,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAAC,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAA;CAAE;AACnF,2FAA2F;AAC3F,MAAM,WAAW,qBAAqB;IACpC,iDAAiD;IACjD,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,gDAAgD;IAChD,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,sEAAsE;IACtE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,oEAAoE;IACpE,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,oEAAoE;IACpE,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;CACrB;AACD,sGAAsG;AACtG,MAAM,MAAM,gBAAgB,GAAG,eAAe,GAAG,iBAAiB,CAAC;AACnE,yGAAyG;AACzG,qBAAa,cAAe,SAAQ,KAAK;IAE3B,QAAQ,CAAC,MAAM,EAAE,gBAAgB;IAD7C,sDAAsD;gBACjC,MAAM,EAAE,gBAAgB;CAG9C;AA4FD;;;;;;;;;;;;;;;GAeG;AACH,wBAAgB,gBAAgB,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,eAAe,GAAG,kBAAkB,CA2C3F;AACD;;;;;;;;;GASG;AACH,wBAAgB,oBAAoB,CAAC,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,eAAe,GAAG,kBAAkB,CAM/F;AACD;;;;;;;;;;;GAWG;AACH,wBAAgB,mBAAmB,CAAC,OAAO,EAAE,SAAS,gBAAgB,EAAE,EAAE,MAAM,EAAE,eAAe,GAAG,qBAAqB,CAsBxH"}

package/dist/mailSanitize.js

@@ -0,0 +1,257 @@
+/**
+ * @fileoverview Deterministic pre-storage sanitization of mail text: address tokens, credential assignments, known
+ * credential shapes, private-key blocks and long mixed tokens are replaced with fixed marks, and a final gate
+ * suppresses any line that still carries a credential cue. Every count is a number; no matched value, offset or
+ * reverse mapping is kept, nothing is logged and no model or network is called.
+ * @module agentos-ext-gmail-source/mailSanitize
+ */
+/** A text helper's fixed refusal; its message is `gmail_text_<reason>` and never carries source text. */
+export class GmailTextFault extends Error {
+    reason;
+    /** @param reason Why the helper refused its input. */
+    constructor(reason) {
+        super(`gmail_text_${reason}`);
+        this.reason = reason;
+        this.name = 'GmailTextFault';
+    }
+}
+const addressMark = '[address removed]';
+const secretMark = '[secret removed]';
+const lineMark = '[line removed by the secret gate]';
+// Named character references decoded during canonicalization; any other name stays as written.
+const namedEntities = Object.freeze({
+    amp: '&', lt: '<', gt: '>', quot: '"', apos: "'", commat: '@', period: '.',
+    colon: ':', sol: '/', bsol: '\\', equals: '=', percnt: '%', num: '#',
+    lowbar: '_', hyphen: '-', semi: ';', plus: '+', quest: '?', nbsp: ' ',
+    Tab: '\t', NewLine: '\n',
+});
+// Credential shapes replaced in place. They are a conservative policy, not a claim about every provider's format. A
+// prefix right after a letter or digit is part of a word (`risk-adjusted`), not a credential. A JWT is matched only
+// from the start of its run of token characters, so a long `-` joined run cannot make the scan quadratic.
+const shapeSources = [
+    String.raw `(?<![A-Za-z0-9])(?:GOCSPX-|ya29\.|1//0|ghp_|github_pat_|xox[bpas]-|sk-ant-|sk-proj-|sk-|tvly-|whsec_)[A-Za-z0-9._~+/=-]{6,}`,
+    String.raw `\b(?:AKIA|ASIA)[A-Z0-9]{16}\b`,
+    String.raw `\bfc-[0-9a-f]{32}\b`,
+    String.raw `\bAIza[A-Za-z0-9_-]{20,}`,
+    String.raw `\bre_[A-Za-z0-9]{6,}_[A-Za-z0-9_]{10,}`,
+    String.raw `\b[rs]k_(?:live|test)_[A-Za-z0-9]{10,}`,
+    String.raw `(?<![A-Za-z0-9_-])eyJ[A-Za-z0-9_-]{4,}\.[A-Za-z0-9_-]{4,}\.[A-Za-z0-9_-]{4,}\b`,
+];
+// A credential label followed by `:` or `=`: the whole line is replaced. The password words (also plural), `token`,
+// `secret`, `secret key` and the compound labels count at the end of any identifier (`DB_PASSWORD=`, `PGPASSWORD=`,
+// `GITHUB_TOKEN=`, `authToken:`, `SECRET_KEY=`, `Passwords:`). The short or common labels count only after a character
+// other than a letter or digit, so `Compass:` and `Preauthorization:` stay.
+const assignment = /(?:(?<![a-z0-9])(?:pass|pwd|pw|authorization|cookie|set-cookie|database[ _-]?url|signing[ _-]?key|one[ _-]?time[ _-]?code)|pass(?:word|wd|code|phrase)s?|api[ _-]?key|access[ _-]?(?:key|token)|refresh[ _-]?token|client[ _-]?secret|secret(?:[ _-]?key)?|token|auth[ _-]?key|provisioning[ _-]?key|verification[ _-]?code)["']?\s*[:=]\s*/i;
+// The final gate's cues: a line still carrying one after replacement is suppressed whole. The password words (also
+// plural) and the compound cues count at the end of any identifier (`DB_PASSWORD`, `rsaPrivateKey`); `bearer` and the
+// SSH cues need a character other than a letter or digit before them. No cue continues into a letter or digit.
+const residual = /(?:(?<![a-z0-9])(?:bearer|ssh-(?:rsa|ed25519)|sshpass)|pass(?:word|wd|phrase)s?|api[ _-]?(?:key|token)|access[ _-]?token|refresh[ _-]?token|client[ _-]?secret|private[ _-]?key)(?![a-z0-9])/i;
+const beginKey = /-----BEGIN [A-Z0-9 ]*PRIVATE KEY-----/;
+const endKey = /-----END [A-Z0-9 ]*PRIVATE KEY-----/;
+// Encoded bytes still present after the bounded decoding passes.
+const unresolved = /%[0-9a-f]{2}|&#(?:x[0-9a-f]+|[0-9]+);?/i;
+// A named reference the decoder knows, still present after the passes: the text was nested deeper than four levels.
+const unresolvedName = new RegExp(`&(?:${Object.keys(namedEntities).join('|')});`);
+function validate(policy) {
+    for (const value of [policy.maxInputCharacters, policy.maxLineCharacters, policy.maxOutputCharacters]) {
+        if (!Number.isSafeInteger(value) || value < 1)
+            throw new GmailTextFault('configuration');
+    }
+    if (policy.maxInputCharacters > 2_097_152 || policy.maxLineCharacters > 65_536
+        || policy.maxOutputCharacters > policy.maxInputCharacters)
+        throw new GmailTextFault('configuration');
+}
+// At most four passes of percent and character-reference decoding, Unicode compatibility normalization, line-break
+// unification and removal of format and control characters. The cap is checked before decoding and again after each
+// pass, since normalization can lengthen text.
+function canonical(input, limit) {
+    if (input.length > limit)
+        throw new GmailTextFault('input_too_large');
+    let text = input;
+    for (let pass = 0; pass < 4; pass++) {
+        const previous = text;
+        text = text.replace(/(?:%[0-9a-f]{2})+/gi, (run) => {
+            try {
+                return decodeURIComponent(run);
+            }
+            catch {
+                return run;
+            }
+        }).replace(/&(#x[0-9a-f]+|#[0-9]+|[a-z][a-z0-9]+);/gi, (whole, entity) => {
+            if (!entity.startsWith('#'))
+                return namedEntities[entity] ?? whole;
+            const hex = entity[1]?.toLowerCase() === 'x';
+            const point = Number.parseInt(entity.slice(hex ? 2 : 1), hex ? 16 : 10);
+            return Number.isSafeInteger(point) && point >= 0 && point <= 0x10ffff
+                && !(point >= 0xd800 && point <= 0xdfff) ? String.fromCodePoint(point) : whole;
+        });
+        text = text.normalize('NFKC').replace(/\r\n?/g, '\n').replace(/[\u2028\u2029]/g, '\n')
+            .replace(/\p{Cf}/gu, '').replace(/[\x00-\x08\x0b\x0c\x0e-\x1f\x7f-\x9f]/g, '');
+        if (text.length > limit)
+            throw new GmailTextFault('input_too_large');
+        if (text === previous)
+            break;
+    }
+    return text;
+}
+// Clips without leaving the high half of a surrogate pair at the end.
+function clipped(text, maximum) {
+    let end = Math.min(text.length, maximum);
+    if (end > 0 && end < text.length && /[\uD800-\uDBFF]/.test(text[end - 1])
+        && /[\uDC00-\uDFFF]/.test(text[end]))
+        end--;
+    return text.slice(0, end);
+}
+// Whether the line holds a URL with a user name or password, or an unparsable URL with an `@`. The scheme is at most
+// 64 characters, so a long run of scheme characters cannot make the scan quadratic; a longer one still leaves the `@`
+// for the address rule.
+function containsAuthenticatedUrl(line) {
+    for (const match of line.matchAll(/\b[a-z][a-z0-9+.-]{0,63}:\/\/[^\s<>"']+/gi)) {
+        try {
+            const url = new URL(match[0]);
+            if (url.username || url.password)
+                return true;
+        }
+        catch {
+            if (match[0].includes('@'))
+                return true;
+        }
+    }
+    return false;
+}
+/**
+ * Pure pre-storage transformation. It never records matched values or invokes a model.
+ *
+ * Decodes repeated percent and character-reference encoding (at most four passes), normalizes Unicode and line breaks,
+ * then works line by line: private-key blocks, credential assignments and authenticated URLs are replaced, credential
+ * shapes and long mixed tokens become `[secret removed]`, address tokens become `[address removed]`, and a line with
+ * unresolved encoded bytes or character references, above the line cap or still carrying a credential cue becomes
+ * `[line removed by the secret gate]`. An assignment with an empty value, a block scalar or an unclosed quote also
+ * suppresses the rest of the text. A replaced assignment and a line the final gate suppresses keep their quote markers
+ * (`>`). The complete input is checked before the output cap clips the result.
+ *
+ * @param input Decoded text: a body or one display field.
+ * @param policy The input, line and output bounds.
+ * @returns The sanitized text and content-free counts.
+ * @throws {GmailTextFault} `configuration` for an invalid policy, `input_too_large` above the input cap.
+ */
+export function sanitizeMailText(input, policy) {
+    validate(policy);
+    const normalized = canonical(input, policy.maxInputCharacters);
+    let addressRedactions = 0;
+    let secretRedactions = 0;
+    let suppressedLines = 0;
+    let insideKey = false;
+    let credentialTail = false;
+    const lines = [];
+    for (const raw of normalized.split('\n')) {
+        if (credentialTail) {
+            suppressedLines++;
+            lines.push(lineMark);
+            continue;
+        }
+        if (insideKey || beginKey.test(raw)) {
+            if (!insideKey)
+                secretRedactions++;
+            insideKey = !endKey.test(raw);
+            lines.push(secretMark);
+            continue;
+        }
+        if (raw.length > policy.maxLineCharacters || unresolved.test(raw) || unresolvedName.test(raw)) {
+            suppressedLines++;
+            lines.push(lineMark);
+            continue;
+        }
+        // Preserve quote depth so removal cannot turn quoted history into fresh text.
+        const quote = raw.match(/^([ \t]*(?:>[ \t]*)+)/)?.[0] ?? '';
+        let line = raw.slice(quote.length);
+        if (assignment.test(line) || containsAuthenticatedUrl(line) || /\bsshpass\s+-p\b/i.test(line)) {
+            const found = assignment.exec(line);
+            const value = found ? line.slice(found.index + found[0].length).trim() : null;
+            credentialTail = value !== null && (value === '' || /^[|>][+-]?$/.test(value) || ((value[0] === '"' || value[0] === "'") && value.lastIndexOf(value[0]) === 0));
+            secretRedactions++;
+            lines.push(quote + secretMark);
+            continue;
+        }
+        for (const source of shapeSources) {
+            line = line.replace(new RegExp(source, 'g'), () => { secretRedactions++; return secretMark; });
+        }
+        line = line.replace(/[A-Za-z0-9_+/=-]{32,}/g, (token) => {
+            if ((/[a-z]/i.test(token) && /[0-9]/.test(token)) || /^[0-9a-f]{40,}$/i.test(token)) {
+                secretRedactions++;
+                return secretMark;
+            }
+            return token;
+        });
+        line = line.replace(/"[^"\n]{1,512}"\s*@[^\s<>;,"']+/gu, () => { addressRedactions++; return addressMark; });
+        line = line.replace(/[^\s<>;,"'()\[\]]+/gu, (token) => { if (!token.includes('@'))
+            return token; addressRedactions++; return addressMark; });
+        const inspected = line.split(secretMark).join('').split(lineMark).join('');
+        if (residual.test(inspected) || endKey.test(inspected)) {
+            suppressedLines++;
+            lines.push(quote + lineMark);
+            continue;
+        }
+        lines.push(quote + line);
+    }
+    const complete = lines.join('\n');
+    const text = clipped(complete, policy.maxOutputCharacters);
+    return { text, addressRedactions, secretRedactions, suppressedLines, truncated: text.length < complete.length };
+}
+/**
+ * Use for every filename, label name and display field, after MIME decoding. Applies the same rules as
+ * `sanitizeMailText` to the complete value, folds each whitespace run to one space and trims, then applies the
+ * output cap.
+ *
+ * @param input One decoded metadata value.
+ * @param policy The input, line and output bounds.
+ * @returns The sanitized one-line value and content-free counts.
+ * @throws {GmailTextFault} `configuration` for an invalid policy, `input_too_large` above the input cap.
+ */
+export function sanitizeMailMetadata(input, policy) {
+    validate(policy);
+    const safe = sanitizeMailText(input, { ...policy, maxOutputCharacters: policy.maxInputCharacters });
+    const complete = safe.text.replace(/\s+/gu, ' ').trim();
+    const text = clipped(complete, policy.maxOutputCharacters);
+    return { ...safe, text, truncated: safe.truncated || text.length < complete.length };
+}
+/**
+ * Input values must already have passed bounded MIME header decoding. Retains only Subject, Date, From, To and Cc,
+ * matched without regard to case, and sanitizes each as metadata: the first Subject (at most 200 characters) and Date
+ * (at most 1,000), and every From, To and Cc value joined with commas (at most 4,000 each), each also within the
+ * policy's output cap.
+ *
+ * @param headers At most 1,000 decoded headers, whose names and values together fit the input cap.
+ * @param policy The input, line and output bounds.
+ * @returns The five retained fields; a missing header is an empty string.
+ * @throws {GmailTextFault} `configuration` for an invalid policy, `input_too_large` for more than 1,000 headers or
+ * text above the input cap.
+ */
+export function sanitizeMailHeaders(headers, policy) {
+    validate(policy);
+    if (headers.length > 1000)
+        throw new GmailTextFault('input_too_large');
+    let characters = 0;
+    const retained = new Map();
+    for (const header of headers) {
+        characters += header.name.length + header.value.length;
+        if (characters > policy.maxInputCharacters)
+            throw new GmailTextFault('input_too_large');
+        const name = header.name.toLowerCase();
+        if (!['subject', 'date', 'from', 'to', 'cc'].includes(name))
+            continue;
+        const values = retained.get(name) ?? [];
+        values.push(header.value);
+        retained.set(name, values);
+    }
+    const field = (name, single, maximum) => {
+        const values = retained.get(name) ?? [];
+        const raw = single ? values[0] ?? '' : values.join(', ');
+        return sanitizeMailMetadata(raw, { ...policy, maxOutputCharacters: Math.min(maximum, policy.maxOutputCharacters) }).text;
+    };
+    return {
+        subject: field('subject', true, 200), date: field('date', true, 1000),
+        from: field('from', false, 4000), to: field('to', false, 4000), cc: field('cc', false, 4000),
+    };
+}
+//# sourceMappingURL=mailSanitize.js.map

package/dist/mailSanitize.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"mailSanitize.js","sourceRoot":"","sources":["../src/mailSanitize.ts"],"names":[],"mappings":"AAAA;;;;;;GAMG;AA4CH,yGAAyG;AACzG,MAAM,OAAO,cAAe,SAAQ,KAAK;IAElB;IADrB,sDAAsD;IACtD,YAAqB,MAAwB;QAC3C,KAAK,CAAC,cAAc,MAAM,EAAE,CAAC,CAAC;QADX,WAAM,GAAN,MAAM,CAAkB;QACZ,IAAI,CAAC,IAAI,GAAG,gBAAgB,CAAC;IAC9D,CAAC;CACF;AAED,MAAM,WAAW,GAAG,mBAAmB,CAAC;AACxC,MAAM,UAAU,GAAG,kBAAkB,CAAC;AACtC,MAAM,QAAQ,GAAG,mCAAmC,CAAC;AACrD,+FAA+F;AAC/F,MAAM,aAAa,GAAqC,MAAM,CAAC,MAAM,CAAC;IACpE,GAAG,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,EAAE,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG;IAC1E,KAAK,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,IAAI,EAAE,IAAI,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG;IACpE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,KAAK,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG;IACrE,GAAG,EAAE,IAAI,EAAE,OAAO,EAAE,IAAI;CACzB,CAAC,CAAC;AACH,oHAAoH;AACpH,oHAAoH;AACpH,0GAA0G;AAC1G,MAAM,YAAY,GAAsB;IACtC,MAAM,CAAC,GAAG,CAAA,6HAA6H;IACvI,MAAM,CAAC,GAAG,CAAA,+BAA+B;IACzC,MAAM,CAAC,GAAG,CAAA,qBAAqB;IAC/B,MAAM,CAAC,GAAG,CAAA,0BAA0B;IACpC,MAAM,CAAC,GAAG,CAAA,wCAAwC;IAClD,MAAM,CAAC,GAAG,CAAA,wCAAwC;IAClD,MAAM,CAAC,GAAG,CAAA,gFAAgF;CAC3F,CAAC;AACF,oHAAoH;AACpH,oHAAoH;AACpH,uHAAuH;AACvH,4EAA4E;AAC5E,MAAM,UAAU,GAAG,8UAA8U,CAAC;AAClW,mHAAmH;AACnH,sHAAsH;AACtH,+GAA+G;AAC/G,MAAM,QAAQ,GAAG,+LAA+L,CAAC;AACjN,MAAM,QAAQ,GAAG,uCAAuC,CAAC;AACzD,MAAM,MAAM,GAAG,qCAAqC,CAAC;AACrD,iEAAiE;AACjE,MAAM,UAAU,GAAG,yCAAyC,CAAC;AAC7D,oHAAoH;AACpH,MAAM,cAAc,GAAG,IAAI,MAAM,CAAC,OAAO,MAAM,CAAC,IAAI,CAAC,aAAa,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;AAEnF,SAAS,QAAQ,CAAC,MAAuB;IACvC,KAAK,MAAM,KAAK,IAAI,CAAC,MAAM,CAAC,kBAAkB,EAAE,MAAM,CAAC,iBAAiB,EAAE,MAAM,CAAC,mBAAmB,CAAC,EAAE,CAAC;QACtG,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,KAAK,GAAG,CAAC;YAAE,MAAM,IAAI,cAAc,CAAC,eAAe,CAAC,CAAC;IAC3F,CAAC;IACD,IAAI,MAAM,CAAC,kBAAkB,GAAG,SAAS,IAAI,MAAM,CAAC,iBAAiB,GAAG,MAAM;WACzE,MAAM,CAAC,mBAAmB,GAAG,MAAM,CAAC,kBAAkB;QAAE,MAAM,IAAI,cAAc,CAAC,eAAe,CAAC,CAAC;AACzG,CAAC;AACD,mHAAmH;AACnH,oHAAoH;AACpH,+CAA+C;AAC/C,SAAS,SAAS,CAAC,KAAa,EAAE,KAAa;IAC7C,IAAI,KAAK,CAAC,MAAM,GAAG,KAAK;QAAE,MAAM,IAAI,cAAc,CAAC,iBAAiB,CAAC,CAAC;IACtE,IAAI,IAAI,GAAG,KAAK,CAAC;IACjB,KAAK,IAAI,IAAI,GAAG,CAAC,EAAE,IAAI,GAAG,CAAC,EAAE,IAAI,EAAE,EAAE,CAAC;QACpC,MAAM,QAAQ,GAAG,IAAI,CAAC;QACtB,IAAI,GAAG,IAAI,CAAC,OAAO,CAAC,qBAAqB,EAAE,CAAC,GAAG,EAAE,EAAE;YACjD,IAAI,CAAC;gBAAC,OAAO,kBAAkB,CAAC,GAAG,CAAC,CAAC;YAAC,CAAC;YAAC,MAAM,CAAC;gBAAC,OAAO,GAAG,CAAC;YAAC,CAAC;QAC/D,CAAC,CAAC,CAAC,OAAO,CAAC,0CAA0C,EAAE,CAAC,KAAK,EAAE,MAAc,EAAE,EAAE;YAC/E,IAAI,CAAC,MAAM,CAAC,UAAU,CAAC,GAAG,CAAC;gBAAE,OAAO,aAAa,CAAC,MAAM,CAAC,IAAI,KAAK,CAAC;YACnE,MAAM,GAAG,GAAG,MAAM,CAAC,CAAC,CAAC,EAAE,WAAW,EAAE,KAAK,GAAG,CAAC;YAC7C,MAAM,KAAK,GAAG,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;YACxE,OAAO,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,IAAI,KAAK,IAAI,QAAQ;mBAChE,CAAC,CAAC,KAAK,IAAI,MAAM,IAAI,KAAK,IAAI,MAAM,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC;QACnF,CAAC,CAAC,CAAC;QACH,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC,OAAO,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC,OAAO,CAAC,iBAAiB,EAAE,IAAI,CAAC;aACnF,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC,OAAO,CAAC,wCAAwC,EAAE,EAAE,CAAC,CAAC;QACjF,IAAI,IAAI,CAAC,MAAM,GAAG,KAAK;YAAE,MAAM,IAAI,cAAc,CAAC,iBAAiB,CAAC,CAAC;QACrE,IAAI,IAAI,KAAK,QAAQ;YAAE,MAAM;IAC/B,CAAC;IACD,OAAO,IAAI,CAAC;AACd,CAAC;AACD,sEAAsE;AACtE,SAAS,OAAO,CAAC,IAAY,EAAE,OAAe;IAC5C,IAAI,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACzC,IAAI,GAAG,GAAG,CAAC,IAAI,GAAG,GAAG,IAAI,CAAC,MAAM,IAAI,iBAAiB,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,GAAG,CAAC,CAAE,CAAC;WACrE,iBAAiB,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAE,CAAC;QAAE,GAAG,EAAE,CAAC;IAC/C,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;AAC5B,CAAC;AACD,qHAAqH;AACrH,sHAAsH;AACtH,wBAAwB;AACxB,SAAS,wBAAwB,CAAC,IAAY;IAC5C,KAAK,MAAM,KAAK,IAAI,IAAI,CAAC,QAAQ,CAAC,2CAA2C,CAAC,EAAE,CAAC;QAC/E,IAAI,CAAC;YACH,MAAM,GAAG,GAAG,IAAI,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;YAC9B,IAAI,GAAG,CAAC,QAAQ,IAAI,GAAG,CAAC,QAAQ;gBAAE,OAAO,IAAI,CAAC;QAChD,CAAC;QAAC,MAAM,CAAC;YACP,IAAI,KAAK,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,GAAG,CAAC;gBAAE,OAAO,IAAI,CAAC;QAC1C,CAAC;IACH,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AACD;;;;;;;;;;;;;;;GAeG;AACH,MAAM,UAAU,gBAAgB,CAAC,KAAa,EAAE,MAAuB;IACrE,QAAQ,CAAC,MAAM,CAAC,CAAC;IACjB,MAAM,UAAU,GAAG,SAAS,CAAC,KAAK,EAAE,MAAM,CAAC,kBAAkB,CAAC,CAAC;IAC/D,IAAI,iBAAiB,GAAG,CAAC,CAAC;IAAC,IAAI,gBAAgB,GAAG,CAAC,CAAC;IAAC,IAAI,eAAe,GAAG,CAAC,CAAC;IAC7E,IAAI,SAAS,GAAG,KAAK,CAAC;IAAC,IAAI,cAAc,GAAG,KAAK,CAAC;IAClD,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,KAAK,MAAM,GAAG,IAAI,UAAU,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC;QAAC,IAAI,cAAc,EAAE,CAAC;YAAC,eAAe,EAAE,CAAC;YAAC,KAAK,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;YAAC,SAAS;QAAC,CAAC;QACpH,IAAI,SAAS,IAAI,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;YACpC,IAAI,CAAC,SAAS;gBAAE,gBAAgB,EAAE,CAAC;YACnC,SAAS,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;YAC9B,KAAK,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;YAAC,SAAS;QACnC,CAAC;QACD,IAAI,GAAG,CAAC,MAAM,GAAG,MAAM,CAAC,iBAAiB,IAAI,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,cAAc,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC;YAC9F,eAAe,EAAE,CAAC;YAAC,KAAK,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;YAAC,SAAS;QACpD,CAAC;QACD,8EAA8E;QAC9E,MAAM,KAAK,GAAG,GAAG,CAAC,KAAK,CAAC,uBAAuB,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;QAC5D,IAAI,IAAI,GAAG,GAAG,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;QACnC,IAAI,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,wBAAwB,CAAC,IAAI,CAAC,IAAI,mBAAmB,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;YAC9F,MAAM,KAAK,GAAG,UAAU,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;YAAC,MAAM,KAAK,GAAG,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC;YAAC,cAAc,GAAG,KAAK,KAAK,IAAI,IAAI,CAAC,KAAK,KAAK,EAAE,IAAI,aAAa,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,KAAK,CAAC,CAAC,CAAC,KAAK,GAAG,CAAC,IAAI,KAAK,CAAC,WAAW,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;YAAC,gBAAgB,EAAE,CAAC;YAAC,KAAK,CAAC,IAAI,CAAC,KAAK,GAAG,UAAU,CAAC,CAAC;YAAC,SAAS;QACpV,CAAC;QACD,KAAK,MAAM,MAAM,IAAI,YAAY,EAAE,CAAC;YAClC,IAAI,GAAG,IAAI,CAAC,OAAO,CAAC,IAAI,MAAM,CAAC,MAAM,EAAE,GAAG,CAAC,EAAE,GAAG,EAAE,GAAG,gBAAgB,EAAE,CAAC,CAAC,OAAO,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC;QACjG,CAAC;QACD,IAAI,GAAG,IAAI,CAAC,OAAO,CAAC,wBAAwB,EAAE,CAAC,KAAK,EAAE,EAAE;YACtD,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,IAAI,kBAAkB,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;gBACpF,gBAAgB,EAAE,CAAC;gBAAC,OAAO,UAAU,CAAC;YACxC,CAAC;YACD,OAAO,KAAK,CAAC;QACf,CAAC,CAAC,CAAC;QACH,IAAI,GAAG,IAAI,CAAC,OAAO,CAAC,mCAAmC,EACrD,GAAG,EAAE,GAAG,iBAAiB,EAAE,CAAC,CAAC,OAAO,WAAW,CAAC,CAAC,CAAC,CAAC,CAAC;QACtD,IAAI,GAAG,IAAI,CAAC,OAAO,CAAC,sBAAsB,EACxC,CAAC,KAAK,EAAE,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,QAAQ,CAAC,GAAG,CAAC;YAAE,OAAO,KAAK,CAAC,CAAC,iBAAiB,EAAE,CAAC,CAAC,OAAO,WAAW,CAAC,CAAC,CAAC,CAAC,CAAC;QACnG,MAAM,SAAS,GAAG,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QAC3E,IAAI,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,CAAC;YACvD,eAAe,EAAE,CAAC;YAAC,KAAK,CAAC,IAAI,CAAC,KAAK,GAAG,QAAQ,CAAC,CAAC;YAAC,SAAS;QAC5D,CAAC;QACD,KAAK,CAAC,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC,CAAC;IAC3B,CAAC;IACD,MAAM,QAAQ,GAAG,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAClC,MAAM,IAAI,GAAG,OAAO,CAAC,QAAQ,EAAE,MAAM,CAAC,mBAAmB,CAAC,CAAC;IAC3D,OAAO,EAAE,IAAI,EAAE,iBAAiB,EAAE,gBAAgB,EAAE,eAAe,EAAE,SAAS,EAAE,IAAI,CAAC,MAAM,GAAG,QAAQ,CAAC,MAAM,EAAE,CAAC;AAClH,CAAC;AACD;;;;;;;;;GASG;AACH,MAAM,UAAU,oBAAoB,CAAC,KAAa,EAAE,MAAuB;IACzE,QAAQ,CAAC,MAAM,CAAC,CAAC;IACjB,MAAM,IAAI,GAAG,gBAAgB,CAAC,KAAK,EAAE,EAAE,GAAG,MAAM,EAAE,mBAAmB,EAAE,MAAM,CAAC,kBAAkB,EAAE,CAAC,CAAC;IACpG,MAAM,QAAQ,GAAG,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,OAAO,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;IACxD,MAAM,IAAI,GAAG,OAAO,CAAC,QAAQ,EAAE,MAAM,CAAC,mBAAmB,CAAC,CAAC;IAC3D,OAAO,EAAE,GAAG,IAAI,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,CAAC,SAAS,IAAI,IAAI,CAAC,MAAM,GAAG,QAAQ,CAAC,MAAM,EAAE,CAAC;AACvF,CAAC;AACD;;;;;;;;;;;GAWG;AACH,MAAM,UAAU,mBAAmB,CAAC,OAAoC,EAAE,MAAuB;IAC/F,QAAQ,CAAC,MAAM,CAAC,CAAC;IACjB,IAAI,OAAO,CAAC,MAAM,GAAG,IAAI;QAAE,MAAM,IAAI,cAAc,CAAC,iBAAiB,CAAC,CAAC;IACvE,IAAI,UAAU,GAAG,CAAC,CAAC;IACnB,MAAM,QAAQ,GAAG,IAAI,GAAG,EAAoB,CAAC;IAC7C,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;QAC7B,UAAU,IAAI,MAAM,CAAC,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,MAAM,CAAC;QACvD,IAAI,UAAU,GAAG,MAAM,CAAC,kBAAkB;YAAE,MAAM,IAAI,cAAc,CAAC,iBAAiB,CAAC,CAAC;QACxF,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC;QACvC,IAAI,CAAC,CAAC,SAAS,EAAE,MAAM,EAAE,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC;YAAE,SAAS;QACtE,MAAM,MAAM,GAAG,QAAQ,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC;QACxC,MAAM,CAAC,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QAAC,QAAQ,CAAC,GAAG,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC;IACxD,CAAC;IACD,MAAM,KAAK,GAAG,CAAC,IAAY,EAAE,MAAe,EAAE,OAAe,EAAU,EAAE;QACvE,MAAM,MAAM,GAAG,QAAQ,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,EAAE,CAAC;QACxC,MAAM,GAAG,GAAG,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACzD,OAAO,oBAAoB,CAAC,GAAG,EAAE,EAAE,GAAG,MAAM,EAAE,mBAAmB,EAAE,IAAI,CAAC,GAAG,CAAC,OAAO,EAAE,MAAM,CAAC,mBAAmB,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC;IAC3H,CAAC,CAAC;IACF,OAAO;QACL,OAAO,EAAE,KAAK,CAAC,SAAS,EAAE,IAAI,EAAE,GAAG,CAAC,EAAE,IAAI,EAAE,KAAK,CAAC,MAAM,EAAE,IAAI,EAAE,IAAI,CAAC;QACrE,IAAI,EAAE,KAAK,CAAC,MAAM,EAAE,KAAK,EAAE,IAAI,CAAC,EAAE,EAAE,EAAE,KAAK,CAAC,IAAI,EAAE,KAAK,EAAE,IAAI,CAAC,EAAE,EAAE,EAAE,KAAK,CAAC,IAAI,EAAE,KAAK,EAAE,IAAI,CAAC;KAC7F,CAAC;AACJ,CAAC"}

package/dist/mapThread.d.ts

@@ -0,0 +1,66 @@
+import { type SourceCachePermission, type SourceCacheHeaders, type SourceCacheDecision, type SourceSyncReadResult } from '@framers/agentos-ext-account-sources';
+import { type GmailMappingBounds } from './mime.js';
+/** The per-read values of one `map` call. The host supplies every value; none comes from a browser request. */
+export interface GmailMapThreadOptions {
+    /** The requested thread id; the response must carry the same id. */
+    readonly sourceId: string;
+    /**
+     * The host's authenticated source-detail path, used as `SourceRef.url`: a relative path of segments made of letters,
+     * digits, `_` and `-`, with no query, fragment or origin, at most 512 characters. Never a Gmail link.
+     */
+    readonly sourceDetailUrl: string;
+    /** The selection cutoff, epoch milliseconds, no later than `receivedAt`: a message received after it is left out. */
+    readonly cutoff: number;
+    /**
+     * The message window in milliseconds, 1 to 90 days. A message received at or before `cutoff - windowMs`, or whose
+     * window has ended by `receivedAt`, is left out.
+     */
+    readonly windowMs: number;
+    /** When the thread request started, epoch milliseconds. */
+    readonly readStartedAt: number;
+    /** When the thread response was complete, epoch milliseconds; it becomes `SourceRef.readAt`. */
+    readonly receivedAt: number;
+    /** The host's evidenced cache permission; null refuses durable content. */
+    readonly permission: SourceCachePermission | null;
+    /** The thread response's own cache headers. */
+    readonly cacheHeaders: SourceCacheHeaders;
+    /** Label names from the cycle's labels response, by label id. */
+    readonly labelNames: Readonly<Record<string, string>>;
+    /** The labels response's own cache decision. Null, a refusal or an expired decision keeps ids without names. */
+    readonly labelCache: SourceCacheDecision | null;
+    /** Cancels the call between steps. */
+    readonly signal?: AbortSignal;
+}
+/**
+ * Maps one parsed Gmail thread into a sanitized account-source document: one thread, one document, with each retained
+ * message's id, dates, sanitized headers, labels, attachment listing and body span. It calls no endpoint, writes no log
+ * and keeps no state between calls.
+ */
+export declare class GmailThreadMapper {
+    private readonly bounds;
+    /**
+     * @param bounds Every operational ceiling; see `GmailMappingBounds`.
+     * @throws {GmailMappingFault} `configuration` for an invalid bound.
+     */
+    constructor(bounds: GmailMappingBounds);
+    private policy;
+    private normalize;
+    /**
+     * Validates the whole response, filters each message by its internal date and by the spam, trash and draft labels,
+     * keeps the newest eligible messages up to the cap with ties broken by id, spends the thread's character budget
+     * newest first and renders the kept messages oldest first. A message's text is its sanitized header prefix followed
+     * by its sanitized, trimmed body, and its span covers the body alone. A message that cannot be read is counted under
+     * `metadata.omitted` by a fixed reason. The cache expiry is the earliest of the permission's retention, the read start
+     * plus `maximumCacheMs`, the response's cache headers, every kept message's window end and the label names' expiry.
+     *
+     * @param input The parsed `threads.get` response in the `full` format, from a transport that capped its bytes.
+     * @param options The per-read values.
+     * @returns A document, or a skip with a fixed reason: a thread-wide bound was exceeded, no message survived, or the
+     * cache policy refused durable content.
+     * @throws {GmailMappingFault} `configuration` for invalid options.
+     * @throws {SourceSyncFault} `invalid_response` for an invalid response, never an empty success; `cancelled` once the
+     * signal is aborted.
+     */
+    map(input: unknown, options: GmailMapThreadOptions): Promise<SourceSyncReadResult>;
+}
+//# sourceMappingURL=mapThread.d.ts.map

package/dist/mapThread.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"mapThread.d.ts","sourceRoot":"","sources":["../src/mapThread.ts"],"names":[],"mappings":"AAQA,OAAO,EAEL,KAAK,qBAAqB,EAAE,KAAK,kBAAkB,EAAE,KAAK,mBAAmB,EACjD,KAAK,oBAAoB,EACtD,MAAM,sCAAsC,CAAC;AAC9C,OAAO,EAGL,KAAK,kBAAkB,EACxB,MAAM,WAAW,CAAC;AAOnB,+GAA+G;AAC/G,MAAM,WAAW,qBAAqB;IACpC,oEAAoE;IACpE,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B;;;OAGG;IACH,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAC;IACjC,qHAAqH;IACrH,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB;;;OAGG;IACH,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,2DAA2D;IAC3D,QAAQ,CAAC,aAAa,EAAE,MAAM,CAAC;IAC/B,gGAAgG;IAChG,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;IAC5B,2EAA2E;IAC3E,QAAQ,CAAC,UAAU,EAAE,qBAAqB,GAAG,IAAI,CAAC;IAClD,+CAA+C;IAC/C,QAAQ,CAAC,YAAY,EAAE,kBAAkB,CAAC;IAC1C,iEAAiE;IACjE,QAAQ,CAAC,UAAU,EAAE,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC,CAAC;IACtD,gHAAgH;IAChH,QAAQ,CAAC,UAAU,EAAE,mBAAmB,GAAG,IAAI,CAAC;IAChD,sCAAsC;IACtC,QAAQ,CAAC,MAAM,CAAC,EAAE,WAAW,CAAC;CAC/B;AAyCD;;;;GAIG;AACH,qBAAa,iBAAiB;IAC5B,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAqB;IAC5C;;;OAGG;gBACS,MAAM,EAAE,kBAAkB;IAEtC,OAAO,CAAC,MAAM;YAKA,SAAS;IAyBvB;;;;;;;;;;;;;;;OAeG;IACG,GAAG,CAAC,KAAK,EAAE,OAAO,EAAE,OAAO,EAAE,qBAAqB,GAAG,OAAO,CAAC,oBAAoB,CAAC;CAmFzF"}