@bobfrankston/rmfmail 1.1.225 → 1.1.228

package/docs/outlook.md

@@ -0,0 +1,215 @@
+# Outlook.com / Microsoft 365 Support
+
+Status (2026-06-05): **scaffolding present, not usable.** An account auto-detects
+the right hosts but cannot authenticate — there is no Microsoft OAuth wired, and
+Microsoft has disabled basic-auth (password) for outlook.com / office365, so the
+IMAP path can't fall back to a password either. Gmail is the only fully-wired
+OAuth provider today.
+
+This doc scopes what it takes to make Outlook a first-class account.
+
+---
+
+## What already exists
+
+| Piece | Where | State |
+|---|---|---|
+| Graph API provider | `@bobfrankston/mailx-sync/outlook.ts` (re-exported `packages/mailx-imap/providers/outlook-api.ts`) | **Read-only**: `listFolders`, `fetchSince/ByDate/ByUids/One`, `getUids`. No send, no write-back, no drafts/attachments. |
+| Host auto-config | `packages/mailx-settings/index.ts:503` (`outlook.com`, `hotmail.com` → `outlook.office365.com:993` / `smtp.office365.com:587`, `auth: "oauth2"`) | Done |
+| MX-based detection | `bin/mailx.ts:1307` (`*.outlook.com` / `*.protection.outlook.com` MX → Microsoft 365) | Done |
+| Generic OAuth flow | `@bobfrankston/oauthsupport` `OAuthTokenManager.ts:369` — validates `client_id/client_secret/auth_uri/token_uri` from the creds file; localhost loopback auth-code flow | **Provider-agnostic** — not Google-specific |
+| OAuth SMTP send | `packages/mailx-imap/index.ts:1533` builds `{type:"OAuth2", user, accessToken}` from `config.tokenProvider()` | Works for any provider with a tokenProvider |
+
+The single thing that makes Gmail work and Outlook not: the **tokenProvider** at
+`packages/mailx-imap/index.ts:798-848` is built only for Google. It hardcodes the
+Google credentials file (`~/.mailx/google-credentials.json`) and the all-Google
+scope string. Nothing constructs a Microsoft tokenProvider, so
+`OutlookApiProvider` is never instantiated and the office365 IMAP host has no
+token to present.
+
+---
+
+## Decision: Graph API vs IMAP/XOAUTH2
+
+Both need the same Azure app registration + Microsoft OAuth. The difference is
+the sync transport once you have a token.
+
+**Recommended: Graph API.** It mirrors the Gmail-API model the codebase already
+follows (`isGmailAccount()` branch → REST provider, no IMAP client). Wins: no
+connection-limit/timeout class of bugs (the entire `inactivityTimeout` /
+`greetingTimeout` / chunk-size tuning at `index.ts:862` is IMAP-only pain);
+provider is already half-written; same `MailProvider` interface so Android reuses
+it verbatim. Cost: the provider is read-only — send + flag/move/delete + drafts
+must be added (Graph endpoints, ~a day). Sharp edge: `idToUid()`
+(`outlook.ts` ~135) hashes Graph's long string IDs to 48-bit ints, and
+`fetchByUids`/`fetchOne` re-list the **whole** folder then filter — O(folder) per
+fetch with collision risk. Real sync needs a UID↔providerId map (we already keep
+`provider_id` in the DB) and direct `GET /messages/{id}` instead of list-and-scan.
+
+**Alternative: IMAP + XOAUTH2.** Reuses the existing iflow-direct sync path
+unchanged — just feed it a Microsoft token instead of a Google one. Wins: zero
+new sync/send code; flag/move/delete/append already work over IMAP. Cost:
+inherits every IMAP fragility we've spent months hardening for Dovecot/Gmail, now
+against Exchange Online's quirks. Sharp edge: Microsoft is steering third-party
+clients toward Graph and away from consumer IMAP; betting the integration on
+office365 IMAP is the less future-proof path.
+
+Pragmatic call: **IMAP/XOAUTH2 first** as the fast path to a working account (it's
+mostly OAuth plumbing), then migrate sync to Graph for parity with Gmail. The
+OAuth work below is shared, so it's not wasted either way.
+
+---
+
+## Work breakdown
+
+### 1. Azure app registration  [S, one-time, external]
+Register an app in Entra/Azure AD:
+- Account type: "personal Microsoft accounts" (consumer outlook.com) and/or
+  "any org directory" (Microsoft 365). For both, use the `/common` authority.
+- Redirect URI: `http://localhost` (loopback) — matches oauthsupport's flow
+  (`OAuthTokenManager.ts:463`, parses the loopback callback).
+- API permissions (delegated):
+    - Graph path: `Mail.ReadWrite`, `Mail.Send`, `offline_access`, `openid`,
+      `profile`.
+    - IMAP path: `https://outlook.office.com/IMAP.AccessAsUser.All`,
+      `https://outlook.office.com/SMTP.Send`, `offline_access`.
+- Produce a `microsoft-credentials.json` shaped like the Google one, with
+  `auth_uri: https://login.microsoftonline.com/common/oauth2/v2.0/authorize`,
+  `token_uri: https://login.microsoftonline.com/common/oauth2/v2.0/token`,
+  `redirect_uris: ["http://localhost"]`, plus `client_id` / `client_secret`.
+
+Store at `~/.mailx/microsoft-credentials.json` (parallel to
+`google-credentials.json`).
+
+### 2. Microsoft tokenProvider  [M]
+Generalize `index.ts:798-848` so the provider isn't Google-only:
+- Pick the creds file + scope by account kind (gmail vs outlook), keyed off the
+  same signal `account.imap.auth === "oauth2"` plus host/email domain.
+- For Outlook, point `authenticateOAuth` at `microsoft-credentials.json` with the
+  Microsoft scope set. The flow itself is unchanged — oauthsupport already reads
+  the endpoints from the creds file, so no oauthsupport changes are required
+  (confirm consumer-account PKCE: Microsoft may require `code_challenge` for the
+  loopback client; if so, add PKCE support in `OAuthTokenManager.getToken`).
+- Token cache dir reuses the existing `tokens/<user>/` convention (`index.ts:814`).
+
+This single change makes the **IMAP/XOAUTH2 path work end-to-end** — the office365
+host config already exists, and SMTP OAuth (`index.ts:1533`) consumes the same
+tokenProvider. Verify the token is minted with Outlook IMAP/SMTP scopes, not Graph
+scopes (they're different audiences).
+
+### 3. Provider selection  [S]
+Add an `isOutlookAccount()` sibling to `isGmailAccount()` (`index.ts:898`) and an
+`getOutlookProvider()` mirroring `getGmailProvider()` (`index.ts:906`) that does
+`new OutlookApiProvider(config.tokenProvider)`. Then extend each
+`isGmailAccount()` branch (sync, periodic STATUS, IDLE-skip, etc. — ~10 sites) to
+a three-way provider switch. *(Only needed for the Graph path; the IMAP path
+needs none of this — it flows through the normal IMAP code.)*
+
+### 4. Graph write-back + send  [M] *(Graph path only)*
+Add to `OutlookApiProvider`:
+- `sendMail`: `POST /sendMail` with the MIME or the Graph message object.
+- mark read/flag: `PATCH /messages/{id}` (`isRead`, `flag`).
+- move: `POST /messages/{id}/move`.
+- delete: `DELETE /messages/{id}` (or move to deleteditems).
+- draft create/update: `POST /messages` / `PATCH`.
+- attachments: `GET /messages/{id}/attachments`.
+Replace list-and-scan in `fetchByUids`/`fetchOne` with direct
+`GET /messages/{providerId}` using the DB's `provider_id` mapping.
+
+### 5. Setup UX  [S]
+`bin/mailx.ts` `knownOAuth` list (~line 1320) already includes `outlook.com` /
+`hotmail.com`. Once the tokenProvider exists, first-run setup triggers the
+Microsoft consent screen the same way Gmail does — no extra UI. Add an error
+banner case for "Microsoft credentials missing" pointing at
+`microsoft-credentials.json`.
+
+---
+
+## Step-by-step (with URLs)
+
+### A. Register the Azure app — the part only you can do
+1. Sign in to the Entra admin center → **App registrations**:
+   https://entra.microsoft.com/#view/Microsoft_AAD_RegisteredApps/ApplicationsListBlade
+   (equivalently Azure portal → "Microsoft Entra ID" → "App registrations":
+   https://portal.azure.com)
+2. **New registration** (guide:
+   https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app):
+   - Name: `rmfmail`.
+   - Supported account types: **"Accounts in any organizational directory and
+     personal Microsoft accounts"** (covers both outlook.com consumer and M365).
+   - Redirect URI: platform **"Mobile and desktop applications"**, value
+     **`http://localhost`** (loopback — what oauthsupport's flow listens on).
+     Desktop/loopback registration:
+     https://learn.microsoft.com/en-us/entra/identity-platform/scenario-desktop-app-registration
+3. Copy the **Application (client) ID** from the app's Overview page.
+4. **Certificates & secrets** → **New client secret** → copy the secret *value*
+   (not the ID). (Public-client/PKCE is the alternative if you'd rather not ship a
+   secret — see Sharp edges.)
+5. **API permissions** → **Add a permission** → **Microsoft Graph** → **Delegated**
+   (permissions reference:
+   https://learn.microsoft.com/en-us/graph/permissions-reference):
+   - Graph path: `Mail.ReadWrite`, `Mail.Send`, `offline_access`, `openid`,
+     `profile`.
+   - IMAP path instead/also: `IMAP.AccessAsUser.All`, `SMTP.Send`,
+     `offline_access` (these live under the *Office 365 Exchange Online* API, not
+     Graph). Microsoft's IMAP/SMTP-OAuth guide:
+     https://learn.microsoft.com/en-us/exchange/client-developer/legacy-protocols/how-to-authenticate-an-imap-pop-smtp-application-by-using-oauth
+   - Click **Grant admin consent** if it's a work tenant (personal accounts
+     consent at first sign-in).
+
+Reference for the endpoints used below:
+- Authorize: `https://login.microsoftonline.com/common/oauth2/v2.0/authorize`
+- Token: `https://login.microsoftonline.com/common/oauth2/v2.0/token`
+- Auth-code flow spec:
+  https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-auth-code-flow
+- Graph mail API overview:
+  https://learn.microsoft.com/en-us/graph/api/resources/mail-api-overview
+
+### B. Drop in the credentials file
+Create `~/.mailx/microsoft-credentials.json` mirroring the Google one's shape so
+oauthsupport's `OAuthTokenManager` (reads `client_id`/`client_secret`/`auth_uri`/
+`token_uri`, `OAuthTokenManager.ts:369`) consumes it unchanged:
+
+    {
+        "installed": {
+            "client_id": "<Application (client) ID>",
+            "client_secret": "<secret value>",
+            "auth_uri": "https://login.microsoftonline.com/common/oauth2/v2.0/authorize",
+            "token_uri": "https://login.microsoftonline.com/common/oauth2/v2.0/token",
+            "redirect_uris": ["http://localhost"]
+        }
+    }
+
+### C. Wire the code (steps 2-3 from the work breakdown)
+1. In `packages/mailx-imap/index.ts` (~798), branch the tokenProvider on account
+   kind: Outlook accounts use `microsoft-credentials.json` + the Microsoft scope
+   string (IMAP scopes for the IMAP path, Graph scopes for the Graph path).
+2. For the **IMAP path** that's the whole change — the office365 host config
+   (`mailx-settings/index.ts:503`) and OAuth-SMTP (`index.ts:1533`) already
+   consume the tokenProvider. Test sign-in: first send triggers the loopback
+   consent at `http://localhost`.
+3. For the **Graph path**, add `isOutlookAccount()`/`getOutlookProvider()` next to
+   the Gmail equivalents (`index.ts:898/906`) and fill the provider's write-back
+   gaps (`sendMail`, flag/move/delete, drafts).
+
+## Sharp edges / open questions
+
+- **Consumer vs M365 differ.** Personal outlook.com and Microsoft 365 work
+  accounts use different scope audiences and consent behavior. `/common` covers
+  both at the authority level, but test both — a token good for Graph isn't valid
+  for IMAP and vice-versa.
+- **PKCE.** Microsoft increasingly requires PKCE for loopback clients. If consent
+  fails with `invalid_request`, add `code_challenge`/`code_verifier` to
+  oauthsupport's auth-code flow (it's the one oauthsupport change that might be
+  needed).
+- **Basic auth is gone.** Don't add a password fallback for office365 — it will
+  always fail. The error path should say "Outlook requires sign-in", not "wrong
+  password".
+- **Drive/OneDrive is unrelated.** `cloud.ts` already sketches MS Graph
+  `Files.ReadWrite` scopes for OneDrive storage — that's a separate app/scope from
+  mail and shares none of this wiring (and OneDrive cloud storage was removed;
+  GDrive only).
+- **`idToUid` collisions.** A 32-bit hash of Graph IDs across a 10-year mailbox
+  will eventually collide. The Graph path must key on `provider_id`
+  (string) as identity, with the integer UID as a display/sort convenience only —
+  same lesson as `imap_uid_not_identity`.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@bobfrankston/rmfmail",
-  "version": "1.1.225",
+  "version": "1.1.228",
   "description": "Local-first email client with IMAP sync and standalone native app",
   "type": "module",
   "main": "bin/mailx.js",
@@ -45,7 +45,7 @@
     "@bobfrankston/msger": "^0.1.384",
     "@bobfrankston/node-tcp-transport": "^0.1.8",
     "@bobfrankston/oauthsupport": "^1.0.31",
-    "@bobfrankston/rmf-tiny": "^0.1.28",
+    "@bobfrankston/rmf-tiny": "^0.1.29",
     "@bobfrankston/smtp-direct": "^0.1.8",
     "@bobfrankston/tcp-transport": "^0.1.6",
     "@capacitor/android": "^8.3.0",
@@ -125,7 +125,7 @@
       "@bobfrankston/msger": "^0.1.384",
       "@bobfrankston/node-tcp-transport": "^0.1.8",
       "@bobfrankston/oauthsupport": "^1.0.31",
-      "@bobfrankston/rmf-tiny": "^0.1.28",
+      "@bobfrankston/rmf-tiny": "^0.1.29",
       "@bobfrankston/smtp-direct": "^0.1.8",
       "@bobfrankston/tcp-transport": "^0.1.6",
       "@capacitor/android": "^8.3.0",