protonmail-bridge-agent-skill 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Luke Ramsden
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/NOTICE ADDED
@@ -0,0 +1,4 @@
1
+ This project is an independent, unofficial tool and is not affiliated with,
2
+ endorsed by, or sponsored by Proton AG. "Proton" and "Proton Mail" are
3
+ trademarks of Proton AG. Proton Mail Bridge is required and is available
4
+ from Proton at https://proton.me/mail/bridge.
package/README.md ADDED
@@ -0,0 +1,89 @@
1
+ # Proton Mail Bridge Agent Skill
2
+
3
+ Give coding agents read-only access to a Proton Mail inbox — with instant full-text search across the entire archive — through the local [Proton Mail Bridge](https://proton.me/mail/bridge) IMAP server.
4
+
5
+ The skill ships `pmail`, a zero-dependency Python CLI (stdlib only) that talks IMAP to Bridge on `127.0.0.1:1143`, maintains a local SQLite+FTS5 cache of the whole mailbox, and serves bm25-ranked keyword search with snippets in well under a second (measured against an ~80k-message archive where Bridge's own server-side search takes ~15s and returns unranked UIDs).
6
+
7
+ ## Requirements
8
+
9
+ - macOS (credentials are stored in the system Keychain)
10
+ - [Proton Mail Bridge](https://proton.me/mail/bridge) installed, signed in, and running
11
+ - Python 3.9+ (no packages to install — stdlib only)
12
+
13
+ ## Install
14
+
15
+ Install with the Skills CLI:
16
+
17
+ ```bash
18
+ npx skills add lukeramsden/protonmail-bridge-agent-skill@protonmail-bridge
19
+ ```
20
+
21
+ Install globally for all supported agents:
22
+
23
+ ```bash
24
+ npx skills add lukeramsden/protonmail-bridge-agent-skill@protonmail-bridge -g -a '*' -y
25
+ ```
26
+
27
+ List the skills available in this repository without installing:
28
+
29
+ ```bash
30
+ npx skills add lukeramsden/protonmail-bridge-agent-skill --list
31
+ ```
32
+
33
+ ## Quick start
34
+
35
+ ```bash
36
+ ./doctor # verify all runtime dependencies
37
+ ./pmail setup # account email + Bridge password -> Keychain
38
+ ./pmail sync --all --full # backfill the whole archive (resumable)
39
+ ./pmail search "invoice" # ranked full-text search, sub-second
40
+ ./pmail list INBOX --unseen
41
+ ./pmail read 34655
42
+ ./pmail save-attach 34655 1
43
+ ```
44
+
45
+ All commands print JSON on stdout; progress and errors go to stderr.
46
+
47
+ ## What the skill provides
48
+
49
+ - **Full-archive FTS search** — SQLite FTS5 with bm25 ranking and match snippets; phrases, `OR`/`NOT`, and filters (`--from`, `--since`, `--unseen`, `--mailbox`)
50
+ - **Resumable backfill** — cursor-persisted batches; interrupt and resume freely; automatic wipe/resync if Bridge's UIDVALIDITY changes
51
+ - **Cache-first reads** — `list`, `read`, and `search` serve from the local cache with an implicit incremental sync at most once per minute; `--live` and `--no-sync` escape hatches
52
+ - **Attachment download** — `save-attach` extracts parts to the cache's attachments directory
53
+ - **Self-diagnostics** — `doctor` checks python/sqlite, credentials, Bridge reachability + login, cache health, and disk space, printing an actionable fix per failure
54
+ - **Read-only by construction** — only `BODY.PEEK` and `EXAMINE` on the wire; no STORE/COPY/EXPUNGE/APPEND anywhere, so reading mail never sets `\Seen`
55
+
56
+ ## Privacy and security
57
+
58
+ - Everything is local: pmail only ever connects to the Bridge listener on your own machine
59
+ - The Bridge password lives in macOS Keychain (service `pmail-bridge`); the only on-disk config is your account email in `~/.local/share/pmail/config.json` (mode 600)
60
+ - The cache (`~/.local/share/pmail/`) is fully rebuildable — delete it anytime and re-run `pmail sync --all --full`
61
+
62
+ ## Repository layout
63
+
64
+ ```text
65
+ skills/
66
+ └── protonmail-bridge/
67
+ ├── SKILL.md # agent-facing instructions and command reference
68
+ ├── pmail # the CLI (Python stdlib only, executable)
69
+ └── doctor # runtime dependency checker
70
+ ```
71
+
72
+ ## License
73
+
74
+ This repository is available under the MIT License. See [NOTICE](NOTICE) for trademark attribution.
75
+
76
+ ## Install as a pi package
77
+
78
+ ```bash
79
+ pi install npm:protonmail-bridge-agent-skill
80
+ ```
81
+
82
+ ## Releasing
83
+
84
+ ```bash
85
+ npm version patch|minor|major && git push --follow-tags
86
+ ```
87
+
88
+ The `v*` tag triggers `.github/workflows/publish.yml`, which publishes to npm
89
+ with OIDC trusted publishing and provenance.
package/package.json ADDED
@@ -0,0 +1,15 @@
1
+ {
2
+ "name": "protonmail-bridge-agent-skill",
3
+ "version": "1.0.0",
4
+ "description": "Agent skill: read-only Proton Mail access via Proton Mail Bridge IMAP with a local SQLite FTS cache (pmail CLI, Python stdlib only).",
5
+ "license": "MIT",
6
+ "author": "lukeramsden",
7
+ "repository": { "type": "git", "url": "git+https://github.com/lukeramsden/protonmail-bridge-agent-skill.git" },
8
+ "homepage": "https://github.com/lukeramsden/protonmail-bridge-agent-skill#readme",
9
+ "bugs": "https://github.com/lukeramsden/protonmail-bridge-agent-skill/issues",
10
+ "keywords": [ "pi-package", "pi-skill", "pi-coding-agent", "agent-skill", "protonmail-bridge" ],
11
+ "pi": { "skills": [ "./skills" ] },
12
+ "bin": { "pmail": "skills/protonmail-bridge/pmail" },
13
+ "files": [ "skills", "README.md", "LICENSE", "NOTICE" ],
14
+ "scripts": { "verify": "python3 -m py_compile skills/*/pmail skills/*/doctor" }
15
+ }
@@ -0,0 +1,99 @@
1
+ ---
2
+ name: protonmail-bridge
3
+ description: Read, list, and full-text search email from a Proton Mail account via the local Proton Mail Bridge IMAP server and a fast local SQLite cache. Use for checking the inbox, reading messages, searching the entire mailbox by keyword/sender, or downloading attachments. Read-only; cannot send or modify mail. Requires the Proton Mail Bridge app running locally.
4
+ ---
5
+
6
+ # Proton Mail Bridge (`pmail`)
7
+
8
+ Read-only email access through Proton Mail Bridge (IMAP on 127.0.0.1:1143) with a
9
+ local SQLite+FTS5 cache for instant full-text search across the whole mailbox.
10
+
11
+ The CLI is `pmail` in this skill's directory. Run commands from the skill
12
+ directory as `./pmail <command> ...` (or invoke it by its absolute path).
13
+
14
+ All data commands print **JSON on stdout**; progress and errors go to stderr.
15
+ Pipe through `jq` when you need to reshape output.
16
+
17
+ **If pmail fails or behaves unexpectedly, run the doctor script first:**
18
+
19
+ ```bash
20
+ ./doctor
21
+ ```
22
+
23
+ It checks every runtime dependency (python/sqlite, account configuration,
24
+ Keychain credentials, Bridge reachability + login, cache database, disk space)
25
+ and prints an actionable fix for each failure. If it reports Bridge
26
+ unreachable, stop and ask the user to start the Proton Mail Bridge app —
27
+ do not retry in a loop.
28
+
29
+ ## First-time setup
30
+
31
+ Run `./doctor` — it reports anything missing. On a fresh machine it will ask
32
+ you to run `./pmail setup`, which prompts for the Proton account email and the
33
+ Bridge-generated password (Bridge app > Mailbox details), verifies them against
34
+ Bridge, stores the password in macOS Keychain, and writes the account email to
35
+ `~/.local/share/pmail/config.json`.
36
+
37
+ ## Commands
38
+
39
+ ```bash
40
+ pmail mailboxes # folders + labels, message counts, cached counts
41
+ pmail list [mailbox] [--limit N] [--unseen] [--live]
42
+ pmail read <uid> [--mailbox M] [--max-chars N] [--raw]
43
+ pmail search <words...> [--mailbox M] [--from X] [--since YYYY-MM-DD] [--unseen] [--limit N] [--live]
44
+ pmail sync [--mailbox M | --all] [--full] [--skip-all-mail]
45
+ pmail status # cache state per mailbox, db size
46
+ pmail save-attach <uid> <index> [--mailbox M] # index comes from `read` output
47
+ ```
48
+
49
+ ## Behavior notes
50
+
51
+ - **Mailboxes**: `INBOX`, `Archive`, `Sent`, `Drafts`, `Trash`, `Spam`, `Starred`,
52
+ `All Mail` (everything), plus user labels under `Labels/<name>` and folders
53
+ under `Folders/<name>`. `search` defaults to `All Mail`; `list`/`read` default
54
+ to `INBOX`.
55
+ - **Cache-first**: `list`, `read`, and `search` run an implicit incremental sync
56
+ (at most once per minute) and then serve from the local cache. Use `--live` to
57
+ force a server query or `--no-sync` to skip the implicit sync.
58
+ - **Backfill**: `pmail sync --mailbox "All Mail" --full` caches the entire mailbox
59
+ (resumable if interrupted; roughly 300 msg/s through Bridge). Until it has run,
60
+ `search` only covers what has been synced so far; use `--live` for an
61
+ uncached server-side search.
62
+ - **Search syntax**: cached search is SQLite FTS5 — plain words are AND-ed,
63
+ `"quoted phrases"` and `OR`/`NOT` work. Results are bm25-ranked with snippets
64
+ (sub-second over an ~80k-message archive). FTS matches whole tokens;
65
+ `--live` IMAP search matches substrings and also scans headers, so hit counts
66
+ differ slightly between the two.
67
+ - **Bodies**: most real-world mail is HTML-only; pmail indexes the stripped text
68
+ of the HTML part so search covers those messages too.
69
+ - **Read-only guarantee**: bodies are fetched with BODY.PEEK and mailboxes are
70
+ opened read-only, so reading mail here never marks it `\Seen`.
71
+ - **UIDs are per-mailbox**: a UID from a `search` on `All Mail` must be read
72
+ back with `--mailbox "All Mail"`.
73
+
74
+ ## Credentials
75
+
76
+ The Bridge password is read from macOS Keychain (service `pmail-bridge`) or
77
+ `$PMAIL_PASSWORD`; the account email comes from `$PMAIL_ACCOUNT` or the config
78
+ written by `pmail setup`. To store or rotate credentials:
79
+
80
+ ```bash
81
+ pmail setup # prompts for account email + Bridge password, verifies login
82
+ ```
83
+
84
+ Bridge rotates its local password when it re-authenticates — if logins start
85
+ failing, get the new password from Bridge > Mailbox details and re-run setup.
86
+
87
+ ## Data and configuration
88
+
89
+ - Cache: `~/.local/share/pmail/mail.db` (SQLite, WAL mode) +
90
+ `~/.local/share/pmail/attachments/`. Fully rebuildable from Bridge — safe to
91
+ delete; the next `pmail sync --all --full` repopulates it. A UIDVALIDITY
92
+ change on the server triggers an automatic per-mailbox wipe and resync.
93
+ - The full-archive cache can grow to several GB (raw bodies incl. HTML are
94
+ stored).
95
+ - Env overrides: `PMAIL_HOST` (127.0.0.1), `PMAIL_PORT` (1143),
96
+ `PMAIL_ACCOUNT`, `PMAIL_PASSWORD`, `PMAIL_DATA` (cache directory).
97
+ - The first command after a minute of quiet runs a fast incremental sync
98
+ (~1-2s); subsequent commands within the minute are instant. `--no-sync`
99
+ skips this; `--live` bypasses the cache entirely.
@@ -0,0 +1,178 @@
1
+ #!/usr/bin/env python3
2
+ """doctor - diagnose pmail's runtime dependencies.
3
+
4
+ Checks every layer the pmail CLI needs, in order, and prints PASS/FAIL with
5
+ an actionable fix for each failure. Exit code 0 if all hard checks pass,
6
+ 1 otherwise. Safe to run anytime: it changes nothing (except creating the
7
+ data directory if missing).
8
+
9
+ Run: ./doctor from the skill directory.
10
+ """
11
+
12
+ import os
13
+ import shutil
14
+ import socket
15
+ import sqlite3
16
+ import ssl
17
+ import subprocess
18
+ import sys
19
+ import tempfile
20
+
21
+ IMAP_HOST = os.environ.get("PMAIL_HOST", "127.0.0.1")
22
+ IMAP_PORT = int(os.environ.get("PMAIL_PORT", "1143"))
23
+ KEYCHAIN_SERVICE = "pmail-bridge"
24
+ DATA_DIR = os.path.expanduser(os.environ.get("PMAIL_DATA", "~/.local/share/pmail"))
25
+ DB_PATH = os.path.join(DATA_DIR, "mail.db")
26
+ CONFIG_PATH = os.path.join(DATA_DIR, "config.json")
27
+
28
+ def configured_account():
29
+ """$PMAIL_ACCOUNT, else the account written by `pmail setup`. May be None."""
30
+ a = os.environ.get("PMAIL_ACCOUNT")
31
+ if a:
32
+ return a
33
+ try:
34
+ import json
35
+ with open(CONFIG_PATH) as f:
36
+ return json.load(f).get("account")
37
+ except (OSError, ValueError):
38
+ return None
39
+
40
+ ACCOUNT = configured_account()
41
+
42
+ failures = 0
43
+
44
+ def check(ok, label, fix=None, info=None):
45
+ global failures
46
+ mark = "PASS" if ok else "FAIL"
47
+ line = f"[{mark}] {label}"
48
+ if ok and info:
49
+ line += f" ({info})"
50
+ print(line)
51
+ if not ok:
52
+ failures += 1
53
+ if fix:
54
+ print(f" fix: {fix}")
55
+ return ok
56
+
57
+ def note(text):
58
+ print(f" {text}")
59
+
60
+ # 1. python version (pmail needs 3.9+ for imaplib's timeout parameter)
61
+ v = sys.version_info
62
+ check(v >= (3, 9), f"python3 version", info=".".join(map(str, v[:3])),
63
+ fix="install Python 3.9+ (e.g. `brew install python3`)")
64
+
65
+ # 2. sqlite3 with FTS5 (the search index needs it)
66
+ try:
67
+ db = sqlite3.connect(":memory:")
68
+ db.execute("CREATE VIRTUAL TABLE t USING fts5(x)")
69
+ check(True, "sqlite3 FTS5 extension", info=sqlite3.sqlite_version)
70
+ except Exception as e:
71
+ check(False, "sqlite3 FTS5 extension", fix=f"python3 sqlite3 lacks FTS5: {e}")
72
+
73
+ # 3. macOS `security` CLI (Keychain access)
74
+ check(shutil.which("security") is not None, "`security` CLI on PATH",
75
+ fix="expected on macOS; if missing, set PMAIL_PASSWORD instead")
76
+
77
+ # 4. account configured, then credentials available; never print the secret
78
+ acct_ok = check(ACCOUNT is not None, "account configured",
79
+ info="$PMAIL_ACCOUNT" if os.environ.get("PMAIL_ACCOUNT") else "pmail setup config",
80
+ fix="run `pmail setup` (stores only the email address in "
81
+ f"{CONFIG_PATH}), or set PMAIL_ACCOUNT")
82
+ pw = os.environ.get("PMAIL_PASSWORD")
83
+ cred_source = "PMAIL_PASSWORD env var" if pw else None
84
+ if not pw and acct_ok:
85
+ r = subprocess.run(
86
+ ["security", "find-generic-password", "-s", KEYCHAIN_SERVICE, "-a", ACCOUNT, "-w"],
87
+ capture_output=True, text=True)
88
+ if r.returncode == 0 and r.stdout.strip():
89
+ pw = r.stdout.strip()
90
+ cred_source = f"Keychain (service '{KEYCHAIN_SERVICE}')"
91
+ check(pw is not None, "Bridge password available", info=cred_source,
92
+ fix="run `pmail setup` to store the Bridge password in Keychain "
93
+ "(Bridge app > Mailbox details), or set PMAIL_PASSWORD")
94
+
95
+ # 5. Bridge TCP port reachable
96
+ try:
97
+ with socket.create_connection((IMAP_HOST, IMAP_PORT), timeout=5):
98
+ check(True, f"Bridge listening on {IMAP_HOST}:{IMAP_PORT}")
99
+ bridge_up = True
100
+ except OSError as e:
101
+ check(False, f"Bridge listening on {IMAP_HOST}:{IMAP_PORT}",
102
+ fix=f"start the Proton Mail Bridge app ({e})")
103
+ bridge_up = False
104
+
105
+ # 6. full IMAP handshake: STARTTLS + LOGIN + LIST
106
+ if bridge_up and pw:
107
+ try:
108
+ import imaplib
109
+ ctx = ssl.create_default_context()
110
+ ctx.check_hostname = False
111
+ ctx.verify_mode = ssl.CERT_NONE
112
+ m = imaplib.IMAP4(IMAP_HOST, IMAP_PORT, timeout=15)
113
+ banner = m.welcome.decode(errors="replace")
114
+ m.starttls(ctx)
115
+ m.login(ACCOUNT or "", pw)
116
+ typ, boxes = m.list()
117
+ n = len(boxes or [])
118
+ m.logout()
119
+ check(True, "IMAP STARTTLS + login + LIST",
120
+ info=f"{n} mailboxes; {banner.split(' - ')[0].lstrip('* OK ')}")
121
+ except imaplib.IMAP4.error as e:
122
+ check(False, "IMAP STARTTLS + login + LIST",
123
+ fix=f"Bridge rejected login: {e}. Passwords rotate when Bridge "
124
+ f"re-auths — get the current one from Bridge > Mailbox details "
125
+ f"and run `pmail setup`")
126
+ except (OSError, ssl.SSLError) as e:
127
+ check(False, "IMAP STARTTLS + login + LIST", fix=f"handshake failed: {e}")
128
+ elif not bridge_up:
129
+ note("skipping IMAP handshake (Bridge unreachable)")
130
+ else:
131
+ note("skipping IMAP handshake (no credentials)")
132
+
133
+ # 7. data directory present and writable
134
+ try:
135
+ os.makedirs(DATA_DIR, exist_ok=True)
136
+ with tempfile.TemporaryFile(dir=DATA_DIR):
137
+ pass
138
+ check(True, "data directory writable", info=DATA_DIR)
139
+ except OSError as e:
140
+ check(False, "data directory writable",
141
+ fix=f"check permissions on {DATA_DIR} ({e})")
142
+
143
+ # 8. cache database opens, has schema, and is consistent (informational)
144
+ if os.path.exists(DB_PATH):
145
+ try:
146
+ db = sqlite3.connect(f"file:{DB_PATH}?mode=ro", uri=True, timeout=10)
147
+ counts = dict(db.execute("SELECT mailbox, COUNT(*) FROM messages GROUP BY mailbox"))
148
+ total = sum(counts.values())
149
+ sync = db.execute("SELECT MAX(synced_at) FROM sync_state").fetchone()[0]
150
+ db.close()
151
+ size_mb = os.path.getsize(DB_PATH) / 1e6
152
+ check(True, "cache database readable",
153
+ info=f"{total:,} messages in {len(counts)} mailboxes, {size_mb:,.0f} MB")
154
+ if sync:
155
+ from datetime import datetime, timezone
156
+ age_h = (__import__("time").time() - sync) / 3600
157
+ note(f"last sync: {datetime.fromtimestamp(sync, timezone.utc):%Y-%m-%d %H:%M} UTC "
158
+ f"({age_h:.1f}h ago)")
159
+ if total == 0:
160
+ note("cache is empty - run `pmail sync --all --full` to backfill")
161
+ except sqlite3.Error as e:
162
+ check(False, "cache database readable",
163
+ fix=f"database looks corrupt ({e}); it is fully rebuildable: "
164
+ f"delete {DB_PATH} and run `pmail sync --all --full`")
165
+ else:
166
+ note(f"no cache database yet at {DB_PATH}")
167
+ note("run `pmail sync --all --full` to build it (search/list work live until then)")
168
+
169
+ # 9. free disk space for cache growth (informational)
170
+ free_gb = shutil.disk_usage(DATA_DIR).free / 1e9
171
+ check(free_gb > 1, "free disk space", info=f"{free_gb:.0f} GB available",
172
+ fix="free up disk space; the full-archive cache needs several GB")
173
+
174
+ print()
175
+ if failures:
176
+ print(f"doctor: {failures} check(s) FAILED - see fixes above")
177
+ sys.exit(1)
178
+ print("doctor: all checks passed")
@@ -0,0 +1,844 @@
1
+ #!/usr/bin/env python3
2
+ """pmail - read ProtonMail via the local Proton Mail Bridge IMAP server.
3
+
4
+ Zero dependencies (Python 3 stdlib only). Strictly read-only: bodies are
5
+ fetched with BODY.PEEK, mailboxes are opened with EXAMINE, and no command
6
+ ever stores flags, moves, or deletes mail.
7
+
8
+ Cache: ~/.local/share/pmail/mail.db (SQLite + FTS5, rebuildable).
9
+ Credentials: macOS Keychain (service "pmail-bridge") or the PMAIL_PASSWORD
10
+ env var; account email from PMAIL_ACCOUNT or `pmail setup`. Override the
11
+ server with PMAIL_HOST / PMAIL_PORT.
12
+
13
+ All data commands print JSON on stdout; progress and errors go to stderr.
14
+ """
15
+
16
+ import argparse
17
+ import getpass
18
+ import imaplib
19
+ import json
20
+ import os
21
+ import re
22
+ import sqlite3
23
+ import ssl
24
+ import subprocess
25
+ import sys
26
+ import time
27
+ from datetime import datetime, timezone
28
+ from email import policy
29
+ from email.parser import BytesParser
30
+ from email.utils import parsedate_to_datetime
31
+
32
+ IMAP_HOST = os.environ.get("PMAIL_HOST", "127.0.0.1")
33
+ IMAP_PORT = int(os.environ.get("PMAIL_PORT", "1143"))
34
+ KEYCHAIN_SERVICE = "pmail-bridge"
35
+ DATA_DIR = os.path.expanduser(os.environ.get("PMAIL_DATA", "~/.local/share/pmail"))
36
+ DB_PATH = os.path.join(DATA_DIR, "mail.db")
37
+ ATTACH_DIR = os.path.join(DATA_DIR, "attachments")
38
+ CONFIG_PATH = os.path.join(DATA_DIR, "config.json")
39
+
40
+ def account():
41
+ """Proton account email: $PMAIL_ACCOUNT, else the config written by setup."""
42
+ a = os.environ.get("PMAIL_ACCOUNT")
43
+ if a:
44
+ return a
45
+ try:
46
+ with open(CONFIG_PATH) as f:
47
+ a = json.load(f).get("account")
48
+ if a:
49
+ return a
50
+ except (OSError, json.JSONDecodeError):
51
+ pass
52
+ die("no account configured - run `pmail setup` first (or set PMAIL_ACCOUNT)")
53
+
54
+ DEFAULT_LIMIT = 20
55
+ SYNC_STALENESS = 60 # seconds; implicit sync runs at most this often
56
+ QUICK_SEED = 500 # newest N UIDs synced on every incremental sync
57
+ BACKFILL_BATCH = 200 # UID window per backfill step (cursor persisted per step)
58
+ FETCH_CHUNK = 100 # messages per IMAP FETCH command
59
+
60
+ # ---------------------------------------------------------------- utilities
61
+
62
+ def die(msg, code=1):
63
+ print(f"pmail: error: {msg}", file=sys.stderr)
64
+ sys.exit(code)
65
+
66
+ def log(msg):
67
+ print(msg, file=sys.stderr)
68
+
69
+ def emit(obj):
70
+ print(json.dumps(obj, indent=2, ensure_ascii=False))
71
+
72
+ def quote_mailbox(name):
73
+ return '"' + name.replace("\\", "\\\\").replace('"', '\\"') + '"'
74
+
75
+ def iso(ts):
76
+ if ts is None:
77
+ return None
78
+ return datetime.fromtimestamp(ts, timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
79
+
80
+ # ------------------------------------------------------------- credentials
81
+
82
+ def get_password():
83
+ pw = os.environ.get("PMAIL_PASSWORD")
84
+ if pw:
85
+ return pw
86
+ r = subprocess.run(
87
+ ["security", "find-generic-password", "-s", KEYCHAIN_SERVICE, "-a", account(), "-w"],
88
+ capture_output=True, text=True)
89
+ if r.returncode != 0:
90
+ die(f"no credentials: store the Bridge password with `pmail setup`, "
91
+ f"or set PMAIL_PASSWORD (Keychain service '{KEYCHAIN_SERVICE}', account '{account()}')")
92
+ return r.stdout.rstrip("\n")
93
+
94
+ # ------------------------------------------------------------------- IMAP
95
+
96
+ def connect():
97
+ """Connect + login. Dies with an actionable message if Bridge is down."""
98
+ ctx = ssl.create_default_context()
99
+ ctx.check_hostname = False
100
+ ctx.verify_mode = ssl.CERT_NONE
101
+ try:
102
+ m = imaplib.IMAP4(IMAP_HOST, IMAP_PORT, timeout=30)
103
+ m.starttls(ctx)
104
+ except (OSError, ssl.SSLError) as e:
105
+ die(f"cannot reach Proton Mail Bridge at {IMAP_HOST}:{IMAP_PORT} - "
106
+ f"is the Proton Mail Bridge app running? ({e})")
107
+ try:
108
+ m.login(account(), get_password())
109
+ except imaplib.IMAP4.error as e:
110
+ die(f"Bridge login failed for {account()}: {e}. "
111
+ f"Get the current password from Bridge > Mailbox details and run `pmail setup`.")
112
+ return m
113
+
114
+ def try_connect():
115
+ """Like connect(), but returns None instead of dying (for implicit sync)."""
116
+ try:
117
+ return connect()
118
+ except SystemExit:
119
+ return None
120
+
121
+ UID_RE = re.compile(rb"UID (\d+)")
122
+ FLAGS_RE = re.compile(rb"FLAGS \(([^)]*)\)")
123
+
124
+ def parse_fetch_meta(header):
125
+ uid_m = UID_RE.search(header)
126
+ flags_m = FLAGS_RE.search(header)
127
+ return (int(uid_m.group(1)) if uid_m else None,
128
+ flags_m.group(1).decode(errors="replace") if flags_m else "")
129
+
130
+ def iter_fetch(data):
131
+ """Yield (uid, flags, literal_bytes) from an imaplib FETCH response.
132
+ Gluon appends 'UID n' AFTER the literal, so combine each tuple's header
133
+ with the trailing bytes item that follows it."""
134
+ for i, item in enumerate(data):
135
+ if isinstance(item, tuple):
136
+ blob = item[0]
137
+ if i + 1 < len(data) and isinstance(data[i + 1], bytes):
138
+ blob += data[i + 1]
139
+ uid, flags = parse_fetch_meta(blob)
140
+ if uid is not None and isinstance(item[1], bytes):
141
+ yield uid, flags, item[1]
142
+
143
+ def fetch_full(m, uids):
144
+ """UID FETCH (FLAGS BODY.PEEK[]) for the given UIDs, in chunks.
145
+ Yields (uid, flags, raw_bytes)."""
146
+ for i in range(0, len(uids), FETCH_CHUNK):
147
+ chunk = uids[i:i + FETCH_CHUNK]
148
+ typ, data = m.uid("FETCH", ",".join(map(str, chunk)), "(FLAGS BODY.PEEK[])")
149
+ if typ != "OK":
150
+ log(f"warning: FETCH failed for chunk starting at UID {chunk[0]}, skipping")
151
+ continue
152
+ yield from iter_fetch(data)
153
+
154
+ def fetch_headers(m, uids):
155
+ """UID FETCH (FLAGS BODY.PEEK[HEADER.FIELDS ...]) -> list of summary dicts."""
156
+ out = []
157
+ fields = "(FLAGS BODY.PEEK[HEADER.FIELDS (FROM TO CC SUBJECT DATE)])"
158
+ for i in range(0, len(uids), FETCH_CHUNK):
159
+ chunk = uids[i:i + FETCH_CHUNK]
160
+ typ, data = m.uid("FETCH", ",".join(map(str, chunk)), fields)
161
+ if typ != "OK":
162
+ continue
163
+ for uid, flags, hdr_bytes in iter_fetch(data):
164
+ msg = BytesParser(policy=policy.default).parsebytes(hdr_bytes)
165
+ out.append({
166
+ "uid": uid,
167
+ "date": iso(parse_date(msg.get("Date"))),
168
+ "from": safe_str(msg.get("From")),
169
+ "to": safe_str(msg.get("To")),
170
+ "subject": safe_str(msg.get("Subject")),
171
+ "flags": flags,
172
+ })
173
+ return out
174
+
175
+ # ----------------------------------------------------------------- parsing
176
+
177
+ def safe_str(header):
178
+ if header is None:
179
+ return ""
180
+ try:
181
+ return str(header)
182
+ except Exception:
183
+ return str(header.raw) if hasattr(header, "raw") else ""
184
+
185
+ def parse_date(header):
186
+ if header is None:
187
+ return None
188
+ try:
189
+ dt = parsedate_to_datetime(str(header))
190
+ if dt.tzinfo is None:
191
+ dt = dt.replace(tzinfo=timezone.utc)
192
+ return int(dt.timestamp())
193
+ except Exception:
194
+ return None
195
+
196
+ def part_text(part):
197
+ try:
198
+ content = part.get_content()
199
+ if isinstance(content, bytes):
200
+ return content.decode("utf-8", errors="replace")
201
+ return content or ""
202
+ except Exception:
203
+ try:
204
+ payload = part.get_payload(decode=True)
205
+ return payload.decode("utf-8", errors="replace") if payload else ""
206
+ except Exception:
207
+ return ""
208
+
209
+ def parse_message(raw):
210
+ """Parse a raw RFC822 message into a cache record."""
211
+ msg = BytesParser(policy=policy.default).parsebytes(raw)
212
+ text_parts, html_parts, attachments = [], [], []
213
+ for part in msg.walk():
214
+ if part.get_content_maintype() == "multipart":
215
+ continue
216
+ ctype = part.get_content_type()
217
+ filename = part.get_filename()
218
+ if part.get_content_disposition() == "attachment" or filename:
219
+ try:
220
+ payload = part.get_payload(decode=True)
221
+ except Exception:
222
+ payload = None
223
+ attachments.append({
224
+ "index": len(attachments) + 1,
225
+ "name": filename or "unnamed",
226
+ "mime": ctype,
227
+ "size": len(payload) if payload else 0,
228
+ })
229
+ elif ctype == "text/plain":
230
+ text_parts.append(part_text(part))
231
+ elif ctype == "text/html":
232
+ html_parts.append(part_text(part))
233
+ body_text = "\n".join(p for p in text_parts if p).strip()
234
+ body_html = "\n".join(p for p in html_parts if p).strip()
235
+ if not body_text and body_html:
236
+ # ~97% of real mail is HTML-only; index the stripped text so FTS works
237
+ body_text = html_to_text(body_html)
238
+ else:
239
+ body_text = INVISIBLES_RE.sub("", body_text)
240
+ return {
241
+ "message_id": safe_str(msg.get("Message-ID")),
242
+ "date": parse_date(msg.get("Date")),
243
+ "from_addr": safe_str(msg.get("From")),
244
+ "to_addr": safe_str(msg.get("To")),
245
+ "cc": safe_str(msg.get("Cc")),
246
+ "subject": safe_str(msg.get("Subject")),
247
+ "size": len(raw),
248
+ "body_text": body_text,
249
+ "body_html": body_html,
250
+ "attachments": json.dumps(attachments, ensure_ascii=False),
251
+ }
252
+
253
+ HTML_TAG_RE = re.compile(r"<[^>]+>")
254
+ HTML_STRIP_RE = re.compile(r"(?is)<(script|style|head)[^>]*>.*?</\1>|<!--.*?-->")
255
+ INVISIBLES_RE = re.compile(
256
+ "[\u00ad\u034f\u200b-\u200f\u202a-\u202e\u2060\ufeff]") # zero-width/format junk
257
+
258
+ def html_to_text(html):
259
+ txt = HTML_STRIP_RE.sub(" ", html) # drop style/script/head bodies + comments
260
+ txt = HTML_TAG_RE.sub(" ", txt)
261
+ txt = INVISIBLES_RE.sub("", txt)
262
+ txt = re.sub(r"[ \t]+", " ", txt)
263
+ txt = re.sub(r"\n\s*\n\s*\n+", "\n\n", txt)
264
+ for ent, ch in (("&amp;", "&"), ("&lt;", "<"), ("&gt;", ">"),
265
+ ("&quot;", '"'), ("&#39;", "'"), ("&nbsp;", " ")):
266
+ txt = txt.replace(ent, ch)
267
+ return txt.strip()
268
+
269
+ # ------------------------------------------------------------------- cache
270
+
271
+ SCHEMA = """
272
+ PRAGMA journal_mode=WAL;
273
+ CREATE TABLE IF NOT EXISTS messages (
274
+ mailbox TEXT NOT NULL,
275
+ uid INTEGER NOT NULL,
276
+ message_id TEXT,
277
+ date INTEGER,
278
+ from_addr TEXT,
279
+ to_addr TEXT,
280
+ cc TEXT,
281
+ subject TEXT,
282
+ flags TEXT DEFAULT '',
283
+ size INTEGER,
284
+ body_text TEXT,
285
+ body_html TEXT,
286
+ attachments TEXT DEFAULT '[]',
287
+ synced_at INTEGER,
288
+ PRIMARY KEY (mailbox, uid)
289
+ );
290
+ CREATE VIRTUAL TABLE IF NOT EXISTS mail_fts USING fts5(
291
+ subject, from_addr, to_addr, body_text,
292
+ content='messages', content_rowid='rowid'
293
+ );
294
+ CREATE TRIGGER IF NOT EXISTS messages_ai AFTER INSERT ON messages BEGIN
295
+ INSERT INTO mail_fts(rowid, subject, from_addr, to_addr, body_text)
296
+ VALUES (new.rowid, new.subject, new.from_addr, new.to_addr, new.body_text);
297
+ END;
298
+ CREATE TRIGGER IF NOT EXISTS messages_ad AFTER DELETE ON messages BEGIN
299
+ INSERT INTO mail_fts(mail_fts, rowid, subject, from_addr, to_addr, body_text)
300
+ VALUES ('delete', old.rowid, old.subject, old.from_addr, old.to_addr, old.body_text);
301
+ END;
302
+ CREATE TRIGGER IF NOT EXISTS messages_au AFTER UPDATE ON messages BEGIN
303
+ INSERT INTO mail_fts(mail_fts, rowid, subject, from_addr, to_addr, body_text)
304
+ VALUES ('delete', old.rowid, old.subject, old.from_addr, old.to_addr, old.body_text);
305
+ INSERT INTO mail_fts(rowid, subject, from_addr, to_addr, body_text)
306
+ VALUES (new.rowid, new.subject, new.from_addr, new.to_addr, new.body_text);
307
+ END;
308
+ CREATE TABLE IF NOT EXISTS sync_state (
309
+ mailbox TEXT PRIMARY KEY,
310
+ uidvalidity INTEGER,
311
+ backfill_cursor INTEGER DEFAULT 0,
312
+ uidnext INTEGER,
313
+ synced_at INTEGER
314
+ );
315
+ """
316
+
317
+ UPSERT = """
318
+ INSERT INTO messages
319
+ (mailbox, uid, message_id, date, from_addr, to_addr, cc, subject,
320
+ flags, size, body_text, body_html, attachments, synced_at)
321
+ VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?)
322
+ ON CONFLICT(mailbox, uid) DO UPDATE SET
323
+ message_id=excluded.message_id, date=excluded.date,
324
+ from_addr=excluded.from_addr, to_addr=excluded.to_addr, cc=excluded.cc,
325
+ subject=excluded.subject, flags=excluded.flags, size=excluded.size,
326
+ body_text=excluded.body_text, body_html=excluded.body_html,
327
+ attachments=excluded.attachments, synced_at=excluded.synced_at
328
+ """
329
+
330
+ def open_db():
331
+ os.makedirs(DATA_DIR, exist_ok=True)
332
+ db = sqlite3.connect(DB_PATH)
333
+ db.executescript(SCHEMA)
334
+ return db
335
+
336
+ def insert_record(db, box, rec, uid, flags):
337
+ db.execute(UPSERT, (
338
+ box, uid, rec["message_id"], rec["date"], rec["from_addr"], rec["to_addr"],
339
+ rec["cc"], rec["subject"], flags, rec["size"], rec["body_text"],
340
+ rec["body_html"], rec["attachments"], int(time.time())))
341
+
342
+ def cached_count(db, box):
343
+ return db.execute("SELECT COUNT(*) FROM messages WHERE mailbox=?", (box,)).fetchone()[0]
344
+
345
+ def get_sync_row(db, box):
346
+ return db.execute(
347
+ "SELECT uidvalidity, backfill_cursor, uidnext, synced_at FROM sync_state WHERE mailbox=?",
348
+ (box,)).fetchone()
349
+
350
+ # -------------------------------------------------------------------- sync
351
+
352
+ def mailbox_status(m, box):
353
+ typ, st = m.status(quote_mailbox(box), "(UIDVALIDITY UIDNEXT MESSAGES)")
354
+ if typ != "OK" or not st or not st[0]:
355
+ die(f"STATUS failed for mailbox '{box}'")
356
+ vals = dict(re.findall(r"(\w+) (\d+)", st[0].decode(errors="replace")))
357
+ return {"uidvalidity": int(vals.get("UIDVALIDITY", 0)),
358
+ "uidnext": int(vals.get("UIDNEXT", 1)),
359
+ "messages": int(vals.get("MESSAGES", 0))}
360
+
361
+ def fetch_missing(db, m, box, uids):
362
+ """Fetch and cache any of `uids` not already in the db. Returns counts."""
363
+ if not uids:
364
+ return 0, 0
365
+ placeholders = ",".join("?" * len(uids))
366
+ have = {r[0] for r in db.execute(
367
+ f"SELECT uid FROM messages WHERE mailbox=? AND uid IN ({placeholders})",
368
+ (box, *uids))}
369
+ missing = [u for u in uids if u not in have]
370
+ fetched = 0
371
+ for uid, flags, raw in fetch_full(m, missing):
372
+ insert_record(db, box, parse_message(raw), uid, flags)
373
+ fetched += 1
374
+ db.commit()
375
+ return fetched, len(uids) - len(missing)
376
+
377
+ def reconcile_flags(db, m, box):
378
+ """Pull FLAGS for every UID in the mailbox; update only rows that changed."""
379
+ typ, data = m.uid("FETCH", "1:*", "(FLAGS)")
380
+ if typ != "OK":
381
+ return 0
382
+ changed = 0
383
+ for item in data:
384
+ blob = item[0] if isinstance(item, tuple) else item
385
+ if not isinstance(blob, bytes):
386
+ continue
387
+ uid_m, flags_m = UID_RE.search(blob), FLAGS_RE.search(blob)
388
+ if uid_m and flags_m:
389
+ flags = flags_m.group(1).decode(errors="replace")
390
+ cur = db.execute(
391
+ "UPDATE messages SET flags=? WHERE mailbox=? AND uid=? AND flags != ?",
392
+ (flags, box, int(uid_m.group(1)), flags))
393
+ changed += cur.rowcount
394
+ db.commit()
395
+ return changed
396
+
397
+ def sync_mailbox(db, m, box, full=False, batch=BACKFILL_BATCH, quick=QUICK_SEED,
398
+ progress=True):
399
+ m.select(quote_mailbox(box), readonly=True)
400
+ st = mailbox_status(m, box)
401
+ row = get_sync_row(db, box)
402
+ if row is None:
403
+ cursor = 0
404
+ db.execute("INSERT INTO sync_state (mailbox, uidvalidity, backfill_cursor) VALUES (?,?,0)",
405
+ (box, st["uidvalidity"]))
406
+ db.commit()
407
+ elif row[0] != st["uidvalidity"]:
408
+ if progress:
409
+ log(f"{box}: UIDVALIDITY changed ({row[0]} -> {st['uidvalidity']}); "
410
+ f"wiping {cached_count(db, box)} cached rows")
411
+ db.execute("DELETE FROM messages WHERE mailbox=?", (box,))
412
+ cursor = 0
413
+ else:
414
+ cursor = row[1]
415
+
416
+ typ, d = m.uid("SEARCH", "ALL")
417
+ server_uids = sorted(int(u) for u in d[0].split()) if d and d[0] else []
418
+ fetched = skipped = 0
419
+
420
+ # 1. recent-first seed: newest `quick` UIDs
421
+ f, s = fetch_missing(db, m, box, server_uids[-quick:])
422
+ fetched += f; skipped += s
423
+ if progress and f:
424
+ log(f"{box}: quick-seeded {f} recent message(s)")
425
+
426
+ # 2. resumable backfill over the contiguous UID range
427
+ if full and server_uids:
428
+ top = server_uids[-1]
429
+ while cursor < top:
430
+ lo, hi = cursor + 1, cursor + batch
431
+ chunk = [u for u in server_uids if lo <= u <= hi]
432
+ f, s = fetch_missing(db, m, box, chunk)
433
+ fetched += f; skipped += s
434
+ cursor = hi
435
+ db.execute(
436
+ "UPDATE sync_state SET backfill_cursor=?, uidvalidity=?, uidnext=?, synced_at=? "
437
+ "WHERE mailbox=?",
438
+ (cursor, st["uidvalidity"], st["uidnext"], int(time.time()), box))
439
+ db.commit()
440
+ if progress:
441
+ log(f"{box}: backfilled UID range ..{hi} "
442
+ f"(+{f} new, {cached_count(db, box)} cached total)")
443
+
444
+ # 3. flags reconcile (explicit sync only)
445
+ changed = reconcile_flags(db, m, box)
446
+ db.execute(
447
+ "UPDATE sync_state SET uidnext=?, synced_at=? WHERE mailbox=?",
448
+ (st["uidnext"], int(time.time()), box))
449
+ db.commit()
450
+ return {"mailbox": box, "server_messages": st["messages"],
451
+ "cached": cached_count(db, box), "fetched": fetched,
452
+ "already_cached": skipped, "flags_updated": changed,
453
+ "backfill_cursor": cursor, "uidnext": st["uidnext"]}
454
+
455
+ def implicit_sync(db, box, no_sync):
456
+ """Best-effort incremental sync; never fatal, keeps cache fresh on reads."""
457
+ if no_sync:
458
+ return
459
+ row = get_sync_row(db, box)
460
+ if row and row[3] and time.time() - row[3] < SYNC_STALENESS:
461
+ return
462
+ m = try_connect()
463
+ if m is None:
464
+ return # Bridge down: serve whatever the cache has
465
+ try:
466
+ st = mailbox_status(m, box)
467
+ if get_sync_row(db, box) is None or get_sync_row(db, box)[0] != st["uidvalidity"]:
468
+ sync_mailbox(db, m, box, full=False, progress=False)
469
+ else:
470
+ m.select(quote_mailbox(box), readonly=True)
471
+ typ, d = m.uid("SEARCH", "ALL")
472
+ server_uids = sorted(int(u) for u in d[0].split()) if d and d[0] else []
473
+ fetch_missing(db, m, box, server_uids[-QUICK_SEED:])
474
+ db.execute("UPDATE sync_state SET uidnext=?, synced_at=? WHERE mailbox=?",
475
+ (st["uidnext"], int(time.time()), box))
476
+ db.commit()
477
+ finally:
478
+ try:
479
+ m.logout()
480
+ except Exception:
481
+ pass
482
+
483
+ # ----------------------------------------------------------- command impls
484
+
485
+ def cmd_setup(args):
486
+ acct = input("Proton account email: ").strip()
487
+ if not acct:
488
+ die("empty account")
489
+ pw = getpass.getpass(f"Bridge password for {acct} (Bridge app > Mailbox details): ")
490
+ if not pw:
491
+ die("empty password")
492
+ r = subprocess.run(
493
+ ["security", "add-generic-password", "-s", KEYCHAIN_SERVICE, "-a", acct,
494
+ "-w", pw, "-U"], capture_output=True, text=True)
495
+ if r.returncode != 0:
496
+ die(f"Keychain store failed: {r.stderr.strip()}")
497
+ os.makedirs(DATA_DIR, exist_ok=True)
498
+ with open(CONFIG_PATH, "w") as f:
499
+ json.dump({"account": acct}, f)
500
+ os.chmod(CONFIG_PATH, 0o600)
501
+ m = connect()
502
+ m.logout()
503
+ log(f"Account '{acct}' configured; password stored in macOS Keychain and "
504
+ f"verified against Bridge.")
505
+
506
+ def cmd_mailboxes(args):
507
+ db = open_db()
508
+ m = connect()
509
+ typ, boxes = m.list()
510
+ if typ != "OK":
511
+ die("LIST failed")
512
+ out = []
513
+ for b in boxes:
514
+ if not isinstance(b, bytes):
515
+ continue
516
+ mm = re.match(rb"\(([^)]*)\)\s+\"[^\"]*\"\s+(.*)$", b)
517
+ if not mm:
518
+ continue
519
+ attrs = mm.group(1).decode(errors="replace")
520
+ name = mm.group(2).decode(errors="replace").strip('"')
521
+ entry = {"mailbox": name, "attributes": attrs}
522
+ if "\\Noselect" not in attrs:
523
+ try:
524
+ st = mailbox_status(m, name)
525
+ entry["messages"] = st["messages"]
526
+ except SystemExit:
527
+ entry["messages"] = None
528
+ entry["cached"] = cached_count(db, name)
529
+ out.append(entry)
530
+ m.logout()
531
+ emit(out)
532
+
533
+ def cmd_list(args):
534
+ db = open_db()
535
+ implicit_sync(db, args.mailbox, args.no_sync)
536
+ use_cache = (not args.live) and cached_count(db, args.mailbox) > 0
537
+ if use_cache:
538
+ sql = ("SELECT uid, date, from_addr, to_addr, subject, flags FROM messages "
539
+ "WHERE mailbox=?")
540
+ params = [args.mailbox]
541
+ if args.unseen:
542
+ sql += " AND flags NOT LIKE '%\\Seen%'"
543
+ sql += " ORDER BY uid DESC LIMIT ?"
544
+ params.append(args.limit)
545
+ rows = [{"uid": r[0], "date": iso(r[1]), "from": r[2], "to": r[3],
546
+ "subject": r[4], "flags": r[5], "source": "cache"}
547
+ for r in db.execute(sql, params)]
548
+ emit({"mailbox": args.mailbox, "source": "cache", "count": len(rows),
549
+ "messages": rows})
550
+ return
551
+ m = connect()
552
+ m.select(quote_mailbox(args.mailbox), readonly=True)
553
+ typ, d = m.uid("SEARCH", "UNSEEN" if args.unseen else "ALL")
554
+ uids = sorted(int(u) for u in d[0].split()) if d and d[0] else []
555
+ latest = uids[-args.limit:][::-1]
556
+ rows = fetch_headers(m, latest)
557
+ m.logout()
558
+ for r in rows:
559
+ r["source"] = "live"
560
+ emit({"mailbox": args.mailbox, "source": "live", "count": len(rows),
561
+ "messages": rows})
562
+
563
+ def render_message(box, uid, rec, flags, max_chars):
564
+ body, source = rec["body_text"], "text"
565
+ if not body and rec["body_html"]:
566
+ body, source = html_to_text(rec["body_html"]), "html-stripped"
567
+ truncated = False
568
+ if max_chars and len(body) > max_chars:
569
+ body = body[:max_chars]
570
+ truncated = True
571
+ return {
572
+ "mailbox": box, "uid": uid,
573
+ "date": iso(rec["date"]), "from": rec["from_addr"], "to": rec["to_addr"],
574
+ "cc": rec["cc"], "subject": rec["subject"], "flags": flags,
575
+ "body_source": source, "truncated": truncated,
576
+ "attachments": json.loads(rec["attachments"] or "[]"),
577
+ "body": body,
578
+ }
579
+
580
+ def cmd_read(args):
581
+ if args.raw:
582
+ m = connect()
583
+ m.select(quote_mailbox(args.mailbox), readonly=True)
584
+ for uid, flags, raw in fetch_full(m, [args.uid]):
585
+ sys.stdout.buffer.write(raw)
586
+ m.logout()
587
+ return
588
+ m.logout()
589
+ die(f"UID {args.uid} not found in {args.mailbox}")
590
+ db = open_db()
591
+ implicit_sync(db, args.mailbox, args.no_sync)
592
+ row = db.execute(
593
+ "SELECT message_id, date, from_addr, to_addr, cc, subject, flags, size,"
594
+ " body_text, body_html, attachments FROM messages WHERE mailbox=? AND uid=?",
595
+ (args.mailbox, args.uid)).fetchone()
596
+ if row is not None:
597
+ rec = {"date": row[1], "from_addr": row[2], "to_addr": row[3], "cc": row[4],
598
+ "subject": row[5], "body_text": row[8], "body_html": row[9],
599
+ "attachments": row[10]}
600
+ out = render_message(args.mailbox, args.uid, rec, row[6], args.max_chars)
601
+ out["source"] = "cache"
602
+ emit(out)
603
+ return
604
+ # cache miss -> live fetch, cache it, render
605
+ m = connect()
606
+ m.select(quote_mailbox(args.mailbox), readonly=True)
607
+ for uid, flags, raw in fetch_full(m, [args.uid]):
608
+ rec = parse_message(raw)
609
+ insert_record(db, args.mailbox, rec, uid, flags)
610
+ db.commit()
611
+ m.logout()
612
+ out = render_message(args.mailbox, uid, rec, flags, args.max_chars)
613
+ out["source"] = "live"
614
+ emit(out)
615
+ return
616
+ m.logout()
617
+ die(f"UID {args.uid} not found in {args.mailbox}")
618
+
619
+ def cmd_search(args):
620
+ db = open_db()
621
+ if not args.live:
622
+ implicit_sync(db, args.mailbox, args.no_sync)
623
+ if cached_count(db, args.mailbox) > 0:
624
+ return search_fts(db, args)
625
+ log("pmail: cache empty for this mailbox; falling back to --live search")
626
+ m = connect()
627
+ m.select(quote_mailbox(args.mailbox), readonly=True)
628
+ crit = []
629
+ if args.unseen:
630
+ crit.append("UNSEEN")
631
+ if args.sender:
632
+ crit += ["FROM", f'"{args.sender}"']
633
+ if args.subject:
634
+ crit += ["SUBJECT", f'"{args.subject}"']
635
+ if args.since:
636
+ crit += ["SINCE", args.since]
637
+ for w in args.words:
638
+ crit += ["TEXT", f'"{w}"']
639
+ if not crit:
640
+ die("search needs at least one word or filter")
641
+ typ, d = m.uid("SEARCH", *crit)
642
+ if typ != "OK":
643
+ die(f"SEARCH failed: {d}")
644
+ uids = sorted(int(u) for u in d[0].split()) if d and d[0] else []
645
+ latest = uids[-args.limit:][::-1]
646
+ rows = fetch_headers(m, latest)
647
+ m.logout()
648
+ emit({"mailbox": args.mailbox, "source": "live", "total_hits": len(uids),
649
+ "count": len(rows), "messages": rows})
650
+
651
+ def search_fts(db, args):
652
+ where, params = ["mail_fts MATCH ?"], [args.query]
653
+ where.append("m.mailbox=?"); params.append(args.mailbox)
654
+ if args.sender:
655
+ where.append("m.from_addr LIKE ?"); params.append(f"%{args.sender}%")
656
+ if args.since:
657
+ try:
658
+ ts = int(datetime.fromisoformat(args.since).replace(tzinfo=timezone.utc).timestamp())
659
+ where.append("m.date >= ?"); params.append(ts)
660
+ except ValueError:
661
+ die(f"--since must be ISO date (e.g. 2026-01-01), got '{args.since}'")
662
+ if args.unseen:
663
+ where.append("m.flags NOT LIKE '%\\Seen%'")
664
+ # rank in a CTE first (CROSS JOIN pins scan order: FTS first, rowid probes),
665
+ # then compute the expensive snippet() only for the rows that survive LIMIT
666
+ sql = f"""
667
+ WITH top AS (
668
+ SELECT m.rowid AS rid, bm25(mail_fts, 10.0, 5.0, 1.0, 1.0) AS r
669
+ FROM mail_fts CROSS JOIN messages m ON m.rowid = mail_fts.rowid
670
+ WHERE {' AND '.join(where)}
671
+ ORDER BY r LIMIT ?)
672
+ SELECT m.uid, m.date, m.from_addr, m.subject, m.flags,
673
+ snippet(mail_fts, 3, '>>>', '<<<', '...', 24), top.r
674
+ FROM top JOIN messages m ON m.rowid = top.rid
675
+ JOIN mail_fts ON mail_fts.rowid = top.rid
676
+ ORDER BY top.r"""
677
+ params.append(args.limit)
678
+ try:
679
+ rows = db.execute(sql, params).fetchall()
680
+ except sqlite3.OperationalError:
681
+ # not a valid FTS5 query: treat words as an AND of quoted terms
682
+ terms = args.query.split()
683
+ if not terms:
684
+ die("empty search query")
685
+ params[0] = " AND ".join('"' + t.replace('"', '') + '"' for t in terms)
686
+ try:
687
+ rows = db.execute(sql, params).fetchall()
688
+ except sqlite3.OperationalError as e:
689
+ die(f"search failed: {e}")
690
+ out = [{"uid": r[0], "date": iso(r[1]), "from": r[2], "subject": r[3],
691
+ "flags": r[4],
692
+ "snippet": INVISIBLES_RE.sub("", re.sub(r"\s+", " ", r[5] or "")).strip(),
693
+ "rank": round(r[6], 3)} for r in rows]
694
+ emit({"mailbox": args.mailbox, "source": "cache-fts", "query": args.query,
695
+ "count": len(out), "messages": out})
696
+
697
+ def cmd_sync(args):
698
+ db = open_db()
699
+ m = connect()
700
+ if args.all:
701
+ typ, boxes = m.list()
702
+ targets = []
703
+ for b in boxes or []:
704
+ if isinstance(b, bytes):
705
+ mm = re.match(rb"\(([^)]*)\)\s+\"[^\"]*\"\s+(.*)$", b)
706
+ if mm and b"\\Noselect" not in mm.group(1):
707
+ targets.append(mm.group(2).decode(errors="replace").strip('"'))
708
+ if len(targets) > 1 and "All Mail" in targets and args.skip_all_mail:
709
+ targets.remove("All Mail")
710
+ # backfill the biggest (All Mail) last so smaller boxes finish first
711
+ targets.sort(key=lambda t: (t == "All Mail", t))
712
+ else:
713
+ targets = [args.mailbox]
714
+ results = []
715
+ for box in targets:
716
+ m.select(quote_mailbox(box), readonly=True)
717
+ results.append(sync_mailbox(db, m, box, full=args.full, batch=args.batch))
718
+ m.logout()
719
+ emit({"synced": results})
720
+
721
+ def cmd_status(args):
722
+ db = open_db()
723
+ rows = []
724
+ for box, uidvalidity, cursor, uidnext, synced_at in db.execute(
725
+ "SELECT mailbox, uidvalidity, backfill_cursor, uidnext, synced_at "
726
+ "FROM sync_state ORDER BY mailbox"):
727
+ rows.append({
728
+ "mailbox": box, "cached": cached_count(db, box),
729
+ "server_messages": (uidnext - 1) if uidnext else None,
730
+ "backfill_complete": bool(uidnext and cursor >= uidnext - 1),
731
+ "backfill_cursor": cursor, "uidnext": uidnext,
732
+ "last_sync": iso(synced_at),
733
+ })
734
+ db_size = os.path.getsize(DB_PATH) if os.path.exists(DB_PATH) else 0
735
+ attach = []
736
+ if os.path.isdir(ATTACH_DIR):
737
+ attach = os.listdir(ATTACH_DIR)
738
+ emit({"db_path": DB_PATH, "db_size_mb": round(db_size / 1e6, 1),
739
+ "mailboxes": rows, "attachments_saved": len(attach)})
740
+
741
+ def cmd_save_attach(args):
742
+ db = open_db()
743
+ os.makedirs(ATTACH_DIR, exist_ok=True)
744
+ m = connect()
745
+ m.select(quote_mailbox(args.mailbox), readonly=True)
746
+ for uid, flags, raw in fetch_full(m, [args.uid]):
747
+ msg = BytesParser(policy=policy.default).parsebytes(raw)
748
+ idx = 0
749
+ for part in msg.walk():
750
+ if part.get_content_maintype() == "multipart":
751
+ continue
752
+ filename = part.get_filename()
753
+ if part.get_content_disposition() == "attachment" or filename:
754
+ idx += 1
755
+ if idx == args.index:
756
+ payload = part.get_payload(decode=True)
757
+ if payload is None:
758
+ m.logout()
759
+ die(f"attachment {args.index} has no decodable payload")
760
+ safe = re.sub(r"[^\w.\-]+", "_", filename or f"attachment-{idx}")
761
+ path = os.path.join(ATTACH_DIR,
762
+ f"{args.mailbox.replace('/', '_')}-{uid}-{idx}-{safe}")
763
+ with open(path, "wb") as f:
764
+ f.write(payload)
765
+ m.logout()
766
+ emit({"saved": path, "name": filename, "mime": part.get_content_type(),
767
+ "size": len(payload)})
768
+ return
769
+ m.logout()
770
+ die(f"message UID {args.uid} has only {idx} attachment(s)")
771
+ m.logout()
772
+ die(f"UID {args.uid} not found in {args.mailbox}")
773
+
774
+ # --------------------------------------------------------------------- CLI
775
+
776
+ def build_parser():
777
+ p = argparse.ArgumentParser(
778
+ prog="pmail",
779
+ description="Read ProtonMail via local Bridge IMAP. Read-only. JSON on stdout.")
780
+ sub = p.add_subparsers(dest="cmd", required=True)
781
+
782
+ sp = sub.add_parser("setup", help="store Bridge password in macOS Keychain")
783
+ sp.set_defaults(fn=cmd_setup)
784
+
785
+ sp = sub.add_parser("mailboxes", help="list folders/labels with counts")
786
+ sp.set_defaults(fn=cmd_mailboxes)
787
+
788
+ sp = sub.add_parser("list", help="list messages (default: INBOX, cache-first)")
789
+ sp.add_argument("mailbox", nargs="?", default="INBOX")
790
+ sp.add_argument("--limit", type=int, default=DEFAULT_LIMIT)
791
+ sp.add_argument("--unseen", action="store_true")
792
+ sp.add_argument("--live", action="store_true", help="bypass cache, query Bridge")
793
+ sp.add_argument("--no-sync", action="store_true", help="skip implicit incremental sync")
794
+ sp.set_defaults(fn=cmd_list)
795
+
796
+ sp = sub.add_parser("read", help="read one message by UID (cache-first)")
797
+ sp.add_argument("uid", type=int)
798
+ sp.add_argument("--mailbox", default="INBOX")
799
+ sp.add_argument("--raw", action="store_true", help="dump untouched RFC822 to stdout")
800
+ sp.add_argument("--max-chars", type=int, default=20000, help="0 = no limit")
801
+ sp.add_argument("--no-sync", action="store_true")
802
+ sp.set_defaults(fn=cmd_read)
803
+
804
+ sp = sub.add_parser("search", help="full-text search (FTS5 cache) or --live IMAP search")
805
+ sp.add_argument("words", nargs="*", help="keywords (AND-ed); quotes/phrases passed to FTS5")
806
+ sp.add_argument("--mailbox", default="All Mail")
807
+ sp.add_argument("--from", dest="sender")
808
+ sp.add_argument("--subject")
809
+ sp.add_argument("--since", help="ISO date, e.g. 2026-01-01")
810
+ sp.add_argument("--unseen", action="store_true")
811
+ sp.add_argument("--limit", type=int, default=DEFAULT_LIMIT)
812
+ sp.add_argument("--live", action="store_true")
813
+ sp.add_argument("--no-sync", action="store_true")
814
+ sp.set_defaults(fn=cmd_search)
815
+
816
+ sp = sub.add_parser("sync", help="sync mailbox(es) into the local cache")
817
+ sp.add_argument("--mailbox", default="INBOX")
818
+ sp.add_argument("--all", action="store_true", help="sync every folder/label")
819
+ sp.add_argument("--skip-all-mail", action="store_true",
820
+ help="with --all: skip the 'All Mail' virtual mailbox (it duplicates the others)")
821
+ sp.add_argument("--full", action="store_true", help="run/resume full backfill")
822
+ sp.add_argument("--batch", type=int, default=BACKFILL_BATCH)
823
+ sp.set_defaults(fn=cmd_sync)
824
+
825
+ sp = sub.add_parser("status", help="cache state per mailbox")
826
+ sp.set_defaults(fn=cmd_status)
827
+
828
+ sp = sub.add_parser("save-attach", help="download attachment N of message UID")
829
+ sp.add_argument("uid", type=int)
830
+ sp.add_argument("index", type=int, help="attachment index from `read` output (1-based)")
831
+ sp.add_argument("--mailbox", default="INBOX")
832
+ sp.set_defaults(fn=cmd_save_attach)
833
+ return p
834
+
835
+ def main():
836
+ args = build_parser().parse_args()
837
+ if args.cmd == "search":
838
+ args.query = " ".join(args.words)
839
+ if not args.live and not args.query and not (args.sender or args.subject):
840
+ die("search needs at least one word (cached search is text-based)")
841
+ args.fn(args)
842
+
843
+ if __name__ == "__main__":
844
+ main()