fetch2gmail 1.1.9__tar.gz → 1.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (27) hide show
  1. {fetch2gmail-1.1.9/src/fetch2gmail.egg-info → fetch2gmail-1.2.0}/PKG-INFO +14 -5
  2. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/README.md +12 -4
  3. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/pyproject.toml +2 -1
  4. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0/src/fetch2gmail.egg-info}/PKG-INFO +14 -5
  5. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetch2gmail.egg-info/SOURCES.txt +3 -1
  6. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetch2gmail.egg-info/requires.txt +1 -0
  7. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/cli.py +2 -0
  8. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/gmail_client.py +3 -0
  9. fetch2gmail-1.2.0/src/fetcher/idle_watcher.py +742 -0
  10. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/imap_client.py +117 -11
  11. fetch2gmail-1.2.0/src/fetcher/run.py +558 -0
  12. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/web_ui.py +143 -71
  13. fetch2gmail-1.2.0/tests/test_idle_watcher.py +309 -0
  14. fetch2gmail-1.1.9/src/fetcher/run.py +0 -482
  15. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/LICENSE +0 -0
  16. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/setup.cfg +0 -0
  17. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetch2gmail.egg-info/dependency_links.txt +0 -0
  18. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetch2gmail.egg-info/entry_points.txt +0 -0
  19. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetch2gmail.egg-info/top_level.txt +0 -0
  20. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/__init__.py +0 -0
  21. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/auth_server.py +0 -0
  22. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/auth_ui.py +0 -0
  23. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/config.py +0 -0
  24. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/env_file.py +0 -0
  25. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/log_buffer.py +0 -0
  26. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/state.py +0 -0
  27. {fetch2gmail-1.1.9 → fetch2gmail-1.2.0}/src/fetcher/ui_auth.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: fetch2gmail
3
- Version: 1.1.9
3
+ Version: 1.2.0
4
4
  Summary: Self-hosted email fetcher: IMAP to Gmail API import with idempotent state tracking
5
5
  Author: Fetch2Gmail Contributors
6
6
  License-Expression: MIT
@@ -24,6 +24,7 @@ Requires-Dist: python-dotenv>=1.0.0
24
24
  Requires-Dist: python-multipart>=0.0.6
25
25
  Requires-Dist: cryptography>=42.0.0
26
26
  Requires-Dist: bcrypt>=4.0.0
27
+ Requires-Dist: imapclient>=3.0.0
27
28
  Provides-Extra: dev
28
29
  Requires-Dist: pytest>=7.0; extra == "dev"
29
30
  Requires-Dist: pytest-asyncio>=0.21; extra == "dev"
@@ -33,8 +34,8 @@ Dynamic: license-file
33
34
 
34
35
  Self-hosted email fetcher: **IMAP (ISP mailbox) → Gmail API import**. Replaces Gmail’s deprecated POP3 fetch.
35
36
 
36
- - Polls an ISP mailbox via **IMAPS** (port 993).
37
- - Imports messages with **Gmail API** (not SMTP); applies a Gmail label, preserves headers and date.
37
+ - Watches an ISP mailbox via **IMAPS** (port 993), using **IMAP IDLE** for near-realtime delivery when the server supports it (falls back to polling otherwise).
38
+ - Imports messages with **Gmail API** (not SMTP); applies a Gmail label, preserves headers and date. By default uses **`neverMarkSpam`** so Gmail’s spam classifier does not bury ISP→Gmail imports.
38
39
  - Deletes from ISP only after Gmail confirms import. **Idempotent** (UID + message hash).
39
40
 
40
41
  **Requirements:** Python 3.11+, IMAP credentials, Google Cloud project with Gmail API and OAuth2 (Web application).
@@ -211,9 +212,17 @@ Your Google OAuth client must use **Web application** (not Desktop) and have the
211
212
 
212
213
  Google does not allow redirect URIs that use an IP address. That is why the token must be obtained on a machine where the app can use localhost (Use case 1 steps 1–4 on a PC; Use case 2 on the same machine).
213
214
 
214
- ### Poll interval
215
+ ### IMAP IDLE and poll interval
215
216
 
216
- The app runs the fetch every 5 minutes by default (background poller in the same process as the web UI). You can change the interval in the web UI (Config) or in **config.json** (`poll_interval_minutes`).
217
+ The background watcher in the web UI process prefers **IMAP IDLE** (RFC 2177) when the server advertises the `IDLE` capability. New mail can trigger an import immediately instead of waiting for the next poll.
218
+
219
+ - **`imap.use_idle`** (default `true`): enable IDLE when supported. Disable to force polling.
220
+ - **`imap.idle_refresh_minutes`** (default `25`): leave IDLE, send `NOOP`, and re-enter IDLE before the typical ~29-minute server timeout. Also enables TCP `SO_KEEPALIVE` and reconnects with exponential backoff on silent drops.
221
+ - **`poll_interval_minutes`** (default `5`): used when IDLE is missing or disabled.
222
+
223
+ All three can be changed in the web UI (Config) or in **config.json**.
224
+
225
+ **Manual smoke checks:** with IDLE supported, send yourself mail and confirm an import in the UI logs within seconds (look for `IDLE: mailbox update`). Set `use_idle` to `false` (or use a server without IDLE) and confirm fetches still run on the poll interval.
217
226
 
218
227
  ### Data directory
219
228
 
@@ -2,8 +2,8 @@
2
2
 
3
3
  Self-hosted email fetcher: **IMAP (ISP mailbox) → Gmail API import**. Replaces Gmail’s deprecated POP3 fetch.
4
4
 
5
- - Polls an ISP mailbox via **IMAPS** (port 993).
6
- - Imports messages with **Gmail API** (not SMTP); applies a Gmail label, preserves headers and date.
5
+ - Watches an ISP mailbox via **IMAPS** (port 993), using **IMAP IDLE** for near-realtime delivery when the server supports it (falls back to polling otherwise).
6
+ - Imports messages with **Gmail API** (not SMTP); applies a Gmail label, preserves headers and date. By default uses **`neverMarkSpam`** so Gmail’s spam classifier does not bury ISP→Gmail imports.
7
7
  - Deletes from ISP only after Gmail confirms import. **Idempotent** (UID + message hash).
8
8
 
9
9
  **Requirements:** Python 3.11+, IMAP credentials, Google Cloud project with Gmail API and OAuth2 (Web application).
@@ -180,9 +180,17 @@ Your Google OAuth client must use **Web application** (not Desktop) and have the
180
180
 
181
181
  Google does not allow redirect URIs that use an IP address. That is why the token must be obtained on a machine where the app can use localhost (Use case 1 steps 1–4 on a PC; Use case 2 on the same machine).
182
182
 
183
- ### Poll interval
183
+ ### IMAP IDLE and poll interval
184
184
 
185
- The app runs the fetch every 5 minutes by default (background poller in the same process as the web UI). You can change the interval in the web UI (Config) or in **config.json** (`poll_interval_minutes`).
185
+ The background watcher in the web UI process prefers **IMAP IDLE** (RFC 2177) when the server advertises the `IDLE` capability. New mail can trigger an import immediately instead of waiting for the next poll.
186
+
187
+ - **`imap.use_idle`** (default `true`): enable IDLE when supported. Disable to force polling.
188
+ - **`imap.idle_refresh_minutes`** (default `25`): leave IDLE, send `NOOP`, and re-enter IDLE before the typical ~29-minute server timeout. Also enables TCP `SO_KEEPALIVE` and reconnects with exponential backoff on silent drops.
189
+ - **`poll_interval_minutes`** (default `5`): used when IDLE is missing or disabled.
190
+
191
+ All three can be changed in the web UI (Config) or in **config.json**.
192
+
193
+ **Manual smoke checks:** with IDLE supported, send yourself mail and confirm an import in the UI logs within seconds (look for `IDLE: mailbox update`). Set `use_idle` to `false` (or use a server without IDLE) and confirm fetches still run on the poll interval.
186
194
 
187
195
  ### Data directory
188
196
 
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "fetch2gmail"
7
- version = "1.1.9"
7
+ version = "1.2.0"
8
8
  description = "Self-hosted email fetcher: IMAP to Gmail API import with idempotent state tracking"
9
9
  readme = "README.md"
10
10
  license = "MIT"
@@ -31,6 +31,7 @@ dependencies = [
31
31
  "python-multipart>=0.0.6",
32
32
  "cryptography>=42.0.0",
33
33
  "bcrypt>=4.0.0",
34
+ "imapclient>=3.0.0",
34
35
  ]
35
36
 
36
37
  [project.optional-dependencies]
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: fetch2gmail
3
- Version: 1.1.9
3
+ Version: 1.2.0
4
4
  Summary: Self-hosted email fetcher: IMAP to Gmail API import with idempotent state tracking
5
5
  Author: Fetch2Gmail Contributors
6
6
  License-Expression: MIT
@@ -24,6 +24,7 @@ Requires-Dist: python-dotenv>=1.0.0
24
24
  Requires-Dist: python-multipart>=0.0.6
25
25
  Requires-Dist: cryptography>=42.0.0
26
26
  Requires-Dist: bcrypt>=4.0.0
27
+ Requires-Dist: imapclient>=3.0.0
27
28
  Provides-Extra: dev
28
29
  Requires-Dist: pytest>=7.0; extra == "dev"
29
30
  Requires-Dist: pytest-asyncio>=0.21; extra == "dev"
@@ -33,8 +34,8 @@ Dynamic: license-file
33
34
 
34
35
  Self-hosted email fetcher: **IMAP (ISP mailbox) → Gmail API import**. Replaces Gmail’s deprecated POP3 fetch.
35
36
 
36
- - Polls an ISP mailbox via **IMAPS** (port 993).
37
- - Imports messages with **Gmail API** (not SMTP); applies a Gmail label, preserves headers and date.
37
+ - Watches an ISP mailbox via **IMAPS** (port 993), using **IMAP IDLE** for near-realtime delivery when the server supports it (falls back to polling otherwise).
38
+ - Imports messages with **Gmail API** (not SMTP); applies a Gmail label, preserves headers and date. By default uses **`neverMarkSpam`** so Gmail’s spam classifier does not bury ISP→Gmail imports.
38
39
  - Deletes from ISP only after Gmail confirms import. **Idempotent** (UID + message hash).
39
40
 
40
41
  **Requirements:** Python 3.11+, IMAP credentials, Google Cloud project with Gmail API and OAuth2 (Web application).
@@ -211,9 +212,17 @@ Your Google OAuth client must use **Web application** (not Desktop) and have the
211
212
 
212
213
  Google does not allow redirect URIs that use an IP address. That is why the token must be obtained on a machine where the app can use localhost (Use case 1 steps 1–4 on a PC; Use case 2 on the same machine).
213
214
 
214
- ### Poll interval
215
+ ### IMAP IDLE and poll interval
215
216
 
216
- The app runs the fetch every 5 minutes by default (background poller in the same process as the web UI). You can change the interval in the web UI (Config) or in **config.json** (`poll_interval_minutes`).
217
+ The background watcher in the web UI process prefers **IMAP IDLE** (RFC 2177) when the server advertises the `IDLE` capability. New mail can trigger an import immediately instead of waiting for the next poll.
218
+
219
+ - **`imap.use_idle`** (default `true`): enable IDLE when supported. Disable to force polling.
220
+ - **`imap.idle_refresh_minutes`** (default `25`): leave IDLE, send `NOOP`, and re-enter IDLE before the typical ~29-minute server timeout. Also enables TCP `SO_KEEPALIVE` and reconnects with exponential backoff on silent drops.
221
+ - **`poll_interval_minutes`** (default `5`): used when IDLE is missing or disabled.
222
+
223
+ All three can be changed in the web UI (Config) or in **config.json**.
224
+
225
+ **Manual smoke checks:** with IDLE supported, send yourself mail and confirm an import in the UI logs within seconds (look for `IDLE: mailbox update`). Set `use_idle` to `false` (or use a server without IDLE) and confirm fetches still run on the poll interval.
217
226
 
218
227
  ### Data directory
219
228
 
@@ -14,9 +14,11 @@ src/fetcher/cli.py
14
14
  src/fetcher/config.py
15
15
  src/fetcher/env_file.py
16
16
  src/fetcher/gmail_client.py
17
+ src/fetcher/idle_watcher.py
17
18
  src/fetcher/imap_client.py
18
19
  src/fetcher/log_buffer.py
19
20
  src/fetcher/run.py
20
21
  src/fetcher/state.py
21
22
  src/fetcher/ui_auth.py
22
- src/fetcher/web_ui.py
23
+ src/fetcher/web_ui.py
24
+ tests/test_idle_watcher.py
@@ -7,6 +7,7 @@ python-dotenv>=1.0.0
7
7
  python-multipart>=0.0.6
8
8
  cryptography>=42.0.0
9
9
  bcrypt>=4.0.0
10
+ imapclient>=3.0.0
10
11
 
11
12
  [dev]
12
13
  pytest>=7.0
@@ -284,6 +284,7 @@ def _write_default_config(dest: Path) -> None:
284
284
  "mailbox": "INBOX",
285
285
  "use_ssl": True,
286
286
  "delete_after_import": True,
287
+ "never_mark_spam": True,
287
288
  "since_date": None,
288
289
  },
289
290
  # Canonical: gmail_accounts (multi-account). Single-account setups will have one entry.
@@ -318,6 +319,7 @@ def config_wizard_interactive() -> None:
318
319
  "mailbox": mailbox,
319
320
  "use_ssl": True,
320
321
  "delete_after_import": True,
322
+ "never_mark_spam": True,
321
323
  "since_date": None,
322
324
  },
323
325
  "gmail_accounts": [{"use_label": bool(label), "label": label or "ISP Mail", "credentials_path": cred_path, "token_path": token_path}],
@@ -128,11 +128,13 @@ def import_message(
128
128
  inbox_label_id: str | None = None,
129
129
  unread_label_id: str | None = None,
130
130
  mark_unread: bool = True,
131
+ never_mark_spam: bool = True,
131
132
  ) -> str:
132
133
  """
133
134
  Import a single message using users.messages.import.
134
135
  Preserves original headers and sets internalDate when possible.
135
136
  If mark_unread is False (message was read on ISP), the message is imported as read.
137
+ If never_mark_spam is True, ignore Gmail's spam classifier for this import.
136
138
  Returns Gmail message ID.
137
139
  """
138
140
  raw = _normalize_from_header(raw)
@@ -152,6 +154,7 @@ def import_message(
152
154
  lambda: service.users().messages().import_(
153
155
  userId=user_id,
154
156
  body=body,
157
+ neverMarkSpam=never_mark_spam,
155
158
  ).execute()
156
159
  )
157
160
  return msg["id"]