mcp-gee-sweet 0.9.2.dev215__tar.gz → 0.9.2.dev217__tar.gz

{mcp_gee_sweet-0.9.2.dev215 → mcp_gee_sweet-0.9.2.dev217}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: mcp-gee-sweet
-Version: 0.9.2.dev215
+Version: 0.9.2.dev217
 Summary: MCP server for Google Workspace — Sheets, Drive, Docs (HTML or Markdown), and Calendar.
 Project-URL: Homepage, https://github.com/khuisman/mcp-gee-sweet
 Project-URL: Bug Tracker, https://github.com/khuisman/mcp-gee-sweet/issues

{mcp_gee_sweet-0.9.2.dev215 → mcp_gee_sweet-0.9.2.dev217}/docs/qa/tests/drive_transfer.md

@@ -1787,6 +1787,56 @@ Unit tests: full suite 1883 passed / 3 skipped; ruff clean. Fast-path re-verific
 
 ---
 
+### TC-D279: downloads write to a temp file and rename it into place; a leftover temp file is never uploaded (issue #844) ⚠️ local-filesystem ⚠️ destructive
+
+**Background:** `sync_folder`, `download_file`, and `download_folder` used to stream straight into the destination. A download that failed partway left a truncated file with a current mtime, so the next bidirectional sync read it as locally newer and uploaded it over the intact Drive copy. Every download now goes through `_write_atomically`: it writes a sibling `.gee-sweet-partial-<16 hex>` file, stamps `sync_folder`'s Drive mtime on it, and renames it onto the destination only when complete. A temp file survives only a hard kill mid-write, and `sync_folder` and `upload_local_folder` skip one. The failure path itself (a dropped connection mid-download) can't be triggered through the tool surface; it's unit-tested in `tests/drive/test_transfer.py::TestSyncFolderFailedDownloadLeavesNoPartial`, `TestDownloadToolsFailedDownloadLeavesNoPartial`, and `TestWriteAtomically`. This case checks the success path through the new write, the mode and mtime it leaves, and the leftover-temp-file skip.
+
+**Setup**
+> 1. Call `create_folder(name="qa-279", parent_folder_id="{FOLDER_ID}")`; call the returned ID `{F}`.
+> 2. Run `rm -rf /tmp/qa-279 && mkdir -p /tmp/qa-279/src /tmp/qa-279/sync && head -c 3000000 /dev/urandom > /tmp/qa-279/src/big.bin`.
+> 3. Call `upload_local_file(local_path="/tmp/qa-279/src/big.bin", parent_folder_id="{F}")`; call the returned ID `{BIG}`. Call `get_file_metadata(file_id="{BIG}")` and note `modified_time` and `md5_checksum`.
+> 4. Plant a stale local copy and a leftover temp file: `printf 'stale' > /tmp/qa-279/sync/big.bin && chmod 640 /tmp/qa-279/sync/big.bin && touch -t 202001010000 /tmp/qa-279/sync/big.bin && printf 'half a file' > /tmp/qa-279/sync/.gee-sweet-partial-0123456789abcdef`.
+
+**Steps**
+> 1. `sync_folder(folder_id="{F}", local_path="/tmp/qa-279/sync/")` (real run, default `bidirectional`).
+> 2. Run `ls -la /tmp/qa-279/sync/ && md5 -q /tmp/qa-279/sync/big.bin && stat -f '%Sp %Sm' -t '%Y-%m-%dT%H:%M:%S' /tmp/qa-279/sync/big.bin` (Linux: `md5sum`, `stat -c '%A %y'`).
+> 3. `list_files(folder_id="{F}")`.
+> 4. `sync_folder(folder_id="{F}", local_path="/tmp/qa-279/sync/")` again.
+> 5. `upload_local_folder(local_path="/tmp/qa-279/sync/", parent_folder_id="{F}", skip_if_exists=false)`, then `list_files(folder_id="{F}")`.
+> 6. `download_file(file_id="{BIG}", local_path="/tmp/qa-279/sync/big.bin")`, then `ls -la /tmp/qa-279/sync/`.
+> 7. `download_folder(folder_id="{F}", local_path="/tmp/qa-279/dl/", skip_if_exists=false)`, then `ls -la /tmp/qa-279/dl/`.
+> 8. Make a read-only local copy: `printf 'protected' > /tmp/qa-279/ro.bin && chmod 444 /tmp/qa-279/ro.bin && chmod 4644 /tmp/qa-279/dl/big.bin`. Call `download_file(file_id="{BIG}", local_path="/tmp/qa-279/ro.bin")`. Then call `download_file(file_id="{BIG}", local_path="/tmp/qa-279/dl/big.bin")` and run `ls -la /tmp/qa-279/ /tmp/qa-279/dl/ && cat /tmp/qa-279/ro.bin`. (Not as root: root can write a `0444` file.)
+
+**Checks**
+- Step 1: `downloaded` is `["big.bin"]`. `uploaded` is empty, and `.gee-sweet-partial-0123456789abcdef` appears in no result list. `failed` is empty.
+- Step 2: `big.bin` is 3000000 bytes, its md5 matches setup step 3's `md5_checksum`, its mode is still `-rw-r-----` (the replaced file's mode is kept), and its mtime matches setup step 3's `modified_time` to the second (local time vs UTC aside). The only `.gee-sweet-partial-*` file in the directory is the planted `…0123456789abcdef`; no new one was left behind.
+- Step 3: `{F}` holds exactly one file, `big.bin`. No `.gee-sweet-partial-*` file was uploaded.
+- Step 4: `big.bin` in `skipped`; `uploaded` and `downloaded` empty (the restamped mtime round-trips, as in #346).
+- Step 5: `uploaded` lists only `big.bin` (a second copy, since `skip_if_exists=false`); the planted temp file is not uploaded and not in `failed`. `list_files` shows two `big.bin` and no `.gee-sweet-partial-*`.
+- Step 6: succeeds with `size_bytes` 3000000. The directory still holds only `big.bin` and the planted temp file.
+- Step 7: `downloaded` is `["big.bin"]` and `failed` has one duplicate-filename entry for the second `big.bin` (PR #351). `/tmp/qa-279/dl/` holds only `big.bin`, 3000000 bytes, and no `.gee-sweet-partial-*` file.
+- Step 8 (PR #884 QA round 1): the first `download_file` fails with a permission error. `ro.bin` still reads `protected`, still `-r--r--r--`, and no `.gee-sweet-partial-*` file is in `/tmp/qa-279/`. The second call succeeds, and `dl/big.bin` is `-rw-r--r--`: the setuid bit is not carried onto the new content.
+
+**Teardown**
+`delete_file` on `{F}` (takes its contents with it). `rm -rf /tmp/qa-279`.
+
+**Result (2026-10-04) ✅ PASS** — Sky, PR #884 round 1 (`b74ec2b`), OAuth, throwaway `qa-279` folder under `TEST_FOLDER_ID`. Drive `big.bin`: 3000000 B, md5 `f606776b…2c1a`, `modified_time` `17:42:20Z`.
+- **Step 1:** `downloaded: ["big.bin"]`; `uploaded`, `conflicts`, `failed` empty; the planted temp file appears in no list.
+- **Step 2:** `big.bin` 3000000 B, md5 matches, mode `-rw-r-----` kept, mtime `10:42:20-0700` (= `17:42:20Z`). Only the planted `…0123456789abcdef` temp file present.
+- **Step 3:** one file, `big.bin`.
+- **Step 4:** `skipped: ["big.bin"]`, nothing transferred.
+- **Step 5:** `uploaded: ["big.bin"]` only; `list_files` shows two `big.bin`, no temp file.
+- **Step 6:** `size_bytes` 3000000; directory still holds only `big.bin` + the planted temp file.
+- **Step 7:** `downloaded: ["big.bin"]`, one duplicate-filename `failed` entry; `dl/` holds only `big.bin`, 3000000 B.
+
+The case passes, but the PR goes back for a regression this case doesn't cover: a read-only (`0444`) local destination is now silently replaced. Live, after this case: `printf protected > big.bin && chmod 444 big.bin`, then `download_file(file_id="{BIG}", local_path=".../big.bin")` returned success and left a 3000000 B `-r--r--r--` file. Before this PR the `open("wb")` raised `PermissionError` and left the file alone. See the PR comment.
+
+**Result (2026-10-04) ✅ PASS** — Sky, PR #884 round 2 (`b4e7287`), step 8 only, after reconnecting to the fix. Steps 1–7 code paths are unchanged apart from the new up-front `os.access` check and the `0o777` mask. Fresh `qa-279` folder; setup plus step 7 (`download_folder` → `dl/big.bin`, 3000000 B) re-run first.
+- **Step 8, `ro.bin`:** `download_file` failed with `[Errno 13] Permission denied: '/tmp/qa-279/ro.bin'`. `ro.bin` still reads `protected`, still `-r--r--r--`, and there's no `.gee-sweet-partial-*` in `/tmp/qa-279/`.
+- **Step 8, `dl/big.bin` (`chmod 4644` first):** succeeded, `size_bytes` 3000000, and the file is now `-rw-r--r--`: setuid dropped.
+
+---
+
 ## `list_revisions`
 
 ### TC-D146: List revisions for a spreadsheet

{mcp_gee_sweet-0.9.2.dev215 → mcp_gee_sweet-0.9.2.dev217}/docs/roadmap.md

@@ -315,7 +315,7 @@ Same shape as v0.8.1 — stabilize on defects that surfaced since v0.9.0 shipped
 - [x] `reply_to_message` should honor Reply-To and handle replies to your own sent mail ([#791](https://github.com/khuisman/mcp-gee-sweet/issues/791)) (PR #815)
 - [x] Decode message bodies using the part's charset; tolerate missing base64 padding ([#792](https://github.com/khuisman/mcp-gee-sweet/issues/792)) (PR #816)
 - [x] `get_message`/`get_thread` return no body when Gmail delivers a large (~MB) `text/plain`/`text/html` part by `attachmentId`. Found in #803's live QA ([#825](https://github.com/khuisman/mcp-gee-sweet/issues/825)) (PR #829)
-- [ ] `get_thread`/`get_message`: add a body-less option — a long thread trips the response-size cap and there's no other way to get its message IDs, so it can't be read at all ([#793](https://github.com/khuisman/mcp-gee-sweet/issues/793))
+- [x] `get_thread`/`get_message`: add a body-less option — a long thread trips the response-size cap and there's no other way to get its message IDs, so it can't be read at all ([#793](https://github.com/khuisman/mcp-gee-sweet/issues/793)) (PR #885)
 - [ ] Reply-all self-exclusion (silent `getProfile` failure, send-as aliases), attachment reads blocking the event loop, attachment MIME guessing ([#802](https://github.com/khuisman/mcp-gee-sweet/issues/802))
 - [ ] Deduplicate `send_message`/`create_draft` compose and `list_messages`/`list_threads` kwargs/pagination helpers ([#795](https://github.com/khuisman/mcp-gee-sweet/issues/795))
 - [ ] Local-file read policy across local-path tools — in a remote/SSE deployment these read the server's own filesystem (token, SA key), and Gmail attachments can now mail them out. One consistent policy (e.g. a config switch), not per-tool patches. Decision-needed; Joy scopes ([#796](https://github.com/khuisman/mcp-gee-sweet/issues/796))

{mcp_gee_sweet-0.9.2.dev215 → mcp_gee_sweet-0.9.2.dev217}/src/mcp_gee_sweet/tools/drive/transfer.py

@@ -1,15 +1,19 @@
 import asyncio
 import base64
+import contextlib
+import errno
 import hashlib
 import io
 import logging
 import mimetypes
 import os
-from collections.abc import Iterable
+import re
+import secrets
+from collections.abc import Callable, Iterable
 from dataclasses import dataclass
 from datetime import datetime, timezone
 from pathlib import Path
-from typing import Any, Literal
+from typing import Any, BinaryIO, Literal
 
 import markdown as _md
 from googleapiclient.errors import HttpError
@@ -39,6 +43,15 @@ _EXPORT_MIME: dict[str, tuple[str, str]] = {
 
 _SYSTEM_FILES = {".DS_Store", "Thumbs.db", "desktop.ini", ".localized"}
 
+# Every download writes to a sibling temp file with this name and renames it
+# onto the destination only once complete (_write_atomically, #844). A temp
+# file only outlives its download when the process is killed mid-write;
+# sync_folder and upload_local_folder skip one so a leftover partial is never
+# uploaded. No part of the destination's name is embedded, so the temp name
+# can't exceed the filesystem's name-length limit.
+_PARTIAL_DOWNLOAD_PREFIX = ".gee-sweet-partial-"
+_PARTIAL_DOWNLOAD_RE = re.compile(r"\.gee-sweet-partial-[0-9a-f]{16}")
+
 _SYNC_MTIME_TOLERANCE = 5  # seconds — absorbs clock skew and upload-time drift
 
 # use_checksum hashes every both-sides pair with a Drive md5Checksum (#716), so
@@ -375,6 +388,55 @@ def _drive_time_str(dt: datetime) -> str:
     return dt.strftime("%Y-%m-%dT%H:%M:%S.000Z")
 
 
+def _is_partial_download(name: str) -> bool:
+    """Whether `name` is a temp file _write_atomically left behind."""
+    return _PARTIAL_DOWNLOAD_RE.fullmatch(name) is not None
+
+
+def _write_atomically(
+    dest: Path, write: Callable[[BinaryIO], object], mtime: float | None = None
+) -> None:
+    """Run `write` against a sibling temp file, then rename it onto `dest`.
+
+    `dest` only ever holds its previous content or the complete new content.
+    Writing straight into `dest` left a truncated file behind when a download
+    failed partway, with a current mtime that made sync_folder treat it as the
+    newer copy and upload it over the intact Drive file (#844). `mtime`, when
+    given, is stamped on the temp file before the rename, so a failed restamp
+    also leaves `dest` untouched. The temp file is removed on any failure.
+
+    A symlink at `dest` is written through, as the old `open("wb")` did: the
+    temp file goes next to the link's target and replaces the target. The new
+    file gets the mode a fresh `open("wb")` would (0o666 less the umask), or the
+    replaced file's permission bits when there was one (never its
+    setuid/setgid/sticky bits, which an in-place write would have cleared). A
+    hard link to the old file keeps the old content, since the rename swaps in
+    a new inode.
+
+    A rename needs write permission on the directory, not the file, so an
+    existing file this process couldn't open for writing is refused up front
+    with the same PermissionError `open("wb")` raised; otherwise a file the
+    user made read-only would be silently replaced (PR #884 QA round 1).
+    """
+    target = Path(os.path.realpath(dest)) if dest.is_symlink() else dest
+    if target.exists() and not os.access(target, os.W_OK):
+        raise PermissionError(errno.EACCES, os.strerror(errno.EACCES), str(target))
+    tmp = target.with_name(_PARTIAL_DOWNLOAD_PREFIX + secrets.token_hex(8))
+    fd = os.open(tmp, os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_BINARY", 0), 0o666)
+    try:
+        with os.fdopen(fd, "wb") as fh:
+            write(fh)
+        with contextlib.suppress(FileNotFoundError):
+            os.chmod(tmp, target.stat().st_mode & 0o777)
+        if mtime is not None:
+            os.utime(tmp, (mtime, mtime))
+        os.replace(tmp, target)
+    except BaseException:
+        with contextlib.suppress(OSError):
+            os.unlink(tmp)
+        raise
+
+
 def _local_mtime_str(path: Path) -> str:
     """_local_mtime_dt in the RFC 3339 form Drive's modifiedTime field takes."""
     return _drive_time_str(_local_mtime_dt(path))
@@ -1328,6 +1390,8 @@ async def _sync_level(
                 continue
             if skip_system_files and p.name in _SYSTEM_FILES:
                 continue
+            if _is_partial_download(p.name):
+                continue
             # Only a backslash (a separator on Windows, legal in a POSIX name) can
             # fail this for a name the OS listed. Such a file would upload under
             # a name this check then refuses on the Drive side, so it'd never
@@ -1899,6 +1963,15 @@ async def _sync_level(
                 # The plan already routed unsafe names to 'unsafe_name'; this is
                 # the shared backstop, and a refusal lands as download_fail.
                 dest_file = _safe_local_dest(dest_dir, name, resolved_dest_dir)
+                # Mirror what the upload branch above does in reverse: set the
+                # local file's mtime to Drive's modifiedTime so the round trip
+                # stays within _SYNC_MTIME_TOLERANCE on the next sync. Without
+                # this the local mtime is "now" (write time), which is always
+                # later than Drive's original timestamp — the next sync sees the
+                # file as locally newer and re-uploads it, indefinitely (#346).
+                # _write_atomically stamps it before the rename, so a failed
+                # download never leaves a "newer" partial in place (#844).
+                drive_ts = _drive_mtime(entry).timestamp()
                 if is_workspace:
                     target_mime = _EXPORT_MIME[export_format][0]
                     content = await execute_in_thread(
@@ -1907,7 +1980,9 @@ async def _sync_level(
                     )
                     if not isinstance(content, bytes):
                         content = content.encode("utf-8")
-                    await asyncio.to_thread(dest_file.write_bytes, content)
+                    await asyncio.to_thread(
+                        _write_atomically, dest_file, lambda fh, c=content: fh.write(c), drive_ts
+                    )
                 else:
 
                     def _download_to_completion(fid=fid, dest_file=dest_file) -> None:
@@ -1915,21 +1990,16 @@ async def _sync_level(
                             fileId=fid, supportsAllDrives=True
                         )
                         request.http = thread_http(drive_service)
-                        with dest_file.open("wb") as fh:
+
+                        def _stream(fh: BinaryIO) -> None:
                             downloader = MediaIoBaseDownload(fh, request)
                             done = False
                             while not done:
                                 _, done = downloader.next_chunk()
 
+                        _write_atomically(dest_file, _stream, drive_ts)
+
                     await asyncio.to_thread(_download_to_completion)
-                # Mirror what the upload branch above does in reverse: set the
-                # local file's mtime to Drive's modifiedTime so the round trip
-                # stays within _SYNC_MTIME_TOLERANCE on the next sync. Without
-                # this the local mtime is "now" (write time), which is always
-                # later than Drive's original timestamp — the next sync sees the
-                # file as locally newer and re-uploads it, indefinitely (#346).
-                dtime = _drive_mtime(entry)
-                os.utime(dest_file, (dtime.timestamp(), dtime.timestamp()))
                 size = dest_file.stat().st_size
                 logger.debug("Synced (download) Drive → %s%s (%d bytes)", rel_prefix, name, size)
                 return {"kind": "download_ok", "name": name, "bytes": size}
@@ -2628,7 +2698,9 @@ def register(tool):
         if not folder.is_dir():
             raise ValueError(f"No directory found at {local_path!r}")
 
-        candidates = [p for p in folder.iterdir() if p.is_file()]
+        candidates = [
+            p for p in folder.iterdir() if p.is_file() and not _is_partial_download(p.name)
+        ]
         if skip_system_files:
             candidates = [p for p in candidates if p.name not in _SYSTEM_FILES]
 
@@ -2867,18 +2939,21 @@ def register(tool):
             )
             if not isinstance(content, bytes):
                 content = content.encode("utf-8")
-            await asyncio.to_thread(dest.write_bytes, content)
+            await asyncio.to_thread(_write_atomically, dest, lambda fh: fh.write(content))
         else:
 
             def _download_to_completion() -> None:
                 request = drive_service.files().get_media(fileId=file_id, supportsAllDrives=True)
                 request.http = thread_http(drive_service)
-                with dest.open("wb") as fh:
+
+                def _stream(fh: BinaryIO) -> None:
                     downloader = MediaIoBaseDownload(fh, request)
                     done = False
                     while not done:
                         _, done = downloader.next_chunk()
 
+                _write_atomically(dest, _stream)
+
             await asyncio.to_thread(_download_to_completion)
 
         size = dest.stat().st_size
@@ -3086,7 +3161,9 @@ def register(tool):
                     )
                     if not isinstance(content, bytes):
                         content = content.encode("utf-8")
-                    await asyncio.to_thread(candidate.dest_file.write_bytes, content)
+                    await asyncio.to_thread(
+                        _write_atomically, candidate.dest_file, lambda fh: fh.write(content)
+                    )
                 else:
 
                     def _download_to_completion(
@@ -3096,12 +3173,15 @@ def register(tool):
                             fileId=fid, supportsAllDrives=True
                         )
                         request.http = thread_http(drive_service)
-                        with dest_file.open("wb") as fh:
+
+                        def _stream(fh: BinaryIO) -> None:
                             downloader = MediaIoBaseDownload(fh, request)
                             done = False
                             while not done:
                                 _, done = downloader.next_chunk()
 
+                        _write_atomically(dest_file, _stream)
+
                     await asyncio.to_thread(_download_to_completion)
 
                 size = candidate.dest_file.stat().st_size