asanypath 2.1.1__tar.gz → 2.2.0__tar.gz

{asanypath-2.1.1 → asanypath-2.2.0}/.github/workflows/unittests.yml

@@ -49,11 +49,8 @@ jobs:
       - name: Ruff checks
         run: uvx ruff check src tests && uvx ruff format --check src tests
 
-      - name: Create virtual environment
-        run: uv venv --python "${{ matrix.python-version }}"
-
-      - name: Install project and native extension
-        run: uv pip install -e ".[dev]"
+      - name: Install project and native extension from uv.lock
+        run: uv sync --locked --extra dev --python "${{ matrix.python-version }}"
 
       - name: Test with pytest and coverage
         env:

{asanypath-2.1.1 → asanypath-2.2.0}/.pre-commit-config.yaml

@@ -22,4 +22,10 @@ repos:
       entry: cargo check --manifest-path rust/Cargo.toml --locked
       language: system
       files: ^rust/.*\.(rs|toml|lock)$
-      pass_filenames: false
+      pass_filenames: false
+    # uv.lock must only reference the public index (a mirror URL leaks internal hosts).
+    - id: uv-lock-public-index
+      name: uv.lock uses only public index URLs
+      language: pygrep
+      entry: '(registry|url) = "https?://(?!pypi\.org/|files\.pythonhosted\.org/)'
+      files: ^uv\.lock$

{asanypath-2.1.1 → asanypath-2.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: asanypath
-Version: 2.1.1
+Version: 2.2.0
 Summary: Async pathlib for local and cloud storage
 License-Expression: MIT
 License-File: LICENSE
@@ -19,11 +19,12 @@ Classifier: Programming Language :: Python :: 3.14
 Classifier: Topic :: Software Development :: Libraries :: Python Modules
 Requires-Python: >=3.10
 Requires-Dist: anyio>=4
-Requires-Dist: asanypath-native~=2.1.0
+Requires-Dist: asanypath-native~=2.2.0
 Requires-Dist: msgspec>=0.22.0
 Requires-Dist: yarl>=1.2
 Provides-Extra: all
 Requires-Dist: aioftp>=0.21; extra == 'all'
+Requires-Dist: diskcache>=5.6; extra == 'all'
 Requires-Dist: keyring>=24; extra == 'all'
 Provides-Extra: bench
 Requires-Dist: adlfs; extra == 'bench'
@@ -37,12 +38,15 @@ Requires-Dist: gcloud-aio-storage; extra == 'bench'
 Requires-Dist: gcsfs; extra == 'bench'
 Requires-Dist: requests; extra == 'bench'
 Requires-Dist: s3fs; extra == 'bench'
+Provides-Extra: cache
+Requires-Dist: diskcache>=5.6; extra == 'cache'
 Provides-Extra: cli
 Requires-Dist: rich-click>=1.9.7; extra == 'cli'
 Provides-Extra: credstore
 Requires-Dist: keyring>=24; extra == 'credstore'
 Provides-Extra: dev
 Requires-Dist: aioftp>=0.21; extra == 'dev'
+Requires-Dist: diskcache>=5.6; extra == 'dev'
 Requires-Dist: keyring>=24; extra == 'dev'
 Requires-Dist: pre-commit>=4; extra == 'dev'
 Requires-Dist: pytest-asyncio>=1.3; extra == 'dev'
@@ -161,11 +165,131 @@ for child in p.parent.iterdir():  # yields synchronously
 Local paths (`file://` or bare `/path`) bypass the proxy entirely and use
 `pathlib.Path` directly — zero threading overhead.
 
+### Large uploads stream automatically
+
+Writing or copying a large object never buffers the whole thing in memory — it
+streams in bounded-size parts using each backend's native mechanism (S3
+multipart, Azure block blobs, GCS resumable, SSH chunked writes, FTP data
+connection, Artifactory streamed `PUT`). This is the **default** behaviour — no
+flag, no API change:
+
+```python
+# open("wb") spills to disk past 16 MiB and uploads in parts on close
+async with AsAnyPath("s3://bucket/huge.bin").open("wb") as f:
+    f.write(data)
+
+# local -> cloud copy streams straight from the source file (async and sync)
+await AsAnyPath("/data/movie.mp4").copy("s3://bucket/movie.mp4")
+```
+
+Peak memory per upload is one part — **8 MiB by default**. Objects smaller than
+one part upload in a single request. (`write_bytes(data)` / `write_text(...)`
+take the whole payload in memory by definition — use `open("wb")` or `copy()`
+for large data.)
+
+**Tuning the memory footprint.** The part size *is* the max RAM held per
+in-flight chunk (parts upload sequentially, so there is no concurrency
+multiplier). Set it globally via env, or per call:
+
+```bash
+export ASANYPATH_UPLOAD_CHUNK_SIZE=33554432   # 32 MiB parts (bytes)
+export ASANYPATH_SPOOL_MAX_SIZE=67108864      # 64 MiB before open("wb") spills to disk
+```
+
+```python
+# per call: copy(chunk_size=...) or, for writes, open(..., buffering=...)
+await src.copy("s3://bucket/huge.bin", chunk_size=32 * 1024 * 1024)
+async with dst.open("wb", buffering=32 * 1024 * 1024) as f:
+    f.write(data)
+```
+
+Bigger parts mean fewer requests and a larger single-object ceiling (S3 allows
+10,000 parts, so 8 MiB parts cap an object at ~80 GB). Backend minimums are
+applied automatically: **S3** clamps to ≥ 5 MiB parts, **GCS** rounds down to a
+256 KiB multiple.
+
 Unknown URI schemes now construct as `UnsupportedProtocolPath` placeholders.
 Pure path operations (e.g. `.name`, `.parent`, joins) still work, while backend
 operations (e.g. `.exists()`, `.read_bytes()`, `.open()`) raise
 `UnsupportedProtocolError` on use.
 
+### Download caching (opt-in)
+
+Reading the same object or URL twice always re-downloads it — there is **no
+cache by default**, and the read path is unchanged unless you opt in, so
+non-caching callers pay zero overhead.
+
+Enable it with the optional `cache` extra (`pip install asanypath[cache]`,
+backed by [`diskcache`](https://pypi.org/project/diskcache/) — a pure-Python,
+zero-dependency store). Opt in per read with `use_cache=True`:
+
+```python
+from asanypath import AsAnyPath
+
+p = AsAnyPath("s3://bucket/model.bin")
+data = await p.read_bytes(use_cache=True)   # first call fetches + stores
+data = await p.read_bytes(use_cache=True)   # subsequent calls served from disk
+text = await p.read_text(use_cache=True)
+```
+
+Configure the location and limits once (location defaults to
+`~/.cache/asanypath`, honoring `$XDG_CACHE_HOME` and `$ASANYPATH_CACHE_DIR`):
+
+```python
+from asanypath import configure_cache
+
+configure_cache(
+    directory="/var/cache/asanypath",  # default: ~/.cache/asanypath
+    size_limit=2 * 1024**3,            # total on-disk cap in bytes (default 1 GiB, LRU)
+    ttl=3600,                          # per-entry expiry in seconds (default: none)
+    enabled=True,                      # make use_cache default to True process-wide
+)
+```
+
+Or via environment: `ASANYPATH_CACHE_DIR`, `ASANYPATH_CACHE_SIZE_LIMIT` (bytes),
+`ASANYPATH_CACHE_TTL` (seconds).
+
+Behaviour and trade-offs:
+
+- **Off by default, opt-in per call.** With `configure_cache(enabled=True)` the
+  default flips process-wide; an explicit `use_cache=False` always bypasses.
+- **Works for every remote backend** (S3, GCS, Azure, Artifactory, HTTP, SSH,
+  FTP). Local paths are never cached.
+- **Total-size bounded, LRU-evicted;** disk I/O runs in a worker thread so the
+  event loop stays responsive. Concurrent reads of the same key collapse to a
+  single fetch.
+- **Staleness is on you.** Entries are keyed by URL with no revalidation — a
+  cached object that changes upstream is served stale until it expires (`ttl`)
+  or is evicted. Prefer caching immutable/versioned objects, or set a `ttl`.
+
+### Resumable downloads
+
+`download(dst)` streams an object to a local file and **resumes an interrupted
+transfer** instead of starting over:
+
+```python
+await AsAnyPath("s3://bucket/huge.bin").download("/data/huge.bin")
+# process killed mid-download... run the same call again — it continues
+await AsAnyPath("s3://bucket/huge.bin").download("/data/huge.bin")
+```
+
+Bytes stream into `<dst>.part` with a `<dst>.part.meta` sidecar recording the
+object's ETag (or `Last-Modified`) and total size. On resume the identity is
+re-checked: if the upstream object is unchanged the download continues from the
+saved offset; if it changed, the partial is discarded and it restarts cleanly.
+On completion `<dst>.part` is atomically renamed to `dst`.
+
+```python
+# tune the range-request size; disable resume to always start fresh
+await path.download("/data/huge.bin", chunk_size=32 * 1024 * 1024, resume=False)
+```
+
+Resume needs range reads **and** a HEAD probe, so it's available on the object
+stores (S3/GCS/Azure) and Artifactory/HTTP where the server supports it.
+Backends without those (e.g. FTP) download in a single pass automatically. This
+needs no extra dependency — resume state is plain sidecar files, independent of
+the optional download cache.
+
 ## Installation
 
 Most cloud backends (S3, GCS, Azure, Artifactory, HTTP) are included in the base
@@ -438,10 +562,11 @@ Factory that returns the appropriate path instance based on the URI scheme.
 
 All path implementations support:
 
-- `open(mode, buffering)` — file-like streaming with lazy range reads
-- `copy(dst, recursive)` — copy file or tree (cross-backend)
-- `read_bytes()` / `write_bytes(data)`
-- `read_text(encoding)` / `write_text(data, encoding)`
+- `open(mode, buffering)` — file-like streaming with lazy range reads; for writes, `buffering` sets the upload part size
+- `copy(dst, recursive, chunk_size)` — copy file or tree (cross-backend); `chunk_size` tunes the upload part size
+- `download(dst, resume, chunk_size)` — stream an object to a local file, resuming an interrupted transfer
+- `read_bytes()` / `write_bytes(data)` — `read_bytes(use_cache=True)` serves from / stores in the opt-in [download cache](#download-caching-opt-in)
+- `read_text(encoding)` / `write_text(data, encoding)` — `read_text` also accepts `use_cache`
 - `exists()` / `is_file()` / `is_dir()`
 - `stat()` / `checksums()` (cloud backends)
 - `mkdir(parents, exist_ok)` / `touch(exist_ok)`

{asanypath-2.1.1 → asanypath-2.2.0}/README.md

@@ -106,11 +106,131 @@ for child in p.parent.iterdir():  # yields synchronously
 Local paths (`file://` or bare `/path`) bypass the proxy entirely and use
 `pathlib.Path` directly — zero threading overhead.
 
+### Large uploads stream automatically
+
+Writing or copying a large object never buffers the whole thing in memory — it
+streams in bounded-size parts using each backend's native mechanism (S3
+multipart, Azure block blobs, GCS resumable, SSH chunked writes, FTP data
+connection, Artifactory streamed `PUT`). This is the **default** behaviour — no
+flag, no API change:
+
+```python
+# open("wb") spills to disk past 16 MiB and uploads in parts on close
+async with AsAnyPath("s3://bucket/huge.bin").open("wb") as f:
+    f.write(data)
+
+# local -> cloud copy streams straight from the source file (async and sync)
+await AsAnyPath("/data/movie.mp4").copy("s3://bucket/movie.mp4")
+```
+
+Peak memory per upload is one part — **8 MiB by default**. Objects smaller than
+one part upload in a single request. (`write_bytes(data)` / `write_text(...)`
+take the whole payload in memory by definition — use `open("wb")` or `copy()`
+for large data.)
+
+**Tuning the memory footprint.** The part size *is* the max RAM held per
+in-flight chunk (parts upload sequentially, so there is no concurrency
+multiplier). Set it globally via env, or per call:
+
+```bash
+export ASANYPATH_UPLOAD_CHUNK_SIZE=33554432   # 32 MiB parts (bytes)
+export ASANYPATH_SPOOL_MAX_SIZE=67108864      # 64 MiB before open("wb") spills to disk
+```
+
+```python
+# per call: copy(chunk_size=...) or, for writes, open(..., buffering=...)
+await src.copy("s3://bucket/huge.bin", chunk_size=32 * 1024 * 1024)
+async with dst.open("wb", buffering=32 * 1024 * 1024) as f:
+    f.write(data)
+```
+
+Bigger parts mean fewer requests and a larger single-object ceiling (S3 allows
+10,000 parts, so 8 MiB parts cap an object at ~80 GB). Backend minimums are
+applied automatically: **S3** clamps to ≥ 5 MiB parts, **GCS** rounds down to a
+256 KiB multiple.
+
 Unknown URI schemes now construct as `UnsupportedProtocolPath` placeholders.
 Pure path operations (e.g. `.name`, `.parent`, joins) still work, while backend
 operations (e.g. `.exists()`, `.read_bytes()`, `.open()`) raise
 `UnsupportedProtocolError` on use.
 
+### Download caching (opt-in)
+
+Reading the same object or URL twice always re-downloads it — there is **no
+cache by default**, and the read path is unchanged unless you opt in, so
+non-caching callers pay zero overhead.
+
+Enable it with the optional `cache` extra (`pip install asanypath[cache]`,
+backed by [`diskcache`](https://pypi.org/project/diskcache/) — a pure-Python,
+zero-dependency store). Opt in per read with `use_cache=True`:
+
+```python
+from asanypath import AsAnyPath
+
+p = AsAnyPath("s3://bucket/model.bin")
+data = await p.read_bytes(use_cache=True)   # first call fetches + stores
+data = await p.read_bytes(use_cache=True)   # subsequent calls served from disk
+text = await p.read_text(use_cache=True)
+```
+
+Configure the location and limits once (location defaults to
+`~/.cache/asanypath`, honoring `$XDG_CACHE_HOME` and `$ASANYPATH_CACHE_DIR`):
+
+```python
+from asanypath import configure_cache
+
+configure_cache(
+    directory="/var/cache/asanypath",  # default: ~/.cache/asanypath
+    size_limit=2 * 1024**3,            # total on-disk cap in bytes (default 1 GiB, LRU)
+    ttl=3600,                          # per-entry expiry in seconds (default: none)
+    enabled=True,                      # make use_cache default to True process-wide
+)
+```
+
+Or via environment: `ASANYPATH_CACHE_DIR`, `ASANYPATH_CACHE_SIZE_LIMIT` (bytes),
+`ASANYPATH_CACHE_TTL` (seconds).
+
+Behaviour and trade-offs:
+
+- **Off by default, opt-in per call.** With `configure_cache(enabled=True)` the
+  default flips process-wide; an explicit `use_cache=False` always bypasses.
+- **Works for every remote backend** (S3, GCS, Azure, Artifactory, HTTP, SSH,
+  FTP). Local paths are never cached.
+- **Total-size bounded, LRU-evicted;** disk I/O runs in a worker thread so the
+  event loop stays responsive. Concurrent reads of the same key collapse to a
+  single fetch.
+- **Staleness is on you.** Entries are keyed by URL with no revalidation — a
+  cached object that changes upstream is served stale until it expires (`ttl`)
+  or is evicted. Prefer caching immutable/versioned objects, or set a `ttl`.
+
+### Resumable downloads
+
+`download(dst)` streams an object to a local file and **resumes an interrupted
+transfer** instead of starting over:
+
+```python
+await AsAnyPath("s3://bucket/huge.bin").download("/data/huge.bin")
+# process killed mid-download... run the same call again — it continues
+await AsAnyPath("s3://bucket/huge.bin").download("/data/huge.bin")
+```
+
+Bytes stream into `<dst>.part` with a `<dst>.part.meta` sidecar recording the
+object's ETag (or `Last-Modified`) and total size. On resume the identity is
+re-checked: if the upstream object is unchanged the download continues from the
+saved offset; if it changed, the partial is discarded and it restarts cleanly.
+On completion `<dst>.part` is atomically renamed to `dst`.
+
+```python
+# tune the range-request size; disable resume to always start fresh
+await path.download("/data/huge.bin", chunk_size=32 * 1024 * 1024, resume=False)
+```
+
+Resume needs range reads **and** a HEAD probe, so it's available on the object
+stores (S3/GCS/Azure) and Artifactory/HTTP where the server supports it.
+Backends without those (e.g. FTP) download in a single pass automatically. This
+needs no extra dependency — resume state is plain sidecar files, independent of
+the optional download cache.
+
 ## Installation
 
 Most cloud backends (S3, GCS, Azure, Artifactory, HTTP) are included in the base
@@ -383,10 +503,11 @@ Factory that returns the appropriate path instance based on the URI scheme.
 
 All path implementations support:
 
-- `open(mode, buffering)` — file-like streaming with lazy range reads
-- `copy(dst, recursive)` — copy file or tree (cross-backend)
-- `read_bytes()` / `write_bytes(data)`
-- `read_text(encoding)` / `write_text(data, encoding)`
+- `open(mode, buffering)` — file-like streaming with lazy range reads; for writes, `buffering` sets the upload part size
+- `copy(dst, recursive, chunk_size)` — copy file or tree (cross-backend); `chunk_size` tunes the upload part size
+- `download(dst, resume, chunk_size)` — stream an object to a local file, resuming an interrupted transfer
+- `read_bytes()` / `write_bytes(data)` — `read_bytes(use_cache=True)` serves from / stores in the opt-in [download cache](#download-caching-opt-in)
+- `read_text(encoding)` / `write_text(data, encoding)` — `read_text` also accepts `use_cache`
 - `exists()` / `is_file()` / `is_dir()`
 - `stat()` / `checksums()` (cloud backends)
 - `mkdir(parents, exist_ok)` / `touch(exist_ok)`

{asanypath-2.1.1 → asanypath-2.2.0}/pyproject.toml

@@ -20,7 +20,7 @@ asanypath-native = { path = "rust" }
 root = "src"
 
 [tool.hatch.build.targets.sdist]
-exclude = ["rust"]
+exclude = ["rust", "uv.lock"]
 
 [tool.hatch.metadata.hooks.custom]
 
@@ -58,6 +58,7 @@ dev = [
 cli = ["rich-click>=1.9.7"]
 ftp = ["aioftp>=0.21"]
 credstore = ["keyring>=24"]
+cache = ["diskcache>=5.6"]
 bench = [
     "asyncssh>=2.14",
     "fsspec",
@@ -71,7 +72,7 @@ bench = [
     "aiofiles",
     "requests",
 ]
-all = ["asanypath[credstore,ftp]"]
+all = ["asanypath[credstore,ftp,cache]"]
 [project.scripts]
 asanypath = "asanypath.cli:main"
 
@@ -102,7 +103,7 @@ disable_error_code = ["attr-defined", "arg-type", "assignment"]
 module = "asanypath.local"
 disallow_untyped_defs = false
 warn_return_any = false
-disable_error_code = ["type-var", "misc", "override", "return-value"]
+disable_error_code = ["type-var", "misc", "override", "return-value", "assignment", "no-redef"]
 
 [[tool.mypy.overrides]]
 module = "asanypath.common"
@@ -120,24 +121,40 @@ disable_error_code = ["override", "type-arg", "return-value", "assignment", "att
 module = ["asanypath.s3", "asanypath.gcs", "asanypath.azure", "asanypath.artifactory"]
 disallow_untyped_defs = false
 warn_return_any = false
-disable_error_code = ["no-any-return", "attr-defined", "assignment", "override"]
+disable_error_code = ["no-any-return", "attr-defined", "assignment", "override", "arg-type", "misc"]
+
+[[tool.mypy.overrides]]
+module = ["asanypath.ssh", "asanypath.ftp"]
+disallow_untyped_defs = false
+warn_return_any = false
+disable_error_code = ["attr-defined", "override", "arg-type", "misc", "no-any-return", "assignment"]
+
+[[tool.mypy.overrides]]
+module = "asanypath.sync"
+disallow_untyped_defs = false
+warn_return_any = false
+disable_error_code = ["no-any-return", "misc", "attr-defined", "arg-type"]
 
 [[tool.mypy.overrides]]
 module = "asanypath.cloud"
 disallow_untyped_defs = false
 warn_return_any = false
-disable_error_code = ["no-any-return", "arg-type"]
+disable_error_code = ["no-any-return", "arg-type", "attr-defined", "misc", "assignment"]
 
 [[tool.mypy.overrides]]
 module = "asanypath.asanypath"
 disallow_untyped_defs = false
 warn_return_any = false
-disable_error_code = ["no-any-return", "misc", "union-attr", "type-var"]
+disable_error_code = ["no-any-return", "misc", "union-attr", "type-var", "attr-defined"]
 
 [[tool.mypy.overrides]]
 module = "asanypath.batcher"
 disallow_untyped_defs = false
 
+[[tool.mypy.overrides]]
+module = "diskcache.*"
+ignore_missing_imports = true
+
 [tool.pytest.ini_options]
 asyncio_mode = "auto"
 testpaths = ["tests"]

{asanypath-2.1.1 → asanypath-2.2.0}/src/asanypath/__init__.py

@@ -13,6 +13,7 @@ from asanypath.asanypath import (
     register_protocol,
 )
 from asanypath.azure import AzurePath
+from asanypath.cache import configure_cache
 from asanypath.cloud import CloudPathMixin
 from asanypath.common import CommonPurePathMixin
 from asanypath.exceptions import (
@@ -62,6 +63,7 @@ __all__ = [
     "GCSPath",
     "AzurePath",
     "ArtifactoryPath",
+    "configure_cache",
     "register_protocol",
     "list_protocols",
     "AsAnyPathError",

asanypath-2.2.0/src/asanypath/_resume.py

@@ -0,0 +1,43 @@
+# Copyright (C) 2026 Axis Communications AB, Lund, Sweden
+#
+# Use of this source code is governed by an MIT-style
+# license that can be found in the LICENSE file or at
+# https://opensource.org/licenses/MIT.
+
+"""Sidecar state for resumable downloads.
+
+A partial download lives in ``<dst>.part`` with a ``<dst>.part.meta`` sidecar
+recording the remote object's identity (ETag/Last-Modified) and total size. On
+resume the identity is re-checked so a changed upstream object restarts cleanly
+rather than stitching mismatched bytes. State is plain files — no dependency on
+the optional download cache.
+"""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import msgspec
+
+
+class ResumeMeta(msgspec.Struct, frozen=True):
+    """Identity and size of the object a ``.part`` file is downloading."""
+
+    identity: str | None
+    size: int
+
+
+_encoder = msgspec.json.Encoder()
+_decoder = msgspec.json.Decoder(ResumeMeta)
+
+
+def read_meta(path: Path) -> ResumeMeta | None:
+    """Return the sidecar meta at *path*, or ``None`` if missing/corrupt."""
+    try:
+        return _decoder.decode(path.read_bytes())
+    except (OSError, msgspec.DecodeError):
+        return None
+
+
+def write_meta(path: Path, meta: ResumeMeta) -> None:
+    path.write_bytes(_encoder.encode(meta))

{asanypath-2.1.1 → asanypath-2.2.0}/src/asanypath/_version.py

@@ -18,7 +18,7 @@ version_tuple: tuple[int | str, ...]
 commit_id: str | None
 __commit_id__: str | None
 
-__version__ = version = '2.1.1'
-__version_tuple__ = version_tuple = (2, 1, 1)
+__version__ = version = '2.2.0'
+__version_tuple__ = version_tuple = (2, 2, 0)
 
 __commit_id__ = commit_id = None

{asanypath-2.1.1 → asanypath-2.2.0}/src/asanypath/artifactory.py

@@ -9,6 +9,9 @@
 from __future__ import annotations
 
 import json
+import os
+import shutil
+import tempfile
 from collections.abc import AsyncIterator, Mapping
 from datetime import datetime, timezone
 from os import getenv
@@ -31,11 +34,18 @@ from asanypath_native import (
     art_list_repos,
     art_put_batch,
     art_put_permission_target,
+    art_put_stream,
     art_storage_info,
 )
 
 from asanypath.cloud import CloudPathMixin
-from asanypath.options import AccessGrant, AccessPolicy, AccessPolicyPatch, BackendOptions
+from asanypath.options import (
+    UPLOAD_CHUNK_SIZE,
+    AccessGrant,
+    AccessPolicy,
+    AccessPolicyPatch,
+    BackendOptions,
+)
 
 
 def _jfrog_conf_token(base_url: str) -> str | None:
@@ -95,6 +105,9 @@ class ArtifactoryPath(CloudPathMixin):
 
     protocol: str = "art"
     _supports_range_read: bool = True
+    # Artifactory has no multipart API: objects at/above this size stream a single
+    # PUT body from disk (bounded memory) instead of buffering the whole object.
+    _STREAM_THRESHOLD = UPLOAD_CHUNK_SIZE
     _copy_batch_fn = staticmethod(art_copy_batch)
 
     @staticmethod
@@ -194,6 +207,44 @@ class ArtifactoryPath(CloudPathMixin):
     # Backend-specific operations
     # ------------------------------------------------------------------
 
+    async def _upload_buffer(
+        self, fileobj, size: int, *, backend_options=None, chunk_size: int | None = None
+    ) -> None:
+        """Upload a spooled write buffer; large objects stream a single PUT from disk."""
+        if size < (chunk_size or self._STREAM_THRESHOLD):
+            data = fileobj.read()
+            if backend_options is None:
+                await self.write_bytes(data)
+            else:
+                await self.write_bytes(data, backend_options=backend_options)
+            return
+        # Stream from a real file path when we have one (e.g. a local copy
+        # source); otherwise spill the anonymous buffer to a named temp file.
+        name = getattr(fileobj, "name", None)
+        if isinstance(name, str) and os.path.exists(name):
+            await self._stream_upload(name, size, backend_options=backend_options)
+            return
+        with tempfile.NamedTemporaryFile(delete=False) as tmp:
+            shutil.copyfileobj(fileobj, tmp, 1024 * 1024)
+            tmp_path = tmp.name
+        try:
+            await self._stream_upload(tmp_path, size, backend_options=backend_options)
+        finally:
+            os.unlink(tmp_path)
+
+    async def _stream_upload(self, file_path: str, size: int, *, backend_options=None) -> None:
+        options_json = (
+            msgspec.json.encode(backend_options).decode() if backend_options is not None else "{}"
+        )
+        await art_put_stream(
+            key=self._item_path,
+            file_path=file_path,
+            size=size,
+            options_json=options_json,
+            **self._native_kwargs,
+        )
+        type(self)._listing_cache.pop(self._listing_cache_key(), None)
+
     async def _storage_info(self) -> dict:
         """Fetch item metadata from the Artifactory storage API (cached).
 

{asanypath-2.1.1 → asanypath-2.2.0}/src/asanypath/azure.py

@@ -12,6 +12,7 @@ https://learn.microsoft.com/en-us/rest/api/storageservices/blob-service-rest-api
 
 from __future__ import annotations
 
+import base64
 from collections.abc import Mapping
 from datetime import datetime, timezone
 from os import getenv
@@ -31,11 +32,19 @@ from asanypath_native import (
     az_list_containers,
     az_presign,
     az_put_batch,
+    az_put_block,
+    az_put_block_list,
     az_put_container_acl,
 )
 
 from asanypath.cloud import CloudPathMixin
-from asanypath.options import AccessGrant, AccessPolicy, AccessPolicyPatch, BackendOptions
+from asanypath.options import (
+    UPLOAD_CHUNK_SIZE,
+    AccessGrant,
+    AccessPolicy,
+    AccessPolicyPatch,
+    BackendOptions,
+)
 
 AZURE_API_VERSION = "2023-11-03"
 
@@ -82,6 +91,8 @@ class AzurePath(CloudPathMixin):
 
     protocol: str = "az"
     _supports_range_read: bool = True
+    # Bytes per staged block; objects below one block upload in a single PUT.
+    _MULTIPART_PART_SIZE = UPLOAD_CHUNK_SIZE
     _is_dir_batch_fn = staticmethod(az_is_dir_batch)
     _copy_batch_fn = staticmethod(az_copy_batch)
 
@@ -180,6 +191,46 @@ class AzurePath(CloudPathMixin):
     # Backend-specific operations
     # ------------------------------------------------------------------
 
+    async def _upload_buffer(
+        self, fileobj, size: int, *, backend_options=None, chunk_size: int | None = None
+    ) -> None:
+        """Upload a spooled write buffer, using block blobs for large objects."""
+        part = chunk_size or self._MULTIPART_PART_SIZE
+        if size < part:
+            data = fileobj.read()
+            if backend_options is None:
+                await self.write_bytes(data)
+            else:
+                await self.write_bytes(data, backend_options=backend_options)
+            return
+        await self._block_upload(fileobj, part, backend_options=backend_options)
+
+    async def _block_upload(self, fileobj, part_size: int, *, backend_options=None) -> None:
+        """Stage fixed-size blocks then commit the list (memory bounded to one block)."""
+        options_json = (
+            msgspec.json.encode(backend_options).decode() if backend_options is not None else "{}"
+        )
+        block_ids: list[str] = []
+        index = 0
+        while True:
+            chunk = fileobj.read(part_size)
+            if not chunk:
+                break
+            # Block ids must be equal-length base64 strings.
+            block_id = base64.b64encode(f"{index:048d}".encode()).decode()
+            await az_put_block(
+                blob=self._item_path, block_id=block_id, data=chunk, **self._native_kwargs
+            )
+            block_ids.append(block_id)
+            index += 1
+        await az_put_block_list(
+            blob=self._item_path,
+            block_ids=block_ids,
+            options_json=options_json,
+            **self._native_kwargs,
+        )
+        type(self)._listing_cache.pop(self._listing_cache_key(), None)
+
     async def is_dir(self) -> bool:
         """Check if path is a directory prefix (has blobs with this prefix)."""
         return await az_is_dir(prefix=self._blob_path, **self._native_kwargs)