central-storage-platform-compute 0.1.0__tar.gz

central_storage_platform_compute-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 phasuwut
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

central_storage_platform_compute-0.1.0/PKG-INFO

@@ -0,0 +1,54 @@
+Metadata-Version: 2.4
+Name: central-storage-platform-compute
+Version: 0.1.0
+Summary: Provider-neutral compute transfer client for Central Storage Platform
+Author-email: phasuwut <phasuwut.share@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://central-storage-platform-api.phasuwut.com
+Keywords: storage,s3,transfer,resume,multipart,compute
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: System :: Archiving
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# Central Storage Platform Compute client
+
+```bash
+pip install central-storage-platform-compute
+```
+
+The package is published to PyPI, so compute nodes (RunPod, Colab, a bare GPU box) install it without access to the platform's private repositories. It has no third-party dependencies and needs Python 3.10+. To work on the client itself, clone this repository and run `pip install -e compute-python`.
+
+```python
+from central_storage_compute import ComputeClient
+
+client = ComputeClient("https://api.example.invalid", token="cpt_<show-once-token>")
+result = client.download("<file-id>", "./data.bin")
+print(result.bytes_written, result.sha256)
+
+upload = client.upload("./result.bin", destination="results/", mode="auto")
+print(upload.file_id, upload.mode, upload.sha256)
+```
+
+`download(mode="auto")` follows the transfer mode and concurrency returned by the API. Use `mode="parallel"` to request bounded HTTP Range workers; the client falls back to streaming mode when the storage endpoint does not support ranges. `upload(mode="auto")` tries the single path and switches to the token-scoped multipart path when the API requires it.
+
+The token is supplied at runtime. The client never writes it or a presigned URL to a resume manifest or log. Resume manifests contain only file identity, expected size/checksum and completed byte ranges. All upload completion calls carry a fresh `Idempotency-Key`.
+
+## Releasing
+
+`central-storage-platform-compute` is published to PyPI from this repository. Bump `__version__` in `src/central_storage_compute/__init__.py` (`pyproject.toml` reads the version from there), then tag:
+
+```bash
+git tag compute-client-v0.2.0 && git push origin compute-client-v0.2.0
+```
+
+`.github/workflows/publish-compute-client.yml` checks the tag against `__version__`, runs the tests, builds the sdist and wheel, and uploads them through PyPI Trusted Publishing — no API token is stored in the repository. PyPI never lets a version number be reused, so a bad release needs a new patch version rather than a re-upload.

central_storage_platform_compute-0.1.0/README.md

@@ -0,0 +1,32 @@
+# Central Storage Platform Compute client
+
+```bash
+pip install central-storage-platform-compute
+```
+
+The package is published to PyPI, so compute nodes (RunPod, Colab, a bare GPU box) install it without access to the platform's private repositories. It has no third-party dependencies and needs Python 3.10+. To work on the client itself, clone this repository and run `pip install -e compute-python`.
+
+```python
+from central_storage_compute import ComputeClient
+
+client = ComputeClient("https://api.example.invalid", token="cpt_<show-once-token>")
+result = client.download("<file-id>", "./data.bin")
+print(result.bytes_written, result.sha256)
+
+upload = client.upload("./result.bin", destination="results/", mode="auto")
+print(upload.file_id, upload.mode, upload.sha256)
+```
+
+`download(mode="auto")` follows the transfer mode and concurrency returned by the API. Use `mode="parallel"` to request bounded HTTP Range workers; the client falls back to streaming mode when the storage endpoint does not support ranges. `upload(mode="auto")` tries the single path and switches to the token-scoped multipart path when the API requires it.
+
+The token is supplied at runtime. The client never writes it or a presigned URL to a resume manifest or log. Resume manifests contain only file identity, expected size/checksum and completed byte ranges. All upload completion calls carry a fresh `Idempotency-Key`.
+
+## Releasing
+
+`central-storage-platform-compute` is published to PyPI from this repository. Bump `__version__` in `src/central_storage_compute/__init__.py` (`pyproject.toml` reads the version from there), then tag:
+
+```bash
+git tag compute-client-v0.2.0 && git push origin compute-client-v0.2.0
+```
+
+`.github/workflows/publish-compute-client.yml` checks the tag against `__version__`, runs the tests, builds the sdist and wheel, and uploads them through PyPI Trusted Publishing — no API token is stored in the repository. PyPI never lets a version number be reused, so a bad release needs a new patch version rather than a re-upload.

central_storage_platform_compute-0.1.0/pyproject.toml

@@ -0,0 +1,39 @@
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "central-storage-platform-compute"
+description = "Provider-neutral compute transfer client for Central Storage Platform"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+license-files = ["LICENSE"]
+authors = [{ name = "phasuwut", email = "phasuwut.share@gmail.com" }]
+keywords = ["storage", "s3", "transfer", "resume", "multipart", "compute"]
+dependencies = []
+dynamic = ["version"]
+
+classifiers = [
+  "Development Status :: 4 - Beta",
+  "Intended Audience :: Developers",
+  "Programming Language :: Python :: 3",
+  "Programming Language :: Python :: 3.10",
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Topic :: System :: Archiving",
+  "Typing :: Typed",
+]
+
+[project.urls]
+Homepage = "https://central-storage-platform-api.phasuwut.com"
+
+[tool.setuptools.dynamic]
+version = { attr = "central_storage_compute.__version__" }
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+central_storage_compute = ["py.typed"]

central_storage_platform_compute-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

central_storage_platform_compute-0.1.0/src/central_storage_compute/__init__.py

@@ -0,0 +1,5 @@
+from .client import ComputeApiError, ComputeClient, DownloadResult, UploadResult
+
+__version__ = "0.1.0"
+
+__all__ = ["ComputeApiError", "ComputeClient", "DownloadResult", "UploadResult", "__version__"]

central_storage_platform_compute-0.1.0/src/central_storage_compute/client.py

@@ -0,0 +1,361 @@
+from __future__ import annotations
+
+import hashlib
+import json
+import base64
+import mimetypes
+import os
+import tempfile
+import time
+import urllib.error
+import urllib.request
+from concurrent.futures import ThreadPoolExecutor, as_completed
+from dataclasses import dataclass
+from pathlib import Path
+from threading import Lock
+from typing import BinaryIO
+from uuid import uuid4
+
+
+@dataclass(frozen=True)
+class DownloadResult:
+    file_id: str
+    destination: Path
+    bytes_written: int
+    sha256: str
+    mode: str = "normal"
+
+
+@dataclass(frozen=True)
+class UploadResult:
+    upload_id: str
+    file_id: str
+    source: Path
+    bytes_read: int
+    sha256: str
+    mode: str
+
+
+class ComputeApiError(RuntimeError):
+    """Safe API error that never includes a bearer token or presigned URL."""
+
+    def __init__(self, status: int, code: str, message: str) -> None:
+        super().__init__(f"Compute API request failed ({status}): {code}")
+        self.status = status
+        self.code = code
+        self.message = message
+
+
+class _RangeUnsupported(RuntimeError):
+    pass
+
+
+class ComputeClient:
+    """Provider-neutral client; credentials and presigned URLs stay out of manifests/logs."""
+
+    def __init__(self, api_url: str, token: str, timeout: float = 30.0, retry_count: int = 3) -> None:
+        if not token.startswith("cpt_"):
+            raise ValueError("Compute token must use the cpt_ prefix")
+        self.api_url = api_url.rstrip("/")
+        self._token = token
+        self.timeout = timeout
+        self.retry_count = max(1, retry_count)
+
+    def download(
+        self,
+        file_id: str,
+        destination: str | os.PathLike[str],
+        *,
+        verify_sha256: str | None = None,
+        mode: str = "auto",
+        max_connections: int | None = None,
+        resume: bool = True,
+    ) -> DownloadResult:
+        if mode not in {"auto", "normal", "parallel"}:
+            raise ValueError("mode must be auto, normal, or parallel")
+        payload = self._request("POST", f"/api/v1/compute/files/{file_id}/download")
+        target = Path(destination)
+        target.parent.mkdir(parents=True, exist_ok=True)
+        expected_size = _as_int(payload.get("sizeBytes"))
+        configured_mode = str(payload.get("transferMode") or "NORMAL").lower()
+        connections = max(1, min(int(max_connections or payload.get("maxConcurrency") or 1), 32))
+        selected_mode = "parallel" if mode == "parallel" or (mode == "auto" and configured_mode == "parallel") else "normal"
+        if expected_size is None or expected_size < 1 or connections < 2:
+            selected_mode = "normal"
+        if selected_mode == "parallel":
+            try:
+                return self._download_parallel(file_id, target, str(payload["url"]), expected_size, verify_sha256, connections, resume)
+            except _RangeUnsupported:
+                selected_mode = "normal"
+        result = self._download_normal(file_id, target, str(payload["url"]), expected_size, verify_sha256)
+        return DownloadResult(result.file_id, result.destination, result.bytes_written, result.sha256, selected_mode)
+
+    def benchmark(self, file_id: str, *, sample_path: str | os.PathLike[str] | None = None, max_connections: int | None = None) -> dict[str, float | int | str | list[dict[str, float | int | str]]]:
+        allowed = max(1, min(int(max_connections or 8), 32))
+        candidates = [connections for connections in (1, 4, 8) if connections <= allowed]
+        runs: list[dict[str, float | int | str]] = []
+        for connections in candidates:
+            scratch = Path(sample_path) if sample_path else Path(tempfile.gettempdir()) / f"csp-benchmark-{file_id}-{connections}"
+            started = time.perf_counter()
+            result = self.download(file_id, scratch, mode="normal" if connections == 1 else "parallel", max_connections=connections, resume=False)
+            duration = max(time.perf_counter() - started, 0.000001)
+            runs.append({"connections": connections, "bytes": result.bytes_written, "seconds": duration, "bytesPerSecond": result.bytes_written / duration, "mode": result.mode, "sha256": result.sha256})
+            if sample_path is None:
+                scratch.unlink(missing_ok=True)
+        recommended = max(runs, key=lambda run: (float(run["bytesPerSecond"]), -int(run["connections"])))
+        return {"fileId": file_id, "bytes": int(recommended["bytes"]), "seconds": float(recommended["seconds"]), "bytesPerSecond": float(recommended["bytesPerSecond"]), "sha256": str(recommended["sha256"]), "mode": str(recommended["mode"]), "recommendedConcurrency": int(recommended["connections"]), "runs": runs}
+
+    def upload(
+        self,
+        source: str | os.PathLike[str],
+        *,
+        destination: str | None = None,
+        mime_type: str | None = None,
+        mode: str = "auto",
+        max_connections: int | None = None,
+        include_checksum: bool = True,
+    ) -> UploadResult:
+        if mode not in {"auto", "single", "multipart"}:
+            raise ValueError("mode must be auto, single, or multipart")
+        path = Path(source)
+        size = path.stat().st_size
+        digest = _sha256_file(path)
+        checksum = _base64_sha256(path) if include_checksum else None
+        body: dict[str, object] = {"filename": path.name, "sizeBytes": size}
+        if destination:
+            body["destination"] = destination
+        if mime_type or mimetypes.guess_type(path.name)[0]:
+            body["mimeType"] = mime_type or mimetypes.guess_type(path.name)[0]
+        if checksum:
+            body["checksum"] = checksum
+        payload: dict[str, object] | None = None
+        if mode in {"auto", "single"}:
+            try:
+                payload = self._request("POST", "/api/v1/compute/uploads/create", body)
+            except ComputeApiError as error:
+                if mode == "single" or error.code != "MULTIPART_REQUIRED":
+                    raise
+        if payload is not None and str(payload.get("mode")) == "single":
+            self._put_file(str(payload["uploadUrl"]), path, str(payload.get("headers", {}).get("content-type", "")), checksum)
+            completed = self._request("POST", f"/api/v1/compute/uploads/{payload['uploadId']}/complete", {"checksum": checksum} if checksum else {}, idempotency_key=str(uuid4()))
+            return UploadResult(str(payload["uploadId"]), str(completed["fileId"]), path, size, digest, "single")
+        multipart = payload if payload is not None else self._request("POST", "/api/v1/compute/uploads/multipart/create", body)
+        return self._upload_multipart(path, multipart, checksum, digest, max_connections)
+
+    def _upload_multipart(self, path: Path, payload: dict[str, object], checksum: str | None, digest: str, max_connections: int | None) -> UploadResult:
+        upload_id = str(payload["uploadId"])
+        part_size = _as_int(payload.get("partSizeBytes")) or 8 * 1024 * 1024
+        total_parts = _as_int(payload.get("totalParts")) or ((path.stat().st_size + part_size - 1) // part_size)
+        concurrency = max(1, min(int(max_connections or payload.get("maxConcurrency") or 4), 32))
+        signed: dict[int, str] = {}
+        for offset in range(0, total_parts, 500):
+            numbers = list(range(offset + 1, min(total_parts, offset + 500) + 1))
+            response = self._request("POST", f"/api/v1/compute/uploads/{upload_id}/multipart/parts/sign", {"partNumbers": numbers})
+            for item in response.get("parts", []):
+                signed[int(item["partNumber"])] = str(item["uploadUrl"])
+
+        def put(part_number: int) -> tuple[int, str]:
+            start = (part_number - 1) * part_size
+            length = min(part_size, path.stat().st_size - start)
+            last_error: Exception | None = None
+            for attempt in range(self.retry_count):
+                try:
+                    with path.open("rb") as source:
+                        source.seek(start)
+                        body = source.read(length)
+                    etag = self._put_bytes(signed[part_number], body)
+                    return part_number, etag
+                except (urllib.error.URLError, TimeoutError, OSError) as error:
+                    last_error = error
+                    if attempt + 1 < self.retry_count:
+                        time.sleep(min(2**attempt, 4))
+            raise RuntimeError(f"Multipart part {part_number} failed") from last_error
+
+        try:
+            parts: list[tuple[int, str]] = []
+            with ThreadPoolExecutor(max_workers=concurrency) as pool:
+                futures = [pool.submit(put, part_number) for part_number in range(1, total_parts + 1)]
+                for future in as_completed(futures):
+                    parts.append(future.result())
+            completed = self._request("POST", f"/api/v1/compute/uploads/{upload_id}/multipart/complete", {"parts": [{"partNumber": number, "etag": etag} for number, etag in sorted(parts)], **({"checksum": checksum} if checksum else {})}, idempotency_key=str(uuid4()))
+        except Exception:
+            self._request("POST", f"/api/v1/compute/uploads/{upload_id}/multipart/abort", {})
+            raise
+        return UploadResult(upload_id, str(completed["fileId"]), path, path.stat().st_size, digest, "multipart")
+
+    def _put_file(self, url: str, path: Path, content_type: str, checksum: str | None) -> None:
+        headers = {"Content-Length": str(path.stat().st_size)}
+        if content_type:
+            headers["Content-Type"] = content_type
+        if checksum:
+            headers["x-amz-checksum-sha256"] = checksum
+        request = urllib.request.Request(url, method="PUT", headers=headers)
+        with path.open("rb") as source:
+            request.data = source
+            try:
+                with urllib.request.urlopen(request, timeout=self.timeout) as response:
+                    if response.status < 200 or response.status >= 300:
+                        raise RuntimeError(f"Compute upload failed ({response.status})")
+            except urllib.error.HTTPError as error:
+                raise RuntimeError(f"Compute upload failed ({error.code})") from error
+
+    @staticmethod
+    def _put_bytes(url: str, body: bytes) -> str:
+        request = urllib.request.Request(url, data=body, method="PUT", headers={"Content-Length": str(len(body))})
+        try:
+            with urllib.request.urlopen(request) as response:
+                if response.status < 200 or response.status >= 300:
+                    raise RuntimeError(f"Compute multipart upload failed ({response.status})")
+                return response.headers.get("ETag", "").strip('"')
+        except urllib.error.HTTPError as error:
+            raise RuntimeError(f"Compute multipart upload failed ({error.code})") from error
+
+    def _download_normal(self, file_id: str, target: Path, url: str, expected_size: int | None, verify_sha256: str | None) -> DownloadResult:
+        last_error: Exception | None = None
+        for attempt in range(self.retry_count):
+            try:
+                if attempt:
+                    payload = self._request("POST", f"/api/v1/compute/files/{file_id}/download")
+                    url = str(payload["url"])
+                digest = hashlib.sha256()
+                with urllib.request.urlopen(urllib.request.Request(url), timeout=self.timeout) as response, target.open("wb") as output:
+                    count = self._stream(response, output, digest)
+                self._verify_size_and_checksum(count, digest.hexdigest(), expected_size, verify_sha256)
+                return DownloadResult(file_id=file_id, destination=target, bytes_written=count, sha256=digest.hexdigest())
+            except (urllib.error.URLError, TimeoutError, OSError, ComputeApiError) as error:
+                last_error = error
+                if attempt + 1 < self.retry_count:
+                    time.sleep(min(2**attempt, 4))
+        raise RuntimeError("Compute download failed after retries") from last_error
+
+    def _download_parallel(self, file_id: str, target: Path, url: str, expected_size: int, verify_sha256: str | None, connections: int, resume: bool) -> DownloadResult:
+        manifest = target.with_name(f".{target.name}.csp-resume.json")
+        chunk_size = max(1 * 1024 * 1024, min(64 * 1024 * 1024, (expected_size + connections * 8 - 1) // (connections * 8)))
+        chunks = [(start, min(start + chunk_size, expected_size) - 1) for start in range(0, expected_size, chunk_size)]
+        completed = self._load_manifest(manifest, file_id, expected_size, verify_sha256) if resume else set()
+        if not target.exists() or target.stat().st_size != expected_size:
+            with target.open("wb") as output:
+                output.truncate(expected_size)
+            completed = set()
+        write_lock = Lock()
+
+        def fetch(chunk: tuple[int, int]) -> tuple[int, int]:
+            if chunk in completed:
+                return chunk
+            start, end = chunk
+            last_error: Exception | None = None
+            for attempt in range(self.retry_count):
+                try:
+                    request = urllib.request.Request(url, headers={"Range": f"bytes={start}-{end}"})
+                    with urllib.request.urlopen(request, timeout=self.timeout) as response:
+                        if response.status != 206:
+                            raise _RangeUnsupported()
+                        content_range = response.headers.get("Content-Range", "")
+                        if not content_range.startswith(f"bytes {start}-{end}/"):
+                            raise _RangeUnsupported()
+                        body = response.read(end - start + 1)
+                    if len(body) != end - start + 1:
+                        raise IOError("Range response length mismatch")
+                    with write_lock, target.open("r+b") as output:
+                        output.seek(start)
+                        output.write(body)
+                    return chunk
+                except _RangeUnsupported:
+                    raise
+                except (urllib.error.URLError, TimeoutError, OSError) as error:
+                    last_error = error
+                    if attempt + 1 < self.retry_count:
+                        time.sleep(min(2**attempt, 4))
+            raise RuntimeError("Range download failed after retries") from last_error
+
+        with ThreadPoolExecutor(max_workers=connections) as pool:
+            futures = [pool.submit(fetch, chunk) for chunk in chunks if chunk not in completed]
+            for future in as_completed(futures):
+                completed.add(future.result())
+                self._save_manifest(manifest, file_id, expected_size, verify_sha256, completed)
+        digest = hashlib.sha256()
+        count = 0
+        with target.open("rb") as output:
+            count = self._stream(output, None, digest)
+        self._verify_size_and_checksum(count, digest.hexdigest(), expected_size, verify_sha256)
+        manifest.unlink(missing_ok=True)
+        return DownloadResult(file_id=file_id, destination=target, bytes_written=count, sha256=digest.hexdigest(), mode="parallel")
+
+    @staticmethod
+    def _stream(response: BinaryIO, output: BinaryIO | None, digest: "hashlib._Hash") -> int:
+        count = 0
+        while chunk := response.read(1024 * 1024):
+            if output is not None:
+                output.write(chunk)
+            digest.update(chunk)
+            count += len(chunk)
+        return count
+
+    @staticmethod
+    def _verify_size_and_checksum(count: int, digest: str, expected_size: int | None, verify_sha256: str | None) -> None:
+        if expected_size is not None and count != expected_size:
+            raise ValueError(f"Downloaded file size does not match expected size ({expected_size})")
+        if verify_sha256 and digest.lower() != verify_sha256.lower():
+            raise ValueError("Downloaded file checksum does not match")
+
+    @staticmethod
+    def _load_manifest(path: Path, file_id: str, expected_size: int, checksum: str | None) -> set[tuple[int, int]]:
+        try:
+            value = json.loads(path.read_text())
+            if value.get("fileId") != file_id or value.get("sizeBytes") != expected_size or value.get("checksum") != checksum:
+                return set()
+            return {(int(item[0]), int(item[1])) for item in value.get("completed", [])}
+        except (OSError, ValueError, TypeError, KeyError):
+            return set()
+
+    @staticmethod
+    def _save_manifest(path: Path, file_id: str, expected_size: int, checksum: str | None, completed: set[tuple[int, int]]) -> None:
+        value = {"fileId": file_id, "sizeBytes": expected_size, "checksum": checksum, "completed": sorted([list(item) for item in completed])}
+        path.write_text(json.dumps(value, separators=(",", ":")))
+
+    def _request(self, method: str, path: str, body: dict[str, object] | None = None, idempotency_key: str | None = None) -> dict[str, object]:
+        headers = {"Authorization": f"Bearer {self._token}", "Accept": "application/json"}
+        request = urllib.request.Request(f"{self.api_url}{path}", method=method, headers=headers)
+        if body is not None:
+            request.data = json.dumps(body).encode()
+            request.add_header("Content-Type", "application/json")
+        if idempotency_key:
+            request.add_header("Idempotency-Key", idempotency_key)
+        try:
+            with urllib.request.urlopen(request, timeout=self.timeout) as response:
+                value = json.load(response)
+                if not isinstance(value, dict):
+                    raise RuntimeError("Compute API returned an invalid response")
+                return value
+        except urllib.error.HTTPError as error:
+            try:
+                payload = json.load(error)
+                envelope = payload.get("error", {}) if isinstance(payload, dict) else {}
+                code = str(envelope.get("code", "HTTP_ERROR")) if isinstance(envelope, dict) else "HTTP_ERROR"
+                message = str(envelope.get("message", "Request failed")) if isinstance(envelope, dict) else "Request failed"
+            except (ValueError, OSError):
+                code, message = "HTTP_ERROR", "Request failed"
+            raise ComputeApiError(error.code, code, message) from error
+
+
+def _as_int(value: object) -> int | None:
+    try:
+        return int(value) if value is not None else None
+    except (TypeError, ValueError):
+        return None
+
+
+def _sha256_file(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as source:
+        while chunk := source.read(1024 * 1024):
+            digest.update(chunk)
+    return digest.hexdigest()
+
+
+def _base64_sha256(path: Path) -> str:
+    digest = hashlib.sha256()
+    with path.open("rb") as source:
+        while chunk := source.read(1024 * 1024):
+            digest.update(chunk)
+    return base64.b64encode(digest.digest()).decode("ascii")

central_storage_platform_compute-0.1.0/src/central_storage_platform_compute.egg-info/PKG-INFO

@@ -0,0 +1,54 @@
+Metadata-Version: 2.4
+Name: central-storage-platform-compute
+Version: 0.1.0
+Summary: Provider-neutral compute transfer client for Central Storage Platform
+Author-email: phasuwut <phasuwut.share@gmail.com>
+License-Expression: MIT
+Project-URL: Homepage, https://central-storage-platform-api.phasuwut.com
+Keywords: storage,s3,transfer,resume,multipart,compute
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: System :: Archiving
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# Central Storage Platform Compute client
+
+```bash
+pip install central-storage-platform-compute
+```
+
+The package is published to PyPI, so compute nodes (RunPod, Colab, a bare GPU box) install it without access to the platform's private repositories. It has no third-party dependencies and needs Python 3.10+. To work on the client itself, clone this repository and run `pip install -e compute-python`.
+
+```python
+from central_storage_compute import ComputeClient
+
+client = ComputeClient("https://api.example.invalid", token="cpt_<show-once-token>")
+result = client.download("<file-id>", "./data.bin")
+print(result.bytes_written, result.sha256)
+
+upload = client.upload("./result.bin", destination="results/", mode="auto")
+print(upload.file_id, upload.mode, upload.sha256)
+```
+
+`download(mode="auto")` follows the transfer mode and concurrency returned by the API. Use `mode="parallel"` to request bounded HTTP Range workers; the client falls back to streaming mode when the storage endpoint does not support ranges. `upload(mode="auto")` tries the single path and switches to the token-scoped multipart path when the API requires it.
+
+The token is supplied at runtime. The client never writes it or a presigned URL to a resume manifest or log. Resume manifests contain only file identity, expected size/checksum and completed byte ranges. All upload completion calls carry a fresh `Idempotency-Key`.
+
+## Releasing
+
+`central-storage-platform-compute` is published to PyPI from this repository. Bump `__version__` in `src/central_storage_compute/__init__.py` (`pyproject.toml` reads the version from there), then tag:
+
+```bash
+git tag compute-client-v0.2.0 && git push origin compute-client-v0.2.0
+```
+
+`.github/workflows/publish-compute-client.yml` checks the tag against `__version__`, runs the tests, builds the sdist and wheel, and uploads them through PyPI Trusted Publishing — no API token is stored in the repository. PyPI never lets a version number be reused, so a bad release needs a new patch version rather than a re-upload.

central_storage_platform_compute-0.1.0/src/central_storage_platform_compute.egg-info/SOURCES.txt

@@ -0,0 +1,11 @@
+LICENSE
+README.md
+pyproject.toml
+src/central_storage_compute/__init__.py
+src/central_storage_compute/client.py
+src/central_storage_compute/py.typed
+src/central_storage_platform_compute.egg-info/PKG-INFO
+src/central_storage_platform_compute.egg-info/SOURCES.txt
+src/central_storage_platform_compute.egg-info/dependency_links.txt
+src/central_storage_platform_compute.egg-info/top_level.txt
+tests/test_client.py

central_storage_platform_compute-0.1.0/src/central_storage_platform_compute.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

central_storage_platform_compute-0.1.0/src/central_storage_platform_compute.egg-info/top_level.txt

@@ -0,0 +1 @@
+central_storage_compute

central_storage_platform_compute-0.1.0/tests/test_client.py

@@ -0,0 +1,74 @@
+import json
+import tempfile
+import unittest
+from pathlib import Path
+from unittest.mock import Mock, patch
+
+from central_storage_compute.client import ComputeClient
+
+
+class ComputeClientTests(unittest.TestCase):
+    def test_manifest_contains_identity_only(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            path = Path(directory) / "resume.json"
+            ComputeClient._save_manifest(path, "file-1", 10, "checksum", {(0, 9)})
+            value = json.loads(path.read_text())
+            self.assertEqual(value, {"fileId": "file-1", "sizeBytes": 10, "checksum": "checksum", "completed": [[0, 9]]})
+            self.assertNotIn("cpt_secret", path.read_text())
+            self.assertNotIn("X-Amz-Signature", path.read_text())
+
+    def test_manifest_is_discarded_when_identity_changes(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            path = Path(directory) / "resume.json"
+            ComputeClient._save_manifest(path, "file-1", 10, None, {(0, 9)})
+            self.assertEqual(ComputeClient._load_manifest(path, "file-2", 10, None), set())
+            self.assertEqual(ComputeClient._load_manifest(path, "file-1", 11, None), set())
+
+    def test_token_prefix_is_required(self) -> None:
+        with self.assertRaises(ValueError):
+            ComputeClient("https://api.example.invalid", "secret")
+
+    def test_single_upload_uses_compute_namespace_and_completion_key(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            source = Path(directory) / "output.txt"
+            source.write_text("hello")
+            client = ComputeClient("https://api.example.invalid", "cpt_token.secret")
+            client._request = Mock(side_effect=[{"uploadId": "upload-1", "mode": "single", "uploadUrl": "https://s3.example/put", "headers": {}}, {"fileId": "file-1"}])
+            client._put_file = Mock()
+
+            result = client.upload(source, mode="single")
+
+            self.assertEqual(result.file_id, "file-1")
+            self.assertEqual(result.bytes_read, 5)
+            self.assertEqual(result.mode, "single")
+            self.assertEqual(client._request.call_args_list[1].args[1], "/api/v1/compute/uploads/upload-1/complete")
+            self.assertIn("idempotency_key", client._request.call_args_list[1].kwargs)
+
+    def test_multipart_upload_retries_parts_and_completes_in_order(self) -> None:
+        with tempfile.TemporaryDirectory() as directory:
+            source = Path(directory) / "output.bin"
+            source.write_bytes(b"abcde")
+            client = ComputeClient("https://api.example.invalid", "cpt_token.secret")
+            client._request = Mock(side_effect=[{"parts": [{"partNumber": 1, "uploadUrl": "https://s3.example/1"}, {"partNumber": 2, "uploadUrl": "https://s3.example/2"}]}, {"fileId": "file-1"}])
+            client._put_bytes = Mock(side_effect=lambda url, _body: "etag-1" if url.endswith("/1") else "etag-2")
+
+            result = client._upload_multipart(source, {"uploadId": "upload-1", "partSizeBytes": 3, "totalParts": 2, "maxConcurrency": 2}, None, "digest", 2)
+
+            self.assertEqual(result.mode, "multipart")
+            completion = client._request.call_args_list[1].args[2]
+            self.assertEqual([part["partNumber"] for part in completion["parts"]], [1, 2])
+            self.assertEqual([part["etag"] for part in completion["parts"]], ["etag-1", "etag-2"])
+
+    def test_benchmark_respects_connection_limit_and_prefers_fewer_on_tie(self) -> None:
+        client = ComputeClient("https://api.example.invalid", "cpt_token.secret")
+        client.download = Mock(side_effect=lambda _file_id, destination, **kwargs: type("Result", (), {"file_id": "file-1", "destination": Path(destination), "bytes_written": 100, "sha256": "digest", "mode": "normal" if kwargs["max_connections"] == 1 else "parallel"})())
+
+        with patch("central_storage_compute.client.time.perf_counter", side_effect=[0.0, 1.0, 10.0, 11.0]):
+            result = client.benchmark("file-1", max_connections=4)
+
+        self.assertEqual([run["connections"] for run in result["runs"]], [1, 4])
+        self.assertEqual(result["recommendedConcurrency"], 1)
+
+
+if __name__ == "__main__":
+    unittest.main()