py-storebridge 1.0.0__tar.gz

py_storebridge-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Asif Khan
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

py_storebridge-1.0.0/PKG-INFO

@@ -0,0 +1,562 @@
+Metadata-Version: 2.4
+Name: py_storebridge
+Version: 1.0.0
+Summary: A unified, production-ready cloud storage SDK for Python. Switch providers with one config line.
+Author-email: Asif Khan <ak2311581@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/ak9347128658/py_storebridge
+Project-URL: Documentation, https://github.com/ak9347128658/py_storebridge#readme
+Project-URL: Repository, https://github.com/ak9347128658/py_storebridge
+Project-URL: Issues, https://github.com/ak9347128658/py_storebridge/issues
+Project-URL: Changelog, https://github.com/ak9347128658/py_storebridge/blob/main/CHANGELOG.md
+Keywords: storage,s3,gcs,azure,cloudflare,r2,minio,supabase,firebase,appwrite,backblaze,digitalocean,spaces,cloud,upload,pydantic
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Internet :: File Transfer Protocol (FTP)
+Classifier: Typing :: Typed
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic>=2.5
+Provides-Extra: s3
+Requires-Dist: boto3>=1.34.0; extra == "s3"
+Provides-Extra: r2
+Requires-Dist: boto3>=1.34.0; extra == "r2"
+Provides-Extra: gcs
+Requires-Dist: google-cloud-storage>=2.14.0; extra == "gcs"
+Provides-Extra: azure
+Requires-Dist: azure-storage-blob>=12.19.0; extra == "azure"
+Provides-Extra: spaces
+Requires-Dist: boto3>=1.34.0; extra == "spaces"
+Provides-Extra: b2
+Requires-Dist: boto3>=1.34.0; extra == "b2"
+Provides-Extra: minio
+Requires-Dist: boto3>=1.34.0; extra == "minio"
+Provides-Extra: supabase
+Requires-Dist: httpx>=0.27.0; extra == "supabase"
+Provides-Extra: firebase
+Requires-Dist: google-cloud-storage>=2.14.0; extra == "firebase"
+Requires-Dist: google-auth>=2.23.0; extra == "firebase"
+Provides-Extra: appwrite
+Requires-Dist: httpx>=0.27.0; extra == "appwrite"
+Provides-Extra: all
+Requires-Dist: boto3>=1.34.0; extra == "all"
+Requires-Dist: google-cloud-storage>=2.14.0; extra == "all"
+Requires-Dist: google-auth>=2.23.0; extra == "all"
+Requires-Dist: azure-storage-blob>=12.19.0; extra == "all"
+Requires-Dist: httpx>=0.27.0; extra == "all"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pytest-cov>=5.0; extra == "dev"
+Provides-Extra: build
+Requires-Dist: build>=1.2; extra == "build"
+Requires-Dist: twine>=5.0; extra == "build"
+Dynamic: license-file
+
+# py_storebridge
+
+**A unified, production-ready cloud storage SDK for Python.**
+
+Switch storage providers by changing a single config line — no code changes required.
+
+Python port of [storebridge](https://www.npmjs.com/package/storebridge) (Node.js / TypeScript), with Pydantic validation and **cross-provider transfer** of files, folders, and entire buckets.
+
+---
+
+## Features
+
+- **10 storage providers** — one consistent API
+- **Pydantic v2** — every config object and public operation is validated
+- **Retry system** — exponential backoff with full jitter
+- **Middleware** — composable upload pipeline (metadata injection, key prefixes, …)
+- **Hooks** — lifecycle events (before/after upload, delete, copy, move, transfer)
+- **Multipart upload** — high-level helper + low-level lifecycle API
+- **Signed / presigned URLs** — GET, PUT, DELETE
+- **Streaming** — upload from paths / file objects, download as streams or bytes
+- **MIME auto-detection** — from file extension
+- **CDN / custom domain** support
+- **Lazy provider loading** — only the SDK you use is imported
+- **Cross-provider transfer** — move existing files, folders, or a whole bucket from one provider to another
+
+---
+
+## Provider support
+
+| Provider | Extra | Install |
+| --- | --- | --- |
+| AWS S3 | `s3` | `pip install py_storebridge[s3]` |
+| Cloudflare R2 | `r2` | `pip install py_storebridge[r2]` |
+| Google Cloud Storage | `gcs` | `pip install py_storebridge[gcs]` |
+| Azure Blob Storage | `azure` | `pip install py_storebridge[azure]` |
+| DigitalOcean Spaces | `spaces` | `pip install py_storebridge[spaces]` |
+| Backblaze B2 | `b2` | `pip install py_storebridge[b2]` |
+| MinIO | `minio` | `pip install py_storebridge[minio]` |
+| Supabase Storage | `supabase` | `pip install py_storebridge[supabase]` |
+| Firebase Storage | `firebase` | `pip install py_storebridge[firebase]` |
+| Appwrite Storage | `appwrite` | `pip install py_storebridge[appwrite]` |
+
+Provider SDKs are optional extras. The core package depends only on **Pydantic**.
+
+```bash
+pip install py_storebridge
+pip install py_storebridge[s3]
+pip install py_storebridge[all]
+```
+
+---
+
+## Feature matrix
+
+| Feature | S3 | R2 | GCS | Azure | Spaces | B2 | MinIO | Supabase | Firebase | Appwrite |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| Upload | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| Delete | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| Exists | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| Metadata | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| List | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| Signed URLs | yes | yes | yes | yes | yes | yes | yes | yes | yes | limited |
+| Copy / move | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| Stream upload | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| Multipart | native | native | fallback | native blocks | native | native | native | fallback | fallback | fallback |
+| ACL / visibility | yes | limited | yes | limited | yes | limited | limited | yes | yes | yes |
+| CDN / custom domain | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+| **Transfer to another provider** | yes | yes | yes | yes | yes | yes | yes | yes | yes | yes |
+
+---
+
+## Quick start
+
+```python
+from py_storebridge import create_storage
+
+storage = create_storage({
+    "provider": "s3",
+    "credentials": {
+        "access_key_id": "AKI…",
+        "secret_access_key": "…",
+        "region": "us-east-1",
+        "bucket": "my-bucket",
+    },
+    "visibility": "private",
+})
+
+result = storage.upload(key="users/profile.png", file=image_bytes, content_type="image/png")
+print(result.url)
+
+url = storage.get_signed_url("users/profile.png", expires_in=3600)
+exists = storage.exists("users/profile.png")
+storage.delete(key="users/profile.png")
+```
+
+CamelCase credential keys from the Node.js SDK (`accessKeyId`, `secretAccessKey`, …) are accepted as well.
+
+---
+
+## Configuration
+
+All providers share the same top-level shape. Credentials differ per provider. Pydantic validates the payload before any SDK is loaded.
+
+```python
+from py_storebridge import create_storage
+
+storage = create_storage(
+    provider="s3",
+    credentials={
+        "access_key_id": "…",
+        "secret_access_key": "…",
+        "region": "us-east-1",
+        "bucket": "my-bucket",
+    },
+    visibility="private",          # default visibility
+    retry={"max_retries": 3, "base_delay": 200, "max_delay": 10000},
+    log_level="warn",              # debug | info | warn | error | silent
+)
+```
+
+Retry delays are milliseconds, matching the Node.js package.
+
+---
+
+## API
+
+### upload
+
+```python
+result = storage.upload(
+    key="images/photo.jpg",
+    file=buffer,                    # bytes | str | pathlib.Path | BinaryIO
+    content_type="image/jpeg",      # optional — detected from the key
+    visibility="public",
+    metadata={"author": "alice"},
+    cache_control="max-age=31536000",
+    on_progress=lambda uploaded, total: print(uploaded, total),
+)
+# result.key, result.url, result.etag, result.size, result.provider
+
+storage.upload_file("local/photo.jpg", key="images/photo.jpg")
+```
+
+### delete / exists / metadata / list
+
+```python
+storage.delete(key="images/photo.jpg")
+storage.exists("images/photo.jpg")
+meta = storage.get_metadata("images/photo.jpg")
+
+page = storage.list(prefix="images/", limit=50, cursor=None, delimiter="/")
+for entry in storage.iter_files(prefix="images/"):
+    print(entry.key, entry.size)
+
+storage.delete_prefix("tmp/")
+```
+
+### URLs
+
+```python
+storage.get_url("public/logo.png")
+storage.get_signed_url("private/report.pdf", expires_in=900)
+storage.get_signed_url("uploads/raw.csv", method="PUT", expires_in=300, content_type="text/csv")
+```
+
+### copy / move / rename
+
+These operate **inside one provider**.
+
+```python
+storage.copy(source_key="original.png", destination_key="copies/backup.png")
+storage.move(source_key="temp/upload.png", destination_key="final/upload.png")
+storage.rename("old-name.png", "new-name.png")
+```
+
+### streams and buffers
+
+```python
+stream = storage.get_stream("videos/intro.mp4")
+data = storage.get_buffer("config/settings.json")
+storage.download_file("config/settings.json", "settings.json")
+```
+
+### multipart
+
+```python
+result = storage.upload_multipart({"key": "large/dataset.parquet", "file": payload}, chunk_size=10 * 1024 * 1024)
+
+session = storage.create_multipart_upload("huge.bin", "application/octet-stream")
+parts = [
+    storage.upload_part(upload_id=session.upload_id, key=session.key, part_number=i + 1, body=chunk)
+    for i, chunk in enumerate(chunks)
+]
+storage.complete_multipart_upload(session, parts)
+# storage.abort_multipart_upload(session)
+```
+
+---
+
+## Transfer between providers
+
+This is the capability the Node.js `storebridge` package does not have. `transfer()` copies (or moves) **existing** objects from one provider to another: a single file, a folder prefix, an explicit key list, or an entire bucket.
+
+```python
+from py_storebridge import create_storage, transfer
+
+s3 = create_storage({"provider": "s3", "credentials": {…}})
+gcs = create_storage({"provider": "gcs", "credentials": {…}})
+
+# One file
+transfer(s3, gcs, source_key="users/profile.png", destination_key="migrated/profile.png")
+
+# A folder (every key under the prefix)
+transfer(
+    s3,
+    gcs,
+    prefix="images/",
+    destination_prefix="backup/images/",
+    concurrency=8,
+    preserve_metadata=True,
+    preserve_visibility=True,
+)
+
+# Entire bucket
+result = s3.transfer_to(gcs)
+
+# Move (copy then delete from the source)
+transfer(s3, gcs, prefix="uploads/", delete_source=True)
+
+# Dry run
+plan = transfer(s3, gcs, prefix="images/", dry_run=True)
+print(plan.planned, plan.items)
+
+# Skip objects that already exist on the destination
+transfer(s3, gcs, prefix="images/", skip_existing=True, continue_on_error=True)
+```
+
+`TransferResult` includes `transferred`, `skipped`, `failed`, `bytes_transferred`, and per-item status. Set `continue_on_error=False` to stop on the first failure (`TransferError` then carries the partial result).
+
+---
+
+## Middleware
+
+Middleware wraps every `upload()` call (onion model).
+
+```python
+def inject_metadata(ctx, nxt):
+    ctx.options.metadata = {
+        **(ctx.options.metadata or {}),
+        "uploaded_by": "user-123",
+    }
+    return nxt()
+
+def force_prefix(ctx, nxt):
+    if not ctx.options.key.startswith("uploads/"):
+        ctx.options.key = f"uploads/{ctx.options.key}"
+    return nxt()
+
+storage.use(inject_metadata)
+storage.use(force_prefix)
+```
+
+---
+
+## Hooks
+
+```python
+storage.on("after:upload", lambda result: print(result.key, result.provider))
+storage.on("error", lambda payload: log(payload["operation"], payload["error"]))
+storage.on("after:transfer", lambda result: print(result.transferred, result.failed))
+
+storage.off("after:upload", handler)
+```
+
+Events: `before:upload`, `after:upload`, `before:delete`, `after:delete`, `before:copy`, `after:copy`, `before:move`, `after:move`, `before:list`, `after:list`, `before:getSignedUrl`, `after:getSignedUrl`, `before:transfer`, `after:transfer`, `error`.
+
+---
+
+## Errors
+
+All errors extend `StoreBridgeError` (`StorixError` is an alias).
+
+```python
+from py_storebridge import (
+    StoreBridgeError,
+    FileNotFoundError,
+    PermissionError,
+    MaxRetriesExceededError,
+    ValidationError,
+    MissingDependencyError,
+    TransferError,
+)
+
+try:
+    storage.get_metadata("missing.png")
+except FileNotFoundError as err:
+    print(err.key, err.status_code, err.provider)
+except MaxRetriesExceededError as err:
+    print(err.attempts)
+except StoreBridgeError as err:
+    print(err.code)
+```
+
+Install the matching extra if you see `MissingDependencyError`:
+
+```text
+pip install py_storebridge[s3]
+```
+
+---
+
+## Provider examples
+
+### AWS S3
+
+```python
+storage = create_storage({
+    "provider": "s3",
+    "credentials": {
+        "access_key_id": "…",
+        "secret_access_key": "…",
+        "region": "us-east-1",
+        "bucket": "my-bucket",
+        "cdn_url": "https://cdn.example.com",
+        "custom_domain": "https://files.example.com",
+    },
+    "visibility": "public",
+})
+```
+
+### Cloudflare R2
+
+```python
+storage = create_storage({
+    "provider": "r2",
+    "credentials": {
+        "account_id": "…",
+        "access_key_id": "…",
+        "secret_access_key": "…",
+        "bucket": "my-bucket",
+        "cdn_url": "https://pub-xxx.r2.dev",
+    },
+})
+```
+
+### Google Cloud Storage
+
+```python
+storage = create_storage({
+    "provider": "gcs",
+    "credentials": {
+        "project_id": "my-gcp-project",
+        "bucket": "my-bucket",
+        "credentials": {
+            "client_email": "…",
+            "private_key": "-----BEGIN PRIVATE KEY-----\\n…\\n-----END PRIVATE KEY-----\\n",
+        },
+    },
+})
+```
+
+### Azure Blob Storage
+
+```python
+storage = create_storage({
+    "provider": "azure",
+    "credentials": {
+        "account_name": "…",
+        "account_key": "…",
+        "container_name": "uploads",
+    },
+})
+```
+
+### DigitalOcean Spaces
+
+```python
+storage = create_storage({
+    "provider": "spaces",
+    "credentials": {
+        "access_key_id": "…",
+        "secret_access_key": "…",
+        "region": "nyc3",
+        "bucket": "my-space",
+        "cdn_url": "https://my-space.nyc3.cdn.digitaloceanspaces.com",
+    },
+})
+```
+
+### Backblaze B2
+
+```python
+storage = create_storage({
+    "provider": "b2",
+    "credentials": {
+        "application_key_id": "…",
+        "application_key": "…",
+        "bucket": "my-bucket",
+        "region": "us-west-004",
+    },
+})
+```
+
+### MinIO
+
+```python
+storage = create_storage({
+    "provider": "minio",
+    "credentials": {
+        "end_point": "minio.example.com",
+        "port": 9000,
+        "use_ssl": True,
+        "access_key": "…",
+        "secret_key": "…",
+        "bucket": "my-bucket",
+    },
+})
+```
+
+### Supabase Storage
+
+```python
+storage = create_storage({
+    "provider": "supabase",
+    "credentials": {
+        "url": "https://xyz.supabase.co",
+        "service_role_key": "…",
+        "bucket": "avatars",
+    },
+    "visibility": "public",
+})
+```
+
+### Firebase Storage
+
+```python
+storage = create_storage({
+    "provider": "firebase",
+    "credentials": {
+        "project_id": "my-project",
+        "bucket": "my-project.appspot.com",
+        "service_account": {…},  # full service-account JSON
+    },
+})
+```
+
+### Appwrite Storage
+
+Appwrite file IDs cannot contain `/`. py_storebridge stores a stable SHA-256 of the key as the file ID and keeps the original key as the file name, so `users/profile.png` still works.
+
+```python
+storage = create_storage({
+    "provider": "appwrite",
+    "credentials": {
+        "endpoint": "https://cloud.appwrite.io/v1",
+        "project_id": "…",
+        "api_key": "…",
+        "bucket_id": "…",
+    },
+})
+```
+
+---
+
+## Custom provider
+
+```python
+from py_storebridge import BaseProvider, create_storage_from_provider
+
+class MyProvider(BaseProvider):
+    provider_name = "my-provider"
+
+    def _upload(self, options):
+        ...
+
+storage = create_storage_from_provider(MyProvider())
+```
+
+---
+
+## Development
+
+```bash
+python -m venv .venv
+.venv\Scripts\activate          # Windows
+# source .venv/bin/activate     # macOS / Linux
+
+pip install -e ".[dev]"
+pytest
+```
+
+Publishing to PyPI is documented in [docs/HOW_TO_DEPLOY.md](docs/HOW_TO_DEPLOY.md).
+
+---
+
+## License
+
+MIT © [Asif Khan](https://github.com/ak9347128658)