@chidchanun/bcp 0.1.26 → 0.1.28

package/docs/s3-storage.md

@@ -1,8 +1,10 @@
 # S3-Compatible Storage
 
-BCP Framework `0.1.26` adds an S3-compatible `StorageAdapter` implementation through `bcp/server`.
+BCP Framework `0.1.26` introduced an S3-compatible streaming `StorageAdapter`. BCP `0.1.27` extends the same `createS3Storage()` API with object listing, native copy/move, user metadata, bulk deletion and presigned read/write URLs.
 
-The adapter is designed for AWS S3 and services that implement the S3 API, including common S3-compatible providers such as Cloudflare R2 and MinIO. Provider-specific behavior can still differ, so production deployments should verify their chosen provider with representative uploads, ranges and overwrite rules.
+> `0.1.27` behavior is an unreleased development target until RC validation and publication complete.
+
+The adapter is designed for AWS S3 and services that implement the S3 API, including common S3-compatible providers such as Cloudflare R2 and MinIO. Provider-specific behavior can differ, so production deployments should verify their chosen service with representative object operations.
 
 ## Create an adapter
 
@@ -54,7 +56,7 @@ const storage =
 
 ## Prefix objects
 
-A deployment can reserve a logical prefix without changing application storage keys:
+A deployment can reserve a logical provider prefix without changing application-facing keys:
 
 ```ts
 const storage =
@@ -73,24 +75,22 @@ await storage.put(
 );
 ```
 
-The application-facing key remains:
+Application key:
 
 ```text
 avatars/user-101.webp
 ```
 
-while the provider object key becomes:
+Provider key:
 
 ```text
 production/avatars/user-101.webp
 ```
 
-Prefixes and object keys reject traversal segments such as `..`.
+All `0.1.27` list/copy/move/delete/signing operations preserve this logical/provider-key boundary.
 
 ## Standard operations
 
-The S3 adapter implements the same `StorageAdapter` surface as local storage:
-
 ```ts
 await storage.put(
     "documents/report.pdf",
@@ -124,7 +124,7 @@ Buffered `put()` uses a conditional S3 write when `overwrite` is not enabled. Ex
 
 ## Streaming upload
 
-`createS3Storage()` supports `putStream()` and works with the generic `putStorageStream()` helper:
+`createS3Storage()` supports `putStream()` and works with `putStorageStream()`:
 
 ```ts
 import {
@@ -144,21 +144,10 @@ await putStorageStream(
 );
 ```
 
-BCP uses the AWS SDK multipart upload helper for streams whose total size may not be known when the upload begins. This avoids requiring the complete object to be buffered in application memory.
-
-If a stream exceeds `maxBytes`, the upload is aborted and `StorageError` uses:
-
-```text
-code: STREAM_TOO_LARGE
-status: 413
-```
-
-Multipart uploads use automatic cleanup on failure (`leavePartsOnError: false`).
+BCP uses AWS SDK multipart upload for streams whose total size may not be known when upload begins. Multipart cleanup remains enabled on failure.
 
 ## Multipart tuning
 
-The adapter accepts optional multipart tuning:
-
 ```ts
 const storage =
     createS3Storage({
@@ -175,12 +164,10 @@ const storage =
     });
 ```
 
-`partSize` must be at least 5 MiB. Higher `queueSize` can improve throughput, but it also increases concurrent network and memory usage.
+`partSize` must be at least 5 MiB.
 
 ## Ranged reads
 
-The adapter preserves BCP's inclusive `start` / `end` range contract:
-
 ```ts
 const bytes =
     await storage.read(
@@ -194,60 +181,221 @@ const bytes =
 
 `readStream()` also supports ranges and is used automatically by `createStorageResponse()`.
 
+## Listing
+
+BCP `0.1.27` enables native S3 `ListObjectsV2` through the generic helper:
+
+```ts
+import {
+    listStorageObjects,
+} from "bcp/server";
+
+const page =
+    await listStorageObjects(
+        storage,
+        {
+            prefix:
+                "documents/",
+            limit:
+                100,
+        }
+    );
+```
+
+When S3 returns another page, `page.cursor` contains its continuation token. Treat this cursor as opaque and pass it back unchanged.
+
+S3 list responses do not include object content type, so listed `StorageObjectMetadata` uses `application/octet-stream` for `contentType`. Use `stat()` when exact object headers are required.
+
+## Native copy and move
+
 ```ts
-return createStorageResponse(
-    request,
+import {
+    copyStorageObject,
+    moveStorageObject,
+} from "bcp/server";
+
+await copyStorageObject(
     storage,
-    "media/video.mp4"
+    "incoming/report.pdf",
+    "archive/report.pdf"
+);
+
+await moveStorageObject(
+    storage,
+    "incoming/avatar.webp",
+    "users/42/avatar.webp"
 );
 ```
 
-This allows the existing file-delivery API to serve local or S3-backed objects with the same `GET`, `HEAD`, ETag, Last-Modified and single-range behavior.
+The S3 adapter uses native `CopyObject`. User metadata and BCP checksum metadata are preserved by normal copies.
+
+Move is copy followed by source deletion. It is not a distributed transaction; applications needing business-level atomicity should coordinate storage state with their database.
+
+## User metadata
+
+```ts
+await storage.put(
+    "documents/report.pdf",
+    bytes,
+    {
+        contentType:
+            "application/pdf",
+        metadata: {
+            owner:
+                "user-42",
+            category:
+                "report",
+        },
+    }
+);
+```
+
+Read or replace metadata:
+
+```ts
+import {
+    getStorageMetadata,
+    setStorageMetadata,
+} from "bcp/server";
+
+const metadata =
+    await getStorageMetadata(
+        storage,
+        "documents/report.pdf"
+    );
+
+await setStorageMetadata(
+    storage,
+    "documents/report.pdf",
+    {
+        owner:
+            "user-42",
+        state:
+            "approved",
+    }
+);
+```
+
+BCP maps application metadata to S3 user metadata and reserves `bcp_sha256` for framework checksum data.
+
+Replacing S3 metadata uses provider copy-to-self semantics with `MetadataDirective: REPLACE` while retaining BCP checksum metadata.
+
+## Bulk deletion
+
+```ts
+import {
+    deleteStorageObjects,
+} from "bcp/server";
+
+const result =
+    await deleteStorageObjects(
+        storage,
+        keys
+    );
+```
+
+BCP batches native S3 multi-delete requests at up to 1000 keys per request.
+
+S3 delete semantics are idempotent: deleting a missing key is normally reported as successful. Therefore S3 results commonly place such keys in `deleted` rather than `missing`.
+
+## Presigned read URLs
+
+```ts
+import {
+    createStorageSignedReadUrl,
+} from "bcp/server";
+
+const url =
+    await createStorageSignedReadUrl(
+        storage,
+        "videos/demo.mp4",
+        {
+            expiresIn:
+                300,
+        }
+    );
+```
+
+The default expiry is 15 minutes. The maximum accepted expiry is seven days, matching the SigV4 presigning boundary.
+
+## Presigned write URLs
+
+```ts
+import {
+    createStorageSignedWriteUrl,
+} from "bcp/server";
+
+const url =
+    await createStorageSignedWriteUrl(
+        storage,
+        "uploads/video.mp4",
+        {
+            expiresIn:
+                300,
+            contentType:
+                "video/mp4",
+            metadata: {
+                owner:
+                    "user-42",
+            },
+        }
+    );
+```
+
+The browser or another client can then upload directly to S3-compatible storage instead of proxying the file bytes through BCP.
+
+When content type or metadata are included in the signed operation, the direct uploader must send the headers required by the generated signature/provider.
+
+Never issue a signed URL before application authorization. Signed URLs should be treated as temporary credentials.
 
 ## Capabilities
 
+Use the original capability helper for compatibility:
+
 ```ts
 getStorageCapabilities(storage)
 ```
 
-returns native support for:
+For the `0.1.27` S3 wrapper it reports:
 
 ```text
 streamingRead: true
 streamingWrite: true
 ranges: true
-signedUrls: false
-listing: false
+signedUrls: true
+listing: true
 ```
 
-Signed URLs and listing are intentionally deferred to a later storage milestone rather than expanding the `0.1.26` release surface further.
+For the complete ecosystem surface use:
 
-## Overwrite semantics
+```ts
+getStorageEcosystemCapabilities(storage)
+```
 
-For buffered `put()`, BCP sends a conditional create-only request when `overwrite` is false.
+which additionally reports:
 
-Streaming multipart uploads perform a `HeadObject` preflight before starting when overwrite is disabled. That preserves the normal application behavior, but it cannot provide the same atomic create-only guarantee across every S3-compatible multipart implementation if another writer creates the same key between the preflight and completion.
+```text
+signedReadUrls: true
+signedWriteUrls: true
+copy: true
+move: true
+metadata: true
+bulkDelete: true
+```
 
-Applications that require strict cross-writer serialization should enforce it in their database/business layer or use provider-specific conditional workflows.
+## Overwrite semantics
 
-## Metadata and ETags
+Buffered `put()` retains the `0.1.26` conditional create-only behavior when `overwrite` is false.
 
-S3 response metadata is mapped to `StorageObjectMetadata`:
+Streaming multipart writes still use a preflight existence check because portable S3 multipart APIs do not expose the same atomic create-only condition across every compatible provider.
 
-```ts
-{
-    key,
-    size,
-    contentType,
-    lastModified,
-    etag,
-    checksumSha256?,
-}
-```
+Copy/move destinations also reject existing objects by default unless `overwrite: true` is supplied.
+
+## Metadata and ETags
 
-Buffered BCP writes persist a SHA-256 value in S3 user metadata. Streaming multipart writes rely on provider metadata/ETag behavior and may not expose `checksumSha256` through the generic contract.
+Do not assume an S3 ETag is a plain MD5 checksum. Multipart uploads and provider-specific implementations can use different ETag formats.
 
-Do not assume an S3 ETag is always a plain MD5 checksum; multipart and provider-specific ETag formats can differ.
+Buffered BCP writes persist a SHA-256 value in reserved S3 user metadata. Multipart streams may not expose a portable whole-object SHA-256 through the generic contract.
 
 ## Client lifecycle
 
@@ -263,14 +411,23 @@ When an existing `S3Client` is injected through `client`, BCP does not destroy t
 
 ## Security
 
-Storage keys are not authorization. Authenticate and authorize a user before allowing access to an object key.
+Storage keys are not authorization. Authenticate and authorize before exposing object operations.
 
-Recommended production practices include:
+Recommended production practices:
 
 - keep buckets private by default,
 - use scoped credentials / IAM policies,
 - do not expose access keys to browser bundles,
+- use short presigned URL expirations,
+- avoid logging full signed query strings,
 - validate uploaded content independently of MIME metadata,
-- configure provider-side encryption and retention policies when required,
-- keep application upload limits below infrastructure/proxy limits,
-- verify CORS only when browsers intentionally access the bucket directly.
+- configure encryption/retention policies when required,
+- keep application upload limits below proxy/infrastructure limits,
+- configure CORS only for the browser origins and methods that need direct object access.
+
+## Related documentation
+
+- [Storage and File Delivery](storage.md)
+- [Storage Ecosystem](storage-ecosystem.md)
+- [File Upload](file-upload.md)
+- [BCP Framework 0.1.27](releases/0.1.27.md)