@chidchanun/bcp 0.1.25 → 0.1.26

package/docs/s3-storage.md

@@ -0,0 +1,276 @@
+# S3-Compatible Storage
+
+BCP Framework `0.1.26` adds an S3-compatible `StorageAdapter` implementation through `bcp/server`.
+
+The adapter is designed for AWS S3 and services that implement the S3 API, including common S3-compatible providers such as Cloudflare R2 and MinIO. Provider-specific behavior can still differ, so production deployments should verify their chosen provider with representative uploads, ranges and overwrite rules.
+
+## Create an adapter
+
+```ts
+import {
+    createS3Storage,
+} from "bcp/server";
+
+export const storage =
+    createS3Storage({
+        bucket:
+            process.env.S3_BUCKET!,
+        region:
+            process.env.S3_REGION!,
+        endpoint:
+            process.env.S3_ENDPOINT,
+        accessKeyId:
+            process.env.S3_ACCESS_KEY_ID,
+        secretAccessKey:
+            process.env.S3_SECRET_ACCESS_KEY,
+    });
+```
+
+When `accessKeyId` / `secretAccessKey` are omitted, the underlying AWS SDK client can use its normal server-side credential provider chain.
+
+Do not expose S3 credentials through `BCP_PUBLIC_*` environment variables.
+
+## MinIO / path-style endpoints
+
+Some S3-compatible endpoints require path-style addressing:
+
+```ts
+const storage =
+    createS3Storage({
+        bucket:
+            "uploads",
+        region:
+            "us-east-1",
+        endpoint:
+            "http://127.0.0.1:9000",
+        forcePathStyle:
+            true,
+        accessKeyId:
+            process.env.MINIO_ACCESS_KEY,
+        secretAccessKey:
+            process.env.MINIO_SECRET_KEY,
+    });
+```
+
+## Prefix objects
+
+A deployment can reserve a logical prefix without changing application storage keys:
+
+```ts
+const storage =
+    createS3Storage({
+        bucket:
+            "app-data",
+        region:
+            "auto",
+        prefix:
+            "production",
+    });
+
+await storage.put(
+    "avatars/user-101.webp",
+    bytes
+);
+```
+
+The application-facing key remains:
+
+```text
+avatars/user-101.webp
+```
+
+while the provider object key becomes:
+
+```text
+production/avatars/user-101.webp
+```
+
+Prefixes and object keys reject traversal segments such as `..`.
+
+## Standard operations
+
+The S3 adapter implements the same `StorageAdapter` surface as local storage:
+
+```ts
+await storage.put(
+    "documents/report.pdf",
+    bytes,
+    {
+        contentType:
+            "application/pdf",
+    }
+);
+
+const metadata =
+    await storage.stat(
+        "documents/report.pdf"
+    );
+
+const bytes =
+    await storage.read(
+        "documents/report.pdf"
+    );
+
+await storage.exists(
+    "documents/report.pdf"
+);
+
+await storage.delete(
+    "documents/report.pdf"
+);
+```
+
+Buffered `put()` uses a conditional S3 write when `overwrite` is not enabled. Existing objects therefore produce `StorageError` with code `OBJECT_EXISTS` instead of being silently replaced.
+
+## Streaming upload
+
+`createS3Storage()` supports `putStream()` and works with the generic `putStorageStream()` helper:
+
+```ts
+import {
+    putStorageStream,
+} from "bcp/server";
+
+await putStorageStream(
+    storage,
+    "media/video.mp4",
+    request.body!,
+    {
+        contentType:
+            "video/mp4",
+        maxBytes:
+            500 * 1024 * 1024,
+    }
+);
+```
+
+BCP uses the AWS SDK multipart upload helper for streams whose total size may not be known when the upload begins. This avoids requiring the complete object to be buffered in application memory.
+
+If a stream exceeds `maxBytes`, the upload is aborted and `StorageError` uses:
+
+```text
+code: STREAM_TOO_LARGE
+status: 413
+```
+
+Multipart uploads use automatic cleanup on failure (`leavePartsOnError: false`).
+
+## Multipart tuning
+
+The adapter accepts optional multipart tuning:
+
+```ts
+const storage =
+    createS3Storage({
+        bucket:
+            "uploads",
+        region:
+            "us-east-1",
+        multipart: {
+            partSize:
+                8 * 1024 * 1024,
+            queueSize:
+                2,
+        },
+    });
+```
+
+`partSize` must be at least 5 MiB. Higher `queueSize` can improve throughput, but it also increases concurrent network and memory usage.
+
+## Ranged reads
+
+The adapter preserves BCP's inclusive `start` / `end` range contract:
+
+```ts
+const bytes =
+    await storage.read(
+        "media/video.mp4",
+        {
+            start: 0,
+            end: 1023,
+        }
+    );
+```
+
+`readStream()` also supports ranges and is used automatically by `createStorageResponse()`.
+
+```ts
+return createStorageResponse(
+    request,
+    storage,
+    "media/video.mp4"
+);
+```
+
+This allows the existing file-delivery API to serve local or S3-backed objects with the same `GET`, `HEAD`, ETag, Last-Modified and single-range behavior.
+
+## Capabilities
+
+```ts
+getStorageCapabilities(storage)
+```
+
+returns native support for:
+
+```text
+streamingRead: true
+streamingWrite: true
+ranges: true
+signedUrls: false
+listing: false
+```
+
+Signed URLs and listing are intentionally deferred to a later storage milestone rather than expanding the `0.1.26` release surface further.
+
+## Overwrite semantics
+
+For buffered `put()`, BCP sends a conditional create-only request when `overwrite` is false.
+
+Streaming multipart uploads perform a `HeadObject` preflight before starting when overwrite is disabled. That preserves the normal application behavior, but it cannot provide the same atomic create-only guarantee across every S3-compatible multipart implementation if another writer creates the same key between the preflight and completion.
+
+Applications that require strict cross-writer serialization should enforce it in their database/business layer or use provider-specific conditional workflows.
+
+## Metadata and ETags
+
+S3 response metadata is mapped to `StorageObjectMetadata`:
+
+```ts
+{
+    key,
+    size,
+    contentType,
+    lastModified,
+    etag,
+    checksumSha256?,
+}
+```
+
+Buffered BCP writes persist a SHA-256 value in S3 user metadata. Streaming multipart writes rely on provider metadata/ETag behavior and may not expose `checksumSha256` through the generic contract.
+
+Do not assume an S3 ETag is always a plain MD5 checksum; multipart and provider-specific ETag formats can differ.
+
+## Client lifecycle
+
+Adapters that create their own AWS SDK client expose:
+
+```ts
+storage.destroy();
+```
+
+Call it during application shutdown when appropriate.
+
+When an existing `S3Client` is injected through `client`, BCP does not destroy that externally owned client.
+
+## Security
+
+Storage keys are not authorization. Authenticate and authorize a user before allowing access to an object key.
+
+Recommended production practices include:
+
+- keep buckets private by default,
+- use scoped credentials / IAM policies,
+- do not expose access keys to browser bundles,
+- validate uploaded content independently of MIME metadata,
+- configure provider-side encryption and retention policies when required,
+- keep application upload limits below infrastructure/proxy limits,
+- verify CORS only when browsers intentionally access the bucket directly.