@chidchanun/bcp 0.1.24 → 0.1.26

package/docs/s3-storage.md

@@ -0,0 +1,276 @@
+# S3-Compatible Storage
+
+BCP Framework `0.1.26` adds an S3-compatible `StorageAdapter` implementation through `bcp/server`.
+
+The adapter is designed for AWS S3 and services that implement the S3 API, including common S3-compatible providers such as Cloudflare R2 and MinIO. Provider-specific behavior can still differ, so production deployments should verify their chosen provider with representative uploads, ranges and overwrite rules.
+
+## Create an adapter
+
+```ts
+import {
+    createS3Storage,
+} from "bcp/server";
+
+export const storage =
+    createS3Storage({
+        bucket:
+            process.env.S3_BUCKET!,
+        region:
+            process.env.S3_REGION!,
+        endpoint:
+            process.env.S3_ENDPOINT,
+        accessKeyId:
+            process.env.S3_ACCESS_KEY_ID,
+        secretAccessKey:
+            process.env.S3_SECRET_ACCESS_KEY,
+    });
+```
+
+When `accessKeyId` / `secretAccessKey` are omitted, the underlying AWS SDK client can use its normal server-side credential provider chain.
+
+Do not expose S3 credentials through `BCP_PUBLIC_*` environment variables.
+
+## MinIO / path-style endpoints
+
+Some S3-compatible endpoints require path-style addressing:
+
+```ts
+const storage =
+    createS3Storage({
+        bucket:
+            "uploads",
+        region:
+            "us-east-1",
+        endpoint:
+            "http://127.0.0.1:9000",
+        forcePathStyle:
+            true,
+        accessKeyId:
+            process.env.MINIO_ACCESS_KEY,
+        secretAccessKey:
+            process.env.MINIO_SECRET_KEY,
+    });
+```
+
+## Prefix objects
+
+A deployment can reserve a logical prefix without changing application storage keys:
+
+```ts
+const storage =
+    createS3Storage({
+        bucket:
+            "app-data",
+        region:
+            "auto",
+        prefix:
+            "production",
+    });
+
+await storage.put(
+    "avatars/user-101.webp",
+    bytes
+);
+```
+
+The application-facing key remains:
+
+```text
+avatars/user-101.webp
+```
+
+while the provider object key becomes:
+
+```text
+production/avatars/user-101.webp
+```
+
+Prefixes and object keys reject traversal segments such as `..`.
+
+## Standard operations
+
+The S3 adapter implements the same `StorageAdapter` surface as local storage:
+
+```ts
+await storage.put(
+    "documents/report.pdf",
+    bytes,
+    {
+        contentType:
+            "application/pdf",
+    }
+);
+
+const metadata =
+    await storage.stat(
+        "documents/report.pdf"
+    );
+
+const bytes =
+    await storage.read(
+        "documents/report.pdf"
+    );
+
+await storage.exists(
+    "documents/report.pdf"
+);
+
+await storage.delete(
+    "documents/report.pdf"
+);
+```
+
+Buffered `put()` uses a conditional S3 write when `overwrite` is not enabled. Existing objects therefore produce `StorageError` with code `OBJECT_EXISTS` instead of being silently replaced.
+
+## Streaming upload
+
+`createS3Storage()` supports `putStream()` and works with the generic `putStorageStream()` helper:
+
+```ts
+import {
+    putStorageStream,
+} from "bcp/server";
+
+await putStorageStream(
+    storage,
+    "media/video.mp4",
+    request.body!,
+    {
+        contentType:
+            "video/mp4",
+        maxBytes:
+            500 * 1024 * 1024,
+    }
+);
+```
+
+BCP uses the AWS SDK multipart upload helper for streams whose total size may not be known when the upload begins. This avoids requiring the complete object to be buffered in application memory.
+
+If a stream exceeds `maxBytes`, the upload is aborted and `StorageError` uses:
+
+```text
+code: STREAM_TOO_LARGE
+status: 413
+```
+
+Multipart uploads use automatic cleanup on failure (`leavePartsOnError: false`).
+
+## Multipart tuning
+
+The adapter accepts optional multipart tuning:
+
+```ts
+const storage =
+    createS3Storage({
+        bucket:
+            "uploads",
+        region:
+            "us-east-1",
+        multipart: {
+            partSize:
+                8 * 1024 * 1024,
+            queueSize:
+                2,
+        },
+    });
+```
+
+`partSize` must be at least 5 MiB. Higher `queueSize` can improve throughput, but it also increases concurrent network and memory usage.
+
+## Ranged reads
+
+The adapter preserves BCP's inclusive `start` / `end` range contract:
+
+```ts
+const bytes =
+    await storage.read(
+        "media/video.mp4",
+        {
+            start: 0,
+            end: 1023,
+        }
+    );
+```
+
+`readStream()` also supports ranges and is used automatically by `createStorageResponse()`.
+
+```ts
+return createStorageResponse(
+    request,
+    storage,
+    "media/video.mp4"
+);
+```
+
+This allows the existing file-delivery API to serve local or S3-backed objects with the same `GET`, `HEAD`, ETag, Last-Modified and single-range behavior.
+
+## Capabilities
+
+```ts
+getStorageCapabilities(storage)
+```
+
+returns native support for:
+
+```text
+streamingRead: true
+streamingWrite: true
+ranges: true
+signedUrls: false
+listing: false
+```
+
+Signed URLs and listing are intentionally deferred to a later storage milestone rather than expanding the `0.1.26` release surface further.
+
+## Overwrite semantics
+
+For buffered `put()`, BCP sends a conditional create-only request when `overwrite` is false.
+
+Streaming multipart uploads perform a `HeadObject` preflight before starting when overwrite is disabled. That preserves the normal application behavior, but it cannot provide the same atomic create-only guarantee across every S3-compatible multipart implementation if another writer creates the same key between the preflight and completion.
+
+Applications that require strict cross-writer serialization should enforce it in their database/business layer or use provider-specific conditional workflows.
+
+## Metadata and ETags
+
+S3 response metadata is mapped to `StorageObjectMetadata`:
+
+```ts
+{
+    key,
+    size,
+    contentType,
+    lastModified,
+    etag,
+    checksumSha256?,
+}
+```
+
+Buffered BCP writes persist a SHA-256 value in S3 user metadata. Streaming multipart writes rely on provider metadata/ETag behavior and may not expose `checksumSha256` through the generic contract.
+
+Do not assume an S3 ETag is always a plain MD5 checksum; multipart and provider-specific ETag formats can differ.
+
+## Client lifecycle
+
+Adapters that create their own AWS SDK client expose:
+
+```ts
+storage.destroy();
+```
+
+Call it during application shutdown when appropriate.
+
+When an existing `S3Client` is injected through `client`, BCP does not destroy that externally owned client.
+
+## Security
+
+Storage keys are not authorization. Authenticate and authorize a user before allowing access to an object key.
+
+Recommended production practices include:
+
+- keep buckets private by default,
+- use scoped credentials / IAM policies,
+- do not expose access keys to browser bundles,
+- validate uploaded content independently of MIME metadata,
+- configure provider-side encryption and retention policies when required,
+- keep application upload limits below infrastructure/proxy limits,
+- verify CORS only when browsers intentionally access the bucket directly.

package/docs/storage.md

@@ -0,0 +1,401 @@
+# Storage and File Delivery
+
+BCP Framework `0.1.25` introduced the server-only `StorageAdapter` abstraction and production file-delivery helpers through `bcp/server`.
+
+BCP Framework `0.1.26` completes the next storage milestone with:
+
+- additive streaming reads/writes,
+- capability discovery,
+- local streaming I/O,
+- S3-compatible storage,
+- production multipart-to-storage streaming,
+- streamed full/ranged file delivery.
+
+Existing adapters written for `0.1.25` remain valid because `putStream()` and `readStream()` are optional.
+
+## Local storage
+
+```ts
+import {
+    createLocalStorage,
+} from "bcp/server";
+
+export const storage =
+    createLocalStorage({
+        directory:
+            "./uploads",
+    });
+```
+
+Storage keys are logical relative paths:
+
+```text
+avatars/user-101.webp
+documents/2026/report.pdf
+```
+
+Absolute paths, traversal segments and BCP's reserved metadata directory are rejected.
+
+## S3-compatible storage
+
+```ts
+import {
+    createS3Storage,
+} from "bcp/server";
+
+export const storage =
+    createS3Storage({
+        bucket:
+            process.env.S3_BUCKET!,
+        region:
+            process.env.S3_REGION!,
+        endpoint:
+            process.env.S3_ENDPOINT,
+        accessKeyId:
+            process.env.S3_ACCESS_KEY_ID,
+        secretAccessKey:
+            process.env.S3_SECRET_ACCESS_KEY,
+    });
+```
+
+The same application-facing `StorageAdapter` surface works with local or S3-compatible backends.
+
+Read more: [S3-Compatible Storage](s3-storage.md).
+
+## Storage adapter contract
+
+```ts
+interface StorageAdapter {
+    capabilities?: Partial<
+        StorageAdapterCapabilities
+    >;
+
+    put(
+        key: string,
+        value: StorageWriteValue,
+        options?: StoragePutOptions
+    ): Promise<StorageObjectMetadata>;
+
+    putStream?(
+        key: string,
+        stream: ReadableStream<Uint8Array>,
+        options?: StoragePutStreamOptions
+    ): Promise<StorageObjectMetadata>;
+
+    stat(
+        key: string
+    ): Promise<StorageObjectMetadata | null>;
+
+    read(
+        key: string,
+        options?: StorageReadOptions
+    ): Promise<Uint8Array>;
+
+    readStream?(
+        key: string,
+        options?: StorageReadOptions
+    ): Promise<StorageReadableStream>;
+
+    exists(
+        key: string
+    ): Promise<boolean>;
+
+    delete(
+        key: string
+    ): Promise<boolean>;
+}
+```
+
+The original buffered methods remain required. Streaming methods are additive optional capabilities.
+
+## Capabilities
+
+```ts
+import {
+    getStorageCapabilities,
+} from "bcp/server";
+
+const capabilities =
+    getStorageCapabilities(
+        storage
+    );
+```
+
+The capability model currently reports:
+
+```ts
+{
+    streamingRead,
+    streamingWrite,
+    ranges,
+    signedUrls,
+    listing,
+}
+```
+
+`createLocalStorage()` and `createS3Storage()` in `0.1.26` report:
+
+```ts
+{
+    streamingRead: true,
+    streamingWrite: true,
+    ranges: true,
+    signedUrls: false,
+    listing: false,
+}
+```
+
+Signed URLs and listing are deferred to a later milestone.
+
+## Buffered writes
+
+```ts
+const stored =
+    await storage.put(
+        "notes/hello.txt",
+        "Hello BCP",
+        {
+            contentType:
+                "text/plain",
+        }
+    );
+```
+
+`put()` does not overwrite an existing object unless `overwrite: true` is supplied.
+
+Local storage persists content type and SHA-256 identity in sidecar metadata. Buffered S3 writes persist SHA-256 in S3 user metadata.
+
+## Streaming writes
+
+Use the generic helper so application code works with native or legacy adapters:
+
+```ts
+import {
+    putStorageStream,
+} from "bcp/server";
+
+await putStorageStream(
+    storage,
+    "imports/data.bin",
+    request.body!,
+    {
+        contentType:
+            "application/octet-stream",
+        maxBytes:
+            100 * 1024 * 1024,
+    }
+);
+```
+
+When an adapter provides `putStream()`, BCP uses it. Legacy adapters fall back to collecting the stream and calling `put()`.
+
+The local adapter writes chunks directly to disk and calculates SHA-256 while streaming.
+
+The S3 adapter uses the AWS SDK multipart upload helper so streams whose final size is not known at the start can still be uploaded without buffering the complete object.
+
+### Size limits and abort
+
+```ts
+await putStorageStream(
+    storage,
+    key,
+    source,
+    {
+        maxBytes:
+            50 * 1024 * 1024,
+        signal:
+            abortController.signal,
+    }
+);
+```
+
+If the stream exceeds `maxBytes`, BCP throws:
+
+```text
+StorageError
+code: STREAM_TOO_LARGE
+status: 413
+```
+
+Native adapters clean up/abort incomplete writes where supported.
+
+## Streaming reads
+
+```ts
+import {
+    readStorageStream,
+} from "bcp/server";
+
+const stream =
+    await readStorageStream(
+        storage,
+        "videos/demo.mp4"
+    );
+```
+
+Inclusive byte ranges are supported:
+
+```ts
+const stream =
+    await readStorageStream(
+        storage,
+        "videos/demo.mp4",
+        {
+            start: 0,
+            end: 1023,
+        }
+    );
+```
+
+Legacy adapters fall back to `read()` and expose the result as a stream.
+
+## Multipart uploads
+
+Small forms can continue using the buffered `FormData` API:
+
+```ts
+const formData =
+    await parseMultipartFormData(
+        request
+    );
+
+const file =
+    requireUploadedFile(
+        formData,
+        "file"
+    );
+
+await storeUploadedFile(
+    file,
+    {
+        storage,
+    }
+);
+```
+
+For large production uploads, `0.1.26` adds direct multipart-to-storage streaming:
+
+```ts
+import {
+    storeMultipartFile,
+} from "bcp/server";
+
+const stored =
+    await storeMultipartFile(
+        request,
+        {
+            storage,
+            fieldName:
+                "file",
+            key:
+                "documents/report.pdf",
+            maxBytes:
+                100 * 1024 * 1024,
+            constraints: {
+                maxBytes:
+                    80 * 1024 * 1024,
+                allowedTypes: [
+                    "application/pdf",
+                ],
+                allowedExtensions: [
+                    ".pdf",
+                ],
+            },
+        }
+    );
+```
+
+The multipart parser and storage write operate concurrently, so the complete file is not converted to a `File` / `ArrayBuffer` first.
+
+Read more: [File Upload](file-upload.md).
+
+## File delivery
+
+```ts
+import {
+    createStorageResponse,
+} from "bcp/server";
+
+export function GET(
+    request: Request
+) {
+    return createStorageResponse(
+        request,
+        storage,
+        "documents/report.pdf",
+        {
+            disposition:
+                "attachment",
+            downloadName:
+                "report.pdf",
+        }
+    );
+}
+```
+
+`createStorageResponse()` uses `readStorageStream()` and supports:
+
+- `GET` and `HEAD`,
+- `ETag` / `If-None-Match`,
+- `Last-Modified` / `If-Modified-Since`,
+- `If-Range`,
+- single byte ranges with `206 Partial Content`,
+- `304 Not Modified`,
+- `416 Range Not Satisfiable`,
+- safe `Content-Disposition` filenames,
+- configurable cache control.
+
+The default cache policy is:
+
+```text
+Cache-Control: private, max-age=0, must-revalidate
+```
+
+Public immutable caching must be enabled explicitly.
+
+Multiple byte ranges are intentionally unsupported in `0.1.26`.
+
+## Object metadata
+
+```ts
+interface StorageObjectMetadata {
+    key: string;
+    size: number;
+    contentType: string;
+    lastModified: Date;
+    etag: string;
+    checksumSha256?: string;
+}
+```
+
+`checksumSha256` is optional because not every cloud provider exposes a portable whole-object SHA-256 value for multipart objects.
+
+Do not assume an S3 ETag is an MD5 hash.
+
+## Storage errors
+
+Current codes:
+
+```text
+INVALID_KEY
+OBJECT_NOT_FOUND
+OBJECT_EXISTS
+RANGE_NOT_SATISFIABLE
+STREAM_TOO_LARGE
+```
+
+## Security boundary
+
+Storage keys are not authorization. Authenticate and authorize requests before allowing access to user-selected keys.
+
+MIME type and extension checks are metadata validation, not file-signature verification.
+
+Streaming limits do not replace the framework/proxy request body limit. Configure `server.bodyLimit` / `BCP_BODY_LIMIT` large enough for intended uploads while keeping an explicit outer request boundary.
+
+For S3/object storage:
+
+- keep credentials server-only,
+- prefer private buckets,
+- scope IAM/provider permissions,
+- use provider-side encryption/retention policies where required,
+- validate provider CORS only when direct browser access is intentionally enabled.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@chidchanun/bcp",
-  "version": "0.1.24",
+  "version": "0.1.26",
   "description": "BCP Framework - a React full-stack framework with file-based routing, SSR, APIs, middleware, islands, caching and standalone production builds.",
   "type": "module",
   "license": "MIT",
@@ -55,7 +55,7 @@
     "./database": {
       "types": "./packages/client/src/database.ts",
       "browser": "./packages/client/src/server-only.browser.mjs",
-      "default": "./packages/client/src/database.ts"
+      "default": "./packages/client/src/database.mjs"
     },
     "./auth": {
       "types": "./packages/client/src/auth.ts",
@@ -79,9 +79,12 @@
     "./package.json": "./package.json"
   },
   "dependencies": {
+    "@aws-sdk/client-s3": "^3.1119.0",
+    "@aws-sdk/lib-storage": "^3.1119.0",
     "@babel/core": "^8.0.1",
     "@babel/preset-react": "^8.0.1",
     "@babel/preset-typescript": "^8.0.1",
+    "busboy": "^1.6.0",
     "chokidar": "^5.0.0",
     "esbuild": "^0.28.2",
     "react-refresh": "^0.18.0",