@chidchanun/bcp 0.1.25 → 0.1.26

package/docs/file-upload.md

@@ -1,8 +1,13 @@
 # File Upload
 
-BCP Framework 0.1.24 adds server-only multipart and file-storage helpers through `bcp/server`.
+BCP Framework provides two server-only multipart upload paths through `bcp/server`:
 
-## Basic API route
+- the buffered `FormData` helpers introduced in `0.1.24`,
+- direct multipart-to-storage streaming added in `0.1.26`.
+
+Use the buffered API for small forms and workflows that need normal `FormData` access. Use `storeMultipartFile()` when large files should flow directly from the request body into a `StorageAdapter` without materializing the complete file in application memory.
+
+## Buffered API route
 
 ```ts
 import {
@@ -64,6 +69,125 @@ export async function POST(
 }
 ```
 
+## Production streaming upload
+
+BCP `0.1.26` adds `storeMultipartFile()`:
+
+```ts
+import {
+    createLocalStorage,
+    storeMultipartFile,
+} from "bcp/server";
+
+const uploads =
+    createLocalStorage({
+        directory:
+            "./uploads",
+    });
+
+export async function POST(
+    request: Request
+) {
+    const stored =
+        await storeMultipartFile(
+            request,
+            {
+                storage:
+                    uploads,
+                fieldName:
+                    "file",
+                key:
+                    "incoming/upload.bin",
+                maxBytes:
+                    100 * 1024 * 1024,
+                constraints: {
+                    maxBytes:
+                        80 * 1024 * 1024,
+                    allowedTypes: [
+                        "application/octet-stream",
+                    ],
+                },
+            }
+        );
+
+    return Response.json(
+        stored
+    );
+}
+```
+
+The request body is parsed incrementally. The target file stream is handed to the selected `StorageAdapter` while multipart data is still arriving.
+
+This path is intended for large uploads and shared/object storage. It avoids calling `request.formData()` for the uploaded file.
+
+## Streaming directly to S3-compatible storage
+
+The same API works with `createS3Storage()`:
+
+```ts
+import {
+    createS3Storage,
+    storeMultipartFile,
+} from "bcp/server";
+
+const uploads =
+    createS3Storage({
+        bucket:
+            process.env.S3_BUCKET!,
+        region:
+            process.env.S3_REGION!,
+        endpoint:
+            process.env.S3_ENDPOINT,
+        accessKeyId:
+            process.env.S3_ACCESS_KEY_ID,
+        secretAccessKey:
+            process.env.S3_SECRET_ACCESS_KEY,
+    });
+
+export async function POST(
+    request: Request
+) {
+    return Response.json(
+        await storeMultipartFile(
+            request,
+            {
+                storage:
+                    uploads,
+                fieldName:
+                    "file",
+                key: (
+                    file
+                ) =>
+                    `documents/${crypto.randomUUID()}${file.originalName.endsWith(".pdf") ? ".pdf" : ""}`,
+                constraints: {
+                    maxBytes:
+                        50 * 1024 * 1024,
+                    allowedTypes: [
+                        "application/pdf",
+                    ],
+                    allowedExtensions: [
+                        ".pdf",
+                    ],
+                },
+            }
+        )
+    );
+}
+```
+
+The `key` option can be a fixed string or an async callback receiving:
+
+```ts
+{
+    fieldName,
+    originalName,
+    encoding,
+    contentType,
+}
+```
+
+When `key` is omitted, BCP creates a UUID-based key and preserves a short sanitized extension.
+
 ## Request body limit
 
 BCP's security gateway enforces the framework-wide request body limit before the request reaches the application server.
@@ -79,16 +203,20 @@ import {
 export default defineConfig({
     server: {
         bodyLimit:
-            10 * 1024 * 1024,
+            100 * 1024 * 1024,
     },
 });
 ```
 
 The same setting can be supplied with `BCP_BODY_LIMIT`.
 
-Keep `server.bodyLimit` greater than or equal to the multipart limit used by `parseMultipartFormData()`.
+For buffered uploads, keep `server.bodyLimit` greater than or equal to the multipart limit used by `parseMultipartFormData()`.
+
+For streaming uploads, keep the infrastructure/framework body limit greater than or equal to `storeMultipartFile({ maxBytes })`. Streaming avoids full application-memory buffering, but it does not bypass reverse-proxy, framework or platform request-size limits.
+
+## Multipart request limits
 
-## Multipart parsing
+Buffered parsing:
 
 ```ts
 const formData =
@@ -101,16 +229,31 @@ const formData =
     );
 ```
 
-The helper:
+Streaming parsing:
+
+```ts
+await storeMultipartFile(
+    request,
+    {
+        storage,
+        fieldName:
+            "file",
+        maxBytes:
+            100 * 1024 * 1024,
+    }
+);
+```
+
+Both helpers:
 
-- requires `multipart/form-data`,
-- rejects a declared `Content-Length` above `maxBytes`,
-- validates the parsed form payload size,
-- throws `UploadError` with HTTP-oriented status information on validation failures.
+- require `multipart/form-data`,
+- reject a declared `Content-Length` above the configured request limit,
+- enforce the request limit while processing,
+- throw `UploadError` with HTTP-oriented status information on validation failures.
 
-`maxBytes` is optional. The framework-wide `server.bodyLimit` remains the outer request limit.
+`storeMultipartFile()` additionally applies the file-level `constraints.maxBytes` while bytes are flowing into storage. Native streaming adapters can abort incomplete writes rather than keeping a partial object.
 
-## Reading files
+## Reading buffered files
 
 Use `getUploadedFile()` for optional files:
 
@@ -140,7 +283,7 @@ A missing required file throws `UploadError` with code `FILE_REQUIRED` and statu
 
 ## File constraints
 
-`getUploadedFile()`, `requireUploadedFile()` and `saveUploadedFile()` can validate:
+Buffered and streaming uploads can validate:
 
 ```ts
 {
@@ -160,9 +303,9 @@ Supported checks are:
 - MIME type allowlist,
 - extension allowlist.
 
-MIME type and file extension are metadata supplied with the upload. They are useful validation signals but are not content-signature verification. Applications accepting security-sensitive formats should inspect the file content or magic bytes before trusting the file type.
+MIME type and file extension are multipart metadata. They are useful validation signals but are not content-signature verification. Applications accepting security-sensitive formats should inspect the file content or magic bytes before trusting the file type.
 
-## Saving files
+## Saving buffered files
 
 ```ts
 const saved =
@@ -192,25 +335,31 @@ The result contains:
 
 Files are created with exclusive-write behavior by default. An existing destination produces `FILE_EXISTS` instead of silently overwriting data.
 
-Use `overwrite: true` only when replacement is intentional:
+Use `overwrite: true` only when replacement is intentional.
+
+## Streaming result
+
+`storeMultipartFile()` returns storage metadata together with multipart identity:
 
 ```ts
-await saveUploadedFile(
-    file,
-    {
-        directory:
-            "./uploads",
-        fileName:
-            "avatar.webp",
-        overwrite:
-            true,
-    }
-);
+{
+    key: string;
+    size: number;
+    contentType: string;
+    lastModified: Date;
+    etag: string;
+    checksumSha256?: string;
+    fieldName: string;
+    originalName: string;
+    encoding: string;
+}
 ```
 
+The checksum is optional because not every storage provider exposes a portable whole-object SHA-256 value for multipart uploads.
+
 ## File-name safety
 
-`saveUploadedFile()` sanitizes the final file name and verifies that the resolved destination remains inside the configured directory.
+`saveUploadedFile()` sanitizes the final local file name and verifies that the resolved destination remains inside the configured directory.
 
 You can use the same sanitizer independently:
 
@@ -225,7 +374,7 @@ const safeName =
     );
 ```
 
-Path separators, control characters and operating-system-invalid filename characters are removed or replaced.
+Streaming storage uses logical storage keys instead of direct operating-system paths. Storage adapters validate their own key rules; local/S3 BCP adapters reject traversal segments.
 
 ## Upload errors
 
@@ -273,8 +422,21 @@ INVALID_FILE_NAME
 FILE_EXISTS
 ```
 
-## Storage model
+The streaming helper maps storage `STREAM_TOO_LARGE` and `OBJECT_EXISTS` failures into the corresponding upload-facing error codes.
+
+## Choosing buffered vs streaming
+
+Prefer the buffered API when:
+
+- uploads are small,
+- the route needs several regular form fields/files as `FormData`,
+- application code needs random access to the parsed file before storage.
+
+Prefer `storeMultipartFile()` when:
 
-The 0.1.24 foundation writes to the local filesystem. It does not provide S3/object-storage adapters, multipart streaming directly to object storage, virus scanning or image transcoding yet.
+- files can be large,
+- the file should be written directly to local/shared/object storage,
+- memory usage should stay bounded,
+- S3 multipart upload is required for streams with unknown final size.
 
-For horizontally scaled applications, use shared/object storage rather than relying on instance-local disk.
+The `0.1.26` streaming API intentionally targets one named file field per call. Multiple-file orchestration and richer streaming form-field APIs can be added later without changing the storage contract.

package/docs/releases/0.1.26.md

@@ -0,0 +1,281 @@
+# BCP Framework 0.1.26
+
+BCP Framework `0.1.26` is the cloud-storage and production-streaming milestone.
+
+> Release state: release candidate source complete; not published until the full RC checks pass and npm publication completes.
+
+## Highlights
+
+- Added additive storage streaming capabilities without breaking `0.1.25` adapters.
+- Added native local filesystem streaming reads/writes.
+- Added S3-compatible storage through `createS3Storage()`.
+- Added direct multipart-to-storage streaming through `storeMultipartFile()`.
+- Added streamed full and ranged file delivery.
+- Added stream size limits and abort propagation.
+- Added cleanup/abort behavior for incomplete local and S3 multipart writes.
+- Added S3 integration coverage using a local S3-compatible test endpoint.
+- Added package smoke coverage for cloud-storage dependencies and public exports.
+
+## Storage streaming foundation
+
+New public helpers:
+
+```ts
+import {
+    getStorageCapabilities,
+    putStorageStream,
+    readStorageStream,
+} from "bcp/server";
+```
+
+New public types:
+
+```ts
+StorageAdapterCapabilities
+StoragePutStreamOptions
+StorageReadableStream
+```
+
+The streaming methods on `StorageAdapter` are optional:
+
+```ts
+interface StorageAdapter {
+    put(...): Promise<StorageObjectMetadata>;
+    putStream?(...): Promise<StorageObjectMetadata>;
+
+    read(...): Promise<Uint8Array>;
+    readStream?(...): Promise<StorageReadableStream>;
+
+    stat(...): Promise<StorageObjectMetadata | null>;
+    exists(...): Promise<boolean>;
+    delete(...): Promise<boolean>;
+}
+```
+
+`putStorageStream()` / `readStorageStream()` automatically use native streaming methods when available and preserve compatibility with legacy buffered adapters.
+
+## Local storage streaming
+
+`createLocalStorage()` now advertises:
+
+```ts
+{
+    streamingRead: true,
+    streamingWrite: true,
+    ranges: true,
+    signedUrls: false,
+    listing: false,
+}
+```
+
+Streaming local writes:
+
+- write chunks directly to disk,
+- calculate SHA-256 while consuming the stream,
+- enforce `maxBytes`,
+- honor `AbortSignal`,
+- remove incomplete objects when the operation fails.
+
+Streaming local reads support full objects and inclusive byte ranges without first reading the complete file into memory.
+
+## S3-compatible storage
+
+New API:
+
+```ts
+import {
+    createS3Storage,
+} from "bcp/server";
+
+const storage =
+    createS3Storage({
+        bucket:
+            process.env.S3_BUCKET!,
+        region:
+            process.env.S3_REGION!,
+        endpoint:
+            process.env.S3_ENDPOINT,
+        accessKeyId:
+            process.env.S3_ACCESS_KEY_ID,
+        secretAccessKey:
+            process.env.S3_SECRET_ACCESS_KEY,
+    });
+```
+
+The adapter supports:
+
+- `put()` / conditional create-only writes,
+- `putStream()` through AWS SDK multipart upload,
+- `stat()` through `HeadObject`,
+- `read()` / `readStream()` through `GetObject`,
+- inclusive ranges,
+- `exists()`,
+- `delete()`,
+- logical key prefixes,
+- custom endpoints,
+- path-style endpoints for MinIO/S3-compatible services,
+- AWS SDK default credential provider behavior when explicit credentials are omitted,
+- injected `S3Client` instances for advanced deployments.
+
+The S3 adapter exposes `destroy()` for internally owned AWS SDK clients.
+
+Buffered BCP writes persist SHA-256 as S3 user metadata. Whole-object SHA-256 remains optional for multipart streams because cloud providers do not expose one portable checksum format across all multipart implementations.
+
+## Production multipart streaming
+
+New API:
+
+```ts
+import {
+    storeMultipartFile,
+} from "bcp/server";
+```
+
+Example:
+
+```ts
+const stored =
+    await storeMultipartFile(
+        request,
+        {
+            storage,
+            fieldName:
+                "file",
+            key:
+                "documents/report.pdf",
+            maxBytes:
+                100 * 1024 * 1024,
+            constraints: {
+                maxBytes:
+                    80 * 1024 * 1024,
+                allowedTypes: [
+                    "application/pdf",
+                ],
+                allowedExtensions: [
+                    ".pdf",
+                ],
+            },
+        }
+    );
+```
+
+The multipart parser consumes `Request.body` incrementally and sends the selected file field directly into `putStorageStream()`.
+
+This avoids calling `request.formData()` and avoids requiring the complete file to exist in application memory before storage begins.
+
+The existing buffered APIs remain supported:
+
+```text
+parseMultipartFormData()
+getUploadedFile()
+requireUploadedFile()
+saveUploadedFile()
+storeUploadedFile()
+```
+
+## Upload validation and cleanup
+
+`storeMultipartFile()` supports:
+
+- total multipart request limit (`maxBytes`),
+- per-file limit (`constraints.maxBytes`),
+- MIME allowlists,
+- extension allowlists,
+- fixed or callback-generated storage keys,
+- caller `AbortSignal`,
+- one required named file field per operation.
+
+Storage failures are translated where appropriate:
+
+```text
+STREAM_TOO_LARGE -> FILE_TOO_LARGE
+OBJECT_EXISTS    -> FILE_EXISTS
+```
+
+Partial local objects are removed on failure. S3 multipart uploads use automatic part cleanup when upload completion fails.
+
+## File delivery
+
+`createStorageResponse()` uses `readStorageStream()` for response bodies.
+
+Streaming-capable adapters therefore support:
+
+```text
+200 full object
+206 single byte range
+HEAD without body read
+304 validators
+416 invalid ranges
+```
+
+The existing ETag, Last-Modified, If-Range, safe Content-Disposition and conservative private-cache behavior remain unchanged.
+
+## Package/runtime dependencies
+
+The published framework package now carries the runtime dependencies required by this milestone:
+
+```text
+@aws-sdk/client-s3
+@aws-sdk/lib-storage
+busboy
+```
+
+Package smoke tests verify these dependencies are present in the staged npm artifact.
+
+## Tests
+
+`0.1.26` adds coverage for:
+
+- local streaming writes and ranges,
+- legacy adapter streaming fallbacks,
+- stream size cleanup,
+- streaming multipart uploads to local storage,
+- multipart MIME and size failures,
+- S3 buffered writes,
+- S3 no-overwrite behavior,
+- S3 streaming writes,
+- S3 ranged reads,
+- S3 streamed reads,
+- S3 deletes,
+- public `bcp/server` exports,
+- packaged cloud-storage dependencies/artifacts.
+
+The S3 integration test uses a local S3-compatible HTTP fixture; it does not require an AWS account.
+
+## Backward compatibility
+
+`0.1.26` is additive to the `0.1.25` storage contract.
+
+Existing applications can continue using:
+
+```ts
+storage.put(...)
+storage.read(...)
+storeUploadedFile(...)
+createStorageResponse(...)
+```
+
+Custom `StorageAdapter` implementations do not need to implement streaming methods immediately. Generic helpers fall back to the required buffer APIs.
+
+## Intentional limitations
+
+The following are not part of `0.1.26`:
+
+- presigned URLs,
+- object listing,
+- copy/move helpers,
+- multiple HTTP byte ranges,
+- multi-file streaming multipart orchestration,
+- generated cloud-provider-specific adapters beyond the S3 API.
+
+These can be added in later milestones without expanding this release further.
+
+## Documentation
+
+- [File Upload](../file-upload.md)
+- [Storage and File Delivery](../storage.md)
+- [S3-Compatible Storage](../s3-storage.md)
+
+## Next milestone
+
+The recommended next storage/DX milestone is `0.1.27`, focusing on higher-level storage operations such as signed URLs/listing where needed, plus broader production hardening rather than changing the `0.1.26` core streaming contract.