@allegria/aws-file-manager 0.0.1 → 1.0.2

package/README.md

@@ -1,11 +1,6 @@
 # AWS File Manager
 
-A TypeScript library for managing files in AWS S3.
-
-## Features
-
-- Upload and download files from S3
-- Fully typed with TypeScript
+A TypeScript library for managing files in AWS S3. Handles uploads, signed URL generation, downloads, deletes, copies, and bucket listing — with full TypeScript types and a clean API designed for server-side Node.js applications.
 
 ## Installation
 
@@ -13,21 +8,165 @@ A TypeScript library for managing files in AWS S3.
 npm install @allegria/aws-file-manager
 ```
 
-## Examples
+## Why this library
+
+- **No ACL footgun.** ACL headers are omitted by default, which is correct for all buckets created after April 2023 (BucketOwnerEnforced). Passing an ACL to such a bucket throws a hard AWS error.
+- **UUID filenames.** `generateUniqueFileName: true` replaces the original filename with a UUID while preserving the extension. Timestamps collide under concurrent uploads; UUIDs don't.
+- **Keys and signed URLs are separate.** `upload()` returns only the S3 key (persist this to your database). `getSignedUrl()` is a separate call you make at request time — signed URLs expire and must never be stored.
 
-#### Basic usage
+## Quick start
+
+```ts
+import { AwsFileManager, fromMulterFile } from "@allegria/aws-file-manager";
 
-```js
 const fileManager = new AwsFileManager({
-  region: AWS_REGION,
-  bucketName: AWS_BUCKET_NAME,
-  accessKeyId: AWS_ACCESS_KEY,
-  secretAccessKey: AWS_SECRET_ACCESS_KEY,
+  region: "us-east-1",
+  bucketName: "my-app-uploads",
+  basePath: "uploads",          // optional: namespaces all keys under this prefix
 });
 
-// To upload
-const uploadResult = await fileManager.upload('my-photo.jpg');
+// In an Express/multer route:
+const fileInput = fromMulterFile(req.file);
+
+const result = await fileManager.upload(fileInput, {
+  folder: "avatars",
+  generateUniqueFileName: true,
+});
+
+// Persist result.key to your database — not the URL
+// await db.files.create({ s3Key: result.key });
+
+// Generate a signed URL on demand (e.g. when serving the file to a client)
+const url = await fileManager.getSignedUrl(result.key, {
+  disposition: "inline",
+});
+```
+
+## Core concepts
+
+### FileInput and adapters
+
+`upload()` takes a `FileInput` — a normalised object with `buffer`, `originalName`, `mimeType`, and `size`. Use the provided adapters to construct one:
+
+```ts
+// Express + multer (server-side)
+import { fromMulterFile } from "@allegria/aws-file-manager";
+const fileInput = fromMulterFile(req.file);
+
+// Next.js App Router / Web API File (browser or edge)
+import { fromWebFile } from "@allegria/aws-file-manager";
+const fileInput = await fromWebFile(formData.get("file") as File);
+
+// Or build it directly
+const fileInput: FileInput = {
+  buffer: myBuffer,
+  originalName: "photo.jpg",
+  mimeType: "image/jpeg",
+  size: myBuffer.length,
+};
+```
+
+### Keys vs signed URLs
+
+| What | Where to store | Lifetime |
+|---|---|---|
+| S3 key (`result.key`) | Your database | Permanent |
+| Signed URL | Never store | Minutes to hours |
+
+The S3 key is the stable identifier for a file. Generate a signed URL at request time when you need to give a client access to a private file.
+
+### basePath namespacing
+
+Set `basePath` in the constructor to prefix every key with a sub-folder:
+
+```ts
+const fm = new AwsFileManager({ ..., basePath: "uploads" });
+// upload to folder 'avatars' → key: 'uploads/avatars/<filename>'
+```
+
+## API reference
+
+### Constructor
+
+```ts
+new AwsFileManager(config: AwsFileManagerConfig)
+```
+
+See [Configuration reference](#configuration-reference) below.
+
+### Methods
+
+| Method | Signature | Returns | Notes |
+|---|---|---|---|
+| `upload` | `(file: FileInput, options?: UploadOptions)` | `Promise<UploadResult>` | Stores the file; returns key + metadata |
+| `getSignedUrl` | `(key: string, options?: SignedUrlOptions)` | `Promise<string>` | Short-lived presigned URL for private objects |
+| `download` | `(key: string, mode?: 'buffer' \| 'stream')` | `Promise<DownloadResult \| null>` | Returns `null` when key not found |
+| `delete` | `(key: string)` | `Promise<void>` | Idempotent — missing key does not throw |
+| `deleteMany` | `(keys: string[])` | `Promise<void>` | Chunks at 1 000 keys per S3 request |
+| `copy` | `(sourceKey, destKey, options?)` | `Promise<void>` | Server-side copy within the bucket |
+| `list` | `(options?: ListOptions)` | `Promise<ListResult>` | Paginated via `continuationToken` |
+| `exists` | `(key: string)` | `Promise<boolean>` | Lightweight key existence check |
+| `getS3Client` | `()` | `S3Client` | Access the underlying client for advanced use |
+
+### Adapter functions
+
+| Function | Signature | Notes |
+|---|---|---|
+| `fromMulterFile` | `(multerFile) => FileInput` | Sync — for Express + multer |
+| `fromWebFile` | `(webFile: File) => Promise<FileInput>` | Async — for Web API File / Next.js App Router |
+
+## Examples
+
+The `examples/` directory contains runnable TypeScript snippets for every method:
+
+| File | Description |
+|---|---|
+| [01-setup.ts](examples/01-setup.ts) | Instantiation: explicit credentials, env vars, IAM role |
+| [02-upload-multer.ts](examples/02-upload-multer.ts) | Upload from Express + multer |
+| [03-upload-web.ts](examples/03-upload-web.ts) | Upload from Next.js App Router (Web API File) |
+| [04-signed-urls.ts](examples/04-signed-urls.ts) | Inline, attachment, and custom-TTL signed URLs |
+| [05-download.ts](examples/05-download.ts) | Download as Buffer or stream; pipe to HTTP response |
+| [06-delete.ts](examples/06-delete.ts) | Delete single file or all variants at once |
+| [07-copy.ts](examples/07-copy.ts) | Copy / move objects (server-side, no re-upload) |
+| [08-list-paginated.ts](examples/08-list-paginated.ts) | Paginated listing for reconciliation jobs |
+| [09-exists.ts](examples/09-exists.ts) | Key existence check for integrity validation |
+
+## Configuration reference
+
+```ts
+interface AwsFileManagerConfig {
+  region: string;                // AWS region, e.g. 'us-east-1'
+  bucketName: string;            // S3 bucket name
+  accessKeyId?: string;          // Omit to use environment/IAM resolution
+  secretAccessKey?: string;      // Omit to use environment/IAM resolution
+  basePath?: string;             // Prefix for all keys, e.g. 'uploads'
+  urlExpirationSeconds?: number; // Signed URL TTL (default: 3600)
+  storageClass?: StorageClass;   // Default storage class (default: INTELLIGENT_TIERING)
+}
+```
+
+**Credential resolution order** (when `accessKeyId`/`secretAccessKey` are omitted):
+
+1. `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` environment variables
+2. `~/.aws/credentials` file
+3. EC2/ECS/Lambda instance metadata (IAM role) — recommended for production
+
+## Notes
+
+**ACL behaviour.** No ACL is sent with `PutObjectCommand`. This is correct for all S3 buckets created after April 2023, which use `BucketOwnerEnforced` by default. If your bucket predates that change and requires legacy ACLs, call `getS3Client()` and issue the command directly.
+
+**Storage class.** The default storage class is `INTELLIGENT_TIERING`, which automatically moves objects between access tiers based on usage patterns. Override per-upload via `UploadOptions.storageClass`, or change the instance default via `AwsFileManagerConfig.storageClass`.
+
+**`deleteMany` chunking.** The S3 batch delete API accepts at most 1 000 keys per request. `deleteMany` splits larger arrays into 1 000-key chunks and sends them in parallel automatically.
+
+## Development
+
+```bash
+pnpm test          # run tests once
+pnpm test:watch    # run tests in watch mode
+pnpm build         # compile TypeScript
+```
+
+## License
 
-// To download
-const downloadResult = await fileManager.download('my-photo.jpg', 'buffer');
-```
+MIT

package/dist/cjs/aws-file-manager.d.ts

@@ -0,0 +1,318 @@
+import { S3Client, StorageClass } from "@aws-sdk/client-s3";
+import type { Readable } from "stream";
+/**
+ * Configuration options for AwsFileManager.
+ * Credentials are optional — when omitted, the SDK resolves them from the
+ * environment (IAM role, AWS_ACCESS_KEY_ID, ~/.aws/credentials, etc.).
+ * Prefer environment/IAM resolution in production.
+ */
+export interface AwsFileManagerConfig {
+    /** AWS region, e.g. 'us-east-1' */
+    region: string;
+    /** S3 bucket name */
+    bucketName: string;
+    /** AWS access key ID — omit to rely on environment/IAM resolution */
+    accessKeyId?: string;
+    /** AWS secret access key — omit to rely on environment/IAM resolution */
+    secretAccessKey?: string;
+    /**
+     * Base path prefix prepended to every key (no leading/trailing slashes).
+     * Useful for namespacing all writes to a sub-folder, e.g. 'uploads'.
+     */
+    basePath?: string;
+    /** Signed URL TTL in seconds (default: 3600) */
+    urlExpirationSeconds?: number;
+    /** Default S3 storage class (default: INTELLIGENT_TIERING) */
+    storageClass?: StorageClass;
+}
+/**
+ * A normalised file input that works for both server-side (multer Buffer) and
+ * browser-side (Web API File) callers. Use `fromMullerFile()` or
+ * `fromWebFile()` helpers to construct one, or build it directly.
+ */
+export interface FileInput {
+    /** Raw file bytes */
+    buffer: Buffer;
+    /** Original filename as provided by the uploader */
+    originalName: string;
+    /** MIME type, e.g. 'image/jpeg' */
+    mimeType: string;
+    /** File size in bytes */
+    size: number;
+}
+/** Options that control how a single file is stored in S3. */
+export interface UploadOptions {
+    /**
+     * Full S3 key (path + filename) for this object, relative to basePath.
+     * When provided, `folder` and `generateUniqueFileName` are ignored.
+     * Prefer this in pipeline usage where the calling code owns key generation.
+     */
+    key?: string;
+    /**
+     * Folder path appended to basePath (no leading/trailing slashes).
+     * Used when `key` is not provided.
+     */
+    folder?: string;
+    /**
+     * Custom filename to use instead of originalName.
+     * Used when `key` is not provided.
+     */
+    fileName?: string;
+    /**
+     * When true and `key` is not provided, replaces the filename with a UUID
+     * while preserving the extension. Prevents collisions under concurrent
+     * uploads.
+     */
+    generateUniqueFileName?: boolean;
+    /** Override the detected MIME type */
+    contentType?: string;
+    /** Arbitrary string metadata stored alongside the S3 object */
+    metadata?: Record<string, string>;
+    /** Per-upload storage class override */
+    storageClass?: StorageClass;
+}
+/** Options for listing objects in a folder */
+export interface ListOptions {
+    /** Folder path appended to basePath */
+    folder?: string;
+    /** Additional prefix filter within the folder */
+    prefix?: string;
+    /** Maximum number of keys to return (1–1000, default: 1000) */
+    maxResults?: number;
+    /** Continuation token from a previous list call for pagination */
+    continuationToken?: string;
+}
+/** Options for generating a signed URL for an existing key */
+export interface SignedUrlOptions {
+    /** TTL override in seconds. Falls back to instance default. */
+    expiresIn?: number;
+    /**
+     * Sets Content-Disposition on the response.
+     * 'inline' — browser renders the file (images, PDFs).
+     * 'attachment' — browser downloads the file.
+     */
+    disposition?: "inline" | "attachment";
+    /** Original filename to suggest in Content-Disposition (for attachments) */
+    fileName?: string;
+}
+/** Metadata returned after a successful upload */
+export interface UploadResult {
+    /** Full S3 key for this object — store this in your database */
+    key: string;
+    /** Original filename as provided by the uploader */
+    originalName: string;
+    /** Filename used in S3 (may differ from originalName if UUID was generated) */
+    storedName: string;
+    /** Stored file size in bytes */
+    size: number;
+    /** MIME type */
+    contentType: string;
+    /** S3 ETag (without surrounding quotes) */
+    etag?: string;
+    /** Storage class applied */
+    storageClass: string;
+}
+/** A single entry returned by list() */
+export interface ListEntry {
+    /** Full S3 key */
+    key: string;
+    /** Filename portion of the key */
+    fileName: string;
+    /** Object size in bytes */
+    size: number;
+    /** Last modified timestamp */
+    lastModified: Date;
+    /** Storage class */
+    storageClass?: string;
+    /** ETag (without surrounding quotes) */
+    etag?: string;
+}
+/** Paginated result from list() */
+export interface ListResult {
+    entries: ListEntry[];
+    /** Pass this to the next list() call to fetch the next page */
+    continuationToken?: string;
+    /** True when there are more results beyond this page */
+    hasMore: boolean;
+}
+export type DownloadMode = "buffer" | "stream";
+export type DownloadResult<T extends DownloadMode> = T extends "buffer" ? {
+    buffer: Buffer;
+    metadata: ObjectMetadata;
+} : {
+    stream: Readable;
+    metadata: ObjectMetadata;
+};
+export interface ObjectMetadata {
+    contentType?: string;
+    contentLength?: number;
+    lastModified?: Date;
+    metadata?: Record<string, string>;
+    storageClass?: string;
+}
+/**
+ * Adapts a multer file (Express.Multer.File) to FileInput.
+ *
+ * @example
+ * // In an Express route with multer:
+ * const fileInput = fromMulterFile(req.file);
+ */
+export declare function fromMulterFile(multerFile: {
+    buffer: Buffer;
+    originalname: string;
+    mimetype: string;
+    size: number;
+}): FileInput;
+/**
+ * Adapts a Web API File (browser / Next.js App Router FormData) to FileInput.
+ *
+ * @example
+ * // In a Next.js App Router route handler:
+ * const formData = await request.formData();
+ * const fileInput = await fromWebFile(formData.get('file') as File);
+ */
+export declare function fromWebFile(webFile: File): Promise<FileInput>;
+export declare class AwsFileManager {
+    private readonly s3;
+    private readonly bucketName;
+    private readonly basePath;
+    private readonly urlExpirationSeconds;
+    private readonly storageClass;
+    static readonly DEFAULTS: {
+        readonly storageClass: "INTELLIGENT_TIERING";
+        readonly urlExpirationSeconds: 3600;
+    };
+    constructor(config: AwsFileManagerConfig);
+    /**
+     * Uploads a file to S3 and returns storage metadata (no URL).
+     *
+     * The returned `key` is what you persist to your database. Generate signed
+     * URLs on demand via `getSignedUrl()` — never store them, as they expire.
+     *
+     * @example
+     * // In a pipeline step:
+     * const result = await fileManager.upload(ctx.fileInput, { key: ctx.s3Key });
+     * ctx.uploadResult = result;
+     */
+    upload(file: FileInput, options?: UploadOptions): Promise<UploadResult>;
+    /**
+     * Generates a short-lived signed URL for a private S3 object.
+     *
+     * Call this at request time; never store the returned URL. Signed URLs
+     * are credentials — treat them accordingly.
+     *
+     * @example
+     * const url = await fileManager.getSignedUrl(file.s3Key, {
+     *   disposition: 'inline',
+     *   fileName: file.originalName,
+     * });
+     */
+    getSignedUrl(key: string, options?: SignedUrlOptions): Promise<string>;
+    /**
+     * Downloads an S3 object as a Buffer or a Node.js Readable stream.
+     *
+     * Use `'stream'` (default) when piping to a response or writing to disk.
+     * Use `'buffer'` when you need the full bytes in memory (e.g. for processing).
+     *
+     * Returns `null` when the object does not exist.
+     *
+     * @example
+     * const result = await fileManager.download(key, 'buffer');
+     * if (result) {
+     *   const { buffer, metadata } = result;
+     * }
+     */
+    download<T extends DownloadMode = "stream">(key: string, mode?: T): Promise<DownloadResult<T> | null>;
+    /**
+     * Permanently deletes an object from S3.
+     *
+     * Note: this only removes the S3 object. Your database record should be
+     * soft-deleted first, and this called as a cleanup step after the DB
+     * transaction commits — so a crash mid-way leaves a recoverable orphan
+     * rather than a missing file with an intact DB row.
+     *
+     * @example
+     * await fileManager.delete(file.s3Key);
+     */
+    delete(key: string): Promise<void>;
+    /**
+     * Deletes multiple objects in a single call.
+     * Silently skips keys that do not exist.
+     *
+     * @example
+     * await fileManager.deleteMany([original.s3Key, thumbnail.s3Key]);
+     */
+    deleteMany(keys: string[]): Promise<void>;
+    /**
+     * Copies an object within the same bucket.
+     *
+     * Useful when duplicating files across entities (e.g. cloning a note)
+     * or reorganising keys without re-uploading bytes.
+     *
+     * @example
+     * await fileManager.copy(sourceFile.s3Key, newKey);
+     */
+    copy(sourceKey: string, destinationKey: string, options?: {
+        storageClass?: StorageClass;
+        metadata?: Record<string, string>;
+    }): Promise<void>;
+    /**
+     * Lists objects under a folder prefix.
+     *
+     * Results are paginated. Pass the returned `continuationToken` back
+     * into subsequent calls to walk through large result sets.
+     *
+     * Primarily intended for the server-side storage reconciliation job
+     * (nightly scan to reconcile DB byte counts against S3 actuals).
+     *
+     * @example
+     * let token: string | undefined;
+     * do {
+     *   const result = await fileManager.list({ folder: 'acme-corp', continuationToken: token });
+     *   processBatch(result.entries);
+     *   token = result.continuationToken;
+     * } while (result.hasMore);
+     */
+    list(options?: ListOptions): Promise<ListResult>;
+    /**
+     * Returns true if an object exists at the given key.
+     *
+     * @example
+     * if (!(await fileManager.exists(key))) {
+     *   throw new Error('Referenced file is missing from storage');
+     * }
+     */
+    exists(key: string): Promise<boolean>;
+    /**
+     * Returns the underlying S3Client for operations not covered by this class.
+     * Use sparingly — prefer adding methods here so the abstraction stays coherent.
+     */
+    getS3Client(): S3Client;
+    /**
+     * Resolves the final S3 key and stored filename from upload options.
+     *
+     * Priority order for the key:
+     *   1. options.key (explicit full key — pipeline usage)
+     *   2. basePath + folder + fileName (if provided)
+     *   3. basePath + folder + UUID-based name (if generateUniqueFileName)
+     *   4. basePath + folder + originalName (fallback)
+     */
+    private resolveKey;
+    /**
+     * Builds a full S3 key from basePath, optional folder, and filename.
+     */
+    private buildFullKey;
+    /**
+     * Builds a prefix string for list operations.
+     */
+    private buildPrefix;
+    /**
+     * Generates a UUID-based filename, preserving the original extension.
+     * UUIDs are collision-proof under any upload concurrency.
+     */
+    private makeUniqueFileName;
+    /**
+     * Builds a Content-Disposition header value from SignedUrlOptions.
+     */
+    private buildContentDisposition;
+}