express-storage 1.0.0 → 1.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (48) hide show
  1. package/README.md +519 -348
  2. package/dist/drivers/azure.driver.d.ts +88 -0
  3. package/dist/drivers/azure.driver.d.ts.map +1 -0
  4. package/dist/drivers/azure.driver.js +367 -0
  5. package/dist/drivers/azure.driver.js.map +1 -0
  6. package/dist/drivers/base.driver.d.ts +125 -24
  7. package/dist/drivers/base.driver.d.ts.map +1 -1
  8. package/dist/drivers/base.driver.js +248 -62
  9. package/dist/drivers/base.driver.js.map +1 -1
  10. package/dist/drivers/gcs.driver.d.ts +60 -13
  11. package/dist/drivers/gcs.driver.d.ts.map +1 -1
  12. package/dist/drivers/gcs.driver.js +242 -41
  13. package/dist/drivers/gcs.driver.js.map +1 -1
  14. package/dist/drivers/local.driver.d.ts +89 -12
  15. package/dist/drivers/local.driver.d.ts.map +1 -1
  16. package/dist/drivers/local.driver.js +533 -45
  17. package/dist/drivers/local.driver.js.map +1 -1
  18. package/dist/drivers/s3.driver.d.ts +64 -13
  19. package/dist/drivers/s3.driver.d.ts.map +1 -1
  20. package/dist/drivers/s3.driver.js +269 -41
  21. package/dist/drivers/s3.driver.js.map +1 -1
  22. package/dist/factory/driver.factory.d.ts +35 -29
  23. package/dist/factory/driver.factory.d.ts.map +1 -1
  24. package/dist/factory/driver.factory.js +119 -59
  25. package/dist/factory/driver.factory.js.map +1 -1
  26. package/dist/index.d.ts +23 -22
  27. package/dist/index.d.ts.map +1 -1
  28. package/dist/index.js +26 -46
  29. package/dist/index.js.map +1 -1
  30. package/dist/storage-manager.d.ts +205 -52
  31. package/dist/storage-manager.d.ts.map +1 -1
  32. package/dist/storage-manager.js +644 -73
  33. package/dist/storage-manager.js.map +1 -1
  34. package/dist/types/storage.types.d.ts +243 -18
  35. package/dist/types/storage.types.d.ts.map +1 -1
  36. package/dist/utils/config.utils.d.ts +28 -4
  37. package/dist/utils/config.utils.d.ts.map +1 -1
  38. package/dist/utils/config.utils.js +121 -47
  39. package/dist/utils/config.utils.js.map +1 -1
  40. package/dist/utils/file.utils.d.ts +111 -14
  41. package/dist/utils/file.utils.d.ts.map +1 -1
  42. package/dist/utils/file.utils.js +215 -32
  43. package/dist/utils/file.utils.js.map +1 -1
  44. package/package.json +51 -27
  45. package/dist/drivers/oci.driver.d.ts +0 -37
  46. package/dist/drivers/oci.driver.d.ts.map +0 -1
  47. package/dist/drivers/oci.driver.js +0 -84
  48. package/dist/drivers/oci.driver.js.map +0 -1
package/README.md CHANGED
@@ -1,490 +1,661 @@
1
1
  # Express Storage
2
2
 
3
- A powerful, TypeScript-first file upload and storage management package for Express.js applications. Supports multiple cloud storage providers with a simple, unified API.
3
+ **Secure, unified file uploads for Express.js one API for all cloud providers.**
4
4
 
5
- ## 🚀 Features
5
+ Stop writing separate upload code for every storage provider. Express Storage gives you a single, secure interface that works with AWS S3, Google Cloud Storage, Azure Blob Storage, and local disk. Switch providers by changing one environment variable. No code changes required.
6
6
 
7
- - **TypeScript First**: Fully written in TypeScript with complete type definitions
8
- - **Multiple Storage Drivers**: Support for AWS S3, Google Cloud Storage, Oracle Cloud Infrastructure, and local storage
9
- - **Presigned URLs**: Generate secure, time-limited URLs for direct client uploads
10
- - **Flexible File Handling**: Support for single and multiple file uploads
11
- - **Automatic File Organization**: Files stored in month/year directories for local storage
12
- - **Unique File Naming**: Unix timestamp-based unique filenames with sanitization
13
- - **Environment-based Configuration**: Simple setup using environment variables
14
- - **Class-based API**: Clean, object-oriented interface with `StorageManager`
15
- - **Comprehensive Testing**: Full test coverage with Jest
16
- - **Error Handling**: Consistent error responses with detailed messages
7
+ [![npm version](https://img.shields.io/npm/v/express-storage.svg)](https://www.npmjs.com/package/express-storage)
8
+ [![TypeScript](https://img.shields.io/badge/TypeScript-Ready-blue.svg)](https://www.typescriptlang.org/)
9
+ [![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
17
10
 
18
- ## 📦 Installation
11
+ ---
12
+
13
+ ## Why Express Storage?
14
+
15
+ Every application needs file uploads. And every application gets it wrong at first.
16
+
17
+ You start with local storage, then realize you need S3 for production. You copy-paste upload code from Stack Overflow, then discover it's vulnerable to path traversal attacks. You build presigned URL support, then learn Azure handles it completely differently than AWS.
18
+
19
+ **Express Storage solves these problems once, so you don't have to.**
20
+
21
+ ### What Makes It Different
22
+
23
+ - **One API, Four Providers** — Write upload code once. Deploy to any cloud.
24
+ - **Security Built In** — Path traversal prevention, filename sanitization, file validation, and null byte protection come standard.
25
+ - **Presigned URLs Done Right** — Client-side uploads that bypass your server, with proper validation for each provider's quirks.
26
+ - **TypeScript Native** — Full type safety with intelligent autocomplete. No `any` types hiding bugs.
27
+ - **Zero Config Switching** — Change `FILE_DRIVER=local` to `FILE_DRIVER=s3` and you're done.
28
+
29
+ ---
30
+
31
+ ## Quick Start
32
+
33
+ ### Installation
19
34
 
20
35
  ```bash
21
36
  npm install express-storage
22
37
  ```
23
38
 
24
- ## 🔧 Quick Setup
39
+ ### Basic Setup
40
+
41
+ ```typescript
42
+ import express from "express";
43
+ import multer from "multer";
44
+ import { StorageManager } from "express-storage";
45
+
46
+ const app = express();
47
+ const upload = multer();
48
+ const storage = new StorageManager();
49
+
50
+ app.post("/upload", upload.single("file"), async (req, res) => {
51
+ const result = await storage.uploadFile(req.file, {
52
+ maxSize: 10 * 1024 * 1024, // 10MB limit
53
+ allowedMimeTypes: ["image/jpeg", "image/png", "application/pdf"],
54
+ });
25
55
 
26
- ### 1. Environment Configuration
56
+ if (result.success) {
57
+ res.json({ url: result.fileUrl });
58
+ } else {
59
+ res.status(400).json({ error: result.error });
60
+ }
61
+ });
62
+ ```
27
63
 
28
- Create a `.env` file in your project root:
64
+ ### Environment Configuration
65
+
66
+ Create a `.env` file:
29
67
 
30
68
  ```env
31
- # Required: Choose your storage driver
69
+ # Choose your storage provider
32
70
  FILE_DRIVER=local
33
71
 
34
- # For local storage (optional - defaults to public/express-storage)
35
- LOCAL_PATH=public/uploads
72
+ # For local storage
73
+ LOCAL_PATH=uploads
36
74
 
37
- # For cloud storage (AWS S3 example)
75
+ # For AWS S3
38
76
  FILE_DRIVER=s3
39
77
  BUCKET_NAME=my-bucket
40
78
  AWS_REGION=us-east-1
41
- AWS_ACCESS_KEY=your-access-key
42
- AWS_SECRET_KEY=your-secret-key
79
+ AWS_ACCESS_KEY=your-key
80
+ AWS_SECRET_KEY=your-secret
81
+
82
+ # For Google Cloud Storage
83
+ FILE_DRIVER=gcs
84
+ BUCKET_NAME=my-bucket
85
+ GCS_PROJECT_ID=my-project
43
86
 
44
- # Optional: Presigned URL expiry (default: 600 seconds / 10 minutes)
45
- PRESIGNED_URL_EXPIRY=600
87
+ # For Azure Blob Storage
88
+ FILE_DRIVER=azure
89
+ BUCKET_NAME=my-container
90
+ AZURE_CONNECTION_STRING=your-connection-string
46
91
  ```
47
92
 
48
- ### 2. Basic Usage
93
+ That's it. Your upload code stays the same regardless of which provider you choose.
94
+
95
+ ---
96
+
97
+ ## Supported Storage Providers
98
+
99
+ | Provider | Direct Upload | Presigned URLs | Best For |
100
+ | ---------------- | ------------- | ----------------- | ------------------------- |
101
+ | **Local Disk** | `local` | — | Development, small apps |
102
+ | **AWS S3** | `s3` | `s3-presigned` | Most production apps |
103
+ | **Google Cloud** | `gcs` | `gcs-presigned` | GCP-hosted applications |
104
+ | **Azure Blob** | `azure` | `azure-presigned` | Azure-hosted applications |
105
+
106
+ ---
107
+
108
+ ## Security Features
109
+
110
+ File uploads are one of the most exploited attack vectors in web applications. Express Storage protects you by default.
111
+
112
+ ### Path Traversal Prevention
113
+
114
+ Attackers try filenames like `../../../etc/passwd` to escape your upload directory. We block this:
49
115
 
50
116
  ```typescript
51
- import express from 'express';
52
- import multer from 'multer';
53
- import { StorageManager } from 'express-storage';
117
+ // These malicious filenames are automatically rejected
118
+ "../secret.txt"; // Blocked: path traversal
119
+ "..\\config.json"; // Blocked: Windows path traversal
120
+ "file\0.txt"; // Blocked: null byte injection
121
+ ```
54
122
 
55
- const app = express();
56
- const upload = multer();
123
+ ### Automatic Filename Sanitization
57
124
 
58
- // Initialize storage manager
59
- const storage = new StorageManager();
125
+ User-provided filenames can't be trusted. We transform them into safe, unique identifiers:
60
126
 
61
- // Single file upload
62
- app.post('/upload', upload.single('file'), async (req, res) => {
63
- try {
64
- const result = await storage.uploadFile(req.file!);
65
-
66
- if (result.success) {
67
- res.json({
68
- success: true,
69
- fileName: result.fileName,
70
- fileUrl: result.fileUrl
71
- });
72
- } else {
73
- res.status(400).json({ success: false, error: result.error });
74
- }
75
- } catch (error) {
76
- res.status(500).json({ success: false, error: 'Upload failed' });
77
- }
127
+ ```
128
+ User uploads: "My Photo (1).jpg"
129
+ Stored as: "1706123456789_a1b2c3d4e5_my_photo_1_.jpg"
130
+ ```
131
+
132
+ The format `{timestamp}_{random}_{sanitized_name}` prevents collisions and removes dangerous characters.
133
+
134
+ ### File Validation
135
+
136
+ Validate before processing. Reject before storing.
137
+
138
+ ```typescript
139
+ await storage.uploadFile(file, {
140
+ maxSize: 5 * 1024 * 1024, // 5MB limit
141
+ allowedMimeTypes: ["image/jpeg", "image/png"],
142
+ allowedExtensions: [".jpg", ".png"],
78
143
  });
144
+ ```
145
+
146
+ ### Presigned URL Security
147
+
148
+ For S3 and GCS, file constraints are enforced at the URL level — clients physically cannot upload the wrong file type or size. For Azure (which doesn't support URL-level constraints), we validate after upload and automatically delete invalid files.
149
+
150
+ ---
151
+
152
+ ## Presigned URLs: Client-Side Uploads
153
+
154
+ Large files shouldn't flow through your server. Presigned URLs let clients upload directly to cloud storage.
155
+
156
+ ### The Flow
157
+
158
+ ```
159
+ 1. Client → Your Server: "I want to upload photo.jpg (2MB, image/jpeg)"
160
+ 2. Your Server → Client: "Here's a presigned URL, valid for 10 minutes"
161
+ 3. Client → Cloud Storage: Uploads directly (your server never touches the bytes)
162
+ 4. Client → Your Server: "Upload complete, please verify"
163
+ 5. Your Server: Confirms file exists, returns permanent URL
164
+ ```
165
+
166
+ ### Implementation
167
+
168
+ ```typescript
169
+ // Step 1: Generate upload URL
170
+ app.post("/upload/init", async (req, res) => {
171
+ const { fileName, contentType, fileSize } = req.body;
172
+
173
+ const result = await storage.generateUploadUrl(
174
+ fileName,
175
+ contentType,
176
+ fileSize,
177
+ "user-uploads", // Optional folder
178
+ );
79
179
 
80
- // Multiple files upload
81
- app.post('/upload-multiple', upload.array('files', 10), async (req, res) => {
82
- try {
83
- const results = await storage.uploadFiles(req.files as Express.Multer.File[]);
84
-
85
- const successful = results.filter(r => r.success);
86
- const failed = results.filter(r => !r.success);
87
-
88
180
  res.json({
89
- success: true,
90
- uploaded: successful.length,
91
- failed: failed.length,
92
- results
181
+ uploadUrl: result.uploadUrl,
182
+ reference: result.reference, // Save this for later
93
183
  });
94
- } catch (error) {
95
- res.status(500).json({ success: false, error: 'Upload failed' });
96
- }
184
+ });
185
+
186
+ // Step 2: Confirm upload
187
+ app.post("/upload/confirm", async (req, res) => {
188
+ const { reference, expectedContentType, expectedFileSize } = req.body;
189
+
190
+ const result = await storage.validateAndConfirmUpload(reference, {
191
+ expectedContentType,
192
+ expectedFileSize,
193
+ });
194
+
195
+ if (result.success) {
196
+ res.json({ viewUrl: result.viewUrl });
197
+ } else {
198
+ res.status(400).json({ error: result.error });
199
+ }
97
200
  });
98
201
  ```
99
202
 
100
- ## 🗂️ Supported Storage Drivers
203
+ ### Provider-Specific Behavior
204
+
205
+ | Provider | Content-Type Enforced | File Size Enforced | Post-Upload Validation |
206
+ | -------- | --------------------- | ------------------ | ---------------------- |
207
+ | S3 | At URL level | At URL level | Optional |
208
+ | GCS | At URL level | At URL level | Optional |
209
+ | Azure | **Not enforced** | **Not enforced** | **Required** |
210
+
211
+ For Azure, always call `validateAndConfirmUpload()` with expected values. Invalid files are automatically deleted.
101
212
 
102
- | Driver | Type | Description | Required Environment Variables |
103
- |--------|------|-------------|-------------------------------|
104
- | `local` | Direct | Local file system storage | `LOCAL_PATH` (optional) |
105
- | `s3` | Direct | AWS S3 direct upload | `BUCKET_NAME`, `AWS_REGION`, `AWS_ACCESS_KEY`, `AWS_SECRET_KEY` |
106
- | `s3-presigned` | Presigned | AWS S3 presigned URLs | `BUCKET_NAME`, `AWS_REGION`, `AWS_ACCESS_KEY`, `AWS_SECRET_KEY` |
107
- | `gcs` | Direct | Google Cloud Storage direct upload | `BUCKET_NAME`, `GCS_PROJECT_ID`, `GCS_CREDENTIALS` |
108
- | `gcs-presigned` | Presigned | Google Cloud Storage presigned URLs | `BUCKET_NAME`, `GCS_PROJECT_ID`, `GCS_CREDENTIALS` |
109
- | `oci` | Direct | Oracle Cloud Infrastructure direct upload | `BUCKET_NAME`, `OCI_REGION`, `OCI_CREDENTIALS` |
110
- | `oci-presigned` | Presigned | Oracle Cloud Infrastructure presigned URLs | `BUCKET_NAME`, `OCI_REGION`, `OCI_CREDENTIALS` |
213
+ ---
214
+
215
+ ## Large File Uploads
111
216
 
112
- ## 📋 Environment Variables Reference
217
+ For files larger than 100MB, we recommend using **presigned URLs** instead of direct server uploads. Here's why:
113
218
 
114
- ### Core Configuration
115
- - `FILE_DRIVER` (required): Storage driver to use
116
- - `BUCKET_NAME`: Cloud storage bucket name
117
- - `LOCAL_PATH`: Local storage directory path (default: `public/express-storage`)
118
- - `PRESIGNED_URL_EXPIRY`: Presigned URL expiry in seconds (default: 600)
219
+ ### Memory Efficiency
119
220
 
120
- ### AWS S3 Configuration
121
- - `AWS_REGION`: AWS region (e.g., `us-east-1`)
122
- - `AWS_ACCESS_KEY`: AWS access key ID
123
- - `AWS_SECRET_KEY`: AWS secret access key
221
+ When you upload through your server, the entire file must be buffered in memory (or stored temporarily on disk). For a 500MB video file, that's 500MB of RAM per concurrent upload. With presigned URLs, the file goes directly to cloud storage — your server only handles small JSON requests.
124
222
 
125
- ### Google Cloud Storage Configuration
126
- - `GCS_PROJECT_ID`: Google Cloud project ID
127
- - `GCS_CREDENTIALS`: Path to service account JSON file
223
+ ### Automatic Streaming
128
224
 
129
- ### Oracle Cloud Infrastructure Configuration
130
- - `OCI_REGION`: OCI region (e.g., `us-ashburn-1`)
131
- - `OCI_CREDENTIALS`: Path to OCI credentials file
225
+ For files that must go through your server, Express Storage automatically uses streaming uploads for files larger than 100MB:
132
226
 
133
- ## 🔌 API Reference
227
+ - **S3**: Uses multipart upload with 10MB chunks
228
+ - **GCS**: Uses resumable uploads with streaming
229
+ - **Azure**: Uses block upload with streaming
134
230
 
135
- ### StorageManager Class
231
+ This happens transparently — you don't need to change your code.
136
232
 
137
- The main class for managing file storage operations.
233
+ ### Recommended Approach for Large Files
138
234
 
139
- #### Constructor
140
235
  ```typescript
141
- const storage = new StorageManager();
236
+ // Frontend: Request presigned URL
237
+ const { uploadUrl, reference } = await fetch("/api/upload/init", {
238
+ method: "POST",
239
+ body: JSON.stringify({
240
+ fileName: "large-video.mp4",
241
+ contentType: "video/mp4",
242
+ fileSize: 524288000, // 500MB
243
+ }),
244
+ }).then((r) => r.json());
245
+
246
+ // Frontend: Upload directly to cloud (bypasses your server!)
247
+ await fetch(uploadUrl, {
248
+ method: "PUT",
249
+ body: file,
250
+ headers: { "Content-Type": "video/mp4" },
251
+ });
252
+
253
+ // Frontend: Confirm upload
254
+ await fetch("/api/upload/confirm", {
255
+ method: "POST",
256
+ body: JSON.stringify({ reference }),
257
+ });
142
258
  ```
143
259
 
144
- #### Methods
260
+ ### Size Limits
145
261
 
146
- ##### File Upload
147
- ```typescript
148
- // Single file upload
149
- const result = await storage.uploadFile(file: Express.Multer.File): Promise<FileUploadResult>
262
+ | Scenario | Recommended Limit | Reason |
263
+ | ------------------------------ | ----------------- | ------------------------------ |
264
+ | Direct upload (memory storage) | < 100MB | Node.js memory constraints |
265
+ | Direct upload (disk storage) | < 500MB | Temp file management |
266
+ | Presigned URL upload | 5GB+ | Limited only by cloud provider |
150
267
 
151
- // Multiple files upload
152
- const results = await storage.uploadFiles(files: Express.Multer.File[]): Promise<FileUploadResult[]>
268
+ ---
153
269
 
154
- // Generic upload (handles both single and multiple)
155
- const result = await storage.upload(input: FileInput): Promise<FileUploadResult | FileUploadResult[]>
156
- ```
270
+ ## API Reference
157
271
 
158
- ##### Presigned URLs
159
- ```typescript
160
- // Generate upload URL
161
- const result = await storage.generateUploadUrl(fileName: string): Promise<PresignedUrlResult>
272
+ ### StorageManager
162
273
 
163
- // Generate view URL
164
- const result = await storage.generateViewUrl(fileName: string): Promise<PresignedUrlResult>
274
+ The main class you'll interact with.
165
275
 
166
- // Generate multiple upload URLs
167
- const results = await storage.generateUploadUrls(fileNames: string[]): Promise<PresignedUrlResult[]>
276
+ ```typescript
277
+ import { StorageManager } from "express-storage";
168
278
 
169
- // Generate multiple view URLs
170
- const results = await storage.generateViewUrls(fileNames: string[]): Promise<PresignedUrlResult[]>
279
+ // Use environment variables
280
+ const storage = new StorageManager();
281
+
282
+ // Or configure programmatically
283
+ const storage = new StorageManager({
284
+ driver: "s3",
285
+ credentials: {
286
+ bucketName: "my-bucket",
287
+ awsRegion: "us-east-1",
288
+ maxFileSize: 50 * 1024 * 1024, // 50MB
289
+ },
290
+ logger: console, // Optional: enable debug logging
291
+ });
171
292
  ```
172
293
 
173
- ##### File Deletion
294
+ ### File Upload Methods
295
+
174
296
  ```typescript
175
- // Delete single file
176
- const success = await storage.deleteFile(fileName: string): Promise<boolean>
297
+ // Single file
298
+ const result = await storage.uploadFile(file, validation?, options?);
177
299
 
178
- // Delete multiple files
179
- const results = await storage.deleteFiles(fileNames: string[]): Promise<boolean[]>
300
+ // Multiple files (processed in parallel with concurrency limits)
301
+ const results = await storage.uploadFiles(files, validation?, options?);
302
+
303
+ // Generic upload (auto-detects single vs multiple)
304
+ const result = await storage.upload(input, validation?, options?);
180
305
  ```
181
306
 
182
- ##### Utility Methods
307
+ ### Presigned URL Methods
308
+
183
309
  ```typescript
184
- // Get current configuration
185
- const config = storage.getConfig(): StorageConfig
310
+ // Generate upload URL with constraints
311
+ const result = await storage.generateUploadUrl(fileName, contentType?, fileSize?, folder?);
312
+
313
+ // Generate view URL for existing file
314
+ const result = await storage.generateViewUrl(reference);
186
315
 
187
- // Get driver type
188
- const driverType = storage.getDriverType(): string
316
+ // Validate upload (required for Azure, recommended for all)
317
+ const result = await storage.validateAndConfirmUpload(reference, options?);
189
318
 
190
- // Check if presigned URLs are supported
191
- const isSupported = storage.isPresignedSupported(): boolean
319
+ // Batch operations
320
+ const results = await storage.generateUploadUrls(files, folder?);
321
+ const results = await storage.generateViewUrls(references);
192
322
  ```
193
323
 
194
- ### Static Methods
324
+ ### File Management
195
325
 
196
326
  ```typescript
197
- // Initialize with custom configuration
198
- const storage = StorageManager.initialize({
199
- driver: 's3',
200
- bucketName: 'my-bucket',
201
- awsRegion: 'us-east-1'
202
- });
327
+ // Delete single file
328
+ const success = await storage.deleteFile(reference);
203
329
 
204
- // Get available drivers
205
- const drivers = StorageManager.getAvailableDrivers(): string[]
330
+ // Delete multiple files
331
+ const results = await storage.deleteFiles(references);
206
332
 
207
- // Clear driver cache
208
- StorageManager.clearCache(): void
333
+ // List files with pagination
334
+ const result = await storage.listFiles(prefix?, maxResults?, continuationToken?);
209
335
  ```
210
336
 
211
- ### Convenience Functions
337
+ ### Upload Options
212
338
 
213
339
  ```typescript
214
- import {
215
- uploadFile,
216
- uploadFiles,
217
- generateUploadUrl,
218
- generateViewUrl,
219
- deleteFile,
220
- deleteFiles,
221
- getStorageManager,
222
- initializeStorageManager
223
- } from 'express-storage';
224
-
225
- // Use default storage manager
226
- const result = await uploadFile(file);
227
- const results = await uploadFiles(files);
228
- const urlResult = await generateUploadUrl('filename.jpg');
229
- const success = await deleteFile('filename.jpg');
230
-
231
- // Initialize custom storage manager
232
- const storage = initializeStorageManager({
233
- driver: 'local',
234
- localPath: 'uploads'
340
+ interface UploadOptions {
341
+ contentType?: string; // Override detected type
342
+ metadata?: Record<string, string>; // Custom metadata
343
+ cacheControl?: string; // e.g., 'max-age=31536000'
344
+ contentDisposition?: string; // e.g., 'attachment; filename="doc.pdf"'
345
+ }
346
+
347
+ // Example: Upload with caching headers
348
+ await storage.uploadFile(file, undefined, {
349
+ cacheControl: "public, max-age=31536000",
350
+ metadata: { uploadedBy: "user-123" },
235
351
  });
236
352
  ```
237
353
 
238
- ## 📁 File Organization
354
+ ### Validation Options
239
355
 
240
- ### Local Storage
241
- Files are organized in month/year directories:
242
- ```
243
- public/express-storage/
244
- ├── january/
245
- │ └── 2024/
246
- │ ├── 1703123456_image.jpg
247
- │ └── 1703123457_document.pdf
248
- ├── february/
249
- │ └── 2024/
250
- │ └── 1705800000_video.mp4
251
- └── ...
356
+ ```typescript
357
+ interface FileValidationOptions {
358
+ maxSize?: number; // Maximum file size in bytes
359
+ allowedMimeTypes?: string[]; // e.g., ['image/jpeg', 'image/png']
360
+ allowedExtensions?: string[]; // e.g., ['.jpg', '.png']
361
+ }
252
362
  ```
253
363
 
254
- ### Cloud Storage
255
- Files are stored with unique timestamps:
256
- ```
257
- bucket/
258
- ├── 1703123456_image.jpg
259
- ├── 1703123457_document.pdf
260
- └── 1705800000_video.mp4
261
- ```
364
+ ---
365
+
366
+ ## Environment Variables
367
+
368
+ ### Core Settings
369
+
370
+ | Variable | Description | Default |
371
+ | ---------------------- | ----------------------------------- | ------------------------ |
372
+ | `FILE_DRIVER` | Storage driver to use | `local` |
373
+ | `BUCKET_NAME` | Cloud storage bucket/container name | — |
374
+ | `BUCKET_PATH` | Default folder path within bucket | `""` (root) |
375
+ | `LOCAL_PATH` | Directory for local storage | `public/express-storage` |
376
+ | `PRESIGNED_URL_EXPIRY` | URL validity in seconds | `600` (10 min) |
377
+ | `MAX_FILE_SIZE` | Maximum upload size in bytes | `5368709120` (5GB) |
378
+
379
+ ### AWS S3
262
380
 
263
- ## 🔐 Presigned URLs
381
+ | Variable | Description |
382
+ | ---------------- | ----------------------------------------------- |
383
+ | `AWS_REGION` | AWS region (e.g., `us-east-1`) |
384
+ | `AWS_ACCESS_KEY` | Access key ID (optional if using IAM roles) |
385
+ | `AWS_SECRET_KEY` | Secret access key (optional if using IAM roles) |
264
386
 
265
- For cloud storage providers, you can generate presigned URLs for secure, direct client uploads:
387
+ ### Google Cloud Storage
388
+
389
+ | Variable | Description |
390
+ | ----------------- | ------------------------------------------------ |
391
+ | `GCS_PROJECT_ID` | Google Cloud project ID |
392
+ | `GCS_CREDENTIALS` | Path to service account JSON (optional with ADC) |
393
+
394
+ ### Azure Blob Storage
395
+
396
+ | Variable | Description |
397
+ | ------------------------- | ------------------------------------------------- |
398
+ | `AZURE_CONNECTION_STRING` | Full connection string (recommended) |
399
+ | `AZURE_ACCOUNT_NAME` | Storage account name (alternative) |
400
+ | `AZURE_ACCOUNT_KEY` | Storage account key (alternative) |
401
+
402
+ **Note**: Azure uses `BUCKET_NAME` for the container name (same as S3/GCS).
403
+
404
+ ---
405
+
406
+ ## Utilities
407
+
408
+ Express Storage includes battle-tested utilities you can use directly.
409
+
410
+ ### Retry with Exponential Backoff
266
411
 
267
412
  ```typescript
268
- // Generate upload URL for client-side upload
269
- const uploadResult = await storage.generateUploadUrl('my-file.jpg');
270
- if (uploadResult.success) {
271
- // Client can use uploadResult.uploadUrl to upload directly
272
- console.log(uploadResult.uploadUrl);
273
- }
413
+ import { withRetry } from "express-storage";
274
414
 
275
- // Generate view URL for secure file access
276
- const viewResult = await storage.generateViewUrl('my-file.jpg');
277
- if (viewResult.success) {
278
- // Client can use viewResult.viewUrl to view the file
279
- console.log(viewResult.viewUrl);
280
- }
415
+ const result = await withRetry(() => storage.uploadFile(file), {
416
+ maxAttempts: 3,
417
+ baseDelay: 1000,
418
+ maxDelay: 10000,
419
+ exponentialBackoff: true,
420
+ });
281
421
  ```
282
422
 
283
- ## 🛠️ Advanced Usage Examples
423
+ ### File Type Helpers
284
424
 
285
- ### Custom Configuration
425
+ ```typescript
426
+ import {
427
+ isImageFile,
428
+ isDocumentFile,
429
+ getFileExtension,
430
+ formatFileSize,
431
+ } from "express-storage";
432
+
433
+ isImageFile("image/jpeg"); // true
434
+ isDocumentFile("application/pdf"); // true
435
+ getFileExtension("photo.jpg"); // '.jpg'
436
+ formatFileSize(1048576); // '1 MB'
437
+ ```
438
+
439
+ ### Custom Logging
286
440
 
287
441
  ```typescript
288
- import { StorageManager } from 'express-storage';
289
-
290
- // Initialize with custom config
291
- const storage = StorageManager.initialize({
292
- driver: 's3',
293
- bucketName: 'my-bucket',
294
- awsRegion: 'us-east-1',
295
- awsAccessKey: process.env.AWS_ACCESS_KEY,
296
- awsSecretKey: process.env.AWS_SECRET_KEY,
297
- presignedUrlExpiry: 1800 // 30 minutes
298
- });
442
+ import { StorageManager, Logger } from "express-storage";
443
+
444
+ const logger: Logger = {
445
+ debug: (msg, ...args) => console.debug(`[Storage] ${msg}`, ...args),
446
+ info: (msg, ...args) => console.info(`[Storage] ${msg}`, ...args),
447
+ warn: (msg, ...args) => console.warn(`[Storage] ${msg}`, ...args),
448
+ error: (msg, ...args) => console.error(`[Storage] ${msg}`, ...args),
449
+ };
450
+
451
+ const storage = new StorageManager({ driver: "s3", logger });
299
452
  ```
300
453
 
301
- ### Error Handling
454
+ ---
455
+
456
+ ## Real-World Examples
457
+
458
+ ### Profile Picture Upload
302
459
 
303
460
  ```typescript
304
- app.post('/upload', upload.single('file'), async (req, res) => {
305
- try {
306
- const result = await storage.uploadFile(req.file!);
307
-
461
+ app.post("/users/:id/avatar", upload.single("avatar"), async (req, res) => {
462
+ const result = await storage.uploadFile(
463
+ req.file,
464
+ {
465
+ maxSize: 2 * 1024 * 1024, // 2MB
466
+ allowedMimeTypes: ["image/jpeg", "image/png", "image/webp"],
467
+ },
468
+ {
469
+ cacheControl: "public, max-age=86400",
470
+ metadata: { userId: req.params.id },
471
+ },
472
+ );
473
+
308
474
  if (result.success) {
309
- res.json({
310
- success: true,
311
- fileName: result.fileName,
312
- fileUrl: result.fileUrl
313
- });
475
+ await db.users.update(req.params.id, { avatarUrl: result.fileUrl });
476
+ res.json({ avatarUrl: result.fileUrl });
314
477
  } else {
315
- res.status(400).json({
316
- success: false,
317
- error: result.error
318
- });
478
+ res.status(400).json({ error: result.error });
319
479
  }
320
- } catch (error) {
321
- res.status(500).json({
322
- success: false,
323
- error: error instanceof Error ? error.message : 'Unknown error'
324
- });
325
- }
326
480
  });
327
481
  ```
328
482
 
329
- ### File Validation
483
+ ### Document Upload with Presigned URLs
330
484
 
331
485
  ```typescript
332
- app.post('/upload', upload.single('file'), async (req, res) => {
333
- const file = req.file!;
334
-
335
- // Validate file size (5MB limit)
336
- if (file.size > 5 * 1024 * 1024) {
337
- return res.status(400).json({
338
- success: false,
339
- error: 'File size too large. Maximum 5MB allowed.'
486
+ // Frontend requests upload URL
487
+ app.post("/documents/request-upload", async (req, res) => {
488
+ const { fileName, fileSize } = req.body;
489
+
490
+ const result = await storage.generateUploadUrl(
491
+ fileName,
492
+ "application/pdf",
493
+ fileSize,
494
+ `documents/${req.user.id}`,
495
+ );
496
+
497
+ // Store pending upload in database
498
+ await db.documents.create({
499
+ reference: result.reference,
500
+ userId: req.user.id,
501
+ status: "pending",
340
502
  });
341
- }
342
-
343
- // Validate file type
344
- const allowedTypes = ['image/jpeg', 'image/png', 'image/gif'];
345
- if (!allowedTypes.includes(file.mimetype)) {
346
- return res.status(400).json({
347
- success: false,
348
- error: 'Invalid file type. Only JPEG, PNG, and GIF allowed.'
503
+
504
+ res.json({
505
+ uploadUrl: result.uploadUrl,
506
+ reference: result.reference,
349
507
  });
350
- }
351
-
352
- const result = await storage.uploadFile(file);
353
- res.json(result);
354
508
  });
355
- ```
356
509
 
357
- ### Multiple File Upload with Progress
510
+ // Frontend confirms upload complete
511
+ app.post("/documents/confirm-upload", async (req, res) => {
512
+ const { reference } = req.body;
358
513
 
359
- ```typescript
360
- app.post('/upload-multiple', upload.array('files', 10), async (req, res) => {
361
- const files = req.files as Express.Multer.File[];
362
- const results = await storage.uploadFiles(files);
363
-
364
- const summary = {
365
- total: files.length,
366
- successful: results.filter(r => r.success).length,
367
- failed: results.filter(r => !r.success).length,
368
- files: results.map((result, index) => ({
369
- originalName: files[index].originalname,
370
- success: result.success,
371
- fileName: result.fileName,
372
- fileUrl: result.fileUrl,
373
- error: result.error
374
- }))
375
- };
376
-
377
- res.json(summary);
514
+ const result = await storage.validateAndConfirmUpload(reference, {
515
+ expectedContentType: "application/pdf",
516
+ });
517
+
518
+ if (result.success) {
519
+ await db.documents.update(
520
+ { reference },
521
+ {
522
+ status: "uploaded",
523
+ size: result.actualFileSize,
524
+ },
525
+ );
526
+ res.json({ success: true, viewUrl: result.viewUrl });
527
+ } else {
528
+ await db.documents.delete({ reference });
529
+ res.status(400).json({ error: result.error });
530
+ }
378
531
  });
379
532
  ```
380
533
 
381
- ## 🧪 Testing
534
+ ### Bulk File Upload
382
535
 
383
- Run the test suite:
536
+ ```typescript
537
+ app.post("/gallery/upload", upload.array("photos", 20), async (req, res) => {
538
+ const files = req.files as Express.Multer.File[];
384
539
 
385
- ```bash
386
- # Run all tests
387
- npm test
540
+ const results = await storage.uploadFiles(files, {
541
+ maxSize: 10 * 1024 * 1024,
542
+ allowedMimeTypes: ["image/jpeg", "image/png"],
543
+ });
388
544
 
389
- # Run tests in watch mode
390
- npm run test:watch
545
+ const successful = results.filter((r) => r.success);
546
+ const failed = results.filter((r) => !r.success);
391
547
 
392
- # Run tests with coverage
393
- npm run test:coverage
548
+ res.json({
549
+ uploaded: successful.length,
550
+ failed: failed.length,
551
+ files: successful.map((r) => ({
552
+ fileName: r.fileName,
553
+ url: r.fileUrl,
554
+ })),
555
+ errors: failed.map((r) => r.error),
556
+ });
557
+ });
394
558
  ```
395
559
 
396
- ## 🛠️ Development
560
+ ---
397
561
 
398
- ### Prerequisites
399
- - Node.js >= 16.0.0
400
- - TypeScript >= 5.1.6
562
+ ## Migrating Between Providers
401
563
 
402
- ### Development Commands
564
+ Moving from local development to cloud production? Or switching cloud providers? Here's how.
403
565
 
404
- ```bash
405
- # Install dependencies
406
- npm install
566
+ ### Local to S3
407
567
 
408
- # Build the package
409
- npm run build
568
+ ```env
569
+ # Before (development)
570
+ FILE_DRIVER=local
571
+ LOCAL_PATH=uploads
410
572
 
411
- # Development mode (watch for changes)
412
- npm run dev
573
+ # After (production)
574
+ FILE_DRIVER=s3
575
+ BUCKET_NAME=my-app-uploads
576
+ AWS_REGION=us-east-1
577
+ ```
413
578
 
414
- # Clean build directory
415
- npm run clean
579
+ Your code stays exactly the same. Files uploaded before migration remain in their original location — you'll need to migrate existing files separately if needed.
416
580
 
417
- # Type checking
418
- npm run type-check
581
+ ### S3 to Azure
419
582
 
420
- # Linting
421
- npm run lint
422
- npm run lint:fix
583
+ ```env
584
+ # Before
585
+ FILE_DRIVER=s3
586
+ BUCKET_NAME=my-bucket
587
+ AWS_REGION=us-east-1
423
588
 
424
- # Formatting
425
- npm run format
589
+ # After
590
+ FILE_DRIVER=azure
591
+ BUCKET_NAME=my-container
592
+ AZURE_CONNECTION_STRING=DefaultEndpointsProtocol=https;AccountName=...
426
593
  ```
427
594
 
428
- ### Project Structure
595
+ **Important**: If using presigned URLs, remember that Azure requires post-upload validation. Add `validateAndConfirmUpload()` calls to your confirmation endpoints.
429
596
 
430
- ```
431
- express-storage/
432
- ├── src/
433
- │ ├── types/
434
- │ │ └── storage.types.ts # Type definitions
435
- │ ├── utils/
436
- │ │ ├── config.utils.ts # Configuration utilities
437
- │ │ └── file.utils.ts # File operation utilities
438
- │ ├── drivers/
439
- │ │ ├── base.driver.ts # Abstract base driver
440
- │ │ ├── local.driver.ts # Local storage driver
441
- │ │ ├── s3.driver.ts # AWS S3 driver
442
- │ │ ├── gcs.driver.ts # Google Cloud Storage driver
443
- │ │ └── oci.driver.ts # Oracle Cloud Infrastructure driver
444
- │ ├── factory/
445
- │ │ └── driver.factory.ts # Driver factory
446
- │ ├── storage-manager.ts # Main StorageManager class
447
- │ └── index.ts # Package entry point
448
- ├── tests/ # Test files
449
- ├── examples/ # Usage examples
450
- ├── dist/ # Compiled output
451
- └── package.json
597
+ ---
598
+
599
+ ## TypeScript Support
600
+
601
+ Express Storage is written in TypeScript and exports all types:
602
+
603
+ ```typescript
604
+ import {
605
+ StorageManager,
606
+ StorageDriver,
607
+ FileUploadResult,
608
+ PresignedUrlResult,
609
+ FileValidationOptions,
610
+ UploadOptions,
611
+ Logger,
612
+ } from "express-storage";
613
+
614
+ // Full autocomplete and type checking
615
+ const result: FileUploadResult = await storage.uploadFile(file);
616
+
617
+ if (result.success) {
618
+ console.log(result.fileName); // TypeScript knows this exists
619
+ console.log(result.fileUrl); // TypeScript knows this exists
620
+ }
452
621
  ```
453
622
 
454
- ## 🤝 Contributing
623
+ ---
455
624
 
456
- 1. Fork the repository
457
- 2. Create a feature branch (`git checkout -b feature/amazing-feature`)
458
- 3. Commit your changes (`git commit -m 'Add amazing feature'`)
459
- 4. Push to the branch (`git push origin feature/amazing-feature`)
460
- 5. Open a Pull Request
625
+ ## Contributing
461
626
 
462
- ### Development Guidelines
627
+ Contributions are welcome! Please read our contributing guidelines before submitting a pull request.
463
628
 
464
- - Follow TypeScript best practices
465
- - Write comprehensive tests for new features
466
- - Update documentation for API changes
467
- - Ensure all tests pass before submitting PR
629
+ ```bash
630
+ # Clone the repository
631
+ git clone https://github.com/th3hero/express-storage.git
468
632
 
469
- ## 📄 License
633
+ # Install dependencies
634
+ npm install
635
+
636
+ # Run in development mode
637
+ npm run dev
470
638
 
471
- This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
639
+ # Build for production
640
+ npm run build
472
641
 
473
- ## 🆘 Support
642
+ # Run linting
643
+ npm run lint
644
+ ```
474
645
 
475
- - **Issues**: Report bugs and feature requests on GitHub
476
- - **Documentation**: Check the examples folder for usage patterns
477
- - **Questions**: Open a GitHub discussion for questions
646
+ ---
647
+
648
+ ## License
649
+
650
+ MIT License — use it however you want.
651
+
652
+ ---
478
653
 
479
- ## 🔄 Changelog
654
+ ## Support
480
655
 
481
- ### v1.0.0
482
- - Initial release
483
- - Support for local, S3, GCS, and OCI storage
484
- - Presigned URL generation
485
- - TypeScript-first implementation
486
- - Comprehensive test coverage
656
+ - **Issues**: [GitHub Issues](https://github.com/th3hero/express-storage/issues)
657
+ - **Author**: Alok Kumar ([@th3hero](https://github.com/th3hero))
487
658
 
488
659
  ---
489
660
 
490
- **Made with ❤️ for the Express.js community**
661
+ **Made for developers who are tired of writing upload code from scratch.**