@venizia/ignis-docs 0.2.0 → 0.2.1-1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (174) hide show
  1. package/README.md +14 -14
  2. package/content/best-practices/api-usage-examples.md +39 -19
  3. package/content/best-practices/architectural-patterns.md +24 -13
  4. package/content/best-practices/architecture-decisions.md +29 -16
  5. package/content/best-practices/code-style-standards/advanced-patterns.md +43 -31
  6. package/content/best-practices/code-style-standards/constants-configuration.md +19 -4
  7. package/content/best-practices/code-style-standards/control-flow.md +33 -1
  8. package/content/best-practices/code-style-standards/documentation.md +28 -8
  9. package/content/best-practices/code-style-standards/function-patterns.md +15 -4
  10. package/content/best-practices/code-style-standards/index.md +7 -2
  11. package/content/best-practices/code-style-standards/naming-conventions.md +27 -3
  12. package/content/best-practices/code-style-standards/route-definitions.md +9 -5
  13. package/content/best-practices/code-style-standards/tooling.md +14 -1
  14. package/content/best-practices/code-style-standards/type-safety.md +39 -6
  15. package/content/best-practices/common-pitfalls.md +59 -35
  16. package/content/best-practices/contribution-workflow.md +8 -4
  17. package/content/best-practices/data-modeling.md +78 -62
  18. package/content/best-practices/deployment-strategies.md +56 -19
  19. package/content/best-practices/error-handling.md +247 -153
  20. package/content/best-practices/index.md +6 -0
  21. package/content/best-practices/performance-optimization.md +26 -13
  22. package/content/best-practices/security-guidelines.md +35 -9
  23. package/content/best-practices/testing-strategies.md +7 -2
  24. package/content/best-practices/troubleshooting-tips.md +27 -17
  25. package/content/extensions/components/api-reference.md +109 -323
  26. package/content/extensions/components/authentication/api.md +489 -602
  27. package/content/extensions/components/authentication/errors.md +135 -498
  28. package/content/extensions/components/authentication/index.md +89 -801
  29. package/content/extensions/components/authentication/usage.md +231 -955
  30. package/content/extensions/components/authorization/api.md +991 -644
  31. package/content/extensions/components/authorization/errors.md +204 -208
  32. package/content/extensions/components/authorization/getting-started.md +227 -0
  33. package/content/extensions/components/authorization/index.md +88 -795
  34. package/content/extensions/components/authorization/usage.md +219 -528
  35. package/content/extensions/components/health-check.md +77 -243
  36. package/content/extensions/components/index.md +24 -90
  37. package/content/extensions/components/mail/api.md +553 -285
  38. package/content/extensions/components/mail/errors.md +76 -62
  39. package/content/extensions/components/mail/index.md +111 -463
  40. package/content/extensions/components/mail/usage.md +139 -173
  41. package/content/extensions/components/request-tracker.md +70 -173
  42. package/content/extensions/components/socket-io/api.md +462 -785
  43. package/content/extensions/components/socket-io/errors.md +49 -51
  44. package/content/extensions/components/socket-io/index.md +69 -372
  45. package/content/extensions/components/socket-io/usage.md +212 -105
  46. package/content/extensions/components/static-asset/api.md +461 -141
  47. package/content/extensions/components/static-asset/errors.md +121 -53
  48. package/content/extensions/components/static-asset/index.md +84 -606
  49. package/content/extensions/components/static-asset/usage.md +184 -299
  50. package/content/extensions/components/template/index.md +3 -3
  51. package/content/extensions/components/websocket/api.md +311 -404
  52. package/content/extensions/components/websocket/errors.md +47 -56
  53. package/content/extensions/components/websocket/index.md +75 -407
  54. package/content/extensions/components/websocket/usage.md +120 -338
  55. package/content/extensions/helpers/cron/index.md +52 -160
  56. package/content/extensions/helpers/crypto/index.md +67 -480
  57. package/content/extensions/helpers/crypto/reference.md +528 -0
  58. package/content/extensions/helpers/env/index.md +64 -178
  59. package/content/extensions/helpers/error/index.md +286 -196
  60. package/content/extensions/helpers/index.md +61 -47
  61. package/content/extensions/helpers/inversion/index.md +76 -550
  62. package/content/extensions/helpers/inversion/reference.md +530 -0
  63. package/content/extensions/helpers/kafka/admin.md +23 -3
  64. package/content/extensions/helpers/kafka/compile-binary.md +83 -52
  65. package/content/extensions/helpers/kafka/consumer.md +65 -28
  66. package/content/extensions/helpers/kafka/examples.md +46 -253
  67. package/content/extensions/helpers/kafka/index.md +55 -617
  68. package/content/extensions/helpers/kafka/producer.md +156 -22
  69. package/content/extensions/helpers/kafka/schema-registry.md +59 -79
  70. package/content/extensions/helpers/logger/hf-logger.md +220 -0
  71. package/content/extensions/helpers/logger/index.md +78 -552
  72. package/content/extensions/helpers/logger/pino.md +105 -0
  73. package/content/extensions/helpers/logger/reference.md +937 -0
  74. package/content/extensions/helpers/network/api.md +276 -195
  75. package/content/extensions/helpers/network/index.md +87 -524
  76. package/content/extensions/helpers/queue/index.md +80 -897
  77. package/content/extensions/helpers/queue/reference.md +494 -0
  78. package/content/extensions/helpers/redis/index.md +86 -640
  79. package/content/extensions/helpers/redis/reference.md +784 -0
  80. package/content/extensions/helpers/secrets/index.md +136 -0
  81. package/content/extensions/helpers/socket-io/api.md +325 -204
  82. package/content/extensions/helpers/socket-io/index.md +79 -429
  83. package/content/extensions/helpers/storage/api.md +584 -464
  84. package/content/extensions/helpers/storage/index.md +80 -573
  85. package/content/extensions/helpers/types/index.md +75 -495
  86. package/content/extensions/helpers/types/reference.md +689 -0
  87. package/content/extensions/helpers/uid/index.md +176 -189
  88. package/content/extensions/helpers/websocket/api.md +366 -214
  89. package/content/extensions/helpers/websocket/index.md +68 -503
  90. package/content/extensions/helpers/worker-thread/index.md +62 -396
  91. package/content/extensions/helpers/worker-thread/reference.md +428 -0
  92. package/content/extensions/index.md +38 -39
  93. package/content/extensions/src-details/mcp-server.md +96 -548
  94. package/content/guides/core-concepts/persistent/datasources.md +2 -2
  95. package/content/guides/core-concepts/persistent/index.md +5 -1
  96. package/content/guides/core-concepts/persistent/models.md +1 -1
  97. package/content/guides/core-concepts/persistent/pglite.md +218 -0
  98. package/content/guides/core-concepts/persistent/postgres-drivers.md +21 -15
  99. package/content/guides/core-concepts/persistent/search-meilisearch.md +22 -12
  100. package/content/guides/core-concepts/persistent/search-typesense.md +28 -16
  101. package/content/guides/core-concepts/persistent/sqlite.md +296 -0
  102. package/content/guides/core-concepts/persistent/transactions.md +12 -11
  103. package/content/guides/core-concepts/secrets-vault.md +177 -0
  104. package/content/guides/core-concepts/services.md +1 -1
  105. package/content/guides/get-started/5-minute-quickstart.md +59 -206
  106. package/content/guides/get-started/philosophy.md +135 -670
  107. package/content/guides/get-started/setup.md +53 -74
  108. package/content/guides/migrations/redis-helpers-migration.md +5 -4
  109. package/content/guides/migrations/scoped-rbac-migration.md +5 -5
  110. package/content/guides/migrations/unified-connectors-migration.md +8 -7
  111. package/content/guides/tutorials/ecommerce-api.md +3 -8
  112. package/content/guides/tutorials/realtime-chat.md +1 -1
  113. package/content/references/base/application.md +64 -22
  114. package/content/references/base/components.md +3 -3
  115. package/content/references/base/connectors.md +79 -136
  116. package/content/references/base/controllers.md +12 -12
  117. package/content/references/base/datasources-reference.md +600 -0
  118. package/content/references/base/datasources.md +84 -444
  119. package/content/references/base/dependency-injection.md +25 -46
  120. package/content/references/base/filter-system/application-usage.md +95 -123
  121. package/content/references/base/filter-system/array-operators.md +33 -43
  122. package/content/references/base/filter-system/comparison-operators.md +55 -63
  123. package/content/references/base/filter-system/default-filter.md +144 -350
  124. package/content/references/base/filter-system/fields-order-pagination.md +116 -148
  125. package/content/references/base/filter-system/index.md +114 -253
  126. package/content/references/base/filter-system/json-filtering.md +52 -181
  127. package/content/references/base/filter-system/list-operators.md +28 -44
  128. package/content/references/base/filter-system/logical-operators.md +69 -114
  129. package/content/references/base/filter-system/null-operators.md +41 -98
  130. package/content/references/base/filter-system/pattern-matching.md +48 -51
  131. package/content/references/base/filter-system/quick-reference.md +93 -196
  132. package/content/references/base/filter-system/range-operators.md +22 -38
  133. package/content/references/base/filter-system/tips.md +70 -133
  134. package/content/references/base/filter-system/use-cases.md +173 -232
  135. package/content/references/base/grpc-controllers.md +53 -17
  136. package/content/references/base/index.md +5 -3
  137. package/content/references/base/middlewares.md +43 -28
  138. package/content/references/base/models-reference.md +886 -0
  139. package/content/references/base/models.md +81 -1452
  140. package/content/references/base/providers.md +8 -8
  141. package/content/references/base/repositories/advanced.md +281 -419
  142. package/content/references/base/repositories/index.md +84 -644
  143. package/content/references/base/repositories/mixins.md +25 -21
  144. package/content/references/base/repositories/relations.md +194 -375
  145. package/content/references/base/repositories/soft-deletable.md +67 -55
  146. package/content/references/base/secrets.md +267 -0
  147. package/content/references/base/services.md +8 -6
  148. package/content/references/configuration/environment-variables.md +73 -21
  149. package/content/references/configuration/index.md +51 -31
  150. package/content/references/index.md +1 -1
  151. package/content/references/quick-reference.md +3 -16
  152. package/content/references/utilities/crypto.md +35 -76
  153. package/content/references/utilities/date.md +33 -73
  154. package/content/references/utilities/duration.md +85 -0
  155. package/content/references/utilities/index.md +6 -2
  156. package/content/references/utilities/jsx-reference.md +298 -0
  157. package/content/references/utilities/jsx.md +82 -525
  158. package/content/references/utilities/module.md +81 -61
  159. package/content/references/utilities/parse.md +34 -64
  160. package/content/references/utilities/performance.md +33 -58
  161. package/content/references/utilities/promise.md +28 -62
  162. package/content/references/utilities/request.md +58 -218
  163. package/content/references/utilities/retry.md +139 -0
  164. package/content/references/utilities/schema.md +43 -137
  165. package/content/references/utilities/statuses-reference.md +361 -0
  166. package/content/references/utilities/statuses.md +63 -667
  167. package/dist/mcp-server/index.js +0 -0
  168. package/dist/mcp-server/tools/github/list-project-files.tool.d.ts +1 -1
  169. package/dist/mcp-server/tools/github/list-project-files.tool.d.ts.map +1 -1
  170. package/dist/mcp-server/tools/github/list-project-files.tool.js +3 -3
  171. package/dist/mcp-server/tools/github/search-code.tool.js +1 -1
  172. package/dist/mcp-server/tools/github/verify-dependencies.tool.js +1 -1
  173. package/dist/mcp-server/tools/github/view-source-file.tool.js +1 -1
  174. package/package.json +24 -23
@@ -1,238 +1,88 @@
1
- # Storage
2
-
3
- Unified file storage abstraction with interchangeable backends for S3-compatible object storage, local filesystem, and in-memory key-value caching.
4
-
5
- ## Quick Reference
6
-
7
- | Class | Extends | Backend | Implements |
8
- |-------|---------|---------|------------|
9
- | **MinioHelper** | `BaseStorageHelper` | S3-compatible (MinIO) | `IStorageHelper` |
10
- | **BunS3Helper** | `BaseStorageHelper` | S3-compatible (Bun-native) | `IStorageHelper` |
11
- | **DiskHelper** | `BaseStorageHelper` | Local filesystem | `IStorageHelper` |
12
- | **MemoryStorageHelper** | `BaseHelper` | In-memory key-value | -- |
13
-
14
- #### Import Paths
15
-
16
- ```typescript
17
- // Disk and in-memory storage (from base package)
18
- import { DiskHelper, MemoryStorageHelper } from '@venizia/ignis-helpers';
19
-
20
- // MinIO storage (separate export path)
21
- import { MinioHelper } from '@venizia/ignis-helpers/minio';
22
-
23
- // Bun S3 storage (separate export path, Bun runtime only)
24
- import { BunS3Helper } from '@venizia/ignis-helpers/bun-s3';
25
-
26
- // Types
27
- import type {
28
- IStorageHelper,
29
- IStorageHelperOptions,
30
- IDiskHelperOptions,
31
- IUploadFile,
32
- IUploadResult,
33
- IFileStat,
34
- IBucketInfo,
35
- IObjectInfo,
36
- IListObjectsOptions,
37
- } from '@venizia/ignis-helpers';
38
- import type { IMinioHelperOptions } from '@venizia/ignis-helpers/minio';
39
- import type { IBunS3HelperOptions } from '@venizia/ignis-helpers/bun-s3';
40
- ```
41
-
42
- ## Creating an Instance
43
-
44
- ### MinIO Storage
45
-
46
- `MinioHelper` connects to MinIO or any S3-compatible object storage server. The constructor accepts all `minio.ClientOptions` properties alongside `IStorageHelperOptions`.
47
-
48
- ```typescript
49
- import { MinioHelper } from '@venizia/ignis-helpers/minio';
50
-
51
- const storage = new MinioHelper({
52
- endPoint: 'localhost',
53
- port: 9000,
54
- useSSL: false,
55
- accessKey: 'minioadmin',
56
- secretKey: 'minioadmin',
57
- });
58
- ```
59
-
60
- #### IMinioHelperOptions
61
-
62
- `IMinioHelperOptions` extends both `IStorageHelperOptions` and the minio `ClientOptions` type, so all [minio Client options](https://min.io/docs/minio/linux/developers/javascript/API.html) are accepted.
1
+ ---
2
+ title: Storage
3
+ description: A unified file storage interface with interchangeable S3-compatible, filesystem, and in-memory backends
4
+ difficulty: intermediate
5
+ ---
63
6
 
64
- ```typescript
65
- interface IMinioHelperOptions extends IStorageHelperOptions, ClientOptions {}
66
- ```
67
-
68
- | Option | Type | Default | Description |
69
- |--------|------|---------|-------------|
70
- | `endPoint` | `string` | -- | MinIO server hostname. |
71
- | `port` | `number` | -- | Server port. |
72
- | `useSSL` | `boolean` | -- | Enable HTTPS. |
73
- | `accessKey` | `string` | -- | Access key credential. |
74
- | `secretKey` | `string` | -- | Secret key credential. |
75
- | `scope` | `string` | `'MinioHelper'` | Logger scope name. |
76
- | `identifier` | `string` | `'MinioHelper'` | Helper identifier. |
77
-
78
- > [!NOTE]
79
- > The underlying `minio.Client` is stored as a private property. Use the `IStorageHelper` methods for all operations. If you need direct minio SDK access, extend `MinioHelper` in a subclass.
80
-
81
- ### Bun S3 Storage
82
-
83
- `BunS3Helper` provides S3-compatible storage using Bun's native `S3Client` for high-performance object operations. Bucket management operations (list, create, delete) use AWS Signature V4 signed requests, while object operations use Bun's native S3 API.
84
-
85
- > [!IMPORTANT]
86
- > `BunS3Helper` requires the **Bun runtime**. It uses Bun's built-in `S3Client` class which is not available in Node.js.
87
-
88
- ```typescript
89
- import { BunS3Helper } from '@venizia/ignis-helpers/bun-s3';
90
-
91
- const storage = new BunS3Helper({
92
- accessKey: 'minioadmin',
93
- secretKey: 'minioadmin',
94
- endpoint: 'http://localhost:9000',
95
- region: 'us-east-1',
96
- });
97
- ```
98
-
99
- #### IBunS3HelperOptions
7
+ # Storage
100
8
 
101
- | Option | Type | Default | Description |
102
- |--------|------|---------|-------------|
103
- | `accessKey` | `string` | -- | S3 access key credential. |
104
- | `secretKey` | `string` | -- | S3 secret key credential. |
105
- | `endpoint` | `string` | -- | S3-compatible endpoint URL (e.g., `'http://localhost:9000'`). |
106
- | `region` | `string` | `'us-east-1'` | AWS region for signing. |
107
- | `sessionToken` | `string` | -- | Optional session token for temporary credentials. |
108
- | `scope` | `string` | `'BunS3Helper'` | Logger scope name. |
109
- | `identifier` | `string` | `'BunS3Helper'` | Helper identifier. |
9
+ Storage gives you one file-storage interface with interchangeable backends for S3-compatible object storage, the local filesystem, and in-memory key-value caching.
110
10
 
111
- ### Disk Storage
11
+ ## In one example
112
12
 
113
- `DiskHelper` provides local filesystem storage using a bucket-based directory structure. The `basePath` directory is created automatically if it does not exist.
13
+ `DiskHelper` needs no external server, so it's the fastest way to see the shape of the API. Create a bucket, upload a file, get a link back.
114
14
 
115
15
  ```typescript
116
16
  import { DiskHelper } from '@venizia/ignis-helpers';
117
17
 
118
- const storage = new DiskHelper({
119
- basePath: './app_data/storage',
120
- });
121
- ```
18
+ const storage = new DiskHelper({ basePath: './app_data/storage' });
122
19
 
123
- #### IDiskHelperOptions
20
+ await storage.createBucket({ name: 'uploads' });
124
21
 
125
- | Option | Type | Default | Description |
126
- |--------|------|---------|-------------|
127
- | `basePath` | `string` | -- | Base directory where buckets will be created. Resolved to an absolute path internally. Created automatically if it does not exist. |
128
- | `scope` | `string` | `'DiskHelper'` | Logger scope name. |
129
- | `identifier` | `string` | `'DiskHelper'` | Helper identifier. |
130
-
131
- The resulting directory structure maps buckets to subdirectories:
132
-
133
- ```
134
- app_data/storage/ <-- basePath
135
- ├── bucket-1/ <-- bucket (directory)
136
- │ ├── file1.pdf <-- object (file)
137
- │ └── file2.jpg
138
- └── user-uploads/
139
- ├── avatar.png
140
- └── resume.pdf
141
- ```
142
-
143
- ### In-Memory Storage
144
-
145
- `MemoryStorageHelper` is a standalone, generic key-value store for caching or temporary state within a single process. It does **not** implement `IStorageHelper` and has no bucket or file operations.
146
-
147
- ```typescript
148
- import { MemoryStorageHelper } from '@venizia/ignis-helpers';
149
-
150
- // Direct instantiation
151
- const cache = new MemoryStorageHelper();
152
-
153
- // With custom scope for logging
154
- const cache = new MemoryStorageHelper({ scope: 'SessionCache' });
155
-
156
- // With typed container using the factory method
157
- const cache = MemoryStorageHelper.newInstance<{ counter: number; name: string }>();
158
- ```
159
-
160
- #### Constructor Options
161
-
162
- | Option | Type | Default | Description |
163
- |--------|------|---------|-------------|
164
- | `scope` | `string` | `'MemoryStorageHelper'` | Logger scope name. |
165
-
166
- ## Usage
167
-
168
- `DiskHelper`, `MinioHelper`, and `BunS3Helper` implement the same `IStorageHelper` interface, making them interchangeable. All examples below apply to all three unless noted otherwise.
169
-
170
- ### Uploading Files
171
-
172
- Pass an array of `IUploadFile` objects to `upload()`. The method validates all file names before writing, then uploads in parallel.
173
-
174
- ```typescript
175
- const results = await storage.upload({
176
- bucket: 'my-bucket',
22
+ const [result] = await storage.upload({
23
+ bucket: 'uploads',
177
24
  files: [
178
25
  {
179
26
  originalName: 'report.pdf',
180
27
  mimetype: 'application/pdf',
181
28
  buffer: fileBuffer,
182
29
  size: fileBuffer.length,
183
- encoding: '7bit',
184
30
  },
185
31
  ],
186
32
  });
187
33
 
188
- console.log(results);
189
- // [{ bucketName: 'my-bucket', objectName: 'report.pdf', link: '/static-assets/my-bucket/report.pdf' }]
34
+ console.log(result);
35
+ // { bucketName: 'uploads', objectName: 'report.pdf', link: '/static-resources/uploads/report.pdf' }
190
36
  ```
191
37
 
192
- #### Custom Name and Link Normalization
38
+ `MinioHelper` and `BunS3Helper` accept the same `upload()` call against S3-compatible storage - swap the constructor, keep everything else unchanged.
193
39
 
194
- By default, file names are lowercased with spaces replaced by underscores. The default link prefix differs by backend: MinioHelper and BunS3Helper use `/static-assets/{bucket}/{name}`, DiskHelper uses `/static-resources/{bucket}/{name}`. Override either with custom functions:
40
+ ## How it works
195
41
 
196
- ```typescript
197
- const results = await storage.upload({
198
- bucket: 'my-bucket',
199
- files: files,
200
- normalizeNameFn: ({ originalName, folderPath }) => {
201
- const timestamp = Date.now();
202
- return folderPath
203
- ? `${folderPath}/${timestamp}_${originalName}`
204
- : `${timestamp}_${originalName}`;
205
- },
206
- normalizeLinkFn: ({ bucketName, normalizeName }) => {
207
- return `/files/${bucketName}/${normalizeName}`;
208
- },
209
- });
210
- ```
42
+ - **`BaseStorageHelper` owns the shared logic.** It's an abstract class implementing `IStorageHelper`: name/path validation, MIME type detection, and the `upload()` orchestration itself. Each backend only supplies two protected hooks: `defaultLinkPrefix` and `writeObject()`.
43
+ - **Everything else is per-backend.** `isBucketExists`, `getBuckets`, `createBucket`, `getFile`, `getStat`, `removeObject`, `listObjects`, and the rest of `IStorageHelper` are implemented independently per backend. A filesystem `stat()` and a MinIO `statObject()` share nothing beyond the return shape.
44
+ - **The three backends are interchangeable.** Write services against `IStorageHelper`, not a concrete class, and swap backends by construction only.
45
+ - **`MemoryStorageHelper` is unrelated.** It's a standalone generic key-value store for in-process caching, extending `BaseHelper` directly - no bucket or file concept.
46
+ - **Every write path is validated first.** `originalName` and `folderPath` run through `isValidName()`/`isValidPath()` before touching the filesystem or object store.
47
+ - **Validation blocks four kinds of bad input:** path traversal (`../`), shell-injection characters, hidden files, and folder nesting beyond `maxFolderDepth` (default `2`).
48
+ - **A custom `normalizeNameFn` doesn't get a free pass.** Its output runs through the same check, so a traversal payload smuggled back from application code is rejected too.
49
+ - **Two backends stay optional.** `MinioHelper` and `BunS3Helper` live behind separate sub-path exports. Apps that only need `DiskHelper` or `MemoryStorageHelper` don't pull in the `minio` package or require the Bun runtime.
50
+
51
+ **Backends**
211
52
 
212
- > [!NOTE]
213
- > The output of `normalizeNameFn` is validated with `isValidPath()` before it reaches the filesystem or object store - a traversal payload returned from a custom function (e.g. `../../../etc/cron.d/pwn`) is rejected with `'[upload] Invalid normalized object name | name: {name}'`, not trusted just because it came from application code.
53
+ | Backend | Storage | Mechanism | Import |
54
+ |---|---|---|---|
55
+ | `MinioHelper` | S3-compatible object storage | `minio` SDK | `@venizia/ignis-helpers/minio` |
56
+ | `BunS3Helper` | S3-compatible object storage | Bun's native `S3Client` (Bun runtime only) | `@venizia/ignis-helpers/bun-s3` |
57
+ | `DiskHelper` | Local filesystem, one directory per bucket | Node `fs`/`fs/promises` | `@venizia/ignis-helpers` |
58
+ | `MemoryStorageHelper` | In-process key-value cache | Plain object | `@venizia/ignis-helpers` |
214
59
 
215
- #### Controlling Folder Depth
60
+ ## Common tasks
216
61
 
217
- `upload()` accepts an optional `maxFolderDepth` that overrides `BaseStorageHelper.DEFAULT_MAX_FOLDER_DEPTH` (`2`) for that call. It bounds both the incoming `folderPath` on each file and the folder depth of the resulting normalized object name:
62
+ ### Choose and construct a backend
218
63
 
219
64
  ```typescript
220
- const results = await storage.upload({
221
- bucket: 'my-bucket',
222
- files: files,
223
- maxFolderDepth: 4, // allow up to 4 folder segments instead of the default 2
65
+ import { DiskHelper } from '@venizia/ignis-helpers';
66
+ import { MinioHelper } from '@venizia/ignis-helpers/minio';
67
+
68
+ const disk = new DiskHelper({ basePath: './app_data/storage' });
69
+
70
+ const minio = new MinioHelper({
71
+ endPoint: 'localhost',
72
+ port: 9000,
73
+ useSSL: false,
74
+ accessKey: 'minioadmin',
75
+ secretKey: 'minioadmin',
224
76
  });
225
77
  ```
226
78
 
227
- Omit it to keep the default of `2`.
228
-
229
- #### Upload with Folder Path
79
+ ### Upload with a folder path
230
80
 
231
- When `folderPath` is provided in an `IUploadFile`, the default normalization creates subdirectory-based paths:
81
+ When `folderPath` is set, the default normalization creates a subdirectory-based object name.
232
82
 
233
83
  ```typescript
234
- const results = await storage.upload({
235
- bucket: 'my-bucket',
84
+ const [result] = await storage.upload({
85
+ bucket: 'uploads',
236
86
  files: [
237
87
  {
238
88
  originalName: 'avatar.png',
@@ -246,410 +96,67 @@ const results = await storage.upload({
246
96
  // objectName: 'users/avatar.png'
247
97
  ```
248
98
 
249
- > [!WARNING]
250
- > DiskHelper uses `/static-resources/` as the default link prefix, while MinioHelper and BunS3Helper use `/static-assets/`. Provide a `normalizeLinkFn` if you need consistent links across storage backends.
251
-
252
- ### Downloading Files
99
+ ### Download a file
253
100
 
254
- Retrieve a file as a Node.js `Readable` stream:
101
+ `getFile()` returns a Node.js `Readable` on every backend, so piping to a response or a write stream works identically.
255
102
 
256
103
  ```typescript
257
- const fileStream = await storage.getFile({
258
- bucket: 'my-bucket',
259
- name: 'report.pdf',
260
- });
261
-
262
- // Pipe to an HTTP response
104
+ const fileStream = await storage.getFile({ bucket: 'uploads', name: 'report.pdf' });
263
105
  fileStream.pipe(response);
264
-
265
- // Or write to disk
266
- import fs from 'node:fs';
267
- const writeStream = fs.createWriteStream('./downloads/report.pdf');
268
- fileStream.pipe(writeStream);
269
- ```
270
-
271
- #### MinIO-Specific Options
272
-
273
- MinioHelper supports additional options for server-side encryption and versioning:
274
-
275
- ```typescript
276
- const fileStream = await minioStorage.getFile({
277
- bucket: 'my-bucket',
278
- name: 'report.pdf',
279
- options: {
280
- versionId: 'specific-version-id',
281
- SSECustomerAlgorithm: 'AES256',
282
- SSECustomerKey: 'encryption-key',
283
- SSECustomerKeyMD5: 'key-md5-hash',
284
- },
285
- });
286
- ```
287
-
288
- ### Getting File Metadata
289
-
290
- ```typescript
291
- const stat = await storage.getStat({
292
- bucket: 'my-bucket',
293
- name: 'report.pdf',
294
- });
295
-
296
- console.log(stat);
297
- // {
298
- // size: 204800,
299
- // lastModified: 2025-01-15T10:30:00.000Z,
300
- // metadata: { mimetype: 'application/pdf' },
301
- // etag: 'abc123', // MinioHelper and BunS3Helper only
302
- // versionId: 'v1', // MinioHelper only (if versioning enabled)
303
- // }
304
- ```
305
-
306
- > [!NOTE]
307
- > DiskHelper populates `metadata.mimetype` using the `getMimeType()` extension-based lookup. It does not return `etag` or `versionId`. MinioHelper returns full metadata from the MinIO server including the original upload metadata, `etag`, and `versionId`. BunS3Helper returns `metadata` with `contentType` and `mimetype` from the S3 stat response, plus `etag` and `lastModified`.
308
-
309
- ### Listing Files
310
-
311
- ```typescript
312
- // List all objects in a bucket
313
- const objects = await storage.listObjects({ bucket: 'my-bucket' });
314
-
315
- // List with prefix filter
316
- const docs = await storage.listObjects({
317
- bucket: 'my-bucket',
318
- prefix: 'documents/',
319
- });
320
-
321
- // Recursive listing (includes files in subdirectories)
322
- const allFiles = await storage.listObjects({
323
- bucket: 'my-bucket',
324
- useRecursive: true,
325
- });
326
-
327
- // Limit the number of results
328
- const firstTen = await storage.listObjects({
329
- bucket: 'my-bucket',
330
- maxKeys: 10,
331
- });
332
-
333
- console.log(allFiles);
334
- // [
335
- // { name: 'report.pdf', size: 204800, lastModified: Date, etag: '...' },
336
- // { name: 'avatar.png', size: 51200, lastModified: Date },
337
- // ]
338
106
  ```
339
107
 
340
- ### Deleting Files
108
+ ### List and delete objects
341
109
 
342
110
  ```typescript
343
- // Delete a single object
344
- await storage.removeObject({ bucket: 'my-bucket', name: 'old-file.pdf' });
111
+ const objects = await storage.listObjects({ bucket: 'uploads', useRecursive: true });
345
112
 
346
- // Delete multiple objects
347
113
  await storage.removeObjects({
348
- bucket: 'my-bucket',
349
- names: ['file1.pdf', 'file2.jpg', 'file3.png'],
114
+ bucket: 'uploads',
115
+ names: objects.map(object => object.name!),
350
116
  });
351
117
  ```
352
118
 
353
- > [!NOTE]
354
- > DiskHelper's `removeObject()` throws if the file does not exist. DiskHelper's `removeObjects()` processes deletions sequentially. MinioHelper's `removeObjects()` delegates to the minio SDK's batch removal. BunS3Helper's `removeObjects()` deletes in parallel via `Promise.all()`.
355
-
356
- ### Bucket Operations
357
-
358
- ```typescript
359
- // Check if a bucket exists
360
- const exists = await storage.isBucketExists({ name: 'my-bucket' });
361
-
362
- // Create a new bucket
363
- const bucket = await storage.createBucket({ name: 'my-bucket' });
364
- // Returns: { name: 'my-bucket', creationDate: Date }
365
-
366
- // List all buckets
367
- const buckets = await storage.getBuckets();
368
- // Returns: [{ name: 'bucket-1', creationDate: Date }, ...]
369
-
370
- // Get a specific bucket
371
- const bucket = await storage.getBucket({ name: 'my-bucket' });
372
- // Returns: { name: 'my-bucket', creationDate: Date } | null
373
-
374
- // Remove a bucket
375
- const removed = await storage.removeBucket({ name: 'my-bucket' });
376
- ```
377
-
378
- > [!IMPORTANT]
379
- > DiskHelper's `removeBucket()` requires the bucket directory to be empty. It throws if files remain. Remove all objects first, then remove the bucket.
380
-
381
- ### In-Memory Storage Operations
382
-
383
- `MemoryStorageHelper` provides a simple key-value API, separate from the bucket-based `IStorageHelper` interface:
384
-
385
- ```typescript
386
- const cache = new MemoryStorageHelper();
387
-
388
- // Store a value
389
- cache.set('user:123', { name: 'Alice', role: 'admin' });
390
-
391
- // Retrieve a typed value
392
- const user = cache.get<{ name: string; role: string }>('user:123');
393
-
394
- // Check if a key exists
395
- cache.isBound('user:123'); // true
396
-
397
- // Get all keys
398
- cache.keys(); // ['user:123']
399
-
400
- // Access the underlying container
401
- cache.getContainer(); // { 'user:123': { name: 'Alice', role: 'admin' } }
402
-
403
- // Clear all stored data
404
- cache.clear();
405
- ```
406
-
407
- ### Name Validation
408
-
409
- Storage helpers use two validation methods depending on context:
410
-
411
- - **`isValidName(name)`** - validates a single path segment (bucket names, raw file names). Rejects names that contain `/`, `\`, or `..`.
412
- - **`isValidPath(pathStr)`** - validates a full object path that may include folder segments (e.g., `folder/file.pdf`). Splits on `/` and validates each segment with `isValidName`. Also enforces a max folder depth (default: 2).
413
-
414
- Bucket operations (create, remove) validate the bucket name with `isValidName()`. `upload()` validates each file's `originalName` with `isValidName()` and, when provided, the file's `folderPath` with `isValidPath()` so that folder structures like `2025/uploads` are accepted. Read and delete operations (`getFile`, `getStat`, `removeObject`, `listObjects`) do not re-validate names.
415
-
416
- The following single-segment inputs are rejected by `isValidName()`:
417
-
418
- | Rule | Example | Reason |
419
- |------|---------|--------|
420
- | Contains `..`, `/`, or `\` | `../etc/passwd` | Path traversal |
421
- | Starts with `.` | `.hidden` | Hidden file |
422
- | Contains `;`, `\|`, `&`, `$`, `` ` ``, `<`, `>`, `{`, `}`, `[`, `]`, `!`, `#` | `file;rm -rf` | Shell injection |
423
- | Contains `\n`, `\r`, or `\0` | `file\nname` | Header injection |
424
- | Longer than 255 characters | (very long string) | DoS prevention |
425
- | Empty or whitespace-only | `""`, `" "` | Invalid input |
426
-
427
- ```typescript
428
- // Single-segment name validation (buckets, raw file names)
429
- storage.isValidName('my-file.pdf'); // true
430
- storage.isValidName('../etc/passwd'); // false - contains path separators
431
- storage.isValidName('.hidden'); // false - starts with dot
432
-
433
- // Multi-segment path validation (object names with folder structure)
434
- storage.isValidPath('folder/file.pdf'); // true
435
- storage.isValidPath('../etc/passwd'); // false - path traversal
436
- storage.isValidPath('a/b/c/d/file.pdf'); // false - exceeds max depth (2)
437
- ```
438
-
439
- ### MIME Type Detection
440
-
441
- `getMimeType()` determines the MIME type from a filename's extension:
442
-
443
- ```typescript
444
- storage.getMimeType('photo.jpg'); // 'image/jpeg'
445
- storage.getMimeType('data.csv'); // 'text/csv'
446
- storage.getMimeType('unknown.xyz'); // 'application/octet-stream'
447
- ```
119
+ ### Write storage-agnostic services
448
120
 
449
- `getFileType()` categorizes a MIME type into a broad group:
121
+ Depend on `IStorageHelper`, not a concrete class, so backends swap without touching service code.
450
122
 
451
123
  ```typescript
452
- storage.getFileType({ mimeType: 'image/png' }); // 'image'
453
- storage.getFileType({ mimeType: 'video/mp4' }); // 'video'
454
- storage.getFileType({ mimeType: 'text/plain' }); // 'text'
455
- storage.getFileType({ mimeType: 'application/pdf' }); // 'unknown'
456
- ```
457
-
458
- ### Common Patterns
459
-
460
- #### Storage Abstraction
124
+ import type { IStorageHelper, IUploadFile } from '@venizia/ignis-helpers';
461
125
 
462
- Use `IStorageHelper` to write storage-agnostic code:
463
-
464
- ```typescript
465
126
  class FileService {
466
127
  constructor(private storage: IStorageHelper) {}
467
128
 
468
- async uploadFile(bucket: string, file: IUploadFile) {
129
+ uploadFile(opts: { bucket: string; file: IUploadFile }) {
130
+ const { bucket, file } = opts;
469
131
  return this.storage.upload({ bucket, files: [file] });
470
132
  }
471
133
  }
472
-
473
- // Swap backends without changing service code
474
- const devService = new FileService(new DiskHelper({ basePath: './files' }));
475
- const prodService = new FileService(new MinioHelper({ /* ... */ }));
476
- const bunService = new FileService(new BunS3Helper({ /* ... */ }));
477
134
  ```
478
135
 
479
- #### Environment-Based Selection
136
+ ### Cache values in-memory
480
137
 
481
- ```typescript
482
- import { applicationEnvironment } from '@venizia/ignis-helpers';
483
-
484
- const createStorage = (): IStorageHelper => {
485
- if (applicationEnvironment.get('STORAGE_TYPE') === 'minio') {
486
- return new MinioHelper({
487
- endPoint: applicationEnvironment.get('MINIO_HOST'),
488
- port: Number(applicationEnvironment.get('MINIO_PORT')),
489
- accessKey: applicationEnvironment.get('MINIO_ACCESS_KEY'),
490
- secretKey: applicationEnvironment.get('MINIO_SECRET_KEY'),
491
- useSSL: applicationEnvironment.get('MINIO_USE_SSL') === 'true',
492
- });
493
- }
494
-
495
- return new DiskHelper({
496
- basePath: applicationEnvironment.get('DISK_STORAGE_PATH') || './storage',
497
- });
498
- };
499
- ```
500
-
501
- ## Troubleshooting
502
-
503
- ### "[createBucket] Invalid name to create bucket!"
504
-
505
- **Cause:** The bucket name failed `isValidName()` validation. The name may contain path traversal characters, start with a dot, contain shell-special characters, or exceed 255 characters.
506
-
507
- **Fix:** Use a simple alphanumeric bucket name:
138
+ `MemoryStorageHelper` is a separate, generic key-value store - not a bucket-based backend.
508
139
 
509
140
  ```typescript
510
- // Wrong
511
- await storage.createBucket({ name: '../my-bucket' });
512
- await storage.createBucket({ name: '.hidden-bucket' });
513
-
514
- // Correct
515
- await storage.createBucket({ name: 'my-bucket' });
516
- ```
517
-
518
- ### "[removeBucket] Invalid name to remove bucket!"
519
-
520
- **Cause:** Same as above -- the bucket name failed validation.
521
-
522
- **Fix:** Provide a valid bucket name that passes `isValidName()`.
523
-
524
- ### "[createBucket] Bucket already exists | name: {name}"
525
-
526
- **Cause:** DiskHelper throws when calling `createBucket()` on an existing bucket directory.
527
-
528
- **Fix:** Check existence first:
529
-
530
- ```typescript
531
- const exists = await storage.isBucketExists({ name: 'my-bucket' });
532
- if (!exists) {
533
- await storage.createBucket({ name: 'my-bucket' });
534
- }
535
- ```
536
-
537
- ### "[removeBucket] Bucket does not exist | name: {name}"
538
-
539
- **Cause:** DiskHelper throws when attempting to remove a bucket directory that does not exist.
540
-
541
- **Fix:** Check existence before removal:
542
-
543
- ```typescript
544
- const exists = await storage.isBucketExists({ name: 'my-bucket' });
545
- if (exists) {
546
- await storage.removeBucket({ name: 'my-bucket' });
547
- }
548
- ```
549
-
550
- ### "[removeBucket] Bucket is not empty | name: {name}"
551
-
552
- **Cause:** DiskHelper's `removeBucket()` requires the bucket directory to be empty before removal.
553
-
554
- **Fix:** Remove all objects first:
555
-
556
- ```typescript
557
- const objects = await storage.listObjects({ bucket: 'my-bucket', useRecursive: true });
558
- if (objects.length > 0) {
559
- await storage.removeObjects({
560
- bucket: 'my-bucket',
561
- names: objects.map(o => o.name!),
562
- });
563
- }
564
- await storage.removeBucket({ name: 'my-bucket' });
565
- ```
566
-
567
- ### "[upload] Bucket does not exist | name: {bucket}"
568
-
569
- **Cause:** The target bucket does not exist. DiskHelper, MinioHelper, and BunS3Helper all validate bucket existence before uploading.
570
-
571
- **Fix:** Create the bucket before uploading:
572
-
573
- ```typescript
574
- const exists = await storage.isBucketExists({ name: 'uploads' });
575
- if (!exists) {
576
- await storage.createBucket({ name: 'uploads' });
577
- }
578
- await storage.upload({ bucket: 'uploads', files: [...] });
579
- ```
580
-
581
- ### "[upload] Invalid original file name"
582
-
583
- **Cause:** A file's `originalName` failed `isValidName()` validation.
584
-
585
- **Fix:** Sanitize file names before uploading, or use `normalizeNameFn` to control the stored name:
586
-
587
- ```typescript
588
- await storage.upload({
589
- bucket: 'my-bucket',
590
- files: files,
591
- normalizeNameFn: ({ originalName }) => {
592
- return originalName.replace(/[^a-zA-Z0-9._-]/g, '_');
593
- },
594
- });
595
- ```
596
-
597
- ### "[upload] Invalid file size"
598
-
599
- **Cause:** A file's `size` property is `undefined`, `null`, or negative. A zero-byte file is a legal upload - it is not what triggers this error.
600
-
601
- **Fix:** Ensure every file in the upload array has a valid `size` value:
602
-
603
- ```typescript
604
- const file: IUploadFile = {
605
- originalName: 'doc.pdf',
606
- mimetype: 'application/pdf',
607
- buffer: fileBuffer,
608
- size: fileBuffer.length, // Must be a number >= 0
609
- };
610
- ```
611
-
612
- ### "[upload] Invalid normalized object name | name: {name}"
613
-
614
- **Cause:** A custom `normalizeNameFn` returned a value that fails `isValidPath()` - typically a path-traversal payload (e.g. `../../../etc/cron.d/pwn`) or a name exceeding `maxFolderDepth`.
615
-
616
- **Fix:** Ensure `normalizeNameFn` returns a plain relative name/path with no `..` segments, no leading `/`, and no more folder segments than `maxFolderDepth` (default `2`) allows.
617
-
618
- ### "[getFile] File not found | bucket: {bucket} | name: {name}"
619
-
620
- **Cause:** DiskHelper throws when the requested file does not exist on the filesystem.
621
-
622
- **Fix:** Verify the file exists before attempting to retrieve it, or handle the error:
141
+ import { MemoryStorageHelper } from '@venizia/ignis-helpers';
623
142
 
624
- ```typescript
625
- try {
626
- const stream = await storage.getFile({ bucket: 'my-bucket', name: 'file.pdf' });
627
- } catch (error) {
628
- // File not found -- handle gracefully
629
- }
143
+ const cache = MemoryStorageHelper.newInstance<{ counter: number }>();
144
+ cache.set('counter', 1);
145
+ cache.get<number>('counter'); // 1
630
146
  ```
631
147
 
632
- ### MinioHelper connection errors
633
-
634
- **Cause:** Network or configuration issue between the application and the MinIO server.
635
-
636
- **Checklist:**
637
- - The MinIO server is running and reachable at the configured `endPoint` and `port`
638
- - `useSSL` matches the server's TLS configuration
639
- - `accessKey` and `secretKey` are correct
640
- - Network and firewall rules allow the connection
641
-
642
- ## See Also
148
+ ## See also
643
149
 
644
- - **Other Helpers:**
645
- - [Helpers Index](../index) -- All available helpers
646
- - [Queue Helper](../queue/) -- Message queue processing
150
+ - [Full reference](./api) - every class, method signature, validation rule, and error message
151
+ - [Static Asset Component](/extensions/components/static-asset/) - serving stored files over HTTP
152
+ - [Request Utilities](/references/utilities/request) - `parseMultipartBody` for file uploads
153
+ - [Helpers Index](../index) - all available helpers
154
+ - [Queue Helper](../queue/) - message queue processing
647
155
 
648
- - **References:**
649
- - [Static Asset Component](/extensions/components/static-asset/) -- Serving stored files via HTTP
650
- - [Request Utilities](/references/utilities/request) -- `parseMultipartBody` for file uploads
651
- - [API Reference](./api) -- Full method signatures and types
156
+ **Files:**
652
157
 
653
- - **External Resources:**
654
- - [MinIO Documentation](https://min.io/docs/minio/linux/index.html) -- MinIO object storage
655
- - [MinIO JavaScript SDK](https://min.io/docs/minio/linux/developers/javascript/API.html) -- Full minio client API
158
+ - [`packages/helpers/src/modules/storage/base.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/storage/base.ts) - `BaseStorageHelper`
159
+ - [`packages/helpers/src/modules/storage/minio/helper.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/storage/minio/helper.ts) - `MinioHelper`
160
+ - [`packages/helpers/src/modules/storage/bun-s3/helper.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/storage/bun-s3/helper.ts) - `BunS3Helper`
161
+ - [`packages/helpers/src/modules/storage/disk/helper.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/storage/disk/helper.ts) - `DiskHelper`
162
+ - [`packages/helpers/src/modules/storage/in-memory/helper.ts`](https://github.com/VENIZIA-AI/ignis/blob/main/packages/helpers/src/modules/storage/in-memory/helper.ts) - `MemoryStorageHelper`