@voltro/cli 0.30.1 → 0.31.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (124) hide show
  1. package/CHANGELOG.md +163 -0
  2. package/dist/apiBuild-BOqL1dlg.js +2 -0
  3. package/dist/{apiBuild-DmkzRyv7.js → apiBuild-CLPt-pX6.js} +2 -2
  4. package/dist/appGraph-CvQCte0z.js +2 -0
  5. package/dist/appGraph-gQ_6GkQQ.js +110 -0
  6. package/dist/bin.js +3 -3
  7. package/dist/checkCommand-ClRvH_SN.js +2 -0
  8. package/dist/{checkCommand-C4konUy9.js → checkCommand-D7r6_pk7.js} +439 -530
  9. package/dist/{commands-BkJft9K2.js → commands-CNVVv03h.js} +1941 -1880
  10. package/dist/{dbCommand-qn__kLEt.js → dbCommand-BH8GAYuf.js} +76 -19
  11. package/dist/dbCommand-BwQ_Jl98.js +2 -0
  12. package/dist/{dev-DXhhZ7-f.js → dev-DOZVMVfi.js} +1 -1
  13. package/dist/{dev-C_NPcLDp.js → dev-DdQEn2Ia.js} +1986 -1967
  14. package/dist/{frameworkTableAssembly-Cpgn2e88.js → frameworkTableAssembly-BwIrO5nv.js} +122 -103
  15. package/dist/frameworkTableAssembly-D-EebUQX.js +2 -0
  16. package/dist/index.js +1 -1
  17. package/dist/{inspectMetrics-CkB2J0Zn.js → inspectMetrics-BeEAnjqw.js} +774 -527
  18. package/dist/{sdkgen-wsWLrq4l.js → sdkgen-v9WV8zRi.js} +1 -1
  19. package/dist/{serveCommand-HtI4X5kj.js → serveCommand-KsKXXWn4.js} +294 -293
  20. package/dist/serveEntry.js +2 -2
  21. package/dist/{start-egkbpugc.js → start-CI2QdKy9.js} +351 -346
  22. package/dist/startEntry.js +2 -2
  23. package/package.json +17 -17
  24. package/templates/AGENTS.md +43 -43
  25. package/templates/agent-docs/_index.md +43 -43
  26. package/templates/agent-docs/_manifest.json +84 -42
  27. package/templates/agent-docs/cli.md +4 -2
  28. package/templates/agent-docs/database/migrations.md +14 -1
  29. package/templates/agent-docs/internationalization.md +54 -3
  30. package/templates/agent-docs/plugins/ai-flows.md +148 -0
  31. package/templates/agent-docs/plugins/analytics-postgres.md +90 -0
  32. package/templates/agent-docs/plugins/atlassian.md +378 -0
  33. package/templates/agent-docs/plugins/audit.md +324 -0
  34. package/templates/agent-docs/plugins/auth-auth0.md +92 -0
  35. package/templates/agent-docs/plugins/auth-clerk.md +90 -0
  36. package/templates/agent-docs/plugins/auth-kinde.md +88 -0
  37. package/templates/agent-docs/plugins/auth-oidc.md +95 -0
  38. package/templates/agent-docs/plugins/auth-supabase.md +103 -0
  39. package/templates/agent-docs/plugins/auth-workos.md +105 -0
  40. package/templates/agent-docs/plugins/auth.md +227 -0
  41. package/templates/agent-docs/plugins/billing.md +324 -0
  42. package/templates/agent-docs/plugins/broadcast.md +144 -0
  43. package/templates/agent-docs/plugins/cdc-out.md +116 -0
  44. package/templates/agent-docs/plugins/clickhouse.md +91 -0
  45. package/templates/agent-docs/plugins/datadog.md +105 -0
  46. package/templates/agent-docs/plugins/deactivation.md +114 -0
  47. package/templates/agent-docs/plugins/duckdb.md +82 -0
  48. package/templates/agent-docs/plugins/flags.md +146 -0
  49. package/templates/agent-docs/plugins/governance.md +143 -0
  50. package/templates/agent-docs/plugins/licensing.md +74 -0
  51. package/templates/agent-docs/plugins/logship.md +56 -0
  52. package/templates/agent-docs/plugins/mail.md +334 -0
  53. package/templates/agent-docs/plugins/moderation.md +65 -0
  54. package/templates/agent-docs/plugins/multitenancy.md +99 -0
  55. package/templates/agent-docs/plugins/notifications.md +252 -0
  56. package/templates/agent-docs/plugins/openapi.md +94 -0
  57. package/templates/agent-docs/plugins/postgis.md +140 -0
  58. package/templates/agent-docs/plugins/posthog.md +91 -0
  59. package/templates/agent-docs/plugins/presence.md +115 -0
  60. package/templates/agent-docs/plugins/prometheus.md +124 -0
  61. package/templates/agent-docs/plugins/ratelimit.md +274 -0
  62. package/templates/agent-docs/plugins/rbac.md +270 -0
  63. package/templates/agent-docs/plugins/scim.md +63 -0
  64. package/templates/agent-docs/plugins/search.md +116 -0
  65. package/templates/agent-docs/plugins/sentry.md +192 -0
  66. package/templates/agent-docs/plugins/soft-delete.md +72 -0
  67. package/templates/agent-docs/plugins/sso-saml.md +132 -0
  68. package/templates/agent-docs/plugins/storage.md +709 -0
  69. package/templates/agent-docs/plugins/tinybird.md +92 -0
  70. package/templates/agent-docs/plugins/versioning.md +226 -0
  71. package/templates/agent-docs/plugins/webhooks.md +585 -0
  72. package/templates/agent-docs/security.md +12 -1
  73. package/templates/agent-docs/whats-new.md +81 -37
  74. package/templates/apps/api-ai/package.json +7 -7
  75. package/templates/apps/api-auth/package.json +8 -8
  76. package/templates/apps/api-backend/package.json +7 -7
  77. package/templates/apps/api-backend-deactivation/package.json +7 -7
  78. package/templates/apps/api-backend-mail/package.json +8 -8
  79. package/templates/apps/api-backend-mariadb/package.json +9 -9
  80. package/templates/apps/api-backend-sqlite/package.json +8 -8
  81. package/templates/apps/api-backend-storage/package.json +8 -8
  82. package/templates/apps/api-cms/package.json +10 -10
  83. package/templates/apps/api-collab/package.json +8 -8
  84. package/templates/apps/api-data-advanced/package.json +8 -8
  85. package/templates/apps/api-durable/package.json +8 -8
  86. package/templates/apps/api-feature-flags/package.json +9 -9
  87. package/templates/apps/api-governance/package.json +8 -8
  88. package/templates/apps/api-kv/package.json +8 -8
  89. package/templates/apps/api-moderation/package.json +8 -8
  90. package/templates/apps/api-observability/package.json +8 -8
  91. package/templates/apps/api-ratelimit/package.json +8 -8
  92. package/templates/apps/api-rbac/package.json +8 -8
  93. package/templates/apps/api-rest/package.json +7 -7
  94. package/templates/apps/api-saas/package.json +11 -11
  95. package/templates/apps/api-saas-starter/package.json +10 -10
  96. package/templates/apps/api-search/package.json +8 -8
  97. package/templates/apps/api-status/package.json +8 -8
  98. package/templates/apps/api-versioning/package.json +8 -8
  99. package/templates/apps/api-webhooks/package.json +9 -9
  100. package/templates/apps/changelog/package.json +6 -6
  101. package/templates/apps/edge-functions/package.json +2 -2
  102. package/templates/apps/frontend-admin/package.json +8 -8
  103. package/templates/apps/frontend-app/package.json +8 -8
  104. package/templates/apps/frontend-auth/package.json +8 -8
  105. package/templates/apps/frontend-blank/package.json +7 -7
  106. package/templates/apps/frontend-cms/package.json +9 -9
  107. package/templates/apps/frontend-collab/package.json +9 -9
  108. package/templates/apps/frontend-contact/package.json +7 -7
  109. package/templates/apps/frontend-dashboard/package.json +7 -7
  110. package/templates/apps/frontend-docs/package.json +7 -7
  111. package/templates/apps/frontend-i18n/package.json +6 -6
  112. package/templates/apps/frontend-landing/package.json +7 -7
  113. package/templates/apps/frontend-portal/package.json +8 -8
  114. package/templates/apps/frontend-saas/package.json +8 -8
  115. package/templates/apps/frontend-spa/package.json +7 -7
  116. package/templates/apps/frontend-ssr/package.json +7 -7
  117. package/templates/apps/frontend-ssr-api/package.json +8 -8
  118. package/templates/apps/frontend-static-blog/package.json +6 -6
  119. package/templates/apps/frontend-status/package.json +8 -8
  120. package/templates/apps/mobile-app/package.json +2 -2
  121. package/dist/apiBuild-Ca6Sq5jy.js +0 -2
  122. package/dist/checkCommand-DHD0aBeR.js +0 -2
  123. package/dist/dbCommand-wfGi3FgG.js +0 -2
  124. package/dist/frameworkTableAssembly-Bn_c4Mnl.js +0 -2
@@ -0,0 +1,709 @@
1
+ # Storage
2
+
3
+ > File storage behind one StorageService — public objects served direct from the bucket/CDN, private objects gated by an access policy + per-object grants. S3 / R2 / GCS / MinIO / filesystem / memory providers, presigned URLs, a dashboard browser.
4
+
5
+
6
+
7
+ ---
8
+
9
+ <!-- source: en/plugins/storage.md -->
10
+ ## Storage
11
+
12
+ _File storage behind one StorageService — public objects served direct from the bucket/CDN, private objects gated by an access policy + per-object grants. S3 / R2 / GCS / MinIO / filesystem / memory providers, presigned URLs, a dashboard browser._
13
+
14
+ `@voltro/plugin-storage` is file storage behind one `StorageService`. Wire a
15
+ provider in `app.config.ts`; consume it in handlers and actions via
16
+ `yield* StorageService`. The bytes live in the provider (S3-compatible bucket,
17
+ filesystem, or in-memory); metadata rows live in `_voltro_storage_refs` +
18
+ `_voltro_storage_grants` (both auto-migrated on a SQL store).
19
+
20
+ Every object is **public** or **private**:
21
+
22
+ - **Public** — cacheable + no auth. With `cdnBaseUrl` (or `STORAGE_CDN_URL`)
23
+ set to a CDN / publicly-readable bucket endpoint, the serve route 302s
24
+ straight there and the app never touches the read path (edge-cacheable).
25
+ WITHOUT `cdnBaseUrl`, the serve route streams the bytes through the app
26
+ from the (private) bucket with a 1-year `immutable` cache — still
27
+ browser-cacheable, bucket stays private. A bare S3/MinIO bucket is private
28
+ by default, so the framework never 302s to a raw bucket URL unless you've
29
+ declared a public base via `cdnBaseUrl` (otherwise it would land on a 403).
30
+ - **Private** (the default — secure by default) — gated by an **access policy**
31
+ plus **per-object grants**; delivered via a presigned URL or a short-lived
32
+ signed grant token.
33
+
34
+ ## Quick start
35
+
36
+ ```ts
37
+ // app.config.ts
38
+ import { storagePlugin } from '@voltro/plugin-storage'
39
+
40
+ export default {
41
+ type: 'api' as const,
42
+ name: 'myApi',
43
+ store: 'postgres' as const,
44
+ plugins: [
45
+ // dev: zero-config in-memory. prod: swap the provider (see below).
46
+ storagePlugin({ provider: 'memory' }),
47
+ ],
48
+ }
49
+ ```
50
+
51
+ ```ts
52
+ // actions/uploadAvatar.action.server.ts (+ matching .action.ts)
53
+ import { StorageService } from '@voltro/plugin-storage'
54
+ import { Effect } from 'effect'
55
+
56
+ export default (input: { bytes: Uint8Array; contentType: string }, ctx) =>
57
+ Effect.gen(function* () {
58
+ const storage = yield* StorageService
59
+ const ref = yield* storage.put({
60
+ bytes: input.bytes,
61
+ contentType: input.contentType,
62
+ visibility: 'public', // public avatar
63
+ ownerId: ctx.request.subject.id,
64
+ key: `avatars/${ctx.request.subject.id}/avatar.png`,
65
+ })
66
+ // For a public object this is the CDN/bucket URL (no app round-trip).
67
+ return { url: yield* storage.getUrl(ref.id) }
68
+ })
69
+ ```
70
+
71
+ The ref id is a `file_…` TypeID — URL-safe and unguessable.
72
+
73
+ ## Uploading files — `useUpload()` (client)
74
+
75
+ The one-liner. Works on every provider (filesystem/dev, s3/r2/prod),
76
+ cross-origin, no base64:
77
+
78
+ ```tsx
79
+ import { useUpload } from '@voltro/client'
80
+
81
+ function AvatarUpload() {
82
+ const { upload, progress, status, cancel } = useUpload('myApi')
83
+ return (
84
+ <input type="file" onChange={async (e) => {
85
+ const file = e.target.files?.[0]
86
+ if (!file) return
87
+ const asset = await upload(file, { visibility: 'public', folder: 'avatars' })
88
+ // asset: { id, url, name, mime, size, width?, height? }
89
+ }} />
90
+ )
91
+ }
92
+ ```
93
+
94
+ `useUpload(apiName)` returns `{ upload, uploadMany, progress (0..1), status,
95
+ error, cancel, reset }`. Under the hood it calls the authenticated
96
+ `storage.mintUploadTicket` rpc (which runs your auth chain and returns a
97
+ short-lived signed URL), then POSTs the file's **raw bytes** via XHR with
98
+ real per-byte progress — binary, so a 50 MB video never bloats the rpc
99
+ payload. The stored ref row appears **live** in any `useSubscription` over that
100
+ table (normal reactivity). For an instant preview, pair with
101
+ `URL.createObjectURL(file)` and swap to `asset.url` on completion.
102
+
103
+ **Drag-drop + paste** come for free. The handle also returns `onDrop`, `onPaste`,
104
+ and a `dropzoneProps` you spread onto any element for a zero-boilerplate dropzone:
105
+
106
+ ```tsx
107
+ const { dropzoneProps, onPaste, uploadMany } = useUpload('myApi')
108
+
109
+ <div {...dropzoneProps}>Drop files here</div> // drop → uploadMany
110
+ <textarea onPaste={(e) => onPaste(e, { folder: 'notes' })} /> // paste a screenshot
111
+ ```
112
+
113
+ `onDrop(event, opts?)` / `onPaste(event, opts?)` pull the files off the event and
114
+ call `uploadMany` (bounded concurrency); both resolve `[]` when the event carries
115
+ no files. `dropzoneProps` also prevents the browser's default "navigate to the
116
+ dropped file" behavior.
117
+
118
+ Server-side, the same path is the `mintUploadTicket` action + the
119
+ `POST /_voltro/storage/upload` route (ticket-verified, no auth re-run). The
120
+ `storage.upload` base64 action also exists for tiny files.
121
+
122
+ ### Transports — `prefer`
123
+
124
+ `through-app` (the default) works everywhere and needs nothing extra. Two opt-in
125
+ transports handle large media:
126
+
127
+ ```tsx
128
+ useUpload('myApi').upload(file, { prefer: 'presign' }) // offload the upload leg
129
+ useUpload('myApi').upload(file, { prefer: 'resumable' }) // survive dropped connections
130
+ useUpload('myApi').upload(file, { prefer: 'multipart' }) // multi-GB, direct to bucket
131
+ ```
132
+
133
+ - **`presign`** — on a presigning provider (s3/minio) the browser PUTs straight
134
+ to the bucket (offloading the client→server leg), then the server **fetches the
135
+ bytes back** to scan + checksum + derive and register a **non-degraded** ref
136
+ (`storage.mintPresignedUpload` → direct PUT → `storage.finalizeUpload`). On a
137
+ non-presigning provider it **transparently falls back** to through-app. A virus
138
+ hit fails closed (temp bytes deleted, nothing registered) — so presigned
139
+ uploads are scanned, unlike a raw bucket PUT.
140
+ - **`resumable`** — splits the file into chunks POSTed one at a time to
141
+ `/_voltro/storage/upload/resumable` (`storage.beginResumableUpload`); each chunk
142
+ is retried independently and is idempotent by index, so a dropped connection
143
+ resumes instead of restarting. Works on **every** provider (chunks are stored,
144
+ then assembled through the same scan pipeline). Tune with `chunkSize`.
145
+ - **`multipart`** — for **multi-GB** media: each part is PUT **directly to the
146
+ bucket** via a presigned URL (`storage.beginMultipartUpload` →
147
+ `storage.signMultipartPart` per part → `storage.completeMultipartUpload`), so
148
+ the bytes **never touch the app** — offloaded *and* resumable (a failed part is
149
+ re-signed + re-sent). s3/minio only; **falls back to `resumable`** elsewhere.
150
+ Needs the bucket CORS to **expose the `ETag` header** (`voltro storage cors`
151
+ prints the rule). **Tradeoff:** completing registers the ref straight from the
152
+ bucket object (size via HEAD, etag as checksum) — it does **not** fetch the
153
+ bytes back, so a multipart upload is **not scanned or derived** inline (that
154
+ would mean pulling GB through the app). Scan multi-GB out of band (a
155
+ bucket-event job) and note `checksum` is the S3 etag, not a sha-256.
156
+
157
+ ## Media derivatives + rich metadata
158
+
159
+ `put()` (and every upload) auto-extracts, for raster images (via optional
160
+ `sharp`, graceful when absent):
161
+
162
+ - `width` / `height` (pixels) — the fields your app used to leave null.
163
+ - `placeholder` — a tiny `data:image/webp;base64,…` LQIP (drops into
164
+ `<img src>` / a CSS background; no decoder library).
165
+
166
+ Opt into `storagePlugin({ normalizeImages: true })` to apply EXIF orientation
167
+ and strip ALL metadata (GPS/camera), re-encoding — the normalized bytes are
168
+ what get checksummed + stored.
169
+
170
+ Uploads also carry app metadata onto the ref — `folder`, `tags`, `alt`,
171
+ `caption` — so the ref row is rich enough that **apps no longer need a parallel
172
+ `assets` table**.
173
+
174
+ For **video/audio**, wire a `videoProbe` hook — the core bundles no transcoder
175
+ (ffmpeg is a large binary + licensing), so you plug in ffprobe / a cloud API and
176
+ it runs in `put()` for video/audio content types:
177
+
178
+ ```ts
179
+ storagePlugin({
180
+ videoProbe: async ({ bytes, contentType }) => {
181
+ const { duration, width, height, posterDataUri } = await probeWithFfprobe(bytes)
182
+ return { duration, width, height, poster: posterDataUri } // → ref.duration + ref.placeholder
183
+ },
184
+ })
185
+ ```
186
+
187
+ Best-effort: a `null` / throw / failed probe never blocks the upload. An explicit
188
+ `duration` passed to `put()` wins over the probe.
189
+
190
+ ## Transcoding video — renditions
191
+
192
+ Produce playable variants (720p mp4, webm, a poster…) from an uploaded video.
193
+ The first-party `ffmpegTranscoder` **spawns the ffmpeg binary** (no npm dep — like
194
+ `clamavScanner` talks to a daemon), so install ffmpeg on the host (or pass
195
+ `ffmpegPath`); absent it, transcoding is skipped and the upload still succeeds.
196
+
197
+ ```ts
198
+ import { storagePlugin, ffmpegTranscoder } from '@voltro/plugin-storage'
199
+
200
+ storagePlugin({
201
+ transcode: ffmpegTranscoder({
202
+ renditions: [
203
+ { kind: 'rendition', label: '720p', contentType: 'video/mp4', ext: 'mp4',
204
+ args: ['-vf', 'scale=-2:720', '-c:v', 'libx264', '-crf', '23', '-c:a', 'aac', '-movflags', '+faststart'] },
205
+ { kind: 'poster', label: 'poster', contentType: 'image/jpeg', ext: 'jpg',
206
+ args: ['-frames:v', '1', '-vf', 'thumbnail'] },
207
+ ],
208
+ }),
209
+ })
210
+ ```
211
+
212
+ Each rendition is stored as a **normal storage ref linked to the original**
213
+ (`derivedFrom` = the parent id, `kind` = `'rendition'`/`'poster'`, `caption` =
214
+ the label) — so renditions get serving, grants, transforms and GC for free, and
215
+ inherit the original's tenant + visibility.
216
+
217
+ - **Runs out of band.** Transcoding forks in the background *after* the upload
218
+ returns (never blocks the client). It's ON by default once a `transcode` is
219
+ configured — set `autoTranscode: false` to only run it explicitly.
220
+ - **`storage.renditions(refId)`** — list the derivatives of an asset.
221
+ - Bring your own `transcode` (a cloud video API, HLS packager) by implementing
222
+ the `Transcoder` interface — `ffmpegTranscoder` is just one implementation.
223
+
224
+ ### Durability — the auto-fork is best-effort
225
+
226
+ The background fork is **fire-and-forget**: if the process crashes or is
227
+ redeployed mid-transcode, that rendition is simply never produced (the original
228
+ upload is unaffected). For long or business-critical transcodes, **turn the
229
+ auto-fork off and drive `storage.transcode(refId)` from a durable
230
+ [workflow](/docs/workflows/overview)** — you get retries, backoff, and
231
+ at-least-once execution across restarts:
232
+
233
+ ```ts
234
+ // 1. Disable the fire-and-forget fork:
235
+ storagePlugin({ transcode: ffmpegTranscoder({ renditions }), autoTranscode: false })
236
+
237
+ // 2. Kick a durable workflow when a video ref is created (e.g. from the upload
238
+ // mutation or a reactive hook), and transcode inside it — retried until it
239
+ // succeeds, surviving restarts:
240
+ export const transcodeVideo = workflow('transcodeVideo', (refId: string) =>
241
+ Effect.gen(function* () {
242
+ const storage = yield* StorageService
243
+ const renditions = yield* storage.transcode(refId) // retried by the workflow engine
244
+ return { count: renditions.length }
245
+ }),
246
+ )
247
+ ```
248
+
249
+ So: convenient by default (auto-fork), durable when you need it (workflow + the
250
+ explicit primitive). The two coexist — same `transcode()` call underneath.
251
+
252
+ > ffmpeg **transcode** (re-encoding) is opt-in as above. On-the-fly *image*
253
+ > resizing stays separate (query params on the serve URL, powered by `sharp`).
254
+
255
+ ## Binding an entity to an asset — `assetRef()`
256
+
257
+ ```ts
258
+ import { table, id } from '@voltro/database'
259
+ import { assetRef } from '@voltro/plugin-storage'
260
+
261
+ const employees = table('employees', {
262
+ id: id(),
263
+ avatar: assetRef(), // stores a _voltro_storage_refs id
264
+ })
265
+ ```
266
+
267
+ `assetRef()` is a real FK to `_voltro_storage_refs.id` (nullable, `onDelete:
268
+ 'setNull'` — deleting the blob clears the link) by default; pass `{ fk: false }`
269
+ for a plain typed id column, `{ nullable: false }` to require it, or `{ onDelete:
270
+ 'cascade' }` to change the FK semantics. Resolve the linked asset with the storage
271
+ service (`head` / `mintUrl`).
272
+
273
+ For the one-column shorthand there's a `withStorage()` mixin (applied via `.with`,
274
+ the same convention as `tenant()` / `audit()`), which adds a single `asset`
275
+ column:
276
+
277
+ ```ts
278
+ import { withStorage } from '@voltro/plugin-storage'
279
+
280
+ const posts = table('posts', { id: id() }).with(withStorage()) // → posts.asset
281
+ ```
282
+
283
+ Prefer `assetRef()` inline when you want a custom column name (`avatar`, `cover`)
284
+ or several asset columns.
285
+
286
+ ## Swappable engine
287
+
288
+ The provider is the swap point; the rest of your code is provider-agnostic.
289
+
290
+ ```ts
291
+ storagePlugin({ provider: 'memory' }) // dev / tests
292
+ storagePlugin({ provider: 'database' }) // bytes in your DB — zero extra infra
293
+ storagePlugin({ provider: 'filesystem', root: '.voltro-storage' }) // on-disk dev
294
+ storagePlugin({ provider: 's3', bucket: 'prod', region: 'eu-central-1',
295
+ cdnBaseUrl: 'https://cdn.example.com' }) // AWS S3 + CDN
296
+ storagePlugin({ provider: 'minio', bucket: 'prod',
297
+ endpoint: 'http://localhost:9000' }) // MinIO
298
+ storagePlugin({ provider: 'azure', bucket: 'prod',
299
+ accountName: 'acct', accountKey: process.env.AZURE_STORAGE_KEY }) // Azure Blob
300
+ // Cloudflare R2 / GCS / Backblaze B2 / Wasabi — any S3-compatible bucket via `endpoint`.
301
+ storagePlugin({ provider: 's3', bucket: 'prod',
302
+ endpoint: 'https://<acct>.r2.cloudflarestorage.com',
303
+ cdnBaseUrl: 'https://files.example.com', publicAcl: false })
304
+ ```
305
+
306
+ Two providers stand apart from the S3-compatible family:
307
+
308
+ - **`azure`** — Azure Blob Storage (not S3-compatible). Presigned URLs are
309
+ SAS tokens; needs `accountName` + `accountKey` (or `AZURE_STORAGE_ACCOUNT` /
310
+ `AZURE_STORAGE_KEY`), or a `connectionString`. Requires the optional
311
+ `@azure/storage-blob` dependency.
312
+ - **`database`** — stores the blob bytes IN the app's database
313
+ (`_voltro_storage_blobs`, a `bytes()` / `BYTEA` column) via the same
314
+ DataStore. **Zero extra infra** — no bucket, survives restarts (unlike
315
+ `memory`). Served through the app query (no CDN-direct). Use for small files;
316
+ large blobs belong in object storage.
317
+
318
+ The **`filesystem`** provider writes blobs **atomically** — bytes go to a
319
+ unique temp file in the target directory, then `rename()` over the final path
320
+ (POSIX-atomic). A reader never sees a half-written file and two concurrent
321
+ writers of the same key can't interleave into a corrupt blob. That makes it
322
+ safe on a **shared ReadWriteMany volume** (e.g. CephFS) with multiple app
323
+ replicas — point `STORAGE_ROOT` at the mount. (A `ReadWriteOnce` block volume
324
+ mounts on one pod only, so it can't back more than one replica.)
325
+
326
+ Options resolve from env when omitted: `STORAGE_PROVIDER`, `S3_BUCKET` /
327
+ `STORAGE_BUCKET`, `S3_REGION` / `AWS_REGION`, `S3_ENDPOINT` / `MINIO_ENDPOINT`,
328
+ `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY`, `STORAGE_ROOT`, `STORAGE_CDN_URL`,
329
+ `S3_FORCE_PATH_STYLE=1`. Pass a `StorageProvider` object to bring your own
330
+ backend. `publicAcl: false` skips per-object `public-read` ACLs (use a bucket
331
+ policy + `cdnBaseUrl` instead — required for R2).
332
+
333
+ ## Public objects — minimal middleware
334
+
335
+ A `visibility: 'public'` object is readable with no auth. When `cdnBaseUrl`
336
+ is set, `getUrl(id)` returns `${cdnBaseUrl}/${key}` and the serve route 302s
337
+ there — reads never hit the app. WITHOUT `cdnBaseUrl` (a bare S3/MinIO bucket,
338
+ which is private by default — and on `filesystem` / `memory`), the framework
339
+ does NOT hand out a raw bucket URL (it would 403); the serve route streams the
340
+ bytes through the app instead, with `cache-control: public, max-age=31536000,
341
+ immutable`. So public objects are always cacheable; `cdnBaseUrl` is what moves
342
+ the read path off the app and onto the edge.
343
+
344
+ ## Private objects — the access model
345
+
346
+ Access is the union of two layers; a request is allowed if **either** grants it.
347
+
348
+ ### 1. Inline access policy
349
+
350
+ `put({ access })` takes an **OR-of-rules** array. A rule GRANTS when EVERY
351
+ condition it declares is satisfied (AND); access is allowed if ANY rule grants.
352
+ The owner (`subject.id === ref.ownerId`) is always allowed.
353
+
354
+ ```ts
355
+ yield* storage.put({
356
+ bytes, contentType: 'application/pdf',
357
+ ownerId: ctx.request.subject.id,
358
+ password: 'optional-file-password', // hashed; gates a { password: true } rule
359
+ access: [
360
+ { roles: ['admin'] }, // admins, OR
361
+ { tenant: true }, // any same-tenant subject, OR
362
+ { groups: ['finance'], scopes: ['files:read'] }, // finance AND files:read
363
+ ],
364
+ })
365
+ ```
366
+
367
+ Rule conditions: `owner`, `roles[]`, `groups[]`, `scopes[]`, `tenant`,
368
+ `apiKey`, `password`, `guard` (a named custom guard). Roles/groups come from
369
+ `subject.metadata.roles` / `.groups` by default — override the resolution:
370
+
371
+ ```ts
372
+ storagePlugin({
373
+ access: {
374
+ resolveGroups: (subject) => lookupGroups(subject.id), // sync | Promise
375
+ guards: { vip: ({ subject, ref }) => subject.metadata?.tier === 'vip' },
376
+ },
377
+ })
378
+ ```
379
+
380
+ ### 2. Per-object grants
381
+
382
+ Explicitly share one object with a user / group / api-key, optionally expiring:
383
+
384
+ ```ts
385
+ yield* storage.grant({ refId, principalType: 'user', principalId: 'user_123', permission: 'read' })
386
+ yield* storage.grant({ refId, principalType: 'group', principalId: 'team-a', expiresAt: in7Days })
387
+ yield* storage.revoke(grantId)
388
+ const grants = yield* storage.listGrants(refId)
389
+ ```
390
+
391
+ Grants persist in `_voltro_storage_grants` and are managed from the dashboard's
392
+ **Storage** tab.
393
+
394
+ ## Delivering a private file
395
+
396
+ ```ts
397
+ // Access-checked. Returns a presigned URL (s3/minio) OR an app serve URL
398
+ // carrying a short-lived signed grant token (filesystem/memory). Fails with
399
+ // StorageAccessDenied (403) if the subject isn't allowed.
400
+ const url = yield* storage.mintUrl(fileId, ctx.request.subject, { password })
401
+ ```
402
+
403
+ For large uploads, skip the wire and presign a direct-to-bucket PUT
404
+ (s3/minio): `storage.mintUploadUrl({ key, contentType })`.
405
+
406
+ ## Typed RPC queries (client-facing)
407
+
408
+ The plugin ships queries your client can call with no extra wiring — and their
409
+ errors (`StorageError`, `StorageAccessDenied`) are merged into every
410
+ procedure's wire-error union so they decode typed on the client:
411
+
412
+ `storage.upload`, `storage.mintUploadTicket`, `storage.mintUrl`,
413
+ `storage.mintUploadUrl`, `storage.mintPresignedUpload`, `storage.finalizeUpload`,
414
+ `storage.beginResumableUpload`, `storage.beginMultipartUpload`,
415
+ `storage.signMultipartPart`, `storage.completeMultipartUpload`,
416
+ `storage.abortMultipartUpload`, `storage.ingestUrl`, `storage.share`,
417
+ `storage.revoke`, `storage.listGrants`, `storage.listRefs`. (The upload hook picks
418
+ the right ones per `prefer`; you rarely call them directly.)
419
+
420
+ ### The grant RPCs are owner-only
421
+
422
+ `storage.share`, `storage.revoke` and `storage.listGrants` require that the
423
+ caller OWNS the object (`ref.ownerId === subject.id`), or carries `admin:full`.
424
+
425
+ They used to require nothing. Each took an object id straight off the wire and
426
+ handed it to a service method that performs no authorization — correct for a
427
+ trusted server-side API — with nothing in between. Any authenticated caller
428
+ could grant themselves read or write on any object in the installation, revoke
429
+ anyone else's grants, and enumerate who an object is shared with.
430
+
431
+ A missing object and an unowned object both report the same 403; a 404 would let
432
+ an unauthorized caller probe which ids exist.
433
+
434
+ If a server-side flow legitimately shares on someone else's behalf, call the
435
+ SERVICE (`yield* StorageService` → `.grant(...)`) rather than the RPC — the
436
+ service is deliberately unrestricted, and the gate belongs at the wire. For an
437
+ admin surface, give the subject `admin:full` (see
438
+ [`auth.resolveScopes`](/docs/authentication/strategies) for lifting a role into
439
+ scopes).
440
+
441
+ ## StorageService API
442
+
443
+ | Method | Returns |
444
+ |---|---|
445
+ | `put(input)` | `StorageRef` — `{ id, tenantId, ownerId, bucket, key, contentType, size, checksum, visibility, accessPolicy, createdAt }` |
446
+ | `get(id, opts?)` | `{ bytes, ref }` (tenant-guarded when `opts.tenantId` set) |
447
+ | `getUrl(id, opts?)` | direct/CDN URL (public) or presigned/serve URL (private) |
448
+ | `mintUrl(id, subject, opts?)` | access-checked delivery URL |
449
+ | `mintUploadUrl(input)` | presigned PUT (s3/minio) |
450
+ | `head(id, opts?)` | `StorageRef \| null` |
451
+ | `getRange(id, { start, endInclusive }, opts?)` | `{ bytes, ref, totalSize }` — a byte slice (backs Range serving) |
452
+ | `delete(id, opts?)` | refcount-aware — dedup'd siblings survive |
453
+ | `listRefs(input?)` | `{ refs, nextOffset }` — browse/search the ref index |
454
+ | `checkAccess(ref, subject, opts?)` | `boolean` |
455
+ | `grant` / `revoke` / `listGrants` | manage per-object grants |
456
+
457
+ Failures are a typed `StorageError` (the `transient` flag drives retry — default
458
+ 3 attempts) or `StorageAccessDenied` (403).
459
+
460
+ ### Browse / search files — `listRefs()`
461
+
462
+ The `folder`, `tags`, and `ownerId` you set on each `put()` are queryable
463
+ first-class — a media library or file manager never has to touch
464
+ `_voltro_storage_refs` by hand. `listRefs()` filters by folder **prefix** (a path
465
+ descendant match), all-of `tags` (AND), and `ownerId`, newest-first, with offset
466
+ paging:
467
+
468
+ ```ts
469
+ import { StorageService } from '@voltro/plugin-storage'
470
+ import { Effect } from 'effect'
471
+
472
+ const browse = Effect.gen(function* () {
473
+ const storage = yield* StorageService
474
+ // Page 1: everything under `photos/` (incl. `photos/2026/…`) tagged `hero`.
475
+ const page = yield* storage.listRefs({
476
+ tenantId: 'org_123',
477
+ folder: 'photos',
478
+ tags: ['hero'],
479
+ ownerId: 'user_42',
480
+ limit: 24,
481
+ offset: 0,
482
+ })
483
+ // page.refs: ReadonlyArray<StorageRef>, newest first.
484
+ // page.nextOffset: number | null — pass back as `offset` for the next page,
485
+ // or null when this was the last page.
486
+ return page
487
+ })
488
+ ```
489
+
490
+ - **`folder`** matches the value exactly OR as a `/`-delimited ancestor
491
+ (`photos` ⇒ `photos`, `photos/2026`; never `photos-archive`). A trailing slash
492
+ is ignored.
493
+ - **`tags`** requires ALL listed tags on the ref (AND).
494
+ - **`ownerId`** / **`tenantId`** scope to one owner / tenant (`tenantId: null`
495
+ is the system/global partition; omit `tenantId` only for a trusted admin read
496
+ that spans tenants).
497
+ - Transcode **derivatives** (renditions / posters) are excluded by default; pass
498
+ `includeDerived: true` to include them.
499
+ - **Paging**: `limit` is clamped to `[1, 500]` (default 50); `nextOffset` is
500
+ non-null while another page follows.
501
+
502
+ `listRefs` filters the **index** — it does not run the per-object access policy
503
+ (it delivers no bytes). Scope it to the caller's `tenantId` / `ownerId` for an
504
+ end-user surface, and deliver any listed private object through `mintUrl`. The
505
+ `storage.listRefs` rpc query takes the tenant from the caller's subject
506
+ server-side, so a client can only browse its own tenant.
507
+
508
+ Reach the service with `yield* StorageService` inside an Effect handler — the
509
+ plugin provides it as the handler base layer. The service methods are
510
+ Effect-returning (`storage.put(...)` is an `Effect<StorageRef>`); from a plain
511
+ `async` handler, run them with `Effect.runPromise`, or write the handler in
512
+ Effect form.
513
+
514
+ ## Serve endpoint
515
+
516
+ `GET /_voltro/storage/:id` (mounted pre-auth):
517
+
518
+ - **public** → 302 to the CDN/bucket URL when `cdnBaseUrl` is set; otherwise
519
+ serve the bytes with `cache-control: public, max-age=31536000, immutable`
520
+ (no auth either way).
521
+ - **private** → checks a `?t=<grant-token>` (minted by `mintUrl`) OR the session
522
+ cookie + access policy; then 403, a presigned 302, or a served body with
523
+ `cache-control: private, no-store`.
524
+
525
+ ### HTTP Range — `206 Partial Content` (video seeking, bounded delivery)
526
+
527
+ When the object is served **through the app** (a non-presigning provider, or a
528
+ private object without a bucket presign), the serve route honours an HTTP
529
+ `Range:` request header so a `<video>`/`<audio>` element can seek and a client
530
+ can resume — only the requested slice leaves the backend (via the provider's
531
+ `getRange`; memory/database fall back to a buffered slice):
532
+
533
+ - A **full** `GET` responds `200` with `Accept-Ranges: bytes`.
534
+ - A satisfiable `Range: bytes=<start>-<end>` responds **`206 Partial Content`**
535
+ with `Content-Range: bytes <start>-<end>/<total>` and just that slice. Open-
536
+ ended (`bytes=N-`) reads to EOF; a suffix (`bytes=-N`) reads the last N bytes.
537
+ - A range entirely past the object → **`416 Range Not Satisfiable`** with
538
+ `Content-Range: bytes */<total>`.
539
+ - A malformed `Range` header falls back to the full `200` (per RFC 7233).
540
+
541
+ For **presigning** providers (s3/minio) a private object 302-redirects to the
542
+ bucket, which serves Range natively — the client re-issues its `Range` against
543
+ the presigned URL, so seeking works without the bytes ever passing through the
544
+ app. The `Range`/`Content-Range`/`Accept-Ranges` headers are added to the serve
545
+ route's CORS allow/expose lists so a cross-origin `fetch()` can drive them.
546
+
547
+ ### Cross-origin: absolute URLs + CORS
548
+
549
+ When the api is a **different origin** than your web app (the common deploy —
550
+ `api.example.com` vs your web origin), the serve URL must be **absolute** or a
551
+ browser `<img>` / a server-to-server AI-gateway fetch can't resolve it. Set the
552
+ api's public base and `getUrl`/`mintUrl`/the serve route emit absolute URLs:
553
+
554
+ ```
555
+ VOLTRO_PUBLIC_URL=https://api.example.com # or storagePlugin({ publicBaseUrl })
556
+ ```
557
+
558
+ Without it, serve URLs are **relative** (`/_voltro/storage/:id`) — correct only
559
+ same-origin. For public objects a `cdnBaseUrl` / `STORAGE_CDN_URL` already yields
560
+ an absolute CDN URL; `VOLTRO_PUBLIC_URL` covers the no-CDN + private cases.
561
+
562
+ To let a cross-origin browser `fetch()`/canvas the bytes (a plain `<img>` needs
563
+ nothing), allow its origin:
564
+
565
+ ```
566
+ STORAGE_ALLOWED_ORIGINS=https://app.example.com # comma-separated, or '*'
567
+ ```
568
+
569
+ > Diagnostics: on boot the plugin logs `storage: provider=… presign=… publicUrl=…
570
+ > transforms=…`, and **warns loudly** when `provider=memory` outside tests
571
+ > (uploads are in-memory and lost on restart — empty/placeholder storage env
572
+ > silently falls back to memory) or when serve URLs would be relative on a
573
+ > non-presigning provider with no public base.
574
+
575
+ ## Diagnostics — `voltro storage`
576
+
577
+ Probe the configured backend from the terminal so a broken setup fails LOUD
578
+ before an upload does:
579
+
580
+ ```
581
+ voltro storage doctor # provider + a real read/write round-trip + config sanity
582
+ voltro storage cors # print the recommended bucket CORS for presigned uploads
583
+ ```
584
+
585
+ `doctor` reads your storage env (`STORAGE_PROVIDER` / `S3_*` / `VOLTRO_PUBLIC_URL`
586
+ / `STORAGE_ALLOWED_ORIGINS`), prints the resolved provider banner, does an actual
587
+ `put`→`get`→`delete` round-trip against it, and flags the two silent footguns —
588
+ `provider=memory` (data lost on restart) and relative serve URLs (cross-origin
589
+ breakage). Exit code is non-zero when it finds an issue, so it drops into CI.
590
+
591
+ `cors` prints an S3/R2 `CORSRules` JSON to apply on your bucket — only needed if
592
+ you upload via a presigned direct-to-bucket PUT (`storage.mintUploadUrl`); the
593
+ through-app upload route sets its own CORS from `STORAGE_ALLOWED_ORIGINS`.
594
+
595
+ **Dev restart-safety.** `memory` (the empty-env fallback) loses everything on a
596
+ process/pod restart. For a dev environment that survives restarts, use
597
+ `filesystem` with a mounted volume (`STORAGE_ROOT` → a Docker/k8s volume — a
598
+ `ReadWriteOnce` PVC is fine for a single dev replica) or the `database` provider
599
+ (bytes in your DB, zero extra infra). `voltro storage doctor` warns when you're on
600
+ `memory` so this never surprises you in a deployed dev cluster.
601
+
602
+ ## Upload constraints
603
+
604
+ Reject bad uploads before the bytes ever reach the provider:
605
+
606
+ ```ts
607
+ storagePlugin({
608
+ limits: {
609
+ maxBytes: 10 * 1024 * 1024, // 10 MB cap
610
+ allowedContentTypes: ['image/*', 'application/pdf'], // exact or `type/*`
611
+ sniff: true, // magic-byte check
612
+ },
613
+ })
614
+ ```
615
+
616
+ `sniff` verifies the bytes' magic number matches the declared `contentType`, so
617
+ an executable can't slip in as `image/png`. A violation fails `put` with a typed
618
+ `StorageRejected` — `reason` is `too-large`, `content-type-not-allowed`, or
619
+ `content-mismatch`.
620
+
621
+ ## Virus scanning
622
+
623
+ A `scan` hook runs on every `put` BEFORE the bytes are stored. The first-party
624
+ `clamavScanner` talks to a running [ClamAV](https://www.clamav.net) daemon over
625
+ its INSTREAM protocol (no npm dependency):
626
+
627
+ ```ts
628
+ import { storagePlugin, clamavScanner } from '@voltro/plugin-storage'
629
+
630
+ storagePlugin({ scan: clamavScanner({ host: '127.0.0.1', port: 3310 }) })
631
+ ```
632
+
633
+ A detection fails `put` with a typed `StorageScanRejected` (carrying the threat
634
+ name) and stores nothing. A scanner outage **fails closed** — a transient
635
+ `StorageError`, so a misconfigured scanner never silently waves files through.
636
+ Bring your own engine (VirusTotal, Cloudmersive, an ICAP gateway) by
637
+ implementing `StorageScanner`:
638
+
639
+ ```ts
640
+ const myScanner: StorageScanner = async ({ bytes, contentType }) => {
641
+ const clean = await scanSomehow(bytes)
642
+ return clean ? { clean: true } : { clean: false, threat: 'detected' }
643
+ }
644
+ ```
645
+
646
+ Scanning applies to every path that runs `put` — through-app uploads, the
647
+ resumable finalize, AND `prefer: 'presign'` (which fetches the bytes back and
648
+ runs `finalizeUpload`, so a presigned upload is scanned out of band and fails
649
+ closed on a hit). Only a **raw** `storage.mintUploadUrl` PUT that you finalize
650
+ yourself bypasses the scan — prefer the hook's `presign` transport, which does
651
+ not.
652
+
653
+ ## On-the-fly image transforms
654
+
655
+ Resize / re-encode images via query params on the serve URL — powered by the
656
+ optional [`sharp`](https://sharp.pixelplumbing.com) dependency:
657
+
658
+ ```html
659
+ <img src="/_voltro/storage/file_01j…?w=400&h=300&format=webp&q=80" />
660
+ ```
661
+
662
+ Params: `w`, `h` (px), `format` (`webp` / `avif` / `jpeg` / `png`), `q` (quality
663
+ 1–100), `fit` (`cover` / `contain` / …). Transformed variants are cached.
664
+
665
+ The fast path stays fast: **without** transform params a public object 302s
666
+ straight to the CDN (when `cdnBaseUrl` is set) or streams with the immutable
667
+ cache otherwise; **with** params the app streams + transforms (opt-in cost
668
+ only when you ask for it). Private objects are access-checked first, then
669
+ transformed. If `sharp` isn't installed the original bytes are served unchanged.
670
+ For CDN-direct public images at scale, a CDN-level image resizer (Cloudflare
671
+ Images, etc.) avoids the app round-trip entirely.
672
+
673
+ ## Dashboard
674
+
675
+ The **Storage** tab (local devtools + cloud) lists every object with its
676
+ visibility, size and owner, shows public/private + per-tenant stats, and lets
677
+ you view + share + revoke per-object grants. Backed by
678
+ `/_voltro/inspect/plugins/storage/{refs,stats,grants,share,revoke}`.
679
+
680
+ ## Lifecycle: quotas, GC, URL ingest
681
+
682
+ - **Per-tenant quotas**: `storagePlugin({ quota: { maxBytes, maxCount } })` —
683
+ the headroom is consumed ATOMICALLY (a compare-and-set over the
684
+ `_voltro_storage_usage` counter row) BEFORE bytes are stored, so concurrent
685
+ uploads — even across replicas — cannot overshoot the cap; a failed write
686
+ refunds the reservation, a delete releases it. Over-budget fails with a
687
+ typed `StorageRejected` (`'quota-exceeded'`). `storage.usage(tenantId)`
688
+ returns `{ bytes, count }` for dashboards / limits.
689
+ - **Orphan GC**: `storage.sweepOrphans({ limit, dryRun })` removes dangling refs
690
+ whose bytes are gone from the provider (a crash between store + insert, or a
691
+ manual blob delete). Bounded + repeatable; run it from a scheduled task.
692
+ - **Ingest an external URL**: `storage.ingestUrl(url, opts)` (also the
693
+ `storage.ingestUrl` rpc) fetches a URL server-side and stores it as an asset —
694
+ the one-liner for CMS migration ("adopt 21k legacy image URLs"). It fetches an
695
+ arbitrary URL from the server, so pass **trusted URLs only** (SSRF); front with
696
+ an allow-list if the URL is user-supplied.
697
+
698
+ ## Schema
699
+
700
+ `_voltro_storage_refs` (id, tenantId, ownerId, bucket, key, contentType, size,
701
+ checksum, visibility, accessPolicy, passwordHash, createdAt, + width, height,
702
+ duration, placeholder, folder, tags, alt, caption, derivedFrom, kind),
703
+ `_voltro_storage_grants` (id, refId, principalType, principalId, permission,
704
+ createdAt, expiresAt, createdBy), `_voltro_storage_usage` (id, tenantId
705
+ `UNIQUE`, bytes, count, updatedAt — one counter row per tenant, the atomic
706
+ arbiter for quota enforcement), and `_voltro_storage_blobs` (id, key, data
707
+ `BYTEA`, contentType, size — only written by the `database` provider) are
708
+ framework-internal and auto-migrated on every SQL-backed app — no `*.entity.ts`
709
+ needed. `passwordHash` is server-only and never serialized.