@voltro/cli 0.30.1 → 0.31.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +163 -0
- package/dist/apiBuild-BOqL1dlg.js +2 -0
- package/dist/{apiBuild-DmkzRyv7.js → apiBuild-CLPt-pX6.js} +2 -2
- package/dist/appGraph-CvQCte0z.js +2 -0
- package/dist/appGraph-gQ_6GkQQ.js +110 -0
- package/dist/bin.js +3 -3
- package/dist/checkCommand-ClRvH_SN.js +2 -0
- package/dist/{checkCommand-C4konUy9.js → checkCommand-D7r6_pk7.js} +439 -530
- package/dist/{commands-BkJft9K2.js → commands-CNVVv03h.js} +1941 -1880
- package/dist/{dbCommand-qn__kLEt.js → dbCommand-BH8GAYuf.js} +76 -19
- package/dist/dbCommand-BwQ_Jl98.js +2 -0
- package/dist/{dev-DXhhZ7-f.js → dev-DOZVMVfi.js} +1 -1
- package/dist/{dev-C_NPcLDp.js → dev-DdQEn2Ia.js} +1986 -1967
- package/dist/{frameworkTableAssembly-Cpgn2e88.js → frameworkTableAssembly-BwIrO5nv.js} +122 -103
- package/dist/frameworkTableAssembly-D-EebUQX.js +2 -0
- package/dist/index.js +1 -1
- package/dist/{inspectMetrics-CkB2J0Zn.js → inspectMetrics-BeEAnjqw.js} +774 -527
- package/dist/{sdkgen-wsWLrq4l.js → sdkgen-v9WV8zRi.js} +1 -1
- package/dist/{serveCommand-HtI4X5kj.js → serveCommand-KsKXXWn4.js} +294 -293
- package/dist/serveEntry.js +2 -2
- package/dist/{start-egkbpugc.js → start-CI2QdKy9.js} +351 -346
- package/dist/startEntry.js +2 -2
- package/package.json +17 -17
- package/templates/AGENTS.md +43 -43
- package/templates/agent-docs/_index.md +43 -43
- package/templates/agent-docs/_manifest.json +84 -42
- package/templates/agent-docs/cli.md +4 -2
- package/templates/agent-docs/database/migrations.md +14 -1
- package/templates/agent-docs/internationalization.md +54 -3
- package/templates/agent-docs/plugins/ai-flows.md +148 -0
- package/templates/agent-docs/plugins/analytics-postgres.md +90 -0
- package/templates/agent-docs/plugins/atlassian.md +378 -0
- package/templates/agent-docs/plugins/audit.md +324 -0
- package/templates/agent-docs/plugins/auth-auth0.md +92 -0
- package/templates/agent-docs/plugins/auth-clerk.md +90 -0
- package/templates/agent-docs/plugins/auth-kinde.md +88 -0
- package/templates/agent-docs/plugins/auth-oidc.md +95 -0
- package/templates/agent-docs/plugins/auth-supabase.md +103 -0
- package/templates/agent-docs/plugins/auth-workos.md +105 -0
- package/templates/agent-docs/plugins/auth.md +227 -0
- package/templates/agent-docs/plugins/billing.md +324 -0
- package/templates/agent-docs/plugins/broadcast.md +144 -0
- package/templates/agent-docs/plugins/cdc-out.md +116 -0
- package/templates/agent-docs/plugins/clickhouse.md +91 -0
- package/templates/agent-docs/plugins/datadog.md +105 -0
- package/templates/agent-docs/plugins/deactivation.md +114 -0
- package/templates/agent-docs/plugins/duckdb.md +82 -0
- package/templates/agent-docs/plugins/flags.md +146 -0
- package/templates/agent-docs/plugins/governance.md +143 -0
- package/templates/agent-docs/plugins/licensing.md +74 -0
- package/templates/agent-docs/plugins/logship.md +56 -0
- package/templates/agent-docs/plugins/mail.md +334 -0
- package/templates/agent-docs/plugins/moderation.md +65 -0
- package/templates/agent-docs/plugins/multitenancy.md +99 -0
- package/templates/agent-docs/plugins/notifications.md +252 -0
- package/templates/agent-docs/plugins/openapi.md +94 -0
- package/templates/agent-docs/plugins/postgis.md +140 -0
- package/templates/agent-docs/plugins/posthog.md +91 -0
- package/templates/agent-docs/plugins/presence.md +115 -0
- package/templates/agent-docs/plugins/prometheus.md +124 -0
- package/templates/agent-docs/plugins/ratelimit.md +274 -0
- package/templates/agent-docs/plugins/rbac.md +270 -0
- package/templates/agent-docs/plugins/scim.md +63 -0
- package/templates/agent-docs/plugins/search.md +116 -0
- package/templates/agent-docs/plugins/sentry.md +192 -0
- package/templates/agent-docs/plugins/soft-delete.md +72 -0
- package/templates/agent-docs/plugins/sso-saml.md +132 -0
- package/templates/agent-docs/plugins/storage.md +709 -0
- package/templates/agent-docs/plugins/tinybird.md +92 -0
- package/templates/agent-docs/plugins/versioning.md +226 -0
- package/templates/agent-docs/plugins/webhooks.md +585 -0
- package/templates/agent-docs/security.md +12 -1
- package/templates/agent-docs/whats-new.md +81 -37
- package/templates/apps/api-ai/package.json +7 -7
- package/templates/apps/api-auth/package.json +8 -8
- package/templates/apps/api-backend/package.json +7 -7
- package/templates/apps/api-backend-deactivation/package.json +7 -7
- package/templates/apps/api-backend-mail/package.json +8 -8
- package/templates/apps/api-backend-mariadb/package.json +9 -9
- package/templates/apps/api-backend-sqlite/package.json +8 -8
- package/templates/apps/api-backend-storage/package.json +8 -8
- package/templates/apps/api-cms/package.json +10 -10
- package/templates/apps/api-collab/package.json +8 -8
- package/templates/apps/api-data-advanced/package.json +8 -8
- package/templates/apps/api-durable/package.json +8 -8
- package/templates/apps/api-feature-flags/package.json +9 -9
- package/templates/apps/api-governance/package.json +8 -8
- package/templates/apps/api-kv/package.json +8 -8
- package/templates/apps/api-moderation/package.json +8 -8
- package/templates/apps/api-observability/package.json +8 -8
- package/templates/apps/api-ratelimit/package.json +8 -8
- package/templates/apps/api-rbac/package.json +8 -8
- package/templates/apps/api-rest/package.json +7 -7
- package/templates/apps/api-saas/package.json +11 -11
- package/templates/apps/api-saas-starter/package.json +10 -10
- package/templates/apps/api-search/package.json +8 -8
- package/templates/apps/api-status/package.json +8 -8
- package/templates/apps/api-versioning/package.json +8 -8
- package/templates/apps/api-webhooks/package.json +9 -9
- package/templates/apps/changelog/package.json +6 -6
- package/templates/apps/edge-functions/package.json +2 -2
- package/templates/apps/frontend-admin/package.json +8 -8
- package/templates/apps/frontend-app/package.json +8 -8
- package/templates/apps/frontend-auth/package.json +8 -8
- package/templates/apps/frontend-blank/package.json +7 -7
- package/templates/apps/frontend-cms/package.json +9 -9
- package/templates/apps/frontend-collab/package.json +9 -9
- package/templates/apps/frontend-contact/package.json +7 -7
- package/templates/apps/frontend-dashboard/package.json +7 -7
- package/templates/apps/frontend-docs/package.json +7 -7
- package/templates/apps/frontend-i18n/package.json +6 -6
- package/templates/apps/frontend-landing/package.json +7 -7
- package/templates/apps/frontend-portal/package.json +8 -8
- package/templates/apps/frontend-saas/package.json +8 -8
- package/templates/apps/frontend-spa/package.json +7 -7
- package/templates/apps/frontend-ssr/package.json +7 -7
- package/templates/apps/frontend-ssr-api/package.json +8 -8
- package/templates/apps/frontend-static-blog/package.json +6 -6
- package/templates/apps/frontend-status/package.json +8 -8
- package/templates/apps/mobile-app/package.json +2 -2
- package/dist/apiBuild-Ca6Sq5jy.js +0 -2
- package/dist/checkCommand-DHD0aBeR.js +0 -2
- package/dist/dbCommand-wfGi3FgG.js +0 -2
- package/dist/frameworkTableAssembly-Bn_c4Mnl.js +0 -2
|
@@ -0,0 +1,709 @@
|
|
|
1
|
+
# Storage
|
|
2
|
+
|
|
3
|
+
> File storage behind one StorageService — public objects served direct from the bucket/CDN, private objects gated by an access policy + per-object grants. S3 / R2 / GCS / MinIO / filesystem / memory providers, presigned URLs, a dashboard browser.
|
|
4
|
+
|
|
5
|
+
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
<!-- source: en/plugins/storage.md -->
|
|
10
|
+
## Storage
|
|
11
|
+
|
|
12
|
+
_File storage behind one StorageService — public objects served direct from the bucket/CDN, private objects gated by an access policy + per-object grants. S3 / R2 / GCS / MinIO / filesystem / memory providers, presigned URLs, a dashboard browser._
|
|
13
|
+
|
|
14
|
+
`@voltro/plugin-storage` is file storage behind one `StorageService`. Wire a
|
|
15
|
+
provider in `app.config.ts`; consume it in handlers and actions via
|
|
16
|
+
`yield* StorageService`. The bytes live in the provider (S3-compatible bucket,
|
|
17
|
+
filesystem, or in-memory); metadata rows live in `_voltro_storage_refs` +
|
|
18
|
+
`_voltro_storage_grants` (both auto-migrated on a SQL store).
|
|
19
|
+
|
|
20
|
+
Every object is **public** or **private**:
|
|
21
|
+
|
|
22
|
+
- **Public** — cacheable + no auth. With `cdnBaseUrl` (or `STORAGE_CDN_URL`)
|
|
23
|
+
set to a CDN / publicly-readable bucket endpoint, the serve route 302s
|
|
24
|
+
straight there and the app never touches the read path (edge-cacheable).
|
|
25
|
+
WITHOUT `cdnBaseUrl`, the serve route streams the bytes through the app
|
|
26
|
+
from the (private) bucket with a 1-year `immutable` cache — still
|
|
27
|
+
browser-cacheable, bucket stays private. A bare S3/MinIO bucket is private
|
|
28
|
+
by default, so the framework never 302s to a raw bucket URL unless you've
|
|
29
|
+
declared a public base via `cdnBaseUrl` (otherwise it would land on a 403).
|
|
30
|
+
- **Private** (the default — secure by default) — gated by an **access policy**
|
|
31
|
+
plus **per-object grants**; delivered via a presigned URL or a short-lived
|
|
32
|
+
signed grant token.
|
|
33
|
+
|
|
34
|
+
## Quick start
|
|
35
|
+
|
|
36
|
+
```ts
|
|
37
|
+
// app.config.ts
|
|
38
|
+
import { storagePlugin } from '@voltro/plugin-storage'
|
|
39
|
+
|
|
40
|
+
export default {
|
|
41
|
+
type: 'api' as const,
|
|
42
|
+
name: 'myApi',
|
|
43
|
+
store: 'postgres' as const,
|
|
44
|
+
plugins: [
|
|
45
|
+
// dev: zero-config in-memory. prod: swap the provider (see below).
|
|
46
|
+
storagePlugin({ provider: 'memory' }),
|
|
47
|
+
],
|
|
48
|
+
}
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
```ts
|
|
52
|
+
// actions/uploadAvatar.action.server.ts (+ matching .action.ts)
|
|
53
|
+
import { StorageService } from '@voltro/plugin-storage'
|
|
54
|
+
import { Effect } from 'effect'
|
|
55
|
+
|
|
56
|
+
export default (input: { bytes: Uint8Array; contentType: string }, ctx) =>
|
|
57
|
+
Effect.gen(function* () {
|
|
58
|
+
const storage = yield* StorageService
|
|
59
|
+
const ref = yield* storage.put({
|
|
60
|
+
bytes: input.bytes,
|
|
61
|
+
contentType: input.contentType,
|
|
62
|
+
visibility: 'public', // public avatar
|
|
63
|
+
ownerId: ctx.request.subject.id,
|
|
64
|
+
key: `avatars/${ctx.request.subject.id}/avatar.png`,
|
|
65
|
+
})
|
|
66
|
+
// For a public object this is the CDN/bucket URL (no app round-trip).
|
|
67
|
+
return { url: yield* storage.getUrl(ref.id) }
|
|
68
|
+
})
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
The ref id is a `file_…` TypeID — URL-safe and unguessable.
|
|
72
|
+
|
|
73
|
+
## Uploading files — `useUpload()` (client)
|
|
74
|
+
|
|
75
|
+
The one-liner. Works on every provider (filesystem/dev, s3/r2/prod),
|
|
76
|
+
cross-origin, no base64:
|
|
77
|
+
|
|
78
|
+
```tsx
|
|
79
|
+
import { useUpload } from '@voltro/client'
|
|
80
|
+
|
|
81
|
+
function AvatarUpload() {
|
|
82
|
+
const { upload, progress, status, cancel } = useUpload('myApi')
|
|
83
|
+
return (
|
|
84
|
+
<input type="file" onChange={async (e) => {
|
|
85
|
+
const file = e.target.files?.[0]
|
|
86
|
+
if (!file) return
|
|
87
|
+
const asset = await upload(file, { visibility: 'public', folder: 'avatars' })
|
|
88
|
+
// asset: { id, url, name, mime, size, width?, height? }
|
|
89
|
+
}} />
|
|
90
|
+
)
|
|
91
|
+
}
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
`useUpload(apiName)` returns `{ upload, uploadMany, progress (0..1), status,
|
|
95
|
+
error, cancel, reset }`. Under the hood it calls the authenticated
|
|
96
|
+
`storage.mintUploadTicket` rpc (which runs your auth chain and returns a
|
|
97
|
+
short-lived signed URL), then POSTs the file's **raw bytes** via XHR with
|
|
98
|
+
real per-byte progress — binary, so a 50 MB video never bloats the rpc
|
|
99
|
+
payload. The stored ref row appears **live** in any `useSubscription` over that
|
|
100
|
+
table (normal reactivity). For an instant preview, pair with
|
|
101
|
+
`URL.createObjectURL(file)` and swap to `asset.url` on completion.
|
|
102
|
+
|
|
103
|
+
**Drag-drop + paste** come for free. The handle also returns `onDrop`, `onPaste`,
|
|
104
|
+
and a `dropzoneProps` you spread onto any element for a zero-boilerplate dropzone:
|
|
105
|
+
|
|
106
|
+
```tsx
|
|
107
|
+
const { dropzoneProps, onPaste, uploadMany } = useUpload('myApi')
|
|
108
|
+
|
|
109
|
+
<div {...dropzoneProps}>Drop files here</div> // drop → uploadMany
|
|
110
|
+
<textarea onPaste={(e) => onPaste(e, { folder: 'notes' })} /> // paste a screenshot
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
`onDrop(event, opts?)` / `onPaste(event, opts?)` pull the files off the event and
|
|
114
|
+
call `uploadMany` (bounded concurrency); both resolve `[]` when the event carries
|
|
115
|
+
no files. `dropzoneProps` also prevents the browser's default "navigate to the
|
|
116
|
+
dropped file" behavior.
|
|
117
|
+
|
|
118
|
+
Server-side, the same path is the `mintUploadTicket` action + the
|
|
119
|
+
`POST /_voltro/storage/upload` route (ticket-verified, no auth re-run). The
|
|
120
|
+
`storage.upload` base64 action also exists for tiny files.
|
|
121
|
+
|
|
122
|
+
### Transports — `prefer`
|
|
123
|
+
|
|
124
|
+
`through-app` (the default) works everywhere and needs nothing extra. Two opt-in
|
|
125
|
+
transports handle large media:
|
|
126
|
+
|
|
127
|
+
```tsx
|
|
128
|
+
useUpload('myApi').upload(file, { prefer: 'presign' }) // offload the upload leg
|
|
129
|
+
useUpload('myApi').upload(file, { prefer: 'resumable' }) // survive dropped connections
|
|
130
|
+
useUpload('myApi').upload(file, { prefer: 'multipart' }) // multi-GB, direct to bucket
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
- **`presign`** — on a presigning provider (s3/minio) the browser PUTs straight
|
|
134
|
+
to the bucket (offloading the client→server leg), then the server **fetches the
|
|
135
|
+
bytes back** to scan + checksum + derive and register a **non-degraded** ref
|
|
136
|
+
(`storage.mintPresignedUpload` → direct PUT → `storage.finalizeUpload`). On a
|
|
137
|
+
non-presigning provider it **transparently falls back** to through-app. A virus
|
|
138
|
+
hit fails closed (temp bytes deleted, nothing registered) — so presigned
|
|
139
|
+
uploads are scanned, unlike a raw bucket PUT.
|
|
140
|
+
- **`resumable`** — splits the file into chunks POSTed one at a time to
|
|
141
|
+
`/_voltro/storage/upload/resumable` (`storage.beginResumableUpload`); each chunk
|
|
142
|
+
is retried independently and is idempotent by index, so a dropped connection
|
|
143
|
+
resumes instead of restarting. Works on **every** provider (chunks are stored,
|
|
144
|
+
then assembled through the same scan pipeline). Tune with `chunkSize`.
|
|
145
|
+
- **`multipart`** — for **multi-GB** media: each part is PUT **directly to the
|
|
146
|
+
bucket** via a presigned URL (`storage.beginMultipartUpload` →
|
|
147
|
+
`storage.signMultipartPart` per part → `storage.completeMultipartUpload`), so
|
|
148
|
+
the bytes **never touch the app** — offloaded *and* resumable (a failed part is
|
|
149
|
+
re-signed + re-sent). s3/minio only; **falls back to `resumable`** elsewhere.
|
|
150
|
+
Needs the bucket CORS to **expose the `ETag` header** (`voltro storage cors`
|
|
151
|
+
prints the rule). **Tradeoff:** completing registers the ref straight from the
|
|
152
|
+
bucket object (size via HEAD, etag as checksum) — it does **not** fetch the
|
|
153
|
+
bytes back, so a multipart upload is **not scanned or derived** inline (that
|
|
154
|
+
would mean pulling GB through the app). Scan multi-GB out of band (a
|
|
155
|
+
bucket-event job) and note `checksum` is the S3 etag, not a sha-256.
|
|
156
|
+
|
|
157
|
+
## Media derivatives + rich metadata
|
|
158
|
+
|
|
159
|
+
`put()` (and every upload) auto-extracts, for raster images (via optional
|
|
160
|
+
`sharp`, graceful when absent):
|
|
161
|
+
|
|
162
|
+
- `width` / `height` (pixels) — the fields your app used to leave null.
|
|
163
|
+
- `placeholder` — a tiny `data:image/webp;base64,…` LQIP (drops into
|
|
164
|
+
`<img src>` / a CSS background; no decoder library).
|
|
165
|
+
|
|
166
|
+
Opt into `storagePlugin({ normalizeImages: true })` to apply EXIF orientation
|
|
167
|
+
and strip ALL metadata (GPS/camera), re-encoding — the normalized bytes are
|
|
168
|
+
what get checksummed + stored.
|
|
169
|
+
|
|
170
|
+
Uploads also carry app metadata onto the ref — `folder`, `tags`, `alt`,
|
|
171
|
+
`caption` — so the ref row is rich enough that **apps no longer need a parallel
|
|
172
|
+
`assets` table**.
|
|
173
|
+
|
|
174
|
+
For **video/audio**, wire a `videoProbe` hook — the core bundles no transcoder
|
|
175
|
+
(ffmpeg is a large binary + licensing), so you plug in ffprobe / a cloud API and
|
|
176
|
+
it runs in `put()` for video/audio content types:
|
|
177
|
+
|
|
178
|
+
```ts
|
|
179
|
+
storagePlugin({
|
|
180
|
+
videoProbe: async ({ bytes, contentType }) => {
|
|
181
|
+
const { duration, width, height, posterDataUri } = await probeWithFfprobe(bytes)
|
|
182
|
+
return { duration, width, height, poster: posterDataUri } // → ref.duration + ref.placeholder
|
|
183
|
+
},
|
|
184
|
+
})
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
Best-effort: a `null` / throw / failed probe never blocks the upload. An explicit
|
|
188
|
+
`duration` passed to `put()` wins over the probe.
|
|
189
|
+
|
|
190
|
+
## Transcoding video — renditions
|
|
191
|
+
|
|
192
|
+
Produce playable variants (720p mp4, webm, a poster…) from an uploaded video.
|
|
193
|
+
The first-party `ffmpegTranscoder` **spawns the ffmpeg binary** (no npm dep — like
|
|
194
|
+
`clamavScanner` talks to a daemon), so install ffmpeg on the host (or pass
|
|
195
|
+
`ffmpegPath`); absent it, transcoding is skipped and the upload still succeeds.
|
|
196
|
+
|
|
197
|
+
```ts
|
|
198
|
+
import { storagePlugin, ffmpegTranscoder } from '@voltro/plugin-storage'
|
|
199
|
+
|
|
200
|
+
storagePlugin({
|
|
201
|
+
transcode: ffmpegTranscoder({
|
|
202
|
+
renditions: [
|
|
203
|
+
{ kind: 'rendition', label: '720p', contentType: 'video/mp4', ext: 'mp4',
|
|
204
|
+
args: ['-vf', 'scale=-2:720', '-c:v', 'libx264', '-crf', '23', '-c:a', 'aac', '-movflags', '+faststart'] },
|
|
205
|
+
{ kind: 'poster', label: 'poster', contentType: 'image/jpeg', ext: 'jpg',
|
|
206
|
+
args: ['-frames:v', '1', '-vf', 'thumbnail'] },
|
|
207
|
+
],
|
|
208
|
+
}),
|
|
209
|
+
})
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
Each rendition is stored as a **normal storage ref linked to the original**
|
|
213
|
+
(`derivedFrom` = the parent id, `kind` = `'rendition'`/`'poster'`, `caption` =
|
|
214
|
+
the label) — so renditions get serving, grants, transforms and GC for free, and
|
|
215
|
+
inherit the original's tenant + visibility.
|
|
216
|
+
|
|
217
|
+
- **Runs out of band.** Transcoding forks in the background *after* the upload
|
|
218
|
+
returns (never blocks the client). It's ON by default once a `transcode` is
|
|
219
|
+
configured — set `autoTranscode: false` to only run it explicitly.
|
|
220
|
+
- **`storage.renditions(refId)`** — list the derivatives of an asset.
|
|
221
|
+
- Bring your own `transcode` (a cloud video API, HLS packager) by implementing
|
|
222
|
+
the `Transcoder` interface — `ffmpegTranscoder` is just one implementation.
|
|
223
|
+
|
|
224
|
+
### Durability — the auto-fork is best-effort
|
|
225
|
+
|
|
226
|
+
The background fork is **fire-and-forget**: if the process crashes or is
|
|
227
|
+
redeployed mid-transcode, that rendition is simply never produced (the original
|
|
228
|
+
upload is unaffected). For long or business-critical transcodes, **turn the
|
|
229
|
+
auto-fork off and drive `storage.transcode(refId)` from a durable
|
|
230
|
+
[workflow](/docs/workflows/overview)** — you get retries, backoff, and
|
|
231
|
+
at-least-once execution across restarts:
|
|
232
|
+
|
|
233
|
+
```ts
|
|
234
|
+
// 1. Disable the fire-and-forget fork:
|
|
235
|
+
storagePlugin({ transcode: ffmpegTranscoder({ renditions }), autoTranscode: false })
|
|
236
|
+
|
|
237
|
+
// 2. Kick a durable workflow when a video ref is created (e.g. from the upload
|
|
238
|
+
// mutation or a reactive hook), and transcode inside it — retried until it
|
|
239
|
+
// succeeds, surviving restarts:
|
|
240
|
+
export const transcodeVideo = workflow('transcodeVideo', (refId: string) =>
|
|
241
|
+
Effect.gen(function* () {
|
|
242
|
+
const storage = yield* StorageService
|
|
243
|
+
const renditions = yield* storage.transcode(refId) // retried by the workflow engine
|
|
244
|
+
return { count: renditions.length }
|
|
245
|
+
}),
|
|
246
|
+
)
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
So: convenient by default (auto-fork), durable when you need it (workflow + the
|
|
250
|
+
explicit primitive). The two coexist — same `transcode()` call underneath.
|
|
251
|
+
|
|
252
|
+
> ffmpeg **transcode** (re-encoding) is opt-in as above. On-the-fly *image*
|
|
253
|
+
> resizing stays separate (query params on the serve URL, powered by `sharp`).
|
|
254
|
+
|
|
255
|
+
## Binding an entity to an asset — `assetRef()`
|
|
256
|
+
|
|
257
|
+
```ts
|
|
258
|
+
import { table, id } from '@voltro/database'
|
|
259
|
+
import { assetRef } from '@voltro/plugin-storage'
|
|
260
|
+
|
|
261
|
+
const employees = table('employees', {
|
|
262
|
+
id: id(),
|
|
263
|
+
avatar: assetRef(), // stores a _voltro_storage_refs id
|
|
264
|
+
})
|
|
265
|
+
```
|
|
266
|
+
|
|
267
|
+
`assetRef()` is a real FK to `_voltro_storage_refs.id` (nullable, `onDelete:
|
|
268
|
+
'setNull'` — deleting the blob clears the link) by default; pass `{ fk: false }`
|
|
269
|
+
for a plain typed id column, `{ nullable: false }` to require it, or `{ onDelete:
|
|
270
|
+
'cascade' }` to change the FK semantics. Resolve the linked asset with the storage
|
|
271
|
+
service (`head` / `mintUrl`).
|
|
272
|
+
|
|
273
|
+
For the one-column shorthand there's a `withStorage()` mixin (applied via `.with`,
|
|
274
|
+
the same convention as `tenant()` / `audit()`), which adds a single `asset`
|
|
275
|
+
column:
|
|
276
|
+
|
|
277
|
+
```ts
|
|
278
|
+
import { withStorage } from '@voltro/plugin-storage'
|
|
279
|
+
|
|
280
|
+
const posts = table('posts', { id: id() }).with(withStorage()) // → posts.asset
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
Prefer `assetRef()` inline when you want a custom column name (`avatar`, `cover`)
|
|
284
|
+
or several asset columns.
|
|
285
|
+
|
|
286
|
+
## Swappable engine
|
|
287
|
+
|
|
288
|
+
The provider is the swap point; the rest of your code is provider-agnostic.
|
|
289
|
+
|
|
290
|
+
```ts
|
|
291
|
+
storagePlugin({ provider: 'memory' }) // dev / tests
|
|
292
|
+
storagePlugin({ provider: 'database' }) // bytes in your DB — zero extra infra
|
|
293
|
+
storagePlugin({ provider: 'filesystem', root: '.voltro-storage' }) // on-disk dev
|
|
294
|
+
storagePlugin({ provider: 's3', bucket: 'prod', region: 'eu-central-1',
|
|
295
|
+
cdnBaseUrl: 'https://cdn.example.com' }) // AWS S3 + CDN
|
|
296
|
+
storagePlugin({ provider: 'minio', bucket: 'prod',
|
|
297
|
+
endpoint: 'http://localhost:9000' }) // MinIO
|
|
298
|
+
storagePlugin({ provider: 'azure', bucket: 'prod',
|
|
299
|
+
accountName: 'acct', accountKey: process.env.AZURE_STORAGE_KEY }) // Azure Blob
|
|
300
|
+
// Cloudflare R2 / GCS / Backblaze B2 / Wasabi — any S3-compatible bucket via `endpoint`.
|
|
301
|
+
storagePlugin({ provider: 's3', bucket: 'prod',
|
|
302
|
+
endpoint: 'https://<acct>.r2.cloudflarestorage.com',
|
|
303
|
+
cdnBaseUrl: 'https://files.example.com', publicAcl: false })
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
Two providers stand apart from the S3-compatible family:
|
|
307
|
+
|
|
308
|
+
- **`azure`** — Azure Blob Storage (not S3-compatible). Presigned URLs are
|
|
309
|
+
SAS tokens; needs `accountName` + `accountKey` (or `AZURE_STORAGE_ACCOUNT` /
|
|
310
|
+
`AZURE_STORAGE_KEY`), or a `connectionString`. Requires the optional
|
|
311
|
+
`@azure/storage-blob` dependency.
|
|
312
|
+
- **`database`** — stores the blob bytes IN the app's database
|
|
313
|
+
(`_voltro_storage_blobs`, a `bytes()` / `BYTEA` column) via the same
|
|
314
|
+
DataStore. **Zero extra infra** — no bucket, survives restarts (unlike
|
|
315
|
+
`memory`). Served through the app query (no CDN-direct). Use for small files;
|
|
316
|
+
large blobs belong in object storage.
|
|
317
|
+
|
|
318
|
+
The **`filesystem`** provider writes blobs **atomically** — bytes go to a
|
|
319
|
+
unique temp file in the target directory, then `rename()` over the final path
|
|
320
|
+
(POSIX-atomic). A reader never sees a half-written file and two concurrent
|
|
321
|
+
writers of the same key can't interleave into a corrupt blob. That makes it
|
|
322
|
+
safe on a **shared ReadWriteMany volume** (e.g. CephFS) with multiple app
|
|
323
|
+
replicas — point `STORAGE_ROOT` at the mount. (A `ReadWriteOnce` block volume
|
|
324
|
+
mounts on one pod only, so it can't back more than one replica.)
|
|
325
|
+
|
|
326
|
+
Options resolve from env when omitted: `STORAGE_PROVIDER`, `S3_BUCKET` /
|
|
327
|
+
`STORAGE_BUCKET`, `S3_REGION` / `AWS_REGION`, `S3_ENDPOINT` / `MINIO_ENDPOINT`,
|
|
328
|
+
`S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY`, `STORAGE_ROOT`, `STORAGE_CDN_URL`,
|
|
329
|
+
`S3_FORCE_PATH_STYLE=1`. Pass a `StorageProvider` object to bring your own
|
|
330
|
+
backend. `publicAcl: false` skips per-object `public-read` ACLs (use a bucket
|
|
331
|
+
policy + `cdnBaseUrl` instead — required for R2).
|
|
332
|
+
|
|
333
|
+
## Public objects — minimal middleware
|
|
334
|
+
|
|
335
|
+
A `visibility: 'public'` object is readable with no auth. When `cdnBaseUrl`
|
|
336
|
+
is set, `getUrl(id)` returns `${cdnBaseUrl}/${key}` and the serve route 302s
|
|
337
|
+
there — reads never hit the app. WITHOUT `cdnBaseUrl` (a bare S3/MinIO bucket,
|
|
338
|
+
which is private by default — and on `filesystem` / `memory`), the framework
|
|
339
|
+
does NOT hand out a raw bucket URL (it would 403); the serve route streams the
|
|
340
|
+
bytes through the app instead, with `cache-control: public, max-age=31536000,
|
|
341
|
+
immutable`. So public objects are always cacheable; `cdnBaseUrl` is what moves
|
|
342
|
+
the read path off the app and onto the edge.
|
|
343
|
+
|
|
344
|
+
## Private objects — the access model
|
|
345
|
+
|
|
346
|
+
Access is the union of two layers; a request is allowed if **either** grants it.
|
|
347
|
+
|
|
348
|
+
### 1. Inline access policy
|
|
349
|
+
|
|
350
|
+
`put({ access })` takes an **OR-of-rules** array. A rule GRANTS when EVERY
|
|
351
|
+
condition it declares is satisfied (AND); access is allowed if ANY rule grants.
|
|
352
|
+
The owner (`subject.id === ref.ownerId`) is always allowed.
|
|
353
|
+
|
|
354
|
+
```ts
|
|
355
|
+
yield* storage.put({
|
|
356
|
+
bytes, contentType: 'application/pdf',
|
|
357
|
+
ownerId: ctx.request.subject.id,
|
|
358
|
+
password: 'optional-file-password', // hashed; gates a { password: true } rule
|
|
359
|
+
access: [
|
|
360
|
+
{ roles: ['admin'] }, // admins, OR
|
|
361
|
+
{ tenant: true }, // any same-tenant subject, OR
|
|
362
|
+
{ groups: ['finance'], scopes: ['files:read'] }, // finance AND files:read
|
|
363
|
+
],
|
|
364
|
+
})
|
|
365
|
+
```
|
|
366
|
+
|
|
367
|
+
Rule conditions: `owner`, `roles[]`, `groups[]`, `scopes[]`, `tenant`,
|
|
368
|
+
`apiKey`, `password`, `guard` (a named custom guard). Roles/groups come from
|
|
369
|
+
`subject.metadata.roles` / `.groups` by default — override the resolution:
|
|
370
|
+
|
|
371
|
+
```ts
|
|
372
|
+
storagePlugin({
|
|
373
|
+
access: {
|
|
374
|
+
resolveGroups: (subject) => lookupGroups(subject.id), // sync | Promise
|
|
375
|
+
guards: { vip: ({ subject, ref }) => subject.metadata?.tier === 'vip' },
|
|
376
|
+
},
|
|
377
|
+
})
|
|
378
|
+
```
|
|
379
|
+
|
|
380
|
+
### 2. Per-object grants
|
|
381
|
+
|
|
382
|
+
Explicitly share one object with a user / group / api-key, optionally expiring:
|
|
383
|
+
|
|
384
|
+
```ts
|
|
385
|
+
yield* storage.grant({ refId, principalType: 'user', principalId: 'user_123', permission: 'read' })
|
|
386
|
+
yield* storage.grant({ refId, principalType: 'group', principalId: 'team-a', expiresAt: in7Days })
|
|
387
|
+
yield* storage.revoke(grantId)
|
|
388
|
+
const grants = yield* storage.listGrants(refId)
|
|
389
|
+
```
|
|
390
|
+
|
|
391
|
+
Grants persist in `_voltro_storage_grants` and are managed from the dashboard's
|
|
392
|
+
**Storage** tab.
|
|
393
|
+
|
|
394
|
+
## Delivering a private file
|
|
395
|
+
|
|
396
|
+
```ts
|
|
397
|
+
// Access-checked. Returns a presigned URL (s3/minio) OR an app serve URL
|
|
398
|
+
// carrying a short-lived signed grant token (filesystem/memory). Fails with
|
|
399
|
+
// StorageAccessDenied (403) if the subject isn't allowed.
|
|
400
|
+
const url = yield* storage.mintUrl(fileId, ctx.request.subject, { password })
|
|
401
|
+
```
|
|
402
|
+
|
|
403
|
+
For large uploads, skip the wire and presign a direct-to-bucket PUT
|
|
404
|
+
(s3/minio): `storage.mintUploadUrl({ key, contentType })`.
|
|
405
|
+
|
|
406
|
+
## Typed RPC queries (client-facing)
|
|
407
|
+
|
|
408
|
+
The plugin ships queries your client can call with no extra wiring — and their
|
|
409
|
+
errors (`StorageError`, `StorageAccessDenied`) are merged into every
|
|
410
|
+
procedure's wire-error union so they decode typed on the client:
|
|
411
|
+
|
|
412
|
+
`storage.upload`, `storage.mintUploadTicket`, `storage.mintUrl`,
|
|
413
|
+
`storage.mintUploadUrl`, `storage.mintPresignedUpload`, `storage.finalizeUpload`,
|
|
414
|
+
`storage.beginResumableUpload`, `storage.beginMultipartUpload`,
|
|
415
|
+
`storage.signMultipartPart`, `storage.completeMultipartUpload`,
|
|
416
|
+
`storage.abortMultipartUpload`, `storage.ingestUrl`, `storage.share`,
|
|
417
|
+
`storage.revoke`, `storage.listGrants`, `storage.listRefs`. (The upload hook picks
|
|
418
|
+
the right ones per `prefer`; you rarely call them directly.)
|
|
419
|
+
|
|
420
|
+
### The grant RPCs are owner-only
|
|
421
|
+
|
|
422
|
+
`storage.share`, `storage.revoke` and `storage.listGrants` require that the
|
|
423
|
+
caller OWNS the object (`ref.ownerId === subject.id`), or carries `admin:full`.
|
|
424
|
+
|
|
425
|
+
They used to require nothing. Each took an object id straight off the wire and
|
|
426
|
+
handed it to a service method that performs no authorization — correct for a
|
|
427
|
+
trusted server-side API — with nothing in between. Any authenticated caller
|
|
428
|
+
could grant themselves read or write on any object in the installation, revoke
|
|
429
|
+
anyone else's grants, and enumerate who an object is shared with.
|
|
430
|
+
|
|
431
|
+
A missing object and an unowned object both report the same 403; a 404 would let
|
|
432
|
+
an unauthorized caller probe which ids exist.
|
|
433
|
+
|
|
434
|
+
If a server-side flow legitimately shares on someone else's behalf, call the
|
|
435
|
+
SERVICE (`yield* StorageService` → `.grant(...)`) rather than the RPC — the
|
|
436
|
+
service is deliberately unrestricted, and the gate belongs at the wire. For an
|
|
437
|
+
admin surface, give the subject `admin:full` (see
|
|
438
|
+
[`auth.resolveScopes`](/docs/authentication/strategies) for lifting a role into
|
|
439
|
+
scopes).
|
|
440
|
+
|
|
441
|
+
## StorageService API
|
|
442
|
+
|
|
443
|
+
| Method | Returns |
|
|
444
|
+
|---|---|
|
|
445
|
+
| `put(input)` | `StorageRef` — `{ id, tenantId, ownerId, bucket, key, contentType, size, checksum, visibility, accessPolicy, createdAt }` |
|
|
446
|
+
| `get(id, opts?)` | `{ bytes, ref }` (tenant-guarded when `opts.tenantId` set) |
|
|
447
|
+
| `getUrl(id, opts?)` | direct/CDN URL (public) or presigned/serve URL (private) |
|
|
448
|
+
| `mintUrl(id, subject, opts?)` | access-checked delivery URL |
|
|
449
|
+
| `mintUploadUrl(input)` | presigned PUT (s3/minio) |
|
|
450
|
+
| `head(id, opts?)` | `StorageRef \| null` |
|
|
451
|
+
| `getRange(id, { start, endInclusive }, opts?)` | `{ bytes, ref, totalSize }` — a byte slice (backs Range serving) |
|
|
452
|
+
| `delete(id, opts?)` | refcount-aware — dedup'd siblings survive |
|
|
453
|
+
| `listRefs(input?)` | `{ refs, nextOffset }` — browse/search the ref index |
|
|
454
|
+
| `checkAccess(ref, subject, opts?)` | `boolean` |
|
|
455
|
+
| `grant` / `revoke` / `listGrants` | manage per-object grants |
|
|
456
|
+
|
|
457
|
+
Failures are a typed `StorageError` (the `transient` flag drives retry — default
|
|
458
|
+
3 attempts) or `StorageAccessDenied` (403).
|
|
459
|
+
|
|
460
|
+
### Browse / search files — `listRefs()`
|
|
461
|
+
|
|
462
|
+
The `folder`, `tags`, and `ownerId` you set on each `put()` are queryable
|
|
463
|
+
first-class — a media library or file manager never has to touch
|
|
464
|
+
`_voltro_storage_refs` by hand. `listRefs()` filters by folder **prefix** (a path
|
|
465
|
+
descendant match), all-of `tags` (AND), and `ownerId`, newest-first, with offset
|
|
466
|
+
paging:
|
|
467
|
+
|
|
468
|
+
```ts
|
|
469
|
+
import { StorageService } from '@voltro/plugin-storage'
|
|
470
|
+
import { Effect } from 'effect'
|
|
471
|
+
|
|
472
|
+
const browse = Effect.gen(function* () {
|
|
473
|
+
const storage = yield* StorageService
|
|
474
|
+
// Page 1: everything under `photos/` (incl. `photos/2026/…`) tagged `hero`.
|
|
475
|
+
const page = yield* storage.listRefs({
|
|
476
|
+
tenantId: 'org_123',
|
|
477
|
+
folder: 'photos',
|
|
478
|
+
tags: ['hero'],
|
|
479
|
+
ownerId: 'user_42',
|
|
480
|
+
limit: 24,
|
|
481
|
+
offset: 0,
|
|
482
|
+
})
|
|
483
|
+
// page.refs: ReadonlyArray<StorageRef>, newest first.
|
|
484
|
+
// page.nextOffset: number | null — pass back as `offset` for the next page,
|
|
485
|
+
// or null when this was the last page.
|
|
486
|
+
return page
|
|
487
|
+
})
|
|
488
|
+
```
|
|
489
|
+
|
|
490
|
+
- **`folder`** matches the value exactly OR as a `/`-delimited ancestor
|
|
491
|
+
(`photos` ⇒ `photos`, `photos/2026`; never `photos-archive`). A trailing slash
|
|
492
|
+
is ignored.
|
|
493
|
+
- **`tags`** requires ALL listed tags on the ref (AND).
|
|
494
|
+
- **`ownerId`** / **`tenantId`** scope to one owner / tenant (`tenantId: null`
|
|
495
|
+
is the system/global partition; omit `tenantId` only for a trusted admin read
|
|
496
|
+
that spans tenants).
|
|
497
|
+
- Transcode **derivatives** (renditions / posters) are excluded by default; pass
|
|
498
|
+
`includeDerived: true` to include them.
|
|
499
|
+
- **Paging**: `limit` is clamped to `[1, 500]` (default 50); `nextOffset` is
|
|
500
|
+
non-null while another page follows.
|
|
501
|
+
|
|
502
|
+
`listRefs` filters the **index** — it does not run the per-object access policy
|
|
503
|
+
(it delivers no bytes). Scope it to the caller's `tenantId` / `ownerId` for an
|
|
504
|
+
end-user surface, and deliver any listed private object through `mintUrl`. The
|
|
505
|
+
`storage.listRefs` rpc query takes the tenant from the caller's subject
|
|
506
|
+
server-side, so a client can only browse its own tenant.
|
|
507
|
+
|
|
508
|
+
Reach the service with `yield* StorageService` inside an Effect handler — the
|
|
509
|
+
plugin provides it as the handler base layer. The service methods are
|
|
510
|
+
Effect-returning (`storage.put(...)` is an `Effect<StorageRef>`); from a plain
|
|
511
|
+
`async` handler, run them with `Effect.runPromise`, or write the handler in
|
|
512
|
+
Effect form.
|
|
513
|
+
|
|
514
|
+
## Serve endpoint
|
|
515
|
+
|
|
516
|
+
`GET /_voltro/storage/:id` (mounted pre-auth):
|
|
517
|
+
|
|
518
|
+
- **public** → 302 to the CDN/bucket URL when `cdnBaseUrl` is set; otherwise
|
|
519
|
+
serve the bytes with `cache-control: public, max-age=31536000, immutable`
|
|
520
|
+
(no auth either way).
|
|
521
|
+
- **private** → checks a `?t=<grant-token>` (minted by `mintUrl`) OR the session
|
|
522
|
+
cookie + access policy; then 403, a presigned 302, or a served body with
|
|
523
|
+
`cache-control: private, no-store`.
|
|
524
|
+
|
|
525
|
+
### HTTP Range — `206 Partial Content` (video seeking, bounded delivery)
|
|
526
|
+
|
|
527
|
+
When the object is served **through the app** (a non-presigning provider, or a
|
|
528
|
+
private object without a bucket presign), the serve route honours an HTTP
|
|
529
|
+
`Range:` request header so a `<video>`/`<audio>` element can seek and a client
|
|
530
|
+
can resume — only the requested slice leaves the backend (via the provider's
|
|
531
|
+
`getRange`; memory/database fall back to a buffered slice):
|
|
532
|
+
|
|
533
|
+
- A **full** `GET` responds `200` with `Accept-Ranges: bytes`.
|
|
534
|
+
- A satisfiable `Range: bytes=<start>-<end>` responds **`206 Partial Content`**
|
|
535
|
+
with `Content-Range: bytes <start>-<end>/<total>` and just that slice. Open-
|
|
536
|
+
ended (`bytes=N-`) reads to EOF; a suffix (`bytes=-N`) reads the last N bytes.
|
|
537
|
+
- A range entirely past the object → **`416 Range Not Satisfiable`** with
|
|
538
|
+
`Content-Range: bytes */<total>`.
|
|
539
|
+
- A malformed `Range` header falls back to the full `200` (per RFC 7233).
|
|
540
|
+
|
|
541
|
+
For **presigning** providers (s3/minio) a private object 302-redirects to the
|
|
542
|
+
bucket, which serves Range natively — the client re-issues its `Range` against
|
|
543
|
+
the presigned URL, so seeking works without the bytes ever passing through the
|
|
544
|
+
app. The `Range`/`Content-Range`/`Accept-Ranges` headers are added to the serve
|
|
545
|
+
route's CORS allow/expose lists so a cross-origin `fetch()` can drive them.
|
|
546
|
+
|
|
547
|
+
### Cross-origin: absolute URLs + CORS
|
|
548
|
+
|
|
549
|
+
When the api is a **different origin** than your web app (the common deploy —
|
|
550
|
+
`api.example.com` vs your web origin), the serve URL must be **absolute** or a
|
|
551
|
+
browser `<img>` / a server-to-server AI-gateway fetch can't resolve it. Set the
|
|
552
|
+
api's public base and `getUrl`/`mintUrl`/the serve route emit absolute URLs:
|
|
553
|
+
|
|
554
|
+
```
|
|
555
|
+
VOLTRO_PUBLIC_URL=https://api.example.com # or storagePlugin({ publicBaseUrl })
|
|
556
|
+
```
|
|
557
|
+
|
|
558
|
+
Without it, serve URLs are **relative** (`/_voltro/storage/:id`) — correct only
|
|
559
|
+
same-origin. For public objects a `cdnBaseUrl` / `STORAGE_CDN_URL` already yields
|
|
560
|
+
an absolute CDN URL; `VOLTRO_PUBLIC_URL` covers the no-CDN + private cases.
|
|
561
|
+
|
|
562
|
+
To let a cross-origin browser `fetch()`/canvas the bytes (a plain `<img>` needs
|
|
563
|
+
nothing), allow its origin:
|
|
564
|
+
|
|
565
|
+
```
|
|
566
|
+
STORAGE_ALLOWED_ORIGINS=https://app.example.com # comma-separated, or '*'
|
|
567
|
+
```
|
|
568
|
+
|
|
569
|
+
> Diagnostics: on boot the plugin logs `storage: provider=… presign=… publicUrl=…
|
|
570
|
+
> transforms=…`, and **warns loudly** when `provider=memory` outside tests
|
|
571
|
+
> (uploads are in-memory and lost on restart — empty/placeholder storage env
|
|
572
|
+
> silently falls back to memory) or when serve URLs would be relative on a
|
|
573
|
+
> non-presigning provider with no public base.
|
|
574
|
+
|
|
575
|
+
## Diagnostics — `voltro storage`
|
|
576
|
+
|
|
577
|
+
Probe the configured backend from the terminal so a broken setup fails LOUD
|
|
578
|
+
before an upload does:
|
|
579
|
+
|
|
580
|
+
```
|
|
581
|
+
voltro storage doctor # provider + a real read/write round-trip + config sanity
|
|
582
|
+
voltro storage cors # print the recommended bucket CORS for presigned uploads
|
|
583
|
+
```
|
|
584
|
+
|
|
585
|
+
`doctor` reads your storage env (`STORAGE_PROVIDER` / `S3_*` / `VOLTRO_PUBLIC_URL`
|
|
586
|
+
/ `STORAGE_ALLOWED_ORIGINS`), prints the resolved provider banner, does an actual
|
|
587
|
+
`put`→`get`→`delete` round-trip against it, and flags the two silent footguns —
|
|
588
|
+
`provider=memory` (data lost on restart) and relative serve URLs (cross-origin
|
|
589
|
+
breakage). Exit code is non-zero when it finds an issue, so it drops into CI.
|
|
590
|
+
|
|
591
|
+
`cors` prints an S3/R2 `CORSRules` JSON to apply on your bucket — only needed if
|
|
592
|
+
you upload via a presigned direct-to-bucket PUT (`storage.mintUploadUrl`); the
|
|
593
|
+
through-app upload route sets its own CORS from `STORAGE_ALLOWED_ORIGINS`.
|
|
594
|
+
|
|
595
|
+
**Dev restart-safety.** `memory` (the empty-env fallback) loses everything on a
|
|
596
|
+
process/pod restart. For a dev environment that survives restarts, use
|
|
597
|
+
`filesystem` with a mounted volume (`STORAGE_ROOT` → a Docker/k8s volume — a
|
|
598
|
+
`ReadWriteOnce` PVC is fine for a single dev replica) or the `database` provider
|
|
599
|
+
(bytes in your DB, zero extra infra). `voltro storage doctor` warns when you're on
|
|
600
|
+
`memory` so this never surprises you in a deployed dev cluster.
|
|
601
|
+
|
|
602
|
+
## Upload constraints
|
|
603
|
+
|
|
604
|
+
Reject bad uploads before the bytes ever reach the provider:
|
|
605
|
+
|
|
606
|
+
```ts
|
|
607
|
+
storagePlugin({
|
|
608
|
+
limits: {
|
|
609
|
+
maxBytes: 10 * 1024 * 1024, // 10 MB cap
|
|
610
|
+
allowedContentTypes: ['image/*', 'application/pdf'], // exact or `type/*`
|
|
611
|
+
sniff: true, // magic-byte check
|
|
612
|
+
},
|
|
613
|
+
})
|
|
614
|
+
```
|
|
615
|
+
|
|
616
|
+
`sniff` verifies the bytes' magic number matches the declared `contentType`, so
|
|
617
|
+
an executable can't slip in as `image/png`. A violation fails `put` with a typed
|
|
618
|
+
`StorageRejected` — `reason` is `too-large`, `content-type-not-allowed`, or
|
|
619
|
+
`content-mismatch`.
|
|
620
|
+
|
|
621
|
+
## Virus scanning
|
|
622
|
+
|
|
623
|
+
A `scan` hook runs on every `put` BEFORE the bytes are stored. The first-party
|
|
624
|
+
`clamavScanner` talks to a running [ClamAV](https://www.clamav.net) daemon over
|
|
625
|
+
its INSTREAM protocol (no npm dependency):
|
|
626
|
+
|
|
627
|
+
```ts
|
|
628
|
+
import { storagePlugin, clamavScanner } from '@voltro/plugin-storage'
|
|
629
|
+
|
|
630
|
+
storagePlugin({ scan: clamavScanner({ host: '127.0.0.1', port: 3310 }) })
|
|
631
|
+
```
|
|
632
|
+
|
|
633
|
+
A detection fails `put` with a typed `StorageScanRejected` (carrying the threat
|
|
634
|
+
name) and stores nothing. A scanner outage **fails closed** — a transient
|
|
635
|
+
`StorageError`, so a misconfigured scanner never silently waves files through.
|
|
636
|
+
Bring your own engine (VirusTotal, Cloudmersive, an ICAP gateway) by
|
|
637
|
+
implementing `StorageScanner`:
|
|
638
|
+
|
|
639
|
+
```ts
|
|
640
|
+
const myScanner: StorageScanner = async ({ bytes, contentType }) => {
|
|
641
|
+
const clean = await scanSomehow(bytes)
|
|
642
|
+
return clean ? { clean: true } : { clean: false, threat: 'detected' }
|
|
643
|
+
}
|
|
644
|
+
```
|
|
645
|
+
|
|
646
|
+
Scanning applies to every path that runs `put` — through-app uploads, the
|
|
647
|
+
resumable finalize, AND `prefer: 'presign'` (which fetches the bytes back and
|
|
648
|
+
runs `finalizeUpload`, so a presigned upload is scanned out of band and fails
|
|
649
|
+
closed on a hit). Only a **raw** `storage.mintUploadUrl` PUT that you finalize
|
|
650
|
+
yourself bypasses the scan — prefer the hook's `presign` transport, which does
|
|
651
|
+
not.
|
|
652
|
+
|
|
653
|
+
## On-the-fly image transforms
|
|
654
|
+
|
|
655
|
+
Resize / re-encode images via query params on the serve URL — powered by the
|
|
656
|
+
optional [`sharp`](https://sharp.pixelplumbing.com) dependency:
|
|
657
|
+
|
|
658
|
+
```html
|
|
659
|
+
<img src="/_voltro/storage/file_01j…?w=400&h=300&format=webp&q=80" />
|
|
660
|
+
```
|
|
661
|
+
|
|
662
|
+
Params: `w`, `h` (px), `format` (`webp` / `avif` / `jpeg` / `png`), `q` (quality
|
|
663
|
+
1–100), `fit` (`cover` / `contain` / …). Transformed variants are cached.
|
|
664
|
+
|
|
665
|
+
The fast path stays fast: **without** transform params a public object 302s
|
|
666
|
+
straight to the CDN (when `cdnBaseUrl` is set) or streams with the immutable
|
|
667
|
+
cache otherwise; **with** params the app streams + transforms (opt-in cost
|
|
668
|
+
only when you ask for it). Private objects are access-checked first, then
|
|
669
|
+
transformed. If `sharp` isn't installed the original bytes are served unchanged.
|
|
670
|
+
For CDN-direct public images at scale, a CDN-level image resizer (Cloudflare
|
|
671
|
+
Images, etc.) avoids the app round-trip entirely.
|
|
672
|
+
|
|
673
|
+
## Dashboard
|
|
674
|
+
|
|
675
|
+
The **Storage** tab (local devtools + cloud) lists every object with its
|
|
676
|
+
visibility, size and owner, shows public/private + per-tenant stats, and lets
|
|
677
|
+
you view + share + revoke per-object grants. Backed by
|
|
678
|
+
`/_voltro/inspect/plugins/storage/{refs,stats,grants,share,revoke}`.
|
|
679
|
+
|
|
680
|
+
## Lifecycle: quotas, GC, URL ingest
|
|
681
|
+
|
|
682
|
+
- **Per-tenant quotas**: `storagePlugin({ quota: { maxBytes, maxCount } })` —
|
|
683
|
+
the headroom is consumed ATOMICALLY (a compare-and-set over the
|
|
684
|
+
`_voltro_storage_usage` counter row) BEFORE bytes are stored, so concurrent
|
|
685
|
+
uploads — even across replicas — cannot overshoot the cap; a failed write
|
|
686
|
+
refunds the reservation, a delete releases it. Over-budget fails with a
|
|
687
|
+
typed `StorageRejected` (`'quota-exceeded'`). `storage.usage(tenantId)`
|
|
688
|
+
returns `{ bytes, count }` for dashboards / limits.
|
|
689
|
+
- **Orphan GC**: `storage.sweepOrphans({ limit, dryRun })` removes dangling refs
|
|
690
|
+
whose bytes are gone from the provider (a crash between store + insert, or a
|
|
691
|
+
manual blob delete). Bounded + repeatable; run it from a scheduled task.
|
|
692
|
+
- **Ingest an external URL**: `storage.ingestUrl(url, opts)` (also the
|
|
693
|
+
`storage.ingestUrl` rpc) fetches a URL server-side and stores it as an asset —
|
|
694
|
+
the one-liner for CMS migration ("adopt 21k legacy image URLs"). It fetches an
|
|
695
|
+
arbitrary URL from the server, so pass **trusted URLs only** (SSRF); front with
|
|
696
|
+
an allow-list if the URL is user-supplied.
|
|
697
|
+
|
|
698
|
+
## Schema
|
|
699
|
+
|
|
700
|
+
`_voltro_storage_refs` (id, tenantId, ownerId, bucket, key, contentType, size,
|
|
701
|
+
checksum, visibility, accessPolicy, passwordHash, createdAt, + width, height,
|
|
702
|
+
duration, placeholder, folder, tags, alt, caption, derivedFrom, kind),
|
|
703
|
+
`_voltro_storage_grants` (id, refId, principalType, principalId, permission,
|
|
704
|
+
createdAt, expiresAt, createdBy), `_voltro_storage_usage` (id, tenantId
|
|
705
|
+
`UNIQUE`, bytes, count, updatedAt — one counter row per tenant, the atomic
|
|
706
|
+
arbiter for quota enforcement), and `_voltro_storage_blobs` (id, key, data
|
|
707
|
+
`BYTEA`, contentType, size — only written by the `database` provider) are
|
|
708
|
+
framework-internal and auto-migrated on every SQL-backed app — no `*.entity.ts`
|
|
709
|
+
needed. `passwordHash` is server-only and never serialized.
|