cloudrop 0.1.1 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (50) hide show
  1. package/CHANGELOG.md +115 -0
  2. package/README.md +34 -4
  3. package/dist/adapters/drizzle.d.ts +1 -1
  4. package/dist/adapters/drizzle.js +3 -3
  5. package/dist/adapters/drizzle.js.map +1 -1
  6. package/dist/adapters/prisma.d.ts +1 -1
  7. package/dist/adapters/prisma.js +1 -1
  8. package/dist/adapters/sql.d.ts +4 -1
  9. package/dist/adapters/sql.js +8 -1
  10. package/dist/adapters/sql.js.map +1 -1
  11. package/dist/cli.js +2045 -0
  12. package/dist/{client-Dj6ON4MN.d.ts → client-CP_ajLF4.d.ts} +43 -25
  13. package/dist/{client-Y1zUEbCj.js → client-DxP3uv8V.js} +172 -44
  14. package/dist/client-DxP3uv8V.js.map +1 -0
  15. package/dist/client.d.ts +3 -3
  16. package/dist/client.js +1 -1
  17. package/dist/errors-CUxMmFAe.js.map +1 -1
  18. package/dist/index.d.ts +92 -7
  19. package/dist/index.js +432 -152
  20. package/dist/index.js.map +1 -1
  21. package/dist/next.d.ts +1 -1
  22. package/dist/node.d.ts +1 -1
  23. package/dist/react.d.ts +18 -4
  24. package/dist/react.js +23 -3
  25. package/dist/react.js.map +1 -1
  26. package/dist/{route-CspI1Ypt.js → route-C8-YjzdY.js} +3 -2
  27. package/dist/{route-CspI1Ypt.js.map → route-C8-YjzdY.js.map} +1 -1
  28. package/dist/{store-BfCJejGh.d.ts → store-B539nY3J.d.ts} +41 -3
  29. package/dist/testing.d.ts +81 -0
  30. package/dist/testing.js +1243 -0
  31. package/dist/testing.js.map +1 -0
  32. package/dist/{transactional-DnvnGsVH.js → transactional-CiNwMiDb.js} +215 -105
  33. package/dist/transactional-CiNwMiDb.js.map +1 -0
  34. package/dist/{types-BrAx3dA4.d.ts → types-CFfMw-n_.d.ts} +1 -1
  35. package/docs/README.md +30 -0
  36. package/docs/bucket-setup.md +226 -0
  37. package/docs/client.md +386 -0
  38. package/docs/command.md +129 -0
  39. package/docs/deployment.md +169 -0
  40. package/docs/errors.md +150 -0
  41. package/docs/hono.md +276 -0
  42. package/docs/react.md +508 -0
  43. package/docs/resume.md +253 -0
  44. package/docs/server.md +603 -0
  45. package/docs/store.md +804 -0
  46. package/docs/testing-your-app.md +328 -0
  47. package/docs/troubleshooting.md +176 -0
  48. package/package.json +16 -8
  49. package/dist/client-Y1zUEbCj.js.map +0 -1
  50. package/dist/transactional-DnvnGsVH.js.map +0 -1
package/docs/hono.md ADDED
@@ -0,0 +1,276 @@
1
+ # Hono with better-auth and Drizzle
2
+
3
+ This page adds cloudrop to an app built like this: a Hono API on Bun with better-auth, Drizzle on Postgres with
4
+ drizzle-kit migrations, a React SPA on another origin of the same site calling the API with cookies, background jobs
5
+ in a worker process, and shadcn on Base UI. The steps are in the order you take them. The working version of every
6
+ snippet is in [`examples/hono`](https://github.com/vcode-sh/cloudrop/blob/main/examples/hono/README.md) (the API and the worker) and
7
+ [`examples/vite-react`](https://github.com/vcode-sh/cloudrop/blob/main/examples/vite-react/README.md) (the web app), which run against RustFS and Postgres in
8
+ Docker with no credentials.
9
+
10
+ The page uses the layout of a monorepo with an API app, a web app, a worker app and shared packages (`@app/db` for the
11
+ schema and the pool, `@app/api` for oRPC). Adapt the names to yours.
12
+
13
+ ## 1. One package owns cloudrop
14
+
15
+ Give the server side a package of its own, say `@app/uploads`, that depends on `cloudrop` and exports a factory and its
16
+ type:
17
+
18
+ ```ts
19
+ // packages/uploads/src/index.ts
20
+ export function createUploads(deps: UploadsDeps) { /* step 5 */ }
21
+ export type Uploads = ReturnType<typeof createUploads>;
22
+ ```
23
+
24
+ Three places import cloudrop, each a different entry:
25
+
26
+ | Who | Imports | Why |
27
+ | --- | --- | --- |
28
+ | `@app/uploads` | `cloudrop`, `cloudrop/adapters/drizzle` | the server, its routes and the store |
29
+ | `@app/db` | `cloudrop/adapters/drizzle` | `cloudropPgTables()` in the schema (step 2) |
30
+ | the web app | `cloudrop/react`, `cloudrop/client` | the hooks and their client; the server only as `import type` |
31
+
32
+ If your repository checks which package may import which SDK, register those three. The web app must reach
33
+ `@app/uploads` only through `import type`: a value import pulls the server, the database driver and the secret's
34
+ settings into the browser bundle.
35
+
36
+ `drizzle-orm` is an optional peer dependency of cloudrop (`>=0.45.0 <1 || >=1.0.0-rc.5 <2`): it uses your app's copy,
37
+ so the store and your queries share one Drizzle.
38
+
39
+ ## 2. Declare the tables in your schema
40
+
41
+ cloudrop keeps two tables, `cloudrop_files` and `cloudrop_usage`. Declare them in a schema file of their own, beside
42
+ your other features:
43
+
44
+ ```ts
45
+ // packages/db/src/schema/cloudrop.ts
46
+ import { cloudropPgTables } from "cloudrop/adapters/drizzle";
47
+
48
+ export const { files: cloudropFiles, usage: cloudropUsage } = cloudropPgTables();
49
+ ```
50
+
51
+ Export it from the schema index (`export * from "./cloudrop"`), as you do for every feature. The time columns are
52
+ `timestamptz(3)`, millisecond precision, the same as `timestamp(name, { withTimezone: true, precision: 3 })`.
53
+
54
+ The tables are your app's: read them with your own Drizzle queries, which is how you list a user's files (step 6).
55
+ cloudrop's file API has no listing call because you already have the better one, a query you can join and page as
56
+ you like. Write to them only through cloudrop, which keeps each owner's usage in step with their files.
57
+
58
+ ## 3. Generate and apply the migration
59
+
60
+ ```sh
61
+ bun run db:generate # drizzle-kit generate: a migration with both tables, their indexes and CHECKs
62
+ bun run db:migrate # drizzle-kit migrate, as an explicit step
63
+ ```
64
+
65
+ Until the tables exist, every upload fails with a retryable `SERVER_ERROR` and the server logs which table is missing.
66
+ The store needs no session state, so it works through PgBouncer in transaction mode: its one setting, a 2 s
67
+ `lock_timeout` while it claims a file, is set with `set_config(..., true)`, local to the transaction.
68
+
69
+ ## 4. Mount the handler beside better-auth, outside your CORS middleware
70
+
71
+ ```ts
72
+ // apps/server/src/app.ts
73
+ const UPLOADS_PATH = "/api/cloudrop";
74
+
75
+ // X-CSRF-Token is oRPC's CSRF header (step 6).
76
+ const browserCors = cors({ origin: webOrigins, credentials: true, allowHeaders: ["Content-Type", "X-CSRF-Token"] });
77
+ app.use((c, next) => (c.req.path.startsWith(`${UPLOADS_PATH}/`) ? next() : browserCors(c, next)));
78
+
79
+ app.on(["GET", "POST"], "/api/auth/*", (c) => deps.handleAuth(c.req.raw));
80
+ app.on(["POST", "OPTIONS"], `${UPLOADS_PATH}/*`, (c) => deps.uploads.handler(c.req.raw));
81
+ ```
82
+
83
+ - The handler serves `POST` (`/upload`, `/sign`, `/complete`, `/abort`) and `OPTIONS`, their preflight. Any other
84
+ method gets Hono's 404; with `app.all` it gets cloudrop's `METHOD_NOT_ALLOWED`.
85
+ - **Skip cloudrop's path in a global `hono/cors`.** This is the trap. `hono/cors` answers every `OPTIONS` itself and
86
+ never calls the route, so cloudrop's own preflight, which checks the origin against its `trustedOrigins` and allows
87
+ the headers the page asks for, never runs. Uploads with cookies still pass while the app's list happens to allow
88
+ `Content-Type`; a client that sends an `Authorization` header ([bearer token](client.md#send-a-bearer-token)), or an
89
+ app list without `POST`, fails in the browser's preflight with a CORS error that names no cloudrop code. cloudrop sets
90
+ the CORS headers on its other answers too, so it needs nothing from the middleware. The browser lane proves the
91
+ preflight reaches cloudrop through the real app.
92
+ - `basePath` stays `/api/cloudrop`, the default, so the server needs no option for it.
93
+
94
+ ## 5. Authorize with the session
95
+
96
+ `authorize` reads the session the same way the API does, from the request's headers, and returns the user's id. With a
97
+ store, `owner` says whose quota a file counts against:
98
+
99
+ ```ts
100
+ // packages/uploads/src/index.ts
101
+ export function createUploads(deps: UploadsDeps) {
102
+ const store = drizzleStore(deps.db, { dialect: "postgres", transaction: true });
103
+ const signedIn = async ({ request }: { request: Request }) => {
104
+ const session = await deps.readSession(request.headers); // auth.api.getSession({ headers })
105
+ return session ? { userId: session.user.id } : unauthorized("Sign in to upload files.");
106
+ };
107
+ const uploads = createCloudrop({
108
+ storage: deps.storage,
109
+ store,
110
+ quota: () => deps.quotaBytes,
111
+ trustedOrigins: deps.webOrigins, // the web app's exact origins, as in better-auth's trustedOrigins
112
+ routes: { avatar, attachments }, // step 6
113
+ });
114
+ return uploads;
115
+ }
116
+ ```
117
+
118
+ The composition root builds it once: `createUploads({ db, storage: r2(), readSession: (headers) =>
119
+ auth.api.getSession({ headers }), webOrigins, quotaBytes })`. `trustedOrigins` is required because the web app is on
120
+ another origin: without it every call gets `ORIGIN_NOT_ALLOWED`. The cookie needs nothing special: the web app and the
121
+ API are the same site, so a host-only `SameSite=Lax` cookie on the API's host travels with `credentials: "include"`
122
+ ([Deployment](deployment.md#cross-origin-an-api-on-another-origin)).
123
+
124
+ ## 6. Define the routes
125
+
126
+ ```ts
127
+ avatar: store.route(
128
+ // replace: one avatar per user; the next completed avatar deletes the previous one.
129
+ { accept: "image/*", maxSize: "2MB", replace: true, authorize: signedIn },
130
+ {
131
+ prefix: ({ userId }) => `users/${userId}/avatar`,
132
+ owner: ({ userId }) => userId,
133
+ onComplete: async ({ tx, userId, file }) => {
134
+ const row = { key: file.key, fileId: file.id, updatedAt: new Date() };
135
+ await tx.insert(avatar).values({ userId, ...row }).onConflictDoUpdate({ target: avatar.userId, set: row });
136
+ },
137
+ },
138
+ ),
139
+ attachments: store.route(
140
+ { accept: "*/*", maxSize: "100MB", maxFiles: 10, authorize: signedIn },
141
+ { prefix: ({ userId }) => `users/${userId}/attachments`, owner: ({ userId }) => userId },
142
+ ),
143
+ ```
144
+
145
+ - `store.route` gives `onComplete` Drizzle's transaction as `tx`: the avatar row commits with cloudrop's record of the
146
+ upload, or neither does. Keep network calls out of it. `replace: true` deletes the previous avatar's row in that same
147
+ commit, and its object right after ([One file per owner](store.md#one-file-per-owner-replace)).
148
+ - Files above 64 MiB go in parts (`multipartThreshold` changes it; the example uses 8 MB so a demo file shows parts).
149
+ - To list a user's files, call `uploads.api.files({ owner: userId })`, a page at a time, or select from
150
+ `cloudropFiles` with your app's Drizzle where `owner` is the user and `status` is `completed`: the tables are yours.
151
+ To delete one, check its owner with `uploads.api.file(id)`, then call `uploads.api.deleteFile(id)`, which removes the
152
+ object and frees its bytes in the quota. The example's oRPC router `files` (`list`, `delete`, `avatar`) does both.
153
+ - Protect the RPC router against cross-site requests. oRPC accepts a `multipart/form-data` body, which a page on
154
+ another origin of the same site can post without a preflight, and the browser sends the `SameSite=Lax` session
155
+ cookie with it. Add oRPC's pair: `SimpleCsrfProtectionHandlerPlugin` from `@orpc/server/plugins` on the
156
+ `RPCHandler`, `SimpleCsrfProtectionLinkPlugin` from `@orpc/client/plugins` on the `RPCLink`, and allow its
157
+ `X-CSRF-Token` header in your CORS middleware. cloudrop's own endpoints need nothing: they require
158
+ `application/json` and check `Origin`.
159
+
160
+ ## 7. The web client
161
+
162
+ ```ts
163
+ // apps/web/src/lib/upload.ts
164
+ import { createCloudropClient } from "cloudrop/client";
165
+ import { createUploadHooks } from "cloudrop/react";
166
+ import { createContext, useContext } from "react";
167
+ import type { Uploads as Server } from "@app/uploads"; // types only
168
+
169
+ function createUploads(userId: string) {
170
+ const client = createCloudropClient<Server>({ basePath: `${API_ORIGIN}/api/cloudrop`, namespace: userId });
171
+ return { client, ...createUploadHooks<Server>({ client }) };
172
+ }
173
+ export type Uploads = ReturnType<typeof createUploads>;
174
+
175
+ const byUser = new Map<string, Uploads>();
176
+ export function uploadsFor(userId: string): Uploads {
177
+ const known = byUser.get(userId) ?? createUploads(userId);
178
+ byUser.set(userId, known);
179
+ return known;
180
+ }
181
+ export const UploadsContext = createContext(uploadsFor(""));
182
+ export const useUploads = () => useContext(UploadsContext);
183
+
184
+ // apps/web/src/app.tsx: keyed by the user, so a new user's components mount with that user's hooks.
185
+ const userId = authClient.useSession().data?.user.id ?? "";
186
+ useCloseOtherUsers(userId); // from Signed-in users in the React guide
187
+ // React 19 renders a context as its provider; on React 18 write <UploadsContext.Provider>.
188
+ <UploadsContext key={userId} value={uploadsFor(userId)}>
189
+ {/* the app */}
190
+ </UploadsContext>;
191
+
192
+ // In a component:
193
+ const { useUpload } = useUploads();
194
+ const { files, getInputProps } = useUpload("attachments", { multiple: true });
195
+ ```
196
+
197
+ - `basePath` is absolute: the handler is on the API's origin, not the page's.
198
+ - The client sends cookies with every call to it; the oRPC link and the better-auth client need
199
+ `credentials: "include"` of their own.
200
+ - **One client per user, with the user's id as `namespace`** ([Resume](resume.md#namespace-and-sign-out)). A client
201
+ lists and resumes only its namespace's records, so the next user of the browser never sees the last one's
202
+ unfinished uploads, even when that user closed the tab or let the session expire without signing out, and the first
203
+ user still finds them on their next sign-in. The namespace is fixed when the client is created, which is why the app
204
+ keys the context by the user: one module-level client, without a namespace, would show every user's records to
205
+ whoever signs in next. The cost is a context and a component remount when the user changes.
206
+ - At sign-out, await `useUploads().close({ abort: true })`, which cancels the user's uploads on every route and tells
207
+ the server, so their rows end `aborted`, then `useUploads().client.forget()`: the namespace hides the records,
208
+ `forget()` deletes them and the upload tokens they hold, and aborts none. Closing first keeps a file that would get
209
+ its ticket afterwards from asking for it. `useCloseOtherUsers` in [Signed-in users](react.md#signed-in-users) closes
210
+ the last user's client when the user changes without a sign-out on this page, keeping its unfinished uploads for
211
+ that user's next sign-in; the example's `src/lib/upload.ts` has it, and `src/components/auth-panel.tsx` the
212
+ sign-out.
213
+ - To drop one unfinished upload, call `useResumable().discard(upload)`: the server aborts it, which with the store
214
+ ends its row and frees the bytes it reserved, and the browser deletes its record
215
+ ([Discard one upload](resume.md#discard-one-upload)). The example's panel has Discard per row and Discard all, each
216
+ through `discard`.
217
+ - The hooks: [React](react.md). The example's drop zone (`getDropProps`, `getInputProps`, `isDragOver`), upload list,
218
+ unfinished-uploads panel and avatar picker are in `examples/vite-react/src/components`.
219
+
220
+ ## 8. The sweep, as a scheduled job
221
+
222
+ Abandoned uploads keep their reservation in the quota and may leave parts in the bucket until a sweep ends them
223
+ ([The sweep](store.md#the-sweep)). Run it from the worker as a scheduled job, not in the API:
224
+
225
+ ```ts
226
+ // The worker process, with BullMQ: a job scheduler adds the sweep job every hour. Any queue with cron schedules
227
+ // works the same way.
228
+ import { Queue, Worker } from "bullmq";
229
+
230
+ const connection = { host: "127.0.0.1", port: 6379 };
231
+ await new Queue("upload-sweep", { connection }).upsertJobScheduler(
232
+ "hourly",
233
+ { pattern: "41 * * * *", tz: "UTC" },
234
+ // BullMQ tries a job once unless told otherwise: here a failed sweep is retried after one minute, then after two.
235
+ { name: "sweep", opts: { attempts: 3, backoff: { type: "exponential", delay: 60_000 } } },
236
+ );
237
+ new Worker(
238
+ "upload-sweep",
239
+ async () => {
240
+ // Every page in one call; the time limit leaves the rest to the next run.
241
+ const { data, error } = await uploads.api.sweep({ all: true, signal: AbortSignal.timeout(10 * 60_000) });
242
+ if (error) throw error; // the queue retries the job
243
+ for (const stuck of data.reported) log.warn({ action: "upload_completion_stuck", id: stuck.id });
244
+ },
245
+ { connection },
246
+ );
247
+ ```
248
+
249
+ The worker builds its own `createUploads` with the same storage and database; it serves no requests, so its
250
+ `readSession` can answer `null`. Two sweeps at once are safe. `examples/hono/src/worker.ts` runs the same sweep on a
251
+ timer. An account deletion runs `uploads.api.eraseOwner(userId)` from the same job until it reports `done`
252
+ ([Erase an owner](store.md#erase-an-owner)).
253
+
254
+ ## 9. The bucket's CORS rule names the web origin
255
+
256
+ The browser sends the bytes from the page, so the bucket's CORS rule lists the web app's origins, never the API's, with
257
+ `PUT`, the four headers cloudrop signs (`content-type`, `content-disposition`, `if-none-match`,
258
+ `x-amz-meta-cloudrop-id`) and `ETag` and `Retry-After` exposed. Add a lifecycle rule that aborts incomplete multipart
259
+ uploads. Both, for R2 and S3: [Bucket setup](bucket-setup.md).
260
+
261
+ ## 10. Map uploads onto your Attachment component
262
+
263
+ shadcn's `Attachment` (in its registry) has five states. A file from `useUpload` maps onto them by
264
+ its `status`:
265
+
266
+ | cloudrop `status` | `Attachment` state | Second line |
267
+ | --- | --- | --- |
268
+ | `queued`, `paused` | `idle` | "Waiting", "Paused" |
269
+ | `uploading` | `uploading` | percent of the size, with a progress bar |
270
+ | `completing` | `processing` | the server verifies the object and runs `onComplete` |
271
+ | `done` | `done` | the size |
272
+ | `failed` | `error` | `file.error.code` and `file.error.message`, with Retry |
273
+ | `aborted` | `idle` | "Cancelled" |
274
+
275
+ `examples/vite-react/src/components/upload-row.tsx` is that mapping, with Cancel while the file runs and Retry once it
276
+ failed.