cloudrop 0.1.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +115 -0
- package/README.md +34 -4
- package/dist/adapters/drizzle.d.ts +1 -1
- package/dist/adapters/drizzle.js +3 -3
- package/dist/adapters/drizzle.js.map +1 -1
- package/dist/adapters/prisma.d.ts +1 -1
- package/dist/adapters/prisma.js +1 -1
- package/dist/adapters/sql.d.ts +4 -1
- package/dist/adapters/sql.js +8 -1
- package/dist/adapters/sql.js.map +1 -1
- package/dist/cli.js +2045 -0
- package/dist/{client-Dj6ON4MN.d.ts → client-CP_ajLF4.d.ts} +43 -25
- package/dist/{client-Y1zUEbCj.js → client-DxP3uv8V.js} +172 -44
- package/dist/client-DxP3uv8V.js.map +1 -0
- package/dist/client.d.ts +3 -3
- package/dist/client.js +1 -1
- package/dist/errors-CUxMmFAe.js.map +1 -1
- package/dist/index.d.ts +92 -7
- package/dist/index.js +432 -152
- package/dist/index.js.map +1 -1
- package/dist/next.d.ts +1 -1
- package/dist/node.d.ts +1 -1
- package/dist/react.d.ts +18 -4
- package/dist/react.js +23 -3
- package/dist/react.js.map +1 -1
- package/dist/{route-CspI1Ypt.js → route-C8-YjzdY.js} +3 -2
- package/dist/{route-CspI1Ypt.js.map → route-C8-YjzdY.js.map} +1 -1
- package/dist/{store-BfCJejGh.d.ts → store-B539nY3J.d.ts} +41 -3
- package/dist/testing.d.ts +81 -0
- package/dist/testing.js +1243 -0
- package/dist/testing.js.map +1 -0
- package/dist/{transactional-DnvnGsVH.js → transactional-CiNwMiDb.js} +215 -105
- package/dist/transactional-CiNwMiDb.js.map +1 -0
- package/dist/{types-BrAx3dA4.d.ts → types-CFfMw-n_.d.ts} +1 -1
- package/docs/README.md +30 -0
- package/docs/bucket-setup.md +226 -0
- package/docs/client.md +386 -0
- package/docs/command.md +129 -0
- package/docs/deployment.md +169 -0
- package/docs/errors.md +150 -0
- package/docs/hono.md +276 -0
- package/docs/react.md +508 -0
- package/docs/resume.md +253 -0
- package/docs/server.md +603 -0
- package/docs/store.md +804 -0
- package/docs/testing-your-app.md +328 -0
- package/docs/troubleshooting.md +176 -0
- package/package.json +16 -8
- package/dist/client-Y1zUEbCj.js.map +0 -1
- package/dist/transactional-DnvnGsVH.js.map +0 -1
package/docs/hono.md
ADDED
|
@@ -0,0 +1,276 @@
|
|
|
1
|
+
# Hono with better-auth and Drizzle
|
|
2
|
+
|
|
3
|
+
This page adds cloudrop to an app built like this: a Hono API on Bun with better-auth, Drizzle on Postgres with
|
|
4
|
+
drizzle-kit migrations, a React SPA on another origin of the same site calling the API with cookies, background jobs
|
|
5
|
+
in a worker process, and shadcn on Base UI. The steps are in the order you take them. The working version of every
|
|
6
|
+
snippet is in [`examples/hono`](https://github.com/vcode-sh/cloudrop/blob/main/examples/hono/README.md) (the API and the worker) and
|
|
7
|
+
[`examples/vite-react`](https://github.com/vcode-sh/cloudrop/blob/main/examples/vite-react/README.md) (the web app), which run against RustFS and Postgres in
|
|
8
|
+
Docker with no credentials.
|
|
9
|
+
|
|
10
|
+
The page uses the layout of a monorepo with an API app, a web app, a worker app and shared packages (`@app/db` for the
|
|
11
|
+
schema and the pool, `@app/api` for oRPC). Adapt the names to yours.
|
|
12
|
+
|
|
13
|
+
## 1. One package owns cloudrop
|
|
14
|
+
|
|
15
|
+
Give the server side a package of its own, say `@app/uploads`, that depends on `cloudrop` and exports a factory and its
|
|
16
|
+
type:
|
|
17
|
+
|
|
18
|
+
```ts
|
|
19
|
+
// packages/uploads/src/index.ts
|
|
20
|
+
export function createUploads(deps: UploadsDeps) { /* step 5 */ }
|
|
21
|
+
export type Uploads = ReturnType<typeof createUploads>;
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Three places import cloudrop, each a different entry:
|
|
25
|
+
|
|
26
|
+
| Who | Imports | Why |
|
|
27
|
+
| --- | --- | --- |
|
|
28
|
+
| `@app/uploads` | `cloudrop`, `cloudrop/adapters/drizzle` | the server, its routes and the store |
|
|
29
|
+
| `@app/db` | `cloudrop/adapters/drizzle` | `cloudropPgTables()` in the schema (step 2) |
|
|
30
|
+
| the web app | `cloudrop/react`, `cloudrop/client` | the hooks and their client; the server only as `import type` |
|
|
31
|
+
|
|
32
|
+
If your repository checks which package may import which SDK, register those three. The web app must reach
|
|
33
|
+
`@app/uploads` only through `import type`: a value import pulls the server, the database driver and the secret's
|
|
34
|
+
settings into the browser bundle.
|
|
35
|
+
|
|
36
|
+
`drizzle-orm` is an optional peer dependency of cloudrop (`>=0.45.0 <1 || >=1.0.0-rc.5 <2`): it uses your app's copy,
|
|
37
|
+
so the store and your queries share one Drizzle.
|
|
38
|
+
|
|
39
|
+
## 2. Declare the tables in your schema
|
|
40
|
+
|
|
41
|
+
cloudrop keeps two tables, `cloudrop_files` and `cloudrop_usage`. Declare them in a schema file of their own, beside
|
|
42
|
+
your other features:
|
|
43
|
+
|
|
44
|
+
```ts
|
|
45
|
+
// packages/db/src/schema/cloudrop.ts
|
|
46
|
+
import { cloudropPgTables } from "cloudrop/adapters/drizzle";
|
|
47
|
+
|
|
48
|
+
export const { files: cloudropFiles, usage: cloudropUsage } = cloudropPgTables();
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Export it from the schema index (`export * from "./cloudrop"`), as you do for every feature. The time columns are
|
|
52
|
+
`timestamptz(3)`, millisecond precision, the same as `timestamp(name, { withTimezone: true, precision: 3 })`.
|
|
53
|
+
|
|
54
|
+
The tables are your app's: read them with your own Drizzle queries, which is how you list a user's files (step 6).
|
|
55
|
+
cloudrop's file API has no listing call because you already have the better one, a query you can join and page as
|
|
56
|
+
you like. Write to them only through cloudrop, which keeps each owner's usage in step with their files.
|
|
57
|
+
|
|
58
|
+
## 3. Generate and apply the migration
|
|
59
|
+
|
|
60
|
+
```sh
|
|
61
|
+
bun run db:generate # drizzle-kit generate: a migration with both tables, their indexes and CHECKs
|
|
62
|
+
bun run db:migrate # drizzle-kit migrate, as an explicit step
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Until the tables exist, every upload fails with a retryable `SERVER_ERROR` and the server logs which table is missing.
|
|
66
|
+
The store needs no session state, so it works through PgBouncer in transaction mode: its one setting, a 2 s
|
|
67
|
+
`lock_timeout` while it claims a file, is set with `set_config(..., true)`, local to the transaction.
|
|
68
|
+
|
|
69
|
+
## 4. Mount the handler beside better-auth, outside your CORS middleware
|
|
70
|
+
|
|
71
|
+
```ts
|
|
72
|
+
// apps/server/src/app.ts
|
|
73
|
+
const UPLOADS_PATH = "/api/cloudrop";
|
|
74
|
+
|
|
75
|
+
// X-CSRF-Token is oRPC's CSRF header (step 6).
|
|
76
|
+
const browserCors = cors({ origin: webOrigins, credentials: true, allowHeaders: ["Content-Type", "X-CSRF-Token"] });
|
|
77
|
+
app.use((c, next) => (c.req.path.startsWith(`${UPLOADS_PATH}/`) ? next() : browserCors(c, next)));
|
|
78
|
+
|
|
79
|
+
app.on(["GET", "POST"], "/api/auth/*", (c) => deps.handleAuth(c.req.raw));
|
|
80
|
+
app.on(["POST", "OPTIONS"], `${UPLOADS_PATH}/*`, (c) => deps.uploads.handler(c.req.raw));
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
- The handler serves `POST` (`/upload`, `/sign`, `/complete`, `/abort`) and `OPTIONS`, their preflight. Any other
|
|
84
|
+
method gets Hono's 404; with `app.all` it gets cloudrop's `METHOD_NOT_ALLOWED`.
|
|
85
|
+
- **Skip cloudrop's path in a global `hono/cors`.** This is the trap. `hono/cors` answers every `OPTIONS` itself and
|
|
86
|
+
never calls the route, so cloudrop's own preflight, which checks the origin against its `trustedOrigins` and allows
|
|
87
|
+
the headers the page asks for, never runs. Uploads with cookies still pass while the app's list happens to allow
|
|
88
|
+
`Content-Type`; a client that sends an `Authorization` header ([bearer token](client.md#send-a-bearer-token)), or an
|
|
89
|
+
app list without `POST`, fails in the browser's preflight with a CORS error that names no cloudrop code. cloudrop sets
|
|
90
|
+
the CORS headers on its other answers too, so it needs nothing from the middleware. The browser lane proves the
|
|
91
|
+
preflight reaches cloudrop through the real app.
|
|
92
|
+
- `basePath` stays `/api/cloudrop`, the default, so the server needs no option for it.
|
|
93
|
+
|
|
94
|
+
## 5. Authorize with the session
|
|
95
|
+
|
|
96
|
+
`authorize` reads the session the same way the API does, from the request's headers, and returns the user's id. With a
|
|
97
|
+
store, `owner` says whose quota a file counts against:
|
|
98
|
+
|
|
99
|
+
```ts
|
|
100
|
+
// packages/uploads/src/index.ts
|
|
101
|
+
export function createUploads(deps: UploadsDeps) {
|
|
102
|
+
const store = drizzleStore(deps.db, { dialect: "postgres", transaction: true });
|
|
103
|
+
const signedIn = async ({ request }: { request: Request }) => {
|
|
104
|
+
const session = await deps.readSession(request.headers); // auth.api.getSession({ headers })
|
|
105
|
+
return session ? { userId: session.user.id } : unauthorized("Sign in to upload files.");
|
|
106
|
+
};
|
|
107
|
+
const uploads = createCloudrop({
|
|
108
|
+
storage: deps.storage,
|
|
109
|
+
store,
|
|
110
|
+
quota: () => deps.quotaBytes,
|
|
111
|
+
trustedOrigins: deps.webOrigins, // the web app's exact origins, as in better-auth's trustedOrigins
|
|
112
|
+
routes: { avatar, attachments }, // step 6
|
|
113
|
+
});
|
|
114
|
+
return uploads;
|
|
115
|
+
}
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
The composition root builds it once: `createUploads({ db, storage: r2(), readSession: (headers) =>
|
|
119
|
+
auth.api.getSession({ headers }), webOrigins, quotaBytes })`. `trustedOrigins` is required because the web app is on
|
|
120
|
+
another origin: without it every call gets `ORIGIN_NOT_ALLOWED`. The cookie needs nothing special: the web app and the
|
|
121
|
+
API are the same site, so a host-only `SameSite=Lax` cookie on the API's host travels with `credentials: "include"`
|
|
122
|
+
([Deployment](deployment.md#cross-origin-an-api-on-another-origin)).
|
|
123
|
+
|
|
124
|
+
## 6. Define the routes
|
|
125
|
+
|
|
126
|
+
```ts
|
|
127
|
+
avatar: store.route(
|
|
128
|
+
// replace: one avatar per user; the next completed avatar deletes the previous one.
|
|
129
|
+
{ accept: "image/*", maxSize: "2MB", replace: true, authorize: signedIn },
|
|
130
|
+
{
|
|
131
|
+
prefix: ({ userId }) => `users/${userId}/avatar`,
|
|
132
|
+
owner: ({ userId }) => userId,
|
|
133
|
+
onComplete: async ({ tx, userId, file }) => {
|
|
134
|
+
const row = { key: file.key, fileId: file.id, updatedAt: new Date() };
|
|
135
|
+
await tx.insert(avatar).values({ userId, ...row }).onConflictDoUpdate({ target: avatar.userId, set: row });
|
|
136
|
+
},
|
|
137
|
+
},
|
|
138
|
+
),
|
|
139
|
+
attachments: store.route(
|
|
140
|
+
{ accept: "*/*", maxSize: "100MB", maxFiles: 10, authorize: signedIn },
|
|
141
|
+
{ prefix: ({ userId }) => `users/${userId}/attachments`, owner: ({ userId }) => userId },
|
|
142
|
+
),
|
|
143
|
+
```
|
|
144
|
+
|
|
145
|
+
- `store.route` gives `onComplete` Drizzle's transaction as `tx`: the avatar row commits with cloudrop's record of the
|
|
146
|
+
upload, or neither does. Keep network calls out of it. `replace: true` deletes the previous avatar's row in that same
|
|
147
|
+
commit, and its object right after ([One file per owner](store.md#one-file-per-owner-replace)).
|
|
148
|
+
- Files above 64 MiB go in parts (`multipartThreshold` changes it; the example uses 8 MB so a demo file shows parts).
|
|
149
|
+
- To list a user's files, call `uploads.api.files({ owner: userId })`, a page at a time, or select from
|
|
150
|
+
`cloudropFiles` with your app's Drizzle where `owner` is the user and `status` is `completed`: the tables are yours.
|
|
151
|
+
To delete one, check its owner with `uploads.api.file(id)`, then call `uploads.api.deleteFile(id)`, which removes the
|
|
152
|
+
object and frees its bytes in the quota. The example's oRPC router `files` (`list`, `delete`, `avatar`) does both.
|
|
153
|
+
- Protect the RPC router against cross-site requests. oRPC accepts a `multipart/form-data` body, which a page on
|
|
154
|
+
another origin of the same site can post without a preflight, and the browser sends the `SameSite=Lax` session
|
|
155
|
+
cookie with it. Add oRPC's pair: `SimpleCsrfProtectionHandlerPlugin` from `@orpc/server/plugins` on the
|
|
156
|
+
`RPCHandler`, `SimpleCsrfProtectionLinkPlugin` from `@orpc/client/plugins` on the `RPCLink`, and allow its
|
|
157
|
+
`X-CSRF-Token` header in your CORS middleware. cloudrop's own endpoints need nothing: they require
|
|
158
|
+
`application/json` and check `Origin`.
|
|
159
|
+
|
|
160
|
+
## 7. The web client
|
|
161
|
+
|
|
162
|
+
```ts
|
|
163
|
+
// apps/web/src/lib/upload.ts
|
|
164
|
+
import { createCloudropClient } from "cloudrop/client";
|
|
165
|
+
import { createUploadHooks } from "cloudrop/react";
|
|
166
|
+
import { createContext, useContext } from "react";
|
|
167
|
+
import type { Uploads as Server } from "@app/uploads"; // types only
|
|
168
|
+
|
|
169
|
+
function createUploads(userId: string) {
|
|
170
|
+
const client = createCloudropClient<Server>({ basePath: `${API_ORIGIN}/api/cloudrop`, namespace: userId });
|
|
171
|
+
return { client, ...createUploadHooks<Server>({ client }) };
|
|
172
|
+
}
|
|
173
|
+
export type Uploads = ReturnType<typeof createUploads>;
|
|
174
|
+
|
|
175
|
+
const byUser = new Map<string, Uploads>();
|
|
176
|
+
export function uploadsFor(userId: string): Uploads {
|
|
177
|
+
const known = byUser.get(userId) ?? createUploads(userId);
|
|
178
|
+
byUser.set(userId, known);
|
|
179
|
+
return known;
|
|
180
|
+
}
|
|
181
|
+
export const UploadsContext = createContext(uploadsFor(""));
|
|
182
|
+
export const useUploads = () => useContext(UploadsContext);
|
|
183
|
+
|
|
184
|
+
// apps/web/src/app.tsx: keyed by the user, so a new user's components mount with that user's hooks.
|
|
185
|
+
const userId = authClient.useSession().data?.user.id ?? "";
|
|
186
|
+
useCloseOtherUsers(userId); // from Signed-in users in the React guide
|
|
187
|
+
// React 19 renders a context as its provider; on React 18 write <UploadsContext.Provider>.
|
|
188
|
+
<UploadsContext key={userId} value={uploadsFor(userId)}>
|
|
189
|
+
{/* the app */}
|
|
190
|
+
</UploadsContext>;
|
|
191
|
+
|
|
192
|
+
// In a component:
|
|
193
|
+
const { useUpload } = useUploads();
|
|
194
|
+
const { files, getInputProps } = useUpload("attachments", { multiple: true });
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
- `basePath` is absolute: the handler is on the API's origin, not the page's.
|
|
198
|
+
- The client sends cookies with every call to it; the oRPC link and the better-auth client need
|
|
199
|
+
`credentials: "include"` of their own.
|
|
200
|
+
- **One client per user, with the user's id as `namespace`** ([Resume](resume.md#namespace-and-sign-out)). A client
|
|
201
|
+
lists and resumes only its namespace's records, so the next user of the browser never sees the last one's
|
|
202
|
+
unfinished uploads, even when that user closed the tab or let the session expire without signing out, and the first
|
|
203
|
+
user still finds them on their next sign-in. The namespace is fixed when the client is created, which is why the app
|
|
204
|
+
keys the context by the user: one module-level client, without a namespace, would show every user's records to
|
|
205
|
+
whoever signs in next. The cost is a context and a component remount when the user changes.
|
|
206
|
+
- At sign-out, await `useUploads().close({ abort: true })`, which cancels the user's uploads on every route and tells
|
|
207
|
+
the server, so their rows end `aborted`, then `useUploads().client.forget()`: the namespace hides the records,
|
|
208
|
+
`forget()` deletes them and the upload tokens they hold, and aborts none. Closing first keeps a file that would get
|
|
209
|
+
its ticket afterwards from asking for it. `useCloseOtherUsers` in [Signed-in users](react.md#signed-in-users) closes
|
|
210
|
+
the last user's client when the user changes without a sign-out on this page, keeping its unfinished uploads for
|
|
211
|
+
that user's next sign-in; the example's `src/lib/upload.ts` has it, and `src/components/auth-panel.tsx` the
|
|
212
|
+
sign-out.
|
|
213
|
+
- To drop one unfinished upload, call `useResumable().discard(upload)`: the server aborts it, which with the store
|
|
214
|
+
ends its row and frees the bytes it reserved, and the browser deletes its record
|
|
215
|
+
([Discard one upload](resume.md#discard-one-upload)). The example's panel has Discard per row and Discard all, each
|
|
216
|
+
through `discard`.
|
|
217
|
+
- The hooks: [React](react.md). The example's drop zone (`getDropProps`, `getInputProps`, `isDragOver`), upload list,
|
|
218
|
+
unfinished-uploads panel and avatar picker are in `examples/vite-react/src/components`.
|
|
219
|
+
|
|
220
|
+
## 8. The sweep, as a scheduled job
|
|
221
|
+
|
|
222
|
+
Abandoned uploads keep their reservation in the quota and may leave parts in the bucket until a sweep ends them
|
|
223
|
+
([The sweep](store.md#the-sweep)). Run it from the worker as a scheduled job, not in the API:
|
|
224
|
+
|
|
225
|
+
```ts
|
|
226
|
+
// The worker process, with BullMQ: a job scheduler adds the sweep job every hour. Any queue with cron schedules
|
|
227
|
+
// works the same way.
|
|
228
|
+
import { Queue, Worker } from "bullmq";
|
|
229
|
+
|
|
230
|
+
const connection = { host: "127.0.0.1", port: 6379 };
|
|
231
|
+
await new Queue("upload-sweep", { connection }).upsertJobScheduler(
|
|
232
|
+
"hourly",
|
|
233
|
+
{ pattern: "41 * * * *", tz: "UTC" },
|
|
234
|
+
// BullMQ tries a job once unless told otherwise: here a failed sweep is retried after one minute, then after two.
|
|
235
|
+
{ name: "sweep", opts: { attempts: 3, backoff: { type: "exponential", delay: 60_000 } } },
|
|
236
|
+
);
|
|
237
|
+
new Worker(
|
|
238
|
+
"upload-sweep",
|
|
239
|
+
async () => {
|
|
240
|
+
// Every page in one call; the time limit leaves the rest to the next run.
|
|
241
|
+
const { data, error } = await uploads.api.sweep({ all: true, signal: AbortSignal.timeout(10 * 60_000) });
|
|
242
|
+
if (error) throw error; // the queue retries the job
|
|
243
|
+
for (const stuck of data.reported) log.warn({ action: "upload_completion_stuck", id: stuck.id });
|
|
244
|
+
},
|
|
245
|
+
{ connection },
|
|
246
|
+
);
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
The worker builds its own `createUploads` with the same storage and database; it serves no requests, so its
|
|
250
|
+
`readSession` can answer `null`. Two sweeps at once are safe. `examples/hono/src/worker.ts` runs the same sweep on a
|
|
251
|
+
timer. An account deletion runs `uploads.api.eraseOwner(userId)` from the same job until it reports `done`
|
|
252
|
+
([Erase an owner](store.md#erase-an-owner)).
|
|
253
|
+
|
|
254
|
+
## 9. The bucket's CORS rule names the web origin
|
|
255
|
+
|
|
256
|
+
The browser sends the bytes from the page, so the bucket's CORS rule lists the web app's origins, never the API's, with
|
|
257
|
+
`PUT`, the four headers cloudrop signs (`content-type`, `content-disposition`, `if-none-match`,
|
|
258
|
+
`x-amz-meta-cloudrop-id`) and `ETag` and `Retry-After` exposed. Add a lifecycle rule that aborts incomplete multipart
|
|
259
|
+
uploads. Both, for R2 and S3: [Bucket setup](bucket-setup.md).
|
|
260
|
+
|
|
261
|
+
## 10. Map uploads onto your Attachment component
|
|
262
|
+
|
|
263
|
+
shadcn's `Attachment` (in its registry) has five states. A file from `useUpload` maps onto them by
|
|
264
|
+
its `status`:
|
|
265
|
+
|
|
266
|
+
| cloudrop `status` | `Attachment` state | Second line |
|
|
267
|
+
| --- | --- | --- |
|
|
268
|
+
| `queued`, `paused` | `idle` | "Waiting", "Paused" |
|
|
269
|
+
| `uploading` | `uploading` | percent of the size, with a progress bar |
|
|
270
|
+
| `completing` | `processing` | the server verifies the object and runs `onComplete` |
|
|
271
|
+
| `done` | `done` | the size |
|
|
272
|
+
| `failed` | `error` | `file.error.code` and `file.error.message`, with Retry |
|
|
273
|
+
| `aborted` | `idle` | "Cancelled" |
|
|
274
|
+
|
|
275
|
+
`examples/vite-react/src/components/upload-row.tsx` is that mapping, with Cancel while the file runs and Retry once it
|
|
276
|
+
failed.
|