@benjavicente/convex-r2 0.0.1 → 0.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,17 @@
1
+ # @benjavicente/convex-r2
2
+
3
+ ## 0.0.2
4
+
5
+ ### Patch Changes
6
+
7
+ - 5eb15b2: Breaking: `confirm` returns only `{ status }`, plus `reason` when rejected. Anyone with an upload id may call it, so it no longer returns the name, key, bucket or SHA-256; read those in `onConfirm` or with `r2.get`.
8
+ - 5eb15b2: Breaking: `r2.store` returns `StoreResult`, `{ uploadId, result }`, and PUTs once, then runs `onConfirm` in one mutation without a HEAD: R2 only accepts the signed PUT when the bytes match.
9
+ - 5eb15b2: A file that doesn't match its request is rejected with "The file doesn't match its upload request". The expected size, type and hash are logged instead of returned.
10
+ - 5eb15b2: The sweep waits a minute plus a second per MB after the URL expires, up to 30 minutes, instead of a fixed 10 minutes.
11
+ - 5eb15b2: When the sweep discards a request whose file hadn't arrived, it deletes the key again a day later, so a slow PUT that lands afterwards doesn't stay in the bucket.
12
+ - 5eb15b2: The generated component types match the validators again, so `confirm`'s result type no longer promises an `upload` field.
13
+ - 5eb15b2: The README compares the same upload with `@convex-dev/r2`, and the package has its own flow tests.
14
+
15
+ ## 0.0.1
16
+
17
+ First release.
package/README.md CHANGED
@@ -4,7 +4,7 @@ Uploads to Cloudflare R2 from a Convex app, where every object belongs to a row
4
4
 
5
5
  - The app defines one request mutation per upload type. It authorizes the user, then asks the component for a URL.
6
6
  - The component generates the key (`<prefix>/<upload id>`). It signs a 5-minute PUT bound to the declared `Content-Type`, `Content-Length` and SHA-256, plus `If-None-Match: *`, so the PUT can't overwrite an existing object.
7
- - After the PUT, the server checks the real object against the request. On a match, it runs the app's `onConfirm` mutation with `{ upload, context }` and marks the upload ready, in one transaction. On a mismatch, or when `onConfirm` throws, it deletes both the object and the request at once.
7
+ - After the PUT, the server checks the real object against the request. On a match, it runs the app's `onConfirm` mutation with `{ upload, context }` and marks the upload ready, in one transaction. On a mismatch, or when `onConfirm` throws, it deletes the request and schedules the object's deletion.
8
8
  - Requests that are never confirmed are checked after they expire: they are confirmed if the object matches, and deleted otherwise.
9
9
  - Credentials come from the component's env vars and never become function arguments. There are no runtime dependencies (SigV4 on WebCrypto).
10
10
 
@@ -85,7 +85,8 @@ export const onUploaded = internalMutation({
85
85
  },
86
86
  handler: async (ctx, { upload, context }) => {
87
87
  // Throwing rejects the upload and deletes it. The message of a
88
- // ConvexError("...") reaches the client as the rejection reason. Any other
88
+ // ConvexError("...") reaches the client as the rejection reason, and anyone
89
+ // who confirms this upload id sees it, so leave file names out. Any other
89
90
  // error rejects it too, including a missing function after a rename, so
90
91
  // keep this name while uploads are pending.
91
92
  await ctx.db.insert("companyDocuments", {
@@ -96,19 +97,21 @@ export const onUploaded = internalMutation({
96
97
  },
97
98
  });
98
99
 
99
- // Needs no authorization: it only compares R2 with the stored request.
100
+ // Needs no authorization: it only compares R2 with the stored request, and
101
+ // returns a status (and a reason when rejected), never the file's details. A
102
+ // mismatch is reported without the expected values, which are logged instead.
100
103
  export const confirmUpload = action({
101
104
  args: { uploadId: v.string() },
102
105
  handler: (ctx, { uploadId }) => r2.confirm(ctx, uploadId),
103
106
  });
104
107
  ```
105
108
 
106
- From an action, `r2.store` uploads bytes the server already has. It goes through the same request, PUT and confirm steps, so `onConfirm` and `context` work the same, and it returns the same result as `confirm`:
109
+ From an action, `r2.store` uploads bytes the server already has. It records the request, PUTs the file, then runs `onConfirm` with your `context`, as a browser upload would. It skips the HEAD: R2 only accepts a PUT signed for the file's size and SHA-256 when the bytes match. It returns the `uploadId` and the same result as `confirm`:
107
110
 
108
111
  ```ts
109
112
  export const writeNote = action({
110
113
  args: {},
111
- handler: async (ctx): Promise<ConfirmResult> => {
114
+ handler: async (ctx): Promise<StoreResult> => {
112
115
  const user = await ctx.runQuery(api.users.me, {});
113
116
  if (!user) throw new ConvexError("Sign in first");
114
117
  return r2.store(ctx, {
@@ -144,7 +147,7 @@ const { confirmed } = await uploadFile(file, {
144
147
  });
145
148
  ```
146
149
 
147
- `uploadFile` hashes the file (reading it into memory), PUTs it with XHR for progress, then calls `confirm`. Without `confirm`, the upload is confirmed after the URL expires plus 10 minutes of grace.
150
+ `uploadFile` hashes the file (reading it into memory), PUTs it with XHR for progress, then calls `confirm`. Without `confirm`, the upload is confirmed after the URL expires plus a grace period: a minute, plus a second per MB, up to 30 minutes.
148
151
 
149
152
  ## Compared with `@convex-dev/r2`
150
153
 
@@ -159,11 +162,107 @@ Measured against version 0.10.4.
159
162
  | Refusing an upload | Throw in `checkUpload`; the object stays in the bucket. | Throw in `onConfirm`; the object and its request are deleted, and the `ConvexError` message is returned. |
160
163
  | Uploads never confirmed | Stay in the bucket. | Checked after the URL expires: confirmed if they match, deleted otherwise. |
161
164
  | Credentials | App env vars, passed to the component as function arguments, so scheduled jobs store them. | Component env vars bound by reference. They are never function arguments. |
162
- | Browser | React and Svelte hooks. | `uploadFile`, framework-agnostic, with hashing, progress and `AbortSignal`. |
165
+ | Browser | React, Svelte and Vue hooks. | `uploadFile`, framework-agnostic, with hashing, progress and `AbortSignal`. |
163
166
  | Signing | AWS SDK v3. | About 100 lines of SigV4 on WebCrypto, tested against the AWS SDK's signer. |
164
167
 
165
168
  `@convex-dev/r2` also lists every object's metadata, and accepts custom keys and per-bucket credentials. Here the app's own tables reference their uploads, so listing is the app's own query; keys are always server-generated; and one component instance uses one Cloudflare account.
166
169
 
170
+ ### The same upload in both
171
+
172
+ A document upload from [the demo](../../examples/r2-demo/convex/documents.ts): signed-in users only, at most 40 KB of PDF, text or Markdown, with a title, and a coin it costs checked when the URL is requested and again when the file arrives.
173
+
174
+ With this package, the checks, the title and the callback travel with the request:
175
+
176
+ ```ts
177
+ export const requestUpload = mutation({
178
+ args: { title: v.string(), ...fileValidator.fields },
179
+ handler: async (ctx, { title, ...file }): Promise<RequestedUpload> => {
180
+ const ownerId = await currentUserId(ctx);
181
+ title = requiredText(title, "A document needs a title");
182
+ await assertCoins(ctx, ownerId, DOCUMENTS.cost);
183
+ return r2.requestUpload(ctx, {
184
+ keyPrefix: `documents/${ownerId}`,
185
+ file,
186
+ rules: { maxSize: DOCUMENTS.maxSize, contentTypes: DOCUMENTS.contentTypes },
187
+ onConfirm: internal.documents.onUploaded,
188
+ context: { ownerId, title },
189
+ });
190
+ },
191
+ });
192
+
193
+ export const onUploaded = internalMutation({
194
+ args: {
195
+ upload: uploadValidator,
196
+ context: v.object({ ownerId: v.id("users"), title: v.string() }),
197
+ },
198
+ handler: async (ctx, { upload, context }) => {
199
+ await assertCoins(ctx, context.ownerId, DOCUMENTS.cost);
200
+ const { uploadId, name, size } = upload;
201
+ await ctx.db.insert("documents", { ...context, uploadId, fileName: name, size });
202
+ },
203
+ });
204
+ ```
205
+
206
+ With `@convex-dev/r2`, `syncMetadata` receives only the key, so the app keeps the request in its own table. The URL doesn't bind size or type, so they are checked after the upload, and a refused object is deleted by hand:
207
+
208
+ ```ts
209
+ // schema.ts: one more table for requests in flight.
210
+ pendingDocuments: defineTable({
211
+ key: v.string(),
212
+ ownerId: v.id("users"),
213
+ title: v.string(),
214
+ fileName: v.string(),
215
+ }).index("key", ["key"]),
216
+
217
+ // documents.ts
218
+ export const r2 = new R2(components.r2);
219
+ const callbacks: R2Callbacks = internal.documents;
220
+
221
+ // The client calls this instead of the generated `generateUploadUrl`,
222
+ // which takes no arguments.
223
+ export const requestUpload = mutation({
224
+ args: { title: v.string(), fileName: v.string() },
225
+ handler: async (ctx, { title, fileName }) => {
226
+ const ownerId = await currentUserId(ctx);
227
+ title = requiredText(title, "A document needs a title");
228
+ await assertCoins(ctx, ownerId, DOCUMENTS.cost);
229
+ const { key, url } = await r2.generateUploadUrl(`documents/${ownerId}/${crypto.randomUUID()}`);
230
+ await ctx.db.insert("pendingDocuments", { key, ownerId, title, fileName });
231
+ return { key, url };
232
+ },
233
+ });
234
+
235
+ export const { syncMetadata, onSyncMetadata } = r2.clientApi<DataModel>({
236
+ callbacks,
237
+ checkUpload: async (ctx) => {
238
+ await currentUserId(ctx);
239
+ },
240
+ // Runs in a later transaction, after the component's action read the object.
241
+ // `syncMetadata` has already returned, so a throw here doesn't reach the
242
+ // client; it has to watch a query to learn the outcome.
243
+ onSyncMetadata: async (ctx, { key }) => {
244
+ const pending = await ctx.db
245
+ .query("pendingDocuments")
246
+ .withIndex("key", (q) => q.eq("key", key))
247
+ .unique();
248
+ if (!pending) return;
249
+ await ctx.db.delete(pending._id);
250
+ const { ownerId, title, fileName } = pending;
251
+ const metadata = await r2.getMetadata(ctx, key);
252
+ const size = metadata?.size ?? 0;
253
+ const accepted =
254
+ size > 0 &&
255
+ size <= DOCUMENTS.maxSize &&
256
+ DOCUMENTS.contentTypes.includes(metadata?.contentType ?? "") &&
257
+ (await coinsLeft(ctx, ownerId)) >= DOCUMENTS.cost;
258
+ if (!accepted) return await r2.deleteObject(ctx, key);
259
+ await ctx.db.insert("documents", { ownerId, title, key, fileName, size });
260
+ },
261
+ });
262
+ ```
263
+
264
+ On top of that, `useUploadFile` calls the generated `generateUploadUrl`, so the client needs its own upload code. A request whose file never arrives stays in `pendingDocuments` until a cron cleans it up. A file uploaded without `syncMetadata` is never checked: it stays in the bucket and no row refers to it.
265
+
167
266
  ### Size
168
267
 
169
268
  Each figure is the bundle growth over a control module with one trivial function. The bundles are built with esbuild settings that reproduce Convex's bundler: renamed identifiers, whitespace kept, external source maps. A real Convex push of the `@convex-dev/r2` app side came out slightly larger: 583 KB of JS and 703 KB of maps.
@@ -5,7 +5,7 @@ export { fileValidator, uploadValidator, type ConfirmResult, type FileInfo, type
5
5
  type RunQueryCtx = Pick<GenericQueryCtx<GenericDataModel>, "runQuery">;
6
6
  type RunMutationCtx = Pick<GenericMutationCtx<GenericDataModel>, "runMutation">;
7
7
  type RunActionCtx = Pick<GenericActionCtx<GenericDataModel>, "runAction">;
8
- type StoreCtx = Pick<GenericActionCtx<GenericDataModel>, "runMutation" | "runAction">;
8
+ type StoreCtx = Pick<GenericActionCtx<GenericDataModel>, "runMutation">;
9
9
  /** The app's mutation that runs when an upload is confirmed. */
10
10
  export type OnConfirm<Context> = FunctionReference<"mutation", "internal", {
11
11
  upload: Upload;
@@ -31,6 +31,11 @@ export type StoreArgs<Context> = {
31
31
  /** Passed to `onConfirm`; must match its `context` argument. */
32
32
  context: NoInfer<Context>;
33
33
  };
34
+ /** What `store` returns: the upload's id and the result of confirming it. */
35
+ export type StoreResult = {
36
+ uploadId: string;
37
+ result: ConfirmResult;
38
+ };
34
39
  /**
35
40
  * What `requestUpload` returns. Annotate the app's request handler with it:
36
41
  * the handler references `internal.*`, which TypeScript can't infer through.
@@ -63,35 +68,19 @@ export declare class R2 {
63
68
  requestUpload<Context>(ctx: RunMutationCtx, args: RequestUploadArgs<Context>): Promise<RequestedUpload>;
64
69
  private requestArgs;
65
70
  /**
66
- * Uploads bytes from an action, through the same request, PUT and confirm
67
- * steps as a browser upload: `onConfirm` runs once R2 holds the file, and a
68
- * failed store is swept like any unconfirmed upload.
71
+ * Uploads bytes from an action: records the request, PUTs the file, then
72
+ * runs `onConfirm`. R2 accepting the signed PUT proves the object matches,
73
+ * so there is no HEAD. A store that fails midway is swept like any
74
+ * unconfirmed upload.
69
75
  */
70
- store<Context>(ctx: StoreCtx, args: StoreArgs<Context>): Promise<ConfirmResult>;
76
+ store<Context>(ctx: StoreCtx, args: StoreArgs<Context>): Promise<StoreResult>;
71
77
  /**
72
78
  * Checks the uploaded object against its request and, if it matches, runs
73
79
  * `onConfirm`. It trusts nothing but the upload id, so a public action may
74
- * call it without authorization.
80
+ * call it without authorization: the result is only a status and, when
81
+ * rejected, the reason.
75
82
  */
76
- confirm(ctx: RunActionCtx, uploadId: string): Promise<{
77
- status: "ready";
78
- upload: {
79
- bucket: string;
80
- contentType: string;
81
- key: string;
82
- name: string;
83
- sha256: string;
84
- size: number;
85
- uploadId: string;
86
- };
87
- } | {
88
- status: "notUploaded";
89
- } | {
90
- reason: string;
91
- status: "rejected";
92
- } | {
93
- status: "missing";
94
- }>;
83
+ confirm(ctx: RunActionCtx, uploadId: string): Promise<ConfirmResult>;
95
84
  /** A ready upload with its Cloudflare dashboard link (null otherwise). */
96
85
  get(ctx: RunQueryCtx, uploadId: string): Promise<{
97
86
  bucket: string;
@@ -32,9 +32,10 @@ export class R2 {
32
32
  };
33
33
  }
34
34
  /**
35
- * Uploads bytes from an action, through the same request, PUT and confirm
36
- * steps as a browser upload: `onConfirm` runs once R2 holds the file, and a
37
- * failed store is swept like any unconfirmed upload.
35
+ * Uploads bytes from an action: records the request, PUTs the file, then
36
+ * runs `onConfirm`. R2 accepting the signed PUT proves the object matches,
37
+ * so there is no HEAD. A store that fails midway is swept like any
38
+ * unconfirmed upload.
38
39
  */
39
40
  async store(ctx, args) {
40
41
  const { file: blob, name, contentType = blob.type, ...rest } = args;
@@ -44,12 +45,14 @@ export class R2 {
44
45
  const response = await fetch(url, { method: "PUT", headers, body: blob });
45
46
  if (!response.ok)
46
47
  throw new Error(`R2 PUT failed: ${response.status} ${await response.text()}`);
47
- return this.confirm(ctx, uploadId);
48
+ const result = await ctx.runMutation(this.component.uploads.confirmStored, { uploadId });
49
+ return { uploadId, result };
48
50
  }
49
51
  /**
50
52
  * Checks the uploaded object against its request and, if it matches, runs
51
53
  * `onConfirm`. It trusts nothing but the upload id, so a public action may
52
- * call it without authorization.
54
+ * call it without authorization: the result is only a status and, when
55
+ * rejected, the reason.
53
56
  */
54
57
  async confirm(ctx, uploadId) {
55
58
  return ctx.runAction(this.component.uploads.confirm, { uploadId });
@@ -24,15 +24,18 @@ export type ComponentApi<Name extends string | undefined = string | undefined> =
24
24
  uploadId: string;
25
25
  }, {
26
26
  status: "ready";
27
- upload: {
28
- bucket: string;
29
- contentType: string;
30
- key: string;
31
- name: string;
32
- sha256: string;
33
- size: number;
34
- uploadId: string;
35
- };
27
+ } | {
28
+ status: "notUploaded";
29
+ } | {
30
+ reason: string;
31
+ status: "rejected";
32
+ } | {
33
+ status: "missing";
34
+ }, Name>;
35
+ confirmStored: FunctionReference<"mutation", "internal", {
36
+ uploadId: string;
37
+ }, {
38
+ status: "ready";
36
39
  } | {
37
40
  status: "notUploaded";
38
41
  } | {
@@ -1,7 +1,12 @@
1
1
  /** Lifetime of an upload URL, in seconds. */
2
2
  export declare const UPLOAD_URL_EXPIRES_IN: number;
3
- /** Time after URL expiry for an upload that started in time to finish. */
4
- export declare const SWEEP_GRACE_MS: number;
3
+ /**
4
+ * Time after URL expiry for an upload that started in time to finish: a
5
+ * minute, plus the file at 1 MB/s, up to 30 minutes. R2 checks expiry when the
6
+ * PUT starts, so a slower upload can still be discarded mid-way; `discard`
7
+ * deletes its key again later.
8
+ */
9
+ export declare function sweepGraceMs(size: number): number;
5
10
  /**
6
11
  * Records an upload request and signs a single-use PUT URL bound to the
7
12
  * declared file. The caller authorizes; this only checks the file against
@@ -36,21 +41,12 @@ export declare const request: import("convex/server").RegisteredMutation<"public
36
41
  /**
37
42
  * Checks the object in R2 against its request. On a match, marks the upload
38
43
  * ready and runs `onConfirm` in one transaction. Safe to call from untrusted
39
- * clients: it only reads R2.
44
+ * clients: it only reads R2, and its result has no upload details.
40
45
  */
41
46
  export declare const confirm: import("convex/server").RegisteredAction<"public", {
42
47
  uploadId: string;
43
48
  }, Promise<{
44
49
  status: "ready";
45
- upload: {
46
- bucket: string;
47
- contentType: string;
48
- key: string;
49
- name: string;
50
- sha256: string;
51
- size: number;
52
- uploadId: string;
53
- };
54
50
  } | {
55
51
  status: "notUploaded";
56
52
  } | {
@@ -85,15 +81,24 @@ export declare const finalize: import("convex/server").RegisteredMutation<"inter
85
81
  uploadId: string;
86
82
  }, Promise<{
87
83
  status: "ready";
88
- upload: {
89
- bucket: string;
90
- contentType: string;
91
- key: string;
92
- name: string;
93
- sha256: string;
94
- size: number;
95
- uploadId: string;
96
- };
84
+ } | {
85
+ status: "notUploaded";
86
+ } | {
87
+ reason: string;
88
+ status: "rejected";
89
+ } | {
90
+ status: "missing";
91
+ }>>;
92
+ /**
93
+ * Runs `onConfirm` for an upload the server PUT itself (`R2.store`), without
94
+ * reading R2: the PUT was signed for the declared size and SHA-256, so R2
95
+ * accepting it proves the object matches. Trusts its caller, so only call it
96
+ * after a successful PUT.
97
+ */
98
+ export declare const confirmStored: import("convex/server").RegisteredMutation<"public", {
99
+ uploadId: string;
100
+ }, Promise<{
101
+ status: "ready";
97
102
  } | {
98
103
  status: "notUploaded";
99
104
  } | {
@@ -114,7 +119,11 @@ export declare const sweep: import("convex/server").RegisteredAction<"internal",
114
119
  key: string;
115
120
  uploadId: import("convex/values").GenericId<"uploads">;
116
121
  }, Promise<void>>;
117
- /** Deletes a request that is still pending, and its object. */
122
+ /**
123
+ * Deletes a request whose object wasn't there when swept, and deletes the key
124
+ * again a day later: a slow PUT that started before the URL expired can still
125
+ * land after the sweep, and nothing else would ever remove it.
126
+ */
118
127
  export declare const discard: import("convex/server").RegisteredMutation<"internal", {
119
128
  uploadId: import("convex/values").GenericId<"uploads">;
120
129
  }, Promise<void>>;
@@ -5,8 +5,17 @@ import { dashboardLink, deleteObject as deleteR2Object, headObject, hexToBase64,
5
5
  import { confirmResultValidator, fileValidator, rulesValidator, uploadValidator, } from "./validators.js";
6
6
  /** Lifetime of an upload URL, in seconds. */
7
7
  export const UPLOAD_URL_EXPIRES_IN = 5 * 60;
8
- /** Time after URL expiry for an upload that started in time to finish. */
9
- export const SWEEP_GRACE_MS = 10 * 60 * 1000;
8
+ /**
9
+ * Time after URL expiry for an upload that started in time to finish: a
10
+ * minute, plus the file at 1 MB/s, up to 30 minutes. R2 checks expiry when the
11
+ * PUT starts, so a slower upload can still be discarded mid-way; `discard`
12
+ * deletes its key again later.
13
+ */
14
+ export function sweepGraceMs(size) {
15
+ return Math.min(60_000 + size / 1000, 30 * 60_000);
16
+ }
17
+ /** How long after a sweep a PUT that started in time might still land. */
18
+ const LATE_PUT_MS = 24 * 60 * 60 * 1000;
10
19
  /** R2's limit for a single PUT. */
11
20
  const MAX_PUT_SIZE = 5 * 1024 ** 3;
12
21
  /** R2 calls retried by scheduled jobs: 4 s, 16 s, … up to about 6 hours in total. */
@@ -107,7 +116,7 @@ export const request = mutation({
107
116
  });
108
117
  const key = `${keyPrefix}/${uploadId}`;
109
118
  await ctx.db.patch("uploads", uploadId, { key });
110
- await ctx.scheduler.runAt(expiresAt + SWEEP_GRACE_MS, internal.uploads.sweep, {
119
+ await ctx.scheduler.runAt(expiresAt + sweepGraceMs(file.size), internal.uploads.sweep, {
111
120
  uploadId,
112
121
  bucket,
113
122
  key,
@@ -131,7 +140,7 @@ export const request = mutation({
131
140
  /**
132
141
  * Checks the object in R2 against its request. On a match, marks the upload
133
142
  * ready and runs `onConfirm` in one transaction. Safe to call from untrusted
134
- * clients: it only reads R2.
143
+ * clients: it only reads R2, and its result has no upload details.
135
144
  */
136
145
  export const confirm = action({
137
146
  args: { uploadId: v.string() },
@@ -141,7 +150,7 @@ export const confirm = action({
141
150
  if (!row)
142
151
  return { status: "missing" };
143
152
  if (!row.pending)
144
- return { status: "ready", upload: toUpload(row) };
153
+ return { status: "ready" };
145
154
  const head = await headObject(row.bucket, row.key);
146
155
  if (!head)
147
156
  return { status: "notUploaded" };
@@ -166,28 +175,50 @@ export const finalize = internalMutation({
166
175
  const row = await loadRow(ctx, uploadId);
167
176
  if (!row)
168
177
  return { status: "missing" };
169
- const upload = toUpload(row);
170
178
  if (!row.pending)
171
- return { status: "ready", upload };
172
- const { onConfirm, context } = row.pending;
179
+ return { status: "ready" };
173
180
  const reason = mismatch(row, head);
174
181
  if (reason) {
182
+ // Details stay in the logs: anyone with the upload id may confirm it.
183
+ console.warn(`Upload ${uploadId} rejected: ${reason}`);
175
184
  await removeRow(ctx, row);
176
- return { status: "rejected", reason };
177
- }
178
- try {
179
- // A throwing callback rolls back its own writes only, so the app
180
- // refuses an upload by throwing and the rejection below still commits.
181
- await ctx.runMutation(onConfirm, { upload, context });
182
- }
183
- catch (error) {
184
- await removeRow(ctx, row);
185
- return { status: "rejected", reason: refusal(error) };
185
+ return { status: "rejected", reason: "The file doesn't match its upload request" };
186
186
  }
187
- await ctx.db.patch("uploads", row._id, { pending: undefined });
188
- return { status: "ready", upload };
187
+ return accept(ctx, row, row.pending);
189
188
  },
190
189
  });
190
+ /**
191
+ * Runs `onConfirm` for an upload the server PUT itself (`R2.store`), without
192
+ * reading R2: the PUT was signed for the declared size and SHA-256, so R2
193
+ * accepting it proves the object matches. Trusts its caller, so only call it
194
+ * after a successful PUT.
195
+ */
196
+ export const confirmStored = mutation({
197
+ args: { uploadId: v.string() },
198
+ returns: confirmResultValidator,
199
+ handler: async (ctx, { uploadId }) => {
200
+ const row = await loadRow(ctx, uploadId);
201
+ if (!row)
202
+ return { status: "missing" };
203
+ if (!row.pending)
204
+ return { status: "ready" };
205
+ return accept(ctx, row, row.pending);
206
+ },
207
+ });
208
+ /** Runs `onConfirm` and marks the upload ready, or deletes it if `onConfirm` throws. */
209
+ async function accept(ctx, row, { onConfirm, context }) {
210
+ try {
211
+ // A throwing callback rolls back its own writes only, so the app
212
+ // refuses an upload by throwing and the rejection below still commits.
213
+ await ctx.runMutation(onConfirm, { upload: toUpload(row), context });
214
+ }
215
+ catch (error) {
216
+ await removeRow(ctx, row);
217
+ return { status: "rejected", reason: refusal(error) };
218
+ }
219
+ await ctx.db.patch("uploads", row._id, { pending: undefined });
220
+ return { status: "ready" };
221
+ }
191
222
  /**
192
223
  * Runs once per request after its URL expired: confirms an upload the client
193
224
  * never confirmed, or deletes the request and any object. When the row is
@@ -220,13 +251,23 @@ export const sweep = internalAction({
220
251
  }
221
252
  },
222
253
  });
223
- /** Deletes a request that is still pending, and its object. */
254
+ /**
255
+ * Deletes a request whose object wasn't there when swept, and deletes the key
256
+ * again a day later: a slow PUT that started before the URL expired can still
257
+ * land after the sweep, and nothing else would ever remove it.
258
+ */
224
259
  export const discard = internalMutation({
225
260
  args: { uploadId: v.id("uploads") },
226
261
  handler: async (ctx, { uploadId }) => {
227
262
  const row = await ctx.db.get("uploads", uploadId);
228
- if (row?.pending)
229
- await removeRow(ctx, row);
263
+ if (!row?.pending)
264
+ return;
265
+ await removeRow(ctx, row);
266
+ await ctx.scheduler.runAfter(LATE_PUT_MS, internal.uploads.deleteObject, {
267
+ bucket: row.bucket,
268
+ key: row.key,
269
+ attempt: 1,
270
+ });
230
271
  },
231
272
  });
232
273
  export const deleteObject = internalAction({
@@ -39,17 +39,12 @@ export declare const uploadValidator: import("convex/values").VObject<{
39
39
  key: import("convex/values").VString<string, "required">;
40
40
  }, "required", "bucket" | "contentType" | "key" | "name" | "sha256" | "size" | "uploadId">;
41
41
  export type Upload = Infer<typeof uploadValidator>;
42
+ /**
43
+ * What `confirm` returns. It carries no upload details: anyone who has an
44
+ * upload id may confirm it, so names and keys are left to `onConfirm` and `get`.
45
+ */
42
46
  export declare const confirmResultValidator: import("convex/values").VUnion<{
43
47
  status: "ready";
44
- upload: {
45
- bucket: string;
46
- contentType: string;
47
- key: string;
48
- name: string;
49
- sha256: string;
50
- size: number;
51
- uploadId: string;
52
- };
53
48
  } | {
54
49
  status: "notUploaded";
55
50
  } | {
@@ -59,35 +54,9 @@ export declare const confirmResultValidator: import("convex/values").VUnion<{
59
54
  status: "missing";
60
55
  }, [import("convex/values").VObject<{
61
56
  status: "ready";
62
- upload: {
63
- bucket: string;
64
- contentType: string;
65
- key: string;
66
- name: string;
67
- sha256: string;
68
- size: number;
69
- uploadId: string;
70
- };
71
57
  }, {
72
58
  status: import("convex/values").VLiteral<"ready", "required">;
73
- upload: import("convex/values").VObject<{
74
- bucket: string;
75
- contentType: string;
76
- key: string;
77
- name: string;
78
- sha256: string;
79
- size: number;
80
- uploadId: string;
81
- }, {
82
- name: import("convex/values").VString<string, "required">;
83
- contentType: import("convex/values").VString<string, "required">;
84
- size: import("convex/values").VFloat64<number, "required">;
85
- sha256: import("convex/values").VString<string, "required">;
86
- uploadId: import("convex/values").VString<string, "required">;
87
- bucket: import("convex/values").VString<string, "required">;
88
- key: import("convex/values").VString<string, "required">;
89
- }, "required", "bucket" | "contentType" | "key" | "name" | "sha256" | "size" | "uploadId">;
90
- }, "required", "status" | "upload" | "upload.bucket" | "upload.contentType" | "upload.key" | "upload.name" | "upload.sha256" | "upload.size" | "upload.uploadId">, import("convex/values").VObject<{
59
+ }, "required", "status">, import("convex/values").VObject<{
91
60
  status: "notUploaded";
92
61
  }, {
93
62
  status: import("convex/values").VLiteral<"notUploaded", "required">;
@@ -101,5 +70,5 @@ export declare const confirmResultValidator: import("convex/values").VUnion<{
101
70
  status: "missing";
102
71
  }, {
103
72
  status: import("convex/values").VLiteral<"missing", "required">;
104
- }, "required", "status">], "required", "reason" | "status" | "upload" | "upload.bucket" | "upload.contentType" | "upload.key" | "upload.name" | "upload.sha256" | "upload.size" | "upload.uploadId">;
73
+ }, "required", "status">], "required", "reason" | "status">;
105
74
  export type ConfirmResult = Infer<typeof confirmResultValidator>;
@@ -17,7 +17,11 @@ export const uploadValidator = v.object({
17
17
  key: v.string(),
18
18
  ...fileValidator.fields,
19
19
  });
20
- export const confirmResultValidator = v.union(v.object({ status: v.literal("ready"), upload: uploadValidator }),
20
+ /**
21
+ * What `confirm` returns. It carries no upload details: anyone who has an
22
+ * upload id may confirm it, so names and keys are left to `onConfirm` and `get`.
23
+ */
24
+ export const confirmResultValidator = v.union(v.object({ status: v.literal("ready") }),
21
25
  /** The object isn't in R2 yet. */
22
26
  v.object({ status: v.literal("notUploaded") }),
23
27
  /** The object didn't match the request, or `onConfirm` threw; both were deleted. */
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@benjavicente/convex-r2",
3
- "version": "0.0.1",
3
+ "version": "0.0.2",
4
4
  "description": "Cloudflare R2 uploads for Convex: server-owned keys, bound presigned URLs and typed confirmation callbacks.",
5
5
  "keywords": [
6
6
  "cloudflare",
@@ -19,6 +19,7 @@
19
19
  "directory": "components/r2"
20
20
  },
21
21
  "files": [
22
+ "CHANGELOG.md",
22
23
  "dist",
23
24
  "src",
24
25
  "!src/**/*.test.ts"
@@ -22,7 +22,7 @@ export {
22
22
  type RunQueryCtx = Pick<GenericQueryCtx<GenericDataModel>, "runQuery">;
23
23
  type RunMutationCtx = Pick<GenericMutationCtx<GenericDataModel>, "runMutation">;
24
24
  type RunActionCtx = Pick<GenericActionCtx<GenericDataModel>, "runAction">;
25
- type StoreCtx = Pick<GenericActionCtx<GenericDataModel>, "runMutation" | "runAction">;
25
+ type StoreCtx = Pick<GenericActionCtx<GenericDataModel>, "runMutation">;
26
26
 
27
27
  /** The app's mutation that runs when an upload is confirmed. */
28
28
  export type OnConfirm<Context> = FunctionReference<
@@ -53,6 +53,9 @@ export type StoreArgs<Context> = {
53
53
  context: NoInfer<Context>;
54
54
  };
55
55
 
56
+ /** What `store` returns: the upload's id and the result of confirming it. */
57
+ export type StoreResult = { uploadId: string; result: ConfirmResult };
58
+
56
59
  /**
57
60
  * What `requestUpload` returns. Annotate the app's request handler with it:
58
61
  * the handler references `internal.*`, which TypeScript can't infer through.
@@ -101,11 +104,12 @@ export class R2 {
101
104
  }
102
105
 
103
106
  /**
104
- * Uploads bytes from an action, through the same request, PUT and confirm
105
- * steps as a browser upload: `onConfirm` runs once R2 holds the file, and a
106
- * failed store is swept like any unconfirmed upload.
107
+ * Uploads bytes from an action: records the request, PUTs the file, then
108
+ * runs `onConfirm`. R2 accepting the signed PUT proves the object matches,
109
+ * so there is no HEAD. A store that fails midway is swept like any
110
+ * unconfirmed upload.
107
111
  */
108
- async store<Context>(ctx: StoreCtx, args: StoreArgs<Context>): Promise<ConfirmResult> {
112
+ async store<Context>(ctx: StoreCtx, args: StoreArgs<Context>): Promise<StoreResult> {
109
113
  const { file: blob, name, contentType = blob.type, ...rest } = args;
110
114
  const file = { name, contentType, size: blob.size, sha256: await sha256Hex(blob) };
111
115
  const rules = { maxSize: file.size, contentTypes: [contentType] };
@@ -115,15 +119,17 @@ export class R2 {
115
119
  );
116
120
  const response = await fetch(url, { method: "PUT", headers, body: blob });
117
121
  if (!response.ok) throw new Error(`R2 PUT failed: ${response.status} ${await response.text()}`);
118
- return this.confirm(ctx, uploadId);
122
+ const result = await ctx.runMutation(this.component.uploads.confirmStored, { uploadId });
123
+ return { uploadId, result };
119
124
  }
120
125
 
121
126
  /**
122
127
  * Checks the uploaded object against its request and, if it matches, runs
123
128
  * `onConfirm`. It trusts nothing but the upload id, so a public action may
124
- * call it without authorization.
129
+ * call it without authorization: the result is only a status and, when
130
+ * rejected, the reason.
125
131
  */
126
- async confirm(ctx: RunActionCtx, uploadId: string) {
132
+ async confirm(ctx: RunActionCtx, uploadId: string): Promise<ConfirmResult> {
127
133
  return ctx.runAction(this.component.uploads.confirm, { uploadId });
128
134
  }
129
135
 
@@ -28,18 +28,17 @@ export type ComponentApi<Name extends string | undefined = string | undefined> =
28
28
  "action",
29
29
  "internal",
30
30
  { uploadId: string },
31
- | {
32
- status: "ready";
33
- upload: {
34
- bucket: string;
35
- contentType: string;
36
- key: string;
37
- name: string;
38
- sha256: string;
39
- size: number;
40
- uploadId: string;
41
- };
42
- }
31
+ | { status: "ready" }
32
+ | { status: "notUploaded" }
33
+ | { reason: string; status: "rejected" }
34
+ | { status: "missing" },
35
+ Name
36
+ >;
37
+ confirmStored: FunctionReference<
38
+ "mutation",
39
+ "internal",
40
+ { uploadId: string },
41
+ | { status: "ready" }
43
42
  | { status: "notUploaded" }
44
43
  | { reason: string; status: "rejected" }
45
44
  | { status: "missing" },
@@ -34,8 +34,17 @@ import {
34
34
 
35
35
  /** Lifetime of an upload URL, in seconds. */
36
36
  export const UPLOAD_URL_EXPIRES_IN = 5 * 60;
37
- /** Time after URL expiry for an upload that started in time to finish. */
38
- export const SWEEP_GRACE_MS = 10 * 60 * 1000;
37
+ /**
38
+ * Time after URL expiry for an upload that started in time to finish: a
39
+ * minute, plus the file at 1 MB/s, up to 30 minutes. R2 checks expiry when the
40
+ * PUT starts, so a slower upload can still be discarded mid-way; `discard`
41
+ * deletes its key again later.
42
+ */
43
+ export function sweepGraceMs(size: number) {
44
+ return Math.min(60_000 + size / 1000, 30 * 60_000);
45
+ }
46
+ /** How long after a sweep a PUT that started in time might still land. */
47
+ const LATE_PUT_MS = 24 * 60 * 60 * 1000;
39
48
  /** R2's limit for a single PUT. */
40
49
  const MAX_PUT_SIZE = 5 * 1024 ** 3;
41
50
  /** R2 calls retried by scheduled jobs: 4 s, 16 s, … up to about 6 hours in total. */
@@ -141,7 +150,7 @@ export const request = mutation({
141
150
  });
142
151
  const key = `${keyPrefix}/${uploadId}`;
143
152
  await ctx.db.patch("uploads", uploadId, { key });
144
- await ctx.scheduler.runAt(expiresAt + SWEEP_GRACE_MS, internal.uploads.sweep, {
153
+ await ctx.scheduler.runAt(expiresAt + sweepGraceMs(file.size), internal.uploads.sweep, {
145
154
  uploadId,
146
155
  bucket,
147
156
  key,
@@ -167,7 +176,7 @@ export const request = mutation({
167
176
  /**
168
177
  * Checks the object in R2 against its request. On a match, marks the upload
169
178
  * ready and runs `onConfirm` in one transaction. Safe to call from untrusted
170
- * clients: it only reads R2.
179
+ * clients: it only reads R2, and its result has no upload details.
171
180
  */
172
181
  export const confirm = action({
173
182
  args: { uploadId: v.string() },
@@ -175,7 +184,7 @@ export const confirm = action({
175
184
  handler: async (ctx, { uploadId }): Promise<ConfirmResult> => {
176
185
  const row = await ctx.runQuery(internal.uploads.load, { uploadId });
177
186
  if (!row) return { status: "missing" };
178
- if (!row.pending) return { status: "ready", upload: toUpload(row) };
187
+ if (!row.pending) return { status: "ready" };
179
188
  const head = await headObject(row.bucket, row.key);
180
189
  if (!head) return { status: "notUploaded" };
181
190
  return ctx.runMutation(internal.uploads.finalize, { uploadId, head });
@@ -200,27 +209,53 @@ export const finalize = internalMutation({
200
209
  handler: async (ctx, { uploadId, head }): Promise<ConfirmResult> => {
201
210
  const row = await loadRow(ctx, uploadId);
202
211
  if (!row) return { status: "missing" };
203
- const upload = toUpload(row);
204
- if (!row.pending) return { status: "ready", upload };
205
- const { onConfirm, context } = row.pending;
212
+ if (!row.pending) return { status: "ready" };
206
213
  const reason = mismatch(row, head);
207
214
  if (reason) {
215
+ // Details stay in the logs: anyone with the upload id may confirm it.
216
+ console.warn(`Upload ${uploadId} rejected: ${reason}`);
208
217
  await removeRow(ctx, row);
209
- return { status: "rejected", reason };
210
- }
211
- try {
212
- // A throwing callback rolls back its own writes only, so the app
213
- // refuses an upload by throwing and the rejection below still commits.
214
- await ctx.runMutation(onConfirm, { upload, context });
215
- } catch (error) {
216
- await removeRow(ctx, row);
217
- return { status: "rejected", reason: refusal(error) };
218
+ return { status: "rejected", reason: "The file doesn't match its upload request" };
218
219
  }
219
- await ctx.db.patch("uploads", row._id, { pending: undefined });
220
- return { status: "ready", upload };
220
+ return accept(ctx, row, row.pending);
221
+ },
222
+ });
223
+
224
+ /**
225
+ * Runs `onConfirm` for an upload the server PUT itself (`R2.store`), without
226
+ * reading R2: the PUT was signed for the declared size and SHA-256, so R2
227
+ * accepting it proves the object matches. Trusts its caller, so only call it
228
+ * after a successful PUT.
229
+ */
230
+ export const confirmStored = mutation({
231
+ args: { uploadId: v.string() },
232
+ returns: confirmResultValidator,
233
+ handler: async (ctx, { uploadId }): Promise<ConfirmResult> => {
234
+ const row = await loadRow(ctx, uploadId);
235
+ if (!row) return { status: "missing" };
236
+ if (!row.pending) return { status: "ready" };
237
+ return accept(ctx, row, row.pending);
221
238
  },
222
239
  });
223
240
 
241
+ /** Runs `onConfirm` and marks the upload ready, or deletes it if `onConfirm` throws. */
242
+ async function accept(
243
+ ctx: MutationCtx,
244
+ row: Doc<"uploads">,
245
+ { onConfirm, context }: NonNullable<Doc<"uploads">["pending"]>,
246
+ ): Promise<ConfirmResult> {
247
+ try {
248
+ // A throwing callback rolls back its own writes only, so the app
249
+ // refuses an upload by throwing and the rejection below still commits.
250
+ await ctx.runMutation(onConfirm, { upload: toUpload(row), context });
251
+ } catch (error) {
252
+ await removeRow(ctx, row);
253
+ return { status: "rejected", reason: refusal(error) };
254
+ }
255
+ await ctx.db.patch("uploads", row._id, { pending: undefined });
256
+ return { status: "ready" };
257
+ }
258
+
224
259
  /**
225
260
  * Runs once per request after its URL expired: confirms an upload the client
226
261
  * never confirmed, or deletes the request and any object. When the row is
@@ -248,12 +283,22 @@ export const sweep = internalAction({
248
283
  },
249
284
  });
250
285
 
251
- /** Deletes a request that is still pending, and its object. */
286
+ /**
287
+ * Deletes a request whose object wasn't there when swept, and deletes the key
288
+ * again a day later: a slow PUT that started before the URL expired can still
289
+ * land after the sweep, and nothing else would ever remove it.
290
+ */
252
291
  export const discard = internalMutation({
253
292
  args: { uploadId: v.id("uploads") },
254
293
  handler: async (ctx, { uploadId }) => {
255
294
  const row = await ctx.db.get("uploads", uploadId);
256
- if (row?.pending) await removeRow(ctx, row);
295
+ if (!row?.pending) return;
296
+ await removeRow(ctx, row);
297
+ await ctx.scheduler.runAfter(LATE_PUT_MS, internal.uploads.deleteObject, {
298
+ bucket: row.bucket,
299
+ key: row.key,
300
+ attempt: 1,
301
+ });
257
302
  },
258
303
  });
259
304
 
@@ -24,8 +24,12 @@ export const uploadValidator = v.object({
24
24
  });
25
25
  export type Upload = Infer<typeof uploadValidator>;
26
26
 
27
+ /**
28
+ * What `confirm` returns. It carries no upload details: anyone who has an
29
+ * upload id may confirm it, so names and keys are left to `onConfirm` and `get`.
30
+ */
27
31
  export const confirmResultValidator = v.union(
28
- v.object({ status: v.literal("ready"), upload: uploadValidator }),
32
+ v.object({ status: v.literal("ready") }),
29
33
  /** The object isn't in R2 yet. */
30
34
  v.object({ status: v.literal("notUploaded") }),
31
35
  /** The object didn't match the request, or `onConfirm` threw; both were deleted. */