@sproutboat/runtime 0.6.6 → 0.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,45 @@
1
1
  # @sproutboat/runtime
2
2
 
3
+ ## 0.7.1
4
+
5
+ ### Patch Changes
6
+
7
+ - b0c5bbd: Fix #177: `new Response(obj.body)` on an `R2Object` UTF-8-re-encoded raw
8
+ bytes, corrupting any binary content (each byte 0x80-0xFF doubled on the
9
+ wire). Added `R2Object.toResponse(init?)`, which serves the bytes
10
+ unmodified via the same `x-sb-raw-body` marker `__sbRawBodyResponse` already
11
+ uses for assets/fetch/service-binding responses. Defaults `content-type`
12
+ from `httpMetadata` and `etag` from `httpEtag` when not set in `init`.
13
+
14
+ ## 0.7.0
15
+
16
+ ### Minor Changes
17
+
18
+ - 6998a15: R2 object bytes no longer live inside SQLite, on either transport (baronunread/sproutboat#56). Every `put()` used to bind the whole object into a `body` column, and every `get()` read it back out — doubling the object through SQLite's own page cache/WAL machinery on top of the buffer it already arrived in, which was the real driver of #56's "large uploads make memory skyrocket" complaint (the JSON-escaping half of that was already fixed in #63).
19
+
20
+ Objects now live in their own file under `r2-blobs/`, named by a hash of bucket+key (so unicode keys and directory traversal are non-issues):
21
+
22
+ - **Embedded (standalone):** `<data-dir>/r2-blobs/` — `sb_r2_put_c`/`sb_r2_get_c` write/read the file directly via C `fopen`/`fwrite`/`fread`, no SQLite blob column at all. `sb_r2_delete_blob` removes the file on delete.
23
+ - **Broker:** colocated with whichever store file already holds that bucket's metadata — the main `db` for a bare-string binding, the resource's own file under `resourceDir` for an account-level one (#74) — so blobs persist across a redeploy exactly when that metadata does. `:memory:` stores (tests, local dev) fall back to an in-memory map.
24
+
25
+ No migration: this is a schema change for new writes going forward, not a converter for objects already stored inline from before.
26
+
27
+ Verified end-to-end against both transports (kitchen-sink's full conformance suite, 24/24 standalone + 27/27 broker) and with new unit tests for file-backed persistence, the `:memory:` fallback, and a real binary round-trip over the v1 wire protocol.
28
+
29
+ **Known limitation found while verifying this, not fixed here:** a binary `put()` body (bytes ≥ 0x80) is corrupted on the way *in*, before it ever reaches storage — `porf_native_fetch_read_value` (the function every inline-C string read in the runtime goes through) always UTF-8-encodes a Latin-1-range string on read, which is correct for genuine text but wrong for opaque bytes coming off a raw HTTP body. This predates this change (the old SQLite-blob path had the identical corruption) and is not something the storage backend can fix — it needs an input-side counterpart to `#176`'s `x-sb-raw-body` marker, or real `ArrayBuffer`/`Uint8Array` body support (the upstream ask already filed as Draft G in `patches/UPSTREAM.md`).
30
+
31
+ ### Patch Changes
32
+
33
+ - 188d8e4: Fix (baronunread/sproutboat#184): a `put()` body (R2, both transports) with bytes ≥ 0x80 was corrupted on the way *in*, before storage — `porf_native_fetch_read_value` (the one function every inline-C string read in the runtime funnels through) always UTF-8-encodes a Latin-1-range value, correct for genuine text, wrong for opaque bytes off a raw HTTP upload. A 256-byte binary upload landed as 384 stored bytes.
34
+
35
+ `__sbR2PutRaw` (embedded) and `__sbCallBin` (broker, the only caller that ever passes it a real body) now try `porf_native_fetch_read_raw_bytes` first, falling back to the normal encoding path only for a genuine multi-byte JS string (which isn't a "bytestring" and so isn't eligible for the raw read anyway). This predates the #56 storage change — the old SQLite-blob `put()` called the identical `read_value`, so it was equally corrupted; the existing R2 conformance check never caught it because it only ever exercises a plain ASCII string.
36
+
37
+ Also fixes an adjacent, independently pre-existing bug found while verifying this: `R2Object.text()`/`.json()` returned the raw stored bytes without UTF-8-decoding them, producing mojibake for any object holding real text beyond ASCII — the same gap `__sbRawBodyResponse` already closed for assets/fetch/service-binding responses. Same fix here (lazy, memoized `__sbFromUtf8` decode).
38
+
39
+ Verified end-to-end against both transports with a real binary upload (`curl --data-binary`, not a synthetic string): stored bytes and `.body`/`.text()` now round-trip byte-for-byte/correctly for binary content, Latin-1 text, and multi-byte Unicode (CJK). Full kitchen-sink conformance stays green on both transports (24/24 standalone, 27/27 broker).
40
+
41
+ **Known remaining gap, not fixed here:** a genuine astral-plane character (an emoji, anything outside the Basic Multilingual Plane, encoded as a UTF-16 surrogate pair) still encodes to the wrong number of UTF-8 bytes on write — confirmed identical before and after this fix, so it's a separate, deeper bug in Porffor's own UTF-16→UTF-8 encoder (the `porf_native_fetch_read_value` "string" branch), not something in sproutboat's own code. Filing separately if it recurs; out of scope for this fix.
42
+
3
43
  ## 0.6.6
4
44
 
5
45
  ### Patch Changes
package/package.json CHANGED
@@ -1,8 +1,13 @@
1
1
  {
2
2
  "name": "@sproutboat/runtime",
3
- "version": "0.6.6",
3
+ "version": "0.7.1",
4
4
  "license": "MIT",
5
5
  "type": "module",
6
+ "repository": {
7
+ "type": "git",
8
+ "url": "git+https://github.com/baronunread/sproutboat-packages.git",
9
+ "directory": "packages/runtime"
10
+ },
6
11
  "main": "./src/index.ts",
7
12
  "types": "./src/index.ts",
8
13
  "scripts": {
@@ -1104,11 +1104,33 @@ function __sbR2Object(meta, body) {
1104
1104
  };
1105
1105
  if (body != null) {
1106
1106
  obj.body = body;
1107
+ // #184 — `body` is raw bytes (a bytestring), not decoded text: `.text()`
1108
+ // returning it verbatim produced mojibake for any object holding real
1109
+ // UTF-8 content, the same gap __sbRawBodyResponse already closed for
1110
+ // assets/fetch/service-binding responses. Same fix here: decode lazily,
1111
+ // once, on first read.
1112
+ let decoded = null;
1107
1113
  obj.text = function () {
1108
- return body;
1114
+ if (decoded === null) decoded = __sbFromUtf8(body);
1115
+ return decoded;
1109
1116
  };
1110
1117
  obj.json = function () {
1111
- return JSON.parse(body);
1118
+ return JSON.parse(obj.text());
1119
+ };
1120
+ // #177 — `new Response(obj.body)` UTF-8-re-encodes the raw bytes (they're
1121
+ // a bytestring, same ambiguity #172/#176 exist to resolve): any byte
1122
+ // 0x80-0xFF doubles into two bytes, corrupting binary content on the way
1123
+ // out. `toResponse()` is the fix already used for assets/fetch/service
1124
+ // bindings (`__sbRawBodyResponse`, below) — same reserved x-sb-raw-body
1125
+ // marker, applied here too.
1126
+ obj.toResponse = function (init) {
1127
+ const opts = init || {};
1128
+ const headers = new Headers(opts.headers || {});
1129
+ headers.set("x-sb-raw-body", "1");
1130
+ if (!headers.has("content-type") && obj.httpMetadata.contentType)
1131
+ headers.set("content-type", obj.httpMetadata.contentType);
1132
+ if (!headers.has("etag")) headers.set("etag", obj.httpEtag);
1133
+ return __sbRawBodyResponse(body, { status: opts.status, headers });
1112
1134
  };
1113
1135
  }
1114
1136
  return obj;
@@ -0,0 +1,65 @@
1
+ // sproutboat #177 — `new Response(obj.body)` UTF-8-re-encodes an R2 object's
2
+ // raw bytes (they're a bytestring, indistinguishable from Latin-1 text at
3
+ // that layer). `obj.toResponse()` is the fix: same reserved x-sb-raw-body
4
+ // marker __sbRawBodyResponse already uses for assets/fetch/service bindings.
5
+ // The prelude is prepended as text and cannot be imported, so this test lifts
6
+ // the relevant functions out and exercises them directly, against real
7
+ // Response/Headers (both are Bun globals). If any of these move or their
8
+ // boundary markers change, this fails loudly rather than silently testing
9
+ // nothing.
10
+ import { expect, test } from "bun:test";
11
+ import { readFileSync } from "node:fs";
12
+ import { fileURLToPath } from "node:url";
13
+
14
+ const src = readFileSync(fileURLToPath(new URL("./native-fetch-prelude.js", import.meta.url)), "utf8");
15
+ const utf8Start = src.indexOf("function __sbFromUtf8(");
16
+ const rawStart = src.indexOf("function __sbRawBodyResponse(");
17
+ const utf8End = rawStart;
18
+ const rawEnd = src.indexOf("function __sbEqCt(");
19
+ const r2Start = src.indexOf("function __sbR2Object(");
20
+ const r2End = src.indexOf("// Installed only when the project declares bindings.");
21
+ if (utf8Start === -1 || utf8End === -1) throw new Error("__sbFromUtf8 block not found in prelude");
22
+ if (rawStart === -1 || rawEnd === -1) throw new Error("__sbRawBodyResponse block not found in prelude");
23
+ if (r2Start === -1 || r2End === -1 || r2End < r2Start) throw new Error("__sbR2Object block not found in prelude");
24
+
25
+ // oxlint-disable-next-line no-function-constructor -- lifting pure blocks out of the text-only prelude for test
26
+ const load = new Function(
27
+ `${src.slice(utf8Start, utf8End)}\n${src.slice(rawStart, rawEnd)}\n${src.slice(r2Start, r2End)}\nreturn { __sbR2Object };`,
28
+ );
29
+ const { __sbR2Object } = load();
30
+
31
+ const meta = { key: "f.bin", size: 3, etag: "abc123", uploaded: "2026-09-14T00:00:00.000Z" };
32
+
33
+ // The byte-for-byte round trip itself happens in the compiled runtime's
34
+ // write_response_value (#176's C-side scan, packages/toolchain/src/patch.ts) --
35
+ // a plain-JS Response, Bun's included, re-encodes a bytestring the same way
36
+ // Porffor's does, so this pure-JS test can only prove toResponse() sets the
37
+ // marker that steers that C-side read; the actual byte preservation was
38
+ // verified manually against a real compiled standalone binary (#177).
39
+ test("toResponse() tags the response with the runtime's raw-body marker", () => {
40
+ const bytes = [0x80, 0x81, 0xff].map((b) => String.fromCharCode(b)).join("");
41
+ const obj = __sbR2Object(meta, bytes);
42
+ const fixed = obj.toResponse();
43
+ expect(fixed.headers.get("x-sb-raw-body")).toBe("1");
44
+ });
45
+
46
+ test("toResponse() defaults content-type from httpMetadata and etag from httpEtag", () => {
47
+ const obj = __sbR2Object({ ...meta, httpMetadata: { contentType: "image/png" } }, "x");
48
+ const resp = obj.toResponse();
49
+ expect(resp.headers.get("content-type")).toBe("image/png");
50
+ expect(resp.headers.get("etag")).toBe(obj.httpEtag);
51
+ });
52
+
53
+ test("toResponse() lets caller-supplied headers/status override the defaults", () => {
54
+ const obj = __sbR2Object({ ...meta, httpMetadata: { contentType: "image/png" } }, "x");
55
+ const resp = obj.toResponse({ status: 206, headers: { "content-type": "application/octet-stream" } });
56
+ expect(resp.status).toBe(206);
57
+ expect(resp.headers.get("content-type")).toBe("application/octet-stream");
58
+ });
59
+
60
+ test("toResponse() on an empty body still returns bytes, not the string \"\"", async () => {
61
+ const obj = __sbR2Object(meta, "");
62
+ const resp = obj.toResponse();
63
+ const buf = new Uint8Array(await resp.arrayBuffer());
64
+ expect(buf.length).toBe(0);
65
+ });
@@ -19,6 +19,13 @@
19
19
 
20
20
  // oxlint-disable-next-line no-unused-expressions -- Porffor.c`...` is inline C the compiler consumes, not a JS expression.
21
21
  Porffor.c`
22
+ // baronunread/sproutboat#176 — defined in render.js, alongside
23
+ // porf_native_fetch_read_value (which every inline-C block below already
24
+ // calls with no forward declaration of its own — that one gets one from
25
+ // Porffor's own generated prototypes, this one doesn't, since it's a local
26
+ // patch addition, not something Porffor knows to prototype early).
27
+ extern int porf_native_fetch_read_raw_bytes(jsval value, const char** out_buf, size_t* out_len);
28
+
22
29
  static int sb_io_all(int fd, unsigned char* buf, size_t len, int writing) {
23
30
  size_t done = 0;
24
31
  while (done < len) {
@@ -237,8 +244,15 @@ function __sbCallBin(reqJson, body) {
237
244
  Porffor.c`
238
245
  const char* __j; size_t __jl; char* __jo = 0;
239
246
  porf_native_fetch_read_value(reqJson, &__j, &__jl, &__jo);
247
+ // baronunread/sproutboat#184 -- the only real (non-empty) body this ever
248
+ // carries today is an R2 put(); read raw, not through read_value's
249
+ // bytestring branch, which always UTF-8-encodes (right for text, wrong
250
+ // for opaque bytes off a raw HTTP upload). A genuine multi-byte JS string
251
+ // isn't a bytestring, so it falls back to the encoding path unaffected.
240
252
  const char* __b; size_t __bl; char* __bo = 0;
241
- porf_native_fetch_read_value(body, &__b, &__bl, &__bo);
253
+ if (porf_native_fetch_read_raw_bytes(body, &__b, &__bl) != 0) {
254
+ porf_native_fetch_read_value(body, &__b, &__bl, &__bo);
255
+ }
242
256
  char* __resp = 0; size_t __resplen = 0;
243
257
  int __rc = sb_broker_roundtrip_v1(__j, __jl, __b, __bl, &__resp, &__resplen);
244
258
  if (__jo) free(__jo);
@@ -53,6 +53,13 @@ extern int64_t sqlite3_last_insert_rowid(sqlite3*);
53
53
  extern const char* sqlite3_errmsg(sqlite3*);
54
54
  extern int sqlite3_close(sqlite3*);
55
55
 
56
+ // baronunread/sproutboat#176 — defined in render.js, alongside
57
+ // porf_native_fetch_read_value (which every inline-C block below already
58
+ // calls with no forward declaration of its own — that one gets one from
59
+ // Porffor's own generated prototypes, this one doesn't, since it's a local
60
+ // patch addition, not something Porffor knows to prototype early).
61
+ extern int porf_native_fetch_read_raw_bytes(jsval value, const char** out_buf, size_t* out_len);
62
+
56
63
  #define SB_SQLITE_ROW 100
57
64
  #define SB_SQLITE_DONE 101
58
65
  #define SB_SQLITE_TRANSIENT ((void*)-1)
@@ -240,11 +247,14 @@ static int sb_bind_params(sqlite3_stmt* st, const char* json) {
240
247
  return 0;
241
248
  }
242
249
 
243
- // --- R2 bodies, without the JSON detour -------------------------------------
244
- // An object's bytes never enter a JSON frame: they arrive as their own string
245
- // parameter and go straight into a BLOB column. That matters for size as much
246
- // as correctness — escaping a binary body inflates it several times over, and
247
- // the arena grows to fit the biggest thing it ever had to hold.
250
+ // --- R2 bodies, without the JSON detour or the SQLite detour ----------------
251
+ // An object's bytes never enter a JSON frame (they arrive as their own string
252
+ // parameter) and never enter SQLite either (baronunread/sproutboat#56): they
253
+ // go straight to their own file under "<data-dir>/r2-blobs/", named by a hash
254
+ // of bucket+key so unicode keys and directory traversal are both non-issues.
255
+ // Doubling every object through SQLite's own page cache on top of the buffer
256
+ // it already arrived in was the real cost behind #56's "large uploads make
257
+ // memory skyrocket" complaint, not just the JSON-escaping #63 already fixed.
248
258
 
249
259
  static void sb_hex32(const unsigned char* in, char* out) {
250
260
  static const char* d = "0123456789abcdef";
@@ -269,7 +279,24 @@ static void sb_iso_now(char* out, size_t cap) {
269
279
  strftime(out, cap, "%Y-%m-%dT%H:%M:%S.000Z", &g);
270
280
  }
271
281
 
272
- // Returns malloc'd JSON metadata; the body itself is never serialised.
282
+ // "<dir-of-store_path>/r2-blobs/<sha256(bucket 0x1e key)>.blob" — deterministic,
283
+ // so no extra column is needed to find an object's file back.
284
+ static void sb_r2_blob_path(char* out, size_t cap, const char* store_path, const char* bucket, const char* key) {
285
+ const char* slash = strrchr(store_path, '/');
286
+ int dirlen = slash ? (int)(slash - store_path) : 0;
287
+ br_sha256_context ctx;
288
+ br_sha256_init(&ctx);
289
+ br_sha256_update(&ctx, bucket, strlen(bucket));
290
+ br_sha256_update(&ctx, "\x1e", 1); // separator: a bucket/key split never collides across it
291
+ br_sha256_update(&ctx, key, strlen(key));
292
+ unsigned char digest[32];
293
+ br_sha256_out(&ctx, digest);
294
+ char hex[65];
295
+ sb_hex32(digest, hex);
296
+ snprintf(out, cap, "%.*s/r2-blobs/%s.blob", dirlen, store_path, hex);
297
+ }
298
+
299
+ // Returns malloc'd JSON metadata; the body itself is never serialised into it.
273
300
  static char* sb_r2_put_c(const char* path, const char* bucket, const char* key, const char* body, size_t bodylen,
274
301
  const char* http_json, const char* custom_json) {
275
302
  sb_buf out = { 0, 0, 0 };
@@ -282,10 +309,19 @@ static char* sb_r2_put_c(const char* path, const char* bucket, const char* key,
282
309
  char uploaded[40];
283
310
  sb_iso_now(uploaded, sizeof(uploaded));
284
311
 
312
+ char blobpath[1024];
313
+ sb_r2_blob_path(blobpath, sizeof(blobpath), path, bucket, key);
314
+ sb_mkdirs(blobpath);
315
+ FILE* f = fopen(blobpath, "wb");
316
+ if (!f) { sb_puts(&out, "{\"ok\":false,\"error\":\"cannot write object\"}"); return out.p; }
317
+ size_t written = bodylen ? fwrite(body, 1, bodylen, f) : 0;
318
+ fclose(f);
319
+ if (written != bodylen) { sb_puts(&out, "{\"ok\":false,\"error\":\"short write\"}"); return out.p; }
320
+
285
321
  sqlite3_stmt* st = 0;
286
322
  const char* sql =
287
- "INSERT INTO r2 (bucket, key, body, size, etag, uploaded, http_json, custom_json) VALUES (?1,?2,?3,?4,?5,?6,?7,?8) "
288
- "ON CONFLICT (bucket, key) DO UPDATE SET body=?3, size=?4, etag=?5, uploaded=?6, http_json=?7, custom_json=?8";
323
+ "INSERT INTO r2 (bucket, key, size, etag, uploaded, http_json, custom_json) VALUES (?1,?2,?3,?4,?5,?6,?7) "
324
+ "ON CONFLICT (bucket, key) DO UPDATE SET size=?3, etag=?4, uploaded=?5, http_json=?6, custom_json=?7";
289
325
  if (sqlite3_prepare_v2(db, sql, -1, &st, 0) != 0 || !st) {
290
326
  sb_puts(&out, "{\"ok\":false,\"error\":");
291
327
  const char* m = sqlite3_errmsg(db);
@@ -295,12 +331,11 @@ static char* sb_r2_put_c(const char* path, const char* bucket, const char* key,
295
331
  }
296
332
  sqlite3_bind_text(st, 1, bucket, -1, SB_SQLITE_TRANSIENT);
297
333
  sqlite3_bind_text(st, 2, key, -1, SB_SQLITE_TRANSIENT);
298
- sqlite3_bind_blob(st, 3, body, (int)bodylen, SB_SQLITE_STATIC);
299
- sqlite3_bind_int64(st, 4, (int64_t)bodylen);
300
- sqlite3_bind_text(st, 5, etag, -1, SB_SQLITE_TRANSIENT);
301
- sqlite3_bind_text(st, 6, uploaded, -1, SB_SQLITE_TRANSIENT);
302
- sqlite3_bind_text(st, 7, http_json && *http_json ? http_json : "{}", -1, SB_SQLITE_TRANSIENT);
303
- sqlite3_bind_text(st, 8, custom_json && *custom_json ? custom_json : "{}", -1, SB_SQLITE_TRANSIENT);
334
+ sqlite3_bind_int64(st, 3, (int64_t)bodylen);
335
+ sqlite3_bind_text(st, 4, etag, -1, SB_SQLITE_TRANSIENT);
336
+ sqlite3_bind_text(st, 5, uploaded, -1, SB_SQLITE_TRANSIENT);
337
+ sqlite3_bind_text(st, 6, http_json && *http_json ? http_json : "{}", -1, SB_SQLITE_TRANSIENT);
338
+ sqlite3_bind_text(st, 7, custom_json && *custom_json ? custom_json : "{}", -1, SB_SQLITE_TRANSIENT);
304
339
  int rc = sqlite3_step(st);
305
340
  sqlite3_finalize(st);
306
341
  if (rc != SB_SQLITE_DONE) { sb_puts(&out, "{\"ok\":false,\"error\":\"r2 put failed\"}"); return out.p; }
@@ -311,30 +346,50 @@ static char* sb_r2_put_c(const char* path, const char* bucket, const char* key,
311
346
  return out.p;
312
347
  }
313
348
 
314
- // Body bytes straight into a Porffor bytestring.
315
- //
316
- // The allocation happens while the statement is still open, so sqlite's own
317
- // buffer is the source and there is no intermediate copy: one 8 MB object means
318
- // one 8 MB allocation, not two. Returns the bytestring pointer, or 0.
349
+ // Reads the object's file into a Porffor bytestring. One malloc (the file
350
+ // read) plus the bytestring's own allocation — no SQLite page-cache copy on
351
+ // top, and no growing single database file holding every object ever put.
352
+ // Returns the bytestring pointer, or 0.
319
353
  static u32 sb_r2_get_c(const char* path, const char* bucket, const char* key, int* found) {
320
354
  *found = 0;
321
355
  int idx = sb_db_for(path);
322
356
  if (idx < 0) return 0;
323
357
  sqlite3_stmt* st = 0;
324
- if (sqlite3_prepare_v2(sb_dbs[idx], "SELECT body FROM r2 WHERE bucket = ? AND key = ?", -1, &st, 0) != 0 || !st) return 0;
358
+ if (sqlite3_prepare_v2(sb_dbs[idx], "SELECT 1 FROM r2 WHERE bucket = ? AND key = ?", -1, &st, 0) != 0 || !st) return 0;
325
359
  sqlite3_bind_text(st, 1, bucket, -1, SB_SQLITE_TRANSIENT);
326
360
  sqlite3_bind_text(st, 2, key, -1, SB_SQLITE_TRANSIENT);
327
- u32 out = 0;
328
- if (sqlite3_step(st) == SB_SQLITE_ROW) {
329
- int n = sqlite3_column_bytes(st, 0);
330
- const void* blob = sqlite3_column_blob(st, 0);
331
- out = porf_native_fetch_alloc_bytestring((const char*)(blob ? blob : ""), (size_t)(n > 0 ? n : 0));
332
- *found = 1;
333
- }
361
+ int has_row = sqlite3_step(st) == SB_SQLITE_ROW;
334
362
  sqlite3_finalize(st);
363
+ if (!has_row) return 0;
364
+
365
+ char blobpath[1024];
366
+ sb_r2_blob_path(blobpath, sizeof(blobpath), path, bucket, key);
367
+ FILE* f = fopen(blobpath, "rb");
368
+ // A metadata row with no blob file (deleted out from under it, disk issue)
369
+ // reads as not-found rather than crashing.
370
+ if (!f) return 0;
371
+ fseek(f, 0, SEEK_END);
372
+ long size = ftell(f);
373
+ rewind(f);
374
+ if (size < 0) { fclose(f); return 0; }
375
+ char* buf = (char*)malloc((size_t)size > 0 ? (size_t)size : 1);
376
+ size_t rd = size > 0 ? fread(buf, 1, (size_t)size, f) : 0;
377
+ fclose(f);
378
+ if (!buf || rd != (size_t)size) { free(buf); return 0; }
379
+ u32 out = porf_native_fetch_alloc_bytestring(buf, (size_t)size);
380
+ free(buf);
381
+ *found = 1;
335
382
  return out;
336
383
  }
337
384
 
385
+ // Best-effort: a metadata row deleted with no blob file (or vice versa) is
386
+ // still a correct end state, so a missing file here is not an error.
387
+ static void sb_r2_delete_blob(const char* path, const char* bucket, const char* key) {
388
+ char blobpath[1024];
389
+ sb_r2_blob_path(blobpath, sizeof(blobpath), path, bucket, key);
390
+ unlink(blobpath);
391
+ }
392
+
338
393
  // Run a script: one or more statements separated by semicolons. sqlite3_exec
339
394
  // handles the whole string, which prepare/step does not — it compiles the first
340
395
  // statement and silently ignores the rest, so a schema built from one exec call
@@ -929,10 +984,18 @@ function __sbR2PutRaw(path, bucket, key, body, httpJson, customJson) {
929
984
  char* __custom = (char*)malloc(__cl + 1); memcpy(__custom, __c, __cl); __custom[__cl] = 0;
930
985
  if (__co) free(__co);
931
986
 
932
- // The body is read in place and handed straight to sqlite3_bind_blob: no
933
- // copy beyond what the binding needs, and no escaping at all.
987
+ // baronunread/sproutboat#184 -- read raw, not through porf_native_fetch_
988
+ // read_value's bytestring branch, which always UTF-8-encodes (correct for
989
+ // genuine text, wrong for opaque bytes -- request.body off a raw HTTP
990
+ // upload is exactly that). R2 is object storage: put()/get() need to be
991
+ // byte-preserving, not text-preserving. Only a bytestring-typed value can
992
+ // be read raw; a genuine multi-byte JS string (real Unicode, "string" type
993
+ // rather than "bytestring") isn't one, so it falls back to the normal
994
+ // encoding path, which already handles that type correctly.
934
995
  const char* __b; size_t __bl; char* __bo = 0;
935
- porf_native_fetch_read_value(body, &__b, &__bl, &__bo);
996
+ if (porf_native_fetch_read_raw_bytes(body, &__b, &__bl) != 0) {
997
+ porf_native_fetch_read_value(body, &__b, &__bl, &__bo);
998
+ }
936
999
  char* __out = sb_r2_put_c(__path, __bucket, __key, __b, __bl, __http, __custom);
937
1000
  if (__bo) free(__bo);
938
1001
 
@@ -974,6 +1037,32 @@ function __sbR2GetRaw(path, bucket, key) {
974
1037
  return res;
975
1038
  }
976
1039
 
1040
+ // R2 delete: also removes the object's own file, not just the metadata row
1041
+ // (#56 — sb_r2_delete_blob is best-effort, a missing file is not an error).
1042
+ // oxlint-disable-next-line no-unused-vars -- read inside the RawC block below, not by JS.
1043
+ function __sbR2DeleteRaw(path, bucket, key) {
1044
+ // oxlint-disable-next-line no-unused-expressions -- Porffor.c`...` is inline C the compiler consumes, not a JS expression.
1045
+ Porffor.c`
1046
+ const char* __p; size_t __pl; char* __po = 0;
1047
+ porf_native_fetch_read_value(path, &__p, &__pl, &__po);
1048
+ char* __path = (char*)malloc(__pl + 1); memcpy(__path, __p, __pl); __path[__pl] = 0;
1049
+ if (__po) free(__po);
1050
+
1051
+ const char* __bk; size_t __bkl; char* __bko = 0;
1052
+ porf_native_fetch_read_value(bucket, &__bk, &__bkl, &__bko);
1053
+ char* __bucket = (char*)malloc(__bkl + 1); memcpy(__bucket, __bk, __bkl); __bucket[__bkl] = 0;
1054
+ if (__bko) free(__bko);
1055
+
1056
+ const char* __k; size_t __kl; char* __ko = 0;
1057
+ porf_native_fetch_read_value(key, &__k, &__kl, &__ko);
1058
+ char* __key = (char*)malloc(__kl + 1); memcpy(__key, __k, __kl); __key[__kl] = 0;
1059
+ if (__ko) free(__ko);
1060
+
1061
+ sb_r2_delete_blob(__path, __bucket, __key);
1062
+ free(__path); free(__bucket); free(__key);
1063
+ `;
1064
+ }
1065
+
977
1066
  // Multi-statement exec. Same string-param pattern as __sbSqlRaw.
978
1067
  // oxlint-disable-next-line no-unused-vars -- read inside the RawC block below, not by JS.
979
1068
  function __sbSqlScriptRaw(path, sql) {
@@ -1158,9 +1247,11 @@ function __sbEnsureSchema() {
1158
1247
  s,
1159
1248
  "CREATE TABLE IF NOT EXISTS kv (ns TEXT NOT NULL, key TEXT NOT NULL, value TEXT NOT NULL, PRIMARY KEY (ns, key))",
1160
1249
  );
1250
+ // #56 — no `body` column: an object's bytes live in their own file under
1251
+ // "<data-dir>/r2-blobs/" (sb_r2_put_c/sb_r2_get_c), never in the sqlite row.
1161
1252
  __sbSql(
1162
1253
  s,
1163
- "CREATE TABLE IF NOT EXISTS r2 (bucket TEXT NOT NULL, key TEXT NOT NULL, body TEXT NOT NULL, size INTEGER NOT NULL, " +
1254
+ "CREATE TABLE IF NOT EXISTS r2 (bucket TEXT NOT NULL, key TEXT NOT NULL, size INTEGER NOT NULL, " +
1164
1255
  "etag TEXT NOT NULL, uploaded TEXT NOT NULL, http_json TEXT NOT NULL DEFAULT '{}', custom_json TEXT NOT NULL DEFAULT '{}', " +
1165
1256
  "PRIMARY KEY (bucket, key))",
1166
1257
  );
@@ -1350,38 +1441,33 @@ function __sbEmbeddedDispatch(msg) {
1350
1441
  return { ok: true, path: dest, bytes: reply.bytes };
1351
1442
  }
1352
1443
 
1444
+ // r2.put/get route through the same raw C path (__sbR2PutRaw/__sbR2GetRaw,
1445
+ // sb_r2_put_c/sb_r2_get_c) the R2 binding's own put()/get() use — reachable
1446
+ // separately only via a raw `__sbCall(op:"r2.put"/"r2.get")`, which nothing
1447
+ // in this codebase does today, but broker.ts's dispatch() answers the same
1448
+ // two ops, so the contract stays identical on both backends either way
1449
+ // (#56: neither one keeps object bytes in SQLite any more).
1353
1450
  if (op === "r2.put") {
1354
- const body = String(msg.body == null ? "" : msg.body);
1355
- const etag = __sbHex(16);
1356
- __sbSql(
1357
- store,
1358
- "INSERT INTO r2 (bucket, key, body, size, etag, uploaded, http_json, custom_json) VALUES (?1,?2,?3,?4,?5,?6,?7,?8) " +
1359
- "ON CONFLICT (bucket, key) DO UPDATE SET body=?3, size=?4, etag=?5, uploaded=?6, http_json=?7, custom_json=?8",
1360
- [
1361
- msg.bucket,
1362
- msg.key,
1363
- body,
1364
- body.length,
1365
- etag,
1366
- new Date().toISOString(),
1451
+ const reply = JSON.parse(
1452
+ __sbR2PutRaw(
1453
+ store,
1454
+ String(msg.bucket),
1455
+ String(msg.key),
1456
+ String(msg.body == null ? "" : msg.body),
1367
1457
  JSON.stringify(msg.httpMetadata || {}),
1368
1458
  JSON.stringify(msg.customMetadata || {}),
1369
- ],
1459
+ ),
1370
1460
  );
1371
- return {
1372
- ok: true,
1373
- object: { key: msg.key, size: body.length, etag, uploaded: new Date().toISOString() },
1374
- };
1461
+ if (reply.ok === false) throw new Error("r2 put: " + reply.error);
1462
+ return { ok: true, object: { key: msg.key, size: reply.size, etag: reply.etag, uploaded: reply.uploaded } };
1375
1463
  }
1376
1464
  if (op === "r2.get" || op === "r2.head") {
1377
- // head must not select `body`: reading an 8 MB blob only to drop it costs
1378
- // the read, the text conversion, and a full JSON escape of the row.
1465
+ // head must not read the blob file: reading an 8 MB object only to drop
1466
+ // it costs the read and a full JSON escape for nothing.
1379
1467
  const wantsBody = op === "r2.get";
1380
1468
  const r = __sbSql(
1381
1469
  store,
1382
- wantsBody
1383
- ? "SELECT body, size, etag, uploaded, http_json, custom_json FROM r2 WHERE bucket = ? AND key = ?"
1384
- : "SELECT '', size, etag, uploaded, http_json, custom_json FROM r2 WHERE bucket = ? AND key = ?",
1470
+ "SELECT size, etag, uploaded, http_json, custom_json FROM r2 WHERE bucket = ? AND key = ?",
1385
1471
  [msg.bucket, msg.key],
1386
1472
  );
1387
1473
  if (!r.rows.length) return { ok: true, found: false };
@@ -1392,17 +1478,18 @@ function __sbEmbeddedDispatch(msg) {
1392
1478
  found: true,
1393
1479
  object: {
1394
1480
  key: msg.key,
1395
- size: Number(row[1]),
1396
- etag: row[2],
1397
- uploaded: row[3],
1398
- httpMetadata: JSON.parse(row[4] || "{}"),
1399
- customMetadata: JSON.parse(row[5] || "{}"),
1481
+ size: Number(row[0]),
1482
+ etag: row[1],
1483
+ uploaded: row[2],
1484
+ httpMetadata: JSON.parse(row[3] || "{}"),
1485
+ customMetadata: JSON.parse(row[4] || "{}"),
1400
1486
  },
1401
- body: wantsBody ? row[0] : undefined,
1487
+ body: wantsBody ? __sbR2GetRaw(store, String(msg.bucket), String(msg.key)) : undefined,
1402
1488
  };
1403
1489
  }
1404
1490
  if (op === "r2.delete") {
1405
1491
  __sbSql(store, "DELETE FROM r2 WHERE bucket = ? AND key = ?", [msg.bucket, msg.key]);
1492
+ __sbR2DeleteRaw(store, String(msg.bucket), String(msg.key));
1406
1493
  return { ok: true };
1407
1494
  }
1408
1495
  if (op === "r2.list") {