@sproutboat/runtime 0.5.0 → 0.6.1

package/CHANGELOG.md

@@ -1,5 +1,87 @@
 # @sproutboat/runtime
 
+## 0.6.1
+
+### Patch Changes
+
+- 806c21e: Fix (baronunread/sproutboat#176): #172's bytestring UTF-8 encoding fix broke
+  the assets binding, which reads a file's bytes off disk into a `bytestring`
+  specifically so they pass through the wire untouched — a `bytestring`
+  carrying real text a handler built and one carrying opaque file bytes are the
+  same type with no way to tell them apart, so #172's blanket encoding also
+  re-encoded already-finished bytes, corrupting any binary asset (and any
+  proxied `fetch()`/service-binding response body).
+  
+  Adds a second, dedicated `porf_native_fetch_read_raw_bytes` (zero-copy,
+  bytestring-only, no encoding, ever) alongside the untouched original —
+  `porf_native_fetch_read_value` is called from ~30 places across sproutboat's
+  own inline C, all genuine text, so its signature and behavior stay exactly as
+  #172 left them. `write_response_value` picks between the two based on a new
+  reserved `x-sb-raw-body` response header (same pattern as #163's
+  `x-sb-remote-addr`), which the assets binding, outbound `fetch()`, and
+  service-binding `fetch()` now set — the three places sproutboat's own
+  prelude constructs a Response from wire bytes rather than a string a handler
+  built. Verified end-to-end: a 256-byte all-values fixture now round-trips
+  byte-for-byte through a real standalone build's assets binding, and #172's
+  original UTF-8 fix still holds for genuine text.
+  
+  Not fixed here: `env.<R2>.get()`'s `.body` is exposed directly to handler
+  code, which then constructs its own `new Response(obj.body)` — no
+  sproutboat-owned call site to attach the marker to, so R2 binary objects
+  served this way carry the same corruption. Needs either a real binary body
+  type upstream in Porffor or a different API shape; tracked as a known gap on
+  #176, not resolved by this fix.
+
+## 0.6.0
+
+### Minor Changes
+
+- ae8c7a8: `ctx.waitUntil` (baronunread/sproutboat#57): `fetch(request, ctx)` now takes a
+  second argument with `waitUntil(promise)`. Registered promises are drained,
+  in-process and sequentially, after the handler returns and before the turn
+  completes, capped at 25s for the whole batch — a task still running past that
+  keeps running, but stops holding up the response. A rejected task is
+  swallowed rather than failing the response.
+  
+  `DurableObjectState.waitUntil` was a silent no-op stub; it now actually queues
+  and drains the same way, scoped to the instance. `alarm()` also used to be
+  fire-and-forget (its promise and any `state.waitUntil()` it queued were
+  dropped the moment the 204 went out) — both are now awaited.
+  
+  Additive: a handler that ignores the second `fetch` argument is unaffected.
+- ae8c7a8: `ctx.waitUntil` for `scheduled` and `queue` (baronunread/sproutboat#171):
+  both now take the same second `ctx` argument as `fetch`, drained the same way.
+  
+  Also fixes a real bug found while wiring it up: `scheduled` and `queue` were
+  fire-and-forget regardless of `ctx` — an async handler's own promise was
+  dropped the instant the reply went out. For `queue` specifically, the default
+  "unhandled messages are acked" pass ran synchronously right after the
+  (discarded) call to the handler, so `ack()`/`retry()` calls made after the
+  handler's first `await` never reached the response. Both are now properly
+  awaited, on every delivery path: broker-dispatched, and the embedded
+  standalone binary's own local cron/queue/DO-alarm timers (the latter had the
+  same gap for `DurableObjectState.waitUntil` from #57 — that local path
+  bypassed the fix there entirely).
+  
+  Additive: a handler that ignores the second argument is unaffected.
+
+### Patch Changes
+
+- 5b76f6b: Fix (baronunread/sproutboat#132): the banned-API capability check matched a bare
+  identifier (`\bprocess\b`), so a locally-declared `function process()` — zod v4
+  declares exactly that — failed the check as readily as a real read of the Node
+  global. Now requires a member access (`process.env`, no whitespace around the
+  `.`, to avoid matching a sentence like "...unique to this process. The...") or
+  `new Buffer(...)`; `node:` now only matches as the start of a quoted string
+  (specifier-shaped), not as a substring anywhere. A handler that imports zod (or
+  anything built on it, e.g. better-auth) and only uses APIs the compiler
+  otherwise supports now passes the capability check and builds.
+  
+  Not fixed here: zod still throws an uncaught `TypeError` at runtime on
+  `.safeParse()` even for the simplest schema (`z.string()`) — a separate,
+  deeper Porffor compatibility gap this change does not touch. This fix removes
+  an incorrect early rejection; it does not make zod usable end to end.
+
 ## 0.5.0
 
 ### Minor Changes

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@sproutboat/runtime",
-  "version": "0.5.0",
+  "version": "0.6.1",
   "license": "MIT",
   "type": "module",
   "main": "./src/index.ts",

package/src/native-fetch-prelude.js

@@ -1140,6 +1140,11 @@ globalThis.__sbInstallBindings = function (target, bindings) {
         const headers = {};
         if (r.type) headers["content-type"] = r.type;
         if (r.found) headers["etag"] = '"' + r.hash + '"';
+        // #176: these bytes came off disk already-finished (UTF-8 text or
+        // binary, doesn't matter which) -- the reserved x-sb-raw-body header
+        // tells the C write path to pass them through untouched instead of
+        // re-encoding as if this were a JS string a handler built.
+        if (r.body != null) headers["x-sb-raw-body"] = "1";
         return new Response(r.body == null ? "" : r.body, { status: r.status || (r.found ? 200 : 404), headers });
       },
     };
@@ -1168,6 +1173,10 @@ globalThis.__sbInstallBindings = function (target, bindings) {
         });
         const respHeaders = new Headers();
         for (let j = 0; j < (r.headers || []).length; j++) respHeaders.set(r.headers[j][0], r.headers[j][1]);
+        // #176: wire bytes from another deployment, not a string this handler
+        // built -- must not be re-encoded if the handler proxies it straight
+        // through (see the assets binding for the same reasoning).
+        if (r.body != null) respHeaders.set("x-sb-raw-body", "1");
         return new Response(r.body == null ? "" : r.body, { status: r.status || 502, headers: respHeaders });
       },
     };
@@ -1190,6 +1199,10 @@ globalThis.__sbInstallBindings = function (target, bindings) {
       });
       const respHeaders = new Headers();
       for (let j = 0; j < (r.headers || []).length; j++) respHeaders.set(r.headers[j][0], r.headers[j][1]);
+      // #176: bytes from an outbound HTTP response, not a string this handler
+      // built -- must not be re-encoded if the handler proxies it straight
+      // through (see the assets binding for the same reasoning).
+      if (r.body != null) respHeaders.set("x-sb-raw-body", "1");
       return new Response(r.body == null ? "" : r.body, { status: r.status || 502, headers: respHeaders });
     };
   }
@@ -1248,7 +1261,10 @@ function __sbMakeDONamespace(binding, className) {
             req = new Request(url, { method: opts.method || "GET", headers });
             if (opts.body != null) req.body = String(opts.body);
           }
-          return __sbGetDOInstance(className, idStr).fetch(req);
+          const inst = __sbGetDOInstance(className, idStr);
+          const res = inst.fetch(req);
+          if (inst.__sbTasks.length || (res && __sbIsFn(res.then))) return __sbFetchWithDrain(res, inst.__sbTasks);
+          return res;
         },
       };
     },
@@ -1267,6 +1283,11 @@ function __sbGetDOInstance(cls, id) {
   const cacheKey = cls + " " + id;
   let inst = __sbDOInstances[cacheKey];
   if (!inst) {
+    // #57 — was a no-op: a DO calling `state.waitUntil(p)` had `p` vanish with
+    // no error. The instance outlives any one call (cached in
+    // `__sbDOInstances`), so the queue lives on `state` too and is drained by
+    // whichever call site invoked `fetch`/`alarm` this turn.
+    const __sbTasks = [];
     const state = {
       id: {
         toString() {
@@ -1277,9 +1298,12 @@ function __sbGetDOInstance(cls, id) {
       blockConcurrencyWhile(fn) {
         return fn();
       },
-      waitUntil() {},
+      waitUntil(p) {
+        if (p && __sbIsFn(p.then)) __sbTasks.push(p);
+      },
     };
     inst = new Ctor(state, globalThis.env);
+    inst.__sbTasks = __sbTasks;
     __sbDOInstances[cacheKey] = inst;
   }
   return inst;
@@ -1488,6 +1512,71 @@ function __sbClientIp(request) {
   return peer;
 }
 
+// #57 — ctx.waitUntil: work that must run before the turn completes but must
+// not block the response the handler already built. `waitUntil(p)` just
+// records `p`; nothing awaits it until the handler itself has returned.
+//
+// The wall-clock cap #25 was meant to gate this on turned out not to apply:
+// #25 was per-tenant abuse control (rate limits, cgroup isolation against a
+// hostile *other* caller) and was closed as not-planned — this deployment
+// model is single-admin, nobody else's handler to isolate from. A drain that
+// never lets go of the worker slot is a local bug either way, so it gets its
+// own timeout rather than waiting on that. Kept under the edge's own
+// `SPROUTBOAT_REQUEST_TIMEOUT_MS` (30s default, services/edge/src/main.ts) so
+// the sprout still gets to answer before the edge gives up on it.
+//
+// ponytail: drained sequentially, one timeout for the whole batch (not one
+// per task — N tasks should not cost N timeouts). A task still running past
+// the cap keeps running; nothing here can cancel a promise it didn't create.
+// No Promise.all (unproven in this runtime, sequential is fine at this
+// volume). A rejected task is swallowed rather than failing the response:
+// dropping a background task's error beats dropping the response it doesn't
+// own.
+const __SB_WAITUNTIL_TIMEOUT_MS = 25000;
+
+function __sbTimeout(ms) {
+  return new Promise((resolve) => setTimeout(resolve, ms));
+}
+
+async function __sbDrainWaitUntil(tasks) {
+  if (!tasks.length) return;
+  const pending = tasks.splice(0, tasks.length);
+  await Promise.race([__sbDrainAll(pending), __sbTimeout(__SB_WAITUNTIL_TIMEOUT_MS)]);
+}
+
+async function __sbDrainAll(pending) {
+  for (let i = 0; i < pending.length; i++) {
+    try {
+      await pending[i];
+    } catch {
+      /* a background task's rejection must not fail the response */
+    }
+  }
+}
+
+// Tail-called from `__sbEntry`/DO dispatch so the promise this returns is the
+// one *this* async function creates, not a `.then()` derived from someone
+// else's — see the note below on why that distinction matters here.
+async function __sbFetchWithDrain(res, tasks) {
+  const r = res && __sbIsFn(res.then) ? await res : res;
+  await __sbDrainWaitUntil(tasks);
+  return r;
+}
+
+// Shared by alarm and scheduled: both reply with an empty 204 once their
+// handler (and anything it queued via `waitUntil`) has settled.
+async function __sb204WithDrain(res, tasks) {
+  if (res && __sbIsFn(res.then)) await res;
+  await __sbDrainWaitUntil(tasks);
+  return new Response("", { status: 204 });
+}
+
+async function __sbQueueWithDrain(result, tasks) {
+  const r = result && __sbIsFn(result.then) ? await result : result;
+  await __sbDrainWaitUntil(tasks);
+  return new Response(JSON.stringify(r), { headers: { "content-type": "application/json" } });
+}
+
 globalThis.__sbEntry = function (handlers, request) {
   const trigger = request.headers.get("x-sb-trigger");
   if (!trigger) {
@@ -1506,8 +1595,16 @@ globalThis.__sbEntry = function (handlers, request) {
     // tag on hangs the request forever. cpuMs is documented as absent for
     // async handlers (see LogEvent in services/edge) — that is this.
     const __t0 = __sbCpuMs();
-    const __res = handlers.fetch(request);
-    if (__res && __sbIsFn(__res.then)) return __res;
+    const __sbTasks = [];
+    const ctx = {
+      waitUntil(p) {
+        if (p && __sbIsFn(p.then)) __sbTasks.push(p);
+      },
+    };
+    const __res = handlers.fetch(request, ctx);
+    // Same promise-identity rule as above applies to `__sbFetchWithDrain`'s
+    // own return, which is why it's tail-called rather than chained on here.
+    if (__sbTasks.length || (__res && __sbIsFn(__res.then))) return __sbFetchWithDrain(__res, __sbTasks);
     return __sbTagCpu(__res, __t0);
   }
   if (!__sbTriggerAuthed(request)) return new Response("forbidden", { status: 403 });
@@ -1515,21 +1612,50 @@ globalThis.__sbEntry = function (handlers, request) {
   if (trigger === "scheduled") {
     if (!__sbIsFn(handlers.scheduled)) return new Response("no scheduled handler", { status: 404 });
     const body = __sbReadJson(request);
-    handlers.scheduled({ cron: body.cron || "", scheduledTime: body.scheduledTime || Date.now(), noRetry() {} });
+    // #171 — ctx.waitUntil, same shape as fetch's. Was also fire-and-forget
+    // before this: an async `scheduled()`'s own promise was dropped the
+    // moment 204 went out, same bug alarm() had.
+    const __sbTasks = [];
+    const ctx = {
+      waitUntil(p) {
+        if (p && __sbIsFn(p.then)) __sbTasks.push(p);
+      },
+    };
+    const __sres = handlers.scheduled(
+      { cron: body.cron || "", scheduledTime: body.scheduledTime || Date.now(), noRetry() {} },
+      ctx,
+    );
+    if (__sbTasks.length || (__sres && __sbIsFn(__sres.then))) return __sb204WithDrain(__sres, __sbTasks);
     return new Response("", { status: 204 });
   }
 
   if (trigger === "queue") {
     if (!__sbIsFn(handlers.queue)) return new Response("no queue handler", { status: 404 });
-    const result = __sbRunQueueBatch(handlers, __sbReadJson(request));
-    return new Response(JSON.stringify(result), { headers: { "content-type": "application/json" } });
+    // #171 — same ctx.waitUntil shape. `__sbRunQueueBatch` was also
+    // fire-and-forget for an async `queue()`: it computed the default
+    // ack/retry and answered the broker before the handler's own awaits ran,
+    // so any ack()/retry() past the first `await` never made it into the
+    // response. It now awaits the handler itself when it returns a promise.
+    const __sbTasks = [];
+    const ctx = {
+      waitUntil(p) {
+        if (p && __sbIsFn(p.then)) __sbTasks.push(p);
+      },
+    };
+    // Always a promise now (`__sbRunQueueBatch` is async, to await the
+    // handler above), so always tail-call the drain wrapper — same
+    // promise-identity reasoning as `__sbFetchWithDrain`.
+    return __sbQueueWithDrain(__sbRunQueueBatch(handlers, __sbReadJson(request), ctx), __sbTasks);
   }
 
   if (trigger === "alarm") {
     const body = __sbReadJson(request);
     const inst = __sbGetDOInstance(String(body.cls || ""), String(body.id || ""));
     if (!__sbIsFn(inst.alarm)) return new Response("no alarm handler", { status: 404 });
-    inst.alarm();
+    const __ares = inst.alarm();
+    // Was fire-and-forget before: an async `alarm()` and any `state.waitUntil()`
+    // it queued were both dropped the moment 204 went out. Await both.
+    if (inst.__sbTasks.length || (__ares && __sbIsFn(__ares.then))) return __sb204WithDrain(__ares, inst.__sbTasks);
     return new Response("", { status: 204 });
   }
 
@@ -1543,7 +1669,7 @@ globalThis.__sbEntry = function (handlers, request) {
  * (deployed, and the phase-0 standalone launcher), or straight from the local
  * timer in an embedded binary that has no broker to be delivered from.
  */
-function __sbRunQueueBatch(handlers, body) {
+async function __sbRunQueueBatch(handlers, body, ctx) {
   const acked = [];
   const retried = [];
   const raw = body.messages || [];
@@ -1574,7 +1700,11 @@ function __sbRunQueueBatch(handlers, body) {
       for (let i = 0; i < messages.length; i++) messages[i].retry();
     },
   };
-  handlers.queue(batch);
+  const __qres = handlers.queue(batch, ctx);
+  // Was fire-and-forget: an async `queue()` had this default-ack pass run (and
+  // the response go out) before its own `await`s did, so any ack()/retry()
+  // past the first one never counted.
+  if (__qres && __sbIsFn(__qres.then)) await __qres;
   // default: any message neither acked nor retried is treated as acked
   for (let i = 0; i < messages.length; i++) {
     if (acked.indexOf(messages[i].id) === -1 && retried.indexOf(messages[i].id) === -1) acked.push(messages[i].id);

package/src/source.ts

@@ -11,7 +11,20 @@ const alwaysForbidden: Array<[RegExp, string]> = [
   [/\bimport\s*\(/, "dynamic import() is not supported: nothing can resolve it at build time"],
   [/\brequire\s*\(/, "CommonJS require is not supported"],
   [/\b(WebSocket|XMLHttpRequest)\s*\(/, "WebSocket / XMLHttpRequest are not supported"],
-  [/\b(process|Bun|Deno|Buffer|node:)\b/, "Node, Bun, and Deno APIs are not supported"],
+  // baronunread/sproutboat#132 — a bare identifier match flags a *local*
+  // `function process()` as readily as the global: zod v4 declares exactly
+  // that (its internal `process(schema, ctx)`), so importing zod alone used
+  // to fail this check pointing at code the handler author never wrote. A
+  // Node/Bun/Deno API is always reached through a member access or `new`;
+  // requiring that shape lets a local binding of the same name through.
+  // No whitespace is allowed around the `.`: real member access never has
+  // any (`process.env`, always contiguous), while prose mentioning the word
+  // does (a bundled comment ending "...unique to this process. The id...").
+  [/\b(process|Bun|Deno|Buffer)\.[a-zA-Z_$]|\bnew\s+Buffer\s*\(/, "Node, Bun, and Deno APIs are not supported"],
+  // `node:` only means something as an import specifier; a bare substring
+  // match would also flag it appearing in an ordinary string a dependency
+  // happens to construct (a doc link, a log message).
+  [/['"`]node:/, "Node, Bun, and Deno APIs are not supported"],
   // Porffor alpha-4 compiles `new Proxy(...)` and then ignores the handler: a
   // trapped property reads back as `undefined`, with no throw. Rejecting it
   // here is the difference between a build error and a 502 nobody can explain.

package/src/transport-embedded.js

@@ -1668,7 +1668,7 @@ globalThis.__sbStartLocalTriggers = function (handlers, bindings) {
   const store = __sbStore();
 
   if (crons.length > 0 && __sbIsFn(handlers.scheduled)) {
-    setInterval(function () {
+    setInterval(async function () {
       const now = new Date();
       const stamp =
         now.getUTCFullYear() +
@@ -1684,14 +1684,24 @@ globalThis.__sbStartLocalTriggers = function (handlers, bindings) {
       __sbLastCronTick = stamp;
       for (let i = 0; i < crons.length; i++) {
         if (__sbCronMatches(crons[i], now)) {
-          handlers.scheduled({ cron: crons[i], scheduledTime: now.getTime(), noRetry() {} });
+          // #171 — ctx.waitUntil, and stop dropping the handler's own promise
+          // (was fire-and-forget, same bug the broker-dispatched path had).
+          const __sbTasks = [];
+          const ctx = {
+            waitUntil(p) {
+              if (p && __sbIsFn(p.then)) __sbTasks.push(p);
+            },
+          };
+          const res = handlers.scheduled({ cron: crons[i], scheduledTime: now.getTime(), noRetry() {} }, ctx);
+          if (res && __sbIsFn(res.then)) await res;
+          if (__sbTasks.length) await __sbDrainWaitUntil(__sbTasks);
         }
       }
     }, 15000);
   }
 
   if (queues.length > 0 && __sbIsFn(handlers.queue)) {
-    setInterval(function () {
+    setInterval(async function () {
       __sbEnsureSchema();
       const now = Date.now();
       for (let q = 0; q < queues.length; q++) {
@@ -1709,7 +1719,19 @@ globalThis.__sbStartLocalTriggers = function (handlers, bindings) {
           __sbSql(store, "UPDATE mq SET visible_at = ? WHERE id = ?", [now + 30000, row[0]]);
           messages.push({ id: row[0], body: row[1], timestamp: now, attempts: Number(row[2]) + 1 });
         }
-        const result = __sbRunQueueBatch(handlers, { queue: name, messages });
+        // #171 — same ctx.waitUntil as the broker-delivered path, and the same
+        // fix for the handler's own promise: `__sbRunQueueBatch` used to be
+        // called fire-and-forget here too, so an async `queue()`'s ack/retry
+        // calls made after its first `await` never reached these SQL writes.
+        const __sbTasks = [];
+        const ctx = {
+          waitUntil(p) {
+            if (p && __sbIsFn(p.then)) __sbTasks.push(p);
+          },
+        };
+        let result = __sbRunQueueBatch(handlers, { queue: name, messages }, ctx);
+        if (result && __sbIsFn(result.then)) result = await result;
+        if (__sbTasks.length) await __sbDrainWaitUntil(__sbTasks);
         for (let i = 0; i < result.ack.length; i++) {
           __sbSql(store, "DELETE FROM mq WHERE id = ?", [result.ack[i]]);
         }
@@ -1725,7 +1747,7 @@ globalThis.__sbStartLocalTriggers = function (handlers, bindings) {
   }
 
   if (dos.length > 0) {
-    setInterval(function () {
+    setInterval(async function () {
       __sbEnsureSchema();
       const now = Date.now();
       const due = __sbSql(store, "SELECT cls, id, at, attempts FROM do_alarm WHERE at <= ? ORDER BY at LIMIT 10", [
@@ -1738,7 +1760,14 @@ globalThis.__sbStartLocalTriggers = function (handlers, bindings) {
         // afterwards would erase it. Same rule as the broker (#125).
         __sbSql(store, "DELETE FROM do_alarm WHERE cls = ? AND id = ?", [cls, id]);
         const instance = __sbGetDOInstance(cls, id);
-        if (__sbIsFn(instance.alarm)) instance.alarm();
+        if (__sbIsFn(instance.alarm)) {
+          // #57 gap: this local path bypasses `__sbEntry`'s alarm dispatch
+          // entirely, so it kept the old fire-and-forget behaviour after that
+          // fix landed. Same treatment: await alarm() and drain its waitUntil.
+          const res = instance.alarm();
+          if (res && __sbIsFn(res.then)) await res;
+          if (instance.__sbTasks.length) await __sbDrainWaitUntil(instance.__sbTasks);
+        }
       }
     }, 500);
   }