@mcp-b/do-runtime 0.4.0 → 0.5.0

package/CHANGELOG.md

@@ -1,5 +1,29 @@
 # Changelog
 
+## 0.5.0
+
+### Minor Changes
+
+- bef6bd8: `storage.put()` now reads its arguments the way JSG does, instead of silently dropping writes.
+
+  A non-string, non-object key previously fell into the multi-key overload, where `Object.entries` on a primitive is `[]` — so the call resolved having written nothing. It now stringifies the key as the `kj::String` alternative does: `put(123, v)` writes `"123"`, `put(null, v)` writes `"null"`, `put(undefined, v)` writes `"undefined"`, `put([["a", 1]], v)` writes `"a,1"`, and a symbol key throws V8's own conversion error.
+
+  Once the key does unwrap as a dictionary, a second argument that is a non-null primitive is now refused with upstream's own message rather than spread into the options — `put({a: 1}, "v")` wrote key `a` with `{0: "v"}` for options. The refusal models workerd's all-optional options struct exactly: `null`, arrays and functions all unwrap to default options and the write proceeds, as measured on real workerd. Functions take the dictionary alternative in the KEY position too, so `put(function f(){}, v)` is refused rather than writing a row keyed on the function's source text.
+
+  Measured on real workerd and pinned by a conformance row that runs on all three lanes.
+
+- bef6bd8: `sql.exec()` and `sql.ingest()` now refuse the SQL forms workerd's authorizer refuses but its regulator never sees. SQL that uses them throws where it previously ran.
+
+  `ATTACH`, `DETACH`, the temp-schema creations, and virtual-table modules outside upstream's four reach SQLite action codes rather than `SqlStorageRegulator` callbacks, so porting the regulator whole left them unguarded. Each refuses with workerd's own message, `not authorized: SQLITE_AUTH`, except `VACUUM`, which carries SQLite's `cannot VACUUM from within a transaction: SQLITE_ERROR` because upstream refuses it by the transaction a Durable Object always has open.
+
+  `ATTACH` was an isolation boundary and not only a fidelity gap: both backends open a real database file, so application SQL could attach another actor's database and read every application table in it — the reserved-name scan sees only the submitted statement — and `VACUUM INTO` could write any file the process can. The temp schema is refused by both of its spellings, `CREATE TEMP TABLE` and `CREATE TABLE temp.t`; the second was the live gap, where the table was created, written and read back. Of the virtual-table modules, `dbstat` is the one that mattered: it reports a row per table, so it enumerated the runtime's own `_cf_` tables and their sizes without the statement ever naming them. The table name and the module accept every quoting SQLite does — double quotes, backticks, brackets, and the misquoting feature's single-quoted string — with the module resolved before the allowlist, exactly as workerd's authorizer sees it; a `CREATE VIRTUAL TABLE` whose module cannot be read is refused outright.
+
+  A leading `;` now counts as trivia for every leading-keyword refusal in the file, including the pre-existing transaction-control and pragma checks. `node:sqlite` reports an empty first statement as part of the span it compiled, so `;ATTACH …` read as a statement whose keyword was `;` and passed all three.
+
+  `PRAGMA page_size` is now allowed with no argument, as upstream's allowlist has it for the R\*Tree module's own internal read; `PRAGMA page_size = N` is still refused. That was a false refusal — code that runs on Cloudflare failed here.
+
+  Every form, refused and allowed alike, was measured on real workerd and is pinned by a conformance row that runs on all three lanes.
+
 ## 0.4.0
 
 ### Minor Changes

package/README.md

@@ -238,7 +238,9 @@ The browser cannot reproduce every workerd facility. Where it cannot, the runtim
 | `DurableObjectState.abort()` | Breaks later storage and re-entry; cannot synchronously terminate the calling JavaScript slice. |
 | Stored value wire bytes | Browser-safe versioned structured-clone encoding rather than V8's private format; public value types align and legacy JSON rows remain readable. |
 | SQL row counters | Local `rowsRead`/`rowsWritten`, including `sql.ingest()`, use returned rows and SQLite changes; workerd uses unavailable libsql billing counters. |
-| Reserved SQL names | `_cf_` detected from tokenized SQL text, which can reject more than workerd's authorizer. |
+| Reserved SQL names | `_cf_` detected from tokenized SQL text, which can reject more than workerd's authorizer. `ANALYZE` on a reserved table is refused where workerd allows it. |
+| Authorizer-only SQL forms | `ATTACH`, `DETACH`, the temp-schema creations (both `CREATE TEMP …` and the `temp.` qualifier), `VACUUM`, and virtual-table modules outside upstream's four (`fts5`, `fts5vocab`, `rtree`, `rtree_i32`) are refused from the leading keyword, with workerd's own messages. These reach the authorizer's own decisions — action codes and its temp-schema rule — rather than `SqlStorageRegulator` callbacks, so porting the regulator did not carry them. The refused and allowed forms are matched through SQLite's identifier quoting, whitespace or none. `EXPLAIN` in front of a refused form still compiles here, where workerd's authorizer refuses it. |
+| SQL function allowlist | Not enforced. Workerd's authorizer denies any function outside its 138-name `ALLOWED_SQLITE_FUNCTIONS` list; this runtime allows every function the backend compiled, including build-detail readers such as `sqlite_version()` and `sqlite_source_id()`. |
 | PRAGMA allowlist | Workerd's allowlist enforced from tokenized SQL text. A `pragma_*` table-valued function with a string or bound argument is authorized by pragma name only, where workerd's authorizer also sees the resolved argument; the pinned conformance row is the contract. |
 | Node SQLite length limit | Bound and returned strings and blobs are capped at 4 MiB; `node:sqlite` cannot cap an unreturned SQL-computed value. The browser backend uses SQLite's native limit. |
 | Response BYOB readers | Refused; their continuation cannot be re-gated. Use a default reader or `arrayBuffer()`. |

package/dist/index.js

@@ -999,14 +999,24 @@ function notSerializableType(value) {
 *     `abstract` in the types — JSG nested types have no JS constructor — so the
 *     faithful shape is a constructor that refuses. `sql.Cursor` exists for
 *     `instanceof`, which is all upstream exposes it for.
-*  3. **The regulator is ported whole; what is missing is the authorizer that
-*     calls it.** All three callbacks are here and none of them needed the
+*  3. **The regulator is ported whole, and that is not the whole authorizer.**
+*     Four of its five members are here as callbacks, and none needed the
 *     authorizer to compute anything — `isAllowedName` is a prefix test,
-*     `isAllowedTrigger` is `return true`, `allowTransactions` throws. What the
-*     authorizer supplied was the *identifiers*, not the decisions. With no
-*     authorizer the statement text is the only source, so `exec` tokenizes it
+*     `isAllowedTrigger` is `return true`, `allowTransactions` throws,
+*     `shouldAddQueryStats` is a constant. The fifth, `onError`, is not a
+*     callback at all in this port: it is every `throw new Error(message)`
+*     below, which is all its upstream body does with a refusal message.
+*     For those, what the authorizer supplied was the *identifiers*:
+*     with none, the statement text is the only source, so `exec` tokenizes it
 *     and runs `isAllowedName` over every identifier-shaped token. That is
 *     deliberately STRICTER than upstream — see `SQL_RESERVED_PREFIX_MESSAGE`.
+*
+*     But the authorizer also makes decisions no callback ever sees, and those
+*     do NOT arrive with the regulator: `SQLITE_ATTACH` / `SQLITE_DETACH`, the
+*     `SQLITE_CREATE_TEMP_*` family and the `temp` schema, `SQLITE_PRAGMA`, and
+*     the `SQLITE_CREATE_VTABLE` module list. Each is refused from the text in
+*     `refuseUnauthorizedForms` and `requireAllowedPragmas`. `SQLITE_FUNCTION`
+*     is the one still unported — see the README divergence row.
 *  4. **`ingest` stays at upstream's SQLite seam.** `SqliteDatabase.ingest()`
 *     executes every complete statement and returns the partial tail, using the
 *     same compiled boundaries and regulator as `exec`.
@@ -1039,8 +1049,15 @@ var SQL_RESERVED_PREFIX_MESSAGE = "not authorized: a SQL statement may not name
 * `SQLITE_SAVEPOINT`. The same set `util/sqlite.ts` classifies, read here from
 * the leading keyword because the untrusted path has to refuse them before the
 * trusted one applies them.
+*
+* `;` counts as leading trivia here and in every other leading-keyword refusal
+* below. A statement boundary comes from the backend, and `node:sqlite` reports
+* an empty leading statement as part of the span it compiled: the `sourceSQL`
+* for `;ATTACH ...` is the whole string, so an anchor of `^\s*` would read the
+* keyword as `;` and let the compiled ATTACH through. The browser backend cuts
+* the same input at the first `;` and refuses the empty statement instead.
 */
-var TRANSACTION_CONTROL = /^\s*(?:BEGIN|COMMIT|END|ROLLBACK|SAVEPOINT|RELEASE)\b/i;
+var TRANSACTION_CONTROL = /^[\s;]*(?:BEGIN|COMMIT|END|ROLLBACK|SAVEPOINT|RELEASE)\b/i;
 /** Cheap pre-test, so the tokenizer below runs only on a statement that could fail it. */
 var RESERVED_PREFIX_HINT = /_cf_/i;
 /** A SQL identifier. Double-quoted and bracketed forms are still identifiers, so only the
@@ -1057,7 +1074,15 @@ var IDENTIFIER = /[A-Za-z_][A-Za-z0-9_$]*/g;
 */
 var NOT_CODE = /'(?:[^']|'')*'|--[^\n]*|\/\*[\s\S]*?\*\//g;
 /**
-* ← `SqlStorageRegulator` (`sql.h:15-22`, `sql.c++:143-173`), whole.
+* Comments are never code. String literals STAY, for both of this regex's callers: pragma
+* arguments may be quoted, and SQLite's misquoting feature reads a single-quoted string as an
+* identifier — `CREATE TABLE 'temp'.t(x)` really creates a temp-schema table — so the checks
+* that read identifier positions must still see it. `NOT_CODE` blanks literals, which is right
+* for the token scans and would be a bypass for these callers.
+*/
+var NOT_COMMENT = /--[^\n]*|\/\*[\s\S]*?\*\//g;
+/**
+* ← `SqlStorageRegulator` (`sql.h:15-22`, `sql.c++:141-165`), whole.
 *
 * Upstream reaches these through the SQLite authorizer while a statement is
 * being compiled. `exec` calls them from the statement text instead, which is
@@ -1123,20 +1148,108 @@ function refuseTransactionControl(statement) {
 * byte-identical so a caller matching on it ports unchanged.
 */
 var SQL_NOT_AUTHORIZED_MESSAGE = "not authorized: SQLITE_AUTH";
-/** Comments are never code. String literals STAY: pragma arguments may be quoted. */
-var NOT_COMMENT = /--[^\n]*|\/\*[\s\S]*?\*\//g;
+/**
+* The forms neither the regulator nor any callback sees: the authorizer's own
+* action codes, plus `VACUUM`, which SQLite itself refuses by precondition.
+*
+* Porting `SqlStorageRegulator` whole (point 3 in the file header) carried over its members, but
+* not the authorizer's own action codes: `SQLITE_ATTACH`, `SQLITE_DETACH`, `SQLITE_CREATE_TEMP_*`
+* and `SQLITE_CREATE_VTABLE` consult no callback, so nothing here refused them. Every form below
+* was measured on real workerd through the conformance oracle, not inferred.
+*
+* `ATTACH` and `DETACH` are also the isolation boundary rather than a fidelity detail: both
+* backends open a real file, so on `node:sqlite` an `ATTACH` reads another actor's database and
+* a `VACUUM INTO` writes anywhere the process can. The reserved-name scan does not cover it,
+* because that scan tokenizes the SUBMITTED statement — `other._cf_KV` is caught, and every
+* application table in the same attached database is not.
+*/
+var DATABASE_ATTACHMENT = /^[\s;]*(?:ATTACH|DETACH)\b/i;
+/**
+* ← the `SQLITE_CREATE_TEMP_*` denials (`sqlite.c++:1323`) and the `dbName == temp` rule
+* (`sqlite.c++:1073`), which permits a temp-schema database name only `READ` and `UPDATE`.
+* Upstream's own reason to deny them applies here unchanged: a temporary table makes SQLite
+* open a separate temporary file that the storage engine knows nothing about.
+*
+* Two spellings, one refusal each by its own upstream path: `CREATE TEMP TABLE t(x)` is the
+* keyword and hits the action codes; `CREATE TABLE temp.t(x)` is the schema qualifier and hits
+* the `dbName` rule — and the qualifier was the live gap, where the table was created, written
+* and read back here while workerd refused it outright. The qualifier accepts every quoting
+* SQLite does, single quotes included, with or without whitespace after the keyword (measured:
+* workerd refuses `CREATE TABLE 'temp'.t(x)` and `CREATE TABLE"temp".t(x)` the same way), and
+* is matched only in the object-name position, so an application table merely NAMED `tempest`
+* or `temporary_log` is untouched. The keyword spelling needs no qualifier arm of its own here:
+* `TEMP_SCHEMA` already refuses every `CREATE TEMP…` before this pattern is consulted.
+*
+* The rest of the `dbName` rule goes unported on purpose: with no way to create a temp-schema
+* object, `INSERT`/`DELETE`/`DROP` against one die in SQLite as `no such table`, and the one
+* silent form measures identically — workerd allows `DROP TABLE IF EXISTS temp.ghost` too.
+*/
+var TEMP_SCHEMA = /^[\s;]*CREATE\s+(?:TEMP|TEMPORARY)\b/i;
+var TEMP_QUALIFIED = /^[\s;]*CREATE\s+(?:UNIQUE\s+|VIRTUAL\s+)?(?:TABLE|VIEW|TRIGGER|INDEX)\s*(?:IF\s+NOT\s+EXISTS\s*)?(?:"temp"|'temp'|`temp`|\[temp\]|temp)\s*\./i;
+/**
+* ← `SQLITE_CREATE_VTABLE` (`sqlite.c++:1298-1316`): a virtual table is native-code callbacks, so
+* upstream allows exactly four modules — FTS5 and its `fts5vocab` companion, R*Tree and its
+* `rtree_i32` variant — and denies every other module SQLite was compiled with.
+*
+* `dbstat` is why this is not only fidelity: it reports a row per table with page counts and
+* byte sizes, so `SELECT name FROM d` enumerates `_cf_KV` and the rest of the runtime's own
+* tables without the statement ever naming them — around `requireAllowedNames`, which can only
+* see the text it was given.
+*
+* The table name and the module accept every quoting SQLite does — double quotes, backticks,
+* brackets, and the misquoting feature's single-quoted string, with or without whitespace
+* before them — because the module has to be read from PAST the name, and a guessed name
+* boundary is a bypass in both directions: an unparseable name skipped the check, and
+* `"a USING fts5 b" USING dbstat` read its module out of the quoted name. Measured: workerd
+* refuses both, refuses `CREATE VIRTUAL TABLE"d"USING dbstat`, resolves `USING "dbstat"` to
+* the same denial, and allows `USING 'fts5'`. A `CREATE VIRTUAL TABLE` whose module the
+* pattern cannot read is refused outright — the deliberately stricter direction the
+* unparseable-PRAGMA fallback below already takes.
+*/
+var SQL_IDENTIFIER_SOURCE = /"(?:[^"]|"")*"|'(?:[^']|'')*'|`(?:[^`]|``)*`|\[[^\]]*\]|[A-Za-z_\u0080-\uffff][A-Za-z0-9_$\u0080-\uffff]*/.source;
+var VIRTUAL_TABLE = /^[\s;]*CREATE\s+VIRTUAL\s+TABLE\b/i;
+var VIRTUAL_TABLE_MODULE = new RegExp(String.raw`^[\s;]*CREATE\s+VIRTUAL\s+TABLE\s*(?:IF\s+NOT\s+EXISTS\s*)?(?:${SQL_IDENTIFIER_SOURCE})\s*(?:\.\s*(?:${SQL_IDENTIFIER_SOURCE})\s*)?USING\s*(${SQL_IDENTIFIER_SOURCE})`, "i");
+var ALLOWED_VIRTUAL_TABLE_MODULES = /* @__PURE__ */ new Set([
+	"fts5",
+	"fts5vocab",
+	"rtree",
+	"rtree_i32"
+]);
+/**
+* `VACUUM` has no action code of its own, so the authorizer never sees it. What refuses it
+* upstream is SQLite's own `cannot VACUUM from within a transaction` precondition, against the
+* transaction a Durable Object always has open — and that message is what the oracle returned,
+* for `VACUUM`, `VACUUM main` and `VACUUM INTO` alike. Byte-identical for the same reason
+* `SQL_NOT_AUTHORIZED_MESSAGE` is: a caller matching on it ports unchanged.
+*
+* Refused here unconditionally rather than by transaction state, which is a divergence only in
+* mechanism: this runtime never runs a statement where upstream would have allowed it.
+*/
+var VACUUM_STATEMENT = /^[\s;]*VACUUM\b/i;
+var SQL_VACUUM_REFUSED_MESSAGE = "cannot VACUUM from within a transaction: SQLITE_ERROR";
+/** Refuse the authorizer-only forms against one SQLite-decided statement boundary. */
+function refuseUnauthorizedForms(statement) {
+	const code = statement.replace(NOT_COMMENT, " ");
+	if (VACUUM_STATEMENT.test(code)) throw new Error(SQL_VACUUM_REFUSED_MESSAGE);
+	if (DATABASE_ATTACHMENT.test(code) || TEMP_SCHEMA.test(code) || TEMP_QUALIFIED.test(code)) throw new Error(SQL_NOT_AUTHORIZED_MESSAGE);
+	if (VIRTUAL_TABLE.test(code)) {
+		const moduleName = VIRTUAL_TABLE_MODULE.exec(code)?.[1];
+		if (moduleName === void 0 || !ALLOWED_VIRTUAL_TABLE_MODULES.has(unquoted(moduleName).toLowerCase())) throw new Error(SQL_NOT_AUTHORIZED_MESSAGE);
+	}
+}
 /** Cheap pre-test; `PRAGMA` and the `pragma_` functions both contain it. */
 var PRAGMA_HINT = /pragma/i;
 /** `PRAGMA [schema.]name`, then `= value`, `(argument)`, or nothing. */
-var PRAGMA_STATEMENT = /^\s*PRAGMA\s+(?:[A-Za-z_][A-Za-z0-9_$]*\s*\.\s*)?([A-Za-z_][A-Za-z0-9_$]*)(?:\s*=\s*([\s\S]+?)|\s*\(\s*([\s\S]*?)\s*\))?\s*;?\s*$/i;
+var PRAGMA_STATEMENT = /^[\s;]*PRAGMA\s+(?:[A-Za-z_][A-Za-z0-9_$]*\s*\.\s*)?([A-Za-z_][A-Za-z0-9_$]*)(?:\s*=\s*([\s\S]+?)|\s*\(\s*([\s\S]*?)\s*\))?\s*;?\s*$/i;
 /**
-* ← `ALLOWED_PRAGMAS` and `PragmaSignature` (`util/sqlite.c++:525-563`),
+* ← `ALLOWED_PRAGMAS` (`util/sqlite.c++:543-571`) and `PragmaSignature` (`:528-535`),
 * verbatim. `table_list`, `table_info`, and `table_xinfo` are special-cased
 * ahead of the table in the authorizer, exactly as upstream's `SQLITE_PRAGMA`
 * case does (`util/sqlite.c++:1194-1273`).
 */
 var ALLOWED_PRAGMAS = /* @__PURE__ */ new Map([
 	["data_version", "NO_ARG"],
+	["page_size", "NO_ARG"],
 	["case_sensitive_like", "BOOLEAN"],
 	["foreign_keys", "BOOLEAN"],
 	["defer_foreign_keys", "BOOLEAN"],
@@ -1228,7 +1341,11 @@ var SQLITE_PRAGMA_NAMES = /* @__PURE__ */ new Set([
 ]);
 /** kj's `tryParseAs` is decimal; keep the same acceptance. */
 var DECIMAL = /^[+-]?\d+$/;
-/** One layer of SQL quoting off a pragma argument, any of the four forms. */
+/**
+* One layer of SQL quoting, any of the four forms — a pragma argument, or the module token
+* `VIRTUAL_TABLE_MODULE` captured. Doubled inner quotes stay doubled, which cannot change a
+* verdict here: no allowlisted comparison target contains a quote character.
+*/
 function unquoted(argument) {
 	const first = argument[0];
 	const last = argument[argument.length - 1];
@@ -1282,7 +1399,7 @@ function requireAllowedPragmas(statement) {
 		if (!isAllowedPragma(name, argument === "" ? void 0 : argument)) throw new Error(SQL_NOT_AUTHORIZED_MESSAGE);
 		return;
 	}
-	if (/^\s*PRAGMA\b/i.test(code)) throw new Error(SQL_NOT_AUTHORIZED_MESSAGE);
+	if (/^[\s;]*PRAGMA\b/i.test(code)) throw new Error(SQL_NOT_AUTHORIZED_MESSAGE);
 	const literalFree = code.replace(NOT_CODE, " ");
 	for (const [token] of literalFree.matchAll(IDENTIFIER)) {
 		if (token.length <= 7 || token.slice(0, 7).toLowerCase() !== "pragma_") continue;
@@ -1294,6 +1411,7 @@ function requireAllowedPragmas(statement) {
 /** Everything the untrusted path refuses at one statement boundary. */
 function regulateUntrustedStatement(statement) {
 	refuseTransactionControl(statement);
+	refuseUnauthorizedForms(statement);
 	requireAllowedPragmas(statement);
 }
 /** ← `JSG_INHERIT_INTRINSIC(v8::kIteratorPrototype)` (`jsg/iterator.h:1044`). */
@@ -1686,6 +1804,26 @@ var OP_PUT_ALARM = "setAlarm()";
 var OP_DELETE = "delete()";
 var OP_DELETE_ALARM = "deleteAlarm()";
 var OP_ROLLBACK = "rollback()";
+/** ← `actor-state.c++:455`, verbatim: the one message both overloads' misuse produces. */
+var PUT_OVERLOAD_MESSAGE = "put() may only be called with a single key-value pair and optional options as put(key, value, options) or with multiple key-value pairs and optional options as put(entries, options)";
+/**
+* ← the `kj::OneOf<kj::String, jsg::Dict<…>>` unwrap on put()'s first parameter: `jsg::Dict`
+* takes any JS object except an Array — functions and Maps included — and `kj::String` takes
+* everything else by coercion. A type predicate, so the overload split narrows without a cast.
+*/
+function isEntriesArgument(value) {
+	return (typeof value === "object" || typeof value === "function") && value !== null && !Array.isArray(value);
+}
+/**
+* ← the struct wrapper (`jsg/struct.h:246-258`), which is NOT the Dict wrapper: `PutOptions` is
+* all-optional fields, so `null` unwraps to default options, and any object does — arrays and
+* functions included, because the wrapper checks `IsObject()` with no Array exclusion. Only a
+* non-null primitive fails to unwrap. Measured on real workerd: `put({k: 1}, null)`, `…, [])`
+* and `…, function () {})` all write, and `put({k: 1}, "v")` alone is the overload error.
+*/
+function isPutOptions(value) {
+	return value === null || typeof value === "object" || typeof value === "function";
+}
 /** The key immediately after `k` in byte order is `k` plus this. */
 var NULL_CHARACTER = "\0";
 /** ← the `0xff` upstream strips from the tail of a prefix, in UTF-16 code units. */
@@ -1889,10 +2027,11 @@ var DurableObjectStorageOperations = class {
 	}
 	put(keyOrEntries, valueOrOptions, maybeOptions) {
 		requireInputLock(this.ctx, OP_PUT);
-		if (typeof keyOrEntries === "string") {
+		if (!isEntriesArgument(keyOrEntries)) {
 			if (valueOrOptions === void 0) throw new TypeError("put() called with undefined value.");
-			return this.#putOne(keyOrEntries, valueOrOptions, { ...maybeOptions });
+			return this.#putOne(`${keyOrEntries}`, valueOrOptions, { ...maybeOptions });
 		}
+		if (valueOrOptions !== void 0 && !isPutOptions(valueOrOptions)) throw new TypeError(PUT_OVERLOAD_MESSAGE);
 		return this.#putMultiple(keyOrEntries, { ...valueOrOptions });
 	}
 	delete(keyOrKeys, maybeOptions) {