jseption 0.0.3 → 0.0.4

package/README.md

@@ -104,6 +104,82 @@ checksum only catches accidental corruption; bytecode that passes through storag
 service you do not control needs a signature you verify before loading. See
 `docs/BYTECODE.md` for the format and the measurements.
 
+## Running TypeScript
+
+TypeScript runs the way Node runs it with type stripping: the erasable syntax is
+removed and the rest runs as JavaScript, with no type checking. Pass
+`JS_EVAL_FLAG_STRIP_TYPES` to `JS_Eval`; the command line strips `.ts`, `.mts` and
+`.cts` files, and `--strip-types` applies it to `-e`.
+
+```ts
+JS_Eval(ctx, 'const id = <T,>(x: T): T => x; id<number>(41) + 1', 'app.ts',
+        JS_EVAL_TYPE_GLOBAL | JS_EVAL_FLAG_STRIP_TYPES); // 42
+```
+
+The stripping is built into the parser, so error positions and stacks are those
+of the JavaScript with the types blanked, and plain JavaScript compiles as fast as
+before. As in Node, `enum`, namespaces with values, parameter properties,
+`import x = require(...)` and `export =` are rejected because they need generated
+code. `docs/TYPESCRIPT.md` lists what is stripped and how ambiguous syntax is read.
+
+## JSX and TSX
+
+`JS_EVAL_FLAG_JSX` compiles JSX in the parser, as TypeScript compiles it. With
+`JS_EVAL_FLAG_STRIP_TYPES` as well, it compiles TSX. There is no build step.
+
+* **Modules** use the automatic runtime (`jsx: "react-jsx"`). Elements are calls of
+  `jsx`, `jsxs` and `Fragment`, imported from `react/jsx-runtime`. A
+  `/** @jsxImportSource x */` pragma imports them from `x/jsx-runtime` instead.
+* **Scripts** use the classic runtime (`React.createElement`, or the `@jsx` and
+  `@jsxFrag` pragmas).
+* **The module loader** compiles `.tsx` ids (or a `type: 'tsx'` result) as TSX and
+  `.jsx` ids as JSX.
+* **The command line** runs `.tsx` and `.jsx` files as modules, and `--jsx` applies
+  the flag to `-e`.
+
+```ts
+JS_SetModuleLoader(rt, { load: (id) => files[id] });   // '/app.tsx', 'react/jsx-runtime', ...
+await guestPromiseToHost(ctx, JS_ImportModule(ctx, '/app.tsx'));
+```
+
+`docs/JSX.md` covers the runtimes, pragmas, text and entity rules, and what is not
+supported.
+
+## Modules
+
+ES modules work as in QuickJS: `import` and `export` in every form, live bindings,
+cycles, `export *` with its ambiguity rules, namespace objects, top-level `await`,
+dynamic `import()`, `import.meta`, and JSON modules (`with { type: "json" }`). The
+embedder decides what a specifier means. With no loader set, nothing can be imported,
+so the sandbox stays closed by default.
+
+```ts
+import { JS_SetModuleLoader, JS_ImportModule, guestPromiseToHost, drainJobs } from 'jseption';
+
+JS_SetModuleLoader(rt, {
+  // optional; the default resolves ./, ../ and / against the importing module, URL style
+  resolve: (specifier, referrer) => new URL(specifier, referrer ?? location.href).href,
+  // source text, module bytecode, or { source, url, type, exports }, or a promise of one
+  load: async (url) => ({ source: await (await fetch(url)).text(), url }),
+});
+const ns = await guestPromiseToHost(ctx, JS_ImportModule(ctx, 'https://example.test/app.js'));
+```
+
+The loader may be asynchronous: the whole graph is fetched first, in parallel where
+possible, then linked and evaluated. A synchronous loader keeps everything synchronous.
+`load` can return a host module, `{ exports: { fetch } }`, whose values are marshalled
+into the guest, so guest code can `import { fetch } from "host:net"`. It can also
+return bytecode from `JS_WriteObject`, so a precompiled graph loads without parsing.
+`JS_EvalModule(ctx, source, url)` evaluates source text as a module.
+`JS_Eval` with `JS_EVAL_TYPE_MODULE` keeps QuickJS's behaviour and returns the
+evaluation promise.
+
+The command line runs `.mjs` and `.mts` files as modules, detects `.js` and `.ts` files
+that start with `import` or `export`, and takes `-m` to force module mode or `--script`
+to force a script. Its loader reads relative and absolute paths from disk; bare
+specifiers are not resolved. `docs/MODULES.md` describes the design and the loader
+contract.
+
 ## Passing values between host and guest
 
 Guest values are host values, but the sandbox forbids handing the guest an object whose
@@ -137,8 +213,9 @@ src/atoms.ts        atom table (JS_ATOM_*), tagged integer atoms, atom -> proper
 src/values.ts       JSValue, object metadata, bytecode & frame structures
 src/opcodes.ts      opcode table generated from quickjs-opcode.h
 src/quickjs.ts      runtime, context, property layer over Reflect, calls, conversions, jobs
-src/parser.ts       tokenizer, parser, bytecode emitter, variable/label resolution, eval
+src/parser.ts       tokenizer, parser (with TypeScript type stripping and JSX), bytecode emitter, variable/label resolution, eval
 src/bytecode.ts     bytecode serialization (JS_WriteObject / JS_ReadObject)
+src/module.ts       ES modules: loader, linking, evaluation, namespaces, import(), import.meta
 src/interpreter.ts  JS_CallInternal, closures, generators, async functions (fallback tier)
 src/jit.ts          bytecode -> host JavaScript compiler (stack-to-register, relooper)
 src/jit_runtime.ts  the helpers generated code calls into (the only engine touchpoint)
@@ -214,9 +291,9 @@ Differences from the C implementation:
 * Memory management relies on the host garbage collector: there is no reference
   counting, `JS_FreeValue`/`JS_DupValue` are no-ops, and `WeakRef` / `FinalizationRegistry`
   use the host weak references.
-* ES modules (`import` / `export`), `Atomics` and `SharedArrayBuffer` threads are not
-  ported. `JS_WriteObject` / `JS_ReadObject` serialize compiled scripts (bytecode
-  function trees) only, not arbitrary values, in a jseption-specific format.
+* `Atomics` and `SharedArrayBuffer` threads are not ported. `JS_WriteObject` /
+  `JS_ReadObject` serialize compiled scripts and modules only, not arbitrary values,
+  in a jseption-specific format.
 * Number formatting and parsing use the host conversions (they are specified by
   ECMAScript, so the results are identical).
 

package/dist/bytecode.js

@@ -35,7 +35,8 @@
  */
 import { JS_ATOM_END, JS_ATOM_raw, JS_ATOM_TAG_INT, JS_ATOM_TYPE_STRING, __JS_AtomIsTaggedInt } from "./atoms.js";
 import { JS_GetFunctionData, JS_ThrowSyntaxError, JS_ThrowTypeError, JS_NewArray, JS_DefinePropertyValue, JS_DefinePropertyValueUint32, JS_IsException, JS_PROP_ENUMERABLE, hooks } from "./quickjs.js";
-import { JSClosureVar, JSFunctionBytecode, JSVarDef } from "./values.js";
+import { JSClosureVar, JSFunctionBytecode, JSVarDef, JSModuleDef, JSReqModuleEntry, JSExportEntry, JSStarExportEntry, JSImportEntry } from "./values.js";
+import { js_module_set_function, hooks_bytecode } from "./module.js";
 import { short_opcode_info } from "./opcodes.js";
 import { JSEPTION_VERSION } from "./version.js";
 /* keep each function's source text (Function.prototype.toString); off by default */
@@ -320,6 +321,51 @@ class BCWriter {
         for (const v of b.cpool)
             this.value(v);
     }
+    /* a compiled module (JS_WriteModule): its requests, entries and body function */
+    module(m) {
+        const out = this.out;
+        out.u8(11 /* BCTag.MODULE */);
+        this.atom(m.module_name);
+        out.leb(m.req_module_entries.length);
+        for (const rme of m.req_module_entries) {
+            this.atom(rme.module_name);
+            if (rme.attributes === null) {
+                out.leb(0);
+            }
+            else {
+                const keys = Object.keys(rme.attributes);
+                out.leb(keys.length + 1);
+                for (const k of keys) {
+                    out.leb(this.string_ref(k));
+                    out.leb(this.string_ref(rme.attributes[k]));
+                }
+            }
+        }
+        out.leb(m.export_entries.length);
+        for (const me of m.export_entries) {
+            out.u8(me.export_type);
+            if (me.export_type === 0 /* JSExportType.LOCAL */) {
+                out.leb(me.var_idx);
+            }
+            else {
+                out.leb(me.req_module_idx);
+                this.atom(me.local_name);
+            }
+            this.atom(me.export_name);
+        }
+        out.leb(m.star_export_entries.length);
+        for (const se of m.star_export_entries)
+            out.leb(se.req_module_idx);
+        out.leb(m.import_entries.length);
+        for (const mi of m.import_entries) {
+            out.leb(mi.var_idx);
+            out.u8(mi.is_star ? 1 : 0);
+            this.atom(mi.import_name);
+            out.leb(mi.req_module_idx);
+        }
+        out.u8(m.has_tla ? 1 : 0);
+        this.function(m.func_bytecode);
+    }
     /* the byte code with its atom operands renumbered (JS_WriteFunctionBytecode) */
     byte_code(b) {
         const len = b.byte_code_len;
@@ -453,6 +499,25 @@ function checksum(buf, end) {
    bytecode function that does not close over variables). Returns null with a pending
    exception on failure. */
 export function JS_WriteObject(ctx, obj, flags = 0) {
+    if (obj instanceof JSModuleDef) {
+        /* a module compiled with JS_EVAL_TYPE_MODULE | JS_EVAL_FLAG_COMPILE_ONLY */
+        if (obj.kind !== 0 /* JSModuleKind.JS */ || obj.func_bytecode === null) {
+            JS_ThrowTypeError(ctx, 'JS_WriteObject: only JavaScript modules can be written');
+            return null;
+        }
+        const w = new BCWriter(ctx, flags);
+        try {
+            w.module(obj);
+            return w.finish(w.out);
+        }
+        catch (e) {
+            if (e instanceof BCWriteError) {
+                JS_ThrowTypeError(ctx, 'JS_WriteObject: ' + e.message);
+                return null;
+            }
+            throw e;
+        }
+    }
     const fd = JS_GetFunctionData(obj);
     if (fd === null) {
         JS_ThrowTypeError(ctx, 'JS_WriteObject: not a bytecode function (expected the result of JS_EVAL_FLAG_COMPILE_ONLY)');
@@ -810,6 +875,93 @@ class BCReader {
         this.depth--;
         return b;
     }
+    /* a compiled module (JS_ReadModule) */
+    module() {
+        const m = new JSModuleDef(this.atom());
+        m.realm = this.ctx;
+        const nreq = this.count();
+        for (let i = 0; i < nreq; i++) {
+            const name = this.atom();
+            const n = this.leb();
+            let attrs = null;
+            if (n > 0) {
+                if (n - 1 > this.end - this.pos)
+                    this.truncated();
+                attrs = Object.create(null);
+                for (let j = 0; j < n - 1; j++) {
+                    const k = this.string();
+                    attrs[k] = this.string();
+                }
+            }
+            m.req_module_entries.push(new JSReqModuleEntry(name, attrs));
+        }
+        const req_idx = () => {
+            const i = this.leb();
+            if (i >= nreq)
+                this.fail(`invalid module request index ${i}`);
+            return i;
+        };
+        const nexp = this.count();
+        const local_exports = [];
+        for (let i = 0; i < nexp; i++) {
+            const type = this.u8();
+            let me;
+            if (type === 0 /* JSExportType.LOCAL */) {
+                const var_idx = this.leb();
+                local_exports.push(var_idx);
+                me = new JSExportEntry(0, this.atom(), 0 /* JSExportType.LOCAL */);
+                me.var_idx = var_idx;
+                me.local_name = me.export_name;
+            }
+            else if (type === 1 /* JSExportType.INDIRECT */) {
+                const idx = req_idx();
+                const local_name = this.atom();
+                me = new JSExportEntry(local_name, this.atom(), 1 /* JSExportType.INDIRECT */);
+                me.req_module_idx = idx;
+            }
+            else {
+                this.fail('invalid export entry');
+            }
+            m.export_entries.push(me);
+        }
+        const nstar = this.count();
+        for (let i = 0; i < nstar; i++)
+            m.star_export_entries.push(new JSStarExportEntry(req_idx()));
+        const nimp = this.count();
+        const imports = [];
+        for (let i = 0; i < nimp; i++) {
+            const var_idx = this.leb();
+            const is_star = this.u8();
+            if (is_star > 1)
+                this.fail('invalid import entry');
+            const mi = new JSImportEntry(var_idx, is_star === 1, this.atom());
+            mi.req_module_idx = req_idx();
+            m.import_entries.push(mi);
+            imports.push({ var_idx, is_star: is_star === 1 });
+        }
+        const tla = this.u8();
+        if (tla > 1)
+            this.fail('invalid module flags');
+        m.has_tla = tla === 1;
+        if (this.u8() !== 9 /* BCTag.FUNCTION_BYTECODE */)
+            this.fail('the module body is not a function');
+        const b = this.function();
+        if (b.func_kind !== 2 /* JSFunctionKind.ASYNC */)
+            this.fail('the module body is not an async function');
+        /* the entries index the closure variables of the body */
+        const ncv = b.closure_var.length;
+        for (const v of local_exports)
+            if (v >= ncv)
+                this.fail(`invalid export variable index ${v}`);
+        for (const { var_idx, is_star } of imports) {
+            if (var_idx >= ncv)
+                this.fail(`invalid import variable index ${var_idx}`);
+            if (b.closure_var[var_idx].is_local !== is_star)
+                this.fail('invalid import variable');
+        }
+        js_module_set_function(m, b);
+        return m;
+    }
     /* patch the atom operands and check the instruction boundaries and the constant pool
        indices (JS_ReadFunctionBytecode) */
     fixup_byte_code(b) {
@@ -891,7 +1043,14 @@ export function JS_ReadObject(ctx, buf) {
         r.header();
         r.string_table();
         r.atom_table();
-        if (r.u8() !== 9 /* BCTag.FUNCTION_BYTECODE */)
+        const tag = r.u8();
+        if (tag === 11 /* BCTag.MODULE */) {
+            const m = r.module();
+            if (r.pos !== r.end)
+                r.fail('trailing bytes after the root value');
+            return m;
+        }
+        if (tag !== 9 /* BCTag.FUNCTION_BYTECODE */)
             r.fail('the root value is not a function');
         b = r.function();
         if (r.pos !== r.end)
@@ -908,4 +1067,5 @@ export function JS_ReadObject(ctx, buf) {
         return JS_ThrowSyntaxError(ctx, 'JS_ReadObject: the root function has closure variables');
     return hooks.js_closure(ctx, b, null, null);
 }
+hooks_bytecode.read = (ctx, bytes) => JS_ReadObject(ctx, bytes);
 //# sourceMappingURL=bytecode.js.map