jseption 0.0.2 → 0.0.4

package/README.md

@@ -33,6 +33,7 @@ JSEPTION_JIT=0 bun test        # run the suite on the interpreter tier only
 JSEPTION_JIT=always bun test   # compile every function that runs (no lazy tier-up)
 JSEPTION_JIT_DUMP=1 bun run src/qjs.ts file.js   # print the generated host code
 bun run bench/inproc.ts        # host vs tier modes, incl. cold-code bundles
+bun run bench/bytecode.ts file.js   # compile vs read precompiled bytecode vs run
 ```
 
 ## Installing
@@ -72,6 +73,113 @@ engine metadata kept in a `WeakMap`). Engine code signals exceptions with the
 the two conventions meet at the call boundary. See `docs/PORTING.md` and
 `docs/HOST_VALUES.md` for the internals.
 
+## Precompiled bytecode
+
+A script can be compiled ahead of time and loaded without parsing, as with QuickJS's
+`JS_WriteObject` / `JS_ReadObject`:
+
+```ts
+import { JS_Eval, JS_EVAL_TYPE_GLOBAL, JS_EVAL_FLAG_COMPILE_ONLY, JS_WriteObject, JS_ReadObject,
+         JS_EvalFunction, JS_IsException } from 'jseption';
+
+/* at build time */
+const fn = JS_Eval(ctx, src, 'runtime.js', JS_EVAL_TYPE_GLOBAL | JS_EVAL_FLAG_COMPILE_ONLY);
+const bytes = JS_WriteObject(ctx, fn);          /* Uint8Array, or null with an exception pending */
+
+/* at run time, in any runtime of the same jseption version */
+const loaded = JS_ReadObject(ctx2, bytes);      /* JS_EXCEPTION (a SyntaxError) on bad input */
+if (!JS_IsException(loaded)) JS_EvalFunction(ctx2, loaded);
+```
+
+Reading is 18x (host JIT on) to 37x (JIT off, as on iOS) faster than parsing a 300 KB
+bundle. Line numbers in error stacks are kept; function source text (for
+`Function.prototype.toString`) is dropped unless `JS_WRITE_OBJ_SOURCE` is passed. The
+bytes record the jseption version and only load in that version.
+
+**Only load bytecode you produced yourself.** As in QuickJS, the reader checks the format
+(version, bounds, tags, indices, instruction boundaries, a checksum) but not the meaning
+of the byte code: hand-made bytecode can reach engine states the parser never produces,
+so loading untrusted bytecode is not safe even though running untrusted source is. The
+checksum only catches accidental corruption; bytecode that passes through storage or a
+service you do not control needs a signature you verify before loading. See
+`docs/BYTECODE.md` for the format and the measurements.
+
+## Running TypeScript
+
+TypeScript runs the way Node runs it with type stripping: the erasable syntax is
+removed and the rest runs as JavaScript, with no type checking. Pass
+`JS_EVAL_FLAG_STRIP_TYPES` to `JS_Eval`; the command line strips `.ts`, `.mts` and
+`.cts` files, and `--strip-types` applies it to `-e`.
+
+```ts
+JS_Eval(ctx, 'const id = <T,>(x: T): T => x; id<number>(41) + 1', 'app.ts',
+        JS_EVAL_TYPE_GLOBAL | JS_EVAL_FLAG_STRIP_TYPES); // 42
+```
+
+The stripping is built into the parser, so error positions and stacks are those
+of the JavaScript with the types blanked, and plain JavaScript compiles as fast as
+before. As in Node, `enum`, namespaces with values, parameter properties,
+`import x = require(...)` and `export =` are rejected because they need generated
+code. `docs/TYPESCRIPT.md` lists what is stripped and how ambiguous syntax is read.
+
+## JSX and TSX
+
+`JS_EVAL_FLAG_JSX` compiles JSX in the parser, as TypeScript compiles it. With
+`JS_EVAL_FLAG_STRIP_TYPES` as well, it compiles TSX. There is no build step.
+
+* **Modules** use the automatic runtime (`jsx: "react-jsx"`). Elements are calls of
+  `jsx`, `jsxs` and `Fragment`, imported from `react/jsx-runtime`. A
+  `/** @jsxImportSource x */` pragma imports them from `x/jsx-runtime` instead.
+* **Scripts** use the classic runtime (`React.createElement`, or the `@jsx` and
+  `@jsxFrag` pragmas).
+* **The module loader** compiles `.tsx` ids (or a `type: 'tsx'` result) as TSX and
+  `.jsx` ids as JSX.
+* **The command line** runs `.tsx` and `.jsx` files as modules, and `--jsx` applies
+  the flag to `-e`.
+
+```ts
+JS_SetModuleLoader(rt, { load: (id) => files[id] });   // '/app.tsx', 'react/jsx-runtime', ...
+await guestPromiseToHost(ctx, JS_ImportModule(ctx, '/app.tsx'));
+```
+
+`docs/JSX.md` covers the runtimes, pragmas, text and entity rules, and what is not
+supported.
+
+## Modules
+
+ES modules work as in QuickJS: `import` and `export` in every form, live bindings,
+cycles, `export *` with its ambiguity rules, namespace objects, top-level `await`,
+dynamic `import()`, `import.meta`, and JSON modules (`with { type: "json" }`). The
+embedder decides what a specifier means. With no loader set, nothing can be imported,
+so the sandbox stays closed by default.
+
+```ts
+import { JS_SetModuleLoader, JS_ImportModule, guestPromiseToHost, drainJobs } from 'jseption';
+
+JS_SetModuleLoader(rt, {
+  // optional; the default resolves ./, ../ and / against the importing module, URL style
+  resolve: (specifier, referrer) => new URL(specifier, referrer ?? location.href).href,
+  // source text, module bytecode, or { source, url, type, exports }, or a promise of one
+  load: async (url) => ({ source: await (await fetch(url)).text(), url }),
+});
+const ns = await guestPromiseToHost(ctx, JS_ImportModule(ctx, 'https://example.test/app.js'));
+```
+
+The loader may be asynchronous: the whole graph is fetched first, in parallel where
+possible, then linked and evaluated. A synchronous loader keeps everything synchronous.
+`load` can return a host module, `{ exports: { fetch } }`, whose values are marshalled
+into the guest, so guest code can `import { fetch } from "host:net"`. It can also
+return bytecode from `JS_WriteObject`, so a precompiled graph loads without parsing.
+`JS_EvalModule(ctx, source, url)` evaluates source text as a module.
+`JS_Eval` with `JS_EVAL_TYPE_MODULE` keeps QuickJS's behaviour and returns the
+evaluation promise.
+
+The command line runs `.mjs` and `.mts` files as modules, detects `.js` and `.ts` files
+that start with `import` or `export`, and takes `-m` to force module mode or `--script`
+to force a script. Its loader reads relative and absolute paths from disk; bare
+specifiers are not resolved. `docs/MODULES.md` describes the design and the loader
+contract.
+
 ## Passing values between host and guest
 
 Guest values are host values, but the sandbox forbids handing the guest an object whose
@@ -105,7 +213,9 @@ src/atoms.ts        atom table (JS_ATOM_*), tagged integer atoms, atom -> proper
 src/values.ts       JSValue, object metadata, bytecode & frame structures
 src/opcodes.ts      opcode table generated from quickjs-opcode.h
 src/quickjs.ts      runtime, context, property layer over Reflect, calls, conversions, jobs
-src/parser.ts       tokenizer, parser, bytecode emitter, variable/label resolution, eval
+src/parser.ts       tokenizer, parser (with TypeScript type stripping and JSX), bytecode emitter, variable/label resolution, eval
+src/bytecode.ts     bytecode serialization (JS_WriteObject / JS_ReadObject)
+src/module.ts       ES modules: loader, linking, evaluation, namespaces, import(), import.meta
 src/interpreter.ts  JS_CallInternal, closures, generators, async functions (fallback tier)
 src/jit.ts          bytecode -> host JavaScript compiler (stack-to-register, relooper)
 src/jit_runtime.ts  the helpers generated code calls into (the only engine touchpoint)
@@ -181,8 +291,9 @@ Differences from the C implementation:
 * Memory management relies on the host garbage collector: there is no reference
   counting, `JS_FreeValue`/`JS_DupValue` are no-ops, and `WeakRef` / `FinalizationRegistry`
   use the host weak references.
-* ES modules (`import` / `export`), `Atomics`, `SharedArrayBuffer` threads and the
-  bytecode serialization (`JS_WriteObject` / `JS_ReadObject`) are not ported.
+* `Atomics` and `SharedArrayBuffer` threads are not ported. `JS_WriteObject` /
+  `JS_ReadObject` serialize compiled scripts and modules only, not arbitrary values,
+  in a jseption-specific format.
 * Number formatting and parsing use the host conversions (they are specified by
   ECMAScript, so the results are identical).
 

package/dist/bytecode.d.ts

@@ -0,0 +1,6 @@
+import { JSContext, JSValue } from './quickjs.ts';
+export declare const JS_WRITE_OBJ_SOURCE: number;
+export declare const BC_MAGIC: number[];
+export declare const BC_FORMAT = 1;
+export declare function JS_WriteObject(ctx: JSContext, obj: JSValue, flags?: number): Uint8Array | null;
+export declare function JS_ReadObject(ctx: JSContext, buf: Uint8Array): JSValue;