@genroc/eval-node 0.0.0-edge.f77553c → 0.0.0-edge.f8a2edb

package/README.md

@@ -90,21 +90,41 @@ queue worker rather than the HTTP sidecar it used to be:
   that genroc had to be able to reach. A worker only needs outbound access, so it can live
   anywhere — behind NAT, in another trust zone.
 
+The cost of that inversion is that genroc cannot reach a running worker, so anything it needs
+to say has to ride the renewal the worker already makes. **Renewing is therefore mandatory,
+not an optimisation** — a worker that stops renewing is indistinguishable from one that died.
+The response answers per token:
+
+| list | what it means | what this worker does |
+|---|---|---|
+| `renewed` | still yours | carry on |
+| `lost` | already someone else's | stop; do **not** release — that would bump the new holder's claim |
+| `cancelled` | still yours, nobody wants it | abort the script and release the claim |
+
+`cancelled` is how an operator's `genctl cancel` reaches work already running: the evaluation
+is aborted through an `AbortSignal`, its realm is terminated, and no outcome is submitted —
+the process is stopping, so there is nothing left to answer. `renew_before_ms` on the claim
+says how long the worker may wait before renewing again, so the interval is a value it reads
+rather than one it guesses.
+
 ## The task input
 
 The `input` of the external task IS the evaluation request:
 
 ```jsonc
 {
-  "code": "return { fee: input.amount * 0.1 };",  // required — an async function body
-  "input": { "amount": 250 },                     // optional — bound as `input`
-  "timeout_ms": 5000                              // optional — default 5000
+  "code": "export default (input) => ({ fee: input.amount * 0.1 });",  // required — a module
+  "input": { "amount": 250 },                                         // optional — its argument
+  "timeout_ms": 5000                                                  // optional — default 5000
 }
 ```
 
-`code` is the **body of an async function**, so `await` works and the value reaches genroc
-through `return`. It is compiled with `input` and `require` as parameters, under
-`"use strict"` — `require` is what a bundled `import` of a node builtin lands on.
+`code` is an **ES module**, and the evaluator imports it and calls its **default export** with
+`input`. The value it returns reaches genroc; the function may be async, and so may the module's
+top level. A module exporting anything else — or nothing — is a `compile_error`: there is
+nothing to call, and so is one that will not parse; a throw from its top level is a `threw`
+like any other. `import` of a node builtin resolves as it does anywhere else, and that is what
+a bundled builtin lands on.
 
 ## The answer — the failure kind IS the error code
 
@@ -140,12 +160,14 @@ no retry policy to write.
     type: external
     input:
       code: |
-        if (input.amount > 100) {
-          const e = new Error('amount over the limit');
-          e.name = 'LimitExceeded';
-          throw e;
+        export default function (input) {
+          if (input.amount > 100) {
+            const e = new Error('amount over the limit');
+            e.name = 'LimitExceeded';
+            throw e;
+          }
+          return { fee: input.amount * 0.1 };
         }
-        return { fee: input.amount * 0.1 };
       input: "$: input"
     result_schema: { type: object, properties: { fee: { type: number } }, required: [fee] }
     raises:
@@ -164,7 +186,7 @@ no retry policy to write.
 
 - id: script_failed
   switch:
-    - case: 'error.data.name == "LimitExceeded"'
+    - case: 'last_error.data.name == "LimitExceeded"'
       raise: { code: limit_exceeded, message: "the script rejected the amount" }
     - raise: { code: script_failed, message: "the script failed" }
 ```
@@ -227,12 +249,14 @@ the inferred type of what the definition passes; `Output` is what it declares
 `tsc --noEmit`, and bundles — so **a type error is a failed apply**, and a stored definition
 cannot hold code that failed to typecheck.
 
-The bundle is emitted as CJS and wrapped as a function body, so the evaluator needs to know
-nothing about modules. Imports resolve through TypeScript under the same config the check
-ran with, so a `paths` alias that typechecks also bundles. They are inlined at build time, so the string a
-definition version stores is self-contained forever — with one exception: **node builtins
-stay as `require` calls**, which the realm satisfies. A package is frozen into the
-definition; `node:fs` is resolved by whatever runner executes it.
+The bundle is one self-contained ES module whose default export is the author's own — nothing
+wraps or rewrites it, so what the editor checks and what the realm runs are the same module, and
+**a missing default export is a failed apply** rather than a fault at run time. Imports resolve
+through TypeScript under the same config the check ran with, so a `paths` alias that typechecks
+also bundles. They are inlined at build time, so the string a definition version stores is
+self-contained forever — with one exception: **node builtins stay as imports**, which the realm
+resolves. A package is frozen into the definition; `node:fs` is resolved by whatever runner
+executes it.
 
 ### Your tsconfig, your types
 
@@ -258,8 +282,9 @@ never read by genroc, because genctl doubles every `$` on splice.
 
 ## The realm — one Worker per execution
 
-`evaluate()` starts a Worker (`realm.ts`), posts the code into it, and races the reply against the budget;
-`terminate()` runs on every path. That thread is what the contract rests on, and it buys
+`evaluate()` starts a Worker (`realm.ts`), posts the code into it — where a module loader hook
+serves it to `import()`, so the engine numbers the script's frames from the author's own source —
+and races the reply against the budget; `terminate()` runs on every path. That thread is what the contract rests on, and it buys
 exactly three things the previous in-process evaluator could not:
 
 - **The budget is enforced, not merely reported.** A synchronous `while(true){}` never
@@ -271,16 +296,15 @@ exactly three things the previous in-process evaluator could not:
 - **The script's mistakes stay the script's.** An uncaught throw, and `process.exit()`, end
   the realm and come back as a `422` — neither reaches the runner.
 
-It costs about **50ms per execution** end to end for a trivial script (Node 24, M-series
-laptop), a 200 KiB body about 63ms. Roughly 19ms of that is Node re-stripping `worker.ts`'s
-types on every realm — precompiling it to JavaScript would buy that back, and is deliberately
-not done: a build artefact that goes stale against its source fails silently, and this file
-is the one where a wrong line number is invisible.
+It costs about **27ms per execution** end to end for a trivial script (Node 24, M-series
+laptop), a 200 KiB body about 30ms. Roughly 11ms of that is Node re-stripping `realm.ts`'s
+types on every realm — precompiling it to JavaScript measures 17ms and would buy that back, and
+is deliberately not done: a build artefact that goes stale against its source fails silently,
+and this file is the one where a wrong line number is invisible.
 
-That also changes an old trade-off. A subprocess per execution measures ~48ms here — within
-noise of the thread — where on the previous runtime it was ten times the thread's cost. The
-thread no longer wins on price, and a subprocess contains the two things a thread cannot
-(below), so it is the live upgrade path rather than a theoretical one.
+A subprocess per execution measures ~55ms here, twice the thread rather than the ten times it
+cost on the previous runtime. It contains the two things a thread cannot (below), so the upgrade
+path is a live one at a price worth naming rather than a theoretical one.
 
 ## What this is not
 
@@ -291,7 +315,7 @@ thread no longer wins on price, and a subprocess contains the two things a threa
   it needed was surface with nothing behind it. A value that must survive a retry belongs in
   the definition, passed through `input`.
 - **Not a sandbox.** The realm isolates *execution*, not *authority*: a script gets the
-  worker's filesystem, network and environment, and `require` of any node builtin. That is
+  worker's filesystem, network and environment, and any node builtin it imports. That is
   deliberate — a script task is meant to do real work — but the trust boundary stays the
   same-trust-domain one (your genroc, your worker host). It is not the multi-tenant story, and
   nothing here should be mistaken for one. Pulling does move the boundary in one useful way:
@@ -305,9 +329,9 @@ thread no longer wins on price, and a subprocess contains the two things a threa
 - **Concurrency is capped by this worker, not by genroc.** Each evaluation is a thread, and
   `CONCURRENCY` is how many it will claim at once. Raising it past what the host can run turns
   a queue back into threads fighting over a core.
-- **Not where imports and type checking happen.** The evaluator still takes one
-  self-contained function body and knows nothing about TypeScript; `import.ts` is what turns
-  a module into that body, at author time. See above.
+- **Not where imports and type checking happen.** The evaluator takes one self-contained
+  module and knows nothing about TypeScript; `import.ts` is what bundles an author's script and
+  its dependencies into that module, at author time. See above.
 - **`eval.ts` and `realm.ts` know nothing about genroc.** `worker.ts` is the entire
   queue-facing half, which is what keeps the containment strategy swappable — and what lets
   the realm's own properties be tested by calling `evaluate()` directly.

package/dist/eval.js

@@ -1,6 +1,7 @@
-// Evaluation core: run a code string in its OWN realm and classify every outcome into one of
-// the failure kinds in README.md. Nothing here knows about genroc — worker.ts is the only
-// thing that talks to the queue — so this stays testable and the containment stays swappable.
+// Evaluation core: import a script module in its OWN realm, call its default export, and
+// classify every outcome into one of the failure kinds in README.md. Nothing here knows about
+// genroc — worker.ts is the only thing that talks to the queue — so this stays testable and
+// the containment stays swappable.
 //
 // The containment is a Worker per execution (realm.ts). It is what makes the budget real:
 // a synchronous busy loop never yields, so no in-process timer can interrupt it, and only a
@@ -19,13 +20,31 @@ class RealmFault extends Error {
         this.name = "RealmFault";
     }
 }
-export async function evaluate(req) {
+/** Thrown when `signal` aborts: the work was cancelled server-side, so there is no outcome to
+ *  report and worker.ts must release rather than answer. Distinct from RealmFault because the
+ *  runner did not fault -- nothing is wrong, the answer is simply no longer wanted. */
+export class Cancelled extends Error {
+    constructor() {
+        super("cancelled");
+        this.name = "Cancelled";
+    }
+}
+export async function evaluate(req, signal) {
     const budget = typeof req.timeout_ms === "number" ? req.timeout_ms : DEFAULT_TIMEOUT_MS;
     const worker = new Worker(REALM_URL);
     let timer;
+    let onAbort;
     try {
         return await new Promise((resolve, reject) => {
             timer = setTimeout(() => resolve(timedOut(budget)), budget);
+            // The abort is wired to the same promise as the budget, so both settle through the one
+            // finally below -- which is what guarantees the thread is gone before either returns.
+            if (signal) {
+                if (signal.aborted)
+                    reject(new Cancelled());
+                onAbort = () => reject(new Cancelled());
+                signal.addEventListener("abort", onAbort, { once: true });
+            }
             worker.once("message", (reply) => resolve(reply));
             // A script may end its own realm (`process.exit()`), which is not a throw and would
             // otherwise present as a hang until the budget expired. Our own terminate() raises
@@ -37,6 +56,8 @@ export async function evaluate(req) {
     }
     finally {
         clearTimeout(timer);
+        if (signal && onAbort)
+            signal.removeEventListener("abort", onAbort);
         // Awaited, and the whole point: on the timeout path a thread is still burning a core, and
         // resolving before it is gone would report an evaluation the machine is still running.
         await worker.terminate();

package/dist/import.js

@@ -7,9 +7,10 @@
 // code; "build" typechecks and bundles. A separate types hook would mean a second `tsc`
 // over the same project.
 import { spawn } from "node:child_process";
+import { existsSync } from "node:fs";
 import { access, mkdir, writeFile } from "node:fs/promises";
 import { builtinModules } from "node:module";
-import { dirname, join, relative } from "node:path";
+import { dirname, join, relative, resolve } from "node:path";
 import { fileURLToPath } from "node:url";
 import commonjs from "@rollup/plugin-commonjs";
 import json from "@rollup/plugin-json";
@@ -86,7 +87,9 @@ function objectType(s, used) {
     const required = new Set(s.required ?? []);
     const lines = [];
     for (const [key, sub] of Object.entries(props)) {
-        const doc = typeof sub.description === "string" ? `  /** ${sub.description} */\n` : "";
+        const doc = typeof sub.description === "string"
+            ? `  /** ${sub.description} */\n`
+            : "";
         lines.push(`${doc}  ${propKey(key)}${required.has(key) ? "" : "?"}: ${tsType(sub, used)};`);
     }
     if (s.additionalProperties && typeof s.additionalProperties === "object") {
@@ -102,7 +105,7 @@ function union(parts) {
 }
 const IDENT = /^[A-Za-z_$][A-Za-z0-9_$]*$/;
 const propKey = (k) => (IDENT.test(k) ? k : JSON.stringify(k));
-const identifier = (n) => (IDENT.test(n) ? n : `Def_${n.replace(/[^A-Za-z0-9_$]/g, "_")}`);
+const identifier = (n) => IDENT.test(n) ? n : `Def_${n.replace(/[^A-Za-z0-9_$]/g, "_")}`;
 function deref(s, defs) {
     let cur = s;
     for (let i = 0; cur && typeof cur.$ref === "string" && i < 16; i++) {
@@ -110,21 +113,14 @@ function deref(s, defs) {
     }
     return cur;
 }
-/** The manifest's `input` is the type of the whole ACTION input — for /eval that is
- *  `{code, input, timeout_ms, …}`, and only its `input` field is bound as the script's
- *  parameter. genroc cannot know that; this file owns the evaluator's wire contract, so
- *  the navigation belongs here rather than in genctl. */
-function scriptInput(site, defs) {
-    const action = deref(site.input, defs);
-    const bound = action?.properties?.input;
-    return bound ?? site.input;
-}
 /** Emits one named type per reachable $def rather than inlining: a task output may
  *  reference itself (specs/recursive-type-inference.md) and inlining would not terminate. */
-function declarations(site, defs) {
+function declarations(at) {
+    const { site, where } = at;
+    const defs = where.$defs ?? {};
     const used = new Set();
-    const input = tsType(scriptInput(site, defs), used);
-    const output = tsType(site.output, used);
+    const input = tsType(site.types?.Input, used);
+    const output = tsType(site.types?.Output, used);
     const emitted = [];
     const done = new Set();
     while (true) {
@@ -137,7 +133,7 @@ function declarations(site, defs) {
     }
     return [
         "// Generated by genroc. Do not edit - regenerate with `genctl types`.",
-        `// ${site.process}${site.task ? ` / ${site.task}` : ""}  (${site.pointer})`,
+        `// ${where.name}  (${address(site.pointer)})`,
         "",
         ...emitted,
         emitted.length ? "" : "",
@@ -165,7 +161,7 @@ async function nearestTsconfig(from, root) {
             return null;
     }
 }
-async function typecheck(root, sites) {
+async function typecheck(sites) {
     // NOT `.genroc`: that is the project config FILE, and a directory of the same name cannot
     // coexist with it. The suffix is what keeps the scratch area out of its way.
     const dir = join(root, ".genroc-cache");
@@ -173,13 +169,13 @@ async function typecheck(root, sites) {
     // One tsc per distinct base config: `extends` takes a single base, so merging two would
     // check each script under the other author's options.
     const groups = new Map();
-    for (const site of sites) {
-        const base = (await nearestTsconfig(dirname(site.path), root)) ?? "";
+    for (const at of sites) {
+        const base = (await nearestTsconfig(dirname(at.file), root)) ?? "";
         const group = groups.get(base);
         if (group)
-            group.push(site);
+            group.push(at);
         else
-            groups.set(base, [site]);
+            groups.set(base, [at]);
     }
     let n = 0;
     for (const [base, group] of groups) {
@@ -200,7 +196,10 @@ async function typecheck(root, sites) {
                 // base config there is nothing to opt in with, so the default stays none.
                 ...(base ? {} : { types: [] }),
             },
-            files: group.flatMap((s) => [relative(dir, s.path), relative(dir, typesPathFor(s.path))]),
+            files: group.flatMap((s) => [
+                relative(dir, s.file),
+                relative(dir, typesPathFor(s.file)),
+            ]),
             // `files` overrides the base's, but a base `include` survives beside it and would
             // drag the author's whole tree in, to be checked under the worker lib.
             include: [],
@@ -250,7 +249,10 @@ const transpile = {
         return { code: out.outputText, map: out.sourceMapText ?? null };
     },
 };
-const BUILTIN = new Set([...builtinModules, ...builtinModules.map((m) => `node:${m}`)]);
+const BUILTIN = new Set([
+    ...builtinModules,
+    ...builtinModules.map((m) => `node:${m}`),
+]);
 /** Resolves imports through TYPESCRIPT, using the same config the typecheck ran under, so a
  *  `paths` alias that compiles also bundles. Reimplementing `paths` here would be a second
  *  resolver to keep in agreement with tsc; this one cannot disagree.
@@ -270,22 +272,25 @@ function tsResolve(configPath) {
             const { resolvedModule } = ts.resolveModuleName(source, importer, options, ts.sys);
             if (!resolvedModule || resolvedModule.isExternalLibraryImport)
                 return null;
-            return resolvedModule.resolvedFileName.endsWith(".d.ts") ? null : resolvedModule.resolvedFileName;
+            return resolvedModule.resolvedFileName.endsWith(".d.ts")
+                ? null
+                : resolvedModule.resolvedFileName;
         },
     };
 }
-/** Bundles to CJS and wraps it as an async function BODY, which is what /eval compiles.
- *  The runtime stays unchanged: bundling is entirely the importer's job, and the string it
- *  produces is self-contained, so a definition version pins its code forever. */
-async function bundle(site, root) {
-    // Builtins are EXTERNALISED as `require` calls that worker.ts satisfies. Anything else
-    // unresolved is a REFUSAL, not an external: rollup's default is to leave it as a require
+/** Bundles to a self-contained ES module, which is what the evaluator imports: the default
+ *  export it calls is the author's own, so nothing wraps or rewrites the code between the two.
+ *  Bundling is entirely the importer's job, so a definition version pins its code forever. */
+async function bundle(at) {
+    const site = at.site;
+    // Builtins are EXTERNALISED as imports the realm resolves natively. Anything else
+    // unresolved is a REFUSAL, not an external: rollup's default is to leave it as an import
     // of a module that will not be there, which bundles clean and fails at runtime.
     const built = await rollup({
-        input: site.path,
+        input: at.file,
         external: (id) => BUILTIN.has(id),
         plugins: [
-            tsResolve(await nearestTsconfig(dirname(site.path), root)),
+            tsResolve(await nearestTsconfig(dirname(at.file), root)),
             nodeResolve({ extensions: [".ts", ".tsx", ".mjs", ".js", ".json"] }),
             commonjs(),
             // A `.json` import is a data file inlined at build time, which the previous bundler
@@ -295,22 +300,21 @@ async function bundle(site, root) {
         ],
         onwarn(warning) {
             if (warning.code === "UNRESOLVED_IMPORT") {
-                die(`${site.path}: cannot resolve ${warning.exporter ?? "an import"} — is it installed?`);
+                die(`${at.file}: cannot resolve ${warning.exporter ?? "an import"} — is it installed?`);
             }
         },
-    }).catch((e) => die(`${site.path}: ${e instanceof Error ? e.message : String(e)}`));
-    const { output } = await built.generate({ format: "cjs", exports: "auto", inlineDynamicImports: true });
+    }).catch((e) => die(`${at.file}: ${e instanceof Error ? e.message : String(e)}`));
+    const { output } = await built.generate({
+        format: "es",
+        inlineDynamicImports: true,
+    });
     await built.close();
-    const cjs = output[0].code;
-    return [
-        "var module = { exports: {} }, exports = module.exports;",
-        cjs,
-        "var __genroc_main = module.exports.default ?? module.exports;",
-        'if (typeof __genroc_main !== "function") {',
-        `  throw new Error(${JSON.stringify(`${site.path} has no default export function`)});`,
-        "}",
-        "return await __genroc_main(input);",
-    ].join("\n");
+    // Refused here rather than in the realm: the evaluator can only report it against a running
+    // instance, and the file it names is on this machine.
+    if (!output[0].exports.includes("default")) {
+        die(`${at.file}: a script must \`export default\` the function to run`);
+    }
+    return output[0].code;
 }
 // ── main ───────────────────────────────────────────────────────────────────────
 const stdin = await new Promise((resolve, reject) => {
@@ -321,32 +325,73 @@ const stdin = await new Promise((resolve, reject) => {
     process.stdin.on("error", reject);
 });
 const manifest = JSON.parse(stdin);
-if (!manifest || !Array.isArray(manifest.sites))
+// genctl runs a resolver with the project root as its cwd, so the manifest need not say so.
+const root = process.cwd();
+if (!manifest || !Array.isArray(manifest.processes))
     die("stdin is not a genroc resolver manifest");
+/** A pointer as the address it is, so a generated comment or an error can be pasted into
+ *  `genctl schema`. A key no identifier can spell is bracketed, as the grammar spells it. */
+function address(pointer) {
+    return pointer
+        .map((seg) => typeof seg === "string" && /^[A-Za-z_][A-Za-z0-9_]*$/.test(seg)
+        ? `.${seg}`
+        : `[${JSON.stringify(seg)}]`)
+        .join("")
+        .replace(/^\./, "");
+}
+const located = manifest.processes.flatMap((where) => where.sites.map((site) => ({
+    site,
+    where,
+    file: resolve(where.dir, site.argument),
+})));
+// genctl is agnostic about what a script is for; the contract that an evaluation request carries
+// its module in `code` is THIS resolver's, so it is the one that checks a directive landed there.
+// `child` is the shape the scaffold uses — a call to a process that forwards to the evaluator —
+// and `external` is the same request made directly.
+for (const at of located) {
+    // Two halves, from the two places that carry them: the action's KIND is a field, and WHERE in
+    // it the directive sits is the pointer. A child call is the shape the scaffold generates — to a
+    // process that forwards to the evaluator — and an external task is the same request made
+    // directly.
+    const kind = at.site.level === "action" ? (at.site.action ?? "") : "";
+    const slot = at.site.pointer.slice(-2).join(".");
+    if ((kind !== "child" && kind !== "external") || slot !== "input.code") {
+        die(`${join(at.where.dir, at.where.file)}: ${address(at.site.pointer)}: an evaluated script ` +
+            "belongs in the `code` field of a child or external task's input, and this is " +
+            `${kind ? `a ${kind} task's ` : ""}\`${slot}\`.`);
+    }
+}
+// genctl no longer stats the argument — it does not know it is a file — so the resolver that
+// does is the one that must say when it is not there.
+for (const at of located) {
+    if (!existsSync(at.file)) {
+        die(`${join(at.where.dir, at.where.file)}: ${address(at.site.pointer)}: ` +
+            `"$import: ${at.site.argument}" names no file (looked at ${at.file})`);
+    }
+}
 // One script at two sites with different input types is a refusal, not a union: the union
 // is sound and would typecheck a body that is wrong at one of the sites.
 const byPath = new Map();
-for (const site of manifest.sites) {
-    const seen = byPath.get(site.path);
-    const defsOf = (x) => (manifest.schemas[x.process]?.$defs ?? {});
+for (const at of located) {
+    const seen = byPath.get(at.file);
     if (seen &&
-        JSON.stringify(scriptInput(seen, defsOf(seen))) !==
-            JSON.stringify(scriptInput(site, defsOf(site)))) {
-        die(`${site.path} is imported at ${seen.pointer} and ${site.pointer} with different input types.\n` +
+        JSON.stringify(seen.site.types) !== JSON.stringify(at.site.types)) {
+        die(`${at.file} is imported at ${address(seen.site.pointer)} and ` +
+            `${address(at.site.pointer)} with different input types.\n` +
             "Split it into two scripts, or make the two call sites pass the same shape.");
     }
-    byPath.set(site.path, site);
+    byPath.set(at.file, at);
 }
-for (const site of byPath.values()) {
-    const defs = (manifest.schemas[site.process]?.$defs ?? {});
-    await write(typesPathFor(site.path), declarations(site, defs));
+for (const at of byPath.values()) {
+    await write(typesPathFor(at.file), declarations(at));
 }
 if (manifest.mode === "types") {
+    console.log(JSON.stringify(manifest, null, 2));
     process.exit(0);
 }
-await typecheck(manifest.root, [...byPath.values()]);
+await typecheck([...byPath.values()]);
 const code = [];
-for (const site of manifest.sites) {
-    code.push(await bundle(site, manifest.root));
+for (const at of located) {
+    code.push(await bundle(at));
 }
 process.stdout.write(JSON.stringify({ code }));