exprforge 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,5 +1,7 @@
1
1
  # ExprForge 🔢🔨
2
2
 
3
+ ![ExprForge — a blacksmith forging 3a² + 2ab − b² = (a+b)² − 2ab on a glowing anvil](https://raw.githubusercontent.com/theraccoonbear/exprforge/main/assets/expression-forge.png)
4
+
3
5
  [![npm version](https://img.shields.io/npm/v/exprforge.svg)](https://www.npmjs.com/package/exprforge)
4
6
  [![TypeScript](https://github.com/theraccoonbear/exprforge/actions/workflows/test-typescript.yml/badge.svg)](https://github.com/theraccoonbear/exprforge/actions/workflows/test-typescript.yml)
5
7
  [![Python](https://github.com/theraccoonbear/exprforge/actions/workflows/test-python.yml/badge.svg)](https://github.com/theraccoonbear/exprforge/actions/workflows/test-python.yml)
@@ -18,29 +20,34 @@
18
20
  [![Scheme](https://github.com/theraccoonbear/exprforge/actions/workflows/test-scheme.yml/badge.svg)](https://github.com/theraccoonbear/exprforge/actions/workflows/test-scheme.yml)
19
21
  [![COBOL](https://github.com/theraccoonbear/exprforge/actions/workflows/test-cobol.yml/badge.svg)](https://github.com/theraccoonbear/exprforge/actions/workflows/test-cobol.yml)
20
22
 
21
- Author a math expression once, as a small AST, and emit verified,
22
- identical-behavior implementations in JavaScript, TypeScript, Python, C#,
23
- Lua, QB64, C, Java, Go, Rust, Perl, PHP, Julia, Fortran, Zig, Scheme
24
- (Guile), and COBOL (GnuCOBOL) — plus a native in-process evaluator and a
25
- printer for exprforge's own readable syntax (see `fn`/`expr` below).
23
+ ## Brief
26
24
 
27
- No required dependencies. You can build the AST directly with plain JS
28
- functions, or author it as readable infix text via `expr`/`fn` (see
29
- below) — either way, the same tree is walked once per target.
25
+ ExprForge authors a math formula once, as a small AST, and emits
26
+ verified, identical-behavior implementations in JavaScript, TypeScript,
27
+ Python, C#, Lua, QB64, C, Java, Go, Rust, Perl, PHP, Julia, Fortran, Zig,
28
+ Scheme (Guile), and COBOL (GnuCOBOL) — plus a native in-process
29
+ evaluator and a printer for its own readable syntax. No required
30
+ dependencies.
30
31
 
31
32
  **[â–¶ Try it live](https://theraccoonbear.github.io/exprforge/)** — write
32
- a `` fn`...` `` formula in the browser and watch it emitted across every
33
- target language at once, no install required. Runs the real, current
34
- library (see `playground/`), not a frozen demo build.
33
+ a formula in the browser and watch it emitted across every target
34
+ language at once, no install required. Runs the real, current library
35
+ (see `playground/`), not a frozen demo build.
35
36
 
36
- ## Why
37
+ ## Motivation
37
38
 
38
- Codegen tools like SymPy already turn math expressions into code for
39
- mainstream languages. This exists for two things SymPy doesn't do:
39
+ This grew out of a real, recurring problem in a larger multi-language
40
+ system (internally: SSS) that needed the *same* math to be true in
41
+ several independently-deployed pieces written in different languages at
42
+ once — not "translate this code," but "prove these N implementations of
43
+ one formula actually agree," which is a narrower, checkable claim.
44
+ Codegen tools like SymPy already turn expressions into code for
45
+ mainstream languages; this exists for the two things that leaves open:
40
46
 
41
- - Targets like QB64/BASIC that no general codegen project supports.
42
- - A conformance test harness that actually proves the emitted targets
43
- agree numerically, not just that they compile.
47
+ - Targets like QB64/BASIC or COBOL that no general codegen project
48
+ reaches at all.
49
+ - A conformance harness that actually *proves* the emitted targets agree
50
+ numerically, compiled and run for real — not just that they compile.
44
51
 
45
52
  Two shapes of real use this tends to fall into:
46
53
 
@@ -55,49 +62,63 @@ Two shapes of real use this tends to fall into:
55
62
  target and the new one, and let the conformance suite prove they agree
56
63
  before cutover, not after.
57
64
 
58
- Neither is "translate my code for me" — it's "prove two independent
59
- implementations of one formula actually match," which is a narrower,
60
- checkable claim.
61
-
62
- ## Layered by design
63
-
64
- Everything here is layered, and the layers don't reach back into each
65
- other — worth being explicit about, especially if you're evaluating this
66
- for something like a migration and want to know exactly what you're
67
- trusting:
68
-
69
- - **The AST and its emitters — the whole value proposition.** `ast.js`'s
70
- builders (`num`, `v`, `add`, `mul`, `letIn`, `select`, `outputs`, ...)
71
- build a plain tree of plain objects; every `emitters/<lang>.js` file
72
- turns that tree into target-language source text. This is also the
73
- *entire* dependency graph for it: every emitter requires only
74
- `emitters/base.js` and `ast.js` — nothing else in this repo. No parser,
75
- no custom syntax, no interpreter sits between your AST and the code it
76
- emits. Every example in `samples/` is built this way, and the library
77
- worked exactly this way for its first two published releases, before
78
- anything below existed.
79
- - **`expr`/`fn` — optional authoring sugar.** Nested builder calls
80
- (`add(mul(v("a"), v("b")), num(1))`) get hard to read past a few terms.
81
- `expr`/`fn` are a small hand-rolled tokenizer and recursive-descent
82
- parser that turn ordinary infix text (`` expr`a * b + 1` ``) into
83
- *exactly* the same tree the builders would — checked by structural unit
84
- tests and a full print-reparse-evaluate round trip across every sample
85
- this project has (see "Testing" below), not just asserted. It's
86
- genuinely optional: nothing in the AST/emitter layer above calls into
87
- it or imports it. Don't want a parser in your dependency graph, for a
88
- security review or otherwise? Don't call `expr`/`fn` — build the tree
89
- with the plain functions instead, and every emitter behaves identically
90
- either way.
91
- - **The native evaluator and the `expr`-syntax printer — additive
92
- conveniences.** `evaluate()` (compute a result in-process, no
93
- target-language toolchain needed) and the `expr` emitter target (print
94
- any AST back out as readable text) sit off to the side the same way
95
- `expr`/`fn` do — nothing else depends on them either.
65
+ ## Intents
66
+
67
+ **What it does**: turns one small, pure-arithmetic AST into
68
+ identical-behavior source text for 16 real target languages, a native
69
+ evaluator, and its own readable printer — all from the same tree, walked
70
+ once per target.
71
+
72
+ **What it deliberately won't do** — not gaps waiting on a future
73
+ release, but a boundary held on purpose everywhere in this project:
74
+
75
+ - **No control flow.** No loops, no branches, no generated function
76
+ calling another generated function *at runtime*. This is an
77
+ expression AST, not a program AST. `loadMacro`/`loadExpr` (see below)
78
+ let one definition reference another, but only by inline expansion at
79
+ build time, resolved in declaration order — never a real call, never
80
+ recursion (which would need a stack this library doesn't have), never
81
+ a call graph.
82
+ - **No RNG.** Can't be made to produce identical output across
83
+ languages, so it isn't offered as if it could.
84
+ - **No arbitrary precision / complex numbers.** `float64` only, for now.
85
+
86
+ **What it is not**: a general transpiler ("translate my code for me").
87
+ It's narrower and more checkable than that — "prove two independent
88
+ implementations of one formula actually match."
89
+
90
+ **Trust only the layer you need.** Everything below is genuinely
91
+ layered, and the layers don't reach back into each other:
92
+
93
+ - **The AST and its emitters are the whole value proposition, and the
94
+ entire dependency graph.** `ast.js`'s builders (`num`, `v`, `add`,
95
+ `mul`, `letIn`, `select`, `outputs`, ...) build a plain tree of plain
96
+ objects; every `emitters/<lang>.js` file turns that tree into
97
+ target-language source text. Every emitter requires only
98
+ `emitters/base.js` and `ast.js` — nothing else in this repo. No
99
+ parser, no custom syntax, no interpreter sits between your AST and the
100
+ code it emits. The library worked exactly this way for its first two
101
+ published releases, before anything below existed.
102
+ - **`expr`/`fn`/macros — optional authoring sugar**, all the way up to
103
+ the flashiest multi-function syntax below. Every layer of sugar turns
104
+ into *exactly* the same tree the raw builders would — checked by
105
+ structural unit tests and a full print-reparse-evaluate round trip
106
+ across every sample this project has (see "Testing"), not just
107
+ asserted. It's genuinely optional: nothing in the AST/emitter layer
108
+ calls into or imports any of it. Don't want a parser in your
109
+ dependency graph, for a security review or otherwise? Don't call
110
+ `expr`/`fn`/`loadMacro` — build the tree with the plain functions
111
+ instead, and every emitter behaves identically either way.
112
+ - **The native evaluator and the `expr`-syntax printer are additive
113
+ conveniences** that sit off to the side the same way — nothing else
114
+ depends on them either.
96
115
 
97
116
  If you only care about "does this correctly turn my AST into
98
117
  COBOL/Java/whatever" — `ast.js` and the one `emitters/<lang>.js` file you
99
- care about are the entire surface that matters. Everything else is there
100
- if you want it, invisible if you don't.
118
+ care about are the entire surface that matters. The rest of this README
119
+ walks in from the flashiest end first, then works back down toward that
120
+ same bottom layer, one step of sugar removed at a time — skip straight
121
+ to whichever depth you actually plan to trust.
101
122
 
102
123
  ## Install
103
124
 
@@ -105,10 +126,125 @@ if you want it, invisible if you don't.
105
126
  npm install exprforge
106
127
  ```
107
128
 
129
+ ## Examples, flashiest first
130
+
131
+ The showcase feature: several function definitions in one buffer, a
132
+ later one referencing an earlier one by name, with dot-field access into
133
+ a multi-output result — all inline-expanded at parse time, never a real
134
+ runtime call (see "Intents" above for why that distinction is
135
+ load-bearing, and "Macros and externs" below for the full mechanism).
136
+ This is exactly what the [live playground](https://theraccoonbear.github.io/exprforge/)'s
137
+ editor buffer accepts:
138
+
139
+ ```js
140
+ const { loadExprSource, evaluate, emit } = require("exprforge");
141
+
142
+ const defs = loadExprSource(`
143
+ cross3(ax, ay, az, bx, by, bz):
144
+ let rx = ay * bz - az * by;
145
+ let ry = az * bx - ax * bz;
146
+ let rz = ax * by - ay * bx;
147
+ return { rx, ry, rz };
148
+
149
+ crossLength(ax, ay, az, bx, by, bz):
150
+ let c = cross3(ax, ay, az, bx, by, bz);
151
+ return sqrt(c.rx^2 + c.ry^2 + c.rz^2);
152
+ `);
153
+
154
+ evaluate(defs.crossLength, [1, 0, 0, 0, 1, 0]); // 1
155
+ emit(defs.crossLength, "rust").source; // a real fn crossLength(...) -- no trace of cross3 left
156
+ ```
157
+
158
+ `cross3` never appears in `crossLength`'s emitted output, in any target —
159
+ by the time `loadExprSource` returns, `defs.crossLength` is
160
+ self-contained arithmetic, `cross3`'s formula copied in and simplified
161
+ away. This is also why calling `cross3` from *inside itself* isn't just
162
+ discouraged, it's structurally impossible: a definition only becomes
163
+ referenceable by whatever's declared *after* it, never by itself —
164
+ covered in full under "Macros and externs" below, including exactly what
165
+ happens if you try.
166
+
167
+ This is the top of the sugar. The rest of this README works back down
168
+ from here, one layer at a time, showing the *exact same formula* —
169
+ cross product magnitude — at each level of undress.
170
+
171
+ ### One layer down: `fn` + `loadMacro`, no file/buffer needed
172
+
173
+ Same result, without a multi-definition buffer: register `cross3` once
174
+ with `loadMacro`, then reference it from an ordinary `fn` template.
175
+ `loadExprSource` above is sugar for exactly this loop, run once per
176
+ definition in the buffer.
177
+
178
+ ```js
179
+ const { loadMacro, fn, evaluate } = require("exprforge");
180
+
181
+ loadMacro("cross3", fn`
182
+ cross3(ax, ay, az, bx, by, bz):
183
+ let rx = ay * bz - az * by;
184
+ let ry = az * bx - ax * bz;
185
+ let rz = ax * by - ay * bx;
186
+ return { rx, ry, rz };
187
+ `);
188
+
189
+ const crossLength = fn`
190
+ crossLength(ax, ay, az, bx, by, bz):
191
+ let c = cross3(ax, ay, az, bx, by, bz);
192
+ return sqrt(c.rx^2 + c.ry^2 + c.rz^2);
193
+ `;
194
+
195
+ evaluate(crossLength, [1, 0, 0, 0, 1, 0]); // 1
196
+ ```
197
+
198
+ ### Another layer down: `expr`, hand-inlined, no macro at all
199
+
200
+ Drop the macro entirely and write the whole thing as one infix
201
+ expression — `cross3`'s formula copied in by hand, exactly what the
202
+ macro layer above did for you automatically:
203
+
204
+ ```js
205
+ const { expr, evaluate } = require("exprforge");
206
+
207
+ const body = expr`sqrt((ay*bz - az*by)^2 + (az*bx - ax*bz)^2 + (ax*by - ay*bx)^2)`;
208
+ const crossLength = { name: "crossLength", params: ["ax", "ay", "az", "bx", "by", "bz"], body };
209
+
210
+ evaluate(crossLength, [1, 0, 0, 0, 1, 0]); // 1
211
+ ```
212
+
213
+ ### The bottom: raw AST builders, no parser involved at all
214
+
215
+ The actual library API — no `expr`/`fn`/macros in the dependency graph
216
+ whatsoever, just plain function calls building a plain tree of plain
217
+ objects. Everything above compiles down to exactly this shape:
218
+
219
+ ```js
220
+ const { v, add, sub, mul, call, letChain, evaluate } = require("exprforge");
221
+
222
+ const rx = sub(mul(v("ay"), v("bz")), mul(v("az"), v("by")));
223
+ const ry = sub(mul(v("az"), v("bx")), mul(v("ax"), v("bz")));
224
+ const rz = sub(mul(v("ax"), v("by")), mul(v("ay"), v("bx")));
225
+
226
+ const crossLength = {
227
+ name: "crossLength",
228
+ params: ["ax", "ay", "az", "bx", "by", "bz"],
229
+ body: letChain(
230
+ [["rx", rx], ["ry", ry], ["rz", rz]],
231
+ call("sqrt", add(mul(v("rx"), v("rx")), mul(v("ry"), v("ry")), mul(v("rz"), v("rz")))),
232
+ ),
233
+ };
234
+
235
+ evaluate(crossLength, [1, 0, 0, 0, 1, 0]); // 1
236
+ ```
237
+
238
+ If you're only willing to trust *this* layer — no parser, no macro
239
+ expansion, nothing but `ast.js` and one `emitters/<lang>.js` file — this
240
+ is the entire surface you need to read. Everything above it is sugar
241
+ that provably lowers to this same shape (see "Testing"); nothing below
242
+ it exists.
243
+
108
244
  ## Usage
109
245
 
110
246
  ```js
111
- const { expr, emitAll } = require("exprforge");
247
+ const { expr, emit, emitMany } = require("exprforge");
112
248
 
113
249
  const fn = {
114
250
  name: "lerp",
@@ -116,18 +252,16 @@ const fn = {
116
252
  body: expr`(b - a) * t + a`,
117
253
  };
118
254
 
119
- const outputs = emitAll(fn);
255
+ console.log(emit(fn, "rust").source);
256
+ console.log(emit(fn, "c").source);
257
+
258
+ // Need several targets at once? emitMany() runs each in isolation --
259
+ // one target's failure shows up as { source: null, error } for that
260
+ // target alone, not a thrown exception that blanks every other result.
261
+ const outputs = emitMany(fn, ["rust", "c", "python"]); // omit langs for every registered target
120
262
  console.log(outputs.rust.source);
121
- console.log(outputs.c.source);
122
263
  ```
123
264
 
124
- `` expr`(b - a) * t + a` `` and `add(v("a"), mul(sub(v("b"), v("a")), v("t")))`
125
- build the *exact same tree* — `expr` (see below) is optional infix syntax
126
- sugar over the same builders, not a different API. Every example below
127
- still uses the builders directly, since that's what `expr` compiles down
128
- to and what you'll reach for once a formula needs `${...}`-spliced
129
- sub-expressions.
130
-
131
265
  ## Samples
132
266
 
133
267
  `samples/` has worked, non-trivial examples (also exported from the
@@ -136,7 +270,8 @@ package individually, or together as `samples`):
136
270
  - `samples/catmull-rom.js` — uniform Catmull-Rom spline interpolation.
137
271
  - `samples/fibonacci.js` — nth Fibonacci number via Binet's closed form.
138
272
  There's no loop/recursion version because exprforge has no control flow
139
- (see below) — this is what "fibonacci" looks like as a pure expression.
273
+ (see "Intents" above) — this is what "fibonacci" looks like as a pure
274
+ expression.
140
275
  - `samples/spline-frame.js` — Gram-Schmidt frame construction for spline
141
276
  paths (worldUp selection, tangent normalization with a safe-division
142
277
  fallback, roll). 4 suites exercising `letIn`/`cmp`/`select`/`outputs` on
@@ -147,11 +282,16 @@ package individually, or together as `samples`):
147
282
  - `samples/kitchen-sink.js` — not a worked example: a synthetic function
148
283
  that calls all 22 supported Math functions in one expression, existing
149
284
  purely as a conformance-test fixture. It's what caught Go's and Rust's
150
- `sign()` disagreeing with everyone else at exactly zero (see below) —
151
- the other samples between them only ever exercised 5 of the 22.
285
+ `sign()` disagreeing with everyone else at exactly zero (see "Testing")
286
+ — the other samples between them only ever exercised 5 of the 22.
152
287
  - `samples/math-demo.js` — also not a worked example: a conformance-test
153
288
  fixture exercising every `exprforge/math` helper (see below) in one
154
289
  suite.
290
+ - `samples/macro-demo.js` — also not a worked example: a conformance-test
291
+ fixture specifically for the `loadMacro(name, fn\`...\`)` AST-function
292
+ tier, proving its internal gensym'd let-renaming produces valid
293
+ identifiers on every real target, not just `evaluate()` (which can't
294
+ see codegen at all — see "Macros and externs").
155
295
 
156
296
  `npm run build` emits all of them, for every target language, into `out/`.
157
297
 
@@ -161,7 +301,14 @@ package individually, or together as `samples`):
161
301
  floor ceil round trunc sign min max hypot`
162
302
 
163
303
  Add more by extending a target's `calls` table in `emitters/<lang>.js`.
164
- Requesting an unmapped function throws at build time, not silently.
304
+ Requesting an unmapped function throws at build time, not silently. Each
305
+ one takes exactly 1 argument (everything above `pow`) or 2 (`pow`
306
+ onward) — calling one with the wrong count throws too, for every target
307
+ including the `expr` printer: unlike an unmapped *name* (which `expr`
308
+ prints through unchanged, having no fixed math library of its own to
309
+ validate against), a wrong argument *count* is a structurally malformed
310
+ call regardless of target, checked unconditionally at the same tier as
311
+ `checkUnboundVars` — see `primitives.js`.
165
312
 
166
313
  ## Math utilities (`exprforge/math`)
167
314
 
@@ -203,6 +350,179 @@ conversion, Rodrigues rotation, and a full Gram-Schmidt frame — see
203
350
  `samples/spline-frame.js` for those, and
204
351
  `docs/v0.2.0-math-utilities.md` for the full design rationale.
205
352
 
353
+ `require("exprforge/math")` also registers every one of these (except
354
+ `clamp`) as macros, so they're usable directly inside `fn`/`expr`
355
+ template **text**, not just from JS-authoring — see the next section.
356
+
357
+ ## Macros and externs (`loadMacro` / `loadExtern`)
358
+
359
+ The 22 Math functions above are fixed and built in — call them
360
+ **primitives**. `loadMacro`/`loadExtern` register additional names usable
361
+ the same way, inside `fn`/`expr` template text (and in `.expr` files, see
362
+ `loadExpr` below) — this is the mechanism behind every example in
363
+ "Examples, flashiest first" above, with very different guarantees
364
+ depending on which one you reach for:
365
+
366
+ ```js
367
+ const { loadMacro, fn, evaluate } = require("exprforge");
368
+ require("exprforge/math"); // registers dot3/len3/cross3/normalize3/safeDiv as macros
369
+
370
+ const rodrigues = fn`
371
+ rodrigues(t0x, t0y, t0z, t1x, t1y, t1z):
372
+ let b = cross3(t0x, t0y, t0z, t1x, t1y, t1z);
373
+ let bLen = sqrt(b.x^2 + b.y^2 + b.z^2);
374
+ return bLen;
375
+ `;
376
+ ```
377
+
378
+ - **`loadMacro(name, def)`** — `def` is a plain JS function
379
+ `(...argNodes) => Node` or `(...argNodes) => { field: Node, ... }`,
380
+ built entirely from existing `ast.js` primitives/other macros
381
+ (`exprforge/math`'s own `dot3`/`len3`/`cross3`/`normalize3`/`safeDiv`
382
+ are registered exactly this way — see `math/index.js`).
383
+ **Inline-expanded** into the caller's AST at build time, never emitted
384
+ as a real call in any target: the emitted output is self-contained
385
+ arithmetic, identical in spirit to writing the expansion out by hand.
386
+ Safe by construction — if `def` returns real `ast.js` Nodes, the result
387
+ is exactly as trustworthy as anything else this library emits. `def`
388
+ can also be an AST function definition directly (e.g. straight out of
389
+ `fn\`...\``: `loadMacro("foo", fn\`foo(x): return x * 2;\`)`), sugar
390
+ for the same thing — this is exactly how "Examples, flashiest first"
391
+ above registers `cross3`.
392
+
393
+ A macro returning multiple named values (like `cross3`'s `{x, y, z}`)
394
+ must be bound with `let` before its fields are readable — `let b =
395
+ cross3(...); b.x` — using it bare inside a larger expression throws a
396
+ clear error rather than guessing which field you meant.
397
+
398
+ **Gotcha: the `let` name itself never becomes a real value.** `let b =
399
+ cross3(...);` doesn't bind `b` to anything you can reference bare —
400
+ `b` is consumed entirely as a naming *prefix* for `b`'s flattened
401
+ fields (internally, something like `b__x`/`b__y`/`b__z`). Referencing
402
+ `b` on its own — including accidentally, via `fn`'s own `return { b };`
403
+ shorthand — throws immediately, naming the fields that actually exist:
404
+ `"b" is bound to a multi-output macro result (fields: x, y, z) --
405
+ reference a field directly (e.g. "b.x"), not the bare name`. This is
406
+ easy to trip over precisely because the shorthand return syntax (see
407
+ "Full-program syntax" below) makes `{ b }` look like it should mean
408
+ "the whole thing," the way it would for an ordinary scalar `let`.
409
+
410
+ **The classic macro trade-off still applies**: expansion is pure
411
+ substitution, so if a macro's body references one of its own
412
+ parameters more than once, the caller's argument expression gets
413
+ duplicated in the output everywhere that parameter appears — not
414
+ shared, not auto-let-bound (`safeDiv`'s own doc comment above already
415
+ flags exactly this for its twice-referenced `denominatorExpr`; it's
416
+ general to every macro now, not one helper). Pass an already-let-bound
417
+ `v(name)` as the argument instead of a raw expensive expression if that
418
+ duplication matters to you.
419
+
420
+ - **`loadExtern(name, def)`** — `def` is a plain per-target mapping
421
+ object instead of a function, e.g. `{ evaluate: (x) => ..., js: ([x])
422
+ => \`myLib.f(${x})\`, zig: ([x]) => \`mylib.f(${x})\` }`. A **real
423
+ native call**, same mechanism as the 22 built-in primitives — just
424
+ supplied by you instead of shipped here. Only the targets you provide a
425
+ key for resolve; every other target still throws "no mapping" for that
426
+ name, same as an unmapped primitive. **ExprForge can't verify the named
427
+ symbol actually exists in a given target, or that it behaves
428
+ identically across every target you register a mapping for — that's
429
+ entirely on you**, the same way linking an unfamiliar library is in any
430
+ other compiled language. Reach for this only when the math genuinely
431
+ can't be expressed by composing existing macros/primitives; prefer a
432
+ macro whenever it can.
433
+
434
+ Names are a single shared namespace with the 22 built-in primitives and
435
+ with each other — `loadMacro`/`loadExtern` throw on a collision rather
436
+ than silently shadowing anything.
437
+
438
+ ### What this doesn't buy you: no recursion, no loops, no mutable state
439
+
440
+ Now that one definition can reference another by name, it's a natural
441
+ guess that a definition could call **itself**, or that two definitions
442
+ could call each other back and forth. Neither works, on purpose — try it
443
+ with the `cross3`/`crossLength` example above and have `cross3` reference
444
+ itself, and here's exactly what happens:
445
+
446
+ - **A macro can't call itself, directly or through a cycle.** This is
447
+ enforced two different ways depending on how the macro was defined, and
448
+ it's worth knowing which one applies: a macro defined as an AST
449
+ function (straight `fn\`...\`` text, or every function in a `.expr`
450
+ file/buffer) is resolved and **inline-expanded once, at
451
+ registration/load time**, against whatever's registered so far — a
452
+ definition only becomes referenceable *after* it's fully registered,
453
+ so a self/forward reference simply survives as an ordinary, unmapped
454
+ `call` node, failing later with the same "no mapping for Math
455
+ function" error an unrelated typo would (even once that name
456
+ eventually DOES get registered elsewhere) — no recursion detection
457
+ needed, the ordering alone rules it out. A macro defined as a **plain
458
+ JS function** (`loadMacro(name, someFn)`) runs fresh on every use
459
+ instead, so it legitimately CAN reference other real macros each
460
+ time — which means a genuine self/cyclic reference there needs an
461
+ explicit runtime check instead of relying on ordering, and gets one:
462
+ `"<name>" can't call itself, directly or through a cycle`, not a
463
+ crash.
464
+ - **No loops.** A macro's body is built from the exact same primitives
465
+ every other ExprForge expression is — `select`/`cmp` for a conditional
466
+ *value*, nothing that iterates.
467
+ - **No mutable state.** AST nodes are values, not locations — there's
468
+ nothing to assign to.
469
+ - **Emitted output isn't a call, it's a copy.** Every use of a macro
470
+ expands its full arithmetic in place again — unlike a real function,
471
+ there's no shared implementation at the call site, so a macro used many
472
+ times in one formula makes the emitted source (correspondingly) larger
473
+ each time, not smaller. This is a size/readability trade-off to know
474
+ about, not a correctness concern.
475
+
476
+ This isn't a launch-day limitation waiting on a future release — it's the
477
+ same "expression AST, not a program AST" boundary declared under
478
+ "Intents" above, applied to this feature specifically because it's the
479
+ one place someone's most likely to assume otherwise.
480
+
481
+ ### Sessions (`createSession`)
482
+
483
+ Every `loadMacro`/`loadExtern`/`evaluate`/`emit`/`emitMany`/`loadExpr`/
484
+ `loadExprSource` above registers into (or resolves against) one
485
+ process-wide registry by default — fine for a single program registering
486
+ its own fixed set of macros once, at startup. `createSession()` gives you
487
+ an independent, additively-scoped registry instead, for whenever more
488
+ than one is genuinely needed at once — e.g. a service evaluating math
489
+ defined by several different users/tenants, where one user's
490
+ `loadMacro("helper", ...)` must never resolve inside another user's
491
+ expression just because they happened to pick the same name:
492
+
493
+ ```js
494
+ const { createSession, num, v, mul, call } = require("exprforge");
495
+
496
+ const session = createSession();
497
+ session.loadMacro("double", (x) => mul(x, num(2)));
498
+
499
+ const doubled = { name: "f", params: ["x"], body: call("double", v("x")) };
500
+ session.evaluate(doubled, [21]); // 42
501
+
502
+ // The global loadMacro/evaluate never see "double" at all -- it exists
503
+ // only inside this one session's own registry.
504
+ ```
505
+
506
+ - **Purely additive**: the process-wide default registry (what every bare
507
+ `loadMacro`/`loadExtern`/`evaluate`/`emit` call above already uses) is
508
+ completely unaffected by a session's existence, and vice versa —
509
+ nothing here changes what any existing call site does.
510
+ - **Every session-bound method mirrors its global counterpart 1:1**:
511
+ `session.loadMacro`, `session.loadExtern`, `session.evaluate`,
512
+ `session.emit`, `session.emitMany`, `session.loadExpr`,
513
+ `session.loadExprSource`, `session.expandMacros` — same signatures,
514
+ scoped to that session's own registry instead of the default one.
515
+ - **Two sessions never see each other's registrations**, even when both
516
+ register the same name — registering `"helper"` in session A never
517
+ collides with, or shadows, an unrelated `"helper"` in session B.
518
+ - **Built-in primitives (`sqrt`, `pow`, ...) work identically everywhere**
519
+ — they're fixed and not registry-backed at all, so a session doesn't
520
+ need, and can't be given, its own copy of them.
521
+ - **No removal API.** A session's registry is a plain object,
522
+ garbage-collected normally once you drop your reference to it — rebuild
523
+ a fresh `createSession()` instead of trying to unregister one
524
+ macro/extern out of an existing one.
525
+
206
526
  ## Adding a language
207
527
 
208
528
  Write `emitters/<lang>.js` exporting an `Emitter` instance (see any
@@ -259,7 +579,8 @@ expression model without introducing control flow:
259
579
  next, closing parens piling up at the end with no real hierarchy behind
260
580
  them — just bookkeeping to get everything hoisted before it's used.
261
581
  **`letChain(bindings, body)`** is that same nesting, built for you from a
262
- flat, ordered list instead:
582
+ flat, ordered list instead — exactly what "The bottom: raw AST builders"
583
+ above uses for `crossLength`'s `rx`/`ry`/`rz`:
263
584
 
264
585
  ```js
265
586
  const { v, num, mul, add, letChain, outputs } = require("exprforge");
@@ -306,7 +627,8 @@ select" pattern is wrong.
306
627
  `add(mul(v("a"), v("b")), num(1))` is exactly what gets built, but it's
307
628
  not what a human reads at a glance. `expr` is a tagged template literal
308
629
  that parses ordinary infix math syntax into that same tree — same nodes,
309
- different spelling, no new capability:
630
+ different spelling, no new capability. See "Another layer down" above
631
+ for a full worked example (`crossLength`, hand-inlined, no macro).
310
632
 
311
633
  ```js
312
634
  const { v, expr } = require("exprforge");
@@ -325,13 +647,14 @@ expr`(-b + sqrt(b^2 - 4*a*c)) / (2*a)`
325
647
  | `-x` | `neg(x)` |
326
648
  | `name(args...)` | `call("name", ...args)` — not checked against the 22 known functions at parse time, same deferred-to-emission-time error every hand-built `call()` already gets |
327
649
  | bare `name` | `v("name")` |
650
+ | `name.field` | `field(v("name"), "field")` — only meaningful when `name` is bound to a multi-output macro result (see "Macros and externs"); binds tighter than `^`, chainable (`a.b.c`) |
328
651
  | `cond ? then : else` | `select(cmp(left, op, right), then, else)` — the **only** place a comparison (`> < >= <= == !=`) is valid, matching `cmp()`'s own documented constraint that it's never a general boolean expression. A bare `a > b` with no `?` is a parse-time error, not a deferred one. Chains naturally: `a>0 ? 1 : b>0 ? 2 : 3`. |
329
652
  | `${...}` | Splices in an existing AST node as-is, or a plain JS number (auto-wrapped via `num()`). Anything else throws immediately. Plain strings aren't interpolatable — a bare identifier in the template text already means "variable", with no `${}` needed. |
330
653
  | `# ...` | An end-of-line comment — runs to the next newline, produces no tokens. Works across `${...}` interpolation boundaries too: a value interpolated inside an open comment is silently dropped, never validated (not even for what would otherwise be an invalid interpolation). |
331
654
 
332
655
  Deliberately **not** in the grammar: `let`/`outputs` blocks (it's a pure
333
- expression grammar, same "expression AST, not a program AST" boundary as
334
- the rest of exprforge — wrap the result in `letIn`/`letChain`/`outputs`,
656
+ expression grammar, same "expression AST, not a program AST" boundary
657
+ declared under "Intents" — wrap the result in `letIn`/`letChain`/`outputs`,
335
658
  or reach for `fn` below, which adds exactly that) and `&&`/`||` (the AST
336
659
  has no boolean-combinator node to lower them to).
337
660
 
@@ -346,7 +669,9 @@ letIn("mag", expr`sqrt(x^2 + y^2)`, expr`x / mag`)
346
669
  `let` bindings plus a `return`, on top of the exact same expression
347
670
  grammar (every expression inside a `fn` template is parsed by the same
348
671
  engine `expr` uses). Lowers to real `letChain`/`outputs` calls, same
349
- "same nodes, different spelling" guarantee as `expr` itself:
672
+ "same nodes, different spelling" guarantee as `expr` itself. See "One
673
+ layer down" above for a full worked example (`cross3`/`crossLength`,
674
+ via `loadMacro`).
350
675
 
351
676
  ```js
352
677
  const { fn } = require("exprforge");
@@ -366,6 +691,7 @@ const normalize2 = { name: "normalize2", params: ["x", "y"], body };
366
691
  | `let name = expr;` | one `[name, valueNode]` pair, in order — a later `let` can reference an earlier one's name |
367
692
  | `return expr;` | the chain's final expression |
368
693
  | `return { name: expr, ... };` | `outputs({ name: node, ... })` as the chain's final expression |
694
+ | `return { name, ... };` | shorthand for `return { name: name, ... };` — same convention JS object literals use for a property whose value is a same-named variable. Freely mixes with the explicit form: `return { rx, ry: ry * 2, rz };` |
369
695
 
370
696
  Duplicate `let` names aren't rejected by the parser itself — same
371
697
  deferred-to-`collectLets` behavior every hand-built `letIn`/`letChain`
@@ -379,7 +705,7 @@ one-off — but `fn` can carry them too, with a leading `name(params):`
379
705
  line:
380
706
 
381
707
  ```js
382
- const { fn, emitAll, evaluate } = require("exprforge");
708
+ const { fn, emit, evaluate } = require("exprforge");
383
709
 
384
710
  const normalize2 = fn`
385
711
  normalize2(x, y):
@@ -390,7 +716,7 @@ const normalize2 = fn`
390
716
  // no wrapping object needed.
391
717
 
392
718
  evaluate(normalize2, [3, 4]); // { nx: 0.6, ny: 0.8 }
393
- emitAll(normalize2).rust.source; // ready to use immediately
719
+ emit(normalize2, "rust").source; // ready to use immediately
394
720
  ```
395
721
 
396
722
  This changes `fn`'s return type based on what you wrote, deliberately:
@@ -401,28 +727,103 @@ signature is told apart from a statement by the same rule that tells
401
727
  `let`/`return` apart from any other identifier, so naming a function
402
728
  `let` just parses as (and fails as) a `let` statement instead.
403
729
 
730
+ ### Grammar reference
731
+
732
+ Everything above, as one formal grammar instead of two separate tables —
733
+ copied verbatim from `expr.js`'s/`fn.js`'s own header comments, not a
734
+ paraphrase, so it can't drift out of sync with what the parser actually
735
+ does:
736
+
737
+ ```
738
+ program := signature? stmt* returnStmt
739
+ signature := IDENT "(" (IDENT ("," IDENT)*)? ")" ":"
740
+ stmt := "let" IDENT "=" expression ";"
741
+ returnStmt := "return" expression ";"
742
+ | "return" "{" field ("," field)* "}" ";"
743
+ field := IDENT (":" expression)?
744
+
745
+ expression := ternary
746
+ ternary := additive ( compOp additive "?" expression ":" expression )?
747
+ compOp := ">" | "<" | ">=" | "<=" | "==" | "!="
748
+ additive := multiplicative ( ("+"|"-") multiplicative )*
749
+ multiplicative := unary ( ("*"|"/") unary )*
750
+ unary := "-" unary | power
751
+ power := postfix ( "^" unary )?
752
+ postfix := primary ( "." IDENT )*
753
+ primary := NUMBER | IDENT ("(" args ")")? | "(" expression ")" | HOLE
754
+ args := expression ("," expression)*
755
+ ```
756
+
757
+ `` expr`...` `` is exactly `expression` on its own — one formula, no
758
+ `let`/`return`. `` fn`...` `` is `program` — `expression`'s entire
759
+ grammar embedded unchanged inside every `let`'s value and every
760
+ `return`, parsed by the exact same `Parser` class both tags share (not a
761
+ reimplementation — `fn` literally imports `expr.js`'s tokenizer and
762
+ parser rather than forking either).
763
+
764
+ Not shown above (lexical, not grammar): `# ...` end-of-line comments
765
+ (run to the next newline, produce no tokens); `${...}` interpolation,
766
+ which splices an existing `Node` or a plain number in directly (see
767
+ "Infix expression syntax" above) and becomes a `HOLE` token in the
768
+ grammar above; and that `let`/`return` are ordinary identifiers
769
+ *everywhere except* statement-start position — `` expr`let * 2` `` still
770
+ means `v("let") * 2`, not a syntax error, since `expr`'s own grammar has
771
+ no `stmt`/`signature` rules to make either one special.
772
+
404
773
  ## Printing an AST back out, and a native evaluator
405
774
 
406
775
  Two things that fall out of `fn` existing: `emitters.expr` is a real,
407
- registered 18th target that prints any AST *back out* as `fn`/`expr`
408
- source text (the reverse of parsing it) — useful for debugging a
409
- formula built from several composed helpers, or just getting a readable
410
- string to log or paste into a future `fn`/`expr` call. And `evaluate(fn,
411
- args)` (also exported from the main package) is a native tree-walking
412
- interpreter over the same AST, computing a result directly in JS with no
413
- codegen or compile step — the same node types every emitter already
414
- handles, backed by the real `Math.*` functions.
776
+ registered target that prints any AST *back out* as `fn`/`expr` source
777
+ text (the reverse of parsing it) — useful for debugging a formula built
778
+ from several composed helpers, or just getting a readable string to log
779
+ or paste into a future `fn`/`expr` call. And `evaluate(fn, args)` (also
780
+ exported from the main package) is a native tree-walking interpreter
781
+ over the same AST, computing a result directly in JS with no codegen or
782
+ compile step — the same node types every emitter already handles,
783
+ backed by the real `Math.*` functions.
415
784
 
416
785
  ```js
417
- const { emitAll, evaluate } = require("exprforge");
786
+ const { emit, evaluate } = require("exprforge");
418
787
 
419
- emitAll(normalize2).expr.source;
788
+ emit(normalize2, "expr").source;
420
789
  // "normalize2(x, y):\n let mag = sqrt(((x^2) + (y^2)));\n return { nx: (x / mag), ny: (y / mag) };\n"
421
790
 
422
791
  evaluate(normalize2, [3, 4]);
423
792
  // { nx: 0.6, ny: 0.8 }
424
793
  ```
425
794
 
795
+ ### Loading a `.expr` file (`loadExpr`)
796
+
797
+ `loadExpr(path)` goes the other direction from `emit(fn, "expr")` above:
798
+ reads a `.expr` file (that same round-trip text format) and parses it as
799
+ zero or more `name(params): let ...; return ...;` definitions, each
800
+ usable directly with `evaluate()`/`emit()`/`emitMany()` — this is the
801
+ file-backed sibling of `loadExprSource` in "Examples, flashiest first"
802
+ above:
803
+
804
+ ```js
805
+ const { loadExpr, evaluate } = require("exprforge");
806
+
807
+ const defs = loadExpr("./formulas/vectors.expr");
808
+ evaluate(defs.hyp, [3, 4]); // 5
809
+ ```
810
+
811
+ A function defined earlier in the file is available to a function defined
812
+ **later** in the same file — as an inline macro, the exact same
813
+ "expanded, not called" model `loadMacro` itself uses above (see that
814
+ section for why). Every definition needs a `name(params):` signature line
815
+ (nothing later in the file, or the caller, could refer to one that
816
+ didn't), and a `.expr` file can reference globally loaded macros too, not
817
+ just earlier definitions in the same file — the two sources merge.
818
+
819
+ `loadExpr(path)` is a thin `fs.readFileSync` wrapper around
820
+ **`loadExprSource(text, label?)`** — the same parser, given source text
821
+ directly. Use that one wherever the text isn't coming from a real file on
822
+ disk (a browser text buffer, an HTTP response, ...) — the playground's
823
+ editor uses it exactly this way to let one buffer hold several
824
+ definitions. `label` (default `"loadExprSource()"`) identifies the source
825
+ in error messages, the way a file path does for `loadExpr`.
826
+
426
827
  ## Multiple named outputs
427
828
 
428
829
  `outputs({ name: Node, ... })` computes several named values from ONE
@@ -468,13 +869,52 @@ the order instead (same reason Lua's return, also positional, gets one).
468
869
  C#'s tuple has no such risk — a tuple literal's element names aren't
469
870
  pre-declared locals the way Go's named returns are.
470
871
 
471
- ## What this deliberately doesn't do
472
-
473
- - No control flow (loops, branches, calling other generated functions) —
474
- this is an expression AST, not a program AST.
475
- - No RNG — can't be made to produce identical output across languages,
476
- so it isn't offered as if it could.
477
- - No arbitrary precision / complex numbers — float64 only, for now.
872
+ ## Security considerations
873
+
874
+ "Trust only the layer you need" (see "Intents" above) is a claim, not just
875
+ a description — the raw AST layer specifically has to be safe to build
876
+ from untrusted input, since it's the one this README recommends reaching
877
+ for when you want the least amount of magic between your formula and the
878
+ code it emits. A few concrete guarantees that follow from that:
879
+
880
+ - **The raw builders validate their own inputs.** `num`/`v`/`bin`/`call`/
881
+ `letIn`/`cmp`/`outputs`/`field` all reject anything that isn't a safe
882
+ value — an identifier must match the exact same rule `expr`/`fn`'s own
883
+ tokenizer already enforces on text it parses (start with a letter/`_`,
884
+ then letters/digits/`_` only), an operator must be one of the fixed
885
+ `+`/`-`/`*`/`/` (or, for `cmp`, `>`/`<`/`>=`/`<=`/`==`/`!=`) set, and a
886
+ number must actually be finite. Without this, the raw builder layer
887
+ would have been the *least* safe one to build from untrusted input, not
888
+ the most — `fn`/`expr`'s own tokenizer never produces anything but a
889
+ safe identifier to begin with, so this was the one place a malformed or
890
+ malicious name (e.g. `v("x); process.exit(1); //")`) could otherwise
891
+ reach emitted output completely unescaped, verbatim, in every one of 16
892
+ targets at once.
893
+ - **`expr`/`fn` cap how deeply an expression can nest.** Parens,
894
+ function-call arguments, and ternary branches can nest up to 100 levels
895
+ deep (`MAX_EXPRESSION_DEPTH` in `expr.js`) before parsing fails with one
896
+ clear, controlled error — comfortably below where a pathologically
897
+ nested input (`"((((...))))"` or `"f(f(f(...)))"` thousands deep,
898
+ plausible if this ever parses genuinely untrusted, unbounded-size text)
899
+ would otherwise blow the real JS call stack with a raw "Maximum call
900
+ stack size exceeded". A wide-but-shallow expression (many terms, no
901
+ real nesting) is unaffected regardless of length — only genuine nesting
902
+ depth is bounded.
903
+ - **`createSession()` isolates macro/extern registrations** between
904
+ independent users/tenants sharing one process — see "Sessions" above.
905
+ - **A caller-supplied macro/extern throwing reports which one.** A macro
906
+ function, or an extern's own `evaluate`/per-target template, is code
907
+ *you* (or whoever registered it) supplied — if it has a bug, the error
908
+ it throws is wrapped with context naming the macro/extern/target
909
+ responsible, rather than propagating bare with no indication of where
910
+ it came from.
911
+
912
+ What this doesn't cover, deliberately: `loadExtern`'s per-target templates
913
+ are real native code you supply — ExprForge can't verify the named symbol
914
+ actually exists in a given target, or that it behaves identically across
915
+ every target you provide a mapping for (see "Macros and externs" above).
916
+ That risk is inherent to what an extern *is*, not something a validation
917
+ layer could close without also closing off the feature itself.
478
918
 
479
919
  ## Testing
480
920
 
@@ -582,6 +1022,11 @@ compiling/running against a real toolchain rather than assumed to work:
582
1022
  `select()`'s own contract exactly. The native 2-argument `SIGN(A, B)`
583
1023
  ("magnitude of A, sign of B") is *not* this project's `sign(x)` —
584
1024
  `SIGN(1.0, 0.0)` returns `1.0`, not `0.0` — built from `MERGE` instead.
1025
+ Every gensym'd identifier this library ever introduces internally
1026
+ (e.g. a macro's own alpha-renamed `let`, see "Macros and externs")
1027
+ starts with a letter, never `_` — Fortran is the one target that
1028
+ rejects a leading underscore outright, confirmed against a real
1029
+ compiler ("Invalid character in name").
585
1030
  - **Zig**: `std.debug.print` writes to **stderr** by design, not
586
1031
  stdout — the conformance harness has to use
587
1032
  `std.io.getStdOut().writer()` instead, or every result silently comes
@@ -620,10 +1065,11 @@ compiling/running against a real toolchain rather than assumed to work:
620
1065
 
621
1066
  One test (`normalizeX`) is deliberately excluded from the QB64 check
622
1067
  only: it exists specifically to demonstrate the "don't guard division
623
- with `select`" pitfall from the section above, and QB64 is the one
624
- target where that pitfall actually produces `NaN` (every other target,
625
- including Lua's `and`/`or`, genuinely short-circuits around it) — that's
626
- the AST being correctly unsafe on purpose, not an emitter bug.
1068
+ with `select`" pitfall from "Named subexpressions and conditional
1069
+ values" above, and QB64 is the one target where that pitfall actually
1070
+ produces `NaN` (every other target, including Lua's `and`/`or`, genuinely
1071
+ short-circuits around it) — that's the AST being correctly unsafe on
1072
+ purpose, not an emitter bug.
627
1073
 
628
1074
  ## License
629
1075