porffor 0.1.1 → 0.2.0-c6c8c81

package/README.md

@@ -1,28 +1,34 @@
-# porffor
-a basic experimental wip *aot* optimizing js -> wasm engine/compiler/runtime in js. not serious/intended for (real) use. (this is a straight forward, honest readme)<br>
-age: ~1 month
+# Porffor &nbsp;<sup><sub>/ˈpɔrfɔr/ &nbsp;*(poor-for)*</sup></sub>
+A from-scratch experimental **AOT** optimizing JS -> Wasm/C engine/compiler/runtime in JS. Not serious/intended for (real) use. (this is a straight forward, honest readme)<br>
+Age: ~6 months (very on and off)
 
-## design
-porffor is a very unique js engine, due a very different approach. it is seriously limited, but what it can do, it does pretty well. key differences:
-- 100% aot compiled *(not jit)*
-- everything is a number
-- no constant runtime/preluded code
-- least Wasm imports possible (only stdio)
+## Design
+Porffor is a very unique JS engine, due many wildly different approaches. It is seriously limited, but what it can do, it does pretty well. Key differences:
+- 100% AOT compiled (no JIT)
+- No constant runtime/preluded code
+- Least Wasm imports possible (only I/O)
 
-porffor is mostly built from scratch, the only thing that is not is the parser (using [acorn](https://github.com/acornjs/acorn)). binaryen/etc is not used, we make final wasm binaries ourself. you could imagine it as compiling a language which is a sub (some things unsupported) and super (new/custom apis) set of javascript. not based on any particular spec version, focusing on function/working over spec compliance.
+Porffor is primarily built from scratch, the only thing that is not is the parser (using [Acorn](https://github.com/acornjs/acorn)). Binaryen/etc is not used, we make final wasm binaries ourself. You could imagine it as compiling a language which is a sub (some things unsupported) and super (new/custom apis) set of javascript. Not based on any particular spec version, focusing on function/working over spec compliance.
 
-## limitations
-- no full object support yet
-- little built-ins/prototype
-- no async/promise/await
-- no variables between scopes (except args and globals)
-- literal callees only in calls (eg `print()` works, `a = print; a()` does not)
+## Limitations
+- No full object support yet
+- Little built-ins/prototype
+- No async/promise/await
+- No variables between scopes (except args and globals)
+- Literal callees only in calls (eg `print()` works, `a = print; a()` does not)
+- No `eval()` etc (since it is AOT)
 
-## supported
-see [optimizations](#optimizations) for opts implemented/supported.
+## Rhemyn
+Rhemyn is Porffor's own regex engine; it compiles literal regex to Wasm bytecode AOT (remind you of anything?). It is quite basic and WIP. See [its readme](rhemyn/README.md) for more details.
 
-### proposals
-these include some early (stage 1/0) and/or dead (last commit years ago) proposals but *I* think they are pretty neat, so.
+## 2c
+2c is Porffor's own Wasm -> C compiler, using generated Wasm bytecode and internal info to generate specific and efficient/fast C code. Little boilerplate/preluded code or required external files, just for CLI binaries (not like wasm2c very much).
+
+## Supported
+See [optimizations](#optimizations) for opts implemented/supported.
+
+### Proposals
+These include some early (stage 1/0) and/or dead (last commit years ago) proposals but *I* think they are pretty neat, so.
 
 #### `Math` proposals (stage 1/0)
 
@@ -30,124 +36,139 @@ these include some early (stage 1/0) and/or dead (last commit years ago) proposa
 - [`Math` Extensions Proposal](https://github.com/rwaldron/proposal-math-extensions): `Math.scale`, `Math.radians`, `Math.degrees`, `Math.RAD_PER_DEG`, `Math.DEG_PER_RAD` (stage 1 - last commit september 2020)
 - [`Math.signbit` Proposal](https://github.com/tc39/proposal-Math.signbit): `Math.signbit` (stage 1 - last commit february 2020)
 
-### language
+### Language
 
-- number literals
-- declaring functions
-- calling functions *literal callees only*
+- Number literals
+- Declaring functions
+- Calling functions *literal callees only*
 - `return`
 - `let`/`const`/`var` basic declarations
-- some basic integer operators (`+-/*%`)
-- some basic integer bitwise operators (`&|`)
-- equality operators (`==`, `!=`, etc)
-- gt/lt operators (`>`, `<`, `>=`, etc)
-- some unary operators (`!`, `+`, `-`)
-- logical operators (`&&`, `||`)
-- declaring multiple variables in one (`let a, b = 0`)
-- global variables (`var`/none in top scope)
-- functions returning 1 number
-- bool literals as ints (not real type)
+- Some basic integer operators (`+-/*%`)
+- Some basic integer bitwise operators (`&|`)
+- Equality operators (`==`, `!=`, etc)
+- GT/LT operators (`>`, `<`, `>=`, etc)
+- Some unary operators (`!`, `+`, `-`)
+- Logical operators (`&&`, `||`)
+- Declaring multiple variables in one (`let a, b = 0`)
+- Global variables (`var`/none in top scope)
+- Functions returning 1 number
+- Bool literals as ints (not real type)
 - `if` and `if ... else`
-- anonymous functions
-- setting functions using vars (`const foo = function() { ... }`)
-- arrow functions
+- Anonymous functions
+- Setting functions using vars (`const foo = function() { ... }`)
+- Arrow functions
 - `undefined`/`null` as ints (hack)
-- update expressions (`a++`, `++b`, `c--`, etc)
+- Update expressions (`a++`, `++b`, `c--`, etc)
 - `for` loops (`for (let i = 0; i < N; i++)`, etc)
-- hack for "chars" as ints (`'X'` -> `88`)
-- *basic* objects (hack)
+- *Basic* objects (hack)
 - `console.log` (hack)
 - `while` loops
 - `break` and `continue`
-- named export funcs
-- iife support
-- assignment operators (`+=`, `-=`, `>>=`, `&&=`, etc)
-- conditional/ternary operator (`cond ? a : b`)
-- recursive functions
-- bare returns (`return`)
+- Named export funcs
+- IIFE support
+- Assignment operators (`+=`, `-=`, `>>=`, `&&=`, etc)
+- Conditional/ternary operator (`cond ? a : b`)
+- Recursive functions
+- Bare returns (`return`)
 - `throw` (literals only)
-- basic `try { ... } catch { ... }` (no error given)
-- calling functions with non-matching arguments (eg `f(a, b); f(0); f(1, 2, 3);`)
-- `typeof` mostly (static-ish)
-- runtime errors for undeclared variables (`ReferenceError`), not functions (`TypeError`)
-- array creation via `[]` (eg `let arr = [ 1, 2, 3 ]`)
-- array member access via `arr[ind]` (eg `arr[0]`)
-- string literals (`'hello world'`)
-- string member (char) access via `str[ind]` (eg `str[0]`)
-- string concat (`+`) (eg `'a' + 'b'`)
-- truthy/falsy (eg `!'' == true`)
-
-### built-ins
+- Basic `try { ... } catch { ... }` (no error given)
+- Calling functions with non-matching arguments (eg `f(a, b); f(0); f(1, 2, 3);`)
+- `typeof`
+- Runtime errors for undeclared variables (`ReferenceError`), not functions (`TypeError`)
+- Array creation via `[]` (eg `let arr = [ 1, 2, 3 ]`)
+- Array member access via `arr[ind]` (eg `arr[0]`)
+- String literals (`'hello world'`)
+- String member (char) access via `str[ind]` (eg `str[0]`)
+- String concat (`+`) (eg `'a' + 'b'`)
+- Truthy/falsy (eg `!'' == true`)
+- String comparison (eg `'a' == 'a'`, `'a' != 'b'`)
+- Nullish coalescing operator (`??`)
+- `for...of` (arrays and strings)
+- Array member setting (`arr[0] = 2`, `arr[0] += 2`, etc)
+- Array constructor (`Array(5)`, `new Array(1, 2, 3)`)
+
+### Built-ins
 
 - `NaN` and `Infinity` (f64 only)
 - `isNaN()` and `isFinite()` (f64 only)
-- most of `Number` (`MAX_VALUE`, `MIN_VALUE`, `MAX_SAFE_INTEGER`, `MIN_SAFE_INTEGER`, `POSITIVE_INFINITY`, `NEGATIVE_INFINITY`, `EPSILON`, `NaN`, `isNaN`, `isFinite`, `isInteger`, `isSafeInteger`) (some f64 only)
-- some `Math` funcs (`Math.sqrt`, `Math.abs`, `Math.floor`, `Math.sign`, `Math.round`, `Math.trunc`, `Math.clz32`, `Math.fround`, `Math.random`) (f64 only)
-- basic `globalThis` support
-- basic `Boolean` and `Number`
-- basic `eval` (literals only)
+- Most of `Number` (`MAX_VALUE`, `MIN_VALUE`, `MAX_SAFE_INTEGER`, `MIN_SAFE_INTEGER`, `POSITIVE_INFINITY`, `NEGATIVE_INFINITY`, `EPSILON`, `NaN`, `isNaN`, `isFinite`, `isInteger`, `isSafeInteger`) (some f64 only)
+- Some `Math` funcs (`Math.sqrt`, `Math.abs`, `Math.floor`, `Math.sign`, `Math.round`, `Math.trunc`, `Math.clz32`, `Math.fround`, `Math.random`) (f64 only)
+- Basic `globalThis` support
+- Basic `Boolean` and `Number`
+- Basic `eval` for literals
 - `Math.random()` using self-made xorshift128+ PRNG
-- some of `performance` (`now()`)
-- some of `Array.prototype` (`at`, `push`, `pop`, `shift`)
-- some of `String.prototype` (`at`, `charAt`, `charCodeAt`)
-
-### custom
-
-- basic `assert` func
-- supports i32, i64, and f64 for valtypes
-- wip SIMD api (docs needed)
-- intrinsic functions (see below)
-- inlining wasm via ``asm`...``\` "macro"
-
-## soon todo
-- arrays
-  - member setting (`arr[0] = 2`)
-  - more of `Array` prototype
-  - arrays/strings inside arrays
-- strings
-  - member setting
-  - equality
-- more math operators (`**`, etc)
+- Some of `performance` (`now()`)
+- Some of `Array.prototype` (`at`, `push`, `pop`, `shift`, `fill`)
+- Some of `String.prototype` (`at`, `charAt`, `charCodeAt`)
+
+### Custom
+
+- Supports i32, i64, and f64 for valtypes
+- Start of a SIMD api (docs needed)
+- Intrinsic functions (see below)
+- Inlining wasm via ``asm`...``\` "macro"
+
+## Todo
+No particular order and no guarentees, just what could happen soon™
+
+- Arrays
+  - More of `Array` prototype
+  - Arrays/strings inside arrays
+  - Destructuring
+- Objects
+  - Basic object expressions (eg `{}`, `{ a: 0 }`)
+- Wasm
+  - *Basic* Wasm engine (interpreter) in js
+- More math operators (`**`, etc)
 - `do { ... } while (...)`
-- exceptions
-  - `try { } finally {}`
-  - rethrowing inside catch
-- optimizations
-  - rewrite local indexes per func for smallest local header and remove unused idxs
-  - smarter inline selection (snapshots?)
-  - remove const ifs (`if (true)`, etc)
-  - use data segments for initing arrays
-
-## test262
-porffor can run test262 via some hacks/transforms which remove unsupported features whilst still doing the same asserts (eg simpler error messages using literals only). it currently passes >10% (see latest commit desc for latest and details). use `node test262` to test, it will also show a difference of overall results between the last commit and current results.
-
-## optimizations
-mostly for reducing size. do not really care about compiler perf/time as long as it is reasonable. we do not use/rely on external opt tools (`wasm-opt`, etc), instead doing optimization inside the compiler itself creating even smaller code sizes than `wasm-opt` itself can produce as we have more internal information. (this also enables fast + small runtime use as a potential cursed jit in frontend).
-
-### traditional opts
-- inlining functions (wip, limited)
-- inline const math ops
-- tail calls (behind flag `-tail-call`)
-
-### wasm transforms
+- Rewrite `console.log` to work with strings/arrays
+- Exceptions
+  - Rewrite to use actual strings (optional?)
+  - `try { } finally { }`
+  - Rethrowing inside catch
+- Optimizations
+  - Rewrite local indexes per func for smallest local header and remove unused idxs
+  - Smarter inline selection (snapshots?)
+  - Remove const ifs (`if (true)`, etc)
+  - Use type(script) information to remove unneeded typechecker code
+
+## Performance
+*For the things it supports most of the time*, Porffor is blazingly fast compared to most interpreters, and common engines running without JIT. For those with JIT, it is not that much slower like a traditional interpreter would be; mostly the same or a bit faster/slower depending on what.
+
+![Screenshot of comparison chart](https://github.com/CanadaHonk/porffor/assets/19228318/76c75264-cc68-4be1-8891-c06dc389d97a)
+
+## Optimizations
+Mostly for reducing size. I do not really care about compiler perf/time as long as it is reasonable. We do not use/rely on external opt tools (`wasm-opt`, etc), instead doing optimization inside the compiler itself creating even smaller code sizes than `wasm-opt` itself can produce as we have more internal information.
+
+### Traditional opts
+- Inlining functions (WIP, limited)
+- Inline const math ops
+- Tail calls (behind flag `-tail-call`)
+
+### Wasm transforms
 - `local.set`, `local.get` -> `local.tee`
 - `i32.const 0`, `i32.eq` -> `i32.eqz`
 - `i64.extend_i32_s`, `i32.wrap_i64` -> ``
 - `f64.convert_i32_u`, `i32.trunc_sat_f64_s` -> ``
 - `return`, `end` -> `end`
-- remove some redundant sets/gets
-- remove unneeded single just used vars
-- remove unneeded blocks (no `br`s inside)
-- remove unused imports
+- Change const, convert to const of converted valtype (eg `f64.const`, `i32.trunc_sat_f64_s -> `i32.const`)
+- Remove some redundant sets/gets
+- Remove unneeded single just used vars
+- Remove unneeded blocks (no `br`s inside)
+- Remove unused imports
+- Use data segments for initing arrays/strings
+
+### Wasm module
+- Type cache/index (no repeated types)
+- No main func if empty (and other exports)
 
-### wasm module
-- type cache/index (no repeated types)
-- no main func if empty (and other exports)
+## Test262
+Porffor can run Test262 via some hacks/transforms which remove unsupported features whilst still doing the same asserts (eg simpler error messages using literals only). It currently passes >10% (see latest commit desc for latest and details). Use `node test262` to test, it will also show a difference of overall results between the last commit and current results.
 
-## codebase
+## Codebase
 - `compiler`: contains the compiler itself
   - `builtins.js`: all built-ins of the engine (spec, custom. vars, funcs)
-  - `codeGen.js`: code (wasm) generation, ast -> wasm, the bulk of the effort
+  - `codeGen.js`: code (wasm) generation, ast -> wasm. The bulk of the effort
   - `decompile.js`: basic wasm decompiler for debug info
   - `embedding.js`: utils for embedding consts
   - `encoding.js`: utils for encoding things as bytes as wasm expects
@@ -164,25 +185,34 @@ mostly for reducing size. do not really care about compiler perf/time as long as
   - `info.js`: runs with extra info printed
   - `repl.js`: basic repl (uses `node:repl`)
 
+- `rhemyn`: contains [Rhemyn](#rhemyn) - our regex engine (used by Porffor)
+  - `compile.js`: compiles regex ast into wasm bytecode
+  - `parse.js`: own regex parser
+
 - `test`: contains many test files for majority of supported features
 - `test262`: test262 runner and utils
 
-## usecases
-basically none (other than giving people headaches). potential as a tiny fast advanced expression evaluator (for math)?
+## Usecases
+Basically none (other than giving people headaches). Potential ideas to come?
 
-## usage
-basically nothing will work :). see files in `test` for examples.
+## Usage
+Basically nothing will work :). See files in `test` for examples.
 
-1. clone repo
+1. Clone repo
 2. `npm install`
-3. `node test` to run tests (all should pass)
+3. `node test` to run tests (some will fail)
 4. `node runner path/to/code.js` to run a file (or `node runner` to use wip repl)
 
-you can also use deno (`deno run -A ...` instead of `node ...`), or bun (`bun ...` instead of `node ...`)
+You can also use Deno (`deno run -A ...` instead of `node ...`), or Bun (`bun ...` instead of `node ...`).
 
-### flags
-- `-raw` for no info logs (just raw js output)
-- `-valtype=i32|i64|f64` to set valtype, f64 by default
+### Options
+- `-target=wasm|c|native` (default: `wasm`) to set target output (native compiles c output to binary, see args below)
+- `-target=c|native` only:
+  - `-o=out.c|out.exe|out` to set file to output c or binary
+- `-target=native` only:
+  - `-compiler=clang` to set compiler binary (path/name) to use to compile
+  - `-cO=O3` to set compiler opt argument
+- `-valtype=i32|i64|f64` (default: `f64`) to set valtype
 - `-O0` to disable opt
 - `-O1` (default) to enable basic opt (simplify insts, treeshake wasm imports)
 - `-O2` to enable advanced opt (inlining)
@@ -190,86 +220,36 @@ you can also use deno (`deno run -A ...` instead of `node ...`), or bun (`bun ..
 - `-no-run` to not run wasm output, just compile
 - `-opt-log` to log some opts
 - `-code-log` to log some codegen (you probably want `-funcs`)
-- `-funcs` to log funcs (internal representations)
+- `-regex-log` to log some regex
+- `-funcs` to log funcs
+- `-ast-log` to log AST
 - `-opt-funcs` to log funcs after opt
 - `-sections` to log sections as hex
 - `-opt-no-inline` to not inline any funcs
-- `-tail-call` to enable tail calls (not widely implemented)
-
-## vscode extension
-there is a vscode extension in `porffor-for-vscode` which tweaks js syntax highlighting to be nicer with porffor features (eg highlighting wasm inside of inline asm).
-
-## wasm output
-porffor optimizes for size as much as possible. current output is ~as small as possible (even with manual asm editing) for some simple functions.
-
-### example
-this javascript (159 bytes unminified):
-```js
-function isPrime(number) {
-  if (number === 1) return false;
-
-  for (let i = 2; i < number; i++) {
-    if (number % i == 0) return false;
-  }
-
-  return true;
-}
-```
-
-compiles into this wasm, in 7.3ms (just compile time), 90 bytes large (including module):
-```wasm
-(i32) -> (i32) ;; isPrime
-  local.get 0 ;; number
-  i32.const 1
-  i32.eq
-  if ;; label @2
-    i32.const 0
-    return
-  end
-  i32.const 2
-  local.set 1 ;; i
-  loop ;; label @2
-    local.get 1 ;; i
-    local.get 0 ;; number
-    i32.lt_s
-    if ;; label @3
-      local.get 0 ;; number
-      local.get 1 ;; i
-      i32.rem_s
-      i32.eqz
-      if ;; label @4
-        i32.const 0
-        return
-      end
-      local.get 1 ;; i
-      i32.const 1
-      i32.add
-      local.set 1 ;; i
-      br 1 ;; goto @2
-    end
-  end
-  i32.const 1
-end
-```
-
-## isn't this the same as assemblyscript?
-no. they are not alike at all internally and have different goals/ideals:
-- porffor is made as a generic js engine, not for wasm stuff specifically
-- porffor takes in js, not a different language or typescript like assemblyscript
-- porffor is made in pure js and compiles itself, not using binaryen/etc
-- (also I didn't know it existed when I started this)
-
-## faq
-
-### 1. why name
-`purple` in Welsh is `porffor`. why purple?
-- no other js engine is purple colored
-- purple is pretty cool
-- purple apparently represents "ambition", which is.. one word to describe this project
-- the hard to speak name is also the noise your brain makes in reaction to this idea
-
-### 2. why at all
-yes.
-
-### 3. but what about spec compliance?
-lol, no. (sorry.)
+- `-tail-call` to enable tail calls (experimental + not widely implemented)
+- `-compile-hints` to enable V8 compilation hints (experimental + doesn't seem to do much?)
+
+## VSCode extension
+There is a vscode extension in `porffor-for-vscode` which tweaks js syntax highlighting to be nicer with porffor features (eg highlighting wasm inside of inline asm).
+
+## Isn't this the same as AssemblyScript/other Wasm langs?
+No. they are not alike at all internally and have very different goals/ideals:
+- Porffor is made as a generic JS engine, not for Wasm stuff specifically
+- Porffor takes in JS, not a different language or typescript
+- Porffor is made in pure JS and compiles itself, not using Binaryen/etc
+- (Also I didn't know it existed when I started this, lol)
+
+## FAQ
+
+### 1. Why the name?
+`purple` in Welsh is `porffor`. Why purple?
+- No other js engine is purple colored
+- Purple is pretty cool
+- Purple apparently represents "ambition", which is.. one word to describe this project
+- The hard to speak name is also the noise your brain makes in reaction to this idea!
+
+### 2. Why at all?
+Yes!
+
+### 3. But what about spec compliance?
+Lol, no. (sorry.)