functionalscript 0.3.3 → 0.3.5

package/doc/vm/README.md

@@ -1,24 +0,0 @@
-# VM
-
-Two options:
-- using instances
-- using types. In this case, if we need multiple VMs in the same process, we need multiple types.
-
-## Rust Interface
-
-```rust
-trait Any {
-}
-
-trait String {
-}
-
-trait Bigint {
-}
-
-trait Object {
-}
-
-trait Array {
-}
-```

package/doc/vm-rearchitecture.md

@@ -1,129 +0,0 @@
-# NaNVM Re-Architecture
-
-About a year ago (Nov 2023), the FunctionalScript team started a new project called [NaNVM](https://github.com/functionalscript/nanvm). We have limited resources (⌛💰) for the projects, so progress has been slow. Since then, we’ve implemented several components from scratch in Rust:
-- An interface and multiple implementations for [Memory Management](https://github.com/functionalscript/nanvm/tree/main/nanvm-lib/src/mem), such as a [global](https://github.com/functionalscript/nanvm/blob/main/nanvm-lib/src/mem/global.rs) memory manager using standard `alloc/dealloc`, a [local](https://github.com/functionalscript/nanvm/blob/main/nanvm-lib/src/mem/local.rs) manager with a reference counter, and a simple [arena](https://github.com/functionalscript/nanvm/blob/main/nanvm-lib/src/mem/arena.rs) implementation.
-- [All FS data types](https://github.com/functionalscript/nanvm/tree/main/nanvm-lib/src/js) in the VM, such as [string](https://github.com/functionalscript/nanvm/blob/main/nanvm-lib/src/js/js_string.rs), [bigint](https://github.com/functionalscript/nanvm/blob/main/nanvm-lib/src/js/js_bigint.rs), [array](https://github.com/functionalscript/nanvm/blob/main/nanvm-lib/src/js/js_array.rs), [object](https://github.com/functionalscript/nanvm/blob/main/nanvm-lib/src/js/js_object.rs), and [any](https://github.com/functionalscript/nanvm/blob/main/nanvm-lib/src/js/any.rs).
-- And, of course, we've implemented a parser for JSON and DJS in Rust, which works well. See [this article about DJS](https://medium.com/@sasha.gil/bridging-the-gap-from-json-to-javascript-without-dsls-fee273573f1b) for more information.
-
-All of this code is written in Rust. Rust excels at runtime performance and system-level programming. However, developers often struggle when working with high-level and business logic. Its verbosity and various memory management models make developing components like parsers in Rust slow and complex. It’s great for implementing a memory manager, a VM, or a big integer but less suited for tasks like implementing a parser efficiently (and yes, we are aware of third-party parser generators). That’s why we want to use a high-level language for this purpose. Of course, we don't need to search for one because we already have FunctionalScript and JavaScript as glue. Additionally, because of our limited resources, we aim to use only a few repositories. As a result, we plan to merge the NaNVM code into the [FunctionalScript repo](https://github.com/functionalscript/functionalscript).
-
-## Stage 0: Current State
-
-Before we start rearchitecting the project, we should understand our current architecture. Here's the current module dependency graph for [nanvm_lib](https://github.com/functionalscript/nanvm/tree/main/nanvm-lib/src):
-
-```mermaid
-flowchart TB
-  app[Application] --> parser[Parser] --> js[VM] --> mem[Memory Manager]
-```
-
-The parser is written in Rust using VM types, such as `JsString` and `JsBigInt`. The application can parse JSON and DJS files and output in any of these formats as a standalone module.
-
-## Stage 1: Using a Third-Party JS Engine for Parsing
-
-Because FunctionalScript is a subset of JavaScript, we can use third-party JavaScript engines to bootstrap our parser, written on FunctionalScript, without circular dependencies. In Rust, we only need to implement a generic byte code deserializer that reads byte code and invokes VM API functions. We selected [Deno](https://deno.com/) and its [deno_core](https://crates.io/crates/deno_core/) package as a third-party JS engine because it's also written in Rust, has a crate and it's easy to integrate with our project.
-
-```mermaid
-flowchart TB
-  app[Application] --> deno(deno_core)
-  app --> bcd[Byte Code Deserializer] --> js[VM] --> mem[Memory Manager]
-```
-
-### Requirements 
-1. To restore previous functionality, we still need a parser that can convert FunctionalScript or DJS into byte code,
-2. We have to design byte code for FunctionalScript and implement its deserializer in Rust.
-
-### Build Process
-
-To satisfy the first requirement, we need a parser written in FunctionalScript. The build process should then take the parser source code and embed it into the application. See [include_str](https://doc.rust-lang.org/std/macro.include_str.html) for more details.
-
-```mermaid
-flowchart TB
-  app[Application] --> deno(deno_core)
-  app --> bcd[Byte Code Deserializer] --> js[VM] --> mem[Memory Manager]
-  app --> fsp[(Functional Script Parser Source Code)]
-```
-
-### Run-Time Process
-
-1. The application loads the parser source code from memory into the Deno engine.
-2. The application executes the parser in the JS engine with the command-line parameters provided by the user.
-3. After the parser generates the byte code, the application sends this byte code to the VM.
-
-This stage provides an interim solution to parse FunctionalScript using Deno while we develop the self-hosted parser.
-
-## Stage 2: Moving the Deno into `dev-dependencies`
-
-Once our parser can parse itself and convert it into byte code, we can move the `deno_core` to development dependencies. This means that we need it only for build time. This transition eliminates the runtime dependency on Deno, streamlining deployment and reducing overhead in production environments.
-
-### Build Process
-
-1. Run the parser on itself using Deno and generate byte code for the parser.
-2. Embed the generated byte code into our application ([include_bytes](https://doc.rust-lang.org/std/macro.include_bytes.html)).
-
-### Run-Time Process
-
-```mermaid
-flowchart TB
-  app[Application] --> bcd[Byte Code Deserializer] --> js[VM] --> mem[Memory Manager]
-  app --> fspbc[(Functional Script Parser Byte Code)]
-```
-
-The application can do two things:
-1. Convert FunctionalScript code into byte code using the parser source code and the VM.
-2. Execute byte code using the VM.
-
-## Byte code
-
-The byte code format is designed for fast and straightforward deserialization and doesn't depend on a particular VM implementation. This design ensures that the byte code remains portable and does not rely on a specific VM implementation, allowing flexibility in integrating with other environments, including content-addressable implementation of FunctionalScript.
-
-### Requirements
-
-- **Simple deserialization**: `string` is UTF16, `number` and `bigint` in a binary format, `usize` is 32 bits.
-- **No imports**: byte code describes a standalone module without dependencies. A parser should resolve all imports.
-- **No syntax sugar**: a parser should convert all syntax sugar operations into basic commands.
-- **One unit is one byte**: Byte code can be serialized into a byte array or a file.
-- **Least significant byte first**: Our `bigint` implementation is LSB first, and most current CPU architectures are.
-
-### Pseudo-Code for the byte code binary format
-
-We use syntax that looks like Rust to describe the binary format.
-
-```rust
-struct Array<T> {
-    len: u32,
-    array: [T; self.len],
-}
- 
-type String = Array<u16>;
-
-// LSB first.
-type BigUInt = Array<u64>;
-
-type Object = Array<(String, Any)>;
-
-type Tag = u8
-
-// This is the main structure for serialization.
-type Code = Array<u8>;
-```
-
-### Tags/commands for JSON and DJS.
-
-|format|any           |Tag|                       |
-|------|--------------|---|-----------------------|
-|JSON  |null          | 00|                       |
-|      |number        | 01|u64                    |
-|      |false         | 02|                       |
-|      |true          | 03|                       |
-|      |string        | 04|String                 |
-|      |object        | 05|Object                 |
-|      |array         | 06|Array<Any>             |
-|DJS   |reference     | 07|u32                    |
-|      |bigint+       | 08|BigUInt                |
-|      |bigint-       | 09|BigUInt                |
-
-We will add new tags for FunctionalScript byte code in the future as they are not needed for Stage 1. 
-
-## Summary
-
-If you find this architecture promising and want to support the development of a standalone FunctionalScript VM, please consider [sponsoring the project ❤️](https://opencollective.com/functionalscript). Additionally, we welcome contributions and feedback from the open-source community to accelerate this initiative.

package/doc/vm.md

@@ -1,150 +0,0 @@
-# NaNVM
-
-https://en.wikipedia.org/wiki/Double-precision_floating-point_format
-
-https://anniecherkaev.com/the-secret-life-of-nan
-https://brionv.com/log/2018/05/17/javascript-engine-internals-nan-boxing/
-
-
-- 1 bit - sign (S)
-- 11 bit - exponent (E)
-- 52 bit - fraction (F)
-
-|SE |F             |Value           |
-|---|--------------|----------------|
-|000|00000_00000000|+000000_00000000|
-|...|              |                |
-|3FF|00000_00000000|+000000_00000001|
-|...|              |                |
-|434|00000_00000000|+200000_00000000|
-|...|              |                |
-|7FF|00000_00000000|+inf            |
-|...|              |NaN             |
-|800|00000_00000000|-0.0            |
-|...|              |                |
-|BFF|00000_00000000|-000000_00000001|
-|...|              |                |
-|C34|00000_00000000|-200000_00000000|
-|...|              |                |
-|FFF|00000_00000000|-inf            |
-|...|              |NaN             |
-
-integer range: `[-2^53; +2^53]`.
-
-## 6-bit Id String
-
-|symbol  |code          |# |sum|
-|--------|--------------|--|---|
-|`$`     |`\x24`        | 1|  1|
-|`0`..`9`|`\x30`..`\x39`| A|  B|
-|`A`..`Z`|`\x41`..`\x5A`|1A| 25|
-|`_`     |`\x5F`        | 1| 26|
-|`a`..`z`|`\x61`..`\x7A`|1A| 40|
-
-## 7FF & FFF
-
-53 bits.
-
-Other values:
-
-- `NaN`
-- `+Inf`: 0x7FF00000_00000000
-- `-Inf`: 0xFFF00000_00000000
-- pointer + null:
-  - 32 bit for 32 bit platforms.
-  - 48 bit for current AMD64 https://en.wikipedia.org/wiki/X86-64#Canonical_form_addresses and ARM64
-    note: with alignments it can be further narrowed to 44-45 bit.
-- `true`
-- `false`
-- `undefined`
-
-Optimization for
-- string
-- bigInt
-
-Least used letters in English: Q, J, Z and X.
-
-### Layout 52
-
-Starts with `0xFFF`
-
-|      |  |             |type       |
-|------|--|-------------|-----------|
-|`1111`|48|stringUInt48 |`string`   |
-|`1110`|48|8 x 6 string |           |
-|`1101`|48|7 x 7 stringA|           |
-|`1100`|48|7 x 7 stringB|           |
-|`1011`|48|6 x 8 string |           |
-|`1010`|45|5 x 9 string |           |
-|`1001`|48|4 x 12 string|           |
-|`1000`|48|3 x 16 string|           |
-|`0111`|32|2 x 16 string|           |
-|`0110`|16|1 x 16 string|           |
-|`0101`| 0|empty string |           |
-|`0100`|48|ptr          |?          |
-|`0011`| 0|undefined    |`undefined`|
-|`0010`|48|bigInt48     |`bigint`   |
-|`0001`| 0|bool         |`bool`     |
-|`0000`| 0|-inf         |`number`   |
-
-## Pointer Kind
-
-Alignment 8 bytes. 3 bits.
-
-|    |type      |               |
-|----|----------|---------------|
-|00.0|`object`  |object         |
-|00.1|          |array          |
-|01.0|`string`  |string         |
-|01.1|          |               |
-|10.0|`function`|function       |
-|10.1|          |static function|
-|11.0|`bigint`  |bigint         |
-|11.1|          |               |
-
-## Object
-
-```rust
-struct Object {
-  propertySet: PropertySet
-  iterable: Func<Iterator>
-}
-
-struct Array {
-  length: u32,
-  array: [Value; self.length],
-}
-```
-
-## String
-
-```rust
-struct String {
-  length: u32,
-  array: [u16; self.length],
-}
-```
-
-## Function
-
-```rust
-struct Function {
-  pointer: FunctionBody,
-  array: [Value; length],
-}
-```
-
-## BigInt
-
-```rust
-struct BigInt {
-  length: u32,
-  array: [u64; self.length],
-}
-```
-
-## Order of object properties
-
-See https://262.ecma-international.org/6.0/#sec-ordinary-object-internal-methods-and-internal-slots-ownpropertykeys and https://262.ecma-international.org/6.0/#sec-object-type
-
-An integer index for Node.js, Deno and Bun means a value from `0` to `4294967294` including. 4_294_967_294 = 0xFFFF_FFFE. But an integer index in the ES6 standard is +0..2^53-1.

package/fsc/README.md

@@ -1,111 +0,0 @@
-# FunctionalScript Compiler
-
-## Tokenizer
-
-- `!` - logicalNot
-  - `!=` - nonStrictNotEqual
-  - `!==` - notEqual
-- `"` - doubleStringBegin
-- `$` - idBegin
-- `%` - remainder
-  - `%=` - remainder assignment
-- `&` - bitwiseAnd
-  - `&&` - logicalAnd
-  - `&&=` - logicalAndAssignment
-  - `&=` - bitwiseAndAssignment
-- `'` - singleStringBegin
-- `(` - groupingBegin
-- `)` - groupingEnd
-- `*` - multiplication
-  - `**` - exponential
-  - `**=` - exponentialAssignment
-  - `*/` - commentEnd
-  - `*=` - multiplicationAssignment
-- `+` - addition
-  - `++` - increment
-  - `+=` - additionAssignment
-- `,` - comma
-- `-` - subtraction
-  - `--` - decrement
-  - `-=` - subtractionAssignment
-- `.` - dot
-  - `...` - spread
-- `/` - division
-  - `/*` - commentBegin
-  - `//` - oneLineComment
-  - `/=` - divisionAssignment
-- `0..9` - numberBegin
-- `:` - colon
-- `;` - semicolon
-- `<` - less
-  - `<<` - leftShift
-  - `<<=` - leftShiftAssignment
-  - `<=` - lessEqual
-- `=` - assignment
-  - `==` - nonStrictEqual ?
-  - `===` - equal
-  - `=>` - arrow
-- `>` - greater
-  - `<=` - greaterEqual
-  - `>>` - rightShift
-  - `>>>` - unsignedRightShift
-  - `>>>=` unsignedRightShiftAssignment
-- `?` - question
-  - `?.` - optional chaining
-  - `??` - nullish coalescing
-- `A..Z` - idBegin
-  - `Infinity` - infinity
-  - `NaN` - nan
-- `[` - propertyBegin
-- `]` - propertyEnd`
-- `^` - bitwiseXor
-  - `^=` - bitwiseXorAssignment
-- `_` - idBegin
-- '`' - templateBegin
-- `a..z` - idBegin
-  - `async` ?
-  - `await` ?
-  - `break`
-  - `case`
-  - `catch`
-  - `class`
-  - `const`
-  - `continue`
-  - `debugger`
-  - `delete` ?
-  - `do` ?
-  - `else`
-  - `export`
-  - `false`
-  - `function` ?
-  - `globalThis` ?
-  - `for`
-  - `if`
-  - `import`
-  - `in`
-  - `instanceof`
-  - `let`
-  - `new` ?
-  - `null`
-  - `of`
-  - `return`
-  - `super`
-  - `switch`
-  - `this` ?
-  - `throw`
-  - `true`
-  - `try`
-  - `typeof`
-  - `undefined`
-  - `var`
-  - `void`
-  - `while`
-  - `yield`
-- `{` - objectBegin
-- `|` - bitwiseOr
-  - `|=` - bitwiseOrAssignment
-  - `||` - logicalOr
-  - `||=` - logicalOrAssignment
-- `}` - objectEnd
-- `~` - bitwiseNot
-  - `~=` - bitwiseNotAssignment

package/fsc/module.f.ts

@@ -1,122 +0,0 @@
-import * as operator from '../types/function/operator/module.f.ts'
-import {
-    merge as rangeMapMerge,
-    fromRange,
-    get,
-    type RangeMapArray,
-    type RangeMerge,
-} from '../types/range_map/module.f.ts'
-import * as list from '../types/list/module.f.ts'
-const { reduce: listReduce } = list
-import { range as asciiRange } from '../text/ascii/module.f.ts'
-import { fn } from '../types/function/module.f.ts'
-import { one, type Range } from '../types/range/module.f.ts'
-const { toArray, map } = list
-
-const fromCharCode = String.fromCharCode
-
-type Result = readonly[readonly string[], ToResult]
-
-type ToResult = (codePoint: number) => Result
-
-type CreateToResult<T> = (state: T) => ToResult
-
-type State<T> = RangeMapArray<CreateToResult<T>>
-
-const unexpectedSymbol
-    : ToResult
-    = codePoint => [[`unexpected symbol ${codePoint}`], unexpectedSymbol]
-
-const def
-    : <T>(state: T) => ToResult
-    = () => unexpectedSymbol
-
-const union
-    : <T>(a: CreateToResult<T>) => (b: CreateToResult<T>) => CreateToResult<T>
-    = a => b => {
-    if (a === def || a === b) { return b }
-    if (b === def) { return a }
-    throw [a, b]
-}
-
-const empty
-    : readonly never[]
-    = []
-
-const reduce = <T>(a: list.List<State<T>>): State<T> => {
-    const merge
-    : RangeMerge<CreateToResult<T>>
-    = rangeMapMerge({
-        union,
-        equal: operator.strictEqual,
-    })
-    return toArray(listReduce(merge)(empty)(a))
-}
-
-const codePointRange = fromRange(def)
-
-const range = fn(asciiRange).then(codePointRange).result
-
-const rangeSet
-= (l: readonly string[]) => <T>(f: CreateToResult<T>): State<T> => {
-
-    const codePointRange
-    : (a: Range) => (f: CreateToResult<T>) => State<T>
-    = fromRange(def)
-
-    const g
-    : (r: string) => State<T>
-    = r => codePointRange(asciiRange(r))(f)
-
-    return reduce(map(g)(l))
-}
-
-const create = <T>(a: list.List<State<T>>): CreateToResult<T> => {
-    const i = reduce(a)
-    const x
-        : (v: number) => (i: State<T>) => (v: T) => ToResult
-        = get(def)
-    return v => c => x(c)(i)(v)(c)
-}
-
-export const terminal = -1
-
-const toInit
-    : () => ToResult
-    = () => () => [[], init]
-
-export const init
-: ToResult
-= create([
-    codePointRange(one(terminal))(toInit),
-    rangeSet(['\t', ' ', '\n', '\r'])(toInit),
-    range('!')(() => () => [['!'], unexpectedSymbol]),
-    range('"')(() => () => [['"'], unexpectedSymbol]),
-    rangeSet(['$', '_', 'AZ', 'az'])(() => c => [[fromCharCode(c)], unexpectedSymbol]),
-    range('%')(() => () => [['%'], unexpectedSymbol]),
-    range('&')(() => () => [['&'], unexpectedSymbol]),
-    range("'")(() => () => [["'"], unexpectedSymbol]),
-    range('(')(() => () => [['('], unexpectedSymbol]),
-    range(')')(() => () => [[')'], unexpectedSymbol]),
-    range('*')(() => () => [['*'], unexpectedSymbol]),
-    range('+')(() => () => [['+'], unexpectedSymbol]),
-    range(',')(() => () => [[','], unexpectedSymbol]),
-    range('-')(() => () => [['-'], unexpectedSymbol]),
-    range('.')(() => () => [['.'], unexpectedSymbol]),
-    range('/')(() => () => [['/'], unexpectedSymbol]),
-    range('09')(() => a => [[fromCharCode(a)], unexpectedSymbol]),
-    range(':')(() => () => [[':'], unexpectedSymbol]),
-    range(';')(() => () => [[';'], unexpectedSymbol]),
-    range('<')(() => () => [['<'], unexpectedSymbol]),
-    range('=')(() => () => [['='], unexpectedSymbol]),
-    range('>')(() => () => [['>'], unexpectedSymbol]),
-    range('?')(() => () => [['?'], unexpectedSymbol]),
-    range('[')(() => () => [['['], unexpectedSymbol]),
-    range(']')(() => () => [[']'], unexpectedSymbol]),
-    range('^')(() => () => [['^'], unexpectedSymbol]),
-    range('`')(() => () => [['`'], unexpectedSymbol]),
-    range('{')(() => () => [['{'], unexpectedSymbol]),
-    range('|')(() => () => [['|'], unexpectedSymbol]),
-    range('}')(() => () => [['}'], unexpectedSymbol]),
-    range('~')(() => () => [['~'], unexpectedSymbol]),
-])(undefined)

package/fsc/test.f.ts

@@ -1,21 +0,0 @@
-import * as _ from './module.f.ts'
-import * as ascii from '../text/ascii/module.f.ts'
-const { one } = ascii
-import * as j from '../json/module.f.ts'
-const { stringify } = j
-const s = stringify(i => i)
-
-/** @type {} */
-const f
-    : (v: string) => string
-    = v => {
-    const n = one(v)
-    return s(_.init(n)[0])
-}
-
-export default {
-    a: () => {
-        const x = f('1')
-        if (x != '["1"]') { throw x }
-    }
-}

package/fsm/README.md

@@ -1,113 +0,0 @@
-# FSM
-
-This module is about a [finite-state machine](https://en.wikipedia.org/wiki/Finite-state_machine). It's not about
-[Flying Spaghetti Monster](https://en.wikipedia.org/wiki/Flying_Spaghetti_Monster).
-
-## FA Expression
-
-- `symbol(s)`
-- `next state`
-
-## FA Rule
-
-- `init state`
-- `FA Epression`
-
-## Example
-
-FunctionalScript Identifier
-
-### Grammar (EBNF)
-
-```
-id ::= `_$a..zA..Z` id2
-id2 ::= `_$a..zA..Z0..9` id2
-```
-
-### FA
-
-```js
-const lowercaseAlpha = byteSet.range('a')('z')
-const uppercaseAlpha = byteSet.range('A')('Z')
-const alpha = byteSet.union(lowercaseAlpha)(uppercaseAlpha)
-const idSymbol = byteSet.union(byteSet.one('_'))(byteSet.one('$'))
-const idBegin = byteSet.union(alpha)(idSymbol)
-
-const digit = byteSet.range('0')('9')
-const idNext = byteSet.union(idBegin)(digit)
-
-const dot = byteSet('.')
-
-const grammar = [
-  ['', digit, 'int'],
-  ['int', digit, 'int'],
-  //
-  ['', digit, 'floatBegin'],
-  ['floatBegin', digit, 'floatBegin'],
-  ['floatBegin', dot, 'floatDot'],
-  ['floatDot', digit, 'float'],
-  ['float', digit, 'float'],
-  //
-  ['', idBegin, 'id'],
-  ['id', idNext, 'id'],
-]
-```
-
-```js
-const result = {
-  "['']": {
-    digit: "['floatBegin','int']",
-    idBegin: "['id']"
-  },
-  "['floatBegin','int']": {
-    digit: "['floatBegin','int']",
-    dot: "['floatDot']",
-  },
-  "['floatDot']": {
-    digit: "['float']"
-  },
-  "['float']": {
-    digit: "['float']"
-  },
-  "['id']": {
-    idNext: "['id']"
-  }
-}
-```
-
-```js
-const result = [
-  { // 0
-    digit: 1,
-    idBegin: 4
-  },
-  { // 1
-    digit: 1,
-    dot: 2,
-  },
-  { // 2
-    digit: 3
-  },
-  { // 3
-    digit: 3
-  },
-  { // 4
-    idNext: 4
-  }
-}
-```
-
-## How to Add a Property # 1
-
-```js
-const a = { x: 5, y: 6 }
-const b = { ...a, z: 7 }
-```
-
-## How to Add a Property # 2
-
-```js
-import map from './types/map/module.f.mjs'
-const a = map.fromEntries(Object.entries({ x: 5, y: 6 }))
-const b = map.setReplace('z')(7)(a)
-```