ranger-compiler 3.1.0 → 3.1.1

package/CHANGELOG.md

@@ -7,6 +7,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [3.1.1] - 2026-06-23
+
+### Changed
+
+- Version bump for npm publish — recommended for cloud CI and projects that install `ranger-compiler` from npm (e.g. koodisampo) instead of a sibling `../agent/Ranger` checkout
+- Default `npm test` / `prepublishOnly` skips `compiler-llvm.test.ts` (experimental LLVM/WAT backend); run `npm run test:llvm` when working on native/WASM codegen
+
+### Added
+
+- **IsoDate stdlib** — `lib/IsoDate/` (`DateMath`, `IsoDateParse`, `IsoCalendar`) and `lib/IsoDateLib.rgr` for portable ISO calendar dates without host `Date`; see [ai/ISO_DATE.md](ai/ISO_DATE.md)
+- **IsoDate compiler intrinsics** — `iso_add_days`, `iso_compare`, `iso_between` in `Lang.rgr` (Kotlin/Java `java.time.LocalDate`, ES6 UTC-safe helper)
+- **IsoDate regression** — `tests/fixtures/iso_date_ops.rgr` in Kotlin compiler tests
+- **Regex stdlib** — `lib/Regex/` (`RegexMatch`) and `lib/RegexLib.rgr` for string-pattern matching without `/literal/` syntax; see [ai/REGEX.md](ai/REGEX.md)
+- **Regex compiler intrinsic** — `regex_test(pattern, haystack)` in `Lang.rgr` (Kotlin/Java `Regex`/`Pattern`, ES6 `RegExp`, Swift `range(of:options:)`)
+- **Regex regression** — `tests/fixtures/regex_test_ops.rgr` in Kotlin compiler tests
+
+- **JavaScript/TypeScript source maps (`-sourcemap`)** — `SourceMapBuilder` with VLQ encoding (`compiler/ng_SourceMap.rgr`); `CodeWriter` line/column tracking, `walkNodeStack`, and `outMapped()`; embedded `sourcesContent` for `.rgr` sources; `.js.map` + `//# sourceMappingURL=` on save; statement/expression mappings via `LiveCompiler.WalkNode` walk context; expression `names` from `node.vref` / parameter names; regression `tests/compiler-sourcemap.test.ts`; README section *JavaScript / TypeScript source maps*
+
+### Fixed
+
+- **Source map VLQ line breaks** — `buildMappingsString` no longer resets source/original relative state on `;` (fixes DevTools breakpoints on `.rgr` sources)
+- **Source map original line** — `addMappingFromNode` uses `node.getLine()` from `sp` instead of stale `node.row`
+- **Kotlin `floor` / `int2double`** — `int2double` emits `.toDouble()`; reserved parameter names escaped (`val`, `object`, …)
+- **`proc_send` dispatch wrapping** — turn boundaries around handler calls; `ProcessRuntime.beginDispatchTurn` / `endDispatchTurn`
+
 ## [3.1.0] - 2026-06-02
 
 ### Added

package/README.md

@@ -1,6 +1,6 @@
 # Ranger cross language compiler
 
-**Version 3.1.0** | Status: `experimental`
+**Version 3.1.1** | Status: `experimental`
 
 Ranger is a self-hosting cross-language compiler for writing portable algorithms, parsers, generators, and small tools once and compiling them to multiple target languages.
 
@@ -56,7 +56,7 @@ const host = ProcessUiHost.__singleton();
 host.notifyPath = (path) => { /* sync view model + re-render */ };
 ```
 
-**Docs:** [PROCESS_MVP.md](PROCESS_MVP.md) (scope), [PROCESS_STATUS.md](PROCESS_STATUS.md) (compiler checklist), [PROCESS_UI_NOTIFY.md](PROCESS_UI_NOTIFY.md) (notify batching), [PROCESS_UI_VIEW_MODELS.md](PROCESS_UI_VIEW_MODELS.md) (view DTO assignment). **Gallery:** [process_counter_board](gallery/process_counter_board/README.md). **Pilot:** [realtrainer `app-ranger` Active Workout demo](https://github.com/terotests/realtrainer/tree/copilot/create-watch-ui-components/app-ranger/demo/active-workout-process).
+**Docs:** [PROCESS_MVP.md](PROCESS_MVP.md) (scope), [PROCESS_STATUS.md](PROCESS_STATUS.md) (compiler checklist), [PROCESS_RUNTIME_INVARIANTS.md](PROCESS_RUNTIME_INVARIANTS.md) (dispatch turn / one notify), [PROCESS_UI_NOTIFY.md](PROCESS_UI_NOTIFY.md) (notify batching), [PROCESS_UI_VIEW_MODELS.md](PROCESS_UI_VIEW_MODELS.md) (view DTO assignment). **Gallery:** [process_counter_board](gallery/process_counter_board/README.md) (Vite + React host for `@process`).
 
 ## Where To Start
 
@@ -69,6 +69,7 @@ host.notifyPath = (path) => { /* sync view model + re-render */ };
 - `gallery/js_parser` - substantial parser example with benchmarks and README
 - `gallery/pdf_writer` - EVG / TSX document tooling and preview server
 - `gallery/invaders` - cross-target demo game
+- `gallery/invaders/llvm/invaders.ll` - checked-in LLVM IR sample from the experimental `-l=llvm` backend
 
 ## Compatibility Snapshot
 
@@ -80,6 +81,7 @@ The project can target `JavaScript`, `Java`, `Go`, `Swift`, `PHP`, `C++`, `C#`,
 | Self-hosting | Actively used, but full compiler generation quality is strongest in JavaScript |
 | JavaScript / ES6 | Best baseline target and most reliable place to start |
 | Go / Swift / Rust / Kotlin / C++ | Useful and increasingly capable, but expect edge cases and target-specific gaps |
+| **LLVM / WASM** (`-l=llvm`) | **Experimental.** Lowers to LLVM IR; optional WAT export for freestanding WASM. Native libc builds work for demos like Space Invaders; browser WASM is still rough. See `npm run test:llvm`, `npm run game:build:llvm`. |
 | Gallery examples | Good for understanding direction and capability, but some require manual setup or platform-specific tooling |
 
 ## What's New in Version 3.0
@@ -115,9 +117,9 @@ See [CHANGELOG.md](CHANGELOG.md) for full version history and [PLAN_3.md](PLAN_3
 The compiler is _self hosting_ which means that it has been written using the compiler itself and thus it can be hosted
 on several platforms. At the moment the official platform is node.js, because external plugins are only available as npm packages.
 
-The target languages supported are `JavaScript`, `Java`, `Go`, `Swift`, `PHP`, `C++`, `C#`, `Scala`, `Python`, and `Rust`. The quality
-of the target translation still varies and at the moment of this writing the compiler can only be compiled fully to JavaScript
-target. However, most targets already can compile reasonably good code.
+The target languages supported are `JavaScript`, `Java`, `Go`, `Swift`, `PHP`, `C++`, `C#`, `Scala`, `Python`, and `Rust`. An **experimental LLVM backend** (`-l=llvm`) emits LLVM IR and optional freestanding WAT for WASM toolchains; it is not a separate surface language but another codegen path from the same Ranger sources.
+
+The quality of the target translation still varies and at the moment of this writing the compiler can only be compiled fully to JavaScript target. However, most targets already can compile reasonably good code.
 
 ## Recent Updates (December 2025)
 
@@ -394,11 +396,12 @@ See `gallery/pdf_writer/examples/test_for_loop.tsx` for a complete demonstration
 
 ### Space Invaders Demo Game
 
-A complete terminal-based Space Invaders game demonstrating Ranger's cross-language capabilities. The same source code compiles to **4 different targets**:
+A complete terminal-based Space Invaders game demonstrating Ranger's cross-language capabilities. The same source code compiles to **several targets**:
 
-| Target         | Executable          | Build Command               |
+| Target         | Output              | Build Command               |
 | -------------- | ------------------- | --------------------------- |
 | ES6/JavaScript | `invaders.js`       | `npm run game:compile`      |
+| **LLVM native**| `tmp/invaders-native/invaders` | `npm run game:build:llvm` |
 | Rust           | `invaders_rust.exe` | `npm run game:build:rust`   |
 | Go             | `invaders_go.exe`   | `npm run game:build:go`     |
 | Kotlin         | `invaders.jar`      | `npm run game:build:kotlin` |
@@ -415,8 +418,24 @@ npm run game:build:all
 npm run game:run        # JavaScript
 npm run game:run:rust   # Rust
 npm run game:run:go     # Go
+./tmp/invaders-native/invaders   # LLVM native (after game:build:llvm)
+```
+
+#### Experimental LLVM backend (Space Invaders)
+
+Ranger can lower the same `invaders.rgr` through a **Low IR → LLVM IR** pipeline (`-l=llvm`), then link with `clang` and a small C runtime (`runtime/ranger_term.c`) for terminal I/O.
+
+```bash
+npm run compile              # refresh bin/output.js after compiler changes
+npm run game:build:llvm      # invaders.rgr → tmp/invaders-native/invaders.ll → native binary
+npm run test:llvm            # LLVM/WASM fixture tests (vitest)
+npm run demo:wasm            # smaller freestanding WASM demo (tests/fixtures/llvm_wasm_demo.rgr)
 ```
 
+**Checked-in sample:** [`gallery/invaders/llvm/invaders.ll`](gallery/invaders/llvm/invaders.ll) is LLVM IR kept in the repo so you can inspect codegen without building. Regenerate with `cp tmp/invaders-native/invaders.ll gallery/invaders/llvm/invaders.ll` after `game:build:llvm` (see [`gallery/invaders/llvm/README.md`](gallery/invaders/llvm/README.md)).
+
+Status: experimental — libc-linked native builds are the most reliable path; freestanding WAT/WASM for the full game is still incomplete (terminal imports, control-flow lowering). Smaller WASM demos under `npm run demo:wasm` are closer to working in the browser.
+
 #### Cross-Compiling the Game
 
 The Space Invaders game demonstrates cross-platform compilation from a single source file.
@@ -470,6 +489,15 @@ npm run game:compile:swift  # Generates invaders.swift
 swiftc invaders.swift -o invaders_swift
 ```
 
+**LLVM native (macOS / Linux, experimental)**
+
+```bash
+npm run game:build:llvm
+./tmp/invaders-native/invaders
+```
+
+Requires `clang` on `PATH`. On macOS the script picks `arm64-apple-macos` or `x86_64-apple-macos` automatically.
+
 #### Platform-Specific Keyboard Input
 
 The game uses `on_keypress` and `poll_keypress` operators with platform-specific implementations:
@@ -492,6 +520,7 @@ The Space Invaders game provides an interesting comparison of how the same Range
 | Target     | Generated File   | Size (bytes) | Lines | Notes                           |
 | ---------- | ---------------- | ------------ | ----- | ------------------------------- |
 | **Ranger** | `invaders.rgr`   | 11,289       | ~400  | Original source                 |
+| **LLVM IR**| `llvm/invaders.ll` | ~60,000    | ~1,700 | Low-level IR (sample in repo)   |
 | Python     | `invaders.py`    | 9,271        | ~330  | Most compact generated code     |
 | JavaScript | `invaders.js`    | 10,301       | ~350  | Clean, readable output          |
 | Swift      | `invaders.swift` | 12,554       | ~470  | Verbose type annotations        |
@@ -499,11 +528,12 @@ The Space Invaders game provides an interesting comparison of how the same Range
 | C++        | `invaders.cpp`   | 14,148       | ~500  | Headers and type declarations   |
 | Rust       | `invaders.rs`    | 17,918       | ~600  | Most verbose (ownership, types) |
 
-**Executable Sizes (Windows):**
+**Executable Sizes (native binaries):**
 
 | Target | Executable           | Size   | Notes                             |
 | ------ | -------------------- | ------ | --------------------------------- |
-| Swift  | `invaders_swift.exe` | 76 KB  | Smallest native binary            |
+| **LLVM** | `tmp/invaders-native/invaders` | ~34 KB (arm64 macOS) | Smallest in recent local builds; libc + minimal runtime |
+| Swift  | `invaders_swift.exe` | 76 KB  | Dynamic link to system libraries  |
 | Rust   | `invaders_rust.exe`  | 291 KB | Optimized, statically linked      |
 | Go     | `invaders_go.exe`    | 2.3 MB | Includes Go runtime               |
 | C++    | `invaders_cpp.exe`   | 3.0 MB | Static linking with MinGW/pthread |
@@ -766,6 +796,7 @@ Flags: -<flag>
   -nodecli       Insert node.js command line header #!/usr/bin/env node to the beginning of the JavaScript file
   -nodemodule    Export classes as CommonJS modules using module.exports (disables static main function)
   -esm           Export classes as ES6/ESM modules using export keyword (disables static main function)
+  -sourcemap     Emit .js.map / .ts.map with embedded .rgr sourcesContent (ES6/TypeScript only)
   -client        the code is ment to be run in the client environment
   -scalafiddle   scalafiddle.io compatible output
   -compiler      recompile the compiler
@@ -800,6 +831,47 @@ node bin/output.js -es6 -esm myfile.rgr -o=myfile.mjs
 **File Extensions:**
 The compiler automatically detects JavaScript-related extensions (`.js`, `.ts`, `.mjs`, `.cjs`) and won't double-add them. You can safely specify the full filename with extension.
 
+### JavaScript / TypeScript source maps (`-sourcemap`)
+
+Use `-sourcemap` with `-es6` or `-typescript` to emit a sibling `.js.map` / `.ts.map` file and append `//# sourceMappingURL=…` to the generated output. Kotlin, Swift, and other non-JS targets ignore this flag.
+
+**What you get**
+
+| Output | Purpose |
+| --- | --- |
+| `output.js.map` | Source map v3 (VLQ `mappings`, `names`, `sources`) |
+| `sourcesContent` | Full `.rgr` source embedded in the map — Chrome DevTools can open `.rgr` files without a separate file server |
+| Statement + expression mappings | `LiveCompiler.WalkNode` walk context plus `outMapped()` on identifiers, calls, literals |
+
+**Compile example**
+
+```bash
+# Standalone ES module + map
+node bin/output.js -es6 -esm -nodemodule -sourcemap ./myapp/App.rgr -o=app.js
+
+# Result: bin/app.js and bin/app.js.map
+```
+
+**Debug in Chrome / Edge**
+
+1. Serve the generated `.js` (and `.map` beside it). Vite/webpack are optional when `sourcesContent` is embedded.
+2. Open DevTools → **Sources**. Original `.rgr` files appear under the map tree (from `sourcesContent`).
+3. Set breakpoints on **executable** lines (e.g. `def`, `if`, `return`) — not only blank lines or signatures.
+4. Breakpoints must be **solid red**. A hollow/grey breakpoint means no mapping for that line; rebuild with `-sourcemap` and hard-refresh (disable cache).
+
+**Tests**
+
+```bash
+npm run compile
+npx vitest run tests/compiler-sourcemap.test.ts
+```
+
+**Implementation notes** (for compiler hackers)
+
+- `compiler/ng_SourceMap.rgr` — `SourceMapBuilder`, VLQ encoder, `addMappingFromNode()` uses `node.getLine()` + `node.code.getColumn(sp)` (not stale `node.row`).
+- `compiler/ng_writer.rgr` — `lineNumber` / `columnNumber` on emit, `walkNodeStack`, `outMapped()`, `.map` write in `CodeFileSystem.saveTo`.
+- Flag: `compiler/ng_Compiler.rgr` → `flag sourcemap`; enabled in `VirtualCompiler.rgr` via `fileSystem.enableSourceMaps()`.
+
 ## Getting started with Hello World
 
 Create file `hello.rgr`