jscall 1.0.1 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 97c7ab8cd5d710cfca8c77f6e906643252b9c54cc5973709763b89aef3830dee
-  data.tar.gz: e205fad16cc87df640b186ac99190186374f2db821a7e3d499563add9016b097
+  metadata.gz: a5fb1072aa12a5e884d9c92537f6ae7a5c632d4940715008865ec9760940f1c2
+  data.tar.gz: 597c3199f5ee21f98d80a42b05bf9822b729a22311d7c9f43e2cf45d54594983
 SHA512:
-  metadata.gz: f25ebf232e77432bb5c79f511c4fc99de78e2d7861e4d3831edcf3662e747021d5d4a207036c8a7add2fe1822c9fb8fdb7439b1e17e67627abea090c1285662c
-  data.tar.gz: 8e02fe7a9f3d7f0d82ac70f24d33335c2dffe1261133e2dec4a1c4ff65ec9b87bb9fad8f0049dee3fb681c20e7b3f47d9a97890b45958199f1e78c0b76fd69b5
+  metadata.gz: 54c16c0015f2296309523ecf5d966bbf2071303136afef04e0bbdf139aa204c4b3db332a73ea65c0194fce7cea795503db17577805b4110a56edda796fca0a0b
+  data.tar.gz: ada3a68d6c19093c22087982e520ed18a0f3f3709584fc2c30978aed7909c1b6b8568eb48e24fde6c57cc759dc235b1a824524b1c0d43053d4d80512767339a8

data/README.md

@@ -1,5 +1,7 @@
 # Jscall
 
+[![Ruby](https://github.com/csg-tokyo/jscall/actions/workflows/ruby.yml/badge.svg)](https://github.com/csg-tokyo/jscall/actions/workflows/ruby.yml)
+
 Jscall allows executing a program in JavaScript on node.js or a web browser.
 By default, node.js is used for the execution.
 To choose a web browser, call `Jscall.config`.
@@ -19,10 +21,14 @@ This returns `2`.  The argument passed to `Jscall.exec` can be
 multipe lines.  It is executed as source code written in JavaScript.
 
 `Jscall.exec` returns a resulting value.  Numbers, character strings (and symbols), boolean values, and `nil` (and `null`)
-are copied when passing between Ruby and JavaScript.  An array is also shallow-copied.
+are copied when passing between Ruby and JavaScript.  An array is shallow-copied.
 Other objects are not copied.  When they are passed, a remote reference is created at the destination.
+When a `Map` object is returned from JavaScript to Ruby, it is also
+shallow-copied but into a `Hash` object in Ruby.
 
-A remote reference is a proxy object.  A method call on a remote reference invokes a method on the corresponding object on the remote site.  For example,
+A remote reference is a local reference to a proxy object.
+A method call on a remote reference invokes a method on the corresponding
+object on the remote site.  For example,
 
 ```
 js_obj = Jscall.exec '({ foo: (x) => x + 1, bar: 7 })'
@@ -35,10 +41,29 @@ js_obj.baz       # 9
 The `foo` method is executed in JavaScript.
 Since `bar` is not a function, its value is returned to Ruby as it is.
 
-Setting a object property to a given value is also
+Setting an object property to a given value is also
 allowed.  The expression `js_obj.baz = 9` above sets
 the object property `baz` to 9.
 
+An argument to a JavaScript method is copied from Ruby to
+JavaScript unless it is an object.  When an argument is a Ruby object,
+a proxy object is created in JavaScript.  The rule is the same as the
+rule for returning a value from JavaScript to Ruby.  A primitive
+value is copied but an object is not.  An array is shallow-copied.
+
+A `Hash` object in Ruby is also shallow-copied into JavaScript but a normal
+object is created in JavaScript.  Recall that a JavaScript object is
+regarded as an associative array, or a hash table as in Ruby.
+For example, in Ruby,
+
+```
+obj = { a: 2, b: 3 }
+```
+
+when this ruby object is passed to JavaScript as an argument,
+a normal object `{ a: 2, b: 3 }` is created as its copy in JavaScript
+and passed to a JavaScript method.
+
 To call a JavaScript function from Ruby, call a mehtod on `Jscall`.
 For example,
 
@@ -54,9 +79,22 @@ Jscall.foo(7)    # 8
 `Jscall.foo(7)` invokes the JavaScript function with the name following `Jscall.`
 with the given argument.  In this case,
 the `foo` function is executed with the argument `7`.
+Arguments and a return value are passed to/from a function
+as they are passed to/from a method.
+
+`Jscall` can be used for obtaining a remote reference to access a global variable
+in JavaScript.  For example,
+
+```
+Jscall.console.log('Hello')
+```
+
+This prints `Hello` on a JavaScript console.  `Jscall.console` returns a remote
+refererence to the value of `console` in JavaScript.  Then, `.log('Hello')`
+calls the `log` method on `console` in JavaScript.
 
 When a Ruby object is passed to a JavaScript function/method,
-it can call a method on the passed Ruby object.
+you can call a method on the passed Ruby object.
 
 ```
 Jscall.exec <<CODE
@@ -84,10 +122,19 @@ CODE
 Jscall.foo()
 ```
 
-`Jscall.foo()` returns the result of evaluating `RUBY_VERSION`
-in Ruby.
+`Jscall.foo()` returns the result of evaluating given Ruby code
+`RUBY_VERSION` in Ruby.
 Don't forget to `await` a call to `Ruby.exec`.
 
+### Remote references
+
+A remote reference is implemented by a local reference to a proxy
+object representing the remote object that the remote reference refers to.
+When a proxy object is passed as an argument or a return value
+from Ruby to JavaScript (or vice versa), the corresponding JavaScript
+(or Ruby) object is passed to the destination.  In other words,
+a remote reference passed is converted back to a local reference.
+
 Remote references will be automatically reclaimed when they are no
 longer used.  To reclaim them immediately, call:
 
@@ -95,6 +142,25 @@ longer used.  To reclaim them immediately, call:
 Jscall.scavenge_references
 ```
 
+As mentioned above, a remote reference is a local reference
+to a proxy object.  In Ruby,
+even a proxy object provides a number of methods inherited from `Object` class,
+such as `clone`, `to_s`, and `inspect`.  A call to such a method is not
+delegated to the corresponding JavaScript object.  To invoke such a method
+on a JavaScript object, call `send` on its proxy object.
+For example,
+
+```
+js_obj = Jscall.exec '({ to_s: (x, y) => x + y })'
+puts js_obj.to_s(3, 4)            # error
+puts js_obj.send('to_s', 3, 4)    # 7
+```
+
+The `send` method invokes the JavaScript method with the name specified
+by the first argument.  The remaining arguments passed to `send` are passed
+to that JavaScript method.
+
+
 ## DOM manipulation
 
 When JavaScript code is run on a browser, some utility methods
@@ -109,7 +175,16 @@ links `mystyle.css` in the current directory.
 - `Jscall.dom.print(msg)`
 
 This adds a `p` element to the DOM tree.
+Its inner text is the character string passed as `msg`.
 
+- `Jscall.dom.append_to_body(html_source)`
+
+This inserts the given `html_source` at the end of the `body` element.
+It is a shorthand for
+
+```
+Jscall.document.body.insertAdjacentHTML('beforeend', html_source)
+```
 
 ## Variable scope
 
@@ -142,16 +217,33 @@ for loading a CommonJS module.  For example,
 Jscall.exec "mime = require('./mime.js')"
 ```
 
-The file `./mime.js` is loaded and the module is bound to a global variable `mime`.
+The file `./mime.js` is loaded and the module is bound to a global variable `mime` in JavaScript.
+
+You can directly call `require` on `Jscall` in Ruby.
+
+```
+parser = Jscall.require("@babel/parser")
+ast = parser.parse('const i = 3')
+Jscall.console.log(ast)
+```
+
+`require` will search `./node_modules/` for `@babel/parser`.
+This is equivalent to the following JavaScript code.
 
-You may want to call `Jscall.dyn_import` in Ruby.
+```
+parser = require("@babel/parser")
+ast = parser.parse('const i = 3')
+console.log(ast)
+```
+
+Dynamic importing is also available.  Call `Jscall.dyn_import` in Ruby.
 
 ```
 fs = Jscall.dyn_import('fs')
 ```
 
 This executes dynamic importing in JavaScript.
-For node.js, the file name of the imported module should be a full path name.  For a web browser, the root directory is a current working directory.  So `Jscall.dyn_import('/mine.mjs')` loads the file `./mine.mjs`.
+For node.js, the file name of the imported module should be a full path name.  For a web browser, the root directory is the current working directory.  So `Jscall.dyn_import('/mine.mjs')` loads the file `./mine.mjs`.
 
 `Jscall.dyn_import` takes the second argument.  If it is given,
 a global variable in JavaScript is bound to the loaded module.
@@ -166,49 +258,112 @@ This is quite equivalent to the following JavaScript code:
 fs_module = await load('fs')
 ```
 
+## Promise
+
+If a program attempts to pass a `Promise` object from JavaScript to Ruby,
+it waits until the promise is fullfilled.  Then Jscall passes
+the value of that promise from JavaScript to Ruby instead of that
+promise itself (or a remote reference to that promise).  When that promise
+is rejected, an error object is passed to Ruby
+so that the error will be raised in Ruby.
+This design reflects the fact that an `async` function in JavaScript
+also returns a `Promise` object but this object must not be returned
+to Ruby as is when that `async` function is called from Ruby.
+Jscall cannot determine whether a promise should be passed as is to Ruby
+or its value must be passed to Ruby after the promise is fullfilled.
+
+When enforcing Jscall to pass a `Promise` object from JavaScript to Ruby,
+`.async` must be inserted between a receiver and a method name.
+
+```
+Jscall.exec(<<CODE)
+  function make_promise() {
+    return { a: Promise.resolve(7) }
+  }
+CODE
+
+obj = Jscall.make_promise
+result = obj.a                # 7
+prom = obj.async.a            # promise
+prom.then(->(r) { puts r })   # 7
+```
+
 
 ## Configuration
 
-To import JavaScript modules when node.js starts,
+Jscall supports several configuration options.
+Call `Jscall.config` with necessary options.
+
+### module_names:
+
+To import JavaScript modules when node.js or a web browser starts,
 
 ```
-Jscall.config(module_names: [["Foo", "./foo.mjs"], ["Bar", "./bar.mjs"]], options: "--use-strict")
+Jscall.config(module_names: [["Foo", "./js", "/lib/foo.mjs"], ["Bar", "./js", "/lib/bar.mjs"]])
 ```
 
-This specifies that `./foo.mjs` and `./bar.mjs` are impoted when node.js starts.
+This specifies that `./js/lib/foo.mjs` and `./js/lib/bar.mjs` are imported
+at the beginning.
 This is equivalent to the following import declarations:
 
 ```
-import * as "Foo" from "./foo.mjs"
-import * as "Bar" from "./bar.mjs"
+import * as "Foo" from "./js/lib/foo.mjs"
+import * as "Bar" from "./js/lib/bar.mjs"
+```
+
+Note that each array element given to `module_names:` is
+
+```
+[<module_name> <root> <path>]
 ```
 
-`'--use-strict'` is passed to node.js as a command line argument.
+`<path>` must start with `/`.  It is used as a part of the URL when a browser
+accesses a module.
+When importing a module for node.js, `<root>` and `<path>` are concatenated
+to form a full path name.
 
-`module_names:` and `options:` are optional arguments to `Jscall.config`.
+`<path>` must not start with `/jscall` or `/cmd`.  They are reserved for
+internal use.
 
-To change the name of the node command,
+### options:
+
+To specify a command line argument passed to node.js,
 
 ```
-Jscall::PipeToJs.node_command = "node.exe"
+Jscall.config(options: '--use-strict')
 ```
 
-The default command name is `"node"`.
+This call specifies that
+`--use-strict` is passed as a command line argument.
+
+### browser: and port:
 
 When running JavaScript code on a web browser,
 
 ```
-Jscall.config(browser: true, options: {port: 10082})
+Jscall.config(browser: true, port: 10082)
 ```
 
-`options:` is an optional argument.
+Passing `true` for `browser:` switches the execution engine to a web browser.
+The default engine is node.js.
+To switch the engine back to node.js, pass `false` for `browser:`.
+Call `Jscall.close` to detach the current execution engine.
+A new enigine with a new configuration will be created.
+
+`port:` specifies the port number of an http server.  It is optional.
 The example above specifies that Ruby receives http requests
 sent to http://localhost:10082 from JavaScript on a web browser.
 
-Passing `false` for `browser:` to `Jscall.config` switches
-the execution engine to node.js.
-Call `Jscall.close` to detach the current execution engine.
-A new enigine with a new configuration will be created.
+
+### Other configurations
+
+To change the name of the node command,
+
+```
+Jscall::PipeToJs.node_command = "node.exe"
+```
+
+The default command name is `"node"`.
 
 To change the command for launching a web browser,
 
@@ -224,6 +379,12 @@ Jscall launches a web browser by the command like the following:
 open http://localhost:10082/jscall/jscall.html
 ```
 
+Jscall generates a verbose error message if its debug level is more than 0.
+
+```
+Jscall.debug = 1
+```
+
 ## Installation
 
 Add this line to your application's Gemfile:

data/lib/jscall/browser.mjs

@@ -2,24 +2,27 @@
 
 export class HttpStream {
     constructor() {
-        this.send_buffer = ['start']
+        this.recv_buffer = []
         this.send_callback = null
+        this.fetching = null
+        this.call_queue = []
+        this.puts('start')
     }
 
     [Symbol.asyncIterator]() {
         const http_stream = this
         return {
             next() {
-                let msg = http_stream.send_buffer.shift()
-                if (msg === undefined)
+                let next_data = http_stream.recv_buffer.shift()
+                if (next_data === undefined)
                     return new Promise((resolve, reject) => {
                         if (http_stream.send_callback === null)
                             http_stream.send_callback = resolve
                         else
                             throw new Error('(fatal) send_callback is not null!')
-                    }).then(() => this.next())
+                    })
                 else
-                    return http_stream.do_fetch(msg)
+                    return next_data
             }
         }
     }
@@ -40,12 +43,30 @@ export class HttpStream {
     }
 
     puts(msg) {
-        this.send_buffer.push(msg)
-        if (this.send_callback !== null) {
-            const callback = this.send_callback
-            this.send_callback = null
-            callback()
-        }
+        if (this.fetching === null)
+            this.fetching = this.puts0(msg)
+        else
+            this.call_queue.push(msg)
+    }
+
+    puts0(msg) {
+        return this.do_fetch(msg)
+                .then((data) => {
+                    if (this.send_callback === null)
+                        this.recv_buffer.push(data)
+                    else {
+                        const callback = this.send_callback
+                        this.send_callback = null
+                        callback(data)
+                    }
+
+                    if (this.call_queue.length > 0) {
+                        const msg2 = this.call_queue.shift()
+                        this.fetching = this.puts0(msg2)
+                    }
+                    else
+                        this.fetching = null
+                })
     }
 
     setEncoding(encoding) {}
@@ -64,4 +85,8 @@ export const Jscall = new class {
         link.href = file_name
         document.head.append(link)
     }
+
+    append_to_body(html_source) {
+        document.body.insertAdjacentHTML('beforeend', html_source)
+    }
 }

data/lib/jscall/browser.rb

@@ -8,6 +8,7 @@ module Jscall
     WEBrick::HTTPUtils::DefaultMimeTypes['mjs'] ||= "application/javascript"
 
     class Dom
+        # see Jscall class in browser.mjs
         def method_missing(name, *args)
             Jscall.__getpipe__.funcall(nil, "Jscall.#{name}", args)
         end
@@ -23,13 +24,27 @@ module Jscall
     end
 
     class PipeToBrowser < PipeToJs
-        def startJS(module_names, options)
-            port = 10081
-            port = options[:port] if options.is_a?(Hash) && options.has_key?(:port)
-            @pipe = FetchServer.new(port)
+        def startJS(module_names, config)
+            urls = {}
+            module_names.each_index do |i|
+                mod = module_names[i]
+                urls[mod[2]] = mod
+            end
+            port = config[:port] || 10081
+            @pipe = FetchServer.new(port, urls)
             @pipe.open
         end
 
+        def setup(config)
+            if config[:browser]
+                module_names = config[:module_names] || []
+                module_names.each_index do |i|
+                    mod = module_names[i]
+                    Jscall.dyn_import(mod[2], mod[0])
+                end
+            end
+        end
+
         def close
             @pipe.shutdown
             sleep(0.5)
@@ -58,7 +73,8 @@ module Jscall
             @@run_cmd = name
         end
 
-        def initialize(port)
+        def initialize(port, urls)
+            @url_map = urls
             @send_buffer = Thread::Queue.new
             @receive_buffer = Thread::Queue.new
             @server_running = false
@@ -96,10 +112,16 @@ module Jscall
         end
 
         def read_file(req, res)
-            if req.path.start_with?('/jscall/')
+            path = req.path
+            if path.start_with?('/jscall/')
                 root = "#{__dir__}/../"
             else
-                root = @server.config[:DocumentRoot]
+                value = @url_map[path]
+                if value.nil?
+                    root = @server.config[:DocumentRoot]
+                else
+                    root = value[1]
+                end
             end
             WEBrick::HTTPServlet::FileHandler.new(@server, root).service(req, res)
         end

data/lib/jscall/main.mjs

@@ -3,10 +3,16 @@
 const cmd_eval = 1
 const cmd_call = 2
 const cmd_reply = 3
+const cmd_async_call = 4
+const cmd_async_eval = 5
+const cmd_retry = 6
+const cmd_reject = 7
+
 const param_array = 0
 const param_object = 1
 const param_local_object = 2
 const param_error = 3
+const param_hash = 4
 
 const table_size = 100
 
@@ -64,8 +70,9 @@ const exported = new class {
     }
 }
 
-const RemoteRef = class {
+class RemoteRef extends Function {
     constructor(id) {
+        super()
         this.id = id
     }
 }
@@ -74,12 +81,20 @@ const remoteRefHandler = new class {
     get(obj, name) {
         if (name === '__self__')
             return obj
+        else if (name === 'then')
+            // to prevent the Promise from handling RemoteRefs as thenable
+            // e.g., `Jscall.exec("{ call: (x) => Promise.resolve(x) }").call(obj)' should return that obj itself
+            return undefined
         else
             return (...args) => {
                 // this returns Promise
                 return funcall_to_ruby(obj.id, name, args)
             }
     }
+    apply(obj, self, args) {
+        // this returns Promise
+        return funcall_to_ruby(obj.id, 'call', args)
+    }
 }
 
 const imported = new class {
@@ -94,7 +109,8 @@ const imported = new class {
         if (obj !== null && obj !== undefined)
             return obj
         else {
-            const rref = new Proxy(new RemoteRef(index), remoteRefHandler)
+            const ref = new RemoteRef(index)
+            const rref = new Proxy(ref, remoteRefHandler)
             this.objects[index] = new WeakRef(rref)
             return rref
         }
@@ -104,14 +120,15 @@ const imported = new class {
 
     // collect reclaimed RemoteRef objects.
     dead_references() {
-        if (this.canary === null || this.canary.deref() === undefined)
+        // In Safari, deref() may return null
+        if (this.canary === null || this.canary.deref() == undefined)
             this.canary = new WeakRef(new RemoteRef(-1))
         else
             return []   // the canary is alive, so no GC has happened yet.
 
         const deads = []
         this.objects.forEach((wref, index) => {
-            if (wref !== null && wref !== undefined && wref.deref() === undefined) {
+            if (wref !== null && wref !== undefined && wref.deref() == undefined) {
                 this.objects[index] = null
                 deads.push(index)
             }
@@ -127,15 +144,21 @@ const encode_obj = obj => {
         return null
     else if (obj.constructor === Array)
         return [param_array, obj.map(e => encode_obj(e))]
+    else if (obj instanceof Map) {
+        const hash = {}
+        for (const [key, value] of obj.entries())
+            hash[key] = value
+        return [param_hash, hash]
+    }
     else if (obj instanceof RemoteRef)
         return [param_local_object, exported.export_remoteref(obj)]
     else
         return [param_object, exported.export(obj)]
 }
 
-const encode_error = msg => [param_error, msg]
+const encode_error = msg => [param_error, msg.toString()]
 
-const RubyError = class {
+class RubyError {
     constructor(msg) { this.message = msg }
     get() { return 'RubyError: ' + this.message }
 }
@@ -146,6 +169,12 @@ const decode_obj = obj => {
     else if (obj.constructor === Array && obj.length == 2)
         if (obj[0] == param_array)
             return obj[1].map(e => decode_obj(e))
+        else if (obj[0] == param_hash) {
+            const hash = {}
+            for (const [key, value] of Object.entries(obj[1]))
+                hash[key] = decode_obj(value)
+            return hash
+        }
         else if (obj[0] == param_object)
             return imported.import(obj[1])
         else if (obj[0] == param_local_object)
@@ -159,9 +188,9 @@ const decode_obj = obj => {
 const js_eval = eval
 
 const funcall_from_ruby = cmd => {
-    const receiver = decode_obj(cmd[1])
-    const name = cmd[2]
-    const args = cmd[3].map(e => decode_obj(e))
+    const receiver = decode_obj(cmd[2])
+    const name = cmd[3]
+    const args = cmd[4].map(e => decode_obj(e))
     if (name.endsWith('=')) {
         const name2 = name.substring(0, name.length - 1)
         if (receiver === null)
@@ -189,34 +218,41 @@ const funcall_from_ruby = cmd => {
                 return f    // obtain a propety
     }
 
-    throw `unknown function/method was called, ${name}`
+    throw `unknown JS function/method was called: ${name} on <${receiver}>`
 }
 
 let stdout_puts = console.log
+let num_generated_ids = 0
 
-const reply = value => {
-    if (value instanceof Promise)
-        value.then(result => { reply(result) })
-             .catch(err => reply_error(err))
-    else {
-        if (last_callback_stack_depth < callback_stack.length)
-            throw 'Ruby code was called without await.  Call Jscall.close for recovery'
+const fresh_id = () => {
+    num_generated_ids += 1
+    return num_generated_ids
+}
 
+const reply = (message_id, value, sync_mode) => {
+    if (sync_mode && value instanceof Promise)
+        value.then(result => { reply(message_id, result, true) })
+             .catch(err => reply_error(message_id, err))
+    else {
         try {
-            const cmd = reply_with_piggyback([cmd_reply, encode_obj(value)])
+            const cmd = reply_with_piggyback([cmd_reply, message_id, encode_obj(value)])
             const data = JSON.stringify(cmd)
             stdout_puts(data)
         } catch (e) {
-            reply_error(e)
+            reply_error(message_id, e)
         }
     }
 }
 
-const reply_error = e => {
-    const cmd = reply_with_piggyback([cmd_reply, encode_error(e)])
+const reply_error = (message_id, e) => {
+    const cmd = reply_with_piggyback([cmd_reply, message_id, encode_error(e)])
     stdout_puts(JSON.stringify(cmd))
 }
 
+const puts_retry_cmd = msg_id => {
+    stdout_puts(JSON.stringify([cmd_retry, msg_id, encode_obj(false)]))
+}
+
 export const scavenge_references = () => {
     reply_counter = 200
     imported.kill_canary()
@@ -230,7 +266,7 @@ const reply_with_piggyback = cmd => {
         const dead_refs = imported.dead_references()
         if (dead_refs.length > 0) {
             const cmd2 = cmd.concat()
-            cmd2[4] = dead_refs
+            cmd2[5] = dead_refs
             return cmd2
         }
     }
@@ -239,55 +275,140 @@ const reply_with_piggyback = cmd => {
 }
 
 const callback_stack = []
-let last_callback_stack_depth = 0
 let reply_counter = 0
 
 export const exec = src => {
     return new Promise((resolve, reject) => {
-        const cmd = reply_with_piggyback([cmd_eval, src])
-        callback_stack.push([resolve, reject])
+        const message_id = fresh_id()
+        const cmd = reply_with_piggyback([cmd_eval, message_id, src])
+        callback_stack.push([message_id, resolve, reject])
         stdout_puts(JSON.stringify(cmd))
     })
 }
 
 const funcall_to_ruby = (receiver_id, name, args) => {
     return new Promise((resolve, reject) => {
+        const message_id = fresh_id()
         const receiver = [param_local_object, receiver_id]
         const encoded_args = args.map(e => encode_obj(e))
-        const cmd = reply_with_piggyback([cmd_call, receiver, name, encoded_args])
-        callback_stack.push([resolve, reject])
+        const cmd = reply_with_piggyback([cmd_call, message_id, receiver, name, encoded_args])
+        callback_stack.push([message_id, resolve, reject])
         stdout_puts(JSON.stringify(cmd))
     })
 }
 
 const returned_from_callback = cmd => {
-    const result = decode_obj(cmd[1])
-    const callback = callback_stack.pop()
-    if (result instanceof RubyError)
-        callback[1](result.get())
-    else
-        callback[0](result)
+    const message_id = cmd[1]
+    const result = decode_obj(cmd[2])
+    for (let i = callback_stack.length - 1; i >= 0; i--) {
+        // check the most recent element first since callbacks are
+        // assumed to be synchronously executed
+        if (callback_stack[i][0] === message_id) {
+            const [[_, resolve, reject]] = callback_stack.splice(i, 1)
+            if (result instanceof RubyError)
+                reject(result.get())
+            else
+                resolve(result)
+        }
+    }
 }
 
-class JsonReader {
+export class MessageReader {
+    static HeaderSize = 6
+
     constructor(stream) {
-        this.stream     = stream
-        this.acc        = ""
+        this.stream = stream
+        this.state = "header"
+        this.acc = ""
+        this.bodySize = 0
+    }
+
+    parseHeader(pos) {
+        // skip leading whitespace characters as a countermeasure against leftover "\n"
+        while (pos < this.acc.length && /\s/.test(this.acc[pos]))
+            pos++
+
+        if (this.acc.length >= MessageReader.HeaderSize) {
+            const start = pos
+            pos += MessageReader.HeaderSize
+            return [parseInt(this.acc.slice(start, pos), 16), pos]
+        }
+        else
+            return undefined
+    }
+
+    parseBody(pos) {
+        if (this.acc.length >= this.bodySize) {
+            const start = pos
+            pos += this.bodySize
+            return [this.acc.slice(start, pos), pos]
+        }
+        else
+            return undefined
+    }
+
+    consume(pos) {
+        if (pos > 0)
+            this.acc = this.acc.slice(pos)
     }
 
     async *[Symbol.asyncIterator]() {
-        for await (let data of this.stream) {
+        for await (const data of this.stream) {
             this.acc += data
-            try {
-                yield JSON.parse(this.acc)
-                this.acc = ""
-            } catch {
-                // keep data in this.acc
+            let pos = 0
+            while (true) {
+                const result = this.iteratorBody(pos)
+                if (result[0] === false)
+                    break
+                else if (result[0] !== true)
+                    yield result[0]     // result[0] is a string
+
+                pos = result[1]
+                if (this.checkEmptiness(pos))
+                    break
+            }
+        }
+        this.checkEOS()
+    }
+
+    iteratorBody(pos) {
+        if (this.state === "header") {
+            const header = this.parseHeader(pos)
+            if (header === undefined) {
+                this.consume(pos)
+                return [false, pos]
+            } else {
+                this.bodySize = header[0]
+                pos = header[1]
+                this.state = "body"
+            }
+        }
+        if (this.state === "body") {
+            const body = this.parseBody(pos)
+            if (body === undefined) {
+                this.consume(pos)
+                return [false, pos]
+            } else {
+                this.state = "header"
+                return body
             }
         }
-        if (this.acc != "") {
-            yield JSON.parse(this.acc)
+        return [true, pos]
+    }
+
+    checkEmptiness(pos) {
+        if (pos == this.acc.length || (pos == this.acc.length - 1
+                                       && this.acc[this.acc.length - 1] === "\n")) {
+            this.acc = ""
+            return true
         }
+        else
+            return false
+    }
+
+    checkEOS() {
+        if (this.acc.length > 0)
+            throw new Error("The pipe closed after receiving an incomplete message")
     }
 }
 
@@ -298,30 +419,41 @@ export const start = async (stdin, use_stdout) => {
         stdout_puts = (m) => stdin.puts(m)  // on browser
 
     stdin.setEncoding('utf8')
-    for await (const cmd of new JsonReader(stdin)) {
+    for await (const json_data of new MessageReader(stdin)) {
+        let cmd
         try {
-            last_callback_stack_depth = callback_stack.length
+            cmd = JSON.parse(json_data)
 
             // scavenge remote references
-            if (cmd.length > 4)
-                cmd[4].forEach(i => exported.remove(i))
+            if (cmd.length > 5)
+                cmd[5].forEach(i => exported.remove(i))
 
             if (cmd[0] == cmd_eval) {
-                const result = js_eval(cmd[1])
-                reply(result)
+                const result = js_eval(cmd[2])
+                reply(cmd[1], result, true)
             }
             else if (cmd[0] == cmd_call) {
                 const result = funcall_from_ruby(cmd)
-                reply(result)
+                reply(cmd[1], result, true)
             }
             else if (cmd[0] == cmd_reply)
                 returned_from_callback(cmd)
-            else
-                reply_error('invalid command')
+            else if (cmd[0] == cmd_async_call) {
+                const result = funcall_from_ruby(cmd)
+                reply(cmd[1], result, false)
+            }
+            else if (cmd[0] == cmd_async_eval) {
+                const result = js_eval(cmd[2])
+                reply(cmd[1], result, false)
+            }
+            else if (cmd[0] == cmd_reject)
+                puts_retry_cmd(cmd[1])
+            else // cmd_retry and other unknown commands
+                reply_error(cmd[1], `invalid command ${cmd[0]}`)
         } catch (error) {
             const msg = typeof error === 'string' ? error : error.toString() +
                                                             '\n  ---\n' + error.stack
-            reply_error(msg)
+            reply_error(cmd[1], msg)
         }
     }
 }

data/lib/jscall/version.rb

@@ -3,5 +3,5 @@
 # Copyright (C) 2022- Shigeru Chiba.  All rights reserved.
 
 module Jscall
-  VERSION = "1.0.1"
+  VERSION = "1.1.0"
 end

data/lib/jscall.rb

@@ -9,6 +9,16 @@ require_relative "jscall/version"
 require_relative "jscall/browser"
 
 module Jscall
+    @@debug = 0
+
+    # Current debug level (>= 0)
+    def self.debug() @@debug end
+
+    # Sets the current debug level.
+    def self.debug=(level)
+        @@debug = level
+    end
+
     TableSize = 100
 
     class JavaScriptError < RuntimeError
@@ -77,17 +87,50 @@ module Jscall
     end
 
     class RemoteRef
-        attr_reader :id
-
         def initialize(id)
             @id = id
         end
 
+        def __get_id
+            @id
+        end
+
+        def async
+            AsyncRemoteRef.new(@id)
+        end
+
+        # override Object#then
+        def then(*args)
+            send('then', *args)
+        end
+
+        # puts() calls this
+        def to_ary
+            ["#<RemoteRef @id=#{@id}>"]
+        end
+
+        # override Object#send
+        def send(name, *args)
+            Jscall.__getpipe__.funcall(self, name, args)
+        end
+
+        def async_send(name, *args)
+            Jscall.__getpipe__.async_funcall(self, name, args)
+        end
+
         def method_missing(name, *args)
             Jscall.__getpipe__.funcall(self, name, args)
         end
     end
 
+    class AsyncRemoteRef < RemoteRef
+        alias send async_send
+
+        def method_missing(name, *args)
+            Jscall.__getpipe__.async_funcall(self, name, args)
+        end
+    end
+
     class Imported      # outbound references from Ruby to Node
         attr_reader :objects
 
@@ -137,10 +180,19 @@ module Jscall
         CMD_EVAL = 1
         CMD_CALL = 2
         CMD_REPLY = 3
+        CMD_ASYNC_CALL = 4
+        CMD_ASYNC_EVAL = 5
+        CMD_RETRY = 6
+        CMD_REJECT = 7
+
         Param_array = 0
         Param_object = 1
         Param_local_object = 2
         Param_error = 3
+        Param_hash = 4
+
+        Header_size = 6
+        Header_format = '%%0%dx' % Header_size
 
         @@node_cmd = 'node'
 
@@ -148,21 +200,35 @@ module Jscall
             @@node_cmd = cmd
         end
 
-        # do import * as 'module_names[i][0]' from 'module_names[i][1]' 
-        #
-        def initialize(module_names=[], options='')
-            startJS(module_names, options)
+        def initialize(config)
             @exported = Exported.new
             @imported = Imported.new
             @send_counter = 0
+            @num_generated_ids = 0
+            @pending_replies = {}
+            module_names = config[:module_names] || []
+            startJS(module_names, config)
         end
 
-        # module_names: an array of [module_name, module_file_name]
+        def setup(config)
+            # called just after executing new PipeToJs(config)
+        end
+
+        # Config options.
+        #
+        # module_names: an array of [module_name, module_root, module_file_name]
+        #   For example,
+        #     [['Foo', '/home/jscall', '/lib/foo.mjs']]
+        #   this does
+        #     import * as Foo from "/home/jscall/lib/foo.mjs"
+        #
+        # options: options passed to node.js
         #
-        def startJS(module_names, options)
+        def startJS(module_names, config)
+            options = config[:options] || ''
             script2 = ''
             module_names.each_index do |i|
-                script2 += "import * as m#{i + 2} from \"#{module_names[i][1]}\"; globalThis.#{module_names[i][0]} = m#{i + 2}; "
+                script2 += "import * as m#{i + 2} from \"#{module_names[i][1]}#{module_names[i][2]}\"; globalThis.#{module_names[i][0]} = m#{i + 2}; "
             end
             script2 += "import { createRequire } from \"node:module\"; globalThis.require = createRequire(\"file://#{Dir.pwd}/\");"
             script = "'import * as m1 from \"#{__dir__}/jscall/main.mjs\"; globalThis.Ruby = m1; #{script2}; Ruby.start(process.stdin, true)'"
@@ -184,8 +250,12 @@ module Jscall
                 obj
             elsif obj.is_a?(Array)
                 [Param_array, obj.map {|e| encode_obj(e)}]
+            elsif obj.is_a?(Hash)
+                hash2 = {}
+                obj.each {|key, value| hash2[key] = encode_obj(value) }
+                [Param_hash, hash2]
             elsif obj.is_a?(RemoteRef)
-                [Param_local_object, obj.id]
+                [Param_local_object, obj.__get_id]
             else
                 [Param_object, @exported.export(obj)]
             end
@@ -199,11 +269,15 @@ module Jscall
             if obj.is_a?(Numeric) || obj.is_a?(String) || obj == true || obj == false || obj == nil
                 obj
             elsif obj.is_a?(Array) && obj.size == 2
-                if (obj[0] == Param_array)
+                if obj[0] == Param_array
                     obj[1].map {|e| decode_obj(e)}
-                elsif (obj[0] == Param_object)
+                elsif obj[0] == Param_hash
+                    hash = {}
+                    obj[1].each {|key, value| hash[key] = decode_obj(value)}
+                    hash
+                elsif obj[0] == Param_object
                     @imported.import(obj[1])
-                elsif (obj[0] == Param_local_object)
+                elsif obj[0] == Param_local_object
                     @exported.find(obj[1])
                 else  # if Param_error
                     JavaScriptError.new(obj[1])
@@ -213,20 +287,38 @@ module Jscall
             end
         end
 
+        def fresh_id
+            @num_generated_ids += 1
+        end
+
         def funcall(receiver, name, args)
-            cmd = [CMD_CALL, encode_obj(receiver), name, args.map {|e| encode_obj(e)}]
+            cmd = [CMD_CALL, nil, encode_obj(receiver), name, args.map {|e| encode_obj(e)}]
+            send_command(cmd)
+        end
+
+        def async_funcall(receiver, name, args)
+            cmd = [CMD_ASYNC_CALL, nil, encode_obj(receiver), name, args.map {|e| encode_obj(e)}]
             send_command(cmd)
         end
 
         def exec(src)
-            cmd = [CMD_EVAL, src]
+            cmd = [CMD_EVAL, nil, src]
+            send_command(cmd)
+        end
+
+        def async_exec(src)
+            cmd = [CMD_ASYNC_EVAL, nil, src]
             send_command(cmd)
         end
 
         def encode_eval_error(e)
             traces = e.backtrace
             location = if traces.size > 0 then traces[0] else '' end
-            encode_error(location + ' ' + e.to_s)
+            if Jscall.debug > 0
+                encode_error("\n#{e.full_message}")
+            else
+                encode_error(location + ' ' + e.to_s)
+            end
         end
 
         def scavenge
@@ -243,7 +335,7 @@ module Jscall
                 dead_refs = @imported.dead_references()
                 if (dead_refs.length > 0)
                     cmd2 = cmd.dup
-                    cmd2[4] = dead_refs
+                    cmd2[5] = dead_refs
                     return cmd2
                 end
             end
@@ -251,55 +343,103 @@ module Jscall
         end
 
         def send_command(cmd)
+            message_id = (cmd[1] ||= fresh_id)
             json_data = JSON.generate(send_with_piggyback(cmd))
-            @pipe.puts(json_data)
-            reply_data = @pipe.gets
-            reply = JSON.parse(reply_data || '[]')
-            if reply.length > 4
-                reply[4].each {|idx| @exported.remove(idx) }
+            header = (Header_format % json_data.length)
+            if header.length != Header_size
+                raise "message length limit exceeded"
             end
-            if @pipe.closed?
-                raise RuntimeError.new("connection closed: #{reply}")
-            elsif reply[0] == CMD_REPLY
-                result = decode_obj(reply[1])
-                if result.is_a?(JavaScriptError)
-                    raise result
-                else
-                    return result
-                end
-            elsif reply[0] == CMD_EVAL
-                begin
-                    result = Object::TOPLEVEL_BINDING.eval(reply[1])
-                    encoded = encode_obj(result)
-                rescue => e
-                    encoded = encode_eval_error(e)
+            json_data_with_header = header + json_data
+            @pipe.puts(json_data_with_header)
+
+            while true
+                reply_data = @pipe.gets
+                reply = JSON.parse(reply_data || '[]')
+                if reply.length > 5
+                    reply[5].each {|idx| @exported.remove(idx) }
+                    reply[5] = nil
                 end
-                send_command([CMD_REPLY, encoded])
-            elsif reply[0] == CMD_CALL
-                begin
-                    receiver = decode_obj(reply[1])
-                    name = reply[2]
-                    args = reply[3].map {|e| decode_obj(e)}
-                    result = receiver.public_send(name, *args)
-                    encoded = encode_obj(result)
-                rescue => e
-                    encoded = encode_eval_error(e)
+                if @pipe.closed?
+                    raise RuntimeError.new("connection closed: #{reply}")
+                elsif reply[0] == CMD_REPLY
+                    result = decode_obj(reply[2])
+                    if reply[1] != message_id
+                        @pending_replies[reply[1]] = result
+                        send_reply(reply[1], nil, false, CMD_REJECT)
+                    elsif result.is_a?(JavaScriptError)
+                        raise result
+                    else
+                        return result
+                    end
+                elsif reply[0] == CMD_EVAL
+                    begin
+                        result = Object::TOPLEVEL_BINDING.eval(reply[2])
+                        send_reply(reply[1], result)
+                    rescue => e
+                        send_error(reply[1], e)
+                    end
+                elsif reply[0] == CMD_CALL
+                    begin
+                        receiver = decode_obj(reply[2])
+                        name = reply[3]
+                        args = reply[4].map {|e| decode_obj(e)}
+                        result = receiver.public_send(name, *args)
+                        send_reply(reply[1], result)
+                    rescue => e
+                        send_error(reply[1], e)
+                    end
+                elsif reply[0] == CMD_RETRY
+                    if reply[1] != message_id
+                        send_reply(reply[1], nil, false, CMD_REJECT)
+                    else
+                        result = @pending_replies.delete(message_id)
+                        if result.nil?
+                            raise RuntimeError.new("bad CMD_RETRY: #{reply}")
+                        elsif result.is_a?(JavaScriptError)
+                            raise result
+                        else
+                            return result
+                        end
+                    end
+                else
+                    # CMD_REJECT and other unknown commands
+                    raise RuntimeError.new("bad message: #{reply}")
                 end
-                send_command([CMD_REPLY, encoded])
+            end
+        end
+
+        def send_reply(message_id, value, erroneous = false, cmd_id=CMD_REPLY)
+            if erroneous
+                encoded = encode_eval_error(value)
             else
-                raise RuntimeError.new("bad reply: #{reply}")
+                encoded = encode_obj(value)
+            end
+            json_data = JSON.generate(send_with_piggyback([cmd_id, message_id, encoded]))
+            header = (Header_format % json_data.length)
+            if header.length != Header_size
+                raise "message length limit exceeded"
             end
+            json_data_with_header = header + json_data
+            @pipe.puts(json_data_with_header)
+        end
+
+        def send_error(message_id, e)
+            send_reply(message_id, e, true)
         end
     end
 
     @pipe = nil
-    @module_names = []
-    @options = ''
+    @configurations = {}
     @pipeToJsClass = PipeToJs
 
-    def self.config(module_names: [], options: '', browser: false)
-        @module_names = module_names
-        @options = options
+    #def self.config(module_names: [], options: '', browser: false)
+    def self.config(**kw)
+        if kw.nil? || kw == {}
+            @configurations = {}
+        else
+            @configurations = @configurations.merge!(kw)
+        end
+        browser = @configurations[:browser]
         @pipeToJsClass = if browser then PipeToBrowser else PipeToJs end
         nil
     end
@@ -310,35 +450,65 @@ module Jscall
 
     Signal.trap(0) { self.close }  # close before termination
 
-    def self.exec(src)
-        __getpipe__.exec(src)
-    end
-
-    def self.dyn_import(name, var_name=nil)
-        __getpipe__.funcall(nil, 'Ruby.dyn_import', [name, var_name])
-    end
-
-    # name is a string object.
-    # Evaluating this string in JavaScript results in a JavaScript function.
-    #
-    def self.funcall(name, *args)
-        __getpipe__.funcall(nil, name, args)
-    end
-
     # reclaim unused remote references.
     #
     def self.scavenge_references
         __getpipe__.scavenge
     end
 
-    def self.method_missing(name, *args)
-        __getpipe__.funcall(nil, name, args)
-    end
-
     def self.__getpipe__
         if @pipe.nil?
-            @pipe = @pipeToJsClass.new(@module_names, @options)
+            @pipe = @pipeToJsClass.new(@configurations)
+            @pipe.setup(@configurations)
         end
         @pipe
     end
+
+    module Interface
+        def exec(src)
+            __getpipe__.exec(src)
+        end
+
+        def async_exec(src)
+            __getpipe__.async_exec(src)
+        end
+
+        # name is a string object.
+        # Evaluating this string in JavaScript results in a JavaScript function.
+        #
+        def funcall(name, *args)
+            __getpipe__.funcall(nil, name, args)
+        end
+
+        def async_funcall(name, *args)
+            __getpipe__.async_funcall(nil, name, args)
+        end
+
+        def dyn_import(name, var_name=nil)
+            funcall('Ruby.dyn_import', name, var_name)
+        end
+
+        def method_missing(name, *args)
+            funcall(name, *args)
+        end
+    end
+
+    extend Interface
+
+    module AsyncInterface
+        include Interface
+
+        alias exec async_exec
+        alias funcall async_funcall
+    end
+
+    def self.async
+        @async ||= Class.new do
+            def __getpipe__
+                Jscall.__getpipe__
+            end
+
+            include AsyncInterface
+        end.new
+    end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: jscall
 version: !ruby/object:Gem::Version
-  version: 1.0.1
+  version: 1.1.0
 platform: ruby
 authors:
 - Shigeru Chiba
-autorequire: 
+autorequire:
 bindir: exe
 cert_chain: []
-date: 2022-07-05 00:00:00.000000000 Z
+date: 2022-08-11 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: webrick
@@ -50,7 +50,7 @@ licenses:
 metadata:
   homepage_uri: https://github.com/csg-tokyo/jscall
   source_code_uri: https://github.com/csg-tokyo/jscall
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -65,8 +65,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.1.6
-signing_key: 
+rubygems_version: 3.3.3
+signing_key:
 specification_version: 4
 summary: a library for calling JavaScript functions
 test_files: []