@ai-ecoverse/wasix-ruby 3.4.11-2 → 3.4.11-21

package/PRESTAGE.md

@@ -1,32 +1,69 @@
-# wasix-ruby 3.4.11-2
+# wasix-ruby 3.4.11-21
 
-Built against wasix-sysroot **2025.9.30-14**. **No epoll**: SLICC returns ENOSYS (52)
+Built against wasix-sysroot **2025.9.30-39**. **No epoll**: SLICC returns ENOSYS (52)
 for `epoll_create`; MRI is compiled with `USE_MN_THREADS=0` and
 `HAVE_SYS_EPOLL_H=0` (also kqueue/eventfd/timerfd off) so it uses poll()/timer-thread.
 
-Wasmer implements epoll, so a wasmer PRESTAGE of `ruby -v` is **not** sufficient.
-Host gate: `ruby.wasm` must **not** contain the `epoll_create (errno:%d)` rb_bug string.
+Configure `--prefix=/nonexistent-ruby-prefix` so baked-in `$LOAD_PATH` never matches a real
+VFS path. The manifest's `WASIX_RUBY_PREFIX=${package}` puts the package's stdlib on the initial load path.
+
+## Asyncify (buffer sizing — priority over process/fork)
+
+- **POSTLINK** (from configure `wasi*`): `$(WASMOPT) --asyncify $(wasmoptflags)`
+  — **no** `asyncify-ignore-imports`, **no** restricted import list.
+- Spill buffers `WASM_{SETJMP,FIBER,SCAN}_STACK_BUFFER_SIZE=65536`.
 
 ## OpenSSL
-linked against /tmp/wasix-openssl-prefix (OpenSSL 3.x; honours SSL_CERT_FILE). Socket C ext omitted this cut (WASIX lacks CMSG_*); net/http HTTPS and remote gem install need Socket — local .gem files only until then.
+linked against @ai-ecoverse/wasix-openssl 3.5.9-4 (OpenSSL 3.5.9; honours SSL_CERT_FILE). Socket C ext built with HAVE_STRUCT_MSGHDR_MSG_CONTROL / CMSG_* forced off (WASIX ancillary data incomplete).
+
+## Relocation / WASIX_RUBY_PREFIX
+- Stdlib under `lib/ruby/<API>` (e.g. **3.4.0**), arch `…/wasm32-wasi/rbconfig.rb`.
+- The initial load path is `$WASIX_RUBY_PREFIX/lib/ruby/<API>` and its `/wasm32-wasi`, with no
+  site or vendor directory (homescoop#597); the manifest sets no `RUBYLIB`, so a caller's goes first.
+- **Certify only** at `/shared/lib/node_modules/@ai-ecoverse/wasix-ruby` with
+  manifest env. Never at `/ruby` (hides a missing load path — -3 false pass).
+
+## Gems / flock / uid (-5)
+- `GEM_HOME=${HOME}/.local/share/gem/ruby/3.4.0` (writable under SLICC HOME).
+- `GEM_PATH=${HOME}/.local/share/gem/ruby/3.4.0:${package}/lib/ruby/gems/3.4.0`.
+- `File#flock`: libc's flock(2) through slicc.flock, per open file description (since 3.4.11-18,
+  wasix-sysroot -34; -17 whole-file fcntl record locks, homescoop#578; up to -16 a no-op success).
+- `Process.uid`/`euid`/`gid`/`groups`, `Etc.getpwuid` and `Dir.home` come from
+  slicc-kernel's process credentials (wasix-sysroot -20, slicc-kernel >= 1.44.0):
+  root is uid 0 with home /root; a `kernel.users.add` user gets its own ids.
+
+## Gems / PATH / SLICC #3735
+- `PATH=${package}/bin:${PATH}`.
+- `gem`/`bundle`/`rake` `args: ["${package}/bin/…"]` need **SLICC PR #3735**.
+  Until merged, those commands only work in harnesses that have it.
 
 ## PRESTAGE (host)
-- `ruby.wasm` lacks `epoll_create (errno:` (this build fails otherwise)
-- `date_core` + `openssl` linked (Init_* strings present)
-- F_SETFD: `fcntl(1,F_SETFD,0)` then `F_GETFD==0` against the same sysroot
+- no `epoll_create (errno:` in wasm; date_core linked; `nt_start_wasm_trampoline`
+- package.json `WASIX_RUBY_PREFIX` is `${package}`, and it sets no `RUBYLIB`
+- package.json `GEM_HOME` is under `${HOME}/.local/share/gem`, not `${package}`
 
-## PRESTAGE (SLICC — run this; wasmer hides the epoll bug)
+## PRESTAGE (SLICC — real install path)
 ```ruby
 RUBY_VERSION
 system("echo", "x")
-`echo x`
-IO.popen("echo x", &:read)
+IO.popen(["echo", "x"], &:read)
 Process.spawn("echo", "x").then { Process.wait _1 }
 require "json"; JSON.parse('{"a":1}')
-require "zlib"
-require "yaml"
-require "date"; Date.today
+require "zlib"; require "yaml"; require "date"; Date.today
 Fiber.new { :ok }.resume
-File.write("/tmp/r.txt", "ok"); File.read("/tmp/r.txt")
+$stdout.sync = true
+t = Thread.new { puts "CHILD"; 42 }; puts t.value
+require "openssl"; OpenSSL::OPENSSL_VERSION
+require "socket"
+Process.uid # => 0 (root, the kernel's default user)
+f = File.open("flock-test", "w"); f.flock(File::LOCK_EX); f.close # no Errno::EINVAL; a second process gets false for LOCK_EX|LOCK_NB
 ```
-Plus: `ruby -v`, `gem --version`, `bundle --version`, `bundle exec rake` on a tiny Rakefile.
+Plus Queue/CV/Mutex; `bundle exec rake` when #3735 is available.
+
+## Install acceptance (-5) — certify at real VFS path only
+At `/shared/lib/node_modules/@ai-ecoverse/wasix-ruby` with manifest env:
+1. `gem env home` → `…/.local/share/gem/ruby/3.4.0` (not under package).
+2. `bundle install --local` creates lockfile without `Errno::EINVAL @ rb_file_flock`.
+3. `gem install --local --force` from packaged `rake-13.2.1.gem`; spec under
+   `HOME/.local/share/gem/ruby/3.4.0`.
+Hold npm publish until those three pass.

package/README.md

@@ -1,3 +1,167 @@
 # `@ai-ecoverse/wasix-ruby`
 
 MRI Ruby for slicc WASIX. No epoll (poll fallback). See PRESTAGE.md.
+
+Built on wasix-sysroot 2025.9.30-39 (the WASIX libc and headers; see
+PRESTAGE.md).
+
+## The interpreter path (3.4.11-12)
+
+`Gem.ruby` and `RbConfig.ruby` are the first `ruby` on PATH outside the
+package, which is slicc-kernel's `/usr/bin/ruby` command. Run through it,
+ruby gets the command's `RUBYLIB`, `GEM_HOME` and `GEM_PATH` defaults, so
+`system(Gem.ruby, "-e", "exit 3")` exits 3 from any process, and RubyGems
+writes `#!/usr/bin/ruby` into the gem executables it installs. Without a
+`ruby` on PATH they fall back to the package's `bin/ruby`.
+
+`bin/ruby` (`RbConfig::CONFIG["bindir"]`) is the wasm module itself, so the
+kernel can exec it by path and use it in a shebang. It doesn't get the
+command's environment: started from ruby (which passes its own `RUBYLIB` and
+`GEM_*` on) it works fully, but started from a shell without them ruby can't
+find RubyGems (`RubyGems were not loaded`). Use `ruby` or `/usr/bin/ruby` from a
+shell. The package's `gem`, `bundle`, `rake`, `irb` and `erb` scripts start
+with `#!/usr/bin/env ruby`, so they also run by path.
+
+Up to 3.4.11-11 `bin/ruby` was a 7-byte `#!wasm` marker that the kernel
+can't exec: `system(Gem.ruby, …)` gave 127, and the executables RubyGems
+installed (shebang `#!…/wasix-ruby/bin/ruby`) gave "bad interpreter". Those
+executables now run when started from ruby; to make them run from a shell
+too, rewrite their shebangs with `gem pristine --only-executables <gem>…`
+for the gems you installed (`--all` would also rewrite the package's own
+gems).
+
+Since 3.4.11-8, single-fd waits (`IO#wait_readable`, blocking `TCPSocket.new`,
+`Net::HTTP`, blocking `SSLSocket#connect`/`#accept`) use poll(): WASIX's select()
+rejects an exception set with ENOSYS, so `IO.select` with a third array still
+fails. `io/console` is built, so `irb` and reline work.
+
+`TZ` is ignored: wasix-libc's localtime is UTC only (homescoop#193).
+
+## 3.4.11-21: bundled gems with C extensions, interruptible locks, a clean load path
+
+- **Bundled gems with a C extension** (homescoop#596): bigdecimal, racc (`racc/cparse`), nkf, syslog, debug and rbs (the versions in `gems/bundled_gems`) are installed, and their extensions are linked into `ruby`. `require "bigdecimal"` works, and so do activesupport and anything else that needs it. `gem list` shows them, and Bundler takes them from the package. A newer version of one of them can't be installed with `gem install`, because that needs a compiler. Up to 3.4.11-20 they were left out and `require "bigdecimal"` was a LoadError.
+- **Interruptible locks** (homescoop#869): `Timeout.timeout`, `Thread#raise` and `Thread#kill` interrupt a blocking `File#flock` or `IO#fcntl(F_SETLKW)`. On slicc the wait is a loop of non-blocking tries with a 20 ms sleep in between, so a lock can come free up to 20 ms before ruby takes it. Up to 3.4.11-20 the interrupt waited until the holder let go.
+- **Load path** (homescoop#597): `$LOAD_PATH` holds only directories that exist. The package's stdlib comes from `WASIX_RUBY_PREFIX` (the command sets it to the package), so the command no longer sets `RUBYLIB`. Yours comes first, and there's no site or vendor directory. Up to 3.4.11-20, eight missing `/nonexistent-ruby-prefix` directories were probed on every require that missed the stdlib.
+- **RbConfig** (homescoop#1114): `RbConfig::CONFIG` names the build tree `/nonexistent-ruby-build` instead of the CI runner's checkout, so the bytes don't depend on where it was built.
+- **Sysroot:** wasix-sysroot 2025.9.30-39, whose `malloc` refreshes a thread's view of grown shared memory (homescoop#1287).
+
+## 3.4.11-20: inputs recorded by version
+
+`package.json`'s `homescoop.openssl` names the wasix-openssl package
+version and the OpenSSL version it links, as THIRD-PARTY-NOTICES.md does;
+up to 3.4.11-19 it was the CI runner's build path (homescoop#637). Same
+sources and inputs as 3.4.11-19.
+
+## 3.4.11-18: real File#flock (wasix-sysroot -34; slicc-kernel ≥ 1.120.0)
+
+- **What changed:** `File#flock` is libc's `flock(2)`, through slicc-kernel's `slicc.flock`.
+- **Semantics, as on Linux:**
+  - a lock belongs to the open file description, on any IO: a file, a pipe, a terminal, a socket or a directory;
+  - a second `File.open` of the same file, or a pipe's other end, conflicts;
+  - a `dup` and a spawned child's inherited descriptor share the lock;
+  - `LOCK_EX` on a read-only file excludes.
+- **Before:** -17 used whole-file fcntl record locks, which belonged to the process, covered regular files only, and on a read-only handle took a read lock instead of `LOCK_EX`.
+- **Kernel:** this needs slicc-kernel ≥ 1.120.0 (engines).
+
+## 3.4.11-17: File#flock excludes other processes (up to -17)
+
+`File#flock` takes whole-file `fcntl` record locks (`LOCK_SH` → read lock,
+`LOCK_EX` → write lock, `LOCK_UN` → unlock; `LOCK_NB` doesn't wait), which
+slicc-kernel (≥ 1.100.0) enforces between processes (homescoop#578). Up to
+-16 every `flock` returned 0 and excluded nobody, so two `bundle install` or
+`gem install` runs on one bundle path or gem directory didn't wait for each
+other. Now a second process's `flock(File::LOCK_EX | File::LOCK_NB)` is
+`false` while another holds the lock, and a blocking one waits.
+
+These are `fcntl` semantics, as wasix-perl's `flock` has them since
+5.42.0-15: a lock belongs to the process, so a second handle on the same
+file in the same process isn't refused, and closing any handle on the file
+drops the process's lock on it. `fcntl` wants a writable handle for a write
+lock, so `LOCK_EX` on a read-only handle takes a read lock: it keeps writers
+out, but two read-only `LOCK_EX` holders are both granted; open a lock file
+for writing to exclude (homescoop#758). `flock` on something that isn't a
+regular file (a pipe, STDOUT, `/dev/null`, a directory) succeeds and excludes
+nobody, since slicc-kernel keeps record locks on regular files only
+(ai-ecoverse/slicc-kernel#757).
+
+A real rebuild, on wasix-sysroot 2025.9.30-32.
+
+## 3.4.11-16: the engines floor, and bundle exec
+
+The `engines` floor (slicc-kernel ≥ 1.100.0, below) is enforced from
+slicc-kernel 1.106.2 on: its command catalog refuses a package whose
+`engines.slicc-kernel` the running kernel doesn't meet
+(ai-ecoverse/slicc-kernel#797). Older kernels install the package without a
+warning, and there `fcntl` record locks aren't real: use slicc-kernel ≥
+1.100.0 (homescoop#651).
+
+From slicc-kernel 1.106.2 the commands' `/usr/bin` entries are real files
+(slicc-kernel#762), so `bundle exec ruby …` no longer warns
+`/usr/bin/ruby is empty` (homescoop#379). Older kernels print that warning,
+and the command still runs.
+
+## 3.4.11-15: needs slicc-kernel ≥ 1.100.0
+
+A packaging-only release of 3.4.11-14 (the same files): `engines` now asks
+for slicc-kernel ≥ 1.100.0, the first kernel with `slicc.fcntl_lock`. On an
+older kernel the libc grants every `IO#fcntl` record lock locally, so two
+processes could both hold a write lock on the same bytes and lose data;
+refusing to install there is the honest answer (homescoop#552). Up to -14
+`engines` said ≥ 1.44.0.
+
+## 3.4.11-14: wasix-sysroot 2025.9.30-30
+
+- POSIX record locks: `require "fcntl"` now defines `Fcntl::F_GETLK`,
+  `F_SETLK`, `F_SETLKW` and `F_RDLCK`/`F_WRLCK`/`F_UNLCK` (up to 3.4.11-13
+  they were missing: `NameError`), and `IO#fcntl` passes them to slicc-kernel
+  (≥ 1.100.0), which keeps the locks per process, as Linux does. A
+  `Process.spawn`ed ruby asking for a write lock on a range another process
+  holds gets `Errno::EAGAIN`, `F_SETLKW` waits for the unlock, and `F_GETLK`
+  names the holder. On an older kernel every lock is granted locally and
+  `F_GETLK` answers `F_UNLCK` (hence -15's `engines` floor). The struct is `[type, whence, start, len,
+  pid].pack("s s x4 q q l x4")` (32 bytes). `File#flock` was unchanged: an
+  advisory no-op that succeeded (real locks since 3.4.11-17).
+- `RUBYLIB` names only the package's library directories that exist,
+  `lib/ruby/3.4.0` and its `wasm32-wasi`. Up to 3.4.11-13 it also named
+  `site_ruby` and `vendor_ruby` (and their `wasm32-wasi`), which npm never
+  packs because they are empty, so every require of the standard library
+  first probed four missing directories. `ruby -rirb -e 1` makes about 16%
+  fewer file-system calls (slicc-kernel#774).
+- wasix-sysroot -29 and -30 also bring `poll` with `events` 0 and named
+  semaphores (no Ruby API uses them).
+
+## 3.4.11-13: wasix-sysroot 2025.9.30-28
+
+A rebuild on wasix-sysroot -28, whose libc keeps real signal masks through
+slicc-kernel (≥ 1.88.0), so MRI's own `pthread_sigmask`/`sigprocmask` calls
+now take effect. Ruby has no API of its own for masks, so
+nothing changes at the Ruby level: `Signal.trap` handlers still run when
+other threads are busy.
+
+## 3.4.11-11: wasix-sysroot 2025.9.30-27
+
+- `File.utime(nil, nil, f)` and `FileUtils.touch(f)` set the times to now;
+  up to 3.4.11-10 they set them to 0 (1970), from wasix-sysroot's broken
+  `utimensat(NULL)` (homescoop#294).
+- `Process.times` reports the process CPU time slicc-kernel keeps; system and
+  children's times are 0.
+- An empty path is ENOENT (`File.exist?("")` is false), as on Linux.
+- Files have owners on slicc-kernel ≥ 1.59.0 with users on: `File.chown`,
+  `File::Stat#uid`/`#gid` and `File.owned?` are the kernel's, and
+  `File.readable?`/`writable?`/`executable?` ask it (slicc-kernel ≥ 1.63.0).
+
+## Gem executables on PATH
+
+Command env puts user gem bins ahead of the package:
+
+```text
+${HOME}/.local/share/gem/ruby/3.4.0/bin:${package}/bin:${PATH}
+```
+
+Those are defaults: a `PATH` the caller already has wins, as it does when you
+start `ruby` from bash. To run gem executables **by name**, add the user gem
+bin directory to your PATH, e.g. in `~/.bashrc`:
+
+```bash
+export PATH="$HOME/.local/share/gem/ruby/3.4.0/bin:$PATH"
+```