microsandbox-rb 0.5.7

data/README.md

@@ -0,0 +1,328 @@
+# microsandbox-rb
+
+Lightweight microVM sandboxes for Ruby — run AI agents and untrusted code with hardware-level isolation.
+
+The `microsandbox-rb` gem provides native bindings to the [microsandbox](https://github.com/superradcompany/microsandbox) runtime via a Rust extension (magnus). It spins up real microVMs (not containers) in under 100 ms, runs standard OCI (Docker) images, and gives you full control over command execution, the guest filesystem, networking, and metrics — all from an idiomatic, **synchronous** Ruby API. There is no daemon to install and no server to connect to: the runtime is embedded directly in your process.
+
+This is an **unofficial, community-maintained** Ruby implementation — not part of the official SDK family ([Rust](https://github.com/superradcompany/microsandbox/tree/main/sdk), TypeScript, Python, Go) — though it wraps the same core engine.
+
+## Features
+
+- **Hardware isolation** — each sandbox is a real VM with its own Linux kernel
+- **Sub-100 ms boot** — no daemon, no server setup, embedded directly in your app
+- **OCI image support** — pull and run images from Docker Hub, GHCR, ECR, or any OCI registry
+- **Command execution** — run commands or shell scripts and collect output
+- **Guest filesystem access** — read, write, list, copy, stat files inside a running sandbox
+- **Metrics & logs** — CPU, memory, disk and network I/O; captured stdout/stderr/system logs
+- **Idiomatic Ruby** — keyword arguments, block-scoped lifecycle, a typed error hierarchy
+- **Thread-friendly** — the GVL is released during sandbox calls, so other Ruby threads keep running
+
+## Requirements
+
+- **Ruby** >= 3.1
+- **Linux** with KVM enabled, or **macOS** on Apple Silicon (M-series)
+- Building from source additionally needs a **Rust** toolchain (stable >= 1.91)
+
+## Installation
+
+The gem is published as **`microsandbox-rb`**, but you still `require "microsandbox"`
+(the `microsandbox` package name was already taken on RubyGems):
+
+```ruby
+# Gemfile
+gem "microsandbox-rb", require: "microsandbox"
+```
+
+```bash
+bundle install
+# or
+gem install microsandbox-rb
+```
+
+The first build downloads the `msb` runtime and `libkrunfw` firmware into
+`~/.microsandbox`. You can (re)provision them explicitly at any time:
+
+```ruby
+Microsandbox.install unless Microsandbox.installed?
+```
+
+## Quick start
+
+```ruby
+require "microsandbox"
+
+Microsandbox::Sandbox.create("hello", image: "python") do |sb|
+  output = sb.exec("python", ["-c", "print('Hello, World!')"])
+  puts output.stdout      # => "Hello, World!\n"
+  puts output.success?    # => true
+end
+# the sandbox is stopped automatically when the block returns
+```
+
+## Usage
+
+### Lifecycle
+
+```ruby
+# Block form — recommended; stops the sandbox automatically (even on error)
+Microsandbox::Sandbox.create("box", image: "alpine") do |sb|
+  # ...
+end
+
+# Manual form — you are responsible for stopping it
+sb = Microsandbox::Sandbox.create("box", image: "alpine")
+begin
+  # ...
+ensure
+  sb.stop          # graceful (sb.stop(timeout: 5) to bound the wait)
+  # sb.kill        # force (SIGKILL)
+end
+
+# Inspect / manage existing sandboxes
+Microsandbox::Sandbox.list            # => [Microsandbox::SandboxInfo, ...]
+Microsandbox::Sandbox.get("box")      # => Microsandbox::SandboxInfo
+Microsandbox::Sandbox.start("box")    # restart a stopped sandbox
+Microsandbox::Sandbox.remove("box")   # remove a stopped sandbox
+```
+
+### Configuration
+
+```ruby
+Microsandbox::Sandbox.create(
+  "configured",
+  image:    "python",
+  cpus:     2,
+  memory:   1024,                      # MiB
+  env:      { "API_BASE" => "https://example.com" },
+  workdir:  "/app",
+  labels:   { "team" => "research" },
+  ports:    { 8080 => 80 },            # host => guest (TCP)
+  network:  "public_only",             # or "none" for airgapped
+  replace:  true                       # replace an existing sandbox of the same name
+) do |sb|
+  # ...
+end
+```
+
+### Executing commands
+
+```ruby
+Microsandbox::Sandbox.create("exec-demo", image: "alpine") do |sb|
+  # Direct command (no shell)
+  out = sb.exec("ls", ["-la", "/etc"], cwd: "/", timeout: 30)
+  out.exit_code   # => 0
+  out.success?    # => true
+  out.stdout      # => "..." (UTF-8)
+  out.stderr_bytes # => raw ASCII-8BIT bytes
+
+  # Shell script (pipes, redirects, &&)
+  sb.shell("cat /etc/os-release | grep VERSION").stdout
+
+  # Environment, stdin, working directory
+  sb.exec("cat", [], stdin: "piped data")
+  sb.exec("sh", ["-c", "echo $GREETING"], env: { "GREETING" => "hi" })
+end
+```
+
+A non-zero exit is **not** an error — inspect `exit_code`/`success?`. Spawn-time
+failures (e.g. command not found) and timeouts raise typed errors (see below).
+
+### Guest filesystem
+
+```ruby
+Microsandbox::Sandbox.create("fs-demo", image: "alpine") do |sb|
+  sb.fs.write("/tmp/data.txt", "hello")
+  sb.fs.read_text("/tmp/data.txt")     # => "hello"  (UTF-8)
+  sb.fs.read("/tmp/data.txt")          # => raw bytes (ASCII-8BIT)
+  sb.fs.exists?("/tmp/data.txt")       # => true
+
+  sb.fs.mkdir("/tmp/sub")
+  sb.fs.copy("/tmp/data.txt", "/tmp/sub/copy.txt")
+  sb.fs.rename("/tmp/sub/copy.txt", "/tmp/sub/renamed.txt")
+  sb.fs.list("/tmp/sub")               # => [Microsandbox::FsEntry, ...]
+  sb.fs.stat("/tmp/data.txt")          # => Microsandbox::FsMetadata
+
+  # Host <-> guest copies
+  sb.fs.copy_from_host("./local.txt", "/tmp/local.txt")
+  sb.fs.copy_to_host("/tmp/out.txt", "./out.txt")
+end
+```
+
+### Metrics & logs
+
+```ruby
+Microsandbox::Sandbox.create("obs", image: "alpine") do |sb|
+  m = sb.metrics                       # => Microsandbox::Metrics
+  m.cpu_percent
+  m.memory_bytes
+  m.uptime_secs
+
+  sb.logs(tail: 100, sources: ["stdout", "stderr"]).each do |entry|
+    puts "[#{entry.source}] #{entry.text}"
+  end
+end
+```
+
+### Streaming output
+
+For long-running commands, stream events as they arrive instead of waiting:
+
+```ruby
+Microsandbox::Sandbox.create("stream", image: "python") do |sb|
+  handle = sb.exec_stream("python", ["-u", "-c", "import time\nfor i in range(3): print(i); time.sleep(1)"])
+  handle.each do |event|       # ExecHandle is Enumerable
+    print event.text if event.stdout?
+  end
+  # or: out = handle.collect  → ExecOutput  (drain to the end)
+  # interactive stdin:
+  #   sink = handle.stdin; sink.write("data\n"); sink.close
+  # control: handle.signal(15), handle.kill, handle.resize(rows, cols)
+end
+```
+
+### Images
+
+Manage the local OCI image cache (images are pulled automatically on `create`):
+
+```ruby
+Microsandbox::Image.list           # => [Microsandbox::ImageInfo, ...]
+Microsandbox::Image.get("alpine")  # => Microsandbox::ImageInfo
+Microsandbox::Image.inspect("alpine").layers  # => [{...}, ...]
+Microsandbox::Image.remove("alpine", force: true)
+report = Microsandbox::Image.prune
+report.bytes_reclaimed
+```
+
+### Named volumes
+
+Persistent storage that outlives individual sandboxes:
+
+```ruby
+Microsandbox::Volume.create("cache", kind: "disk", size_mib: 512)
+Microsandbox::Volume.list           # => [Microsandbox::VolumeInfo, ...]
+
+Microsandbox::Sandbox.create("with-vol", image: "alpine",
+                             volumes: { "/data" => { named: "cache" } }) do |sb|
+  sb.fs.write("/data/state.txt", "persisted")
+end
+
+Microsandbox::Volume.remove("cache")
+```
+
+`volumes:` accepts a host path String (bind mount) or `{ bind: "/host" }` /
+`{ named: "volume-name" }` per guest path. Boot from a snapshot with
+`Sandbox.create(name, from_snapshot: "snap-name-or-path")`.
+
+### Error handling
+
+All errors descend from `Microsandbox::Error` and carry a stable `#code`:
+
+```ruby
+begin
+  Microsandbox::Sandbox.create("dup", image: "alpine")
+  Microsandbox::Sandbox.create("dup", image: "alpine")  # name clash
+rescue Microsandbox::SandboxAlreadyExistsError => e
+  warn "#{e.code}: #{e.message}"       # => "sandbox-already-exists: ..."
+rescue Microsandbox::Error => e
+  warn "microsandbox failed: #{e.message}"
+end
+```
+
+| Class | `#code` |
+|-------|---------|
+| `InvalidConfigError` | `invalid-config` |
+| `SandboxNotFoundError` | `sandbox-not-found` |
+| `SandboxAlreadyExistsError` | `sandbox-already-exists` |
+| `SandboxStillRunningError` | `sandbox-still-running` |
+| `ExecTimeoutError` | `exec-timeout` |
+| `ExecFailedError` | `exec-failed` |
+| `FilesystemError` | `filesystem-error` |
+| `ImageNotFoundError` | `image-not-found` |
+| `MetricsDisabledError` / `MetricsUnavailableError` | `metrics-disabled` / `metrics-unavailable` |
+| … | (see `lib/microsandbox/errors.rb`) |
+
+## Runtime configuration
+
+The `msb` runtime path is resolved in this order: the `MSB_PATH` environment
+variable → an SDK-set override → the config file → `~/.microsandbox/bin/msb` →
+`msb` on `PATH`.
+
+```ruby
+Microsandbox.installed?            # => true/false
+Microsandbox.install               # download + install the runtime (idempotent)
+Microsandbox.runtime_path          # => "/Users/you/.microsandbox/bin/msb"
+Microsandbox.runtime_path = "/opt/microsandbox/bin/msb"  # override
+```
+
+## Development
+
+```bash
+bin/setup            # bundle install
+bundle exec rake compile          # build the native extension (debug)
+bundle exec rake compile:release  # build optimized
+bundle exec rake spec             # run unit specs (no runtime needed)
+MICROSANDBOX_INTEGRATION=1 bundle exec rake spec:all   # + real microVM specs
+```
+
+Unit specs run without a runtime. Integration specs boot real microVMs and are
+opt-in via `MICROSANDBOX_INTEGRATION=1` (override the test image with
+`MICROSANDBOX_TEST_IMAGE`).
+
+The native extension depends on the `microsandbox` core crate via a pinned git
+tag, so it builds without an adjacent checkout. To develop against a sibling
+`microsandbox/` checkout instead (faster, reflects local runtime changes):
+
+```bash
+cp .cargo/config.toml.example .cargo/config.toml   # gitignored path override
+bundle exec rake compile
+```
+
+## Releasing
+
+Releases are automated by `.github/workflows/release.yml` via RubyGems
+**Trusted Publishing** (OIDC) — there is no API key to store as a secret.
+
+**One-time setup** (before the first release), create a *pending* trusted
+publisher at <https://rubygems.org/profile/oidc/pending_trusted_publishers>:
+
+| Field | Value |
+|-------|-------|
+| RubyGems gem name | `microsandbox-rb` |
+| Repository owner | `ya-luotao` |
+| Repository name | `microsandbox-rb` |
+| Workflow filename | `release.yml` |
+| Environment | *(leave blank)* |
+
+On the first successful push the pending publisher auto-converts to a permanent
+one bound to the gem.
+
+**Each release:**
+
+1. Bump `Microsandbox::VERSION` (and the `tag = "vX.Y.Z"` on the core-crate
+   dependency in `ext/microsandbox/Cargo.toml`) to match the upstream runtime,
+   update `CHANGELOG.md`.
+2. *(Recommended first time)* run the workflow's manual `dry_run` dispatch to
+   build all platform gems without publishing — confirms the `arm64-darwin`
+   cross-build succeeds.
+3. Push a `vX.Y.Z` tag. CI builds precompiled, multi-ABI platform gems
+   (`x86_64-linux`, `aarch64-linux`, `arm64-darwin`) with `rake-compiler-dock`,
+   plus the source gem, and pushes them to RubyGems. The publish job uses
+   `rubygems/configure-rubygems-credentials` (OIDC) with `id-token: write` — no
+   `RUBYGEMS_API_KEY` secret required.
+
+See [DESIGN.md](DESIGN.md) for the architecture and the implemented-surface
+section for what's covered today vs. on the roadmap. Covered: full sandbox
+lifecycle (including the async `request_stop`/`request_kill`/`request_drain`/
+`wait_until_stopped`/`detach`/`owns_lifecycle?` controls and label-filtered
+`list_with`), `exec`/`shell` (collected and streaming), the full guest
+filesystem, metrics (per-sandbox, `Microsandbox.all_sandbox_metrics`, and
+streaming `metrics_stream`/`log_stream`), logs, OCI image-cache management,
+named volumes, and snapshots (create/list/verify/export/import +
+boot-from-snapshot). Create options span resources, network policy presets,
+`log_level`/`security`/`rlimits`/`pull_policy`/`secrets` and more; `exec`/`shell`
+take per-call `rlimits`. Still on the roadmap: custom per-rule network policies,
+file patches, registry auth, interactive `attach`, SSH, and the raw agent client.
+
+## License
+
+Apache-2.0. See [LICENSE](LICENSE).

data/ext/microsandbox/Cargo.toml

@@ -0,0 +1,45 @@
+[package]
+# Distinct from the core `microsandbox` package: Cargo refuses two packages with
+# the same name in one lockfile. The native artifact is loaded internally as
+# `microsandbox/microsandbox_rb` and stays hidden behind `require "microsandbox"`.
+name = "microsandbox_rb"
+description = "Ruby SDK native extension for microsandbox — secure, fast microVM-based sandboxing."
+version = "0.5.7"
+authors = ["Super Rad Company <development@superrad.company>"]
+repository = "https://github.com/superradcompany/microsandbox"
+license = "Apache-2.0"
+edition = "2021"
+publish = false
+
+[lib]
+# Produces libmicrosandbox_rb.{dylib,so}; rb-sys renames it to
+# microsandbox/microsandbox_rb.{bundle,so}. The magnus #[init] entrypoint is
+# therefore Init_microsandbox_rb (matches the loaded basename).
+name = "microsandbox_rb"
+crate-type = ["cdylib"]
+
+[dependencies]
+magnus = "0.8"
+# Low-level Ruby C API — used directly for rb_thread_call_without_gvl (GVL release
+# during blocking sandbox calls). Kept in lock-step with magnus's own rb-sys.
+rb-sys = "0.9"
+
+# The microsandbox core runtime, wrapped via FFI exactly like the official
+# Python (pyo3) and Node (napi) SDKs. Pinned to the matching upstream tag so the
+# gem is self-contained and buildable anywhere (CI, release containers, end-user
+# source installs) without an adjacent checkout. For local development against a
+# sibling checkout, use the `paths` override in `.cargo/config.toml` (see
+# `.cargo/config.toml.example`). "ssh" matches the feature set the Python/Node
+# SDKs ship with; default features add "prebuilt" (provisions msb + libkrunfw at
+# build time), "net", and "keyring".
+microsandbox = { git = "https://github.com/superradcompany/microsandbox", tag = "v0.5.7", default-features = true, features = ["ssh"] }
+microsandbox-network = { git = "https://github.com/superradcompany/microsandbox", tag = "v0.5.7" }
+
+# Async core bridged to Ruby's synchronous API via a blocking tokio runtime.
+tokio = { version = "1", features = ["rt-multi-thread", "sync", "time"] }
+futures = "0.3"
+
+# Shared (de)serialization + value types used across the binding surface.
+serde_json = "1"
+chrono = "0.4"
+ipnetwork = { version = "0.21", features = ["serde"] }

data/ext/microsandbox/extconf.rb

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+require "mkmf"
+require "rb_sys/mkmf"
+
+# Builds the Rust cdylib and installs it as lib/microsandbox/microsandbox.{bundle,so}.
+# The Cargo profile is selected via the RB_SYS_CARGO_PROFILE env var (defaults to
+# release for installed gems, dev for `rake compile`).
+#
+# Cargo features for the embedded microsandbox runtime (e.g. "ssh") are enabled
+# directly on the dependency in ext/microsandbox/Cargo.toml, mirroring the official
+# Python/Node SDKs. The crate's "prebuilt" default feature provisions the msb
+# runtime + libkrunfw into ~/.microsandbox at build time.
+create_rust_makefile("microsandbox/microsandbox_rb")

data/ext/microsandbox/src/conv.rs

@@ -0,0 +1,74 @@
+//! Helpers for reading a Ruby options `Hash` (string keys) into Rust values.
+//!
+//! The Ruby layer normalizes keyword arguments into a string-keyed Hash before
+//! handing it to the native layer, so lookups here are by `&str`.
+
+use std::collections::HashMap;
+
+use magnus::{value::ReprValue, Error, RHash, TryConvert, Value};
+
+/// Fetch a non-nil value for `key`, if present.
+fn get(hash: RHash, key: &str) -> Option<Value> {
+    match hash.get(key) {
+        Some(v) if !v.is_nil() => Some(v),
+        _ => None,
+    }
+}
+
+/// Generic typed fetch: `None` if the key is absent/nil, else converted.
+pub fn opt<T: TryConvert>(hash: RHash, key: &str) -> Result<Option<T>, Error> {
+    match get(hash, key) {
+        Some(v) => Ok(Some(T::try_convert(v)?)),
+        None => Ok(None),
+    }
+}
+
+pub fn opt_string(hash: RHash, key: &str) -> Result<Option<String>, Error> {
+    opt(hash, key)
+}
+
+pub fn opt_bool(hash: RHash, key: &str) -> Result<bool, Error> {
+    Ok(opt::<bool>(hash, key)?.unwrap_or(false))
+}
+
+pub fn opt_u8(hash: RHash, key: &str) -> Result<Option<u8>, Error> {
+    opt(hash, key)
+}
+
+pub fn opt_u32(hash: RHash, key: &str) -> Result<Option<u32>, Error> {
+    opt(hash, key)
+}
+
+pub fn opt_f64(hash: RHash, key: &str) -> Result<Option<f64>, Error> {
+    opt(hash, key)
+}
+
+/// String→String map (e.g. `env`, `labels`, `scripts`). Empty if absent.
+pub fn opt_string_map(hash: RHash, key: &str) -> Result<Vec<(String, String)>, Error> {
+    match get(hash, key) {
+        Some(v) => {
+            let map = HashMap::<String, String>::try_convert(v)?;
+            Ok(map.into_iter().collect())
+        }
+        None => Ok(Vec::new()),
+    }
+}
+
+/// Array of strings (e.g. `entrypoint`, `args`). Empty if absent.
+pub fn opt_string_vec(hash: RHash, key: &str) -> Result<Vec<String>, Error> {
+    match get(hash, key) {
+        Some(v) => Ok(Vec::<String>::try_convert(v)?),
+        None => Ok(Vec::new()),
+    }
+}
+
+/// `u16`→`u16` port map (host→guest TCP). Empty if absent.
+pub fn opt_port_map(hash: RHash, key: &str) -> Result<Vec<(u16, u16)>, Error> {
+    match get(hash, key) {
+        Some(v) => {
+            let map = HashMap::<u16, u16>::try_convert(v)?;
+            Ok(map.into_iter().collect())
+        }
+        None => Ok(Vec::new()),
+    }
+}

data/ext/microsandbox/src/error.rs

@@ -0,0 +1,72 @@
+//! Map the core `MicrosandboxError` enum onto the Ruby exception hierarchy.
+//!
+//! Mirrors `sdk/python/src/error.rs`: each handled variant is routed to a
+//! specific `Microsandbox::*Error` class (defined in `lib/microsandbox/errors.rb`),
+//! every other variant falls back to the base `Microsandbox::Error`. The message
+//! is always the core error's `to_string()`.
+
+use magnus::{value::ReprValue, Error, ExceptionClass, Module, RClass, RModule, Ruby};
+use microsandbox::{AgentClientError, MicrosandboxError};
+
+/// The Ruby class (relative to the `Microsandbox` module) for a core error.
+/// `"Error"` is the base class; anything else is a named subclass.
+fn class_name(err: &MicrosandboxError) -> &'static str {
+    use MicrosandboxError::*;
+    match err {
+        InvalidConfig(_) => "InvalidConfigError",
+        SandboxNotFound(_) => "SandboxNotFoundError",
+        SandboxAlreadyExists(_) => "SandboxAlreadyExistsError",
+        SandboxStillRunning(_) => "SandboxStillRunningError",
+        ExecTimeout(_) => "ExecTimeoutError",
+        ExecFailed(_) => "ExecFailedError",
+        SandboxFsOps(_) => "FilesystemError",
+        ImageNotFound(_) => "ImageNotFoundError",
+        ImageInUse(_) => "ImageInUseError",
+        VolumeNotFound(_) => "VolumeNotFoundError",
+        VolumeAlreadyExists(_) => "VolumeAlreadyExistsError",
+        Io(_) => "IoError",
+        MetricsDisabled(_) => "MetricsDisabledError",
+        MetricsUnavailable(_) => "MetricsUnavailableError",
+        AgentClient(AgentClientError::UnsupportedOperation { .. }) => "UnsupportedOperationError",
+        _ => "Error",
+    }
+}
+
+/// Look up `Microsandbox::<name>` as an exception class.
+fn exception_class(ruby: &Ruby, name: &str) -> Option<ExceptionClass> {
+    let module: RModule = ruby.class_object().const_get("Microsandbox").ok()?;
+    let class: RClass = module.const_get(name).ok()?;
+    ExceptionClass::from_value(class.as_value())
+}
+
+/// Convert a core error into a Ruby exception, preserving the typed class.
+// The `exception::runtime_error()` fallbacks fire only off a Ruby thread (which
+// never happens from a bound method); there is no handle-based alternative there.
+#[allow(deprecated)]
+pub fn to_ruby(err: MicrosandboxError) -> Error {
+    let message = err.to_string();
+    let ruby = match Ruby::get() {
+        Ok(ruby) => ruby,
+        // Not on a Ruby thread (should never happen from a bound method).
+        Err(_) => return Error::new(magnus::exception::runtime_error(), message),
+    };
+
+    match exception_class(&ruby, class_name(&err)) {
+        Some(class) => Error::new(class, message),
+        None => Error::new(ruby.exception_runtime_error(), message),
+    }
+}
+
+/// A plain `Microsandbox::Error` (base) with a custom message — used for
+/// binding-level validation errors that have no core counterpart.
+#[allow(deprecated)]
+pub fn base_error(message: impl Into<String>) -> Error {
+    let message = message.into();
+    match Ruby::get() {
+        Ok(ruby) => match exception_class(&ruby, "Error") {
+            Some(class) => Error::new(class, message),
+            None => Error::new(ruby.exception_runtime_error(), message),
+        },
+        Err(_) => Error::new(magnus::exception::runtime_error(), message),
+    }
+}