microsandbox-rb 0.17.2 → 0.17.4

data/docs/runtime.md

@@ -104,6 +104,121 @@ operations (create/start/stop/remove/get/list, one-shot exec, follow log
 streaming); unsupported operations raise `Microsandbox::UnsupportedError`
 (with `#operation` and `#hint`).
 
+## Forking (Puma, Resque, Solid Queue)
+
+Forking a process that has already used the SDK is supported: preload the
+application, use the SDK in the master, fork workers, use it in the workers.
+The first SDK call in a child sets up its own runtime threads and its own
+backend (a new catalog connection pool, or a new HTTP client for the cloud
+backend), chosen the same way as the parent's: an explicit
+`set_default_backend`, an enclosing `with_backend` block, or — when neither
+was used — the environment and config file as the child sees them at that
+call. Everything addressed by name works in the child from then on:
+`Sandbox.create`/`get`/`list`/`start`, `Volume`, `Image`, `Snapshot` — except
+that on macOS a forked child must not do anything that builds a network
+client (see [macOS](#macos-no-network-clients-in-a-forked-child) below).
+
+If the parent's choice cannot be made again in the child (say the child's
+environment now selects the cloud backend without credentials), SDK calls
+there raise the configuration error until that is repaired. Selecting a
+backend in the child — `set_default_backend`, or entering `with_backend` —
+does not depend on the inherited choice and works regardless.
+
+```ruby
+Microsandbox::Sandbox.list             # the parent has used the SDK
+
+pid = fork do
+  # On macOS "alpine" must already be in the image cache (see below).
+  Microsandbox::Sandbox.create("job-#{Process.pid}", image: "alpine") do |sb|
+    puts sb.exec("echo", ["hello from the worker"]).stdout
+  end
+end
+Process.wait(pid)
+```
+
+What does **not** cross a fork is an object created before it. A `Sandbox`,
+`SandboxHandle`, `ExecHandle`, `ExecStdin`, `PullSession`, log/metrics/file
+stream, `SshClient`/`SftpClient`/`SshServer`, `AgentClient` or `VolumeFs`
+belongs to the process that created it. In a forked child, any call that acts
+on one raises `Microsandbox::ForkedHandleError` (the message names the
+pids involved); the original keeps working in the parent. Reading `name` or `id` — and
+the snapshot fields of a `SandboxHandle` — still works, so a child can open
+its own handle:
+
+```ruby
+sandbox = Microsandbox::Sandbox.create("shared", image: "alpine")
+
+pid = fork do
+  begin
+    sandbox.exec("true")                 # the parent's handle
+  rescue Microsandbox::ForkedHandleError => e
+    warn e.message
+  end
+  mine = Microsandbox::Sandbox.get(sandbox.name).connect_or_start
+  puts mine.exec("echo", ["hello"]).stdout
+end
+Process.wait(pid)
+sandbox.stop
+```
+
+Things to know:
+
+- **Cleanup code inherited by the child.** The block forms
+  (`Sandbox.create(...) { }`, `connect_or_create`) swallow the
+  `ForkedHandleError` their teardown hits when a forked child leaves the
+  block, so the child does not stop the parent's sandbox. An `ensure
+  sandbox.stop` of your own raises it in the child; rescue it there, or run
+  the cleanup only in the process that created the sandbox.
+- **Leaving a `with_backend` block in the child.** When the backend to
+  restore was the parent's — the child was forked inside the block, or the
+  block is the child's first use of the SDK — leaving the block installs a
+  backend chosen the way that one had been. The block's own temporary backend
+  is not needed for that: if only it can no longer be chosen in the child (a
+  cloud profile that is gone there), leaving the block still restores the
+  previous backend. If the choice to restore no longer resolves in the child,
+  the block is closed anyway and its temporary selection stays in effect.
+  `with_backend` then raises the configuration error — unless the block
+  itself raised, in which case the block's exception propagates and the
+  restore failure is reported as a warning.
+- **A child keeps the parent's sandboxes alive.** A child inherits the
+  parent's open file descriptors, among them the pipe through which a sandbox
+  notices that its owner is gone. If the parent exits while such a child is
+  still running, the sandbox is not torn down until that child exits as well.
+  Fork before creating sandboxes, or create them in the children.
+- **Do not fork while another thread is inside an SDK call.** That thread
+  does not exist in the child, and a lock it held at that instant stays
+  locked there.
+- **Memory.** A child leaks, once, the native state it inherited (the
+  parent's runtime, backend and whichever inherited handles it collects)
+  instead of destroying it — destroying it is what used to crash.
+
+### macOS: no network clients in a forked child
+
+On macOS, a forked child that has not called `exec` dies — `SIGSEGV` or
+`SIGABRT`, not an exception, so it cannot be rescued — as soon as the SDK
+builds a network client or reads registry credentials from the Keychain
+there. macOS's system frameworks do not support being used after `fork`
+without `exec`, and every HTTP client the SDK builds looks up the system proxy
+settings through them. In a forked child on macOS, do not:
+
+- use a cloud backend at all, however it is selected (`set_default_backend`,
+  `with_backend`, `MSB_BACKEND`/`MSB_API_KEY`, a profile);
+- `Sandbox.create` with an image that is not in the image cache yet, or with
+  `pull_policy: :always`;
+- `Image.load`;
+- install the runtime: `Microsandbox.install`, `Microsandbox.setup`, or the
+  first-use install `ensure_runtime!` performs.
+
+It makes no difference whether the parent used the SDK, and
+`OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES` does not help. Local work is
+unaffected, including `Sandbox.create` with an image that is already cached.
+On Linux the cloud backend works in a forked child.
+
+Do that work in a process that was not forked: in the parent before it forks
+(install the runtime and pull the images the workers will use — creating a
+sandbox from the image pulls it), or in a fresh process started with
+`Process.spawn` (or `fork` followed by `exec`).
+
 ## One runtime line per home
 
 The runtime home (`MSB_HOME`, default `~/.microsandbox`) belongs to one

data/docs/surface.md

@@ -57,6 +57,12 @@ counterpart, and [DESIGN.md](../DESIGN.md) explains how the layers fit.
 - **Registries** — `registry_auth`/`registry_insecure`/`registry_ca_certs` on
   `create`.
 - **Errors** — the typed hierarchy in [errors.md](errors.md).
+- **Process model** — the SDK may be used before and after `fork`; objects
+  created before a fork are refused in the child with `ForkedHandleError`
+  (see [runtime.md](runtime.md#forking-puma-resque-solid-queue)). On macOS a
+  forked child must not build network clients — cloud backends, image pulls,
+  `Image.load`, runtime install — or it crashes (see
+  [runtime.md](runtime.md#macos-no-network-clients-in-a-forked-child)).
 
 Create options span resources (`cpus`/`memory`/`max_cpus`/`max_memory`),
 `env`/`workdir`/`labels`/`user`/`hostname`/`shell`, named `scripts`, the image

data/ext/microsandbox/Cargo.toml

@@ -8,7 +8,7 @@ description = "Ruby SDK native extension for microsandbox — secure, fast micro
 # returns this via env!("CARGO_PKG_VERSION") and version_spec.rb asserts equality.
 # The core-crate dependencies below stay on their own tag, which is
 # Microsandbox::RUNTIME_VERSION and moves independently of this version.
-version = "0.17.2"
+version = "0.17.4"
 authors = ["Super Rad Company <development@superrad.company>"]
 repository = "https://github.com/superradcompany/microsandbox"
 license = "Apache-2.0"
@@ -25,12 +25,19 @@ crate-type = ["cdylib"]
 [dependencies]
 # "rb-sys" exposes `magnus::rb_sys::protect` (rb_protect), which the GVL bridge
 # uses to check interrupts without letting Ruby longjmp over Rust frames.
-magnus = { version = "0.8", features = ["rb-sys"] }
+# Default features are off: `old-api` (the free functions that conjure a `Ruby`
+# handle unchecked in release builds) must not creep back, and `io` only gates
+# `magnus::io`, which the extension does not use.
+magnus = { version = "0.9", default-features = false, features = ["rb-sys"] }
 # Low-level Ruby C API — used directly for rb_thread_call_without_gvl2 (GVL
 # release during blocking sandbox calls) and rb_thread_check_ints. Kept in
 # lock-step with magnus's own rb-sys.
 rb-sys = "0.9"
 
+# `pthread_atfork`, for the fork generation (runtime::ProcessIdentity). Already
+# in the dependency graph; listed so the binding can call it directly.
+libc = "0.2"
+
 # The microsandbox core runtime, wrapped via FFI exactly like the official
 # Python (pyo3) and Node (napi) SDKs. Pinned to the matching upstream tag so the
 # gem is self-contained and buildable anywhere (CI, release containers, end-user

data/ext/microsandbox/src/agent.rs

@@ -14,7 +14,7 @@ use microsandbox::agent::AgentClient as CoreAgentClient;
 use microsandbox::{AgentBridge, BridgeFrame, MicrosandboxError};
 
 use crate::error;
-use crate::runtime::{block_on, ruby};
+use crate::runtime::{block_on, ruby, ForkGuarded};
 
 /// Map an agent-client error onto the Ruby exception hierarchy (via the core
 /// `MicrosandboxError::AgentClient` wrapper, exactly like the Python binding).
@@ -24,13 +24,13 @@ fn to_ruby_agent(err: microsandbox::AgentClientError) -> Error {
 
 #[magnus::wrap(class = "Microsandbox::Native::AgentClient", free_immediately, size)]
 pub struct AgentClient {
-    inner: Arc<AgentBridge>,
+    inner: ForkGuarded<Arc<AgentBridge>>,
 }
 
 impl AgentClient {
     fn from_bridge(bridge: AgentBridge) -> Self {
         Self {
-            inner: Arc::new(bridge),
+            inner: ForkGuarded::new("Microsandbox::AgentClient", Arc::new(bridge)),
         }
     }
 
@@ -60,6 +60,9 @@ impl AgentClient {
 
     /// Resolve a sandbox's agent relay socket path without connecting.
     fn socket_path(name: String) -> Result<String, Error> {
+        // Synchronous, and the core derives the path from the default
+        // backend's config.
+        crate::backend::ensure_this_process_backend()?;
         let path = CoreAgentClient::socket_path(&name).map_err(error::to_ruby)?;
         Ok(path.to_string_lossy().into_owned())
     }
@@ -71,7 +74,7 @@ impl AgentClient {
     /// Send one frame and await a single response frame ({id, flags, body}).
     fn request(&self, flags: u8, body: RString) -> Result<RHash, Error> {
         let body = unsafe { body.as_slice() }.to_vec();
-        let inner = Arc::clone(&self.inner);
+        let inner = Arc::clone(self.inner.get()?);
         let frame =
             block_on(async move { inner.request(flags, body).await })?.map_err(to_ruby_agent)?;
         Ok(frame_to_hash(frame))
@@ -80,7 +83,7 @@ impl AgentClient {
     /// Open a streaming session; returns {id, handle}.
     fn stream_open(&self, flags: u8, body: RString) -> Result<RHash, Error> {
         let body = unsafe { body.as_slice() }.to_vec();
-        let inner = Arc::clone(&self.inner);
+        let inner = Arc::clone(self.inner.get()?);
         let (id, handle) = block_on(async move { inner.stream_open(flags, body).await })?
             .map_err(to_ruby_agent)?;
         let hash = ruby().hash_new();
@@ -91,7 +94,7 @@ impl AgentClient {
 
     /// Pull the next frame from a stream; nil at end-of-stream.
     fn stream_next(&self, handle: u64) -> Result<Option<RHash>, Error> {
-        let inner = Arc::clone(&self.inner);
+        let inner = Arc::clone(self.inner.get()?);
         let frame =
             block_on(async move { inner.stream_next(handle).await })?.map_err(to_ruby_agent)?;
         Ok(frame.map(frame_to_hash))
@@ -99,7 +102,7 @@ impl AgentClient {
 
     /// Close a stream handle. Idempotent.
     fn stream_close(&self, handle: u64) -> Result<(), Error> {
-        let inner = Arc::clone(&self.inner);
+        let inner = Arc::clone(self.inner.get()?);
         block_on(async move { inner.stream_close(handle).await })?;
         Ok(())
     }
@@ -107,19 +110,19 @@ impl AgentClient {
     /// Send a follow-up frame on an existing correlation id.
     fn send(&self, id: u32, flags: u8, body: RString) -> Result<(), Error> {
         let body = unsafe { body.as_slice() }.to_vec();
-        let inner = Arc::clone(&self.inner);
+        let inner = Arc::clone(self.inner.get()?);
         block_on(async move { inner.send(id, flags, body).await })?.map_err(to_ruby_agent)
     }
 
     /// Cached handshake `core.ready` frame body bytes (CBOR).
     fn ready_bytes(&self) -> Result<RString, Error> {
-        let bytes = self.inner.ready_bytes().map_err(to_ruby_agent)?;
+        let bytes = self.inner.get()?.ready_bytes().map_err(to_ruby_agent)?;
         Ok(ruby().str_from_slice(&bytes))
     }
 
     /// Close the connection. Idempotent.
     fn close(&self) -> Result<(), Error> {
-        let inner = Arc::clone(&self.inner);
+        let inner = Arc::clone(self.inner.get()?);
         block_on(async move { inner.close().await })?;
         Ok(())
     }