capybara-simulated 0.9.0 → 0.10.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 21b8b1545d94e237e8740cb84e4255eb7ce9b4b0743f105451176804f4fcb904
-  data.tar.gz: 5fe3aaa88743308bebfb7d6acc90366a0c2e56439afe5ae3bdfcc9eb5c107a5b
+  metadata.gz: d70e169d98f5ead094c27e37f8f19fa01a3ea84aa7d808cf62a5d6dcbe3bf811
+  data.tar.gz: 50c9c13da2f7dfcc558fd7b92617bcf39ac45db930965e7152b632951bfc3770
 SHA512:
-  metadata.gz: 7f5c96e2eb7b0b73351026dc25391d17a7da6a36a00b6f4213dc9f28554164da5991b46dbcfa925a7385a5f2a6faadf01b38b16bb7ef9a6ca2569439540f5b04
-  data.tar.gz: 9e6008b539c30fe918c96f8e38f7d67fe552c1a61034890d63ebfc16accfad9748e3a09770e7d44d0d06548036cfd299ab44875e3fccbffeb491dd8a9506706e
+  metadata.gz: c7856b38b6376c6d5ac68065757f4beccac1b13a1bfd97c4453d2c8f1597ba0c82de66ec5e227ee703a185ca76758f8484e6e5b7ac72a5f90f493bb8d30f5049
+  data.tar.gz: b66c174eca3a6f89bdbc0df104cf302e212a0f3a63d72872b41a11b44d9fcd0b5d18c77d17ce0cb8f99b7bab24b01a4c81085942835deed19b66084a3d7a961b

data/README.md

@@ -22,7 +22,7 @@ capybara-simulated is not a complete replacement for real-browser testing. But i
 - **Drop-in**: the Capybara DSL is unchanged — register `:simulated` and go. Just this gem plus one JS-engine gem.
 - **Held to spec**: a vendored [web-platform-tests](https://github.com/web-platform-tests/wpt) conformance gate (the same DOM / HTML tests Chromium and Firefox hold themselves to), plus the full system suites of five real apps — Redmine / Forem / Avo / Mastodon / Discourse — run against the driver in [capybara-simulated-vs-world](https://github.com/ursm/capybara-simulated-vs-world).
 
-**Reach for a real browser** (Selenium / Cuprite) **when** your tests need what this driver doesn't simulate **by design** — there's no rendering engine, so **pixel-accurate layout** (text measured by real font metrics, inline flow, flex / grid track sizing) and **screenshots** are out. There *is* a coarse box-layout engine — enough that `getBoundingClientRect()`, `elementFromPoint()`, `obscured?`, the spatial selectors, scrolling, and drag-and-drop all work against real boxes — but it answers "where is this, roughly, and what's on top", not "how would this render".
+**Reach for a real browser** (Selenium / Cuprite) **when** your tests need what this driver doesn't simulate **by design** — there's no rendering engine, so **pixel-accurate rendering** (glyph shaping — kerning, ligatures, bidi — the real line-breaking algorithm, multi-line `flex-wrap`) is out. `save_screenshot` does paint a real PNG, but it paints what the layout engine believes — enough to see what a test saw, not a visual-regression baseline. There *is* a coarse box-layout engine — enough that `getBoundingClientRect()`, `elementFromPoint()`, `obscured?`, the spatial selectors, scrolling, and drag-and-drop all work against real boxes — but it answers "where is this, roughly, and what's on top", not "how would this render".
 
 Most of the rest runs in-process — including the things that usually mean "you need a real browser": **`within_frame`**, **multiple windows / tabs**, **WebSocket + Action Cable**, **EventSource**, and **Web Workers** all work. Each has constraints (JS engine, settle-timing, coarse layout); see [Capabilities & limits](#capabilities--limits).
 
@@ -35,19 +35,30 @@ gem 'rusty_racer', group: :test  # JS engine — pick one
 
 `bundle install`. Requires Ruby ≥ 3.3. The gem ships its JS bridge under `lib/capybara/simulated/js/` and the vendored JS deps under `vendor/js/`, so there's no Node toolchain at consume time.
 
+### System libraries
+
+**libvips** — Debian/Ubuntu `libvips42`, Homebrew `vips`, Gentoo `media-libs/vips`. The `ruby-vips`
+gem comes with the driver and binds to it; the driver names the package it wants if the library is
+missing.
+
+**fontconfig** — text is MEASURED from the font file fontconfig resolves each CSS family to (the
+same face a browser gets on the same machine), so a box's height and a line's wrapping depend on
+the fonts installed. A machine with no fonts falls back to an estimate and measures text wider or
+narrower than a real browser would.
+
 ### JS engine
 
 The gem treats the JS engine as a soft dependency. Pick one of:
 
 ```ruby
-gem 'rusty_racer', '>= 0.1.9'  # V8 (JIT, fastest per spec) — default
+gem 'rusty_racer', '>= 0.2.1'  # V8 (JIT, fastest per spec) — default
 gem 'quickjs', '>= 0.19'       # QuickJS (interpreter, smaller per-VM RAM —
 gem 'quickjs-polyfill-intl'    # wins when scaling parallel workers under
                                # a fixed memory budget). Intl lives in the
                                # companion gem since quickjs 0.19.
 ```
 
-The V8 engine comes from [rusty_racer](https://github.com/ursm/rusty_racer), a rusty_v8-based Ruby binding with the native ES Module API, `ScriptCompiler::CachedData` snapshots, and per-frame realm contexts the driver builds on.
+The V8 engine comes from [rusty_racer](https://github.com/ursm/rusty_racer), a rusty_v8-based Ruby binding with the native ES Module API, `ScriptCompiler::CachedData` snapshots, and per-frame realm contexts the driver builds on. 0.2.1 is the floor: the driver needs `Context#eval_void` (0.2.1) to wire a frame's `parent`/`top` without marshalling the WindowProxy it just assigned, and `Module#graph_async?` (0.2.0) to reject top-level await in a service worker.
 
 The engine is auto-detected at boot; if both gems are present V8 wins. Override explicitly with `CSIM_JS_ENGINE=v8|quickjs` or `Capybara::Simulated::Driver.new(app, js_engine: :quickjs)`.
 
@@ -122,17 +133,23 @@ puts page.text
 
 ## Trace
 
-Each Capybara action (`visit`, `click`, `set`, …) is recorded as a step in a per-test trace: URL before / after, console output and network requests during the step, plus elapsed and per-step durations. On action failure (and only then, by default) the post-action DOM is captured too.
+Each Capybara action (`visit`, `click`, `set`, …) is recorded as a step in a per-test trace: URL before / after, console output and network requests during the step, plus elapsed and per-step durations. On action failure (and only then, by default) the post-action DOM is captured too, and a failing example gets one **screenshot** of the state it ended in.
 
-Recording is **on by default** — fully in-memory, no files written unless you opt in via `CSIM_TRACE_DIR`. Wall-time overhead is run-to-run-variance equivalent because the expensive part — DOM serialization — only fires on action error.
+Recording is **on by default** — fully in-memory, no files written unless you opt in via `CSIM_TRACE_DIR`. Wall-time overhead is run-to-run-variance equivalent because the expensive parts fire only where they cannot cost a test anything: the DOM is serialized on an action error and only once per action however many times Capybara retries it, and the screenshot is painted after the example.
 
 ### Modes (`CSIM_TRACE=…`)
 
-| value | recording | DOM snapshot |
-|---|---|---|
-| (unset) / `on-failure` | yes (default) | per step on action error only |
-| `full` | yes | after every action — debug-heavy |
-| `off` | nothing recorded, `record_action` early-exits | — |
+| value | recording | DOM snapshot | screenshot |
+|---|---|---|---|
+| (unset) / `on-failure` | yes (default) | per step on action error only | one, of the state a failing example ended in |
+| `full` | yes | after every action — debug-heavy | per action too — debug-heavy |
+| `off` | nothing recorded, `record_action` early-exits | — | — |
+
+A screenshot is painted from the layout the driver already holds (see [Screenshots](#works-with-constraints)), so it shows what the test *saw*, and rides inline as a `data:` URL so a trace stays one file and the viewer still opens from `file://`.
+
+**Where** it is taken matters more than it sounds. A paint is 33 ms on V8 and 517 ms on QuickJS for a small page — 236 ms and 1.6 s for a 2000-row table — so painting an action's *failure* would put it inside Capybara's retry window. Capybara retries for the whole wait: one failing click records **183 attempts** in its 2 s window here, and photographing them turned a click waiting on an overlay to clear from 35 ms into 563 ms, which is enough to turn an action a retry would have rescued into a failure.
+
+So by default the trace paints exactly once, **after** the example and only if it failed, where no wait window is running. `CSIM_TRACE=full` adds a shot per action that *succeeded* — never a failing attempt, for the reason above, and the successful attempt is the interesting frame of a retried action anyway. Each shot is ~60-75 KB of inline base64, so a long trace in `full` mode is a multi-MB single-file HTML.
 
 ### Inspecting traces
 
@@ -176,7 +193,9 @@ capybara-simulated trace tmp/csim-traces/checkout_flow.json
 # wrote /tmp/checkout_flow.html   (then opens it in your browser)
 ```
 
-By default the HTML is written to a temp file and opened in your browser. The viewer works straight from `file://` — the trace JSON is embedded inline, so no server is needed — and shows a step-by-step UI: a timeline of actions, and per step the URL before/after, console output, network requests, the error, and a rendered preview of the post-action DOM snapshot. Its **Load JSON…** button / drag-and-drop swaps in any other trace file.
+By default the HTML is written to a temp file and opened in your browser. The viewer works straight from `file://` — the trace JSON is embedded inline, and there is no webfont or CDN to reach for, so it opens offline from a CI download.
+
+It opens **on the step that failed** (that is what you came for, and on a long trace it is nowhere near the top), keeps that failure in a banner while you read any other step, and marks the step list: `!` where an action failed, `·` where one only logged a warning or error. Per step you get the URL before / after, console output, network requests — click a row for its headers and bodies — the error, and the post-action DOM snapshot as HTML. Screenshots sit in a side rail, which distinguishes the state the example ENDED in from the state at one step. `j` / `k` move, `f` jumps to the failure, and **Load JSON…** / drag-and-drop swaps in any other trace file.
 
 `-o PATH` writes the HTML somewhere specific (`-o -` to stdout); `--no-open` skips launching the browser. Browser launching uses [launchy](https://rubygems.org/gems/launchy) when it's installed (`gem 'launchy'`, recommended for reliable cross-platform / WSL opening) and falls back to the platform opener (`xdg-open` / `open` / `start`) otherwise.
 
@@ -212,6 +231,10 @@ end
       "url_before":  null,
       "url_after":   "http://www.example.com/checkout",
       "dom_after":   null,          // populated only on action error or in `full` mode
+      "shot_after":  null,          // …and the same moment PAINTED, as a `data:image/png;base64,…`
+                                    // URL — `CSIM_TRACE=full` only, and only for an action that
+                                    // SUCCEEDED. A failing example's final state is painted once
+                                    // into `metadata.screenshot` instead (see above).
       "console":     [{ "severity": "info", "message": "Stripe.js loaded" }],
       "network":     [{ "method": "GET",    "url": "/checkout", "status": 200 }],
       "elapsed_ms":  0,
@@ -234,6 +257,7 @@ Per visit, `<script src>`-referenced libraries (jQuery, Stimulus, …) re-evalua
 
 - **`<script src>` parsing** dominates `visit` on JS-heavy pages. Each external script is fetched through the in-process Rack app, compiled, and run in the JS engine with bytecode cache hits from the base snapshot warmup.
 - **CSS cascade resolution**: stylesheets are parsed once per distinct set of sources and cached content-addressably, so repeat visits and subsequent finds on the same page reuse the resolved cascade instead of re-parsing.
+- **HTTP cache**: in-process fetches go through an RFC 9111 cache that is process-wide, like a persistent browser profile. `Capybara.reset_sessions!` keeps what that profile would: fresh `Cache-Control: immutable` responses (fingerprinted bundles), still-fresh `<script src>` / `<link rel=stylesheet>` sources and `@font-face` files; other responses are dropped so test-local server state reaches the app. A test whose app serves *new* bytes at a cacheable URL it already served — a stylesheet digested from a DB row that a rolled-back example reuses — gets the cold cache a fresh Playwright / Cuprite context starts with via `page.driver.clear_http_cache` (or `Capybara::Simulated.clear_http_cache` from a hook). Parsed stylesheets, compiled bytecode and decoded images are memoized by content, so they never go stale and are not affected.
 - **DOM ops stay inside the JS engine** — find / has_? / event dispatch never cross the Ruby ↔ JS boundary for the actual tree walk; only the resulting handle ids do. Modify-heavy tests (SortableJS dragging thousands of items) run at JS-engine speed, not at host-call-IPC speed.
 - **Polling** (Capybara `default_max_wait_time`) advances a *virtual* JS clock — timers fire as polling steps the clock forward, not in real time. A page that schedules `setTimeout(2000, x)` doesn't block for 2 s; the callback fires once polling has advanced the clock past it.
 
@@ -243,7 +267,8 @@ Most features run in-process; the notes below are mostly "works, but…", follow
 
 ### Works, with constraints
 
-- **Layout-backed geometry** — a coarse box-layout engine (block flow, absolute / relative / fixed, a simple grid pass, percentage and viewport units, overflow clipping, the flat tree through shadow roots and slots, and cross-realm frames) backs the page-visible geometry, so there is *one* geometry that both the driver and the page's own JS read. That gives you `obscured?` (real occlusion, including out through nested iframes), the spatial selectors (`:above` / `:below` / `:left_of` / `:right_of` / `:near`), `scroll_to` / `scroll_by` clamped to the real scrollable range, geometry that follows `resize_to` (so a mobile-breakpoint test measures mobile boxes), and `drag_to` — which drives jQuery UI, SortableJS, Dragula and jsTree. It is laid out once per mutation generation and only when something asks for geometry. What it does *not* do: glyph metrics, inline flow, flex / grid track sizing — see [Out of scope](#out-of-scope-by-design--use-selenium--cuprite).
+- **Layout-backed geometry** — a coarse box-layout engine (block flow, absolute / relative / fixed, flex and grid track sizing, percentage and viewport units, overflow clipping, the flat tree through shadow roots and slots, and cross-realm frames) backs the page-visible geometry, so there is *one* geometry that both the driver and the page's own JS read. That gives you `obscured?` (real occlusion, including out through nested iframes), the spatial selectors (`:above` / `:below` / `:left_of` / `:right_of` / `:near`), `scroll_to` / `scroll_by` clamped to the real scrollable range, geometry that follows `resize_to` (so a mobile-breakpoint test measures mobile boxes), and `drag_to` — which drives jQuery UI, SortableJS, Dragula and jsTree. Text runs are measured with the font's own advance widths, inline content shares a line, tables get real CSS Tables 3 column sizing, and a flex line resolves its items together (CSS Flexbox §9.7: bases, grow / shrink, the automatic minimum, gaps, `auto` margins, `justify-content` / `align-items`), so on the block, inline, absolute, flex and table shapes an app page is built from the boxes it reports match Chrome's to the sub-pixel (measured against headless Chrome, fixture by fixture). It is laid out once per mutation generation and only when something asks for geometry. What it does *not* do: glyph shaping (kerning / ligatures / bidi), the real line-breaking algorithm, multi-line flex (`flex-wrap`) — see [Out of scope](#out-of-scope-by-design--use-selenium--cuprite).
+- **Screenshots** — `save_screenshot` (and `full: true` for the whole document) rasters the laid-out page: backgrounds, borders, images, text runs, `overflow` clipping, scroll offsets and `z-index` order. It reads the same boxes every geometry query reads, so a screenshot shows what the driver believes — which makes it useful for seeing what a failing test saw, and unsuitable as a pixel baseline. Not painted: `background-image`, `border-radius`, `opacity`, dashed / dotted borders (drawn solid), SVG, and stacking CONTEXTS (`z-index` is compared globally, not within a parent context). Form controls carry the UA box a browser gives them — border, padding, background, and their own font — so a `<button>` paints as one; what they don't paint is the WIDGET a browser draws inside it (a checkbox's tick, a select's arrow, the value inside a text field).
 - **`within_frame` / `switch_to_frame`** (V8 engine) — each `<iframe>` runs its own scripts in its own per-frame realm; the DSL routes finds, reads, interactions, `evaluate_script`, and navigation into the active frame, nested frames included — the target frame's realm is rebuilt from the fetched document, the top page untouched. QuickJS has no nested browsing context, so `within_frame` raises there.
 - **Multiple windows / tabs** (both engines) — each window is its own Browser + JS VM (own DOM, sessionStorage, history; cookies + localStorage shared). `open_new_window` / `within_window` / `switch_to_window` / `window_opened_by` drive them; JS `window.open` opens a real window, `window.opener` links back, and `postMessage` crosses windows. Only the active window's event loop runs, so a message is delivered when you switch to its window. `target="_blank"` opens with no opener (modern-browser default). `postMessage` carries real structured data (not a lossy JSON hop) — `Map` / `Set` / `Date` / `BigInt` / typed arrays / cyclic graphs all round-trip on V8 — and a `transfer`-list buffer moves **zero-copy** (backing store by token, source detached); only bare `undefined` collapses to `null` (Ruby has no distinct `undefined`). `resize_to` moves the whole viewport: `innerWidth` / `innerHeight`, the `@media` cascade, `matchMedia` change + `resize` events, **and** the layout the geometry surface reports — so a mobile-breakpoint test measures mobile boxes. Each window has its own viewport. `maximize` / `fullscreen` restore the display size (a fixed 1024×768 — that part isn't configurable).
 - **WebSocket + Action Cable** — `new WebSocket(url)` works in-process over the `rack.hijack` socket the Rack app hijacks (hand-rolled RFC6455, including subprotocol negotiation). The real `@rails/actioncable` consumer connects, subscribes, and receives broadcasts, so `turbo_stream_from` live updates work. Constraints: server pushes land at settle (not instant); the Cable app must use the **async / in-process** adapter (a real Redis adapter needs real Redis); binary frames are V8-only (QuickJS corrupts raw bytes across the host boundary — text, hence Action Cable, works on both engines). `EventSource` and Web Workers are likewise real (background reader threads draining at settle).
@@ -252,8 +277,7 @@ Most features run in-process; the notes below are mostly "works, but…", follow
 
 ### Out of scope (by design — use Selenium / Cuprite)
 
-- **Pixel-accurate rendering.** The layout engine (above) is coarse by design: no glyph metrics (text height is estimated from its length), no inline flow, and no flex or grid track sizing. (`resize_to` *does* move the viewport — what's fixed is the display it sits on, 1024×768.) Anything asserting *rendered appearance* — exact text wrapping, a sticky header's pixel offset, whether two boxes overlap by 3px — needs a real browser.
-- **Screenshots.**
+- **Pixel-accurate rendering.** The layout engine (above) is coarse by design: text is measured from the font's advance widths but not SHAPED (no kerning, ligatures or bidi), lines wrap on an estimate rather than the real line-breaking algorithm, and a wrapping flex line (`flex-wrap`) stays on one line. (`resize_to` *does* move the viewport — what's fixed is the display it sits on, 1024×768.) Anything asserting *rendered appearance* — exact text wrapping, a sticky header's pixel offset, whether two boxes overlap by 3px — needs a real browser. That includes comparing screenshots: `save_screenshot` paints from this same coarse layout, so it will differ from a browser's PNG wherever the layout does.
 
 ## Architecture
 

data/lib/capybara/simulated/asset_cache.rb

@@ -35,11 +35,12 @@ module Capybara
     # (sha256(body)), so identical body → identical bytecode falls out
     # naturally.
     #
-    # No Mutex: MRI's Hash `[]`/`[]=` are atomic under the GVL, and Capybara
-    # test suites are serial per-process (parallel_tests forks rather than
-    # threads). A reader briefly racing a `refresh` may see a partial
-    # `stored_at`/`max_age` pair — that just means freshness is computed
-    # against a transient mix, never corrupted.
+    # No Mutex: MRI's Hash `[]`/`[]=` are atomic under the GVL. Concurrent access is
+    # real — async image loads (and keepalive fetches) run `rack_fetch` on background
+    # threads while the main thread fetches too — but every operation here is a single
+    # GVL-atomic Hash read or write of an immutable-once-built Entry, so the worst a
+    # racing reader sees is a partial `stored_at`/`max_age` pair on `refresh`: freshness
+    # computed against a transient mix, never a corrupted structure.
     class AssetCache
       Entry = Struct.new(:status, :headers, :body, :stored_at, :max_age, :no_cache, :immutable, keyword_init: true) do
         # `must-revalidate` (RFC 9111 §5.2.2.2) only forbids reusing a
@@ -83,6 +84,12 @@ module Capybara
       # adds to sprockets-served assets.
       SAFE_VARY_FIELDS = %w[accept-encoding].freeze
 
+      # Immutable entries live for the PROCESS now (they survive every reset), and a long app
+      # suite mints fresh content-addressed URLs per example (Discourse's per-test themes and
+      # colour schemes → new `/theme-javascripts/<sha1>.js` / `/stylesheets/*_<hash>.css`), so
+      # the map is bounded the way `@@asset_src` is: wipe and restart when it fills.
+      MAX_ENTRIES = 4096
+
       def initialize
         @entries = {}
       end
@@ -93,10 +100,15 @@ module Capybara
       # Per-test reset path: keep entries the server marked
       # `Cache-Control: immutable` (declared not to change for their
       # freshness lifetime, so a kept entry can't shadow a later test's
-      # response) and drop everything else, so test-local DB state
-      # reaches the app on the next visit.
+      # response — a real browser keeps them across navigations the same
+      # way) and drop everything else, so test-local DB state reaches the
+      # app on the next visit. This read `reject!` — the exact inverse —
+      # from the day it was written: every immutable asset was re-fetched
+      # once per example (Discourse's `/extra-locales/<sha1>/en/mf.js`
+      # alone is a 41 ms Rails render), while max-age'd non-immutable
+      # responses survived into the next test.
       def clear_volatile
-        @entries.reject! {|_, e| e.immutable }
+        @entries.select! {|_, e| e.immutable }
       end
 
       def store(url, status, headers, body)
@@ -116,6 +128,7 @@ module Capybara
         # Nothing useful to cache without a freshness signal or a
         # validator to revalidate against.
         return if max_age.nil? && h['etag'].nil? && h['last-modified'].nil?
+        @entries.clear if @entries.size >= MAX_ENTRIES && !@entries.key?(url)
         @entries[url] = Entry.new(
           status:    status,
           headers:   h,
@@ -166,10 +179,10 @@ module Capybara
       # at one day. Without this, a response carrying only `Last-Modified` is
       # revalidated on every fetch, which is what a real browser AVOIDS for e.g.
       # Discourse's content-hashed `/assets/*.js` (shipped with `Last-Modified`
-      # and no `Cache-Control`). In the volatile asset cache, cross-visit
-      # staleness is bounded by `clear_volatile` dropping non-immutable entries
-      # per visit; Browser's cross-visit `@@asset_src` cache has its own
-      # argument (it only holds content-stable assets at content-hashed URLs).
+      # and no `Cache-Control`). Here, cross-test staleness is bounded by
+      # `clear_volatile` dropping non-immutable entries at `reset!`; Browser's
+      # cross-session `@@asset_src` cache has its own argument (it only holds
+      # content-stable assets at content-hashed URLs).
       HEURISTIC_FRESHNESS_CAP = 24 * 60 * 60
 
       def heuristic_freshness(headers)