capybara-lightpanda 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: c46e211cf39dfb4b533c59ec88106fd9d4c72d59ed8ef11d99af0d9488885f9e
+  data.tar.gz: 00f22e0a391ccc5c015291d72fa2d53f1d10062c0413f1fb62298a13a7006307
+SHA512:
+  metadata.gz: e3d08ffeb41acac97492d9c1cdf0aaca8499487b38afa6f1b8a1598f5c060cead1f0b4d5572f23ffeee0a49afc555f790a6e358c88816e143494619bd1f88128
+  data.tar.gz: 7a2bfcafc663cba5614bd4385db9957f68d58255f6cfe4ee4a3090a447a5302f2a09db6dc2bcea9ce56795cb2e8739d1f440143e0c8da4d393e7078d76d9b229

data/CHANGELOG.md

@@ -0,0 +1,50 @@
+# Changelog
+
+## [0.1.0] - 2026-04-27
+
+Initial release. Capybara driver for the [Lightpanda](https://github.com/lightpanda-io/browser) headless browser.
+
+### Driver
+
+- Capybara driver registered as `:lightpanda`
+- Auto-downloads the Lightpanda binary on first use; binary version exposed via `Browser#binary_version`
+- Reliable navigation: `Page.loadEventFired` + `document.readyState` polling fallback
+- Crash recovery: detects WebSocket disconnects on heavy SPAs and reconnects transparently
+
+### CDP client (no external browser-client gem)
+
+- WebSocket transport, command dispatch, process management — all in-gem
+- `Capybara::Lightpanda::Utils::Event` (iteration-counted `Concurrent::Event` wrapper, Ferrum-style)
+
+### Nodes
+
+- Identity via CDP remote object IDs (`Runtime.callFunctionOn`)
+- `Node#[]` resolves URLs for `src`/`href`/`action` and returns live property values for boolean attributes
+- `Node#rect`, `Node#obscured?`, `Node#shadow_root`, `Node#moving?`, `Node#wait_for_stop_moving`
+- Whitespace-normalized `Node#text` / `#all_text` (works around Lightpanda's `textContent` divergence from Chrome)
+
+### JavaScript polyfills (auto-injected via `Page.addScriptToEvaluateOnNewDocument`)
+
+- XPath 1.0 evaluator (`document.evaluate` + `XPathResult` shim — Lightpanda doesn't implement XPath natively)
+- `#id` selector rewriter for `querySelector{,All}` (Turbo Drive snapshot+swap workaround)
+- `requestSubmit` polyfill
+- Turbo activity tracking sentinels for event-driven `wait_for_turbo` / `wait_for_idle`
+- `fetch()` + body-swap submit pipeline (works around Lightpanda's no-op `form.submit()`)
+
+### Cookies
+
+- Typed `Cookie` wrapper (Ferrum-style: `name`, `value`, `domain`, `httponly?`, `secure?`, `same_site`, `expires`)
+- `Cookies#store` / `Cookies#load` — YAML round-trip
+- Cross-origin `Cookies#clear` sweep via `visited_origins` tracking (works around `Network.clearBrowserCookies` returning `InvalidParams` on current Lightpanda nightly)
+
+### Frames & modals
+
+- Frame switching via `contentDocument` scoping; XPath polyfill inherited
+- Frame metadata view populated from CDP frame events (`Frame#parent_id`, etc.)
+- Modal capture via `Page.javascriptDialogOpening`. `accept_modal(:alert)` and `dismiss_modal(:confirm|:prompt)` work; `accept_modal(:confirm|:prompt)` cannot override Lightpanda's auto-dismiss
+
+### Tested against
+
+- Capybara `>= 3.0, < 5` — runs Capybara's shared spec suite
+- Ruby 3.3 and 4.0
+- Lightpanda nightly (verified against `1.0.0-nightly.5812+b3257754`, 2026-04-26)

data/LICENSE.txt

@@ -0,0 +1,27 @@
+MIT License
+
+Copyright (c) 2026 Navid Emad
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+
+---
+
+Portions of this gem adapt patterns from Ferrum, Cuprite, and lightpanda-ruby,
+all distributed under the MIT License. See NOTICE.md for the original copyright
+notices and license terms.

data/NOTICE.md

@@ -0,0 +1,101 @@
+# Third-Party Notices
+
+`capybara-lightpanda` adapts patterns and conventions from the following
+MIT-licensed projects. Their copyright notices and license terms are reproduced
+below in accordance with the MIT License.
+
+No verbatim source files from these projects are included; the work referenced
+here covers API shape, error-handling conventions, and architectural patterns
+re-implemented for Lightpanda's CDP surface (notably in `Cookies`, `Frame`,
+`Node`, `Driver`, and `Utils::Event`). Code-level pointers to the inspiring
+upstream patterns are kept as comments in the relevant files.
+
+---
+
+## Ferrum
+
+<https://github.com/rubycdp/ferrum>
+
+```
+MIT License
+
+Copyright (c) 2019-2026 Dmitry Vorotilin
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+```
+
+---
+
+## Cuprite
+
+<https://github.com/rubycdp/cuprite>
+
+```
+MIT License
+
+Copyright (c) 2018-2022 Dmitry Vorotilin
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
+```
+
+---
+
+## lightpanda-ruby
+
+<https://github.com/marcoroth/lightpanda-ruby>
+
+```
+The MIT License (MIT)
+
+Copyright (c) 2025 Marco Roth
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.
+```

data/README.md

@@ -0,0 +1,215 @@
+# Capybara::Lightpanda
+
+A [Capybara](https://github.com/teamcapybara/capybara) driver for [Lightpanda](https://lightpanda.io/), the fast headless browser built in Zig.
+
+This gem provides a **self-contained, production-ready** Capybara driver with a built-in CDP client. No external browser-client gem required — just install and go:
+
+- **Reliable navigation** — falls back to `document.readyState` polling when `Page.loadEventFired` doesn't fire (a known Lightpanda limitation on pages with complex JS)
+- **XPath polyfill** — auto-injected after each navigation so Capybara's internal XPath selectors work (`find`, `click_on`, `fill_in`, `assert_selector`, etc.)
+- **Cookie management** — `set_cookie`, `clear_cookies`, `remove_cookie` on the driver + graceful fallback when `Network.clearBrowserCookies` crashes the CDP connection
+- **Drop-in Capybara integration** — registers a `:lightpanda` driver, configure and go
+
+## Architecture
+
+Similar to how [Cuprite](https://github.com/rubycdp/cuprite) builds on [Ferrum](https://github.com/rubycdp/ferrum), but as a single gem:
+
+```
+Capybara  →  capybara-lightpanda (driver + CDP client)  →  Lightpanda browser
+```
+
+## Installation
+
+### 1. Install the Lightpanda browser
+
+```bash
+# macOS
+brew install lightpanda-io/lightpanda/lightpanda
+
+# Linux (Debian/Ubuntu) — see https://lightpanda.io/docs/
+```
+
+### 2. Add the gem
+
+```ruby
+# Gemfile
+group :test do
+  gem "capybara-lightpanda"
+end
+```
+
+```bash
+bundle install
+```
+
+## Usage
+
+### Basic setup
+
+```ruby
+# test/support/capybara.rb or spec/support/capybara.rb
+require "capybara-lightpanda"
+
+Capybara::Lightpanda.configure do |config|
+  config.host = "127.0.0.1"
+  config.port = 9222
+  config.timeout = 15
+  config.browser_path = "/usr/local/bin/lightpanda" # optional, auto-detected
+end
+
+Capybara.default_driver = :lightpanda
+Capybara.javascript_driver = :lightpanda
+```
+
+### Dual-driver setup (recommended)
+
+Run most tests with Chrome, use Lightpanda for fast DOM-only tests:
+
+```ruby
+if ENV["BROWSER"] == "lightpanda"
+  require "capybara-lightpanda"
+
+  Capybara::Lightpanda.configure do |config|
+    config.timeout = 15
+  end
+
+  Capybara.default_driver = :lightpanda
+  Capybara.javascript_driver = :lightpanda
+else
+  # Your existing Chrome/Cuprite setup
+  Capybara.default_driver = :cuprite
+end
+```
+
+```bash
+# Run with Lightpanda
+BROWSER=lightpanda bundle exec rails test test/system/
+
+# Run with Chrome (default)
+bundle exec rails test test/system/
+```
+
+### Setting cookies (e.g. login helper)
+
+```ruby
+class ApplicationSystemTestCase < ActionDispatch::SystemTestCase
+  def login_as(user)
+    session = user.sessions.first_or_create!
+    cookie_jar = ActionDispatch::TestRequest.create({ "REQUEST_METHOD" => "GET" }).cookie_jar
+    cookie_jar.signed[:session_id] = { value: session.id }
+
+    page.driver.set_cookie(
+      "session_id",
+      cookie_jar[:session_id],
+      domain: "127.0.0.1",
+      httpOnly: true,
+      secure: false,
+    )
+  end
+end
+```
+
+## What works
+
+- Navigation (`visit`, `click_link`, `go_back`, `go_forward`, `refresh`)
+- JavaScript execution (V8 engine) — `evaluate_script`, `execute_script`, `evaluate_async_script`
+- Forms — `fill_in`, `click_button`, `select`, `choose`, `check`, `uncheck`
+- Finding — `find`, `all`, `within`, CSS and XPath selectors
+- Matchers — `assert_selector`, `assert_text`, `assert_current_path`, `has_field?`, `has_select?`
+- Cookies — set/get/clear/remove via CDP
+- Frames — `within_frame`, scoped finding
+- Keyboard — `send_keys` with modifiers and special keys
+- Network — traffic tracking, custom headers, idle waiting
+
+### Turbo Rails support
+
+The gem handles Turbo-enabled Rails apps transparently:
+
+| Feature | Status | How |
+|---------|--------|-----|
+| **Turbo Frames** | Works natively | Lazy-loading (`src=`), scoped link navigation |
+| **Turbo Drive** | Auto-disabled | Gem disables Drive (body replacement fails in Lightpanda) — standard link navigation restored |
+| **Form submission** | Auto-handled | When Turbo is present, forms submit via `fetch()` + `document.write()` to bypass Turbo's interception |
+| **Turbo Streams** | Not supported | Depends on Turbo's fetch pipeline which Lightpanda can't render |
+
+**Root cause**: Lightpanda's `document.body` is read-only — Turbo Drive's body replacement and frame form responses can't be applied. The gem works around this automatically.
+
+## Known limitations
+
+These are Lightpanda browser limitations, not driver limitations:
+
+| Feature | Status |
+|---------|--------|
+| Screenshots | Not supported (no rendering engine) |
+| `window.getComputedStyle()` | Returns defaults (no CSS engine) |
+| `scroll_to`, `resize` | No layout engine |
+| Complex Stimulus controllers | Some may not execute fully |
+| XPath axes/functions | Polyfill covers ~80% of Capybara usage |
+| File uploads | Not yet supported |
+| Turbo Streams | Not supported (Turbo's fetch-then-render pipeline) |
+
+## Benchmark
+
+Tested on a Rails 8.1 app (Turbo + Stimulus), 24 DOM-only tests:
+
+| Driver | Tests | Time | Speed |
+|--------|-------|------|-------|
+| **Lightpanda** | 24/24 pass | 6.89s | 3.48 tests/s |
+| **Chrome** | 24/24 pass | 7.09s | 3.38 tests/s |
+
+Lightpanda's advantage is expected to grow on larger suites due to faster startup and lower memory usage.
+
+## Configuration
+
+```ruby
+Capybara::Lightpanda.configure do |config|
+  config.host = "127.0.0.1"       # Lightpanda bind host
+  config.port = 9222              # Lightpanda CDP port
+  config.timeout = 15             # Navigation/command timeout (seconds)
+  config.process_timeout = 10     # Browser process startup timeout
+  config.browser_path = nil       # Path to lightpanda binary (auto-detected)
+end
+```
+
+### Dynamic port (parallel tests)
+
+```ruby
+def available_port
+  server = TCPServer.new("127.0.0.1", 0)
+  port = server.addr[1]
+  server.close
+  port
+end
+
+Capybara::Lightpanda.configure do |config|
+  config.port = ENV.fetch("LIGHTPANDA_PORT", available_port).to_i
+end
+```
+
+## How it works
+
+| Component | Description |
+|-----------|-------------|
+| `Browser` | High-level API with readyState polling fallback when `Page.loadEventFired` never fires |
+| `Cookies` | Catches `BrowserError` from unsupported `Network.clearBrowserCookies`, deletes cookies individually |
+| `XPathPolyfill` | Provides `document.evaluate` + `XPathResult` shim for Capybara's XPath selectors |
+| `Client` | CDP command dispatch over WebSocket with timeout and event subscription |
+| `Driver` | Complete Capybara driver with `set_cookie`, `clear_cookies`, `remove_cookie` |
+| `Node` | DOM interactions via JavaScript evaluation |
+
+## Credits
+
+- [Lightpanda](https://lightpanda.io/) — the headless browser
+- [Capybara](https://github.com/teamcapybara/capybara) — the test framework
+- Inspired by the [Cuprite](https://github.com/rubycdp/cuprite) / [Ferrum](https://github.com/rubycdp/ferrum) architecture and [`lightpanda-ruby`](https://github.com/marcoroth/lightpanda-ruby)
+
+Patterns adapted from these MIT-licensed projects (cookies API, frame switching,
+node call/error conventions, retry/event utilities) are acknowledged with the
+original copyright notices in [NOTICE.md](NOTICE.md).
+
+## Contributing
+
+Bug reports and pull requests are welcome on [GitHub](https://github.com/navidemad/capybara-lightpanda).
+
+## License
+
+[MIT License](LICENSE.txt)

data/lib/capybara/lightpanda/binary.rb

@@ -0,0 +1,190 @@
+# frozen_string_literal: true
+
+require "fileutils"
+require "net/http"
+require "open3"
+require "rbconfig"
+require "uri"
+
+module Capybara
+  module Lightpanda
+    class Binary
+      Result = Struct.new(:stdout, :stderr, :status) do
+        def success?
+          status.success?
+        end
+
+        def exit_code
+          status.exitstatus
+        end
+
+        def output
+          stdout.empty? ? stderr : stdout
+        end
+      end
+
+      GITHUB_RELEASE_URL = "https://github.com/lightpanda-io/browser/releases/download/nightly"
+
+      PLATFORMS = {
+        %w[x86_64 linux] => "lightpanda-x86_64-linux",
+        %w[aarch64 darwin] => "lightpanda-aarch64-macos",
+        %w[arm64 darwin] => "lightpanda-aarch64-macos",
+      }.freeze
+
+      class << self
+        def path
+          @path ||= find_or_download
+        end
+
+        def find_or_download
+          find || download
+        end
+
+        # Always return the nightly binary, downloading if missing or stale.
+        # Skips PATH lookup so the system binary is never used.
+        def ensure_nightly(max_age: 86_400)
+          path = default_binary_path
+          download if !File.executable?(path) || (Time.now - File.mtime(path)) > max_age
+          path
+        end
+
+        def run(*)
+          stdout, stderr, status = Open3.capture3(path, *)
+
+          Result.new(stdout: stdout, stderr: stderr, status: status)
+        rescue Errno::ENOENT
+          raise BinaryNotFoundError, "Lightpanda binary not found"
+        end
+
+        def exec(*)
+          Kernel.exec(path, *)
+        end
+
+        def fetch(url)
+          result = run("fetch", "--dump", url)
+          raise BinaryError, result.stderr unless result.success?
+
+          result.stdout
+        end
+
+        def version
+          result = run("version")
+          result.output.strip
+        end
+
+        def find
+          env_path = ENV.fetch("LIGHTPANDA_PATH", nil)
+          return env_path if env_path && File.executable?(env_path)
+
+          path_binary = find_in_path
+          return path_binary if path_binary
+
+          default_path = default_binary_path
+          return default_path if File.executable?(default_path)
+
+          nil
+        end
+
+        def download
+          binary_name = platform_binary
+          url = "#{GITHUB_RELEASE_URL}/#{binary_name}"
+          destination = default_binary_path
+
+          FileUtils.mkdir_p(File.dirname(destination))
+
+          download_file(url, destination)
+          FileUtils.chmod(0o755, destination)
+
+          destination
+        end
+
+        def platform_binary
+          arch = normalize_arch(RbConfig::CONFIG["host_cpu"])
+          os = normalize_os(RbConfig::CONFIG["host_os"])
+
+          PLATFORMS[[arch, os]] || raise(UnsupportedPlatformError, "Unsupported platform: #{arch}-#{os}")
+        end
+
+        def default_binary_path
+          cache_dir = ENV.fetch("XDG_CACHE_HOME") { File.expand_path("~/.cache") }
+
+          File.join(cache_dir, "lightpanda", "lightpanda")
+        end
+
+        private
+
+        def find_in_path
+          ENV["PATH"].to_s.split(File::PATH_SEPARATOR).each do |dir|
+            path = File.join(dir, "lightpanda")
+
+            return path if File.executable?(path) && native_binary?(path)
+          end
+
+          nil
+        end
+
+        def native_binary?(path)
+          header = File.binread(path, 4)
+
+          return true if elf_binary?(header)
+          return true if mach_o_binary?(header)
+
+          false
+        rescue StandardError
+          false
+        end
+
+        def elf_binary?(header)
+          header.start_with?("\x7FELF")
+        end
+
+        def mach_o_binary?(header)
+          header.start_with?("\xCF\xFA\xED\xFE")
+        end
+
+        def normalize_arch(arch)
+          case arch
+          when /x86_64|amd64/i then "x86_64"
+          when /aarch64|arm64/i then "aarch64"
+          else arch
+          end
+        end
+
+        def normalize_os(os)
+          case os
+          when /darwin|mac/i then "darwin"
+          when /linux/i then "linux"
+          else os
+          end
+        end
+
+        def download_file(url, destination)
+          uri = URI.parse(url)
+
+          follow_redirects(uri, destination)
+        end
+
+        def follow_redirects(uri, destination, limit = 10)
+          raise BinaryNotFoundError, "Too many redirects" if limit.zero?
+
+          Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https") do |http|
+            request = Net::HTTP::Get.new(uri)
+
+            http.request(request) do |response|
+              case response
+              when Net::HTTPSuccess
+                File.open(destination, "wb") do |file|
+                  response.read_body { |chunk| file.write(chunk) }
+                end
+              when Net::HTTPRedirection
+                follow_redirects(URI.parse(response["location"]), destination, limit - 1)
+              else
+                raise BinaryNotFoundError, "Failed to download binary: #{response.code} #{response.message}"
+              end
+            end
+          end
+        end
+      end
+    end
+  end
+end