lightpanda 0.0.1 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 585fa864b9a0946daa70334644abf23b36a940fc6a283f5a839e8a3f6febe6c0
-  data.tar.gz: ec6ab4fc8101a82150d9c3861d883825966b280e56d9ab2ca076172ca6676335
+  metadata.gz: 3e916e825715e3501b088ac44898d31bab07985528782eec4da975fd91bfe009
+  data.tar.gz: e290aabcd4c89547255ca9e6090eb4eb5bc22dc4551d3811e9bf7f5f03a3bbe4
 SHA512:
-  metadata.gz: 3d09767f01eaf1f49eb91a5e20fc3b2e79f7569fec2b9d1c4d95eaa3b785f6f8b12a2403e70a48d01ad24f3ac8d27df84ab86519bd7c1ea23ac141c0742c0b2c
-  data.tar.gz: b8515873cfc8a1beb3f4942a4f8b774892dff3fce1fa4d4c8463be42558911480289a6d3e9329da098331e24aaa715dcaf4d990a323e371a69b3e577cb95d9c5
+  metadata.gz: e98478d21b3044f719470994b8ef2d90ca484e26173097829ebc196612e9bad1e8e0da1764abef32d605cf9d38686cc267e5b3a4571ec4cbb23f58189a523fe0
+  data.tar.gz: ecf37c64b4349e8d9a2013979c3bfac94c40952afa5e835711ddbfdf49e8035677cb7b863bd9a2922e9a8d595f0c4cfd3299dfa1996ac115d77eefc714f03fd1

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2025 Marco Roth
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -1,39 +1,213 @@
-# Lightpanda
+# Lightpanda for Ruby
 
-TODO: Delete this and the text below, and describe your gem
+Ruby client for the [Lightpanda](https://lightpanda.io) [open-source](https://github.com/lightpanda-io/browser) headless browser via CDP (Chrome DevTools Protocol).
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/lightpanda`. To experiment with that code, run `bin/console` for an interactive prompt.
+Lightpanda is a fast, lightweight headless browser built for web automation, AI agents, and scraping. This gem provides a high-level Ruby API to control Lightpanda, similar to [Ferrum](https://github.com/rubycdp/ferrum) for Chrome.
 
-## Installation
+> [!NOTE]
+> This gem is experimental. [Lightpanda itself](https://github.com/lightpanda-io/browser?tab=readme-ov-file#status) is in Beta and currently a work in progress. Stability and coverage are improving, but you may still encounter errors or crashes. This gem's API will evolve as Lightpanda matures. See [Limitations](#limitations).
 
-TODO: Replace `UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG` with your gem name right after releasing it to RubyGems.org. Please do not do it earlier due to security reasons. Alternatively, replace this section with instructions to install your gem from git if you don't plan to release to RubyGems.org.
 
-Install the gem and add to the application's Gemfile by executing:
+## Features
 
-```bash
-bundle add UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG
+- High-level browser automation API
+- CDP (Chrome DevTools Protocol) client
+- Capybara driver included
+- Auto-downloads Lightpanda binary if not found
+- Ruby 3.2+
+
+## Installation
+
+Add to your Gemfile:
+
+```ruby
+gem "lightpanda"
 ```
 
-If bundler is not being used to manage dependencies, install the gem by executing:
+Or install directly:
 
 ```bash
-gem install UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG
+gem install lightpanda
 ```
 
+The Lightpanda binary will be automatically downloaded on first use if not found in your `PATH`.
+
 ## Usage
 
-TODO: Write usage instructions here
+### Basic Browser Control
 
-## Development
+**Create a browser instance**
+
+```ruby
+require "lightpanda"
+
+browser = Lightpanda::Browser.new
+```
+
+**Navigate to a page**
+
+```ruby
+browser.go_to("https://example.com")
+```
+
+**Get page info**
+
+```ruby
+browser.current_url  # => "https://example.com/"
+browser.title        # => "Example Domain"
+browser.body         # => "<html>...</html>"
+```
+
+**Evaluate JavaScript**
+
+```ruby
+browser.evaluate("1 + 1")                                    # => 2
+browser.evaluate("document.querySelector('h1').textContent") # => "Example Domain"
+```
+
+**Execute JavaScript (no return value)**
+
+```ruby
+browser.execute("console.log('Hello from Lightpanda!')")
+```
+
+**Send raw CDP commands**
+
+```ruby
+browser.command("Browser.getVersion")
+# => {"protocolVersion"=>"1.3", "product"=>"Chrome/124.0.6367.29", ...}
+```
+
+**Clean up**
+
+```ruby
+browser.quit
+```
+
+### Configuration Options
+
+```ruby
+browser = Lightpanda.new(
+  host: "127.0.0.1",        # CDP server host
+  port: 9222,               # CDP server port
+  timeout: 5,               # Command timeout in seconds
+  process_timeout: 10,      # Process startup timeout
+  window_size: [1024, 768],
+  browser_path: "/path/to/lightpanda"  # Custom binary path
+)
+```
+
+### Binary Management
+
+**Get binary path (downloads if needed)**
+
+```ruby
+Lightpanda::Binary.path  # => "/Users/you/.cache/lightpanda/lightpanda"
+```
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+**Get version**
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+```ruby
+Lightpanda::Binary.version  # => "7c976209"
+```
+
+**Run arbitrary commands**
+
+```ruby
+result = Lightpanda::Binary.run("--help")
+result.stdout   # => ""
+result.stderr   # => "usage: lightpanda command [options] [URL]..."
+result.success? # => false (help exits with 1)
+result.output   # => returns stderr if stdout empty
+```
+
+**Fetch a URL directly (no browser instance needed)**
+
+```ruby
+html = Lightpanda::Binary.fetch("https://example.com")
+# => "<!DOCTYPE html><html>..."
+```
+
+### Global Configuration
+
+```ruby
+Lightpanda.configure do |config|
+  config.binary_path = "/path/to/lightpanda"
+end
+```
+
+If not explicitly set, the binary path is auto-discovered on first use and cached for subsequent calls. The discovery order is:
+
+1. `LIGHTPANDA_PATH` environment variable
+2. `lightpanda` executable in your `PATH`
+3. Auto-download to `~/.cache/lightpanda/lightpanda` (or `$XDG_CACHE_HOME/lightpanda/lightpanda`)
+
+### Capybara Integration
+
+**Basic usage**
+
+```ruby
+require "lightpanda/capybara"
+
+Capybara.default_driver = :lightpanda
+
+visit "https://example.com"
+find("h1").text  # => "Example Domain"
+all("p").count   # => 2
+```
 
-## Contributing
+**Configuration**
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/marcoroth/lightpanda. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/marcoroth/lightpanda/blob/main/CODE_OF_CONDUCT.md).
+```ruby
+Lightpanda::Capybara.configure do |config|
+  config.host = "127.0.0.1"
+  config.port = 9222
+  config.timeout = 5
+end
+```
+
+**In tests**
+
+```ruby
+class FeatureTest < Minitest::Spec
+  include Capybara::DSL
+
+  def setup
+    Capybara.default_driver = :lightpanda
+  end
+
+  def teardown
+    Capybara.reset_sessions!
+  end
+
+  it "shows the homepage" do
+    visit "https://example.com"
+    assert find("h1").text == "Example Domain"
+  end
+end
+```
+
+## Environment Variables
+
+- `LIGHTPANDA_PATH` - Custom path to Lightpanda binary
+- `LIGHTPANDA_DEFAULT_TIMEOUT` - Default command timeout (default: 5)
+- `LIGHTPANDA_PROCESS_TIMEOUT` - Process startup timeout (default: 10)
+
+## Limitations
+
+Lightpanda is a lightweight browser with some limitations compared to Chrome:
+
+- Single browser context only (no incognito/multi-context)
+- No XPath support (`XPathResult` not implemented)
+- Limited CDP command coverage
+
+## Development
+
+```bash
+bundle install
+bundle exec minitest
+```
 
-## Code of Conduct
+## License
 
-Everyone interacting in the Lightpanda project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/marcoroth/lightpanda/blob/main/CODE_OF_CONDUCT.md).
+MIT License. See [LICENSE.txt](LICENSE.txt).

data/exe/lightpanda

@@ -0,0 +1,8 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require_relative "../lib/lightpanda"
+
+args = ARGV.empty? ? ["help"] : ARGV
+
+Lightpanda::Binary.exec(*args)

data/lib/lightpanda/binary.rb

@@ -0,0 +1,180 @@
+# frozen_string_literal: true
+
+require "fileutils"
+require "net/http"
+require "open3"
+require "rbconfig"
+require "uri"
+
+module Lightpanda
+  class Binary
+    Result = Struct.new(:stdout, :stderr, :status, keyword_init: true) do
+      def success?
+        status.success?
+      end
+
+      def exit_code
+        status.exitstatus
+      end
+
+      def output
+        stdout.empty? ? stderr : stdout
+      end
+    end
+
+    GITHUB_RELEASE_URL = "https://github.com/lightpanda-io/browser/releases/download/nightly"
+
+    PLATFORMS = {
+      ["x86_64", "linux"] => "lightpanda-x86_64-linux",
+      ["aarch64", "darwin"] => "lightpanda-aarch64-macos",
+      ["arm64", "darwin"] => "lightpanda-aarch64-macos",
+    }.freeze
+
+    class << self
+      def path
+        Lightpanda.configuration.binary_path ||= find_or_download
+      end
+
+      def find_or_download
+        find || download
+      end
+
+      def run(*)
+        stdout, stderr, status = Open3.capture3(path, *)
+
+        Result.new(stdout: stdout, stderr: stderr, status: status)
+      rescue Errno::ENOENT
+        raise BinaryNotFoundError, "Lightpanda binary not found"
+      end
+
+      def exec(*)
+        Kernel.exec(path, *)
+      end
+
+      def fetch(url)
+        result = run("fetch", "--dump", url)
+        raise BinaryError, result.stderr unless result.success?
+
+        result.stdout
+      end
+
+      def version
+        result = run("version")
+        result.output.strip
+      end
+
+      def find
+        env_path = ENV.fetch("LIGHTPANDA_PATH", nil)
+        return env_path if env_path && File.executable?(env_path)
+
+        path_binary = find_in_path
+        return path_binary if path_binary
+
+        default_path = default_binary_path
+        return default_path if File.executable?(default_path)
+
+        nil
+      end
+
+      def download
+        binary_name = platform_binary
+        url = "#{GITHUB_RELEASE_URL}/#{binary_name}"
+        destination = default_binary_path
+
+        FileUtils.mkdir_p(File.dirname(destination))
+
+        download_file(url, destination)
+        FileUtils.chmod(0o755, destination)
+
+        destination
+      end
+
+      def platform_binary
+        arch = normalize_arch(RbConfig::CONFIG["host_cpu"])
+        os = normalize_os(RbConfig::CONFIG["host_os"])
+
+        PLATFORMS[[arch, os]] || raise(UnsupportedPlatformError, "Unsupported platform: #{arch}-#{os}")
+      end
+
+      def default_binary_path
+        cache_dir = ENV.fetch("XDG_CACHE_HOME") { File.expand_path("~/.cache") }
+
+        File.join(cache_dir, "lightpanda", "lightpanda")
+      end
+
+      private
+
+      def find_in_path
+        ENV["PATH"].to_s.split(File::PATH_SEPARATOR).each do |dir|
+          path = File.join(dir, "lightpanda")
+
+          return path if File.executable?(path) && native_binary?(path)
+        end
+
+        nil
+      end
+
+      def native_binary?(path)
+        header = File.binread(path, 4)
+
+        return true if elf_binary?(header)
+        return true if mach_o_binary?(header)
+
+        false
+      rescue StandardError
+        false
+      end
+
+      def elf_binary?(header)
+        header.start_with?("\x7FELF")
+      end
+
+      def mach_o_binary?(header)
+        header.start_with?("\xCF\xFA\xED\xFE")
+      end
+
+      def normalize_arch(arch)
+        case arch
+        when /x86_64|amd64/i then "x86_64"
+        when /aarch64|arm64/i then "aarch64"
+        else arch
+        end
+      end
+
+      def normalize_os(os)
+        case os
+        when /darwin|mac/i then "darwin"
+        when /linux/i then "linux"
+        else os
+        end
+      end
+
+      def download_file(url, destination)
+        uri = URI.parse(url)
+
+        follow_redirects(uri, destination)
+      end
+
+      def follow_redirects(uri, destination, limit = 10)
+        raise BinaryNotFoundError, "Too many redirects" if limit.zero?
+
+        Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https") do |http|
+          request = Net::HTTP::Get.new(uri)
+
+          http.request(request) do |response|
+            case response
+            when Net::HTTPSuccess
+              File.open(destination, "wb") do |file|
+                response.read_body { |chunk| file.write(chunk) }
+              end
+            when Net::HTTPRedirection
+              follow_redirects(URI.parse(response["location"]), destination, limit - 1)
+            else
+              raise BinaryNotFoundError, "Failed to download binary: #{response.code} #{response.message}"
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/lightpanda/browser.rb

@@ -0,0 +1,212 @@
+# frozen_string_literal: true
+
+require "forwardable"
+
+module Lightpanda
+  class Browser
+    extend Forwardable
+
+    attr_reader :options, :process, :client, :target_id, :session_id
+
+    delegate [:on, :off] => :client
+
+    def initialize(options = {})
+      @options = Options.new(options)
+      @process = nil
+      @client = nil
+      @target_id = nil
+      @session_id = nil
+      @started = false
+      @page_events_enabled = false
+
+      start
+    end
+
+    def start
+      return if @started
+
+      if @options.ws_url?
+        @client = Client.new(@options.ws_url, @options)
+      else
+        @process = Process.new(@options)
+        @process.start
+        @client = Client.new(@process.ws_url, @options)
+      end
+
+      create_page
+
+      @started = true
+    end
+
+    def create_page
+      result = @client.command("Target.createTarget", { url: "about:blank" })
+      @target_id = result["targetId"]
+
+      attach_result = @client.command("Target.attachToTarget", { targetId: @target_id, flatten: true })
+      @session_id = attach_result["sessionId"]
+    end
+
+    def restart
+      quit
+      start
+    end
+
+    def quit
+      @client&.close
+      @process&.stop
+      @client = nil
+      @process = nil
+      @started = false
+    end
+
+    def command(method, **params)
+      @client.command(method, params)
+    end
+
+    def page_command(method, **params)
+      @client.command(method, params, session_id: @session_id)
+    end
+
+    def go_to(url, wait: true)
+      enable_page_events
+
+      if wait
+        loaded = Concurrent::Event.new
+
+        handler = proc { loaded.set }
+        @client.on("Page.loadEventFired", &handler)
+
+        result = page_command("Page.navigate", url: url)
+
+        loaded.wait(@options.timeout)
+
+        @client.off("Page.loadEventFired", handler)
+
+        result
+      else
+        page_command("Page.navigate", url: url)
+      end
+    end
+    alias goto go_to
+
+    def enable_page_events
+      return if @page_events_enabled
+
+      page_command("Page.enable")
+      @page_events_enabled = true
+    end
+
+    def back
+      page_command("Page.navigateToHistoryEntry", entryId: current_entry_id - 1)
+    end
+
+    def forward
+      page_command("Page.navigateToHistoryEntry", entryId: current_entry_id + 1)
+    end
+
+    def refresh
+      page_command("Page.reload")
+    end
+    alias reload refresh
+
+    def current_url
+      evaluate("window.location.href")
+    end
+
+    def title
+      evaluate("document.title")
+    end
+
+    def body
+      evaluate("document.documentElement.outerHTML")
+    end
+    alias html body
+
+    def evaluate(expression)
+      response = page_command("Runtime.evaluate", expression: expression, returnByValue: true, awaitPromise: true)
+
+      handle_evaluate_response(response)
+    end
+
+    def execute(expression)
+      page_command("Runtime.evaluate", expression: expression, returnByValue: false, awaitPromise: false)
+      nil
+    end
+
+    def css(selector)
+      node_ids = page_command("DOM.querySelectorAll", nodeId: document_node_id, selector: selector)
+      node_ids["nodeIds"] || []
+    end
+
+    def at_css(selector)
+      result = page_command("DOM.querySelector", nodeId: document_node_id, selector: selector)
+
+      result["nodeId"]
+    end
+
+    def screenshot(path: nil, format: :png, quality: nil, full_page: false, encoding: :binary)
+      params = { format: format.to_s }
+      params[:quality] = quality if quality && format == :jpeg
+
+      if full_page
+        metrics = page_command("Page.getLayoutMetrics")
+        content_size = metrics["contentSize"]
+
+        params[:clip] = {
+          x: 0,
+          y: 0,
+          width: content_size["width"],
+          height: content_size["height"],
+          scale: 1,
+        }
+      end
+
+      result = page_command("Page.captureScreenshot", **params)
+      data = result["data"]
+
+      if encoding == :base64
+        data
+      else
+        decoded = Base64.decode64(data)
+
+        if path
+          File.binwrite(path, decoded)
+          path
+        else
+          decoded
+        end
+      end
+    end
+
+    def network
+      @network ||= Network.new(self)
+    end
+
+    def cookies
+      @cookies ||= Cookies.new(self)
+    end
+
+    private
+
+    def document_node_id
+      result = page_command("DOM.getDocument")
+
+      result.dig("root", "nodeId")
+    end
+
+    def current_entry_id
+      result = page_command("Page.getNavigationHistory")
+
+      result["currentIndex"]
+    end
+
+    def handle_evaluate_response(response)
+      raise JavaScriptError, response if response["exceptionDetails"]
+
+      result = response["result"]
+      return nil if result["type"] == "undefined"
+
+      result["value"]
+    end
+  end
+end