lightpanda 0.0.1 → 0.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 585fa864b9a0946daa70334644abf23b36a940fc6a283f5a839e8a3f6febe6c0
-  data.tar.gz: ec6ab4fc8101a82150d9c3861d883825966b280e56d9ab2ca076172ca6676335
+  metadata.gz: 591cd0a182a0ce6cfababbdfa954162a4ac47c0968f9a84663d34fb0acb71c20
+  data.tar.gz: e4783c19ac142387b2d3eb9144115bae6d252a9d7ca4c3c944d5692aeb8bfb12
 SHA512:
-  metadata.gz: 3d09767f01eaf1f49eb91a5e20fc3b2e79f7569fec2b9d1c4d95eaa3b785f6f8b12a2403e70a48d01ad24f3ac8d27df84ab86519bd7c1ea23ac141c0742c0b2c
-  data.tar.gz: b8515873cfc8a1beb3f4942a4f8b774892dff3fce1fa4d4c8463be42558911480289a6d3e9329da098331e24aaa715dcaf4d990a323e371a69b3e577cb95d9c5
+  metadata.gz: 1aec3d07fd8fb537e82c1e1c938b6008727d90f8fca579caf0d2120501d9acb2659e4bb177b3568e43bbeb2d27f83084bf3a9e61673aa89e11830ff7409c152b
+  data.tar.gz: 300e97b99c1b1f5524c53bec47cded120cde15c680d260da0be7472e447f8e15c939c6068172b36ef408d78bde75a8c591ee633863bb5393235611a0ae0e1eb5

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2025 Marco Roth
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -1,39 +1,195 @@
 # Lightpanda
 
-TODO: Delete this and the text below, and describe your gem
+Ruby client for the [Lightpanda](https://lightpanda.io) headless browser via CDP (Chrome DevTools Protocol).
 
-Welcome to your new gem! In this directory, you'll find the files you need to be able to package up your Ruby library into a gem. Put your Ruby code in the file `lib/lightpanda`. To experiment with that code, run `bin/console` for an interactive prompt.
+Lightpanda is a fast, lightweight headless browser built for web automation, AI agents, and scraping. This gem provides a high-level Ruby API to control Lightpanda, similar to [Ferrum](https://github.com/rubycdp/ferrum) for Chrome.
 
-## Installation
+## Features
+
+- High-level browser automation API
+- CDP (Chrome DevTools Protocol) client
+- Capybara driver included
+- Auto-downloads Lightpanda binary if not found
+- Ruby 3.1+
 
-TODO: Replace `UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG` with your gem name right after releasing it to RubyGems.org. Please do not do it earlier due to security reasons. Alternatively, replace this section with instructions to install your gem from git if you don't plan to release to RubyGems.org.
+## Installation
 
-Install the gem and add to the application's Gemfile by executing:
+Add to your Gemfile:
 
-```bash
-bundle add UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG
+```ruby
+gem "lightpanda"
 ```
 
-If bundler is not being used to manage dependencies, install the gem by executing:
+Or install directly:
 
 ```bash
-gem install UPDATE_WITH_YOUR_GEM_NAME_IMMEDIATELY_AFTER_RELEASE_TO_RUBYGEMS_ORG
+gem install lightpanda
 ```
 
+The Lightpanda binary will be automatically downloaded on first use if not found in your `PATH`.
+
 ## Usage
 
-TODO: Write usage instructions here
+### Basic Browser Control
 
-## Development
+**Create a browser instance**
+
+```ruby
+require "lightpanda"
+
+browser = Lightpanda::Browser.new
+```
+
+**Navigate to a page**
+
+```ruby
+browser.go_to("https://example.com")
+```
+
+**Get page info**
+
+```ruby
+browser.current_url  # => "https://example.com/"
+browser.title        # => "Example Domain"
+browser.body         # => "<html>...</html>"
+```
+
+**Evaluate JavaScript**
 
-After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake test` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+```ruby
+browser.evaluate("1 + 1")                                    # => 2
+browser.evaluate("document.querySelector('h1').textContent") # => "Example Domain"
+```
+
+**Execute JavaScript (no return value)**
+
+```ruby
+browser.execute("console.log('Hello from Lightpanda!')")
+```
+
+**Send raw CDP commands**
+
+```ruby
+browser.command("Browser.getVersion")
+# => {"protocolVersion"=>"1.3", "product"=>"Chrome/124.0.6367.29", ...}
+```
+
+**Clean up**
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+```ruby
+browser.quit
+```
+
+### Configuration Options
+
+```ruby
+browser = Lightpanda.new(
+  host: "127.0.0.1",        # CDP server host
+  port: 9222,               # CDP server port
+  timeout: 5,               # Command timeout in seconds
+  process_timeout: 10,      # Process startup timeout
+  window_size: [1024, 768],
+  browser_path: "/path/to/lightpanda"  # Custom binary path
+)
+```
+
+### Binary Management
 
-## Contributing
+**Get binary path (downloads if needed)**
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/marcoroth/lightpanda. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/marcoroth/lightpanda/blob/main/CODE_OF_CONDUCT.md).
+```ruby
+Lightpanda::Binary.path  # => "/Users/you/.cache/lightpanda/lightpanda"
+```
+
+**Get version**
+
+```ruby
+Lightpanda::Binary.version  # => "7c976209"
+```
+
+**Run arbitrary commands**
+
+```ruby
+result = Lightpanda::Binary.run("--help")
+result.stdout   # => ""
+result.stderr   # => "usage: lightpanda command [options] [URL]..."
+result.success? # => false (help exits with 1)
+result.output   # => returns stderr if stdout empty
+```
+
+**Fetch a URL directly (no browser instance needed)**
+
+```ruby
+html = Lightpanda::Binary.fetch("https://example.com")
+# => "<!DOCTYPE html><html>..."
+```
+
+### Capybara Integration
+
+**Basic usage**
+
+```ruby
+require "lightpanda/capybara"
+
+Capybara.default_driver = :lightpanda
+
+visit "https://example.com"
+find("h1").text  # => "Example Domain"
+all("p").count   # => 2
+```
+
+**Configuration**
+
+```ruby
+Lightpanda::Capybara.configure do |config|
+  config.host = "127.0.0.1"
+  config.port = 9222
+  config.timeout = 5
+end
+```
+
+**In tests**
+
+```ruby
+class FeatureTest < Minitest::Spec
+  include Capybara::DSL
+
+  def setup
+    Capybara.default_driver = :lightpanda
+  end
+
+  def teardown
+    Capybara.reset_sessions!
+  end
+
+  it "shows the homepage" do
+    visit "https://example.com"
+    assert find("h1").text == "Example Domain"
+  end
+end
+```
+
+## Environment Variables
+
+- `LIGHTPANDA_PATH` - Custom path to Lightpanda binary
+- `LIGHTPANDA_DEFAULT_TIMEOUT` - Default command timeout (default: 5)
+- `LIGHTPANDA_PROCESS_TIMEOUT` - Process startup timeout (default: 10)
+
+## Limitations
+
+Lightpanda is a lightweight browser with some limitations compared to Chrome:
+
+- Single browser context only (no incognito/multi-context)
+- No XPath support (`XPathResult` not implemented)
+- Limited CDP command coverage
+
+## Development
+
+```bash
+bundle install
+bundle exec mtest
+```
 
-## Code of Conduct
+## License
 
-Everyone interacting in the Lightpanda project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/marcoroth/lightpanda/blob/main/CODE_OF_CONDUCT.md).
+MIT License. See [LICENSE.txt](LICENSE.txt).

data/lib/lightpanda/binary.rb

@@ -0,0 +1,158 @@
+# frozen_string_literal: true
+
+require "fileutils"
+require "net/http"
+require "open3"
+require "rbconfig"
+require "uri"
+
+module Lightpanda
+  class Binary
+    Result = Struct.new(:stdout, :stderr, :status, keyword_init: true) do
+      def success?
+        status.success?
+      end
+
+      def exit_code
+        status.exitstatus
+      end
+
+      def output
+        stdout.empty? ? stderr : stdout
+      end
+    end
+
+    GITHUB_RELEASE_URL = "https://github.com/lightpanda-io/browser/releases/download/nightly"
+
+    PLATFORMS = {
+      %w[x86_64 linux] => "lightpanda-x86_64-linux",
+      %w[aarch64 darwin] => "lightpanda-aarch64-macos",
+      %w[arm64 darwin] => "lightpanda-aarch64-macos"
+    }.freeze
+
+    class << self
+      def path
+        find_or_download
+      end
+
+      def find_or_download
+        find || download
+      end
+
+      def run(*args)
+        binary = find_or_download
+        stdout, stderr, status = Open3.capture3(binary, *args)
+
+        Result.new(stdout: stdout, stderr: stderr, status: status)
+      rescue Errno::ENOENT
+        raise BinaryNotFoundError, "Lightpanda binary not found"
+      end
+
+      def fetch(url)
+        result = run("fetch", "--dump", url)
+        raise BinaryError, result.stderr unless result.success?
+
+        result.stdout
+      end
+
+      def version
+        result = run("version")
+        result.output.strip
+      end
+
+      def find
+        env_path = ENV.fetch("LIGHTPANDA_PATH", nil)
+        return env_path if env_path && File.executable?(env_path)
+
+        path_binary = find_in_path
+        return path_binary if path_binary
+
+        default_path = default_binary_path
+        return default_path if File.executable?(default_path)
+
+        nil
+      end
+
+      def download
+        binary_name = platform_binary
+        url = "#{GITHUB_RELEASE_URL}/#{binary_name}"
+        destination = default_binary_path
+
+        FileUtils.mkdir_p(File.dirname(destination))
+
+        download_file(url, destination)
+        FileUtils.chmod(0o755, destination)
+
+        destination
+      end
+
+      def platform_binary
+        arch = normalize_arch(RbConfig::CONFIG["host_cpu"])
+        os = normalize_os(RbConfig::CONFIG["host_os"])
+
+        PLATFORMS[[arch, os]] || raise(UnsupportedPlatformError, "Unsupported platform: #{arch}-#{os}")
+      end
+
+      def default_binary_path
+        cache_dir = ENV.fetch("XDG_CACHE_HOME") { File.expand_path("~/.cache") }
+
+        File.join(cache_dir, "lightpanda", "lightpanda")
+      end
+
+      private
+
+      def find_in_path
+        ENV["PATH"].to_s.split(File::PATH_SEPARATOR).each do |dir|
+          path = File.join(dir, "lightpanda")
+
+          return path if File.executable?(path)
+        end
+
+        nil
+      end
+
+      def normalize_arch(arch)
+        case arch
+        when /x86_64|amd64/i then "x86_64"
+        when /aarch64|arm64/i then "aarch64"
+        else arch
+        end
+      end
+
+      def normalize_os(os)
+        case os
+        when /darwin|mac/i then "darwin"
+        when /linux/i then "linux"
+        else os
+        end
+      end
+
+      def download_file(url, destination)
+        uri = URI.parse(url)
+
+        follow_redirects(uri, destination)
+      end
+
+      def follow_redirects(uri, destination, limit = 10)
+        raise BinaryNotFoundError, "Too many redirects" if limit.zero?
+
+        Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == "https") do |http|
+          request = Net::HTTP::Get.new(uri)
+
+          http.request(request) do |response|
+            case response
+            when Net::HTTPSuccess
+              File.open(destination, "wb") do |file|
+                response.read_body { |chunk| file.write(chunk) }
+              end
+            when Net::HTTPRedirection
+              follow_redirects(URI.parse(response["location"]), destination, limit - 1)
+            else
+              raise BinaryNotFoundError, "Failed to download binary: #{response.code} #{response.message}"
+            end
+          end
+        end
+      end
+    end
+  end
+end

data/lib/lightpanda/browser.rb

@@ -0,0 +1,212 @@
+# frozen_string_literal: true
+
+require "forwardable"
+
+module Lightpanda
+  class Browser
+    extend Forwardable
+
+    attr_reader :options, :process, :client, :target_id, :session_id
+
+    delegate %i[on off] => :client
+
+    def initialize(options = {})
+      @options = Options.new(options)
+      @process = nil
+      @client = nil
+      @target_id = nil
+      @session_id = nil
+      @started = false
+      @page_events_enabled = false
+
+      start
+    end
+
+    def start
+      return if @started
+
+      if @options.ws_url?
+        @client = Client.new(@options.ws_url, @options)
+      else
+        @process = Process.new(@options)
+        @process.start
+        @client = Client.new(@process.ws_url, @options)
+      end
+
+      create_page
+
+      @started = true
+    end
+
+    def create_page
+      result = @client.command("Target.createTarget", { url: "about:blank" })
+      @target_id = result["targetId"]
+
+      attach_result = @client.command("Target.attachToTarget", { targetId: @target_id, flatten: true })
+      @session_id = attach_result["sessionId"]
+    end
+
+    def restart
+      quit
+      start
+    end
+
+    def quit
+      @client&.close
+      @process&.stop
+      @client = nil
+      @process = nil
+      @started = false
+    end
+
+    def command(method, **params)
+      @client.command(method, params)
+    end
+
+    def page_command(method, **params)
+      @client.command(method, params, session_id: @session_id)
+    end
+
+    def go_to(url, wait: true)
+      enable_page_events
+
+      if wait
+        loaded = Concurrent::Event.new
+
+        handler = proc { loaded.set }
+        @client.on("Page.loadEventFired", &handler)
+
+        result = page_command("Page.navigate", url: url)
+
+        loaded.wait(@options.timeout)
+
+        @client.off("Page.loadEventFired", handler)
+
+        result
+      else
+        page_command("Page.navigate", url: url)
+      end
+    end
+    alias goto go_to
+
+    def enable_page_events
+      return if @page_events_enabled
+
+      page_command("Page.enable")
+      @page_events_enabled = true
+    end
+
+    def back
+      page_command("Page.navigateToHistoryEntry", entryId: current_entry_id - 1)
+    end
+
+    def forward
+      page_command("Page.navigateToHistoryEntry", entryId: current_entry_id + 1)
+    end
+
+    def refresh
+      page_command("Page.reload")
+    end
+    alias reload refresh
+
+    def current_url
+      evaluate("window.location.href")
+    end
+
+    def title
+      evaluate("document.title")
+    end
+
+    def body
+      evaluate("document.documentElement.outerHTML")
+    end
+    alias html body
+
+    def evaluate(expression)
+      response = page_command("Runtime.evaluate", expression: expression, returnByValue: true, awaitPromise: true)
+
+      handle_evaluate_response(response)
+    end
+
+    def execute(expression)
+      page_command("Runtime.evaluate", expression: expression, returnByValue: false, awaitPromise: false)
+      nil
+    end
+
+    def css(selector)
+      node_ids = page_command("DOM.querySelectorAll", nodeId: document_node_id, selector: selector)
+      node_ids["nodeIds"] || []
+    end
+
+    def at_css(selector)
+      result = page_command("DOM.querySelector", nodeId: document_node_id, selector: selector)
+
+      result["nodeId"]
+    end
+
+    def screenshot(path: nil, format: :png, quality: nil, full_page: false, encoding: :binary)
+      params = { format: format.to_s }
+      params[:quality] = quality if quality && format == :jpeg
+
+      if full_page
+        metrics = page_command("Page.getLayoutMetrics")
+        content_size = metrics["contentSize"]
+
+        params[:clip] = {
+          x: 0,
+          y: 0,
+          width: content_size["width"],
+          height: content_size["height"],
+          scale: 1
+        }
+      end
+
+      result = page_command("Page.captureScreenshot", **params)
+      data = result["data"]
+
+      if encoding == :base64
+        data
+      else
+        decoded = Base64.decode64(data)
+
+        if path
+          File.binwrite(path, decoded)
+          path
+        else
+          decoded
+        end
+      end
+    end
+
+    def network
+      @network ||= Network.new(self)
+    end
+
+    def cookies
+      @cookies ||= Cookies.new(self)
+    end
+
+    private
+
+    def document_node_id
+      result = page_command("DOM.getDocument")
+
+      result.dig("root", "nodeId")
+    end
+
+    def current_entry_id
+      result = page_command("Page.getNavigationHistory")
+
+      result["currentIndex"]
+    end
+
+    def handle_evaluate_response(response)
+      raise JavaScriptError, response if response["exceptionDetails"]
+
+      result = response["result"]
+      return nil if result["type"] == "undefined"
+
+      result["value"]
+    end
+  end
+end