puppeteer-ruby 0.45.6 → 0.50.0.alpha5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ba80a8570a4df72a141dff37a81899373827c461a5327311cc28cae8fc5be16f
-  data.tar.gz: 7811d540f215426009b28906723ee03c93eb0504ccbfacd321d1c219be5105a1
+  metadata.gz: 2a20cad1d3b0ad0947041ff8086f0336e381d783f1f6edb1aee45263054f5b40
+  data.tar.gz: 70c27f24141dc2326762a419cd32c780fcb9c1b5474246e2dd339dd5246b2090
 SHA512:
-  metadata.gz: f047d5dea4f9e7dc34c6578fa22887ef147be87f123f47c16d5b684ae9ac34cf6d2f3797f6d2513b039cf872bf02ff49d3e0505af7cfae98f5fa11df133f6315
-  data.tar.gz: 580d0082d1e2bb05615d5eb472b5df5dff060c291bffd11a2ce08c16ca6592451f071f2b5192ab156141aa65c21d03071eb0af17162395b548dca7a056ab1fff
+  metadata.gz: 89f6ed85a37682ced989ece6cddbac7ad9489bd6a1a31fb98f557f96527a8d298b572c078a6438d7030bd6e8c14d3b5a92042d4c1fd5caad1fc1a031335381a1
+  data.tar.gz: fe22e70a18f3319fcb7face324b2de32944f2ebfc0d594a8b33ad2bc95ceb2a0bd8a75bf63fa81369cd892fb8a125edf5c4221e0650476c618fef8455c794d0a

data/.rubocop.yml

@@ -80,6 +80,7 @@ Layout/IndentationWidth:
 
 Layout/LeadingCommentSpace:
   Enabled: true
+  AllowRBSInlineAnnotation: true
 
 Layout/SpaceAfterColon:
   Enabled: true
@@ -294,9 +295,6 @@ Style/AccessModifierDeclarations:
   Enabled: true
   EnforcedStyle: inline
 
-Style/ClassAndModuleChildren:
-  EnforcedStyle: compact
-
 Style/TrailingCommaInArguments:
   Enabled: true
   EnforcedStyleForMultiline: comma

data/AGENTS.md

@@ -0,0 +1,169 @@
+# Repository Guidelines
+
+## Start Here (Project-Specific Guidance)
+
+- Read `CLAUDE.md` and `CLAUDE/` first; they define the CDP architecture, porting workflow, testing strategy, and concurrency plan.
+- This repository is a CDP-based Ruby port of Puppeteer, **focused on Chrome/Chromium only**.
+- CI covers Ruby 3.2, 3.3, 3.4 with latest Chrome.
+
+### Technical Details
+
+- **Ruby version**: Minimum is 3.2+
+- **Concurrency**: Uses `socketry/async` (version 2.35.1+) for Fiber-based concurrency. See `CLAUDE/concurrency.md` for details.
+
+## Project Structure & Module Organization
+
+- `lib/puppeteer/`: core implementation (entry points: `puppeteer.rb`, `browser.rb`, `page.rb`, `frame.rb`, `element_handle.rb`, `connection.rb`, `cdp_session.rb`).
+- `spec/integration/`: browser-driven specs; fixtures in `spec/assets/`.
+- `spec/puppeteer/`: unit tests that do not require a browser.
+- `docs/api_coverage.md`: API implementation status.
+
+## Build, Test, and Development Commands
+
+- Run all tests: `bundle exec rspec`
+- Run a single file: `bundle exec rspec spec/integration/page_spec.rb`
+- Debug (non-headless): `DEBUG=1 bundle exec rspec spec/integration/page_spec.rb`
+- Lint: `bundle exec rubocop` (auto-fix: `bundle exec rubocop -a`)
+
+### Useful Environment Variables
+
+- `PUPPETEER_EXECUTABLE_PATH_RSPEC`: custom Chrome path
+- `PUPPETEER_CHANNEL_RSPEC`: Chrome channel (for example `chrome-dev`)
+- `PUPPETEER_NO_SANDBOX_RSPEC`: add `--no-sandbox` flag
+
+## Coding Style & Naming Conventions
+
+- Follow `.rubocop.yml`; prefer explicit keyword arguments for public APIs.
+- Public APIs mirror Puppeteer naming but use Ruby `snake_case`.
+- Custom errors inherit from `Puppeteer::Error`.
+- Use `Puppeteer::AsyncUtils` for async operations; see `CLAUDE/concurrency.md` for patterns.
+
+## Type Annotations (rbs-inline)
+
+- Add `# rbs_inline: enabled` at the top of the file (after `# frozen_string_literal: true` if present).
+- Use doc-style `# @rbs` annotations for parameters and return types.
+- Always include descriptions with `--` (example: `# @rbs url: String -- Target URL`).
+- Use `A?` for nullable types and `A | B | nil` for unions that include nil.
+- Avoid `@rbs!` blocks (RubyMine doesn't recognize them).
+- Avoid `**options` in public APIs; RubyMine shows it as `untyped`. Prefer explicit keyword args.
+
+**Generate signatures:**
+- `bundle exec rake rbs` (writes to `sig/`, which is gitignored)
+- `bundle exec steep check` (after generating RBS)
+
+**Manually maintained signatures (tracked):**
+- `sig/_external.rbs` for external dependency stubs.
+- `sig/_supplementary.rbs` for types rbs-inline cannot infer (e.g., `extend self`, singleton helpers).
+
+**Example:**
+```ruby
+# frozen_string_literal: true
+# rbs_inline: enabled
+
+class Example
+  attr_reader :name #: String
+
+  # @rbs name: String -- The name to set
+  # @rbs return: void -- No return value
+  def initialize(name)
+    @name = name
+  end
+end
+```
+
+## Testing Guidelines
+
+- Integration tests are `spec/integration/` with `type: :puppeteer` helpers.
+- Use `sinatra: true` for tests that need a local server.
+- Use `match_golden` for screenshot comparisons where applicable.
+
+## Agent Notes (Porting/Review)
+
+### Source Code Porting
+
+- When porting from upstream, use `packages/puppeteer-core/src/cdp/` as the primary source.
+- Mirror upstream behavior, error messages, and option handling as closely as possible.
+- Enable required CDP domains before relying on their events (see `CLAUDE/cdp_protocol.md`).
+
+### Test Porting Guidelines
+
+When porting tests from upstream `test/src/*.spec.ts` to `spec/integration/*_spec.rb`:
+
+**Structure & Order**
+- Keep `it` blocks in the **exact same order** as upstream
+- Use the **same test names** (translated to Ruby style, e.g., `'should type into a textarea'`)
+- Do NOT add extra `context`/`describe` wrappers unless upstream has them
+- Do NOT add Ruby-specific tests in the middle; add them at the end if needed
+
+**Ruby-Specific Tests → `*_ext_spec.rb`**
+- When porting, separate Ruby-only features into `*_ext_spec.rb` files (e.g., `keyboard_ext_spec.rb`)
+- Ruby-specific features include: block DSL (`page.keyboard { ... }`), `press('Shift') { press('Key') }` syntax
+- Keep upstream-equivalent tests in the main spec file for easy comparison
+- Example: `keyboard_spec.rb` (upstream port) + `keyboard_ext_spec.rb` (Ruby extensions)
+
+**Test State Setup**
+- Use `with_test_state` block instead of `include_context 'with test state'`
+- Access test helpers via block arguments: `page:`, `server:`, `https_server:`, `browser:`, `browser_context:`
+- Example:
+  ```ruby
+  it 'should click button' do
+    with_test_state do |page:, server:, **|
+      page.goto("#{server.prefix}/input/button.html")
+      page.click('button')
+      expect(page.evaluate('() => globalThis.result')).to eq('Clicked')
+    end
+  end
+  ```
+
+**Asset Files**
+- `spec/assets/` files must be **identical** to upstream `test/assets/`
+- Fetch assets directly: `wget https://raw.githubusercontent.com/puppeteer/puppeteer/main/test/assets/xxx`
+- Do NOT hand-edit asset files; if upstream changes, re-fetch
+
+**Code Translation**
+- `page.evaluate(() => expr)` → `page.evaluate('() => expr')` (string form)
+- `page.$('selector')` → `page.query_selector('selector')`
+- `page.$$('selector')` → `page.query_selector_all('selector')`
+- `page.$eval()` → `page.eval_on_selector()`
+- `await expect(...)` assertions → RSpec `expect(...).to eq(...)`
+- `toThrow`/`rejects.toThrow` → `raise_error` matcher
+- Platform checks: `os.platform() !== 'darwin'` → `skip(...) unless Puppeteer.env.darwin?`
+
+**JavaScript Object Comparisons**
+- When comparing JS objects, use `eval_on_selector` or `evaluate` returning a hash
+- Compare with Ruby hash: `expect(result).to eq({ 'key' => 'value' })`
+- Note: JS object keys become string keys in Ruby hashes
+
+**Upstream Test Location**
+- Tests: `https://github.com/puppeteer/puppeteer/tree/main/test/src`
+- Assets: `https://github.com/puppeteer/puppeteer/tree/main/test/assets`
+
+See `CLAUDE/porting_puppeteer.md` for detailed examples.
+- Update `docs/api_coverage.md` when new APIs are added.
+- `CHANGELOG.md` is being retired; do not update it for new changes.
+- Porting plan (CDP + async):
+  - Aim for Node.js Puppeteer fidelity, but use `socketry/async` (like puppeteer-bidi) instead of JS async/await.
+  - Use puppeteer-bidi as the Ruby porting reference: follow its approach and patterns.
+  - For `Puppeteer::Page`, mirror upstream `api/Page.ts` and `cdp/Page.ts`, similar to how puppeteer-bidi uses `api/Page.ts` + `bidi/Page.ts`.
+  - Follow puppeteer-bidi's `Page` patterns: wrap `:request` handlers to enqueue interception actions (WeakMap to keep `on/off` mapping), serialize interception with `Async::Semaphore`, and keep `Page` mostly as a thin delegator to `Frame`.
+  - Mirror upstream timeout/deferred behavior with `Async::Promise` + `AsyncUtils.async_timeout` (timeout 0 = infinite) and keep error messages aligned (e.g., file chooser/network idle).
+  - Screenshot parity: apply default option behavior and clip rounding (`normalize/round`), and convert clip coordinates when `captureBeyondViewport` is false (visualViewport offsets).
+  - Serialize screenshot operations (bidi uses `browserContext.waitForScreenshotOperations`); keep `ScreenshotTaskQueue`/guards to avoid overlapping captures.
+  - `evaluate_on_new_document`: build IIFE expressions and serialize args the same way as puppeteer-bidi's `build_evaluation_expression`/`serialize_arg_for_preload`.
+  - `wait_for_network_idle`: track inflight requests and reset idle timers on changes (mirrors Page.ts inflight counter logic).
+  - `wait_for_file_chooser`: resolve all pending waiters and keep timeout error messaging aligned (`Waiting for \`FileChooser\` failed: <ms> exceeded`).
+  - RxJS Observable flows are replaced with manual EventEmitter listeners + `Async::Promise` (no `fromEmitterEvent`/`merge`/`timeout`), so compose waits explicitly (e.g., `wait_for_navigation`, `set_content`).
+  - Use `AsyncUtils` (Barrier-based Promise.all/race + `async_timeout`) instead of RxJS `combineLatest`/`firstValueFrom`/`timeout`.
+  - `WaitTask`/`TaskManager` mirror Puppeteer's polling waits; no AbortSignal, use Async tasks + cancellation.
+  - `ReactorRunner` runs a dedicated Async reactor thread and proxies sync calls into it (wrap/unwrap).
+  - Use `Core::EventEmitter` + symbols instead of RxJS event streams; clean up listeners explicitly.
+  - Disposable patterns: `Core::Disposable::DisposableStack`/`DisposableMixin` stand in for JS DisposableStack/AsyncDisposableStack.
+
+## Security & Configuration Tips
+
+- Do not commit credentials or secrets.
+- Be cautious with network-fetched/generated files; review diffs carefully.
+
+## Release & CI
+
+- Release/tagging workflow follows the current project practice (no new rules).

data/CLAUDE/README.md

@@ -0,0 +1,41 @@
+# Detailed Implementation Documentation
+
+This directory contains in-depth documentation for AI agents and developers working on puppeteer-ruby.
+
+## Documentation Index
+
+| Document | Description |
+|----------|-------------|
+| [architecture.md](./architecture.md) | CDP-based architecture and component relationships |
+| [testing.md](./testing.md) | Testing strategies, patterns, and debugging |
+| [cdp_protocol.md](./cdp_protocol.md) | Chrome DevTools Protocol usage |
+| [concurrency.md](./concurrency.md) | Concurrency model and migration plans |
+| [porting_puppeteer.md](./porting_puppeteer.md) | Guide for porting from TypeScript Puppeteer |
+| [rbs_type_checking.md](./rbs_type_checking.md) | RBS type annotations and Steep type checking |
+
+## Quick Navigation
+
+### For New Feature Implementation
+
+1. Read [porting_puppeteer.md](./porting_puppeteer.md) for workflow
+2. Understand the feature's CDP calls in [cdp_protocol.md](./cdp_protocol.md)
+3. Review [architecture.md](./architecture.md) for component relationships
+
+### For Bug Fixes
+
+1. Review [architecture.md](./architecture.md) to understand component relationships
+2. Write tests following patterns in [testing.md](./testing.md)
+3. Consider concurrency issues per [concurrency.md](./concurrency.md)
+
+### For Code Review
+
+1. Ensure alignment with [architecture.md](./architecture.md)
+2. Verify test coverage per [testing.md](./testing.md)
+3. Review concurrency patterns per [concurrency.md](./concurrency.md)
+
+## Related Resources
+
+- [CLAUDE.md](../CLAUDE.md) - High-level project summary
+- [docs/api_coverage.md](../docs/api_coverage.md) - API implementation status
+- [Puppeteer TypeScript source](https://github.com/puppeteer/puppeteer) - Reference implementation
+- [Chrome DevTools Protocol docs](https://chromedevtools.github.io/devtools-protocol/) - CDP reference

data/CLAUDE/architecture.md

@@ -0,0 +1,253 @@
+# Architecture Overview
+
+This document describes the architecture of puppeteer-ruby.
+
+## High-Level Architecture
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                        User-Facing API                          │
+│  Puppeteer.launch / Puppeteer.connect                          │
+└─────────────────────────────────────────────────────────────────┘
+                              │
+                              ▼
+┌─────────────────────────────────────────────────────────────────┐
+│                         Browser                                 │
+│  - Manages browser process lifecycle                            │
+│  - Creates browser contexts (incognito sessions)                │
+│  - Tracks all targets (pages, workers, etc.)                    │
+└─────────────────────────────────────────────────────────────────┘
+                              │
+                              ▼
+┌─────────────────────────────────────────────────────────────────┐
+│                      BrowserContext                             │
+│  - Represents an incognito session or default context           │
+│  - Manages permissions, cookies, geolocation                    │
+│  - Creates new pages within the context                         │
+└─────────────────────────────────────────────────────────────────┘
+                              │
+                              ▼
+┌─────────────────────────────────────────────────────────────────┐
+│                          Page                                   │
+│  - Main user interaction point                                  │
+│  - Navigation, content, screenshots, PDF                        │
+│  - Delegates to Frame, Keyboard, Mouse, etc.                    │
+└─────────────────────────────────────────────────────────────────┘
+                              │
+            ┌─────────────────┼─────────────────┐
+            ▼                 ▼                 ▼
+      ┌──────────┐     ┌──────────┐     ┌──────────────┐
+      │  Frame   │     │ Keyboard │     │    Mouse     │
+      │          │     │          │     │              │
+      └──────────┘     └──────────┘     └──────────────┘
+            │
+            ▼
+      ┌──────────────────────────────────────────┐
+      │              IsolatedWorld               │
+      │  - Execution context for JavaScript      │
+      │  - Query selectors, evaluate JS          │
+      └──────────────────────────────────────────┘
+            │
+            ▼
+      ┌──────────────────────────────────────────┐
+      │         ElementHandle / JSHandle         │
+      │  - References to DOM elements / JS objs  │
+      │  - Click, type, evaluate, etc.           │
+      └──────────────────────────────────────────┘
+```
+
+## Core Components
+
+### Connection Layer
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                        Connection                               │
+│  lib/puppeteer/connection.rb                                    │
+│                                                                 │
+│  - WebSocket connection to browser's DevTools endpoint          │
+│  - Manages message routing to correct CDPSession                │
+│  - Handles connection lifecycle                                 │
+└─────────────────────────────────────────────────────────────────┘
+                              │
+                              ▼
+┌─────────────────────────────────────────────────────────────────┐
+│                       CDPSession                                │
+│  lib/puppeteer/cdp_session.rb                                   │
+│                                                                 │
+│  - Represents a CDP session to a specific target                │
+│  - send_message: Send CDP commands                              │
+│  - on/once: Subscribe to CDP events                             │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+### Target Management
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│                    ChromeTargetManager                          │
+│  lib/puppeteer/chrome_target_manager.rb                         │
+│                                                                 │
+│  - Discovers and tracks all targets in browser                  │
+│  - Creates Target objects for new pages, workers, etc.          │
+│  - Handles target attachment and auto-attach                    │
+└─────────────────────────────────────────────────────────────────┘
+
+```
+
+### Page Architecture
+
+```
+Page
+ ├── FrameManager          # Manages all frames in the page
+ │    ├── Frame (main)     # Main frame
+ │    └── Frame (child)... # Iframes
+ │         └── IsolatedWorld
+ │              ├── Main world (page's JS context)
+ │              └── Puppeteer utility world (isolated context)
+ │
+ ├── NetworkManager        # Request interception, network events
+ │    └── NetworkEventManager
+ │
+ ├── EmulationManager      # Viewport, device emulation
+ │
+ ├── Keyboard              # Keyboard input
+ ├── Mouse                 # Mouse input
+ └── TouchScreen           # Touch input
+```
+
+## Key Patterns
+
+### Event Handling
+
+Components use `EventCallbackable` mixin for event handling:
+
+```ruby
+class Page
+  include Puppeteer::EventCallbackable
+
+  def initialize
+    @frame_manager.on('load') do
+      emit_event('load')
+    end
+  end
+end
+
+# Usage
+page.on('load') { puts 'Page loaded!' }
+```
+
+### CDP Command/Response Pattern
+
+```ruby
+# Send CDP command and wait for response
+result = @client.send_message('Page.navigate', url: url)
+
+# Subscribe to CDP events
+@client.on('Network.requestWillBeSent') do |event|
+  handle_request(event)
+end
+```
+
+### Resource Cleanup
+
+Use `begin/ensure` blocks for cleanup:
+
+```ruby
+def screenshot(options = {})
+  original_viewport = @viewport
+  begin
+    self.viewport = options[:viewport] if options[:viewport]
+    # Take screenshot...
+  ensure
+    self.viewport = original_viewport if options[:viewport]
+  end
+end
+```
+
+## File Organization
+
+```
+lib/puppeteer/
+├── puppeteer.rb              # Module with launch/connect methods
+├── puppeteer/
+│   ├── browser.rb            # Browser class
+│   ├── browser_context.rb    # BrowserContext class
+│   ├── page.rb               # Page class (largest file)
+│   ├── frame.rb              # Frame class
+│   ├── frame_manager.rb      # FrameManager class
+│   ├── element_handle.rb     # ElementHandle class
+│   ├── js_handle.rb          # JSHandle class
+│   ├── connection.rb         # WebSocket connection
+│   ├── cdp_session.rb        # CDP session
+│   ├── keyboard.rb           # Keyboard input
+│   ├── mouse.rb              # Mouse input
+│   ├── network_manager.rb    # Network interception
+│   ├── launcher/
+│   │   ├── chrome.rb         # Chrome-specific launch
+│   └── ...
+```
+
+## Data Flow Examples
+
+### Page Navigation
+
+```
+User: page.goto('https://example.com')
+  │
+  ▼
+Page#goto
+  │
+  ├── FrameManager#navigate_frame
+  │     │
+  │     ├── CDPSession.send('Page.navigate', url: ...)
+  │     │
+  │     └── LifecycleWatcher.new(wait_until: 'load')
+  │           │
+  │           └── Waits for 'Page.loadEventFired' event
+  │
+  └── Returns HTTPResponse
+```
+
+### Element Click
+
+```
+User: element.click
+  │
+  ▼
+ElementHandle#click
+  │
+  ├── scroll_into_view_if_needed
+  │     └── CDPSession.send('DOM.scrollIntoViewIfNeeded')
+  │
+  ├── clickable_point
+  │     └── CDPSession.send('DOM.getContentQuads')
+  │
+  └── Page#mouse.click(x, y)
+        │
+        ├── Mouse#move(x, y)
+        │     └── CDPSession.send('Input.dispatchMouseEvent', type: 'mouseMoved')
+        │
+        ├── Mouse#down
+        │     └── CDPSession.send('Input.dispatchMouseEvent', type: 'mousePressed')
+        │
+        └── Mouse#up
+              └── CDPSession.send('Input.dispatchMouseEvent', type: 'mouseReleased')
+```
+
+### JavaScript Evaluation
+
+```
+User: page.evaluate('() => document.title')
+  │
+  ▼
+Page#evaluate
+  │
+  └── Frame#evaluate
+        │
+        └── IsolatedWorld#evaluate
+              │
+              ├── CDPSession.send('Runtime.callFunctionOn', ...)
+              │
+              └── RemoteObject.new(result).value
+```