playwright-ruby-client 0.7.1 → 1.14.beta1

data/documentation/docs/article/guides/playwright_on_alpine_linux.md

@@ -0,0 +1,91 @@
+---
+sidebar_position: 7
+---
+
+# Playwright on Alpine Linux
+
+**NOTE: This feature is EXPERIMENTAL.**
+
+Playwright actually requires a permission for shell command execution, and many run-time dependencies for each browser.
+
+![all-in-one](https://user-images.githubusercontent.com/11763113/124934388-9c9c9100-e03f-11eb-8f13-324afac3be2a.png)
+
+This all-in-one architecture is reasonable for browser automation in our own computers.
+
+However we may have trouble with bringing Playwright into:
+
+* Docker
+  * Alpine Linux
+* Serverless computing
+  * AWS Lambda
+  * Google Cloud Functions
+* PaaS
+  * Heroku
+  * Google App Engine
+
+This article introduces a way to separate environments into client (for executing Playwright script) and server (for working with browsers). The main use-case assumes Docker (using Alpine Linux), however the way can be applied also into other use-cases.
+
+## Overview
+
+Playwrignt Ruby client is running on Alpine Linux. It just sends/receives JSON messages of Playwright-protocol via WebSocket.
+
+Playwright server is running on a container of [official Docker image](https://hub.docker.com/_/microsoft-playwright). It just operates browsers in response to the JSON messages from WebSocket.
+
+![overview](https://user-images.githubusercontent.com/11763113/124934448-ad4d0700-e03f-11eb-942e-b9f3282bb703.png)
+
+## Playwright client
+
+Many example uses `Playwright#create`, which internally uses Pipe (stdin/stdout) transport for Playwright-protocol messaging. Instead, **just use `Playwright#connect_to_playwright_server(endpoint)`** for WebSocket transport.
+
+```ruby {3}
+require 'playwright'
+
+Playwright.connect_to_playwright_server('wss://example.com:8888/ws') do |playwright|
+  playwright.chromium.launch do |browser|
+    page = browser.new_page
+    page.goto('https://github.com/microsoft/playwright')
+    page.screenshot(path: 'github-microsoft-playwright.png')
+  end
+end
+```
+
+`wss://example.com:8888/ws` is an example of endpoint URL of the Playwright server. In local development environment, it is typically `"ws://127.0.0.1:#{port}/ws"`.
+
+## Playwright server
+
+With the [official Docker image](https://hub.docker.com/_/microsoft-playwright) or in the local development environment with Node.js, just execute `npx playwright install && npx playwright run-server $PORT`. (`$PORT` is a port number of the server)
+
+If custom Docker image is preferred, build it as follows:
+
+```Dockerfile
+FROM mcr.microsoft.com/playwright:focal
+
+WORKDIR /root
+RUN npm install playwright@1.12.3 && ./node_modules/.bin/playwright install
+
+ENV PORT 8888
+CMD ["./node_modules/.bin/playwright", "run-server", "$PORT"]
+```
+
+## Debugging for connection
+
+The client and server are really quiet. This chapter shows how to check if the communication on the WebSocket works well or not.
+
+### Show JSON message on client
+
+Just set an environment variable `DEBUG=1`.
+
+```
+DEBUG=1 bundle exec ruby some-automation-with-playwright.rb
+```
+
+
+### Enable verbose logging on server
+
+Just set an environment variable `DEBUG=pw:*` or `DEBUG=pw:server`
+
+```
+DEBUG=pw:* npx playwright run-server 8888
+```
+
+See [the official documentation](https://playwright.dev/docs/debug/#verbose-api-logs) for details.

data/documentation/docs/article/guides/semi_automation.md

@@ -1,3 +1,7 @@
+---
+sidebar_position: 5
+---
+
 # Semi-automation
 
 Playwright Browser context is isolated and not persisted by default. But we can also use persistent browser context using [BrowserType#launch_persistent_context](/docs/api/browser_type#launch_persistent_context).
@@ -10,7 +14,7 @@ Keep in mind repeatedly that persistent browser context is NOT RECOMMENDED for m
 
 ## Pause automation for manual operation
 
-`Page#pause` is not implemented yet, however we can use `binding.pry`  (with `pry-byebug` installed) instead.
+We can simply use `binding.pry`  (with `pry-byebug` installed).
 
 ```ruby {4}
 playwright.chromium.launch_persistent_context('./data/', headless: false) do |context|

data/documentation/docs/include/api_coverage.md

@@ -26,6 +26,8 @@
 * json
 * ok
 * request
+* security_details
+* server_addr
 * status
 * status_text
 * text
@@ -63,7 +65,7 @@
 
 ## Touchscreen
 
-* ~~tap_point~~
+* tap_point
 
 ## JSHandle
 
@@ -91,6 +93,7 @@
 * hover
 * inner_html
 * inner_text
+* input_value
 * checked?
 * disabled?
 * editable?
@@ -113,9 +116,9 @@
 * wait_for_element_state
 * wait_for_selector
 
-## ~~Accessibility~~
+## Accessibility
 
-* ~~snapshot~~
+* snapshot
 
 ## FileChooser
 
@@ -134,6 +137,7 @@
 * content
 * dblclick
 * dispatch_event
+* drag_and_drop
 * eval_on_selector
 * eval_on_selector_all
 * evaluate
@@ -146,6 +150,7 @@
 * hover
 * inner_html
 * inner_text
+* input_value
 * checked?
 * detached?
 * disabled?
@@ -153,6 +158,7 @@
 * enabled?
 * hidden?
 * visible?
+* locator
 * name
 * page
 * parent_frame
@@ -172,14 +178,14 @@
 * wait_for_load_state
 * expect_navigation
 * wait_for_selector
-* ~~wait_for_timeout~~
+* wait_for_timeout
 * wait_for_url
 
 ## Worker
 
-* ~~evaluate~~
-* ~~evaluate_handle~~
-* ~~url~~
+* evaluate
+* evaluate_handle
+* url
 
 ## Selectors
 
@@ -200,6 +206,17 @@
 * message
 * type
 
+## Download
+
+* cancel
+* delete
+* failure
+* page
+* path
+* save_as
+* suggested_filename
+* url
+
 ## Page
 
 * add_init_script
@@ -213,6 +230,7 @@
 * context
 * dblclick
 * dispatch_event
+* drag_and_drop
 * emulate_media
 * eval_on_selector
 * eval_on_selector_all
@@ -231,6 +249,7 @@
 * hover
 * inner_html
 * inner_text
+* input_value
 * checked?
 * closed?
 * disabled?
@@ -238,9 +257,10 @@
 * enabled?
 * hidden?
 * visible?
+* locator
 * main_frame
 * opener
-* ~~pause~~
+* pause
 * pdf
 * press
 * query_selector
@@ -276,11 +296,11 @@
 * expect_request_finished
 * expect_response
 * wait_for_selector
-* ~~wait_for_timeout~~
+* wait_for_timeout
 * wait_for_url
 * expect_websocket
-* ~~expect_worker~~
-* ~~workers~~
+* expect_worker
+* workers
 * ~~wait_for_event~~
 * accessibility
 * keyboard
@@ -291,7 +311,7 @@
 
 * add_cookies
 * add_init_script
-* ~~background_pages~~
+* background_pages
 * browser
 * clear_cookies
 * clear_permissions
@@ -304,13 +324,13 @@
 * new_page
 * pages
 * route
-* ~~service_workers~~
+* service_workers
 * set_default_navigation_timeout
 * set_default_timeout
 * set_extra_http_headers
 * set_geolocation
 * set_offline
-* ~~storage_state~~
+* storage_state
 * unroute
 * expect_event
 * expect_page
@@ -357,6 +377,49 @@
 * start
 * stop
 
+## Locator
+
+* all_inner_texts
+* all_text_contents
+* bounding_box
+* check
+* click
+* count
+* dblclick
+* dispatch_event
+* element_handle
+* element_handles
+* evaluate
+* evaluate_all
+* evaluate_handle
+* fill
+* first
+* focus
+* get_attribute
+* hover
+* inner_html
+* inner_text
+* input_value
+* checked?
+* disabled?
+* editable?
+* enabled?
+* hidden?
+* visible?
+* last
+* locator
+* nth
+* press
+* screenshot
+* scroll_into_view_if_needed
+* select_option
+* select_text
+* set_input_files
+* tap_point
+* text_content
+* type
+* uncheck
+
 ## Android
 
 * devices

data/lib/playwright.rb

@@ -17,7 +17,6 @@ require 'playwright/utils'
 require 'playwright/api_implementation'
 require 'playwright/channel'
 require 'playwright/channel_owner'
-require 'playwright/download'
 require 'playwright/http_headers'
 require 'playwright/input_files'
 require 'playwright/connection'
@@ -59,7 +58,8 @@ module Playwright
   # and we *must* call execution.stop on the end.
   # The instance of playwright is available by calling execution.playwright
   module_function def create(playwright_cli_executable_path:, &block)
-    connection = Connection.new(playwright_cli_executable_path: playwright_cli_executable_path)
+    transport = Transport.new(playwright_cli_executable_path: playwright_cli_executable_path)
+    connection = Connection.new(transport)
     connection.async_run
 
     execution =
@@ -82,7 +82,39 @@ module Playwright
     end
   end
 
-  module_function def instance
-    @playwright_instance
+  # Connects to Playwright server, launched by `npx playwright run-server` via WebSocket transport.
+  #
+  # Playwright.connect_to_playwright_server(...) do |playwright|
+  #   browser = playwright.chromium.launch
+  #   ...
+  # end
+  #
+  # @experimental
+  module_function def connect_to_playwright_server(ws_endpoint, &block)
+    require 'playwright/web_socket_client'
+    require 'playwright/web_socket_transport'
+
+    transport = WebSocketTransport.new(ws_endpoint: ws_endpoint)
+    connection = Connection.new(transport)
+    connection.async_run
+
+    execution =
+      begin
+        playwright = connection.wait_for_object_with_known_name('Playwright')
+        Execution.new(connection, PlaywrightApi.wrap(playwright))
+      rescue
+        connection.stop
+        raise
+      end
+
+    if block
+      begin
+        block.call(execution.playwright)
+      ensure
+        execution.stop
+      end
+    else
+      execution
+    end
   end
 end

data/lib/playwright/accessibility_impl.rb

@@ -0,0 +1,50 @@
+module Playwright
+  define_api_implementation :AccessibilityImpl do
+    def initialize(channel)
+      @channel = channel
+    end
+
+    def snapshot(interestingOnly: nil, root: nil)
+      params = {
+        interestingOnly: interestingOnly,
+        root: root&.channel,
+      }.compact
+      result = @channel.send_message_to_server('accessibilitySnapshot', params)
+      format_ax_node_from_protocol(result) if result
+      result
+    end
+
+    # original JS implementation create a new Hash from ax_node,
+    # but this implementation directly modify ax_node and don't return hash.
+    private def format_ax_node_from_protocol(ax_node)
+      value = ax_node.delete('valueNumber') || ax_node.delete('valueString')
+      ax_node['value'] = value unless value.nil?
+
+      checked =
+        case ax_node['checked']
+        when 'checked'
+          true
+        when 'unchecked'
+          false
+        else
+          ax_node['checked']
+        end
+      ax_node['checked'] = checked unless checked.nil?
+
+      pressed =
+        case ax_node['pressed']
+        when 'pressed'
+          true
+        when 'released'
+          false
+        else
+          ax_node['pressed']
+        end
+      ax_node['pressed'] = pressed unless pressed.nil?
+
+      ax_node['children']&.each do |child|
+        format_ax_node_from_protocol(child)
+      end
+    end
+  end
+end

data/lib/playwright/channel_owners/artifact.rb

@@ -26,5 +26,9 @@ module Playwright
     def delete
       @channel.send_message_to_server('delete')
     end
+
+    def cancel
+      @channel.send_message_to_server('cancel')
+    end
   end
 end

data/lib/playwright/channel_owners/browser_context.rb

@@ -8,9 +8,11 @@ module Playwright
 
     private def after_initialize
       @pages = Set.new
-      @routes = Set.new
+      @routes = []
       @bindings = {}
       @timeout_settings = TimeoutSettings.new
+      @service_workers = Set.new
+      @background_pages = Set.new
 
       @tracing = TracingImpl.new(@channel, self)
       @channel.on('bindingCall', ->(params) { on_binding(ChannelOwners::BindingCall.from(params['binding'])) })
@@ -19,6 +21,12 @@ module Playwright
       @channel.on('route', ->(params) {
         on_route(ChannelOwners::Route.from(params['route']), ChannelOwners::Request.from(params['request']))
       })
+      @channel.on('backgroundPage', ->(params) {
+        on_background_page(ChannelOwners::Page.from(params['page']))
+      })
+      @channel.on('serviceWorker', ->(params) {
+        on_service_worker(ChannelOwners::Worker.from(params['worker']))
+      })
       @channel.on('request', ->(params) {
         on_request(
           ChannelOwners::Request.from(params['request']),
@@ -56,6 +64,11 @@ module Playwright
       page.send(:emit_popup_event_from_browser_context)
     end
 
+    private def on_background_page(page)
+      @background_pages << page
+      emit(Events::BrowserContext::BackgroundPage, page)
+    end
+
     private def on_route(route, request)
       # It is not desired to use PlaywrightApi.wrap directly.
       # However it is a little difficult to define wrapper for `handler` parameter in generate_api.
@@ -98,6 +111,20 @@ module Playwright
       page&.emit(Events::Page::Response, response)
     end
 
+    private def on_service_worker(worker)
+      worker.context = self
+      @service_workers << worker
+      emit(Events::BrowserContext::ServiceWorker, worker)
+    end
+
+    def background_pages
+      @background_pages.to_a
+    end
+
+    def service_workers
+      @service_workers.to_a
+    end
+
     def new_cdp_session(page)
       resp = @channel.send_message_to_server('newCDPSession', page: page.channel)
       ChannelOwners::CDPSession.from(resp)
@@ -210,8 +237,8 @@ module Playwright
     end
 
     def route(url, handler)
-      entry = RouteHandlerEntry.new(url, handler)
-      @routes << entry
+      entry = RouteHandlerEntry.new(url, base_url, handler)
+      @routes.unshift(entry)
       if @routes.count >= 1
         @channel.send_message_to_server('setNetworkInterceptionEnabled', enabled: true)
       end
@@ -250,10 +277,45 @@ module Playwright
       raise unless safe_close_error?(err)
     end
 
+    # REMARK: enable_debug_console is playwright-ruby-client specific method.
+    def enable_debug_console!
+      # Ruby is not supported in Playwright officially,
+      # and causes error:
+      #
+      #  Error:
+      #  ===============================
+      #  Unsupported language: 'ruby'
+      #  ===============================
+      #
+      # So, launch inspector as Python app.
+      # NOTE: This should be used only for Page#pause at this moment.
+      @channel.send_message_to_server('recorderSupplementEnable', language: :python)
+      @debug_console_enabled = true
+    end
+
+    class DebugConsoleNotEnabledError < StandardError
+      def initialize
+        super('Debug console should be enabled in advance, by calling `browser_context.enable_debug_console!`')
+      end
+    end
+
     def pause
+      unless @debug_console_enabled
+        raise DebugConsoleNotEnabledError.new
+      end
       @channel.send_message_to_server('pause')
     end
 
+    def storage_state(path: nil)
+      @channel.send_message_to_server_result('storageState', {}).tap do |result|
+        if path
+          File.open(path, 'w') do |f|
+            f.write(JSON.dump(result))
+          end
+        end
+      end
+    end
+
     def expect_page(predicate: nil, timeout: nil)
       params = {
         predicate: predicate,
@@ -267,6 +329,14 @@ module Playwright
       @pages.delete(page)
     end
 
+    private def remove_background_page(page)
+      @background_pages.delete(page)
+    end
+
+    private def remove_service_worker(worker)
+      @service_workers.delete(worker)
+    end
+
     # called from Page with send(:_timeout_settings), so keep private.
     private def _timeout_settings
       @timeout_settings
@@ -275,5 +345,9 @@ module Playwright
     private def has_record_video_option?
       @options.key?(:recordVideo)
     end
+
+    private def base_url
+      @options[:baseURL]
+    end
   end
 end