btape 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 83f37a580ea032f35ec04083a301339d43bb35bd13881799774d275527558e43
-  data.tar.gz: 6b47a03f994310102fc3534c621d7fe78e782e6ab39653a6c227b56838ba17c0
+  metadata.gz: 18261c2d64fe2120e746ab4e2991a85d8c6327aa32ad497125e6b8b1efb3779a
+  data.tar.gz: aa731da95d316a63404d3ae6a89f03d3c435bea95ff26eeda824c699f8390ef7
 SHA512:
-  metadata.gz: 493ebb303b92ba896387e67236c61deaf5cfaed5ba746990e67c9be9ad06d88d1ea34985d3719a87b6edfbbafe06c59b8d052720b05471fc1215589db23da43d
-  data.tar.gz: b5cf6dcfc0995a623f080f30bd5903dc8fed6f456d42f697534a720494d7c8418bcfdeb3917cb0c4537120216382578456642671b849d07b2365c486e9f0a329
+  metadata.gz: dbb660a8f316b473a9c0a62e9842f3cc131e28d5061b49227cdc7b474be90abae39a7dfd5104e5cca26931458e60140a3423c41043de72a2d0e27b2adb51e155
+  data.tar.gz: 89ddb3f16ff21c0710dce00bfdef3a5588e3ee70a0d7df29373afb32b8e0dbf976ba30f63b83971aedd460ad8ff8c16b980f086f0d58f8d4699fe335554c1144

data/README.md

@@ -1,5 +1,9 @@
 # btape
 
+<p align="center">
+  <img src="assets/logo.png" alt="btape logo" width="480">
+</p>
+
 `btape` is a small, VHS-inspired Ruby CLI that runs browser actions from a
 `.tape` file and records them as an animated GIF. Ferrum controls Chromium
 and captures PNG frames, and a pure-Ruby encoder produces the GIF. It
@@ -10,9 +14,16 @@ does not require Playwright, Selenium, ffmpeg, or an external service.
 ```text
 Output <path>
 Viewport <width>x<height>
+Set <name> <value>
 Goto <url>
 Click <CSS selector or text=Text>
 Type <CSS selector> <text>
+Press <key> [count]
+Frame <CSS selector or main>
+Evaluate <javascript>
+WaitFor <CSS selector or text=Text> [duration]
+WaitForJS <javascript> [duration]
+Screenshot [name]
 Sleep <number>ms|s
 ```
 
@@ -29,6 +40,37 @@ Type "#email" "demo@example.com"
 Sleep 1s
 ```
 
+`Evaluate` runs JavaScript in the current frame, which is how a tape reaches
+an API the page exposes rather than clicking at it. `Frame` points the
+commands that follow at an iframe, and `Frame main` returns to the page;
+navigating returns to the page too, since the frame belonged to the page that
+was left. `WaitFor` and `WaitForJS` poll instead of guessing at a `Sleep`.
+
+`Screenshot` captures a frame there and then. With a name it also lands at a
+predictable path, for picking one particular frame out of a run.
+
+## Settings
+
+`Set NAME VALUE` configures a run. Every setting can also be given on the
+command line with `--set NAME=VALUE`, which wins over the tape, so one tape
+can run in more than one place.
+
+| Name | Default | Meaning |
+| --- | --- | --- |
+| `WsUrl` | — | Connect to a browser already running at this CDP url instead of launching one |
+| `CaptureMode` | `interval` | `interval` records continuously; `manual` captures only where `Screenshot` says to |
+| `Framerate` | `10` | Captures per second in interval mode |
+| `FrameDelay` | `100ms` | How long each frame is shown in the GIF |
+| `Loop` | `0` | Times to loop; 0 is forever |
+| `Scale` | `1.0` | Scale the output down |
+| `OutputWidth` | — | Output width in pixels; overrides `Scale` and keeps the aspect ratio |
+| `Quantizer` | `adaptive` | `adaptive` picks the palette from the frames; `rgb332` uses a fixed one |
+| `Timeout` | `120s` | Give up on the whole run after this |
+| `WaitTimeout` | `10s` | Give up on a `WaitFor` or `WaitForJS` after this |
+| `WaitInterval` | `100ms` | How often those two check |
+| `WaitStable` | `1` | How many checks in a row must pass before a wait is satisfied |
+| `MaxFrames` | `600` | Stop rather than record a hung page until the disk fills |
+
 ## Install and run
 
 Chromium must be installed and discoverable by Ferrum. Then:
@@ -39,9 +81,79 @@ bundle exec btape demo.tape
 bundle exec rake spec
 ```
 
-The recording interval is 100 ms (10 fps). Temporary PNG frames are removed
-after a successful run. They are also isolated in the system temporary
-directory and cleaned when an error unwinds the run.
+```text
+Usage: btape [options] SCRIPT.tape
+
+  --ws-url URL     Connect to a browser already running at this CDP url
+  --set NAME=VALUE Override a setting, as a Set line would
+  --frames-dir DIR Write the PNG frames here and keep them
+  --verbose        Report each command on stderr as it runs
+```
+
+`BTAPE_WS_URL` is used when neither `--ws-url` nor `--set WsUrl=` is given.
+
+### A browser running somewhere else
+
+btape launches its own Chromium by default. Point it at one that is already
+running — a `browserless`/`chrome` container, say — and no browser needs to be
+in the image btape runs from:
+
+```sh
+btape --ws-url ws://chrome:3000 examples/thumbnails.tape
+```
+
+Each connection gets its own browser context, so concurrent runs against one
+shared browser do not see each other. The viewport is applied over the wire,
+since a browser that is already running cannot be told its window size at
+launch.
+
+### Frames, not just the GIF
+
+Frames are normally written to a temporary directory and removed as the run
+unwinds. `--frames-dir` keeps them:
+
+```sh
+btape --frames-dir frames examples/thumbnails.tape
+```
+
+## From Ruby
+
+`Runner#run` returns a `Btape::Result`:
+
+```ruby
+commands = Btape::Parser.new.parse(File.read('deck.tape'))
+
+result = Btape::Runner.new(logger: Rails.logger).run(
+  commands,
+  base_directory: File.dirname('deck.tape'),
+  settings: { ws_url: ENV['CHROME_WS_URL'] },
+  frames_directory: frames,
+  on_frame: ->(path, index) { logger.debug("captured #{index}: #{path}") }
+)
+
+result.output_path            # where the GIF went
+result.frame_count            # frames that went into it
+result.frame_paths            # the frames, when frames_directory was given
+result.named_frames['page-01'] # the frame a Screenshot named
+```
+
+Nothing has to touch the filesystem. Pass an IO to write the GIF into:
+
+```ruby
+buffer = StringIO.new(+''.b)
+Btape::Runner.new.run(commands, base_directory: '.', output: buffer)
+
+# `buffer.string` is the GIF. Hand it to whatever holds on to it — an Active
+# Storage attachment on one of your own records, say:
+deck = Deck.find(params[:id])
+deck.animation.attach(io: StringIO.new(buffer.string), filename: 'deck.gif')
+```
+
+or use the encoder on its own, with PNG paths or ChunkyPNG images:
+
+```ruby
+Btape::GifEncoder.new(delay: 150, width: 640).encode(frame_paths) # => String
+```
 
 ## Container development with dip or wip
 
@@ -60,11 +172,31 @@ wip dispatch btape examples/demo.tape
 `examples/demo.tape` drives a small static page bundled at
 `examples/demo_app.html`, so the demo is self-contained and needs no other
 service running. Edit the tape (or point `Goto` at a different URL) to record
-something else.
+something else. `examples/thumbnails.tape` shows the other shape of run: one
+frame per page of a deck, against a browser running elsewhere.
+
+## Limitations
+
+The palette is chosen from the frames being encoded, which tracks gradients
+and text edges far more closely than the fixed RGB332 palette earlier versions
+used — but banding compresses well and fidelity does not, so the files are
+larger than they were. `Set Scale` or `Set OutputWidth` are the levers to pull
+back; identical consecutive frames are already collapsed into one held for
+longer. `Set Quantizer rgb332` restores the old palette.
+
+The first matching element is used for `Click` and `Type`.
+
+## Upgrading to 0.2
+
+`Runner#run` returns a `Btape::Result` rather than the output path. Read
+`result.output_path` where the path was used before.
+
+## Contributing
 
-## MVP limitations
+Bug reports and pull requests are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md)
+for the development setup, PR conventions, and how releases are generated.
+This project follows the [Code of Conduct](CODE_OF_CONDUCT.md).
 
-The GIF encoder uses a fixed 256-colour RGB332 palette to stay dependency-light.
-This favors portability over photographic colour fidelity and file size. The
-first matching element is used for `Click` and `Type`.
+## License
 
+[MIT](LICENSE)

data/lib/btape/cli.rb

@@ -1,26 +1,39 @@
 # frozen_string_literal: true
 
+require 'logger'
+require 'optparse'
+
 module Btape
   # Entry point invoked by the `btape` executable: parses argv, runs the
   # script, and reports success or failure.
   class CLI
-    def initialize(out: $stdout, err: $stderr, runner: Runner.new)
+    Options = Struct.new(:help, :settings, :frames_directory, :verbose)
+
+    USAGE = 'Usage: btape [options] SCRIPT.tape'
+    HELP_ARGUMENTS = %w[help -h --help].freeze
+    HELP_COMMANDS = ['Output PATH', 'Viewport WIDTHxHEIGHT', 'Goto URL', 'Click SELECTOR',
+                     'Type SELECTOR TEXT', 'Sleep DURATION', 'Set NAME VALUE', 'Screenshot [NAME]',
+                     'Evaluate JAVASCRIPT', 'WaitFor SELECTOR [TIMEOUT]',
+                     'WaitForJS JAVASCRIPT [TIMEOUT]', 'Frame SELECTOR|main',
+                     'Press KEY [COUNT]'].freeze
+    # The two ways a flag can name a browser: --ws-url and --set WsUrl=.
+    WS_URL_KEYS = [:ws_url, 'WsUrl'].freeze
+
+    # The runner is built after the flags are read, so that --verbose can
+    # reach it. Pass one to use it as given.
+    def initialize(out: $stdout, err: $stderr, runner: nil)
       @out = out
       @err = err
       @runner = runner
     end
 
-    HELP_COMMANDS = ['Output PATH', 'Viewport WIDTHxHEIGHT', 'Goto URL', 'Click SELECTOR',
-                     'Type SELECTOR TEXT', 'Sleep DURATION'].freeze
-
     def run(argv)
-      return print_help if argv.empty? || %w[help -h --help].include?(argv.first)
-      raise Error, 'usage: btape SCRIPT.tape' unless argv.length == 1
+      options = Options.new(false, {}, nil, false)
+      arguments = parse_options(argv, options)
+      return print_help if options.help || arguments.empty? || HELP_ARGUMENTS.include?(arguments.first)
+      raise Error, USAGE unless arguments.length == 1
 
-      script = File.expand_path(argv.first)
-      commands = Parser.new.parse(File.read(script))
-      output = @runner.run(commands, base_directory: File.dirname(script))
-      @out.puts "Created #{output}"
+      record(arguments.first, options)
       0
     rescue Error, SystemCallError => e
       @err.puts "btape: #{e.message}"
@@ -29,11 +42,88 @@ module Btape
 
     private
 
+    def record(argument, options)
+      script = File.expand_path(argument)
+      commands = Parser.new.parse(File.read(script))
+      result = runner(options).run(
+        commands,
+        base_directory: File.dirname(script),
+        settings: settings(options),
+        frames_directory: options.frames_directory
+      )
+      @out.puts "Created #{result.output_path}"
+      report_frames(result)
+    end
+
+    def runner(options)
+      @runner || Runner.new(logger: logger(options))
+    end
+
+    def logger(options)
+      return NullLogger.new unless options.verbose
+
+      Logger.new(@err, level: Logger::DEBUG, formatter: ->(_severity, _time, _program, message) { "#{message}\n" })
+    end
+
+    def report_frames(result)
+      return if result.frame_paths.empty?
+
+      @out.puts "Kept #{result.frame_paths.length} frame(s) in #{File.dirname(result.frame_paths.first)}"
+    end
+
+    # The tape's own `Set` lines are the baseline; these flags override them,
+    # which is what lets one tape run against a local and a remote browser.
+    #
+    # BTAPE_WS_URL is the last resort, and only applies when neither flag named
+    # a browser. Settings resolves --ws-url and --set WsUrl to the same
+    # attribute, so merging the environment in unconditionally would let it
+    # overwrite the --set value rather than fall back to it.
+    def settings(options)
+      return options.settings if WS_URL_KEYS.any? { |key| options.settings.key?(key) }
+
+      options.settings.merge(ws_url: ENV.fetch('BTAPE_WS_URL', nil))
+    end
+
+    def parse_options(argv, options)
+      option_parser(options).parse(argv)
+    rescue OptionParser::ParseError => e
+      raise Error, e.message
+    end
+
+    def option_parser(options)
+      OptionParser.new do |parser|
+        parser.banner = USAGE
+        parser.on('--ws-url URL', 'Connect to a browser already running at this CDP url') do |url|
+          options.settings[:ws_url] = url
+        end
+        parser.on('--set NAME=VALUE', 'Override a setting, as a Set line would') do |pair|
+          name, value = pair.split('=', 2)
+          raise Error, '--set expects NAME=VALUE' if value.nil?
+
+          options.settings[name] = value
+        end
+        parser.on('--frames-dir DIR', 'Write the PNG frames here and keep them') do |directory|
+          options.frames_directory = directory
+        end
+        parser.on('--verbose', 'Report each command on stderr as it runs') { options.verbose = true }
+        parser.on('-h', '--help', 'Show this message') { options.help = true }
+      end
+    end
+
     def print_help
-      @out.puts 'Usage: btape SCRIPT.tape'
+      @out.puts USAGE
+      @out.puts
+      @out.puts 'Options:'
+      @out.puts '  --ws-url URL     Connect to a browser already running at this CDP url'
+      @out.puts '  --set NAME=VALUE Override a setting, as a Set line would'
+      @out.puts '  --frames-dir DIR Write the PNG frames here and keep them'
+      @out.puts '  --verbose        Report each command on stderr as it runs'
       @out.puts
       @out.puts 'Commands:'
       HELP_COMMANDS.each { |command| @out.puts "  #{command}" }
+      @out.puts
+      @out.puts 'Settings:'
+      Settings::DEFINITIONS.each_key { |name| @out.puts "  #{name}" }
       0
     end
   end

data/lib/btape/duration.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+module Btape
+  # The duration literals .tape scripts use ("500ms", "1.5s"). The parser,
+  # the runner and Settings all need the same format, so it lives here once
+  # rather than as a regexp repeated at each call site.
+  module Duration
+    PATTERN = /\A(\d+(?:\.\d+)?)(ms|s)\z/
+    DESCRIPTION = 'must use ms or s (for example, 500ms or 1.5s)'
+
+    module_function
+
+    def valid?(value)
+      PATTERN.match?(value.to_s)
+    end
+
+    # Returns the duration in seconds.
+    def parse(value)
+      match = PATTERN.match(value.to_s)
+      raise ArgumentError, "duration #{DESCRIPTION}" unless match
+
+      amount, unit = match.captures
+      amount.to_f / (unit == 'ms' ? 1000 : 1)
+    end
+  end
+end

data/lib/btape/error.rb

@@ -3,6 +3,10 @@
 module Btape
   class Error < StandardError; end
 
+  # Raised when a run outlasts Set Timeout. A page that never finishes
+  # loading would otherwise record until the disk filled.
+  class TimeoutError < Error; end
+
   # Raised for a problem in the .tape script itself, carrying the line
   # number so the CLI can report where the script went wrong.
   class ScriptError < Error

data/lib/btape/executor.rb

@@ -0,0 +1,186 @@
+# frozen_string_literal: true
+
+require 'monitor'
+require_relative 'duration'
+require_relative 'null_logger'
+
+module Btape
+  # Performs the parsed commands against a browser, turning any failure into a
+  # ScriptError that points back at the line it came from.
+  #
+  # It lives apart from Runner because Runner's job is the recording session
+  # around the script — the browser, the frames, the GIF — while this is the
+  # script itself, and only this grows with each new command.
+  class Executor
+    # Commands that do something at run time, and the method that does it.
+    # Output, Viewport and Set are read before the run starts and have no
+    # behaviour of their own here.
+    HANDLERS = {
+      'Goto' => :goto, 'Click' => :click, 'Type' => :enter, 'Sleep' => :pause,
+      'Screenshot' => :screenshot, 'Evaluate' => :evaluate,
+      'WaitFor' => :wait_for_element, 'WaitForJS' => :wait_for_js,
+      'Frame' => :frame, 'Press' => :press
+    }.freeze
+
+    MAIN_FRAME = 'main'
+
+    def initialize(browser:, recorder:, settings:, lock: Monitor.new, logger: NullLogger.new)
+      @browser = browser
+      # Elements and JavaScript are looked up in the current frame, which
+      # starts as the page itself and moves when a Frame command says so.
+      @target = browser
+      @recorder = recorder
+      @settings = settings
+      @lock = lock
+      @logger = logger
+    end
+
+    def call(commands)
+      commands.each do |command|
+        @logger.debug("btape: line #{command.line_number}: #{command.name}")
+        perform(command)
+      # The run's own deadline arrives asynchronously and usually lands inside
+      # a command, which would otherwise be reported as that command failing.
+      # A caller has to be able to tell "the run outlasted Set Timeout" from
+      # "this line is broken", so it goes out as it came in.
+      rescue TimeoutError
+        raise
+      rescue StandardError => e
+        raise ScriptError.new(command.line_number, "#{command.name} failed: #{e.message}")
+      end
+    end
+
+    private
+
+    # Held across each exchange with the browser, and shared with the
+    # recorder, so a screenshot taken on the recorder's thread is never in
+    # flight at the same time as a command on this one.
+    #
+    # It is deliberately not held for a whole command: Sleep and the waiting
+    # commands spend most of their time not talking to the browser at all,
+    # and holding the lock through that would leave interval recording with
+    # nothing to capture for the duration.
+    def locked(&)
+      @lock.synchronize(&)
+    end
+
+    def perform(command)
+      handler = HANDLERS[command.name]
+      send(handler, *command.arguments) if handler
+    end
+
+    def goto(url)
+      locked { @browser.go_to(url) }
+      # Whatever frame we were in belongs to the page we just left.
+      @target = @browser
+    end
+
+    # Points the following commands at an iframe, so a tape can reach an API
+    # inside it — a slide deck's own navigation, say — rather than only what
+    # the outer document exposes. Nesting works by switching again from
+    # within, and `Frame main` returns to the page.
+    def frame(selector)
+      return @target = @browser if selector == MAIN_FRAME
+
+      # Reading .frame off the node is a further exchange with the browser, so
+      # it belongs under the lock as much as the lookup that found the node.
+      node = find(selector)
+      @target = locked { node.frame } || raise("#{selector} is not a frame")
+    end
+
+    def press(key, count = '1')
+      Integer(count, 10).times { locked { @browser.keyboard.type(key.to_sym) } }
+    end
+
+    def click(selector)
+      locked { find(selector).click }
+    end
+
+    def enter(selector, text)
+      locked do
+        element = find(selector)
+        element.focus
+        element.type(text)
+      end
+    end
+
+    def pause(duration)
+      sleep(Duration.parse(duration))
+    end
+
+    def screenshot(name = nil)
+      @recorder.capture(name: name)
+    end
+
+    def evaluate(expression)
+      locked { @target.evaluate(expression) }
+    end
+
+    def wait_for_element(selector, timeout = nil)
+      wait_until(timeout, "#{selector} to appear") { element(selector) }
+    end
+
+    def wait_for_js(expression, timeout = nil)
+      wait_until(timeout, "#{expression} to be true") { evaluate(expression) }
+    end
+
+    # Polls until the block has been satisfied WaitStable times in a row. The
+    # streak matters for pages that report readiness before they have settled:
+    # a single true reading can land mid-render, several in a row cannot.
+    #
+    # A block that raises counts as not-yet-satisfied, since a page part-way
+    # through loading will happily throw on a property that is about to
+    # exist. The last error is reported if the wait times out, so a broken
+    # expression still surfaces rather than being silently polled forever.
+    def wait_until(timeout, description)
+      timeout = timeout ? Duration.parse(timeout) : @settings.wait_timeout
+      deadline = monotonic + timeout
+      stable = 0
+      failure = nil
+
+      loop do
+        satisfied = begin
+          yield
+        rescue StandardError => e
+          failure = e
+          false
+        end
+
+        stable = satisfied ? stable + 1 : 0
+        return if stable >= @settings.wait_stable
+        raise Error, timed_out(description, timeout, failure) if monotonic >= deadline
+
+        sleep(@settings.wait_interval)
+      end
+    end
+
+    def timed_out(description, timeout, failure)
+      message = "timed out after #{timeout}s waiting for #{description}"
+      failure ? "#{message} (last error: #{failure.message})" : message
+    end
+
+    def monotonic
+      Process.clock_gettime(Process::CLOCK_MONOTONIC)
+    end
+
+    def find(selector)
+      element(selector) || raise("element not found: #{selector}")
+    end
+
+    def element(selector)
+      locked do
+        next @target.at_css(selector) unless selector.start_with?('text=')
+
+        literal = xpath_literal(selector.delete_prefix('text='))
+        @target.at_xpath("//*[normalize-space(text())=#{literal}]")
+      end
+    end
+
+    def xpath_literal(text)
+      return %("#{text}") unless text.include?('"')
+
+      parts = text.split('"', -1).map { |part| %("#{part}") }
+      "concat(#{parts.join(%q(, '"', ))})"
+    end
+  end
+end