btape 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 83f37a580ea032f35ec04083a301339d43bb35bd13881799774d275527558e43
-  data.tar.gz: 6b47a03f994310102fc3534c621d7fe78e782e6ab39653a6c227b56838ba17c0
+  metadata.gz: f7ea547bca5c9c270e16d779e170c68b0ab30b8bc4156cde58d4cfe7560ebbe4
+  data.tar.gz: fb55fff428c21bfbc889aa653708a1d02c77ffe795c515238716ad9e246fecf6
 SHA512:
-  metadata.gz: 493ebb303b92ba896387e67236c61deaf5cfaed5ba746990e67c9be9ad06d88d1ea34985d3719a87b6edfbbafe06c59b8d052720b05471fc1215589db23da43d
-  data.tar.gz: b5cf6dcfc0995a623f080f30bd5903dc8fed6f456d42f697534a720494d7c8418bcfdeb3917cb0c4537120216382578456642671b849d07b2365c486e9f0a329
+  metadata.gz: 715e86c7d8e3d18834de6350eacac2ddad337d031faf3fae46f57d86364c445679509d9ef9db25e74688df5ac5c8c341a34418d85fb597b82ec312f2d02fd9d9
+  data.tar.gz: 3b67abd203cd1619bc2de88fb4259437a994ec970ef58e81c937ca68efc0ebebc71588942942efa06e6dc56191af8935891c945d5241d87df3c5440787fa18be

data/README.md

@@ -1,18 +1,34 @@
 # btape
 
+<p align="center">
+  <img src="assets/logo.png" alt="btape logo" width="480">
+</p>
+
 `btape` is a small, VHS-inspired Ruby CLI that runs browser actions from a
 `.tape` file and records them as an animated GIF. Ferrum controls Chromium
 and captures PNG frames, and a pure-Ruby encoder produces the GIF. It
 does not require Playwright, Selenium, ffmpeg, or an external service.
 
+Tapes are written by hand, or asked of a language model running on the same
+machine: `btape generate` describes the language to LM Studio, Ollama or
+anything else speaking their API, and holds the answer to the parser before
+handing it over.
+
 ## Commands
 
 ```text
 Output <path>
 Viewport <width>x<height>
+Set <name> <value>
 Goto <url>
 Click <CSS selector or text=Text>
 Type <CSS selector> <text>
+Press <key> [count]
+Frame <CSS selector or main>
+Evaluate <javascript>
+WaitFor <CSS selector or text=Text> [duration]
+WaitForJS <javascript> [duration]
+Screenshot [name]
 Sleep <number>ms|s
 ```
 
@@ -29,6 +45,37 @@ Type "#email" "demo@example.com"
 Sleep 1s
 ```
 
+`Evaluate` runs JavaScript in the current frame, which is how a tape reaches
+an API the page exposes rather than clicking at it. `Frame` points the
+commands that follow at an iframe, and `Frame main` returns to the page;
+navigating returns to the page too, since the frame belonged to the page that
+was left. `WaitFor` and `WaitForJS` poll instead of guessing at a `Sleep`.
+
+`Screenshot` captures a frame there and then. With a name it also lands at a
+predictable path, for picking one particular frame out of a run.
+
+## Settings
+
+`Set NAME VALUE` configures a run. Every setting can also be given on the
+command line with `--set NAME=VALUE`, which wins over the tape, so one tape
+can run in more than one place.
+
+| Name | Default | Meaning |
+| --- | --- | --- |
+| `WsUrl` | — | Connect to a browser already running at this CDP url instead of launching one |
+| `CaptureMode` | `interval` | `interval` records continuously; `manual` captures only where `Screenshot` says to |
+| `Framerate` | `10` | Captures per second in interval mode |
+| `FrameDelay` | `100ms` | How long each frame is shown in the GIF |
+| `Loop` | `0` | Times to loop; 0 is forever |
+| `Scale` | `1.0` | Scale the output down |
+| `OutputWidth` | — | Output width in pixels; overrides `Scale` and keeps the aspect ratio |
+| `Quantizer` | `adaptive` | `adaptive` picks the palette from the frames; `rgb332` uses a fixed one |
+| `Timeout` | `120s` | Give up on the whole run after this |
+| `WaitTimeout` | `10s` | Give up on a `WaitFor` or `WaitForJS` after this |
+| `WaitInterval` | `100ms` | How often those two check |
+| `WaitStable` | `1` | How many checks in a row must pass before a wait is satisfied |
+| `MaxFrames` | `600` | Stop rather than record a hung page until the disk fills |
+
 ## Install and run
 
 Chromium must be installed and discoverable by Ferrum. Then:
@@ -39,13 +86,234 @@ bundle exec btape demo.tape
 bundle exec rake spec
 ```
 
-The recording interval is 100 ms (10 fps). Temporary PNG frames are removed
-after a successful run. They are also isolated in the system temporary
-directory and cleaned when an error unwinds the run.
+```text
+Usage: btape [options] SCRIPT.tape
+
+  --ws-url URL     Connect to a browser already running at this CDP url
+  --set NAME=VALUE Override a setting, as a Set line would
+  --frames-dir DIR Write the PNG frames here and keep them
+  --verbose        Report each command on stderr as it runs
+
+Subcommands:
+  generate DESCRIPTION  Write a tape by asking a local model; btape generate --help
+```
+
+`BTAPE_WS_URL` is used when neither `--ws-url` nor `--set WsUrl=` is given.
+
+### Fonts, and text that is not Latin
+
+Glyphs come from the fonts the browser can see, which is not necessarily the
+machine btape runs on: the host when btape launches Chromium itself, the other
+machine when `Set WsUrl` points at one, and the image when either of those is
+a container. Nothing raises when a script has no coverage — the page records as
+rows of tofu boxes instead — so a font missing from a headless image shows up
+in the GIF and nowhere earlier.
+
+Install fonts covering the scripts the tapes visit. On Debian or Ubuntu:
+
+```sh
+apt-get install fonts-noto-core                # most scripts, Latin included
+apt-get install fonts-noto-cjk                 # Chinese, Japanese, Korean
+apt-get install fonts-ipafont fonts-ipaexfont  # Japanese, as IPAGothic, IPAexGothic, IPAMincho
+```
+
+Fontconfig reads `/usr/share/fonts`, `/usr/local/share/fonts`,
+`~/.local/share/fonts` and `~/.fonts`, and font files copied in by hand need
+an `fc-cache -f` after them. The home directory in that list is the one
+belonging to whoever launches the browser, which under a service manager is
+often not the user who installed the font — `sudo -u deploy fc-list` settles
+that faster than another recording does. Chromium reads the configuration as
+it starts, and btape starts one browser per run; a browser shared over `WsUrl`
+keeps the fonts it was launched with until it is restarted.
+
+Where one installed font covers a script, that is the whole job: the browser
+falls back to it even for a page that asked for `sans-serif`. Naming a family
+matters when several cover the same script — Noto CJK and IPA together, or a
+developer's macOS with Hiragino already on it. Either force it from the tape:
+
+```text
+Evaluate "document.head.insertAdjacentHTML('beforeend', '<style>*{font-family:IPAexGothic!important}</style>')"
+```
+
+which lasts as long as the document it ran in, so it is repeated after each
+`Goto` and inside a `Frame`; or prefer it for everything the machine renders,
+in `/etc/fonts/local.conf`:
+
+```xml
+<?xml version="1.0"?>
+<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
+<fontconfig>
+  <alias><family>sans-serif</family><prefer><family>IPAexGothic</family></prefer></alias>
+  <alias><family>serif</family><prefer><family>IPAexMincho</family></prefer></alias>
+  <alias><family>monospace</family><prefer><family>IPAGothic</family></prefer></alias>
+</fontconfig>
+```
+
+An alias only answers for the generic families. A page naming `Helvetica,
+Arial` ahead of `sans-serif` keeps whatever those resolve to — Liberation Sans,
+in an image that has it — so such a page is served by forcing the family from
+the tape, or by matching those names in the fontconfig file as well. macOS has
+no fontconfig at all, and there the tape is the only route.
+
+A tape can check that the font arrived rather than trust the image it runs in,
+since a family that is not installed measures the same as one that does not
+exist:
+
+```text
+WaitForJS "(() => { const c = document.createElement('canvas').getContext('2d'); const w = (f) => { c.font = '48px ' + f; return c.measureText('AあÄ0').width; }; return w('IPAexGothic') !== w('__missing__'); })()" 3s
+```
+
+Tape files themselves are read as UTF-8 whatever the locale says, so a `Type`
+line or a `text=` selector can be written in any script.
+
+### A browser running somewhere else
+
+btape launches its own Chromium by default. Point it at one that is already
+running — a `browserless`/`chrome` container, say — and no browser needs to be
+in the image btape runs from:
+
+```sh
+btape --ws-url ws://chrome:3000 examples/thumbnails.tape
+```
+
+Each connection gets its own browser context, so concurrent runs against one
+shared browser do not see each other. The viewport is applied over the wire,
+since a browser that is already running cannot be told its window size at
+launch.
+
+### Frames, not just the GIF
+
+Frames are normally written to a temporary directory and removed as the run
+unwinds. `--frames-dir` keeps them:
+
+```sh
+btape --frames-dir frames examples/thumbnails.tape
+```
+
+## Writing a tape with a local model
+
+`btape generate` describes the language to a model running on your own
+machine and asks it for a tape:
+
+```sh
+btape generate "record signing in at localhost:3000 and landing on the dashboard" -o signin.tape
+btape signin.tape
+```
+
+The default is `http://localhost:1234/v1`, which is where LM Studio serves.
+Anything else speaking the same API answers just as well — Ollama on
+`http://localhost:11434/v1`, llama.cpp's server, vLLM — so nothing about the
+description or the page being recorded leaves the machine unless you point
+`--llm-url` somewhere that it does.
+
+```text
+Usage: btape generate [options] DESCRIPTION
+
+  --llm-url URL    The OpenAI-compatible model server to ask
+  --model NAME     Ask for this model rather than whichever one is loaded
+  --temperature N  How freely the model writes; 0.2 by default
+  --context FILE   Give the model this file as context: selectors, notes, markup
+  -o, --out FILE   Write the tape here rather than to standard output
+  --verbose        Report each attempt on stderr
+```
+
+`BTAPE_LLM_URL`, `BTAPE_LLM_MODEL` and `BTAPE_LLM_KEY` stand in for the first
+two flags and for a key, which a local server rarely wants and a proxy in
+front of one usually does. With no model named, the server is asked which one
+it has loaded — the name a download was given is not worth remembering.
+
+The description can be piped in rather than quoted, which is easier for
+anything longer than a line:
+
+```sh
+btape generate --context app/views/sessions/new.html.erb < what-to-record.txt
+```
+
+What comes back is parsed before you see it, and a tape that does not parse
+goes back to the model with the parser's own complaint — `line 4: unknown
+command "Navigate"` — for it to fix, up to three times. That loop is why this
+is worth more than pasting the command list into a chat window: a small model
+reliably invents a command or drops a quote, and just as reliably repairs it
+when told which line. What it cannot know is your markup, so a tape it wrote
+still names selectors that have to be checked against the page. Read it before
+you run it, the way you would read anything else generated for you.
+
+### A model that is not on this machine
+
+`--llm-url` is the whole of the configuration, so a hosted endpoint speaking
+the same API works as well as a local one. Name the model rather than leaving
+it to be discovered: these servers answer `/v1/models` with a catalogue rather
+than with the one thing they have loaded, and the first entry of it is not
+necessarily something that holds a conversation.
+
+```sh
+BTAPE_LLM_KEY=sk-... btape generate --llm-url https://api.openai.com/v1 --model gpt-4.1 "record signing in"
+```
+
+Anthropic serves an OpenAI-compatible layer on the same host as its own API,
+so `--llm-url https://api.anthropic.com/v1 --model claude-opus-5` records too.
+It is meant for trying models rather than for living on, but nothing btape
+asks of it is among the parts that are missing. A model that refuses
+`--temperature` at anything but its default wants `--temperature 1`.
+
+Sending the work somewhere else is the thing to weigh, not the flag. A local
+model keeps the description and the `--context` file on the machine that ran
+the command; a hosted one is handed both, and a context file is usually a page
+of your own markup rather than something you would have published.
+
+## From Ruby
+
+`Runner#run` returns a `Btape::Result`:
+
+```ruby
+commands = Btape::Parser.new.parse(File.read('deck.tape'))
+
+result = Btape::Runner.new(logger: Rails.logger).run(
+  commands,
+  base_directory: File.dirname('deck.tape'),
+  settings: { ws_url: ENV['CHROME_WS_URL'] },
+  frames_directory: frames,
+  on_frame: ->(path, index) { logger.debug("captured #{index}: #{path}") }
+)
+
+result.output_path            # where the GIF went
+result.frame_count            # frames that went into it
+result.frame_paths            # the frames, when frames_directory was given
+result.named_frames['page-01'] # the frame a Screenshot named
+```
+
+Nothing has to touch the filesystem. Pass an IO to write the GIF into:
+
+```ruby
+buffer = StringIO.new(+''.b)
+Btape::Runner.new.run(commands, base_directory: '.', output: buffer)
+
+# `buffer.string` is the GIF. Hand it to whatever holds on to it — an Active
+# Storage attachment on one of your own records, say:
+deck = Deck.find(params[:id])
+deck.animation.attach(io: StringIO.new(buffer.string), filename: 'deck.gif')
+```
+
+or use the encoder on its own, with PNG paths or ChunkyPNG images:
+
+```ruby
+Btape::GifEncoder.new(delay: 150, width: 640).encode(frame_paths) # => String
+```
+
+The generator is a plain object too, so an application that already knows what
+it wants recorded can go from a sentence to a GIF without a file in between:
+
+```ruby
+generator = Btape::LLM::Generator.new(client: Btape::LLM::Client.new(base_url: ENV['BTAPE_LLM_URL']))
+tape = generator.call('record the dashboard loading', context: page_markup)
+
+Btape::Runner.new.run(Btape::Parser.new.parse(tape), base_directory: '.', output: buffer)
+```
 
 ## Container development with dip or wip
 
-The development image contains Ruby and Chromium.
+The development image contains Ruby, Chromium and Latin fonts; tapes that
+record other scripts need fonts for them added to it.
 
 ```sh
 dip provision
@@ -60,11 +328,31 @@ wip dispatch btape examples/demo.tape
 `examples/demo.tape` drives a small static page bundled at
 `examples/demo_app.html`, so the demo is self-contained and needs no other
 service running. Edit the tape (or point `Goto` at a different URL) to record
-something else.
+something else. `examples/thumbnails.tape` shows the other shape of run: one
+frame per page of a deck, against a browser running elsewhere.
+
+## Limitations
+
+The palette is chosen from the frames being encoded, which tracks gradients
+and text edges far more closely than the fixed RGB332 palette earlier versions
+used — but banding compresses well and fidelity does not, so the files are
+larger than they were. `Set Scale` or `Set OutputWidth` are the levers to pull
+back; identical consecutive frames are already collapsed into one held for
+longer. `Set Quantizer rgb332` restores the old palette.
+
+The first matching element is used for `Click` and `Type`.
+
+## Upgrading to 0.2
+
+`Runner#run` returns a `Btape::Result` rather than the output path. Read
+`result.output_path` where the path was used before.
+
+## Contributing
 
-## MVP limitations
+Bug reports and pull requests are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md)
+for the development setup, PR conventions, and how releases are generated.
+This project follows the [Code of Conduct](CODE_OF_CONDUCT.md).
 
-The GIF encoder uses a fixed 256-colour RGB332 palette to stay dependency-light.
-This favors portability over photographic colour fidelity and file size. The
-first matching element is used for `Click` and `Type`.
+## License
 
+[MIT](LICENSE)

data/lib/btape/cli.rb

@@ -1,26 +1,42 @@
 # frozen_string_literal: true
 
+require 'logger'
+require 'optparse'
+require_relative 'generate_command'
+require_relative 'parser'
+
 module Btape
   # Entry point invoked by the `btape` executable: parses argv, runs the
   # script, and reports success or failure.
   class CLI
-    def initialize(out: $stdout, err: $stderr, runner: Runner.new)
+    Options = Struct.new(:help, :settings, :frames_directory, :verbose)
+
+    USAGE = 'Usage: btape [options] SCRIPT.tape'
+    HELP_ARGUMENTS = %w[help -h --help].freeze
+    HELP_COMMANDS = Parser::SIGNATURES.map { |name, arguments| "#{name} #{arguments}".strip }.freeze
+    GENERATE = 'generate'
+    # The two ways a flag can name a browser: --ws-url and --set WsUrl=.
+    WS_URL_KEYS = [:ws_url, 'WsUrl'].freeze
+
+    # The runner is built after the flags are read, so that --verbose can
+    # reach it. Pass one to use it as given.
+    def initialize(out: $stdout, err: $stderr, stdin: $stdin, runner: nil, generator: nil)
       @out = out
       @err = err
+      @stdin = stdin
       @runner = runner
+      @generator = generator
     end
 
-    HELP_COMMANDS = ['Output PATH', 'Viewport WIDTHxHEIGHT', 'Goto URL', 'Click SELECTOR',
-                     'Type SELECTOR TEXT', 'Sleep DURATION'].freeze
-
     def run(argv)
-      return print_help if argv.empty? || %w[help -h --help].include?(argv.first)
-      raise Error, 'usage: btape SCRIPT.tape' unless argv.length == 1
+      return generate(argv.drop(1)) if argv.first == GENERATE
 
-      script = File.expand_path(argv.first)
-      commands = Parser.new.parse(File.read(script))
-      output = @runner.run(commands, base_directory: File.dirname(script))
-      @out.puts "Created #{output}"
+      options = Options.new(false, {}, nil, false)
+      arguments = parse_options(argv, options)
+      return print_help if options.help || arguments.empty? || HELP_ARGUMENTS.include?(arguments.first)
+      raise Error, USAGE unless arguments.length == 1
+
+      record(arguments.first, options)
       0
     rescue Error, SystemCallError => e
       @err.puts "btape: #{e.message}"
@@ -29,11 +45,104 @@ module Btape
 
     private
 
+    def generate(argv)
+      GenerateCommand.new(out: @out, err: @err, stdin: @stdin, generator: @generator).run(argv)
+    end
+
+    def record(argument, options)
+      script = File.expand_path(argument)
+      commands = Parser.new.parse(read_script(script))
+      result = runner(options).run(
+        commands,
+        base_directory: File.dirname(script),
+        settings: settings(options),
+        frames_directory: options.frames_directory
+      )
+      @out.puts "Created #{result.output_path}"
+      report_frames(result)
+    end
+
+    # Tapes are UTF-8, not whatever the locale happens to be. A tape that
+    # types text or matches on it is as likely to be written in Japanese or
+    # Greek as in ASCII, and read through the default external encoding a
+    # machine with LANG unset would reject those bytes while parsing rather
+    # than while recording.
+    def read_script(path)
+      File.read(path, encoding: Encoding::UTF_8)
+    end
+
+    def runner(options)
+      @runner || Runner.new(logger: logger(options))
+    end
+
+    def logger(options)
+      return NullLogger.new unless options.verbose
+
+      Logger.new(@err, level: Logger::DEBUG, formatter: ->(_severity, _time, _program, message) { "#{message}\n" })
+    end
+
+    def report_frames(result)
+      return if result.frame_paths.empty?
+
+      @out.puts "Kept #{result.frame_paths.length} frame(s) in #{File.dirname(result.frame_paths.first)}"
+    end
+
+    # The tape's own `Set` lines are the baseline; these flags override them,
+    # which is what lets one tape run against a local and a remote browser.
+    #
+    # BTAPE_WS_URL is the last resort, and only applies when neither flag named
+    # a browser. Settings resolves --ws-url and --set WsUrl to the same
+    # attribute, so merging the environment in unconditionally would let it
+    # overwrite the --set value rather than fall back to it.
+    def settings(options)
+      return options.settings if WS_URL_KEYS.any? { |key| options.settings.key?(key) }
+
+      options.settings.merge(ws_url: ENV.fetch('BTAPE_WS_URL', nil))
+    end
+
+    def parse_options(argv, options)
+      option_parser(options).parse(argv)
+    rescue OptionParser::ParseError => e
+      raise Error, e.message
+    end
+
+    def option_parser(options)
+      OptionParser.new do |parser|
+        parser.banner = USAGE
+        parser.on('--ws-url URL', 'Connect to a browser already running at this CDP url') do |url|
+          options.settings[:ws_url] = url
+        end
+        parser.on('--set NAME=VALUE', 'Override a setting, as a Set line would') do |pair|
+          name, value = pair.split('=', 2)
+          raise Error, '--set expects NAME=VALUE' if value.nil?
+
+          options.settings[name] = value
+        end
+        parser.on('--frames-dir DIR', 'Write the PNG frames here and keep them') do |directory|
+          options.frames_directory = directory
+        end
+        parser.on('--verbose', 'Report each command on stderr as it runs') { options.verbose = true }
+        parser.on('-h', '--help', 'Show this message') { options.help = true }
+      end
+    end
+
     def print_help
-      @out.puts 'Usage: btape SCRIPT.tape'
+      @out.puts USAGE
+      @out.puts
+      @out.puts 'Options:'
+      @out.puts '  --ws-url URL     Connect to a browser already running at this CDP url'
+      @out.puts '  --set NAME=VALUE Override a setting, as a Set line would'
+      @out.puts '  --frames-dir DIR Write the PNG frames here and keep them'
+      @out.puts '  --verbose        Report each command on stderr as it runs'
+      @out.puts
+      @out.puts 'Subcommands:'
+      @out.puts '  generate DESCRIPTION  Write a tape by asking a local model; btape generate --help'
       @out.puts
       @out.puts 'Commands:'
       HELP_COMMANDS.each { |command| @out.puts "  #{command}" }
+      @out.puts
+      @out.puts 'Settings:'
+      Settings::DEFINITIONS.each_key { |name| @out.puts "  #{name}" }
       0
     end
   end

data/lib/btape/duration.rb

@@ -0,0 +1,26 @@
+# frozen_string_literal: true
+
+module Btape
+  # The duration literals .tape scripts use ("500ms", "1.5s"). The parser,
+  # the runner and Settings all need the same format, so it lives here once
+  # rather than as a regexp repeated at each call site.
+  module Duration
+    PATTERN = /\A(\d+(?:\.\d+)?)(ms|s)\z/
+    DESCRIPTION = 'must use ms or s (for example, 500ms or 1.5s)'
+
+    module_function
+
+    def valid?(value)
+      PATTERN.match?(value.to_s)
+    end
+
+    # Returns the duration in seconds.
+    def parse(value)
+      match = PATTERN.match(value.to_s)
+      raise ArgumentError, "duration #{DESCRIPTION}" unless match
+
+      amount, unit = match.captures
+      amount.to_f / (unit == 'ms' ? 1000 : 1)
+    end
+  end
+end

data/lib/btape/error.rb

@@ -3,6 +3,10 @@
 module Btape
   class Error < StandardError; end
 
+  # Raised when a run outlasts Set Timeout. A page that never finishes
+  # loading would otherwise record until the disk filled.
+  class TimeoutError < Error; end
+
   # Raised for a problem in the .tape script itself, carrying the line
   # number so the CLI can report where the script went wrong.
   class ScriptError < Error