vtparser 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d622edd5fbb0b4f47cb9831699c466aeabc3da9c6bed99370cc7d99d169c8c02
-  data.tar.gz: 33eccc5b843d8c5d734a06d5b2b3d389462c23ea568bf9456ed55a38d6f1a50a
+  metadata.gz: cbb3fedd9971980bdb4389680f7ad7f52a82ec573b9bdecc3a30ecd1fb22f102
+  data.tar.gz: 5f12efd5ffe0f23656dcd2520f835b64815a423f9cbb7353b98eac372b0a2303
 SHA512:
-  metadata.gz: 0e5c6dd30582e24e0ad63b142446280b1be31d51cbae543b78c0019bf626ce2d477ea7bf002ab3707c7dd8056df886d33304eaa2a9509c30a9cac7ac6148e4b9
-  data.tar.gz: ab1f60a0cdd8e079274bb8a7d6a285287b81a3fe1304462086ccdb71910fe09451faadaa4f3ce8935b6a63fdc579a856e069558e7183c8a7c659924e7ad32417
+  metadata.gz: 60c450670c31b4fee417acdefab3f6ed5287351929a6afcbdda46d68d4ba72e5db76731184136d014b9c656b026649fbed0fe5811740532b5402ce7961946fc2
+  data.tar.gz: 2d7ec6bdb03a5ef18df04e461cc199a9fe60d4b838d39f772db14d9aec70b95cbc90f5fc2b2cf42314a9a91d0aff70e5ca48f7185de754ead3e70061acdb501d

data/CHANGELOG.md

@@ -1,5 +1,20 @@
 ## [Unreleased]
 
+## [0.3.0] - 2024-10-05
+
+- Added more examples `colorswap.rb`, `analyze.rb`, `roundtrip.rb`
+- Renamed KeyboardEvent to KeyEvent
+- Added support for BEL instead of ESC to terminate OSC sequences (xterm uses this)
+- Encapsulate action in class Action and added instance methods
+- Fixed serialization issue for private mode CSI commands
+- Added PtySupport::spawn to easily spawn a command and link stdin/out to the parser
+- Switch PTY to `raw` mode to capture keys, disable echo.
+- Added support for piping output to files from examples.
+
+## [0.2.0] - 2024-10-03
+
+- Add keyevent handling
+
 ## [0.1.0] - 2024-10-01
 
 - Initial release

data/README.md

@@ -1,6 +1,10 @@
 # VT 100 Parser Gem
 
-This gem is a parser for VT100 terminal escape sequences. It is based on the C code from https://github.com/haberman/vtparse/.
+This gem is a parser for VT100 terminal escape sequences. It is based on the C code from https://github.com/haberman/vtparse/ and implements the statemachine from https://www.vt100.net/emu/dec_ansi_parser.
+
+The purpose of this Gem is to have a relatively easy way to filter/modify the output of child/sub-processes (for instance launched via `PTY::spawn`) which use animation or colors. 
+
+Uses keyboard mapping logic from https://github.com/vidarh/keyboard_map/
 
 ## Background on VT100 Escape Sequences
 
@@ -22,26 +26,46 @@ gem install vtparser
 
 ## Basic Usage
 
-See the minimal example below and the [`examples`](https://github.com/coezbek/vtparser/examples) directory for more examples.
+See the minimal example below:
 
 ```ruby
-require 'vtparser'
+require_relative '../lib/vtparser'
 
 # Instantiate the parser with a block to handle actions
-parser = VTParser.new do |action, ch, intermediate_chars, params|
+parser = VTParser.new do |action|
 
   # For this minimal example, we'll just turn everything back strings to print
-  print VtParser::to_ansi(action, ch, intermediate_chars, params)
+  print action.to_ansi
 
 end
 
-# Sample input containing ANSI escape sequences
-input = "\e[31mHello, \e[1mWorld!\e[0m"
+# Sample input containing ANSI escape sequences (red text, bold text)
+input = "\e[31mHello, \e[1mWorld!\e[0m\n"
 
 # Parse the input
 parser.parse(input)
 ```
 
+Further samples in the [`examples directory`](https://github.com/coezbek/vtparser/tree/main/examples):
+
+- [`echo_keys.rb`](https://github.com/coezbek/vtparser/tree/main/examples/echo_keys.rb): Echoes the keys pressed by the user
+- [`indent_cli.rb`](https://github.com/coezbek/vtparser/tree/main/examples/indent_cli.rb): Indents the output of simple command line tools
+- [`colorswap.rb`](https://github.com/coezbek/vtparser/tree/main/examples/colorswap.rb): Swaps the colors red / green in the input program
+- [`analyze.rb`](https://github.com/coezbek/vtparser/tree/main/examples/analyze.rb): Output all VT100 escape sequences written by the subprocess.
+- [`roundtrip.rb`](https://github.com/coezbek/vtparser/tree/main/examples/roundtrip.rb): Runs the given command and compares characters written by the command to the output of running the characters through the parser and serializing the actions back `to_ansi`. If the parser works correctly the output should be the same.
+
+## Limitations
+
+- The parser is based on the implementation https://github.com/haberman/vtparse/ and based on a state machine which precedes Unicode. As such it does not have state transitions for Unicode characters. Rather, it will output them as `:ignore` actions. In case unicode characters are used inside escape sequences, the parser will likely not be able to handle them correctly.
+
+- The state machine does not expose all input characters to the implementation in relationship to the `DSC` (Device Control String) sequences. In particular the "Final Character" is swallowed by the statemachine from https://www.vt100.net/emu/dec_ansi_parser. To circumvent this limitation, I have modified the parser to expose the final character as intermediate_chars to the `:hook` action.
+
+- The parser only outputs full `actions`. So triggering an event for the `ESC` key doesn't work (as expected).
+
+- The parser does not emit actions for commands which are 'interrupted' by another command."
+
+- The parser does not explain any of the actions.
+
 ## Development
 
 After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.

data/examples/analyze.rb

@@ -0,0 +1,21 @@
+require_relative '../lib/vtparser'
+require 'pty'
+require 'tty-prompt' # for winsize call below
+
+#
+# This example demonstrates how to use the VTParser to analyze the VT100 escape sequences outputted by a program.
+#
+
+# Get the command from ARGV
+command = ARGV.join(' ')
+if command.empty?
+  puts "Usage: ruby indent_cli.rb '<command>'"
+  exit 1
+end
+
+# Instantiate the parser with a block to handle actions
+parser = VTParser.new do |action|
+  puts action.inspect + "\r\n"
+end
+
+parser.spawn(command)

data/examples/colors.rb

@@ -0,0 +1,18 @@
+#!/usr/bin/env ruby
+
+#
+# This script outputs alternating red and green text to demonstrate the color-swapping functionality.
+#
+
+# ANSI escape sequences for colors
+COLOR_RED = "\e[31m"
+COLOR_GREEN = "\e[32m"
+RESET_COLOR = "\e[0m"
+
+# Infinite loop to print red and green text alternately
+loop do
+  print "\r#{COLOR_RED}This is red text#{RESET_COLOR}    "
+  sleep 1
+  print "\r#{COLOR_GREEN}This is green text#{RESET_COLOR}"
+  sleep 1
+end

data/examples/colorswap.rb

@@ -0,0 +1,40 @@
+require 'pty'
+require 'rainbow/refinement' # for colorizing output
+using Rainbow
+require_relative '../lib/vtparser'
+
+#
+# 'swap_colors_cli.rb' - Example for vtparser
+#
+# This example demonstrates how to use VTParser to swap colors (red to green, green to red) of a simple tty program.
+#
+# Run with `ruby swap_colors_cli.rb <command>`, where <command> is the command you want to run.
+#
+
+# Get the command from ARGV
+command = ARGV.join(' ')
+if command.empty?
+  puts "Usage: ruby swap_colors_cli.rb '<command>'"
+  exit 1
+end
+
+# VT100 color codes for red and green
+COLOR_RED = "\e[31m"
+COLOR_GREEN = "\e[32m"
+RESET_COLOR = "\e[0m"
+
+# VTParser block to swap red and green colors in the output
+parser = VTParser.new do |action, ch, intermediate_chars, params|
+  to_output = VTParser::to_ansi(action, ch, intermediate_chars, params)
+
+  case to_output
+  when COLOR_RED
+    print COLOR_GREEN # Swap red to green
+  when COLOR_GREEN
+    print COLOR_RED # Swap green to red
+  else
+    print to_output # Default behavior for other output
+  end
+end
+
+parser.spawn(command)

data/examples/echo_keys.rb

@@ -0,0 +1,27 @@
+require 'io/console'
+require_relative '../lib/vtparser'
+
+#
+# Example for how to switch to raw mode to output infos about each keypress
+#
+STDIN.raw do |io|
+
+  parser = VTParser.new do |action, ch, intermediate_chars, params|
+
+    puts "  New VTParser action: #{action}, ch: #{ch.inspect}, ch0x: #{ch.ord.to_s(16)}, intermediate_chars: #{intermediate_chars}, params: #{params}\r\n"
+    
+    parser.to_key(action, ch, intermediate_chars, params) do |event|
+
+      puts "    Keyevent: #{event.to_sym.inspect} #{event.inspect}\r\n"
+      exit(1) if event.to_sym == :ctrl_c
+    
+    end
+  end
+
+  loop do
+    ch = $stdin.getch
+    
+    puts "Getch: #{ch.inspect}\r\n"
+    parser.parse ch
+  end
+end

data/examples/indent_cli.rb

@@ -4,6 +4,19 @@ require 'rainbow/refinement' # for colorizing output
 using Rainbow
 require_relative '../lib/vtparser'
 
+#
+# 'indent_cli.rb' - Example for vtparser
+#
+# This example demonstrates how to use the VTParser to indent the output of simple (!) tty programs 
+# with colorized or animated output.
+#
+# Run with `ruby indent_cli.rb <command>`` where <command> is the command you want to run.
+#
+# Two simple examples are included:
+# - A simple spinner animation is included in `examples/spinner.rb`: `ruby indent_cli.rb 'ruby spinner.rb'`
+# - A simple progress bar animation is included in `examples/progress.rb`: `ruby indent_cli.rb 'ruby progress.rb'`
+#
+
 # Get the command from ARGV
 command = ARGV.join(' ')
 if command.empty?
@@ -12,56 +25,103 @@ if command.empty?
 end
 
 line_indent = '   ▐  '.yellow
+line_indent_length = 6
+
+#
+# Use VTParser to process the VT100 escape sequences outputted by nested program and prepend the line_indent text.
+#
 first_line = true
-parser = VTParser.new do |action, ch, intermediate_chars, params|
+parser = VTParser.new do |action|
+
+  ch = action.ch
+  intermediate_chars = action.intermediate_chars
+  params = action.params
+  action_type = action.action_type
+  to_output = action.to_ansi
+
   print line_indent if first_line
   first_line = false
 
-  to_output = VTParser::to_ansi(action, ch, intermediate_chars, params)
+  if $DEBUG && (action_type != :print || !(ch =~ /\P{Cc}/))
+    puts action.inspect
+  end
 
-  case action
+  # Handle newlines, carriage returns, and cursor movement 
+  case action_type
   when :print, :execute, :put, :osc_put
-    if ch == "\n" || ch == "\r"
+    if ch == "\r" # || ch == "\n" 
       print ch
       print line_indent
       next
     end
   when :csi_dispatch
-    if to_output == "\e[2K"
+    if to_output == "\e[2K" # Clear line
       print "\e[2K"
       print line_indent
       next
     else
-      if ch == 'G'
-        # puts "to_output: #{to_output.inspect} action: #{action} ch: #{ch.inspect}"
-        # && parser.params.size == 1
-        print "\e[#{parser.params[0] + 6}G"
-
+      if ch == 'G' # Cursor movement to column
+        print "\e[#{parser.params[0] + line_indent_length}G"
         next
       end
     end
   end
 
+  if $DEBUG && (action_type != :print || !(ch =~ /\P{Cc}/))
+    puts "\r\n"
+    puts action.inspect
+    puts "\r\n"
+    # sleep 5
+  end
+
   print to_output
 end
 
 begin
   PTY.spawn(command) do |stdout_and_stderr, stdin, pid|
 
-    Thread.new do     
-      while pid != nil
-        stdin.write(STDIN.readpartial(1024)) # Requires user to press enter!
+    # Input Thread
+    input_thread = Thread.new do
+
+      STDIN.raw do |io|
+        loop do
+          break if pid.nil?
+          begin
+            if io.wait_readable(0.1)
+              data = io.read_nonblock(1024)
+              stdin.write data
+            end
+          rescue IO::WaitReadable
+            # No input available right now
+          rescue EOFError
+            break
+          rescue Errno::EIO
+            break
+          end
+        end
       end
-    rescue => e
-      puts "Error: #{e}"
-      exit(0)
     end
 
+    # Pipe stdout and stderr to the parser
     begin
-      stdout_and_stderr.winsize = $stdout.winsize
 
+      begin
+        winsize = $stdout.winsize
+      rescue Errno::ENOTTY
+        winsize = [0, 120] # Default to 120 columns
+      end
+      # Ensure the child process has the proper window size, because 
+      #  - tools such as yarn use it to identify tty mode
+      #  - some tools use it to determine the width of the terminal for formatting
+      stdout_and_stderr.winsize = [winsize.first, winsize.last - line_indent_length]
+      
       stdout_and_stderr.each_char do |char|
 
+        char = block.call(char) if block_given?
+        next if char.nil?
+        
+        # puts Action.inspect_char(char) + "\r\n"  
+        # Pass to parser
         parser.parse(char)
 
       end
@@ -72,13 +132,12 @@ begin
     # Wait for the child process to exit
     Process.wait(pid)
     pid = nil
-    exit_status = $?.exitstatus
-    result = exit_status == 0
-
-    # Clear the line, reset the cursor to the start of the line
-    print "\e[2K\e[1G"
+    input_thread.join  
   end
 
 rescue PTY::ChildExited => e
   puts "The child process exited: #{e}"
-end
+end
+
+# Clear and reset the cursor to the start of the line
+puts "\e[2K\e[1G"

data/examples/minimal.rb

@@ -0,0 +1,15 @@
+require_relative '../lib/vtparser'
+
+# Instantiate the parser with a block to handle actions
+parser = VTParser.new do |action, ch, intermediate_chars, params|
+
+  # For this minimal example, we'll just turn everything back strings to print
+  print VTParser::to_ansi(action, ch, intermediate_chars, params)
+
+end
+
+# Sample input containing ANSI escape sequences (red text, bold text)
+input = "\e[31mHello, \e[1mWorld!\e[0m\n"
+
+# Parse the input
+parser.parse(input)

data/examples/progress.rb

@@ -0,0 +1,19 @@
+#!/usr/bin/env ruby
+
+#
+# Helper script to be used to demonstrate `indent_cli.rb`. It displays a full-width animated progress bar.
+# 
+
+require 'ruby-progressbar'
+
+# Hide the cursor
+print "\e[?25l"
+
+loop do
+  progressbar = ProgressBar.create
+  99.times { 
+    progressbar.increment 
+    sleep 0.02
+  }
+  print "\r"
+end

data/examples/roundtrip.rb

@@ -0,0 +1,83 @@
+require_relative '../lib/vtparser'
+require 'pty'
+require 'tty-prompt' # for winsize call below
+
+COLOR_GREEN = "\e[32m"
+RESET_COLOR = "\e[0m"
+
+#
+# This example checks if the output of VTParser is identical to the data fed into parser.
+#
+# Examples you can try:
+#
+# ruby roundtrip.rb 'vim'  # Exit with :q
+# ruby roundtrip.rb 'ls -la'
+# ruby roundtrip.rb 'less' # Exit with q
+# ruby roundtrip.rb 'top'
+# Screensavers:
+# ruby roundtrip.rb 'cmatrix' # sudo apt-get install cmatrix
+# ruby roundtrip.rb 'neo'     # install from https://github.com/st3w/neo
+#    Note: that upon termination a mismatch will be detected, because neo will cancel an on-going CSI sequence by sending an ESC character.
+# 
+
+# Get the command from ARGV
+command = ARGV.join(' ')
+if command.empty?
+  puts "Usage: ruby indent_cli.rb '<command>'"
+  exit 1
+end
+
+captured = []
+previous_actions = []
+previous_characters = []
+
+# Instantiate the parser with a block to handle actions
+parser = VTParser.new do |action|
+  
+  from_parser = action.to_ansi
+  from_parser.each_char.with_index do |char, i|
+    if captured.empty?
+      puts "ERROR: Parser has extra character after parsing: #{char.inspect}\r\n"
+      exit(1)
+    end
+    if char != captured.first
+      puts "\r\n"
+      puts "ERROR: Parser output does not match input with index #{i}: #{char.inspect} != #{captured.first.inspect}\r\n"
+      puts "Current captured characters: #{captured.inspect}\r\n"
+      puts "Current parser output: #{from_parser.inspect}\r\n"
+      puts "Previous characters: #{previous_characters.join.inspect}\r\n"
+
+      puts "\r\n"
+      previous_actions.each_with_index do |prev_action, i|
+        puts "Previous action -#{(previous_actions.length - i).to_s.rjust(2)}: #{prev_action.inspect}\r\n"
+      end
+      puts "Current  action    : #{action.inspect}\r\n"
+      
+      exit(1)
+    end
+    matched_char = captured.shift
+    previous_characters << matched_char
+    if previous_characters.size > 20
+      previous_characters.shift
+    end
+
+    print "#{COLOR_GREEN}.#{RESET_COLOR}"
+  end
+
+  previous_actions << action
+  if previous_actions.size > 20
+    previous_actions.shift
+  end
+
+end
+
+parser.spawn(command) do |char|
+
+  captured << char
+
+  next char
+end
+
+puts
+puts "#{COLOR_GREEN}SUCCESS: Parser output matches input.#{RESET_COLOR}"
+

data/examples/spinner.rb

@@ -0,0 +1,19 @@
+#!/usr/bin/env ruby
+
+#
+# Helper script to be used to demonstrate `indent_cli.rb`
+# 
+
+# See: https://github.com/sindresorhus/cli-spinners/blob/main/spinners.json
+frames = [
+  "⠋", "⠙", "⠹", "⠸", "⠼", "⠴", "⠦", "⠧", "⠇", "⠏"
+]
+interval = 0.08 # 80 milliseconds
+
+# Infinite loop to display the animation until interrupted
+loop do
+  frames.each do |frame|
+    print "\r  #{frame}  " # "\r" moves the cursor back to the start of the line
+    sleep(interval)
+  end
+end

data/lib/vtparser/keymap.rb

@@ -0,0 +1,177 @@
+require 'set'
+
+#
+# Logic taken from vidarh/keyboard_map gem
+#
+# https://github.com/vidarh/keyboard_map
+#
+# See examples/keymap.rb for usage
+#
+
+class KeyEvent
+  attr_reader :modifiers, :key, :args
+
+  def initialize(key, *modifiers, args: nil)
+    @key = key
+    @args = args
+    @modifiers = modifiers.map(&:to_sym).to_set
+  end
+
+  def to_s
+    (modifiers.to_a.sort << key).join('_')
+  end
+
+  def to_sym
+    to_s.to_sym
+  end
+
+  def ==(other)
+    case other
+    when KeyEvent
+      self.modifiers == other.modifiers && self.key == other.key
+    when Symbol
+      self.to_sym == other
+    else
+      self.to_s == other
+    end
+  end
+end
+
+module Keymap
+
+  SINGLE_KEY_EVENT = {
+    "\t" => :tab,
+    "\r" => :enter,
+    "\n" => :enter,
+    "\u007F" => :backspace
+  }.freeze
+
+  CSI_BASIC_MAP = {
+    "A" => :up,
+    "B" => :down,
+    "C" => :right,
+    "D" => :left,
+    "E" => :keypad_5,
+    "F" => :end,
+    "H" => :home,
+  }.freeze
+
+  CSI_TILDE_MAP = {
+    "1"  => :home,
+    "2"  => :insert,
+    "3"  => :delete,
+    "4"  => :end,
+    "5"  => :page_up,
+    "6"  => :page_down,
+    "15" => :f5,
+    "17" => :f6,
+    "18" => :f7,
+    "19" => :f8,
+    "20" => :f9,
+    "21" => :f10,
+    "23" => :f11,
+    "24" => :f12,
+  }.freeze
+
+  SS3_KEY_MAP = {
+    "P" => :f1,
+    "Q" => :f2,
+    "R" => :f3,
+    "S" => :f4,
+  }.freeze
+  
+  def map_modifiers(mod)
+    return [] if mod.nil? || mod < 2
+    modifiers = []
+    mod = mod - 1  # Subtract 1 to align with modifier bits
+    modifiers << :shift if mod & 1 != 0
+    modifiers << :alt   if mod & 2 != 0
+    modifiers << :ctrl  if mod & 4 != 0
+    modifiers
+  end
+
+  def to_key(action, ch, intermediate_chars, params, &block)
+
+    case action
+    when :execute, :print, :ignore
+      # Control characters (e.g., Ctrl+C)
+      key = map_control_character(ch)
+      yield key if key
+    #when :print, :ignore
+      # Regular printable characters
+      #yield KeyEvent.new(ch)
+    when :esc_dispatch
+      # ESC sequences without intermediates
+      if intermediate_chars == ''
+        key = process_esc_sequence(ch)
+        yield key if key
+      else
+        # Handle other ESC sequences if necessary
+      end
+    when :csi_dispatch
+      key = process_csi_sequence(params, intermediate_chars, ch)
+      yield key if key
+    when :collect, :param, :clear
+      # Handled internally; no action needed here
+    else
+      # Handle other actions if necessary
+    end
+  end
+
+  def map_control_character(ch)
+    if SINGLE_KEY_EVENT.key?(ch)
+      return KeyEvent.new(SINGLE_KEY_EVENT[ch])
+    elsif ch.ord.between?(0x01, 0x1A)
+      # Ctrl+A to Ctrl+Z
+      key = (ch.ord + 96).chr
+      return KeyEvent.new(key, :ctrl)
+    else
+      return KeyEvent.new(ch)
+    end
+  end
+
+  def process_esc_sequence(final_char)
+    case final_char
+    when 'Z'
+      # Shift+Tab
+      return KeyEvent.new(:tab, :shift)
+    when "\e"
+      # Double ESC
+      return KeyEvent.new(:esc)
+    else
+      # Meta key (Alt) combinations
+      if final_char.ord.between?(0x20, 0x7E)
+        return KeyEvent.new(final_char, :meta)
+      else
+        # Handle other ESC sequences if necessary
+      end
+    end
+  end
+
+  def process_csi_sequence(params, intermediate_chars, final_char)
+    key = nil
+    modifiers = []
+    params = params.map(&:to_i)
+
+    if intermediate_chars == ''
+      if final_char == '~'
+        # Sequences like ESC [ 1 ~
+        key = CSI_TILDE_MAP[params[0].to_s]
+        modifiers = map_modifiers(params[1]) if params.size > 1
+      else
+        # Sequences like ESC [ A
+        key = CSI_BASIC_MAP[final_char]
+        modifiers = map_modifiers(params[0]) if params.size > 0
+      end
+    else
+      # Handle intermediates if necessary
+    end
+
+    if key
+      return KeyEvent.new(key, *modifiers)
+    else
+      # Handle unrecognized sequences
+    end
+  end
+
+end

data/lib/vtparser/parser.rb

@@ -1,10 +1,76 @@
+require_relative "keymap"
+require_relative "pty_support"
+
+class Action
+
+  attr_reader :action_type, :ch, :private_mode_intermediate_char, :intermediate_chars, :params
+
+  def initialize(action_type, ch, private_mode_intermediate_char, intermediate_chars, params)
+    @action_type = action_type
+    @ch = ch
+    @intermediate_chars = intermediate_chars
+    @private_mode_intermediate_char = private_mode_intermediate_char
+    @params = params
+  end
+
+  def to_s
+    to_ansi
+  end
+
+  def inspect
+    "ansi: #{to_ansi.inspect.ljust(16)} " + 
+    "action: #{@action_type.to_s.ljust(12)} #{Action.inspect_char(@ch)} " + 
+    "private mode: #{"'#{@private_mode_intermediate_char}'".ljust(3)} " + 
+    "params: '#{@params.inspect.ljust(20)}' " + 
+    "intermediate_chars: '#{@intermediate_chars}'"
+  end
+
+  def self.inspect_char(ch)
+    "ch: #{ch.inspect.ljust(4)} (ord=#{ch ? ch.ord.to_s.rjust(4) : "    "}, hex=0x#{ch ? ("%02x" % ch.ord).rjust(6) : "      "})"
+  end
+
+  def to_ansi
+    
+    case @action_type
+    when :print, :execute, :put, :osc_put, :ignore
+      # Output the character
+      return @ch if @ch
+    when :hook
+      return "\eP#{@intermediate_chars}"
+    when :esc_dispatch
+      return "\e#{@intermediate_chars}#{@ch}"
+    when :csi_dispatch
+      # Output ESC [ followed by parameters, intermediates, and final character
+      return "\e[#{@private_mode_intermediate_char}#{@params.join(';')}#{@intermediate_chars}#{@ch}"
+    when :osc_start
+      return "\e]"
+    when :osc_end
+      return '' # "\x07"  # BEL character to end OSC
+    when :unhook
+      return "" # \e must come from ESCAPE state
+    when :clear # Clear action is called when a command is interrupted by a new command (there is unfinished content in the parser)
+
+    else
+      raise "Unknown action type: #{@action_type}"
+    end
+
+    raise
+  end
+
+end
+
+
 class VTParser
-  attr_reader :intermediate_chars, :params
+  attr_reader :private_mode_intermediate_char, :intermediate_chars, :params
+
+  include Keymap
+  include PtySupport
 
   def initialize(&block)
     @callback = block
     @state = :GROUND
     @intermediate_chars = ''
+    @private_mode_intermediate_char = ''
     @params = []
     @ignore_flagged = false
     initialize_states
@@ -64,7 +130,7 @@ class VTParser
         0x3a => :CSI_IGNORE,
         (0x30..0x39) => [:param, :CSI_PARAM],
         0x3b => [:param, :CSI_PARAM],
-        (0x3c..0x3f) => [:collect, :CSI_PARAM],
+        (0x3c..0x3f) => [:private_mode_collect, :CSI_PARAM],
         (0x40..0x7e) => [:csi_dispatch, :GROUND],
       },
       :CSI_PARAM => {
@@ -145,7 +211,9 @@ class VTParser
       },
       :OSC_STRING => {
         :on_entry => :osc_start,
-        (0x00..0x17) => :ignore,
+        (0x00..0x06) => :ignore,
+        (0x07) => [:osc_put, :GROUND],       # BEL character for xterm compatibility
+        (0x08..0x17) => :ignore,
         0x19 => :ignore,
         (0x1c..0x1f) => :ignore,
         (0x20..0x7f) => :osc_put,
@@ -249,15 +317,15 @@ class VTParser
 
   def handle_action(action, ch)
     case action
-    when :execute, :print, :esc_dispatch, :csi_dispatch, :hook, :put, :unhook, :osc_start, :osc_put, :osc_end
-      @callback.call(action, ch, intermediate_chars, params) if @callback
-    when :ignore
-      # Do nothing
-      @callback.call(action, ch, intermediate_chars, params) if @callback 
+    when :private_mode_collect
+      raise "Private mode intermediate char already set" unless @private_mode_intermediate_char.empty?
+      @private_mode_intermediate_char = ch
+      return
     when :collect
       unless @ignore_flagged
         @intermediate_chars << ch
       end
+      return
     when :param
       if ch == ';'
         @params << 0
@@ -267,39 +335,25 @@ class VTParser
         end
         @params[-1] = @params[-1] * 10 + (ch.ord - '0'.ord)
       end
+      return
     when :clear
+
+      # Warning: If ESC is sent in the middle of a command, the command is cleared and there is no callback being called.
+
       @intermediate_chars = ''
+      @private_mode_intermediate_char = ''
       @params = []
       @ignore_flagged = false
-    else
-      @callback.call(:error, ch, intermediate_chars, params) if @callback
-    end
-  end
 
-  def self.to_ansi(action, ch, intermediate_chars, params)
-      
-    case action
-    when :print, :execute, :put, :osc_put, :ignore
-      # Output the character
-      return ch if ch
-    when :hook
-      return "\eP#{intermediate_chars}"
-    when :esc_dispatch
-      return "\e#{intermediate_chars}#{ch}"
-    when :csi_dispatch
-      # Output ESC [ followed by parameters, intermediates, and final character
-      return "\e[#{params.join(';')}#{intermediate_chars}#{ch}"
-    when :osc_start
-      return "\e]"
-    when :osc_end
-      return "\x07"  # BEL character to end OSC
-    when :unhook
-      return "" # \e must come from ESCAPE state
     else
-      raise "Unknown action: #{action}"
-    end
+      # when :execute, :print, :esc_dispatch, :csi_dispatch, :hook, :put, :unhook, :osc_start, :osc_put, :osc_end, :ignore
+      @callback.call(Action.new(action, ch, private_mode_intermediate_char, intermediate_chars, params)) if @callback
 
-    raise
+      @intermediate_chars = ''
+      @private_mode_intermediate_char = ''
+      @params = []
+      @ignore_flagged = false
+    end
   end
 
 end

data/lib/vtparser/pty_support.rb

@@ -0,0 +1,81 @@
+#
+# This module can only be included in class which has a parse method to receive the next character read.
+# 
+module PtySupport
+
+#
+# Spawn the given command using PTY::spawn, connect pipes and calls parse for each character read.
+#
+# Caution: While this command is running, STDIN will be in raw mode.
+#
+# If a block is given, it will be called for each character read, and the character 
+# will be replaced by the block's return value. If nil is returned by block, the character will be dropped. 
+#
+def spawn(command, &block)
+
+  begin
+    PTY.spawn(command) do |stdout_and_stderr, stdin, pid|
+
+      # Input Thread
+      input_thread = Thread.new do
+
+        STDIN.raw do |io|
+          loop do
+            break if pid.nil?
+            begin
+              if io.wait_readable(0.1)
+                data = io.read_nonblock(1024)
+                stdin.write data
+              end
+            rescue IO::WaitReadable
+              # No input available right now
+            rescue EOFError
+              break
+            rescue Errno::EIO
+              break
+            end
+          end
+        end
+      end
+
+      # Pipe stdout and stderr to the parser
+      begin
+
+        begin
+          winsize = $stdout.winsize
+        rescue Errno::ENOTTY
+          winsize = [0, 120] # Default to 120 columns
+        end
+        # Ensure the child process has the proper window size, because 
+        #  - tools such as yarn use it to identify tty mode
+        #  - some tools use it to determine the width of the terminal for formatting
+        stdout_and_stderr.winsize = winsize
+        
+        stdout_and_stderr.each_char do |char|
+
+          char = block.call(char) if block_given?
+          next if char.nil?
+          
+          # puts Action.inspect_char(char) + "\r\n"  
+          # Pass to parser
+          parse(char)
+
+        end
+      rescue Errno::EIO
+        # End of output
+      end
+
+      # Wait for the child process to exit
+      Process.wait(pid)
+      pid = nil
+      input_thread.join
+      return exit_status = $?.exitstatus
+
+  end
+
+  rescue PTY::ChildExited => e
+    puts "The child process exited: #{e}"
+  end
+end
+
+end

data/lib/vtparser/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Vtparser
-  VERSION = "0.1.0"
+  VERSION = "0.3.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: vtparser
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Christopher Oezbek
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2024-10-01 00:00:00.000000000 Z
+date: 2024-10-05 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: tty-prompt
@@ -58,14 +58,28 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.40.0
+        version: '0.40'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.40.0
+        version: '0.40'
+- !ruby/object:Gem::Dependency
+  name: ruby-progressbar
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: A pure Ruby VT100 parser that can be used to parse ANSI escape sequences.
 email:
 - c.oezbek@gmail.com
@@ -73,13 +87,22 @@ executables: []
 extensions: []
 extra_rdoc_files: []
 files:
-- ".bash_history"
 - CHANGELOG.md
 - README.md
 - Rakefile
+- examples/analyze.rb
+- examples/colors.rb
+- examples/colorswap.rb
+- examples/echo_keys.rb
 - examples/indent_cli.rb
+- examples/minimal.rb
+- examples/progress.rb
+- examples/roundtrip.rb
+- examples/spinner.rb
 - lib/vtparser.rb
+- lib/vtparser/keymap.rb
 - lib/vtparser/parser.rb
+- lib/vtparser/pty_support.rb
 - lib/vtparser/version.rb
 homepage: https://github.com/coezbek/vtparser
 licenses: []
@@ -103,7 +126,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.4.19
+rubygems_version: 3.5.20
 signing_key: 
 specification_version: 4
 summary: Pure Ruby VT100 parser

data/.bash_history

@@ -1,117 +0,0 @@
-rspec
-rspec
-rspec
-rspec
-rspec
-rspec
-rspec
-rspec
-rspec
-rspec
-rspec ./spec/vtparser_spec.rb:153
-rspec ./spec/vtparser_spec.rb:153
-rspec ./spec/vtparser_spec.rb:153
-rspec ./spec/vtparser_spec.rb:153
-rspec ./spec/vtparser_spec.rb:153
-rspec ./spec/vtparser_spec.rb:153
-rspec
-rspec
-rspec
-rspec ./spec/vtparser_spec.rb:37
-rspec ./spec/vtparser_spec.rb:37
-rspec
-rspec
-rspec       expect(output).to eq(input)
-rspec ./spec/vtparser_spec.rb:156
-rspec ./spec/vtparser_spec.rb:156
-rspec ./spec/vtparser_spec.rb:156
-rspec ./spec/vtparser_spec.rb:156
-rspec 
-rspec 
-D
-rspec ./spec/vtparser_spec.rb
-rspec ./spec/vtparser_spec.rb:137 
-rspec ./spec/vtparser_spec.rb:137 
-rspec ./spec/vtparser_spec.rb:137 
-rspec ./spec/vtparser_spec.rb:137 
-rspec ./spec/vtparser_spec.rb:137 
-rspec ./spec/vtparser_spec.rb:137 
-rspec ./spec/vtparser_spec.rb:137 
-rspec ./spec/vtparser_spec.rb:137 
-rspec ./spec/vtparser_spec.rb
-rspec ./spec/vtparser_spec.rb
-rspec ./spec/vtparser_spec.rb
-bundle
-rspec ./spec/vtparser_spec.rb
-rspec ./spec/vtparser_spec.rb
-ruby lib/vtparser.rb 'yarn add --dev esbuild from "."'
-ruby lib/vtparser/parser.rb 'yarn add --dev esbuild from "."'
-ruby lib/vtparser/parser.rb 'yarn add --dev esbuild from "."'
-ruby lib/vtparser/parser.rb 'yarn add --dev esbuild from "."'
-ruby lib/vtparser/parser.rb 'yarn add --dev esbuild from "."'
-ruby lib/vtparser/parser.rb 'yarn add --dev esbuild from "."'
-bundle
-bundle
-ruby lib/vtparser/parser.rb 'yarn add --dev esbuild from "."'
-irb
-ruby lib/vtparser/parser.rb 'yarn add --dev esbuild from "."'
-ruby lib/vtparser/parser.rb 'yarn add --dev esbuild from "."'
-ruby lib/vtparser/parser.rb 'yarn add --dev esbuild from "."'
-ruby lib/vtparser/parser.rb 'joe'
-joe
-ruby lib/vtparser/parser.rb 'vim'
-vim
-ruby example/indent_cli.rb 'vim'
-ruby examples/indent_cli.rb 'vim'
-ruby examples/indent_cli.rb 'vim'
-ruby examples/indent_cli.rb 'vim'
-ruby examples/indent_cli.rb 'joe'
-ruby examples/indent_cli.rb 'joe'
-ruby examples/indent_cli.rb 'joe'
-lsss
-less
-less README.md 
-ls
-exit
-ls
-ruby examples/indent_cli.rb 'less'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-less README.md
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/indent_cli.rb 'less README.md'
-ruby examples/test.rb 
-kkllllijjaaa
-ruby examples/test.rb 
-ruby examples/test.rb 
-ruby examples/test.rb 
-ruby examples/test.rb 
-ruby examples/test.rb 
-ruby examples/test.rb 
-ruby examples/test.rb 
-ruby examples/test.rb 
-ruby examples/test.rb 
-ruby examples/test.rb 
-ruby examples/indent_cli.rb
-ruby examples/indent_cli.rb 'bundle gem test17'
-ruby examples/indent_cli.rb 'bundle gem test18'