kaitai-struct-visualizer 0.5 → 0.11

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: '0294e63ea9c355016ef56c48aed683eec1204ee4'
-  data.tar.gz: 49acc6230547d1c10c1978d626cb498a29d2cd0f
+SHA256:
+  metadata.gz: a4e2b67e9f53f3b692de0e98c7c46c9106c2c88e001ef228e0bd3351f83037d2
+  data.tar.gz: 623ee7e33ececcc081b5309746dd8ac9e557e5bc57ae52e3229e71e8ae5a94cf
 SHA512:
-  metadata.gz: 7962d68bdafde103d6cf56408436ec79299e3fc25ed7b76dee888bde59d06e65bc71231732d4c9fe6ba5dfd3ebb5c727c2f394aa9c081af1123c69f39fb20380
-  data.tar.gz: 07ac40819e35d59ae48e3fcdba3f965df4288ec0c6fe45867f236ec3859f0f06ed62b5e1ea5e6e59fdae80c4a1a7d688fd2ff7658da81767e61fb1b389e41302
+  metadata.gz: b7ce146db58aab7d2a3c356263e4fde24a3f52517c729329beca1e6e1a26ae7d9b7ad4de84d0320d8b06f9c81054035a3d4a07dee5b61f6834f7f0adfc39c61a
+  data.tar.gz: '0867f535300fe5ae4d54bbd11562915249b35d51ff6cb57adae13a343c985c34d40cb916be25fe2baf3f111c9ca055bb2c9d5190ccd44bf640aa3b8df4e314fe'

data/LICENSE

@@ -1,7 +1,7 @@
                     GNU GENERAL PUBLIC LICENSE
                        Version 3, 29 June 2007
 
- Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
+ Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
  Everyone is permitted to copy and distribute verbatim copies
  of this license document, but changing it is not allowed.
 
@@ -645,7 +645,7 @@ the "copyright" line and a pointer to where the full notice is found.
     GNU General Public License for more details.
 
     You should have received a copy of the GNU General Public License
-    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+    along with this program.  If not, see <https://www.gnu.org/licenses/>.
 
 Also add information on how to contact you by electronic and paper mail.
 
@@ -664,11 +664,11 @@ might be different; for a GUI interface, you would use an "about box".
   You should also get your employer (if you work as a programmer) or school,
 if any, to sign a "copyright disclaimer" for the program, if necessary.
 For more information on this, and how to apply and follow the GNU GPL, see
-<http://www.gnu.org/licenses/>.
+<https://www.gnu.org/licenses/>.
 
   The GNU General Public License does not permit incorporating your program
 into proprietary programs.  If your program is a subroutine library, you
 may consider it more useful to permit linking proprietary applications with
 the library.  If this is what you want to do, use the GNU Lesser General
 Public License instead of this License.  But first, please read
-<http://www.gnu.org/philosophy/why-not-lgpl.html>.
+<https://www.gnu.org/licenses/why-not-lgpl.html>.

data/README.md

@@ -1,65 +1,125 @@
 # Kaitai Struct: visualizer
 
-This is a simple visualizer for [Kaitai Struct](https://github.com/kaitai-io/kaitai_struct) project.
+This is a console visualizer for the [Kaitai Struct](https://kaitai.io/) project.
+
+![screenshot](screenshot.png)
 
 Kaitai Struct is a declarative language used for describe various
 binary data structures, laid out in files or in memory: i.e. binary
 file formats, network stream packet formats, etc.
 
 The main idea is that a particular format is described in Kaitai
-Struct language (`.ksy` files) only once and then can be compiled with
-this compiler into source files in one of the supported programming
-languages. These modules will include a generated code for a parser
-that can read described data structure from a file / stream and give
-access to it in a nice, easy-to-comprehend API.
+Struct language (`.ksy` file) and then can be compiled with
+`ksc` into source files in one of the supported programming
+languages. These modules will contain generated code for a parser
+that can read the described data structure from a file / stream and provide
+access to it using a nice, easy-to-understand API.
 
-Please refer to [documentation in Kaitai Struct project](https://github.com/kaitai-io/kaitai_struct)
-for details on `.ksy` files and general usage patterns.
+See the [Kaitai Struct homepage](https://kaitai.io/) for details on `.ksy` files and general usage patterns.
 
 ## Downloading and installing
 
-### From Ruby Gems repository
+### Requirements
+
+- [ksc](https://kaitai.io/#download) — `kaitai-struct-compiler`
+- [Java](https://whichjdk.com/) (the latest LTS version 21 recommended, at least Java 8 required),
+  [JDK or JRE](https://whichjdk.com/#what-is-the-difference-between-jdk-and-jre) at your option
+- [Ruby](https://www.ruby-lang.org/) (the latest Ruby 3.x recommended, at least Ruby 2.4 required)
+
+### From the RubyGems repository
 
-KS visualizer is written in [Ruby](https://www.ruby-lang.org/) and is
-available as
-[.gem package](https://rubygems.org/gems/kaitai-struct-visualizer). Thus,
-you'll need Ruby (RubyGems package manager comes bundled with Ruby
-since v1.9) installed on your box, and then you can just run:
+Kaitai Struct visualizer is written in [Ruby](https://www.ruby-lang.org/) and is
+available [on RubyGems](https://rubygems.org/gems/kaitai-struct-visualizer). Thus,
+you'll need Ruby installed on your box and then you can just run:
 
 ```shell
 gem install kaitai-struct-visualizer
 ```
 
+---
+
+You can use `ksv --version` to check what versions of `ksv` and its dependencies are installed, for example:
+
+```console
+$ ksv --version
+kaitai-struct-visualizer 0.11
+kaitai-struct-compiler 0.11
+kaitai-struct 0.11 (Kaitai Struct runtime library for Ruby)
+```
+
+The versions of `kaitai-struct-compiler` and `kaitai-struct` should match. If not, see https://kaitai.io/#download for instructions how to install the latest version of `kaitai-struct-compiler` and/or use `gem update kaitai-struct` to update the [kaitai-struct](https://rubygems.org/gems/kaitai-struct) gem if needed.
+
 ### Source code
 
 If you're interested in developing the visualizer itself, you can check
-out source code in repository:
+out the source code from the [kaitai_struct_visualizer](https://github.com/kaitai-io/kaitai_struct_visualizer) GitHub repository:
 
-    git clone https://github.com/kaitai-io/kaitai_struct_visualizer
+```shell
+git clone https://github.com/kaitai-io/kaitai_struct_visualizer.git
+```
+
+Then run `bundle install` to install dependencies. After that, you can run [`bin/ksv`](bin/ksv) or [`bin/ksdump`](bin/ksdump) right away (without having to install the `kaitai-struct-visualizer` gem first), which makes development easier.
 
 ## Usage
 
-`ksv <binary-file> <ksy-file>...`
+There are two executables provided by this package:
+
+* `ksv` — interactive console visualizer with GUI
+* `ksdump` — command-line tool for dumping parsed data in JSON, XML or YAML format to standard output (_stdout_)
+
+The basic usage is similar for both programs:
+
+```shell
+ksv <file_to_parse.bin> <format.ksy>|<format.rb>
+```
+
+For `ksdump`, it may be useful to change the output format with the `-f` option (the default is `yaml`) and redirect the output to a file so that your terminal is not flooded with thousands of lines (for larger input files):
+
+```shell
+ksdump -f json <file_to_parse.bin> <format.ksy> > output.json
+```
+
+### Running with Docker
+
+This project is also available via the [kaitai/ksv](https://hub.docker.com/r/kaitai/ksv) image on Docker Hub. The default entrypoint is `ksv` (the interactive visualizer):
+
+```shell
+docker run --rm -it -v "$(pwd):/share" kaitai/ksv <file_to_parse.bin> <format.ksy>
+```
+
+You can specify `ksdump` as the entrypoint like this (and note that we don't need the `-it` flags anymore because `ksdump` is not interactive — omitting them in fact allows you to distinguish the `ksdump`'s output to _stdout_ and _stderr_, see [this comment](https://github.com/kaitai-io/kaitai_struct_visualizer/issues/56#issuecomment-1666629764)):
+
+```shell
+docker run --rm -v "$(pwd):/share" --entrypoint ksdump kaitai/ksv -f json <file_to_parse.bin> <format.ksy> > output.json
+```
+
+---
+
+Building the Docker image locally:
+```shell
+docker build . --tag docker.io/kaitai/ksv
+```
 
 ## Licensing
 
-Kaitai Struct visualizer itself is copyright (C) 2015-2016 Kaitai
-Project.
+Kaitai Struct visualizer is copyright (C) 2015-2025 Kaitai Project.
 
 This program is free software: you can redistribute it and/or modify
 it under the terms of the GNU General Public License as published by
-the Free Software Foundation, either version 3 of the License, or (at
-your option) any later version.
+the Free Software Foundation, either version 3 of the License, or
+(at your option) any later version.
 
-This program is distributed in the hope that it will be useful, but
-WITHOUT ANY WARRANTY; without even the implied warranty of
-MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-General Public License for more details.
+This program is distributed in the hope that it will be useful,
+but WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+GNU General Public License for more details.
 
 You should have received a copy of the GNU General Public License
-along with this program.  If not, see <http://www.gnu.org/licenses/>.
+along with this program.  If not, see <https://www.gnu.org/licenses/>.
+
+---
 
-Note that it applies only to compiler itself, not `.ksy` input files
+Note that it applies only to visualizer itself, not `.ksy` input files
 that one supplies in normal process of compilation, nor to compiler's
-output files — that consitutes normal usage process and you obviously
+output files — that constitutes normal usage process and you obviously
 keep copyright to both.

data/bin/ksdump

@@ -0,0 +1,96 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'optparse'
+
+# Some additional magic to make it work right after repo checkout,
+# without installation to proper Ruby library dirs
+
+# The `__dir__` method is supposed to provide the real absolute path of the
+# `bin/` directory (which this script is in), but it can technically return
+# `nil` or a dot `"."` (in an "irb" session) instead. Normally, it shouldn't
+# happen unless some very unusual startup method is used, but if it happens,
+# we'll skip adding the dirs to $LOAD_PATH and wait for a miracle (which might
+# be that the dependencies we need are already available via $LOAD_PATH because
+# they are installed as gems, for example).
+#
+# See also
+# * https://stackoverflow.com/a/2206731
+# * https://makandracards.com/makandra/42122-ruby-__file__-__dir__-and-symlinks
+script_dir = __dir__
+unless script_dir.nil? || script_dir == '.'
+  $LOAD_PATH << File.expand_path('../lib', script_dir)
+  $LOAD_PATH << File.expand_path('../../runtime/ruby/lib', script_dir)
+end
+
+require 'kaitai/struct/visualizer'
+require 'kaitai/struct/visualizer/obj_to_h'
+
+# ======================================================================
+
+FORMATS = %w[json xml yaml].freeze
+
+prog_name = File.basename($PROGRAM_NAME)
+options = { format: 'yaml' }
+parser = OptionParser.new do |opts|
+  opts.banner = "Usage: #{prog_name} [options] <file_to_parse.bin> <format.ksy>..."
+  opts.separator ''
+
+  opts.on('-I', '--import-path [DIRECTORIES]', '.ksy library search path(s) for imports (see also KSPATH env variable)') do |v|
+    options[:import_path] = v
+  end
+
+  opts.on('-f', '--format FORMAT', FORMATS, "choose dump format - #{FORMATS.join(', ')} (default: #{options[:format]})") do |v|
+    options[:format] = v
+  end
+
+  opts.on('--version', 'show versions of kaitai-struct-visualizer, kaitai-struct-compiler and kaitai-struct (Kaitai Struct runtime library for Ruby)') do |_v|
+    puts "kaitai-struct-visualizer #{Kaitai::Struct::Visualizer::VERSION}"
+    if system('kaitai-struct-compiler', '--version').nil?
+      $stderr.puts "#{prog_name}: unable to find and execute kaitai-struct-compiler in your PATH"
+      exit 1
+    end
+    require 'kaitai/struct/struct'
+    puts "kaitai-struct #{Kaitai::Struct::VERSION} (Kaitai Struct runtime library for Ruby)"
+    exit 0
+  end
+end
+
+begin
+  parser.parse!
+rescue OptionParser::InvalidOption => e
+  puts e
+  puts parser
+  exit 1
+end
+
+if ARGV.size < 2
+  puts parser
+  exit 1
+end
+
+c = Kaitai::Struct::Visualizer::KSYCompiler.new(options, prog_name)
+app = Kaitai::Struct::Visualizer::Parser.new(c, ARGV[0], ARGV[1..-1], options)
+exc = app.load
+raise exc if exc
+
+# ======================================================================
+
+tree = Kaitai::Struct::Visualizer.obj_to_h(app.data)
+r = nil
+
+case options[:format]
+when 'json'
+  require 'json'
+  r = JSON.pretty_generate(tree)
+when 'xml'
+  require 'active_support'
+  require 'active_support/core_ext'
+  r = tree.to_xml
+when 'yaml'
+  require 'yaml'
+  r = tree.to_yaml
+  r = r[4..-1] if r[0..3] == "---\n"
+end
+
+puts r

data/bin/ksv

@@ -1,20 +1,66 @@
 #!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require 'optparse'
 
 # Some additional magic to make it work right after repo checkout,
 # without installation to proper Ruby library dirs
 
-full_bin_path = File.realpath($PROGRAM_NAME)
-dist_path = File.expand_path(File.dirname(full_bin_path) + '/..')
-
-$LOAD_PATH << "#{dist_path}/lib"
-$LOAD_PATH << "#{dist_path}/../runtime/ruby/lib"
+# See `bin/ksdump` for more comments
+script_dir = __dir__
+unless script_dir.nil? || script_dir == '.'
+  $LOAD_PATH << File.expand_path('../lib', script_dir)
+  $LOAD_PATH << File.expand_path('../../runtime/ruby/lib', script_dir)
+end
 
 require 'kaitai/struct/visualizer'
 
+# ======================================================================
+
+prog_name = File.basename($PROGRAM_NAME)
+options = {}
+parser = OptionParser.new do |opts|
+  opts.banner = "Usage: #{File.basename($PROGRAM_NAME)} [options] <file_to_parse.bin> <format.ksy>...|<format.rb>"
+  opts.separator ''
+
+  opts.on('-I', '--import-path [DIRECTORIES]', '.ksy library search path(s) for imports (see also KSPATH env variable)') do |v|
+    options[:import_path] = v
+  end
+
+  opts.on('--opaque-types [BOOLEAN]', 'opaque types allowed, default: false') do |v|
+    options[:opaque_types] = v
+  end
+
+  opts.on('-r', '--require [PATH]', 'load ("require") .rb file into Ruby process') do |v|
+    $LOAD_PATH << '.' unless $LOAD_PATH.include?('.')
+    require v
+  end
+
+  opts.on('--version', 'show versions of kaitai-struct-visualizer, kaitai-struct-compiler and kaitai-struct (Kaitai Struct runtime library for Ruby)') do |_v|
+    puts "kaitai-struct-visualizer #{Kaitai::Struct::Visualizer::VERSION}"
+    if system('kaitai-struct-compiler', '--version').nil?
+      $stderr.puts "#{prog_name}: unable to find and execute kaitai-struct-compiler in your PATH"
+      exit 1
+    end
+    require 'kaitai/struct/struct'
+    puts "kaitai-struct #{Kaitai::Struct::VERSION} (Kaitai Struct runtime library for Ruby)"
+    exit 0
+  end
+end
+
+begin
+  parser.parse!
+rescue OptionParser::InvalidOption => e
+  puts e
+  puts parser
+  exit 1
+end
+
 if ARGV.size < 2
-  puts "Usage: #{File.basename($PROGRAM_NAME)} <file_to_parse.bin> <format.yaml>..."
+  puts parser
   exit 1
 end
 
-v = Kaitai::Struct::Visualizer::ExternalCompilerVisualizer.new(ARGV[0], ARGV[1..-1])
+c = Kaitai::Struct::Visualizer::KSYCompiler.new(options, prog_name)
+v = Kaitai::Struct::Visualizer::Visualizer.new(c, ARGV[0], ARGV[1..-1], options)
 v.run

data/lib/kaitai/console_ansi.rb

@@ -1,119 +1,124 @@
-# coding: utf-8
+# frozen_string_literal: true
+
 require 'io/console'
 require 'readline'
 
 module Kaitai
+  class ConsoleANSI
+    attr_reader :cols, :rows
 
-class ConsoleANSI
-  attr_reader :cols
-  attr_reader :rows
+    def initialize
+      load_term_size
 
-  def initialize
-    # Normal POSIX way to determine console parameters
-    @cols = `tput cols`.to_i
-    @rows = `tput lines`.to_i
+      @seq_clear = `tput clear`
+      @seq_sgr0 = `tput sgr0`
 
-    @seq_clear = `tput clear`
-    @seq_sgr0 = `tput sgr0`
+      @seq_fgcolor = []
+      @seq_bgcolor = []
 
-    @seq_fgcolor = []
-    @seq_bgcolor = []
-  end
+      @on_resize = nil
 
-  def clear
-    print @seq_clear
-  end
+      Signal.trap('SIGWINCH', proc {
+        load_term_size
+        @on_resize&.call(true)
+      })
+    end
 
-  ##
-  # Put the cursor up to screen position (x, y). First line is 0,
-  # first column is 0.
-  def goto(x, y)
-    #print `tput cup #{y} #{x}`
-    printf "\e[%d;%dH", y + 1, x + 1
-  end
+    attr_writer :on_resize
 
-  COLORS = {
-    :black => 0,
-    :gray => 7,
-    :gray0 => 232,
-    :gray1 => 233,
-    :gray2 => 234,
-    :gray3 => 235,
-    :gray4 => 236,
-    :gray5 => 237,
-    :gray6 => 238,
-    :gray7 => 239,
-    :gray8 => 240,
-    :gray9 => 241,
-    :gray10 => 242,
-    :gray11 => 243,
-    :gray12 => 244,
-    :gray13 => 245,
-    :gray14 => 246,
-    :gray15 => 247,
-    :gray16 => 248,
-    :gray17 => 249,
-    :gray18 => 250,
-    :gray19 => 251,
-    :gray20 => 252,
-    :gray21 => 253,
-    :gray22 => 254,
-    :gray23 => 255,
-  }
-
-  def fg_color=(col)
-    #print @seq_fgcolor[col] ||= `tput setaf #{col}`
-    code = COLORS[col]
-    raise "Invalid color: #{col}" unless code
-    print "\e[38;5;#{code}m"
-  end
+    def load_term_size
+      @rows, @cols = IO.console.winsize
+    end
 
-  def bg_color=(col)
-    #print @seq_bgcolor[col] ||= `tput setab #{col}`
-    code = COLORS[col]
-    raise "Invalid color: #{col}" unless code
-    print "\e[48;5;#{code}m"
-  end
+    def clear
+      print @seq_clear
+    end
 
-  def reset_colors
-    print @seq_sgr0
-  end
+    # Put the cursor up to screen position (x, y). First line is 0, first column is 0.
+    def goto(x, y)
+      # print `tput cup #{y} #{x}`
+      printf "\e[%d;%dH", y + 1, x + 1
+    end
+
+    COLORS = {
+      black: 0,
+      red: 1,
+      green: 2,
+      yellow: 3,
+      blue: 4,
+      magenta: 5,
+      cyan: 6,
+      white: 7,
+      gray: 8,
+      bright_red: 9,
+      bright_green: 10,
+      bright_yellow: 11,
+      bright_blue: 12,
+      bright_magenta: 13,
+      bright_cyan: 14,
+      bright_white: 15
+    }.freeze
+
+    def fg_color=(col)
+      # print @seq_fgcolor[col] ||= `tput setaf #{col}`
+      code = COLORS[col]
+      raise "Invalid color: #{col}" unless code
+
+      print "\e[38;5;#{code}m"
+    end
 
-  # Reads keypresses from the user including 2 and 3 escape character sequences.
-  def read_char
-    $stdin.echo = false
-    $stdin.raw!
+    def bg_color=(col)
+      # print @seq_bgcolor[col] ||= `tput setab #{col}`
+      code = COLORS[col]
+      raise "Invalid color: #{col}" unless code
 
-    input = $stdin.getc.chr
-    if input == "\e" then
-      input << $stdin.read_nonblock(3) rescue nil
-      input << $stdin.read_nonblock(2) rescue nil
+      print "\e[48;5;#{code}m"
     end
-  ensure
-    $stdin.echo = true
-    $stdin.cooked!
 
-    return input
-  end
+    def reset_colors
+      print @seq_sgr0
+    end
 
-  def read_char_mapped
-    c = read_char
-    c2 = KEY_MAP[c]
-    c2 ? c2 : c
-  end
+    # Reads keypresses from the user including 2 and 3 escape character sequences.
+    def read_char
+      $stdin.echo = false
+      $stdin.raw!
+
+      input = $stdin.getc.chr
+      if input == "\e"
+        begin
+          # may return less than 3 bytes because it can only read as many bytes as are
+          # currently available
+          up_to_3bytes = $stdin.read_nonblock(3)
+          input << up_to_3bytes
+        rescue IO::WaitReadable
+          # not an ANSI sequence - the user probably just pressed the Esc key
+        end
+      end
+
+      $stdin.echo = true
+      $stdin.cooked!
+
+      input
+    end
 
-  KEY_MAP = {
-    "\t" => :tab,
-    "\r" => :enter,
-    "\e[A" => :up_arrow,
-    "\e[B" => :down_arrow,
-    "\e[C" => :right_arrow,
-    "\e[D" => :left_arrow,
-    "\e[5~" => :pg_up,
-    "\e[6~" => :pg_dn,
-    "\e[H" => :home,
-    "\e[F" => :end,
-  }
-end
+    def read_char_mapped
+      c = read_char
+      c2 = KEY_MAP[c]
+      c2 || c
+    end
 
+    KEY_MAP = {
+      "\t" => :tab,
+      "\r" => :enter,
+      "\e[A" => :up_arrow,
+      "\e[B" => :down_arrow,
+      "\e[C" => :right_arrow,
+      "\e[D" => :left_arrow,
+      "\e[5~" => :pg_up,
+      "\e[6~" => :pg_dn,
+      "\e[H" => :home,
+      "\e[F" => :end
+    }.freeze
+  end
 end