rundoc 4.1.3 → 5.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: b65a04eca7d7099223b2a626c47eb78e14a8e102266cf880cddc747a0d9fc4a5
-  data.tar.gz: 840585e8f0c6b3cddbb1d15354f4fdf862d2440b20054cd1b5344c026adb23d9
+  metadata.gz: 85fbfa9f84b4e98eb72d487721e7f35001f7820404e30a138de4b4505d788f53
+  data.tar.gz: 1ff08ba1cdd468b190fc54eef124b7ab434d700ffe87ba2b4c8e676d03fb515e
 SHA512:
-  metadata.gz: 99661d14543918c21a988bd1c4cffc1d5c9d73a852a95c16b0905b5ddde312dc986d970b4082921752cd2eac4d786a0ae37f42f3577fdea2df54c6de50376810
-  data.tar.gz: 6819d3babc4cb2a7ad1f76386e3d7e7f28886a989e6a31ef052ea87b3b226a6979cc1c2092653fa7aadadd34d3f8af9b268464c2a14c70f385b559ee8a13ca9a
+  metadata.gz: 8e87c610bcd8c8589407e68b503eee0578a1dc23b959baa138258185f7d08e6daa3ee17097e4b66b02fea896ee9e5be2bde936e2a601a2b18d1b1a2fc24b36a2
+  data.tar.gz: 9c9f4d7446beaedc74d61354794327cd30f372f7e5e16500678247f7bc549bb7b04992d0875c06c1fa06ae894a36c35902e07930cf55ab4e9fc01bb8b29c44d3

data/.github/workflows/ci.yml

@@ -11,9 +11,9 @@ jobs:
       fail-fast: false
       matrix:
         ruby:
-          - 3.1
           - 3.2
           - 3.3
+          - "4.0"
           - head
     steps:
       - name: Checkout code
@@ -34,6 +34,7 @@ jobs:
       matrix:
         ruby:
           - 3.3
+          - "4.0"
           - head
     steps:
       - name: Checkout code

data/.standard.yml

@@ -1,6 +1,6 @@
 fix: true               # default: false
 parallel: true          # default: false
 format: progress        # default: Standard::Formatter
-ruby_version: 3.1       # default: RUBY_VERSION
+ruby_version: 3.2       # default: RUBY_VERSION
 ignore:                 # default: []
   - 'test/fixtures/**/*'

data/CHANGELOG.md

@@ -1,5 +1,17 @@
 ## HEAD
 
+## 5.0.0
+
+- Added comment syntax. Use an octothorpe (`#`) after the visibility markers to comment out any commands and make them a no-op.
+- Changed: Minimum Ruby version is now 3.2 (Ruby 3.1 reached EOL in March 2025).
+- Changed: All code commands now use an Args + Runner class pattern. `register_code_command` now requires keyword arguments: `keyword:`, `args_klass:`, and `runner_klass:`.
+- Fix: `seattle_method` parser rule no longer matches across newlines. Previously, a command with no same-line arguments (e.g. `:::>> rundoc`) would consume the next line as its argument, losing the newline between content lines. (https://github.com/zombocom/rundoc/pull/118)
+
+## 4.1.4
+
+- Fix: Net::ReadTimeout errors on `website.visit` are now retried by default (https://github.com/zombocom/rundoc/pull/103)
+- Fix: When a background process (`background.start`) does not exit cleanly, print its logs for help with debugging. (https://github.com/zombocom/rundoc/pull/104)
+
 ## 4.1.3
 
 - Fix: Internal error in `background.wait` introduced in 4.1.2 (https://github.com/zombocom/rundoc/pull/97)

data/README.md

@@ -660,36 +660,63 @@ This command `filter_sensitive` can be called multiple times with different valu
 
 ## Writing a new command
 
+> Note: This is an advanced topic and this interface is unstable.
+
 Rundoc does not have a stable internal command interface. You can define your own commands, but unless it is committed in this repo, it may break on a minor version change.
 
 To add a new command it needs to be parsed and called. Examples of commands being implemented are seen in `lib/rundoc/code_command`.
 
-A new command needs to be registered:
+Each command is split into two classes: an **Args** class that handles argument parsing/validation and a **Runner** class that handles execution. Both must be registered:
 
+```ruby
+Rundoc.register_code_command(
+  keyword: :lol,
+  args_klass: Rundoc::CodeCommand::LolArgs,
+  runner_klass: Rundoc::CodeCommand::LolRunner
+)
 ```
-Rundoc.register_code_command(:lol, Rundoc::CodeCommand::Lol)
-```
 
-They should inherit from Rundoc::CodeCommand:
+The Args class is a plain Ruby class. Its initialize method receives input from the document and exposes parsed values via `attr_reader`:
+
+```ruby
+class Rundoc::CodeCommand::LolArgs
+  attr_reader :message
 
+  def initialize(message)
+    @message = message
+  end
+end
 ```
-class Rundoc::CodeCommand::Lol < Rundoc::CodeCommand
-  def initialize(line)
+
+The Runner class is namespaced under `Rundoc::CodeCommand` and receives the args instance via `user_args:`:
+
+```ruby
+class Rundoc::CodeCommand::LolRunner
+  def initialize(user_args:, **)
+    @message = user_args.message
+  end
+
+  def to_md(env = {})
+    ""
+  end
+
+  def call(env = {})
+    @message
   end
 end
 ```
 
-The initialize method is called with input from the document. The command is rendered (`:::>-`) by the output of the `def call` method. The contents produced by the command (`:::->`) are rendered by the `def to_md` method.
+The command is rendered (`:::>-`) by the output of the `def call` method. The contents produced by the command (`:::->`) are rendered by the `def to_md` method.
 
 The syntax for commands is ruby-ish but it is a custom grammar implemented in `lib/peg_parser.rb` for more info on manipulating the grammar see this tutorial on how I added keword-like/hash-like syntax https://github.com/schneems/implement_ruby_hash_syntax_with_parslet_example.
 
-Command initialize methods natively support:
+Args class initialize methods natively support:
 
 - Barewords as a single string input
 - Keyword arguments
 - A combination of the two
 
-Anything that is passed to the command via "stdin" is available via a method `self.contents`. The interplay between the input and `self.contents` is not strongly defined.
+Anything that is passed to the command via "stdin" is available on the Runner via a method `self.contents`. The interplay between the input and `self.contents` is not strongly defined.
 
 ## Copyright
 

data/lib/rundoc/cli.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 module Rundoc
   class CLI
     module DEFAULTS
@@ -159,7 +161,8 @@ module Rundoc
       Dir.chdir(execution_context.output_dir) do
         parser = Rundoc::Document.new(
           source_contents,
-          context: execution_context
+          context: execution_context,
+          io: io
         )
         output = begin
           parser.to_md
@@ -176,8 +179,8 @@ module Rundoc
       Rundoc::CodeCommand::Background::ProcessSpawn.tasks.each do |name, task|
         next unless task.alive?
 
-        io.puts "Warning background task is still running, cleaning up: name: #{name}"
-        task.stop
+        io.puts "Warning background task is still running, cleaning up: `#{name}`"
+        task.stop(print_io: io)
       end
 
       if execution_context.output_dir.exist?

data/lib/rundoc/code_command/background/log/clear.rb

@@ -1,7 +1,17 @@
+# frozen_string_literal: true
+
 class Rundoc::CodeCommand::Background::Log
-  class Clear < Rundoc::CodeCommand
+  class ClearArgs
+    attr_reader :name
+
     def initialize(name:)
       @name = name
+    end
+  end
+
+  class ClearRunner
+    def initialize(user_args:, render_command:, render_result:, io: nil, contents: nil)
+      @name = user_args.name
       @background = nil
     end
 
@@ -19,4 +29,4 @@ class Rundoc::CodeCommand::Background::Log
     end
   end
 end
-Rundoc.register_code_command(:"background.log.clear", Rundoc::CodeCommand::Background::Log::Clear)
+Rundoc.register_code_command(keyword: :"background.log.clear", args_klass: Rundoc::CodeCommand::Background::Log::ClearArgs, runner_klass: Rundoc::CodeCommand::Background::Log::ClearRunner)

data/lib/rundoc/code_command/background/log/read.rb

@@ -1,7 +1,17 @@
+# frozen_string_literal: true
+
 class Rundoc::CodeCommand::Background::Log
-  class Read < Rundoc::CodeCommand
+  class ReadArgs
+    attr_reader :name
+
     def initialize(name:)
       @name = name
+    end
+  end
+
+  class ReadRunner
+    def initialize(user_args:, render_command:, render_result:, io: nil, contents: nil)
+      @name = user_args.name
       @background = nil
     end
 
@@ -18,4 +28,4 @@ class Rundoc::CodeCommand::Background::Log
     end
   end
 end
-Rundoc.register_code_command(:"background.log.read", Rundoc::CodeCommand::Background::Log::Read)
+Rundoc.register_code_command(keyword: :"background.log.read", args_klass: Rundoc::CodeCommand::Background::Log::ReadArgs, runner_klass: Rundoc::CodeCommand::Background::Log::ReadRunner)

data/lib/rundoc/code_command/background/process_spawn.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 require "shellwords"
 require "timeout"
 require "fileutils"
@@ -45,16 +47,16 @@ class Rundoc::CodeCommand::Background
     attr_reader :log, :pid, :command
 
     def initialize(command, timeout: 5, log: Tempfile.new("log"), out: "2>&1")
-      @command = command
+      @original_command = command
       @timeout_value = timeout
-      @log_reference = log # https://twitter.com/schneems/status/1285289971083907075
+      @log_reference = log # Need to keep a reference to `Tempfile` or it will be deleted. Pathname does not retain the passed in reference
 
       @log = Pathname.new(log)
       @log.dirname.mkpath
       FileUtils.touch(@log)
       @pipe_output, @pipe_input = IO.pipe
 
-      @command = "/usr/bin/env bash -c #{@command.shellescape} >> #{@log} #{out}"
+      @command = "/usr/bin/env bash -c #{@original_command.shellescape} >> #{@log} #{out}"
       @pid = nil
     end
 
@@ -120,13 +122,15 @@ class Rundoc::CodeCommand::Background
       contents
     end
 
-    def stop
+    def stop(print_io: nil)
       return unless alive?
       @pipe_input.close
       Process.kill("TERM", -Process.getpgid(@pid))
       Process.wait(@pid)
     rescue Errno::ESRCH => e
-      puts "Error stopping process (command: #{command}): #{e}"
+      print_io&.puts "Error stopping process (command: #{command}): #{e}"
+    ensure
+      print_io&.puts "Log contents for `#{command}`:\n#{@log.read}"
     end
 
     def check_alive!

data/lib/rundoc/code_command/background/start.rb

@@ -1,18 +1,37 @@
+# frozen_string_literal: true
+
 require "tempfile"
 
 class Rundoc::CodeCommand::Background
-  class Start < Rundoc::CodeCommand
+  class StartArgs
+    attr_reader :command, :name, :wait, :timeout, :log, :out, :allow_fail
+
     def initialize(command, name:, wait: nil, timeout: 5, log: Tempfile.new("log"), out: "2>&1", allow_fail: false)
-      @timeout = timeout
       @command = command
       @name = name
       @wait = wait
-      @allow_fail = allow_fail
+      @timeout = timeout
       @log = log
-      @redirect = out
+      @out = out
+      @allow_fail = allow_fail
+    end
+  end
+
+  class StartRunner
+    attr_reader :io
+
+    def initialize(user_args:, render_command:, render_result:, io:, contents: nil)
+      @timeout = user_args.timeout
+      @command = user_args.command
+      @name = user_args.name
+      @wait = user_args.wait
+      @allow_fail = user_args.allow_fail
+      @log = user_args.log
+      @redirect = user_args.out
       FileUtils.touch(@log)
 
       @background = nil
+      @io = io
     end
 
     def background
@@ -22,7 +41,7 @@ class Rundoc::CodeCommand::Background
         log: @log,
         out: @redirect
       ).tap do |spawn|
-        puts "Spawning commmand: `#{spawn.command}`"
+        io.puts "Spawning commmand: `#{spawn.command}`"
         ProcessSpawn.add(@name, spawn)
       end
     end
@@ -44,4 +63,4 @@ class Rundoc::CodeCommand::Background
   end
 end
 
-Rundoc.register_code_command(:"background.start", Rundoc::CodeCommand::Background::Start)
+Rundoc.register_code_command(keyword: :"background.start", args_klass: Rundoc::CodeCommand::Background::StartArgs, runner_klass: Rundoc::CodeCommand::Background::StartRunner)

data/lib/rundoc/code_command/background/stdin_write.rb

@@ -1,14 +1,27 @@
+# frozen_string_literal: true
+
 class Rundoc::CodeCommand::Background
-  # Will send contents to the background process via STDIN along with a newline
-  #
-  #
-  class StdinWrite < Rundoc::CodeCommand
+  class StdinWriteArgs
+    attr_reader :contents, :name, :wait, :timeout, :ending
+
     def initialize(contents, name:, wait:, timeout: 5, ending: $/)
       @contents = contents
-      @ending = ending
-      @wait = wait
       @name = name
-      @timeout_value = Integer(timeout)
+      @wait = wait
+      @timeout = Integer(timeout)
+      @ending = ending
+    end
+  end
+
+  class StdinWriteRunner
+    attr_reader :contents
+
+    def initialize(user_args:, render_command:, render_result:, io: nil, contents: nil)
+      @contents = user_args.contents
+      @ending = user_args.ending
+      @wait = user_args.wait
+      @name = user_args.name
+      @timeout_value = user_args.timeout
       @contents_written = nil
       @background = nil
     end
@@ -38,4 +51,4 @@ class Rundoc::CodeCommand::Background
     end
   end
 end
-Rundoc.register_code_command(:"background.stdin_write", Rundoc::CodeCommand::Background::StdinWrite)
+Rundoc.register_code_command(keyword: :"background.stdin_write", args_klass: Rundoc::CodeCommand::Background::StdinWriteArgs, runner_klass: Rundoc::CodeCommand::Background::StdinWriteRunner)

data/lib/rundoc/code_command/background/stop.rb

@@ -1,7 +1,17 @@
+# frozen_string_literal: true
+
 class Rundoc::CodeCommand::Background
-  class Stop < Rundoc::CodeCommand
+  class StopArgs
+    attr_reader :name
+
     def initialize(name:)
       @name = name
+    end
+  end
+
+  class StopRunner
+    def initialize(user_args:, render_command:, render_result:, io: nil, contents: nil)
+      @name = user_args.name
       @background = nil
     end
 
@@ -19,4 +29,4 @@ class Rundoc::CodeCommand::Background
     end
   end
 end
-Rundoc.register_code_command(:"background.stop", Rundoc::CodeCommand::Background::Stop)
+Rundoc.register_code_command(keyword: :"background.stop", args_klass: Rundoc::CodeCommand::Background::StopArgs, runner_klass: Rundoc::CodeCommand::Background::StopRunner)

data/lib/rundoc/code_command/background/wait.rb

@@ -1,9 +1,21 @@
+# frozen_string_literal: true
+
 class Rundoc::CodeCommand::Background
-  class Wait < Rundoc::CodeCommand
+  class WaitArgs
+    attr_reader :name, :wait, :timeout
+
     def initialize(name:, wait:, timeout: 5)
       @name = name
       @wait = wait
-      @timeout_value = Integer(timeout)
+      @timeout = Integer(timeout)
+    end
+  end
+
+  class WaitRunner
+    def initialize(user_args:, render_command:, render_result:, io: nil, contents: nil)
+      @name = user_args.name
+      @wait = user_args.wait
+      @timeout_value = user_args.timeout
       @background = nil
     end
 
@@ -21,4 +33,4 @@ class Rundoc::CodeCommand::Background
     end
   end
 end
-Rundoc.register_code_command(:"background.wait", Rundoc::CodeCommand::Background::Wait)
+Rundoc.register_code_command(keyword: :"background.wait", args_klass: Rundoc::CodeCommand::Background::WaitArgs, runner_klass: Rundoc::CodeCommand::Background::WaitRunner)

data/lib/rundoc/code_command/background.rb

@@ -1,3 +1,5 @@
+# frozen_string_literal: true
+
 class Rundoc::CodeCommand::Background
 end
 

data/lib/rundoc/code_command/bash/cd.rb

@@ -1,9 +1,9 @@
-class Rundoc::CodeCommand::Bash
-  # special purpose class to persist cd behavior across the entire program
-  # we change the directory of the parent program (rundoc) rather than
-  # changing the directory of a spawned child (via exec, ``, system, etc.)
-  class Cd < Rundoc::CodeCommand::Bash
-    def initialize(line)
+# frozen_string_literal: true
+
+class Rundoc::CodeCommand::BashRunner
+  class Cd < Rundoc::CodeCommand::BashRunner
+    def initialize(line, io: $stdout)
+      @io = io
       @line = line
     end
 
@@ -23,7 +23,7 @@ class Rundoc::CodeCommand::Bash
 
     def call(env)
       line = @line.sub("cd", "").strip
-      puts "running $ cd #{line}"
+      @io.puts "running $ cd #{line}"
 
       supress_chdir_warning do
         Dir.chdir(line)

data/lib/rundoc/code_command/bash.rb

@@ -1,18 +1,32 @@
-class Rundoc::CodeCommand::Bash < Rundoc::CodeCommand
-  # line = "cd ..""
-  # line = "pwd"
-  # line = "ls"
+# frozen_string_literal: true
+
+class Rundoc::CodeCommand::BashArgs
+  attr_reader :line
+
   def initialize(line)
     @line = line
-    @contents = ""
+  end
+end
+
+class Rundoc::CodeCommand::BashRunner
+  attr_reader :io, :contents
+
+  def initialize(user_args:, render_command:, render_result:, io:, contents: nil)
+    @io = io
+    @contents = contents.dup if contents && !contents.empty?
+    @line = user_args.line
     @delegate = case @line.split(" ").first.downcase
     when "cd"
-      Cd.new(@line)
+      Cd.new(@line, io: io)
     else
       false
     end
   end
 
+  def raise_on_error?
+    true
+  end
+
   def to_md(env = {})
     return @delegate.to_md(env) if @delegate
 
@@ -34,30 +48,40 @@ class Rundoc::CodeCommand::Bash < Rundoc::CodeCommand
     cmd = "(#{cmd}) 2>&1"
     msg = "Running: $ '#{cmd}'"
     msg << " with stdin: '#{stdin.inspect}'" if stdin && !stdin.empty?
-    puts msg
+    io.puts msg
 
-    result = ""
-    IO.popen(cmd, "w+") do |io|
-      io << stdin if stdin
-      io.close_write
+    result = +""
+    IO.popen(cmd, "w+") do |pipe|
+      pipe << stdin if stdin
+      pipe.close_write
 
-      until io.eof?
-        buffer = io.gets
-        puts "    #{buffer}"
+      until pipe.eof?
+        buffer = pipe.gets
+        io.puts "    #{buffer}"
 
         result << sanitize_escape_chars(buffer)
       end
     end
 
-    unless $?.success?
-      raise "Command `#{@line}` exited with non zero status: #{result}" unless keyword.to_s.include?("fail")
+    if raise_on_error? && !$?.success?
+      raise "Command `#{@line}` exited with non zero status: #{result}"
     end
     result
   end
 end
 
-Rundoc.register_code_command(:bash, Rundoc::CodeCommand::Bash)
-Rundoc.register_code_command(:"$", Rundoc::CodeCommand::Bash)
-Rundoc.register_code_command(:"fail.$", Rundoc::CodeCommand::Bash)
+class Rundoc::CodeCommand::BashRunnerFailOk < Rundoc::CodeCommand::BashRunner
+  def raise_on_error?
+    false
+  end
+end
+
+Rundoc.register_code_command(keyword: :bash, args_klass: Rundoc::CodeCommand::BashArgs, runner_klass: Rundoc::CodeCommand::BashRunner)
+Rundoc.register_code_command(keyword: :"$", args_klass: Rundoc::CodeCommand::BashArgs, runner_klass: Rundoc::CodeCommand::BashRunner)
+Rundoc.register_code_command(
+  keyword: :"fail.$",
+  args_klass: Rundoc::CodeCommand::BashArgs,
+  runner_klass: Rundoc::CodeCommand::BashRunnerFailOk
+)
 
 require "rundoc/code_command/bash/cd"

data/lib/rundoc/code_command/comment.rb

@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+class Rundoc::CodeCommand::CommentArgs
+  attr_reader :line
+
+  def initialize(line = nil)
+    @line = line
+  end
+end
+
+class Rundoc::CodeCommand::CommentRunner
+  def initialize(user_args:, render_command:, render_result:, io:, contents: nil)
+    @io = io
+    @line = user_args&.line
+    @contents = contents
+  end
+
+  def call(env = {})
+    @io.puts "Skipping command (commented out): # #{@line}\n#{@contents}".strip
+    ""
+  end
+
+  def to_md(env = {})
+    ""
+  end
+end
+
+Rundoc.register_code_command(
+  keyword: :"#",
+  args_klass: Rundoc::CodeCommand::CommentArgs,
+  runner_klass: Rundoc::CodeCommand::CommentRunner,
+  always_hidden: true
+)

data/lib/rundoc/code_command/deferred.rb

@@ -0,0 +1,66 @@
+# frozen_string_literal: true
+
+module Rundoc
+  module CodeCommand
+    # Hold enough information to construct commands, but don't yet
+    #
+    # Allows us to separate parse time constructs from runtime injectables
+    # (such as IO). Which gives us a cleaner running model.
+    class Deferred
+      attr_accessor :render_result, :render_command,
+        :contents, :keyword, :original_args
+
+      alias_method :render_result?, :render_result
+      alias_method :render_command?, :render_command
+
+      attr_reader :runner_klass
+
+      def initialize(args_instance:, runner_klass:, always_hidden: false)
+        @args_instance = args_instance
+        @runner_klass = runner_klass
+        @always_hidden = always_hidden
+      end
+
+      def hidden?
+        !render_command? && !render_result?
+      end
+
+      def not_hidden?
+        !hidden?
+      end
+
+      def push(contents)
+        @contents ||= +""
+        @contents << contents
+      end
+      alias_method :<<, :push
+
+      def build(io: $stdout)
+        @built ||= begin
+          runner = @runner_klass.new(
+            user_args: @args_instance,
+            render_command: render_command,
+            render_result: render_result,
+            contents: @contents,
+            io: io
+          )
+          if @always_hidden
+            @render_command = false
+            @render_result = false
+          end
+          runner
+        end
+      rescue UnknownCommand
+        raise "No such command registered with rundoc #{keyword.inspect} for `#{keyword} #{original_args}`"
+      end
+
+      def call(env = {})
+        build.call(env)
+      end
+
+      def to_md(env = {})
+        build.to_md(env)
+      end
+    end
+  end
+end