toys-core 0.13.1 → 0.14.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7d34e973aef54758dbd85e689d69bd68660024a41e344c40bdc64b84459e5b31
-  data.tar.gz: 1c488073e098663bf9b2e6b9434c8c1932eeb66411855a4810cb57eca4ea1445
+  metadata.gz: 49f6342ac835f193218570f20cc534fc7f1d4fab14742602df3a7174cfc8a554
+  data.tar.gz: 27b8298897b5c8fd08319fffec33540c057675ed8655700280b4837f413cc635
 SHA512:
-  metadata.gz: '09e199be2b41bc50a6a8e2d8a03f91be92c2f04f022abc5a6a64f4077f745a445777369e4a20787a4754fe8b709e5cfde138e65eee84ec3f4f6d21ca089072fc'
-  data.tar.gz: b874efa699f0c7d76ffaa6848fb30668d830b9a1b8a342fed13283586829069cefdbe7a379a90749c365c510affa06394d82d21e601c3b6bd868f001f4287350
+  metadata.gz: 55dc00e57b5c6b3212ed24d45bb3a595dae8b5b6fd076ddefdbe4835d621dfeb70d58b8551ae69a8190dc3884a66968e78b800d93b07eed6b61d42eef70faea7
+  data.tar.gz: eb6b5c306c815947290a8fb99834a4a809052edf7e6c781f1b562c61217cf49c63a9609beb168b0e06c5e09ace51083bbb7da15ad58746e72fc5017e04f9546d

data/CHANGELOG.md

@@ -1,5 +1,33 @@
 # Release History
 
+### v0.14.1 / 2022-10-03
+
+* (No significant changes)
+
+### v0.14.0 / 2022-10-03
+
+Toys-Core 0.14.0 is a major release with pager support, support for tees and pipes in the Exec utility, some cleanup of the behavior of Acceptors, and other improvements.
+
+Fixes that are potentially breaking:
+
+* Disallowed acceptors on flags that are explicitly boolean.
+* Acceptors no longer sometimes apply to the boolean setting of a flag with an optional value.
+
+New functionality:
+
+* Implemented a utility class and mixin for output pagers.
+* Builtin commands that display data can format as either YAML or JSON.
+* The Exec utility and mixin can tee (i.e. duplicate and split) output streams.
+* The Exec utility and mixin can take pipes as input and output streams.
+* The Exec mixin provides a `verbosity_flags` convenience method.
+* `Loader#list_subtools` takes separate arguments for filtering namespaces and non-runnable tools
+
+Fixes:
+
+* Contents of preload directories are loaded in sorted order.
+* Removed some potential deadlocks if a `Toys::Loader` is accessed from multiple threads
+* Various clarifications, fixes, and updates to the users guide and documentation.
+
 ### v0.13.1 / 2022-03-01
 
 * FIXED: Toys::Utils::Gems no longer fails to install a bundle if it had locked to a different version of a builtin gem

data/README.md

@@ -25,23 +25,15 @@ recommend first walking through the tutorial in the
 you are running a unix-like system such as Linux or macOS. Some commands might
 need to be modified if you're running on Windows.
 
-### Install Toys
+### Install Toys-Core
 
 Install the **toys-core** gem using:
 
     $ gem install toys-core
 
-You may also install the **toys** gem, which brings in **toys-core** as a
+You can also install the **toys** gem, which brings in **toys-core** as a
 dependency.
 
-Toys-Core requires Ruby 2.4 or later.
-
-Most parts of Toys-Core work on JRuby. However, JRuby is not recommended
-because of JVM boot latency, lack of support for Kernel#fork, and other issues.
-
-Most parts of Toys-Core work on TruffleRuby. However, TruffleRuby is not
-recommended because it has a few known bugs that affect Toys.
-
 ### Create a new executable
 
 We'll start by creating an executable Ruby script. Using your favorite text
@@ -77,7 +69,7 @@ use to write Toys files. You could point your executable at a directory
 containing actual Toys files, but the simplest option is to provide the
 information to the Toys CLI object in a block.
 
-Let's add some functionality.
+Let's add some functionality to `mycmd`.
 
     #!/usr/bin/env ruby
 
@@ -107,7 +99,7 @@ look familiar. Let's run it now, and experiment with passing flags to it.
 Notice that we did not create a `tool` block, but instead set up description,
 flags, and functionality directly in the configuration block. This configures
 the "root tool", i.e. what happens when you run the executable without passing
-a tool name to it. (Note, it's legal to do this in Toys as well, by setting
+a tool name to it. (In fact, it's legal to do this in Toys as well, by setting
 functionality at the "top level" of a `.toys.rb` file without including any
 `tool` block.)
 
@@ -172,8 +164,8 @@ modifies error handling and delimiter parsing.
     #### Pass some additional options to the CLI constructor ...
     cli = Toys::CLI.new(
       extra_delimiters: ":",
-      error_handler: ->(_err) {
-        puts "Dude, an error happened..."
+      error_handler: ->(err) {
+        puts "Aww shucks, an error happened: #{err.message}"
         return 1
       }
     )
@@ -209,8 +201,8 @@ Toys _middleware_ are objects that provide common functionality for all the
 tools in your executable. For example, a middleware adds the `--help` flag to
 your tools by default.
 
-The next example modifies the middleware stack to alter this common tool
-functionality.
+The next example provides a custom middleware stack, resulting in a different
+set of common tool functionality.
 
     #!/usr/bin/env ruby
 
@@ -339,6 +331,16 @@ handling errors, controlling log output, and providing your own mixins,
 templates, and middleware, in the
 [Toys-Core User Guide](https://dazuma.github.io/toys/gems/toys-core/latest/file.guide.html).
 
+## System requirements
+
+Toys-Core requires Ruby 2.4 or later.
+
+Most parts of Toys-Core work on JRuby. However, JRuby is not recommended
+because of JVM boot latency, lack of support for Kernel#fork, and other issues.
+
+Most parts of Toys-Core work on TruffleRuby. However, TruffleRuby is not
+recommended because it has a few known bugs that affect Toys.
+
 ## License
 
 Copyright 2019-2022 Daniel Azuma and the Toys contributors

data/docs/guide.md

@@ -29,7 +29,7 @@ sophisticated command line tools.
 
 ## Conceptual overview
 
-Toys-Core is a command line *framework* in the traditional sense. It is
+Toys-Core is a **command line framework** in the traditional sense. It is
 intended to be used to write custom command line executables in Ruby. The
 framework provides common facilities such as argument parsing and online help,
 while your executable chooses and configures those facilities, and implements
@@ -44,7 +44,7 @@ written in **toys files** or in **blocks** passed to the CLI. It uses the same
 DSL used by Toys itself, and supports tools, subtools, flags, arguments, help
 text, and all the other features of Toys.
 
-An executable may customize its own facilities for writing tools by providing
+An executable can customize its own facilities for writing tools by providing
 **built-in mixins** and **built-in templates**, and can implement default
 behavior across all tools by providing **middleware**.
 
@@ -52,7 +52,7 @@ Most executables will provide a set of **static tools**, but it is possible to
 support user-provided tools as Toys does. Executables can customize how tool
 definitions are searched and loaded from the file system.
 
-Finally, an executable may customize many aspects of its behavior, such as the
+Finally, an executable can customize many aspects of its behavior, such as the
 **logging output**, **error handling**, and even shell **tab completion**.
 
 ## Using the CLI object
@@ -96,7 +96,109 @@ Following is a simple "hello world" example using the CLI:
 
 This section provides some detail on how a CLI executes your code.
 
-(TODO)
+When you call {Toys::CLI#run}, the CLI runs through three phases:
+
+ *  **Loading** in which the CLI identifies which tool to run, and loads the
+    tool from a tool **source**, which could be a block passed to the CLI, or a
+    file loaded from the file system, git, or other location.
+ *  **Building context**, in which the CLI parses the command-line arguments
+    according to the flags and arguments declared by the tool, instantiates the
+    tool, and populates the {Toys::Context} object (which is `self` when the
+    tool is executed)
+ *  **Execution**, which involves running any initializers defined on the tool,
+    applying middleware, running the tool, and handling errors.
+
+#### The Loader
+
+When the CLI needs the definition of a tool, it queries the {Toys::Loader}. The
+loader object is configured with a set of tool _sources_ representing ways to
+define a tool. These sources may be blocks passed directly to the CLI,
+directories and files in the file system, and remote git repositories. When a
+tool is requested by name, the loader is responsible for locating the tool
+definition in those sources, and constructing the tool definition object,
+represented by {Toys::ToolDefinition}.
+
+One important property of the loader is that it is _lazy_. It queries tool
+sources only if it has reason to believe that a tool it is looking for may be
+defined there. For example, if your tools are defined in a directory structure,
+the `foo bar` tool might live in the file `foo/bar.rb`. The loader will open
+that file, if it exists, only when the `foo bar` tool is requested. If instead
+`foo qux` is requested, the `foo/bar.rb` file is never even opened.
+
+Perhaps more subtly, if you call {Toys::CLI#add_config_block} to define tools,
+the block is stored in the loader object _but not called immediately_. Only
+when a tool is requested does the block actually execute. Furthermore, if you
+have `tool` blocks inside the block, the loader will execute only those that
+are relevant to a tool it wants. Hence:
+
+    cli.add_config_block do
+      tool "foo" do
+        def run
+          puts "foo called"
+        end
+      end
+
+      tool "bar" do
+        def run
+          puts "bar called"
+        end
+      end
+    end
+
+If only `foo` is requested, the loader will execute the `tool "foo" do` block
+to get that tool definition, but will not execute the `tool "bar" do` block.
+
+We will discuss more about the features of the loader below in the section on
+[defining functionality](#Defining_functionality).
+
+#### Building context
+
+Once a tool is defined, the CLI prepares it for execution by building a
+{Toys::Context} object. This object is `self` during tool runtime, and it
+includes:
+
+ *  The tool's methods, including its `run` entrypoint method.
+ *  Access to core tool functionality such as exit codes and logging.
+ *  The results from parsing the command line arguments
+ *  The runtime environment, including the tool's name, where the tool was
+    defined, detailed results from argumet parsing, and so forth.
+
+Much of this information is stored in a data hash, whose keys are defined as
+constants under {Toys::Context::Key}.
+
+Argument parsing is directed by the {Toys::ArgParser} class. This class, for
+the most part, replicates the semantics of the standard Ruby OptionParser
+class, but it implements a few extra features and cleans up a few ambiguities.
+
+#### Tool execution and error handling
+
+The execution phase involves:
+
+ *  Running the tool's initializers, if any, in order.
+ *  Running the tool's middleware. Each middleware "wraps" the execution of
+    subsequent middleware and the final tool execution, and has the opportunity
+    to inject functionality before and after the main execution, or even to
+    forgo or replace the main functionality, similar to Rack middleware.
+ *  Executing the tool itself by calling its `run` method.
+
+During execution, exceptions are caught and reported along with the location in
+the tool source where it was triggered. This logic is handled by the
+{Toys::ContextualError} class.
+
+The CLI can be configured with an error handler that responds to any exceptions
+raised during execution. An error handler is simply a callable object (such as
+a `Proc`) that takes an exception as an argument. The provided
+{Toys::CLI::DefaultErrorHandler} class provides the default behavior of the
+normal `toys` CLI, but you can provide any object that duck types the `call`
+method.
+
+#### Multiple runs
+
+The {Toys::CLI} object can be reused to run multiple tools. This may save on
+loading overhead, as the tools can be loaded just once and their definitions
+reused for multiple executions. It can even perform multiple executions
+concurrently in separate threads, assuming the tool implementations themselves
+are thread-safe.
 
 ### Configuring the CLI
 
@@ -117,8 +219,9 @@ These features include:
 
 Each of the actual parameters is covered in detail in the documentation for
 {Toys::CLI#initialize}. The configuration of a CLI cannot be changed once the
-CLI is constructed. If you need to a CLI with a modified configuration, use
-{Toys::CLI#child}.
+CLI is constructed. If you need a CLI with modified configuration, use
+{Toys::CLI#child}, which creates a _copy_ of the CLI with any modifications you
+request.
 
 ## Defining functionality
 
@@ -151,3 +254,5 @@ CLI is constructed. If you need to a CLI with a modified configuration, use
 ### Tab completion
 
 ## Packaging your executable
+
+## Overview of Toys-Core classes

data/lib/toys/acceptor.rb

@@ -83,7 +83,7 @@ module Toys
       # When given a valid input, return an array in which the first element is
       # the original input string, and the remaining elements (which may be
       # empty) comprise any additional information that may be useful during
-      # conversion. If there is no additional information, you may return the
+      # conversion. If there is no additional information, you can return the
       # original input string by itself without wrapping in an array.
       #
       # When given an invalid input, return a falsy value such as `nil`.
@@ -95,8 +95,7 @@ module Toys
       # as the only array element, indicating all inputs are valid. You can
       # override this method to provide a different validation function.
       #
-      # @param str [String,nil] The input argument string. May be `nil` if the
-      #     value is optional and not provided.
+      # @param str [String] The input argument string.
       # @return [String,Array,nil]
       #
       def match(str)
@@ -110,15 +109,14 @@ module Toys
       # original input string and any other values returned from {#match}. It
       # must return the final converted value to use.
       #
-      # @param str [String,nil] Original argument string. May be `nil` if the
-      #     value is optional and not provided.
+      # @param str [String] Original argument string.
       # @param extra [Object...] Zero or more additional arguments comprising
       #     additional elements returned from the match function.
       # @return [Object] The converted argument as it should be stored in the
       #     context data.
       #
       def convert(str, *extra) # rubocop:disable Lint/UnusedMethodArgument
-        str.nil? ? true : str
+        str
       end
 
       ##
@@ -183,7 +181,7 @@ module Toys
       #
       def match(str)
         result = @function.call(str) rescue REJECT # rubocop:disable Style/RescueModifier
-        result.equal?(REJECT) ? nil : [str, result]
+        REJECT.equal?(result) ? nil : [str, result]
       end
 
       ##
@@ -240,7 +238,7 @@ module Toys
       # @private
       #
       def match(str)
-        str.nil? ? [nil] : @regex.match(str)
+        @regex.match(str)
       end
 
       ##
@@ -293,7 +291,7 @@ module Toys
       # @private
       #
       def match(str)
-        str.nil? ? [nil, nil] : @values.find { |s, _e| s == str }
+        @values.find { |s, _e| s == str }
       end
 
       ##
@@ -418,9 +416,7 @@ module Toys
     #
     NUMERIC_CONVERTER =
       proc do |s|
-        if s.nil?
-          nil
-        elsif s.include?("/")
+        if s.include?("/")
           Rational(s)
         elsif s.include?(".") || (s.include?("e") && s !~ /\A-?0x/)
           Float(s)
@@ -429,6 +425,41 @@ module Toys
         end
       end
 
+    ##
+    # A set of strings that are considered true for boolean acceptors.
+    # Currently set to `["+", "true", "yes"]`.
+    # @return [Array<String>]
+    #
+    TRUE_STRINGS = ["+", "true", "yes"].freeze
+
+    ##
+    # A set of strings that are considered false for boolean acceptors.
+    # Currently set to `["-", "false", "no", "nil"]`.
+    # @return [Array<String>]
+    #
+    FALSE_STRINGS = ["-", "false", "no", "nil"].freeze
+
+    ##
+    # A converter proc that handles boolean strings. Recognizes {TRUE_STRINGS}
+    # and {FALSE_STRINGS}. Useful for Simple acceptors.
+    # @return [Proc]
+    #
+    BOOLEAN_CONVERTER =
+      proc do |s|
+        if s.empty?
+          REJECT
+        else
+          s = s.downcase
+          if TRUE_STRINGS.any? { |t| t.start_with?(s) }
+            true
+          elsif FALSE_STRINGS.any? { |f| f.start_with?(s) }
+            false
+          else
+            REJECT
+          end
+        end
+      end
+
     class << self
       ##
       # Lookup a standard acceptor name recognized by OptionParser.
@@ -521,7 +552,11 @@ module Toys
       end
 
       ##
-      # @private
+      # Take the various ways to express an acceptor spec, and convert them to
+      # a canonical form expressed as a single object. This is called from the
+      # DSL to generate a spec object that can be stored.
+      #
+      # @private This interface is internal and subject to change without warning.
       #
       def scalarize_spec(spec, options, block)
         spec ||= block
@@ -564,8 +599,8 @@ module Toys
           ::Float => build_float,
           ::Rational => build_rational,
           ::Numeric => build_numeric,
-          ::TrueClass => build_boolean(::TrueClass, true),
-          ::FalseClass => build_boolean(::FalseClass, false),
+          ::TrueClass => build_boolean(::TrueClass),
+          ::FalseClass => build_boolean(::FalseClass),
           ::Array => build_array,
           ::Regexp => build_regexp,
         }
@@ -603,77 +638,48 @@ module Toys
         Simple.new(NUMERIC_CONVERTER, type_desc: "number", well_known_spec: ::Numeric)
       end
 
-      TRUE_STRINGS = ["+", "true", "yes"].freeze
-      FALSE_STRINGS = ["-", "false", "no", "nil"].freeze
-      private_constant :TRUE_STRINGS, :FALSE_STRINGS
-
-      def build_boolean(spec, default)
-        Simple.new(type_desc: "boolean", well_known_spec: spec) do |s|
-          if s.nil?
-            default
-          elsif s.empty?
-            REJECT
-          else
-            s = s.downcase
-            if TRUE_STRINGS.any? { |t| t.start_with?(s) }
-              true
-            elsif FALSE_STRINGS.any? { |f| f.start_with?(s) }
-              false
-            else
-              REJECT
-            end
-          end
-        end
+      def build_boolean(spec)
+        Simple.new(BOOLEAN_CONVERTER, type_desc: "boolean", well_known_spec: spec)
       end
 
       def build_array
         Simple.new(type_desc: "string array", well_known_spec: ::Array) do |s|
-          if s.nil?
-            nil
-          else
-            s.split(",").collect { |elem| elem unless elem.empty? }
-          end
+          s.split(",").collect { |elem| elem unless elem.empty? }
         end
       end
 
       def build_regexp
         Simple.new(type_desc: "regular expression", well_known_spec: ::Regexp) do |s|
-          if s.nil?
-            nil
-          else
-            flags = 0
-            if (match = %r{\A/((?:\\.|[^\\])*)/([[:alpha:]]*)\z}.match(s))
-              s = match[1]
-              opts = match[2] || ""
-              flags |= ::Regexp::IGNORECASE if opts.include?("i")
-              flags |= ::Regexp::MULTILINE if opts.include?("m")
-              flags |= ::Regexp::EXTENDED if opts.include?("x")
-            end
-            ::Regexp.new(s, flags)
+          flags = 0
+          if (match = %r{\A/((?:\\.|[^\\])*)/([[:alpha:]]*)\z}.match(s))
+            s = match[1]
+            opts = match[2] || ""
+            flags |= ::Regexp::IGNORECASE if opts.include?("i")
+            flags |= ::Regexp::MULTILINE if opts.include?("m")
+            flags |= ::Regexp::EXTENDED if opts.include?("x")
           end
+          ::Regexp.new(s, flags)
         end
       end
 
       def build_decimal_integer
         Simple.new(type_desc: "decimal integer",
                    well_known_spec: ::OptionParser::DecimalInteger) do |s|
-          s.nil? ? nil : Integer(s, 10)
+          Integer(s, 10)
         end
       end
 
       def build_octal_integer
         Simple.new(type_desc: "octal integer",
                    well_known_spec: ::OptionParser::OctalInteger) do |s|
-          s.nil? ? nil : Integer(s, 8)
+          Integer(s, 8)
         end
       end
 
       def build_decimal_numeric
         Simple.new(type_desc: "decimal number",
                    well_known_spec: ::OptionParser::DecimalNumeric) do |s|
-          if s.nil?
-            nil
-          elsif s.include?(".") || (s.include?("e") && s !~ /\A-?0x/)
+          if s.include?(".") || (s.include?("e") && s !~ /\A-?0x/)
             Float(s)
           else
             Integer(s, 10)

data/lib/toys/arg_parser.rb

@@ -439,7 +439,7 @@ module Toys
       return false unless @active_flag_def
       result = @active_flag_def.value_type == :required || !arg.start_with?("-")
       add_data(@active_flag_def.key, @active_flag_def.handler, @active_flag_def.acceptor,
-               result ? arg : nil, :flag, @active_flag_arg)
+               result ? arg : true, :flag, @active_flag_arg)
       @seen_flag_keys << @active_flag_def.key
       @active_flag_def = nil
       @active_flag_arg = nil
@@ -481,7 +481,7 @@ module Toys
           @active_flag_def = flag_def
           @active_flag_arg = name
         else
-          add_data(flag_def.key, flag_def.handler, flag_def.acceptor, nil, :flag, name)
+          add_data(flag_def.key, flag_def.handler, flag_def.acceptor, true, :flag, name)
         end
       else
         add_data(flag_def.key, flag_def.handler, flag_def.acceptor, following, :flag, name)
@@ -540,7 +540,7 @@ module Toys
     end
 
     def add_data(key, handler, accept, value, type_name, display_name)
-      if accept
+      if accept && value.is_a?(::String)
         match = accept.match(value)
         unless match
           error_class = type_name == :flag ? FlagValueUnacceptableError : ArgValueUnacceptableError
@@ -562,7 +562,7 @@ module Toys
           @errors << FlagValueMissingError.new(name: @active_flag_arg)
         else
           add_data(@active_flag_def.key, @active_flag_def.handler, @active_flag_def.acceptor,
-                   nil, :flag, @active_flag_arg)
+                   true, :flag, @active_flag_arg)
         end
       end
     end

data/lib/toys/cli.rb

@@ -226,9 +226,9 @@ module Toys
     end
 
     ##
-    # Make a clone with the same settings but no no config blocks and no paths
-    # in the loader. This is sometimes useful for calling another tool that has
-    # to be loaded from a different configuration.
+    # Make a clone with the same settings but no config blocks and no paths in
+    # the loader. This is sometimes useful for calling another tool that has to
+    # be loaded from a different configuration.
     #
     # @param opts [keywords] Any configuration arguments that should be
     #     modified from the original. See {#initialize} for a list of

data/lib/toys/completion.rb

@@ -435,7 +435,11 @@ module Toys
     end
 
     ##
-    # @private
+    # Take the various ways to express a completion spec, and convert them to a
+    # canonical form expressed as a single object. This is called from the DSL
+    # DSL to generate a spec object that can be stored.
+    #
+    # @private This interface is internal and subject to change without warning.
     #
     def self.scalarize_spec(spec, options, block)
       spec ||= block

data/lib/toys/context.rb

@@ -302,7 +302,7 @@ module Toys
     end
 
     ##
-    # Exit immediately with the given status code
+    # Exit immediately with the given status code.
     #
     # @param code [Integer] The status code, which should be 0 for no error,
     #     or nonzero for an error condition. Default is 0.
@@ -313,7 +313,8 @@ module Toys
     end
 
     ##
-    # Exit immediately with the given status code
+    # Exit immediately with the given status code. This class method can be
+    # called if the instance method is or could be replaced by the tool.
     #
     # @param code [Integer] The status code, which should be 0 for no error,
     #     or nonzero for an error condition. Default is 0.
@@ -330,7 +331,7 @@ module Toys
     #
     # @param data [Hash]
     #
-    # @private
+    # @private This interface is internal and subject to change without warning.
     #
     def initialize(data)
       @__data = data

data/lib/toys/core.rb

@@ -9,7 +9,7 @@ module Toys
     # Current version of Toys core.
     # @return [String]
     #
-    VERSION = "0.13.1"
+    VERSION = "0.14.1"
   end
 
   ##

data/lib/toys/errors.rb

@@ -45,7 +45,11 @@ module Toys
   #
   class ContextualError < ::StandardError
     ##
-    # @private
+    # Construct a ContextualError. This exception type is thrown from
+    # {ContextualError.capture} and {ContextualError.capture_path} and should
+    # not be constructed directly.
+    #
+    # @private This interface is internal and subject to change without warning.
     #
     def initialize(cause, banner,
                    config_path: nil, config_line: nil,
@@ -118,7 +122,11 @@ module Toys
 
     class << self
       ##
-      # @private
+      # Execute the given block, and wrap any exceptions thrown with a
+      # ContextualError. This is intended for loading a config file from the
+      # given path, and wraps any Ruby parsing errors.
+      #
+      # @private This interface is internal and subject to change without warning.
       #
       def capture_path(banner, path, **opts)
         yield
@@ -140,7 +148,10 @@ module Toys
       end
 
       ##
-      # @private
+      # Execute the given block, and wrap any exceptions thrown with a
+      # ContextualError.
+      #
+      # @private This interface is internal and subject to change without warning.
       #
       def capture(banner, **opts)
         yield