toys 0.13.1 → 0.14.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 76a2e6d85d0d1baa0dc154bc6572bbd600c855c9e3cf6c6e64594630a33c58ed
-  data.tar.gz: c701ff4fd53eb2a7dd85561b4c19e1614a43a9788f73abdbeaa86e8a9a17fb68
+  metadata.gz: 5605477c7993fad15f1c79740f8de6b90f4aef19becdc00a236e71853cb4d9a8
+  data.tar.gz: 1c63369d5c4302356f412e25904bd6db43ca93b9896c687c6ae5bbfd4fe24fc8
 SHA512:
-  metadata.gz: da3b8b9da43c2bc81e286206fa649fc994b5083376357cf3c4f7d3289b8d8d928da8faffa42a32e4313074a24666ec40a1eabc73d8b1c82304bc6cd1eed99feb
-  data.tar.gz: 8f36a970ed0292a604dcb28cb5a48ad4697fc0a5fbb27389f6dc937e9ea543488c2b28bfc1c148f93044eb26aed4f03c8911f9df921fafb54f352020144f041b
+  metadata.gz: c312c18f2fa8d57f6dbeb8def6b434a611030387e289a370bcb96faadf90a49b948cfc7ac17b14f9e7b97a942a85b133d384ad37705aed9b833fca3f650517d1
+  data.tar.gz: 1f7125dbf036acb25b0352de30b68f95c229610572c754b8cdf3e1c633e03f5a6318eeb5de56df1d229402f4915d16aacb794d62fcd571007a604aec4ad6070f

data/CHANGELOG.md

@@ -1,5 +1,42 @@
 # Release History
 
+### v0.14.2 / 2022-10-09
+
+* ADDED: The tool directive supports the delegate_relative argument, as a preferred alternative over alias_tool.
+* FIXED: The toys file reference now properly appears in error messages on Ruby 3.1.
+* FIXED: Error messages show the correct toys file line number on TruffleRuby.
+* FIXED: Inspect strings for tool classes are less opaque and include the tool name.
+* FIXED: The presence of an acceptor forces an ambiguous flag to take a value rather than erroring.
+
+### v0.14.1 / 2022-10-03
+
+* FIXED: Fixed a crash due to a missing file in the gem
+
+### v0.14.0 / 2022-10-03
+
+Toys 0.14.0 is a major release with new system tools for introspecting defined tools, pager support, support for tees and pipes in the Exec utility, some cleanup of the behavior of Acceptors, and other improvements.
+
+Fixes that are potentially breaking:
+
+* Disallowed acceptors on flags that are explicitly boolean.
+* Acceptors no longer sometimes apply to the boolean setting of a flag with an optional value.
+
+New functionality:
+
+* Implemented new builtins for obtaining information about defined tools: `system tools show` and `system tools list`.
+* Implemented a utility class and mixin for output pagers.
+* Builtin commands that display data can format as either YAML or JSON.
+* The Exec utility and mixin can tee (i.e. duplicate and split) output streams.
+* The Exec utility and mixin can take pipes as input and output streams.
+* The Exec mixin provides a `verbosity_flags` convenience method.
+
+Fixes:
+
+* The `system test` builtin no longer requires toys to be installed as a gem.
+* Fixed a failure when passing a relative path to `system test --directory`.
+* Contents of preload directories are loaded in sorted order.
+* Various clarifications, fixes, and updates to the users guide and documentation.
+
 ### v0.13.1 / 2022-03-01
 
 * FIXED: Bundler integration no longer fails if a bundle was locked to a different version of a builtin gem

data/README.md

@@ -19,7 +19,7 @@ gem. For more info on using toys-core, see
 
 Here's a tutorial to help you get a feel of what Toys can do.
 
-### Install Toys
+### Install and try out Toys
 
 Install the **toys** gem using:
 
@@ -50,14 +50,6 @@ Toys does not yet specially implement tab completion for zsh or other shells.
 However, if you are using zsh, installing bash completion using `bashcompinit`
 *mostly* works.
 
-Toys requires Ruby 2.4 or later.
-
-Most parts of Toys work on JRuby. However, JRuby is not recommended because of
-JVM boot latency, lack of support for Kernel#fork, and other issues.
-
-Most parts of Toys work on TruffleRuby. However, TruffleRuby is not recommended
-because it has a few known bugs that affect Toys.
-
 ### Write your first tool
 
 You can define tools by creating a *Toys file*. Go into any directory, and,
@@ -244,8 +236,7 @@ you may need to add Toys to your Gemfile and use Bundler to invoke Toys (i.e.
 `bundle exec toys test`). This is because Toys is just calling the Rake API to
 run your task, and the Rake task might require the bundle. However, when Toys
 is not wrapping Rake, typical practice is actually *not* to use `bundle exec`.
-Toys provides its own mechanisms to setup a bundle, or to activate and even
-install individual gems.
+Toys provides its own mechanisms to manage bundles or install gems for you.
 
 So far, we've made Toys a front-end for your Rake tasks. This may be useful by
 itself. Toys lets you pass command line arguments "normally" to tools, whereas
@@ -256,11 +247,11 @@ than Rake does.
 But you also might find Toys a more natural way to *write* tasks, and indeed
 you can often rewrite an entire Rakefile as a Toys file and get quite a bit of
 benefit in readability and maintainability. For an example, see the
-[Toys file for the Toys repo itself](https://github.com/dazuma/toys/blob/main/toys/.toys.rb).
-It contains the Toys scripts that I use to develop, test, and release Toys
-itself. Yes, Toys is self-hosted. You'll notice most of this Toys file consists
-of template expansions. Toys provides templates for a lot of common build,
-test, and release tasks for Ruby projects.
+[Toys file for the Toys gem itself](https://github.com/dazuma/toys/blob/main/toys/.toys/.toys.rb).
+It contains Toys scripts that I use to develop, test, and release Toys itself.
+Yes, Toys is self-hosted. You'll notice most of this Toys file consists of
+template expansions. Toys provides templates for a lot of common build, test,
+and release tasks for Ruby projects.
 
 If you're feeling adventurous, try translating some of your Rake tasks into
 native Toys tools. You can do so in your existing `.toys.rb` file. Keep the
@@ -305,7 +296,7 @@ them. Furthermore, when writing new scripts, I was repeating the same
 OptionParser boilerplate and common functionality.
 
 Toys was designed to address those problems by providing a framework for
-writing *and organizing* your own command line scripts. You provide the actual
+writing and organizing your own command line scripts. You provide the actual
 functionality by writing Toys files, and Toys takes care of all the other
 details expected from a good command line tool. It provides a streamlined
 interface for defining and handling command line flags and positional
@@ -320,6 +311,16 @@ scripts written for Toys can be invoked and passed arguments and flags using
 familiar unix command line conventions. The Toys github repo itself comes with
 Toys scripts instead of Rakefiles.
 
+## System requirements
+
+Toys requires Ruby 2.4 or later.
+
+Most parts of Toys work on JRuby. However, JRuby is not recommended because of
+JVM boot latency, lack of support for Kernel#fork, and other issues.
+
+Most parts of Toys work on TruffleRuby. However, TruffleRuby is not recommended
+because it has a few known bugs that affect Toys.
+
 ## License
 
 Copyright 2019-2022 Daniel Azuma and the Toys contributors

data/builtins/system/.toys.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+desc "A set of system commands for Toys"
+
+long_desc "Contains tools that inspect, configure, and update Toys itself."
+
+tool "version" do
+  desc "Print the current Toys version"
+
+  def run
+    puts ::Toys::VERSION
+  end
+end
+
+mixin "output-tools" do
+  on_include do
+    flag :output_format, "--format=FORMAT" do
+      accept ["json", "json-compact", "yaml"]
+      desc 'The output format. Recognized values are "yaml" (the default), "json", and ' \
+           '"json-compact".'
+    end
+  end
+
+  def generate_output(object)
+    case output_format
+    when "json"
+      require "json"
+      ::JSON.pretty_generate(object)
+    when "json-compact"
+      require "json"
+      ::JSON.generate(object)
+    when nil, "yaml"
+      require "psych"
+      ::Psych.dump(object)
+    else
+      logger.error("Unknown output format: #{format}")
+      exit(1)
+    end
+  end
+end

data/builtins/system/git-cache.rb

@@ -23,15 +23,16 @@ tool "list" do
     desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
   end
 
+  include "output-tools"
+
   def run
-    require "psych"
     require "toys/utils/git_cache"
     git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
     output = {
       "cache_dir" => git_cache.cache_dir,
       "remotes" => git_cache.remotes,
     }
-    puts(::Psych.dump(output))
+    puts(generate_output(output))
   end
 end
 
@@ -48,8 +49,9 @@ tool "show" do
     desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
   end
 
+  include "output-tools"
+
   def run
-    require "psych"
     require "toys/utils/git_cache"
     git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
     info = git_cache.repo_info(remote)
@@ -57,7 +59,7 @@ tool "show" do
       logger.fatal("Unknown remote: #{remote}")
       exit(1)
     end
-    puts(::Psych.dump(info.to_h))
+    puts(generate_output(info.to_h))
   end
 end
 
@@ -130,8 +132,9 @@ tool "remove" do
     desc "Remove all repositories. Required unless specific remotes are provided."
   end
 
+  include "output-tools"
+
   def run
-    require "psych"
     require "toys/utils/git_cache"
     if remotes.empty? == !all
       logger.fatal("You must specify at least one remote to clear, or --all to clear all remotes.")
@@ -142,7 +145,7 @@ tool "remove" do
     output = {
       "removed" => removed,
     }
-    puts(::Psych.dump(output))
+    puts(generate_output(output))
   end
 end
 
@@ -175,8 +178,9 @@ tool "remove-refs" do
     desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
   end
 
+  include "output-tools"
+
   def run
-    require "psych"
     require "toys/utils/git_cache"
     git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
     removed = git_cache.remove_refs(remote, refs: refs)
@@ -188,7 +192,7 @@ tool "remove-refs" do
       "remote" => remote,
       "removed_refs" => removed.map(&:to_h),
     }
-    puts(::Psych.dump(output))
+    puts(generate_output(output))
   end
 end
 
@@ -220,8 +224,9 @@ tool "remove-sources" do
     desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
   end
 
+  include "output-tools"
+
   def run
-    require "psych"
     require "toys/utils/git_cache"
     git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
     removed = git_cache.remove_sources(remote, commits: commits)
@@ -233,6 +238,6 @@ tool "remove-sources" do
       "remote" => remote,
       "removed_sources" => removed.map(&:to_h),
     }
-    puts(::Psych.dump(output))
+    puts(generate_output(output))
   end
 end

data/builtins/system/test.rb

@@ -30,7 +30,6 @@ include :terminal
 
 def run
   load_minitest_gems
-  ::Dir.chdir(tool_dir)
   test_files = find_test_files
   result = exec_ruby(ruby_args, in: :controller, log_cmd: "Starting minitest...") do |controller|
     controller.in.puts("gem 'minitest', '= #{::Minitest::VERSION}'")
@@ -43,7 +42,6 @@ def run
       controller.in.puts("gem 'minitest-rg', '= #{::MiniTest::RG::VERSION}'")
       controller.in.puts("require 'minitest/rg'")
     end
-    controller.in.puts("gem 'toys', '= #{::Toys::VERSION}'")
     controller.in.puts("require 'toys'")
     controller.in.puts("require 'toys/testing'")
     if directory
@@ -79,7 +77,8 @@ end
 def find_test_files
   glob = ".test/**/test_*.rb"
   glob = "**/#{glob}" if recursive
-  test_files = ::Dir.glob(glob)
+  glob = "#{tool_dir}/#{glob}"
+  test_files = Dir.glob(glob)
   if test_files.empty?
     logger.warn("No test files found")
     exit

data/builtins/system/tools.rb

@@ -0,0 +1,145 @@
+# frozen_string_literal: true
+
+mixin "tool-methods" do
+  def format_tool(tool, namespace, detailed: false)
+    output = {
+      "name" => tool.full_name[namespace.length..-1].join(" "),
+      "desc" => tool.desc.to_s,
+      "runnable" => tool.runnable?,
+    }
+    if detailed
+      output["exists"] = true
+      output["long_desc"] = tool.long_desc.map(&:to_s)
+    end
+    output
+  end
+
+  def choose_loader(local, from_dir)
+    return cli.loader unless local || from_dir
+    if from_dir
+      unless ::File.directory?(from_dir)
+        logger.fatal("Not a directory: #{from_dir}")
+        exit(1)
+      end
+      if ::File.basename(from_dir) == ".toys"
+        from_dir = ::File.dirname(from_dir)
+      elsif from_dir =~ %r{(^|/)\.toys/}
+        logger.fatal("Directory is inside a toys directory: #{from_dir}")
+        exit(1)
+      end
+    end
+    special_cli = if local
+                    cli.child.add_search_path(from_dir || ::Dir.getwd)
+                  else
+                    ::Toys::StandardCLI.new(cur_dir: from_dir)
+                  end
+    special_cli.loader
+  end
+end
+
+desc "Tools that introspect available tools"
+
+long_desc \
+  "Tools that introspect the available tools."
+
+tool "list" do
+  desc "Output a list of the tools under the given namespace."
+
+  long_desc \
+    "Outputs a list of the tools under the given namespace, in YAML format, to the standard " \
+      "output stream."
+
+  remaining_args :namespace
+
+  flag :local do
+    desc "List only tools defined locally in the current directory."
+  end
+  flag :recursive, "--[no-]recursive" do
+    desc "Recursively list subtools"
+  end
+  flag :flatten, "--[no-]flatten" do
+    desc "Display a flattened list of tools"
+  end
+  flag :from_dir, "--dir=PATH" do
+    desc "List tools from the given directory."
+  end
+  flag :show_all, "--all" do
+    desc "Show all tools, including hidden tools and non-runnable namespaces"
+  end
+
+  include "tool-methods"
+  include "output-tools"
+
+  def run
+    loader = choose_loader(local, from_dir)
+    words = namespace
+    words = loader.split_path(words.first) if words.size == 1
+    tool_list = loader.list_subtools(words,
+                                     recursive: recursive,
+                                     include_hidden: show_all,
+                                     include_namespaces: show_all || !flatten,
+                                     include_non_runnable: show_all)
+    output = {
+      "namespace" => words.join(" "),
+      "tools" => [],
+    }
+    if flatten
+      output["tools"] = tool_list.map { |tool| format_tool(tool, words) }
+    else
+      format_tool_list(tool_list, output, words)
+    end
+    puts(generate_output(output))
+  end
+
+  def format_tool_list(tool_list, toplevel, cur_ns)
+    stack = [toplevel]
+    tool_list.each do |tool|
+      tool_name_size = tool.full_name.size
+      while cur_ns.size >= tool_name_size
+        stack.pop
+        cur_ns.pop
+      end
+      formatted = format_tool(tool, cur_ns)
+      (stack.last["tools"] ||= []) << formatted
+      stack.push(formatted)
+      cur_ns.push(tool.full_name.last)
+    end
+  end
+end
+
+tool "show" do
+  desc "Show detailed information about a single tool"
+
+  long_desc \
+    "Outputs details about the given tool, in YAML format, to the standard output stream."
+
+  remaining_args :name
+
+  flag :local do
+    desc "Show only tools defined locally in the current directory."
+  end
+  flag :from_dir, "--dir=PATH" do
+    desc "Show a tool accessible from the given directory."
+  end
+
+  include "tool-methods"
+  include "output-tools"
+
+  def run
+    loader = choose_loader(local, from_dir)
+    words = name
+    words = loader.split_path(words.first) if words.size == 1
+    tool = loader.lookup_specific(words)
+    output =
+      if tool.nil?
+        {
+          "name" => words.join(" "),
+          "exists" => false,
+        }
+      else
+        format_tool(tool, [], detailed: true)
+      end
+    puts(generate_output(output))
+    exit(tool.nil? ? 1 : 0)
+  end
+end

data/core-docs/toys/acceptor.rb

@@ -84,7 +84,7 @@ module Toys
       # When given a valid input, return an array in which the first element is
       # the original input string, and the remaining elements (which may be
       # empty) comprise any additional information that may be useful during
-      # conversion. If there is no additional information, you may return the
+      # conversion. If there is no additional information, you can return the
       # original input string by itself without wrapping in an array.
       #
       # When given an invalid input, return a falsy value such as `nil`.
@@ -96,8 +96,7 @@ module Toys
       # as the only array element, indicating all inputs are valid. You can
       # override this method to provide a different validation function.
       #
-      # @param str [String,nil] The input argument string. May be `nil` if the
-      #     value is optional and not provided.
+      # @param str [String] The input argument string.
       # @return [String,Array,nil]
       #
       def match(str)
@@ -111,8 +110,7 @@ module Toys
       # original input string and any other values returned from {#match}. It
       # must return the final converted value to use.
       #
-      # @param str [String,nil] Original argument string. May be `nil` if the
-      #     value is optional and not provided.
+      # @param str [String] Original argument string.
       # @param extra [Object...] Zero or more additional arguments comprising
       #     additional elements returned from the match function.
       # @return [Object] The converted argument as it should be stored in the
@@ -337,9 +335,7 @@ module Toys
     #
     NUMERIC_CONVERTER =
       proc do |s|
-        if s.nil?
-          nil
-        elsif s.include?("/")
+        if s.include?("/")
           Rational(s)
         elsif s.include?(".") || (s.include?("e") && s !~ /\A-?0x/)
           Float(s)
@@ -348,6 +344,41 @@ module Toys
         end
       end
 
+    ##
+    # A set of strings that are considered true for boolean acceptors.
+    # Currently set to `["+", "true", "yes"]`.
+    # @return [Array<String>]
+    #
+    TRUE_STRINGS = ["+", "true", "yes"].freeze
+
+    ##
+    # A set of strings that are considered false for boolean acceptors.
+    # Currently set to `["-", "false", "no", "nil"]`.
+    # @return [Array<String>]
+    #
+    FALSE_STRINGS = ["-", "false", "no", "nil"].freeze
+
+    ##
+    # A converter proc that handles boolean strings. Recognizes {TRUE_STRINGS}
+    # and {FALSE_STRINGS}. Useful for Simple acceptors.
+    # @return [Proc]
+    #
+    BOOLEAN_CONVERTER =
+      proc do |s|
+        if s.empty?
+          REJECT
+        else
+          s = s.downcase
+          if TRUE_STRINGS.any? { |t| t.start_with?(s) }
+            true
+          elsif FALSE_STRINGS.any? { |f| f.start_with?(s) }
+            false
+          else
+            REJECT
+          end
+        end
+      end
+
     class << self
       ##
       # Lookup a standard acceptor name recognized by OptionParser.
@@ -427,6 +458,17 @@ module Toys
       def create(spec = nil, **options, &block)
         # Source available in the toys-core gem
       end
+
+      ##
+      # Take the various ways to express an acceptor spec, and convert them to
+      # a canonical form expressed as a single object. This is called from the
+      # DSL to generate a spec object that can be stored.
+      #
+      # @private This interface is internal and subject to change without warning.
+      #
+      def scalarize_spec(spec, options, block)
+        # Source available in the toys-core gem
+      end
     end
   end
 end

data/core-docs/toys/cli.rb

@@ -193,9 +193,9 @@ module Toys
     end
 
     ##
-    # Make a clone with the same settings but no no config blocks and no paths
-    # in the loader. This is sometimes useful for calling another tool that has
-    # to be loaded from a different configuration.
+    # Make a clone with the same settings but no config blocks and no paths in
+    # the loader. This is sometimes useful for calling another tool that has to
+    # be loaded from a different configuration.
     #
     # @param opts [keywords] Any configuration arguments that should be
     #     modified from the original. See {#initialize} for a list of

data/core-docs/toys/completion.rb

@@ -325,5 +325,16 @@ module Toys
     def self.create(spec = nil, **options, &block)
       # Source available in the toys-core gem
     end
+
+    ##
+    # Take the various ways to express a completion spec, and convert them to a
+    # canonical form expressed as a single object. This is called from the DSL
+    # DSL to generate a spec object that can be stored.
+    #
+    # @private This interface is internal and subject to change without warning.
+    #
+    def self.scalarize_spec(spec, options, block)
+      # Source available in the toys-core gem
+    end
   end
 end

data/core-docs/toys/context.rb

@@ -297,7 +297,7 @@ module Toys
     end
 
     ##
-    # Exit immediately with the given status code
+    # Exit immediately with the given status code.
     #
     # @param code [Integer] The status code, which should be 0 for no error,
     #     or nonzero for an error condition. Default is 0.
@@ -308,7 +308,8 @@ module Toys
     end
 
     ##
-    # Exit immediately with the given status code
+    # Exit immediately with the given status code. This class method can be
+    # called if the instance method is or could be replaced by the tool.
     #
     # @param code [Integer] The status code, which should be 0 for no error,
     #     or nonzero for an error condition. Default is 0.
@@ -317,5 +318,18 @@ module Toys
     def self.exit(code = 0)
       # Source available in the toys-core gem
     end
+
+    ##
+    # Create a Context object. Applications generally will not need to create
+    # these objects directly; they are created by the tool when it is preparing
+    # for execution.
+    #
+    # @param data [Hash]
+    #
+    # @private This interface is internal and subject to change without warning.
+    #
+    def initialize(data)
+      # Source available in the toys-core gem
+    end
   end
 end

data/core-docs/toys/core.rb

@@ -9,6 +9,6 @@ module Toys
     # Current version of Toys core.
     # @return [String]
     #
-    VERSION = "0.13.1"
+    VERSION = "0.14.2"
   end
 end

data/core-docs/toys/dsl/tool.rb

@@ -263,7 +263,8 @@ module Toys
       #       end
       #     end
       #
-      # The following example defines a tool that runs one of its subtools.
+      # The following example uses `delegate_to` to define a tool that runs one
+      # of its subtools.
       #
       #     tool "test", delegate_to: ["test", "unit"] do
       #       tool "unit" do
@@ -284,19 +285,22 @@ module Toys
       #     delegate to another tool, specified by the full path. This path may
       #     be given as an array of strings, or a single string possibly
       #     delimited by path separators.
+      # @param delegate_relative [String,Array<String>] Optional. Similar to
+      #     delegate_to, but takes a delegate name relative to the context in
+      #     which this tool is being defined.
       # @param block [Proc] Defines the subtool.
       # @return [self]
       #
-      def tool(words, if_defined: :combine, delegate_to: nil, &block)
+      def tool(words, if_defined: :combine, delegate_to: nil, delegate_relative: nil, &block)
         # Source available in the toys-core gem
       end
 
       ##
       # Create an alias, representing an "alternate name" for a tool.
       #
-      # This is functionally equivalent to creating a subtool with the
-      # `delegate_to` option, except that `alias_tool` takes a _relative_ name
-      # for the delegate.
+      # Note: This is functionally equivalent to creating a tool with the
+      # `:delegate_relative` option. As such, `alias_tool` is considered
+      # deprecated.
       #
       # ### Example
       #
@@ -309,20 +313,24 @@ module Toys
       #       end
       #     end
       #     alias_tool "t", "test"
+      #     # Note: the following is preferred over alias_tool:
+      #     # tool "t", delegate_relative: "test"
       #
       # @param word [String] The name of the alias
       # @param target [String,Array<String>] Relative path to the target of the
       #     alias. This path may be given as an array of strings, or a single
       #     string possibly delimited by path separators.
       # @return [self]
+      # @deprecated Use {#tool} and pass `:delegate_relative` instead
       #
       def alias_tool(word, target)
         # Source available in the toys-core gem
       end
 
       ##
-      # Causes the current tool to delegate to another tool. When run, it
-      # simply invokes the target tool with the same arguments.
+      # Causes the current tool to delegate to another tool, specified by the
+      # full tool name. When run, it simply invokes the target tool with the
+      # same arguments.
       #
       # ### Example
       #