toys 0.13.1 → 0.14.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 76a2e6d85d0d1baa0dc154bc6572bbd600c855c9e3cf6c6e64594630a33c58ed
-  data.tar.gz: c701ff4fd53eb2a7dd85561b4c19e1614a43a9788f73abdbeaa86e8a9a17fb68
+  metadata.gz: fc447cfb397077cb652af00a19031a4b1c439ed483117fecf9e855bf286dbeb5
+  data.tar.gz: 951fb9c8104eb35ac1275efd2c0e7e0c784538b9b878b560861ea7bcb380144a
 SHA512:
-  metadata.gz: da3b8b9da43c2bc81e286206fa649fc994b5083376357cf3c4f7d3289b8d8d928da8faffa42a32e4313074a24666ec40a1eabc73d8b1c82304bc6cd1eed99feb
-  data.tar.gz: 8f36a970ed0292a604dcb28cb5a48ad4697fc0a5fbb27389f6dc937e9ea543488c2b28bfc1c148f93044eb26aed4f03c8911f9df921fafb54f352020144f041b
+  metadata.gz: 1fcdf6af97a7c1516f37760206aa67445f522df7fe0898e61a0ddbc568411b404bab109e68cbdedf21e755bf7a2bf6e3be7c4deb3a52a9e8e36fe29db0148e6b
+  data.tar.gz: fbaaea69be3d5ebffe89167ab01bd51dcab5006239fb20ba00dab31c14f3d0f335cac57c81aad541519e355ae699a5b8c2ed23f8c8ab4db8b8ed64ff239c2c9b

data/CHANGELOG.md

@@ -1,5 +1,34 @@
 # Release History
 
+### v0.14.1 / 2022-10-03
+
+* FIXED: Fixed a crash due to a missing file in the gem
+
+### v0.14.0 / 2022-10-03
+
+Toys 0.14.0 is a major release with new system tools for introspecting defined tools, pager support, support for tees and pipes in the Exec utility, some cleanup of the behavior of Acceptors, and other improvements.
+
+Fixes that are potentially breaking:
+
+* Disallowed acceptors on flags that are explicitly boolean.
+* Acceptors no longer sometimes apply to the boolean setting of a flag with an optional value.
+
+New functionality:
+
+* Implemented new builtins for obtaining information about defined tools: `system tools show` and `system tools list`.
+* Implemented a utility class and mixin for output pagers.
+* Builtin commands that display data can format as either YAML or JSON.
+* The Exec utility and mixin can tee (i.e. duplicate and split) output streams.
+* The Exec utility and mixin can take pipes as input and output streams.
+* The Exec mixin provides a `verbosity_flags` convenience method.
+
+Fixes:
+
+* The `system test` builtin no longer requires toys to be installed as a gem.
+* Fixed a failure when passing a relative path to `system test --directory`.
+* Contents of preload directories are loaded in sorted order.
+* Various clarifications, fixes, and updates to the users guide and documentation.
+
 ### v0.13.1 / 2022-03-01
 
 * FIXED: Bundler integration no longer fails if a bundle was locked to a different version of a builtin gem

data/README.md

@@ -19,7 +19,7 @@ gem. For more info on using toys-core, see
 
 Here's a tutorial to help you get a feel of what Toys can do.
 
-### Install Toys
+### Install and try out Toys
 
 Install the **toys** gem using:
 
@@ -50,14 +50,6 @@ Toys does not yet specially implement tab completion for zsh or other shells.
 However, if you are using zsh, installing bash completion using `bashcompinit`
 *mostly* works.
 
-Toys requires Ruby 2.4 or later.
-
-Most parts of Toys work on JRuby. However, JRuby is not recommended because of
-JVM boot latency, lack of support for Kernel#fork, and other issues.
-
-Most parts of Toys work on TruffleRuby. However, TruffleRuby is not recommended
-because it has a few known bugs that affect Toys.
-
 ### Write your first tool
 
 You can define tools by creating a *Toys file*. Go into any directory, and,
@@ -244,8 +236,7 @@ you may need to add Toys to your Gemfile and use Bundler to invoke Toys (i.e.
 `bundle exec toys test`). This is because Toys is just calling the Rake API to
 run your task, and the Rake task might require the bundle. However, when Toys
 is not wrapping Rake, typical practice is actually *not* to use `bundle exec`.
-Toys provides its own mechanisms to setup a bundle, or to activate and even
-install individual gems.
+Toys provides its own mechanisms to manage bundles or install gems for you.
 
 So far, we've made Toys a front-end for your Rake tasks. This may be useful by
 itself. Toys lets you pass command line arguments "normally" to tools, whereas
@@ -256,11 +247,11 @@ than Rake does.
 But you also might find Toys a more natural way to *write* tasks, and indeed
 you can often rewrite an entire Rakefile as a Toys file and get quite a bit of
 benefit in readability and maintainability. For an example, see the
-[Toys file for the Toys repo itself](https://github.com/dazuma/toys/blob/main/toys/.toys.rb).
-It contains the Toys scripts that I use to develop, test, and release Toys
-itself. Yes, Toys is self-hosted. You'll notice most of this Toys file consists
-of template expansions. Toys provides templates for a lot of common build,
-test, and release tasks for Ruby projects.
+[Toys file for the Toys gem itself](https://github.com/dazuma/toys/blob/main/toys/.toys/.toys.rb).
+It contains Toys scripts that I use to develop, test, and release Toys itself.
+Yes, Toys is self-hosted. You'll notice most of this Toys file consists of
+template expansions. Toys provides templates for a lot of common build, test,
+and release tasks for Ruby projects.
 
 If you're feeling adventurous, try translating some of your Rake tasks into
 native Toys tools. You can do so in your existing `.toys.rb` file. Keep the
@@ -305,7 +296,7 @@ them. Furthermore, when writing new scripts, I was repeating the same
 OptionParser boilerplate and common functionality.
 
 Toys was designed to address those problems by providing a framework for
-writing *and organizing* your own command line scripts. You provide the actual
+writing and organizing your own command line scripts. You provide the actual
 functionality by writing Toys files, and Toys takes care of all the other
 details expected from a good command line tool. It provides a streamlined
 interface for defining and handling command line flags and positional
@@ -320,6 +311,16 @@ scripts written for Toys can be invoked and passed arguments and flags using
 familiar unix command line conventions. The Toys github repo itself comes with
 Toys scripts instead of Rakefiles.
 
+## System requirements
+
+Toys requires Ruby 2.4 or later.
+
+Most parts of Toys work on JRuby. However, JRuby is not recommended because of
+JVM boot latency, lack of support for Kernel#fork, and other issues.
+
+Most parts of Toys work on TruffleRuby. However, TruffleRuby is not recommended
+because it has a few known bugs that affect Toys.
+
 ## License
 
 Copyright 2019-2022 Daniel Azuma and the Toys contributors

data/builtins/system/.toys.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+desc "A set of system commands for Toys"
+
+long_desc "Contains tools that inspect, configure, and update Toys itself."
+
+tool "version" do
+  desc "Print the current Toys version"
+
+  def run
+    puts ::Toys::VERSION
+  end
+end
+
+mixin "output-tools" do
+  on_include do
+    flag :output_format, "--format=FORMAT" do
+      accept ["json", "json-compact", "yaml"]
+      desc 'The output format. Recognized values are "yaml" (the default), "json", and ' \
+           '"json-compact".'
+    end
+  end
+
+  def generate_output(object)
+    case output_format
+    when "json"
+      require "json"
+      ::JSON.pretty_generate(object)
+    when "json-compact"
+      require "json"
+      ::JSON.generate(object)
+    when nil, "yaml"
+      require "psych"
+      ::Psych.dump(object)
+    else
+      logger.error("Unknown output format: #{format}")
+      exit(1)
+    end
+  end
+end

data/builtins/system/git-cache.rb

@@ -23,15 +23,16 @@ tool "list" do
     desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
   end
 
+  include "output-tools"
+
   def run
-    require "psych"
     require "toys/utils/git_cache"
     git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
     output = {
       "cache_dir" => git_cache.cache_dir,
       "remotes" => git_cache.remotes,
     }
-    puts(::Psych.dump(output))
+    puts(generate_output(output))
   end
 end
 
@@ -48,8 +49,9 @@ tool "show" do
     desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
   end
 
+  include "output-tools"
+
   def run
-    require "psych"
     require "toys/utils/git_cache"
     git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
     info = git_cache.repo_info(remote)
@@ -57,7 +59,7 @@ tool "show" do
       logger.fatal("Unknown remote: #{remote}")
       exit(1)
     end
-    puts(::Psych.dump(info.to_h))
+    puts(generate_output(info.to_h))
   end
 end
 
@@ -130,8 +132,9 @@ tool "remove" do
     desc "Remove all repositories. Required unless specific remotes are provided."
   end
 
+  include "output-tools"
+
   def run
-    require "psych"
     require "toys/utils/git_cache"
     if remotes.empty? == !all
       logger.fatal("You must specify at least one remote to clear, or --all to clear all remotes.")
@@ -142,7 +145,7 @@ tool "remove" do
     output = {
       "removed" => removed,
     }
-    puts(::Psych.dump(output))
+    puts(generate_output(output))
   end
 end
 
@@ -175,8 +178,9 @@ tool "remove-refs" do
     desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
   end
 
+  include "output-tools"
+
   def run
-    require "psych"
     require "toys/utils/git_cache"
     git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
     removed = git_cache.remove_refs(remote, refs: refs)
@@ -188,7 +192,7 @@ tool "remove-refs" do
       "remote" => remote,
       "removed_refs" => removed.map(&:to_h),
     }
-    puts(::Psych.dump(output))
+    puts(generate_output(output))
   end
 end
 
@@ -220,8 +224,9 @@ tool "remove-sources" do
     desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
   end
 
+  include "output-tools"
+
   def run
-    require "psych"
     require "toys/utils/git_cache"
     git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
     removed = git_cache.remove_sources(remote, commits: commits)
@@ -233,6 +238,6 @@ tool "remove-sources" do
       "remote" => remote,
       "removed_sources" => removed.map(&:to_h),
     }
-    puts(::Psych.dump(output))
+    puts(generate_output(output))
   end
 end

data/builtins/system/test.rb

@@ -30,7 +30,6 @@ include :terminal
 
 def run
   load_minitest_gems
-  ::Dir.chdir(tool_dir)
   test_files = find_test_files
   result = exec_ruby(ruby_args, in: :controller, log_cmd: "Starting minitest...") do |controller|
     controller.in.puts("gem 'minitest', '= #{::Minitest::VERSION}'")
@@ -43,7 +42,6 @@ def run
       controller.in.puts("gem 'minitest-rg', '= #{::MiniTest::RG::VERSION}'")
       controller.in.puts("require 'minitest/rg'")
     end
-    controller.in.puts("gem 'toys', '= #{::Toys::VERSION}'")
     controller.in.puts("require 'toys'")
     controller.in.puts("require 'toys/testing'")
     if directory
@@ -79,7 +77,8 @@ end
 def find_test_files
   glob = ".test/**/test_*.rb"
   glob = "**/#{glob}" if recursive
-  test_files = ::Dir.glob(glob)
+  glob = "#{tool_dir}/#{glob}"
+  test_files = Dir.glob(glob)
   if test_files.empty?
     logger.warn("No test files found")
     exit

data/builtins/system/tools.rb

@@ -0,0 +1,145 @@
+# frozen_string_literal: true
+
+mixin "tool-methods" do
+  def format_tool(tool, namespace, detailed: false)
+    output = {
+      "name" => tool.full_name[namespace.length..-1].join(" "),
+      "desc" => tool.desc.to_s,
+      "runnable" => tool.runnable?,
+    }
+    if detailed
+      output["exists"] = true
+      output["long_desc"] = tool.long_desc.map(&:to_s)
+    end
+    output
+  end
+
+  def choose_loader(local, from_dir)
+    return cli.loader unless local || from_dir
+    if from_dir
+      unless ::File.directory?(from_dir)
+        logger.fatal("Not a directory: #{from_dir}")
+        exit(1)
+      end
+      if ::File.basename(from_dir) == ".toys"
+        from_dir = ::File.dirname(from_dir)
+      elsif from_dir =~ %r{(^|/)\.toys/}
+        logger.fatal("Directory is inside a toys directory: #{from_dir}")
+        exit(1)
+      end
+    end
+    special_cli = if local
+                    cli.child.add_search_path(from_dir || ::Dir.getwd)
+                  else
+                    ::Toys::StandardCLI.new(cur_dir: from_dir)
+                  end
+    special_cli.loader
+  end
+end
+
+desc "Tools that introspect available tools"
+
+long_desc \
+  "Tools that introspect the available tools."
+
+tool "list" do
+  desc "Output a list of the tools under the given namespace."
+
+  long_desc \
+    "Outputs a list of the tools under the given namespace, in YAML format, to the standard " \
+      "output stream."
+
+  remaining_args :namespace
+
+  flag :local do
+    desc "List only tools defined locally in the current directory."
+  end
+  flag :recursive, "--[no-]recursive" do
+    desc "Recursively list subtools"
+  end
+  flag :flatten, "--[no-]flatten" do
+    desc "Display a flattened list of tools"
+  end
+  flag :from_dir, "--dir=PATH" do
+    desc "List tools from the given directory."
+  end
+  flag :show_all, "--all" do
+    desc "Show all tools, including hidden tools and non-runnable namespaces"
+  end
+
+  include "tool-methods"
+  include "output-tools"
+
+  def run
+    loader = choose_loader(local, from_dir)
+    words = namespace
+    words = loader.split_path(words.first) if words.size == 1
+    tool_list = loader.list_subtools(words,
+                                     recursive: recursive,
+                                     include_hidden: show_all,
+                                     include_namespaces: show_all || !flatten,
+                                     include_non_runnable: show_all)
+    output = {
+      "namespace" => words.join(" "),
+      "tools" => [],
+    }
+    if flatten
+      output["tools"] = tool_list.map { |tool| format_tool(tool, words) }
+    else
+      format_tool_list(tool_list, output, words)
+    end
+    puts(generate_output(output))
+  end
+
+  def format_tool_list(tool_list, toplevel, cur_ns)
+    stack = [toplevel]
+    tool_list.each do |tool|
+      tool_name_size = tool.full_name.size
+      while cur_ns.size >= tool_name_size
+        stack.pop
+        cur_ns.pop
+      end
+      formatted = format_tool(tool, cur_ns)
+      (stack.last["tools"] ||= []) << formatted
+      stack.push(formatted)
+      cur_ns.push(tool.full_name.last)
+    end
+  end
+end
+
+tool "show" do
+  desc "Show detailed information about a single tool"
+
+  long_desc \
+    "Outputs details about the given tool, in YAML format, to the standard output stream."
+
+  remaining_args :name
+
+  flag :local do
+    desc "Show only tools defined locally in the current directory."
+  end
+  flag :from_dir, "--dir=PATH" do
+    desc "Show a tool accessible from the given directory."
+  end
+
+  include "tool-methods"
+  include "output-tools"
+
+  def run
+    loader = choose_loader(local, from_dir)
+    words = name
+    words = loader.split_path(words.first) if words.size == 1
+    tool = loader.lookup_specific(words)
+    output =
+      if tool.nil?
+        {
+          "name" => words.join(" "),
+          "exists" => false,
+        }
+      else
+        format_tool(tool, [], detailed: true)
+      end
+    puts(generate_output(output))
+    exit(tool.nil? ? 1 : 0)
+  end
+end

data/core-docs/toys/acceptor.rb

@@ -84,7 +84,7 @@ module Toys
       # When given a valid input, return an array in which the first element is
       # the original input string, and the remaining elements (which may be
       # empty) comprise any additional information that may be useful during
-      # conversion. If there is no additional information, you may return the
+      # conversion. If there is no additional information, you can return the
       # original input string by itself without wrapping in an array.
       #
       # When given an invalid input, return a falsy value such as `nil`.
@@ -96,8 +96,7 @@ module Toys
       # as the only array element, indicating all inputs are valid. You can
       # override this method to provide a different validation function.
       #
-      # @param str [String,nil] The input argument string. May be `nil` if the
-      #     value is optional and not provided.
+      # @param str [String] The input argument string.
       # @return [String,Array,nil]
       #
       def match(str)
@@ -111,8 +110,7 @@ module Toys
       # original input string and any other values returned from {#match}. It
       # must return the final converted value to use.
       #
-      # @param str [String,nil] Original argument string. May be `nil` if the
-      #     value is optional and not provided.
+      # @param str [String] Original argument string.
       # @param extra [Object...] Zero or more additional arguments comprising
       #     additional elements returned from the match function.
       # @return [Object] The converted argument as it should be stored in the
@@ -337,9 +335,7 @@ module Toys
     #
     NUMERIC_CONVERTER =
       proc do |s|
-        if s.nil?
-          nil
-        elsif s.include?("/")
+        if s.include?("/")
           Rational(s)
         elsif s.include?(".") || (s.include?("e") && s !~ /\A-?0x/)
           Float(s)
@@ -348,6 +344,41 @@ module Toys
         end
       end
 
+    ##
+    # A set of strings that are considered true for boolean acceptors.
+    # Currently set to `["+", "true", "yes"]`.
+    # @return [Array<String>]
+    #
+    TRUE_STRINGS = ["+", "true", "yes"].freeze
+
+    ##
+    # A set of strings that are considered false for boolean acceptors.
+    # Currently set to `["-", "false", "no", "nil"]`.
+    # @return [Array<String>]
+    #
+    FALSE_STRINGS = ["-", "false", "no", "nil"].freeze
+
+    ##
+    # A converter proc that handles boolean strings. Recognizes {TRUE_STRINGS}
+    # and {FALSE_STRINGS}. Useful for Simple acceptors.
+    # @return [Proc]
+    #
+    BOOLEAN_CONVERTER =
+      proc do |s|
+        if s.empty?
+          REJECT
+        else
+          s = s.downcase
+          if TRUE_STRINGS.any? { |t| t.start_with?(s) }
+            true
+          elsif FALSE_STRINGS.any? { |f| f.start_with?(s) }
+            false
+          else
+            REJECT
+          end
+        end
+      end
+
     class << self
       ##
       # Lookup a standard acceptor name recognized by OptionParser.
@@ -427,6 +458,17 @@ module Toys
       def create(spec = nil, **options, &block)
         # Source available in the toys-core gem
       end
+
+      ##
+      # Take the various ways to express an acceptor spec, and convert them to
+      # a canonical form expressed as a single object. This is called from the
+      # DSL to generate a spec object that can be stored.
+      #
+      # @private This interface is internal and subject to change without warning.
+      #
+      def scalarize_spec(spec, options, block)
+        # Source available in the toys-core gem
+      end
     end
   end
 end

data/core-docs/toys/cli.rb

@@ -193,9 +193,9 @@ module Toys
     end
 
     ##
-    # Make a clone with the same settings but no no config blocks and no paths
-    # in the loader. This is sometimes useful for calling another tool that has
-    # to be loaded from a different configuration.
+    # Make a clone with the same settings but no config blocks and no paths in
+    # the loader. This is sometimes useful for calling another tool that has to
+    # be loaded from a different configuration.
     #
     # @param opts [keywords] Any configuration arguments that should be
     #     modified from the original. See {#initialize} for a list of

data/core-docs/toys/completion.rb

@@ -325,5 +325,16 @@ module Toys
     def self.create(spec = nil, **options, &block)
       # Source available in the toys-core gem
     end
+
+    ##
+    # Take the various ways to express a completion spec, and convert them to a
+    # canonical form expressed as a single object. This is called from the DSL
+    # DSL to generate a spec object that can be stored.
+    #
+    # @private This interface is internal and subject to change without warning.
+    #
+    def self.scalarize_spec(spec, options, block)
+      # Source available in the toys-core gem
+    end
   end
 end

data/core-docs/toys/context.rb

@@ -297,7 +297,7 @@ module Toys
     end
 
     ##
-    # Exit immediately with the given status code
+    # Exit immediately with the given status code.
     #
     # @param code [Integer] The status code, which should be 0 for no error,
     #     or nonzero for an error condition. Default is 0.
@@ -308,7 +308,8 @@ module Toys
     end
 
     ##
-    # Exit immediately with the given status code
+    # Exit immediately with the given status code. This class method can be
+    # called if the instance method is or could be replaced by the tool.
     #
     # @param code [Integer] The status code, which should be 0 for no error,
     #     or nonzero for an error condition. Default is 0.
@@ -317,5 +318,18 @@ module Toys
     def self.exit(code = 0)
       # Source available in the toys-core gem
     end
+
+    ##
+    # Create a Context object. Applications generally will not need to create
+    # these objects directly; they are created by the tool when it is preparing
+    # for execution.
+    #
+    # @param data [Hash]
+    #
+    # @private This interface is internal and subject to change without warning.
+    #
+    def initialize(data)
+      # Source available in the toys-core gem
+    end
   end
 end

data/core-docs/toys/core.rb

@@ -9,6 +9,6 @@ module Toys
     # Current version of Toys core.
     # @return [String]
     #
-    VERSION = "0.13.1"
+    VERSION = "0.14.1"
   end
 end

data/core-docs/toys/errors.rb

@@ -51,6 +51,19 @@ module Toys
   # thrown during tool execution.
   #
   class ContextualError < ::StandardError
+    ##
+    # Construct a ContextualError. This exception type is thrown from
+    # {ContextualError.capture} and {ContextualError.capture_path} and should
+    # not be constructed directly.
+    #
+    # @private This interface is internal and subject to change without warning.
+    #
+    def initialize(cause, banner,
+                   config_path: nil, config_line: nil,
+                   tool_name: nil, tool_args: nil)
+      # Source available in the toys-core gem
+    end
+
     ##
     # The underlying exception
     # @return [::StandardError]
@@ -88,6 +101,26 @@ module Toys
     attr_reader :tool_args
 
     class << self
+      ##
+      # Execute the given block, and wrap any exceptions thrown with a
+      # ContextualError. This is intended for loading a config file from the
+      # given path, and wraps any Ruby parsing errors.
+      #
+      # @private This interface is internal and subject to change without warning.
+      #
+      def capture_path(banner, path, **opts)
+        # Source available in the toys-core gem
+      end
+
+      ##
+      # Execute the given block, and wrap any exceptions thrown with a
+      # ContextualError.
+      #
+      # @private This interface is internal and subject to change without warning.
+      #
+      def capture(banner, **opts)
+        # Source available in the toys-core gem
+      end
     end
   end
 end

data/core-docs/toys/flag.rb

@@ -77,6 +77,7 @@ module Toys
       # The type of flag (`:boolean` or `:value`)
       # @return [:boolean] if this is a boolean flag (i.e. no value)
       # @return [:value] if this flag takes a value (even if optional)
+      # @return [nil] if this flag is indeterminate
       #
       attr_reader :flag_type
 
@@ -111,6 +112,16 @@ module Toys
       # @return [String]
       #
       attr_reader :canonical_str
+
+      ##
+      # This method is accessible for testing only.
+      #
+      # @private This interface is internal and subject to change without warning.
+      #
+      def configure_canonical(canonical_flag_type, canonical_value_type,
+                              canonical_value_label, canonical_value_delim)
+        # Source available in the toys-core gem
+      end
     end
 
     ##