toys 0.22.0 → 0.24.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 7c91d8f8914503e8d8baf7eae17725fa3eee2470273311917561c0c1dfa9b341
-  data.tar.gz: 01a7b93c64123371a1f82afbb708c15377efc9b63b0bd2c52fd5dd0e9acee8c7
+  metadata.gz: 1a607f5cce147c21bdd90781026767b310054e69bbbff8c9a1103fdbd3d77b6b
+  data.tar.gz: ab0b2ff711816c75e2a50206e93ae19673c21f88618a3332e0642387b7baa6f3
 SHA512:
-  metadata.gz: f888f8e2b005d13c943a7856add1ae37d56739dc0553612fa09b13472a4b43b002a594b11b8d77723f4ddafdf05170a99c2ade023ed247136cd946d7345784fa
-  data.tar.gz: a8e90f01ab91878bf4175bde290c39d1d126985a77d23779134aa008ba217801f1f2b2e876733b4bf1ef61336506758b7fb4f9dab0bd3a5da7c03ae9aba9d141
+  metadata.gz: a99af47c6944f33da01db83f1aeb9d77021424cf2ef3b739c7f34caeaffac8dfbcfa25fff7e6ce73d742c4d4d44cabd1ab2b6521d3b64aeaf7d9adb5aeff0ffa
+  data.tar.gz: 3d853bf3958d6894a5388830c5f4c2595f3631a13280741d93cc70410d43374c690273a23314bd1758b558a523b0d3d4372446f3054a2d616cf9b78db0d526bd

data/.yardopts

@@ -1,7 +1,6 @@
 --no-private
 --title=Toys
 --markup=markdown
---markup-provider redcarpet
 --main=README.md
 ./core-docs/toys/**/*.rb
 ./core-docs/toys-core.rb

data/CHANGELOG.md

@@ -1,5 +1,78 @@
 # Release History
 
+### v0.24.0 / 2026-09-11
+
+* Feature: The `toys do` builtin provides an `--on-missing-gem` flag that governs how the `--gem` flag handles gems that are not installed.
+* BREAKING CHANGE: Removed deprecated `gem_version:` argument from `:rspec` and `:minitest` templates.
+
+### v0.23.0 / 2026-09-09
+
+This is a major release, with several new features. It also includes a significant refactor of some of the underlying layers, which should be mostly invisible. It is a release candidate for the upcoming version 1.0.
+
+Highlights include:
+
+* Added the ability to treat unknown flags as positional args via the `treat_unknown_flags_as_args` DSL directive. Useful for wrapping and delegating to other commands.
+* The `do` builtin tool supports `--gem=`, `--git=`, and `--path=` flags that make additional tool sources available.
+* Running tools from within a tool should generally now be done via the new `Toys::Runner` object, now available in the runtime context, instead of via the `Toys::CLI`. The new object provides finer-grained control over error handling and verbosity.
+* Updates to error handling to improve the output and fix some long-standing issues related to tool delegation.
+* Several fixes to bundler integration, especially with Bundler 4.
+
+Details follow.
+
+* Changes to tool definition:
+    * Added the `treat_unknown_flags_as_args` DSL directive (and the corresponding `ToolDefinition#treat_unknown_flags_as_args` setter and `ToolDefinition#unknown_flags_are_args?` query), which redirects unrecognized flags to the tool's positional arguments instead of reporting a usage error. Useful for wrapping and delegating to other commands.
+    * The `set_context_directory` DSL directive can now take Pathname arguments and handles relative paths (by expanding to absolute paths).
+    * When two or more flags are declared using the same context key, within a flag group, the flag group's requirements now evaluate correctly.
+    * A flag or optional/remaining positional argument declared with a `nil` default no longer clears default data already set for the same key by an earlier declaration.
+    * Toys now raises `Toys::ToolDefinitionError` if you attempt to "reset" a tool previously defined using a `Toys::Tool` subclass.
+    * SECURITY FIX: Toys now raises `Toys::ToolSourceError` if a tool loaded from a RubyGem attempts to load tools from the local file system.
+    * BREAKING CHANGE: Toys now raises `Toys::ToolDefintionError` if you attempt to create or descend into a subtool from a `subtool_apply` block or a middleware-based config.
+    * Caught additional cases of incorrectly located `Toys::Tool` subclasses and raised `Toys::ToolDefinitionError`
+    * Toys directory contents are loaded in deterministic (sorted) order.
+
+* Changes to tool execution:
+    * Added `Toys::Context#runner` for access to the `Toys::Runner` object. This is now the preferred way to run tools from within other tools (instead of `Toys::CLI#run`), unless you need to change the configuration.
+    * Added `Toys::Context#loader` for access to the `Toys::Loader` object.
+    * BREAKING CHANGE: `Toys::Context#context_directory`, and `Toys::Context#get` with the associated context key, now never return nil. If no context directory is set for the tool, these now return the current working directory.
+    * BREAKING CHANGE: Removed git- and gem- specific attributes from `Toys::SourceInfo` and replaced with `Toys::SourceInfo#origin`.
+    * Standard context keys and sentinels use the new class `Toys::UniqueKey`, which displays useful names in diagnostic output.
+    * Delegation now propagates the caller's verbosity to the delegate target, rather than resetting it to zero.
+    * When a tool delegates, a `SignalException` unhandled by the inner tool now propagates outward so each tool in the delegation chain gets a chance at its own `on_interrupt` / `on_signal` handler.
+    * A nested run that shares a logger with the run that called it now uses the base level already in effect for that logger, so verbosity no longer compounds across nested runs.
+    * Revamped the output and stack trace printed when an exception occurs.
+
+* Changes to shell completion:
+    * Fixed shell completion for non-flag words containing `=` or `:`: completions are now computed against the whole word, and the engine trims candidates to the span the shell will actually replace.
+    * Zsh completion now replaces the entire word rather than only the text after an `=` or `:`, matching how zsh actually handles word breaks; bash continues to break at `=` and `:`.
+    * Word-break trimming for bash now ignores `=` and `:` characters that were quoted or backslash-escaped, since the shell does not break a word at a quoted character.
+    * Subtool completion now uses the loader's configured delimiters to decide where a tool path ends, rather than a separate regex over the fragment prefix, so completing a partially typed tool path works consistently with any configured extra delimiters.
+
+* Changes to builtin tools:
+    * The `do` builtin tool supports `--gem=`, `--git=`, and `--path=` flags that make additional tool sources available.
+
+* Changes to CLI interfaces:
+    * `Toys::CLI#child` accepts `copy_sources: true`, which populates the new CLI's loader with the same sources as the original. This makes it easy to create a new CLI with additional sources on top of the current.
+    * You can get the `Toys::Runner` for a CLI using `Toys::CLI#runner`, providing a way to customize the run process more closely than using `Toys::CLI#run`.
+    * Added `add_source` method that takes an instance of the new `Toys::SourceSpec` types that describe a tool source. This method replaces the now deprecated `add_config_*` methods (although those methods will continue to be supported for a time). In general, the "config" terminology is being retired in favor of "source" which I think better describes what is going on.
+    * BREAKING CHANGE: The `config_file_name` and `config_dir_name` arguments to CLI have been renamed to `toplevel_tool_file_name` and `toplevel_tool_dir_name`, respectively, as part of a general removal of the "config" term. The old names are not aliased.
+    * BREAKING CHANGE: `Toys::CLI#run` no longer accepts the `delegated_from:` keyword argument. Tool delegation is now handled internally and its runtime is not exposed in the public interface.
+    * BREAKING CHANGE: Renamed the `StandardCLI::CONFIG_DIR_NAME` to `StandardCLI::TOPLEVEL_TOOL_DIR_NAME` and `StandardCLI::CONFIG_FILE_NAME` to `StandardCLI::TOPLEVEL_TOOL_FILE_NAME`.
+    * BREAKING CHANGE: Removed the constants `INDEX_FILE_NAME`, `PRELOAD_DIR_NAME`, `PRELOAD_FILE_NAME`, `DATA_DIR_NAME`, and `LIB_DIR_NAME` from `StandardCLI`.
+    * The `Toys::StandardCLI` constructor now takes `git_cache` and `gems_util` keyword arguments letting you customize the objects used to resolve git and gem sources.
+    * Path parameters for the methods that add sources, such as `path` and `context_directory`, can now take Pathname objects and can handle relative paths properly (by expanding them into absolute paths).
+
+* Rubygems/Bundler integration changes:
+    * Bundler integration requires Bundler 2.4 or later. This is the default on Ruby 3.2, but requires that Bundler is updated on earlier Rubies.
+    * Bundler integration properly detects provenance attributes such as `path:` and `git:`.
+    * Calling bundler integration with a nonempty group list no longer loses gems that were already loaded.
+    * Bundler integration now honors `BUNDLE_LOCKFILE` (new in Bundler 4) and stops leaking it from the modified bundle.
+
+* Other fixes
+    * Terminal mixin now raises the correct ArgumentError (instead of NameError) if given an unknown style code.
+    * `Toys::Loader#list_subtools` now ensures returned tools have been finished by their middleware. This affects listing tools not underneath the currently running tool.
+
+A number of additional minor breaking changes were made to internal interfaces in the toys-core gem classes. It is unlikely you will encounter these unless you are extending the framework, but you can see the toys-core changelog for details.
+
 ### v0.22.0 / 2026-05-05
 
 Toys 0.22 is a major release focused on polish and cleanup in preparation for version 1.0. It includes a number of small breaking changes where needed to clean up the interfaces. (Note that many of the changes listed below are actually in the `toys-core` gem.)

data/builtins/do.rb

@@ -15,7 +15,12 @@ long_desc \
   "You may change the delimiter using the --delim flag. For example:",
   ["    toys do --delim=/ rails build --staging / deploy --migrate"],
   "The --delim flag must appear first before the tools to run. Any flags that appear later in" \
-    " the command line will be passed to the tools themselves."
+    " the command line will be passed to the tools themselves.",
+  "",
+  "You may also load tools from additional sources, by passing the --gem, --git, and --path" \
+    " flags. Any number of these flags can be added; each adds a source of tools, with earlier" \
+    " flags taking priority over later. For example:",
+  ["    toys do --gem=my-tools --git=https://github.com/dazuma/example deploy --migrate"]
 
 flag :delim do
   flags "-d", "--delim=VALUE"
@@ -24,6 +29,82 @@ flag :delim do
   long_desc "Sets the delimiter that separates tool invocations. The default value is \",\"."
 end
 
+# The source flags all share the :sources key, and their handlers append to
+# the same array, tagging each value with the kind of source requested. That
+# way the array records the order in which the flags appeared on the command
+# line, across all of the flags, which is the order that determines priority.
+# The handlers store the raw value; parsing happens later, in build_cli,
+# because an error raised from a handler would come out of the argument parser
+# as a stack trace rather than as a simple message.
+
+flag :sources do
+  flags "--gem=GEM"
+  handler { |val, prev| prev + [[:gem, val]] }
+  default []
+  desc "Make the tools from the given gem available"
+  long_desc \
+    "Adds the tools from the given gem. If the specified gem is not currently installed," \
+      " follows the policy set by the --on-missing-gem flag, which is normally to prompt" \
+      " whether to install.",
+    "",
+    "The value is the gem name, optionally followed by any number of version requirements," \
+      " all separated by commas. Whitespace surrounding each element is ignored. The version" \
+      " requirements use the same syntax as Rubygems and Bundler. For example:",
+    ["    --gem=\"my-tools, ~> 1.5, >= 1.5.2\""]
+end
+
+flag :sources do
+  flags "--git=SPEC"
+  handler { |val, prev| prev + [[:git, val]] }
+  default []
+  desc "Make the tools from the given git repository available"
+  long_desc \
+    "Adds the tools from the given git repository, fetching the repository into a local cache" \
+      " if it is not already there.",
+    "",
+    "The value is the git remote (i.e. the repository URL or path), optionally followed by any" \
+      " number of \"key=value\" elements, all separated by commas. There is no way to escape" \
+      " a comma appearing within the value. Whitespace surrounding each element, and surrounding" \
+      " each equals sign, is ignored. For example:",
+    ["    --git=\"https://github.com/dazuma/example, path=toys, commit=main\""],
+    "",
+    "The recognized keys are:",
+    "* \"path\" : The file or directory within the repository to load. By default, the entire" \
+      " repository is loaded.",
+    "* \"commit\" : The SHA, tag, or branch to load. By default, the repository head is used.",
+    "* \"update\" : Whether to refresh a previously cached repository. Pass \"true\" or" \
+      " \"false\", or a number of seconds, to refresh only if the cache is at least that old." \
+      " The default is \"false\"."
+end
+
+flag :sources do
+  flags "--path=PATH"
+  handler { |val, prev| prev + [[:path, val]] }
+  default []
+  complete_values :file_system
+  desc "Make the tools from the given path available"
+  long_desc \
+    "Adds the tools from the given file system path. The path must name either a directory" \
+      " of tools, or a single Ruby file defining tools. For example:",
+    ["    --path=/path/to/my-tools"]
+end
+
+flag :on_missing_gem do
+  flags "--on-missing-gem=WHAT"
+  accept [:confirm, :install, :error]
+  complete_values ["confirm", "install", "error"]
+  desc "Specify what to do if a gem is not installed"
+  long_desc \
+    "Specifies what to do if a gem referenced by the --gem flag is not installed." \
+      " Possible values are:",
+    "* \"confirm\" : Prompt for what to do.",
+    "* \"install\" : Do not prompt, but just install the needed gem.",
+    "* \"error\" : Display an error and abort.",
+    "",
+    "Defaults to the existing CLI configuration, normally \"confirm\". Note this does not affect" \
+      " any other gem references such as via the \"gem\" or \"load_gem\" directives."
+end
+
 remaining_args :commands do
   complete do |context|
     commands = context.arg_parser.data[:commands]
@@ -37,10 +118,139 @@ end
 enforce_flags_before_args
 
 def run
+  tool_cli = build_cli
   commands
     .chunk { |arg| arg == delim ? :_separator : true }
     .each do |_, action|
-      code = cli.run(action)
+      code = tool_cli.run(action)
       exit(code) unless code.zero?
     end
 end
+
+# Returns the CLI used to run the requested tools. Normally this is simply the
+# current CLI, but if any sources were requested, we need a new CLI with those
+# sources added. Any new sources are added in reverse order so that the first
+# flag on the command line ends up with the highest priority.
+#
+# All the flag values are parsed up front, in command line order, so that a
+# malformed value is reported before any source is added, and is reported
+# against the first offending flag rather than the last. All added sources are
+# then proactively resolved and failures reported.
+def build_cli
+  specs = sources.map { |kind, value| parse_source_request(kind, value) }
+  return cli if specs.empty?
+  tool_cli = cli.child(copy_sources: true)
+  specs.reverse_each { |spec| tool_cli.add_source(spec, high_priority: true) }
+  begin
+    tool_cli.loader.resolve_sources
+  rescue ::Toys::ToolSourceError => e
+    logger.error("Unable to resolve the given tool sources: #{e.message}")
+    exit(1)
+  end
+  tool_cli
+end
+
+# Parses and checks a single source request, returning the source spec to add
+# to the CLI later.
+def parse_source_request(kind, value)
+  case kind
+  when :gem
+    parse_gem_request(value)
+  when :git
+    parse_git_request(value)
+  when :path
+    parse_path_request(value)
+  end
+end
+
+# Splits a --gem flag value into the gem name and its version requirements,
+# and checks that both are valid. The requirements are validated by Rubygems
+# itself, but are passed along as the original strings.
+def parse_gem_request(gem_request)
+  gem_name, *gem_version = gem_request.split(",", -1).map(&:strip)
+  if gem_name.nil? || gem_name.empty? || gem_version.any?(&:empty?)
+    logger.fatal("Invalid --gem value: #{gem_request.inspect}")
+    exit(1)
+  end
+  begin
+    ::Gem::Requirement.create(*gem_version)
+  rescue ::Gem::Requirement::BadRequirementError => e
+    logger.fatal("Invalid version requirement for gem #{gem_name.inspect}: #{e.message}")
+    exit(1)
+  end
+  ::Toys::SourceSpec.gem(gem_name, version: gem_version, on_missing: on_missing_gem)
+end
+
+# Splits a --git flag value into the git remote and the options that follow it,
+# and checks that all of them are valid. Unrecognized and duplicate keys are
+# errors rather than being ignored or resolved as last-wins, because a mistyped
+# key would otherwise silently load something other than what was asked for.
+def parse_git_request(git_request)
+  git_remote, *elements = git_request.split(",", -1).map(&:strip)
+  git_error(git_request, "the git remote is required") if git_remote.nil? || git_remote.empty?
+  opts = { path: nil, commit: nil, update: false }
+  seen_keys = []
+  elements.each do |element|
+    key, value = parse_git_element(git_request, element, seen_keys)
+    case key
+    when "path"
+      opts[:path] = value
+    when "commit"
+      opts[:commit] = value
+    when "update"
+      opts[:update] = parse_git_update(git_request, value)
+    else
+      git_error(git_request, "unrecognized key #{key.inspect}")
+    end
+  end
+  ::Toys::SourceSpec.git(git_remote, **opts)
+end
+
+# Splits one element following the git remote into its key and value, and
+# checks that it is well-formed and does not repeat an earlier key.
+def parse_git_element(git_request, element, seen_keys)
+  key, value = element.split("=", 2).map(&:strip)
+  if value.nil? || key.empty?
+    git_error(git_request, "expected \"key=value\" but got #{element.inspect}")
+  end
+  git_error(git_request, "empty value for key #{key.inspect}") if value.empty?
+  git_error(git_request, "duplicate key #{key.inspect}") if seen_keys.include?(key)
+  seen_keys << key
+  [key, value]
+end
+
+# Interprets the value of the "update" key in a --git flag value, which is
+# either a boolean or a number of seconds.
+def parse_git_update(git_request, value)
+  case value
+  when "true"
+    true
+  when "false"
+    false
+  when /\A\d+\z/
+    value.to_i
+  else
+    git_error(git_request, "invalid update value #{value.inspect}")
+  end
+end
+
+# Reports a malformed --git flag value, naming the part of the value that was
+# not understood.
+def git_error(git_request, message)
+  logger.fatal("Invalid --git value: #{git_request.inspect}: #{message}")
+  exit(1)
+end
+
+# Checks that a --path flag value is present. Whether the path actually names
+# tools is left to the loader to determine, so that this tool does not have to
+# duplicate that rule. The spec carries no context directory, like the gem and
+# git sources, because it is injected from the command line rather than found
+# in a project.
+def parse_path_request(path_request)
+  path = path_request.strip
+  if path.empty?
+    logger.fatal("Invalid --path value: #{path_request.inspect}")
+    exit(1)
+  end
+  ::Toys::SourceSpec.path(path)
+end

data/builtins/system/bash-completion.rb

@@ -30,7 +30,7 @@ tool "eval" do
 
   def run
     require "toys/utils/completion_engine"
-    result = ::Toys::Utils::CompletionEngine::Bash.new(cli).run
+    result = ::Toys::Utils::CompletionEngine::Bash.new(cli.completion, cli.loader).run
     if result > 1
       logger.fatal("This tool must be invoked as a bash completion command.")
     end

data/builtins/system/test.rb

@@ -208,7 +208,7 @@ end
 
 def tool_dir
   @tool_dir ||= begin
-    words = cli.loader.split_path(tool)
+    words = cli.tool_name_splitter.split(tool)
     dir = base_dir
     unless words.empty?
       dir = ::File.join(dir, *words)
@@ -232,7 +232,7 @@ def base_dir
     else
       dir = ::File.realpath(::Dir.getwd)
       loop do
-        candidate = ::File.join(dir, ::Toys::StandardCLI::CONFIG_DIR_NAME)
+        candidate = ::File.join(dir, ::Toys::StandardCLI::TOPLEVEL_TOOL_DIR_NAME)
         break candidate if ::File.directory?(candidate)
         parent = ::File.dirname(dir)
         if parent == dir

data/builtins/system/tools.rb

@@ -73,7 +73,7 @@ tool "list" do
   def run
     loader = choose_loader(local, from_dir)
     words = namespace
-    words = loader.split_path(words.first) if words.size == 1
+    words = loader.tool_name_splitter.split(words.first) if words.size == 1
     tool_list = loader.list_subtools(words,
                                      recursive: recursive,
                                      include_hidden: show_all,
@@ -128,7 +128,7 @@ tool "show" do
   def run
     loader = choose_loader(local, from_dir)
     words = name
-    words = loader.split_path(words.first) if words.size == 1
+    words = loader.tool_name_splitter.split(words.first) if words.size == 1
     tool = loader.lookup_specific(words)
     output =
       if tool.nil?

data/builtins/system/zsh-completion.rb

@@ -32,7 +32,7 @@ tool "eval" do
 
   def run
     require "toys/utils/completion_engine"
-    result = ::Toys::Utils::CompletionEngine::Zsh.new(cli).run
+    result = ::Toys::Utils::CompletionEngine::Zsh.new(cli.completion, cli.loader).run
     if result > 1
       logger.fatal("This tool must be invoked as a zsh completion command.")
     end

data/core-docs/toys/acceptor.rb

@@ -1,7 +1,5 @@
 module Toys
   ##
-  # **_Defined in the toys-core gem_**
-  #
   # An Acceptor validates and converts arguments. It is designed to be
   # compatible with the OptionParser accept mechanism.
   #
@@ -16,13 +14,15 @@ module Toys
   # Finally, an acceptor has a name that may appear in help text for flags and
   # arguments that use it.
   #
+  # **_Defined in the toys-core gem_**
+  #
   module Acceptor
     ##
     # A sentinel that may be returned from a function-based acceptor to
     # indicate invalid input.
     # @return [Object]
     #
-    REJECT = ::Object.new.freeze
+    REJECT = ::Toys::UniqueKey.new("Toys::Acceptor::REJECT")
 
     ##
     # The default type description.
@@ -31,8 +31,6 @@ module Toys
     DEFAULT_TYPE_DESC = "string"
 
     ##
-    # **_Defined in the toys-core gem_**
-    #
     # A base class for acceptors.
     #
     # The base acceptor does not do any validation (i.e. it accepts all
@@ -40,6 +38,8 @@ module Toys
     # subclass this base class and override the {#match} and {#convert} methods
     # to implement an acceptor.
     #
+    # **_Defined in the toys-core gem_**
+    #
     class Base
       ##
       # Create a base acceptor.
@@ -146,13 +146,13 @@ module Toys
     DEFAULT = Base.new(type_desc: "string", well_known_spec: ::Object)
 
     ##
-    # **_Defined in the toys-core gem_**
-    #
     # An acceptor that uses a simple function to validate and convert input.
     # The function must take the input string as its argument, and either
     # return the converted object to indicate success, or raise an exception or
     # return the sentinel {Toys::Acceptor::REJECT} to indicate invalid input.
     #
+    # **_Defined in the toys-core gem_**
+    #
     class Simple < Base
       ##
       # Create a simple acceptor.
@@ -178,12 +178,12 @@ module Toys
     end
 
     ##
-    # **_Defined in the toys-core gem_**
-    #
     # An acceptor that uses a regex to validate input. It also supports a
     # custom conversion function that generates the final value from the match
     # results.
     #
+    # **_Defined in the toys-core gem_**
+    #
     class Pattern < Base
       ##
       # Create a pattern acceptor.
@@ -216,8 +216,6 @@ module Toys
     end
 
     ##
-    # **_Defined in the toys-core gem_**
-    #
     # An acceptor that recognizes a fixed set of values.
     #
     # You provide a list of valid values. The input argument string will be
@@ -230,6 +228,8 @@ module Toys
     # converter will yield the integer `3`. If an argument of "three" is
     # passed in, the match will fail.
     #
+    # **_Defined in the toys-core gem_**
+    #
     class Enum < Base
       ##
       # Create an acceptor.
@@ -254,8 +254,6 @@ module Toys
     end
 
     ##
-    # **_Defined in the toys-core gem_**
-    #
     # An acceptor that recognizes a range of values.
     #
     # The input argument is matched against the given range. For example, you
@@ -281,6 +279,8 @@ module Toys
     #  *  If the range beginning and/or end are not numeric types, then no
     #     conversion is done by default.
     #
+    # **_Defined in the toys-core gem_**
+    #
     class Range < Simple
       ##
       # Create an acceptor.