toys 0.12.1 → 0.13.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 26f6e4aff09af80cf73363b7721e59aa06063390d6c598935e46857c8e3b56fd
-  data.tar.gz: 775ea27e5a94f94a4b16750b15b3e5c684fd7a077b6a60d89704319df544ce38
+  metadata.gz: 76a2e6d85d0d1baa0dc154bc6572bbd600c855c9e3cf6c6e64594630a33c58ed
+  data.tar.gz: c701ff4fd53eb2a7dd85561b4c19e1614a43a9788f73abdbeaa86e8a9a17fb68
 SHA512:
-  metadata.gz: 2560dedb8daf223ec208ba2b3e5087b7ccc25f6eaa3361d568238bcd0cb324af6d6add06016f6119353fc60ce8905d90ae542d7433ab0ce7088a56a320ae4acc
-  data.tar.gz: de3028932e124f34c7f4cfb1d8e255dc45d61527083bbfa17d91cff16efe3e6e0e401a18020fe13c4439480a36062cc72e45566662a96d2603742c9da01f83fd
+  metadata.gz: da3b8b9da43c2bc81e286206fa649fc994b5083376357cf3c4f7d3289b8d8d928da8faffa42a32e4313074a24666ec40a1eabc73d8b1c82304bc6cd1eed99feb
+  data.tar.gz: 8f36a970ed0292a604dcb28cb5a48ad4697fc0a5fbb27389f6dc937e9ea543488c2b28bfc1c148f93044eb26aed4f03c8911f9df921fafb54f352020144f041b

data/.yardopts

@@ -3,6 +3,8 @@
 --markup=markdown
 --markup-provider redcarpet
 --main=README.md
+./core-docs/toys/**/*.rb
+./core-docs/toys-core.rb
 ./lib/toys/**/*.rb
 ./lib/toys.rb
 -

data/CHANGELOG.md

@@ -1,5 +1,49 @@
 # Release History
 
+### v0.13.1 / 2022-03-01
+
+* FIXED: Bundler integration no longer fails if a bundle was locked to a different version of a builtin gem
+
+### v0.13.0 / 2022-02-08
+
+Toys 0.13.0 is a major release with significant improvements to the testing framework and git cache, along with compatibility improvements and bug fixes.
+
+Breaking changes:
+
+* Passing `--directory=` to `toys system test` now restricts the search to *only* that directory, excluding global and builtin tools.
+* Renamed `Toys::Testing#exec_tool` to `Toys::Testing#toys_exec_tool`.
+* Removed the methods of `Toys::Testing` that were specific to "capture", "control", or "separate" spawning (i.e. `#exec_separate_tool`, `#capture_tool`, `#capture_separate_tool`, `#control_tool`, and `#control_separate_tool`). The general `Toys::Testing#toys_exec_tool` now either controls or captures depending on whether a block is present. Spawn-based execution is no longer available.
+
+New functionality:
+
+* Provided several builtin system tools for viewing and managing the git cache.
+* The `load_git` directive and the underlying `Toys::Utils::GitCache` class now support updating from git based on cache age.
+* The `Toys::Utils::GitCache` class supports supports copying git content into a provided directory, querying repo information, and deleting cache data.
+* The `Toys::Utils::GitCache` class makes files read-only, to help prevent clients from interfering with one another.
+* The `:terminal` mixin and the underlying `Toys::Utils::Terminal` class now honor the `NO_COLOR` environment variable.
+* `toys system test` now understands `--minitest-focus` and `--minitest-rg` to activate those plugins.
+* `toys system test` now understands `--minitest-version=` to set the minitest version requirement.
+* Added `Toys::Testing#toys_run_tool` to run a tool in-process for testing.
+* Added `Toys::Testing#toys_load_tool` to load a tool in-process and provide a way to test individual methods.
+* Added `:custom_paths` and `:include_builtins` arguments to the `Toys::StandardCLI` constructor.
+
+Fixes and compatibility:
+
+* Bundler install/updates are now spawned in subprocesses for compatibility with bundler 2.3. The bundler integration also now requires bundler 2.2 or later.
+* The `exec_tool` and `exec_proc` methods in the `:exec` mixin now log their execution in the same way as other exec functions.
+* Minor compatibility fixes to provide partial support for TruffleRuby.
+
+Other notes:
+
+* The internal GitCache representation has changed significantly to support additional features and improve robustness and performance. This will force existing caches to update, but should not break existing usage.
+* Significant updates to the readme and user guide, including a section in the user guide on the test framework.
+* Toys-core docs are now embedded into the toys gem, so that all the links will work in rubydoc.info.
+
+### v0.12.2 / 2021-08-30
+
+* FIXED: Tool context inspect string is no longer overwhelmingly long
+* FIXED: Fixed an exception in GitCache and load_git when updating a changed ref
+
 ### v0.12.1 / 2021-08-17
 
 * FIXED: Fixed a regression in 0.12.0 where bundler could use the wrong Gemfile if you set a custom context directory

data/LICENSE.md

@@ -1,6 +1,6 @@
 # License
 
-Copyright 2019-2020 Daniel Azuma and the Toys contributors
+Copyright 2019-2022 Daniel Azuma and the Toys contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -55,6 +55,9 @@ Toys requires Ruby 2.4 or later.
 Most parts of Toys work on JRuby. However, JRuby is not recommended because of
 JVM boot latency, lack of support for Kernel#fork, and other issues.
 
+Most parts of Toys work on TruffleRuby. However, TruffleRuby is not recommended
+because it has a few known bugs that affect Toys.
+
 ### Write your first tool
 
 You can define tools by creating a *Toys file*. Go into any directory, and,
@@ -134,7 +137,7 @@ tool to the end of the file.)
         exec "git init"
         File.write ".gitignore", <<~CONTENT
           tmp
-          .DS_STORE
+          .DS_Store
         CONTENT
         # You can add additional files here.
         exec "git add ."
@@ -279,8 +282,8 @@ and take advantage of a variety of third-party libraries such as Highline and
 TTY. Finally, if your `.toys.rb` files are growing too large or complicated,
 you can replace them with `.toys` directories that contain tool definitions in
 separate files. Such directories are versatile, letting you organize your tool
-definitions, along with shared code, normal Ruby classes, and even data files
-for use by tools.
+definitions, along with shared code, normal Ruby classes, tests, and even data
+files for use by tools.
 
 Unlike most command line frameworks, Toys is *not primarily* designed to help
 you build and ship a custom command line executable written in Ruby. However,
@@ -319,7 +322,7 @@ Toys scripts instead of Rakefiles.
 
 ## License
 
-Copyright 2019-2020 Daniel Azuma and the Toys contributors
+Copyright 2019-2022 Daniel Azuma and the Toys contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/builtins/system/git-cache.rb

@@ -0,0 +1,238 @@
+# frozen_string_literal: true
+
+desc "Git-cache management tools"
+
+long_desc \
+  "Tools that manage the git cache.",
+  "",
+  "The Toys::Utils::GitCache class manages a cache of files from remote git repoistories." \
+    " It is used when loading tools from git, and can also be used directly by tools to access" \
+    " files from a remote git repository such as from GitHub.",
+  "",
+  "The tools under the `system git-cache` namespace manage the contents of the git-cache," \
+    " including querying the cache, getting information about cache status, and removing old data."
+
+tool "list" do
+  desc "Output a list of the git repositories in the cache."
+
+  long_desc \
+    "Outputs a list of the git remotes for the repositories in the cache, in YAML format, to" \
+      " the standard output stream."
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  def run
+    require "psych"
+    require "toys/utils/git_cache"
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    output = {
+      "cache_dir" => git_cache.cache_dir,
+      "remotes" => git_cache.remotes,
+    }
+    puts(::Psych.dump(output))
+  end
+end
+
+tool "show" do
+  desc "Output information about the specified git repo in the cache."
+
+  long_desc \
+    "Outputs information about the git repo specified by the given remote, in YAML format, to" \
+      " the standard output stream."
+
+  required_arg :remote, desc: "The git remote identifying the repo. Required."
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  def run
+    require "psych"
+    require "toys/utils/git_cache"
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    info = git_cache.repo_info(remote)
+    if info.nil?
+      logger.fatal("Unknown remote: #{remote}")
+      exit(1)
+    end
+    puts(::Psych.dump(info.to_h))
+  end
+end
+
+tool "get" do
+  desc "Get files from the git cache, loading from the repo if necessary."
+
+  long_desc \
+    "Get files from the git cache, loading from the repo if necessary, and output the path to" \
+      " the files to the standard output stream.",
+    "",
+    "The resulting files are either served from a shared source directory, or copied into aa" \
+      " directory specified by the `--into=` flag. If you use the shared directory, do not" \
+      " modify the files or directory structure, or other callers will see your modifications."
+
+  required_arg :remote, desc: "The git remote identifying the repo. Required."
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  flag :path, "--path=PATH" do
+    desc "A path to a specific file or directory in the repository. Optional. Defaults to the" \
+         " entire repository."
+  end
+
+  flag :commit, "--commit=REF", "--ref=REF" do
+    desc "The commit, which may be a SHA, branch, tag, or HEAD (the default). Optional."
+  end
+
+  flag :update do
+    desc "Update refs, such as branches, from the remote repo."
+  end
+
+  flag :into, "--into=DIR" do
+    desc "Copy files into the given directory rather than returning a shared directory."
+  end
+
+  def run
+    require "toys/utils/git_cache"
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    out = git_cache.get(remote, path: path, commit: commit, into: into, update: update)
+    puts(out)
+  rescue ::Toys::Utils::GitCache::Error => e
+    logger.fatal(e.message)
+    exit(1)
+  end
+end
+
+tool "remove" do
+  desc "Remove the given repositories from the cache."
+
+  long_desc \
+    "Remove the given repositories, including local clones and all shared source directories," \
+      " from the cache. The next time any of these repositories is requested, it will be" \
+      " reloaded from the remote repository from scratch.",
+    "",
+    "You can remove specific repositories by providing their remotes as arguments, or remove all" \
+      " repositories from the cache by specifying the `--all` flag.",
+    "",
+    "Be careful not to clear repos that are currently in use by other processes. This command" \
+      " may delete files that are in use by other git-cache clients."
+
+  remaining_args :remotes, desc: "The git remote(s) identifying the repo(s) to remove."
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  flag :all do
+    desc "Remove all repositories. Required unless specific remotes are provided."
+  end
+
+  def run
+    require "psych"
+    require "toys/utils/git_cache"
+    if remotes.empty? == !all
+      logger.fatal("You must specify at least one remote to clear, or --all to clear all remotes.")
+      exit(2)
+    end
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    removed = git_cache.remove_repos(all ? :all : remotes)
+    output = {
+      "removed" => removed,
+    }
+    puts(::Psych.dump(output))
+  end
+end
+
+tool "remove-refs" do
+  desc "Removes records of the given refs from the cache."
+
+  long_desc \
+    "Removes records of the given refs (i.e. branches, tags, or HEAD) from the cache for the" \
+      " given repo. The next time any of these refs are requested, they will be pulled fresh" \
+      " from the remote repo.",
+    "",
+    "You must provide either the `--all` flag to remove all refs, or at least one `--ref=` flag" \
+      " to remove specific refs.",
+    "",
+    "Outputs a list of the refs actually removed, in YAML format, to the standard output stream."
+
+  required_arg :remote, desc: "The git remote identifying the repo. Required."
+
+  exactly_one_required do
+    flag :refs, "--ref=REF", handler: :push do
+      desc "Remove a specific ref."
+    end
+
+    flag :all do
+      desc "Remove all refs."
+    end
+  end
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  def run
+    require "psych"
+    require "toys/utils/git_cache"
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    removed = git_cache.remove_refs(remote, refs: refs)
+    if removed.nil?
+      logger.fatal("Unknown remote: #{remote}")
+      exit(1)
+    end
+    output = {
+      "remote" => remote,
+      "removed_refs" => removed.map(&:to_h),
+    }
+    puts(::Psych.dump(output))
+  end
+end
+
+tool "remove-sources" do
+  desc "Removes shared sources from the cache."
+
+  long_desc \
+    "Removes the specified shared sources from the cache for the given repository. The next time" \
+      " these files are retrieved, they will be recopied from the repository.",
+    "",
+    "You must provide either the `--all` flag to remove all sources associated with the repo," \
+      " or at least one `--commit=` flag to remove sources for specific commits.",
+    "",
+    "Outputs a list of the sources actually removed, in YAML format, to the standard output stream."
+
+  required_arg :remote, desc: "The git remote identifying the repo. Required."
+
+  exactly_one_required do
+    flag :commits, "--commit=REF", handler: :push do
+      desc "Remove sources associated with a specific commit."
+    end
+
+    flag :all do
+      desc "Remove all sources."
+    end
+  end
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  def run
+    require "psych"
+    require "toys/utils/git_cache"
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    removed = git_cache.remove_sources(remote, commits: commits)
+    if removed.nil?
+      logger.fatal("Unknown remote: #{remote}")
+      exit(1)
+    end
+    output = {
+      "remote" => remote,
+      "removed_sources" => removed.map(&:to_h),
+    }
+    puts(::Psych.dump(output))
+  end
+end

data/builtins/system/test.rb

@@ -17,20 +17,40 @@ flag :recursive, "--[no-]recursive", default: true,
      desc: "Recursively test subtools (default is true)"
 flag :tool, "-t TOOL", "--tool TOOL", default: "",
      desc: "Run tests only for tools under the given path"
+flag :minitest_version, "--minitest-version=VERSION", default: "~> 5.0",
+     desc: "Set the minitest version requirement (default is ~>5.0)"
+flag :minitest_focus, "--minitest-focus[=VERSION]",
+     desc: "Make minitest-focus available during the run"
+flag :minitest_rg, "--minitest-rg[=VERSION]",
+     desc: "Make minitest-rg available during the run"
 
 include :exec
 include :gems
 include :terminal
 
 def run
-  gem "minitest", "~> 5.0"
+  load_minitest_gems
   ::Dir.chdir(tool_dir)
   test_files = find_test_files
   result = exec_ruby(ruby_args, in: :controller, log_cmd: "Starting minitest...") do |controller|
-    controller.in.puts("gem 'minitest', '~> 5.0'")
+    controller.in.puts("gem 'minitest', '= #{::Minitest::VERSION}'")
     controller.in.puts("require 'minitest/autorun'")
+    if minitest_focus
+      controller.in.puts("gem 'minitest-focus', '= #{::Minitest::Test::Focus::VERSION}'")
+      controller.in.puts("require 'minitest/focus'")
+    end
+    if minitest_rg
+      controller.in.puts("gem 'minitest-rg', '= #{::MiniTest::RG::VERSION}'")
+      controller.in.puts("require 'minitest/rg'")
+    end
+    controller.in.puts("gem 'toys', '= #{::Toys::VERSION}'")
     controller.in.puts("require 'toys'")
     controller.in.puts("require 'toys/testing'")
+    if directory
+      dir_str = ::File.absolute_path(directory).inspect
+      controller.in.puts("Toys::Testing.toys_custom_paths(#{dir_str})")
+      controller.in.puts("Toys::Testing.toys_include_builtins(false)")
+    end
     test_files.each do |file|
       controller.in.puts("load '#{file}'")
     end
@@ -41,6 +61,21 @@ def run
   end
 end
 
+def load_minitest_gems
+  gem "minitest", minitest_version
+  require "minitest"
+  if minitest_focus
+    minitest_focus = "~> 1.0" if minitest_focus == true
+    gem "minitest-focus", minitest_focus
+    require "minitest/focus"
+  end
+  if minitest_rg
+    minitest_rg = "~> 5.0" if minitest_rg == true
+    gem "minitest-rg", minitest_rg
+    require "minitest/rg"
+  end
+end
+
 def find_test_files
   glob = ".test/**/test_*.rb"
   glob = "**/#{glob}" if recursive