toys 0.12.2 → 0.13.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 68fdda1df8ea614918dd8fadaf1e5766693b036cc06c737b4327d48e769d9b5b
-  data.tar.gz: 73acd0ec7f1ae55e8ad18625246b0af9632a3e24423f669b0c06e03507f5e8fc
+  metadata.gz: db87e9639c06ca1247cc72180b8cab752962b86535089aebc8d97daa31b3a804
+  data.tar.gz: 6ca990bbc0010806cf57ab6ca2fd1cde84790bfa647ff6eb948c9b19f66b408b
 SHA512:
-  metadata.gz: 9a97a0179a67317b43ff07e922aa81846375f4db1f5df1c9040c850e41a39d8521049369a4545e6e000ed7adda613c7b80c91d92bc3e30a46888bcac49f05fd5
-  data.tar.gz: 7118fd3217f088ea41a8a7ea9bd063dece182cce2db5d0ae5d6e70850fe423ba757156a6f010a46ed0077ea5c5ab4748e0b222be1067c24be0f4e5ff22093e12
+  metadata.gz: 76c394b761f29c3d8891a2bdf527275bbf217c27bbc1e774eaef21fa79c8aa5b7327022f585c60fa4eae08af46216b3c466d52595662ee7f1a40465f0882df63
+  data.tar.gz: '0944f37779985979d90f86be071e6b53f2917f0a005363ea338f20dc993e4900218e864b7e6753f1a7222fed5077c2317e6e2727039496adcb877c9182817259'

data/.yardopts

@@ -3,6 +3,8 @@
 --markup=markdown
 --markup-provider redcarpet
 --main=README.md
+./core-docs/toys/**/*.rb
+./core-docs/toys-core.rb
 ./lib/toys/**/*.rb
 ./lib/toys.rb
 -

data/CHANGELOG.md

@@ -1,5 +1,40 @@
 # Release History
 
+### v0.13.0 / 2022-02-08
+
+Toys 0.13.0 is a major release with significant improvements to the testing framework and git cache, along with compatibility improvements and bug fixes.
+
+Breaking changes:
+
+* Passing `--directory=` to `toys system test` now restricts the search to *only* that directory, excluding global and builtin tools.
+* Renamed `Toys::Testing#exec_tool` to `Toys::Testing#toys_exec_tool`.
+* Removed the methods of `Toys::Testing` that were specific to "capture", "control", or "separate" spawning (i.e. `#exec_separate_tool`, `#capture_tool`, `#capture_separate_tool`, `#control_tool`, and `#control_separate_tool`). The general `Toys::Testing#toys_exec_tool` now either controls or captures depending on whether a block is present. Spawn-based execution is no longer available.
+
+New functionality:
+
+* Provided several builtin system tools for viewing and managing the git cache.
+* The `load_git` directive and the underlying `Toys::Utils::GitCache` class now support updating from git based on cache age.
+* The `Toys::Utils::GitCache` class supports supports copying git content into a provided directory, querying repo information, and deleting cache data.
+* The `Toys::Utils::GitCache` class makes files read-only, to help prevent clients from interfering with one another.
+* The `:terminal` mixin and the underlying `Toys::Utils::Terminal` class now honor the `NO_COLOR` environment variable.
+* `toys system test` now understands `--minitest-focus` and `--minitest-rg` to activate those plugins.
+* `toys system test` now understands `--minitest-version=` to set the minitest version requirement.
+* Added `Toys::Testing#toys_run_tool` to run a tool in-process for testing.
+* Added `Toys::Testing#toys_load_tool` to load a tool in-process and provide a way to test individual methods.
+* Added `:custom_paths` and `:include_builtins` arguments to the `Toys::StandardCLI` constructor.
+
+Fixes and compatibility:
+
+* Bundler install/updates are now spawned in subprocesses for compatibility with bundler 2.3. The bundler integration also now requires bundler 2.2 or later.
+* The `exec_tool` and `exec_proc` methods in the `:exec` mixin now log their execution in the same way as other exec functions.
+* Minor compatibility fixes to provide partial support for TruffleRuby.
+
+Other notes:
+
+* The internal GitCache representation has changed significantly to support additional features and improve robustness and performance. This will force existing caches to update, but should not break existing usage.
+* Significant updates to the readme and user guide, including a section in the user guide on the test framework.
+* Toys-core docs are now embedded into the toys gem, so that all the links will work in rubydoc.info.
+
 ### v0.12.2 / 2021-08-30
 
 * FIXED: Tool context inspect string is no longer overwhelmingly long

data/LICENSE.md

@@ -1,6 +1,6 @@
 # License
 
-Copyright 2019-2020 Daniel Azuma and the Toys contributors
+Copyright 2019-2022 Daniel Azuma and the Toys contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -55,6 +55,9 @@ Toys requires Ruby 2.4 or later.
 Most parts of Toys work on JRuby. However, JRuby is not recommended because of
 JVM boot latency, lack of support for Kernel#fork, and other issues.
 
+Most parts of Toys work on TruffleRuby. However, TruffleRuby is not recommended
+because it has a few known bugs that affect Toys.
+
 ### Write your first tool
 
 You can define tools by creating a *Toys file*. Go into any directory, and,
@@ -134,7 +137,7 @@ tool to the end of the file.)
         exec "git init"
         File.write ".gitignore", <<~CONTENT
           tmp
-          .DS_STORE
+          .DS_Store
         CONTENT
         # You can add additional files here.
         exec "git add ."
@@ -279,8 +282,8 @@ and take advantage of a variety of third-party libraries such as Highline and
 TTY. Finally, if your `.toys.rb` files are growing too large or complicated,
 you can replace them with `.toys` directories that contain tool definitions in
 separate files. Such directories are versatile, letting you organize your tool
-definitions, along with shared code, normal Ruby classes, and even data files
-for use by tools.
+definitions, along with shared code, normal Ruby classes, tests, and even data
+files for use by tools.
 
 Unlike most command line frameworks, Toys is *not primarily* designed to help
 you build and ship a custom command line executable written in Ruby. However,
@@ -319,7 +322,7 @@ Toys scripts instead of Rakefiles.
 
 ## License
 
-Copyright 2019-2020 Daniel Azuma and the Toys contributors
+Copyright 2019-2022 Daniel Azuma and the Toys contributors
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/builtins/system/git-cache.rb

@@ -0,0 +1,238 @@
+# frozen_string_literal: true
+
+desc "Git-cache management tools"
+
+long_desc \
+  "Tools that manage the git cache.",
+  "",
+  "The Toys::Utils::GitCache class manages a cache of files from remote git repoistories." \
+    " It is used when loading tools from git, and can also be used directly by tools to access" \
+    " files from a remote git repository such as from GitHub.",
+  "",
+  "The tools under the `system git-cache` namespace manage the contents of the git-cache," \
+    " including querying the cache, getting information about cache status, and removing old data."
+
+tool "list" do
+  desc "Output a list of the git repositories in the cache."
+
+  long_desc \
+    "Outputs a list of the git remotes for the repositories in the cache, in YAML format, to" \
+      " the standard output stream."
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  def run
+    require "psych"
+    require "toys/utils/git_cache"
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    output = {
+      "cache_dir" => git_cache.cache_dir,
+      "remotes" => git_cache.remotes,
+    }
+    puts(::Psych.dump(output))
+  end
+end
+
+tool "show" do
+  desc "Output information about the specified git repo in the cache."
+
+  long_desc \
+    "Outputs information about the git repo specified by the given remote, in YAML format, to" \
+      " the standard output stream."
+
+  required_arg :remote, desc: "The git remote identifying the repo. Required."
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  def run
+    require "psych"
+    require "toys/utils/git_cache"
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    info = git_cache.repo_info(remote)
+    if info.nil?
+      logger.fatal("Unknown remote: #{remote}")
+      exit(1)
+    end
+    puts(::Psych.dump(info.to_h))
+  end
+end
+
+tool "get" do
+  desc "Get files from the git cache, loading from the repo if necessary."
+
+  long_desc \
+    "Get files from the git cache, loading from the repo if necessary, and output the path to" \
+      " the files to the standard output stream.",
+    "",
+    "The resulting files are either served from a shared source directory, or copied into aa" \
+      " directory specified by the `--into=` flag. If you use the shared directory, do not" \
+      " modify the files or directory structure, or other callers will see your modifications."
+
+  required_arg :remote, desc: "The git remote identifying the repo. Required."
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  flag :path, "--path=PATH" do
+    desc "A path to a specific file or directory in the repository. Optional. Defaults to the" \
+         " entire repository."
+  end
+
+  flag :commit, "--commit=REF", "--ref=REF" do
+    desc "The commit, which may be a SHA, branch, tag, or HEAD (the default). Optional."
+  end
+
+  flag :update do
+    desc "Update refs, such as branches, from the remote repo."
+  end
+
+  flag :into, "--into=DIR" do
+    desc "Copy files into the given directory rather than returning a shared directory."
+  end
+
+  def run
+    require "toys/utils/git_cache"
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    out = git_cache.get(remote, path: path, commit: commit, into: into, update: update)
+    puts(out)
+  rescue ::Toys::Utils::GitCache::Error => e
+    logger.fatal(e.message)
+    exit(1)
+  end
+end
+
+tool "remove" do
+  desc "Remove the given repositories from the cache."
+
+  long_desc \
+    "Remove the given repositories, including local clones and all shared source directories," \
+      " from the cache. The next time any of these repositories is requested, it will be" \
+      " reloaded from the remote repository from scratch.",
+    "",
+    "You can remove specific repositories by providing their remotes as arguments, or remove all" \
+      " repositories from the cache by specifying the `--all` flag.",
+    "",
+    "Be careful not to clear repos that are currently in use by other processes. This command" \
+      " may delete files that are in use by other git-cache clients."
+
+  remaining_args :remotes, desc: "The git remote(s) identifying the repo(s) to remove."
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  flag :all do
+    desc "Remove all repositories. Required unless specific remotes are provided."
+  end
+
+  def run
+    require "psych"
+    require "toys/utils/git_cache"
+    if remotes.empty? == !all
+      logger.fatal("You must specify at least one remote to clear, or --all to clear all remotes.")
+      exit(2)
+    end
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    removed = git_cache.remove_repos(all ? :all : remotes)
+    output = {
+      "removed" => removed,
+    }
+    puts(::Psych.dump(output))
+  end
+end
+
+tool "remove-refs" do
+  desc "Removes records of the given refs from the cache."
+
+  long_desc \
+    "Removes records of the given refs (i.e. branches, tags, or HEAD) from the cache for the" \
+      " given repo. The next time any of these refs are requested, they will be pulled fresh" \
+      " from the remote repo.",
+    "",
+    "You must provide either the `--all` flag to remove all refs, or at least one `--ref=` flag" \
+      " to remove specific refs.",
+    "",
+    "Outputs a list of the refs actually removed, in YAML format, to the standard output stream."
+
+  required_arg :remote, desc: "The git remote identifying the repo. Required."
+
+  exactly_one_required do
+    flag :refs, "--ref=REF", handler: :push do
+      desc "Remove a specific ref."
+    end
+
+    flag :all do
+      desc "Remove all refs."
+    end
+  end
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  def run
+    require "psych"
+    require "toys/utils/git_cache"
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    removed = git_cache.remove_refs(remote, refs: refs)
+    if removed.nil?
+      logger.fatal("Unknown remote: #{remote}")
+      exit(1)
+    end
+    output = {
+      "remote" => remote,
+      "removed_refs" => removed.map(&:to_h),
+    }
+    puts(::Psych.dump(output))
+  end
+end
+
+tool "remove-sources" do
+  desc "Removes shared sources from the cache."
+
+  long_desc \
+    "Removes the specified shared sources from the cache for the given repository. The next time" \
+      " these files are retrieved, they will be recopied from the repository.",
+    "",
+    "You must provide either the `--all` flag to remove all sources associated with the repo," \
+      " or at least one `--commit=` flag to remove sources for specific commits.",
+    "",
+    "Outputs a list of the sources actually removed, in YAML format, to the standard output stream."
+
+  required_arg :remote, desc: "The git remote identifying the repo. Required."
+
+  exactly_one_required do
+    flag :commits, "--commit=REF", handler: :push do
+      desc "Remove sources associated with a specific commit."
+    end
+
+    flag :all do
+      desc "Remove all sources."
+    end
+  end
+
+  flag :cache_dir, "--cache-dir=PATH" do
+    desc "The base directory for the cache. Optional. Defaults to the standard cache directory."
+  end
+
+  def run
+    require "psych"
+    require "toys/utils/git_cache"
+    git_cache = ::Toys::Utils::GitCache.new(cache_dir: cache_dir)
+    removed = git_cache.remove_sources(remote, commits: commits)
+    if removed.nil?
+      logger.fatal("Unknown remote: #{remote}")
+      exit(1)
+    end
+    output = {
+      "remote" => remote,
+      "removed_sources" => removed.map(&:to_h),
+    }
+    puts(::Psych.dump(output))
+  end
+end

data/builtins/system/test.rb

@@ -17,20 +17,40 @@ flag :recursive, "--[no-]recursive", default: true,
      desc: "Recursively test subtools (default is true)"
 flag :tool, "-t TOOL", "--tool TOOL", default: "",
      desc: "Run tests only for tools under the given path"
+flag :minitest_version, "--minitest-version=VERSION", default: "~> 5.0",
+     desc: "Set the minitest version requirement (default is ~>5.0)"
+flag :minitest_focus, "--minitest-focus[=VERSION]",
+     desc: "Make minitest-focus available during the run"
+flag :minitest_rg, "--minitest-rg[=VERSION]",
+     desc: "Make minitest-rg available during the run"
 
 include :exec
 include :gems
 include :terminal
 
 def run
-  gem "minitest", "~> 5.0"
+  load_minitest_gems
   ::Dir.chdir(tool_dir)
   test_files = find_test_files
   result = exec_ruby(ruby_args, in: :controller, log_cmd: "Starting minitest...") do |controller|
-    controller.in.puts("gem 'minitest', '~> 5.0'")
+    controller.in.puts("gem 'minitest', '= #{::Minitest::VERSION}'")
     controller.in.puts("require 'minitest/autorun'")
+    if minitest_focus
+      controller.in.puts("gem 'minitest-focus', '= #{::Minitest::Test::Focus::VERSION}'")
+      controller.in.puts("require 'minitest/focus'")
+    end
+    if minitest_rg
+      controller.in.puts("gem 'minitest-rg', '= #{::MiniTest::RG::VERSION}'")
+      controller.in.puts("require 'minitest/rg'")
+    end
+    controller.in.puts("gem 'toys', '= #{::Toys::VERSION}'")
     controller.in.puts("require 'toys'")
     controller.in.puts("require 'toys/testing'")
+    if directory
+      dir_str = ::File.absolute_path(directory).inspect
+      controller.in.puts("Toys::Testing.toys_custom_paths(#{dir_str})")
+      controller.in.puts("Toys::Testing.toys_include_builtins(false)")
+    end
     test_files.each do |file|
       controller.in.puts("load '#{file}'")
     end
@@ -41,6 +61,21 @@ def run
   end
 end
 
+def load_minitest_gems
+  gem "minitest", minitest_version
+  require "minitest"
+  if minitest_focus
+    minitest_focus = "~> 1.0" if minitest_focus == true
+    gem "minitest-focus", minitest_focus
+    require "minitest/focus"
+  end
+  if minitest_rg
+    minitest_rg = "~> 5.0" if minitest_rg == true
+    gem "minitest-rg", minitest_rg
+    require "minitest/rg"
+  end
+end
+
 def find_test_files
   glob = ".test/**/test_*.rb"
   glob = "**/#{glob}" if recursive