spinel_test-unit 0.1.2 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a36d406fd907591eb344d09238539b48c5c284a629847fca832dd696114b4cd4
-  data.tar.gz: d982c5bafe885e787e93f60b4434b75f295f54df415bb73aa35e25123a5a6150
+  metadata.gz: ef62e948140f04c676b0709a5780239556c38dd521fbdd0fce4968f82b643fe4
+  data.tar.gz: 3212cc091c366bc33b3fc1854f3fbd6a4a164248baa20cdc3816d3896dcd8519
 SHA512:
-  metadata.gz: 6f17eda9a7f16b435bb057fc6718ae3930ba5dd2c492fbafa60c840e7918d09db300851b57917d8d027a1d53b8b06f807b53ecd9bf595a82981494e3b1521739
-  data.tar.gz: c9b3ff1bea0017c169a937121008e2d7527da44b852bde7ca284eeb14d85114bf84d42a6f5262cfc5e62138f8db7f52607d9b7177b0e79378be17451e807af01
+  metadata.gz: 9dcc12b871aa16d10cea18c038de5d335a4e07121098f3de6b3449be0020e82274d39593698b79a39457dd9049ccdb1fcc06c740458d536151dd592ba93b66e9
+  data.tar.gz: 588963b23dc23425e40721efad1070800e577aec65d9dbba959aa537f3055ddb0a06db18df1671ae484733d5d7055f2f0da8e102213cbc2598897a618be0e43b

data/CHANGELOG.md

@@ -1,5 +1,19 @@
 ## Unreleased
 
+## 0.3.0
+
+* Add `match:` to `assert_raise` to check the exception message, as in `assert_match` (a Regexp, or a String meaning a substring)
+* `assert_raise` passes only for the given class itself, not a subclass, as in test-unit. Use the new `assert_raise_kind_of` to also accept a subclass
+* Add `-t`/`--testcase CLASS` to run only the tests of the class named exactly CLASS (`-n FooTest#` also selects `MyFooTest`)
+* `-n` and `-t` take a regexp written as `/regexp/`, as in test-unit
+
+## 0.2.0
+
+* Report where a test failed: `at file:line` lines after the message (the test method's line with Spinel, the filtered backtrace with `--ruby`)
+* Show a line diff when `assert_equal` fails on two multi-line Strings
+* Add `assert_match`, `assert_not_equal`, `assert_not_nil`, `assert_include`, `assert_not_include` and `refute`. `assert_includes` remains as an alias of `assert_include`
+* Add `--ruby` to run the tests with the current Ruby instead of compiling them with Spinel
+
 ## 0.1.2
 
 * Exclude mise.toml from the gem

data/README.md

@@ -39,26 +39,65 @@ end
 $ bundle exec spinel_test-unit                 # test/**/*_test.rb and test/**/test_*.rb
 $ bundle exec spinel_test-unit test/calc_test.rb
 $ bundle exec spinel_test-unit -n test_add     # only tests whose "Class#method" name includes it
+$ bundle exec spinel_test-unit -n '/#test_add\z/'  # or matches the regexp
+$ bundle exec spinel_test-unit -t CalcTest     # only the tests of the class named exactly CalcTest
+$ bundle exec spinel_test-unit -t '/\ACalc/'   # or of the classes whose name matches the regexp
+$ bundle exec spinel_test-unit --ruby          # run the same tests with the current Ruby, no Spinel needed
 ```
 
-Output:
+`--ruby` runs the generated main.rb with the Ruby that runs the CLI, with the same load paths (`-I`, `lib`, `test`, the project root) instead of compiling it; the child also inherits `RUBYOPT` and `RUBYLIB` (e.g. `bundler/setup`), so it may resolve requires that Spinel cannot. Use it to check that code which must work under both CRuby and Spinel behaves the same in each. The same tests run, with the same report format and exit status convention, but results can differ where CRuby and Spinel differ: Spinel freezes string literals (so `s = "a"; s << "b"` raises under Spinel but passes under `--ruby`), and exception messages can be worded differently.
+
+Output when compiled with Spinel:
 
 ```
-..F
+.F
 
 Failure: CalcTest#test_failure_message
 1 + 1 should be 2
+    at test/calc_test.rb:14
+
+2 tests, 2 assertions, 1 failures, 0 errors
+```
+
+Each failure or error is `Failure: Class#test_name` or `Error: Class#test_name`, the message, and up to three `    at file:line` lines (paths relative to the project root):
+
+- Compiled with Spinel, exceptions have no backtrace, so there is one line: the `file:line` of the test method's `def`.
+- With `--ruby`, the lines are the backtrace entries outside this gem and the generated main.rb, so they point at the failing assertion in the test and the callers above it (a test helper, for example). The first entry is always kept, even from outside the project root (an exception raised inside another library shows where); after it only entries under the root are listed. A location equal to the previous line is left out (Ruby 4.0 attributes block-taking builtins such as `Array#each` to their caller's line). If nothing is left after filtering, it falls back to the test method's line.
+- An error raised in `setup`, or in `teardown` when the test itself passed, is reported with the same lines (under Spinel, the test method's `def`).
+- If a test method is defined more than once, the location of the last definition is used.
+
+When `assert_equal` fails on two Strings that both contain a newline, a line diff follows the Expected/Actual lines. `-` lines are only in the expected string, `+` lines only in the actual one, and unchanged lines more than two away from a change are collapsed into `...`. A single hidden line is shown as is. Lines that could be misread (empty, containing `\r`, starting with `"`, equal to `...`, or starting or ending with a space or tab) are shown with `inspect` (`- ""`, `- "a\r"`). Example:
 
-3 tests, 3 assertions, 1 failures, 0 errors
 ```
+Failure: HttpTest#test_to_cgi
+Expected: "a\nb\nc\nd\ne\nf"
+  Actual: "a\nb\nX\nd\ne\nf"
+Diff:
+  a
+  b
+- c
++ X
+  d
+  e
+  f
+    at test/http_test.rb:7
+```
+
+Very large differences are not diffed: when the lines between the common start and end of the two strings would need a table of over a million cells, the diff is replaced by a one-line note and the Expected/Actual lines remain.
 
 Available assertions:
 
 - `assert(test, message = nil)`
 - `assert_equal(expected, actual, message = nil)`
+- `assert_not_equal(expected, actual, message = nil)`
 - `assert_nil(value, message = nil)`
-- `assert_includes(collection, item, message = nil)`
-- `assert_raise(klass) { ... }` (passes for `klass` or a subclass; returns the exception; assertion failures raised inside the block propagate)
+- `assert_not_nil(value, message = nil)`
+- `assert_match(pattern, string, message = nil)` (`pattern` is a Regexp, or a String meaning a substring, and anything else is reported as an error; a `string` that is not a String, such as `nil`, fails)
+- `assert_include(collection, item, message = nil)` (`assert_includes` is an alias)
+- `assert_not_include(collection, item, message = nil)`
+- `refute(test, message = nil)`
+- `assert_raise(klass, match: nil) { ... }` (passes for `klass` itself, not a subclass, as in test-unit; returns the exception; assertion failures raised inside the block propagate). With `match:`, the exception's message must also match it, as in `assert_match` (a Regexp, or a String meaning a substring)
+- `assert_raise_kind_of(klass, match: nil) { ... }` (as `assert_raise`, but also passes for a subclass of `klass`)
 
 `setup` and `teardown` hooks are supported.
 
@@ -83,6 +122,7 @@ The tests are compiled for build speed, not run speed: with `-O0`, and the gener
 - Files are loaded with `require` and the load paths `lib`, `test` and the
   project root; use `require "..."` / `require_relative` that Spinel resolves.
 - Test files under `fixtures/` directories (relative to the project root) are skipped unless named explicitly or the given directory is itself inside `fixtures/`.
+- A method parameter is typed from all of its call sites, so mixing a nil-typed local and, say, an Integer in calls to the same assertion can fail to compile or raise at run time under Spinel. For example, `refute 0` together with `n = nil; refute n` in the test files compiled together fails at run time ("nil can't be coerced into Integer") while `--ruby` passes. Keep the arguments of one assertion to one type across those files.
 - `ScriptError`s (such as `NotImplementedError`) raised by a test are reported as errors; a teardown error after a failure is appended to its message.
 
 ## Development

data/lib/spinel_test/unit/cli.rb

@@ -2,6 +2,7 @@
 
 require "fileutils"
 require "open3"
+require "rbconfig"
 require "tmpdir"
 require_relative "scanner"
 require_relative "generator"
@@ -9,10 +10,12 @@ require_relative "toolchain"
 
 module SpinelTest
   module Unit
-    # spinel_test-unit [PATH...] [-n NAME]
+    # spinel_test-unit [PATH...] [-n NAME] [-t CLASS] [-I DIR] [--ruby]
     #
     # Scans the test files with Prism, generates a main.rb that calls every
-    # test, compiles it with Spinel and runs the binary.
+    # test, compiles it with Spinel and runs the binary. With --ruby it runs the
+    # generated main.rb with the current Ruby instead, so one test file can be
+    # checked under both.
     class CLI
       def self.start(argv, root: Dir.pwd)
         new(argv, root: root).run
@@ -22,20 +25,27 @@ module SpinelTest
         @root = File.expand_path(root)
         @paths = []
         @filter = nil
-        @keep = false
+        @testcase = nil
+        @ruby = false
         @includes = []
         @exit = nil
         parse(argv.dup)
       end
 
       USAGE = <<~TEXT
-        Usage: spinel_test-unit [PATH...] [-n NAME] [-I DIR]
+        Usage: spinel_test-unit [PATH...] [-n NAME] [-t CLASS] [-I DIR] [--ruby]
 
-        Compiles the tests with Spinel and runs them.
+        Compiles the tests with Spinel and runs them (with --ruby, runs them with the
+        current Ruby instead).
 
           PATH             test file or directory (default: test)
-          -n, --name NAME  run only the tests whose "Class#test_name" includes NAME
-          -I DIR           add DIR to Spinel's load path (repeatable)
+          -n, --name NAME  run only the tests whose "Class#test_name" includes NAME,
+                           or matches it when NAME is /regexp/
+          -t, --testcase CLASS
+                           run only the tests of the class named CLASS (exact match),
+                           or whose class name matches it when CLASS is /regexp/
+          -I DIR           add DIR to the load path (repeatable)
+          --ruby           run with the current Ruby instead of compiling with Spinel
           -h, --help       show this help
       TEXT
 
@@ -48,10 +58,9 @@ module SpinelTest
           return 1
         end
 
-        spinel = Toolchain.spinel
-        unless spinel
-          warn "spinel_test-unit: Spinel not found (set SPINEL, put spinel on PATH, or install it with mise)"
-          return 1
+        unless @ruby
+          spinel = find_spinel
+          return 1 unless spinel
         end
 
         entries = build_entries(files)
@@ -62,17 +71,38 @@ module SpinelTest
 
         Dir.mktmpdir("spinel_test-unit") do |dir|
           main = File.join(dir, "main.rb")
-          bin = File.join(dir, "testbin")
-          File.write(main, Generator.generate(entries))
-          return 1 unless compile(spinel, main, bin, entries)
-
-          args = @filter ? ["-n", @filter] : []
-          system(bin, *args) ? 0 : ($?.exitstatus || 1)
+          File.write(main, Generator.generate(entries, @root))
+          @ruby ? run_with_ruby(main, entries) : run_with_spinel(spinel, main, dir, entries)
         end
       end
 
       private
 
+      def find_spinel
+        spinel = Toolchain.spinel
+        warn "spinel_test-unit: Spinel not found (set SPINEL, put spinel on PATH, or install it with mise)" if spinel.nil?
+        spinel
+      end
+
+      def run_with_spinel(spinel, main, dir, entries)
+        bin = File.join(dir, "testbin")
+        return 1 unless compile(spinel, main, bin, entries)
+
+        run_command([bin])
+      end
+
+      def run_with_ruby(main, entries)
+        load_path = include_dirs(entries).flat_map { |dir| ["-I", dir] }
+        run_command([RbConfig.ruby] + load_path + [main])
+      end
+
+      def run_command(command)
+        args = []
+        args += ["-n", @filter] if @filter
+        args += ["-t", @testcase] if @testcase
+        system(*command, *args) ? 0 : ($?.exitstatus || 1)
+      end
+
       def parse(args)
         until args.empty?
           arg = args.shift
@@ -82,16 +112,27 @@ module SpinelTest
             @exit = 0
             return
           when "-n", "--name"
-            if args.empty?
-              return usage_error("#{arg} requires a value")
-            end
+            return usage_error("#{arg} requires a value") if args.empty?
+
             @filter = args.shift
+            return unless valid_pattern?(@filter)
+          when "-t", "--testcase"
+            return usage_error("#{arg} requires a value") if args.empty?
+
+            @testcase = args.shift
+            return unless valid_pattern?(@testcase)
+          when "--ruby" then @ruby = true
           when "-I"
             return usage_error("-I requires a value") if args.empty?
 
             @includes << File.expand_path(args.shift, @root)
           when /\A-I(.+)\z/ then @includes << File.expand_path($1, @root)
-          when /\A--name=(.*)\z/ then @filter = $1
+          when /\A--name=(.*)\z/
+            @filter = $1
+            return unless valid_pattern?(@filter)
+          when /\A--testcase=(.*)\z/
+            @testcase = $1
+            return unless valid_pattern?(@testcase)
           when "--" then @paths.concat(args.shift(args.size))
           when /\A-/ then return usage_error("unknown option: #{arg}")
           else @paths << arg
@@ -99,6 +140,16 @@ module SpinelTest
         end
       end
 
+      def valid_pattern?(value)
+        return true unless value.length >= 2 && value.start_with?("/") && value.end_with?("/")
+
+        Regexp.new(value[1..-2])
+        true
+      rescue RegexpError => e
+        usage_error("invalid regexp #{value}: #{e.message}")
+        false
+      end
+
       def usage_error(message)
         warn "spinel_test-unit: #{message}"
         warn USAGE
@@ -134,6 +185,7 @@ module SpinelTest
           warn "spinel_test-unit: #{display(file)}: #{name} inherits from #{parent}, which is not defined in a scanned file; its tests are ignored"
         end
 
+        result.classes.each { |c| c.locations.transform_values! { |location| display(location) } }
         by_file = result.classes.group_by(&:file)
         order = []
         files.each do |file|
@@ -219,7 +271,13 @@ module SpinelTest
 
       def compile(spinel, main, bin, entries)
         cmd = [spinel] + Toolchain.compile_flags + include_dirs(entries).flat_map { |d| ["-I", d] } + [main, "-o", bin]
-        output, status = Open3.capture2e(*cmd)
+        begin
+          output, status = Open3.capture2e(*cmd)
+        rescue SystemCallError => e
+          hint = ENV["SPINEL"].to_s.empty? ? "" : " (check SPINEL)"
+          warn "spinel_test-unit: cannot run Spinel: #{e.message}#{hint}"
+          return false
+        end
         unless status.success?
           warn output unless output.empty?
           warn "spinel_test-unit: compilation failed (#{cmd.join(" ")})"

data/lib/spinel_test/unit/diff.rb

@@ -0,0 +1,159 @@
+# frozen_string_literal: true
+
+module SpinelTest
+  module Unit
+    class Diff
+      MAX_TABLE_CELLS = 1_000_000
+
+      def self.lines(expected, actual, context = 2)
+        new(expected.split("\n", -1), actual.split("\n", -1), context).render
+      end
+
+      def initialize(expected_lines, actual_lines, context)
+        @expected_lines = expected_lines
+        @actual_lines = actual_lines
+        @context = context
+        @prefix = common_prefix_length
+        @suffix = common_suffix_length
+        @expected_middle = middle(expected_lines)
+        @actual_middle = middle(actual_lines)
+      end
+
+      def render
+        if @expected_middle.length * @actual_middle.length > MAX_TABLE_CELLS
+          return "  (diff skipped: #{@expected_middle.length} expected and #{@actual_middle.length} actual lines differ)"
+        end
+
+        entries = unchanged(@expected_lines[0, @prefix]) + edits + unchanged(@expected_lines[@expected_lines.length - @suffix, @suffix])
+        collapse(entries)
+      end
+
+      private
+
+      def common_prefix_length
+        limit = @expected_lines.length < @actual_lines.length ? @expected_lines.length : @actual_lines.length
+        count = 0
+        count += 1 while count < limit && @expected_lines[count] == @actual_lines[count]
+        count
+      end
+
+      def common_suffix_length
+        limit = (@expected_lines.length < @actual_lines.length ? @expected_lines.length : @actual_lines.length) - @prefix
+        count = 0
+        while count < limit && @expected_lines[@expected_lines.length - 1 - count] == @actual_lines[@actual_lines.length - 1 - count]
+          count += 1
+        end
+        count
+      end
+
+      def middle(lines)
+        lines[@prefix, lines.length - @prefix - @suffix]
+      end
+
+      def unchanged(lines)
+        lines.map { |line| "  " + display(line) }
+      end
+
+      def display(line)
+        ambiguous?(line) ? line.inspect : line
+      end
+
+      def ambiguous?(line)
+        line == "" || line == "..." || control?(line) || line.start_with?("\"") ||
+          line.start_with?(" ", "\t") || line.end_with?(" ", "\t")
+      end
+
+      def control?(line)
+        line.bytes.any? { |b| (b < 32 && b != 9) || b == 127 }
+      end
+
+      def edits
+        lengths = lcs_lengths
+        entries = []
+        expected_index = 0
+        actual_index = 0
+        while expected_index < @expected_middle.length && actual_index < @actual_middle.length
+          if @expected_middle[expected_index] == @actual_middle[actual_index]
+            entries.push("  " + display(@expected_middle[expected_index]))
+            expected_index += 1
+            actual_index += 1
+          elsif lengths[expected_index + 1][actual_index] >= lengths[expected_index][actual_index + 1]
+            entries.push("- " + display(@expected_middle[expected_index]))
+            expected_index += 1
+          else
+            entries.push("+ " + display(@actual_middle[actual_index]))
+            actual_index += 1
+          end
+        end
+        push_rest(entries, "- ", @expected_middle, expected_index)
+        push_rest(entries, "+ ", @actual_middle, actual_index)
+        entries
+      end
+
+      def push_rest(entries, marker, lines, from)
+        index = from
+        while index < lines.length
+          entries.push(marker + display(lines[index]))
+          index += 1
+        end
+      end
+
+      def lcs_lengths
+        lengths = Array.new(@expected_middle.length + 1) { Array.new(@actual_middle.length + 1, 0) }
+        expected_index = @expected_middle.length - 1
+        while expected_index >= 0
+          actual_index = @actual_middle.length - 1
+          while actual_index >= 0
+            if @expected_middle[expected_index] == @actual_middle[actual_index]
+              lengths[expected_index][actual_index] = lengths[expected_index + 1][actual_index + 1] + 1
+            else
+              down = lengths[expected_index + 1][actual_index]
+              right = lengths[expected_index][actual_index + 1]
+              lengths[expected_index][actual_index] = down >= right ? down : right
+            end
+            actual_index -= 1
+          end
+          expected_index -= 1
+        end
+        lengths
+      end
+
+      def collapse(entries)
+        visible = visible_entries(entries)
+        out = []
+        index = 0
+        while index < entries.length
+          run_end = hidden_run_end(visible, index)
+          if run_end - index >= 2
+            out.push("  ...")
+            index = run_end
+          else
+            out.push(entries[index])
+            index += 1
+          end
+        end
+        out.join("\n")
+      end
+
+      def hidden_run_end(visible, from)
+        index = from
+        index += 1 while index < visible.length && !visible[index]
+        index
+      end
+
+      def visible_entries(entries)
+        visible = Array.new(entries.length, false)
+        entries.each_with_index do |entry, index|
+          next if entry.start_with?("  ")
+
+          first = index - @context
+          first = 0 if first < 0
+          last = index + @context
+          last = entries.length - 1 if last >= entries.length
+          first.upto(last) { |position| visible[position] = true }
+        end
+        visible
+      end
+    end
+  end
+end

data/lib/spinel_test/unit/generator.rb

@@ -4,16 +4,18 @@ module SpinelTest
   module Unit
     module Generator
       # files: [{ require: "test/foo_test", classes: [TestClass, ...] }, ...]
-      def self.generate(files)
+      # root: the directory the failure locations are shown relative to.
+      def self.generate(files, root = "")
         out = +"require \"spinel_test/unit\"\n"
         files.each { |f| out << "require #{f[:require].dump}\n" }
         out << ARGV_PARSER
-        out << "runner = SpinelTest::Unit::Runner.new(filter)\n"
+        out << "runner = SpinelTest::Unit::Runner.new(filter, #{root.dump}, testcase)\n"
         files.each do |f|
           f[:classes].each do |klass|
+            locations = klass.locations || {}
             klass.methods.each do |m|
               out << "t = #{klass.name}.new\n"
-              out << "runner.run(#{"#{klass.name}##{m}".dump}, t) { t.#{m} }\n"
+              out << "runner.run(#{"#{klass.name}##{m}".dump}, t, #{locations[m].to_s.dump}) { t.#{m} }\n"
             end
           end
         end
@@ -23,11 +25,15 @@ module SpinelTest
 
       ARGV_PARSER = <<~RUBY
         filter = ""
+        testcase = ""
         i = 0
         while i < ARGV.length
           if ARGV[i] == "-n" && i + 1 < ARGV.length
             filter = ARGV[i + 1]
             i += 1
+          elsif ARGV[i] == "-t" && i + 1 < ARGV.length
+            testcase = ARGV[i + 1]
+            i += 1
           end
           i += 1
         end

data/lib/spinel_test/unit/runner.rb

@@ -3,8 +3,13 @@
 module SpinelTest
   module Unit
     class Runner
-      def initialize(filter = "")
+      MAX_FRAMES = 3
+      FRAMEWORK_DIR = File.dirname(File.expand_path(__FILE__)) + "/"
+
+      def initialize(filter = "", root = "", testcase = "")
         @filter = filter
+        @root = root
+        @testcase = testcase
         @tests = 0
         @assertions = 0
         @failures = 0
@@ -13,27 +18,46 @@ module SpinelTest
       end
 
       def selected?(name)
-        @filter == "" || name.include?(@filter)
+        index = name.index("#")
+        klass = index ? name[0, index] : name
+        matches?(@testcase, klass, true) && matches?(@filter, name, false)
+      end
+
+      def matches?(filter, text, exact)
+        return true if filter == ""
+        return Regexp.new(filter[1, filter.length - 2]).match?(text) if pattern?(filter)
+
+        exact ? text == filter : text.include?(filter)
+      end
+
+      def pattern?(filter)
+        filter.length >= 2 && filter.start_with?("/") && filter.end_with?("/")
       end
 
       # Runs one test: setup, the block (which calls the test method), teardown.
-      def run(name, tc)
+      # location is the "file:line" of the test method, shown when the exception
+      # has no backtrace of its own.
+      def run(name, tc, location = "")
         return unless selected?(name)
 
         @tests += 1
         status = 0
         message = ""
+        frames = []
+        teardown_message = ""
         begin
           tc.setup
           yield
         rescue AssertionFailedError => e
           status = 1
           message = e.message
+          frames = report_frames(e, location)
         rescue Exception => e
           raise e if fatal?(e)
 
           status = 2
           message = e.message
+          frames = report_frames(e, location)
         end
         begin
           tc.teardown
@@ -43,8 +67,9 @@ module SpinelTest
           if status == 0
             status = 2
             message = e.message
+            frames = report_frames(e, location)
           else
-            message = message + "\nTeardown also raised: " + e.message
+            teardown_message = "\nTeardown also raised: " + e.message
           end
         end
         @assertions += tc.assertions
@@ -53,19 +78,69 @@ module SpinelTest
         elsif status == 1
           print "F"
           @failures += 1
-          @details.push("Failure: #{name}\n#{message}")
+          @details.push("Failure: #{name}\n#{message}#{where(frames)}#{teardown_message}")
         else
           print "E"
           @errors += 1
-          @details.push("Error: #{name}\n#{message}")
+          @details.push("Error: #{name}\n#{message}#{where(frames)}#{teardown_message}")
         end
       end
 
+      def report_frames(e, location)
+        backtrace = e.backtrace || []
+        frames = []
+        backtrace.each do |frame|
+          next if framework_frame?(frame)
+          next unless frames.empty? || under_root?(frame)
+
+          shown = relative_path(frame_location(frame))
+          next if frames.length > 0 && frames[frames.length - 1] == shown
+
+          frames.push(shown)
+          break if frames.length >= MAX_FRAMES
+        end
+        frames.push(location) if frames.empty? && location != ""
+        frames
+      end
+
+      def under_root?(frame)
+        @root == "" || frame.start_with?(@root + "/")
+      end
+
+      def framework_frame?(frame)
+        frame.start_with?(FRAMEWORK_DIR) || frame.start_with?($0 + ":")
+      end
+
+      def frame_location(frame)
+        index = frame.index(":in ")
+        index ? frame[0, index] : frame
+      end
+
+      def relative_path(location)
+        return location if @root == ""
+
+        prefix = @root + "/"
+        location.start_with?(prefix) ? location[prefix.length, location.length] : location
+      end
+
+      def where(frames)
+        text = ""
+        frames.each { |frame| text += "\n    at " + frame }
+        text
+      end
+
       # Exceptions that must end the process instead of being reported as a test error.
       def fatal?(e)
         e.is_a?(SystemExit) || e.is_a?(Interrupt)
       end
 
+      def filter_description
+        parts = []
+        parts.push("testcase: " + @testcase) if @testcase != ""
+        parts.push("filter: " + @filter) if @filter != ""
+        parts.empty? ? "" : " (" + parts.join(", ") + ")"
+      end
+
       def passed?
         @failures == 0 && @errors == 0
       end
@@ -84,7 +159,7 @@ module SpinelTest
         puts ""
         puts summary
         if @tests == 0
-          puts "No tests were run#{@filter == "" ? "" : " (filter: " + @filter + ")"}"
+          puts "No tests were run#{filter_description}"
           return 1
         end
         passed? ? 0 : 1

data/lib/spinel_test/unit/scanner.rb

@@ -8,10 +8,10 @@ module SpinelTest
       VISIBILITY = %i[private protected public].freeze
       BASE_NAMES = ["SpinelTest::Unit::TestCase", "::SpinelTest::Unit::TestCase"].freeze
 
-      TestClass = Struct.new(:name, :methods, :file, :files, keyword_init: true)
+      TestClass = Struct.new(:name, :methods, :file, :files, :locations, keyword_init: true)
       # superclass is the raw name as written; scope is the lexical namespace it
       # was written in. Both are qualified in resolve, once every class is known.
-      ClassInfo = Struct.new(:name, :superclass, :methods, :files, :hidden, :scope)
+      ClassInfo = Struct.new(:name, :superclass, :methods, :files, :hidden, :scope, :locations)
       Result = Struct.new(:classes, :warnings, keyword_init: true)
 
       def self.scan(sources)
@@ -68,7 +68,7 @@ module SpinelTest
           visit(node.body, namespace + [path_name(node.constant_path)]) if node.body
         when Prism::ClassNode
           name = (namespace + [path_name(node.constant_path)]).join("::")
-          info = (@classes[name] ||= ClassInfo.new(name, nil, [], []))
+          info = (@classes[name] ||= ClassInfo.new(name, nil, [], [], nil, nil, {}))
           info.files << @file if @file && !info.files.include?(@file)
           if node.superclass
             info.superclass = path_name(node.superclass)
@@ -107,7 +107,10 @@ module SpinelTest
           name = stmt.name.to_s
           next if info.hidden&.include?(name)
 
-          info.methods << name if name.start_with?("test_") && !info.methods.include?(name)
+          next unless name.start_with?("test_")
+
+          info.methods << name unless info.methods.include?(name)
+          info.locations[name] = "#{@file}:#{stmt.location.start_line}" if @file
         end
       end
 
@@ -148,7 +151,8 @@ module SpinelTest
 
           files = chain.reverse.flat_map(&:files).uniq
           file = (targets && info.files.find { |f| targets.include?(f) }) || info.files.first
-          TestClass.new(name: info.name, methods: methods, file: file, files: files)
+          locations = chain.reverse.reduce({}) { |all, link| all.merge(link.locations) }
+          TestClass.new(name: info.name, methods: methods, file: file, files: files, locations: locations)
         end
       end
 

data/lib/spinel_test/unit/test_case.rb

@@ -27,34 +27,67 @@ module SpinelTest
         raise AssertionFailedError, (message || "Failed assertion, no message given.")
       end
 
+      def refute(test, message = nil)
+        @assertions += 1
+        return true unless test
+
+        raise AssertionFailedError, (message || "Failed refutation, no message given.")
+      end
+
       def assert_equal(expected, actual, message = nil)
         @assertions += 1
         return true if expected == actual
 
-        msg = "Expected: #{expected.inspect}\n  Actual: #{actual.inspect}"
-        msg = message + "\n" + msg if message
-        raise AssertionFailedError, msg
+        failure("Expected: #{expected.inspect}\n  Actual: #{actual.inspect}" + diff_section(expected, actual), message)
+      end
+
+      def assert_not_equal(expected, actual, message = nil)
+        @assertions += 1
+        return true unless expected == actual
+
+        failure("Expected #{actual.inspect} not to equal #{expected.inspect}", message)
+      end
+
+      def assert_match(pattern, string, message = nil)
+        @assertions += 1
+        return true if pattern_matches?(pattern, string)
+
+        failure("Expected #{pattern.inspect} to match #{string.inspect}", message)
       end
 
-      def assert_nil(v, message = nil)
+      def assert_nil(value, message = nil)
         @assertions += 1
-        return true if v.nil?
+        return true if value.nil?
 
-        msg = "Expected nil, got #{v.inspect}"
-        msg = message + "\n" + msg if message
-        raise AssertionFailedError, msg
+        failure("Expected nil, got #{value.inspect}", message)
       end
 
-      def assert_includes(coll, item, message = nil)
+      def assert_not_nil(value, message = nil)
         @assertions += 1
-        return true if coll.include?(item)
+        return true unless value.nil?
 
-        msg = "#{coll.inspect} does not include #{item.inspect}"
-        msg = message + "\n" + msg if message
-        raise AssertionFailedError, msg
+        failure("Expected a value other than nil", message)
       end
 
-      def assert_raise(klass)
+      def assert_include(collection, item, message = nil)
+        @assertions += 1
+        return true if collection.include?(item)
+
+        failure("#{collection.inspect} does not include #{item.inspect}", message)
+      end
+
+      def assert_includes(collection, item, message = nil)
+        assert_include(collection, item, message)
+      end
+
+      def assert_not_include(collection, item, message = nil)
+        @assertions += 1
+        return true unless collection.include?(item)
+
+        failure("#{collection.inspect} includes #{item.inspect}", message)
+      end
+
+      def assert_raise(klass, match: nil)
         @assertions += 1
         begin
           yield
@@ -62,12 +95,54 @@ module SpinelTest
           # a failed assertion inside the block is not "the expected error"
           raise e
         rescue StandardError => e
-          return e if e.is_a?(klass)
+          return expected_exception(e, klass, match, e.instance_of?(klass))
+        end
+        raise AssertionFailedError, "Expected #{klass} but nothing was raised"
+      end
 
-          raise AssertionFailedError, "Expected #{klass} but got #{e.class}"
+      def assert_raise_kind_of(klass, match: nil)
+        @assertions += 1
+        begin
+          yield
+        rescue AssertionFailedError => e
+          raise e
+        rescue StandardError => e
+          return expected_exception(e, klass, match, e.is_a?(klass))
         end
         raise AssertionFailedError, "Expected #{klass} but nothing was raised"
       end
+
+      private
+
+      def failure(text, message)
+        raise AssertionFailedError, message ? message + "\n" + text : text
+      end
+
+      def expected_exception(e, klass, match, class_matches)
+        raise AssertionFailedError, "Expected #{klass} but got #{e.class}" unless class_matches
+        return e if match.nil? || pattern_matches?(match, e.message)
+
+        raise AssertionFailedError, "Expected #{klass} with a message matching #{match.inspect} but got #{e.message.inspect}"
+      end
+
+      def diff_section(expected, actual)
+        return "" unless multiline_strings?(expected, actual)
+
+        "\nDiff:\n" + Diff.lines(expected, actual)
+      rescue ArgumentError, EncodingError
+        ""
+      end
+
+      def multiline_strings?(expected, actual)
+        expected.is_a?(String) && actual.is_a?(String) && expected.include?("\n") && actual.include?("\n")
+      end
+
+      def pattern_matches?(pattern, string)
+        return false unless string.is_a?(String)
+        return pattern.match?(string) if pattern.is_a?(Regexp)
+
+        string.include?(pattern)
+      end
     end
   end
 end

data/lib/spinel_test/unit/version.rb

@@ -2,6 +2,6 @@
 
 module SpinelTest
   module Unit
-    VERSION = "0.1.2"
+    VERSION = "0.3.0"
   end
 end

data/lib/spinel_test/unit.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require_relative "unit/version"
+require_relative "unit/diff"
 require_relative "unit/test_case"
 require_relative "unit/runner"
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: spinel_test-unit
 version: !ruby/object:Gem::Version
-  version: 0.1.2
+  version: 0.3.0
 platform: ruby
 authors:
 - Yuji Yaginuma
@@ -58,6 +58,7 @@ files:
 - exe/spinel_test-unit
 - lib/spinel_test/unit.rb
 - lib/spinel_test/unit/cli.rb
+- lib/spinel_test/unit/diff.rb
 - lib/spinel_test/unit/generator.rb
 - lib/spinel_test/unit/runner.rb
 - lib/spinel_test/unit/scanner.rb
@@ -70,6 +71,7 @@ licenses:
 metadata:
   homepage_uri: https://github.com/y-yagi/spinel_test-unit
   source_code_uri: https://github.com/y-yagi/spinel_test-unit
+  rubygems_mfa_required: 'true'
 rdoc_options: []
 require_paths:
 - lib