grntest 1.0.0 → 1.0.1

data/README.md

@@ -6,7 +6,8 @@ grntest
 
 ## Description
 
-Grntest is a testing framework for groonga. You can write a test for groonga by writing groonga commands and expected result.
+Grntest is a testing framework for groonga. You can write a test for
+groonga by writing groonga commands and expected result.
 
 ## Install
 
@@ -14,9 +15,7 @@ Grntest is a testing framework for groonga. You can write a test for groonga by
 % gem install grntest
 ```
 
-## Usage
-
-### Basic usage
+## Basic usage
 
 Write a test script that extension is `.test`. Here is a sample test
 script `select.test`:
@@ -40,7 +39,7 @@ Run `grntest` with `select.test` as command line argument:
 N
 ================================================================================
 .
-  select                                                   0.1667s [not checked]
+  select                                                   0.3866s [not checked]
 ================================================================================
 table_create Users TABLE_HASH_KEY ShortText
 [[0,0.0,0.0],true]
@@ -55,8 +54,9 @@ select Users --query '_key:Alice'
 ================================================================================
 
 
-5.96 tests/sec: 1 tests, 0 passes, 0 failures, 1 not checked_tests
-0% passed in 0.1678s.
+  tests/sec | tests     | passes    | failures  | leaked    | !checked  |
+       2.57 |         1 |         0 |         0 |         0 |         1 |
+0% passed in 0.3885s.
 ```
 
 It generates `select.actual` file that contains actual result. If it
@@ -72,8 +72,9 @@ Run `grntest` again:
 % grntest select.test
 .
 
-6.12 tests/sec: 1 tests, 1 passes, 0 failures, 0 not checked_tests
-100% passed in 0.1635s.
+  tests/sec | tests     | passes    | failures  | leaked    | !checked  |
+       5.76 |         1 |         1 |         0 |         0 |         0 |
+100% passed in 0.1736s.
 ```
 
 It compares actual result and content of `select.expected` and
@@ -103,23 +104,24 @@ Run `grntest` again:
 F
 ================================================================================
 .
-  select                                                        0.1445s [failed]
+  select                                                        0.1767s [failed]
 ================================================================================
---- (actual)
-+++ (expected)
+--- (expected)
++++ (actual)
 @@ -6,5 +6,5 @@
  {"_key": "Bob"}
  ]
  [[0,0.0,0.0],2]
--select Users --query '_key:Bob'
--[[0,0.0,0.0],[[[1],[["_id","UInt32"],["_key","ShortText"]],[2,"Bob"]]]]
-+select Users --query '_key:Alice'
-+[[0,0.0,0.0],[[[1],[["_id","UInt32"],["_key","ShortText"]],[1,"Alice"]]]]
+-select Users --query '_key:Alice'
+-[[0,0.0,0.0],[[[1],[["_id","UInt32"],["_key","ShortText"]],[1,"Alice"]]]]
++select Users --query '_key:Bob'
++[[0,0.0,0.0],[[[1],[["_id","UInt32"],["_key","ShortText"]],[2,"Bob"]]]]
 ================================================================================
 
 
-6.68 tests/sec: 1 tests, 0 passes, 1 failures, 0 not checked_tests
-0% passed in 0.1497s.
+  tests/sec | tests     | passes    | failures  | leaked    | !checked  |
+       4.65 |         1 |         0 |         1 |         0 |         0 |
+0% passed in 0.2153s.
 ```
 
 It says the expected result that is read from `select.expected` and
@@ -140,19 +142,284 @@ Run `grntest` again:
 % grntest select.test
 .
 
-6.97 tests/sec: 1 tests, 1 passes, 0 failures, 0 not checked_tests
-100% passed in 0.1434s.
+  tests/sec | tests     | passes    | failures  | leaked    | !checked  |
+       6.20 |         1 |         1 |         0 |         0 |         0 |
+100% passed in 0.1613s.
 ```
 
 The test is succeeded again.
 
-### Advanced usage
+## Advanced usage
+
+There are more useful features. They are not needed for normal
+users but they are very useful for advanced users.
+
+There are advanced features:
+
+* Comment
+* Continuation line
+* Directives
+
+### Comment
+
+Groonga supports comment line by `#`.
+
+Example:
+
+```
+# This line is comment line.
+select Users
+```
+
+Grntest also supports the syntax. You can use `#` as comment mark.
+
+### Continuation line
+
+You can break a long line by escaping new line with `\`.
+
+Example:
+
+```
+select Users \
+  --match_columns name \
+  --query Ken
+```
+
+The command is processed as the following command:
+
+```
+select Users  --match_columns name   --query Ken
+```
+
+You can make your test readable with this feature.
+
+Groogna doesn't support this feature.
+
+### Directives
+
+Grntest supports directives that control grntest behavior.
+
+Here is directive syntax:
+
+```
+#@NAME [ARGUMENTS...]
+```
 
-See `grntest --help`. It contains many usuful features.
+Here are available `NAME` s:
 
-Some important features are described in this section.
+* `disable-logging`
+* `enable-logging`
+* `suggest-create-dataset`
+* `include`
+* `copy-path`
 
-#### `--n-workers`
+`ARGUMENTS...` are depends on directive. A directive doesn't require
+any arguments but a directive requires arguments.
+
+#### `disable-logging`
+
+Usage:
+
+```
+#@disable-logging
+```
+
+It disables logging executed command and executed result until
+`enable-logging` directive is used. It is useful for executing
+commands that isn't important for test.
+
+Example:
+
+```
+#@disable-logging
+load --table Users
+[
+{"_key": "User1"},
+{"_key": "..."},
+{"_key": "User9999999"}
+]
+#@enable-logging
+
+select Users --query _key:User29
+```
+
+See also: `enable-logging`
+
+### `enable-logging`
+
+Usage:
+
+```
+#@enable-logging
+```
+
+It enables logging that is disabled by `disable-logging` directive.
+
+See also: `disable-logging`
+
+### suggest-create-dataset
+
+Usage:
+
+```
+#@suggest-create-dataset DATASET_NAME
+```
+
+It creates dataset `DATASET_NAME` for suggest feature. It is useful
+for testing suggest feature.
+
+Example:
+
+```
+#@suggest-create-dataset rurema
+load --table event_rurema --each 'suggest_preparer(_id, type, item, sequence, time, pair_rurema)'
+[
+["sequence", "time", "item", "type"],
+["21e80fd5e5bc2126469db1c927b7a48fb3353dd9",1312134098.0,"f",null],
+["21e80fd5e5bc2126469db1c927b7a48fb3353dd9",1312134105.0,"er",null],
+["21e80fd5e5bc2126469db1c927b7a48fb3353dd9",1312134106.0,"erb",null],
+["21e80fd5e5bc2126469db1c927b7a48fb3353dd9",1312134107.0,"erb","submit"]
+]
+```
+
+See also: `--groonga-suggest-create-dataset` option
+
+### include
+
+Usage:
+
+```
+#@include SUB_TEST_FILE_PATH
+```
+
+It includes `SUB_TEST_FILE_PATH` content. It is useful for sharing
+commands by many tests.
+
+You can use `include` in included file. It means that an included file
+is processed in the same way as a test file.
+
+If `SUB_TEST_FILE_PATH` is relative path, `SUB_TEST_FILE_PATH` is
+found from base directory. Base directory can be specified by
+`--base-directory` option.
+
+Example:
+
+init.grn:
+
+```
+#@disable-logging
+#@include ddl.grn
+#@include data.grn
+#@enable-logging
+```
+
+ddl.grn:
+
+```
+table_create Users TABLE_HASH_KEY ShortText
+```
+
+data.grn:
+
+```
+load --table Users
+[
+["_key"],
+["Alice"],
+["Bob"]
+]
+```
+
+user.test:
+
+```
+#@include init.grn
+select Users --query _key:Alice
+```
+
+See also: `--base-directory` option
+
+### `copy-path`
+
+Usage
+
+```
+#@copy-path SOURCE DESTINATION
+```
+
+It copies a path from `SOURCE` to `DESTINATION`. You can use it for
+both file and directory. It is useful for using fixture data.
+
+Example:
+
+```
+#@copy-path fixture/query_expander/tsv/japanese_synonyms.tsv tmp/synonyms.tsv
+register "query_expanders/tsv"
+```
+
+## Options
+
+Grntest has many options. You don't need to specify many of them
+because they use suitable default values.
+
+This section describes some important options. You can see all options
+by `grntest --help`. You will find many usuful features from it.
+
+### `--test`
+
+Usage:
+
+```
+% grntest --test TEST_NAME ...
+% grntest --test /TEST_NAME_REGEXP/ ...
+% grntest --test TEST_NAME1 --test TEST_NAME2 ...
+% grntest --test /TEST_NAME_REGEXP1/ --test /TEST_NAME_REGEXP2/ ...
+```
+
+`--test` option specifies tests that should be ran. It is useful when
+you are interested in only one test.
+
+For example, the following command line runs only `vector-geo-point`
+test.
+
+```
+% grntest --test vector-geo-point ...
+```
+
+You can use `--test` option multiple times to run only multiple
+interested tests.
+
+For example, the following command line runs only `vector-geo-point`
+and `hash` tests.
+
+```
+% grntest --test vector-geo-point --test hash ...
+```
+
+You can use regular expression to select tests by `/.../` syntax.
+
+For example, the following command line runs tests that have `geo` in
+its name.
+
+```
+% grntest --test /geo/ ...
+```
+
+You can also use multiple --test options with regular expression.
+
+See also: `--exclude-test` option
+
+### `--exclude-test`
+
+### `--test-suite`
+
+### `--exclude-test-suite`
+
+### `--gdb`
+
+### `--keep-database`
+
+### `--n-workers`
 
 `--n-workers` option is very useful. You can run many test scripts at
 once. Tests are finished quickly. You should specify one or more
@@ -163,31 +430,62 @@ Here is a sample command line to use `--n-workers`:
 
 ```
 % grntest --n-workers 4 test/function/suite/suggest
+  tests/sec | tests     | passes    | failures  | leaked    | !checked  |
 [0]                                                                   [finished]
-  8.60 tests/sec: 4 tests, 4 passes, 0 failures, 0 not checked_tests
+       9.95 |         5 |         5 |         0 |         0 |         0 |       
 [1]                                                                   [finished]
-  9.85 tests/sec: 5 tests, 5 passes, 0 failures, 0 not checked_tests
+       9.22 |         4 |         4 |         0 |         0 |         0 |       
 [2]                                                                   [finished]
-  8.63 tests/sec: 4 tests, 4 passes, 0 failures, 0 not checked_tests
+       9.11 |         4 |         4 |         0 |         0 |         0 |       
 [3]                                                                   [finished]
-  9.68 tests/sec: 5 tests, 5 passes, 0 failures, 0 not checked_tests
+      10.10 |         5 |         5 |         0 |         0 |         0 |       
 |-----------------------------------------------------------------------| [100%]
 
-34.43 tests/sec: 18 tests, 18 passes, 0 failures, 0 not checked_tests
-100% passed in 0.5228s.
+  tests/sec | tests     | passes    | failures  | leaked    | !checked  |
+      34.61 |        18 |        18 |         0 |         0 |         0 |
+100% passed in 0.5201s.
+```
+
+### `--base-directory`
+
+### `--database=PATH`
+
+Usage:
+
 ```
+% grntest --database /PATH/TO/EXISTING/DATABASE ...
+```
+
+`--database` option specifies an existing database that is used for
+test. Normally, grntest uses a new empty database for test.
+
+`--database` option is very useful for testing against existing
+database.
+
+### `--groonga`
+
+### `--groonga-httpd`
+
+### `--groonga-suggest-create-dataset`
+
+### `--interface`
+
+### `--output-type`
+
+### `--testee`
 
-### Examples
+## Examples
 
-See [test/function/ directory in groonga's
-source](https://github.com/groonga/groonga/tree/master/test/function). It
+See [test/command/ directory in groonga's
+source](https://github.com/groonga/groonga/tree/master/test/command). It
 has many test scripts and uses many useful features. They will help you.
 
 ## Dependencies
 
 * Ruby 1.9.3
+* msgpack gem
 
-### Mailing list
+## Mailing list
 
 * English: [groonga-talk@lists.sourceforge.net](https://lists.sourceforge.net/lists/listinfo/groonga-talk)
 * Japanese: [groonga-dev@lists.sourceforge.jp](http://lists.sourceforge.jp/mailman/listinfo/groonga-dev)
@@ -205,4 +503,5 @@ has many test scripts and uses many useful features. They will help you.
 
 GPLv3 or later. See doc/text/gpl-3.0.txt for details.
 
-(Kouhei Sutou has a right to change the license including contributed patches.)
+(Kouhei Sutou has a right to change the license including contributed
+patches.)

data/doc/text/news.md

@@ -1,5 +1,38 @@
 # News
 
+## 1.0.1: 2012-10-15
+
+This has a backward incompatible change. It is directive syntax.
+
+Old:
+
+    # NAME ARGUMENT
+
+New:
+
+    #@NAME ARGUMENT
+
+This change is for easy to debug. Conssider about we have a typo in
+`NAME`. It is just ignored in old syntax because it is also a comment
+line. It is reported in new syntax because it syntax is only for
+directive. Grntest can know that user want to use a directive. If
+grntest knows that user want to use a directive, grntest can report an
+error for an unknown directive name.
+
+### Improvements
+
+* Inverted expected and actual in diff.
+* Added memory leak check.
+* Documented many features.
+* Changed directive syntax.
+* Added `copy-path` directive.
+* Supported multiple `--test` and `--test-suite` options.
+* Added `--database` option.
+
+### Fixes
+
+* Fixed a problem that test report can't be shown for no tests.
+
 ## 1.0.0: 2012-08-29
 
 The first release!!!

data/lib/grntest/tester.rb

@@ -122,6 +122,13 @@ module Grntest
           tester.base_directory = Pathname(directory)
         end
 
+        parser.on("--database=PATH",
+                  "Use existing database at PATH " +
+                    "instead of creating a new database",
+                  "(creating a new database)") do |path|
+          tester.database_path = path
+        end
+
         parser.on("--diff=DIFF",
                   "Use DIFF as diff command",
                   "(#{tester.diff})") do |diff|
@@ -224,7 +231,7 @@ module Grntest
 
     attr_accessor :groonga, :groonga_httpd, :groonga_suggest_create_dataset
     attr_accessor :interface, :output_type, :testee
-    attr_accessor :base_directory, :diff, :diff_options
+    attr_accessor :base_directory, :database_path, :diff, :diff_options
     attr_accessor :n_workers
     attr_accessor :output
     attr_accessor :gdb, :default_gdb
@@ -239,6 +246,7 @@ module Grntest
       @output_type = "json"
       @testee = "groonga"
       @base_directory = Pathname(".")
+      @database_path = nil
       @reporter = nil
       @n_workers = 1
       @output = $stdout
@@ -288,7 +296,8 @@ module Grntest
     end
 
     def selected_test?(test_name)
-      @test_patterns.all? do |pattern|
+      return true if @test_patterns.empty?
+      @test_patterns.any? do |pattern|
         pattern === test_name
       end
     end
@@ -305,7 +314,8 @@ module Grntest
     end
 
     def selected_test_suite?(test_suite_name)
-      @test_suite_patterns.all? do |pattern|
+      return true if @test_suite_patterns.empty?
+      @test_suite_patterns.any? do |pattern|
         pattern === test_suite_name
       end
     end
@@ -396,12 +406,14 @@ module Grntest
     end
 
     class WorkerResult < Result
-      attr_reader :n_tests, :n_passed_tests, :n_not_checked_tests
+      attr_reader :n_tests, :n_passed_tests, :n_leaked_tests
+      attr_reader :n_not_checked_tests
       attr_reader :failed_tests
       def initialize
         super
         @n_tests = 0
         @n_passed_tests = 0
+        @n_leaked_tests = 0
         @n_not_checked_tests = 0
         @failed_tests = []
       end
@@ -422,6 +434,10 @@ module Grntest
         @failed_tests << name
       end
 
+      def test_leaked(name)
+        @n_leaked_tests += 1
+      end
+
       def test_not_checked
         @n_not_checked_tests += 1
       end
@@ -501,10 +517,16 @@ module Grntest
         @reporter.fail_test(self, result)
       end
 
-      def not_check_test(result)
+      def leaked_test(result)
+        @status = "leaked(#{result.n_leaked_objects})"
+        @result.test_leaked(test_name)
+        @reporter.leaked_test(self, result)
+      end
+
+      def not_checked_test(result)
         @status = "not checked"
         @result.test_not_checked
-        @reporter.not_check_test(self, result)
+        @reporter.not_checked_test(self, result)
       end
 
       def finish_test(result)
@@ -545,6 +567,10 @@ module Grntest
         collect_count(:n_failed_tests)
       end
 
+      def n_leaked_tests
+        collect_count(:n_leaked_tests)
+      end
+
       def n_not_checked_tests
         collect_count(:n_not_checked_tests)
       end
@@ -633,24 +659,33 @@ module Grntest
 
     class TestResult < Result
       attr_accessor :worker_id, :test_name
-      attr_accessor :expected, :actual
+      attr_accessor :expected, :actual, :n_leaked_objects
       def initialize(worker)
         super()
         @worker_id = worker.id
         @test_name = worker.test_name
         @actual = nil
         @expected = nil
+        @n_leaked_objects = 0
       end
 
       def status
         if @expected
           if @actual == @expected
-            :success
+            if @n_leaked_objects.zero?
+              :success
+            else
+              :leaked
+            end
           else
             :failure
           end
         else
-          :not_checked
+          if @n_leaked_objects.zero?
+            :not_checked
+          else
+            :leaked
+          end
         end
       end
     end
@@ -673,7 +708,7 @@ module Grntest
         result.measure do
           result.actual = execute_groonga_script
         end
-        result.actual = normalize_result(result.actual)
+        normalize_actual_result(result)
         result.expected = read_expected_result
         case result.status
         when :success
@@ -683,8 +718,11 @@ module Grntest
           @worker.fail_test(result)
           output_reject_file(result.actual)
           succeeded = false
+        when :leaked
+          @worker.leaked_test(result)
+          succeeded = false
         else
-          @worker.not_check_test(result)
+          @worker.not_checked_test(result)
           output_actual_file(result.actual)
         end
         @worker.finish_test(result)
@@ -695,9 +733,13 @@ module Grntest
       private
       def execute_groonga_script
         create_temporary_directory do |directory_path|
-          db_dir = directory_path + "db"
-          FileUtils.mkdir_p(db_dir.to_s)
-          db_path = db_dir + "db"
+          if @tester.database_path
+            db_path = Pathname(@tester.database_path).expand_path
+          else
+            db_dir = directory_path + "db"
+            FileUtils.mkdir_p(db_dir.to_s)
+            db_path = db_dir + "db"
+          end
           context = Executor::Context.new
           context.temporary_directory_path = directory_path
           context.db_path = db_path
@@ -708,6 +750,7 @@ module Grntest
           run_groonga(context) do |executor|
             executor.execute(test_script_path)
           end
+          check_memory_leak(context)
           context.result
         end
       end
@@ -734,6 +777,10 @@ module Grntest
       end
 
       def run_groonga(context, &block)
+        unless @tester.database_path
+          create_empty_database(context.db_path.to_s)
+        end
+
         case @tester.interface
         when :stdio
           run_groonga_stdio(context, &block)
@@ -760,7 +807,6 @@ module Grntest
             command_line += [
               "--input-fd", input_fd.to_s,
               "--output-fd", output_fd.to_s,
-              "-n",
               context.relative_db_path.to_s,
             ]
             pid = Process.spawn(env, *command_line, spawn_options)
@@ -890,7 +936,6 @@ EOC
             "--port", port.to_s,
             "--protocol", "http",
             "-s",
-            "-n",
             context.relative_db_path.to_s,
           ]
         when "groonga-httpd"
@@ -907,7 +952,6 @@ EOC
       end
 
       def create_config_file(context, host, port, pid_file_path)
-        create_empty_database(context.db_path.to_s)
         config_file_path =
           context.temporary_directory_path + "groonga-httpd.conf"
         config_file_path.open("w") do |config_file|
@@ -950,9 +994,9 @@ EOF
         output_fd.close(true)
       end
 
-      def normalize_result(result)
+      def normalize_actual_result(result)
         normalized_result = ""
-        result.each do |tag, content, options|
+        result.actual.each do |tag, content, options|
           case tag
           when :input
             normalized_result << content
@@ -960,9 +1004,11 @@ EOF
             normalized_result << normalize_output(content, options)
           when :error
             normalized_result << normalize_raw_content(content)
+          when :n_leaked_objects
+            result.n_leaked_objects = content
           end
         end
-        normalized_result
+        result.actual = normalized_result
       end
 
       def normalize_raw_content(content)
@@ -1069,6 +1115,17 @@ EOF
           result_file.print(actual_result)
         end
       end
+
+      def check_memory_leak(context)
+        context.log.each_line do |line|
+          timestamp, log_level, message = line.split(/\|\s*/, 3)
+          _ = timestamp # suppress warning
+          next unless /^grn_fin \((\d+)\)$/ =~ message
+          n_leaked_objects = $1.to_i
+          next if n_leaked_objects.zero?
+          context.result << [:n_leaked_objects, n_leaked_objects, {}]
+        end
+      end
     end
 
     class Executor
@@ -1181,17 +1238,64 @@ EOF
 
       def execute_line(line)
         case line
+        when /\A\#@/
+          directive_content = $POSTMATCH
+          execute_directive(line, directive_content)
         when /\A\s*\z/
           # do nothing
         when /\A\s*\#/
-          comment_content = $POSTMATCH
-          execute_comment(comment_content)
+          # ignore comment
         else
           execute_command_line(line)
         end
       end
 
-      def execute_comment(content)
+      def resolve_path(path)
+        if path.relative?
+          @context.base_directory + path
+        else
+          path
+        end
+      end
+
+      def execute_directive_suggest_create_dataset(line, content, options)
+        dataset_name = options.first
+        if dataset_name.nil?
+          log_input(line)
+          log_error("#|e| [suggest-create-dataset] dataset name is missing")
+          return
+        end
+        execute_suggest_create_dataset(dataset_name)
+      end
+
+      def execute_directive_include(line, content, options)
+        path = options.first
+        if path.nil?
+          log_input(line)
+          log_error("#|e| [include] path is missing")
+          return
+        end
+        execute_script(Pathname(path))
+      end
+
+      def execute_directive_copy_path(line, content, options)
+        source, destination, = options
+        if source.nil? or destination.nil?
+          log_input(line)
+          if source.nil?
+            log_error("#|e| [copy-path] source is missing")
+          end
+          if destiantion.nil?
+            log_error("#|e| [copy-path] destination is missing")
+          end
+          return
+        end
+        source = resolve_path(Pathname(source))
+        destination = resolve_path(Pathname(destination))
+        FileUtils.cp_r(source.to_s, destination.to_s)
+      end
+
+      def execute_directive(line, content)
         command, *options = Shellwords.split(content)
         case command
         when "disable-logging"
@@ -1199,13 +1303,14 @@ EOF
         when "enable-logging"
           @context.logging = true
         when "suggest-create-dataset"
-          dataset_name = options.first
-          return if dataset_name.nil?
-          execute_suggest_create_dataset(dataset_name)
+          execute_directive_suggest_create_dataset(line, content, options)
         when "include"
-          path = options.first
-          return if path.nil?
-          execute_script(Pathname(path))
+          execute_directive_include(line, content, options)
+        when "copy-path"
+          execute_directive_copy_path(line, content, options)
+        else
+          log_input(line)
+          log_error("#|e| unknown directive: <#{command}>")
         end
       end
 
@@ -1227,10 +1332,7 @@ EOF
 
       def execute_script(script_path)
         executor = create_sub_executor(@context)
-        if script_path.relative?
-          script_path = @context.base_directory + script_path
-        end
-        executor.execute(script_path)
+        executor.execute(resolve_path(script_path))
       end
 
       def execute_command_line(command_line)
@@ -1513,6 +1615,7 @@ EOF
       end
 
       def report_summary(result)
+        puts(statistics_header)
         puts(colorize(statistics(result), result))
         pass_ratio = result.pass_ratio
         elapsed_time = result.elapsed_time
@@ -1520,14 +1623,28 @@ EOF
         puts(colorize(summary, result))
       end
 
+      def statistics_header
+        items = [
+          "tests/sec",
+          "tests",
+          "passes",
+          "failures",
+          "leaked",
+          "!checked",
+        ]
+        "  " + ((["%-9s"] * items.size).join(" | ") % items) + " |"
+      end
+
       def statistics(result)
         items = [
-          "#{result.n_tests} tests",
-          "#{result.n_passed_tests} passes",
-          "#{result.n_failed_tests} failures",
-          "#{result.n_not_checked_tests} not checked_tests",
+          "%9.2f" % throughput(result),
+          "%9d" % result.n_tests,
+          "%9d" % result.n_passed_tests,
+          "%9d" % result.n_failed_tests,
+          "%9d" % result.n_leaked_tests,
+          "%9d" % result.n_not_checked_tests,
         ]
-        "#{throughput(result)}: " + items.join(", ")
+        "  " + items.join(" | ") + " |"
       end
 
       def throughput(result)
@@ -1536,7 +1653,7 @@ EOF
         else
           tests_per_second = result.n_tests / result.elapsed_time
         end
-        "%.2f tests/sec" % tests_per_second
+        tests_per_second
       end
 
       def report_failure(result)
@@ -1559,8 +1676,8 @@ EOF
         create_temporary_file("expected", expected) do |expected_file|
           create_temporary_file("actual", actual) do |actual_file|
             diff_options = @tester.diff_options.dup
-            diff_options.concat(["--label", "(actual)", actual_file.path,
-                                 "--label", "(expected)", expected_file.path])
+            diff_options.concat(["--label", "(expected)", expected_file.path,
+                                 "--label", "(actual)", actual_file.path])
             system(@tester.diff, *diff_options)
           end
         end
@@ -1669,6 +1786,8 @@ EOF
         else
           if result.n_failed_tests > 0
             :failure
+          elsif result.n_leaked_tests > 0
+            :leaked
           elsif result.n_not_checked_tests > 0
             :not_checked
           else
@@ -1689,6 +1808,8 @@ EOF
           "%s%s%s" % [success_color, message, reset_color]
         when :failure
           "%s%s%s" % [failure_color, message, reset_color]
+        when :leaked
+          "%s%s%s" % [leaked_color, message, reset_color]
         when :not_checked
           "%s%s%s" % [not_checked_color, message, reset_color]
         else
@@ -1722,6 +1843,19 @@ EOF
                         })
       end
 
+      def leaked_color
+        escape_sequence({
+                          :color => :magenta,
+                          :color_256 => [3, 0, 3],
+                          :background => true,
+                        },
+                        {
+                          :color => :white,
+                          :color_256 => [5, 5, 5],
+                          :bold => true,
+                        })
+      end
+
       def not_checked_color
         escape_sequence({
                           :color => :cyan,
@@ -1823,7 +1957,13 @@ EOF
         end
       end
 
-      def not_check_test(worker, result)
+      def leaked_test(worker, result)
+        synchronize do
+          report_test_result_mark("L(#{result.n_leaked_objects})", result)
+        end
+      end
+
+      def not_checked_test(worker, result)
         synchronize do
           report_test_result_mark("N", result)
           puts
@@ -1849,6 +1989,9 @@ EOF
 
       private
       def report_test_result_mark(mark, result)
+        if @term_width < @current_column + mark.bytesize
+          puts
+        end
         print(colorize(mark, result))
         if @term_width <= @current_column
           puts
@@ -1892,7 +2035,11 @@ EOF
         report_failure(result)
       end
 
-      def not_check_test(worker, result)
+      def leaked_test(worker, result)
+        report_test_result(result, worker.status)
+      end
+
+      def not_checked_test(worker, result)
         report_test_result(result, worker.status)
         report_actual(result)
       end
@@ -1945,7 +2092,14 @@ EOF
         end
       end
 
-      def not_check_test(worker, result)
+      def leaked_test(worker, result)
+        redraw do
+          report_test(worker, result)
+          report_marker(result)
+        end
+      end
+
+      def not_checked_test(worker, result)
         redraw do
           report_test(worker, result)
           report_actual(result)
@@ -1972,6 +2126,7 @@ EOF
 
       private
       def draw
+        draw_statistics_header_line
         @test_suites_result.workers.each do |worker|
           draw_status_line(worker)
           draw_test_line(worker)
@@ -1979,6 +2134,10 @@ EOF
         draw_progress_line
       end
 
+      def draw_statistics_header_line
+        puts(statistics_header)
+      end
+
       def draw_status_line(worker)
         clear_line
         left = "[#{colorize(worker.id, worker.result)}] "
@@ -1994,7 +2153,7 @@ EOF
         if worker.test_name
           label = "  #{worker.test_name}"
         else
-          label = "  #{statistics(worker.result)}"
+          label = statistics(worker.result)
         end
         puts(justify(label, @term_width))
       end
@@ -2002,7 +2161,11 @@ EOF
       def draw_progress_line
         n_done_tests = @test_suites_result.n_tests
         n_total_tests = @test_suites_result.n_total_tests
-        finished_test_ratio = n_done_tests.to_f / n_total_tests
+        if n_total_tests.zero?
+          finished_test_ratio = 0.0
+        else
+          finished_test_ratio = n_done_tests.to_f / n_total_tests
+        end
 
         start_mark = "|"
         finish_mark = "|"
@@ -2054,7 +2217,11 @@ EOF
       end
 
       def n_using_lines
-        n_worker_lines * n_workers + n_progress_lines
+        n_statistics_header_line + n_worker_lines * n_workers + n_progress_lines
+      end
+
+      def n_statistics_header_line
+        1
       end
 
       def n_worker_lines

data/lib/grntest/version.rb

@@ -14,5 +14,5 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 module Grntest
-  VERSION = "1.0.0"
+  VERSION = "1.0.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: grntest
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.0.1
   prerelease: 
 platform: ruby
 authors:
@@ -10,7 +10,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-08-29 00:00:00.000000000 Z
+date: 2012-10-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: json