kimera 0.1.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +13 -0
- data/README.md +14 -1
- data/lib/kimera/cli/doctor.rb +24 -15
- data/lib/kimera/cli/test_command.rb +53 -0
- data/lib/kimera/execution/child_process.rb +1 -1
- data/lib/kimera/execution/harness.rb +1 -1
- data/lib/kimera/execution/isolated_child.rb +9 -1
- data/lib/kimera/execution/isolated_child_command.rb +8 -5
- data/lib/kimera/execution/isolated_child_minitest.rb +12 -7
- data/lib/kimera/execution/isolated_outcome.rb +50 -0
- data/lib/kimera/execution/isolated_plan.rb +2 -2
- data/lib/kimera/execution/isolated_verdict.rb +15 -8
- data/lib/kimera/execution/parallel_test_databases.rb +7 -0
- data/lib/kimera/execution/schemata.rb +12 -5
- data/lib/kimera/frameworks/adapter.rb +13 -0
- data/lib/kimera/frameworks/minitest_adapter.rb +1 -1
- data/lib/kimera/frameworks/rspec_adapter.rb +1 -1
- data/lib/kimera/report/actions.rb +25 -11
- data/lib/kimera/report/sections.rb +1 -1
- data/lib/kimera/support/version.rb +1 -1
- data/lib/kimera/synthesis/overlay.rb +2 -1
- data/lib/kimera/synthesis/overlay_splice.rb +3 -2
- data/lib/kimera/synthesis/superclass_pin.rb +40 -0
- data/skills/kimera/SKILL.md +15 -4
- metadata +4 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 3c0b6544c638890d575523723b8d86dcac500e247c22a989fda9fa13b9f8d68b
|
|
4
|
+
data.tar.gz: 4b354333f7a2173a49932c9761f1731e034cd0a91df2a6534081412bd0598f92
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 84b6ffd6aac361151843796089b6a4b85e1aa34aafdc8b9a78caf407ef18e774b72527235d4596649b08c52454d2a743e1e35d032240d7eaf6c577f6ad80a3d9
|
|
7
|
+
data.tar.gz: 0b1cd61f82643207a14f2fff230b46e05de8be433f8989a8e55df6e9e5e7af0198f55cd009d411a6beeeeedd5f4c68946d119ba1aa77326e27f12cb3406fe566
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,18 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.1.1 (2026-09-26)
|
|
4
|
+
|
|
5
|
+
- Fix a boot crash on case-sensitive filesystems (Linux): `require "English"`
|
|
6
|
+
was spelled `require "english"`.
|
|
7
|
+
- `--isolated` judges a mutant on failed tests, not the child's exit status.
|
|
8
|
+
A child that exits non-zero with zero failures (e.g. a SimpleCov
|
|
9
|
+
`minimum_coverage` floor tripped by the partial run) is now `harness_error`
|
|
10
|
+
instead of a false kill. Kills carry their failing tests. Children run with
|
|
11
|
+
`KIMERA=1`, and `kimera doctor` warns about an ungated coverage floor.
|
|
12
|
+
- Skill: raising `max_ignored` is a judgment any triager may make when the
|
|
13
|
+
entry has earned it (isolated survival, a mechanism `reason:`, the raise in
|
|
14
|
+
the same diff), never a way to make a failing gate pass.
|
|
15
|
+
|
|
3
16
|
## 0.1.0 (2026-09-26)
|
|
4
17
|
|
|
5
18
|
Initial public release.
|
data/README.md
CHANGED
|
@@ -218,6 +218,18 @@ process with the code under test. On ordinary code it reaches the same verdicts
|
|
|
218
218
|
as the warm path. It can also test what the warm path can't: the runtime
|
|
219
219
|
selector and Kimera's own harness-critical files (see below).
|
|
220
220
|
|
|
221
|
+
An isolated child is judged on its failed tests, not its exit status. A child
|
|
222
|
+
that exits non-zero with no failing test is `harness_error` (unjudged, gated
|
|
223
|
+
by `--max-errors`), not a kill. The usual cause is a coverage floor such as
|
|
224
|
+
SimpleCov's `minimum_coverage`, which a partial run always trips. Children run
|
|
225
|
+
with `KIMERA=1` set, so skip the floor there:
|
|
226
|
+
|
|
227
|
+
```ruby
|
|
228
|
+
SimpleCov.start { minimum_coverage(line: 100, branch: 100) unless ENV["KIMERA"] }
|
|
229
|
+
```
|
|
230
|
+
|
|
231
|
+
`kimera doctor` warns when it finds an ungated `minimum_coverage`.
|
|
232
|
+
|
|
221
233
|
### Progress output
|
|
222
234
|
|
|
223
235
|
During a run, a progress bar tracks each phase on stderr. It shows only when
|
|
@@ -260,7 +272,8 @@ fail_on_no_coverage: false
|
|
|
260
272
|
|
|
261
273
|
# The gate fails when more than this many mutants are ignore-listed. Marking
|
|
262
274
|
# one more mutant equivalent means raising this number in the same diff, so
|
|
263
|
-
# the ignore list stays a
|
|
275
|
+
# the ignore list stays a justified decision, reviewed in the diff that makes
|
|
276
|
+
# it, not a shortcut past a test.
|
|
264
277
|
max_ignored: 1
|
|
265
278
|
|
|
266
279
|
# Accepted pre-adoption survivors, merged with the ignore rules below.
|
data/lib/kimera/cli/doctor.rb
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require "open3"
|
|
4
3
|
require_relative "../error"
|
|
5
4
|
require_relative "../scope/config"
|
|
6
5
|
require_relative "../scope/file_set"
|
|
7
6
|
require_relative "flag"
|
|
7
|
+
require_relative "test_command"
|
|
8
8
|
|
|
9
9
|
class Kimera::CLI::Doctor
|
|
10
10
|
OPTIONS = Kimera::FlagTable.new(
|
|
@@ -16,6 +16,8 @@ class Kimera::CLI::Doctor
|
|
|
16
16
|
|
|
17
17
|
FRAMEWORK_FEATURES = { "rspec" => "rspec/core", "minitest" => "minitest" }.freeze
|
|
18
18
|
|
|
19
|
+
HELPERS = %w[.simplecov spec/spec_helper.rb spec/rails_helper.rb test/test_helper.rb].freeze
|
|
20
|
+
|
|
19
21
|
def initialize(io: $stdout, errors: $stderr, root: ".")
|
|
20
22
|
@io = io
|
|
21
23
|
@errors = errors
|
|
@@ -38,7 +40,8 @@ class Kimera::CLI::Doctor
|
|
|
38
40
|
end
|
|
39
41
|
|
|
40
42
|
def checks(config, options)
|
|
41
|
-
all = [configuration(config), framework(config), sources(config), tests(config), git, rails]
|
|
43
|
+
all = [configuration(config), framework(config), sources(config), tests(config), loading(config), git, rails]
|
|
44
|
+
all.concat(floor)
|
|
42
45
|
options[:check_baseline] ? all << baseline(config) : all
|
|
43
46
|
end
|
|
44
47
|
|
|
@@ -76,7 +79,7 @@ class Kimera::CLI::Doctor
|
|
|
76
79
|
def found(count, present, absent) = count.zero? ? ["✗", absent] : ["✓", present]
|
|
77
80
|
|
|
78
81
|
def git
|
|
79
|
-
return ["✓", "Git: incremental runs are available"] if File.
|
|
82
|
+
return ["✓", "Git: incremental runs are available"] if File.exist?(File.join(@root, ".git"))
|
|
80
83
|
["!", "Git: not a repository; `kimera changed` is unavailable"]
|
|
81
84
|
end
|
|
82
85
|
|
|
@@ -87,22 +90,28 @@ class Kimera::CLI::Doctor
|
|
|
87
90
|
|
|
88
91
|
def rails? = File.file?(File.join(@root, "config", "application.rb"))
|
|
89
92
|
|
|
90
|
-
def
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
93
|
+
def floor
|
|
94
|
+
helper = HELPERS.find { |path| floor?(File.join(@root, path)) }
|
|
95
|
+
helper ? [["!", floored(helper)]] : []
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
def floored(helper)
|
|
99
|
+
"Coverage floor: minimum_coverage in #{helper} fails partial --isolated runs; " \
|
|
100
|
+
"skip it when ENV[\"KIMERA\"] is set"
|
|
96
101
|
end
|
|
97
102
|
|
|
98
|
-
def
|
|
99
|
-
|
|
100
|
-
|
|
103
|
+
def floor?(path)
|
|
104
|
+
return false unless File.file?(path)
|
|
105
|
+
source = File.read(path)
|
|
106
|
+
source.include?("minimum_coverage") && !source.include?("KIMERA")
|
|
101
107
|
end
|
|
102
108
|
|
|
103
|
-
def
|
|
104
|
-
|
|
105
|
-
|
|
109
|
+
def baseline(config) = test_command(config).baseline
|
|
110
|
+
|
|
111
|
+
def loading(config) = test_command(config).loading
|
|
112
|
+
|
|
113
|
+
def test_command(config)
|
|
114
|
+
Kimera::CLI::TestCommand.new(config.fetch(:framework, "rspec").to_s, test_files(config), root: @root)
|
|
106
115
|
end
|
|
107
116
|
|
|
108
117
|
def minitest?(config) = config.fetch(:framework, "rspec") == "minitest"
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "open3"
|
|
4
|
+
|
|
5
|
+
class Kimera::CLI::TestCommand
|
|
6
|
+
LOADER = "ARGV.map { |f| File.expand_path(f) }.tap { ARGV.clear }.each { |f| require(f) }"
|
|
7
|
+
DRY_RUN = "require 'minitest'; Minitest.class_variable_set(:@@installed_at_exit, true); #{LOADER}; exit!(0)".freeze
|
|
8
|
+
|
|
9
|
+
def initialize(framework, files, root:)
|
|
10
|
+
@framework = framework
|
|
11
|
+
@files = files
|
|
12
|
+
@root = root
|
|
13
|
+
end
|
|
14
|
+
|
|
15
|
+
def baseline
|
|
16
|
+
check(run, "Baseline: configured test suite is green") { |failure| ["✗", "Baseline: #{advice(failure)}"] }
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
def loading
|
|
20
|
+
check(dry_run, "Test loading: every test file loads") do |failure|
|
|
21
|
+
failure ? ["✗", "Test loading: #{failure}"] : ["!", "Test loading: Bundler is unavailable; not checked"]
|
|
22
|
+
end
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
private
|
|
26
|
+
|
|
27
|
+
def advice(failure)
|
|
28
|
+
return "Bundler is unavailable; run your test suite, then retry" unless failure
|
|
29
|
+
"#{failure} (fix it, then rerun `kimera doctor --check-baseline`)"
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def check(command, passed)
|
|
33
|
+
output, status = Open3.capture2e(*command, chdir: @root)
|
|
34
|
+
status.success? ? ["✓", passed] : yield(summary(output))
|
|
35
|
+
rescue Errno::ENOENT
|
|
36
|
+
yield(nil)
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
def run = minitest? ? ruby(LOADER) : rspec
|
|
40
|
+
|
|
41
|
+
def dry_run = minitest? ? ruby(DRY_RUN) : rspec("--dry-run")
|
|
42
|
+
|
|
43
|
+
def summary(output)
|
|
44
|
+
output.lines.reverse.find { |line| line.match?(/\d+ failures?|\d+ examples?|failed|Error\b/i) }&.strip ||
|
|
45
|
+
"test command failed"
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def ruby(script) = ["bundle", "exec", "ruby", "-Itest", "-e", script, *@files]
|
|
49
|
+
|
|
50
|
+
def rspec(*flags) = ["bundle", "exec", "rspec", *flags, *@files]
|
|
51
|
+
|
|
52
|
+
def minitest? = @framework == "minitest"
|
|
53
|
+
end
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "json"
|
|
3
4
|
require "rspec/core"
|
|
4
5
|
require "stringio"
|
|
5
6
|
|
|
@@ -12,4 +13,11 @@ RSpec.configuration.output_stream = StringIO.new
|
|
|
12
13
|
RSpec.configuration.error_stream = StringIO.new
|
|
13
14
|
RSpec.configuration.deprecation_stream = StringIO.new
|
|
14
15
|
|
|
15
|
-
|
|
16
|
+
ledger, *locations = ARGV
|
|
17
|
+
code = RSpec::Core::Runner.run(locations)
|
|
18
|
+
|
|
19
|
+
failed = RSpec.world.all_examples.select { |example| example.execution_result.status == :failed }
|
|
20
|
+
outside = RSpec.world.non_example_failure ? 1 : 0
|
|
21
|
+
File.write(ledger, JSON.generate(failures: failed.size + outside, failing: failed.map(&:id)))
|
|
22
|
+
|
|
23
|
+
exit code
|
|
@@ -16,13 +16,16 @@ class Kimera::Execution::IsolatedPlan::ChildCommand
|
|
|
16
16
|
@test_files = test_files
|
|
17
17
|
end
|
|
18
18
|
|
|
19
|
-
|
|
20
|
-
|
|
19
|
+
ENV_FLAG = { "KIMERA" => "1" }.freeze
|
|
20
|
+
|
|
21
|
+
def command(mirror, locations, ledger:, paths:)
|
|
22
|
+
includes = paths.flat_map { |path| ["-I", path] }
|
|
23
|
+
bundled(mirror, ENV_FLAG.dup, ["ruby", *includes, "-I", helpers, *argv(locations, ledger)])
|
|
21
24
|
end
|
|
22
25
|
|
|
23
|
-
def argv(locations)
|
|
24
|
-
return [CHILD, *locations] unless minitest?
|
|
25
|
-
[MINITEST_CHILD, *@test_files, "--", *locations]
|
|
26
|
+
def argv(locations, ledger)
|
|
27
|
+
return [CHILD, ledger, *locations] unless minitest?
|
|
28
|
+
[MINITEST_CHILD, ledger, *@test_files, "--", *locations]
|
|
26
29
|
end
|
|
27
30
|
|
|
28
31
|
def helpers
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "json"
|
|
3
4
|
require "minitest"
|
|
4
5
|
|
|
5
6
|
Minitest.seed ||= 1
|
|
@@ -11,11 +12,13 @@ rescue NameError, ArgumentError
|
|
|
11
12
|
end
|
|
12
13
|
end
|
|
13
14
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
15
|
+
ledger, *rest = ARGV
|
|
16
|
+
split = rest.index("--") or abort("usage: <ledger> <test files...> -- <ids...>")
|
|
17
|
+
files = rest[0...split]
|
|
18
|
+
ids = rest[(split + 1)..]
|
|
19
|
+
ARGV.clear
|
|
17
20
|
|
|
18
|
-
files.each { |f|
|
|
21
|
+
files.each { |f| require File.expand_path(f) }
|
|
19
22
|
|
|
20
23
|
methods = {}
|
|
21
24
|
Minitest::Runnable.runnables.each do |runnable|
|
|
@@ -23,12 +26,14 @@ Minitest::Runnable.runnables.each do |runnable|
|
|
|
23
26
|
runnable.runnable_methods.each { |name| methods["#{runnable}##{name}"] = [runnable, name] }
|
|
24
27
|
end
|
|
25
28
|
|
|
26
|
-
|
|
27
|
-
ids.
|
|
29
|
+
killer =
|
|
30
|
+
ids.find do |id|
|
|
28
31
|
klass, name = methods[id]
|
|
29
32
|
next false unless klass
|
|
30
33
|
result = klass.new(name).run
|
|
31
34
|
!result.passed? && !result.skipped?
|
|
32
35
|
end
|
|
33
36
|
|
|
34
|
-
|
|
37
|
+
File.write(ledger, JSON.generate(failures: killer ? 1 : 0, failing: [killer].compact))
|
|
38
|
+
|
|
39
|
+
exit(killer ? 1 : 0)
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
5
|
+
module Kimera
|
|
6
|
+
module Execution
|
|
7
|
+
end
|
|
8
|
+
end
|
|
9
|
+
|
|
10
|
+
class Kimera::Execution::IsolatedOutcome
|
|
11
|
+
attr_reader :status, :failing, :detail
|
|
12
|
+
|
|
13
|
+
def initialize(status, failing = nil, detail = nil)
|
|
14
|
+
@status = status
|
|
15
|
+
@failing = failing
|
|
16
|
+
@detail = detail
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
class << self
|
|
20
|
+
def judge(exit, path)
|
|
21
|
+
return new(:timeout) if exit == :timeout
|
|
22
|
+
ledger = read(path)
|
|
23
|
+
return unreported(exit) unless ledger
|
|
24
|
+
tally(exit, ledger)
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
private
|
|
28
|
+
|
|
29
|
+
def unreported(exit) = new(exit.success? ? :survived : :killed)
|
|
30
|
+
|
|
31
|
+
def tally(exit, ledger)
|
|
32
|
+
failing = Array(ledger["failing"])
|
|
33
|
+
return new(:killed, failing) if ledger["failures"].positive?
|
|
34
|
+
return new(:survived) if exit.success?
|
|
35
|
+
new(:harness_error, nil, unexplained(exit))
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def unexplained(exit)
|
|
39
|
+
"suite exited #{exit.exitstatus || "on signal #{exit.termsig}"} with 0 failures " \
|
|
40
|
+
"(a coverage floor or exit hook? skip it when ENV[\"KIMERA\"] is set)"
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
def read(path)
|
|
44
|
+
ledger = JSON.parse(File.read(path))
|
|
45
|
+
ledger if ledger.is_a?(Hash) && ledger["failures"].is_a?(Integer)
|
|
46
|
+
rescue Errno::ENOENT, JSON::ParserError
|
|
47
|
+
nil
|
|
48
|
+
end
|
|
49
|
+
end
|
|
50
|
+
end
|
|
@@ -30,8 +30,8 @@ class Kimera::Execution::IsolatedPlan
|
|
|
30
30
|
selection.recorded(id)
|
|
31
31
|
end
|
|
32
32
|
|
|
33
|
-
def command(mirror, locations)
|
|
34
|
-
child.command(mirror, locations, paths: mutables)
|
|
33
|
+
def command(mirror, locations, ledger)
|
|
34
|
+
child.command(mirror, locations, ledger: ledger, paths: mutables)
|
|
35
35
|
end
|
|
36
36
|
|
|
37
37
|
def mutables
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "isolated_outcome"
|
|
4
|
+
|
|
3
5
|
module Kimera
|
|
4
6
|
module Execution
|
|
5
7
|
module IsolatedExecutionVerdict
|
|
@@ -17,8 +19,8 @@ module Kimera
|
|
|
17
19
|
trial.file = point.file
|
|
18
20
|
tests = plan.tests(trial.id, point)
|
|
19
21
|
return result(trial, :no_coverage, nil) unless tests
|
|
20
|
-
|
|
21
|
-
result(trial, status, duration, tests)
|
|
22
|
+
outcome, duration = measure(trial, tests)
|
|
23
|
+
result(trial, outcome.status, duration, tests, failing_tests: outcome.failing, detail: outcome.detail)
|
|
22
24
|
end
|
|
23
25
|
|
|
24
26
|
def measure(trial, tests)
|
|
@@ -34,10 +36,15 @@ module Kimera
|
|
|
34
36
|
end
|
|
35
37
|
|
|
36
38
|
def verdict(mirror, locations)
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
39
|
+
Dir.mktmpdir("kimera-ledger") do |dir|
|
|
40
|
+
ledger = File.join(dir, "ledger.json")
|
|
41
|
+
IsolatedOutcome.judge(launch(mirror, locations, ledger), ledger)
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def launch(mirror, locations, ledger)
|
|
46
|
+
env, cmd = plan.command(mirror, locations, ledger)
|
|
47
|
+
waitfor(Process.spawn(env, *cmd, chdir: mirror, pgroup: true, out: File::NULL, err: File::NULL))
|
|
41
48
|
end
|
|
42
49
|
|
|
43
50
|
def with_mirror
|
|
@@ -54,9 +61,9 @@ module Kimera
|
|
|
54
61
|
end
|
|
55
62
|
end
|
|
56
63
|
|
|
57
|
-
def result(trial, status, duration, cover = nil)
|
|
64
|
+
def result(trial, status, duration, cover = nil, **outcome)
|
|
58
65
|
MutantResult.new(
|
|
59
|
-
mutant_id: trial.id, status: status, file: trial.file, duration: duration, covering_tests: cover
|
|
66
|
+
mutant_id: trial.id, status: status, file: trial.file, duration: duration, covering_tests: cover, **outcome
|
|
60
67
|
)
|
|
61
68
|
end
|
|
62
69
|
|
|
@@ -34,9 +34,16 @@ class Kimera::Execution::ParallelTestDatabases
|
|
|
34
34
|
|
|
35
35
|
def cleanup
|
|
36
36
|
return unless active?
|
|
37
|
+
scrub
|
|
37
38
|
ActiveSupport::Testing::Parallelization.run_cleanup_hooks.each(&:call)
|
|
38
39
|
end
|
|
39
40
|
|
|
41
|
+
def scrub
|
|
42
|
+
return unless defined?(ActiveRecord::Base)
|
|
43
|
+
connection = ActiveRecord::Base.connection
|
|
44
|
+
connection.truncate_tables(*connection.tables)
|
|
45
|
+
end
|
|
46
|
+
|
|
40
47
|
def errors = @errors || $stderr
|
|
41
48
|
|
|
42
49
|
def eligible?
|
|
@@ -27,11 +27,20 @@ class Kimera::Execution::Schemata
|
|
|
27
27
|
|
|
28
28
|
def overlay!
|
|
29
29
|
Kimera::Execution::OverlayGuards.install!
|
|
30
|
-
@registry.files
|
|
30
|
+
loaded = weave(@registry.files)
|
|
31
|
+
(loaded + weave(unresolved)).tap { announce }
|
|
31
32
|
end
|
|
32
33
|
|
|
33
34
|
private
|
|
34
35
|
|
|
36
|
+
def weave(paths) = paths.flat_map { |path| load(path) }
|
|
37
|
+
|
|
38
|
+
def unresolved
|
|
39
|
+
@skipped.keys.select { |path| @skipped[path].start_with?("NameError:") }.each { |path| @skipped.delete(path) }
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def announce = @skipped.each { |path, reason| notice(path, reason) }
|
|
43
|
+
|
|
35
44
|
def load(path)
|
|
36
45
|
file = File.join(root, path)
|
|
37
46
|
return [] unless File.file?(file)
|
|
@@ -44,7 +53,7 @@ class Kimera::Execution::Schemata
|
|
|
44
53
|
return [] if result.mutant_ids.empty?
|
|
45
54
|
apply(result, file)
|
|
46
55
|
rescue StandardError, ScriptError, SystemExit => error
|
|
47
|
-
skip(path, error)
|
|
56
|
+
skip(path, "#{error.class}: #{error.message}")
|
|
48
57
|
end
|
|
49
58
|
|
|
50
59
|
def synthesize(path, file)
|
|
@@ -55,10 +64,8 @@ class Kimera::Execution::Schemata
|
|
|
55
64
|
silence { result.install(file) }
|
|
56
65
|
end
|
|
57
66
|
|
|
58
|
-
def skip(path,
|
|
59
|
-
reason = "#{error.class}: #{error.message}"
|
|
67
|
+
def skip(path, reason)
|
|
60
68
|
@skipped[path] = reason
|
|
61
|
-
notice(path, reason)
|
|
62
69
|
[]
|
|
63
70
|
end
|
|
64
71
|
|
|
@@ -79,4 +79,17 @@ class Kimera::Frameworks::Adapter
|
|
|
79
79
|
end
|
|
80
80
|
end
|
|
81
81
|
end
|
|
82
|
+
|
|
83
|
+
private
|
|
84
|
+
|
|
85
|
+
def each_test_file(files, &)
|
|
86
|
+
ARGV.clear
|
|
87
|
+
Array(files).each { |file| load_test_file(file, &) }
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def load_test_file(file)
|
|
91
|
+
yield(File.expand_path(file))
|
|
92
|
+
rescue StandardError, ScriptError => error
|
|
93
|
+
raise(Kimera::Error, "cannot load test file #{file} (#{error.class}: #{error.message})")
|
|
94
|
+
end
|
|
82
95
|
end
|
|
@@ -11,28 +11,42 @@ class Kimera::Report::Actions
|
|
|
11
11
|
end
|
|
12
12
|
|
|
13
13
|
def show(report, path: nil)
|
|
14
|
-
|
|
14
|
+
hints = path ? SavedHints.new(path) : UnsavedHints.new
|
|
15
|
+
lines = [
|
|
16
|
+
advice(report.survived, hints, :survivors), advice(report.uncovered, hints, :uncovered),
|
|
17
|
+
advice(report.errors, hints, :unjudged)
|
|
18
|
+
].compact
|
|
15
19
|
@io.puts("", "Next actions:", *lines) unless lines.empty?
|
|
16
20
|
end
|
|
17
21
|
|
|
18
22
|
private
|
|
19
23
|
|
|
20
|
-
def
|
|
24
|
+
def advice(found, hints, kind)
|
|
21
25
|
first = found.min_by(&:mutant_id)
|
|
22
|
-
|
|
26
|
+
" #{found.size} #{hints.public_send(kind, first.mutant_id)}" if first
|
|
23
27
|
end
|
|
24
28
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
29
|
+
class SavedHints
|
|
30
|
+
def initialize(path)
|
|
31
|
+
@path = path
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def survivors(id) = "surviving mutant(s): inspect ##{id} with `#{mutant(id)}`"
|
|
35
|
+
|
|
36
|
+
def uncovered(_id) = "uncovered mutant(s): kimera report #{@path} --status no_coverage"
|
|
28
37
|
|
|
29
|
-
|
|
38
|
+
def unjudged(id) = "unjudged mutant(s): see why with `#{mutant(id)}`"
|
|
39
|
+
|
|
40
|
+
private
|
|
41
|
+
|
|
42
|
+
def mutant(id) = "kimera mutant #{id} --report #{@path}"
|
|
43
|
+
end
|
|
30
44
|
|
|
31
|
-
|
|
45
|
+
class UnsavedHints
|
|
46
|
+
def survivors(id) = "surviving mutant(s): inspect ##{id} with `kimera run --report tmp/kimera.json`"
|
|
32
47
|
|
|
33
|
-
|
|
48
|
+
def uncovered(_id) = "uncovered mutant(s): rerun with --report tmp/kimera.json"
|
|
34
49
|
|
|
35
|
-
|
|
36
|
-
" #{found.size} #{text}" unless found.empty?
|
|
50
|
+
def unjudged(_id) = "unjudged mutant(s): retry with `kimera run --isolated`"
|
|
37
51
|
end
|
|
38
52
|
end
|
|
@@ -45,6 +45,6 @@ module Kimera::Report::Sections
|
|
|
45
45
|
return if leaks.empty?
|
|
46
46
|
@io.puts
|
|
47
47
|
@io.puts(yellow("State-leak warnings (#{leaks.size}):"))
|
|
48
|
-
leaks.each { |leak| @io.puts(" - #{leak.detail}") }
|
|
48
|
+
leaks.each { |leak| @io.puts(" - ##{leak.mutant_id} #{leak.detail}") }
|
|
49
49
|
end
|
|
50
50
|
end
|
|
@@ -8,6 +8,7 @@ require_relative "../error"
|
|
|
8
8
|
require_relative "../registry/registry"
|
|
9
9
|
require_relative "guard_weaver"
|
|
10
10
|
require_relative "source_map"
|
|
11
|
+
require_relative "superclass_pin"
|
|
11
12
|
|
|
12
13
|
class Kimera::Overlay
|
|
13
14
|
Result =
|
|
@@ -24,7 +25,7 @@ class Kimera::Overlay
|
|
|
24
25
|
|
|
25
26
|
class << self
|
|
26
27
|
def evaluate(source, path)
|
|
27
|
-
Kimera::Warnings.silence { TOPLEVEL_BINDING.eval(source, path) }
|
|
28
|
+
Kimera::Warnings.silence { TOPLEVEL_BINDING.eval(Kimera::SuperclassPin.pin(source), path) }
|
|
28
29
|
end
|
|
29
30
|
end
|
|
30
31
|
|
|
@@ -20,8 +20,8 @@ module Kimera
|
|
|
20
20
|
def gather
|
|
21
21
|
edits = []
|
|
22
22
|
dropped = []
|
|
23
|
-
each { |definition, points| collect(definition, points, edits, dropped) }
|
|
24
|
-
[edits, dropped]
|
|
23
|
+
claimed = each { |definition, points| collect(definition, points, edits, dropped) }
|
|
24
|
+
[edits, dropped + (@safe - claimed)]
|
|
25
25
|
end
|
|
26
26
|
|
|
27
27
|
def collect(definition, points, edits, dropped)
|
|
@@ -60,6 +60,7 @@ module Kimera
|
|
|
60
60
|
def each(&)
|
|
61
61
|
claimed = []
|
|
62
62
|
definitions.each { |definition| visit(definition, claimed, &) }
|
|
63
|
+
claimed
|
|
63
64
|
end
|
|
64
65
|
|
|
65
66
|
def visit(definition, claimed)
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "prism"
|
|
4
|
+
|
|
5
|
+
module Kimera
|
|
6
|
+
module SuperclassPin
|
|
7
|
+
module_function
|
|
8
|
+
|
|
9
|
+
def pin(source)
|
|
10
|
+
classes(Prism.parse(source).value).reverse_each.with_object(source.b) do |node, text|
|
|
11
|
+
location = node.superclass.location
|
|
12
|
+
text[location.start_offset, location.length] = reopened(node)
|
|
13
|
+
end.force_encoding(source.encoding)
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def existing(scope, name)
|
|
17
|
+
owner = scope.is_a?(Module) ? scope : Object
|
|
18
|
+
inherit = false
|
|
19
|
+
owner.const_defined?(name, inherit) && owner.const_get(name, inherit).superclass
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
def reopened(node)
|
|
23
|
+
"(::Kimera::SuperclassPin.existing(self, :#{node.constant_path.name}) || #{node.superclass.slice})".b
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def classes(node, found = [])
|
|
27
|
+
found << node if dynamic?(node)
|
|
28
|
+
node.compact_child_nodes.each { |child| classes(child, found) }
|
|
29
|
+
found
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def dynamic?(node)
|
|
33
|
+
return false unless node.is_a?(Prism::ClassNode) && node.constant_path.is_a?(Prism::ConstantReadNode)
|
|
34
|
+
superclass = node.superclass
|
|
35
|
+
superclass && !constant?(superclass)
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def constant?(node) = node.is_a?(Prism::ConstantReadNode) || node.is_a?(Prism::ConstantPathNode)
|
|
39
|
+
end
|
|
40
|
+
end
|
data/skills/kimera/SKILL.md
CHANGED
|
@@ -91,13 +91,19 @@ bundle exec kimera run --isolated --jobs 4 # oracle mode (see Strengthen)
|
|
|
91
91
|
|
|
92
92
|
## Ignore discipline: hard rules (especially for agents)
|
|
93
93
|
|
|
94
|
-
- **
|
|
95
|
-
|
|
94
|
+
- **Raise `max_ignored` only for an entry that has earned it**: survival
|
|
95
|
+
under `--isolated`, a `reason:` naming the mechanism, and the raise in the
|
|
96
|
+
same diff as the entry. Any triager, human or agent, may make that call when
|
|
97
|
+
all three hold. Never raise it to make a failing gate pass; that is skipping
|
|
98
|
+
triage, not finishing it.
|
|
99
|
+
- If any criterion is missing (no oracle run, a reason that restates the
|
|
100
|
+
verdict, a mutant you haven't read), write the test or leave the survivor
|
|
101
|
+
visible.
|
|
96
102
|
- An ignore entry without an isolated-oracle survival check is inadmissible.
|
|
97
103
|
Warm-path survivors can be measurement artifacts (kimera's own suite once
|
|
98
104
|
had a "clearly equivalent" mutant that 11 specs actually kill).
|
|
99
|
-
-
|
|
100
|
-
|
|
105
|
+
- Reviewers judge the `reason:`, not the number, so the entry and the raise
|
|
106
|
+
land together.
|
|
101
107
|
|
|
102
108
|
## Strengthen
|
|
103
109
|
|
|
@@ -110,6 +116,11 @@ bundle exec kimera run --isolated --jobs 4 # oracle mode (see Strengthen)
|
|
|
110
116
|
runner, global state the suite also touches, kimera's own plumbing):
|
|
111
117
|
adjudicate with `--isolated`. Suites that manipulate shared globals can
|
|
112
118
|
suppress the warm path.
|
|
119
|
+
- `--isolated` judges on failed tests, not the exit status. A child that
|
|
120
|
+
exits non-zero with zero failures (typically a SimpleCov `minimum_coverage`
|
|
121
|
+
floor tripped by the partial run) is `harness_error`, not a kill. Children
|
|
122
|
+
run with `KIMERA=1`, so gate the floor on it (`minimum_coverage ... unless
|
|
123
|
+
ENV["KIMERA"]`); `kimera doctor` warns when it isn't.
|
|
113
124
|
- A cluster of `error`/`timeout` verdicts in one region usually means harness
|
|
114
125
|
fragility or a missing guard, not test strength. Read the cluster before
|
|
115
126
|
counting the detections.
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: kimera
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Giacomo GK
|
|
@@ -95,6 +95,7 @@ files:
|
|
|
95
95
|
- lib/kimera/cli/survivors.rb
|
|
96
96
|
- lib/kimera/cli/survivors/panel.rb
|
|
97
97
|
- lib/kimera/cli/synthesize.rb
|
|
98
|
+
- lib/kimera/cli/test_command.rb
|
|
98
99
|
- lib/kimera/cli/workflows.rb
|
|
99
100
|
- lib/kimera/error.rb
|
|
100
101
|
- lib/kimera/execution/baseline_failure.rb
|
|
@@ -108,6 +109,7 @@ files:
|
|
|
108
109
|
- lib/kimera/execution/isolated_child.rb
|
|
109
110
|
- lib/kimera/execution/isolated_child_command.rb
|
|
110
111
|
- lib/kimera/execution/isolated_child_minitest.rb
|
|
112
|
+
- lib/kimera/execution/isolated_outcome.rb
|
|
111
113
|
- lib/kimera/execution/isolated_plan.rb
|
|
112
114
|
- lib/kimera/execution/isolated_scheduling.rb
|
|
113
115
|
- lib/kimera/execution/isolated_test_selection.rb
|
|
@@ -229,6 +231,7 @@ files:
|
|
|
229
231
|
- lib/kimera/synthesis/overlay_splice.rb
|
|
230
232
|
- lib/kimera/synthesis/project.rb
|
|
231
233
|
- lib/kimera/synthesis/source_map.rb
|
|
234
|
+
- lib/kimera/synthesis/superclass_pin.rb
|
|
232
235
|
- lib/kimera/version.rb
|
|
233
236
|
- skills/kimera/SKILL.md
|
|
234
237
|
homepage: https://github.com/giacope/kimera
|