kimera 0.1.0 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +34 -0
- data/README.md +20 -3
- data/lib/kimera/cli/doctor.rb +24 -15
- data/lib/kimera/cli/run/cycle.rb +3 -1
- data/lib/kimera/cli/survivors.rb +1 -1
- data/lib/kimera/cli/test_command.rb +54 -0
- data/lib/kimera/execution/child_process.rb +1 -1
- data/lib/kimera/execution/harness.rb +1 -1
- data/lib/kimera/execution/isolated_child.rb +9 -1
- data/lib/kimera/execution/isolated_child_command.rb +8 -5
- data/lib/kimera/execution/isolated_child_minitest.rb +12 -7
- data/lib/kimera/execution/isolated_outcome.rb +50 -0
- data/lib/kimera/execution/isolated_plan.rb +2 -2
- data/lib/kimera/execution/isolated_verdict.rb +15 -8
- data/lib/kimera/execution/parallel_test_databases.rb +7 -0
- data/lib/kimera/execution/schemata.rb +18 -6
- data/lib/kimera/execution/suite_env.rb +7 -0
- data/lib/kimera/execution/verdicts.rb +4 -2
- data/lib/kimera/frameworks/adapter.rb +13 -0
- data/lib/kimera/frameworks/minitest_adapter.rb +1 -1
- data/lib/kimera/frameworks/rspec_adapter.rb +1 -1
- data/lib/kimera/incremental/git_diff.rb +3 -1
- data/lib/kimera/registry/mutation_point.rb +14 -1
- data/lib/kimera/report/actions.rb +25 -11
- data/lib/kimera/report/sections.rb +1 -1
- data/lib/kimera/results/result.rb +3 -2
- data/lib/kimera/results/run_report.rb +1 -1
- data/lib/kimera/support/unparse.rb +27 -0
- data/lib/kimera/support/version.rb +1 -1
- data/lib/kimera/synthesis/file_weave.rb +5 -4
- data/lib/kimera/synthesis/overlay.rb +2 -1
- data/lib/kimera/synthesis/overlay_splice.rb +10 -4
- data/lib/kimera/synthesis/splice_reopening.rb +57 -0
- data/lib/kimera/synthesis/superclass_pin.rb +40 -0
- data/skills/kimera/SKILL.md +18 -5
- metadata +6 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 042a730e48edfef4744060425cd10f04e34006025af3f36db3ab17f0be59e0bc
|
|
4
|
+
data.tar.gz: 97a3914ca3f803f01e68564af89ee07667703aadfa6ec74269bbf37470add0fb
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 2c2d774a35384621bca26d92d5c9e77550f3b08ecf1304019b37e695eeacbbc5566622de50fbae65077f61035fa25e241817aa31e917c0455e8ad23b26c00f9b
|
|
7
|
+
data.tar.gz: 7ebd3017fd0edbd7a2abb3c585ee347dfe0b66a03111ffa9efb322b835b6661d29bd2554f863bc075035f242da3bf66f514f098ebaf2b64339b39b90f8dde558
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,39 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.1.2 (2026-09-26)
|
|
4
|
+
|
|
5
|
+
- Kimera sets `KIMERA=1` in its own process before loading the suite, and in
|
|
6
|
+
the `kimera doctor` test commands. A coverage floor gated on
|
|
7
|
+
`ENV["KIMERA"]` no longer fails a passing warm run with exit 2 or blocks
|
|
8
|
+
doctor's dry run. (#6)
|
|
9
|
+
- A string interpolating a local bound by a pattern (`in [:ok, chosen]`,
|
|
10
|
+
`=> chosen`), a `&block` parameter, or a regexp named capture no longer
|
|
11
|
+
sinks its file's mutants. unparser 0.9 could not round-trip it; Kimera now
|
|
12
|
+
teaches unparser those bindings. (#7)
|
|
13
|
+
- The per-method splice fallback now claims mutants in an endless method
|
|
14
|
+
(`def big? = value > 10`), and reopens `Data.define` / `Struct.new`
|
|
15
|
+
constants instead of redefining them, so spliced guards are reached. (#7)
|
|
16
|
+
- Mutants Kimera could not instrument are reported as `unmutatable`, with the
|
|
17
|
+
reason, instead of `no_coverage`. They never gate and stay out of the score,
|
|
18
|
+
so `--fail-on-no-coverage` again means "no test executes this". (#7)
|
|
19
|
+
- `kimera run --since` no longer crashes with "invalid byte sequence in
|
|
20
|
+
US-ASCII" when the locale isn't UTF-8 (e.g. `LANG` unset) and the diff
|
|
21
|
+
contains non-ASCII text. Git's diff output is read as UTF-8, with invalid
|
|
22
|
+
bytes replaced. (#9)
|
|
23
|
+
|
|
24
|
+
## 0.1.1 (2026-09-26)
|
|
25
|
+
|
|
26
|
+
- Fix a boot crash on case-sensitive filesystems (Linux): `require "English"`
|
|
27
|
+
was spelled `require "english"`.
|
|
28
|
+
- `--isolated` judges a mutant on failed tests, not the child's exit status.
|
|
29
|
+
A child that exits non-zero with zero failures (e.g. a SimpleCov
|
|
30
|
+
`minimum_coverage` floor tripped by the partial run) is now `harness_error`
|
|
31
|
+
instead of a false kill. Kills carry their failing tests. Children run with
|
|
32
|
+
`KIMERA=1`, and `kimera doctor` warns about an ungated coverage floor.
|
|
33
|
+
- Skill: raising `max_ignored` is a judgment any triager may make when the
|
|
34
|
+
entry has earned it (isolated survival, a mechanism `reason:`, the raise in
|
|
35
|
+
the same diff), never a way to make a failing gate pass.
|
|
36
|
+
|
|
3
37
|
## 0.1.0 (2026-09-26)
|
|
4
38
|
|
|
5
39
|
Initial public release.
|
data/README.md
CHANGED
|
@@ -218,6 +218,19 @@ process with the code under test. On ordinary code it reaches the same verdicts
|
|
|
218
218
|
as the warm path. It can also test what the warm path can't: the runtime
|
|
219
219
|
selector and Kimera's own harness-critical files (see below).
|
|
220
220
|
|
|
221
|
+
An isolated child is judged on its failed tests, not its exit status. A child
|
|
222
|
+
that exits non-zero with no failing test is `harness_error` (unjudged, gated
|
|
223
|
+
by `--max-errors`), not a kill. The usual cause is a coverage floor such as
|
|
224
|
+
SimpleCov's `minimum_coverage`, which a partial run always trips. Kimera sets
|
|
225
|
+
`KIMERA=1` wherever it loads your suite (the warm process, `--isolated`
|
|
226
|
+
children, and `kimera doctor`'s test commands), so skip the floor there:
|
|
227
|
+
|
|
228
|
+
```ruby
|
|
229
|
+
SimpleCov.start { minimum_coverage(line: 100, branch: 100) unless ENV["KIMERA"] }
|
|
230
|
+
```
|
|
231
|
+
|
|
232
|
+
`kimera doctor` warns when it finds an ungated `minimum_coverage`.
|
|
233
|
+
|
|
221
234
|
### Progress output
|
|
222
235
|
|
|
223
236
|
During a run, a progress bar tracks each phase on stderr. It shows only when
|
|
@@ -260,7 +273,8 @@ fail_on_no_coverage: false
|
|
|
260
273
|
|
|
261
274
|
# The gate fails when more than this many mutants are ignore-listed. Marking
|
|
262
275
|
# one more mutant equivalent means raising this number in the same diff, so
|
|
263
|
-
# the ignore list stays a
|
|
276
|
+
# the ignore list stays a justified decision, reviewed in the diff that makes
|
|
277
|
+
# it, not a shortcut past a test.
|
|
264
278
|
max_ignored: 1
|
|
265
279
|
|
|
266
280
|
# Accepted pre-adoption survivors, merged with the ignore rules below.
|
|
@@ -491,8 +505,11 @@ source ──Prism──▶ registry (mutation points, JSON) ──┬─▶ syn
|
|
|
491
505
|
baseline, or worse, as kills that are really collisions.
|
|
492
506
|
- **Score definition.** `score = killed / evaluable`.
|
|
493
507
|
- Killed includes timeouts and errors (observable misbehaviour).
|
|
494
|
-
- Evaluable excludes `no_coverage`, `ignored`, `isolated_only`,
|
|
495
|
-
`harness_error` mutants. Each is reported separately.
|
|
508
|
+
- Evaluable excludes `no_coverage`, `ignored`, `isolated_only`,
|
|
509
|
+
`unmutatable`, and `harness_error` mutants. Each is reported separately.
|
|
510
|
+
- An `unmutatable` mutant is one Kimera could not instrument for the warm
|
|
511
|
+
run (its file or method failed to re-emit or load); the detail says why.
|
|
512
|
+
It never gates, and it is not `no_coverage`: a test may well run it.
|
|
496
513
|
- A `harness_error` is a mutant Kimera could not judge: its worker died, or
|
|
497
514
|
its reply was unreadable. It is not a kill. It gates via `--max-errors`
|
|
498
515
|
(default 0). A pool that never produces a result aborts the run.
|
data/lib/kimera/cli/doctor.rb
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
-
require "open3"
|
|
4
3
|
require_relative "../error"
|
|
5
4
|
require_relative "../scope/config"
|
|
6
5
|
require_relative "../scope/file_set"
|
|
7
6
|
require_relative "flag"
|
|
7
|
+
require_relative "test_command"
|
|
8
8
|
|
|
9
9
|
class Kimera::CLI::Doctor
|
|
10
10
|
OPTIONS = Kimera::FlagTable.new(
|
|
@@ -16,6 +16,8 @@ class Kimera::CLI::Doctor
|
|
|
16
16
|
|
|
17
17
|
FRAMEWORK_FEATURES = { "rspec" => "rspec/core", "minitest" => "minitest" }.freeze
|
|
18
18
|
|
|
19
|
+
HELPERS = %w[.simplecov spec/spec_helper.rb spec/rails_helper.rb test/test_helper.rb].freeze
|
|
20
|
+
|
|
19
21
|
def initialize(io: $stdout, errors: $stderr, root: ".")
|
|
20
22
|
@io = io
|
|
21
23
|
@errors = errors
|
|
@@ -38,7 +40,8 @@ class Kimera::CLI::Doctor
|
|
|
38
40
|
end
|
|
39
41
|
|
|
40
42
|
def checks(config, options)
|
|
41
|
-
all = [configuration(config), framework(config), sources(config), tests(config), git, rails]
|
|
43
|
+
all = [configuration(config), framework(config), sources(config), tests(config), loading(config), git, rails]
|
|
44
|
+
all.concat(floor)
|
|
42
45
|
options[:check_baseline] ? all << baseline(config) : all
|
|
43
46
|
end
|
|
44
47
|
|
|
@@ -76,7 +79,7 @@ class Kimera::CLI::Doctor
|
|
|
76
79
|
def found(count, present, absent) = count.zero? ? ["✗", absent] : ["✓", present]
|
|
77
80
|
|
|
78
81
|
def git
|
|
79
|
-
return ["✓", "Git: incremental runs are available"] if File.
|
|
82
|
+
return ["✓", "Git: incremental runs are available"] if File.exist?(File.join(@root, ".git"))
|
|
80
83
|
["!", "Git: not a repository; `kimera changed` is unavailable"]
|
|
81
84
|
end
|
|
82
85
|
|
|
@@ -87,22 +90,28 @@ class Kimera::CLI::Doctor
|
|
|
87
90
|
|
|
88
91
|
def rails? = File.file?(File.join(@root, "config", "application.rb"))
|
|
89
92
|
|
|
90
|
-
def
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
93
|
+
def floor
|
|
94
|
+
helper = HELPERS.find { |path| floor?(File.join(@root, path)) }
|
|
95
|
+
helper ? [["!", floored(helper)]] : []
|
|
96
|
+
end
|
|
97
|
+
|
|
98
|
+
def floored(helper)
|
|
99
|
+
"Coverage floor: minimum_coverage in #{helper} fails Kimera's partial runs; " \
|
|
100
|
+
"skip it when ENV[\"KIMERA\"] is set (Kimera sets it)"
|
|
96
101
|
end
|
|
97
102
|
|
|
98
|
-
def
|
|
99
|
-
|
|
100
|
-
|
|
103
|
+
def floor?(path)
|
|
104
|
+
return false unless File.file?(path)
|
|
105
|
+
source = File.read(path)
|
|
106
|
+
source.include?("minimum_coverage") && !source.include?("KIMERA")
|
|
101
107
|
end
|
|
102
108
|
|
|
103
|
-
def
|
|
104
|
-
|
|
105
|
-
|
|
109
|
+
def baseline(config) = test_command(config).baseline
|
|
110
|
+
|
|
111
|
+
def loading(config) = test_command(config).loading
|
|
112
|
+
|
|
113
|
+
def test_command(config)
|
|
114
|
+
Kimera::CLI::TestCommand.new(config.fetch(:framework, "rspec").to_s, test_files(config), root: @root)
|
|
106
115
|
end
|
|
107
116
|
|
|
108
117
|
def minitest?(config) = config.fetch(:framework, "rspec") == "minitest"
|
data/lib/kimera/cli/run/cycle.rb
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "../../execution/suite_env"
|
|
3
4
|
require_relative "../../incremental/selection"
|
|
4
5
|
require_relative "../../incremental/session"
|
|
5
6
|
require_relative "digest"
|
|
@@ -71,7 +72,8 @@ class Kimera::CLI::Run::Cycle
|
|
|
71
72
|
.transform_keys(&:to_s)
|
|
72
73
|
end
|
|
73
74
|
|
|
74
|
-
def harness(remaining, loaded)
|
|
75
|
+
def harness(remaining, loaded, env: ENV)
|
|
76
|
+
env.update(Kimera::Execution::SUITE_ENV)
|
|
75
77
|
adapter = Kimera::Frameworks::Adapter.load(@options[:framework])
|
|
76
78
|
Kimera::CLI::Run::Pass.new(@registry, @options, adapter).call(remaining, loaded)
|
|
77
79
|
end
|
data/lib/kimera/cli/survivors.rb
CHANGED
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "open3"
|
|
4
|
+
require_relative "../execution/suite_env"
|
|
5
|
+
|
|
6
|
+
class Kimera::CLI::TestCommand
|
|
7
|
+
LOADER = "ARGV.map { |f| File.expand_path(f) }.tap { ARGV.clear }.each { |f| require(f) }"
|
|
8
|
+
DRY_RUN = "require 'minitest'; Minitest.class_variable_set(:@@installed_at_exit, true); #{LOADER}; exit!(0)".freeze
|
|
9
|
+
|
|
10
|
+
def initialize(framework, files, root:)
|
|
11
|
+
@framework = framework
|
|
12
|
+
@files = files
|
|
13
|
+
@root = root
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def baseline
|
|
17
|
+
check(run, "Baseline: configured test suite is green") { |failure| ["✗", "Baseline: #{advice(failure)}"] }
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def loading
|
|
21
|
+
check(dry_run, "Test loading: every test file loads") do |failure|
|
|
22
|
+
failure ? ["✗", "Test loading: #{failure}"] : ["!", "Test loading: Bundler is unavailable; not checked"]
|
|
23
|
+
end
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
private
|
|
27
|
+
|
|
28
|
+
def advice(failure)
|
|
29
|
+
return "Bundler is unavailable; run your test suite, then retry" unless failure
|
|
30
|
+
"#{failure} (fix it, then rerun `kimera doctor --check-baseline`)"
|
|
31
|
+
end
|
|
32
|
+
|
|
33
|
+
def check(command, passed)
|
|
34
|
+
output, status = Open3.capture2e(Kimera::Execution::SUITE_ENV, *command, chdir: @root)
|
|
35
|
+
status.success? ? ["✓", passed] : yield(summary(output))
|
|
36
|
+
rescue Errno::ENOENT
|
|
37
|
+
yield(nil)
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
def run = minitest? ? ruby(LOADER) : rspec
|
|
41
|
+
|
|
42
|
+
def dry_run = minitest? ? ruby(DRY_RUN) : rspec("--dry-run")
|
|
43
|
+
|
|
44
|
+
def summary(output)
|
|
45
|
+
output.lines.reverse.find { |line| line.match?(/\d+ failures?|\d+ examples?|failed|Error\b/i) }&.strip ||
|
|
46
|
+
"test command failed"
|
|
47
|
+
end
|
|
48
|
+
|
|
49
|
+
def ruby(script) = ["bundle", "exec", "ruby", "-Itest", "-e", script, *@files]
|
|
50
|
+
|
|
51
|
+
def rspec(*flags) = ["bundle", "exec", "rspec", *flags, *@files]
|
|
52
|
+
|
|
53
|
+
def minitest? = @framework == "minitest"
|
|
54
|
+
end
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "json"
|
|
3
4
|
require "rspec/core"
|
|
4
5
|
require "stringio"
|
|
5
6
|
|
|
@@ -12,4 +13,11 @@ RSpec.configuration.output_stream = StringIO.new
|
|
|
12
13
|
RSpec.configuration.error_stream = StringIO.new
|
|
13
14
|
RSpec.configuration.deprecation_stream = StringIO.new
|
|
14
15
|
|
|
15
|
-
|
|
16
|
+
ledger, *locations = ARGV
|
|
17
|
+
code = RSpec::Core::Runner.run(locations)
|
|
18
|
+
|
|
19
|
+
failed = RSpec.world.all_examples.select { |example| example.execution_result.status == :failed }
|
|
20
|
+
outside = RSpec.world.non_example_failure ? 1 : 0
|
|
21
|
+
File.write(ledger, JSON.generate(failures: failed.size + outside, failing: failed.map(&:id)))
|
|
22
|
+
|
|
23
|
+
exit code
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "suite_env"
|
|
4
|
+
|
|
3
5
|
module Kimera
|
|
4
6
|
module Execution
|
|
5
7
|
end
|
|
@@ -16,13 +18,14 @@ class Kimera::Execution::IsolatedPlan::ChildCommand
|
|
|
16
18
|
@test_files = test_files
|
|
17
19
|
end
|
|
18
20
|
|
|
19
|
-
def command(mirror, locations, paths:)
|
|
20
|
-
|
|
21
|
+
def command(mirror, locations, ledger:, paths:)
|
|
22
|
+
includes = paths.flat_map { |path| ["-I", path] }
|
|
23
|
+
bundled(mirror, Kimera::Execution::SUITE_ENV.dup, ["ruby", *includes, "-I", helpers, *argv(locations, ledger)])
|
|
21
24
|
end
|
|
22
25
|
|
|
23
|
-
def argv(locations)
|
|
24
|
-
return [CHILD, *locations] unless minitest?
|
|
25
|
-
[MINITEST_CHILD, *@test_files, "--", *locations]
|
|
26
|
+
def argv(locations, ledger)
|
|
27
|
+
return [CHILD, ledger, *locations] unless minitest?
|
|
28
|
+
[MINITEST_CHILD, ledger, *@test_files, "--", *locations]
|
|
26
29
|
end
|
|
27
30
|
|
|
28
31
|
def helpers
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "json"
|
|
3
4
|
require "minitest"
|
|
4
5
|
|
|
5
6
|
Minitest.seed ||= 1
|
|
@@ -11,11 +12,13 @@ rescue NameError, ArgumentError
|
|
|
11
12
|
end
|
|
12
13
|
end
|
|
13
14
|
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
15
|
+
ledger, *rest = ARGV
|
|
16
|
+
split = rest.index("--") or abort("usage: <ledger> <test files...> -- <ids...>")
|
|
17
|
+
files = rest[0...split]
|
|
18
|
+
ids = rest[(split + 1)..]
|
|
19
|
+
ARGV.clear
|
|
17
20
|
|
|
18
|
-
files.each { |f|
|
|
21
|
+
files.each { |f| require File.expand_path(f) }
|
|
19
22
|
|
|
20
23
|
methods = {}
|
|
21
24
|
Minitest::Runnable.runnables.each do |runnable|
|
|
@@ -23,12 +26,14 @@ Minitest::Runnable.runnables.each do |runnable|
|
|
|
23
26
|
runnable.runnable_methods.each { |name| methods["#{runnable}##{name}"] = [runnable, name] }
|
|
24
27
|
end
|
|
25
28
|
|
|
26
|
-
|
|
27
|
-
ids.
|
|
29
|
+
killer =
|
|
30
|
+
ids.find do |id|
|
|
28
31
|
klass, name = methods[id]
|
|
29
32
|
next false unless klass
|
|
30
33
|
result = klass.new(name).run
|
|
31
34
|
!result.passed? && !result.skipped?
|
|
32
35
|
end
|
|
33
36
|
|
|
34
|
-
|
|
37
|
+
File.write(ledger, JSON.generate(failures: killer ? 1 : 0, failing: [killer].compact))
|
|
38
|
+
|
|
39
|
+
exit(killer ? 1 : 0)
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "json"
|
|
4
|
+
|
|
5
|
+
module Kimera
|
|
6
|
+
module Execution
|
|
7
|
+
end
|
|
8
|
+
end
|
|
9
|
+
|
|
10
|
+
class Kimera::Execution::IsolatedOutcome
|
|
11
|
+
attr_reader :status, :failing, :detail
|
|
12
|
+
|
|
13
|
+
def initialize(status, failing = nil, detail = nil)
|
|
14
|
+
@status = status
|
|
15
|
+
@failing = failing
|
|
16
|
+
@detail = detail
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
class << self
|
|
20
|
+
def judge(exit, path)
|
|
21
|
+
return new(:timeout) if exit == :timeout
|
|
22
|
+
ledger = read(path)
|
|
23
|
+
return unreported(exit) unless ledger
|
|
24
|
+
tally(exit, ledger)
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
private
|
|
28
|
+
|
|
29
|
+
def unreported(exit) = new(exit.success? ? :survived : :killed)
|
|
30
|
+
|
|
31
|
+
def tally(exit, ledger)
|
|
32
|
+
failing = Array(ledger["failing"])
|
|
33
|
+
return new(:killed, failing) if ledger["failures"].positive?
|
|
34
|
+
return new(:survived) if exit.success?
|
|
35
|
+
new(:harness_error, nil, unexplained(exit))
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def unexplained(exit)
|
|
39
|
+
"suite exited #{exit.exitstatus || "on signal #{exit.termsig}"} with 0 failures " \
|
|
40
|
+
"(a coverage floor or exit hook? skip it when ENV[\"KIMERA\"] is set)"
|
|
41
|
+
end
|
|
42
|
+
|
|
43
|
+
def read(path)
|
|
44
|
+
ledger = JSON.parse(File.read(path))
|
|
45
|
+
ledger if ledger.is_a?(Hash) && ledger["failures"].is_a?(Integer)
|
|
46
|
+
rescue Errno::ENOENT, JSON::ParserError
|
|
47
|
+
nil
|
|
48
|
+
end
|
|
49
|
+
end
|
|
50
|
+
end
|
|
@@ -30,8 +30,8 @@ class Kimera::Execution::IsolatedPlan
|
|
|
30
30
|
selection.recorded(id)
|
|
31
31
|
end
|
|
32
32
|
|
|
33
|
-
def command(mirror, locations)
|
|
34
|
-
child.command(mirror, locations, paths: mutables)
|
|
33
|
+
def command(mirror, locations, ledger)
|
|
34
|
+
child.command(mirror, locations, ledger: ledger, paths: mutables)
|
|
35
35
|
end
|
|
36
36
|
|
|
37
37
|
def mutables
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "isolated_outcome"
|
|
4
|
+
|
|
3
5
|
module Kimera
|
|
4
6
|
module Execution
|
|
5
7
|
module IsolatedExecutionVerdict
|
|
@@ -17,8 +19,8 @@ module Kimera
|
|
|
17
19
|
trial.file = point.file
|
|
18
20
|
tests = plan.tests(trial.id, point)
|
|
19
21
|
return result(trial, :no_coverage, nil) unless tests
|
|
20
|
-
|
|
21
|
-
result(trial, status, duration, tests)
|
|
22
|
+
outcome, duration = measure(trial, tests)
|
|
23
|
+
result(trial, outcome.status, duration, tests, failing_tests: outcome.failing, detail: outcome.detail)
|
|
22
24
|
end
|
|
23
25
|
|
|
24
26
|
def measure(trial, tests)
|
|
@@ -34,10 +36,15 @@ module Kimera
|
|
|
34
36
|
end
|
|
35
37
|
|
|
36
38
|
def verdict(mirror, locations)
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
39
|
+
Dir.mktmpdir("kimera-ledger") do |dir|
|
|
40
|
+
ledger = File.join(dir, "ledger.json")
|
|
41
|
+
IsolatedOutcome.judge(launch(mirror, locations, ledger), ledger)
|
|
42
|
+
end
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def launch(mirror, locations, ledger)
|
|
46
|
+
env, cmd = plan.command(mirror, locations, ledger)
|
|
47
|
+
waitfor(Process.spawn(env, *cmd, chdir: mirror, pgroup: true, out: File::NULL, err: File::NULL))
|
|
41
48
|
end
|
|
42
49
|
|
|
43
50
|
def with_mirror
|
|
@@ -54,9 +61,9 @@ module Kimera
|
|
|
54
61
|
end
|
|
55
62
|
end
|
|
56
63
|
|
|
57
|
-
def result(trial, status, duration, cover = nil)
|
|
64
|
+
def result(trial, status, duration, cover = nil, **outcome)
|
|
58
65
|
MutantResult.new(
|
|
59
|
-
mutant_id: trial.id, status: status, file: trial.file, duration: duration, covering_tests: cover
|
|
66
|
+
mutant_id: trial.id, status: status, file: trial.file, duration: duration, covering_tests: cover, **outcome
|
|
60
67
|
)
|
|
61
68
|
end
|
|
62
69
|
|
|
@@ -34,9 +34,16 @@ class Kimera::Execution::ParallelTestDatabases
|
|
|
34
34
|
|
|
35
35
|
def cleanup
|
|
36
36
|
return unless active?
|
|
37
|
+
scrub
|
|
37
38
|
ActiveSupport::Testing::Parallelization.run_cleanup_hooks.each(&:call)
|
|
38
39
|
end
|
|
39
40
|
|
|
41
|
+
def scrub
|
|
42
|
+
return unless defined?(ActiveRecord::Base)
|
|
43
|
+
connection = ActiveRecord::Base.connection
|
|
44
|
+
connection.truncate_tables(*connection.tables)
|
|
45
|
+
end
|
|
46
|
+
|
|
40
47
|
def errors = @errors || $stderr
|
|
41
48
|
|
|
42
49
|
def eligible?
|
|
@@ -27,11 +27,25 @@ class Kimera::Execution::Schemata
|
|
|
27
27
|
|
|
28
28
|
def overlay!
|
|
29
29
|
Kimera::Execution::OverlayGuards.install!
|
|
30
|
-
@registry.files
|
|
30
|
+
loaded = weave(@registry.files)
|
|
31
|
+
(loaded + weave(unresolved)).tap { announce }
|
|
31
32
|
end
|
|
32
33
|
|
|
33
34
|
private
|
|
34
35
|
|
|
36
|
+
def weave(paths) = paths.flat_map { |path| load(path) }
|
|
37
|
+
|
|
38
|
+
def unresolved
|
|
39
|
+
@skipped.keys.select { |path| @skipped[path].start_with?("NameError:") }.each { |path| @skipped.delete(path) }
|
|
40
|
+
end
|
|
41
|
+
|
|
42
|
+
def announce
|
|
43
|
+
@skipped.each do |path, reason|
|
|
44
|
+
notice(path, reason)
|
|
45
|
+
@registry.at(path).select(&:safe?).each { |point| point.unmutatable!("schemata setup failed (#{reason})") }
|
|
46
|
+
end
|
|
47
|
+
end
|
|
48
|
+
|
|
35
49
|
def load(path)
|
|
36
50
|
file = File.join(root, path)
|
|
37
51
|
return [] unless File.file?(file)
|
|
@@ -44,7 +58,7 @@ class Kimera::Execution::Schemata
|
|
|
44
58
|
return [] if result.mutant_ids.empty?
|
|
45
59
|
apply(result, file)
|
|
46
60
|
rescue StandardError, ScriptError, SystemExit => error
|
|
47
|
-
skip(path, error)
|
|
61
|
+
skip(path, "#{error.class}: #{error.message}")
|
|
48
62
|
end
|
|
49
63
|
|
|
50
64
|
def synthesize(path, file)
|
|
@@ -55,17 +69,15 @@ class Kimera::Execution::Schemata
|
|
|
55
69
|
silence { result.install(file) }
|
|
56
70
|
end
|
|
57
71
|
|
|
58
|
-
def skip(path,
|
|
59
|
-
reason = "#{error.class}: #{error.message}"
|
|
72
|
+
def skip(path, reason)
|
|
60
73
|
@skipped[path] = reason
|
|
61
|
-
notice(path, reason)
|
|
62
74
|
[]
|
|
63
75
|
end
|
|
64
76
|
|
|
65
77
|
def notice(path, reason)
|
|
66
78
|
errors.puts(
|
|
67
79
|
"kimera: #{path} cannot run in warm workers — schemata setup failed " \
|
|
68
|
-
"(#{reason}); its mutants are reported
|
|
80
|
+
"(#{reason}); its mutants are reported unmutatable"
|
|
69
81
|
)
|
|
70
82
|
end
|
|
71
83
|
|
|
@@ -18,7 +18,7 @@ class Kimera::Execution::Verdicts
|
|
|
18
18
|
end
|
|
19
19
|
|
|
20
20
|
def reloadable?(id)
|
|
21
|
-
|
|
21
|
+
point(id).reloadable?
|
|
22
22
|
end
|
|
23
23
|
|
|
24
24
|
def reclassify!(results)
|
|
@@ -45,7 +45,9 @@ class Kimera::Execution::Verdicts
|
|
|
45
45
|
end
|
|
46
46
|
|
|
47
47
|
def quarantine(id)
|
|
48
|
-
|
|
48
|
+
found = point(id)
|
|
49
|
+
status = found.unmutatable? ? :unmutatable : :isolated_only
|
|
50
|
+
Kimera::MutantResult.new(mutant_id: id, status: status, file: found.file, detail: found.unsafe_reason)
|
|
49
51
|
end
|
|
50
52
|
|
|
51
53
|
def point(id)
|
|
@@ -79,4 +79,17 @@ class Kimera::Frameworks::Adapter
|
|
|
79
79
|
end
|
|
80
80
|
end
|
|
81
81
|
end
|
|
82
|
+
|
|
83
|
+
private
|
|
84
|
+
|
|
85
|
+
def each_test_file(files, &)
|
|
86
|
+
ARGV.clear
|
|
87
|
+
Array(files).each { |file| load_test_file(file, &) }
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def load_test_file(file)
|
|
91
|
+
yield(File.expand_path(file))
|
|
92
|
+
rescue StandardError, ScriptError => error
|
|
93
|
+
raise(Kimera::Error, "cannot load test file #{file} (#{error.class}: #{error.message})")
|
|
94
|
+
end
|
|
82
95
|
end
|
|
@@ -27,12 +27,14 @@ class Kimera::Incremental::GitDiff
|
|
|
27
27
|
def parse(text)
|
|
28
28
|
files = {}
|
|
29
29
|
current = nil
|
|
30
|
-
text.each_line { |line| current = advance(files, current, line) }
|
|
30
|
+
decoded(text).each_line { |line| current = advance(files, current, line) }
|
|
31
31
|
files
|
|
32
32
|
end
|
|
33
33
|
|
|
34
34
|
private
|
|
35
35
|
|
|
36
|
+
def decoded(text) = text.dup.force_encoding(Encoding::UTF_8).scrub
|
|
37
|
+
|
|
36
38
|
def advance(files, current, line)
|
|
37
39
|
header = line.match(FILE_HEADER)
|
|
38
40
|
return named(header) if header
|
|
@@ -17,7 +17,7 @@ module Kimera
|
|
|
17
17
|
end
|
|
18
18
|
|
|
19
19
|
def within?(from, to)
|
|
20
|
-
interior = ((from + 1)
|
|
20
|
+
interior = ((from + 1)..to)
|
|
21
21
|
interior.cover?(start_offset) && interior.cover?(finish)
|
|
22
22
|
end
|
|
23
23
|
|
|
@@ -66,6 +66,7 @@ end
|
|
|
66
66
|
|
|
67
67
|
class Kimera::MutationPoint
|
|
68
68
|
CLASS_BODY_REASON = "class-body DSL (runs at load; evaluate with --isolated)"
|
|
69
|
+
UNMUTATABLE = "unmutatable: "
|
|
69
70
|
SAFE = true
|
|
70
71
|
|
|
71
72
|
FIELDS = %i[point_id file operator node_type location original_source method_name mutants unsafe_reason].freeze
|
|
@@ -98,6 +99,18 @@ class Kimera::MutationPoint
|
|
|
98
99
|
unsafe_reason == CLASS_BODY_REASON
|
|
99
100
|
end
|
|
100
101
|
|
|
102
|
+
def reloadable?
|
|
103
|
+
!body? && !unmutatable?
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
def unmutatable?
|
|
107
|
+
unsafe_reason.to_s.start_with?(UNMUTATABLE)
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
def unmutatable!(detail)
|
|
111
|
+
unsafe!("#{UNMUTATABLE}#{detail}")
|
|
112
|
+
end
|
|
113
|
+
|
|
101
114
|
def safe
|
|
102
115
|
@attributes.fetch(:schema_safe, SAFE)
|
|
103
116
|
end
|
|
@@ -11,28 +11,42 @@ class Kimera::Report::Actions
|
|
|
11
11
|
end
|
|
12
12
|
|
|
13
13
|
def show(report, path: nil)
|
|
14
|
-
|
|
14
|
+
hints = path ? SavedHints.new(path) : UnsavedHints.new
|
|
15
|
+
lines = [
|
|
16
|
+
advice(report.survived, hints, :survivors), advice(report.uncovered, hints, :uncovered),
|
|
17
|
+
advice(report.errors, hints, :unjudged)
|
|
18
|
+
].compact
|
|
15
19
|
@io.puts("", "Next actions:", *lines) unless lines.empty?
|
|
16
20
|
end
|
|
17
21
|
|
|
18
22
|
private
|
|
19
23
|
|
|
20
|
-
def
|
|
24
|
+
def advice(found, hints, kind)
|
|
21
25
|
first = found.min_by(&:mutant_id)
|
|
22
|
-
|
|
26
|
+
" #{found.size} #{hints.public_send(kind, first.mutant_id)}" if first
|
|
23
27
|
end
|
|
24
28
|
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
29
|
+
class SavedHints
|
|
30
|
+
def initialize(path)
|
|
31
|
+
@path = path
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def survivors(id) = "surviving mutant(s): inspect ##{id} with `#{mutant(id)}`"
|
|
35
|
+
|
|
36
|
+
def uncovered(_id) = "uncovered mutant(s): kimera report #{@path} --status no_coverage"
|
|
28
37
|
|
|
29
|
-
|
|
38
|
+
def unjudged(id) = "unjudged mutant(s): see why with `#{mutant(id)}`"
|
|
39
|
+
|
|
40
|
+
private
|
|
41
|
+
|
|
42
|
+
def mutant(id) = "kimera mutant #{id} --report #{@path}"
|
|
43
|
+
end
|
|
30
44
|
|
|
31
|
-
|
|
45
|
+
class UnsavedHints
|
|
46
|
+
def survivors(id) = "surviving mutant(s): inspect ##{id} with `kimera run --report tmp/kimera.json`"
|
|
32
47
|
|
|
33
|
-
|
|
48
|
+
def uncovered(_id) = "uncovered mutant(s): rerun with --report tmp/kimera.json"
|
|
34
49
|
|
|
35
|
-
|
|
36
|
-
" #{found.size} #{text}" unless found.empty?
|
|
50
|
+
def unjudged(_id) = "unjudged mutant(s): retry with `kimera run --isolated`"
|
|
37
51
|
end
|
|
38
52
|
end
|
|
@@ -45,6 +45,6 @@ module Kimera::Report::Sections
|
|
|
45
45
|
return if leaks.empty?
|
|
46
46
|
@io.puts
|
|
47
47
|
@io.puts(yellow("State-leak warnings (#{leaks.size}):"))
|
|
48
|
-
leaks.each { |leak| @io.puts(" - #{leak.detail}") }
|
|
48
|
+
leaks.each { |leak| @io.puts(" - ##{leak.mutant_id} #{leak.detail}") }
|
|
49
49
|
end
|
|
50
50
|
end
|
|
@@ -4,9 +4,10 @@ module Kimera
|
|
|
4
4
|
module Status
|
|
5
5
|
KILLING = %i[killed timeout error].freeze
|
|
6
6
|
|
|
7
|
-
UNJUDGED = %i[no_coverage ignored isolated_only harness_error].freeze
|
|
7
|
+
UNJUDGED = %i[no_coverage ignored isolated_only unmutatable harness_error].freeze
|
|
8
8
|
|
|
9
|
-
REPORTED =
|
|
9
|
+
REPORTED =
|
|
10
|
+
([:survived] + (KILLING - [:killed]) + %i[no_coverage harness_error ignored isolated_only unmutatable]).freeze
|
|
10
11
|
|
|
11
12
|
SELF_SUSPECT = ((KILLING - [:killed]) + [:survived]).freeze
|
|
12
13
|
end
|
|
@@ -10,7 +10,7 @@ end
|
|
|
10
10
|
class Kimera::RunReport
|
|
11
11
|
attr_reader :results, :leaks, :registry
|
|
12
12
|
|
|
13
|
-
CONDITIONAL = %i[harness_error ignored isolated_only leaks].freeze
|
|
13
|
+
CONDITIONAL = %i[harness_error ignored isolated_only unmutatable leaks].freeze
|
|
14
14
|
LABELS = { total: "mutants", harness_error: "unjudged" }.freeze
|
|
15
15
|
SCHEMA_VERSION = 1
|
|
16
16
|
|
|
@@ -11,5 +11,32 @@ module Kimera
|
|
|
11
11
|
def parse(source) = Kimera::Warnings.silence { Unparser.parse(source) }
|
|
12
12
|
|
|
13
13
|
def unparse(node) = Kimera::Warnings.silence { Unparser.unparse(node) }
|
|
14
|
+
|
|
15
|
+
module Binders
|
|
16
|
+
private
|
|
17
|
+
|
|
18
|
+
def enter(node)
|
|
19
|
+
super
|
|
20
|
+
Binders.names(node).each { |name| define(name) }
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
class << self
|
|
24
|
+
def names(node)
|
|
25
|
+
case node.type
|
|
26
|
+
when :match_var, :blockarg then [node.children.first].compact
|
|
27
|
+
when :match_with_lvasgn then captures(node.children.first)
|
|
28
|
+
else []
|
|
29
|
+
end
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def captures(regexp)
|
|
33
|
+
*parts, options = regexp.children
|
|
34
|
+
flags = options.children.include?(:x) ? Regexp::EXTENDED : 0
|
|
35
|
+
Regexp.new(parts.sum("") { |part| part.children.first }, flags).names.map(&:to_sym)
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
end
|
|
14
39
|
end
|
|
15
40
|
end
|
|
41
|
+
|
|
42
|
+
Unparser::AST::LocalVariableScopeEnumerator.prepend(Kimera::Unparse::Binders)
|
|
@@ -6,6 +6,8 @@ require_relative "overlay_splice"
|
|
|
6
6
|
class Kimera::Overlay::FileWeave
|
|
7
7
|
include Kimera::OverlaySplice
|
|
8
8
|
|
|
9
|
+
UNGUARDABLE = "unparser could not round-trip its guard"
|
|
10
|
+
|
|
9
11
|
def initialize(file, map, safe, unsafe)
|
|
10
12
|
@file = file
|
|
11
13
|
@map = map
|
|
@@ -43,16 +45,15 @@ class Kimera::Overlay::FileWeave
|
|
|
43
45
|
def finish(viable)
|
|
44
46
|
dropped = @safe - viable
|
|
45
47
|
notice(dropped)
|
|
46
|
-
|
|
47
|
-
result.skipped_unsafe.concat(dropped.flat_map(&:ids))
|
|
48
|
-
result
|
|
48
|
+
dropped.each { |point| point.unmutatable!(UNGUARDABLE) }
|
|
49
|
+
attempt(viable).tap { |result| result.skipped_unsafe.concat(dropped.flat_map(&:ids)) }
|
|
49
50
|
end
|
|
50
51
|
|
|
51
52
|
def notice(dropped)
|
|
52
53
|
io.puts(
|
|
53
54
|
"kimera: #{@file}: #{dropped.size} mutation point(s) at " \
|
|
54
55
|
"#{dropped.map { |p| "line #{p.location.start_line}" }.uniq.join(", ")} " \
|
|
55
|
-
"are unguardable (unparser round-trip); their mutants are reported
|
|
56
|
+
"are unguardable (unparser round-trip); their mutants are reported unmutatable"
|
|
56
57
|
)
|
|
57
58
|
end
|
|
58
59
|
|
|
@@ -8,6 +8,7 @@ require_relative "../error"
|
|
|
8
8
|
require_relative "../registry/registry"
|
|
9
9
|
require_relative "guard_weaver"
|
|
10
10
|
require_relative "source_map"
|
|
11
|
+
require_relative "superclass_pin"
|
|
11
12
|
|
|
12
13
|
class Kimera::Overlay
|
|
13
14
|
Result =
|
|
@@ -24,7 +25,7 @@ class Kimera::Overlay
|
|
|
24
25
|
|
|
25
26
|
class << self
|
|
26
27
|
def evaluate(source, path)
|
|
27
|
-
Kimera::Warnings.silence { TOPLEVEL_BINDING.eval(source, path) }
|
|
28
|
+
Kimera::Warnings.silence { TOPLEVEL_BINDING.eval(Kimera::SuperclassPin.pin(source), path) }
|
|
28
29
|
end
|
|
29
30
|
end
|
|
30
31
|
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "splice_reopening"
|
|
4
|
+
|
|
3
5
|
module Kimera
|
|
4
6
|
module OverlaySplice
|
|
5
7
|
private
|
|
@@ -13,17 +15,19 @@ module Kimera
|
|
|
13
15
|
return unless ready?
|
|
14
16
|
edits, dropped = gather
|
|
15
17
|
return if edits.empty?
|
|
16
|
-
source = apply(edits)
|
|
18
|
+
source = apply(edits + reopenings)
|
|
17
19
|
[edits, dropped, source] if Kimera::Syntax.parse(source).success?
|
|
18
20
|
end
|
|
19
21
|
|
|
20
22
|
def gather
|
|
21
23
|
edits = []
|
|
22
24
|
dropped = []
|
|
23
|
-
each { |definition, points| collect(definition, points, edits, dropped) }
|
|
24
|
-
[edits, dropped]
|
|
25
|
+
claimed = each { |definition, points| collect(definition, points, edits, dropped) }
|
|
26
|
+
[edits, dropped + (@safe - claimed)]
|
|
25
27
|
end
|
|
26
28
|
|
|
29
|
+
def reopenings = Kimera::SpliceReopening.new(@map).edits
|
|
30
|
+
|
|
27
31
|
def collect(definition, points, edits, dropped)
|
|
28
32
|
rewritten = rewrite(definition, points)
|
|
29
33
|
rewritten ? edits << rewritten : dropped.concat(points)
|
|
@@ -32,6 +36,7 @@ module Kimera
|
|
|
32
36
|
def report(edits, dropped, source)
|
|
33
37
|
applied = edits.flat_map(&:last)
|
|
34
38
|
announce(applied, dropped)
|
|
39
|
+
dropped.each { |point| point.unmutatable!("unparser could not round-trip its method") }
|
|
35
40
|
build(applied, dropped, source)
|
|
36
41
|
end
|
|
37
42
|
|
|
@@ -46,7 +51,7 @@ module Kimera
|
|
|
46
51
|
io.puts(
|
|
47
52
|
"kimera: #{@file}: file-level round-trip failed; " \
|
|
48
53
|
"#{applied.size} mutant(s) spliced per method, " \
|
|
49
|
-
"#{dropped.flat_map(&:ids).size} reported
|
|
54
|
+
"#{dropped.flat_map(&:ids).size} reported unmutatable"
|
|
50
55
|
)
|
|
51
56
|
end
|
|
52
57
|
|
|
@@ -60,6 +65,7 @@ module Kimera
|
|
|
60
65
|
def each(&)
|
|
61
66
|
claimed = []
|
|
62
67
|
definitions.each { |definition| visit(definition, claimed, &) }
|
|
68
|
+
claimed
|
|
63
69
|
end
|
|
64
70
|
|
|
65
71
|
def visit(definition, claimed)
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "../rewrite/ast_walk"
|
|
4
|
+
require_relative "guardrail_value_objects"
|
|
5
|
+
|
|
6
|
+
class Kimera::SpliceReopening
|
|
7
|
+
FACTORIES = Kimera::GuardrailValueObjects::VALUE_OBJECT_FACTORIES
|
|
8
|
+
LEXICAL = "(is_a?(::Module) ? self : ::Object)"
|
|
9
|
+
|
|
10
|
+
Assignment =
|
|
11
|
+
Data.define(:scope, :name, :value, :location) do
|
|
12
|
+
def edit = [location.expression.begin_pos, value.location.begin.begin_pos, header, []]
|
|
13
|
+
|
|
14
|
+
def header
|
|
15
|
+
owner = base
|
|
16
|
+
"#{constant} = (#{owner}.const_defined?(:#{name}, false) ? #{owner}.const_get(:#{name}) : " \
|
|
17
|
+
"#{factory}); #{constant}.class_eval "
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def constant = location.expression.begin.join(location.name).source
|
|
21
|
+
|
|
22
|
+
def factory = value.children.first.location.expression.source
|
|
23
|
+
|
|
24
|
+
def base
|
|
25
|
+
return LEXICAL unless scope
|
|
26
|
+
scope.type == :cbase ? "::Object" : scope.location.expression.source
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def initialize(map)
|
|
31
|
+
@map = map
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def edits = constants.map(&:edit)
|
|
35
|
+
|
|
36
|
+
private
|
|
37
|
+
|
|
38
|
+
def constants
|
|
39
|
+
found = []
|
|
40
|
+
Kimera::Rewrite::AstWalk.visit(@map.ast) { |node| found << assignment(node) if reopenable?(node) }
|
|
41
|
+
found
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def assignment(node) = Assignment.new(*node, node.location)
|
|
45
|
+
|
|
46
|
+
def reopenable?(node)
|
|
47
|
+
return false unless node.type == :casgn
|
|
48
|
+
value = node.children.last
|
|
49
|
+
value.is_a?(Parser::AST::Node) && value.type == :block && factory?(value.children.first)
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def factory?(call)
|
|
53
|
+
receiver, selector = call.children
|
|
54
|
+
call.type == :send && receiver.is_a?(Parser::AST::Node) && receiver.type == :const &&
|
|
55
|
+
FACTORIES[receiver.children.last] == selector
|
|
56
|
+
end
|
|
57
|
+
end
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "prism"
|
|
4
|
+
|
|
5
|
+
module Kimera
|
|
6
|
+
module SuperclassPin
|
|
7
|
+
module_function
|
|
8
|
+
|
|
9
|
+
def pin(source)
|
|
10
|
+
classes(Prism.parse(source).value).reverse_each.with_object(source.b) do |node, text|
|
|
11
|
+
location = node.superclass.location
|
|
12
|
+
text[location.start_offset, location.length] = reopened(node)
|
|
13
|
+
end.force_encoding(source.encoding)
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def existing(scope, name)
|
|
17
|
+
owner = scope.is_a?(Module) ? scope : Object
|
|
18
|
+
inherit = false
|
|
19
|
+
owner.const_defined?(name, inherit) && owner.const_get(name, inherit).superclass
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
def reopened(node)
|
|
23
|
+
"(::Kimera::SuperclassPin.existing(self, :#{node.constant_path.name}) || #{node.superclass.slice})".b
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def classes(node, found = [])
|
|
27
|
+
found << node if dynamic?(node)
|
|
28
|
+
node.compact_child_nodes.each { |child| classes(child, found) }
|
|
29
|
+
found
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def dynamic?(node)
|
|
33
|
+
return false unless node.is_a?(Prism::ClassNode) && node.constant_path.is_a?(Prism::ConstantReadNode)
|
|
34
|
+
superclass = node.superclass
|
|
35
|
+
superclass && !constant?(superclass)
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def constant?(node) = node.is_a?(Prism::ConstantReadNode) || node.is_a?(Prism::ConstantPathNode)
|
|
39
|
+
end
|
|
40
|
+
end
|
data/skills/kimera/SKILL.md
CHANGED
|
@@ -72,7 +72,8 @@ bundle exec kimera run --isolated --jobs 4 # oracle mode (see Strengthen)
|
|
|
72
72
|
- Rails `enum` models overlay warm (the re-declaration is idempotent), so
|
|
73
73
|
their method-body mutants are judged like any other. A file labeled
|
|
74
74
|
`unmutatable` could not be overlaid (a distinct, reported reason); it is not
|
|
75
|
-
an `--isolated` case.
|
|
75
|
+
an `--isolated` case. Its mutants report as `unmutatable` (never gating,
|
|
76
|
+
never scored, with the reason in the detail), not `no_coverage`.
|
|
76
77
|
|
|
77
78
|
## Triage a surviving mutant: the only four verdicts
|
|
78
79
|
|
|
@@ -91,13 +92,19 @@ bundle exec kimera run --isolated --jobs 4 # oracle mode (see Strengthen)
|
|
|
91
92
|
|
|
92
93
|
## Ignore discipline: hard rules (especially for agents)
|
|
93
94
|
|
|
94
|
-
- **
|
|
95
|
-
|
|
95
|
+
- **Raise `max_ignored` only for an entry that has earned it**: survival
|
|
96
|
+
under `--isolated`, a `reason:` naming the mechanism, and the raise in the
|
|
97
|
+
same diff as the entry. Any triager, human or agent, may make that call when
|
|
98
|
+
all three hold. Never raise it to make a failing gate pass; that is skipping
|
|
99
|
+
triage, not finishing it.
|
|
100
|
+
- If any criterion is missing (no oracle run, a reason that restates the
|
|
101
|
+
verdict, a mutant you haven't read), write the test or leave the survivor
|
|
102
|
+
visible.
|
|
96
103
|
- An ignore entry without an isolated-oracle survival check is inadmissible.
|
|
97
104
|
Warm-path survivors can be measurement artifacts (kimera's own suite once
|
|
98
105
|
had a "clearly equivalent" mutant that 11 specs actually kill).
|
|
99
|
-
-
|
|
100
|
-
|
|
106
|
+
- Reviewers judge the `reason:`, not the number, so the entry and the raise
|
|
107
|
+
land together.
|
|
101
108
|
|
|
102
109
|
## Strengthen
|
|
103
110
|
|
|
@@ -110,6 +117,12 @@ bundle exec kimera run --isolated --jobs 4 # oracle mode (see Strengthen)
|
|
|
110
117
|
runner, global state the suite also touches, kimera's own plumbing):
|
|
111
118
|
adjudicate with `--isolated`. Suites that manipulate shared globals can
|
|
112
119
|
suppress the warm path.
|
|
120
|
+
- `--isolated` judges on failed tests, not the exit status. A child that
|
|
121
|
+
exits non-zero with zero failures (typically a SimpleCov `minimum_coverage`
|
|
122
|
+
floor tripped by the partial run) is `harness_error`, not a kill. Kimera
|
|
123
|
+
sets `KIMERA=1` wherever it loads the suite (warm, `--isolated`, doctor), so
|
|
124
|
+
gate the floor on it (`minimum_coverage ... unless ENV["KIMERA"]`); `kimera
|
|
125
|
+
doctor` warns when it isn't.
|
|
113
126
|
- A cluster of `error`/`timeout` verdicts in one region usually means harness
|
|
114
127
|
fragility or a missing guard, not test strength. Read the cluster before
|
|
115
128
|
counting the detections.
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: kimera
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Giacomo GK
|
|
@@ -95,6 +95,7 @@ files:
|
|
|
95
95
|
- lib/kimera/cli/survivors.rb
|
|
96
96
|
- lib/kimera/cli/survivors/panel.rb
|
|
97
97
|
- lib/kimera/cli/synthesize.rb
|
|
98
|
+
- lib/kimera/cli/test_command.rb
|
|
98
99
|
- lib/kimera/cli/workflows.rb
|
|
99
100
|
- lib/kimera/error.rb
|
|
100
101
|
- lib/kimera/execution/baseline_failure.rb
|
|
@@ -108,6 +109,7 @@ files:
|
|
|
108
109
|
- lib/kimera/execution/isolated_child.rb
|
|
109
110
|
- lib/kimera/execution/isolated_child_command.rb
|
|
110
111
|
- lib/kimera/execution/isolated_child_minitest.rb
|
|
112
|
+
- lib/kimera/execution/isolated_outcome.rb
|
|
111
113
|
- lib/kimera/execution/isolated_plan.rb
|
|
112
114
|
- lib/kimera/execution/isolated_scheduling.rb
|
|
113
115
|
- lib/kimera/execution/isolated_test_selection.rb
|
|
@@ -129,6 +131,7 @@ files:
|
|
|
129
131
|
- lib/kimera/execution/shift/coverage_channel.rb
|
|
130
132
|
- lib/kimera/execution/shift/killer_memory.rb
|
|
131
133
|
- lib/kimera/execution/shift/leak_guard.rb
|
|
134
|
+
- lib/kimera/execution/suite_env.rb
|
|
132
135
|
- lib/kimera/execution/sweep.rb
|
|
133
136
|
- lib/kimera/execution/transaction_isolation.rb
|
|
134
137
|
- lib/kimera/execution/trial.rb
|
|
@@ -229,6 +232,8 @@ files:
|
|
|
229
232
|
- lib/kimera/synthesis/overlay_splice.rb
|
|
230
233
|
- lib/kimera/synthesis/project.rb
|
|
231
234
|
- lib/kimera/synthesis/source_map.rb
|
|
235
|
+
- lib/kimera/synthesis/splice_reopening.rb
|
|
236
|
+
- lib/kimera/synthesis/superclass_pin.rb
|
|
232
237
|
- lib/kimera/version.rb
|
|
233
238
|
- skills/kimera/SKILL.md
|
|
234
239
|
homepage: https://github.com/giacope/kimera
|