kimera 0.1.1 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +50 -0
- data/README.md +12 -5
- data/lib/kimera/cli/doctor.rb +2 -2
- data/lib/kimera/cli/report_file.rb +1 -1
- data/lib/kimera/cli/run/cycle.rb +3 -1
- data/lib/kimera/cli/survivors.rb +1 -1
- data/lib/kimera/cli/test_command.rb +2 -1
- data/lib/kimera/execution/baseline_failure.rb +4 -8
- data/lib/kimera/execution/baseline_pass.rb +2 -1
- data/lib/kimera/execution/isolated_child.rb +2 -1
- data/lib/kimera/execution/isolated_child_command.rb +13 -6
- data/lib/kimera/execution/isolated_child_minitest.rb +10 -5
- data/lib/kimera/execution/isolated_outcome.rb +25 -5
- data/lib/kimera/execution/isolated_verdict.rb +6 -3
- data/lib/kimera/execution/kinship.rb +27 -0
- data/lib/kimera/execution/overlay_guard_modules.rb +17 -0
- data/lib/kimera/execution/overlay_guards.rb +6 -0
- data/lib/kimera/execution/reflections.rb +35 -0
- data/lib/kimera/execution/reload.rb +33 -39
- data/lib/kimera/execution/reload_errand.rb +39 -0
- data/lib/kimera/execution/schedule.rb +17 -9
- data/lib/kimera/execution/schemata.rb +7 -2
- data/lib/kimera/execution/suite_env.rb +7 -0
- data/lib/kimera/execution/verdicts.rb +4 -2
- data/lib/kimera/frameworks/adapter.rb +5 -1
- data/lib/kimera/frameworks/minitest_adapter.rb +19 -0
- data/lib/kimera/incremental/git_diff.rb +3 -1
- data/lib/kimera/incremental/session.rb +1 -1
- data/lib/kimera/registry/mutation_point.rb +14 -1
- data/lib/kimera/results/result.rb +3 -2
- data/lib/kimera/results/run_report.rb +1 -1
- data/lib/kimera/support/unparse.rb +27 -0
- data/lib/kimera/support/version.rb +1 -1
- data/lib/kimera/synthesis/file_weave.rb +5 -4
- data/lib/kimera/synthesis/overlay_splice.rb +7 -2
- data/lib/kimera/synthesis/splice_reopening.rb +57 -0
- data/skills/kimera/SKILL.md +8 -4
- metadata +6 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 558ffe918ec2624918506fb0616bd8426d8bb1a4c569abb4dd31e4ef9e74217d
|
|
4
|
+
data.tar.gz: 8ab25fa8b1f4c06085fb63606e20b1fb26691017d1bffc06c2d5dfa59b9e743c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 889e8498b185c70c2e933954c2343729bfa8292a9651bd3b9f583080e85f9e7c212c4590c9d23388c6415798e81eff89620a598701ce2de1854956f806873434
|
|
7
|
+
data.tar.gz: bdd9bfab7f3409e107e52921bf6f210f0c25cf56a58b3afa6ebd01485ef1ea806fc5165b7fa732abd19bc1ec3eb59545d6055960f38ba6062c934891bfc9fa72
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,55 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.1.3 (2026-09-27)
|
|
4
|
+
|
|
5
|
+
- `--isolated` no longer scores a child that crashes before reporting results
|
|
6
|
+
as a kill. It is `harness_error`, and its `detail` gives the exit status and
|
|
7
|
+
the ends of the child's stderr. A Minitest child that fails to load a test
|
|
8
|
+
file still reports that as a kill, as the RSpec child already did.
|
|
9
|
+
- `--isolated` passes test ids and test files to the child in a request file
|
|
10
|
+
instead of argv, so a mutant covered by thousands of tests no longer fails
|
|
11
|
+
with `Errno::E2BIG: Argument list too long`.
|
|
12
|
+
- Warm reload-path mutants (for example, statements in a memoized method) run
|
|
13
|
+
one test at a time and stop at the first failure. Tests covering the
|
|
14
|
+
mutant's method run first. The hard timeout now applies to each test, not
|
|
15
|
+
the whole suite, so a mutant a test kills is no longer reported as
|
|
16
|
+
"reload worker produced no result". A test that overruns the timeout is a
|
|
17
|
+
`timeout`, and a worker that dies names its exit status or signal.
|
|
18
|
+
- A scope or association declared in a concern's `included do` block is now
|
|
19
|
+
instrumented in the classes that already included the concern. It used to be
|
|
20
|
+
reported `no_coverage`, or `survived` when it was covered through another
|
|
21
|
+
path.
|
|
22
|
+
- An association overlaid on a parent model now also reaches subclasses that
|
|
23
|
+
declared associations of their own (such as STI children). They used to keep
|
|
24
|
+
the unguarded reflection, so mutants in the association's scope lambda
|
|
25
|
+
survived.
|
|
26
|
+
- The red-baseline hint prints a plain Minitest command for a Minitest suite
|
|
27
|
+
instead of `rspec …`.
|
|
28
|
+
- `kimera report`/`kimera mutant` and incremental sessions read report JSON as
|
|
29
|
+
UTF-8, so they no longer crash under a non-UTF-8 locale when a report
|
|
30
|
+
contains non-ASCII text.
|
|
31
|
+
|
|
32
|
+
## 0.1.2 (2026-09-26)
|
|
33
|
+
|
|
34
|
+
- Kimera sets `KIMERA=1` in its own process before loading the suite, and in
|
|
35
|
+
the `kimera doctor` test commands. A coverage floor gated on
|
|
36
|
+
`ENV["KIMERA"]` no longer fails a passing warm run with exit 2 or blocks
|
|
37
|
+
doctor's dry run. (#6)
|
|
38
|
+
- A string interpolating a local bound by a pattern (`in [:ok, chosen]`,
|
|
39
|
+
`=> chosen`), a `&block` parameter, or a regexp named capture no longer
|
|
40
|
+
sinks its file's mutants. unparser 0.9 could not round-trip it; Kimera now
|
|
41
|
+
teaches unparser those bindings. (#7)
|
|
42
|
+
- The per-method splice fallback now claims mutants in an endless method
|
|
43
|
+
(`def big? = value > 10`), and reopens `Data.define` / `Struct.new`
|
|
44
|
+
constants instead of redefining them, so spliced guards are reached. (#7)
|
|
45
|
+
- Mutants Kimera could not instrument are reported as `unmutatable`, with the
|
|
46
|
+
reason, instead of `no_coverage`. They never gate and stay out of the score,
|
|
47
|
+
so `--fail-on-no-coverage` again means "no test executes this". (#7)
|
|
48
|
+
- `kimera run --since` no longer crashes with "invalid byte sequence in
|
|
49
|
+
US-ASCII" when the locale isn't UTF-8 (e.g. `LANG` unset) and the diff
|
|
50
|
+
contains non-ASCII text. Git's diff output is read as UTF-8, with invalid
|
|
51
|
+
bytes replaced. (#9)
|
|
52
|
+
|
|
3
53
|
## 0.1.1 (2026-09-26)
|
|
4
54
|
|
|
5
55
|
- Fix a boot crash on case-sensitive filesystems (Linux): `require "English"`
|
data/README.md
CHANGED
|
@@ -220,9 +220,13 @@ selector and Kimera's own harness-critical files (see below).
|
|
|
220
220
|
|
|
221
221
|
An isolated child is judged on its failed tests, not its exit status. A child
|
|
222
222
|
that exits non-zero with no failing test is `harness_error` (unjudged, gated
|
|
223
|
-
by `--max-errors`), not a kill.
|
|
224
|
-
|
|
225
|
-
|
|
223
|
+
by `--max-errors`), not a kill. So is a child that dies before reporting any
|
|
224
|
+
results (a boot error, an exit hook, a crash); its `detail` carries the
|
|
225
|
+
child's exit status and the ends of its stderr. The usual cause of a non-zero
|
|
226
|
+
exit with no failures is a coverage floor such as
|
|
227
|
+
SimpleCov's `minimum_coverage`, which a partial run always trips. Kimera sets
|
|
228
|
+
`KIMERA=1` wherever it loads your suite (the warm process, `--isolated`
|
|
229
|
+
children, and `kimera doctor`'s test commands), so skip the floor there:
|
|
226
230
|
|
|
227
231
|
```ruby
|
|
228
232
|
SimpleCov.start { minimum_coverage(line: 100, branch: 100) unless ENV["KIMERA"] }
|
|
@@ -504,8 +508,11 @@ source ──Prism──▶ registry (mutation points, JSON) ──┬─▶ syn
|
|
|
504
508
|
baseline, or worse, as kills that are really collisions.
|
|
505
509
|
- **Score definition.** `score = killed / evaluable`.
|
|
506
510
|
- Killed includes timeouts and errors (observable misbehaviour).
|
|
507
|
-
- Evaluable excludes `no_coverage`, `ignored`, `isolated_only`,
|
|
508
|
-
`harness_error` mutants. Each is reported separately.
|
|
511
|
+
- Evaluable excludes `no_coverage`, `ignored`, `isolated_only`,
|
|
512
|
+
`unmutatable`, and `harness_error` mutants. Each is reported separately.
|
|
513
|
+
- An `unmutatable` mutant is one Kimera could not instrument for the warm
|
|
514
|
+
run (its file or method failed to re-emit or load); the detail says why.
|
|
515
|
+
It never gates, and it is not `no_coverage`: a test may well run it.
|
|
509
516
|
- A `harness_error` is a mutant Kimera could not judge: its worker died, or
|
|
510
517
|
its reply was unreadable. It is not a kill. It gates via `--max-errors`
|
|
511
518
|
(default 0). A pool that never produces a result aborts the run.
|
data/lib/kimera/cli/doctor.rb
CHANGED
|
@@ -96,8 +96,8 @@ class Kimera::CLI::Doctor
|
|
|
96
96
|
end
|
|
97
97
|
|
|
98
98
|
def floored(helper)
|
|
99
|
-
"Coverage floor: minimum_coverage in #{helper} fails partial
|
|
100
|
-
"skip it when ENV[\"KIMERA\"] is set"
|
|
99
|
+
"Coverage floor: minimum_coverage in #{helper} fails Kimera's partial runs; " \
|
|
100
|
+
"skip it when ENV[\"KIMERA\"] is set (Kimera sets it)"
|
|
101
101
|
end
|
|
102
102
|
|
|
103
103
|
def floor?(path)
|
|
@@ -13,7 +13,7 @@ module Kimera::CLI::ReportFile
|
|
|
13
13
|
module_function
|
|
14
14
|
|
|
15
15
|
def parse(path)
|
|
16
|
-
JSON.parse(File.read(path))
|
|
16
|
+
JSON.parse(File.read(path, encoding: Encoding::UTF_8))
|
|
17
17
|
rescue JSON::ParserError => error
|
|
18
18
|
raise(Kimera::UsageError, "unreadable report #{path}: #{error.message}")
|
|
19
19
|
end
|
data/lib/kimera/cli/run/cycle.rb
CHANGED
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "../../execution/suite_env"
|
|
3
4
|
require_relative "../../incremental/selection"
|
|
4
5
|
require_relative "../../incremental/session"
|
|
5
6
|
require_relative "digest"
|
|
@@ -71,7 +72,8 @@ class Kimera::CLI::Run::Cycle
|
|
|
71
72
|
.transform_keys(&:to_s)
|
|
72
73
|
end
|
|
73
74
|
|
|
74
|
-
def harness(remaining, loaded)
|
|
75
|
+
def harness(remaining, loaded, env: ENV)
|
|
76
|
+
env.update(Kimera::Execution::SUITE_ENV)
|
|
75
77
|
adapter = Kimera::Frameworks::Adapter.load(@options[:framework])
|
|
76
78
|
Kimera::CLI::Run::Pass.new(@registry, @options, adapter).call(remaining, loaded)
|
|
77
79
|
end
|
data/lib/kimera/cli/survivors.rb
CHANGED
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require "open3"
|
|
4
|
+
require_relative "../execution/suite_env"
|
|
4
5
|
|
|
5
6
|
class Kimera::CLI::TestCommand
|
|
6
7
|
LOADER = "ARGV.map { |f| File.expand_path(f) }.tap { ARGV.clear }.each { |f| require(f) }"
|
|
@@ -30,7 +31,7 @@ class Kimera::CLI::TestCommand
|
|
|
30
31
|
end
|
|
31
32
|
|
|
32
33
|
def check(command, passed)
|
|
33
|
-
output, status = Open3.capture2e(*command, chdir: @root)
|
|
34
|
+
output, status = Open3.capture2e(Kimera::Execution::SUITE_ENV, *command, chdir: @root)
|
|
34
35
|
status.success? ? ["✓", passed] : yield(summary(output))
|
|
35
36
|
rescue Errno::ENOENT
|
|
36
37
|
yield(nil)
|
|
@@ -12,14 +12,14 @@ class Kimera::Execution::BaselineFailure < Kimera::Error
|
|
|
12
12
|
MAX_MESSAGE = 300
|
|
13
13
|
|
|
14
14
|
class << self
|
|
15
|
-
def build(failed, messages)
|
|
16
|
-
new(summary(failed, messages))
|
|
15
|
+
def build(failed, messages, command)
|
|
16
|
+
new(summary(failed, messages, command))
|
|
17
17
|
end
|
|
18
18
|
|
|
19
|
-
def summary(failed, messages)
|
|
19
|
+
def summary(failed, messages, command)
|
|
20
20
|
text = "baseline suite is not green: #{failed.join(", ")}"
|
|
21
21
|
text += appendix(failed, messages)
|
|
22
|
-
"#{text}\n#{
|
|
22
|
+
"#{text}\n reproduce without kimera: #{command}"
|
|
23
23
|
end
|
|
24
24
|
|
|
25
25
|
def appendix(failed, messages)
|
|
@@ -39,9 +39,5 @@ class Kimera::Execution::BaselineFailure < Kimera::Error
|
|
|
39
39
|
def truncate(text)
|
|
40
40
|
text.length > MAX_MESSAGE ? "#{text[0, MAX_MESSAGE]}…" : text
|
|
41
41
|
end
|
|
42
|
-
|
|
43
|
-
def reproduce(failed)
|
|
44
|
-
" reproduce without kimera: rspec #{failed.first(MAX_DETAILS).join(" ")} --order defined"
|
|
45
|
-
end
|
|
46
42
|
end
|
|
47
43
|
end
|
|
@@ -118,6 +118,7 @@ class Kimera::Execution::BaselinePass
|
|
|
118
118
|
end
|
|
119
119
|
|
|
120
120
|
def failure!(failed)
|
|
121
|
-
|
|
121
|
+
command = @adapter.reproduce(failed.first(Kimera::Execution::BaselineFailure::MAX_DETAILS))
|
|
122
|
+
raise(Kimera::Execution::BaselineFailure.build(failed, @messages, command))
|
|
122
123
|
end
|
|
123
124
|
end
|
|
@@ -13,7 +13,8 @@ RSpec.configuration.output_stream = StringIO.new
|
|
|
13
13
|
RSpec.configuration.error_stream = StringIO.new
|
|
14
14
|
RSpec.configuration.deprecation_stream = StringIO.new
|
|
15
15
|
|
|
16
|
-
ledger,
|
|
16
|
+
ledger, request = ARGV
|
|
17
|
+
locations = JSON.parse(File.read(request, encoding: Encoding::UTF_8)).fetch("tests")
|
|
17
18
|
code = RSpec::Core::Runner.run(locations)
|
|
18
19
|
|
|
19
20
|
failed = RSpec.world.all_examples.select { |example| example.execution_result.status == :failed }
|
|
@@ -1,5 +1,8 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "json"
|
|
4
|
+
require_relative "suite_env"
|
|
5
|
+
|
|
3
6
|
module Kimera
|
|
4
7
|
module Execution
|
|
5
8
|
end
|
|
@@ -16,22 +19,26 @@ class Kimera::Execution::IsolatedPlan::ChildCommand
|
|
|
16
19
|
@test_files = test_files
|
|
17
20
|
end
|
|
18
21
|
|
|
19
|
-
ENV_FLAG = { "KIMERA" => "1" }.freeze
|
|
20
|
-
|
|
21
22
|
def command(mirror, locations, ledger:, paths:)
|
|
23
|
+
File.write(request_path(ledger), JSON.generate(request(locations)))
|
|
22
24
|
includes = paths.flat_map { |path| ["-I", path] }
|
|
23
|
-
bundled(mirror,
|
|
25
|
+
bundled(mirror, Kimera::Execution::SUITE_ENV.dup, ["ruby", *includes, "-I", helpers, *argv(ledger)])
|
|
24
26
|
end
|
|
25
27
|
|
|
26
|
-
def argv(
|
|
27
|
-
|
|
28
|
-
|
|
28
|
+
def argv(ledger)
|
|
29
|
+
[minitest? ? MINITEST_CHILD : CHILD, ledger, request_path(ledger)]
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def request(locations)
|
|
33
|
+
{ "tests" => locations, "files" => @test_files }
|
|
29
34
|
end
|
|
30
35
|
|
|
31
36
|
def helpers
|
|
32
37
|
minitest? ? "test" : "spec"
|
|
33
38
|
end
|
|
34
39
|
|
|
40
|
+
def request_path(ledger) = "#{ledger}.request"
|
|
41
|
+
|
|
35
42
|
private
|
|
36
43
|
|
|
37
44
|
def bundled(mirror, env, argv)
|
|
@@ -12,13 +12,18 @@ rescue NameError, ArgumentError
|
|
|
12
12
|
end
|
|
13
13
|
end
|
|
14
14
|
|
|
15
|
-
ledger,
|
|
16
|
-
|
|
17
|
-
files =
|
|
18
|
-
ids =
|
|
15
|
+
ledger, request = ARGV
|
|
16
|
+
request = JSON.parse(File.read(request, encoding: Encoding::UTF_8))
|
|
17
|
+
files = request.fetch("files")
|
|
18
|
+
ids = request.fetch("tests")
|
|
19
19
|
ARGV.clear
|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
begin
|
|
22
|
+
files.each { |f| require File.expand_path(f) }
|
|
23
|
+
rescue StandardError, ScriptError => error
|
|
24
|
+
File.write(ledger, JSON.generate(failures: 1, failing: [], load_error: "#{error.class}: #{error.message}"))
|
|
25
|
+
exit(1)
|
|
26
|
+
end
|
|
22
27
|
|
|
23
28
|
methods = {}
|
|
24
29
|
Minitest::Runnable.runnables.each do |runnable|
|
|
@@ -8,6 +8,9 @@ module Kimera
|
|
|
8
8
|
end
|
|
9
9
|
|
|
10
10
|
class Kimera::Execution::IsolatedOutcome
|
|
11
|
+
HEAD_LINES = 5
|
|
12
|
+
TAIL_LINES = 15
|
|
13
|
+
|
|
11
14
|
attr_reader :status, :failing, :detail
|
|
12
15
|
|
|
13
16
|
def initialize(status, failing = nil, detail = nil)
|
|
@@ -17,16 +20,33 @@ class Kimera::Execution::IsolatedOutcome
|
|
|
17
20
|
end
|
|
18
21
|
|
|
19
22
|
class << self
|
|
20
|
-
def judge(exit, path)
|
|
23
|
+
def judge(exit, path, stderr = "")
|
|
21
24
|
return new(:timeout) if exit == :timeout
|
|
22
25
|
ledger = read(path)
|
|
23
|
-
return unreported(exit) unless ledger
|
|
26
|
+
return unreported(exit, stderr) unless ledger
|
|
24
27
|
tally(exit, ledger)
|
|
25
28
|
end
|
|
26
29
|
|
|
27
30
|
private
|
|
28
31
|
|
|
29
|
-
def unreported(exit
|
|
32
|
+
def unreported(exit, stderr)
|
|
33
|
+
new(:harness_error, nil, "test child #{status_of(exit)} without reporting results#{tail(stderr)}")
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def tail(stderr)
|
|
37
|
+
lines = stderr.to_s.strip.lines(chomp: true)
|
|
38
|
+
lines.empty? ? "" : ":\n#{ends(lines).join("\n")}"
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
def ends(lines)
|
|
42
|
+
return lines if lines.size <= HEAD_LINES + TAIL_LINES
|
|
43
|
+
[*lines.first(HEAD_LINES), "…", *lines.last(TAIL_LINES)]
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
def status_of(exit)
|
|
47
|
+
code = exit.exitstatus
|
|
48
|
+
code ? "exited #{code}" : "died on signal #{exit.termsig}"
|
|
49
|
+
end
|
|
30
50
|
|
|
31
51
|
def tally(exit, ledger)
|
|
32
52
|
failing = Array(ledger["failing"])
|
|
@@ -36,12 +56,12 @@ class Kimera::Execution::IsolatedOutcome
|
|
|
36
56
|
end
|
|
37
57
|
|
|
38
58
|
def unexplained(exit)
|
|
39
|
-
"suite
|
|
59
|
+
"suite #{status_of(exit)} with 0 failures " \
|
|
40
60
|
"(a coverage floor or exit hook? skip it when ENV[\"KIMERA\"] is set)"
|
|
41
61
|
end
|
|
42
62
|
|
|
43
63
|
def read(path)
|
|
44
|
-
ledger = JSON.parse(File.read(path))
|
|
64
|
+
ledger = JSON.parse(File.read(path, encoding: Encoding::UTF_8))
|
|
45
65
|
ledger if ledger.is_a?(Hash) && ledger["failures"].is_a?(Integer)
|
|
46
66
|
rescue Errno::ENOENT, JSON::ParserError
|
|
47
67
|
nil
|
|
@@ -38,13 +38,16 @@ module Kimera
|
|
|
38
38
|
def verdict(mirror, locations)
|
|
39
39
|
Dir.mktmpdir("kimera-ledger") do |dir|
|
|
40
40
|
ledger = File.join(dir, "ledger.json")
|
|
41
|
-
|
|
41
|
+
stderr = File.join(dir, "stderr.log")
|
|
42
|
+
IsolatedOutcome.judge(launch(mirror, locations, ledger, stderr), ledger, captured(stderr))
|
|
42
43
|
end
|
|
43
44
|
end
|
|
44
45
|
|
|
45
|
-
def
|
|
46
|
+
def captured(path) = File.read(path, encoding: Encoding::UTF_8).scrub
|
|
47
|
+
|
|
48
|
+
def launch(mirror, locations, ledger, stderr)
|
|
46
49
|
env, cmd = plan.command(mirror, locations, ledger)
|
|
47
|
-
waitfor(Process.spawn(env, *cmd, chdir: mirror, pgroup: true, out: File::NULL, err:
|
|
50
|
+
waitfor(Process.spawn(env, *cmd, chdir: mirror, pgroup: true, out: File::NULL, err: stderr))
|
|
48
51
|
end
|
|
49
52
|
|
|
50
53
|
def with_mirror
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "../registry/mutation_point"
|
|
4
|
+
|
|
5
|
+
module Kimera
|
|
6
|
+
module Execution
|
|
7
|
+
end
|
|
8
|
+
end
|
|
9
|
+
|
|
10
|
+
class Kimera::Execution::Kinship
|
|
11
|
+
def initialize(registry, coverage)
|
|
12
|
+
@registry = registry
|
|
13
|
+
@coverage = coverage
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def order(id, tests)
|
|
17
|
+
(related(@registry.index.fetch(id, Kimera::MutationPoint::NONE)) & tests) | tests
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
private
|
|
21
|
+
|
|
22
|
+
def related(point)
|
|
23
|
+
siblings = @registry.at(point.file).sort_by { |other| other.method_name == point.method_name ? 0 : 1 }
|
|
24
|
+
known = @coverage.to_h
|
|
25
|
+
siblings.flat_map { |other| other.ids.flat_map { |mutant| Array(known[mutant]) } }
|
|
26
|
+
end
|
|
27
|
+
end
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require_relative "callback"
|
|
4
|
+
require_relative "reflections"
|
|
4
5
|
|
|
5
6
|
module Kimera
|
|
6
7
|
module Execution
|
|
@@ -56,14 +57,30 @@ module Kimera
|
|
|
56
57
|
define_method(hook) do |base = nil, &block|
|
|
57
58
|
next super(base, &block) unless block && OverlayGuardModules.replacing?(self, stored, base)
|
|
58
59
|
instance_variable_set(stored, block)
|
|
60
|
+
OverlayGuardModules.reapply(self, block)
|
|
59
61
|
end
|
|
60
62
|
end
|
|
61
63
|
end
|
|
62
64
|
|
|
63
65
|
def serialization = SERIALIZEGUARD
|
|
64
66
|
|
|
67
|
+
def reflection = Reflections::GUARD
|
|
68
|
+
|
|
69
|
+
MIXES = Module.instance_method(:include?)
|
|
70
|
+
|
|
65
71
|
def concern = CONCERNGUARD
|
|
66
72
|
|
|
73
|
+
def reapply(concern, block)
|
|
74
|
+
ObjectSpace.each_object(Class).each { |klass| klass.class_eval(&block) if owner?(klass, concern) }
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
def owner?(klass, concern)
|
|
78
|
+
return false if klass.singleton_class? || !mixes?(klass, concern)
|
|
79
|
+
[klass.superclass].compact.none? { |parent| mixes?(parent, concern) }
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
def mixes?(klass, concern) = MIXES.bind_call(klass, concern)
|
|
83
|
+
|
|
67
84
|
def replacing?(target, stored, base)
|
|
68
85
|
Kimera::Execution::OverlayGuards.overlaying? && !base && target.instance_variable_defined?(stored)
|
|
69
86
|
end
|
|
@@ -26,6 +26,7 @@ module Kimera
|
|
|
26
26
|
callbacks!
|
|
27
27
|
concern!
|
|
28
28
|
serialize!
|
|
29
|
+
reflections!
|
|
29
30
|
end
|
|
30
31
|
|
|
31
32
|
def enum!
|
|
@@ -43,6 +44,11 @@ module Kimera
|
|
|
43
44
|
ActiveRecord::Base.singleton_class.prepend(OverlayGuardModules.serialization)
|
|
44
45
|
end
|
|
45
46
|
|
|
47
|
+
def reflections!
|
|
48
|
+
return unless defined?(ActiveRecord::Reflection)
|
|
49
|
+
ActiveRecord::Reflection.singleton_class.prepend(OverlayGuardModules.reflection)
|
|
50
|
+
end
|
|
51
|
+
|
|
46
52
|
def concern!
|
|
47
53
|
return unless defined?(ActiveSupport::Concern)
|
|
48
54
|
ActiveSupport::Concern.prepend(OverlayGuardModules.concern)
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kimera
|
|
4
|
+
module Execution
|
|
5
|
+
module Reflections
|
|
6
|
+
module_function
|
|
7
|
+
|
|
8
|
+
GUARD =
|
|
9
|
+
Module.new do
|
|
10
|
+
def add_reflection(owner, name, reflection)
|
|
11
|
+
key = Reflections.key(owner, name)
|
|
12
|
+
stale = OverlayGuards.overlaying? && owner._reflections[key]
|
|
13
|
+
super
|
|
14
|
+
Reflections.inherit(owner, key, stale, reflection) if stale
|
|
15
|
+
end
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
def key(owner, name)
|
|
19
|
+
text = name.to_s
|
|
20
|
+
owner._reflections.key?(text) ? text : name.to_sym
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def inherit(owner, key, stale, reflection)
|
|
24
|
+
owner.descendants.each { |heir| replace(heir, key, stale, reflection) }
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def replace(heir, key, stale, reflection)
|
|
28
|
+
held = heir._reflections
|
|
29
|
+
return unless held[key].equal?(stale)
|
|
30
|
+
heir.clear_reflections_cache
|
|
31
|
+
heir._reflections = held.merge(key => reflection)
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
end
|
|
@@ -5,38 +5,16 @@ require_relative "../runtime"
|
|
|
5
5
|
require_relative "../synthesis/overlay"
|
|
6
6
|
require_relative "child_process"
|
|
7
7
|
require_relative "overlay_guards"
|
|
8
|
+
require_relative "reload_errand"
|
|
8
9
|
require_relative "verdicts"
|
|
9
10
|
|
|
10
11
|
class Kimera::Execution::Reload
|
|
12
|
+
include Kimera::Execution::ChildProcess
|
|
13
|
+
|
|
11
14
|
Failure =
|
|
12
15
|
Data.define(:exception) do
|
|
13
16
|
def message = "#{exception.class}: #{exception.message}"
|
|
14
17
|
end
|
|
15
|
-
include Kimera::Execution::ChildProcess
|
|
16
|
-
|
|
17
|
-
Errand =
|
|
18
|
-
Struct.new(:id, :reader, :writer) do
|
|
19
|
-
def parent!
|
|
20
|
-
writer.close
|
|
21
|
-
end
|
|
22
|
-
|
|
23
|
-
def child!
|
|
24
|
-
reader.close
|
|
25
|
-
end
|
|
26
|
-
|
|
27
|
-
def emit(payload)
|
|
28
|
-
writer.puts(payload)
|
|
29
|
-
writer.close
|
|
30
|
-
end
|
|
31
|
-
|
|
32
|
-
def await(deadline, &)
|
|
33
|
-
return reader.gets if reader.wait_readable(deadline)
|
|
34
|
-
yield
|
|
35
|
-
nil
|
|
36
|
-
ensure
|
|
37
|
-
reader.close
|
|
38
|
-
end
|
|
39
|
-
end
|
|
40
18
|
|
|
41
19
|
def initialize(registry:, adapter:, isolation:, root:)
|
|
42
20
|
@registry = registry
|
|
@@ -45,8 +23,8 @@ class Kimera::Execution::Reload
|
|
|
45
23
|
@root = root
|
|
46
24
|
end
|
|
47
25
|
|
|
48
|
-
def run(id, deadline:)
|
|
49
|
-
errand = Errand.new(id, *IO.pipe)
|
|
26
|
+
def run(id, deadline:, tests: @adapter.test_ids)
|
|
27
|
+
errand = Errand.new(id, tests, *IO.pipe)
|
|
50
28
|
collect(errand, child(errand), deadline)
|
|
51
29
|
end
|
|
52
30
|
|
|
@@ -59,32 +37,42 @@ class Kimera::Execution::Reload
|
|
|
59
37
|
def work(errand)
|
|
60
38
|
errand.child!
|
|
61
39
|
silence!
|
|
62
|
-
errand.emit(report(errand
|
|
40
|
+
errand.emit(report(errand))
|
|
63
41
|
exit!(0)
|
|
64
42
|
end
|
|
65
43
|
|
|
66
|
-
def report(
|
|
67
|
-
status, fails = evaluate(
|
|
68
|
-
JSON.generate(id: id, status: status.to_s, fails: fails)
|
|
44
|
+
def report(errand)
|
|
45
|
+
status, fails = evaluate(errand)
|
|
46
|
+
JSON.generate(id: errand.id, status: status.to_s, fails: Array(fails).map { |fail| fail.to_s.scrub })
|
|
69
47
|
end
|
|
70
48
|
|
|
71
49
|
def collect(errand, pid, deadline)
|
|
72
50
|
line = errand.await(deadline) { kill(pid) }
|
|
73
|
-
reap(pid)
|
|
74
|
-
verdict(errand.id, line)
|
|
51
|
+
verdict(errand.id, line, reap(pid), deadline)
|
|
75
52
|
end
|
|
76
53
|
|
|
77
|
-
def evaluate(
|
|
78
|
-
mutate(
|
|
54
|
+
def evaluate(errand)
|
|
55
|
+
failed = mutate(errand)
|
|
56
|
+
failed ? failed.verdict : [:survived, []]
|
|
79
57
|
rescue StandardError, ScriptError => error
|
|
80
58
|
[:error, [Failure.new(error).message]]
|
|
81
59
|
end
|
|
82
60
|
|
|
83
|
-
def mutate(
|
|
61
|
+
def mutate(errand)
|
|
62
|
+
id = errand.id
|
|
84
63
|
file = verdicts.point(id).file
|
|
85
64
|
overlay(file, File.join(File.expand_path(@root), file), id)
|
|
86
65
|
Kimera::Runtime.active = nil
|
|
87
|
-
@isolation.around {
|
|
66
|
+
@isolation.around { hunt(errand) }
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
def hunt(errand)
|
|
70
|
+
errand.tests.lazy.map { |test| trial(test, errand) }.reject(&:passed?).first
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
def trial(test, errand)
|
|
74
|
+
errand.tick
|
|
75
|
+
@adapter.run([test])
|
|
88
76
|
end
|
|
89
77
|
|
|
90
78
|
def overlay(file, path, id)
|
|
@@ -92,12 +80,18 @@ class Kimera::Execution::Reload
|
|
|
92
80
|
Kimera::Execution::OverlayGuards.overlay { Kimera::Overlay.evaluate(baked, path) }
|
|
93
81
|
end
|
|
94
82
|
|
|
95
|
-
def verdict(id, line)
|
|
96
|
-
return verdicts.
|
|
83
|
+
def verdict(id, line, status = nil, deadline = nil)
|
|
84
|
+
return verdicts.timeout(id, deadline) if line == :timeout
|
|
85
|
+
return verdicts.unjudged(id, "reload worker produced no result#{exited(status)}") unless line
|
|
97
86
|
message = parse(line)
|
|
98
87
|
message ? verdicts.parse(message) : verdicts.unjudged(id, "reload worker output unparseable")
|
|
99
88
|
end
|
|
100
89
|
|
|
90
|
+
def exited(status)
|
|
91
|
+
return "" unless status
|
|
92
|
+
status.signaled? ? " (died on signal #{status.termsig})" : " (exited #{status.exitstatus})"
|
|
93
|
+
end
|
|
94
|
+
|
|
101
95
|
def verdicts
|
|
102
96
|
Kimera::Execution::Verdicts.new(@registry)
|
|
103
97
|
end
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kimera
|
|
4
|
+
module Execution
|
|
5
|
+
end
|
|
6
|
+
end
|
|
7
|
+
|
|
8
|
+
class Kimera::Execution::Reload; end
|
|
9
|
+
|
|
10
|
+
Kimera::Execution::Reload::Errand =
|
|
11
|
+
Struct.new(:id, :tests, :reader, :writer) do
|
|
12
|
+
tick = "tick\n"
|
|
13
|
+
def parent!
|
|
14
|
+
writer.close
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def child!
|
|
18
|
+
reader.close
|
|
19
|
+
writer.sync = true
|
|
20
|
+
end
|
|
21
|
+
define_method(:tick) { writer.write(tick) }
|
|
22
|
+
def emit(payload)
|
|
23
|
+
writer.puts(payload)
|
|
24
|
+
writer.close
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def await(deadline)
|
|
28
|
+
heard(deadline).tap { |line| yield if line == :timeout }
|
|
29
|
+
ensure
|
|
30
|
+
reader.close
|
|
31
|
+
end
|
|
32
|
+
define_method(:heard) do |deadline|
|
|
33
|
+
while reader.wait_readable(deadline)
|
|
34
|
+
line = reader.gets
|
|
35
|
+
return line unless line == tick
|
|
36
|
+
end
|
|
37
|
+
:timeout
|
|
38
|
+
end
|
|
39
|
+
end
|
|
@@ -1,12 +1,13 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require_relative "priority"
|
|
4
|
+
require_relative "kinship"
|
|
4
5
|
require_relative "reload"
|
|
5
6
|
require_relative "verdicts"
|
|
6
7
|
|
|
7
8
|
class Kimera::Execution::Schedule
|
|
8
9
|
Ledger = Struct.new(:results, :leaks)
|
|
9
|
-
Batch = Struct.new(:safe, :reloadable, :deferred, :ledger)
|
|
10
|
+
Batch = Struct.new(:safe, :reloadable, :deferred, :ledger, :coverage)
|
|
10
11
|
|
|
11
12
|
LOSSES = { timeout: :expired }.freeze
|
|
12
13
|
|
|
@@ -37,14 +38,16 @@ class Kimera::Execution::Schedule
|
|
|
37
38
|
safe, unsafe = ids.partition { |id| verdicts.safe?(id) }
|
|
38
39
|
reloadable, deferred = unsafe.partition { |id| verdicts.reloadable?(id) }
|
|
39
40
|
progress.start(safe.size + unsafe.size, label)
|
|
40
|
-
Batch.new(
|
|
41
|
+
Batch.new(prioritized(safe, coverage), reloadable, deferred, Ledger.new({}, []), coverage)
|
|
41
42
|
end
|
|
42
43
|
|
|
44
|
+
def prioritized(safe, coverage) = Kimera::Execution::Priority.new(coverage).order(safe)
|
|
45
|
+
|
|
43
46
|
def process(batch)
|
|
44
|
-
|
|
45
|
-
pool(safe, ledger)
|
|
46
|
-
reload(
|
|
47
|
-
defer(deferred, ledger)
|
|
47
|
+
ledger = batch.ledger
|
|
48
|
+
pool(batch.safe, ledger)
|
|
49
|
+
reload(batch)
|
|
50
|
+
defer(batch.deferred, ledger)
|
|
48
51
|
end
|
|
49
52
|
|
|
50
53
|
def finish(ledger)
|
|
@@ -58,8 +61,13 @@ class Kimera::Execution::Schedule
|
|
|
58
61
|
progress.tick(result.status)
|
|
59
62
|
end
|
|
60
63
|
|
|
61
|
-
def reload(
|
|
62
|
-
|
|
64
|
+
def reload(batch)
|
|
65
|
+
kinship = Kimera::Execution::Kinship.new(@registry, batch.coverage)
|
|
66
|
+
batch.reloadable.each { |id| record(batch.ledger, id, rerun(id, kinship)) }
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
def rerun(id, kinship)
|
|
70
|
+
reloader.run(id, deadline: hard, tests: kinship.order(id, @options.fetch(:adapter).test_ids))
|
|
63
71
|
end
|
|
64
72
|
|
|
65
73
|
def defer(ids, ledger)
|
|
@@ -97,7 +105,7 @@ class Kimera::Execution::Schedule
|
|
|
97
105
|
def crashed(id) = verdicts.unjudged(id, "worker crashed before result")
|
|
98
106
|
|
|
99
107
|
def reloader
|
|
100
|
-
Kimera::Execution::Reload.new(
|
|
108
|
+
@_reloader ||= Kimera::Execution::Reload.new(
|
|
101
109
|
registry: @registry, adapter: @options.fetch(:adapter),
|
|
102
110
|
isolation: @options.fetch(:isolation), root: @options.fetch(:root)
|
|
103
111
|
)
|
|
@@ -39,7 +39,12 @@ class Kimera::Execution::Schemata
|
|
|
39
39
|
@skipped.keys.select { |path| @skipped[path].start_with?("NameError:") }.each { |path| @skipped.delete(path) }
|
|
40
40
|
end
|
|
41
41
|
|
|
42
|
-
def announce
|
|
42
|
+
def announce
|
|
43
|
+
@skipped.each do |path, reason|
|
|
44
|
+
notice(path, reason)
|
|
45
|
+
@registry.at(path).select(&:safe?).each { |point| point.unmutatable!("schemata setup failed (#{reason})") }
|
|
46
|
+
end
|
|
47
|
+
end
|
|
43
48
|
|
|
44
49
|
def load(path)
|
|
45
50
|
file = File.join(root, path)
|
|
@@ -72,7 +77,7 @@ class Kimera::Execution::Schemata
|
|
|
72
77
|
def notice(path, reason)
|
|
73
78
|
errors.puts(
|
|
74
79
|
"kimera: #{path} cannot run in warm workers — schemata setup failed " \
|
|
75
|
-
"(#{reason}); its mutants are reported
|
|
80
|
+
"(#{reason}); its mutants are reported unmutatable"
|
|
76
81
|
)
|
|
77
82
|
end
|
|
78
83
|
|
|
@@ -18,7 +18,7 @@ class Kimera::Execution::Verdicts
|
|
|
18
18
|
end
|
|
19
19
|
|
|
20
20
|
def reloadable?(id)
|
|
21
|
-
|
|
21
|
+
point(id).reloadable?
|
|
22
22
|
end
|
|
23
23
|
|
|
24
24
|
def reclassify!(results)
|
|
@@ -45,7 +45,9 @@ class Kimera::Execution::Verdicts
|
|
|
45
45
|
end
|
|
46
46
|
|
|
47
47
|
def quarantine(id)
|
|
48
|
-
|
|
48
|
+
found = point(id)
|
|
49
|
+
status = found.unmutatable? ? :unmutatable : :isolated_only
|
|
50
|
+
Kimera::MutantResult.new(mutant_id: id, status: status, file: found.file, detail: found.unsafe_reason)
|
|
49
51
|
end
|
|
50
52
|
|
|
51
53
|
def point(id)
|
|
@@ -51,8 +51,27 @@ class Kimera::Frameworks::MinitestAdapter < Kimera::Frameworks::Adapter
|
|
|
51
51
|
Kimera::Frameworks::RunOutcome.new(passed: failed.empty?, failed_ids: failed, failures: failures)
|
|
52
52
|
end
|
|
53
53
|
|
|
54
|
+
def reproduce(ids)
|
|
55
|
+
"bundle exec ruby -Itest -rminitest/autorun -e #{quote(loads(ids))} -- -n #{quote(filter(ids))} --seed 1"
|
|
56
|
+
end
|
|
57
|
+
|
|
54
58
|
private
|
|
55
59
|
|
|
60
|
+
def loads(ids)
|
|
61
|
+
ids.filter_map { |id| origin(id) }.uniq.map { |file| "require File.expand_path(#{file.dump})" }.join("; ")
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def filter(ids) = "/^(?:#{ids.map { |id| Regexp.escape(id) }.join("|")})$/"
|
|
65
|
+
|
|
66
|
+
def origin(id)
|
|
67
|
+
klass, name = @methods[id]
|
|
68
|
+
return unless klass
|
|
69
|
+
path, = klass.instance_method(name).source_location
|
|
70
|
+
path.delete_prefix("#{Dir.pwd}#{File::SEPARATOR}")
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
def quote(text) = "'#{text.gsub("'") { "'\\''" }}'"
|
|
74
|
+
|
|
56
75
|
def record(id, failed, failures)
|
|
57
76
|
klass, name = @methods[id]
|
|
58
77
|
return unless klass
|
|
@@ -27,12 +27,14 @@ class Kimera::Incremental::GitDiff
|
|
|
27
27
|
def parse(text)
|
|
28
28
|
files = {}
|
|
29
29
|
current = nil
|
|
30
|
-
text.each_line { |line| current = advance(files, current, line) }
|
|
30
|
+
decoded(text).each_line { |line| current = advance(files, current, line) }
|
|
31
31
|
files
|
|
32
32
|
end
|
|
33
33
|
|
|
34
34
|
private
|
|
35
35
|
|
|
36
|
+
def decoded(text) = text.dup.force_encoding(Encoding::UTF_8).scrub
|
|
37
|
+
|
|
36
38
|
def advance(files, current, line)
|
|
37
39
|
header = line.match(FILE_HEADER)
|
|
38
40
|
return named(header) if header
|
|
@@ -24,7 +24,7 @@ class Kimera::Incremental::Session
|
|
|
24
24
|
class << self
|
|
25
25
|
def load(path, registry: nil)
|
|
26
26
|
return new unless path && File.exist?(path)
|
|
27
|
-
session = decode(JSON.parse(File.read(path)))
|
|
27
|
+
session = decode(JSON.parse(File.read(path, encoding: Encoding::UTF_8)))
|
|
28
28
|
session.prune!(registry) if registry
|
|
29
29
|
session
|
|
30
30
|
end
|
|
@@ -17,7 +17,7 @@ module Kimera
|
|
|
17
17
|
end
|
|
18
18
|
|
|
19
19
|
def within?(from, to)
|
|
20
|
-
interior = ((from + 1)
|
|
20
|
+
interior = ((from + 1)..to)
|
|
21
21
|
interior.cover?(start_offset) && interior.cover?(finish)
|
|
22
22
|
end
|
|
23
23
|
|
|
@@ -66,6 +66,7 @@ end
|
|
|
66
66
|
|
|
67
67
|
class Kimera::MutationPoint
|
|
68
68
|
CLASS_BODY_REASON = "class-body DSL (runs at load; evaluate with --isolated)"
|
|
69
|
+
UNMUTATABLE = "unmutatable: "
|
|
69
70
|
SAFE = true
|
|
70
71
|
|
|
71
72
|
FIELDS = %i[point_id file operator node_type location original_source method_name mutants unsafe_reason].freeze
|
|
@@ -98,6 +99,18 @@ class Kimera::MutationPoint
|
|
|
98
99
|
unsafe_reason == CLASS_BODY_REASON
|
|
99
100
|
end
|
|
100
101
|
|
|
102
|
+
def reloadable?
|
|
103
|
+
!body? && !unmutatable?
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
def unmutatable?
|
|
107
|
+
unsafe_reason.to_s.start_with?(UNMUTATABLE)
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
def unmutatable!(detail)
|
|
111
|
+
unsafe!("#{UNMUTATABLE}#{detail}")
|
|
112
|
+
end
|
|
113
|
+
|
|
101
114
|
def safe
|
|
102
115
|
@attributes.fetch(:schema_safe, SAFE)
|
|
103
116
|
end
|
|
@@ -4,9 +4,10 @@ module Kimera
|
|
|
4
4
|
module Status
|
|
5
5
|
KILLING = %i[killed timeout error].freeze
|
|
6
6
|
|
|
7
|
-
UNJUDGED = %i[no_coverage ignored isolated_only harness_error].freeze
|
|
7
|
+
UNJUDGED = %i[no_coverage ignored isolated_only unmutatable harness_error].freeze
|
|
8
8
|
|
|
9
|
-
REPORTED =
|
|
9
|
+
REPORTED =
|
|
10
|
+
([:survived] + (KILLING - [:killed]) + %i[no_coverage harness_error ignored isolated_only unmutatable]).freeze
|
|
10
11
|
|
|
11
12
|
SELF_SUSPECT = ((KILLING - [:killed]) + [:survived]).freeze
|
|
12
13
|
end
|
|
@@ -10,7 +10,7 @@ end
|
|
|
10
10
|
class Kimera::RunReport
|
|
11
11
|
attr_reader :results, :leaks, :registry
|
|
12
12
|
|
|
13
|
-
CONDITIONAL = %i[harness_error ignored isolated_only leaks].freeze
|
|
13
|
+
CONDITIONAL = %i[harness_error ignored isolated_only unmutatable leaks].freeze
|
|
14
14
|
LABELS = { total: "mutants", harness_error: "unjudged" }.freeze
|
|
15
15
|
SCHEMA_VERSION = 1
|
|
16
16
|
|
|
@@ -11,5 +11,32 @@ module Kimera
|
|
|
11
11
|
def parse(source) = Kimera::Warnings.silence { Unparser.parse(source) }
|
|
12
12
|
|
|
13
13
|
def unparse(node) = Kimera::Warnings.silence { Unparser.unparse(node) }
|
|
14
|
+
|
|
15
|
+
module Binders
|
|
16
|
+
private
|
|
17
|
+
|
|
18
|
+
def enter(node)
|
|
19
|
+
super
|
|
20
|
+
Binders.names(node).each { |name| define(name) }
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
class << self
|
|
24
|
+
def names(node)
|
|
25
|
+
case node.type
|
|
26
|
+
when :match_var, :blockarg then [node.children.first].compact
|
|
27
|
+
when :match_with_lvasgn then captures(node.children.first)
|
|
28
|
+
else []
|
|
29
|
+
end
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def captures(regexp)
|
|
33
|
+
*parts, options = regexp.children
|
|
34
|
+
flags = options.children.include?(:x) ? Regexp::EXTENDED : 0
|
|
35
|
+
Regexp.new(parts.sum("") { |part| part.children.first }, flags).names.map(&:to_sym)
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
end
|
|
14
39
|
end
|
|
15
40
|
end
|
|
41
|
+
|
|
42
|
+
Unparser::AST::LocalVariableScopeEnumerator.prepend(Kimera::Unparse::Binders)
|
|
@@ -6,6 +6,8 @@ require_relative "overlay_splice"
|
|
|
6
6
|
class Kimera::Overlay::FileWeave
|
|
7
7
|
include Kimera::OverlaySplice
|
|
8
8
|
|
|
9
|
+
UNGUARDABLE = "unparser could not round-trip its guard"
|
|
10
|
+
|
|
9
11
|
def initialize(file, map, safe, unsafe)
|
|
10
12
|
@file = file
|
|
11
13
|
@map = map
|
|
@@ -43,16 +45,15 @@ class Kimera::Overlay::FileWeave
|
|
|
43
45
|
def finish(viable)
|
|
44
46
|
dropped = @safe - viable
|
|
45
47
|
notice(dropped)
|
|
46
|
-
|
|
47
|
-
result.skipped_unsafe.concat(dropped.flat_map(&:ids))
|
|
48
|
-
result
|
|
48
|
+
dropped.each { |point| point.unmutatable!(UNGUARDABLE) }
|
|
49
|
+
attempt(viable).tap { |result| result.skipped_unsafe.concat(dropped.flat_map(&:ids)) }
|
|
49
50
|
end
|
|
50
51
|
|
|
51
52
|
def notice(dropped)
|
|
52
53
|
io.puts(
|
|
53
54
|
"kimera: #{@file}: #{dropped.size} mutation point(s) at " \
|
|
54
55
|
"#{dropped.map { |p| "line #{p.location.start_line}" }.uniq.join(", ")} " \
|
|
55
|
-
"are unguardable (unparser round-trip); their mutants are reported
|
|
56
|
+
"are unguardable (unparser round-trip); their mutants are reported unmutatable"
|
|
56
57
|
)
|
|
57
58
|
end
|
|
58
59
|
|
|
@@ -1,5 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require_relative "splice_reopening"
|
|
4
|
+
|
|
3
5
|
module Kimera
|
|
4
6
|
module OverlaySplice
|
|
5
7
|
private
|
|
@@ -13,7 +15,7 @@ module Kimera
|
|
|
13
15
|
return unless ready?
|
|
14
16
|
edits, dropped = gather
|
|
15
17
|
return if edits.empty?
|
|
16
|
-
source = apply(edits)
|
|
18
|
+
source = apply(edits + reopenings)
|
|
17
19
|
[edits, dropped, source] if Kimera::Syntax.parse(source).success?
|
|
18
20
|
end
|
|
19
21
|
|
|
@@ -24,6 +26,8 @@ module Kimera
|
|
|
24
26
|
[edits, dropped + (@safe - claimed)]
|
|
25
27
|
end
|
|
26
28
|
|
|
29
|
+
def reopenings = Kimera::SpliceReopening.new(@map).edits
|
|
30
|
+
|
|
27
31
|
def collect(definition, points, edits, dropped)
|
|
28
32
|
rewritten = rewrite(definition, points)
|
|
29
33
|
rewritten ? edits << rewritten : dropped.concat(points)
|
|
@@ -32,6 +36,7 @@ module Kimera
|
|
|
32
36
|
def report(edits, dropped, source)
|
|
33
37
|
applied = edits.flat_map(&:last)
|
|
34
38
|
announce(applied, dropped)
|
|
39
|
+
dropped.each { |point| point.unmutatable!("unparser could not round-trip its method") }
|
|
35
40
|
build(applied, dropped, source)
|
|
36
41
|
end
|
|
37
42
|
|
|
@@ -46,7 +51,7 @@ module Kimera
|
|
|
46
51
|
io.puts(
|
|
47
52
|
"kimera: #{@file}: file-level round-trip failed; " \
|
|
48
53
|
"#{applied.size} mutant(s) spliced per method, " \
|
|
49
|
-
"#{dropped.flat_map(&:ids).size} reported
|
|
54
|
+
"#{dropped.flat_map(&:ids).size} reported unmutatable"
|
|
50
55
|
)
|
|
51
56
|
end
|
|
52
57
|
|
|
@@ -0,0 +1,57 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "../rewrite/ast_walk"
|
|
4
|
+
require_relative "guardrail_value_objects"
|
|
5
|
+
|
|
6
|
+
class Kimera::SpliceReopening
|
|
7
|
+
FACTORIES = Kimera::GuardrailValueObjects::VALUE_OBJECT_FACTORIES
|
|
8
|
+
LEXICAL = "(is_a?(::Module) ? self : ::Object)"
|
|
9
|
+
|
|
10
|
+
Assignment =
|
|
11
|
+
Data.define(:scope, :name, :value, :location) do
|
|
12
|
+
def edit = [location.expression.begin_pos, value.location.begin.begin_pos, header, []]
|
|
13
|
+
|
|
14
|
+
def header
|
|
15
|
+
owner = base
|
|
16
|
+
"#{constant} = (#{owner}.const_defined?(:#{name}, false) ? #{owner}.const_get(:#{name}) : " \
|
|
17
|
+
"#{factory}); #{constant}.class_eval "
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def constant = location.expression.begin.join(location.name).source
|
|
21
|
+
|
|
22
|
+
def factory = value.children.first.location.expression.source
|
|
23
|
+
|
|
24
|
+
def base
|
|
25
|
+
return LEXICAL unless scope
|
|
26
|
+
scope.type == :cbase ? "::Object" : scope.location.expression.source
|
|
27
|
+
end
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def initialize(map)
|
|
31
|
+
@map = map
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def edits = constants.map(&:edit)
|
|
35
|
+
|
|
36
|
+
private
|
|
37
|
+
|
|
38
|
+
def constants
|
|
39
|
+
found = []
|
|
40
|
+
Kimera::Rewrite::AstWalk.visit(@map.ast) { |node| found << assignment(node) if reopenable?(node) }
|
|
41
|
+
found
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def assignment(node) = Assignment.new(*node, node.location)
|
|
45
|
+
|
|
46
|
+
def reopenable?(node)
|
|
47
|
+
return false unless node.type == :casgn
|
|
48
|
+
value = node.children.last
|
|
49
|
+
value.is_a?(Parser::AST::Node) && value.type == :block && factory?(value.children.first)
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def factory?(call)
|
|
53
|
+
receiver, selector = call.children
|
|
54
|
+
call.type == :send && receiver.is_a?(Parser::AST::Node) && receiver.type == :const &&
|
|
55
|
+
FACTORIES[receiver.children.last] == selector
|
|
56
|
+
end
|
|
57
|
+
end
|
data/skills/kimera/SKILL.md
CHANGED
|
@@ -72,7 +72,8 @@ bundle exec kimera run --isolated --jobs 4 # oracle mode (see Strengthen)
|
|
|
72
72
|
- Rails `enum` models overlay warm (the re-declaration is idempotent), so
|
|
73
73
|
their method-body mutants are judged like any other. A file labeled
|
|
74
74
|
`unmutatable` could not be overlaid (a distinct, reported reason); it is not
|
|
75
|
-
an `--isolated` case.
|
|
75
|
+
an `--isolated` case. Its mutants report as `unmutatable` (never gating,
|
|
76
|
+
never scored, with the reason in the detail), not `no_coverage`.
|
|
76
77
|
|
|
77
78
|
## Triage a surviving mutant: the only four verdicts
|
|
78
79
|
|
|
@@ -118,9 +119,12 @@ bundle exec kimera run --isolated --jobs 4 # oracle mode (see Strengthen)
|
|
|
118
119
|
suppress the warm path.
|
|
119
120
|
- `--isolated` judges on failed tests, not the exit status. A child that
|
|
120
121
|
exits non-zero with zero failures (typically a SimpleCov `minimum_coverage`
|
|
121
|
-
floor tripped by the partial run) is `harness_error`, not a kill
|
|
122
|
-
|
|
123
|
-
|
|
122
|
+
floor tripped by the partial run) is `harness_error`, not a kill, and so is
|
|
123
|
+
a child that crashes before reporting (read its `detail` for the stderr).
|
|
124
|
+
Kimera
|
|
125
|
+
sets `KIMERA=1` wherever it loads the suite (warm, `--isolated`, doctor), so
|
|
126
|
+
gate the floor on it (`minimum_coverage ... unless ENV["KIMERA"]`); `kimera
|
|
127
|
+
doctor` warns when it isn't.
|
|
124
128
|
- A cluster of `error`/`timeout` verdicts in one region usually means harness
|
|
125
129
|
fragility or a missing guard, not test strength. Read the cluster before
|
|
126
130
|
counting the detections.
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: kimera
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Giacomo GK
|
|
@@ -116,13 +116,16 @@ files:
|
|
|
116
116
|
- lib/kimera/execution/isolated_verdict.rb
|
|
117
117
|
- lib/kimera/execution/isolated_watchdog.rb
|
|
118
118
|
- lib/kimera/execution/isolation.rb
|
|
119
|
+
- lib/kimera/execution/kinship.rb
|
|
119
120
|
- lib/kimera/execution/null_progress.rb
|
|
120
121
|
- lib/kimera/execution/overlay_guard_modules.rb
|
|
121
122
|
- lib/kimera/execution/overlay_guards.rb
|
|
122
123
|
- lib/kimera/execution/parallel_test_databases.rb
|
|
123
124
|
- lib/kimera/execution/pool_driver.rb
|
|
124
125
|
- lib/kimera/execution/priority.rb
|
|
126
|
+
- lib/kimera/execution/reflections.rb
|
|
125
127
|
- lib/kimera/execution/reload.rb
|
|
128
|
+
- lib/kimera/execution/reload_errand.rb
|
|
126
129
|
- lib/kimera/execution/rig.rb
|
|
127
130
|
- lib/kimera/execution/schedule.rb
|
|
128
131
|
- lib/kimera/execution/schemata.rb
|
|
@@ -131,6 +134,7 @@ files:
|
|
|
131
134
|
- lib/kimera/execution/shift/coverage_channel.rb
|
|
132
135
|
- lib/kimera/execution/shift/killer_memory.rb
|
|
133
136
|
- lib/kimera/execution/shift/leak_guard.rb
|
|
137
|
+
- lib/kimera/execution/suite_env.rb
|
|
134
138
|
- lib/kimera/execution/sweep.rb
|
|
135
139
|
- lib/kimera/execution/transaction_isolation.rb
|
|
136
140
|
- lib/kimera/execution/trial.rb
|
|
@@ -231,6 +235,7 @@ files:
|
|
|
231
235
|
- lib/kimera/synthesis/overlay_splice.rb
|
|
232
236
|
- lib/kimera/synthesis/project.rb
|
|
233
237
|
- lib/kimera/synthesis/source_map.rb
|
|
238
|
+
- lib/kimera/synthesis/splice_reopening.rb
|
|
234
239
|
- lib/kimera/synthesis/superclass_pin.rb
|
|
235
240
|
- lib/kimera/version.rb
|
|
236
241
|
- skills/kimera/SKILL.md
|