kimera 0.1.2 → 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +29 -0
- data/README.md +4 -1
- data/lib/kimera/cli/report_file.rb +1 -1
- data/lib/kimera/execution/baseline_failure.rb +4 -8
- data/lib/kimera/execution/baseline_pass.rb +2 -1
- data/lib/kimera/execution/isolated_child.rb +2 -1
- data/lib/kimera/execution/isolated_child_command.rb +11 -4
- data/lib/kimera/execution/isolated_child_minitest.rb +10 -5
- data/lib/kimera/execution/isolated_outcome.rb +25 -5
- data/lib/kimera/execution/isolated_verdict.rb +6 -3
- data/lib/kimera/execution/kinship.rb +27 -0
- data/lib/kimera/execution/overlay_guard_modules.rb +17 -0
- data/lib/kimera/execution/overlay_guards.rb +6 -0
- data/lib/kimera/execution/reflections.rb +35 -0
- data/lib/kimera/execution/reload.rb +33 -39
- data/lib/kimera/execution/reload_errand.rb +39 -0
- data/lib/kimera/execution/schedule.rb +17 -9
- data/lib/kimera/frameworks/adapter.rb +5 -1
- data/lib/kimera/frameworks/minitest_adapter.rb +19 -0
- data/lib/kimera/incremental/session.rb +1 -1
- data/lib/kimera/support/version.rb +1 -1
- data/skills/kimera/SKILL.md +3 -1
- metadata +4 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 558ffe918ec2624918506fb0616bd8426d8bb1a4c569abb4dd31e4ef9e74217d
|
|
4
|
+
data.tar.gz: 8ab25fa8b1f4c06085fb63606e20b1fb26691017d1bffc06c2d5dfa59b9e743c
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 889e8498b185c70c2e933954c2343729bfa8292a9651bd3b9f583080e85f9e7c212c4590c9d23388c6415798e81eff89620a598701ce2de1854956f806873434
|
|
7
|
+
data.tar.gz: bdd9bfab7f3409e107e52921bf6f210f0c25cf56a58b3afa6ebd01485ef1ea806fc5165b7fa732abd19bc1ec3eb59545d6055960f38ba6062c934891bfc9fa72
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,34 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.1.3 (2026-09-27)
|
|
4
|
+
|
|
5
|
+
- `--isolated` no longer scores a child that crashes before reporting results
|
|
6
|
+
as a kill. It is `harness_error`, and its `detail` gives the exit status and
|
|
7
|
+
the ends of the child's stderr. A Minitest child that fails to load a test
|
|
8
|
+
file still reports that as a kill, as the RSpec child already did.
|
|
9
|
+
- `--isolated` passes test ids and test files to the child in a request file
|
|
10
|
+
instead of argv, so a mutant covered by thousands of tests no longer fails
|
|
11
|
+
with `Errno::E2BIG: Argument list too long`.
|
|
12
|
+
- Warm reload-path mutants (for example, statements in a memoized method) run
|
|
13
|
+
one test at a time and stop at the first failure. Tests covering the
|
|
14
|
+
mutant's method run first. The hard timeout now applies to each test, not
|
|
15
|
+
the whole suite, so a mutant a test kills is no longer reported as
|
|
16
|
+
"reload worker produced no result". A test that overruns the timeout is a
|
|
17
|
+
`timeout`, and a worker that dies names its exit status or signal.
|
|
18
|
+
- A scope or association declared in a concern's `included do` block is now
|
|
19
|
+
instrumented in the classes that already included the concern. It used to be
|
|
20
|
+
reported `no_coverage`, or `survived` when it was covered through another
|
|
21
|
+
path.
|
|
22
|
+
- An association overlaid on a parent model now also reaches subclasses that
|
|
23
|
+
declared associations of their own (such as STI children). They used to keep
|
|
24
|
+
the unguarded reflection, so mutants in the association's scope lambda
|
|
25
|
+
survived.
|
|
26
|
+
- The red-baseline hint prints a plain Minitest command for a Minitest suite
|
|
27
|
+
instead of `rspec …`.
|
|
28
|
+
- `kimera report`/`kimera mutant` and incremental sessions read report JSON as
|
|
29
|
+
UTF-8, so they no longer crash under a non-UTF-8 locale when a report
|
|
30
|
+
contains non-ASCII text.
|
|
31
|
+
|
|
3
32
|
## 0.1.2 (2026-09-26)
|
|
4
33
|
|
|
5
34
|
- Kimera sets `KIMERA=1` in its own process before loading the suite, and in
|
data/README.md
CHANGED
|
@@ -220,7 +220,10 @@ selector and Kimera's own harness-critical files (see below).
|
|
|
220
220
|
|
|
221
221
|
An isolated child is judged on its failed tests, not its exit status. A child
|
|
222
222
|
that exits non-zero with no failing test is `harness_error` (unjudged, gated
|
|
223
|
-
by `--max-errors`), not a kill.
|
|
223
|
+
by `--max-errors`), not a kill. So is a child that dies before reporting any
|
|
224
|
+
results (a boot error, an exit hook, a crash); its `detail` carries the
|
|
225
|
+
child's exit status and the ends of its stderr. The usual cause of a non-zero
|
|
226
|
+
exit with no failures is a coverage floor such as
|
|
224
227
|
SimpleCov's `minimum_coverage`, which a partial run always trips. Kimera sets
|
|
225
228
|
`KIMERA=1` wherever it loads your suite (the warm process, `--isolated`
|
|
226
229
|
children, and `kimera doctor`'s test commands), so skip the floor there:
|
|
@@ -13,7 +13,7 @@ module Kimera::CLI::ReportFile
|
|
|
13
13
|
module_function
|
|
14
14
|
|
|
15
15
|
def parse(path)
|
|
16
|
-
JSON.parse(File.read(path))
|
|
16
|
+
JSON.parse(File.read(path, encoding: Encoding::UTF_8))
|
|
17
17
|
rescue JSON::ParserError => error
|
|
18
18
|
raise(Kimera::UsageError, "unreadable report #{path}: #{error.message}")
|
|
19
19
|
end
|
|
@@ -12,14 +12,14 @@ class Kimera::Execution::BaselineFailure < Kimera::Error
|
|
|
12
12
|
MAX_MESSAGE = 300
|
|
13
13
|
|
|
14
14
|
class << self
|
|
15
|
-
def build(failed, messages)
|
|
16
|
-
new(summary(failed, messages))
|
|
15
|
+
def build(failed, messages, command)
|
|
16
|
+
new(summary(failed, messages, command))
|
|
17
17
|
end
|
|
18
18
|
|
|
19
|
-
def summary(failed, messages)
|
|
19
|
+
def summary(failed, messages, command)
|
|
20
20
|
text = "baseline suite is not green: #{failed.join(", ")}"
|
|
21
21
|
text += appendix(failed, messages)
|
|
22
|
-
"#{text}\n#{
|
|
22
|
+
"#{text}\n reproduce without kimera: #{command}"
|
|
23
23
|
end
|
|
24
24
|
|
|
25
25
|
def appendix(failed, messages)
|
|
@@ -39,9 +39,5 @@ class Kimera::Execution::BaselineFailure < Kimera::Error
|
|
|
39
39
|
def truncate(text)
|
|
40
40
|
text.length > MAX_MESSAGE ? "#{text[0, MAX_MESSAGE]}…" : text
|
|
41
41
|
end
|
|
42
|
-
|
|
43
|
-
def reproduce(failed)
|
|
44
|
-
" reproduce without kimera: rspec #{failed.first(MAX_DETAILS).join(" ")} --order defined"
|
|
45
|
-
end
|
|
46
42
|
end
|
|
47
43
|
end
|
|
@@ -118,6 +118,7 @@ class Kimera::Execution::BaselinePass
|
|
|
118
118
|
end
|
|
119
119
|
|
|
120
120
|
def failure!(failed)
|
|
121
|
-
|
|
121
|
+
command = @adapter.reproduce(failed.first(Kimera::Execution::BaselineFailure::MAX_DETAILS))
|
|
122
|
+
raise(Kimera::Execution::BaselineFailure.build(failed, @messages, command))
|
|
122
123
|
end
|
|
123
124
|
end
|
|
@@ -13,7 +13,8 @@ RSpec.configuration.output_stream = StringIO.new
|
|
|
13
13
|
RSpec.configuration.error_stream = StringIO.new
|
|
14
14
|
RSpec.configuration.deprecation_stream = StringIO.new
|
|
15
15
|
|
|
16
|
-
ledger,
|
|
16
|
+
ledger, request = ARGV
|
|
17
|
+
locations = JSON.parse(File.read(request, encoding: Encoding::UTF_8)).fetch("tests")
|
|
17
18
|
code = RSpec::Core::Runner.run(locations)
|
|
18
19
|
|
|
19
20
|
failed = RSpec.world.all_examples.select { |example| example.execution_result.status == :failed }
|
|
@@ -1,5 +1,6 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
|
+
require "json"
|
|
3
4
|
require_relative "suite_env"
|
|
4
5
|
|
|
5
6
|
module Kimera
|
|
@@ -19,19 +20,25 @@ class Kimera::Execution::IsolatedPlan::ChildCommand
|
|
|
19
20
|
end
|
|
20
21
|
|
|
21
22
|
def command(mirror, locations, ledger:, paths:)
|
|
23
|
+
File.write(request_path(ledger), JSON.generate(request(locations)))
|
|
22
24
|
includes = paths.flat_map { |path| ["-I", path] }
|
|
23
|
-
bundled(mirror, Kimera::Execution::SUITE_ENV.dup, ["ruby", *includes, "-I", helpers, *argv(
|
|
25
|
+
bundled(mirror, Kimera::Execution::SUITE_ENV.dup, ["ruby", *includes, "-I", helpers, *argv(ledger)])
|
|
24
26
|
end
|
|
25
27
|
|
|
26
|
-
def argv(
|
|
27
|
-
|
|
28
|
-
|
|
28
|
+
def argv(ledger)
|
|
29
|
+
[minitest? ? MINITEST_CHILD : CHILD, ledger, request_path(ledger)]
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def request(locations)
|
|
33
|
+
{ "tests" => locations, "files" => @test_files }
|
|
29
34
|
end
|
|
30
35
|
|
|
31
36
|
def helpers
|
|
32
37
|
minitest? ? "test" : "spec"
|
|
33
38
|
end
|
|
34
39
|
|
|
40
|
+
def request_path(ledger) = "#{ledger}.request"
|
|
41
|
+
|
|
35
42
|
private
|
|
36
43
|
|
|
37
44
|
def bundled(mirror, env, argv)
|
|
@@ -12,13 +12,18 @@ rescue NameError, ArgumentError
|
|
|
12
12
|
end
|
|
13
13
|
end
|
|
14
14
|
|
|
15
|
-
ledger,
|
|
16
|
-
|
|
17
|
-
files =
|
|
18
|
-
ids =
|
|
15
|
+
ledger, request = ARGV
|
|
16
|
+
request = JSON.parse(File.read(request, encoding: Encoding::UTF_8))
|
|
17
|
+
files = request.fetch("files")
|
|
18
|
+
ids = request.fetch("tests")
|
|
19
19
|
ARGV.clear
|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
begin
|
|
22
|
+
files.each { |f| require File.expand_path(f) }
|
|
23
|
+
rescue StandardError, ScriptError => error
|
|
24
|
+
File.write(ledger, JSON.generate(failures: 1, failing: [], load_error: "#{error.class}: #{error.message}"))
|
|
25
|
+
exit(1)
|
|
26
|
+
end
|
|
22
27
|
|
|
23
28
|
methods = {}
|
|
24
29
|
Minitest::Runnable.runnables.each do |runnable|
|
|
@@ -8,6 +8,9 @@ module Kimera
|
|
|
8
8
|
end
|
|
9
9
|
|
|
10
10
|
class Kimera::Execution::IsolatedOutcome
|
|
11
|
+
HEAD_LINES = 5
|
|
12
|
+
TAIL_LINES = 15
|
|
13
|
+
|
|
11
14
|
attr_reader :status, :failing, :detail
|
|
12
15
|
|
|
13
16
|
def initialize(status, failing = nil, detail = nil)
|
|
@@ -17,16 +20,33 @@ class Kimera::Execution::IsolatedOutcome
|
|
|
17
20
|
end
|
|
18
21
|
|
|
19
22
|
class << self
|
|
20
|
-
def judge(exit, path)
|
|
23
|
+
def judge(exit, path, stderr = "")
|
|
21
24
|
return new(:timeout) if exit == :timeout
|
|
22
25
|
ledger = read(path)
|
|
23
|
-
return unreported(exit) unless ledger
|
|
26
|
+
return unreported(exit, stderr) unless ledger
|
|
24
27
|
tally(exit, ledger)
|
|
25
28
|
end
|
|
26
29
|
|
|
27
30
|
private
|
|
28
31
|
|
|
29
|
-
def unreported(exit
|
|
32
|
+
def unreported(exit, stderr)
|
|
33
|
+
new(:harness_error, nil, "test child #{status_of(exit)} without reporting results#{tail(stderr)}")
|
|
34
|
+
end
|
|
35
|
+
|
|
36
|
+
def tail(stderr)
|
|
37
|
+
lines = stderr.to_s.strip.lines(chomp: true)
|
|
38
|
+
lines.empty? ? "" : ":\n#{ends(lines).join("\n")}"
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
def ends(lines)
|
|
42
|
+
return lines if lines.size <= HEAD_LINES + TAIL_LINES
|
|
43
|
+
[*lines.first(HEAD_LINES), "…", *lines.last(TAIL_LINES)]
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
def status_of(exit)
|
|
47
|
+
code = exit.exitstatus
|
|
48
|
+
code ? "exited #{code}" : "died on signal #{exit.termsig}"
|
|
49
|
+
end
|
|
30
50
|
|
|
31
51
|
def tally(exit, ledger)
|
|
32
52
|
failing = Array(ledger["failing"])
|
|
@@ -36,12 +56,12 @@ class Kimera::Execution::IsolatedOutcome
|
|
|
36
56
|
end
|
|
37
57
|
|
|
38
58
|
def unexplained(exit)
|
|
39
|
-
"suite
|
|
59
|
+
"suite #{status_of(exit)} with 0 failures " \
|
|
40
60
|
"(a coverage floor or exit hook? skip it when ENV[\"KIMERA\"] is set)"
|
|
41
61
|
end
|
|
42
62
|
|
|
43
63
|
def read(path)
|
|
44
|
-
ledger = JSON.parse(File.read(path))
|
|
64
|
+
ledger = JSON.parse(File.read(path, encoding: Encoding::UTF_8))
|
|
45
65
|
ledger if ledger.is_a?(Hash) && ledger["failures"].is_a?(Integer)
|
|
46
66
|
rescue Errno::ENOENT, JSON::ParserError
|
|
47
67
|
nil
|
|
@@ -38,13 +38,16 @@ module Kimera
|
|
|
38
38
|
def verdict(mirror, locations)
|
|
39
39
|
Dir.mktmpdir("kimera-ledger") do |dir|
|
|
40
40
|
ledger = File.join(dir, "ledger.json")
|
|
41
|
-
|
|
41
|
+
stderr = File.join(dir, "stderr.log")
|
|
42
|
+
IsolatedOutcome.judge(launch(mirror, locations, ledger, stderr), ledger, captured(stderr))
|
|
42
43
|
end
|
|
43
44
|
end
|
|
44
45
|
|
|
45
|
-
def
|
|
46
|
+
def captured(path) = File.read(path, encoding: Encoding::UTF_8).scrub
|
|
47
|
+
|
|
48
|
+
def launch(mirror, locations, ledger, stderr)
|
|
46
49
|
env, cmd = plan.command(mirror, locations, ledger)
|
|
47
|
-
waitfor(Process.spawn(env, *cmd, chdir: mirror, pgroup: true, out: File::NULL, err:
|
|
50
|
+
waitfor(Process.spawn(env, *cmd, chdir: mirror, pgroup: true, out: File::NULL, err: stderr))
|
|
48
51
|
end
|
|
49
52
|
|
|
50
53
|
def with_mirror
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "../registry/mutation_point"
|
|
4
|
+
|
|
5
|
+
module Kimera
|
|
6
|
+
module Execution
|
|
7
|
+
end
|
|
8
|
+
end
|
|
9
|
+
|
|
10
|
+
class Kimera::Execution::Kinship
|
|
11
|
+
def initialize(registry, coverage)
|
|
12
|
+
@registry = registry
|
|
13
|
+
@coverage = coverage
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def order(id, tests)
|
|
17
|
+
(related(@registry.index.fetch(id, Kimera::MutationPoint::NONE)) & tests) | tests
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
private
|
|
21
|
+
|
|
22
|
+
def related(point)
|
|
23
|
+
siblings = @registry.at(point.file).sort_by { |other| other.method_name == point.method_name ? 0 : 1 }
|
|
24
|
+
known = @coverage.to_h
|
|
25
|
+
siblings.flat_map { |other| other.ids.flat_map { |mutant| Array(known[mutant]) } }
|
|
26
|
+
end
|
|
27
|
+
end
|
|
@@ -1,6 +1,7 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require_relative "callback"
|
|
4
|
+
require_relative "reflections"
|
|
4
5
|
|
|
5
6
|
module Kimera
|
|
6
7
|
module Execution
|
|
@@ -56,14 +57,30 @@ module Kimera
|
|
|
56
57
|
define_method(hook) do |base = nil, &block|
|
|
57
58
|
next super(base, &block) unless block && OverlayGuardModules.replacing?(self, stored, base)
|
|
58
59
|
instance_variable_set(stored, block)
|
|
60
|
+
OverlayGuardModules.reapply(self, block)
|
|
59
61
|
end
|
|
60
62
|
end
|
|
61
63
|
end
|
|
62
64
|
|
|
63
65
|
def serialization = SERIALIZEGUARD
|
|
64
66
|
|
|
67
|
+
def reflection = Reflections::GUARD
|
|
68
|
+
|
|
69
|
+
MIXES = Module.instance_method(:include?)
|
|
70
|
+
|
|
65
71
|
def concern = CONCERNGUARD
|
|
66
72
|
|
|
73
|
+
def reapply(concern, block)
|
|
74
|
+
ObjectSpace.each_object(Class).each { |klass| klass.class_eval(&block) if owner?(klass, concern) }
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
def owner?(klass, concern)
|
|
78
|
+
return false if klass.singleton_class? || !mixes?(klass, concern)
|
|
79
|
+
[klass.superclass].compact.none? { |parent| mixes?(parent, concern) }
|
|
80
|
+
end
|
|
81
|
+
|
|
82
|
+
def mixes?(klass, concern) = MIXES.bind_call(klass, concern)
|
|
83
|
+
|
|
67
84
|
def replacing?(target, stored, base)
|
|
68
85
|
Kimera::Execution::OverlayGuards.overlaying? && !base && target.instance_variable_defined?(stored)
|
|
69
86
|
end
|
|
@@ -26,6 +26,7 @@ module Kimera
|
|
|
26
26
|
callbacks!
|
|
27
27
|
concern!
|
|
28
28
|
serialize!
|
|
29
|
+
reflections!
|
|
29
30
|
end
|
|
30
31
|
|
|
31
32
|
def enum!
|
|
@@ -43,6 +44,11 @@ module Kimera
|
|
|
43
44
|
ActiveRecord::Base.singleton_class.prepend(OverlayGuardModules.serialization)
|
|
44
45
|
end
|
|
45
46
|
|
|
47
|
+
def reflections!
|
|
48
|
+
return unless defined?(ActiveRecord::Reflection)
|
|
49
|
+
ActiveRecord::Reflection.singleton_class.prepend(OverlayGuardModules.reflection)
|
|
50
|
+
end
|
|
51
|
+
|
|
46
52
|
def concern!
|
|
47
53
|
return unless defined?(ActiveSupport::Concern)
|
|
48
54
|
ActiveSupport::Concern.prepend(OverlayGuardModules.concern)
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kimera
|
|
4
|
+
module Execution
|
|
5
|
+
module Reflections
|
|
6
|
+
module_function
|
|
7
|
+
|
|
8
|
+
GUARD =
|
|
9
|
+
Module.new do
|
|
10
|
+
def add_reflection(owner, name, reflection)
|
|
11
|
+
key = Reflections.key(owner, name)
|
|
12
|
+
stale = OverlayGuards.overlaying? && owner._reflections[key]
|
|
13
|
+
super
|
|
14
|
+
Reflections.inherit(owner, key, stale, reflection) if stale
|
|
15
|
+
end
|
|
16
|
+
end
|
|
17
|
+
|
|
18
|
+
def key(owner, name)
|
|
19
|
+
text = name.to_s
|
|
20
|
+
owner._reflections.key?(text) ? text : name.to_sym
|
|
21
|
+
end
|
|
22
|
+
|
|
23
|
+
def inherit(owner, key, stale, reflection)
|
|
24
|
+
owner.descendants.each { |heir| replace(heir, key, stale, reflection) }
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def replace(heir, key, stale, reflection)
|
|
28
|
+
held = heir._reflections
|
|
29
|
+
return unless held[key].equal?(stale)
|
|
30
|
+
heir.clear_reflections_cache
|
|
31
|
+
heir._reflections = held.merge(key => reflection)
|
|
32
|
+
end
|
|
33
|
+
end
|
|
34
|
+
end
|
|
35
|
+
end
|
|
@@ -5,38 +5,16 @@ require_relative "../runtime"
|
|
|
5
5
|
require_relative "../synthesis/overlay"
|
|
6
6
|
require_relative "child_process"
|
|
7
7
|
require_relative "overlay_guards"
|
|
8
|
+
require_relative "reload_errand"
|
|
8
9
|
require_relative "verdicts"
|
|
9
10
|
|
|
10
11
|
class Kimera::Execution::Reload
|
|
12
|
+
include Kimera::Execution::ChildProcess
|
|
13
|
+
|
|
11
14
|
Failure =
|
|
12
15
|
Data.define(:exception) do
|
|
13
16
|
def message = "#{exception.class}: #{exception.message}"
|
|
14
17
|
end
|
|
15
|
-
include Kimera::Execution::ChildProcess
|
|
16
|
-
|
|
17
|
-
Errand =
|
|
18
|
-
Struct.new(:id, :reader, :writer) do
|
|
19
|
-
def parent!
|
|
20
|
-
writer.close
|
|
21
|
-
end
|
|
22
|
-
|
|
23
|
-
def child!
|
|
24
|
-
reader.close
|
|
25
|
-
end
|
|
26
|
-
|
|
27
|
-
def emit(payload)
|
|
28
|
-
writer.puts(payload)
|
|
29
|
-
writer.close
|
|
30
|
-
end
|
|
31
|
-
|
|
32
|
-
def await(deadline, &)
|
|
33
|
-
return reader.gets if reader.wait_readable(deadline)
|
|
34
|
-
yield
|
|
35
|
-
nil
|
|
36
|
-
ensure
|
|
37
|
-
reader.close
|
|
38
|
-
end
|
|
39
|
-
end
|
|
40
18
|
|
|
41
19
|
def initialize(registry:, adapter:, isolation:, root:)
|
|
42
20
|
@registry = registry
|
|
@@ -45,8 +23,8 @@ class Kimera::Execution::Reload
|
|
|
45
23
|
@root = root
|
|
46
24
|
end
|
|
47
25
|
|
|
48
|
-
def run(id, deadline:)
|
|
49
|
-
errand = Errand.new(id, *IO.pipe)
|
|
26
|
+
def run(id, deadline:, tests: @adapter.test_ids)
|
|
27
|
+
errand = Errand.new(id, tests, *IO.pipe)
|
|
50
28
|
collect(errand, child(errand), deadline)
|
|
51
29
|
end
|
|
52
30
|
|
|
@@ -59,32 +37,42 @@ class Kimera::Execution::Reload
|
|
|
59
37
|
def work(errand)
|
|
60
38
|
errand.child!
|
|
61
39
|
silence!
|
|
62
|
-
errand.emit(report(errand
|
|
40
|
+
errand.emit(report(errand))
|
|
63
41
|
exit!(0)
|
|
64
42
|
end
|
|
65
43
|
|
|
66
|
-
def report(
|
|
67
|
-
status, fails = evaluate(
|
|
68
|
-
JSON.generate(id: id, status: status.to_s, fails: fails)
|
|
44
|
+
def report(errand)
|
|
45
|
+
status, fails = evaluate(errand)
|
|
46
|
+
JSON.generate(id: errand.id, status: status.to_s, fails: Array(fails).map { |fail| fail.to_s.scrub })
|
|
69
47
|
end
|
|
70
48
|
|
|
71
49
|
def collect(errand, pid, deadline)
|
|
72
50
|
line = errand.await(deadline) { kill(pid) }
|
|
73
|
-
reap(pid)
|
|
74
|
-
verdict(errand.id, line)
|
|
51
|
+
verdict(errand.id, line, reap(pid), deadline)
|
|
75
52
|
end
|
|
76
53
|
|
|
77
|
-
def evaluate(
|
|
78
|
-
mutate(
|
|
54
|
+
def evaluate(errand)
|
|
55
|
+
failed = mutate(errand)
|
|
56
|
+
failed ? failed.verdict : [:survived, []]
|
|
79
57
|
rescue StandardError, ScriptError => error
|
|
80
58
|
[:error, [Failure.new(error).message]]
|
|
81
59
|
end
|
|
82
60
|
|
|
83
|
-
def mutate(
|
|
61
|
+
def mutate(errand)
|
|
62
|
+
id = errand.id
|
|
84
63
|
file = verdicts.point(id).file
|
|
85
64
|
overlay(file, File.join(File.expand_path(@root), file), id)
|
|
86
65
|
Kimera::Runtime.active = nil
|
|
87
|
-
@isolation.around {
|
|
66
|
+
@isolation.around { hunt(errand) }
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
def hunt(errand)
|
|
70
|
+
errand.tests.lazy.map { |test| trial(test, errand) }.reject(&:passed?).first
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
def trial(test, errand)
|
|
74
|
+
errand.tick
|
|
75
|
+
@adapter.run([test])
|
|
88
76
|
end
|
|
89
77
|
|
|
90
78
|
def overlay(file, path, id)
|
|
@@ -92,12 +80,18 @@ class Kimera::Execution::Reload
|
|
|
92
80
|
Kimera::Execution::OverlayGuards.overlay { Kimera::Overlay.evaluate(baked, path) }
|
|
93
81
|
end
|
|
94
82
|
|
|
95
|
-
def verdict(id, line)
|
|
96
|
-
return verdicts.
|
|
83
|
+
def verdict(id, line, status = nil, deadline = nil)
|
|
84
|
+
return verdicts.timeout(id, deadline) if line == :timeout
|
|
85
|
+
return verdicts.unjudged(id, "reload worker produced no result#{exited(status)}") unless line
|
|
97
86
|
message = parse(line)
|
|
98
87
|
message ? verdicts.parse(message) : verdicts.unjudged(id, "reload worker output unparseable")
|
|
99
88
|
end
|
|
100
89
|
|
|
90
|
+
def exited(status)
|
|
91
|
+
return "" unless status
|
|
92
|
+
status.signaled? ? " (died on signal #{status.termsig})" : " (exited #{status.exitstatus})"
|
|
93
|
+
end
|
|
94
|
+
|
|
101
95
|
def verdicts
|
|
102
96
|
Kimera::Execution::Verdicts.new(@registry)
|
|
103
97
|
end
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kimera
|
|
4
|
+
module Execution
|
|
5
|
+
end
|
|
6
|
+
end
|
|
7
|
+
|
|
8
|
+
class Kimera::Execution::Reload; end
|
|
9
|
+
|
|
10
|
+
Kimera::Execution::Reload::Errand =
|
|
11
|
+
Struct.new(:id, :tests, :reader, :writer) do
|
|
12
|
+
tick = "tick\n"
|
|
13
|
+
def parent!
|
|
14
|
+
writer.close
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def child!
|
|
18
|
+
reader.close
|
|
19
|
+
writer.sync = true
|
|
20
|
+
end
|
|
21
|
+
define_method(:tick) { writer.write(tick) }
|
|
22
|
+
def emit(payload)
|
|
23
|
+
writer.puts(payload)
|
|
24
|
+
writer.close
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def await(deadline)
|
|
28
|
+
heard(deadline).tap { |line| yield if line == :timeout }
|
|
29
|
+
ensure
|
|
30
|
+
reader.close
|
|
31
|
+
end
|
|
32
|
+
define_method(:heard) do |deadline|
|
|
33
|
+
while reader.wait_readable(deadline)
|
|
34
|
+
line = reader.gets
|
|
35
|
+
return line unless line == tick
|
|
36
|
+
end
|
|
37
|
+
:timeout
|
|
38
|
+
end
|
|
39
|
+
end
|
|
@@ -1,12 +1,13 @@
|
|
|
1
1
|
# frozen_string_literal: true
|
|
2
2
|
|
|
3
3
|
require_relative "priority"
|
|
4
|
+
require_relative "kinship"
|
|
4
5
|
require_relative "reload"
|
|
5
6
|
require_relative "verdicts"
|
|
6
7
|
|
|
7
8
|
class Kimera::Execution::Schedule
|
|
8
9
|
Ledger = Struct.new(:results, :leaks)
|
|
9
|
-
Batch = Struct.new(:safe, :reloadable, :deferred, :ledger)
|
|
10
|
+
Batch = Struct.new(:safe, :reloadable, :deferred, :ledger, :coverage)
|
|
10
11
|
|
|
11
12
|
LOSSES = { timeout: :expired }.freeze
|
|
12
13
|
|
|
@@ -37,14 +38,16 @@ class Kimera::Execution::Schedule
|
|
|
37
38
|
safe, unsafe = ids.partition { |id| verdicts.safe?(id) }
|
|
38
39
|
reloadable, deferred = unsafe.partition { |id| verdicts.reloadable?(id) }
|
|
39
40
|
progress.start(safe.size + unsafe.size, label)
|
|
40
|
-
Batch.new(
|
|
41
|
+
Batch.new(prioritized(safe, coverage), reloadable, deferred, Ledger.new({}, []), coverage)
|
|
41
42
|
end
|
|
42
43
|
|
|
44
|
+
def prioritized(safe, coverage) = Kimera::Execution::Priority.new(coverage).order(safe)
|
|
45
|
+
|
|
43
46
|
def process(batch)
|
|
44
|
-
|
|
45
|
-
pool(safe, ledger)
|
|
46
|
-
reload(
|
|
47
|
-
defer(deferred, ledger)
|
|
47
|
+
ledger = batch.ledger
|
|
48
|
+
pool(batch.safe, ledger)
|
|
49
|
+
reload(batch)
|
|
50
|
+
defer(batch.deferred, ledger)
|
|
48
51
|
end
|
|
49
52
|
|
|
50
53
|
def finish(ledger)
|
|
@@ -58,8 +61,13 @@ class Kimera::Execution::Schedule
|
|
|
58
61
|
progress.tick(result.status)
|
|
59
62
|
end
|
|
60
63
|
|
|
61
|
-
def reload(
|
|
62
|
-
|
|
64
|
+
def reload(batch)
|
|
65
|
+
kinship = Kimera::Execution::Kinship.new(@registry, batch.coverage)
|
|
66
|
+
batch.reloadable.each { |id| record(batch.ledger, id, rerun(id, kinship)) }
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
def rerun(id, kinship)
|
|
70
|
+
reloader.run(id, deadline: hard, tests: kinship.order(id, @options.fetch(:adapter).test_ids))
|
|
63
71
|
end
|
|
64
72
|
|
|
65
73
|
def defer(ids, ledger)
|
|
@@ -97,7 +105,7 @@ class Kimera::Execution::Schedule
|
|
|
97
105
|
def crashed(id) = verdicts.unjudged(id, "worker crashed before result")
|
|
98
106
|
|
|
99
107
|
def reloader
|
|
100
|
-
Kimera::Execution::Reload.new(
|
|
108
|
+
@_reloader ||= Kimera::Execution::Reload.new(
|
|
101
109
|
registry: @registry, adapter: @options.fetch(:adapter),
|
|
102
110
|
isolation: @options.fetch(:isolation), root: @options.fetch(:root)
|
|
103
111
|
)
|
|
@@ -51,8 +51,27 @@ class Kimera::Frameworks::MinitestAdapter < Kimera::Frameworks::Adapter
|
|
|
51
51
|
Kimera::Frameworks::RunOutcome.new(passed: failed.empty?, failed_ids: failed, failures: failures)
|
|
52
52
|
end
|
|
53
53
|
|
|
54
|
+
def reproduce(ids)
|
|
55
|
+
"bundle exec ruby -Itest -rminitest/autorun -e #{quote(loads(ids))} -- -n #{quote(filter(ids))} --seed 1"
|
|
56
|
+
end
|
|
57
|
+
|
|
54
58
|
private
|
|
55
59
|
|
|
60
|
+
def loads(ids)
|
|
61
|
+
ids.filter_map { |id| origin(id) }.uniq.map { |file| "require File.expand_path(#{file.dump})" }.join("; ")
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def filter(ids) = "/^(?:#{ids.map { |id| Regexp.escape(id) }.join("|")})$/"
|
|
65
|
+
|
|
66
|
+
def origin(id)
|
|
67
|
+
klass, name = @methods[id]
|
|
68
|
+
return unless klass
|
|
69
|
+
path, = klass.instance_method(name).source_location
|
|
70
|
+
path.delete_prefix("#{Dir.pwd}#{File::SEPARATOR}")
|
|
71
|
+
end
|
|
72
|
+
|
|
73
|
+
def quote(text) = "'#{text.gsub("'") { "'\\''" }}'"
|
|
74
|
+
|
|
56
75
|
def record(id, failed, failures)
|
|
57
76
|
klass, name = @methods[id]
|
|
58
77
|
return unless klass
|
|
@@ -24,7 +24,7 @@ class Kimera::Incremental::Session
|
|
|
24
24
|
class << self
|
|
25
25
|
def load(path, registry: nil)
|
|
26
26
|
return new unless path && File.exist?(path)
|
|
27
|
-
session = decode(JSON.parse(File.read(path)))
|
|
27
|
+
session = decode(JSON.parse(File.read(path, encoding: Encoding::UTF_8)))
|
|
28
28
|
session.prune!(registry) if registry
|
|
29
29
|
session
|
|
30
30
|
end
|
data/skills/kimera/SKILL.md
CHANGED
|
@@ -119,7 +119,9 @@ bundle exec kimera run --isolated --jobs 4 # oracle mode (see Strengthen)
|
|
|
119
119
|
suppress the warm path.
|
|
120
120
|
- `--isolated` judges on failed tests, not the exit status. A child that
|
|
121
121
|
exits non-zero with zero failures (typically a SimpleCov `minimum_coverage`
|
|
122
|
-
floor tripped by the partial run) is `harness_error`, not a kill
|
|
122
|
+
floor tripped by the partial run) is `harness_error`, not a kill, and so is
|
|
123
|
+
a child that crashes before reporting (read its `detail` for the stderr).
|
|
124
|
+
Kimera
|
|
123
125
|
sets `KIMERA=1` wherever it loads the suite (warm, `--isolated`, doctor), so
|
|
124
126
|
gate the floor on it (`minimum_coverage ... unless ENV["KIMERA"]`); `kimera
|
|
125
127
|
doctor` warns when it isn't.
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: kimera
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.3
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Giacomo GK
|
|
@@ -116,13 +116,16 @@ files:
|
|
|
116
116
|
- lib/kimera/execution/isolated_verdict.rb
|
|
117
117
|
- lib/kimera/execution/isolated_watchdog.rb
|
|
118
118
|
- lib/kimera/execution/isolation.rb
|
|
119
|
+
- lib/kimera/execution/kinship.rb
|
|
119
120
|
- lib/kimera/execution/null_progress.rb
|
|
120
121
|
- lib/kimera/execution/overlay_guard_modules.rb
|
|
121
122
|
- lib/kimera/execution/overlay_guards.rb
|
|
122
123
|
- lib/kimera/execution/parallel_test_databases.rb
|
|
123
124
|
- lib/kimera/execution/pool_driver.rb
|
|
124
125
|
- lib/kimera/execution/priority.rb
|
|
126
|
+
- lib/kimera/execution/reflections.rb
|
|
125
127
|
- lib/kimera/execution/reload.rb
|
|
128
|
+
- lib/kimera/execution/reload_errand.rb
|
|
126
129
|
- lib/kimera/execution/rig.rb
|
|
127
130
|
- lib/kimera/execution/schedule.rb
|
|
128
131
|
- lib/kimera/execution/schemata.rb
|