kimera 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.kimera.yml.example +59 -0
- data/CHANGELOG.md +36 -0
- data/LICENSE.txt +21 -0
- data/README.md +528 -0
- data/exe/kimera +22 -0
- data/lib/kimera/audit/operator_audit.rb +71 -0
- data/lib/kimera/cli/argv.rb +15 -0
- data/lib/kimera/cli/baseline.rb +87 -0
- data/lib/kimera/cli/changed.rb +27 -0
- data/lib/kimera/cli/ci.rb +27 -0
- data/lib/kimera/cli/completion.rb +37 -0
- data/lib/kimera/cli/doctor.rb +115 -0
- data/lib/kimera/cli/flag.rb +55 -0
- data/lib/kimera/cli/help_text.rb +45 -0
- data/lib/kimera/cli/init.rb +105 -0
- data/lib/kimera/cli/mutant.rb +82 -0
- data/lib/kimera/cli/report_file.rb +20 -0
- data/lib/kimera/cli/run/arguments.rb +58 -0
- data/lib/kimera/cli/run/cycle.rb +78 -0
- data/lib/kimera/cli/run/digest.rb +49 -0
- data/lib/kimera/cli/run/emission.rb +45 -0
- data/lib/kimera/cli/run/gate.rb +43 -0
- data/lib/kimera/cli/run/options.rb +85 -0
- data/lib/kimera/cli/run/pass.rb +100 -0
- data/lib/kimera/cli/run/sources.rb +56 -0
- data/lib/kimera/cli/run.rb +52 -0
- data/lib/kimera/cli/skill.rb +23 -0
- data/lib/kimera/cli/suggestion.rb +36 -0
- data/lib/kimera/cli/survivors/panel.rb +81 -0
- data/lib/kimera/cli/survivors.rb +89 -0
- data/lib/kimera/cli/synthesize.rb +66 -0
- data/lib/kimera/cli/workflows.rb +8 -0
- data/lib/kimera/cli.rb +132 -0
- data/lib/kimera/error.rb +3 -0
- data/lib/kimera/execution/baseline_failure.rb +47 -0
- data/lib/kimera/execution/baseline_pass.rb +123 -0
- data/lib/kimera/execution/boot.rb +50 -0
- data/lib/kimera/execution/callback.rb +33 -0
- data/lib/kimera/execution/child_process.rb +48 -0
- data/lib/kimera/execution/composite_isolation.rb +24 -0
- data/lib/kimera/execution/harness.rb +150 -0
- data/lib/kimera/execution/isolated.rb +64 -0
- data/lib/kimera/execution/isolated_child.rb +15 -0
- data/lib/kimera/execution/isolated_child_command.rb +44 -0
- data/lib/kimera/execution/isolated_child_minitest.rb +34 -0
- data/lib/kimera/execution/isolated_plan.rb +54 -0
- data/lib/kimera/execution/isolated_scheduling.rb +54 -0
- data/lib/kimera/execution/isolated_test_selection.rb +22 -0
- data/lib/kimera/execution/isolated_verdict.rb +72 -0
- data/lib/kimera/execution/isolated_watchdog.rb +59 -0
- data/lib/kimera/execution/isolation.rb +17 -0
- data/lib/kimera/execution/null_progress.rb +17 -0
- data/lib/kimera/execution/overlay_guard_modules.rb +95 -0
- data/lib/kimera/execution/overlay_guards.rb +56 -0
- data/lib/kimera/execution/parallel_test_databases.rb +64 -0
- data/lib/kimera/execution/pool_driver.rb +101 -0
- data/lib/kimera/execution/priority.rb +23 -0
- data/lib/kimera/execution/reload.rb +104 -0
- data/lib/kimera/execution/rig.rb +41 -0
- data/lib/kimera/execution/schedule.rb +105 -0
- data/lib/kimera/execution/schemata.rb +88 -0
- data/lib/kimera/execution/shift/attempt.rb +34 -0
- data/lib/kimera/execution/shift/coverage_channel.rb +43 -0
- data/lib/kimera/execution/shift/killer_memory.rb +20 -0
- data/lib/kimera/execution/shift/leak_guard.rb +34 -0
- data/lib/kimera/execution/shift.rb +137 -0
- data/lib/kimera/execution/sweep.rb +30 -0
- data/lib/kimera/execution/transaction_isolation.rb +32 -0
- data/lib/kimera/execution/trial.rb +16 -0
- data/lib/kimera/execution/verdicts.rb +78 -0
- data/lib/kimera/execution/worker_pool/fleet.rb +104 -0
- data/lib/kimera/execution/worker_pool/stillborn_guard.rb +40 -0
- data/lib/kimera/execution/worker_pool/worker.rb +32 -0
- data/lib/kimera/execution/worker_pool.rb +101 -0
- data/lib/kimera/frameworks/adapter.rb +82 -0
- data/lib/kimera/frameworks/minitest_adapter.rb +102 -0
- data/lib/kimera/frameworks/rspec_adapter.rb +123 -0
- data/lib/kimera/frameworks/rspec_group_index.rb +45 -0
- data/lib/kimera/incremental/git_diff.rb +98 -0
- data/lib/kimera/incremental/selection.rb +27 -0
- data/lib/kimera/incremental/session.rb +109 -0
- data/lib/kimera/memoization.rb +104 -0
- data/lib/kimera/operators/argument_drop.rb +19 -0
- data/lib/kimera/operators/arithmetic.rb +12 -0
- data/lib/kimera/operators/audit.rb +3 -0
- data/lib/kimera/operators/base.rb +40 -0
- data/lib/kimera/operators/binary_swap.rb +23 -0
- data/lib/kimera/operators/boolean.rb +26 -0
- data/lib/kimera/operators/chain_link_deletion.rb +25 -0
- data/lib/kimera/operators/collection_literal.rb +18 -0
- data/lib/kimera/operators/comparison.rb +13 -0
- data/lib/kimera/operators/conditional.rb +36 -0
- data/lib/kimera/operators/default_argument.rb +17 -0
- data/lib/kimera/operators/element_drop.rb +31 -0
- data/lib/kimera/operators/index_fetch.rb +15 -0
- data/lib/kimera/operators/kernel_coercion.rb +17 -0
- data/lib/kimera/operators/method_unwrap.rb +20 -0
- data/lib/kimera/operators/negation.rb +15 -0
- data/lib/kimera/operators/node_swap.rb +20 -0
- data/lib/kimera/operators/numeric_literal.rb +39 -0
- data/lib/kimera/operators/op_assign.rb +18 -0
- data/lib/kimera/operators/plugins.rb +27 -0
- data/lib/kimera/operators/rails_association.rb +24 -0
- data/lib/kimera/operators/rails_callback.rb +49 -0
- data/lib/kimera/operators/rails_declaration.rb +16 -0
- data/lib/kimera/operators/rails_permit.rb +34 -0
- data/lib/kimera/operators/rails_validation.rb +17 -0
- data/lib/kimera/operators/range.rb +15 -0
- data/lib/kimera/operators/regexp_literal.rb +16 -0
- data/lib/kimera/operators/respond_to_guard.rb +24 -0
- data/lib/kimera/operators/return_value.rb +36 -0
- data/lib/kimera/operators/safe_navigation.rb +15 -0
- data/lib/kimera/operators/selector_swap.rb +24 -0
- data/lib/kimera/operators/statement_deletion.rb +26 -0
- data/lib/kimera/operators/string_literal.rb +22 -0
- data/lib/kimera/operators/symbol_literal.rb +26 -0
- data/lib/kimera/operators/vocabulary.rb +82 -0
- data/lib/kimera/operators.rb +141 -0
- data/lib/kimera/plugins.rb +3 -0
- data/lib/kimera/registry/builder.rb +59 -0
- data/lib/kimera/registry/mutation_point.rb +146 -0
- data/lib/kimera/registry/numbering.rb +15 -0
- data/lib/kimera/registry/registry.rb +120 -0
- data/lib/kimera/registry/source_file.rb +71 -0
- data/lib/kimera/registry/tally.rb +11 -0
- data/lib/kimera/registry/walking.rb +55 -0
- data/lib/kimera/report/actions.rb +38 -0
- data/lib/kimera/report/coloring.rb +25 -0
- data/lib/kimera/report/formats.rb +78 -0
- data/lib/kimera/report/live.rb +56 -0
- data/lib/kimera/report/log.rb +46 -0
- data/lib/kimera/report/progress.rb +55 -0
- data/lib/kimera/report/screen.rb +44 -0
- data/lib/kimera/report/sections.rb +50 -0
- data/lib/kimera/report/tally.rb +84 -0
- data/lib/kimera/report/text.rb +104 -0
- data/lib/kimera/results/result.rb +72 -0
- data/lib/kimera/results/run_report.rb +114 -0
- data/lib/kimera/rewrite/ast_walk.rb +20 -0
- data/lib/kimera/rewrite/directive.rb +61 -0
- data/lib/kimera/rewrite/directive_extra_handlers.rb +52 -0
- data/lib/kimera/rewrite/directive_handlers.rb +138 -0
- data/lib/kimera/rewrite/directive_kwarg_handlers.rb +44 -0
- data/lib/kimera/rewrite/numbered_params.rb +50 -0
- data/lib/kimera/runtime.rb +51 -0
- data/lib/kimera/scope/config.rb +90 -0
- data/lib/kimera/scope/file_set.rb +51 -0
- data/lib/kimera/scope/ignore_list.rb +55 -0
- data/lib/kimera/self_protection.rb +56 -0
- data/lib/kimera/support/duration.rb +13 -0
- data/lib/kimera/support/errors.rb +8 -0
- data/lib/kimera/support/operator_protocol.rb +10 -0
- data/lib/kimera/support/syntax.rb +68 -0
- data/lib/kimera/support/syntax_types.rb +20 -0
- data/lib/kimera/support/unparse.rb +15 -0
- data/lib/kimera/support/version.rb +5 -0
- data/lib/kimera/support/warnings.rb +15 -0
- data/lib/kimera/synthesis/file_weave.rb +78 -0
- data/lib/kimera/synthesis/guard_weaver.rb +77 -0
- data/lib/kimera/synthesis/guardrail_dispatch.rb +68 -0
- data/lib/kimera/synthesis/guardrail_interpolation.rb +59 -0
- data/lib/kimera/synthesis/guardrail_value_objects.rb +46 -0
- data/lib/kimera/synthesis/overlay.rb +52 -0
- data/lib/kimera/synthesis/overlay_splice.rb +101 -0
- data/lib/kimera/synthesis/project.rb +58 -0
- data/lib/kimera/synthesis/source_map.rb +42 -0
- data/lib/kimera/version.rb +3 -0
- data/lib/kimera.rb +13 -0
- data/skills/kimera/SKILL.md +118 -0
- metadata +259 -0
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kimera
|
|
4
|
+
module GuardrailInterpolation
|
|
5
|
+
INTERPOLATING = %i[dstr dsym regexp xstr].freeze
|
|
6
|
+
|
|
7
|
+
private
|
|
8
|
+
|
|
9
|
+
def type?(node, type)
|
|
10
|
+
node.is_a?(Parser::AST::Node) && node.type == type
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
def string?(node)
|
|
14
|
+
type?(node, :str)
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def empty?(node)
|
|
18
|
+
string?(node) && node.children.first == ""
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def interpolation?(node)
|
|
22
|
+
type?(node, :begin)
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
def flatten(node)
|
|
26
|
+
spliced = node.children.flat_map { |child| type?(child, :dstr) ? child.children : [child] }
|
|
27
|
+
spliced = merge(spliced) if spliced.any? { |child| interpolation?(child) }
|
|
28
|
+
node.updated(nil, spliced)
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def merge(children)
|
|
32
|
+
children.each_with_object([]) { |child, items| append(items, child) }
|
|
33
|
+
end
|
|
34
|
+
|
|
35
|
+
def append(items, child)
|
|
36
|
+
previous = items.last
|
|
37
|
+
return items << child unless string?(child) && string?(previous)
|
|
38
|
+
items[-1] = ast(:str, previous.children.first + child.children.first)
|
|
39
|
+
end
|
|
40
|
+
|
|
41
|
+
def prune(node)
|
|
42
|
+
Kimera::Rewrite::AstWalk.rebuild(node) do |rebuilt, _original|
|
|
43
|
+
INTERPOLATING.include?(rebuilt.type) ? filter(rebuilt) : rebuilt
|
|
44
|
+
end
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
def filter(node)
|
|
48
|
+
kept = node.children.reject { |child| empty?(child) }
|
|
49
|
+
return ast(:str, "") if node.type == :dstr && kept.empty?
|
|
50
|
+
collapse(node, kept)
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def collapse(node, kept)
|
|
54
|
+
first = kept.first
|
|
55
|
+
return first if node.type == :dstr && kept.one? && %i[str dstr].include?(first.type)
|
|
56
|
+
node.updated(nil, kept)
|
|
57
|
+
end
|
|
58
|
+
end
|
|
59
|
+
end
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kimera
|
|
4
|
+
module GuardrailValueObjects
|
|
5
|
+
VALUE_OBJECT_FACTORIES = { Struct: :new, Data: :define }.freeze
|
|
6
|
+
|
|
7
|
+
private
|
|
8
|
+
|
|
9
|
+
def reopen(node)
|
|
10
|
+
return node unless node.type == :casgn
|
|
11
|
+
scope, name, value = node.children
|
|
12
|
+
return node unless type?(value, :block)
|
|
13
|
+
block(node, scope, name, value)
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
def block(node, scope, name, value)
|
|
17
|
+
factory, _arguments, body = value.children
|
|
18
|
+
return node unless factory?(factory)
|
|
19
|
+
reopened(node, scope, name, factory, body)
|
|
20
|
+
end
|
|
21
|
+
|
|
22
|
+
def reopened(node, scope, name, factory, body)
|
|
23
|
+
ast(
|
|
24
|
+
:begin,
|
|
25
|
+
node.updated(nil, [scope, name, constant(ast(:begin, scope || context), ast(:sym, name), factory)]),
|
|
26
|
+
ast(:block, ast(:send, ast(:const, scope, name), :class_eval), ast(:args), body)
|
|
27
|
+
)
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def constant(base, sym, factory)
|
|
31
|
+
ast(:if, ast(:send, base, :const_defined?, sym, ast(:false)), ast(:send, base, :const_get, sym), factory)
|
|
32
|
+
end
|
|
33
|
+
|
|
34
|
+
def context
|
|
35
|
+
identity = ast(:self)
|
|
36
|
+
root = ast(:cbase)
|
|
37
|
+
ast(:if, ast(:send, identity, :is_a?, ast(:const, root, :Module)), identity, ast(:const, root, :Object))
|
|
38
|
+
end
|
|
39
|
+
|
|
40
|
+
def factory?(call)
|
|
41
|
+
return false unless type?(call, :send)
|
|
42
|
+
receiver, selector = call.children
|
|
43
|
+
type?(receiver, :const) && VALUE_OBJECT_FACTORIES[receiver.children[1]] == selector
|
|
44
|
+
end
|
|
45
|
+
end
|
|
46
|
+
end
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "../operators/base"
|
|
4
|
+
|
|
5
|
+
require_relative "../support/unparse"
|
|
6
|
+
|
|
7
|
+
require_relative "../error"
|
|
8
|
+
require_relative "../registry/registry"
|
|
9
|
+
require_relative "guard_weaver"
|
|
10
|
+
require_relative "source_map"
|
|
11
|
+
|
|
12
|
+
class Kimera::Overlay
|
|
13
|
+
Result =
|
|
14
|
+
Struct.new(:file, :source, :mutant_ids, :skipped_unsafe, keyword_init: true) do
|
|
15
|
+
def install(path)
|
|
16
|
+
Kimera::Overlay.evaluate(source, path)
|
|
17
|
+
mutant_ids
|
|
18
|
+
end
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def initialize(registry)
|
|
22
|
+
@registry = registry
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
class << self
|
|
26
|
+
def evaluate(source, path)
|
|
27
|
+
Kimera::Warnings.silence { TOPLEVEL_BINDING.eval(source, path) }
|
|
28
|
+
end
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
def synthesize(file, source)
|
|
32
|
+
safe, unsafe = @registry.at(file).partition(&:safe?)
|
|
33
|
+
return unsafe(file, source, unsafe) if safe.empty?
|
|
34
|
+
Kimera::Overlay::FileWeave.new(file, Kimera::SourceMap.new(source), safe, unsafe).result
|
|
35
|
+
end
|
|
36
|
+
|
|
37
|
+
def bake(_file, source, id)
|
|
38
|
+
pair = @registry.point(id)
|
|
39
|
+
return source unless pair
|
|
40
|
+
mutant, point = pair
|
|
41
|
+
map = Kimera::SourceMap.new(source)
|
|
42
|
+
map.restore(Kimera::Unparse.unparse(Kimera::Guardrail.new(map, []).bake(point.location, mutant.directive)))
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
private
|
|
46
|
+
|
|
47
|
+
def unsafe(file, source, points)
|
|
48
|
+
Result.new(file: file, source: source, mutant_ids: [], skipped_unsafe: points.flat_map(&:ids))
|
|
49
|
+
end
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
require_relative "file_weave"
|
|
@@ -0,0 +1,101 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Kimera
|
|
4
|
+
module OverlaySplice
|
|
5
|
+
private
|
|
6
|
+
|
|
7
|
+
def splice
|
|
8
|
+
prepared = prepare
|
|
9
|
+
prepared && report(*prepared)
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
def prepare
|
|
13
|
+
return unless ready?
|
|
14
|
+
edits, dropped = gather
|
|
15
|
+
return if edits.empty?
|
|
16
|
+
source = apply(edits)
|
|
17
|
+
[edits, dropped, source] if Kimera::Syntax.parse(source).success?
|
|
18
|
+
end
|
|
19
|
+
|
|
20
|
+
def gather
|
|
21
|
+
edits = []
|
|
22
|
+
dropped = []
|
|
23
|
+
each { |definition, points| collect(definition, points, edits, dropped) }
|
|
24
|
+
[edits, dropped]
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
def collect(definition, points, edits, dropped)
|
|
28
|
+
rewritten = rewrite(definition, points)
|
|
29
|
+
rewritten ? edits << rewritten : dropped.concat(points)
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
def report(edits, dropped, source)
|
|
33
|
+
applied = edits.flat_map(&:last)
|
|
34
|
+
announce(applied, dropped)
|
|
35
|
+
build(applied, dropped, source)
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def build(applied, dropped, source)
|
|
39
|
+
Kimera::Overlay::Result.new(
|
|
40
|
+
file: @file, source: source, mutant_ids: applied,
|
|
41
|
+
skipped_unsafe: (@unsafe + dropped).flat_map(&:ids)
|
|
42
|
+
)
|
|
43
|
+
end
|
|
44
|
+
|
|
45
|
+
def announce(applied, dropped)
|
|
46
|
+
io.puts(
|
|
47
|
+
"kimera: #{@file}: file-level round-trip failed; " \
|
|
48
|
+
"#{applied.size} mutant(s) spliced per method, " \
|
|
49
|
+
"#{dropped.flat_map(&:ids).size} reported no_coverage"
|
|
50
|
+
)
|
|
51
|
+
end
|
|
52
|
+
|
|
53
|
+
def ready?
|
|
54
|
+
Kimera::Guardrail.new(@map, []).tree
|
|
55
|
+
true
|
|
56
|
+
rescue StandardError
|
|
57
|
+
false
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def each(&)
|
|
61
|
+
claimed = []
|
|
62
|
+
definitions.each { |definition| visit(definition, claimed, &) }
|
|
63
|
+
end
|
|
64
|
+
|
|
65
|
+
def visit(definition, claimed)
|
|
66
|
+
mine = points(definition, claimed)
|
|
67
|
+
return if mine.empty?
|
|
68
|
+
claimed.concat(mine)
|
|
69
|
+
yield(definition, mine)
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
def points(definition, claimed)
|
|
73
|
+
from, to = @map.span(definition.location.expression)
|
|
74
|
+
@safe.select { |point| !claimed.include?(point) && point.location.within?(from, to) }
|
|
75
|
+
end
|
|
76
|
+
|
|
77
|
+
def definitions
|
|
78
|
+
nodes = []
|
|
79
|
+
Kimera::Rewrite::AstWalk.visit(@map.ast) { |node| nodes << node if %i[def defs].include?(node.type) }
|
|
80
|
+
nodes
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
def rewrite(definition, points)
|
|
84
|
+
text, applied = Kimera::Guardrail.new(@map, points).rewrite(definition)
|
|
85
|
+
span(definition) + [text, applied]
|
|
86
|
+
rescue StandardError
|
|
87
|
+
nil
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
def span(definition)
|
|
91
|
+
expression = definition.location.expression
|
|
92
|
+
[expression.begin_pos, expression.end_pos]
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
def apply(edits)
|
|
96
|
+
edits.sort_by(&:first).reverse_each.with_object(@map.source.dup) do |(from, to, replacement, _ids), text|
|
|
97
|
+
text[from...to] = replacement
|
|
98
|
+
end
|
|
99
|
+
end
|
|
100
|
+
end
|
|
101
|
+
end
|
|
@@ -0,0 +1,58 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "fileutils"
|
|
4
|
+
require_relative "overlay"
|
|
5
|
+
|
|
6
|
+
module Kimera
|
|
7
|
+
module Synthesis
|
|
8
|
+
end
|
|
9
|
+
end
|
|
10
|
+
|
|
11
|
+
class Kimera::Synthesis::Project
|
|
12
|
+
Manifest =
|
|
13
|
+
Struct.new(:files, :mutant_files, :unsafe_mutants, keyword_init: true) do
|
|
14
|
+
def record(file, dest, result)
|
|
15
|
+
files[file] = dest
|
|
16
|
+
result.mutant_ids.each { |id| mutant_files[id] = file }
|
|
17
|
+
unsafe_mutants.concat(result.skipped_unsafe)
|
|
18
|
+
end
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def initialize(registry, root: ".")
|
|
22
|
+
@registry = registry
|
|
23
|
+
@root = root
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
def write(outdir)
|
|
27
|
+
outdir = File.expand_path(outdir)
|
|
28
|
+
@registry.files.each { |file| copy(file, outdir) }
|
|
29
|
+
manifest
|
|
30
|
+
end
|
|
31
|
+
|
|
32
|
+
private
|
|
33
|
+
|
|
34
|
+
def manifest
|
|
35
|
+
@_manifest ||= Manifest.new(files: {}, mutant_files: {}, unsafe_mutants: [])
|
|
36
|
+
end
|
|
37
|
+
|
|
38
|
+
def copy(file, outdir)
|
|
39
|
+
source = File.join(root, file)
|
|
40
|
+
return unless File.file?(source)
|
|
41
|
+
emit(file, outdir, synth.synthesize(file, File.read(source, encoding: Encoding::UTF_8)))
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
def root
|
|
45
|
+
@_root ||= File.expand_path(@root)
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def synth
|
|
49
|
+
@_synth ||= Kimera::Overlay.new(@registry)
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def emit(file, outdir, result)
|
|
53
|
+
dest = File.join(outdir, file)
|
|
54
|
+
FileUtils.mkdir_p(File.dirname(dest))
|
|
55
|
+
File.write(dest, result.source)
|
|
56
|
+
manifest.record(file, dest, result)
|
|
57
|
+
end
|
|
58
|
+
end
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "../support/unparse"
|
|
4
|
+
require_relative "../rewrite/numbered_params"
|
|
5
|
+
|
|
6
|
+
class Kimera::SourceMap
|
|
7
|
+
attr_reader :source
|
|
8
|
+
|
|
9
|
+
def initialize(source)
|
|
10
|
+
@source = source
|
|
11
|
+
end
|
|
12
|
+
|
|
13
|
+
def ast
|
|
14
|
+
@_ast ||= Kimera::Rewrite::NumberedParams.normalize(Kimera::Unparse.parse(@source))
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def span(expr)
|
|
18
|
+
[bytes[expr.begin_pos], bytes[expr.end_pos]]
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def restore(source)
|
|
22
|
+
prefix = comments.join("\n")
|
|
23
|
+
prefix.empty? ? source : "#{prefix}\n#{source}"
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
MAGIC_COMMENT =
|
|
27
|
+
/\A#\s*(?:-\*-.*-\*-|(?:frozen_string_literal|encoding|coding|warn_indent|shareable_constant_value)\s*:)/i
|
|
28
|
+
private_constant :MAGIC_COMMENT
|
|
29
|
+
|
|
30
|
+
private
|
|
31
|
+
|
|
32
|
+
def comments
|
|
33
|
+
first, second = @source.lines
|
|
34
|
+
shebang = first&.start_with?("#!") ? first : nil
|
|
35
|
+
candidate = shebang ? second : first
|
|
36
|
+
[shebang, (candidate&.match?(MAGIC_COMMENT) ? candidate : nil)].compact.map(&:chomp)
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
def bytes
|
|
40
|
+
@_bytes ||= @source.each_char.map(&:bytesize).each_with_object([0]) { |size, map| map << (map.last + size) }
|
|
41
|
+
end
|
|
42
|
+
end
|
data/lib/kimera.rb
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require_relative "kimera/error"
|
|
4
|
+
require_relative "kimera/runtime"
|
|
5
|
+
require_relative "kimera/version"
|
|
6
|
+
|
|
7
|
+
module Kimera
|
|
8
|
+
autoload :MutationPoint, "kimera/registry/mutation_point"
|
|
9
|
+
autoload :Registry, "kimera/registry/registry"
|
|
10
|
+
autoload :RegistryScan, "kimera/registry/builder"
|
|
11
|
+
autoload :Overlay, "kimera/synthesis/overlay"
|
|
12
|
+
autoload :CLI, "kimera/cli"
|
|
13
|
+
end
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: kimera
|
|
3
|
+
description: >-
|
|
4
|
+
Running, triaging, and gating mutation tests with kimera in a Ruby or Rails
|
|
5
|
+
project. Use when running kimera, triaging surviving mutants, deciding
|
|
6
|
+
whether to add a test, adding or reviewing ignore or baseline entries,
|
|
7
|
+
raising max_ignored, wiring kimera into CI, or reading a kimera report.
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Strengthening a suite with kimera
|
|
11
|
+
|
|
12
|
+
Mutation testing audits *tests*, not code. The unit of work is one surviving
|
|
13
|
+
mutant, and each gets exactly one of four verdicts (see Triage). Never act to
|
|
14
|
+
move the score.
|
|
15
|
+
|
|
16
|
+
## Set up
|
|
17
|
+
|
|
18
|
+
```sh
|
|
19
|
+
bundle exec kimera init # detects the project, writes .kimera.yml
|
|
20
|
+
bundle exec kimera doctor --check-baseline # discovery, git, and a green suite
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
- Gates: keep `max_survivors: 0`; every survivor is a decision, not a
|
|
24
|
+
statistic. `max_ignored` rises only in the same diff as the entry it admits.
|
|
25
|
+
- Adopting on a suite with existing survivors: `kimera baseline create
|
|
26
|
+
REPORT.json --reason TEXT` records them as reviewed debt, so the gate blocks
|
|
27
|
+
only new holes. Burn the baseline down; never grow it to pass a gate.
|
|
28
|
+
- Fix line coverage first. `no_coverage` mutants are plain coverage gaps, and
|
|
29
|
+
mutation results only mean something for code the tests execute.
|
|
30
|
+
- CI shape: **PR gate is incremental** (`kimera ci --since origin/main
|
|
31
|
+
--session tmp/kimera.json`; `ci` defaults to `--max-survivors 0
|
|
32
|
+
--fail-on-no-coverage`): no new surviving mutants on changed lines.
|
|
33
|
+
**Full run nightly**, not per-PR.
|
|
34
|
+
- Exit codes: 0 pass; 1 invalid invocation or unmutated suite not green (fix
|
|
35
|
+
the suite, not kimera); 2 gate failure (survivors, uncovered, ignore budget,
|
|
36
|
+
or unjudged).
|
|
37
|
+
- The green check covers only tests that *cover* an in-scope mutant. A red
|
|
38
|
+
test touching none of them is reported and excluded, so an unrelated flaky
|
|
39
|
+
spec doesn't abort a per-module run.
|
|
40
|
+
|
|
41
|
+
## Operate
|
|
42
|
+
|
|
43
|
+
```sh
|
|
44
|
+
bundle exec kimera changed # changed lines only (vs origin/main)
|
|
45
|
+
bundle exec kimera run # full, per .kimera.yml
|
|
46
|
+
bundle exec kimera report REPORT.json --status survived
|
|
47
|
+
bundle exec kimera mutant ID --report REPORT.json
|
|
48
|
+
bundle exec kimera run --isolated --jobs 4 # oracle mode (see Strengthen)
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
- The progress bar renders on a tty. Redirected/CI runs stay silent until the
|
|
52
|
+
report, except isolated mode, which traces one verdict per line to stderr.
|
|
53
|
+
- `--session FILE` persists per-mutant verdicts and resumes interrupted runs.
|
|
54
|
+
- `--report FILE` writes the machine-readable report. Each result carries
|
|
55
|
+
`mutant_id`, `status`, `file`, `line`, `operator`, and the `original` ->
|
|
56
|
+
`mutated` source; don't re-parse the human log.
|
|
57
|
+
- Minitest/Rails: kimera puts `test/` (or `spec/`) on `$LOAD_PATH`, so test
|
|
58
|
+
files can `require "test_helper"` without `RUBYOPT="-Itest"`.
|
|
59
|
+
- `--tests` on the CLI *replaces* the config `tests:` glob (it does not
|
|
60
|
+
append).
|
|
61
|
+
- `--jobs` sizes the warm pool and isolated mirrors. Coverage-based test
|
|
62
|
+
selection and kill-on-first-failure are automatic.
|
|
63
|
+
- A `timeout` verdict is a *detected* mutant (the suite hung on it), not an
|
|
64
|
+
error.
|
|
65
|
+
- On a Rails app that uses `parallelize`, `--jobs > 1` gives each worker its
|
|
66
|
+
own database, so there's no shared-DB fixture/RLS deadlock.
|
|
67
|
+
- Operators: the default is the conservative core. `--operators all` enables
|
|
68
|
+
the extended families; `--operators rails` (or `comparison,rails`) the
|
|
69
|
+
Rails-aware ones.
|
|
70
|
+
- `isolated_only` marks class-body DSL mutants that only `--isolated` can
|
|
71
|
+
judge. Run it for their verdicts; they never gate or count in the score.
|
|
72
|
+
- Rails `enum` models overlay warm (the re-declaration is idempotent), so
|
|
73
|
+
their method-body mutants are judged like any other. A file labeled
|
|
74
|
+
`unmutatable` could not be overlaid (a distinct, reported reason); it is not
|
|
75
|
+
an `--isolated` case.
|
|
76
|
+
|
|
77
|
+
## Triage a surviving mutant: the only four verdicts
|
|
78
|
+
|
|
79
|
+
1. **Real gap**: write the test. Assert the exact distinction the mutant
|
|
80
|
+
erased (the `<` vs `<=` boundary, the deleted call's observable effect).
|
|
81
|
+
2. **Equivalent**: ignore entry, only with *both* (a) survival in an
|
|
82
|
+
`--isolated` oracle run and (b) a `reason:` naming the **mechanism**
|
|
83
|
+
("IO.pipe write ends are sync; the flush is redundant"), not restating the
|
|
84
|
+
verdict ("this is equivalent"). Entries without `reason:` are rejected.
|
|
85
|
+
3. **Dead code**: delete the code. A survivor on a branch nothing observes
|
|
86
|
+
is YAGNI evidence.
|
|
87
|
+
4. **Wrong level**: the behavior is real but invisible to unit assertions
|
|
88
|
+
(logging, fd hygiene, progress output). Cover it with an integration test
|
|
89
|
+
or accept it visibly. Never stub internals just to kill a mutant; a test
|
|
90
|
+
that pins the implementation makes the suite worse.
|
|
91
|
+
|
|
92
|
+
## Ignore discipline: hard rules (especially for agents)
|
|
93
|
+
|
|
94
|
+
- **Never raise `max_ignored` to make a gate pass.** Raising the budget is a
|
|
95
|
+
human decision. Surface the failing gate and the candidate entry instead.
|
|
96
|
+
- An ignore entry without an isolated-oracle survival check is inadmissible.
|
|
97
|
+
Warm-path survivors can be measurement artifacts (kimera's own suite once
|
|
98
|
+
had a "clearly equivalent" mutant that 11 specs actually kill).
|
|
99
|
+
- A budget raise goes in the same diff as the entry it admits. Reviewers
|
|
100
|
+
judge the `reason:`, not the number.
|
|
101
|
+
|
|
102
|
+
## Strengthen
|
|
103
|
+
|
|
104
|
+
- Prioritize by blast radius: boundaries, money, authz, parsers, state
|
|
105
|
+
machines first; CLI wiring and glue last.
|
|
106
|
+
- **Verify a kill by hand when in doubt**: apply the mutation to the source
|
|
107
|
+
manually, run the covering tests, confirm red, restore. This catches both
|
|
108
|
+
false survivors and false kills.
|
|
109
|
+
- Suspicious verdicts on self-referential or harness-critical code (the test
|
|
110
|
+
runner, global state the suite also touches, kimera's own plumbing):
|
|
111
|
+
adjudicate with `--isolated`. Suites that manipulate shared globals can
|
|
112
|
+
suppress the warm path.
|
|
113
|
+
- A cluster of `error`/`timeout` verdicts in one region usually means harness
|
|
114
|
+
fragility or a missing guard, not test strength. Read the cluster before
|
|
115
|
+
counting the detections.
|
|
116
|
+
- Anti-patterns: chasing the score; adding tests without reading the mutant;
|
|
117
|
+
white-box tests that mirror the implementation; reclassifying killable
|
|
118
|
+
mutants as equivalent to end a triage session.
|