moots 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/CHANGELOG.md +34 -0
- data/CODE_OF_CONDUCT.md +132 -0
- data/LICENSE.txt +21 -0
- data/README.md +312 -0
- data/docs/isolation.md +89 -0
- data/docs/reporting.md +70 -0
- data/exe/moots +6 -0
- data/lib/moots/cli.rb +75 -0
- data/lib/moots/configuration.rb +86 -0
- data/lib/moots/environment.rb +23 -0
- data/lib/moots/error.rb +6 -0
- data/lib/moots/mutation.rb +42 -0
- data/lib/moots/mutation_generator.rb +60 -0
- data/lib/moots/reporter.rb +104 -0
- data/lib/moots/test_result.rb +17 -0
- data/lib/moots/test_runner.rb +60 -0
- data/lib/moots/version.rb +5 -0
- data/lib/moots/worker_pool.rb +76 -0
- data/lib/moots/workspace.rb +137 -0
- data/lib/moots.rb +111 -0
- data/sig/moots.rbs +4 -0
- metadata +100 -0
checksums.yaml
ADDED
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
---
|
|
2
|
+
SHA256:
|
|
3
|
+
metadata.gz: 49077c78f2418214b2e331c9540b4e2f69e365d4bd4107fcd48373eac16395f0
|
|
4
|
+
data.tar.gz: f0be889f311faa2edbb112eb229070adde1667710329901b0cb4f2c9878c9b71
|
|
5
|
+
SHA512:
|
|
6
|
+
metadata.gz: ca28e8c3afc830bb851ef12fe75f8f84cb55ea8e85a46fed48b21576d279dbc97be00a618c4849229e74912e938ba29d430330893845bfa71d9a85032107fd19
|
|
7
|
+
data.tar.gz: dfa2cd560720aa7432b772a8c121b0314a2b1e2191b11db7dc4c7f77bc3d283d4c96d785a8125b4afe8602638bb1e16ebcb875d780534dd958fc0997be855807
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,34 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.1.0 - 2026-10-29
|
|
4
|
+
|
|
5
|
+
- Add optional parallel execution with `--jobs N` or YAML/API `jobs`, keeping one
|
|
6
|
+
worker as the default. Each worker receives a copy of the verified baseline;
|
|
7
|
+
errors and interruption cancel active commands and clean up all copies.
|
|
8
|
+
- Preserve source order in parallel API/JSON results, including partial errors,
|
|
9
|
+
while serializing live progress callbacks on the calling thread. Check the
|
|
10
|
+
independent RSpec/Minitest corpus with both one and two workers.
|
|
11
|
+
- Run baselines and mutations in isolated copies by default, preserving uncommitted
|
|
12
|
+
project files and original report paths/IDs. Add configurable copy exclusions,
|
|
13
|
+
Bundler path rebasing and an explicit `isolation: false` compatibility mode.
|
|
14
|
+
- Verify cleanup and original-source protection, including forced termination;
|
|
15
|
+
reject linked paths, special files and source globs that escape the snapshot.
|
|
16
|
+
- Stream baseline and mutation progress, with IDs, survivor expression diffs and
|
|
17
|
+
rerun arguments; select one mutation using `--mutation ID` or YAML.
|
|
18
|
+
- Add versioned JSON reports (`--format json`) with separate progress output,
|
|
19
|
+
captured test output and explicit partial-error reports. Interruptions exit 130.
|
|
20
|
+
- Add optional Ruby API progress callbacks and declare the JSON runtime dependency.
|
|
21
|
+
- Mutate equality (`==`, `!=`) and strict/inclusive boundaries (`>`, `>=`, `<`, `<=`).
|
|
22
|
+
- Add an independent comparison acceptance corpus for RSpec and Minitest, checking
|
|
23
|
+
exact kills and survivors against strong and deliberately weak test suites.
|
|
24
|
+
- Support maintained CRuby 3.3, 3.4 and 4.0; raise the minimum from 3.1 to 3.3.
|
|
25
|
+
- Replace the runtime parser dependency with Prism and remove OpenStruct.
|
|
26
|
+
- Refresh dependencies and add cross-platform CI, coverage and self-mutation gates.
|
|
27
|
+
- Add the documented `moots` executable and validated, safe YAML configuration.
|
|
28
|
+
- Replace exactly one operator by byte location, preserving repeated expressions,
|
|
29
|
+
multiline source, encodings, comments and line endings.
|
|
30
|
+
- Support keyword boolean operators and explicit arithmetic operator calls.
|
|
31
|
+
- Require a passing baseline and restore original source even after execution errors.
|
|
32
|
+
- Add command deadlines, output capture and process cleanup; distinguish operational
|
|
33
|
+
errors from killed mutations and fail empty mutation runs.
|
|
34
|
+
- Expand unit and subprocess integration tests with full runtime line/branch coverage.
|
data/CODE_OF_CONDUCT.md
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
# Contributor Covenant Code of Conduct
|
|
2
|
+
|
|
3
|
+
## Our Pledge
|
|
4
|
+
|
|
5
|
+
We as members, contributors, and leaders pledge to make participation in our
|
|
6
|
+
community a harassment-free experience for everyone, regardless of age, body
|
|
7
|
+
size, visible or invisible disability, ethnicity, sex characteristics, gender
|
|
8
|
+
identity and expression, level of experience, education, socio-economic status,
|
|
9
|
+
nationality, personal appearance, race, caste, color, religion, or sexual
|
|
10
|
+
identity and orientation.
|
|
11
|
+
|
|
12
|
+
We pledge to act and interact in ways that contribute to an open, welcoming,
|
|
13
|
+
diverse, inclusive, and healthy community.
|
|
14
|
+
|
|
15
|
+
## Our Standards
|
|
16
|
+
|
|
17
|
+
Examples of behavior that contributes to a positive environment for our
|
|
18
|
+
community include:
|
|
19
|
+
|
|
20
|
+
* Demonstrating empathy and kindness toward other people
|
|
21
|
+
* Being respectful of differing opinions, viewpoints, and experiences
|
|
22
|
+
* Giving and gracefully accepting constructive feedback
|
|
23
|
+
* Accepting responsibility and apologizing to those affected by our mistakes,
|
|
24
|
+
and learning from the experience
|
|
25
|
+
* Focusing on what is best not just for us as individuals, but for the overall
|
|
26
|
+
community
|
|
27
|
+
|
|
28
|
+
Examples of unacceptable behavior include:
|
|
29
|
+
|
|
30
|
+
* The use of sexualized language or imagery, and sexual attention or advances of
|
|
31
|
+
any kind
|
|
32
|
+
* Trolling, insulting or derogatory comments, and personal or political attacks
|
|
33
|
+
* Public or private harassment
|
|
34
|
+
* Publishing others' private information, such as a physical or email address,
|
|
35
|
+
without their explicit permission
|
|
36
|
+
* Other conduct which could reasonably be considered inappropriate in a
|
|
37
|
+
professional setting
|
|
38
|
+
|
|
39
|
+
## Enforcement Responsibilities
|
|
40
|
+
|
|
41
|
+
Community leaders are responsible for clarifying and enforcing our standards of
|
|
42
|
+
acceptable behavior and will take appropriate and fair corrective action in
|
|
43
|
+
response to any behavior that they deem inappropriate, threatening, offensive,
|
|
44
|
+
or harmful.
|
|
45
|
+
|
|
46
|
+
Community leaders have the right and responsibility to remove, edit, or reject
|
|
47
|
+
comments, commits, code, wiki edits, issues, and other contributions that are
|
|
48
|
+
not aligned to this Code of Conduct, and will communicate reasons for moderation
|
|
49
|
+
decisions when appropriate.
|
|
50
|
+
|
|
51
|
+
## Scope
|
|
52
|
+
|
|
53
|
+
This Code of Conduct applies within all community spaces, and also applies when
|
|
54
|
+
an individual is officially representing the community in public spaces.
|
|
55
|
+
Examples of representing our community include using an official email address,
|
|
56
|
+
posting via an official social media account, or acting as an appointed
|
|
57
|
+
representative at an online or offline event.
|
|
58
|
+
|
|
59
|
+
## Enforcement
|
|
60
|
+
|
|
61
|
+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
|
62
|
+
reported to the community leaders responsible for enforcement at
|
|
63
|
+
[INSERT CONTACT METHOD].
|
|
64
|
+
All complaints will be reviewed and investigated promptly and fairly.
|
|
65
|
+
|
|
66
|
+
All community leaders are obligated to respect the privacy and security of the
|
|
67
|
+
reporter of any incident.
|
|
68
|
+
|
|
69
|
+
## Enforcement Guidelines
|
|
70
|
+
|
|
71
|
+
Community leaders will follow these Community Impact Guidelines in determining
|
|
72
|
+
the consequences for any action they deem in violation of this Code of Conduct:
|
|
73
|
+
|
|
74
|
+
### 1. Correction
|
|
75
|
+
|
|
76
|
+
**Community Impact**: Use of inappropriate language or other behavior deemed
|
|
77
|
+
unprofessional or unwelcome in the community.
|
|
78
|
+
|
|
79
|
+
**Consequence**: A private, written warning from community leaders, providing
|
|
80
|
+
clarity around the nature of the violation and an explanation of why the
|
|
81
|
+
behavior was inappropriate. A public apology may be requested.
|
|
82
|
+
|
|
83
|
+
### 2. Warning
|
|
84
|
+
|
|
85
|
+
**Community Impact**: A violation through a single incident or series of
|
|
86
|
+
actions.
|
|
87
|
+
|
|
88
|
+
**Consequence**: A warning with consequences for continued behavior. No
|
|
89
|
+
interaction with the people involved, including unsolicited interaction with
|
|
90
|
+
those enforcing the Code of Conduct, for a specified period of time. This
|
|
91
|
+
includes avoiding interactions in community spaces as well as external channels
|
|
92
|
+
like social media. Violating these terms may lead to a temporary or permanent
|
|
93
|
+
ban.
|
|
94
|
+
|
|
95
|
+
### 3. Temporary Ban
|
|
96
|
+
|
|
97
|
+
**Community Impact**: A serious violation of community standards, including
|
|
98
|
+
sustained inappropriate behavior.
|
|
99
|
+
|
|
100
|
+
**Consequence**: A temporary ban from any sort of interaction or public
|
|
101
|
+
communication with the community for a specified period of time. No public or
|
|
102
|
+
private interaction with the people involved, including unsolicited interaction
|
|
103
|
+
with those enforcing the Code of Conduct, is allowed during this period.
|
|
104
|
+
Violating these terms may lead to a permanent ban.
|
|
105
|
+
|
|
106
|
+
### 4. Permanent Ban
|
|
107
|
+
|
|
108
|
+
**Community Impact**: Demonstrating a pattern of violation of community
|
|
109
|
+
standards, including sustained inappropriate behavior, harassment of an
|
|
110
|
+
individual, or aggression toward or disparagement of classes of individuals.
|
|
111
|
+
|
|
112
|
+
**Consequence**: A permanent ban from any sort of public interaction within the
|
|
113
|
+
community.
|
|
114
|
+
|
|
115
|
+
## Attribution
|
|
116
|
+
|
|
117
|
+
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
|
|
118
|
+
version 2.1, available at
|
|
119
|
+
[https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
|
|
120
|
+
|
|
121
|
+
Community Impact Guidelines were inspired by
|
|
122
|
+
[Mozilla's code of conduct enforcement ladder][Mozilla CoC].
|
|
123
|
+
|
|
124
|
+
For answers to common questions about this code of conduct, see the FAQ at
|
|
125
|
+
[https://www.contributor-covenant.org/faq][FAQ]. Translations are available at
|
|
126
|
+
[https://www.contributor-covenant.org/translations][translations].
|
|
127
|
+
|
|
128
|
+
[homepage]: https://www.contributor-covenant.org
|
|
129
|
+
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
|
|
130
|
+
[Mozilla CoC]: https://github.com/mozilla/diversity
|
|
131
|
+
[FAQ]: https://www.contributor-covenant.org/faq
|
|
132
|
+
[translations]: https://www.contributor-covenant.org/translations
|
data/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
The MIT License (MIT)
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2025 Dave Russell
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in
|
|
13
|
+
all copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
|
21
|
+
THE SOFTWARE.
|
data/README.md
ADDED
|
@@ -0,0 +1,312 @@
|
|
|
1
|
+
# Moots
|
|
2
|
+
|
|
3
|
+
Moots is a small, MIT-licensed mutation testing tool for Ruby. It changes one
|
|
4
|
+
operator in your source, runs your tests in a fresh process, and reports whether
|
|
5
|
+
the tests caught the change.
|
|
6
|
+
|
|
7
|
+
A **killed** mutation made the tests fail. A **surviving** mutation left them
|
|
8
|
+
passing and is worth investigating: it may reveal a missing assertion, untested
|
|
9
|
+
behavior, or an equivalent change.
|
|
10
|
+
|
|
11
|
+
Moots is still an early project with a deliberately small set of mutation
|
|
12
|
+
operators. A perfect score measures those operators; it does not prove that a
|
|
13
|
+
program is correct.
|
|
14
|
+
|
|
15
|
+
## Ruby support
|
|
16
|
+
|
|
17
|
+
Supports CRuby **3.3, 3.4 and 4.0**, with a minimum of Ruby 3.3. These are the
|
|
18
|
+
[maintained Ruby series](https://www.ruby-lang.org/en/downloads/branches/).
|
|
19
|
+
CI runs all three on Linux and Ruby 4.0 on Windows and macOS.
|
|
20
|
+
|
|
21
|
+
Parsing uses [Prism](https://github.com/ruby/prism), including modern Ruby syntax.
|
|
22
|
+
The old `parser` gem only handles syntax through Ruby 3.3. Runtime dependencies
|
|
23
|
+
are Prism and JSON; development tools are kept in the Gemfile.
|
|
24
|
+
|
|
25
|
+
## Installation
|
|
26
|
+
|
|
27
|
+
Add Moots to your test/development bundle:
|
|
28
|
+
|
|
29
|
+
```ruby
|
|
30
|
+
group :development, :test do
|
|
31
|
+
gem "moots"
|
|
32
|
+
end
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
To use a development checkout instead, add
|
|
36
|
+
`gem "moots", path: "/path/to/moots"` to the project Gemfile, or run
|
|
37
|
+
`bundle exec rake install` in this repository. Then run `bundle install`.
|
|
38
|
+
|
|
39
|
+
Windows installations need RubyInstaller's MSYS2 development toolchain to build
|
|
40
|
+
Prism and development dependencies (`ridk install`). This repository can also be
|
|
41
|
+
developed under WSL with a supported Linux Ruby.
|
|
42
|
+
|
|
43
|
+
## Usage
|
|
44
|
+
|
|
45
|
+
From the project root:
|
|
46
|
+
|
|
47
|
+
```sh
|
|
48
|
+
bundle exec moots
|
|
49
|
+
bundle exec moots --config config/moots.yml
|
|
50
|
+
bundle exec moots --format json > moots-report.json
|
|
51
|
+
bundle exec moots --mutation MUTATION_ID
|
|
52
|
+
bundle exec moots --jobs 2
|
|
53
|
+
bundle exec moots --help
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Moots reads `.moots.yml` when present. Without it, the defaults below apply:
|
|
57
|
+
|
|
58
|
+
```yaml
|
|
59
|
+
include:
|
|
60
|
+
- app/**/*.rb
|
|
61
|
+
- lib/**/*.rb
|
|
62
|
+
exclude: []
|
|
63
|
+
test_command: bundle exec rspec
|
|
64
|
+
timeout: 60
|
|
65
|
+
mutation: null
|
|
66
|
+
isolation: true
|
|
67
|
+
jobs: 1
|
|
68
|
+
workspace_exclude: [tmp, coverage, pkg, .rspec_status]
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Paths are relative to the current working directory, including when using an
|
|
72
|
+
explicit configuration path. Overlapping include patterns are deduplicated;
|
|
73
|
+
exclude patterns take precedence. Unknown options, malformed YAML and invalid
|
|
74
|
+
option values are errors.
|
|
75
|
+
|
|
76
|
+
`test_command` can also be an argument array, which passes arguments literally
|
|
77
|
+
without shell interpretation:
|
|
78
|
+
|
|
79
|
+
```yaml
|
|
80
|
+
test_command: [bundle, exec, rspec, spec/models]
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
The command must run the relevant tests in a fresh process and reload changed
|
|
84
|
+
source. Avoid persistent test preloaders such as Spring. A command that runs no
|
|
85
|
+
tests can still exit successfully; make sure your test runner rejects empty
|
|
86
|
+
suites when appropriate.
|
|
87
|
+
|
|
88
|
+
Exit statuses:
|
|
89
|
+
|
|
90
|
+
| Status | Meaning |
|
|
91
|
+
| --- | --- |
|
|
92
|
+
| 0 | At least one mutation, all killed (or help/version requested) |
|
|
93
|
+
| 1 | Survivors, or no mutations generated |
|
|
94
|
+
| 2 | Configuration, source, baseline or command execution error |
|
|
95
|
+
| 130 | Interrupted run |
|
|
96
|
+
|
|
97
|
+
## Progress, survivors and CI reports
|
|
98
|
+
|
|
99
|
+
Text output starts before the baseline tests run, then shows `[current/total]`
|
|
100
|
+
progress before each mutation's tests. Each result has an ID and a relative
|
|
101
|
+
`file:line` location. Survivors include an expression diff (`-` original, `+`
|
|
102
|
+
replacement) and a `Rerun argv` argument array. The diff covers the mutated
|
|
103
|
+
expression; it is not a patch for the whole file.
|
|
104
|
+
|
|
105
|
+
Use `--mutation ID` to run a reported mutation with a fresh baseline. It overrides
|
|
106
|
+
the optional `mutation` YAML setting. IDs use the relative file path, byte offset
|
|
107
|
+
and original/replacement bytes, so equivalent checkouts produce the same IDs.
|
|
108
|
+
Source edits can change IDs. An unknown ID is an error; it never silently runs
|
|
109
|
+
the full suite or reports success. Use the same working directory, configuration
|
|
110
|
+
and source revision when reproducing a result.
|
|
111
|
+
|
|
112
|
+
`--format json` writes one JSON document to stdout; progress and diagnostics go
|
|
113
|
+
to stderr. Test-process output is captured inside the document. The schema is
|
|
114
|
+
described in [docs/reporting.md](docs/reporting.md). It includes:
|
|
115
|
+
|
|
116
|
+
- `schema_version: 1` and `status: "completed"` or `"error"`.
|
|
117
|
+
- Summary counts and each completed mutation's ID, source location, expressions,
|
|
118
|
+
killed/survived status, captured test output and rerun arguments.
|
|
119
|
+
- An `error` message on operational failure or interruption. Any partial results
|
|
120
|
+
cover only completed mutations; an aborted run is never marked completed.
|
|
121
|
+
|
|
122
|
+
Completion does not mean all mutations were killed. Continue to check the exit
|
|
123
|
+
status (or the summary) in CI. Invalid UTF-8 in report text is replaced with `�`;
|
|
124
|
+
mutation identity and source restoration still use the original bytes.
|
|
125
|
+
|
|
126
|
+
## Mutation behavior
|
|
127
|
+
|
|
128
|
+
Supported replacements:
|
|
129
|
+
|
|
130
|
+
| Original | Replacement |
|
|
131
|
+
| --- | --- |
|
|
132
|
+
| `+` | `-` |
|
|
133
|
+
| `-` | `+` |
|
|
134
|
+
| `*` | `/` |
|
|
135
|
+
| `/` | `*` |
|
|
136
|
+
| `==` | `!=` |
|
|
137
|
+
| `!=` | `==` |
|
|
138
|
+
| `>` | `>=` |
|
|
139
|
+
| `>=` | `>` |
|
|
140
|
+
| `<` | `<=` |
|
|
141
|
+
| `<=` | `<` |
|
|
142
|
+
| `&&` | `||` |
|
|
143
|
+
| `||` | `&&` |
|
|
144
|
+
| `and` | `or` |
|
|
145
|
+
| `or` | `and` |
|
|
146
|
+
|
|
147
|
+
Each mutation changes exactly one operator token, including explicit operator
|
|
148
|
+
calls such as `a.+(b)`. Other operators on the same line, string contents,
|
|
149
|
+
comments and surrounding formatting are preserved. Byte offsets handle
|
|
150
|
+
multiline expressions, Unicode and CRLF files.
|
|
151
|
+
|
|
152
|
+
Boundary mutations switch between strict and inclusive comparisons, exposing
|
|
153
|
+
missing tests for the value exactly at a limit. Equality mutations invert the
|
|
154
|
+
comparison. Explicit calls such as `a.>=(b)` are supported too.
|
|
155
|
+
|
|
156
|
+
Compound assignments (`+=`, `&&=`, etc.), unary operators, other comparisons
|
|
157
|
+
(`===`, `<=>`, `=~`, `!~`), constants and method removal are not currently mutated.
|
|
158
|
+
Equivalent mutations are not automatically filtered.
|
|
159
|
+
|
|
160
|
+
## Execution and isolation
|
|
161
|
+
|
|
162
|
+
Moots copies the current project into a temporary directory and runs the baseline
|
|
163
|
+
and mutations there. Uncommitted and untracked files are included; no Git commit
|
|
164
|
+
is required. The original checkout is never used for mutation writes. Reported
|
|
165
|
+
paths and mutation IDs still refer to that checkout.
|
|
166
|
+
|
|
167
|
+
A failing baseline aborts before any mutations. Within the copy, files are
|
|
168
|
+
processed sequentially and restored after each mutation, including errors and
|
|
169
|
+
interruptions. Each run uses one copy, so test-generated artifacts persist
|
|
170
|
+
between invocations within that run. The copy is removed when the run finishes
|
|
171
|
+
or raises. Hard termination may leave an orphan copy, but mutation writes still
|
|
172
|
+
cannot affect the original files.
|
|
173
|
+
|
|
174
|
+
The `workspace_exclude` globs omit files or entire directories from the copy;
|
|
175
|
+
they are separate from `exclude`, which only controls mutation targets. The
|
|
176
|
+
default copy exclusions are `tmp`, `coverage`, `pkg` and `.rspec_status`. Supplying
|
|
177
|
+
your own list replaces those defaults; use `[]` if your tests need everything.
|
|
178
|
+
Git, Mercurial and Subversion metadata are always omitted, including nested
|
|
179
|
+
repositories and worktree pointer files. Installed dependencies such as
|
|
180
|
+
`vendor/bundle` are copied, so large projects may need more time and disk space.
|
|
181
|
+
|
|
182
|
+
Links (including directory junctions), special files and source globs that escape
|
|
183
|
+
the project are rejected in isolated mode. Exclude unnecessary linked paths with
|
|
184
|
+
`workspace_exclude`. The system temporary directory must be outside the project.
|
|
185
|
+
See [docs/isolation.md](docs/isolation.md) for path handling and compatibility.
|
|
186
|
+
|
|
187
|
+
Source read/write failures and invalid Ruby syntax are reported as errors;
|
|
188
|
+
they are not silently excluded from a score. Missing commands, shell execution
|
|
189
|
+
failures (exit 126/127), signal termination and timeouts are also errors rather
|
|
190
|
+
than successful kills. Other nonzero exits count as killed mutations.
|
|
191
|
+
|
|
192
|
+
The timeout applies to each test invocation, including the baseline. Standard
|
|
193
|
+
output and standard error are drained concurrently. On POSIX systems the runner
|
|
194
|
+
kills the process group on interruption or timeout; on Windows it uses
|
|
195
|
+
`taskkill /T /F` for the command's process tree.
|
|
196
|
+
|
|
197
|
+
Isolation protects the checkout from Moots' mutation writes; it is not a security
|
|
198
|
+
sandbox. Test commands can still access external files, services and databases.
|
|
199
|
+
Avoid editing the project while the snapshot is being copied. Windows descendants
|
|
200
|
+
whose parent has already exited may outlive tree cleanup.
|
|
201
|
+
|
|
202
|
+
For projects that require their original location or Git metadata, explicitly
|
|
203
|
+
set `isolation: false`. This restores the previous in-place behavior: use a
|
|
204
|
+
disposable checkout and avoid concurrent edits or mutation jobs. An `ensure`
|
|
205
|
+
block restores source bytes after normal failures, but cannot protect against
|
|
206
|
+
hard termination, power loss or a failed restoration write.
|
|
207
|
+
|
|
208
|
+
## Parallel execution
|
|
209
|
+
|
|
210
|
+
`--jobs N` (or `jobs: N` in YAML/the Ruby API) runs up to N test commands at once.
|
|
211
|
+
The default is `1`, preserving sequential execution. Parallel execution requires
|
|
212
|
+
`isolation: true`; in-place concurrent mutations are rejected.
|
|
213
|
+
|
|
214
|
+
Moots runs one baseline, then prepares an independent copy of that verified
|
|
215
|
+
snapshot for each worker. Workers include any artifacts produced by the baseline,
|
|
216
|
+
but never share mutated files. The worker count is capped at the number of
|
|
217
|
+
selected mutations. Empty runs and single-mutation reruns avoid creating a pool.
|
|
218
|
+
|
|
219
|
+
Start with `--jobs 2` and size the pool for your available CPU, memory and disk.
|
|
220
|
+
Each worker needs a full project copy and may itself run a parallel test suite.
|
|
221
|
+
Working copies do not isolate databases, network ports or external services;
|
|
222
|
+
keep `jobs: 1` for suites that cannot safely run concurrently.
|
|
223
|
+
|
|
224
|
+
Progress and completed-result messages can arrive out of order, with each
|
|
225
|
+
mutation retaining its original index and ID. The returned Ruby array and final
|
|
226
|
+
JSON results stay in source order. Progress callbacks execute serially on the
|
|
227
|
+
calling thread. Operational errors, callback failures and interruptions stop
|
|
228
|
+
the pool, terminate active test commands and clean up all working copies.
|
|
229
|
+
Partial JSON error reports retain only completed results, in source order.
|
|
230
|
+
|
|
231
|
+
## Ruby API
|
|
232
|
+
|
|
233
|
+
```ruby
|
|
234
|
+
require "moots"
|
|
235
|
+
|
|
236
|
+
results = Moots.run(
|
|
237
|
+
include: ["lib/**/*.rb"],
|
|
238
|
+
exclude: ["lib/generated/**/*.rb"],
|
|
239
|
+
test_command: ["bundle", "exec", "rspec"],
|
|
240
|
+
timeout: 60
|
|
241
|
+
)
|
|
242
|
+
|
|
243
|
+
results.each do |result|
|
|
244
|
+
puts [result.mutation.file_path, result.mutation.line_number, result.killed].inspect
|
|
245
|
+
puts result.test_result.output
|
|
246
|
+
end
|
|
247
|
+
```
|
|
248
|
+
|
|
249
|
+
`MutationGenerator#generate_mutations(code, file_path: "example.rb")` returns
|
|
250
|
+
mutations with original/replacement expressions and a byte offset.
|
|
251
|
+
`Mutation#apply(code)` returns a changed copy and rejects mismatched source.
|
|
252
|
+
`Mutation#id` returns the rerunnable ID; `Mutation#relative_path` returns its
|
|
253
|
+
project-relative path. Both accept an optional `root:` directory.
|
|
254
|
+
Directly constructed mutations now require `start_offset:`.
|
|
255
|
+
Invalid input raises `Moots::Error`; source I/O errors also propagate from the API.
|
|
256
|
+
|
|
257
|
+
An optional block receives progress hashes while `Moots.run` still returns the
|
|
258
|
+
results array. Events are `:workspace_started` (isolated runs only, before copying),
|
|
259
|
+
`:baseline_started`, `:baseline_passed`,
|
|
260
|
+
`:mutation_started` (with `:mutation`, one-based `:index`, and `:total`) and
|
|
261
|
+
`:mutation_finished` (with `:result`, `:index`, and `:total`). Callbacks execute
|
|
262
|
+
before source modification or after restoration, and callback exceptions propagate.
|
|
263
|
+
After the baseline passes, Moots reads all selected source files to determine the
|
|
264
|
+
mutation count before executing any mutations.
|
|
265
|
+
|
|
266
|
+
## Development and verification
|
|
267
|
+
|
|
268
|
+
```sh
|
|
269
|
+
bundle install
|
|
270
|
+
bundle exec rake spec # randomized RSpec suite
|
|
271
|
+
bundle exec rake acceptance # independent RSpec and Minitest comparison fixtures
|
|
272
|
+
bundle exec rake coverage # 100% line and branch coverage, overall and per file
|
|
273
|
+
bundle exec rake rubocop
|
|
274
|
+
bundle exec rake mutation # run Moots against all lib/**/*.rb files
|
|
275
|
+
bundle exec rake "mutation[2]" # dogfood with two isolated workers
|
|
276
|
+
bundle exec rake verify # lint, coverage, then dogfood
|
|
277
|
+
bundle exec rake build
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
Coverage includes unloaded runtime files. Only `lib/moots/version.rb` is excluded,
|
|
281
|
+
because Bundler loads its release constant before coverage starts; the version
|
|
282
|
+
is checked separately. No runtime implementation file is excluded from
|
|
283
|
+
self-mutation. The mutation task disables coverage thresholds in child suites
|
|
284
|
+
so kills come from test failures rather than uncovered-line enforcement.
|
|
285
|
+
This repository's `.moots.yml` uses RSpec's `--fail-fast`: the passing baseline
|
|
286
|
+
still runs every example, while a mutant stops at its first failing assertion.
|
|
287
|
+
|
|
288
|
+
Tests use temporary projects, real subprocesses and controlled failures. They
|
|
289
|
+
exercise every supported operator, exact mutation locations, modern syntax,
|
|
290
|
+
encodings, configuration and CLI behavior, baseline validation, process output,
|
|
291
|
+
deadlines and byte-for-byte restoration. The integration suite proves that
|
|
292
|
+
strong tests kill mutations, weak tests leave survivors, and broken baselines
|
|
293
|
+
are rejected.
|
|
294
|
+
|
|
295
|
+
The [comparison acceptance corpus](spec/fixtures/comparisons/README.md) uses a
|
|
296
|
+
standalone subject with hand-written expected results, independent of Moots'
|
|
297
|
+
operator tables. RSpec and Minitest each run strong and deliberately weak suites.
|
|
298
|
+
The strong suites kill all six comparison mutations. The weak suites omit the
|
|
299
|
+
exact boundary: equality mutations still die, but all four boundary mutations
|
|
300
|
+
survive. These acceptance checks are also included in the normal spec suite.
|
|
301
|
+
|
|
302
|
+
The self-mutation run must generate at least one mutation and kill every one.
|
|
303
|
+
CI runs it with two workers on all supported Linux Ruby versions. Review surviving
|
|
304
|
+
mutations and improve behavioral assertions; do not exclude implementation
|
|
305
|
+
files just to get a green result.
|
|
306
|
+
|
|
307
|
+
## Contributing and license
|
|
308
|
+
|
|
309
|
+
Bug reports and pull requests are welcome at
|
|
310
|
+
[OkayDave/moots](https://github.com/OkayDave/moots).
|
|
311
|
+
Released under the [MIT License](LICENSE.txt).
|
|
312
|
+
Contributors must follow the [Code of Conduct](CODE_OF_CONDUCT.md).
|
data/docs/isolation.md
ADDED
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
# Isolated working copies
|
|
2
|
+
|
|
3
|
+
`isolation: true` is the default for both the CLI and `Moots.run`. The project
|
|
4
|
+
root is the current working directory. Moots snapshots its current files, runs
|
|
5
|
+
the baseline and every mutation in the snapshot, then removes it. This does not
|
|
6
|
+
require Git and includes uncommitted changes and untracked files.
|
|
7
|
+
|
|
8
|
+
Files are copied independently, never hardlinked to the checkout. File bytes
|
|
9
|
+
and executable permissions are preserved. Hidden files and dependencies are
|
|
10
|
+
included. `.git`, `.hg` and `.svn` entries are always omitted at every depth.
|
|
11
|
+
Tests that need version-control metadata can use explicit in-place operation.
|
|
12
|
+
|
|
13
|
+
## Copy exclusions
|
|
14
|
+
|
|
15
|
+
```yaml
|
|
16
|
+
isolation: true
|
|
17
|
+
workspace_exclude:
|
|
18
|
+
- tmp
|
|
19
|
+
- coverage
|
|
20
|
+
- pkg
|
|
21
|
+
- .rspec_status
|
|
22
|
+
- node_modules
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
Patterns match project-relative paths, with `/` as the separator. `*`, `**`,
|
|
26
|
+
brace alternatives and hidden names are supported. A matching directory omits
|
|
27
|
+
its entire subtree. `cache` matches the root cache directory; `**/cache` matches
|
|
28
|
+
cache directories at any depth. A custom list replaces the defaults. Use `[]`
|
|
29
|
+
to copy all ordinary files, including default-excluded directories.
|
|
30
|
+
|
|
31
|
+
`exclude` still only removes mutation targets. Tests, fixtures and support files
|
|
32
|
+
usually need to be copied even when they are excluded from mutation.
|
|
33
|
+
|
|
34
|
+
Links, directory junctions and special files are rejected with a diagnostic;
|
|
35
|
+
Moots does not follow or preserve them into the snapshot. Copy exclusions are
|
|
36
|
+
applied first, so unnecessary linked directories can be omitted explicitly.
|
|
37
|
+
Read failures and insufficient disk space abort the run and remove partial copies.
|
|
38
|
+
|
|
39
|
+
## Commands, Bundler and paths
|
|
40
|
+
|
|
41
|
+
The parent process keeps its original working directory and environment. Each
|
|
42
|
+
test subprocess starts in the snapshot. Relative paths therefore use the copied
|
|
43
|
+
files. Absolute project-local paths supplied as separate `test_command` array
|
|
44
|
+
arguments are rebased into the snapshot. External paths remain unchanged.
|
|
45
|
+
Use separate arguments for path-bearing options, for example `[ruby, -I, lib,
|
|
46
|
+
test.rb]`. Embedded paths such as `-I/checkout/lib` are not rewritten.
|
|
47
|
+
|
|
48
|
+
Moots also rebases project-local entries in `BUNDLE_GEMFILE`, `BUNDLE_APP_CONFIG`,
|
|
49
|
+
`BUNDLE_BIN_PATH`, `BUNDLE_PATH`, `GEM_HOME`, `GEM_PATH`, `RUBYLIB` and `PATH`,
|
|
50
|
+
including Bundler's saved `BUNDLER_ORIG_` values. Dependencies under `vendor/bundle`
|
|
51
|
+
are copied, preserving relative Bundler install paths. Dependencies outside the
|
|
52
|
+
project continue to be shared.
|
|
53
|
+
|
|
54
|
+
Shell command strings, Ruby snippets, `RUBYOPT`, configuration file contents and
|
|
55
|
+
other environment variables are not rewritten. Prefer relative paths in those
|
|
56
|
+
places. An absolute Bundler path or local gem override stored inside
|
|
57
|
+
`.bundle/config` can still point to the original project. Configure those paths
|
|
58
|
+
relatively, or use `isolation: false` when relocation is incompatible with the
|
|
59
|
+
project. Ensure the test command actually reloads the copied source; persistent
|
|
60
|
+
test preloaders are unsupported.
|
|
61
|
+
|
|
62
|
+
Include/exclude source globs must stay inside the project in isolated mode.
|
|
63
|
+
Results retain the original source paths and IDs; reported rerun commands should
|
|
64
|
+
be executed from the original project directory with the same source and config.
|
|
65
|
+
|
|
66
|
+
## Lifetime and limits
|
|
67
|
+
|
|
68
|
+
With `jobs: 1`, one temporary directory is used for the whole run. With multiple
|
|
69
|
+
workers, the baseline runs once and each worker receives an independent copy of
|
|
70
|
+
that verified snapshot, including baseline-generated artifacts. Workers reuse
|
|
71
|
+
their own copy across assigned mutants; no two workers mutate the same copy.
|
|
72
|
+
Source bytes are restored between mutants, while other test-generated files can
|
|
73
|
+
persist in that worker. Tests should reset their own state between invocations.
|
|
74
|
+
|
|
75
|
+
Cleanup runs on success, failure, timeout and interruption. Active worker test
|
|
76
|
+
commands are stopped before their directories are removed. Hard termination can
|
|
77
|
+
leave `moots-workspace-*` or `moots-workers-*` directories in the system temporary
|
|
78
|
+
directory. The original source bytes remain untouched by mutation writes even
|
|
79
|
+
in that case. Parallel runs need extra disk space and do not isolate shared
|
|
80
|
+
external resources such as databases, services and network ports.
|
|
81
|
+
|
|
82
|
+
The temporary directory must be outside the project to prevent recursive copying.
|
|
83
|
+
The snapshot is not atomic: avoid editing source while it is being copied.
|
|
84
|
+
Isolation does not confine test commands or isolate external databases/services.
|
|
85
|
+
|
|
86
|
+
`isolation: false` uses the original checkout and ignores copy exclusions. It
|
|
87
|
+
retains the previous source restoration behavior and its limits: force-killing
|
|
88
|
+
Moots or losing power while a mutation is active can leave a changed file. Use
|
|
89
|
+
a disposable checkout for this compatibility mode.
|
data/docs/reporting.md
ADDED
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
# Reporting contract (schema version 1)
|
|
2
|
+
|
|
3
|
+
Run `bundle exec moots --format json > moots-report.json`. Standard output is
|
|
4
|
+
one JSON document; progress is flushed to standard error. The normal exit codes
|
|
5
|
+
still apply: 0 for a nonempty all-killed run, 1 for survivors or no mutations,
|
|
6
|
+
2 for operational errors, and 130 for interruption. Help/version requests print
|
|
7
|
+
their usual text instead of JSON. Put `--format json` before other arguments if
|
|
8
|
+
you also need option-parsing errors in JSON.
|
|
9
|
+
|
|
10
|
+
## Document fields
|
|
11
|
+
|
|
12
|
+
| Field | Type | Meaning |
|
|
13
|
+
| --- | --- | --- |
|
|
14
|
+
| `schema_version` | integer | Currently `1` |
|
|
15
|
+
| `status` | string | `completed` or `error` |
|
|
16
|
+
| `summary.mutations` | integer | Number of completed mutation tests |
|
|
17
|
+
| `summary.killed` | integer | Completed mutations that failed their tests |
|
|
18
|
+
| `summary.survived` | integer | Completed mutations that passed their tests |
|
|
19
|
+
| `results` | array | Completed results in execution order |
|
|
20
|
+
| `error` | string | Present on errors; describes why execution stopped |
|
|
21
|
+
|
|
22
|
+
Each result has these fields:
|
|
23
|
+
|
|
24
|
+
| Field | Type | Meaning |
|
|
25
|
+
| --- | --- | --- |
|
|
26
|
+
| `id` | string | 16 lowercase hexadecimal characters identifying the mutation |
|
|
27
|
+
| `file` | string | Source path relative to the project working directory |
|
|
28
|
+
| `line` | integer | One-based line containing the changed operator |
|
|
29
|
+
| `start_offset` | integer | Zero-based byte offset of the replaced expression |
|
|
30
|
+
| `original` | string | Original expression |
|
|
31
|
+
| `mutated` | string | Replacement expression |
|
|
32
|
+
| `status` | string | `killed` or `survived` |
|
|
33
|
+
| `test_output` | string | Captured output from this mutation's test command |
|
|
34
|
+
| `rerun` | array of strings | Executable and literal arguments for rerunning this mutation |
|
|
35
|
+
|
|
36
|
+
IDs are the first 16 hexadecimal characters of SHA-256 over the compact JSON
|
|
37
|
+
array `[relative_path, start_offset, original_hex_bytes, mutated_hex_bytes]`.
|
|
38
|
+
Identical source at the same relative path has the same ID across checkouts.
|
|
39
|
+
The reported line describes the operator, while the byte offset describes the
|
|
40
|
+
whole replaced expression; these can start on different lines.
|
|
41
|
+
|
|
42
|
+
## Reproduction
|
|
43
|
+
|
|
44
|
+
From the original working directory, using the same source revision:
|
|
45
|
+
|
|
46
|
+
```sh
|
|
47
|
+
bundle exec moots --mutation MUTATION_ID
|
|
48
|
+
bundle exec moots --config config/moots.yml --mutation MUTATION_ID
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
The `rerun` array preserves the configuration path, including spaces, without
|
|
52
|
+
shell-specific quoting. A Ruby consumer can execute a result from its own report
|
|
53
|
+
with `system(*result.fetch("rerun"))`. The baseline is always checked again.
|
|
54
|
+
Missing IDs are errors, including when source changes invalidate an earlier ID.
|
|
55
|
+
|
|
56
|
+
## Failure and encoding
|
|
57
|
+
|
|
58
|
+
An error document retains results completed before the failure. Its summary is
|
|
59
|
+
not a score for the whole intended run. A failed or interrupted mutation is not
|
|
60
|
+
recorded as killed or survived. A failed baseline has an empty results array.
|
|
61
|
+
Hard termination cannot produce a final document.
|
|
62
|
+
|
|
63
|
+
With `--jobs N`, progress messages can arrive in completion order. Each mutation
|
|
64
|
+
keeps its original index and ID; final JSON results, including partial error
|
|
65
|
+
results, remain in source order. The rerun command selects one mutation and does
|
|
66
|
+
not need to preserve a CLI worker-count override.
|
|
67
|
+
|
|
68
|
+
Report strings are decoded as UTF-8 with invalid bytes replaced by `�`. IDs use
|
|
69
|
+
the exact source bytes. The test runner combines stdout/stderr and trims leading
|
|
70
|
+
and trailing whitespace before storing `test_output`.
|