mutmut 3.6.0__tar.gz → 3.8.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {mutmut-3.6.0 → mutmut-3.8.0}/HISTORY.rst +59 -1
- {mutmut-3.6.0 → mutmut-3.8.0}/PKG-INFO +204 -10
- {mutmut-3.6.0 → mutmut-3.8.0}/README.rst +196 -3
- mutmut-3.8.0/pyproject.toml +113 -0
- mutmut-3.6.0/pyproject.toml → mutmut-3.8.0/pyproject.toml.orig +9 -8
- mutmut-3.8.0/src/mutmut/__init__.py +48 -0
- mutmut-3.8.0/src/mutmut/__main__.py +1139 -0
- mutmut-3.8.0/src/mutmut/code_coverage.py +105 -0
- {mutmut-3.6.0 → mutmut-3.8.0}/src/mutmut/configuration.py +123 -17
- mutmut-3.8.0/src/mutmut/core.py +25 -0
- mutmut-3.8.0/src/mutmut/models/__init__.py +1 -0
- mutmut-3.8.0/src/mutmut/models/results.py +44 -0
- mutmut-3.8.0/src/mutmut/mutation/data.py +133 -0
- mutmut-3.8.0/src/mutmut/mutation/diff_apply.py +186 -0
- {mutmut-3.6.0 → mutmut-3.8.0}/src/mutmut/mutation/file_mutation.py +202 -16
- {mutmut-3.6.0 → mutmut-3.8.0}/src/mutmut/mutation/mutators.py +22 -0
- {mutmut-3.6.0 → mutmut-3.8.0}/src/mutmut/mutation/pragma_handling.py +96 -8
- mutmut-3.8.0/src/mutmut/mutation/trampoline.py +176 -0
- mutmut-3.8.0/src/mutmut/runners/__init__.py +1 -0
- mutmut-3.8.0/src/mutmut/runners/harness.py +255 -0
- mutmut-3.8.0/src/mutmut/state.py +43 -0
- mutmut-3.8.0/src/mutmut/stats.py +173 -0
- mutmut-3.8.0/src/mutmut/ui/__init__.py +1 -0
- mutmut-3.8.0/src/mutmut/ui/browse.py +254 -0
- mutmut-3.8.0/src/mutmut/ui/terminal.py +35 -0
- mutmut-3.8.0/src/mutmut/utils/file_utils.py +92 -0
- {mutmut-3.6.0 → mutmut-3.8.0}/src/mutmut/utils/format_utils.py +31 -0
- mutmut-3.8.0/src/mutmut/utils/logging_utils.py +74 -0
- {mutmut-3.6.0 → mutmut-3.8.0}/src/mutmut/utils/safe_setproctitle.py +2 -2
- mutmut-3.8.0/src/mutmut/workers/__init__.py +1 -0
- mutmut-3.8.0/src/mutmut/workers/isolation.py +953 -0
- mutmut-3.6.0/src/mutmut/__init__.py +0 -42
- mutmut-3.6.0/src/mutmut/__main__.py +0 -1550
- mutmut-3.6.0/src/mutmut/code_coverage.py +0 -70
- mutmut-3.6.0/src/mutmut/mutation/data.py +0 -61
- mutmut-3.6.0/src/mutmut/mutation/trampoline.py +0 -105
- mutmut-3.6.0/src/mutmut/skills/kill-survivors.md +0 -18
- mutmut-3.6.0/src/mutmut/utils/__init__.py +0 -0
- mutmut-3.6.0/src/mutmut/utils/file_utils.py +0 -14
- {mutmut-3.6.0 → mutmut-3.8.0}/LICENSE +0 -0
- {mutmut-3.6.0 → mutmut-3.8.0}/src/mutmut/mutation/__init__.py +0 -0
- {mutmut-3.6.0 → mutmut-3.8.0}/src/mutmut/mutation/trampoline_templates.py +0 -0
- {mutmut-3.6.0 → mutmut-3.8.0}/src/mutmut/py.typed +0 -0
- {mutmut-3.6.0 → mutmut-3.8.0}/src/mutmut/type_checking.py +0 -0
- {mutmut-3.6.0/src/mutmut → mutmut-3.8.0/src/mutmut/ui}/result_browser_layout.tcss +0 -0
- {mutmut-3.6.0/src/mutmut/threading → mutmut-3.8.0/src/mutmut/utils}/__init__.py +0 -0
- {mutmut-3.6.0/src/mutmut/threading → mutmut-3.8.0/src/mutmut/workers}/timeout.py +0 -0
|
@@ -1,6 +1,64 @@
|
|
|
1
1
|
Changelog
|
|
2
2
|
---------
|
|
3
3
|
|
|
4
|
+
3.8.0
|
|
5
|
+
~~~~~~~~~~
|
|
6
|
+
|
|
7
|
+
* Fix obsolete test names never being removed from ``mutants/mutmut-stats.json`` (`#564`)
|
|
8
|
+
|
|
9
|
+
* Fix ``# pragma: no mutate block`` being silently ignored when placed on an ``else``, ``except``, ``except*`` or ``finally`` header, and on the ``try`` line of a ``try``/``except*`` (`#559`)
|
|
10
|
+
|
|
11
|
+
* Fix ``# pragma: no mutate`` being silently ignored on ``match`` and ``case`` headers (`#554`)
|
|
12
|
+
|
|
13
|
+
* Fix mutants being reported as survived when a test uses ``patch.dict(os.environ, ..., clear=True)`` (`#511`, `#552`)
|
|
14
|
+
|
|
15
|
+
* Fix `mutate_only_covered_lines` mutating code that coverage.py excludes from measurement (`# pragma: no cover`, `exclude_lines`, `exclude_also`) (`#547`)
|
|
16
|
+
|
|
17
|
+
* Fix mutations that delete ignored code. Dropping a `case` from a `match`, or an argument from a call, is a mutation of the enclosing node, so it used to be made even when the removed lines were themselves ignored (`#547`)
|
|
18
|
+
|
|
19
|
+
* Fix methods of decorated classes (for example `@dataclass`) not being mutated (`#480`, `#539`)
|
|
20
|
+
|
|
21
|
+
* Fix time measurement including test setup/teardown instead of only the main test run (`#544`)
|
|
22
|
+
|
|
23
|
+
* Fix mutants stopped by the CPU-time limit being reported as killed instead of timed out (`#565`)
|
|
24
|
+
|
|
25
|
+
* Fix ``mutate_only_covered_lines`` breaking when the test suite imports a dependency that cannot be initialized twice in one process, which raised for numpy and libyaml, produced misleading type errors for cryptography, and segfaulted for asyncpg. Coverage is now gathered in a separate process, so the modules the coverage run imports are no longer unloaded and re-imported in the main process (`#528`, `#566`)
|
|
26
|
+
|
|
27
|
+
* Mutate the condition of ternary expressions (`#196`, `#546`)
|
|
28
|
+
|
|
29
|
+
* Add a ``process_isolation`` config for choosing which process mutant workers are forked from. The new ``forkserver`` strategy keeps mutmut's main process free of pytest and your ``conftest.py``, routing every fork through a dedicated fork server process, for test setups that are not fork-safe (``gevent``, ``grpc``, ``torch``). Tune what it preloads with ``forkserver_warmup`` (`#578`, `#566`)
|
|
30
|
+
|
|
31
|
+
* Add a `badge` command that writes a shields.io endpoint file, for publishing the mutation score (`#549`)
|
|
32
|
+
|
|
33
|
+
* Support python3.15 (`#551`)
|
|
34
|
+
|
|
35
|
+
3.7.0
|
|
36
|
+
~~~~~
|
|
37
|
+
|
|
38
|
+
* Fix the trampoline dropping a generator's return value, so ``yield from`` on a mutated generator no longer yields ``None`` instead of its result
|
|
39
|
+
|
|
40
|
+
* Fix async generators, so that `aclose()`/`athrow()` are handled by the `except`/`finally` blocks of the mutated function instead of being swallowed by the trampoline
|
|
41
|
+
|
|
42
|
+
* Fix `max_stack_depth` when a test changes the working directory (for example via a tmpdir fixture)
|
|
43
|
+
|
|
44
|
+
* Fix `type_check_command` reporting type errors in copied files that have no mutants
|
|
45
|
+
|
|
46
|
+
* Per-function source hashing for incremental cache invalidation — only re-test mutants in functions that changed
|
|
47
|
+
|
|
48
|
+
* Cross-call dependency tracking — invalidate mutants in callers when a called function changes
|
|
49
|
+
|
|
50
|
+
* Use git to detect non-Python dependency file changes; falls back to a curated file list when git is unavailable
|
|
51
|
+
|
|
52
|
+
* Add `cache_invalidation_exclude` config to suppress noisy files from change detection
|
|
53
|
+
|
|
54
|
+
* Add `use_git_change_detection` config (default true) to opt out of git-based detection
|
|
55
|
+
|
|
56
|
+
* Add `uv.lock`, `poetry.lock`, `Pipfile.lock`, and `pdm.lock` to the list of files copied by default
|
|
57
|
+
|
|
58
|
+
* Invalidate cached results automatically when result-affecting config fields change
|
|
59
|
+
|
|
60
|
+
* Make `mutmut browse`, `show` and `apply` much faster and use far less memory, by storing where each mutant is in the mutated file instead of parsing the whole file to find it
|
|
61
|
+
|
|
4
62
|
3.6.0
|
|
5
63
|
~~~~~
|
|
6
64
|
|
|
@@ -559,7 +617,7 @@ Thanks goes out Marcelo Da Cruz Pinto, Savo Kovačević,
|
|
|
559
617
|
|
|
560
618
|
* Dict literals looking like `dict(a=foo)` now have mutated keys. You can also declare synonyms in setup.cfg.
|
|
561
619
|
|
|
562
|
-
* Fix
|
|
620
|
+
* Fix ``from x import *``
|
|
563
621
|
|
|
564
622
|
|
|
565
623
|
0.0.6 (2017-06-13)
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: mutmut
|
|
3
|
-
Version: 3.
|
|
3
|
+
Version: 3.8.0
|
|
4
4
|
Summary: mutation testing for Python 3
|
|
5
5
|
Keywords: mutmut,mutant,mutation,test,testing
|
|
6
6
|
Author: Anders Hovmöller
|
|
@@ -16,19 +16,20 @@ Classifier: Programming Language :: Python :: 3.11
|
|
|
16
16
|
Classifier: Programming Language :: Python :: 3.12
|
|
17
17
|
Classifier: Programming Language :: Python :: 3.13
|
|
18
18
|
Classifier: Programming Language :: Python :: 3.14
|
|
19
|
-
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.15
|
|
20
|
+
Requires-Dist: click>=8.4.2
|
|
20
21
|
Requires-Dist: coverage>=7.3.0
|
|
21
|
-
Requires-Dist: libcst>=1.
|
|
22
|
+
Requires-Dist: libcst>=1.9.0
|
|
22
23
|
Requires-Dist: pytest>=6.2.5
|
|
23
|
-
Requires-Dist: setproctitle>=1.
|
|
24
|
+
Requires-Dist: setproctitle>=1.3.7
|
|
24
25
|
Requires-Dist: textual>=1.0.0
|
|
25
26
|
Requires-Dist: toml>=0.10.2 ; python_full_version < '3.11'
|
|
26
27
|
Requires-Python: >=3.10
|
|
27
|
-
Project-URL: Changelog, https://github.com/boxed/mutmut/blob/main/HISTORY.rst
|
|
28
|
-
Project-URL: Documentation, https://mutmut.readthedocs.io/en/latest/
|
|
29
28
|
Project-URL: Homepage, https://github.com/boxed/mutmut
|
|
30
|
-
Project-URL:
|
|
29
|
+
Project-URL: Documentation, https://mutmut.readthedocs.io/en/latest/
|
|
31
30
|
Project-URL: Repository, https://github.com/boxed/mutmut
|
|
31
|
+
Project-URL: Issues, https://github.com/boxed/mutmut/issues
|
|
32
|
+
Project-URL: Changelog, https://github.com/boxed/mutmut/blob/main/HISTORY.rst
|
|
32
33
|
Description-Content-Type: text/x-rst
|
|
33
34
|
|
|
34
35
|
mutmut - python mutation tester
|
|
@@ -80,7 +81,7 @@ You can get started with a simple:
|
|
|
80
81
|
pip install mutmut
|
|
81
82
|
mutmut run
|
|
82
83
|
|
|
83
|
-
This will
|
|
84
|
+
This will run pytest on tests in the "tests" or "test" folder and
|
|
84
85
|
it will try to figure out where the code to mutate is.
|
|
85
86
|
|
|
86
87
|
|
|
@@ -96,7 +97,7 @@ You can also write a mutant to disk from the `browse` interface, or via
|
|
|
96
97
|
source code control and committed before you apply a mutant!
|
|
97
98
|
|
|
98
99
|
|
|
99
|
-
If during the installation you get an error for the `libcst` dependency mentioning the lack of a rust compiler on your system, it is because your architecture does not have a prebuilt binary for `libcst` and it requires both `rustc` and `cargo` from the
|
|
100
|
+
If during the installation you get an error for the `libcst` dependency mentioning the lack of a rust compiler on your system, it is because your architecture does not have a prebuilt binary for `libcst` and it requires both `rustc` and `cargo` from the `rust toolchain <https://www.rust-lang.org/tools/install>`_ to be built. This is known for at least the `x86_64-darwin` architecture.
|
|
100
101
|
|
|
101
102
|
|
|
102
103
|
Wildcards for testing mutants
|
|
@@ -202,6 +203,11 @@ If you only want to mutate lines that are called (according to coverage.py), you
|
|
|
202
203
|
|
|
203
204
|
mutate_only_covered_lines=true
|
|
204
205
|
|
|
206
|
+
This also honours the lines coverage.py is told to leave out of its measurement, so code
|
|
207
|
+
marked with `# pragma: no cover`, or matched by your `exclude_lines`/`exclude_also`
|
|
208
|
+
settings, is not mutated either. Such code is not held to your test suite, so mutants
|
|
209
|
+
there could only ever show up as survivors.
|
|
210
|
+
|
|
205
211
|
|
|
206
212
|
Filter generated mutants with type checker
|
|
207
213
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
@@ -264,6 +270,108 @@ macOS at your own risk, or to disable it on other platforms), set ``use_setproct
|
|
|
264
270
|
use_setproctitle = false
|
|
265
271
|
|
|
266
272
|
|
|
273
|
+
Process isolation (``fork`` vs ``forkserver``)
|
|
274
|
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
275
|
+
|
|
276
|
+
.. code-block:: toml
|
|
277
|
+
|
|
278
|
+
# pyproject.toml
|
|
279
|
+
[tool.mutmut]
|
|
280
|
+
process_isolation = "fork" # or "forkserver"
|
|
281
|
+
|
|
282
|
+
Both strategies require ``os.fork``, so they are POSIX-only.
|
|
283
|
+
|
|
284
|
+
``fork`` (default)
|
|
285
|
+
Mutmut's main process imports pytest and runs your test suite to collect
|
|
286
|
+
stats, then forks one child per mutant. This is fast and needs no
|
|
287
|
+
configuration, but every mutant worker inherits whatever your test setup
|
|
288
|
+
left behind in that process: monkey-patched sockets, running event loops,
|
|
289
|
+
background threads, open database connections, CUDA contexts. If any of
|
|
290
|
+
that is not fork-safe, the workers can hang, segfault, or report nonsense.
|
|
291
|
+
|
|
292
|
+
``forkserver``
|
|
293
|
+
Mutmut's main process never imports pytest or your ``conftest.py``. It forks
|
|
294
|
+
a dedicated *fork server* child, and that child does the importing once and
|
|
295
|
+
forks a worker per mutant::
|
|
296
|
+
|
|
297
|
+
main process (clean) -> fork server (imports pytest) -> N mutant workers
|
|
298
|
+
|
|
299
|
+
Because the fork server only imports and forks, and never runs a test
|
|
300
|
+
itself, every worker starts from the same predictable state. This is the
|
|
301
|
+
same design as the ``forkserver`` start method in Python's
|
|
302
|
+
``multiprocessing``, and it is still only one pytest import for the whole
|
|
303
|
+
run, so it is not slower than ``fork`` in the normal case.
|
|
304
|
+
|
|
305
|
+
**Which one should you use?** Start with ``fork``. Switch to ``forkserver`` if
|
|
306
|
+
your run hangs, crashes, or gives results you cannot reproduce by running the
|
|
307
|
+
tests by hand, and your test setup touches something known to be fork-unsafe,
|
|
308
|
+
such as ``gevent.monkey.patch_all()``, ``grpc``, or ``torch``. It is also the
|
|
309
|
+
better choice if you simply do not want your session-scoped setup living in the
|
|
310
|
+
process that forks mutants.
|
|
311
|
+
|
|
312
|
+
|
|
313
|
+
Tuning what the fork server preloads
|
|
314
|
+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
315
|
+
|
|
316
|
+
``forkserver_warmup`` controls how much the fork server imports before it starts
|
|
317
|
+
forking workers. It only applies when ``process_isolation = "forkserver"``.
|
|
318
|
+
Anything imported in the fork server is inherited by every worker, and anything
|
|
319
|
+
left out is imported in each worker.
|
|
320
|
+
|
|
321
|
+
.. code-block:: toml
|
|
322
|
+
|
|
323
|
+
# pyproject.toml
|
|
324
|
+
[tool.mutmut]
|
|
325
|
+
process_isolation = "forkserver"
|
|
326
|
+
forkserver_warmup = "collect" # "collect" (default), "import" or "none"
|
|
327
|
+
|
|
328
|
+
``collect`` (default)
|
|
329
|
+
Run ``pytest --collect-only``, which imports your ``conftest.py``, your
|
|
330
|
+
plugins and your test modules. This is the right default for most projects
|
|
331
|
+
and usually the fastest.
|
|
332
|
+
|
|
333
|
+
``import``
|
|
334
|
+
Import only the modules listed in ``preload_modules_file`` (one dotted
|
|
335
|
+
module name per line; blank lines and ``#`` comments are ignored, and
|
|
336
|
+
modules that fail to import are skipped). Use this when test *collection*
|
|
337
|
+
itself has side effects you do not want every worker to share, but you still
|
|
338
|
+
want your slow imports warmed up.
|
|
339
|
+
|
|
340
|
+
``none``
|
|
341
|
+
Import nothing beyond what running a single test requires. This is the
|
|
342
|
+
slowest option, and the most isolated. It is mostly useful for working out
|
|
343
|
+
whether a warmup side effect is what is breaking your run.
|
|
344
|
+
|
|
345
|
+
.. code-block:: toml
|
|
346
|
+
|
|
347
|
+
# pyproject.toml
|
|
348
|
+
[tool.mutmut]
|
|
349
|
+
process_isolation = "forkserver"
|
|
350
|
+
forkserver_warmup = "import"
|
|
351
|
+
preload_modules_file = "mutmut_preload.txt"
|
|
352
|
+
|
|
353
|
+
If the fork server itself crashes, mutmut starts a new one and re-submits the
|
|
354
|
+
mutants that were in flight, up to ``max_forkserver_restarts`` times (default
|
|
355
|
+
``3``) before giving up with a ``ForkServerCrashError``:
|
|
356
|
+
|
|
357
|
+
.. code-block:: toml
|
|
358
|
+
|
|
359
|
+
# pyproject.toml
|
|
360
|
+
[tool.mutmut]
|
|
361
|
+
max_forkserver_restarts = 3
|
|
362
|
+
|
|
363
|
+
The fork server and its workers cannot write to the console without mangling
|
|
364
|
+
mutmut's progress output, so they log to a file instead. Logging is off unless
|
|
365
|
+
you ask for it with ``log_to_file`` (or ``debug``):
|
|
366
|
+
|
|
367
|
+
.. code-block:: toml
|
|
368
|
+
|
|
369
|
+
# pyproject.toml
|
|
370
|
+
[tool.mutmut]
|
|
371
|
+
log_to_file = true
|
|
372
|
+
log_file_path = "mutants/mutmut-debug.log"
|
|
373
|
+
|
|
374
|
+
|
|
267
375
|
Disabling mutation on specific code
|
|
268
376
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
269
377
|
|
|
@@ -434,6 +542,77 @@ You can add and override pytest arguments:
|
|
|
434
542
|
also_copy = ["mutmut_pytest.ini"]
|
|
435
543
|
|
|
436
544
|
|
|
545
|
+
Detecting dependency and config changes
|
|
546
|
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
547
|
+
|
|
548
|
+
Between runs, mutmut only re-tests mutants in functions whose source changed.
|
|
549
|
+
Changes outside your Python source — a dependency upgrade, a data file, a
|
|
550
|
+
config file — cannot be tied to a function, so they would otherwise be missed
|
|
551
|
+
and you would get cached results that no longer reflect reality.
|
|
552
|
+
|
|
553
|
+
To catch this, mutmut detects non-Python files that changed since the last full
|
|
554
|
+
run and warns you about them. If your project is a git repository and git is
|
|
555
|
+
installed, mutmut uses git (a soft dependency no extra package is required) to
|
|
556
|
+
find every changed non-Python file, respecting your `.gitignore`. Python files
|
|
557
|
+
are excluded because their changes are already tracked per function.
|
|
558
|
+
|
|
559
|
+
On a full run with git available, mutmut also records the content hashes of the
|
|
560
|
+
tracked non-Python files. This means a later run in an environment without git
|
|
561
|
+
(for example a different CI stage) can still detect changes to that known set of
|
|
562
|
+
files, even though it cannot discover brand-new ones.
|
|
563
|
+
|
|
564
|
+
When git is unavailable, mutmut falls back to hashing a curated set of build and
|
|
565
|
+
dependency files:
|
|
566
|
+
|
|
567
|
+
- `pyproject.toml`
|
|
568
|
+
- `setup.cfg`
|
|
569
|
+
- `setup.py`
|
|
570
|
+
- `requirements*.txt`
|
|
571
|
+
- `poetry.lock`
|
|
572
|
+
- `uv.lock`
|
|
573
|
+
- `Pipfile`
|
|
574
|
+
- `Pipfile.lock`
|
|
575
|
+
|
|
576
|
+
You can watch additional files (for example data files your tests depend on)
|
|
577
|
+
with the `cache_invalidation_files` config, which accepts glob patterns
|
|
578
|
+
resolved against the project root. These are checked even when git ignores them,
|
|
579
|
+
and are never dropped by the exclusions below:
|
|
580
|
+
|
|
581
|
+
.. code-block:: toml
|
|
582
|
+
|
|
583
|
+
cache_invalidation_files = [ "queries/*.sql", "config/*.yaml" ]
|
|
584
|
+
|
|
585
|
+
Git detection reports every changed non-Python file, so mutmut drops files that
|
|
586
|
+
practically never affect tests (markdown, `LICENSE`, `CHANGELOG`, `docs/`, git
|
|
587
|
+
and editor metadata, ...). Exclude additional noisy files with
|
|
588
|
+
`cache_invalidation_exclude` (glob patterns, `*` spans directories):
|
|
589
|
+
|
|
590
|
+
.. code-block:: toml
|
|
591
|
+
|
|
592
|
+
cache_invalidation_exclude = [ "*.json", "fixtures/snapshots/*" ]
|
|
593
|
+
|
|
594
|
+
When a watched file changes, `on_dependency_change` controls what happens:
|
|
595
|
+
|
|
596
|
+
- `warn` (default): list the changed files and keep the cache.
|
|
597
|
+
- `rerun`: re-test all mutants.
|
|
598
|
+
- `ignore`: do nothing.
|
|
599
|
+
|
|
600
|
+
.. code-block:: toml
|
|
601
|
+
|
|
602
|
+
on_dependency_change = "warn"
|
|
603
|
+
|
|
604
|
+
Git detection is on by default; disable it (forcing the curated-list fallback)
|
|
605
|
+
with:
|
|
606
|
+
|
|
607
|
+
.. code-block:: toml
|
|
608
|
+
|
|
609
|
+
use_git_change_detection = false
|
|
610
|
+
|
|
611
|
+
Changes to mutmut's own result-affecting config (such as `pytest_add_cli_args`,
|
|
612
|
+
`type_check_command`, or the timeout settings) are always detected and
|
|
613
|
+
invalidate the affected cached results automatically.
|
|
614
|
+
|
|
615
|
+
|
|
437
616
|
Unstable configs
|
|
438
617
|
~~~~~~~~~~~~~~~~
|
|
439
618
|
|
|
@@ -473,9 +652,24 @@ This section describes how to work with mutmut to enhance your test suite.
|
|
|
473
652
|
4. Press `r` to rerun the mutant and see if you successfully managed to kill it.
|
|
474
653
|
|
|
475
654
|
Mutmut keeps the data of what it has done and the mutants in the `mutants/`
|
|
476
|
-
directory.If
|
|
655
|
+
directory. If you want to make sure you run a full mutmut run you can delete
|
|
477
656
|
this directory to start from scratch.
|
|
478
657
|
|
|
658
|
+
|
|
659
|
+
Mutation score badges
|
|
660
|
+
---------------------
|
|
661
|
+
|
|
662
|
+
`mutmut badge` turns `mutmut export-cicd-stats` output into `Shields endpoint JSON <https://shields.io/badges/endpoint-badge>`_:
|
|
663
|
+
|
|
664
|
+
.. code-block:: console
|
|
665
|
+
|
|
666
|
+
mutmut export-cicd-stats
|
|
667
|
+
mutmut badge --output mutation-score.json
|
|
668
|
+
|
|
669
|
+
.. code-block:: md
|
|
670
|
+
|
|
671
|
+

|
|
672
|
+
|
|
479
673
|
Contributing to Mutmut
|
|
480
674
|
----------------------
|
|
481
675
|
|
|
@@ -47,7 +47,7 @@ You can get started with a simple:
|
|
|
47
47
|
pip install mutmut
|
|
48
48
|
mutmut run
|
|
49
49
|
|
|
50
|
-
This will
|
|
50
|
+
This will run pytest on tests in the "tests" or "test" folder and
|
|
51
51
|
it will try to figure out where the code to mutate is.
|
|
52
52
|
|
|
53
53
|
|
|
@@ -63,7 +63,7 @@ You can also write a mutant to disk from the `browse` interface, or via
|
|
|
63
63
|
source code control and committed before you apply a mutant!
|
|
64
64
|
|
|
65
65
|
|
|
66
|
-
If during the installation you get an error for the `libcst` dependency mentioning the lack of a rust compiler on your system, it is because your architecture does not have a prebuilt binary for `libcst` and it requires both `rustc` and `cargo` from the
|
|
66
|
+
If during the installation you get an error for the `libcst` dependency mentioning the lack of a rust compiler on your system, it is because your architecture does not have a prebuilt binary for `libcst` and it requires both `rustc` and `cargo` from the `rust toolchain <https://www.rust-lang.org/tools/install>`_ to be built. This is known for at least the `x86_64-darwin` architecture.
|
|
67
67
|
|
|
68
68
|
|
|
69
69
|
Wildcards for testing mutants
|
|
@@ -169,6 +169,11 @@ If you only want to mutate lines that are called (according to coverage.py), you
|
|
|
169
169
|
|
|
170
170
|
mutate_only_covered_lines=true
|
|
171
171
|
|
|
172
|
+
This also honours the lines coverage.py is told to leave out of its measurement, so code
|
|
173
|
+
marked with `# pragma: no cover`, or matched by your `exclude_lines`/`exclude_also`
|
|
174
|
+
settings, is not mutated either. Such code is not held to your test suite, so mutants
|
|
175
|
+
there could only ever show up as survivors.
|
|
176
|
+
|
|
172
177
|
|
|
173
178
|
Filter generated mutants with type checker
|
|
174
179
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
@@ -231,6 +236,108 @@ macOS at your own risk, or to disable it on other platforms), set ``use_setproct
|
|
|
231
236
|
use_setproctitle = false
|
|
232
237
|
|
|
233
238
|
|
|
239
|
+
Process isolation (``fork`` vs ``forkserver``)
|
|
240
|
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
241
|
+
|
|
242
|
+
.. code-block:: toml
|
|
243
|
+
|
|
244
|
+
# pyproject.toml
|
|
245
|
+
[tool.mutmut]
|
|
246
|
+
process_isolation = "fork" # or "forkserver"
|
|
247
|
+
|
|
248
|
+
Both strategies require ``os.fork``, so they are POSIX-only.
|
|
249
|
+
|
|
250
|
+
``fork`` (default)
|
|
251
|
+
Mutmut's main process imports pytest and runs your test suite to collect
|
|
252
|
+
stats, then forks one child per mutant. This is fast and needs no
|
|
253
|
+
configuration, but every mutant worker inherits whatever your test setup
|
|
254
|
+
left behind in that process: monkey-patched sockets, running event loops,
|
|
255
|
+
background threads, open database connections, CUDA contexts. If any of
|
|
256
|
+
that is not fork-safe, the workers can hang, segfault, or report nonsense.
|
|
257
|
+
|
|
258
|
+
``forkserver``
|
|
259
|
+
Mutmut's main process never imports pytest or your ``conftest.py``. It forks
|
|
260
|
+
a dedicated *fork server* child, and that child does the importing once and
|
|
261
|
+
forks a worker per mutant::
|
|
262
|
+
|
|
263
|
+
main process (clean) -> fork server (imports pytest) -> N mutant workers
|
|
264
|
+
|
|
265
|
+
Because the fork server only imports and forks, and never runs a test
|
|
266
|
+
itself, every worker starts from the same predictable state. This is the
|
|
267
|
+
same design as the ``forkserver`` start method in Python's
|
|
268
|
+
``multiprocessing``, and it is still only one pytest import for the whole
|
|
269
|
+
run, so it is not slower than ``fork`` in the normal case.
|
|
270
|
+
|
|
271
|
+
**Which one should you use?** Start with ``fork``. Switch to ``forkserver`` if
|
|
272
|
+
your run hangs, crashes, or gives results you cannot reproduce by running the
|
|
273
|
+
tests by hand, and your test setup touches something known to be fork-unsafe,
|
|
274
|
+
such as ``gevent.monkey.patch_all()``, ``grpc``, or ``torch``. It is also the
|
|
275
|
+
better choice if you simply do not want your session-scoped setup living in the
|
|
276
|
+
process that forks mutants.
|
|
277
|
+
|
|
278
|
+
|
|
279
|
+
Tuning what the fork server preloads
|
|
280
|
+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
281
|
+
|
|
282
|
+
``forkserver_warmup`` controls how much the fork server imports before it starts
|
|
283
|
+
forking workers. It only applies when ``process_isolation = "forkserver"``.
|
|
284
|
+
Anything imported in the fork server is inherited by every worker, and anything
|
|
285
|
+
left out is imported in each worker.
|
|
286
|
+
|
|
287
|
+
.. code-block:: toml
|
|
288
|
+
|
|
289
|
+
# pyproject.toml
|
|
290
|
+
[tool.mutmut]
|
|
291
|
+
process_isolation = "forkserver"
|
|
292
|
+
forkserver_warmup = "collect" # "collect" (default), "import" or "none"
|
|
293
|
+
|
|
294
|
+
``collect`` (default)
|
|
295
|
+
Run ``pytest --collect-only``, which imports your ``conftest.py``, your
|
|
296
|
+
plugins and your test modules. This is the right default for most projects
|
|
297
|
+
and usually the fastest.
|
|
298
|
+
|
|
299
|
+
``import``
|
|
300
|
+
Import only the modules listed in ``preload_modules_file`` (one dotted
|
|
301
|
+
module name per line; blank lines and ``#`` comments are ignored, and
|
|
302
|
+
modules that fail to import are skipped). Use this when test *collection*
|
|
303
|
+
itself has side effects you do not want every worker to share, but you still
|
|
304
|
+
want your slow imports warmed up.
|
|
305
|
+
|
|
306
|
+
``none``
|
|
307
|
+
Import nothing beyond what running a single test requires. This is the
|
|
308
|
+
slowest option, and the most isolated. It is mostly useful for working out
|
|
309
|
+
whether a warmup side effect is what is breaking your run.
|
|
310
|
+
|
|
311
|
+
.. code-block:: toml
|
|
312
|
+
|
|
313
|
+
# pyproject.toml
|
|
314
|
+
[tool.mutmut]
|
|
315
|
+
process_isolation = "forkserver"
|
|
316
|
+
forkserver_warmup = "import"
|
|
317
|
+
preload_modules_file = "mutmut_preload.txt"
|
|
318
|
+
|
|
319
|
+
If the fork server itself crashes, mutmut starts a new one and re-submits the
|
|
320
|
+
mutants that were in flight, up to ``max_forkserver_restarts`` times (default
|
|
321
|
+
``3``) before giving up with a ``ForkServerCrashError``:
|
|
322
|
+
|
|
323
|
+
.. code-block:: toml
|
|
324
|
+
|
|
325
|
+
# pyproject.toml
|
|
326
|
+
[tool.mutmut]
|
|
327
|
+
max_forkserver_restarts = 3
|
|
328
|
+
|
|
329
|
+
The fork server and its workers cannot write to the console without mangling
|
|
330
|
+
mutmut's progress output, so they log to a file instead. Logging is off unless
|
|
331
|
+
you ask for it with ``log_to_file`` (or ``debug``):
|
|
332
|
+
|
|
333
|
+
.. code-block:: toml
|
|
334
|
+
|
|
335
|
+
# pyproject.toml
|
|
336
|
+
[tool.mutmut]
|
|
337
|
+
log_to_file = true
|
|
338
|
+
log_file_path = "mutants/mutmut-debug.log"
|
|
339
|
+
|
|
340
|
+
|
|
234
341
|
Disabling mutation on specific code
|
|
235
342
|
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
236
343
|
|
|
@@ -401,6 +508,77 @@ You can add and override pytest arguments:
|
|
|
401
508
|
also_copy = ["mutmut_pytest.ini"]
|
|
402
509
|
|
|
403
510
|
|
|
511
|
+
Detecting dependency and config changes
|
|
512
|
+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
|
513
|
+
|
|
514
|
+
Between runs, mutmut only re-tests mutants in functions whose source changed.
|
|
515
|
+
Changes outside your Python source — a dependency upgrade, a data file, a
|
|
516
|
+
config file — cannot be tied to a function, so they would otherwise be missed
|
|
517
|
+
and you would get cached results that no longer reflect reality.
|
|
518
|
+
|
|
519
|
+
To catch this, mutmut detects non-Python files that changed since the last full
|
|
520
|
+
run and warns you about them. If your project is a git repository and git is
|
|
521
|
+
installed, mutmut uses git (a soft dependency no extra package is required) to
|
|
522
|
+
find every changed non-Python file, respecting your `.gitignore`. Python files
|
|
523
|
+
are excluded because their changes are already tracked per function.
|
|
524
|
+
|
|
525
|
+
On a full run with git available, mutmut also records the content hashes of the
|
|
526
|
+
tracked non-Python files. This means a later run in an environment without git
|
|
527
|
+
(for example a different CI stage) can still detect changes to that known set of
|
|
528
|
+
files, even though it cannot discover brand-new ones.
|
|
529
|
+
|
|
530
|
+
When git is unavailable, mutmut falls back to hashing a curated set of build and
|
|
531
|
+
dependency files:
|
|
532
|
+
|
|
533
|
+
- `pyproject.toml`
|
|
534
|
+
- `setup.cfg`
|
|
535
|
+
- `setup.py`
|
|
536
|
+
- `requirements*.txt`
|
|
537
|
+
- `poetry.lock`
|
|
538
|
+
- `uv.lock`
|
|
539
|
+
- `Pipfile`
|
|
540
|
+
- `Pipfile.lock`
|
|
541
|
+
|
|
542
|
+
You can watch additional files (for example data files your tests depend on)
|
|
543
|
+
with the `cache_invalidation_files` config, which accepts glob patterns
|
|
544
|
+
resolved against the project root. These are checked even when git ignores them,
|
|
545
|
+
and are never dropped by the exclusions below:
|
|
546
|
+
|
|
547
|
+
.. code-block:: toml
|
|
548
|
+
|
|
549
|
+
cache_invalidation_files = [ "queries/*.sql", "config/*.yaml" ]
|
|
550
|
+
|
|
551
|
+
Git detection reports every changed non-Python file, so mutmut drops files that
|
|
552
|
+
practically never affect tests (markdown, `LICENSE`, `CHANGELOG`, `docs/`, git
|
|
553
|
+
and editor metadata, ...). Exclude additional noisy files with
|
|
554
|
+
`cache_invalidation_exclude` (glob patterns, `*` spans directories):
|
|
555
|
+
|
|
556
|
+
.. code-block:: toml
|
|
557
|
+
|
|
558
|
+
cache_invalidation_exclude = [ "*.json", "fixtures/snapshots/*" ]
|
|
559
|
+
|
|
560
|
+
When a watched file changes, `on_dependency_change` controls what happens:
|
|
561
|
+
|
|
562
|
+
- `warn` (default): list the changed files and keep the cache.
|
|
563
|
+
- `rerun`: re-test all mutants.
|
|
564
|
+
- `ignore`: do nothing.
|
|
565
|
+
|
|
566
|
+
.. code-block:: toml
|
|
567
|
+
|
|
568
|
+
on_dependency_change = "warn"
|
|
569
|
+
|
|
570
|
+
Git detection is on by default; disable it (forcing the curated-list fallback)
|
|
571
|
+
with:
|
|
572
|
+
|
|
573
|
+
.. code-block:: toml
|
|
574
|
+
|
|
575
|
+
use_git_change_detection = false
|
|
576
|
+
|
|
577
|
+
Changes to mutmut's own result-affecting config (such as `pytest_add_cli_args`,
|
|
578
|
+
`type_check_command`, or the timeout settings) are always detected and
|
|
579
|
+
invalidate the affected cached results automatically.
|
|
580
|
+
|
|
581
|
+
|
|
404
582
|
Unstable configs
|
|
405
583
|
~~~~~~~~~~~~~~~~
|
|
406
584
|
|
|
@@ -440,9 +618,24 @@ This section describes how to work with mutmut to enhance your test suite.
|
|
|
440
618
|
4. Press `r` to rerun the mutant and see if you successfully managed to kill it.
|
|
441
619
|
|
|
442
620
|
Mutmut keeps the data of what it has done and the mutants in the `mutants/`
|
|
443
|
-
directory.If
|
|
621
|
+
directory. If you want to make sure you run a full mutmut run you can delete
|
|
444
622
|
this directory to start from scratch.
|
|
445
623
|
|
|
624
|
+
|
|
625
|
+
Mutation score badges
|
|
626
|
+
---------------------
|
|
627
|
+
|
|
628
|
+
`mutmut badge` turns `mutmut export-cicd-stats` output into `Shields endpoint JSON <https://shields.io/badges/endpoint-badge>`_:
|
|
629
|
+
|
|
630
|
+
.. code-block:: console
|
|
631
|
+
|
|
632
|
+
mutmut export-cicd-stats
|
|
633
|
+
mutmut badge --output mutation-score.json
|
|
634
|
+
|
|
635
|
+
.. code-block:: md
|
|
636
|
+
|
|
637
|
+

|
|
638
|
+
|
|
446
639
|
Contributing to Mutmut
|
|
447
640
|
----------------------
|
|
448
641
|
|