pytest-regtest 2.1.0__tar.gz → 2.3.2__tar.gz

pytest_regtest-2.3.2/.gitignore

@@ -0,0 +1,27 @@
+/.*
+!.gitlab-ci.yml
+!.pre-commit-config.yaml
+!.readthedocs.yaml
+
+.ropeproject
+
+
+*.egg-info/
+*.pyc
+.*.sw?
+venv*
+coverage.xml
+
+tests_dev/_regtest_outputs/
+
+/dist
+/build
+/site
+/html
+/sandbox
+
+package.json
+package-lock.json
+uv.lock
+devbox.lock
+node_modules/

pytest_regtest-2.3.2/PKG-INFO

@@ -0,0 +1,91 @@
+Metadata-Version: 2.3
+Name: pytest-regtest
+Version: 2.3.2
+Summary: pytest plugin for snapshot regression testing
+Project-URL: Source, https://gitlab.com/uweschmitt/pytest-regtest
+Project-URL: Documentation, https://pytest-regtest.readthedocs.org
+Author-email: Uwe Schmitt <uwe.schmitt@id.ethz.ch>
+License: MIT License
+License-File: LICENSE.txt
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Dist: pytest>7.2
+Description-Content-Type: text/markdown
+
+![](https://gitlab.com/uweschmitt/pytest-regtest/badges/main/pipeline.svg)
+![](https://gitlab.com/uweschmitt/pytest-regtest/badges/main/coverage.svg?job=coverage)
+
+
+The full documentation for this package are available at
+https://pytest-regtest.readthedocs.org
+
+# About
+
+## Introduction
+
+`pytest-regtest` is a plugin for [pytest](https://pytest.org) to implement
+**regression testing**.
+
+Unlike [functional testing](https://en.wikipedia.org/wiki/Functional_testing),
+[regression testing](https://en.wikipedia.org/wiki/Regression_testing)
+does not test whether the software produces the correct
+results, but whether it behaves as it did before changes were introduced.
+
+More specifically, `pytest-regtest` provides **snapshot testing**, which
+implements regression testing by recording data within a test function
+and comparing this recorded output to a previously recorded reference
+output.
+
+
+## Installation
+
+To install and activate this plugin execute:
+
+    $ pip install pytest-regtest
+
+
+!!! note
+
+     `pytest-regtest` provides some functionality specific to `NumPy`,
+     `pandas`, and `polars`. These dependencies are not installed when
+     you install `pytest-regtest`. For example, if you are using NumPy
+     snapshots, we assume that your production code (the code under
+     test) uses NumPy and therefore should be part of your project's
+     setup.
+
+
+## Use case 1: Changing code with no or little testing setup yet
+If you're working with code that has little or no unit testing, you
+can use regression testing to ensure that your changes don't break or
+alter previous results.
+
+**Example**:
+This can be useful when working with data analysis scripts, which often
+start as one long script and then are restructured into different
+functions as they evolve.
+
+
+## Use case 2:  Testing complex data
+If a unit tests contains many `assert` statements to check a complex
+data structure you can use regression tests instead.
+
+**Example**: To test code which ingests data into a database one can
+use regression tests on textual database dumps.
+
+## Use case 3: Testing NumPy arrays or pandas data frames
+
+If your code generates numerical results, such as `NumPy` arrays,
+`pandas` or `polars` data frames, you can use `pytest-regtest` to simply record such
+results and test them later, taking into account relative and absolute
+tolerances.
+
+
+**Example**:
+A function creates a 10 x 10 matrix. Either you have to write 100
+assert statements or you use summary statistics to test your result.
+In both cases, you may get little debugging information if a test
+fails.

pytest_regtest-2.3.2/README.md

@@ -0,0 +1,73 @@
+![](https://gitlab.com/uweschmitt/pytest-regtest/badges/main/pipeline.svg)
+![](https://gitlab.com/uweschmitt/pytest-regtest/badges/main/coverage.svg?job=coverage)
+
+
+The full documentation for this package are available at
+https://pytest-regtest.readthedocs.org
+
+# About
+
+## Introduction
+
+`pytest-regtest` is a plugin for [pytest](https://pytest.org) to implement
+**regression testing**.
+
+Unlike [functional testing](https://en.wikipedia.org/wiki/Functional_testing),
+[regression testing](https://en.wikipedia.org/wiki/Regression_testing)
+does not test whether the software produces the correct
+results, but whether it behaves as it did before changes were introduced.
+
+More specifically, `pytest-regtest` provides **snapshot testing**, which
+implements regression testing by recording data within a test function
+and comparing this recorded output to a previously recorded reference
+output.
+
+
+## Installation
+
+To install and activate this plugin execute:
+
+    $ pip install pytest-regtest
+
+
+!!! note
+
+     `pytest-regtest` provides some functionality specific to `NumPy`,
+     `pandas`, and `polars`. These dependencies are not installed when
+     you install `pytest-regtest`. For example, if you are using NumPy
+     snapshots, we assume that your production code (the code under
+     test) uses NumPy and therefore should be part of your project's
+     setup.
+
+
+## Use case 1: Changing code with no or little testing setup yet
+If you're working with code that has little or no unit testing, you
+can use regression testing to ensure that your changes don't break or
+alter previous results.
+
+**Example**:
+This can be useful when working with data analysis scripts, which often
+start as one long script and then are restructured into different
+functions as they evolve.
+
+
+## Use case 2:  Testing complex data
+If a unit tests contains many `assert` statements to check a complex
+data structure you can use regression tests instead.
+
+**Example**: To test code which ingests data into a database one can
+use regression tests on textual database dumps.
+
+## Use case 3: Testing NumPy arrays or pandas data frames
+
+If your code generates numerical results, such as `NumPy` arrays,
+`pandas` or `polars` data frames, you can use `pytest-regtest` to simply record such
+results and test them later, taking into account relative and absolute
+tolerances.
+
+
+**Example**:
+A function creates a 10 x 10 matrix. Either you have to write 100
+assert statements or you use summary statistics to test your result.
+In both cases, you may get little debugging information if a test
+fails.

pytest_regtest-2.3.2/pyproject.toml

@@ -0,0 +1,58 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+
+[project]
+name = "pytest-regtest"
+version = "2.3.2"
+description = "pytest plugin for snapshot regression testing"
+readme = "README.md"
+authors = [
+  {name = "Uwe Schmitt", email = "uwe.schmitt@id.ethz.ch"}
+]
+
+license = {text = "MIT License"}
+
+classifiers = [
+    "Intended Audience :: Developers",
+    "Programming Language :: Python :: 3.9",
+    "Programming Language :: Python :: 3.10",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "License :: OSI Approved :: MIT License"
+]
+
+dependencies = [
+  "pytest>7.2",
+]
+
+[project.urls]
+Source = "https://gitlab.com/uweschmitt/pytest-regtest"
+Documentation = "https://pytest-regtest.readthedocs.org"
+
+
+[project.entry-points.pytest11]
+regtest = "pytest_regtest"
+
+[tool.hatch.build.targets.sdist]
+only-include = ["pytest_regtest", "tests/conftest.py", "tests/test_plugin.py"]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/pytest_regtest"]
+
+[tool.ruff]
+line-length = 88
+exclude = ["_regtest_output"]
+
+[tool.ruff.lint]
+ignore = ["E731", "E203"]
+
+[tool.uv]
+dev-dependencies = [
+    "twine", "build", "hatchling", "wheel", "pre-commit", "ruff", "black",
+    "pytest-cov", "numpy", "pandas", "mkdocs", "mkdocs-material", "mistletoe",
+    "mkdocs-awesome-pages-plugin", "jinja2-cli", "mkdocstrings[python]",
+    "numpy>=2", "pandas>=2", "polars>=1.9",
+    "md-transformer>=0.0.3"
+]

pytest_regtest-2.3.2/tests/conftest.py

@@ -0,0 +1,28 @@
+import sys
+
+import numpy
+import pytest
+
+# hack to disable reload numpy in pytester (current code only prevent reload of
+# zope, so we trick pytester:
+# (see also _pytest/pytester.py  __take_sys_modules_snapshot method)
+sys.modules["zope"] = numpy
+
+pytest_plugins = ["pytester"]
+
+# for whatever reason pytester loaded the old bytcode in one test and caused it
+# to fail:
+sys.dont_write_bytecode = True
+
+
+@pytest.fixture
+def assert_outcomes():
+    def check(result, **kw):
+        try:
+            result.assert_outcomes(**kw)
+        except AssertionError:
+            outcome = result.parseoutcomes()
+            message = f"expected: {kw}\ngot: {outcome}\n\n" + result.stdout.str()
+            raise AssertionError(message)
+
+    yield check

pytest-regtest-2.1.0/MANIFEST.in

@@ -1 +0,0 @@
-include LICENSE.txt

pytest-regtest-2.1.0/PKG-INFO

@@ -1,350 +0,0 @@
-Metadata-Version: 2.1
-Name: pytest-regtest
-Version: 2.1.0
-Summary: "pytest plugin for snapshot regression testing"
-Author: Uwe Schmitt
-Author-email: uwe.schmitt@id.ethz.ch
-License: MIT
-Project-URL: Source, https://gitlab.com/uweschmitt/pytest-regtest
-Project-URL: Documentation, https://pytest-regtest.readthedocs.org
-Classifier: Intended Audience :: Developers
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.8
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-License-File: LICENSE.txt
-Requires-Dist: pytest>7.2
-Provides-Extra: dev
-Requires-Dist: twine; extra == "dev"
-Requires-Dist: build; extra == "dev"
-Requires-Dist: wheel; extra == "dev"
-Requires-Dist: pytest; extra == "dev"
-Requires-Dist: pre-commit; extra == "dev"
-
-
-# Home
-
-## About
-
-`pytest-regtest` is a plugin for [pytest](https://pytest.org) to implement
-**regression testing**.
-
-Unlike [functional testing](https://en.wikipedia.org/wiki/Functional_testing),
-[regression testing](https://en.wikipedia.org/wiki/Regression_testing)
-testing does not test whether the software produces the correct
-results, but whether it behaves as it did before changes were introduced.
-
-More specifically, `pytest-regtest` provides **snapshot testing**, which
-implements regression testing by recording the textual output of a test
-function and comparing this recorded output to a reference output.
-
-**Regression testing** is a common technique to implement basic testing
-before refactoring legacy code that lacks a test suite.
-
-Snapshot testing can also be used to implement tests for complex outcomes, such
-as recording textual database dumps or the results of a scientific analysis
-routine.
-
-
-## Installation
-
-To install and activate this plugin execute:
-
-    $ pip install pytest-regtest
-
-## Basic Usage
-
-
-### Write a test
-
-The `pytest-regtest` plugin provides multiple fixtures.
-To record output, use the fixture `regtest` that works like a file handle:
-
-```py
-def test_squares(regtest):
-
-    result = [i*i for i in range(10)]
-
-    # one way to record output:
-    print(result, file=regtest)
-
-    # alternative method to record output:
-    regtest.write("done")
-```
-
-You can also use the `regtest_all` fixture. This enables all output to stdout to be
-recorded in a test function.
-
-
-### Run the test
-
-If you run this test script with *pytest* the first time there is no
-recorded output for this test function so far and thus the test will
-fail with a message including a diff:
-
-```
-$ pytest -v test_squares.py
-============================= test session starts ==============================
-platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0 -- ...
-cachedir: .pytest_cache
-rootdir: ...
-plugins: regtest-2.0.3
-collecting ... collected 1 item
-
-test_squares.py::test_squares FAILED                                     [100%]
-
-=================================== FAILURES ===================================
-_________________________________ test_squares _________________________________
-
-regression test output not recorded yet for test_squares.py::test_squares:
-
-[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
-done
----------------------------- pytest-regtest report -----------------------------
-total number of failed regression tests: 1
-=========================== short test summary info ============================
-FAILED test_squares.py::test_squares
-============================== 1 failed in 0.02s ===============================
-```
-
-This is a diff of the current output `is` to a previously recorded output
-`tobe`. Since we did not record output yet, the diff contains no lines marked
-`+`.
-
-
-### Reset the test
-
-To record the current output, we run *pytest* with the *--reset-regtest*
-flag:
-
-```
-$ pytest -v --regtest-reset test_squares.py
-============================= test session starts ==============================
-platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0 -- ...
-cachedir: .pytest_cache
-rootdir: ...
-plugins: regtest-2.0.3
-collecting ... collected 1 item
-
-test_squares.py::test_squares RESET                                      [100%]
-
----------------------------- pytest-regtest report -----------------------------
-total number of failed regression tests: 0
-the following output files have been reset:
-  _regtest_outputs/test_squares.test_squares.out
-============================== 1 passed in 0.00s ===============================
-```
-
-You can also see from the output that the recorded output is in the
-`_regtest_outputs` folder which in the same folder as the test script.
-Don't forget to commit this folder to your version control system!
-
-### Run the test again
-
-When we run the test again, it succeeds:
-
-```
-$ pytest -v test_squares.py
-============================= test session starts ==============================
-platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0 -- ...
-cachedir: .pytest_cache
-rootdir: ...
-plugins: regtest-2.0.3
-collecting ... collected 1 item
-
-test_squares.py::test_squares PASSED                                     [100%]
-
----------------------------- pytest-regtest report -----------------------------
-total number of failed regression tests: 0
-============================== 1 passed in 0.00s ===============================
-```
-
-### Break the test
-
-Let us break the test by changing the test function to compute
-11 instead of 10 square numbers:
-
-```py
-def test_squares(regtest):
-
-    result = [i*i for i in range(11)]
-
-    # one way to record output:
-    print(result, file=regtest)
-
-    # alternative method to record output:
-    regtest.write("done")
-```
-
-The next run of pytest delivers a nice diff of the current and expected output
-from this test function:
-
-```
-$ pytest -v test_squares.py
-============================= test session starts ==============================
-platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0 -- ...
-cachedir: .pytest_cache
-rootdir: ...
-plugins: regtest-2.0.3
-collecting ... collected 1 item
-
-test_squares.py::test_squares FAILED                                     [100%]
-
-=================================== FAILURES ===================================
-_________________________________ test_squares _________________________________
-
-regression test output differences for test_squares.py::test_squares:
-(recorded output from _regtest_outputs/test_squares.test_squares.out)
-
->   --- current
->   +++ expected
->   @@ -1,2 +1,2 @@
->   -[0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
->   +[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
->    done
-
----------------------------- pytest-regtest report -----------------------------
-total number of failed regression tests: 1
-=========================== short test summary info ============================
-FAILED test_squares.py::test_squares
-============================== 1 failed in 0.02s ===============================
-```
-
-
-## Other features
-
-### Using the `regtest` fixture as context manager
-
-The `regtest` fixture also works as a context manager  to capture
-all output from the wrapped code block:
-
-```py
-def test_squares(regtest):
-
-    result = [i*i for i in range(10)]
-
-    with regtest:
-        print(result)
-```
-
-### The `regtest_all` fixture
-
-The `regtest_all` fixture leads to recording of all output to `stdout` in a
-test function.
-
-```py
-def test_all(regtest_all):
-    print("this line will be recorded.")
-    print("and this line also.")
-```
-
-### Reset individual tests
-
-You can reset recorded output of files and functions individually as:
-
-```sh
-$ py.test --regtest-reset test_demo.py
-$ py.test --regtest-reset test_demo.py::test_squares
-```
-
-### Suppress diff for failed tests
-
-To hide the diff and just show the number of lines changed, use:
-
-```sh
-$ py.test --regtest-nodiff ...
-```
-
-
-### Show all recorded output
-
-
-For complex diffs it helps to see the full recorded output also.
-To enable this use:
-
-```sh
-$ py.test --regtest-tee...
-```
-
-
-### Line endings
-
-Per default `pytest-regtest` ignores different line endings in the output.
-In case you want to disable this feature, use the `-regtest-consider-line-endings`
-flag.
-
-
-## Clean indeterministic output before recording
-
-Output can contain data which is changing from test run to test
-run, e.g. paths created with the `tmpdir` fixture, hexadecimal object ids or
-timestamps.
-
-Per default the plugin helps to make output more deterministic by:
-
-- replacing all temporary folder in the output with `<tmpdir...>` or similar markers,
-  depending on the origin of the temporary folder (`tempfile` module, `tmpdir` fixture,
-   ...)
-- replacing hexadecimal numbers ` 0x...` of arbitary length by the fixed string `0x?????????`.
-
-You can also implement your own cleanup routines as described below.
-
-### Register own cleanup functions
-
-You can register own converters in `conftest.py`:
-
-```py
-import re
-import pytest_regtest
-
-@pytest_regtest.register_converter_pre
-def remove_password_lines(txt):
-    '''modify recorded output BEFORE the default fixes
-    like temp folders or hex object ids are applied'''
-
-    # remove lines with passwords:
-    lines = txt.splitlines(keepends=True)
-    lines = [l for l in lines if "password is" not in l]
-    return "".join(lines)
-
-@pytest_regtest.register_converter_post
-def fix_time_measurements(txt):
-    '''modify recorded output AFTER the default fixes
-    like temp folders or hex object ids are applied'''
-
-    # fix time measurements:
-    return re.sub(
-        "\d+(\.\d+)? seconds",
-        "<SECONDS> seconds",
-        txt
-    )
-```
-
-If you register multiple converters they will be applied in the order of
-registration.
-
-In case your routines replace, improve or conflict with the standard cleanup converters,
-you can use the flag `--regtest-disable-stdconv` to disable the default cleanup
-procedure.
-
-## Command line options summary
-
-These are all supported command line options:
-
-```
-$ pytest --help
-...
-
-regression test plugin:
-  --regtest-reset       do not run regtest but record current output
-  --regtest-tee         print recorded results to console too
-  --regtest-consider-line-endings
-                        do not strip whitespaces at end of recorded lines
-  --regtest-nodiff      do not show diff output for failed regresson tests
-  --regtest-disable-stdconv
-                        do not apply standard output converters to clean up
-                        indeterministic output
-...
-```