pytest-regtest 2.2.0a2__py2.py3-none-any.whl → 2.3.0__py2.py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,356 +0,0 @@
1
- Metadata-Version: 2.3
2
- Name: pytest-regtest
3
- Version: 2.2.0a2
4
- Summary: pytest plugin for snapshot regression testing
5
- Project-URL: Source, https://gitlab.com/uweschmitt/pytest-regtest
6
- Project-URL: Documentation, https://pytest-regtest.readthedocs.org
7
- Author-email: Uwe Schmitt <uwe.schmitt@id.ethz.ch>
8
- License: MIT License
9
- License-File: LICENSE.txt
10
- Classifier: Intended Audience :: Developers
11
- Classifier: License :: OSI Approved :: MIT License
12
- Classifier: Programming Language :: Python :: 3.9
13
- Classifier: Programming Language :: Python :: 3.10
14
- Classifier: Programming Language :: Python :: 3.11
15
- Classifier: Programming Language :: Python :: 3.12
16
- Requires-Dist: pytest>7.2
17
- Provides-Extra: dev
18
- Requires-Dist: black; extra == 'dev'
19
- Requires-Dist: build; extra == 'dev'
20
- Requires-Dist: hatchling; extra == 'dev'
21
- Requires-Dist: mkdocs; extra == 'dev'
22
- Requires-Dist: mkdocs-material; extra == 'dev'
23
- Requires-Dist: numpy; extra == 'dev'
24
- Requires-Dist: pandas; extra == 'dev'
25
- Requires-Dist: pre-commit; extra == 'dev'
26
- Requires-Dist: pytest-cov; extra == 'dev'
27
- Requires-Dist: ruff; extra == 'dev'
28
- Requires-Dist: twine; extra == 'dev'
29
- Requires-Dist: wheel; extra == 'dev'
30
- Description-Content-Type: text/markdown
31
-
32
-
33
- # Home
34
-
35
- ## About
36
-
37
- `pytest-regtest` is a plugin for [pytest](https://pytest.org) to implement
38
- **regression testing**.
39
-
40
- Unlike [functional testing](https://en.wikipedia.org/wiki/Functional_testing),
41
- [regression testing](https://en.wikipedia.org/wiki/Regression_testing)
42
- testing does not test whether the software produces the correct
43
- results, but whether it behaves as it did before changes were introduced.
44
-
45
- More specifically, `pytest-regtest` provides **snapshot testing**, which
46
- implements regression testing by recording the textual output of a test
47
- function and comparing this recorded output to a reference output.
48
-
49
- **Regression testing** is a common technique to implement basic testing
50
- before refactoring legacy code that lacks a test suite.
51
-
52
- Snapshot testing can also be used to implement tests for complex outcomes, such
53
- as recording textual database dumps or the results of a scientific analysis
54
- routine.
55
-
56
-
57
- ## Installation
58
-
59
- To install and activate this plugin execute:
60
-
61
- $ pip install pytest-regtest
62
-
63
- ## Basic Usage
64
-
65
-
66
- ### Write a test
67
-
68
- The `pytest-regtest` plugin provides multiple fixtures.
69
- To record output, use the fixture `regtest` that works like a file handle:
70
-
71
- ```py
72
- def test_squares(regtest):
73
-
74
- result = [i*i for i in range(10)]
75
-
76
- # one way to record output:
77
- print(result, file=regtest)
78
-
79
- # alternative method to record output:
80
- regtest.write("done")
81
- ```
82
-
83
- You can also use the `regtest_all` fixture. This enables all output to stdout to be
84
- recorded in a test function.
85
-
86
-
87
- ### Run the test
88
-
89
- If you run this test script with *pytest* the first time there is no
90
- recorded output for this test function so far and thus the test will
91
- fail with a message including a diff:
92
-
93
- ```
94
- $ pytest -v test_squares.py
95
- ============================= test session starts ==============================
96
- platform darwin -- Python 3.11.4, pytest-8.0.0, pluggy-1.3.0 -- ...
97
- cachedir: .pytest_cache
98
- rootdir: ...
99
- plugins: cov-4.1.0, pytest_regtest-2.1.0
100
- collecting ... collected 1 item
101
-
102
- test_squares.py::test_squares FAILED [100%]
103
-
104
- =================================== FAILURES ===================================
105
- _________________________________ test_squares _________________________________
106
-
107
- regression test output not recorded yet for test_squares.py::test_squares:
108
-
109
- [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
110
- done
111
- ---------------------------- pytest-regtest report -----------------------------
112
- total number of failed regression tests: 1
113
- =========================== short test summary info ============================
114
- FAILED test_squares.py::test_squares
115
- ============================== 1 failed in 0.02s ===============================
116
- ```
117
-
118
- This is a diff of the current output `is` to a previously recorded output
119
- `tobe`. Since we did not record output yet, the diff contains no lines marked
120
- `+`.
121
-
122
-
123
- ### Reset the test
124
-
125
- To record the current output, we run *pytest* with the *--regtest-reset*
126
- flag:
127
-
128
- ```
129
- $ pytest -v --regtest-reset test_squares.py
130
- ============================= test session starts ==============================
131
- platform darwin -- Python 3.11.4, pytest-8.0.0, pluggy-1.3.0 -- ...
132
- cachedir: .pytest_cache
133
- rootdir: ...
134
- plugins: cov-4.1.0, pytest_regtest-2.1.0
135
- collecting ... collected 1 item
136
-
137
- test_squares.py::test_squares RESET [100%]
138
-
139
- ---------------------------- pytest-regtest report -----------------------------
140
- total number of failed regression tests: 0
141
- the following output files have been reset:
142
- _regtest_outputs/test_squares.test_squares.out
143
- ============================== 1 passed in 0.00s ===============================
144
- ```
145
-
146
- You can also see from the output that the recorded output is in the
147
- `_regtest_outputs` folder which in the same folder as the test script.
148
- Don't forget to commit this folder to your version control system!
149
-
150
- ### Run the test again
151
-
152
- When we run the test again, it succeeds:
153
-
154
- ```
155
- $ pytest -v test_squares.py
156
- ============================= test session starts ==============================
157
- platform darwin -- Python 3.11.4, pytest-8.0.0, pluggy-1.3.0 -- ...
158
- cachedir: .pytest_cache
159
- rootdir: ...
160
- plugins: cov-4.1.0, pytest_regtest-2.1.0
161
- collecting ... collected 1 item
162
-
163
- test_squares.py::test_squares PASSED [100%]
164
-
165
- ---------------------------- pytest-regtest report -----------------------------
166
- total number of failed regression tests: 0
167
- ============================== 1 passed in 0.00s ===============================
168
- ```
169
-
170
- ### Break the test
171
-
172
- Let us break the test by changing the test function to compute
173
- 11 instead of 10 square numbers:
174
-
175
- ```py
176
- def test_squares(regtest):
177
-
178
- result = [i*i for i in range(11)]
179
-
180
- # one way to record output:
181
- print(result, file=regtest)
182
-
183
- # alternative method to record output:
184
- regtest.write("done")
185
- ```
186
-
187
- The next run of pytest delivers a nice diff of the current and expected output
188
- from this test function:
189
-
190
- ```
191
- $ pytest -v test_squares.py
192
- ============================= test session starts ==============================
193
- platform darwin -- Python 3.11.4, pytest-8.0.0, pluggy-1.3.0 -- ...
194
- cachedir: .pytest_cache
195
- rootdir: ...
196
- plugins: cov-4.1.0, pytest_regtest-2.1.0
197
- collecting ... collected 1 item
198
-
199
- test_squares.py::test_squares FAILED [100%]
200
-
201
- =================================== FAILURES ===================================
202
- _________________________________ test_squares _________________________________
203
-
204
- regression test output differences for test_squares.py::test_squares:
205
- (recorded output from _regtest_outputs/test_squares.test_squares.out)
206
-
207
- > --- current
208
- > +++ expected
209
- > @@ -1,2 +1,2 @@
210
- > -[0, 1, 4, 9, 16, 25, 36, 49, 64, 81, 100]
211
- > +[0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
212
- > done
213
-
214
- ---------------------------- pytest-regtest report -----------------------------
215
- total number of failed regression tests: 1
216
- =========================== short test summary info ============================
217
- FAILED test_squares.py::test_squares
218
- ============================== 1 failed in 0.02s ===============================
219
- ```
220
-
221
-
222
- ## Other features
223
-
224
- ### Using the `regtest` fixture as context manager
225
-
226
- The `regtest` fixture also works as a context manager to capture
227
- all output from the wrapped code block:
228
-
229
- ```py
230
- def test_squares(regtest):
231
-
232
- result = [i*i for i in range(10)]
233
-
234
- with regtest:
235
- print(result)
236
- ```
237
-
238
- ### The `regtest_all` fixture
239
-
240
- The `regtest_all` fixture leads to recording of all output to `stdout` in a
241
- test function.
242
-
243
- ```py
244
- def test_all(regtest_all):
245
- print("this line will be recorded.")
246
- print("and this line also.")
247
- ```
248
-
249
- ### Reset individual tests
250
-
251
- You can reset recorded output of files and functions individually as:
252
-
253
- ```sh
254
- $ py.test --regtest-reset test_demo.py
255
- $ py.test --regtest-reset test_demo.py::test_squares
256
- ```
257
-
258
- ### Suppress diff for failed tests
259
-
260
- To hide the diff and just show the number of lines changed, use:
261
-
262
- ```sh
263
- $ py.test --regtest-nodiff ...
264
- ```
265
-
266
-
267
- ### Show all recorded output
268
-
269
-
270
- For complex diffs it helps to see the full recorded output also.
271
- To enable this use:
272
-
273
- ```sh
274
- $ py.test --regtest-tee...
275
- ```
276
-
277
-
278
- ### Line endings
279
-
280
- Per default `pytest-regtest` ignores different line endings in the output.
281
- In case you want to disable this feature, use the `-regtest-consider-line-endings`
282
- flag.
283
-
284
-
285
- ## Clean indeterministic output before recording
286
-
287
- Output can contain data which is changing from test run to test
288
- run, e.g. paths created with the `tmpdir` fixture, hexadecimal object ids or
289
- timestamps.
290
-
291
- Per default the plugin helps to make output more deterministic by:
292
-
293
- - replacing all temporary folder in the output with `<tmpdir...>` or similar markers,
294
- depending on the origin of the temporary folder (`tempfile` module, `tmpdir` fixture,
295
- ...)
296
- - replacing hexadecimal numbers ` 0x...` of arbitary length by the fixed string `0x?????????`.
297
-
298
- You can also implement your own cleanup routines as described below.
299
-
300
- ### Register own cleanup functions
301
-
302
- You can register own converters in `conftest.py`:
303
-
304
- ```py
305
- import re
306
- import pytest_regtest
307
-
308
- @pytest_regtest.register_converter_pre
309
- def remove_password_lines(txt):
310
- '''modify recorded output BEFORE the default fixes
311
- like temp folders or hex object ids are applied'''
312
-
313
- # remove lines with passwords:
314
- lines = txt.splitlines(keepends=True)
315
- lines = [l for l in lines if "password is" not in l]
316
- return "".join(lines)
317
-
318
- @pytest_regtest.register_converter_post
319
- def fix_time_measurements(txt):
320
- '''modify recorded output AFTER the default fixes
321
- like temp folders or hex object ids are applied'''
322
-
323
- # fix time measurements:
324
- return re.sub(
325
- "\d+(\.\d+)? seconds",
326
- "<SECONDS> seconds",
327
- txt
328
- )
329
- ```
330
-
331
- If you register multiple converters they will be applied in the order of
332
- registration.
333
-
334
- In case your routines replace, improve or conflict with the standard cleanup converters,
335
- you can use the flag `--regtest-disable-stdconv` to disable the default cleanup
336
- procedure.
337
-
338
- ## Command line options summary
339
-
340
- These are all supported command line options:
341
-
342
- ```
343
- $ pytest --help
344
- ...
345
-
346
- regression test plugin:
347
- --regtest-reset do not run regtest but record current output
348
- --regtest-tee print recorded results to console too
349
- --regtest-consider-line-endings
350
- do not strip whitespaces at end of recorded lines
351
- --regtest-nodiff do not show diff output for failed regresson tests
352
- --regtest-disable-stdconv
353
- do not apply standard output converters to clean up
354
- indeterministic output
355
- ...
356
- ```
@@ -1,12 +0,0 @@
1
- pytest_regtest/__init__.py,sha256=Y9D7u9DiyRbGMGW2X1wsl583tp_BAHTBUKJnKps8kMo,1833
2
- pytest_regtest/numpy_handler.py,sha256=75xwG9UiFVGOp6Px0JXzr7pKxiI6cyezX3nx76h39Jo,6327
3
- pytest_regtest/pandas_handler.py,sha256=wlCVYLQU_CJ0x6r9Nok3ToF4CXm8eydBkoil-Nu2rEk,4249
4
- pytest_regtest/pytest_regtest.py,sha256=LmKSMVpzV7E_gh3Sm3asUqNrpmlCOsUVZ_9GoQTOgag,19105
5
- pytest_regtest/register_third_party_handlers.py,sha256=OU0BqDBBxVPV5OidbWCd_PyI-7P3YSPhuxZhQz_6wkI,926
6
- pytest_regtest/snapshot_handler.py,sha256=8l95Es_BNVSplZv1Nc7SzXXeeh__4atJzLTz9AzNO9A,2085
7
- pytest_regtest/utils.py,sha256=2jYTlV_qL5hH6FCeg7T1HJJvKual-Kux2scJ9_aB1lY,811
8
- pytest_regtest-2.2.0a2.dist-info/METADATA,sha256=CHzX4gf330hgNKd7A3CCI0Sr7jS4RU5XnVmcFGNXW2M,10804
9
- pytest_regtest-2.2.0a2.dist-info/WHEEL,sha256=fl6v0VwpzfGBVsGtkAkhILUlJxROXbA3HvRL6Fe3140,105
10
- pytest_regtest-2.2.0a2.dist-info/entry_points.txt,sha256=4VuIhXeMGhDo0ATbaUfyjND0atofmZjV_P-o6_uEk2s,36
11
- pytest_regtest-2.2.0a2.dist-info/licenses/LICENSE.txt,sha256=Tue36uAzpW79-9WAqzkwPhsDDVd1X-VWUmdZ0MfGYvk,1068
12
- pytest_regtest-2.2.0a2.dist-info/RECORD,,