pytest-regtest 2.0.1__tar.gz → 2.0.2__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: pytest-regtest
3
- Version: 2.0.1
3
+ Version: 2.0.2
4
4
  Summary: "pytest plugin for snapshot regression testing"
5
5
  Author: Uwe Schmitt
6
6
  Author-email: uwe.schmitt@id.ethz.ch
@@ -20,27 +20,30 @@ Provides-Extra: dev
20
20
  Requires-Dist: twine; extra == "dev"
21
21
  Requires-Dist: build; extra == "dev"
22
22
  Requires-Dist: wheel; extra == "dev"
23
+ Requires-Dist: pytest; extra == "dev"
23
24
 
24
- # README
25
+ # pytest-regtest
25
26
 
26
27
  ## About
27
28
 
28
29
  `pytest-regtest` is a plugin for [pytest](https://pytest.org) to implement
29
- [regression testing](https://en.wikipedia.org/wiki/Regression_testing).
30
+ **regression testing**.
30
31
 
31
32
  Unlike [functional testing](https://en.wikipedia.org/wiki/Functional_testing),
32
- regression testing does not test whether the software produces the correct
33
- results, but whether it behaves as it did before the changes were introduced.
33
+ [regression testing](https://en.wikipedia.org/wiki/Regression_testing)
34
+ testing does not test whether the software produces the correct
35
+ results, but whether it behaves as it did before changes were introduced.
34
36
 
35
- **Regression testing** is a common technique to get implement basic testing before
36
- refactoring legacy code that lacks a test suite.
37
+ More specifically, `pytest-regtest` provides **snapshot testing**, which
38
+ implements regression testing by recording the textual output of a test
39
+ function and comparing this recorded output to a reference output.
37
40
 
38
- More specifically, `pytest-regtest` provides **snapshot testing**, which implements
39
- regression testing by recording the textual output of a test function and
40
- comparing this recorded output to a reference output.
41
+ **Regression testing** is a common technique to implement basic testing
42
+ before refactoring legacy code that lacks a test suite.
41
43
 
42
- Snapshot testing can be used to implement tests for complex outcomes, such as
43
- recording textual database dumps or the results of a scientific analysis routine.
44
+ Snapshot testing can also be used to implement tests for complex outcomes, such
45
+ as recording textual database dumps or the results of a scientific analysis
46
+ routine.
44
47
 
45
48
 
46
49
  ## Installation
@@ -51,14 +54,13 @@ To install and activate this plugin execute:
51
54
 
52
55
  ## Basic Usage
53
56
 
54
- *pytest-regtest* plugin provides multiple fixtures.
55
57
 
56
58
  ### Write a test
57
59
 
60
+ *pytest-regtest* plugin provides multiple fixtures.
58
61
  To record output, use the fixture *regtest* that works like a file handle:
59
62
 
60
- ```python
61
-
63
+ ```py
62
64
  def test_squares_up_to_ten(regtest):
63
65
 
64
66
  result = [i*i for i in range(10)]
@@ -68,7 +70,6 @@ def test_squares_up_to_ten(regtest):
68
70
 
69
71
  # alternative method to record output:
70
72
  regtest.write("done")
71
-
72
73
  ```
73
74
 
74
75
  You can also use the `regtest_all` fixture. This enables all output to stdout to be
@@ -82,7 +83,7 @@ recorded output for this test function so far and thus the test will
82
83
  fail with a message including a diff:
83
84
 
84
85
 
85
- ```sh
86
+ ```bash
86
87
  $ pytest test_demo.py
87
88
  ========================= test session starts ==========================
88
89
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -118,7 +119,7 @@ This is a diff of the current output `is` to a previously recorded output
118
119
  To record the current output, we run *pytest* with the *--reset-regtest*
119
120
  flag:
120
121
 
121
- ```sh
122
+ ```bash
122
123
  $ py.test -v --regtest-reset test_demo.py
123
124
  ========================= test session starts ==========================
124
125
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -143,7 +144,7 @@ Don't forget to commit this folder to your version control system!
143
144
 
144
145
  When we run the test again, it succeeds:
145
146
 
146
- ```sh
147
+ ```bash
147
148
  $ py.test test_demo.py
148
149
  ========================= test session starts ==========================
149
150
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -160,10 +161,10 @@ total number of failed regression tests: 0
160
161
 
161
162
  ### Break the test
162
163
 
163
- Let us we break the test by changing the recoreded output to compute
164
+ Let us we break the test by changing the test function to compute
164
165
  11 instead of 10 square numbers:
165
166
 
166
- ```python
167
+ ```py
167
168
  def test_squares_up_to_ten(regtest):
168
169
 
169
170
  result = [i*i for i in range(11)] # changed!
@@ -178,7 +179,7 @@ def test_squares_up_to_ten(regtest):
178
179
  The next run of pytest delivers a nice diff of the current and expected output
179
180
  from this test function:
180
181
 
181
- ```sh
182
+ ```bash
182
183
  $ pytest test_demo.py
183
184
  ========================= test session starts ==========================
184
185
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -215,7 +216,7 @@ FAILED test_demo.py::test_squares_up_to_ten
215
216
  The `regtest` fixture also works as a context manager to capture
216
217
  all output from the wrapped code block:
217
218
 
218
- ```python
219
+ ```py
219
220
  def test_squares_up_to_ten(regtest):
220
221
 
221
222
  result = [i*i for i in range(10)]
@@ -229,7 +230,7 @@ def test_squares_up_to_ten(regtest):
229
230
  The `regtest_all` fixture leads to recording of all output to `stdout` in a
230
231
  test function.
231
232
 
232
- ```python
233
+ ```py
233
234
  def test_all(regtest_all):
234
235
  print("this line will be recorded.")
235
236
  print("and this line also.")
@@ -240,14 +241,18 @@ def test_all(regtest_all):
240
241
 
241
242
  You can reset recorded output of files and functions individually as:
242
243
 
243
- $ py.test --regtest-reset test_demo.py
244
- $ py.test --regtest-reset test_demo.py::test_squares_up_to_ten
244
+ ```bash
245
+ $ py.test --regtest-reset test_demo.py
246
+ $ py.test --regtest-reset test_demo.py::test_squares_up_to_ten
247
+ ```
245
248
 
246
249
  ### Suppress diff for failed tests
247
250
 
248
- To supress the diff and only see the stats use:
251
+ To hide the diff and just show the number of lines changed, use:
249
252
 
250
- $ py.test --regtest-nodiff ...
253
+ ```bash
254
+ $ py.test --regtest-nodiff ...
255
+ ```
251
256
 
252
257
 
253
258
  ### Show all recorded output
@@ -256,7 +261,9 @@ To supress the diff and only see the stats use:
256
261
  For complex diffs it helps to see the full recorded output also.
257
262
  To enable this use:
258
263
 
259
- $ py.test --regtest-tee...
264
+ ```bash
265
+ $ py.test --regtest-tee...
266
+ ```
260
267
 
261
268
 
262
269
  ### Line endings
@@ -284,31 +291,31 @@ Per default the plugin:
284
291
 
285
292
  You can register own converters in `conftest.py`:
286
293
 
287
- ```python
288
- import re
289
- import pytest_regtest
290
-
291
- @pytest_regtest.register_converter_pre
292
- def remove_password_lines(txt):
293
- """modify recorded output BEFORE the default fixes
294
- like temp folders or hex object ids are applied"""
295
-
296
- # remove lines with passwords:
297
- lines = txt.splitlines(keepends=True)
298
- lines = [l for l in lines if "password is" not in l]
299
- return "".join(lines)
300
-
301
- @pytest_regtest.register_converter_post
302
- def fix_time_measurements(txt):
303
- """modify recorded output AFTER the default fixes
304
- like temp folders or hex object ids are applied"""
305
-
306
- # fix time measurements:
307
- return re.sub(
308
- "\d+(\.\d+)? seconds",
309
- "<SECONDS> seconds",
310
- txt
311
- )
294
+ ```py
295
+ import re
296
+ import pytest_regtest
297
+
298
+ @pytest_regtest.register_converter_pre
299
+ def remove_password_lines(txt):
300
+ """modify recorded output BEFORE the default fixes
301
+ like temp folders or hex object ids are applied"""
302
+
303
+ # remove lines with passwords:
304
+ lines = txt.splitlines(keepends=True)
305
+ lines = [l for l in lines if "password is" not in l]
306
+ return "".join(lines)
307
+
308
+ @pytest_regtest.register_converter_post
309
+ def fix_time_measurements(txt):
310
+ """modify recorded output AFTER the default fixes
311
+ like temp folders or hex object ids are applied"""
312
+
313
+ # fix time measurements:
314
+ return re.sub(
315
+ "\d+(\.\d+)? seconds",
316
+ "<SECONDS> seconds",
317
+ txt
318
+ )
312
319
  ```
313
320
 
314
321
  If you register multiple converters they will be applied in the order of
@@ -1,23 +1,25 @@
1
- # README
1
+ # pytest-regtest
2
2
 
3
3
  ## About
4
4
 
5
5
  `pytest-regtest` is a plugin for [pytest](https://pytest.org) to implement
6
- [regression testing](https://en.wikipedia.org/wiki/Regression_testing).
6
+ **regression testing**.
7
7
 
8
8
  Unlike [functional testing](https://en.wikipedia.org/wiki/Functional_testing),
9
- regression testing does not test whether the software produces the correct
10
- results, but whether it behaves as it did before the changes were introduced.
9
+ [regression testing](https://en.wikipedia.org/wiki/Regression_testing)
10
+ testing does not test whether the software produces the correct
11
+ results, but whether it behaves as it did before changes were introduced.
11
12
 
12
- **Regression testing** is a common technique to get implement basic testing before
13
- refactoring legacy code that lacks a test suite.
13
+ More specifically, `pytest-regtest` provides **snapshot testing**, which
14
+ implements regression testing by recording the textual output of a test
15
+ function and comparing this recorded output to a reference output.
14
16
 
15
- More specifically, `pytest-regtest` provides **snapshot testing**, which implements
16
- regression testing by recording the textual output of a test function and
17
- comparing this recorded output to a reference output.
17
+ **Regression testing** is a common technique to implement basic testing
18
+ before refactoring legacy code that lacks a test suite.
18
19
 
19
- Snapshot testing can be used to implement tests for complex outcomes, such as
20
- recording textual database dumps or the results of a scientific analysis routine.
20
+ Snapshot testing can also be used to implement tests for complex outcomes, such
21
+ as recording textual database dumps or the results of a scientific analysis
22
+ routine.
21
23
 
22
24
 
23
25
  ## Installation
@@ -28,14 +30,13 @@ To install and activate this plugin execute:
28
30
 
29
31
  ## Basic Usage
30
32
 
31
- *pytest-regtest* plugin provides multiple fixtures.
32
33
 
33
34
  ### Write a test
34
35
 
36
+ *pytest-regtest* plugin provides multiple fixtures.
35
37
  To record output, use the fixture *regtest* that works like a file handle:
36
38
 
37
- ```python
38
-
39
+ ```py
39
40
  def test_squares_up_to_ten(regtest):
40
41
 
41
42
  result = [i*i for i in range(10)]
@@ -45,7 +46,6 @@ def test_squares_up_to_ten(regtest):
45
46
 
46
47
  # alternative method to record output:
47
48
  regtest.write("done")
48
-
49
49
  ```
50
50
 
51
51
  You can also use the `regtest_all` fixture. This enables all output to stdout to be
@@ -59,7 +59,7 @@ recorded output for this test function so far and thus the test will
59
59
  fail with a message including a diff:
60
60
 
61
61
 
62
- ```sh
62
+ ```bash
63
63
  $ pytest test_demo.py
64
64
  ========================= test session starts ==========================
65
65
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -95,7 +95,7 @@ This is a diff of the current output `is` to a previously recorded output
95
95
  To record the current output, we run *pytest* with the *--reset-regtest*
96
96
  flag:
97
97
 
98
- ```sh
98
+ ```bash
99
99
  $ py.test -v --regtest-reset test_demo.py
100
100
  ========================= test session starts ==========================
101
101
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -120,7 +120,7 @@ Don't forget to commit this folder to your version control system!
120
120
 
121
121
  When we run the test again, it succeeds:
122
122
 
123
- ```sh
123
+ ```bash
124
124
  $ py.test test_demo.py
125
125
  ========================= test session starts ==========================
126
126
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -137,10 +137,10 @@ total number of failed regression tests: 0
137
137
 
138
138
  ### Break the test
139
139
 
140
- Let us we break the test by changing the recoreded output to compute
140
+ Let us we break the test by changing the test function to compute
141
141
  11 instead of 10 square numbers:
142
142
 
143
- ```python
143
+ ```py
144
144
  def test_squares_up_to_ten(regtest):
145
145
 
146
146
  result = [i*i for i in range(11)] # changed!
@@ -155,7 +155,7 @@ def test_squares_up_to_ten(regtest):
155
155
  The next run of pytest delivers a nice diff of the current and expected output
156
156
  from this test function:
157
157
 
158
- ```sh
158
+ ```bash
159
159
  $ pytest test_demo.py
160
160
  ========================= test session starts ==========================
161
161
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -192,7 +192,7 @@ FAILED test_demo.py::test_squares_up_to_ten
192
192
  The `regtest` fixture also works as a context manager to capture
193
193
  all output from the wrapped code block:
194
194
 
195
- ```python
195
+ ```py
196
196
  def test_squares_up_to_ten(regtest):
197
197
 
198
198
  result = [i*i for i in range(10)]
@@ -206,7 +206,7 @@ def test_squares_up_to_ten(regtest):
206
206
  The `regtest_all` fixture leads to recording of all output to `stdout` in a
207
207
  test function.
208
208
 
209
- ```python
209
+ ```py
210
210
  def test_all(regtest_all):
211
211
  print("this line will be recorded.")
212
212
  print("and this line also.")
@@ -217,14 +217,18 @@ def test_all(regtest_all):
217
217
 
218
218
  You can reset recorded output of files and functions individually as:
219
219
 
220
- $ py.test --regtest-reset test_demo.py
221
- $ py.test --regtest-reset test_demo.py::test_squares_up_to_ten
220
+ ```bash
221
+ $ py.test --regtest-reset test_demo.py
222
+ $ py.test --regtest-reset test_demo.py::test_squares_up_to_ten
223
+ ```
222
224
 
223
225
  ### Suppress diff for failed tests
224
226
 
225
- To supress the diff and only see the stats use:
227
+ To hide the diff and just show the number of lines changed, use:
226
228
 
227
- $ py.test --regtest-nodiff ...
229
+ ```bash
230
+ $ py.test --regtest-nodiff ...
231
+ ```
228
232
 
229
233
 
230
234
  ### Show all recorded output
@@ -233,7 +237,9 @@ To supress the diff and only see the stats use:
233
237
  For complex diffs it helps to see the full recorded output also.
234
238
  To enable this use:
235
239
 
236
- $ py.test --regtest-tee...
240
+ ```bash
241
+ $ py.test --regtest-tee...
242
+ ```
237
243
 
238
244
 
239
245
  ### Line endings
@@ -261,31 +267,31 @@ Per default the plugin:
261
267
 
262
268
  You can register own converters in `conftest.py`:
263
269
 
264
- ```python
265
- import re
266
- import pytest_regtest
267
-
268
- @pytest_regtest.register_converter_pre
269
- def remove_password_lines(txt):
270
- """modify recorded output BEFORE the default fixes
271
- like temp folders or hex object ids are applied"""
272
-
273
- # remove lines with passwords:
274
- lines = txt.splitlines(keepends=True)
275
- lines = [l for l in lines if "password is" not in l]
276
- return "".join(lines)
277
-
278
- @pytest_regtest.register_converter_post
279
- def fix_time_measurements(txt):
280
- """modify recorded output AFTER the default fixes
281
- like temp folders or hex object ids are applied"""
282
-
283
- # fix time measurements:
284
- return re.sub(
285
- "\d+(\.\d+)? seconds",
286
- "<SECONDS> seconds",
287
- txt
288
- )
270
+ ```py
271
+ import re
272
+ import pytest_regtest
273
+
274
+ @pytest_regtest.register_converter_pre
275
+ def remove_password_lines(txt):
276
+ """modify recorded output BEFORE the default fixes
277
+ like temp folders or hex object ids are applied"""
278
+
279
+ # remove lines with passwords:
280
+ lines = txt.splitlines(keepends=True)
281
+ lines = [l for l in lines if "password is" not in l]
282
+ return "".join(lines)
283
+
284
+ @pytest_regtest.register_converter_post
285
+ def fix_time_measurements(txt):
286
+ """modify recorded output AFTER the default fixes
287
+ like temp folders or hex object ids are applied"""
288
+
289
+ # fix time measurements:
290
+ return re.sub(
291
+ "\d+(\.\d+)? seconds",
292
+ "<SECONDS> seconds",
293
+ txt
294
+ )
289
295
  ```
290
296
 
291
297
  If you register multiple converters they will be applied in the order of
@@ -0,0 +1,19 @@
1
+ [build-system]
2
+ requires = ["setuptools", "wheel"]
3
+
4
+
5
+ [tool.tox]
6
+ legacy_tox_ini = '''
7
+ [tox]
8
+ isolated_build = true
9
+ envlist = py39,py310,py311,py312
10
+ package = sdist
11
+
12
+ [testenv]
13
+ skip_install = false
14
+ passenv = *
15
+ allowlist_externals = pytest
16
+ extras = dev
17
+ commands =
18
+ pytest -ra -v tests
19
+ '''
@@ -0,0 +1,2 @@
1
+ from importlib.metadata import version as _version
2
+ __version__ = _version(__package__)
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.1
2
2
  Name: pytest-regtest
3
- Version: 2.0.1
3
+ Version: 2.0.2
4
4
  Summary: "pytest plugin for snapshot regression testing"
5
5
  Author: Uwe Schmitt
6
6
  Author-email: uwe.schmitt@id.ethz.ch
@@ -20,27 +20,30 @@ Provides-Extra: dev
20
20
  Requires-Dist: twine; extra == "dev"
21
21
  Requires-Dist: build; extra == "dev"
22
22
  Requires-Dist: wheel; extra == "dev"
23
+ Requires-Dist: pytest; extra == "dev"
23
24
 
24
- # README
25
+ # pytest-regtest
25
26
 
26
27
  ## About
27
28
 
28
29
  `pytest-regtest` is a plugin for [pytest](https://pytest.org) to implement
29
- [regression testing](https://en.wikipedia.org/wiki/Regression_testing).
30
+ **regression testing**.
30
31
 
31
32
  Unlike [functional testing](https://en.wikipedia.org/wiki/Functional_testing),
32
- regression testing does not test whether the software produces the correct
33
- results, but whether it behaves as it did before the changes were introduced.
33
+ [regression testing](https://en.wikipedia.org/wiki/Regression_testing)
34
+ testing does not test whether the software produces the correct
35
+ results, but whether it behaves as it did before changes were introduced.
34
36
 
35
- **Regression testing** is a common technique to get implement basic testing before
36
- refactoring legacy code that lacks a test suite.
37
+ More specifically, `pytest-regtest` provides **snapshot testing**, which
38
+ implements regression testing by recording the textual output of a test
39
+ function and comparing this recorded output to a reference output.
37
40
 
38
- More specifically, `pytest-regtest` provides **snapshot testing**, which implements
39
- regression testing by recording the textual output of a test function and
40
- comparing this recorded output to a reference output.
41
+ **Regression testing** is a common technique to implement basic testing
42
+ before refactoring legacy code that lacks a test suite.
41
43
 
42
- Snapshot testing can be used to implement tests for complex outcomes, such as
43
- recording textual database dumps or the results of a scientific analysis routine.
44
+ Snapshot testing can also be used to implement tests for complex outcomes, such
45
+ as recording textual database dumps or the results of a scientific analysis
46
+ routine.
44
47
 
45
48
 
46
49
  ## Installation
@@ -51,14 +54,13 @@ To install and activate this plugin execute:
51
54
 
52
55
  ## Basic Usage
53
56
 
54
- *pytest-regtest* plugin provides multiple fixtures.
55
57
 
56
58
  ### Write a test
57
59
 
60
+ *pytest-regtest* plugin provides multiple fixtures.
58
61
  To record output, use the fixture *regtest* that works like a file handle:
59
62
 
60
- ```python
61
-
63
+ ```py
62
64
  def test_squares_up_to_ten(regtest):
63
65
 
64
66
  result = [i*i for i in range(10)]
@@ -68,7 +70,6 @@ def test_squares_up_to_ten(regtest):
68
70
 
69
71
  # alternative method to record output:
70
72
  regtest.write("done")
71
-
72
73
  ```
73
74
 
74
75
  You can also use the `regtest_all` fixture. This enables all output to stdout to be
@@ -82,7 +83,7 @@ recorded output for this test function so far and thus the test will
82
83
  fail with a message including a diff:
83
84
 
84
85
 
85
- ```sh
86
+ ```bash
86
87
  $ pytest test_demo.py
87
88
  ========================= test session starts ==========================
88
89
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -118,7 +119,7 @@ This is a diff of the current output `is` to a previously recorded output
118
119
  To record the current output, we run *pytest* with the *--reset-regtest*
119
120
  flag:
120
121
 
121
- ```sh
122
+ ```bash
122
123
  $ py.test -v --regtest-reset test_demo.py
123
124
  ========================= test session starts ==========================
124
125
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -143,7 +144,7 @@ Don't forget to commit this folder to your version control system!
143
144
 
144
145
  When we run the test again, it succeeds:
145
146
 
146
- ```sh
147
+ ```bash
147
148
  $ py.test test_demo.py
148
149
  ========================= test session starts ==========================
149
150
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -160,10 +161,10 @@ total number of failed regression tests: 0
160
161
 
161
162
  ### Break the test
162
163
 
163
- Let us we break the test by changing the recoreded output to compute
164
+ Let us we break the test by changing the test function to compute
164
165
  11 instead of 10 square numbers:
165
166
 
166
- ```python
167
+ ```py
167
168
  def test_squares_up_to_ten(regtest):
168
169
 
169
170
  result = [i*i for i in range(11)] # changed!
@@ -178,7 +179,7 @@ def test_squares_up_to_ten(regtest):
178
179
  The next run of pytest delivers a nice diff of the current and expected output
179
180
  from this test function:
180
181
 
181
- ```sh
182
+ ```bash
182
183
  $ pytest test_demo.py
183
184
  ========================= test session starts ==========================
184
185
  platform darwin -- Python 3.11.4, pytest-7.4.3, pluggy-1.3.0
@@ -215,7 +216,7 @@ FAILED test_demo.py::test_squares_up_to_ten
215
216
  The `regtest` fixture also works as a context manager to capture
216
217
  all output from the wrapped code block:
217
218
 
218
- ```python
219
+ ```py
219
220
  def test_squares_up_to_ten(regtest):
220
221
 
221
222
  result = [i*i for i in range(10)]
@@ -229,7 +230,7 @@ def test_squares_up_to_ten(regtest):
229
230
  The `regtest_all` fixture leads to recording of all output to `stdout` in a
230
231
  test function.
231
232
 
232
- ```python
233
+ ```py
233
234
  def test_all(regtest_all):
234
235
  print("this line will be recorded.")
235
236
  print("and this line also.")
@@ -240,14 +241,18 @@ def test_all(regtest_all):
240
241
 
241
242
  You can reset recorded output of files and functions individually as:
242
243
 
243
- $ py.test --regtest-reset test_demo.py
244
- $ py.test --regtest-reset test_demo.py::test_squares_up_to_ten
244
+ ```bash
245
+ $ py.test --regtest-reset test_demo.py
246
+ $ py.test --regtest-reset test_demo.py::test_squares_up_to_ten
247
+ ```
245
248
 
246
249
  ### Suppress diff for failed tests
247
250
 
248
- To supress the diff and only see the stats use:
251
+ To hide the diff and just show the number of lines changed, use:
249
252
 
250
- $ py.test --regtest-nodiff ...
253
+ ```bash
254
+ $ py.test --regtest-nodiff ...
255
+ ```
251
256
 
252
257
 
253
258
  ### Show all recorded output
@@ -256,7 +261,9 @@ To supress the diff and only see the stats use:
256
261
  For complex diffs it helps to see the full recorded output also.
257
262
  To enable this use:
258
263
 
259
- $ py.test --regtest-tee...
264
+ ```bash
265
+ $ py.test --regtest-tee...
266
+ ```
260
267
 
261
268
 
262
269
  ### Line endings
@@ -284,31 +291,31 @@ Per default the plugin:
284
291
 
285
292
  You can register own converters in `conftest.py`:
286
293
 
287
- ```python
288
- import re
289
- import pytest_regtest
290
-
291
- @pytest_regtest.register_converter_pre
292
- def remove_password_lines(txt):
293
- """modify recorded output BEFORE the default fixes
294
- like temp folders or hex object ids are applied"""
295
-
296
- # remove lines with passwords:
297
- lines = txt.splitlines(keepends=True)
298
- lines = [l for l in lines if "password is" not in l]
299
- return "".join(lines)
300
-
301
- @pytest_regtest.register_converter_post
302
- def fix_time_measurements(txt):
303
- """modify recorded output AFTER the default fixes
304
- like temp folders or hex object ids are applied"""
305
-
306
- # fix time measurements:
307
- return re.sub(
308
- "\d+(\.\d+)? seconds",
309
- "<SECONDS> seconds",
310
- txt
311
- )
294
+ ```py
295
+ import re
296
+ import pytest_regtest
297
+
298
+ @pytest_regtest.register_converter_pre
299
+ def remove_password_lines(txt):
300
+ """modify recorded output BEFORE the default fixes
301
+ like temp folders or hex object ids are applied"""
302
+
303
+ # remove lines with passwords:
304
+ lines = txt.splitlines(keepends=True)
305
+ lines = [l for l in lines if "password is" not in l]
306
+ return "".join(lines)
307
+
308
+ @pytest_regtest.register_converter_post
309
+ def fix_time_measurements(txt):
310
+ """modify recorded output AFTER the default fixes
311
+ like temp folders or hex object ids are applied"""
312
+
313
+ # fix time measurements:
314
+ return re.sub(
315
+ "\d+(\.\d+)? seconds",
316
+ "<SECONDS> seconds",
317
+ txt
318
+ )
312
319
  ```
313
320
 
314
321
  If you register multiple converters they will be applied in the order of
@@ -3,6 +3,8 @@ MANIFEST.in
3
3
  README.md
4
4
  pyproject.toml
5
5
  setup.cfg
6
+ pytest_regtest/__init__.py
7
+ pytest_regtest/pytest_regtest.py
6
8
  pytest_regtest/pytest_regtest.egg-info/PKG-INFO
7
9
  pytest_regtest/pytest_regtest.egg-info/SOURCES.txt
8
10
  pytest_regtest/pytest_regtest.egg-info/dependency_links.txt
@@ -0,0 +1,2 @@
1
+ __init__
2
+ pytest_regtest
@@ -0,0 +1,362 @@
1
+ import difflib
2
+ import os
3
+ import re
4
+ import sys
5
+ import tempfile
6
+ from hashlib import sha512
7
+ from io import StringIO
8
+
9
+ import pytest
10
+ from _pytest._code.code import TerminalRepr
11
+ from _pytest._io import TerminalWriter
12
+
13
+ pytest_plugins = ["pytester"]
14
+
15
+ IS_WIN = sys.platform == "win32"
16
+
17
+
18
+ def ljust(s, *a):
19
+ return s.ljust(*a)
20
+
21
+
22
+ class RegtestException(Exception):
23
+ pass
24
+
25
+
26
+ def pytest_addoption(parser):
27
+ """Add options to control the timeout plugin"""
28
+ group = parser.getgroup("regtest", "regression test plugin")
29
+ group.addoption(
30
+ "--regtest-reset",
31
+ action="store_true",
32
+ help="do not run regtest but record current output",
33
+ )
34
+ group.addoption(
35
+ "--regtest-tee",
36
+ action="store_true",
37
+ default=False,
38
+ help="print recorded results to console too",
39
+ )
40
+ group.addoption(
41
+ "--regtest-consider-line-endings",
42
+ action="store_true",
43
+ default=False,
44
+ help="do not strip whitespaces at end of recorded lines",
45
+ )
46
+ group.addoption(
47
+ "--regtest-nodiff",
48
+ action="store_true",
49
+ default=False,
50
+ help="do not show diff output for failed regresson tests",
51
+ )
52
+
53
+
54
+ def pytest_configure(config):
55
+ config.pluginmanager.register(PytestRegtestPlugin())
56
+
57
+
58
+ class PytestRegtestPlugin:
59
+ def __init__(self):
60
+ self._reset_regtest_outputs = []
61
+ self._failed_regtests = []
62
+
63
+ @pytest.hookimpl(trylast=True)
64
+ def pytest_runtest_call(self, item):
65
+ if hasattr(item, "regtest_stream"):
66
+ self.check_recorded_output(item)
67
+
68
+ def check_recorded_output(self, item):
69
+ test_folder = item.fspath.dirname
70
+ regtest_stream = item.regtest_stream
71
+ identifier = regtest_stream.identifier
72
+
73
+ recorded_output_path = result_file_path(test_folder, item.nodeid, identifier)
74
+
75
+ config = item.config
76
+
77
+ ignore_line_endings = not config.getvalue("--regtest-consider-line-endings")
78
+ reset = config.getvalue("--regtest-reset")
79
+
80
+ if reset:
81
+ os.makedirs(os.path.dirname(recorded_output_path), exist_ok=True)
82
+ with open(recorded_output_path, "w", encoding="utf-8") as fh:
83
+ fh.write("".join(regtest_stream.get_lines()))
84
+ self._reset_regtest_outputs.append(recorded_output_path)
85
+ return
86
+
87
+ if os.path.exists(recorded_output_path):
88
+ with open(recorded_output_path, "r", encoding="utf-8") as fh:
89
+ tobe = fh.readlines()
90
+ else:
91
+ tobe = []
92
+
93
+ current = regtest_stream.get_lines()
94
+ if ignore_line_endings:
95
+ current = [line.rstrip() for line in current]
96
+ tobe = [line.rstrip() for line in tobe]
97
+
98
+ if current != tobe:
99
+ self._failed_regtests.append(item)
100
+ raise RegtestException(current, tobe, regtest_stream)
101
+
102
+ @pytest.hookimpl(hookwrapper=True)
103
+ def pytest_pyfunc_call(self, pyfuncitem):
104
+ stdout = sys.stdout
105
+ if "regtest_all" in pyfuncitem.fixturenames and hasattr(
106
+ pyfuncitem, "regtest_stream"
107
+ ):
108
+ sys.stdout = pyfuncitem.regtest_stream
109
+ yield
110
+ sys.stdout = stdout
111
+
112
+ @pytest.hookimpl(hookwrapper=True)
113
+ def pytest_report_teststatus(self, report, config):
114
+ outcome = yield
115
+ if report.when == "call":
116
+ if config.getvalue("--regtest-reset"):
117
+ result = outcome.get_result()
118
+ if result[0] != "failed":
119
+ outcome.force_result((result[0], "R", "RESET"))
120
+
121
+ def pytest_terminal_summary(self, terminalreporter, exitstatus, config):
122
+ terminalreporter.ensure_newline()
123
+ terminalreporter.section("pytest-regtest report", sep="-", blue=True, bold=True)
124
+ terminalreporter.write("total number of failed regression tests: ", bold=True)
125
+ terminalreporter.line(str(len(self._failed_regtests)))
126
+ if config.getvalue("--regtest-reset"):
127
+ if config.option.verbose:
128
+ terminalreporter.line(
129
+ "the following output files have been reset:", bold=True
130
+ )
131
+ for path in self._reset_regtest_outputs:
132
+ rel_path = os.path.relpath(path)
133
+ terminalreporter.line(" " + rel_path)
134
+ else:
135
+ terminalreporter.write(
136
+ "total number of reset output files: ", bold=True
137
+ )
138
+ terminalreporter.line(str(len(self._reset_regtest_outputs)))
139
+
140
+ @pytest.hookimpl(hookwrapper=True)
141
+ def pytest_runtest_makereport(self, item, call):
142
+ outcome = yield
143
+ if call.when == "teardown" and hasattr(item, "regtest_stream"):
144
+ if item.config.getvalue("--regtest-tee"):
145
+ tw = TerminalWriter()
146
+ tw.line()
147
+ line = "recorded raw output to regtest fixture: "
148
+ line = ljust(line, tw.fullwidth, "-")
149
+ tw.line(line, green=True)
150
+ tw.write(item.regtest_stream.get_output() + "\n", cyan=True)
151
+ tw.line("-" * tw.fullwidth, green=True)
152
+ tw.line()
153
+ tw.flush()
154
+
155
+ test_folder = item.fspath.dirname
156
+ regtest_stream = item.regtest_stream
157
+ identifier = regtest_stream.identifier
158
+
159
+ recorded_output_path = result_file_path(
160
+ test_folder, item.nodeid, identifier
161
+ )
162
+ result = outcome.get_result()
163
+ result.longrepr = CollectErrorRepr(
164
+ [f"wrote recorded output to {recorded_output_path}"], [dict()]
165
+ )
166
+ return
167
+
168
+ if call.when != "call" or not hasattr(item, "regtest_stream"):
169
+ return
170
+
171
+ if call.excinfo is not None and call.excinfo.type is RegtestException:
172
+ current, recorded, regtest_stream = call.excinfo.value.args
173
+
174
+ nodeid = item.nodeid + (
175
+ ""
176
+ if regtest_stream.identifier is None
177
+ else "__" + regtest_stream.identifier
178
+ )
179
+
180
+ nodiff = item.config.getvalue("--regtest-nodiff")
181
+ ignore_line_endings = not item.config.getvalue(
182
+ "--regtest-consider-line-endings"
183
+ )
184
+
185
+ result = outcome.get_result()
186
+
187
+ if not ignore_line_endings:
188
+ # add quotes around lines in diff:
189
+
190
+ current = list(map(repr, current))
191
+ recorded = list(map(repr, recorded))
192
+
193
+ collected = list(
194
+ difflib.unified_diff(current, recorded, "is", "tobe", lineterm="")
195
+ )
196
+
197
+ msg = "\nregression test output differences for {}:\n".format(nodeid)
198
+ if nodiff:
199
+ msg_diff = f"{len(collected)} lines in diff"
200
+ else:
201
+ msg_diff = "> " + "\n> ".join(collected)
202
+
203
+ result.longrepr = CollectErrorRepr(
204
+ [msg, msg_diff + "\n"], [dict(), dict(red=True, bold=True)]
205
+ )
206
+
207
+
208
+ def result_file_path(test_folder, nodeid, identifier):
209
+ file_name, __, test_function = nodeid.partition("::")
210
+ file_name = os.path.basename(file_name)
211
+
212
+ for c in "/\\:*\"'?<>|":
213
+ test_function = test_function.replace(c, "-")
214
+
215
+ # If file name is too long, hash parameters.
216
+ if len(test_function) > 100:
217
+ test_function = (
218
+ test_function[:88]
219
+ + "__"
220
+ + sha512(test_function.encode("utf-8")).hexdigest()[:10]
221
+ )
222
+
223
+ test_function = test_function.replace(" ", "_")
224
+ stem, __ = os.path.splitext(file_name)
225
+ if identifier is not None:
226
+ output_file_name = stem + "." + test_function + "__" + identifier + ".out"
227
+ else:
228
+ output_file_name = stem + "." + test_function + ".out"
229
+
230
+ return os.path.join(test_folder, "_regtest_outputs", output_file_name)
231
+
232
+
233
+ @pytest.fixture
234
+ def regtest(request):
235
+ yield RegtestStream(request)
236
+
237
+
238
+ @pytest.fixture
239
+ def regtest_all(regtest):
240
+ yield regtest
241
+
242
+
243
+ class RegtestStream:
244
+ def __init__(self, request):
245
+ request.node.regtest_stream = self
246
+ self.request = request
247
+ self.buffer = StringIO()
248
+ self.identifier = None
249
+
250
+ def write(self, what):
251
+ self.buffer.write(what)
252
+
253
+ def flush(self, true):
254
+ pass
255
+
256
+ def get_lines(self):
257
+ output = self.buffer.getvalue()
258
+ if not output:
259
+ return []
260
+ output = cleanup(output, self.request)
261
+ lines = output.splitlines(keepends=True)
262
+ return lines
263
+
264
+ def get_output(self):
265
+ return self.buffer.getvalue()
266
+
267
+ def __enter__(self):
268
+ sys.stdout = self
269
+ return self
270
+
271
+ def __exit__(self, *a):
272
+ sys.stdout = sys.__stdout__
273
+ return False # dont suppress exception
274
+
275
+
276
+ def cleanup(output, request):
277
+ for converter in _converters_pre:
278
+ output = converter(output, request)
279
+
280
+ output = _std_conversion(output, request)
281
+
282
+ for converter in _converters_post:
283
+ output = converter(output, request)
284
+
285
+ # in python 3 a string should not contain binary symbols...:
286
+ if contains_binary(output):
287
+ request.raiseerror(
288
+ "recorded output for regression test contains unprintable characters."
289
+ )
290
+
291
+ return output
292
+
293
+
294
+ # the function below is modified version of http://stackoverflow.com/questions/898669/
295
+ textchars = bytearray({7, 8, 9, 10, 12, 13, 27} | set(range(0x20, 0x100)) - {0x7F})
296
+
297
+
298
+ def contains_binary(txt):
299
+ return bool(txt.translate(dict(zip(textchars, " " * 9999))).replace(" ", ""))
300
+
301
+
302
+ _converters_pre = []
303
+ _converters_post = []
304
+
305
+
306
+ def register_converter_pre(function):
307
+ if function not in _converters_pre:
308
+ _converters_pre.append(function)
309
+
310
+
311
+ def register_converter_post(function):
312
+ if function not in _converters_post:
313
+ _converters_post.append(function)
314
+
315
+
316
+ def _std_replacements(request):
317
+ if "tmpdir" in request.fixturenames:
318
+ tmpdir = request.getfixturevalue("tmpdir").strpath + os.path.sep
319
+ yield tmpdir, "<tmpdir_from_fixture>/"
320
+ tmpdir = request.getfixturevalue("tmpdir").strpath
321
+ yield tmpdir, "<tmpdir_from_fixture>"
322
+
323
+ regexp = os.path.join(
324
+ os.path.realpath(tempfile.gettempdir()), "pytest-of-.*", r"pytest-\d+/"
325
+ )
326
+ yield regexp, "<pytest_tempdir>/"
327
+
328
+ regexp = os.path.join(tempfile.gettempdir(), "tmp[_a-zA-Z0-9]+")
329
+
330
+ yield regexp, "<tmpdir_from_tempfile_module>"
331
+ yield os.path.realpath(
332
+ tempfile.gettempdir()
333
+ ) + os.path.sep, "<tmpdir_from_tempfile_module>/"
334
+ yield os.path.realpath(tempfile.gettempdir()), "<tmpdir_from_tempfile_module>"
335
+ yield tempfile.tempdir + os.path.sep, "<tmpdir_from_tempfile_module>/"
336
+ yield tempfile.tempdir, "<tmpdir_from_tempfile_module>"
337
+ yield r"var/folders/.*/pytest-of.*/", "<pytest_tempdir>/"
338
+
339
+ # replace hex object ids in output by 0x?????????
340
+ yield r" 0x[0-9a-fA-F]+", " 0x?????????"
341
+
342
+
343
+ def _std_conversion(output, request):
344
+ fixed = []
345
+ for line in output.splitlines(keepends=True):
346
+ for regex, replacement in _std_replacements(request):
347
+ if IS_WIN:
348
+ # fix windows backwards slashes in regex
349
+ regex = regex.replace("\\", "\\\\")
350
+ line, __ = re.subn(regex, replacement, line)
351
+ fixed.append(line)
352
+ return "".join(fixed)
353
+
354
+
355
+ class CollectErrorRepr(TerminalRepr):
356
+ def __init__(self, messages, colors):
357
+ self.messages = messages
358
+ self.colors = colors
359
+
360
+ def toterminal(self, out):
361
+ for message, color in zip(self.messages, self.colors):
362
+ out.line(message, **color)
@@ -1,6 +1,6 @@
1
1
  [metadata]
2
2
  name = pytest-regtest
3
- version = 2.0.1
3
+ version = 2.0.2
4
4
  license = MIT
5
5
  description = "pytest plugin for snapshot regression testing"
6
6
  long_description = file: README.md
@@ -20,13 +20,11 @@ project_urls =
20
20
  [options]
21
21
  install_requires =
22
22
  pytest>7.2
23
- packages = find:
23
+ package_dir =
24
+ = pytest_regtest
24
25
  zip_safe = False
25
26
  python_requires = >=3.9
26
27
 
27
- [options.packages.find]
28
- where = pytest_regtest/
29
-
30
28
  [options.entry_points]
31
29
  pytest11 =
32
30
  regtest = pytest_regtest
@@ -36,6 +34,7 @@ dev =
36
34
  twine
37
35
  build
38
36
  wheel
37
+ pytest
39
38
 
40
39
  [egg_info]
41
40
  tag_build =
@@ -1,2 +0,0 @@
1
- [build-system]
2
- requires = ["setuptools", "wheel"]