pytest-receptor 1.0.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,100 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project are recorded here. The format follows
4
+ [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and releases follow
5
+ [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
6
+
7
+ ## [Unreleased]
8
+
9
+ ## [1.0.0] - 2026-08-12
10
+
11
+ ### Added
12
+
13
+ - A normalized evidence model shared by terminal rendering and machine output.
14
+ - The opt-in `pytest-receptor.events@1` JSONL artifact and supported reader.
15
+ - Exact text-contract tests and differential pytest/JUnit parity tests.
16
+ - Tested coexistence with xdist, coverage, reruns, subtests, and JUnit.
17
+ - Auditable artifact size limits and reproducible wall-time/peak-RSS benchmarks.
18
+ - Stable root-cause fingerprints and recoverable, auditable terminal truncation.
19
+
20
+ ### Fixed
21
+
22
+ - Mixed valid and nonexistent paths under xdist now retain an actionable invalid
23
+ selection diagnostic while preserving pytest's exit status.
24
+
25
+ ### Changed
26
+
27
+ - CI, documentation, packaging metadata, and release gates now exercise the
28
+ declared Python and pytest support matrix.
29
+ - Rerun attempts and subtests retain distinct normalized identities without
30
+ inflating logical-test counts.
31
+
32
+ ## [0.7.0] - 2026-08-02
33
+
34
+ ### Changed
35
+
36
+ - Consolidated the post-0.6 development guide and proposal triage.
37
+
38
+ ## [0.6.0] - 2026-07-28
39
+
40
+ ### Changed
41
+
42
+ - Rebuilt compact reporting around pytest's public hooks and truth-preserving
43
+ exit semantics.
44
+ - Added owner-only full reports, credential-pattern redaction, deterministic
45
+ failure grouping, warning visibility, progress, and real-suite regressions.
46
+
47
+ ## [0.5.0] - 2026-07-17
48
+
49
+ ### Added
50
+
51
+ - Experimental integration and event-reader work, subsequently redesigned by
52
+ the 0.6 correctness audit.
53
+
54
+ ## [0.4.0] - 2026-07-17
55
+
56
+ ### Added
57
+
58
+ - Experimental CI watchdog behavior, subsequently removed in 0.6.
59
+
60
+ ## [0.3.0] - 2026-07-17
61
+
62
+ ### Added
63
+
64
+ - Initial event-model and semantic-budget experiments.
65
+
66
+ ## [0.2.0] - 2026-07-17
67
+
68
+ ### Added
69
+
70
+ - Initial correctness and outcome handling.
71
+
72
+ ## [0.1.2] - 2026-07-17
73
+
74
+ ### Added
75
+
76
+ - Conda build recipe.
77
+
78
+ ## [0.1.1] - 2026-07-17
79
+
80
+ ### Changed
81
+
82
+ - English documentation and explicit Python support metadata.
83
+
84
+ ## [0.1.0] - 2026-07-17
85
+
86
+ ### Added
87
+
88
+ - Human, LLM, and CI output profiles.
89
+
90
+ [Unreleased]: https://github.com/uibcdf/pytest-receptor/compare/1.0.0...HEAD
91
+ [1.0.0]: https://github.com/uibcdf/pytest-receptor/compare/0.7.0...1.0.0
92
+ [0.7.0]: https://github.com/uibcdf/pytest-receptor/compare/0.6.0...0.7.0
93
+ [0.6.0]: https://github.com/uibcdf/pytest-receptor/compare/0.5.0...0.6.0
94
+ [0.5.0]: https://github.com/uibcdf/pytest-receptor/compare/0.4.0...0.5.0
95
+ [0.4.0]: https://github.com/uibcdf/pytest-receptor/compare/0.3.0...0.4.0
96
+ [0.3.0]: https://github.com/uibcdf/pytest-receptor/compare/0.2.0...0.3.0
97
+ [0.2.0]: https://github.com/uibcdf/pytest-receptor/compare/0.1.2...0.2.0
98
+ [0.1.2]: https://github.com/uibcdf/pytest-receptor/compare/0.1.1...0.1.2
99
+ [0.1.1]: https://github.com/uibcdf/pytest-receptor/compare/0.1.0...0.1.1
100
+ [0.1.0]: https://github.com/uibcdf/pytest-receptor/releases/tag/0.1.0
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Unit of Research on Computational Biology and Drug Design. Mexico Children's Hospital Federico Gómez.
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1 @@
1
+ include CHANGELOG.md
@@ -0,0 +1,440 @@
1
+ Metadata-Version: 2.4
2
+ Name: pytest-receptor
3
+ Version: 1.0.0
4
+ Summary: A pytest plugin to adapt test output for different receptors (humans, LLMs, CI).
5
+ Author: UIBCDF Development Team
6
+ License-Expression: MIT
7
+ Project-URL: Homepage, https://github.com/uibcdf/pytest-receptor
8
+ Project-URL: Repository, https://github.com/uibcdf/pytest-receptor
9
+ Project-URL: Issues, https://github.com/uibcdf/pytest-receptor/issues
10
+ Project-URL: Changelog, https://github.com/uibcdf/pytest-receptor/blob/main/CHANGELOG.md
11
+ Keywords: pytest,plugin,testing,LLM,CI
12
+ Classifier: Development Status :: 5 - Production/Stable
13
+ Classifier: Framework :: Pytest
14
+ Classifier: Intended Audience :: Developers
15
+ Classifier: Operating System :: OS Independent
16
+ Classifier: Programming Language :: Python :: 3
17
+ Classifier: Programming Language :: Python :: 3.11
18
+ Classifier: Programming Language :: Python :: 3.12
19
+ Classifier: Programming Language :: Python :: 3.13
20
+ Classifier: Topic :: Software Development :: Testing
21
+ Requires-Python: <3.14,>=3.11
22
+ Description-Content-Type: text/markdown
23
+ License-File: LICENSE
24
+ Requires-Dist: pytest>=8.0.0
25
+ Provides-Extra: test
26
+ Requires-Dist: packaging>=24; extra == "test"
27
+ Requires-Dist: pytest-cov<8,>=6; extra == "test"
28
+ Requires-Dist: pytest-rerunfailures<17,>=15; extra == "test"
29
+ Requires-Dist: pytest-subtests<0.16,>=0.14; extra == "test"
30
+ Requires-Dist: pytest-xdist<4,>=3.6; extra == "test"
31
+ Provides-Extra: lint
32
+ Requires-Dist: ruff==0.16.1; extra == "lint"
33
+ Provides-Extra: docs
34
+ Requires-Dist: myst-parser<5,>=4; extra == "docs"
35
+ Requires-Dist: sphinx<10,>=8; extra == "docs"
36
+ Requires-Dist: sphinx-rtd-theme<4,>=3; extra == "docs"
37
+ Provides-Extra: benchmark
38
+ Requires-Dist: tiktoken<1,>=0.9; extra == "benchmark"
39
+ Provides-Extra: release
40
+ Requires-Dist: build<2,>=1.2; extra == "release"
41
+ Requires-Dist: packaging>=24; extra == "release"
42
+ Requires-Dist: twine<7,>=6; extra == "release"
43
+ Provides-Extra: dev
44
+ Requires-Dist: build<2,>=1.2; extra == "dev"
45
+ Requires-Dist: myst-parser<5,>=4; extra == "dev"
46
+ Requires-Dist: packaging>=24; extra == "dev"
47
+ Requires-Dist: pytest-cov<8,>=6; extra == "dev"
48
+ Requires-Dist: pytest-rerunfailures<17,>=15; extra == "dev"
49
+ Requires-Dist: pytest-subtests<0.16,>=0.14; extra == "dev"
50
+ Requires-Dist: pytest-xdist<4,>=3.6; extra == "dev"
51
+ Requires-Dist: ruff==0.16.1; extra == "dev"
52
+ Requires-Dist: sphinx<10,>=8; extra == "dev"
53
+ Requires-Dist: sphinx-rtd-theme<4,>=3; extra == "dev"
54
+ Requires-Dist: tiktoken<1,>=0.9; extra == "dev"
55
+ Requires-Dist: twine<7,>=6; extra == "dev"
56
+ Dynamic: license-file
57
+
58
+ # pytest-receptor
59
+
60
+ [![Tests](https://github.com/uibcdf/pytest-receptor/actions/workflows/tests.yml/badge.svg)](https://github.com/uibcdf/pytest-receptor/actions/workflows/tests.yml)
61
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
62
+ [![Python Version](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-blue)](https://www.python.org/)
63
+ [![Pytest Version](https://img.shields.io/badge/pytest-%3E%3D8.0.0-green)](https://docs.pytest.org/)
64
+
65
+ A pytest reporter for coding agents.
66
+
67
+ When pytest is driven by an agent such as Claude Code, Codex, or an autonomous
68
+ TDD loop, its output is read by something that pays for every token and cannot
69
+ scroll back. `pytest-receptor` renders the same run for that consumer: it says
70
+ what happened, groups repeated failures by root cause, and tells the agent
71
+ exactly what to re-run.
72
+
73
+ > **1.0 release candidate.** The reliability, CLI, outcome, and
74
+ > `pytest-receptor.events@1` compatibility contracts are frozen. The receptor
75
+ > never reports an unsuccessful or incomplete run as a success, and a failure
76
+ > inside the receptor itself never costs you the run.
77
+
78
+ ---
79
+
80
+ ## The problem
81
+
82
+ Your agent was told to run the tests, so it runs `pytest`, and pytest answers
83
+ the way it has always answered: for a human sitting at a terminal. Banner,
84
+ `rootdir`, plugin list, progress bar, and the full source of every failing test.
85
+ The agent pays for all of it, on every iteration.
86
+
87
+ Tuning the flags does not fix it. `pytest -q --tb=line` is compact but strips
88
+ the assertion diff, so the agent guesses and loops. And nothing pytest offers
89
+ handles the case that hurts most: one broken fixture fails forty tests, and the
90
+ agent reads the same traceback forty times.
91
+
92
+ ## What it does
93
+
94
+ ```console
95
+ $ pytest --receptor=llm
96
+
97
+ FAIL exit=1 | 38 errors, 90 passed | 12.40s | 1 root cause
98
+
99
+ [1] TypeError | 38 tests | setup
100
+ conftest.py:31
101
+ TypeError: 'NoneType' object is not subscriptable
102
+ tests:
103
+ tests/test_merge.py::test_merge[0]
104
+ tests/test_merge.py::test_merge[1]
105
+ tests/test_merge.py::test_merge[2]
106
+ +35 more
107
+ rerun: pytest tests/test_merge.py -q
108
+ ```
109
+
110
+ That is 109 tokens. Plain `pytest` spends 3,300 on the same run.
111
+
112
+ ---
113
+
114
+ ## Install
115
+
116
+ ```bash
117
+ conda install -c uibcdf pytest-receptor
118
+ ```
119
+
120
+ Version 1.0 is planned as the first PyPI release (`pip install
121
+ pytest-receptor`). Its official `pytest11` metadata makes pytest discover it
122
+ automatically after installation. Until then, use conda or install from source.
123
+ See the [installation guide](https://uibcdf.github.io/pytest-receptor/installation.html).
124
+
125
+ Requires Python 3.11-3.13 and pytest 8 or later. Every combination of
126
+ Python 3.11/3.12/3.13 with pytest 8 and 9 is exercised in CI, so the support
127
+ claim is evidence rather than intent.
128
+
129
+ ## Use
130
+
131
+ ```bash
132
+ pytest # unchanged pytest -- installing this changes nothing
133
+ pytest --receptor=llm # compact output for a coding agent
134
+ pytest --receptor=ci # compact output for a CI log
135
+ pytest --receptor=human # the default, stated explicitly
136
+ pytest --receptor=llm --receptor-full # expand every failure group
137
+ pytest --receptor=llm --receptor-stats # what did this actually save?
138
+ pytest --receptor=llm --receptor-events=events.jsonl # normalized evidence
139
+ pytest --receptor=llm --receptor-events=events.jsonl \
140
+ --receptor-events-max-bytes=104857600 # explicit artifact cap
141
+ ```
142
+
143
+ The opt-in JSONL stream is versioned, owner-only, integrity-checked on
144
+ finalization, and readable with `pytest_receptor.read_artifact`. A missing final
145
+ session record is reported as incomplete. See the
146
+ [artifact guide](https://uibcdf.github.io/pytest-receptor/artifacts.html).
147
+
148
+ **Installing the plugin does not change anything until you ask it to.** The
149
+ default is `human`, and `human` is a true passthrough: the plugin registers
150
+ nothing at all, so `pytest` on its own produces output byte-identical to not
151
+ having it installed. There is a test asserting exactly that.
152
+
153
+ This matters in a shared environment. You can install it for yourself, or for
154
+ your agent, without altering what anyone else sees.
155
+
156
+ **`llm` vs `ci` — one question: will the reader be able to open the on-disk
157
+ report?** `llm` assumes yes (the agent shares the filesystem), so on a
158
+ pathological spread of failures — past ~10 distinct root causes — it shows the
159
+ first ten in full and points at the report for the rest, which the agent opens
160
+ only if it needs to. `ci` assumes no: a CI runner is destroyed at job end and the
161
+ log gets one shot, so nothing is held back and no report path is printed — every
162
+ root cause is expanded inline. Same renderer and same truth-preserving rules;
163
+ they differ only in what survives for the reader, so neither is simply "more"
164
+ than the other. Details in the
165
+ [usage guide](https://uibcdf.github.io/pytest-receptor/usage.html#choosing-between-llm-and-ci).
166
+
167
+ You do not need to combine these with pytest's own quieting flags —
168
+ `--receptor=llm` already sets the equivalent of `-qq --no-header --no-summary`,
169
+ so adding them changes nothing. One caveat: do **not** pass `--tb=line` or
170
+ `--tb=no`. Those control how pytest *builds* the traceback, not how it prints it,
171
+ so they save a handful of tokens and silently cost the receptor the call chain.
172
+ See the [usage guide](https://uibcdf.github.io/pytest-receptor/usage.html).
173
+
174
+ ---
175
+
176
+ ## Behavior
177
+
178
+ ### It always tells you what actually happened
179
+
180
+ The numeric exit status comes from pytest, never from the absence of failure
181
+ reports. The label refines that status only when pytest merges distinct states:
182
+ collection errors and interrupts both use exit 2, and xdist can return exit 5
183
+ for an invocation that contains nonexistent filesystem targets.
184
+
185
+ ```text
186
+ PASS exit=0 | 126 passed, 2 skipped | 4.21s | 3 warnings
187
+ FAIL exit=1 | 2 failed, 87 passed | 12.40s | 2 root causes
188
+ NO_TESTS exit=5
189
+ INTERRUPTED exit=2 | incomplete: 12 of 128 executed
190
+ COLLECTION_ERROR exit=2
191
+ USAGE_ERROR exit=5 | invalid selection
192
+ ```
193
+
194
+ A run stopped by `-x`, `--maxfail`, or an interrupt is marked incomplete even
195
+ when nothing failed, so a partial run cannot be mistaken for a clean one.
196
+
197
+ ### It groups by root cause, not by test
198
+
199
+ Failures are grouped by exception type, phase, crash location, and cause chain.
200
+ Forty tests broken by one fixture become one group that keeps all forty test
201
+ IDs, and a parametrized test failing on twenty inputs is one bug with twenty
202
+ messages rather than twenty bugs. Failures crashing in unrelated places stay
203
+ separate, and so do two failures wrapping different underlying errors.
204
+
205
+ `raise X from Y` reports both, because the wrapper is usually the less
206
+ informative half:
207
+
208
+ ```text
209
+ ValueError: could not build topology
210
+ caused by: KeyError: 'atoms'
211
+ ```
212
+
213
+ ### It gives you everything on stdout
214
+
215
+ Every root cause is rendered in full. Grouping has already collapsed the volume,
216
+ so withholding on top of it saves almost nothing and costs double if you then
217
+ have to read the file back: measured at five distinct causes, holding back saved
218
+ forty tokens and cost two hundred. Only a pathological spread — more than ten
219
+ distinct causes — is summarized, and only when the on-disk report exists to hold
220
+ what was left out.
221
+
222
+ Occurrence lists are the exception, and for the opposite reason: a group of
223
+ thirty-eight failing tests names three and counts the rest, because the rerun
224
+ command already selects all of them.
225
+
226
+ ### It says why tests were skipped
227
+
228
+ A suite built on optional dependencies skips in the hundreds, and `412 skipped`
229
+ does not say which capability is missing:
230
+
231
+ ```text
232
+ skipped: 412 in 3 groups
233
+ x380 | openmm not installed
234
+ x30 | requires a GPU
235
+ x2 | (no reason declared)
236
+ ```
237
+
238
+ The last group is deliberate: a skip nobody documented is worth knowing about.
239
+ Warnings and xfails are grouped the same way. Each section is bounded by the
240
+ variety of reasons rather than the number of tests.
241
+
242
+ ### It degrades safely
243
+
244
+ If the receptor itself raises, you get `RECEPTOR_ERROR`, the underlying
245
+ exception, the raw pytest evidence, and pytest's original exit status. The worst
246
+ case of enabling this plugin is standard pytest plus one line of noise.
247
+
248
+ Test output is treated as untrusted input: ANSI escapes and control characters
249
+ are stripped, values that look like credentials are redacted before anything is
250
+ rendered or written, and no text produced by a test can forge a verdict line.
251
+ The redaction is a conservative net for obvious shapes — `api_key=`, `Bearer ...`
252
+ — not a security boundary.
253
+
254
+ ### It shows the run is alive
255
+
256
+ pytest streams a progress character per test; suppressing those leaves a long
257
+ suite completely silent. Progress now goes to **stderr** — never stdout, so the
258
+ report stays as parseable as before — once per twenty percent of the suite:
259
+
260
+ ```text
261
+ receptor: 20% 1867/9332 52s
262
+ receptor: 40% 3733/9332 108s
263
+ ```
264
+
265
+ Reporting by percentage rather than by clock bounds this at five lines whether
266
+ the run takes five minutes or three hours, and the elapsed time exposes pace: a
267
+ step that suddenly takes four times longer is visible before the run ends.
268
+
269
+ It is a liveness signal, not a hang detector: the line is emitted when a test
270
+ finishes, so a stuck test produces no further output. What survives is how far
271
+ the run got, which is what you want when something kills it.
272
+
273
+ ### It works under `pytest-xdist`
274
+
275
+ A distributed run produces **byte-identical output to a serial one**. Workers
276
+ finish in arbitrary order, so occurrences and groups are given a total order
277
+ before rendering; otherwise the same failure would render differently on every
278
+ run. Counts, grouping, and exit status are unaffected by `-n`.
279
+
280
+ Worker identity is not reported, and that is a decision rather than a gap. The
281
+ signal it would provide — a group of failures landing entirely on one worker —
282
+ is confounded by the distribution mode: under `--dist loadfile` or `loadscope`,
283
+ failures from one file land on one worker by construction. The bare identifier
284
+ without execution order also does not help reproduce anything, which is what
285
+ `-n0` is for.
286
+
287
+ ---
288
+
289
+ ## How it works
290
+
291
+ Worth knowing before you trust it with your suite.
292
+
293
+ **It does not replace pytest's reporter.** Earlier versions unregistered
294
+ pytest's `TerminalReporter` and substituted a subclass of it. This one leaves it
295
+ in place — so any plugin that looks it up still finds it — and quietens it
296
+ through its documented options: `verbose = -2` and `no_header`, an emptied
297
+ `reportchars`, and a wrapper around `pytest_report_teststatus` that drops the
298
+ progress characters while preserving pytest's own categorization.
299
+
300
+ `no_summary` is deliberately *not* used, although it looks like the obvious
301
+ switch. It gates the whole `pytest_terminal_summary` hook, which is where
302
+ third-party plugins write, and setting it swallowed pytest-cov's report
303
+ entirely. Reported upstream as
304
+ [pytest#14724](https://github.com/pytest-dev/pytest/issues/14724).
305
+
306
+ **It collects from public hooks.** `pytest_runtest_logreport` for phase results,
307
+ `pytest_collectreport` for collection failures, `pytest_warning_recorded` for
308
+ warnings, and `pytest_sessionfinish` to render. Rendering happens in
309
+ `sessionfinish` rather than `terminal_summary` because pytest does not call the
310
+ latter for internal errors, and an internal error is exactly when you most need
311
+ to be told the truth.
312
+
313
+ **Grouping is call-site aware.** The key is exception type, phase, crash
314
+ location, and cause chain. The message is deliberately excluded: keying on it
315
+ fragmented a parametrized test into one group per input, which defeats grouping
316
+ exactly where suites are most repetitive. Differing messages are kept as
317
+ variants inside the group and shown. Crash location rather than test line is
318
+ what makes this right — a bug in `merge.py:117` groups every caller.
319
+
320
+ **Tracebacks keep the decisive frame.** Every local frame is kept, because that
321
+ is the code you can change. External frames are pruned to the boundary you
322
+ entered the dependency at and the frame that actually broke, with elisions
323
+ marked:
324
+
325
+ ```text
326
+ frames: tests/test_merge.py:12 -> molsysmt/merge.py:41 -> numpy/core/shape.py:88 (ext) -> ... -> numpy/core/_methods.py:52 (ext)
327
+ ```
328
+
329
+ Dropping external frames entirely is cheaper, and wrong: when a failure
330
+ originates inside NumPy or a serializer, the external frame *is* the answer.
331
+
332
+ **Nothing is thrown away, and nothing is deferred.** Grouping is a presentation
333
+ decision; every occurrence keeps its node ID, phase, and location. The complete
334
+ report is written to `.pytest_cache/d/receptor/last-run.txt` while the run is
335
+ still going, owner-only and refusing symlinks. Detail is only ever held back
336
+ when that file exists to hold it, so a consumer can never be left with
337
+ information reachable solely by running the suite again.
338
+
339
+ **`--tb` is deliberately left alone.** It controls how pytest *builds*
340
+ `longrepr`, not how it prints it. Forcing `--tb=no` would look like a sensible
341
+ way to suppress tracebacks and would silently destroy every frame this plugin
342
+ exists to summarize.
343
+
344
+ ---
345
+
346
+ ## What your agent is doing right now
347
+
348
+ Your agent runs `pytest`. Plain, because that is the obvious command and nobody
349
+ told it otherwise. So every test run spends tokens on a platform banner, a
350
+ `rootdir` line, a plugin list, a progress bar, and the source code of every
351
+ failing test — none of which the agent needs, all of which you pay for, on every
352
+ iteration of every loop.
353
+
354
+ Measured with `tiktoken` (`cl100k_base`):
355
+
356
+ | Scenario | `pytest` | `--receptor=llm` | Change |
357
+ | :--- | ---: | ---: | ---: |
358
+ | Cascade (38 failures, one cause) | 3300 | **105** | **-96.8%** |
359
+ | Green suite (128 tests) | 118 | **15** | -87.3% |
360
+ | Green with warnings | 181 | **45** | -75.1% |
361
+ | Green with many distinct warnings | 1692 | **664** | -60.8% |
362
+ | Single assertion failure | 349 | **165** | -52.7% |
363
+ | Five distinct causes | 405 | **211** | -47.9% |
364
+ | Collection error | 286 | **213** | -25.5% |
365
+ | Mixed states (skip, xfail, xpass) | 124 | **77** | -37.9% |
366
+
367
+ Every scenario is cheaper, most of them by half or better. In a TDD loop that
368
+ runs the suite twenty times, the cascade row alone is sixty thousand tokens.
369
+
370
+ And the saving grows with the suite. Measured on eight thousand tests under
371
+ twelve xdist workers, against `pytest -q -n 12` — a pytest that has *already*
372
+ been quietened:
373
+
374
+ | Scenario | `-q -n 12` | `--receptor=llm -n 12` | Saving |
375
+ | :--- | ---: | ---: | ---: |
376
+ | Whole suite green | 812 | **17** | 97.9% |
377
+ | One fixture breaks 200 tests | 25,681 | **107** | 99.6% |
378
+ | Six unrelated bugs | 1,503 | **278** | 81.5% |
379
+
380
+ `-q` prints one progress character per test, so a *successful* eight-thousand
381
+ test run costs 812 tokens of dots before anything has gone wrong.
382
+
383
+ ### And if you already tuned pytest
384
+
385
+ If you are the kind of person who already runs `pytest -q --no-header
386
+ --tb=short`, the picture is narrower and you deserve to see that too:
387
+
388
+ | Scenario | tuned pytest | `--receptor=llm` | Change |
389
+ | :--- | ---: | ---: | ---: |
390
+ | Cascade (38 failures, one cause) | 2863 | **105** | -96.3% |
391
+ | Green with many distinct warnings | 1598 | **664** | -58.4% |
392
+ | Green with warnings | 87 | **45** | -48.3% |
393
+ | Five distinct causes | 316 | **211** | -33.2% |
394
+ | Green suite (128 tests) | 23 | **15** | -34.8% |
395
+ | Single assertion failure | 197 | **165** | -16.2% |
396
+ | Collection error | 192 | **213** | +10.9% |
397
+ | Mixed states (skip, xfail, xpass) | 31 | **77** | +148.4% |
398
+
399
+ Both positive rows are scenarios of a handful of tests, where any fixed overhead
400
+ looks enormous as a percentage: +148.4% is forty-six tokens. They buy the
401
+ reason behind every skip and xfail and the name of the test that passed
402
+ unexpectedly, where `pytest -q` says `1 skipped, 1 xfailed, 1 xpassed` and
403
+ leaves you to re-run with `-rs` to find out which. Those sections are bounded by
404
+ the variety of reasons, not the number of tests, so four hundred skips across
405
+ three reasons still cost three lines.
406
+
407
+ The cascade row does not move: grouping forty failures into one root cause is
408
+ something no combination of pytest flags does.
409
+
410
+ ### Measure it on your own suite
411
+
412
+ ```bash
413
+ pytest --receptor=llm --receptor-stats
414
+ ```
415
+
416
+ ```text
417
+ receptor stats: 38 tokens vs 148 for pytest as you configured it | 110 fewer (-74.3%) | cl100k_base
418
+ ```
419
+
420
+ The baseline here is *your* pytest configuration, not the strict one used in the
421
+ table above, because the question this answers is personal: against how you
422
+ actually run pytest, what does this save you? It is measured rather than
423
+ estimated — pytest genuinely renders into a temporary file during the same run,
424
+ which is then tokenized and deleted. No second invocation, no extra memory, and
425
+ your own output is unaffected.
426
+
427
+ Reproduce the table above with `python devtools/benchmarks/run_benchmarks.py`.
428
+
429
+ ---
430
+
431
+ ## Documentation
432
+
433
+ Full documentation: [uibcdf.github.io/pytest-receptor](https://uibcdf.github.io/pytest-receptor/)
434
+
435
+ Design notes, the audit that shaped the current scope, and the open work queue
436
+ live in [`devguide/`](devguide/README.md).
437
+
438
+ ## License
439
+
440
+ MIT. See [LICENSE](LICENSE).