pytest-collect-profile 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,8 @@
1
+ __pycache__/
2
+ *.py[cod]
3
+ .pytest_cache/
4
+ .ruff_cache/
5
+ .venv/
6
+ build/
7
+ dist/
8
+ *.egg-info/
@@ -0,0 +1,99 @@
1
+ # Contributing
2
+
3
+ Thanks for your interest in `pytest-collect-profile`. The project has one
4
+ purpose: find what makes pytest collection slow. Its only user-facing interface
5
+ is `pytest --collect-profile`.
6
+
7
+ ## Scope
8
+
9
+ Before proposing a change, make sure it:
10
+
11
+ - helps users find slow pytest collection (not slow tests or general CI costs);
12
+ - works through the single `--collect-profile` interface;
13
+ - does not change pytest's test selection, execution, or exit status;
14
+ - uses public pytest hooks and the standard library where possible.
15
+
16
+ Changes that add options, configuration, or new subsystems before the first
17
+ release are out of scope.
18
+
19
+ ## Workflow
20
+
21
+ 1. Start from the default branch (`main`).
22
+ 2. Create a short-lived topic branch with one of these prefixes:
23
+
24
+ ```text
25
+ feat/<short-slug> new behavior
26
+ fix/<short-slug> defect fix
27
+ docs/<short-slug> documentation only
28
+ test/<short-slug> tests only
29
+ chore/<short-slug> tooling, packaging, maintenance
30
+ ```
31
+
32
+ Keep the slug short and use lowercase kebab-case, for example
33
+ `fix/slow-module-import`.
34
+
35
+ 3. Keep the branch focused on one logical change.
36
+ 4. Bring in changes from `main` by rebasing or merging when needed.
37
+
38
+ ## Local checks
39
+
40
+ Create and activate a virtual environment, then install the project and its
41
+ development tools:
42
+
43
+ ```bash
44
+ python -m pip install --editable . build "ruff==0.16.6"
45
+ ```
46
+
47
+ Run the Python lint, formatting, and test checks before opening a pull request:
48
+
49
+ ```bash
50
+ python -m ruff check .
51
+ python -m ruff format --check .
52
+ python -m pytest -q
53
+ ```
54
+
55
+ ## Commits
56
+
57
+ Use Conventional Commit subjects:
58
+
59
+ ```text
60
+ feat: rank custom collectors without special casing
61
+ fix: keep report silent when the flag is absent
62
+ docs: document interpretation of the summary line
63
+ test: cover fewer-than-10 collector output
64
+ chore: set up minimal packaging
65
+ ```
66
+
67
+ - Write the subject in the imperative mood, in lowercase, without a trailing
68
+ period, and keep it to 72 characters or fewer.
69
+ - The body is optional. Use it to explain *why*, not *what*, and wrap it at
70
+ about 72 characters.
71
+ - Write commits and pull requests in English.
72
+
73
+ ## Releases
74
+
75
+ Versions follow [SemVer](https://semver.org/) (`MAJOR.MINOR.PATCH`) and each
76
+ release is tagged `v<major>.<minor>.<patch>`, e.g. `v0.1.0`.
77
+
78
+ ## Pull requests
79
+
80
+ Use a Conventional Commit-style pull request title that describes the primary
81
+ change, for example `docs: add contribution workflow`.
82
+
83
+ Briefly explain the purpose of the change, summarize what changed, and record
84
+ which checks were run.
85
+
86
+ Pull requests are squash-merged into `main`. Use the pull request title as the
87
+ squash commit subject. Delete the topic branch after merging.
88
+
89
+ Before opening a pull request:
90
+
91
+ - Run the project's test suite (and any documented lint/build checks once
92
+ tooling is established).
93
+ - Add tests for new behavior. For a defect fix, add the smallest regression
94
+ check.
95
+ - Update the documentation the change affects.
96
+ - Verify the diff contains no secrets, machine-specific paths, or unrelated
97
+ work.
98
+
99
+ Keep pull requests small and self-contained.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Jan Mrowiec
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,110 @@
1
+ Metadata-Version: 2.5
2
+ Name: pytest-collect-profile
3
+ Version: 0.1.0
4
+ Summary: Find what makes pytest collection slow
5
+ Project-URL: Repository, https://github.com/janmrow/pytest-collect-profile
6
+ Project-URL: Issues, https://github.com/janmrow/pytest-collect-profile/issues
7
+ Author: Jan Mrowiec
8
+ License-Expression: MIT
9
+ License-File: LICENSE
10
+ Classifier: Framework :: Pytest
11
+ Classifier: Programming Language :: Python :: 3
12
+ Requires-Python: >=3.10
13
+ Requires-Dist: pytest>=8.0
14
+ Description-Content-Type: text/markdown
15
+
16
+ # pytest-collect-profile
17
+
18
+ Find what makes pytest collection slow.
19
+
20
+ `pytest-collect-profile` is a pytest plugin that reports the slowest collector
21
+ operations and the total collection time before tests begin to run. It measures
22
+ pytest's collection model directly, including custom collectors, rather than
23
+ guessing from file layout.
24
+
25
+ ## Installation
26
+
27
+ Install the plugin in the environment where pytest runs:
28
+
29
+ ```bash
30
+ python -m pip install pytest-collect-profile
31
+ ```
32
+
33
+ To install from a source checkout instead:
34
+
35
+ ```bash
36
+ python -m pip install .
37
+ ```
38
+
39
+ ## Quick start
40
+
41
+ Run the normal test suite with collection profiling enabled:
42
+
43
+ ```bash
44
+ pytest --collect-profile
45
+ ```
46
+
47
+ To inspect collection without executing tests, combine the plugin flag with
48
+ pytest's existing `--collect-only` option:
49
+
50
+ ```bash
51
+ pytest --collect-profile --collect-only
52
+ ```
53
+
54
+ `--collect-profile` does not imply `--collect-only`; without the latter, pytest
55
+ continues to execute the selected tests normally.
56
+
57
+ ## Example
58
+
59
+ ```text
60
+ ====================== collect profile ======================
61
+ time collector node
62
+ 1.421s Module tests/integration/test_api.py
63
+ 0.612s Class tests/test_users.py::TestUsers
64
+ 0.184s Directory tests/integration
65
+
66
+ Total collection: 2.340s | 842 items
67
+ ```
68
+
69
+ The banner padding may change with terminal width. The report itself always
70
+ uses seconds with three decimal places and shows at most 10 collector operations,
71
+ ordered by measured duration. An empty root-session node ID appears as
72
+ `<session>`.
73
+
74
+ Each row measures one collector operation. Collector calls may be nested, so row
75
+ times can overlap. `Total collection` is measured separately and includes work
76
+ outside collector calls, such as item modification hooks. Do not add the rows or
77
+ expect their sum to equal the total.
78
+
79
+ The report helps locate where collection time is spent. It does not explain why
80
+ a collector is slow or guarantee that optimizing one displayed row will reduce
81
+ the total by the same amount.
82
+
83
+ ## Behavior
84
+
85
+ - The plugin is silent unless `--collect-profile` is present.
86
+ - The report appears after collection and before test execution.
87
+ - Selection, execution, collection errors, test failures, and exit statuses
88
+ remain pytest's responsibility.
89
+ - Timings and records exist only for the current pytest process; the plugin
90
+ creates no history, cache, network request, or external service.
91
+
92
+ ## Compatibility
93
+
94
+ `pytest-collect-profile` requires Python 3.10 or newer and pytest 8.0 or newer.
95
+ Version `0.1.0` targets Linux, macOS, and Windows.
96
+
97
+ The project's GitHub Actions matrix has passed for Python 3.10 through 3.14,
98
+ representative pytest 8.x and 9.x releases, and Linux, macOS, and Windows.
99
+
100
+ ## Limitations
101
+
102
+ Version `0.1.0` profiles collection only. It does not profile test execution,
103
+ fixtures, functions, or call stacks; explain the cause of slowness; produce JSON,
104
+ HTML, or flamegraphs; store history; compare runs; enforce thresholds; or offer
105
+ configurable ranking and filters. It provides no special xdist guarantees.
106
+
107
+ See the
108
+ [contribution guide](https://github.com/janmrow/pytest-collect-profile/blob/main/CONTRIBUTING.md)
109
+ for contribution guidance. This project is available under the terms of the
110
+ [MIT License](https://github.com/janmrow/pytest-collect-profile/blob/main/LICENSE).
@@ -0,0 +1,95 @@
1
+ # pytest-collect-profile
2
+
3
+ Find what makes pytest collection slow.
4
+
5
+ `pytest-collect-profile` is a pytest plugin that reports the slowest collector
6
+ operations and the total collection time before tests begin to run. It measures
7
+ pytest's collection model directly, including custom collectors, rather than
8
+ guessing from file layout.
9
+
10
+ ## Installation
11
+
12
+ Install the plugin in the environment where pytest runs:
13
+
14
+ ```bash
15
+ python -m pip install pytest-collect-profile
16
+ ```
17
+
18
+ To install from a source checkout instead:
19
+
20
+ ```bash
21
+ python -m pip install .
22
+ ```
23
+
24
+ ## Quick start
25
+
26
+ Run the normal test suite with collection profiling enabled:
27
+
28
+ ```bash
29
+ pytest --collect-profile
30
+ ```
31
+
32
+ To inspect collection without executing tests, combine the plugin flag with
33
+ pytest's existing `--collect-only` option:
34
+
35
+ ```bash
36
+ pytest --collect-profile --collect-only
37
+ ```
38
+
39
+ `--collect-profile` does not imply `--collect-only`; without the latter, pytest
40
+ continues to execute the selected tests normally.
41
+
42
+ ## Example
43
+
44
+ ```text
45
+ ====================== collect profile ======================
46
+ time collector node
47
+ 1.421s Module tests/integration/test_api.py
48
+ 0.612s Class tests/test_users.py::TestUsers
49
+ 0.184s Directory tests/integration
50
+
51
+ Total collection: 2.340s | 842 items
52
+ ```
53
+
54
+ The banner padding may change with terminal width. The report itself always
55
+ uses seconds with three decimal places and shows at most 10 collector operations,
56
+ ordered by measured duration. An empty root-session node ID appears as
57
+ `<session>`.
58
+
59
+ Each row measures one collector operation. Collector calls may be nested, so row
60
+ times can overlap. `Total collection` is measured separately and includes work
61
+ outside collector calls, such as item modification hooks. Do not add the rows or
62
+ expect their sum to equal the total.
63
+
64
+ The report helps locate where collection time is spent. It does not explain why
65
+ a collector is slow or guarantee that optimizing one displayed row will reduce
66
+ the total by the same amount.
67
+
68
+ ## Behavior
69
+
70
+ - The plugin is silent unless `--collect-profile` is present.
71
+ - The report appears after collection and before test execution.
72
+ - Selection, execution, collection errors, test failures, and exit statuses
73
+ remain pytest's responsibility.
74
+ - Timings and records exist only for the current pytest process; the plugin
75
+ creates no history, cache, network request, or external service.
76
+
77
+ ## Compatibility
78
+
79
+ `pytest-collect-profile` requires Python 3.10 or newer and pytest 8.0 or newer.
80
+ Version `0.1.0` targets Linux, macOS, and Windows.
81
+
82
+ The project's GitHub Actions matrix has passed for Python 3.10 through 3.14,
83
+ representative pytest 8.x and 9.x releases, and Linux, macOS, and Windows.
84
+
85
+ ## Limitations
86
+
87
+ Version `0.1.0` profiles collection only. It does not profile test execution,
88
+ fixtures, functions, or call stacks; explain the cause of slowness; produce JSON,
89
+ HTML, or flamegraphs; store history; compare runs; enforce thresholds; or offer
90
+ configurable ranking and filters. It provides no special xdist guarantees.
91
+
92
+ See the
93
+ [contribution guide](https://github.com/janmrow/pytest-collect-profile/blob/main/CONTRIBUTING.md)
94
+ for contribution guidance. This project is available under the terms of the
95
+ [MIT License](https://github.com/janmrow/pytest-collect-profile/blob/main/LICENSE).
@@ -0,0 +1,48 @@
1
+ [build-system]
2
+ requires = ["hatchling>=1.26"]
3
+ build-backend = "hatchling.build"
4
+
5
+ [project]
6
+ name = "pytest-collect-profile"
7
+ version = "0.1.0"
8
+ description = "Find what makes pytest collection slow"
9
+ readme = "README.md"
10
+ requires-python = ">=3.10"
11
+ authors = [{ name = "Jan Mrowiec" }]
12
+ license = "MIT"
13
+ license-files = ["LICENSE"]
14
+ classifiers = [
15
+ "Framework :: Pytest",
16
+ "Programming Language :: Python :: 3",
17
+ ]
18
+ dependencies = [
19
+ "pytest>=8.0",
20
+ ]
21
+
22
+ [project.urls]
23
+ Repository = "https://github.com/janmrow/pytest-collect-profile"
24
+ Issues = "https://github.com/janmrow/pytest-collect-profile/issues"
25
+
26
+ [project.entry-points.pytest11]
27
+ collect-profile = "pytest_collect_profile.plugin"
28
+
29
+ [tool.hatch.build.targets.sdist]
30
+ only-include = [
31
+ "CONTRIBUTING.md",
32
+ "README.md",
33
+ "src",
34
+ "tests",
35
+ ]
36
+
37
+ [tool.hatch.build.targets.wheel]
38
+ packages = ["src/pytest_collect_profile"]
39
+
40
+ [tool.pytest.ini_options]
41
+ testpaths = ["tests"]
42
+ pythonpath = ["src"]
43
+
44
+ [tool.ruff]
45
+ target-version = "py310"
46
+
47
+ [tool.ruff.lint]
48
+ select = ["B", "E4", "E7", "E9", "F", "I", "UP"]
@@ -0,0 +1 @@
1
+ """Profile pytest collection at pytest's collector boundaries."""
@@ -0,0 +1,116 @@
1
+ """Pytest hooks for collection profiling."""
2
+
3
+ from __future__ import annotations
4
+
5
+ from collections.abc import Callable, Iterable
6
+ from dataclasses import dataclass
7
+ from time import perf_counter_ns
8
+
9
+ import pytest
10
+
11
+ _OPTION = "--collect-profile"
12
+ _RUNTIME_PLUGIN_NAME = "pytest-collect-profile-runtime"
13
+ _ROW_LIMIT = 10
14
+
15
+
16
+ @dataclass(frozen=True)
17
+ class _CollectorTiming:
18
+ duration_ns: int
19
+ collector_type: str
20
+ nodeid: str
21
+
22
+
23
+ def pytest_addoption(parser: pytest.Parser) -> None:
24
+ group = parser.getgroup("collect profile")
25
+ group.addoption(
26
+ _OPTION,
27
+ action="store_true",
28
+ default=False,
29
+ help="show the slowest collectors after collection",
30
+ )
31
+
32
+
33
+ def pytest_configure(config: pytest.Config) -> None:
34
+ if config.getoption(_OPTION):
35
+ config.pluginmanager.register(
36
+ _CollectProfilePlugin(config), name=_RUNTIME_PLUGIN_NAME
37
+ )
38
+
39
+
40
+ class _CollectProfilePlugin:
41
+ def __init__(
42
+ self,
43
+ config: pytest.Config,
44
+ clock: Callable[[], int] = perf_counter_ns,
45
+ ) -> None:
46
+ self._config = config
47
+ self._clock = clock
48
+ self._collector_timings: list[_CollectorTiming] = []
49
+
50
+ @pytest.hookimpl(wrapper=True, tryfirst=True)
51
+ def pytest_make_collect_report(self, collector: pytest.Collector):
52
+ started_at = self._clock()
53
+ try:
54
+ result = yield
55
+ finally:
56
+ self._collector_timings.append(
57
+ _CollectorTiming(
58
+ duration_ns=self._clock() - started_at,
59
+ collector_type=type(collector).__name__,
60
+ nodeid=collector.nodeid,
61
+ )
62
+ )
63
+ return result
64
+
65
+ @pytest.hookimpl(wrapper=True, tryfirst=True)
66
+ def pytest_collection(self, session: pytest.Session):
67
+ started_at = self._clock()
68
+ result = yield
69
+ total_duration_ns = self._clock() - started_at
70
+ self._write_report(total_duration_ns, len(session.items))
71
+ return result
72
+
73
+ def _write_report(self, total_duration_ns: int, item_count: int) -> None:
74
+ terminal_reporter = self._config.pluginmanager.get_plugin("terminalreporter")
75
+ if terminal_reporter is None:
76
+ return
77
+
78
+ terminal_reporter.section("collect profile")
79
+ for line in _report_lines(
80
+ self._collector_timings, total_duration_ns, item_count
81
+ ):
82
+ terminal_reporter.write_line(line)
83
+
84
+
85
+ def _rank_timings(timings: Iterable[_CollectorTiming]) -> list[_CollectorTiming]:
86
+ return sorted(
87
+ timings,
88
+ key=lambda timing: (
89
+ -timing.duration_ns,
90
+ timing.nodeid,
91
+ timing.collector_type,
92
+ ),
93
+ )[:_ROW_LIMIT]
94
+
95
+
96
+ def _report_lines(
97
+ timings: Iterable[_CollectorTiming], total_duration_ns: int, item_count: int
98
+ ) -> list[str]:
99
+ lines = [f"{'time':<6} {'collector':<11} node"]
100
+
101
+ for timing in _rank_timings(timings):
102
+ duration = _format_duration(timing.duration_ns)
103
+ nodeid = timing.nodeid or "<session>"
104
+ lines.append(f"{duration:<6} {timing.collector_type:<11} {nodeid}")
105
+
106
+ lines.extend(
107
+ [
108
+ "",
109
+ f"Total collection: {_format_duration(total_duration_ns)} | {item_count} items",
110
+ ]
111
+ )
112
+ return lines
113
+
114
+
115
+ def _format_duration(duration_ns: int) -> str:
116
+ return f"{duration_ns / 1_000_000_000:.3f}s"
@@ -0,0 +1 @@
1
+ pytest_plugins = ["pytester"]
@@ -0,0 +1,273 @@
1
+ """Exercise the public workflow from an installed wheel in a fresh environment."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import argparse
6
+ import os
7
+ import re
8
+ import shlex
9
+ import subprocess
10
+ import sys
11
+ import tempfile
12
+ import venv
13
+ from pathlib import Path
14
+
15
+
16
+ def main() -> None:
17
+ parser = argparse.ArgumentParser()
18
+ parser.add_argument("wheel", type=Path)
19
+ wheel = parser.parse_args().wheel.resolve(strict=True)
20
+
21
+ with tempfile.TemporaryDirectory(prefix="pytest-collect-profile-smoke-") as root:
22
+ root_path = Path(root)
23
+ environment = root_path / "venv"
24
+ suite = root_path / "suite"
25
+ suite.mkdir()
26
+
27
+ venv.EnvBuilder(with_pip=True).create(environment)
28
+ python = _environment_python(environment)
29
+ pytest_command = _environment_pytest(environment)
30
+ clean_environment = _clean_subprocess_environment()
31
+
32
+ _run(
33
+ python,
34
+ "-m",
35
+ "pip",
36
+ "install",
37
+ str(wheel),
38
+ environment=clean_environment,
39
+ timeout=120,
40
+ )
41
+ _write_suite(suite)
42
+ execution_marker = suite / "test-executed"
43
+
44
+ help_result = _run(
45
+ python,
46
+ "-m",
47
+ "pytest",
48
+ "--help",
49
+ cwd=suite,
50
+ environment=clean_environment,
51
+ )
52
+ _require(
53
+ help_result.stdout.count("--collect-profile") == 1,
54
+ "installed pytest did not expose exactly one --collect-profile option",
55
+ help_result,
56
+ )
57
+
58
+ collect_only_result = _run(
59
+ pytest_command,
60
+ "--collect-profile",
61
+ "--collect-only",
62
+ cwd=suite,
63
+ environment=clean_environment,
64
+ )
65
+ _check_collect_only_run(collect_only_result)
66
+ _require(
67
+ not execution_marker.exists(),
68
+ "the README collect-only workflow executed a test",
69
+ collect_only_result,
70
+ )
71
+
72
+ profiled_result = _run(
73
+ python,
74
+ "-m",
75
+ "pytest",
76
+ "-q",
77
+ "-s",
78
+ "--collect-profile",
79
+ cwd=suite,
80
+ environment=clean_environment,
81
+ )
82
+ _check_profiled_run(profiled_result)
83
+ _require(
84
+ execution_marker.exists(),
85
+ "--collect-profile prevented normal test execution",
86
+ profiled_result,
87
+ )
88
+ execution_marker.unlink()
89
+
90
+ unprofiled_result = _run(
91
+ python,
92
+ "-m",
93
+ "pytest",
94
+ "-q",
95
+ "-s",
96
+ cwd=suite,
97
+ environment=clean_environment,
98
+ )
99
+ _check_unprofiled_run(unprofiled_result)
100
+ _require(
101
+ execution_marker.exists(),
102
+ "the unprofiled test did not execute",
103
+ unprofiled_result,
104
+ )
105
+
106
+ print("installed wheel smoke test passed")
107
+
108
+
109
+ def _environment_python(environment: Path) -> Path:
110
+ if os.name == "nt":
111
+ return environment / "Scripts" / "python.exe"
112
+ return environment / "bin" / "python"
113
+
114
+
115
+ def _environment_pytest(environment: Path) -> Path:
116
+ if os.name == "nt":
117
+ return environment / "Scripts" / "pytest.exe"
118
+ return environment / "bin" / "pytest"
119
+
120
+
121
+ def _clean_subprocess_environment() -> dict[str, str]:
122
+ environment = os.environ.copy()
123
+ for name in (
124
+ "PYTHONPATH",
125
+ "PYTEST_ADDOPTS",
126
+ "PYTEST_DISABLE_PLUGIN_AUTOLOAD",
127
+ "PYTEST_PLUGINS",
128
+ ):
129
+ environment.pop(name, None)
130
+ return environment
131
+
132
+
133
+ def _write_suite(suite: Path) -> None:
134
+ (suite / "conftest.py").write_text(
135
+ """
136
+ import time
137
+
138
+ import pytest
139
+
140
+
141
+ class SlowFile(pytest.File):
142
+ def collect(self):
143
+ time.sleep(0.03)
144
+ return []
145
+
146
+
147
+ def pytest_collect_file(file_path, parent):
148
+ if file_path.name == "slow.case":
149
+ return SlowFile.from_parent(parent, path=file_path)
150
+ """.lstrip(),
151
+ encoding="utf-8",
152
+ )
153
+ (suite / "slow.case").write_text("content\n", encoding="utf-8")
154
+ (suite / "test_sample.py").write_text(
155
+ """
156
+ from pathlib import Path
157
+
158
+
159
+ def test_runs():
160
+ Path("test-executed").write_text("executed", encoding="utf-8")
161
+ print("TEST EXECUTED")
162
+ """.lstrip(),
163
+ encoding="utf-8",
164
+ )
165
+
166
+
167
+ def _run(
168
+ *command: str | Path,
169
+ cwd: Path | None = None,
170
+ environment: dict[str, str],
171
+ timeout: int = 30,
172
+ ) -> subprocess.CompletedProcess[str]:
173
+ string_command = [str(part) for part in command]
174
+ result = subprocess.run(
175
+ string_command,
176
+ cwd=cwd,
177
+ env=environment,
178
+ capture_output=True,
179
+ text=True,
180
+ timeout=timeout,
181
+ check=False,
182
+ )
183
+ _require(
184
+ result.returncode == 0,
185
+ f"command failed: {shlex.join(string_command)}",
186
+ result,
187
+ )
188
+ return result
189
+
190
+
191
+ def _check_profiled_run(result: subprocess.CompletedProcess[str]) -> None:
192
+ output = result.stdout
193
+ _require(
194
+ re.search(
195
+ r"(?m)^time\s+collector\s+node\n"
196
+ r"\d+\.\d{3}s\s+SlowFile\s+slow\.case$",
197
+ output,
198
+ )
199
+ is not None,
200
+ "the deliberately slow collector was not the first reported row",
201
+ result,
202
+ )
203
+ _require(
204
+ re.search(r"Total collection: \d+\.\d{3}s \| 1 items", output) is not None,
205
+ "the installed plugin did not print the collection summary",
206
+ result,
207
+ )
208
+ _require(
209
+ "TEST EXECUTED" in output,
210
+ "--collect-profile prevented normal test execution",
211
+ result,
212
+ )
213
+ _require(
214
+ output.index("collect profile") < output.index("TEST EXECUTED"),
215
+ "the collection report did not appear before test execution",
216
+ result,
217
+ )
218
+ _require("1 passed" in output, "the profiled test did not pass", result)
219
+
220
+
221
+ def _check_collect_only_run(result: subprocess.CompletedProcess[str]) -> None:
222
+ output = result.stdout
223
+ _require(
224
+ re.search(
225
+ r"(?m)^time\s+collector\s+node\n"
226
+ r"\d+\.\d{3}s\s+SlowFile\s+slow\.case$",
227
+ output,
228
+ )
229
+ is not None,
230
+ "the README workflow did not rank the deliberately slow collector first",
231
+ result,
232
+ )
233
+ _require(
234
+ re.search(r"Total collection: \d+\.\d{3}s \| 1 items", output) is not None,
235
+ "the README workflow did not print the collection summary",
236
+ result,
237
+ )
238
+ _require(
239
+ "1 test collected" in output,
240
+ "the README workflow did not preserve pytest's collect-only outcome",
241
+ result,
242
+ )
243
+
244
+
245
+ def _check_unprofiled_run(result: subprocess.CompletedProcess[str]) -> None:
246
+ output = result.stdout
247
+ _require(
248
+ "collect profile" not in output,
249
+ "the installed plugin printed a report without its option",
250
+ result,
251
+ )
252
+ _require("TEST EXECUTED" in output, "the unprofiled test did not run", result)
253
+ _require("1 passed" in output, "the unprofiled test did not pass", result)
254
+
255
+
256
+ def _require(
257
+ condition: bool,
258
+ message: str,
259
+ result: subprocess.CompletedProcess[str],
260
+ ) -> None:
261
+ if condition:
262
+ return
263
+
264
+ raise RuntimeError(
265
+ f"{message}\n\nstdout:\n{result.stdout}\nstderr:\n{result.stderr}"
266
+ )
267
+
268
+
269
+ if __name__ == "__main__":
270
+ try:
271
+ main()
272
+ except (OSError, RuntimeError, subprocess.SubprocessError) as error:
273
+ sys.exit(str(error))
@@ -0,0 +1,393 @@
1
+ from __future__ import annotations
2
+
3
+ import re
4
+
5
+ import pytest
6
+
7
+ from pytest_collect_profile import plugin
8
+ from pytest_collect_profile.plugin import (
9
+ _CollectorTiming,
10
+ _CollectProfilePlugin,
11
+ _format_duration,
12
+ _report_lines,
13
+ )
14
+
15
+
16
+ def _timing(duration_ns: int, collector_type: str, nodeid: str) -> _CollectorTiming:
17
+ return _CollectorTiming(duration_ns, collector_type, nodeid)
18
+
19
+
20
+ def test_duration_uses_fixed_seconds_with_three_decimal_places() -> None:
21
+ assert _format_duration(1_421_400_000) == "1.421s"
22
+ assert _format_duration(400_000) == "0.000s"
23
+
24
+
25
+ def test_report_formats_rows_session_label_and_summary() -> None:
26
+ lines = _report_lines(
27
+ [
28
+ _timing(1_421_000_000, "Module", "tests/test_api.py"),
29
+ _timing(612_000_000, "Session", ""),
30
+ ],
31
+ total_duration_ns=2_340_000_000,
32
+ item_count=842,
33
+ )
34
+
35
+ assert lines == [
36
+ "time collector node",
37
+ "1.421s Module tests/test_api.py",
38
+ "0.612s Session <session>",
39
+ "",
40
+ "Total collection: 2.340s | 842 items",
41
+ ]
42
+
43
+
44
+ def test_report_separates_long_collector_type_from_nodeid() -> None:
45
+ lines = _report_lines(
46
+ [_timing(1_000_000_000, "VeryLongCollector", "tests/test_api.py")],
47
+ total_duration_ns=1_000_000_000,
48
+ item_count=1,
49
+ )
50
+
51
+ assert lines[1] == "1.000s VeryLongCollector tests/test_api.py"
52
+
53
+
54
+ def test_report_sorts_raw_timings_before_display_rounding() -> None:
55
+ lines = _report_lines(
56
+ [
57
+ _timing(1_000_400_000, "Module", "tests/a.py"),
58
+ _timing(1_000_499_999, "Module", "tests/z.py"),
59
+ ],
60
+ total_duration_ns=2_000_000_000,
61
+ item_count=2,
62
+ )
63
+
64
+ assert lines[1:3] == [
65
+ "1.000s Module tests/z.py",
66
+ "1.000s Module tests/a.py",
67
+ ]
68
+
69
+
70
+ def test_report_breaks_exact_ties_by_nodeid_then_collector_type() -> None:
71
+ lines = _report_lines(
72
+ [
73
+ _timing(1_000_000_000, "Package", "tests/z.py"),
74
+ _timing(1_000_000_000, "Module", "tests/a.py"),
75
+ _timing(1_000_000_000, "Class", "tests/a.py"),
76
+ ],
77
+ total_duration_ns=3_000_000_000,
78
+ item_count=3,
79
+ )
80
+
81
+ assert lines[1:4] == [
82
+ "1.000s Class tests/a.py",
83
+ "1.000s Module tests/a.py",
84
+ "1.000s Package tests/z.py",
85
+ ]
86
+
87
+
88
+ def test_report_limits_rows_to_ten_without_ellipsis() -> None:
89
+ timings = [
90
+ _timing(index, "Module", f"tests/test_{index}.py") for index in range(12)
91
+ ]
92
+
93
+ lines = _report_lines(timings, total_duration_ns=100, item_count=12)
94
+
95
+ assert len(lines) == 13
96
+ assert "..." not in "\n".join(lines)
97
+ assert "tests/test_11.py" in lines[1]
98
+ assert "tests/test_2.py" in lines[10]
99
+
100
+
101
+ def test_report_keeps_every_row_when_fewer_than_ten_exist() -> None:
102
+ timings = [_timing(index, "Module", f"tests/test_{index}.py") for index in range(3)]
103
+
104
+ lines = _report_lines(timings, total_duration_ns=100, item_count=3)
105
+
106
+ assert len(lines) == 6
107
+
108
+
109
+ def test_nested_measurement_hooks_keep_inclusive_timings(pytester) -> None:
110
+ class ParentCollector:
111
+ nodeid = "parent"
112
+
113
+ class ChildCollector:
114
+ nodeid = "parent::child"
115
+
116
+ clock_values = iter([0, 10, 30, 50])
117
+ runtime = _CollectProfilePlugin(
118
+ config=pytester.parseconfig(),
119
+ clock=lambda: next(clock_values),
120
+ )
121
+
122
+ parent_hook = runtime.pytest_make_collect_report(ParentCollector())
123
+ assert next(parent_hook) is None
124
+ child_hook = runtime.pytest_make_collect_report(ChildCollector())
125
+ assert next(child_hook) is None
126
+
127
+ child_result = object()
128
+ with pytest.raises(StopIteration) as child_finished:
129
+ child_hook.send(child_result)
130
+ assert child_finished.value.value is child_result
131
+
132
+ parent_result = object()
133
+ with pytest.raises(StopIteration) as parent_finished:
134
+ parent_hook.send(parent_result)
135
+ assert parent_finished.value.value is parent_result
136
+
137
+ assert runtime._collector_timings == [
138
+ _timing(20, "ChildCollector", "parent::child"),
139
+ _timing(50, "ParentCollector", "parent"),
140
+ ]
141
+
142
+
143
+ def test_option_is_registered_once(pytester, monkeypatch) -> None:
144
+ monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
145
+
146
+ result = pytester.runpytest_inprocess("--help", plugins=[plugin])
147
+
148
+ result.stdout.fnmatch_lines(["*--collect-profile*slowest collectors*"])
149
+ assert result.stdout.str().count("--collect-profile") == 1
150
+
151
+
152
+ def test_disabled_plugin_is_silent_and_does_not_prevent_execution(
153
+ pytester, monkeypatch
154
+ ) -> None:
155
+ monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
156
+ pytester.makepyfile(
157
+ """
158
+ def test_runs():
159
+ print("TEST EXECUTED")
160
+ """
161
+ )
162
+
163
+ result = pytester.runpytest_inprocess("-q", "-s", plugins=[plugin])
164
+
165
+ result.assert_outcomes(passed=1)
166
+ result.stdout.fnmatch_lines(["*TEST EXECUTED*"])
167
+ assert "collect profile" not in result.stdout.str()
168
+
169
+
170
+ def test_enabled_plugin_ranks_a_slow_collector_and_reports_before_execution(
171
+ pytester, monkeypatch
172
+ ) -> None:
173
+ monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
174
+ pytester.makeconftest(
175
+ """
176
+ import time
177
+ import pytest
178
+
179
+ class SlowFile(pytest.File):
180
+ def collect(self):
181
+ time.sleep(0.03)
182
+ return []
183
+
184
+ def pytest_collect_file(file_path, parent):
185
+ if file_path.name == "slow.case":
186
+ return SlowFile.from_parent(parent, path=file_path)
187
+ """
188
+ )
189
+ pytester.makefile(".case", slow="content")
190
+ pytester.makepyfile(
191
+ """
192
+ def test_runs():
193
+ print("TEST EXECUTED")
194
+ """
195
+ )
196
+
197
+ result = pytester.runpytest_inprocess(
198
+ "--collect-profile", "-q", "-s", plugins=[plugin]
199
+ )
200
+
201
+ result.assert_outcomes(passed=1)
202
+ output = result.stdout.str()
203
+ assert re.search(
204
+ r"(?m)^time\s+collector\s+node\n"
205
+ r"\d+\.\d{3}s\s+SlowFile\s+slow\.case$",
206
+ output,
207
+ )
208
+ assert re.search(r"(?m)^\d+\.\d{3}s\s+Module\s+\S+\.py$", output)
209
+ assert re.search(r"Total collection: \d+\.\d{3}s \| 1 items", output)
210
+ assert output.index("collect profile") < output.index("TEST EXECUTED")
211
+
212
+
213
+ def test_terminal_report_treats_unexpected_nodeid_text_as_data(
214
+ pytester, monkeypatch
215
+ ) -> None:
216
+ monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
217
+ unexpected_nodeid = "$(touch terminal-side-effect).case"
218
+ side_effect = pytester.path / "terminal-side-effect"
219
+ (pytester.path / unexpected_nodeid).write_text("content\n", encoding="utf-8")
220
+ pytester.makeconftest(
221
+ f"""
222
+ import pytest
223
+
224
+ class UnexpectedFile(pytest.File):
225
+ def collect(self):
226
+ return []
227
+
228
+ def pytest_collect_file(file_path, parent):
229
+ if file_path.name == {unexpected_nodeid!r}:
230
+ return UnexpectedFile.from_parent(parent, path=file_path)
231
+ """
232
+ )
233
+ pytester.makepyfile("def test_passes(): pass")
234
+
235
+ result = pytester.runpytest_inprocess("--collect-profile", "-q", plugins=[plugin])
236
+
237
+ result.assert_outcomes(passed=1)
238
+ assert re.search(
239
+ rf"(?m)^\d+\.\d{{3}}s\s+UnexpectedFile\s+{re.escape(unexpected_nodeid)}$",
240
+ result.stdout.str(),
241
+ )
242
+ assert not side_effect.exists()
243
+
244
+
245
+ def test_total_includes_collection_work_outside_collector_reports(
246
+ pytester, monkeypatch
247
+ ) -> None:
248
+ monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
249
+ pytester.makeconftest(
250
+ """
251
+ import time
252
+
253
+ def pytest_collection_modifyitems(items):
254
+ time.sleep(0.03)
255
+ """
256
+ )
257
+ pytester.makepyfile("def test_passes(): pass")
258
+
259
+ result = pytester.runpytest_inprocess(
260
+ "--collect-profile", "--collect-only", "-q", plugins=[plugin]
261
+ )
262
+
263
+ assert result.ret == 0
264
+ result.stdout.fnmatch_lines(["*1 test collected*"])
265
+ output = result.stdout.str()
266
+ row_durations = [
267
+ float(value)
268
+ for value in re.findall(
269
+ r"(?m)^(\d+\.\d{3})s\s+\w+\s+(?:\S.*|<session>)$", output
270
+ )
271
+ ]
272
+ total_match = re.search(r"Total collection: (\d+\.\d{3})s", output)
273
+ assert row_durations
274
+ assert total_match is not None
275
+ assert float(total_match.group(1)) >= max(row_durations) + 0.02
276
+
277
+
278
+ def test_collect_only_reports_without_executing_tests(pytester, monkeypatch) -> None:
279
+ monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
280
+ execution_marker = pytester.path / "test-executed"
281
+ pytester.makepyfile(
282
+ f"""
283
+ from pathlib import Path
284
+
285
+ def test_does_not_run():
286
+ Path({str(execution_marker)!r}).write_text("executed", encoding="utf-8")
287
+ """
288
+ )
289
+
290
+ result = pytester.runpytest_inprocess(
291
+ "--collect-profile", "--collect-only", "-q", plugins=[plugin]
292
+ )
293
+
294
+ assert result.ret == 0
295
+ result.stdout.fnmatch_lines(["*1 test collected*"])
296
+ assert "Total collection:" in result.stdout.str()
297
+ assert not execution_marker.exists()
298
+
299
+
300
+ def test_profile_preserves_selection_and_collected_item_count(
301
+ pytester, monkeypatch
302
+ ) -> None:
303
+ monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
304
+ pytester.makepyfile(
305
+ """
306
+ def test_selected():
307
+ print("SELECTED EXECUTED")
308
+
309
+ def test_not_selected():
310
+ print("UNSELECTED EXECUTED")
311
+ """
312
+ )
313
+
314
+ profiled = pytester.runpytest_inprocess(
315
+ "--collect-profile", "-q", "-s", "-k", "test_selected", plugins=[plugin]
316
+ )
317
+ unprofiled = pytester.runpytest_inprocess(
318
+ "-q", "-s", "-k", "test_selected", plugins=[plugin]
319
+ )
320
+
321
+ assert profiled.ret == unprofiled.ret == 0
322
+ assert (
323
+ profiled.parseoutcomes()
324
+ == unprofiled.parseoutcomes()
325
+ == {
326
+ "passed": 1,
327
+ "deselected": 1,
328
+ }
329
+ )
330
+ assert "Total collection:" in profiled.stdout.str()
331
+ assert "| 1 items" in profiled.stdout.str()
332
+ assert "collect profile" not in unprofiled.stdout.str()
333
+ for result in (profiled, unprofiled):
334
+ assert "SELECTED EXECUTED" in result.stdout.str()
335
+ assert "UNSELECTED EXECUTED" not in result.stdout.str()
336
+
337
+
338
+ def test_profiled_empty_suite_preserves_native_no_tests_outcome(
339
+ pytester, monkeypatch
340
+ ) -> None:
341
+ monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
342
+
343
+ profiled = pytester.runpytest_inprocess("--collect-profile", "-q", plugins=[plugin])
344
+ unprofiled = pytester.runpytest_inprocess("-q", plugins=[plugin])
345
+
346
+ assert profiled.ret == unprofiled.ret == 5
347
+ assert "Total collection:" in profiled.stdout.str()
348
+ assert "| 0 items" in profiled.stdout.str()
349
+ assert "collect profile" not in unprofiled.stdout.str()
350
+
351
+
352
+ def test_collection_error_keeps_native_failure_and_exit_status(
353
+ pytester, monkeypatch
354
+ ) -> None:
355
+ monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
356
+ pytester.makepyfile("import module_that_does_not_exist")
357
+
358
+ profiled = pytester.runpytest_inprocess("--collect-profile", "-q", plugins=[plugin])
359
+ unprofiled = pytester.runpytest_inprocess("-q", plugins=[plugin])
360
+
361
+ assert profiled.ret == unprofiled.ret == 2
362
+ for result in (profiled, unprofiled):
363
+ result.stdout.fnmatch_lines(["*ERROR collecting*"])
364
+ assert "ModuleNotFoundError" in result.stdout.str()
365
+ assert "1 error" in result.stdout.str()
366
+ assert "passed" not in result.stdout.str()
367
+ assert "Total collection:" in profiled.stdout.str()
368
+ assert "| 0 items" in profiled.stdout.str()
369
+ assert "collect profile" not in unprofiled.stdout.str()
370
+
371
+
372
+ def test_failing_test_keeps_native_outcome_after_profile_report(
373
+ pytester, monkeypatch
374
+ ) -> None:
375
+ monkeypatch.setenv("PYTEST_DISABLE_PLUGIN_AUTOLOAD", "1")
376
+ pytester.makepyfile(
377
+ """
378
+ def test_fails():
379
+ assert False, "native failure"
380
+ """
381
+ )
382
+
383
+ profiled = pytester.runpytest_inprocess("--collect-profile", "-q", plugins=[plugin])
384
+ unprofiled = pytester.runpytest_inprocess("-q", plugins=[plugin])
385
+
386
+ assert profiled.ret == unprofiled.ret == 1
387
+ assert profiled.parseoutcomes() == unprofiled.parseoutcomes() == {"failed": 1}
388
+ for result in (profiled, unprofiled):
389
+ assert "native failure" in result.stdout.str()
390
+ output = profiled.stdout.str()
391
+ assert "Total collection:" in output
392
+ assert output.index("collect profile") < output.index("FAILURES")
393
+ assert "collect profile" not in unprofiled.stdout.str()