pytest-parallex 0.1.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- pytest_parallex-0.1.1/.gitignore +28 -0
- pytest_parallex-0.1.1/CHANGELOG.md +45 -0
- pytest_parallex-0.1.1/LICENSE +21 -0
- pytest_parallex-0.1.1/PKG-INFO +237 -0
- pytest_parallex-0.1.1/README.md +210 -0
- pytest_parallex-0.1.1/docs/benchmarks.md +110 -0
- pytest_parallex-0.1.1/docs/fork-safety-references.md +431 -0
- pytest_parallex-0.1.1/pyproject.toml +90 -0
- pytest_parallex-0.1.1/src/pytest_parallex/__init__.py +17 -0
- pytest_parallex-0.1.1/src/pytest_parallex/_loop.py +66 -0
- pytest_parallex-0.1.1/src/pytest_parallex/_reports.py +89 -0
- pytest_parallex-0.1.1/src/pytest_parallex/_safety.py +247 -0
- pytest_parallex-0.1.1/src/pytest_parallex/hooks.py +32 -0
- pytest_parallex-0.1.1/src/pytest_parallex/plugin.py +214 -0
- pytest_parallex-0.1.1/src/pytest_parallex/runner_async.py +88 -0
- pytest_parallex-0.1.1/src/pytest_parallex/runner_fork.py +302 -0
- pytest_parallex-0.1.1/src/pytest_parallex/runner_process.py +199 -0
- pytest_parallex-0.1.1/src/pytest_parallex/runner_thread.py +85 -0
- pytest_parallex-0.1.1/src/pytest_parallex/scheduler.py +77 -0
- pytest_parallex-0.1.1/src/pytest_parallex/session_scope.py +208 -0
- pytest_parallex-0.1.1/testing/conftest.py +1 -0
- pytest_parallex-0.1.1/testing/test_async.py +52 -0
- pytest_parallex-0.1.1/testing/test_plugin.py +57 -0
- pytest_parallex-0.1.1/testing/test_process.py +37 -0
- pytest_parallex-0.1.1/testing/test_regressions.py +378 -0
- pytest_parallex-0.1.1/testing/test_scheduler.py +39 -0
- pytest_parallex-0.1.1/testing/test_thread.py +159 -0
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Python
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*.egg-info/
|
|
5
|
+
.eggs/
|
|
6
|
+
|
|
7
|
+
# Build artifacts
|
|
8
|
+
/dist/
|
|
9
|
+
/build/
|
|
10
|
+
|
|
11
|
+
# uv / venv
|
|
12
|
+
.venv/
|
|
13
|
+
uv.lock.bak
|
|
14
|
+
|
|
15
|
+
# Tool caches
|
|
16
|
+
.pytest_cache/
|
|
17
|
+
.ruff_cache/
|
|
18
|
+
.pyright/
|
|
19
|
+
.mypy_cache/
|
|
20
|
+
.moon/cache/
|
|
21
|
+
|
|
22
|
+
# Editor
|
|
23
|
+
.vscode/
|
|
24
|
+
.idea/
|
|
25
|
+
*.swp
|
|
26
|
+
|
|
27
|
+
# Local session logs
|
|
28
|
+
logs/
|
|
@@ -0,0 +1,45 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here.
|
|
4
|
+
|
|
5
|
+
The version in `pyproject.toml` is the single source of truth, and merging a version bump
|
|
6
|
+
to `main` publishes it — see "Releasing" in the README.
|
|
7
|
+
|
|
8
|
+
## 0.1.1 — first release
|
|
9
|
+
|
|
10
|
+
The first published version is 0.1.1, not 0.1.0: `version-guard` requires a code change to
|
|
11
|
+
carry a version bump, and 0.1.0 never left this repo. Nothing was released as 0.1.0.
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
- `--parallel=fork`: collect once in the controller, then fork warm workers that inherit
|
|
16
|
+
the imported suite and the collected items via copy-on-write.
|
|
17
|
+
- **Controller-side session fixtures.** `scope="session"` runs once, in the controller,
|
|
18
|
+
before the fork; children inherit the cached result. Under pytest-xdist the same fixture
|
|
19
|
+
runs once *per worker*. `--parallex-no-session-scope` restores the per-worker behaviour.
|
|
20
|
+
- `--parallel=thread`, `--parallel=async`, `--parallel=process`.
|
|
21
|
+
- `--parallel-workers=N` (default `auto` = CPU count).
|
|
22
|
+
- Fork-safety refusal: the run stops with a message naming the offending thread or event
|
|
23
|
+
loop rather than forking a process that would deadlock in the child.
|
|
24
|
+
- Logging `QueueListener` threads are stopped and restarted around the fork, so a suite
|
|
25
|
+
that merely imports litestar (or anything using `QueueHandler`) can still fork.
|
|
26
|
+
- Hooks: `pytest_parallex_setup`, `pytest_parallex_teardown`,
|
|
27
|
+
`pytest_parallex_auto_num_workers`.
|
|
28
|
+
- Fixtures: `parallex_worker_id`, `parallex_mode`, `parallex_setup_data`.
|
|
29
|
+
- Dynamic work-claiming: workers race for module groups off a shared counter rather than
|
|
30
|
+
being dealt a static partition, so a slow module can't strand a worker.
|
|
31
|
+
|
|
32
|
+
### Requires
|
|
33
|
+
|
|
34
|
+
- **pytest >= 8.1**, not 8.0: `FixtureManager.getfixturedefs` took a nodeid string until
|
|
35
|
+
8.1 and a `Node` from 8.1 on, and the fork runner passes a `Node`.
|
|
36
|
+
- Tested against pytest 8.1, 8.4, 9.0 and 9.1 across all four modes. CI pins the floor.
|
|
37
|
+
|
|
38
|
+
### Worker-scoped fixtures
|
|
39
|
+
|
|
40
|
+
- A `scope="session"` fixture that requests `worker_id` (directly or transitively) is
|
|
41
|
+
per-worker and is never hoisted to the controller — so `postgres_server` (one container)
|
|
42
|
+
and `database_url(worker_id)` (one database per worker) can coexist, which is what
|
|
43
|
+
xdist's filelock recipe exists to fake.
|
|
44
|
+
- `worker_id` is provided by parallex when `--parallel` is active, registered late enough
|
|
45
|
+
to win the name from pytest-xdist. A plain `pytest -n 4` run keeps xdist's.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Jorge Cardona
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,237 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: pytest-parallex
|
|
3
|
+
Version: 0.1.1
|
|
4
|
+
Summary: Parallel test execution for pytest where scope="session" means session — fork-based warm workers that inherit session fixtures, plus thread, async and process modes
|
|
5
|
+
Project-URL: Homepage, https://gitlab.com/jorgeecardona/pytest-parallex
|
|
6
|
+
Project-URL: Repository, https://gitlab.com/jorgeecardona/pytest-parallex
|
|
7
|
+
Project-URL: Changelog, https://gitlab.com/jorgeecardona/pytest-parallex/-/blob/main/CHANGELOG.md
|
|
8
|
+
Author-email: Jorge Cardona <jorgeecardona@gmail.com>
|
|
9
|
+
License-Expression: MIT
|
|
10
|
+
License-File: LICENSE
|
|
11
|
+
Keywords: concurrency,fork,parallel,pytest,session-fixtures,test-speed,testing,xdist
|
|
12
|
+
Classifier: Development Status :: 4 - Beta
|
|
13
|
+
Classifier: Framework :: Pytest
|
|
14
|
+
Classifier: Intended Audience :: Developers
|
|
15
|
+
Classifier: Operating System :: MacOS
|
|
16
|
+
Classifier: Operating System :: POSIX :: Linux
|
|
17
|
+
Classifier: Programming Language :: Python :: 3
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
22
|
+
Classifier: Topic :: Software Development :: Testing
|
|
23
|
+
Classifier: Typing :: Typed
|
|
24
|
+
Requires-Python: >=3.10
|
|
25
|
+
Requires-Dist: pytest>=8.1
|
|
26
|
+
Description-Content-Type: text/markdown
|
|
27
|
+
|
|
28
|
+
# pytest-parallex
|
|
29
|
+
|
|
30
|
+
Parallel test execution for pytest, with one idea the other plugins can't offer:
|
|
31
|
+
**`scope="session"` means session — once, not once per worker.**
|
|
32
|
+
|
|
33
|
+
```bash
|
|
34
|
+
pip install pytest-parallex
|
|
35
|
+
pytest --parallel=fork
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
## The problem it solves
|
|
39
|
+
|
|
40
|
+
Under pytest-xdist, every worker is a fresh interpreter that runs its own full session, so
|
|
41
|
+
a `scope="session"` fixture executes **once per worker**. Their docs say so plainly, and
|
|
42
|
+
ship a `FileLock`-and-shared-tmpdir recipe as the workaround. It isn't a bug in your test
|
|
43
|
+
suite — their architecture has no moment at which a controller could build something once
|
|
44
|
+
and hand it to everybody.
|
|
45
|
+
|
|
46
|
+
Forking has that moment.
|
|
47
|
+
|
|
48
|
+
```python
|
|
49
|
+
@pytest.fixture(scope="session")
|
|
50
|
+
def postgres():
|
|
51
|
+
with PostgresContainer("postgres:16") as pg: # boots ONCE
|
|
52
|
+
yield pg.get_connection_url() # every worker gets this URL
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
`--parallel=fork` sets session fixtures up in the controller *before* forking. Children
|
|
56
|
+
inherit the `FixtureDef` — cached result and all — through copy-on-write, so they use it
|
|
57
|
+
and never re-run setup. One container, one compiled asset, one login, shared by every
|
|
58
|
+
worker, with no lockfile.
|
|
59
|
+
|
|
60
|
+
Same suite, same four workers, counting how many times the fixture body actually ran:
|
|
61
|
+
|
|
62
|
+
```console
|
|
63
|
+
$ pytest -n 4 # pytest-xdist
|
|
64
|
+
8 passed session fixture booted 4 times
|
|
65
|
+
|
|
66
|
+
$ pytest --parallel=fork --parallel-workers=4 # pytest-parallex
|
|
67
|
+
8 passed session fixture booted 1 time
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
That leaves the scope pytest never had a name for: **the worker**. What xdist calls
|
|
71
|
+
"session" is really per-worker, and conflating the two is what forces the recipe. Here
|
|
72
|
+
they're separate — `session` runs in the controller, and each forked child is a worker.
|
|
73
|
+
|
|
74
|
+
## Modes
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
pytest --parallel=fork --parallel-workers=8
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
| mode | isolation | `scope="session"` runs | use when |
|
|
81
|
+
|---|---|---|---|
|
|
82
|
+
| `fork` | process per worker, warm | **once, in the controller** | Linux/macOS, and the process is quiet at fork time |
|
|
83
|
+
| `thread` | shared address space | once | tests are I/O-bound and tolerate sharing globals |
|
|
84
|
+
| `async` | shared address space | once | same as `thread`, dispatched via asyncio |
|
|
85
|
+
| `process` | fresh interpreter per worker | once **per worker** | tests mutate process-global state, or you're on Windows |
|
|
86
|
+
|
|
87
|
+
`--parallel-workers` defaults to `auto` (CPU count).
|
|
88
|
+
|
|
89
|
+
## What fork actually costs and buys
|
|
90
|
+
|
|
91
|
+
Measured, and worth being precise about, because the intuitive pitch is wrong:
|
|
92
|
+
|
|
93
|
+
- **Removing xdist's worker boot is a flat ~1.5–2.3s.** It does *not* grow with suite size,
|
|
94
|
+
import weight, or worker count. On a 3s local run that's 1.84x and you feel it; on a 292s
|
|
95
|
+
CI suite it's under 1% and it's noise. Redundant imports parallelize across idle cores —
|
|
96
|
+
xdist's N-way collection is wasted *CPU*, not wasted *time*, and you only get paid for
|
|
97
|
+
removing it if the cores were already busy.
|
|
98
|
+
- **Session-fixture reuse is the part that scales.** A 10s container boot across 8 workers
|
|
99
|
+
is 80s under xdist and 10s here. That's the reason to reach for this.
|
|
100
|
+
|
|
101
|
+
Full method, numbers, and three traps that produced convincing wrong answers:
|
|
102
|
+
[docs/benchmarks.md](docs/benchmarks.md).
|
|
103
|
+
|
|
104
|
+
## Fork safety
|
|
105
|
+
|
|
106
|
+
`fork()` duplicates only the calling thread. Anything another thread held — a lock, a
|
|
107
|
+
half-written buffer, a connection's state machine — is duplicated mid-flight and belongs
|
|
108
|
+
to nobody in the child. That's how forked children deadlock, and CPython 3.12 warns about
|
|
109
|
+
exactly it.
|
|
110
|
+
|
|
111
|
+
So parallex refuses rather than warns, and names the offender:
|
|
112
|
+
|
|
113
|
+
```
|
|
114
|
+
--parallel=fork requires a quiet process at fork time, but 1 non-main thread(s)
|
|
115
|
+
are running (Thread-1 (_monitor)). Move the offending setup into a fixture so it
|
|
116
|
+
runs after the fork, or use --parallel=process.
|
|
117
|
+
```
|
|
118
|
+
|
|
119
|
+
Logging `QueueListener` threads (which litestar, and anything using `QueueHandler`, starts
|
|
120
|
+
at import) are stopped and restarted around the fork automatically.
|
|
121
|
+
|
|
122
|
+
The constraint this imposes on a pre-fork session fixture: whatever it builds must survive
|
|
123
|
+
a fork. **An address survives — a URL, a path, a port. A live connection, a thread, or an
|
|
124
|
+
event loop does not.** The natural split is that the controller owns the *server* and each
|
|
125
|
+
worker opens its own *connection* to it:
|
|
126
|
+
|
|
127
|
+
```python
|
|
128
|
+
@pytest.fixture(scope="session")
|
|
129
|
+
def db_url(): # controller: owns the server. Forkable.
|
|
130
|
+
with PostgresContainer("postgres:16") as pg:
|
|
131
|
+
yield pg.get_connection_url()
|
|
132
|
+
|
|
133
|
+
@pytest.fixture # worker: owns the connection. Not forkable, so per-test.
|
|
134
|
+
def db(db_url):
|
|
135
|
+
conn = psycopg.connect(db_url)
|
|
136
|
+
yield conn
|
|
137
|
+
conn.close()
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
If a session fixture can't survive the fork, `--parallex-no-session-scope` defers them all
|
|
141
|
+
to the workers (restoring xdist's once-per-worker behaviour).
|
|
142
|
+
|
|
143
|
+
## The fixture that has to differ per worker
|
|
144
|
+
|
|
145
|
+
Some `scope="session"` fixtures exist precisely *to* differ between workers — the archetype
|
|
146
|
+
being one that carves a database per worker so tests don't tread on each other. Hoisting
|
|
147
|
+
that one would be actively wrong: every worker gets the same database, and a per-test
|
|
148
|
+
`create_all`/`drop_all` then wipes its siblings' schemas mid-test.
|
|
149
|
+
|
|
150
|
+
**A fixture that requests `worker_id` — directly or through another fixture — is
|
|
151
|
+
per-worker, and is never hoisted.** Everything else that says `scope="session"` means it.
|
|
152
|
+
No new syntax, because suites already write it this way:
|
|
153
|
+
|
|
154
|
+
```python
|
|
155
|
+
@pytest.fixture(scope="session")
|
|
156
|
+
def postgres_server(): # -> controller. One container.
|
|
157
|
+
with PostgresContainer("postgres:16") as pg:
|
|
158
|
+
yield pg.get_connection_url()
|
|
159
|
+
|
|
160
|
+
@pytest.fixture(scope="session")
|
|
161
|
+
def database_url(worker_id, postgres_server): # -> per worker. Requests worker_id.
|
|
162
|
+
url = f"{postgres_server}/test_{worker_id}"
|
|
163
|
+
create_database(url)
|
|
164
|
+
yield url
|
|
165
|
+
drop_database(url)
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
One container for the run, one database per worker on it — which is the thing xdist's
|
|
169
|
+
filelock recipe exists to fake. This names the scope xdist conflates: what it calls
|
|
170
|
+
"session" is really per-worker, and here you can have both.
|
|
171
|
+
|
|
172
|
+
`worker_id` is also xdist's fixture name, deliberately — it's what makes existing suites
|
|
173
|
+
work unchanged. When both plugins are installed, `--parallel` takes the name so you get a
|
|
174
|
+
real per-worker id; a plain `pytest -n 4` run leaves xdist's alone. `parallex_worker_id`
|
|
175
|
+
is an alias if you'd rather be explicit, and marks a fixture per-worker just the same.
|
|
176
|
+
|
|
177
|
+
## Fixtures
|
|
178
|
+
|
|
179
|
+
| fixture | gives you |
|
|
180
|
+
|---|---|
|
|
181
|
+
| `worker_id` | `'f0'`, `'w1'`, `'a2'`… or `'main'`. Requesting it marks a fixture per-worker |
|
|
182
|
+
| `parallex_worker_id` | the same value, under a name xdist can't shadow |
|
|
183
|
+
| `parallex_mode` | the active mode, or `None` |
|
|
184
|
+
| `parallex_setup_data` | whatever `pytest_parallex_setup` returned |
|
|
185
|
+
|
|
186
|
+
## Hooks
|
|
187
|
+
|
|
188
|
+
```python
|
|
189
|
+
# conftest.py
|
|
190
|
+
def pytest_parallex_setup(config):
|
|
191
|
+
"""Run once in the controller, before any worker starts. Pre-fork: no I/O handles."""
|
|
192
|
+
return {"token": build_expensive_thing()}
|
|
193
|
+
|
|
194
|
+
def pytest_parallex_teardown(config, data):
|
|
195
|
+
"""Run once in the controller, after every worker has finished."""
|
|
196
|
+
|
|
197
|
+
def pytest_parallex_auto_num_workers(config):
|
|
198
|
+
"""Override --parallel-workers=auto. Default: os.cpu_count()."""
|
|
199
|
+
return 8
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
## Known limitations
|
|
203
|
+
|
|
204
|
+
- **`--maxfail` doesn't stop a fork run early.** Workers run their claimed group to
|
|
205
|
+
completion and the controller replays the reports afterwards, so the count is right but
|
|
206
|
+
the run isn't cut short.
|
|
207
|
+
- **Fork parallelizes by module.** A whole suite in one file uses one worker no matter what
|
|
208
|
+
`--parallel-workers` says.
|
|
209
|
+
- **`fork` is Linux/macOS only.** On Windows, use `--parallel=process`.
|
|
210
|
+
- **`thread`/`async` don't escape the GIL.** They're for suites that wait, not suites that
|
|
211
|
+
compute.
|
|
212
|
+
|
|
213
|
+
## Developing
|
|
214
|
+
|
|
215
|
+
```bash
|
|
216
|
+
uv sync # toolchain
|
|
217
|
+
make install-hooks # one-time, after cloning
|
|
218
|
+
make check # lint + typecheck + test, exactly what CI runs
|
|
219
|
+
```
|
|
220
|
+
|
|
221
|
+
## Releasing
|
|
222
|
+
|
|
223
|
+
The version in `pyproject.toml` is the single source of truth, and releases are automated:
|
|
224
|
+
|
|
225
|
+
1. **`pre-commit` auto-bumps** the patch version whenever a commit touches `src/` (run
|
|
226
|
+
`make install-hooks` once after cloning). Doc/test/config-only commits don't bump. For
|
|
227
|
+
an intentional minor/major release, bump deliberately: `make bump TYPE=minor`.
|
|
228
|
+
2. **`version-guard`** (part of `moon run :ci`) enforces the same rule non-bypassably in
|
|
229
|
+
CI: a push or MR that changes `src/` without a version bump fails the pipeline, catching
|
|
230
|
+
`--no-verify` and unhooked clones.
|
|
231
|
+
3. **`:release`** runs on a green `main` pipeline: if `pyproject`'s version isn't on PyPI
|
|
232
|
+
yet, it publishes via OIDC Trusted Publishing. So merging a version bump to `main`
|
|
233
|
+
releases itself — no second pipeline, no tag required.
|
|
234
|
+
|
|
235
|
+
## License
|
|
236
|
+
|
|
237
|
+
MIT
|
|
@@ -0,0 +1,210 @@
|
|
|
1
|
+
# pytest-parallex
|
|
2
|
+
|
|
3
|
+
Parallel test execution for pytest, with one idea the other plugins can't offer:
|
|
4
|
+
**`scope="session"` means session — once, not once per worker.**
|
|
5
|
+
|
|
6
|
+
```bash
|
|
7
|
+
pip install pytest-parallex
|
|
8
|
+
pytest --parallel=fork
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
## The problem it solves
|
|
12
|
+
|
|
13
|
+
Under pytest-xdist, every worker is a fresh interpreter that runs its own full session, so
|
|
14
|
+
a `scope="session"` fixture executes **once per worker**. Their docs say so plainly, and
|
|
15
|
+
ship a `FileLock`-and-shared-tmpdir recipe as the workaround. It isn't a bug in your test
|
|
16
|
+
suite — their architecture has no moment at which a controller could build something once
|
|
17
|
+
and hand it to everybody.
|
|
18
|
+
|
|
19
|
+
Forking has that moment.
|
|
20
|
+
|
|
21
|
+
```python
|
|
22
|
+
@pytest.fixture(scope="session")
|
|
23
|
+
def postgres():
|
|
24
|
+
with PostgresContainer("postgres:16") as pg: # boots ONCE
|
|
25
|
+
yield pg.get_connection_url() # every worker gets this URL
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
`--parallel=fork` sets session fixtures up in the controller *before* forking. Children
|
|
29
|
+
inherit the `FixtureDef` — cached result and all — through copy-on-write, so they use it
|
|
30
|
+
and never re-run setup. One container, one compiled asset, one login, shared by every
|
|
31
|
+
worker, with no lockfile.
|
|
32
|
+
|
|
33
|
+
Same suite, same four workers, counting how many times the fixture body actually ran:
|
|
34
|
+
|
|
35
|
+
```console
|
|
36
|
+
$ pytest -n 4 # pytest-xdist
|
|
37
|
+
8 passed session fixture booted 4 times
|
|
38
|
+
|
|
39
|
+
$ pytest --parallel=fork --parallel-workers=4 # pytest-parallex
|
|
40
|
+
8 passed session fixture booted 1 time
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
That leaves the scope pytest never had a name for: **the worker**. What xdist calls
|
|
44
|
+
"session" is really per-worker, and conflating the two is what forces the recipe. Here
|
|
45
|
+
they're separate — `session` runs in the controller, and each forked child is a worker.
|
|
46
|
+
|
|
47
|
+
## Modes
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
pytest --parallel=fork --parallel-workers=8
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
| mode | isolation | `scope="session"` runs | use when |
|
|
54
|
+
|---|---|---|---|
|
|
55
|
+
| `fork` | process per worker, warm | **once, in the controller** | Linux/macOS, and the process is quiet at fork time |
|
|
56
|
+
| `thread` | shared address space | once | tests are I/O-bound and tolerate sharing globals |
|
|
57
|
+
| `async` | shared address space | once | same as `thread`, dispatched via asyncio |
|
|
58
|
+
| `process` | fresh interpreter per worker | once **per worker** | tests mutate process-global state, or you're on Windows |
|
|
59
|
+
|
|
60
|
+
`--parallel-workers` defaults to `auto` (CPU count).
|
|
61
|
+
|
|
62
|
+
## What fork actually costs and buys
|
|
63
|
+
|
|
64
|
+
Measured, and worth being precise about, because the intuitive pitch is wrong:
|
|
65
|
+
|
|
66
|
+
- **Removing xdist's worker boot is a flat ~1.5–2.3s.** It does *not* grow with suite size,
|
|
67
|
+
import weight, or worker count. On a 3s local run that's 1.84x and you feel it; on a 292s
|
|
68
|
+
CI suite it's under 1% and it's noise. Redundant imports parallelize across idle cores —
|
|
69
|
+
xdist's N-way collection is wasted *CPU*, not wasted *time*, and you only get paid for
|
|
70
|
+
removing it if the cores were already busy.
|
|
71
|
+
- **Session-fixture reuse is the part that scales.** A 10s container boot across 8 workers
|
|
72
|
+
is 80s under xdist and 10s here. That's the reason to reach for this.
|
|
73
|
+
|
|
74
|
+
Full method, numbers, and three traps that produced convincing wrong answers:
|
|
75
|
+
[docs/benchmarks.md](docs/benchmarks.md).
|
|
76
|
+
|
|
77
|
+
## Fork safety
|
|
78
|
+
|
|
79
|
+
`fork()` duplicates only the calling thread. Anything another thread held — a lock, a
|
|
80
|
+
half-written buffer, a connection's state machine — is duplicated mid-flight and belongs
|
|
81
|
+
to nobody in the child. That's how forked children deadlock, and CPython 3.12 warns about
|
|
82
|
+
exactly it.
|
|
83
|
+
|
|
84
|
+
So parallex refuses rather than warns, and names the offender:
|
|
85
|
+
|
|
86
|
+
```
|
|
87
|
+
--parallel=fork requires a quiet process at fork time, but 1 non-main thread(s)
|
|
88
|
+
are running (Thread-1 (_monitor)). Move the offending setup into a fixture so it
|
|
89
|
+
runs after the fork, or use --parallel=process.
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
Logging `QueueListener` threads (which litestar, and anything using `QueueHandler`, starts
|
|
93
|
+
at import) are stopped and restarted around the fork automatically.
|
|
94
|
+
|
|
95
|
+
The constraint this imposes on a pre-fork session fixture: whatever it builds must survive
|
|
96
|
+
a fork. **An address survives — a URL, a path, a port. A live connection, a thread, or an
|
|
97
|
+
event loop does not.** The natural split is that the controller owns the *server* and each
|
|
98
|
+
worker opens its own *connection* to it:
|
|
99
|
+
|
|
100
|
+
```python
|
|
101
|
+
@pytest.fixture(scope="session")
|
|
102
|
+
def db_url(): # controller: owns the server. Forkable.
|
|
103
|
+
with PostgresContainer("postgres:16") as pg:
|
|
104
|
+
yield pg.get_connection_url()
|
|
105
|
+
|
|
106
|
+
@pytest.fixture # worker: owns the connection. Not forkable, so per-test.
|
|
107
|
+
def db(db_url):
|
|
108
|
+
conn = psycopg.connect(db_url)
|
|
109
|
+
yield conn
|
|
110
|
+
conn.close()
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
If a session fixture can't survive the fork, `--parallex-no-session-scope` defers them all
|
|
114
|
+
to the workers (restoring xdist's once-per-worker behaviour).
|
|
115
|
+
|
|
116
|
+
## The fixture that has to differ per worker
|
|
117
|
+
|
|
118
|
+
Some `scope="session"` fixtures exist precisely *to* differ between workers — the archetype
|
|
119
|
+
being one that carves a database per worker so tests don't tread on each other. Hoisting
|
|
120
|
+
that one would be actively wrong: every worker gets the same database, and a per-test
|
|
121
|
+
`create_all`/`drop_all` then wipes its siblings' schemas mid-test.
|
|
122
|
+
|
|
123
|
+
**A fixture that requests `worker_id` — directly or through another fixture — is
|
|
124
|
+
per-worker, and is never hoisted.** Everything else that says `scope="session"` means it.
|
|
125
|
+
No new syntax, because suites already write it this way:
|
|
126
|
+
|
|
127
|
+
```python
|
|
128
|
+
@pytest.fixture(scope="session")
|
|
129
|
+
def postgres_server(): # -> controller. One container.
|
|
130
|
+
with PostgresContainer("postgres:16") as pg:
|
|
131
|
+
yield pg.get_connection_url()
|
|
132
|
+
|
|
133
|
+
@pytest.fixture(scope="session")
|
|
134
|
+
def database_url(worker_id, postgres_server): # -> per worker. Requests worker_id.
|
|
135
|
+
url = f"{postgres_server}/test_{worker_id}"
|
|
136
|
+
create_database(url)
|
|
137
|
+
yield url
|
|
138
|
+
drop_database(url)
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
One container for the run, one database per worker on it — which is the thing xdist's
|
|
142
|
+
filelock recipe exists to fake. This names the scope xdist conflates: what it calls
|
|
143
|
+
"session" is really per-worker, and here you can have both.
|
|
144
|
+
|
|
145
|
+
`worker_id` is also xdist's fixture name, deliberately — it's what makes existing suites
|
|
146
|
+
work unchanged. When both plugins are installed, `--parallel` takes the name so you get a
|
|
147
|
+
real per-worker id; a plain `pytest -n 4` run leaves xdist's alone. `parallex_worker_id`
|
|
148
|
+
is an alias if you'd rather be explicit, and marks a fixture per-worker just the same.
|
|
149
|
+
|
|
150
|
+
## Fixtures
|
|
151
|
+
|
|
152
|
+
| fixture | gives you |
|
|
153
|
+
|---|---|
|
|
154
|
+
| `worker_id` | `'f0'`, `'w1'`, `'a2'`… or `'main'`. Requesting it marks a fixture per-worker |
|
|
155
|
+
| `parallex_worker_id` | the same value, under a name xdist can't shadow |
|
|
156
|
+
| `parallex_mode` | the active mode, or `None` |
|
|
157
|
+
| `parallex_setup_data` | whatever `pytest_parallex_setup` returned |
|
|
158
|
+
|
|
159
|
+
## Hooks
|
|
160
|
+
|
|
161
|
+
```python
|
|
162
|
+
# conftest.py
|
|
163
|
+
def pytest_parallex_setup(config):
|
|
164
|
+
"""Run once in the controller, before any worker starts. Pre-fork: no I/O handles."""
|
|
165
|
+
return {"token": build_expensive_thing()}
|
|
166
|
+
|
|
167
|
+
def pytest_parallex_teardown(config, data):
|
|
168
|
+
"""Run once in the controller, after every worker has finished."""
|
|
169
|
+
|
|
170
|
+
def pytest_parallex_auto_num_workers(config):
|
|
171
|
+
"""Override --parallel-workers=auto. Default: os.cpu_count()."""
|
|
172
|
+
return 8
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
## Known limitations
|
|
176
|
+
|
|
177
|
+
- **`--maxfail` doesn't stop a fork run early.** Workers run their claimed group to
|
|
178
|
+
completion and the controller replays the reports afterwards, so the count is right but
|
|
179
|
+
the run isn't cut short.
|
|
180
|
+
- **Fork parallelizes by module.** A whole suite in one file uses one worker no matter what
|
|
181
|
+
`--parallel-workers` says.
|
|
182
|
+
- **`fork` is Linux/macOS only.** On Windows, use `--parallel=process`.
|
|
183
|
+
- **`thread`/`async` don't escape the GIL.** They're for suites that wait, not suites that
|
|
184
|
+
compute.
|
|
185
|
+
|
|
186
|
+
## Developing
|
|
187
|
+
|
|
188
|
+
```bash
|
|
189
|
+
uv sync # toolchain
|
|
190
|
+
make install-hooks # one-time, after cloning
|
|
191
|
+
make check # lint + typecheck + test, exactly what CI runs
|
|
192
|
+
```
|
|
193
|
+
|
|
194
|
+
## Releasing
|
|
195
|
+
|
|
196
|
+
The version in `pyproject.toml` is the single source of truth, and releases are automated:
|
|
197
|
+
|
|
198
|
+
1. **`pre-commit` auto-bumps** the patch version whenever a commit touches `src/` (run
|
|
199
|
+
`make install-hooks` once after cloning). Doc/test/config-only commits don't bump. For
|
|
200
|
+
an intentional minor/major release, bump deliberately: `make bump TYPE=minor`.
|
|
201
|
+
2. **`version-guard`** (part of `moon run :ci`) enforces the same rule non-bypassably in
|
|
202
|
+
CI: a push or MR that changes `src/` without a version bump fails the pipeline, catching
|
|
203
|
+
`--no-verify` and unhooked clones.
|
|
204
|
+
3. **`:release`** runs on a green `main` pipeline: if `pyproject`'s version isn't on PyPI
|
|
205
|
+
yet, it publishes via OIDC Trusted Publishing. So merging a version bump to `main`
|
|
206
|
+
releases itself — no second pipeline, no tag required.
|
|
207
|
+
|
|
208
|
+
## License
|
|
209
|
+
|
|
210
|
+
MIT
|
|
@@ -0,0 +1,110 @@
|
|
|
1
|
+
# What fork actually buys — measured, 2026-07-14
|
|
2
|
+
|
|
3
|
+
Written after a session that produced three wrong answers before a right one. The wrong turns are
|
|
4
|
+
kept here on purpose: each one is a trap that looks exactly like a result.
|
|
5
|
+
|
|
6
|
+
## Bottom line
|
|
7
|
+
|
|
8
|
+
**`--parallel=fork` removes xdist's fixed worker-boot cost — a flat ~1.5–2.3s. That is the whole
|
|
9
|
+
win.** It does *not* scale with suite size, import weight, or worker count.
|
|
10
|
+
|
|
11
|
+
That makes fork mode a **local TDD-loop feature, not a CI feature**:
|
|
12
|
+
|
|
13
|
+
| suite | xdist | fork | verdict |
|
|
14
|
+
|---|---|---|---|
|
|
15
|
+
| 3s local run | 3.21s | 1.75s | **1.84x — you feel this** |
|
|
16
|
+
| paas CI `api:test` (292s) | — | −2s | **under 1%. noise.** |
|
|
17
|
+
|
|
18
|
+
## The thesis that died
|
|
19
|
+
|
|
20
|
+
The pitch was: *xdist re-imports and re-collects in every worker (its own FAQ admits it, and
|
|
21
|
+
[#353](https://github.com/pytest-dev/pytest-xdist/issues/353) reports a 4x collection penalty on a
|
|
22
|
+
16k-test suite), so collect-once-then-fork should scale better the heavier the import.*
|
|
23
|
+
|
|
24
|
+
Measured, with a CPU-bound import (not a `sleep` — see trap 3), 8 workers on 16 cores:
|
|
25
|
+
|
|
26
|
+
| import cost | xdist -n8 | fork x8 | ratio | absolute saving |
|
|
27
|
+
|---|---|---|---|---|
|
|
28
|
+
| 0.03s | 3.21s | 1.75s | 1.83x | 1.46s |
|
|
29
|
+
| 0.55s | 3.82s | 2.17s | 1.76x | 1.65s |
|
|
30
|
+
| 2.01s | 5.37s | 3.67s | 1.46x | 1.70s |
|
|
31
|
+
| 4.65s | 8.75s | 6.40s | 1.37x | 2.35s |
|
|
32
|
+
|
|
33
|
+
**The ratio shrinks as the import grows, and the absolute saving is flat.** The prediction was
|
|
34
|
+
backwards, and the reason is the thing to internalize:
|
|
35
|
+
|
|
36
|
+
> **Redundant imports parallelize across idle cores.** Eight workers each importing on eight free
|
|
37
|
+
> cores costs ~1x wall time, not 8x. xdist's N-way collection is wasted *CPU*, not wasted *time*.
|
|
38
|
+
> You only get paid for removing it if the cores were already busy.
|
|
39
|
+
|
|
40
|
+
So xdist's real, addressable overhead is just gateway boot: `subprocess.Popen` of a virgin
|
|
41
|
+
interpreter per worker, plus the execnet handshake. Constant, and small.
|
|
42
|
+
|
|
43
|
+
## Three traps that produced convincing wrong numbers
|
|
44
|
+
|
|
45
|
+
**1. The runner silently never ran.** `pytest_runtestloop` is a `firstresult` hook — the first impl
|
|
46
|
+
to return non-None wins. Registering with `trylast=True` handed the loop to pytest's builtin, which
|
|
47
|
+
returned `True`. Every test still passed, serially, and the "fork" benchmarks were plain serial
|
|
48
|
+
pytest for hours. The tell was CPU utilization: **0.9 cores busy on a 16-worker run** (`user 5m44s`
|
|
49
|
+
/ `real 7m14s`), against xdist's 8.9 (`user 18m30s` / `real 2m10s`). *A parallel runner that isn't
|
|
50
|
+
running looks like a slow parallel runner, not like a broken one.* Check `user`/`real` before
|
|
51
|
+
believing any parallel benchmark.
|
|
52
|
+
|
|
53
|
+
**2. Benchmarking a suite that can't fork.** With the hook fixed, paas's suite refuses outright:
|
|
54
|
+
|
|
55
|
+
```
|
|
56
|
+
--parallel=fork requires a quiet process at fork time, but 1 non-main thread(s)
|
|
57
|
+
are running (Thread-1 (_monitor)).
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
`litestar.logging.standard.LoggingQueueListener` starts a logging thread **at import**. This is the
|
|
61
|
+
predicted #1 hazard arriving immediately: you cannot stop threads you don't own. Android's zygote
|
|
62
|
+
only gets away with fork because `ZygoteHooks.preFork()` calls `Daemons.stop()` and restarts them
|
|
63
|
+
after — it owns the entire runtime. A pytest plugin owns none of it.
|
|
64
|
+
|
|
65
|
+
**3. `time.sleep()` is not an import.** The first synthetic modelled import cost as a sleep. Sleep
|
|
66
|
+
parallelizes perfectly, so it showed fork's advantage *growing* with import weight — the opposite of
|
|
67
|
+
the truth. Model CPU-bound work with CPU-bound work.
|
|
68
|
+
|
|
69
|
+
## What was actually confirmed
|
|
70
|
+
|
|
71
|
+
- **xdist cannot fork, and it isn't close.** execnet's `get_execmodel()` closes over
|
|
72
|
+
`thread`/`main_thread_only`/`eventlet`/`gevent` with an exhaustive `else: raise`; workers are
|
|
73
|
+
spawned `subprocess.Popen([python, "-u", "-c", bootstrap])`. There is nothing to inherit. The
|
|
74
|
+
ground is genuinely unoccupied.
|
|
75
|
+
- **The stdlib `random` frozen-seed hazard does not apply to CPython.** `Lib/random.py:992` does
|
|
76
|
+
`_os.register_at_fork(after_in_child=_inst.seed)` — children reseed automatically. Verified by
|
|
77
|
+
forking after `random.seed(42)`: children diverge. The Spring
|
|
78
|
+
[#113](https://github.com/rails/spring/issues/113) precedent that froze a test seed for five years
|
|
79
|
+
is *Ruby*, whose `Random` has no such hook. Libraries that seed their own state (numpy, faker)
|
|
80
|
+
still need checking; stdlib does not.
|
|
81
|
+
- **The zygote effect is real but narrow.** Forking a parent that has already paid a 904ms
|
|
82
|
+
`create_app()` costs **1.56ms/child, flat out to 248 children**. Whether that is worth anything
|
|
83
|
+
depends entirely on whether the suite is CPU-bound — see below.
|
|
84
|
+
- **Dynamic scheduling beat static by only 6%** on the paas suite (411.5s → 386.5s), far under the
|
|
85
|
+
1.43x predicted from per-module costs. Both numbers are void anyway (trap 1), but the
|
|
86
|
+
claim-counter queue is the right design regardless: a static deal must guess a group's cost from
|
|
87
|
+
its test *count*, and on a suite whose modules ranged 0.1s–167s that guess is bad.
|
|
88
|
+
|
|
89
|
+
## Where this leaves the project
|
|
90
|
+
|
|
91
|
+
Fork mode is worth shipping for the **inner loop** — the same niche Discord's pytest daemon targets
|
|
92
|
+
(median 20s → 2s) without inheriting their hard problem (correct incremental reload of a long-lived
|
|
93
|
+
dirty process; a forked child is a throwaway copy and the parent is never mutated).
|
|
94
|
+
|
|
95
|
+
It is **not** a CI play, and the prior art says so too: DHH removed Spring from Rails defaults in
|
|
96
|
+
[rails/rails#42997](https://github.com/rails/rails/pull/42997) with the reason *"Faster computers
|
|
97
|
+
have meant that most apps won't see a big benefit... the pain of dealing with the occasional spring
|
|
98
|
+
issue is no longer warranted by default."* The cost of a preloader is constant; the benefit shrinks
|
|
99
|
+
with hardware. That is the same shape as the flat 1.5–2.3s measured above.
|
|
100
|
+
|
|
101
|
+
Open questions, in order:
|
|
102
|
+
1. **Can a suite be made forkable?** paas can't fork because of one third-party logging thread. A
|
|
103
|
+
`pytest_parallex_pre_fork` / `post_fork` hook pair would let the suite author quiesce and restart
|
|
104
|
+
what the plugin can't know about (Android's `Daemons.stop()`, Passenger's
|
|
105
|
+
`:starting_worker_process`, Spring's `after_fork` — every prior system converged on this).
|
|
106
|
+
2. **Is the inner-loop win defensible against `--lf`/`-k`?** A dev rerunning one module already
|
|
107
|
+
pays only that module's import. Measure fork against the workflows people actually use, not
|
|
108
|
+
against a full-suite run.
|
|
109
|
+
3. **Does the zygote matter for anyone?** It only pays where per-test setup is expensive AND
|
|
110
|
+
CPU-bound. paas's setup is expensive but I/O-bound (Postgres), so it wouldn't have helped there.
|