pytest-timing 0.2.0__tar.gz → 0.3.1__tar.gz

{pytest_timing-0.2.0 → pytest_timing-0.3.1}/ARCHITECTURE.md

@@ -14,9 +14,9 @@ schedules a run from the previous one. For usage, see the [README](README.md).
 | `cli.py` | `pytest-timing render` and `pytest-timing merge`. |
 | `fixtures.py` | Times shared fixture set-up where tests run and lists dependencies on reports. |
 | `schedule.py` | Duration estimates, the fixture-aware cost model and the planner. Free of xdist. |
-| `demand.py` | Declared CPU demand: the `timing_cpu` marker, the fixture decorator, the declarations a worker sends, and the meter that records each test's CPU work. |
+| `demand.py` | Declared CPU demand: the `timing_cpu` marker, the fixture decorator, the declarations a worker sends, and the meter that records each test's CPU work and resident memory. |
 | `admission.py` | One host's CPU budget, reservations, fair waiting line and pressure feedback. Free of xdist and platform reads. |
-| `telemetry.py` | Platform reads: affinity and cgroup quota, CPU time of a process tree, pressure and throttling. |
+| `telemetry.py` | Platform reads: affinity and cgroup quota, CPU time and resident memory of a process tree, pressure and throttling. |
 | `xdist_scheduler.py` | The xdist scheduler that drives workers with the planner and the admission gate, in `load` and `worksteal` mode. |
 | `xdist_compat.py` | Everything that reads pytest-xdist internals (versions, worker detection, why the controller stopped, the custom worker event and controller command). |
 
@@ -85,10 +85,13 @@ optimistic there.
 
 `Estimates.from_run` subtracts shared set-up and runtime admission waits from each
 attempt's duration to estimate the test's *own* time. Crashed attempts and nonpositive
-results are excluded. For each node id it uses the longest clean attempt, or the
-longest contended attempt when no clean one exists. Tests without a usable estimate
-get the mean of those estimates, or zero if none exist. Fixture dependencies are
-combined across attempts, and each fixture key's set-up cost is the longest recorded.
+results are excluded. For each node id it ranks the attempts clean before contended
+and passing before failing, and takes the median of the best rank present: a failed
+attempt usually stops early, and the longest attempt grows with the number of
+attempts, so a history merged from several runs would drift upward. Tests without a
+usable estimate get the mean of those estimates, or zero if none exist. Fixture
+dependencies are combined across attempts, and each fixture key's set-up cost is the
+median of the recorded ones.
 
 A missing or unreadable history file leaves xdist's scheduler in place unless an
 explicit CPU budget was requested. With that budget the custom scheduler still runs,
@@ -370,11 +373,71 @@ naming it in `/proc/self/cgroup` wins over the `0::` v2 line, whose hierarchy ha
 controller there. Throttling is read from the same group's `cpu.stat` (`throttled_usec`
 on v2, `throttled_time` in nanoseconds on v1).
 
+## Memory admission
+
+Memory goes through the same gate as CPU, as a second `Admission` per domain counted
+in bytes, and a test is granted only when its reservation fits both: `_raise` checks
+every gate with `fits` first and reserves on all of them or none, and a refused worker
+waits in every line with its respective need. Reservations, releases, forced
+admissions, withdrawals and stall resolution act on both gates in step, so their
+`busy` and `idle` states never disagree. The rules that let CPU admission exceed its
+limit are safe for memory for the same reasons they are safe for CPU: backfilling
+never exceeds the limit (it lends out slots pledged to the head), a request above the
+limit is clamped so the test runs alone rather than never, and a forced admission
+happens only when nothing runs in the domain, so nothing else's memory is at risk.
+Pressure feedback moves only the CPU limit.
+
+Demand comes from history, never from declarations. `Estimates.from_run` keeps, per
+test, the largest need any attempt showed (the worst case is what an out-of-memory
+kill depends on), and per shared fixture key the memory the attempt that paid its
+set-up kept resident (`after - base`, split evenly when one attempt paid for
+several). A payer's rise spans its set-ups, so its own need is `peak - after`: what
+it used beyond what stayed, which is the fixtures'. Session- and package-scoped
+fixtures are the exception: what they keep is every worker's baseline
+(`is_baseline`), never attributed, charged or projected. Every worker sets them up
+once and never lets go, so gating on them cannot spare the host; it can only hold
+tests back, or park a worker until stall resolution shuts it down, which is what a
+large session fixture did before this rule. The first attempt on each worker is a
+warm-up window (lazy imports, caches, the allocator's first growth), so its rise and
+residual count only for a test or fixture with no other attempt.
+
+`Costs.charge` adds a `memory` to each `Charge`: the test's need plus what the
+module- and class-scoped fixtures alive around it keep, projected through the lane's
+fixture state exactly as CPU holds are, since workers report nothing about memory at
+run time. A fixture with recorded memory belongs to its tests' families even when its
+set-up was too quick to matter for time. `_need_memory` is the twin of `_need`; an
+idle worker reserves what its fixtures keep, and a worker whose next test could never
+fit next to what the other workers' fixtures keep is parked like one blocked by CPU
+holds. A fixture reached at run time adds its recorded memory to the queued charges
+when its request is granted.
+
+The budget is `--timing-memory`: a size, or `auto`, which takes 80% of the smallest
+memory the domain's workers report (physical memory capped by the cgroup limit,
+`memory.max` on v2 and `memory.limit_in_bytes` on v1, read like the CPU quota). The
+share is deliberate: estimates are rises above each worker's footprint, and the
+footprints (session fixtures included), the controller and the rest of the host are
+not in them. Without a recorded run every estimate is zero and the gate admits
+everything; the summary says so. The planner ignores memory: with tests kept apart by
+the gate, balancing lanes by memory would add little, and the lane plan can still move.
+
+Every `Admission` has a `kind` (`cpu` or `memory`), and a refusal records which kinds
+refused (`refused`), as does passing a test over for what other workers keep. When
+the worker is admitted, the wait it records (`AdmissionWait.gates`) says which gates
+held it, and the collector puts it on the test's `cpu.wait` or `memory.wait`
+accordingly. A worker that leaves while parked, having run nothing it waited for,
+has no test to carry its wait: `_release_runtime` records it in `parked`, and each
+gate's summary reports the parked time and worker count beside the tests' waits.
+
 ## Measuring CPU work
 
 `ProcessTreeClock` uses `os.times()` for worker CPU time and, on POSIX, reaped child
 CPU time. Where available, it adds live descendants through
-`/proc/<pid>/task/*/children` and `/proc/<pid>/stat` on Linux, or psutil otherwise.
+`/proc/<pid>/task/*/children` and `/proc/<pid>/stat` on Linux, through `libproc` on
+macOS (`proc_listchildpids` to find them, `proc_pidinfo` for their task times, in
+Mach time units converted with `mach_timebase_info`), or psutil otherwise. psutil is
+the last resort because its `children` scans the whole process table, about ten
+milliseconds on macOS, and the clock is read several times per test: with psutil
+installed, 3,000 trivial tests went from under a second to over a minute.
 Readings account for a waited-for child moving from the live total into the reaped
 total. Process discovery is a snapshot, so exits during traversal or descendants
 that outlive or detach from their parents can leave gaps. Every record reports its
@@ -389,6 +452,45 @@ the host's PSI `some` share and whether the cgroup was throttled during
 the test; `Estimates.from_run` treats such attempts as contended and prefers a clean
 attempt of the same test when one exists, keeping the contended ones in the file.
 
+## Measuring memory
+
+Memory is recorded so that a later run can keep tests that need a lot of it from
+running at the same time; nothing schedules on it yet. `ResidentMemory` reads the
+resident set size of the process running the tests and, where they can be listed, of
+its live descendants: `/proc/self/statm` and the `/proc` walk shared with CPU
+measurement on Linux, the `libproc` reader shared with the CPU clock on macOS,
+`GetProcessMemoryInfo` on Windows for the worker alone. psutil fills in what the
+platform readers cannot do, today descendants on Windows, and is never preferred
+over a native reader. Descendants are summed, so pages they share count more than
+once; the total errs on the large side.
+
+A high-water mark such as `ru_maxrss` never comes down, so it cannot say what one
+test needed: only the test that first raised the worker's peak would show anything.
+`MemorySampler` therefore polls from a daemon thread while a window is open and
+keeps the highest total. The worker's own size is read every 20 ms; it costs about a
+microsecond on macOS and ten on Linux. Descendants are listed at most every 100 ms,
+and while none are found the interval doubles up to a second, since listing costs an
+order of magnitude more (and far more with psutil). Opening or closing a window never
+lists descendants by itself, so a fast test costs two readings of its own process, a
+few microseconds. The meter opens the window at `pytest_runtest_setup` and closes it
+when the teardown report is made, so shared fixture set-ups are charged to the test
+that paid for them, as their time is. The window goes on the teardown report as
+`timing_memory` and into the JSON as `memory`: `base`, `peak` and `after` in bytes,
+and the coverage (`tree` or `self`). A platform with no reading records nothing.
+The sampler sleeps between windows and is closed at `pytest_unconfigure`.
+
+A test shorter than the sampling interval is seen only at its edges: its record is
+what was resident before and after it, and a buffer allocated and freed inside it
+is missed. Faulting in enough memory to matter takes longer than one interval.
+
+`peak - base` is the attempt's rise: what it needed on top of the worker's footprint.
+`after - base` is what stayed resident, which for the first test of a session fixture
+is roughly the fixture. Both are biased by allocator behaviour: a heap that already
+grew for an earlier test can serve a later one without raising RSS, so a rise can
+undercount a test whose allocations reuse freed heap, and a freed buffer the allocator
+keeps can leave `after` high. Large buffers and subprocess memory, the usual causes of
+an out-of-memory kill, are mapped and unmapped directly and measure well.
+
 ## Feedback
 
 Admission's pressure feedback moves a domain's limit, never its budget, on evidence

{pytest_timing-0.2.0 → pytest_timing-0.3.1}/CHANGELOG.md

@@ -1,5 +1,54 @@
 # Changelog
 
+## 0.3.1
+
+- Memory admission no longer charges what session- and package-scoped fixtures keep
+  resident. Every worker sets them up once and never lets go, so counting them once
+  per lane, and again against every other lane, could only hold tests back or park
+  a worker until it was shut down: on a suite with a large session fixture the gate
+  left two of eight workers nearly idle and made the run almost three times longer
+  while the host had tens of gigabytes free. Their memory is now the worker's
+  baseline, which the budget's headroom covers (#7).
+- Memory estimates are what a test needs of its own. An attempt that set up shared
+  fixtures was charged its whole rise and, again, what the fixtures kept, twice the
+  residual; its own need is now the rise beyond what stayed. The first attempt on
+  each worker, whose window also covers the worker's warm-up, counts only for a
+  test or fixture with no other attempt (#7).
+- Waits say which gate held the test. A test the memory gate held back carries the
+  seconds in `memory.wait`, beside `cpu.wait` for CPU slots, and the summary line
+  reports memory waits even when a CPU budget is set. Time a worker sat parked and
+  then left without running what it waited for is reported per gate as well, instead
+  of vanishing. The HTML report's table gains a "Held" column and its hover details a
+  "held" line, and the terminal's slowest-tests rows say how long each was held (#7).
+
+## 0.3.0
+
+- Estimate a test's cost from the median of its best attempts instead of its
+  longest one. The longest attempt grows with the number of attempts, so a history
+  merged from several runs drifted upward and a flaky test's worst run was taken as
+  its cost. Attempts are ranked clean before contended and passing before failing,
+  and fixture set-up costs use the median too.
+
+- Memory-aware admission under pytest-xdist. `--timing-memory SIZE|auto` (ini
+  `timing_memory`, env `PYTEST_TIMING_MEMORY`) sets a memory budget per host, and the
+  memory each test needed in the run at `--timing-schedule` keeps tests apart whose
+  recorded needs would not fit in it together. Nothing is declared: the first run
+  records, the next one gates. Memory and CPU budgets are checked together, so a
+  test starts only when both fit. A shared fixture keeps what its first test left
+  resident reserved while it is alive. The run's JSON gains `memory` with the budget
+  and admission summary.
+- On macOS, CPU time and memory of live subprocesses are read through `libproc`
+  instead of psutil. psutil's child listing scans the whole process table, and the
+  CPU clock is read several times per test: with psutil installed, `--timing` on
+  3,000 trivial tests took over a minute instead of under a second. psutil is now
+  used only where no native reader covers subprocesses, which is Windows.
+- Record each test's resident memory. A sampler thread in the process running the
+  tests polls the resident set size of the worker (and, on Linux, macOS or with
+  psutil, its live subprocesses) while a test runs. Test JSON records now include `memory` when
+  the platform provides a reading: `base` before the test's set-up, `peak` during
+  it and `after` at its teardown, in bytes, with the measurement coverage. Nothing
+  schedules on it yet.
+
 ## 0.2.0
 
 - Raise the minimum supported pytest-xdist version to 3.7. Running without xdist

{pytest_timing-0.2.0 → pytest_timing-0.3.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: pytest-timing
-Version: 0.2.0
+Version: 0.3.1
 Summary: Record test timings under pytest-xdist and render cargo-style timing reports (ASCII Gantt and HTML).
 Project-URL: Homepage, https://github.com/messense/pytest-timing
 Author-email: messense <messense@icloud.com>
@@ -78,10 +78,11 @@ pip install "pytest-timing[xdist]"   # with pytest-xdist
 ```
 
 Python 3.10+ and pytest 7.3+. Distributed runs need pytest-xdist 3.7+.
-CPU records include live subprocesses on Linux through `/proc`, or through
-[psutil](https://pypi.org/project/psutil/) when installed. Without either, POSIX
-records include children the worker has waited for; Windows records cover only the
-worker itself. Each CPU record identifies its measurement coverage.
+CPU and memory records include live subprocesses on Linux through `/proc`, on macOS
+through `libproc`, and on Windows through [psutil](https://pypi.org/project/psutil/)
+when installed. Without psutil, Windows CPU records cover only the worker itself and
+memory records only the worker; other POSIX platforms count children the worker has
+waited for. Each record identifies its measurement coverage.
 
 ## Options
 
@@ -94,6 +95,7 @@ worker itself. Each CPU record identifies its measurement coverage.
 | `--timing-html-file PATH`, `--timing-json-file PATH`, `--timing-trace-file PATH` | Same, to an explicit path. |
 | `--timing-schedule PATH` | Under xdist, plan the workers' queues from the JSON run at `PATH`. |
 | `--timing-cpus N\|auto` | Under xdist, admit tests against a budget of `N` CPU slots per host (`auto` detects it). |
+| `--timing-memory SIZE\|auto` | Under xdist, keep tests whose recorded memory would not fit in `SIZE` together from running at once (`auto` takes 80% of the host's memory). |
 | `--timing-top=N` | Rows in the slowest-tests section (default 10, `0` hides it). |
 | `--timing-min=SECONDS` | Hide tests shorter than this from the slowest-tests section. |
 | `--timing-ascii-style=unicode\|ascii` | Chart glyphs. |
@@ -107,7 +109,8 @@ and schedule paths are resolved from pytest's root directory.
 
 The ini key `timing` is a boolean; `timing_html`, `timing_json` and `timing_trace`
 accept paths or `true` for the default filenames. Other ini keys are
-`timing_schedule`, `timing_cpus`, `timing_top`, `timing_min` and `timing_ascii_style`.
+`timing_schedule`, `timing_cpus`, `timing_memory`, `timing_top`, `timing_min` and
+`timing_ascii_style`.
 Environment variables use the uppercase key prefixed by `PYTEST_`, for example
 `PYTEST_TIMING=1`, `PYTEST_TIMING_HTML=path` or `PYTEST_TIMING_CPUS=auto`.
 `--timing-width` is command-line only.
@@ -120,7 +123,16 @@ environment, which wins over ini. Timing is enabled if any source requests it:
 
 **JSON** is the plugin's own record of the run: tests with their worker, phases and
 shared fixtures, plus CPU telemetry when available (elapsed time, measured CPU time,
-statically declared demand and time waiting for CPU slots). It also records worker
+statically declared demand and time waiting for CPU slots) and the resident memory
+of the worker's process tree around each test, in bytes: `base` before its set-up,
+`peak` during it, and `after` at its teardown. The difference between `peak` and
+`base` is what the test needed on top of the worker's existing footprint; `after`
+minus `base` is what stayed, such as a shared fixture it set up. With
+`--timing-memory`, the run's `memory` describes the budget and how admission went,
+and a test the memory gate held back carries the seconds it waited in `memory.wait`,
+as `cpu.wait` does for the CPU gate; the HTML report's table and hover details and
+the terminal's slowest-tests rows show them.
+It also records worker
 lifecycles, detected host CPU environments, and how the session ended (`finished`,
 `collect_only`, `interrupted`, `aborted`, `internal_error`, with pytest's reason). It is
 the input to the CLI below and to `--timing-schedule`.
@@ -259,8 +271,8 @@ What to expect:
   forces the oldest blocked request if needed. Such forced reservations can exceed
   the limit and are counted in the summary.
 - The summary reports the budget, the number of tests over one slot, and how long
-  tests waited for slots in total. Each test's JSON record carries its own wait.
-  `cpu.runtime_wait` records admission waits inside a running test. These waits
+  tests waited for slots in total. Each test's JSON record carries its own wait in
+  `cpu.wait`. `cpu.runtime_wait` records admission waits inside a running test. These waits
   are excluded from duration estimates, fixture setup costs and measured CPU rate.
   A runtime request that gets no grant within five minutes cancels the fixture
   setup. It waits to recover the test's original reservation before failing, so
@@ -280,6 +292,53 @@ domains with their own budgets. With `-x` or `--maxfail`, xdist shuts every work
 down; a test a worker was holding without having been admitted is withdrawn and
 never starts.
 
+## Keep memory-hungry tests apart
+
+Two tests that each need several gigabytes are fine on their own and fatal together:
+the kernel kills a worker, or the whole job. Nothing needs declaring for this. Every
+run records how much resident memory each test needed on top of its worker's footprint
+(`memory` in the JSON), and the next run keeps tests apart when their recorded needs
+would not fit in the budget at the same time:
+
+```
+pytest -n 4 --timing-json --timing-schedule pytest-timing.json --timing-memory 12G
+```
+
+`--timing-memory` takes a size with a binary suffix (`512M`, `12G`) or `auto`, which
+takes 80% of physical memory, capped by the cgroup memory limit. It reads the memory
+each test needed from the run at `--timing-schedule`, so the first run only records;
+the summary says how many tests had recorded memory. A test with no record weighs
+nothing until it has run once. A module- or class-scoped fixture keeps what its first
+test left resident (a loaded model, say) reserved for as long as it is alive, on that
+worker. What session- and package-scoped fixtures keep is treated as part of every
+worker's footprint instead: each worker sets them up once and never lets go, so
+gating on them could only delay the run, never spare the host.
+
+What to expect:
+
+- The gate is the same fair waiting line as for CPU: a worker whose next test does
+  not fit waits with its fixtures alive, and the oldest request goes first. Memory
+  and CPU budgets are checked together; a test starts only when both fit.
+- Estimates are the largest need any recorded attempt showed. For an attempt that
+  set up shared fixtures, what stayed resident afterwards is the fixtures' (or the
+  worker's footprint), and the test's own need is what it used beyond that. The
+  first test on each worker also pays for the worker's warm-up, so its record is
+  used only for a test or fixture with no other attempt. Estimates are relative to
+  the worker's footprint, so `auto` leaves a fifth of the host for the workers
+  themselves, their session fixtures, the controller and everything else; set a
+  smaller budget on a shared machine, and a larger share of headroom when session
+  fixtures are big.
+- A test recorded above the budget runs alone, and the summary counts it. Allocator
+  behaviour can make an estimate low: a test that reuses heap an earlier test freed
+  shows a smaller rise than it needs. Large buffers and subprocesses, the usual
+  cause of an out-of-memory kill, measure well.
+- Memory does not enter the planner: lanes are still balanced by duration and fixture
+  cost, and the gate only delays starts. Under `--dist load` and `worksteal`, like
+  CPU admission.
+- The summary says how many tests waited for memory and for how long, and how long
+  workers sat parked at the gate without running what they waited for. Each held
+  test's JSON record carries its wait in `memory.wait`.
+
 ## Re-render or merge saved runs
 
 ```

{pytest_timing-0.2.0 → pytest_timing-0.3.1}/README.md

@@ -52,10 +52,11 @@ pip install "pytest-timing[xdist]"   # with pytest-xdist
 ```
 
 Python 3.10+ and pytest 7.3+. Distributed runs need pytest-xdist 3.7+.
-CPU records include live subprocesses on Linux through `/proc`, or through
-[psutil](https://pypi.org/project/psutil/) when installed. Without either, POSIX
-records include children the worker has waited for; Windows records cover only the
-worker itself. Each CPU record identifies its measurement coverage.
+CPU and memory records include live subprocesses on Linux through `/proc`, on macOS
+through `libproc`, and on Windows through [psutil](https://pypi.org/project/psutil/)
+when installed. Without psutil, Windows CPU records cover only the worker itself and
+memory records only the worker; other POSIX platforms count children the worker has
+waited for. Each record identifies its measurement coverage.
 
 ## Options
 
@@ -68,6 +69,7 @@ worker itself. Each CPU record identifies its measurement coverage.
 | `--timing-html-file PATH`, `--timing-json-file PATH`, `--timing-trace-file PATH` | Same, to an explicit path. |
 | `--timing-schedule PATH` | Under xdist, plan the workers' queues from the JSON run at `PATH`. |
 | `--timing-cpus N\|auto` | Under xdist, admit tests against a budget of `N` CPU slots per host (`auto` detects it). |
+| `--timing-memory SIZE\|auto` | Under xdist, keep tests whose recorded memory would not fit in `SIZE` together from running at once (`auto` takes 80% of the host's memory). |
 | `--timing-top=N` | Rows in the slowest-tests section (default 10, `0` hides it). |
 | `--timing-min=SECONDS` | Hide tests shorter than this from the slowest-tests section. |
 | `--timing-ascii-style=unicode\|ascii` | Chart glyphs. |
@@ -81,7 +83,8 @@ and schedule paths are resolved from pytest's root directory.
 
 The ini key `timing` is a boolean; `timing_html`, `timing_json` and `timing_trace`
 accept paths or `true` for the default filenames. Other ini keys are
-`timing_schedule`, `timing_cpus`, `timing_top`, `timing_min` and `timing_ascii_style`.
+`timing_schedule`, `timing_cpus`, `timing_memory`, `timing_top`, `timing_min` and
+`timing_ascii_style`.
 Environment variables use the uppercase key prefixed by `PYTEST_`, for example
 `PYTEST_TIMING=1`, `PYTEST_TIMING_HTML=path` or `PYTEST_TIMING_CPUS=auto`.
 `--timing-width` is command-line only.
@@ -94,7 +97,16 @@ environment, which wins over ini. Timing is enabled if any source requests it:
 
 **JSON** is the plugin's own record of the run: tests with their worker, phases and
 shared fixtures, plus CPU telemetry when available (elapsed time, measured CPU time,
-statically declared demand and time waiting for CPU slots). It also records worker
+statically declared demand and time waiting for CPU slots) and the resident memory
+of the worker's process tree around each test, in bytes: `base` before its set-up,
+`peak` during it, and `after` at its teardown. The difference between `peak` and
+`base` is what the test needed on top of the worker's existing footprint; `after`
+minus `base` is what stayed, such as a shared fixture it set up. With
+`--timing-memory`, the run's `memory` describes the budget and how admission went,
+and a test the memory gate held back carries the seconds it waited in `memory.wait`,
+as `cpu.wait` does for the CPU gate; the HTML report's table and hover details and
+the terminal's slowest-tests rows show them.
+It also records worker
 lifecycles, detected host CPU environments, and how the session ended (`finished`,
 `collect_only`, `interrupted`, `aborted`, `internal_error`, with pytest's reason). It is
 the input to the CLI below and to `--timing-schedule`.
@@ -233,8 +245,8 @@ What to expect:
   forces the oldest blocked request if needed. Such forced reservations can exceed
   the limit and are counted in the summary.
 - The summary reports the budget, the number of tests over one slot, and how long
-  tests waited for slots in total. Each test's JSON record carries its own wait.
-  `cpu.runtime_wait` records admission waits inside a running test. These waits
+  tests waited for slots in total. Each test's JSON record carries its own wait in
+  `cpu.wait`. `cpu.runtime_wait` records admission waits inside a running test. These waits
   are excluded from duration estimates, fixture setup costs and measured CPU rate.
   A runtime request that gets no grant within five minutes cancels the fixture
   setup. It waits to recover the test's original reservation before failing, so
@@ -254,6 +266,53 @@ domains with their own budgets. With `-x` or `--maxfail`, xdist shuts every work
 down; a test a worker was holding without having been admitted is withdrawn and
 never starts.
 
+## Keep memory-hungry tests apart
+
+Two tests that each need several gigabytes are fine on their own and fatal together:
+the kernel kills a worker, or the whole job. Nothing needs declaring for this. Every
+run records how much resident memory each test needed on top of its worker's footprint
+(`memory` in the JSON), and the next run keeps tests apart when their recorded needs
+would not fit in the budget at the same time:
+
+```
+pytest -n 4 --timing-json --timing-schedule pytest-timing.json --timing-memory 12G
+```
+
+`--timing-memory` takes a size with a binary suffix (`512M`, `12G`) or `auto`, which
+takes 80% of physical memory, capped by the cgroup memory limit. It reads the memory
+each test needed from the run at `--timing-schedule`, so the first run only records;
+the summary says how many tests had recorded memory. A test with no record weighs
+nothing until it has run once. A module- or class-scoped fixture keeps what its first
+test left resident (a loaded model, say) reserved for as long as it is alive, on that
+worker. What session- and package-scoped fixtures keep is treated as part of every
+worker's footprint instead: each worker sets them up once and never lets go, so
+gating on them could only delay the run, never spare the host.
+
+What to expect:
+
+- The gate is the same fair waiting line as for CPU: a worker whose next test does
+  not fit waits with its fixtures alive, and the oldest request goes first. Memory
+  and CPU budgets are checked together; a test starts only when both fit.
+- Estimates are the largest need any recorded attempt showed. For an attempt that
+  set up shared fixtures, what stayed resident afterwards is the fixtures' (or the
+  worker's footprint), and the test's own need is what it used beyond that. The
+  first test on each worker also pays for the worker's warm-up, so its record is
+  used only for a test or fixture with no other attempt. Estimates are relative to
+  the worker's footprint, so `auto` leaves a fifth of the host for the workers
+  themselves, their session fixtures, the controller and everything else; set a
+  smaller budget on a shared machine, and a larger share of headroom when session
+  fixtures are big.
+- A test recorded above the budget runs alone, and the summary counts it. Allocator
+  behaviour can make an estimate low: a test that reuses heap an earlier test freed
+  shows a smaller rise than it needs. Large buffers and subprocesses, the usual
+  cause of an out-of-memory kill, measure well.
+- Memory does not enter the planner: lanes are still balanced by duration and fixture
+  cost, and the gate only delays starts. Under `--dist load` and `worksteal`, like
+  CPU admission.
+- The summary says how many tests waited for memory and for how long, and how long
+  workers sat parked at the gate without running what they waited for. Each held
+  test's JSON record carries its wait in `memory.wait`.
+
 ## Re-render or merge saved runs
 
 ```

{pytest_timing-0.2.0 → pytest_timing-0.3.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "pytest-timing"
-version = "0.2.0"
+version = "0.3.1"
 description = "Record test timings under pytest-xdist and render cargo-style timing reports (ASCII Gantt and HTML)."
 readme = "README.md"
 license = "MIT"

{pytest_timing-0.2.0 → pytest_timing-0.3.1}/src/pytest_timing/__init__.py

@@ -4,6 +4,6 @@ from __future__ import annotations
 
 from pytest_timing.demand import cpu
 
-__version__ = "0.2.0"
+__version__ = "0.3.1"
 
 __all__ = ["__version__", "cpu"]

{pytest_timing-0.2.0 → pytest_timing-0.3.1}/src/pytest_timing/admission.py

@@ -40,8 +40,9 @@ class Admission:
     RECOVER_AFTER = 8
     COOLDOWN = 10.0
 
-    def __init__(self, budget: int, name: str = "local") -> None:
+    def __init__(self, budget: int, name: str = "local", kind: str = "cpu") -> None:
         self.name = name
+        self.kind = kind  # what the slots are: ``cpu`` or ``memory`` (bytes)
         self.budget = max(1, int(budget))
         self.limit = self.budget
         self.reserved: dict[Hashable, Reservation] = {}

{pytest_timing-0.2.0 → pytest_timing-0.3.1}/src/pytest_timing/collector.py

@@ -14,10 +14,20 @@ correct span; they do not change that public occurrence/attempt numbering.
 
 from __future__ import annotations
 
+from collections.abc import Iterable
 from dataclasses import dataclass
 from typing import Any
 
-from pytest_timing.model import PHASES, CpuRecord, Phase, Run, RunInfo, TestSpan, Worker
+from pytest_timing.model import (
+    PHASES,
+    CpuRecord,
+    MemoryRecord,
+    Phase,
+    Run,
+    RunInfo,
+    TestSpan,
+    Worker,
+)
 
 CRASH_WHEN = "???"  # xdist synthesises a report with this ``when`` for crashed items
 RETRIED = "rerun"
@@ -38,6 +48,7 @@ class PhaseReport:
     received: float = 0.0  # epoch seconds when the controller saw the report
     fixtures: dict[str, float | None] | None = None  # shared fixtures, setup and call
     cpu: dict[str, Any] | None = None  # the worker's CPU record, on the teardown report
+    memory: dict[str, Any] | None = None  # the worker's memory record, likewise
     execution: tuple[int, int] | None = None  # collection index, attempt on that worker
 
 
@@ -73,12 +84,19 @@ class Collector:
         worker.items = items
 
     def add_wait(
-        self, worker_id: str, nodeid: str, index: int, attempt: int, seconds: float
+        self,
+        worker_id: str,
+        nodeid: str,
+        index: int,
+        attempt: int,
+        seconds: float,
+        gates: Iterable[str] = (),
     ) -> None:
         """Add an admission delay to the exact execution that waited.
 
         A nodeid can have several selections and retries. Looking up its last span
-        at session finish would put every delay on the final one instead.
+        at session finish would put every delay on the final one instead. ``gates``
+        names what held the test (``cpu``, ``memory``); unnamed, the CPU gate did.
         """
         span = self._executions.get((worker_id, index, attempt))
         if span is None:
@@ -88,9 +106,15 @@ class Collector:
             span = self._last.get((worker_id, nodeid))
             if span is None or span.outcome != "crashed" or span.phases or span.attempt != attempt:
                 return
-        if span.cpu is None:
-            span.cpu = CpuRecord(elapsed=span.duration)
-        span.cpu.wait += seconds
+        gates = set(gates)
+        if "cpu" in gates or not gates:
+            if span.cpu is None:
+                span.cpu = CpuRecord(elapsed=span.duration)
+            span.cpu.wait += seconds
+        if "memory" in gates:
+            if span.memory is None:
+                span.memory = MemoryRecord()  # no reading, but the wait is a fact
+            span.memory.wait += seconds
 
     def worker_down(self, worker_id: str, epoch: float, error: str | None) -> None:
         worker = self._worker(worker_id)
@@ -131,6 +155,8 @@ class Collector:
             span.fixtures.update(report.fixtures)
         if report.cpu:
             span.cpu = CpuRecord.from_dict(report.cpu)
+        if report.memory:
+            span.memory = MemoryRecord.from_dict(report.memory)
         span.start = min(span.start, start)
         span.stop = max(span.stop, stop)
         self._apply_outcome(span, report)

{pytest_timing-0.2.0 → pytest_timing-0.3.1}/src/pytest_timing/demand.py

@@ -1,4 +1,4 @@
-"""CPU declarations and per-test measurement in the process running the tests.
+"""CPU declarations and per-test CPU and memory measurement where the tests run.
 
 Declarations precede xdist collection reports. Dynamic fixture setup requests
 reserve slots before execution; holds follow actual setup/finalization events.
@@ -16,11 +16,13 @@ import pytest
 
 from pytest_timing import xdist_compat
 from pytest_timing.fixtures import FixtureTimer, _key_for, defined_at, fixture_key, fixturedefs
-from pytest_timing.telemetry import Pressure, ProcessTreeClock, host_cpu
+from pytest_timing.telemetry import MemorySampler, Pressure, ProcessTreeClock, host_cpu
 
 MARKER = "timing_cpu"
 FIXTURE_ATTR = "_pytest_timing_cpu"
 REPORT_ATTR = "timing_cpu"
+MEMORY_ATTR = "timing_memory"
+"""Report attribute carrying the attempt's resident-memory window, on teardown."""
 EXECUTION_ATTR = "timing_execution"
 """Private report identity: collection index and attempt, including retries."""
 EVENT = "timing_cpu"
@@ -237,7 +239,7 @@ class Declarations:
 
 
 class CpuMeter:
-    """Measures each test's CPU work and attaches it to the teardown report.
+    """Measures each test's CPU work and memory and attaches them to the teardown report.
 
     Runs wherever tests run: in every xdist worker, or in the main process without
     xdist. After collection it also sends the :class:`Declarations` to the controller
@@ -255,6 +257,9 @@ class CpuMeter:
     set-up's slots (``timing_request``) before the set-up starts and blocks until
     they are granted (``timing_grant``), so such a set-up goes through the same
     gate as a declared one.
+
+    ``memory`` samples resident memory from set-up to the teardown report; the
+    window (``timing_memory``) goes on that report next to the CPU record.
     """
 
     def __init__(
@@ -263,11 +268,13 @@ class CpuMeter:
         send: Callable[[str, dict[str, Any]], None] | None = None,
         clock: ProcessTreeClock | None = None,
         pressure: Pressure | None = None,
+        memory: MemorySampler | None = None,
     ) -> None:
         self.timer = timer
         self.send = send
         self.clock = clock or ProcessTreeClock()
         self.pressure = pressure or Pressure()
+        self.memory = memory
         self.declarations: Declarations | None = None
         self.cancelled: set[int] = set()  # item indices withdrawn by the controller
         self._live = False  # can the controller reach this worker while a test runs
@@ -427,6 +434,8 @@ class CpuMeter:
         self._started = time.perf_counter()
         self._work = self.clock.seconds()
         self._throttled = self.pressure.throttled()
+        if self.memory is not None:
+            self.memory.begin()
 
     @pytest.hookimpl(hookwrapper=True)
     def pytest_runtest_makereport(self, item: pytest.Item, call: Any) -> Generator[None, Any, None]:
@@ -436,6 +445,18 @@ class CpuMeter:
             setattr(outcome.get_result(), EXECUTION_ATTR, (index, self._attempts.get(index, 0)))
         if call.when != "teardown" or not self._started:
             return
+        window = self.memory.end() if self.memory is not None else None
+        if window is not None:
+            setattr(
+                outcome.get_result(),
+                MEMORY_ATTR,
+                {
+                    "base": window.base,
+                    "peak": window.peak,
+                    "after": window.after,
+                    "coverage": window.coverage,
+                },
+            )
         elapsed = max(0.0, time.perf_counter() - self._started - self.timer.wait)
         work = self.clock.seconds() - self._work - self.timer.wait_work
         throttled_now = self.pressure.throttled()
@@ -456,3 +477,7 @@ class CpuMeter:
         }
         setattr(outcome.get_result(), REPORT_ATTR, record)
         self._started = 0.0
+
+    def pytest_unconfigure(self) -> None:
+        if self.memory is not None:
+            self.memory.close()