pytest-timing 0.3.1__tar.gz → 0.4.0__tar.gz

{pytest_timing-0.3.1 → pytest_timing-0.4.0}/ARCHITECTURE.md

@@ -11,9 +11,10 @@ schedules a run from the previous one. For usage, see the [README](README.md).
 | `collector.py` | Folds phase reports and worker events into a `Run`. Knows nothing about pytest objects. |
 | `model.py` | The recorded run: `Run`, `Worker`, `TestSpan`, `Phase`, and the JSON representation. |
 | `outputs.py` | The ASCII, HTML and trace renderers, in one table shared by the plugin and the CLI. |
-| `cli.py` | `pytest-timing render` and `pytest-timing merge`. |
+| `cli.py` | `pytest-timing render`, `merge` and `compare`. |
 | `fixtures.py` | Times shared fixture set-up where tests run and lists dependencies on reports. |
 | `schedule.py` | Duration estimates, the fixture-aware cost model and the planner. Free of xdist. |
+| `compare.py` | Saved-run comparison, final-attempt aggregation and per-test CI budgets. Free of pytest hooks and output I/O. |
 | `demand.py` | Declared CPU demand: the `timing_cpu` marker, the fixture decorator, the declarations a worker sends, and the meter that records each test's CPU work and resident memory. |
 | `admission.py` | One host's CPU budget, reservations, fair waiting line and pressure feedback. Free of xdist and platform reads. |
 | `telemetry.py` | Platform reads: affinity and cgroup quota, CPU time and resident memory of a process tree, pressure and throttling. |
@@ -97,6 +98,14 @@ A missing or unreadable history file leaves xdist's scheduler in place unless an
 explicit CPU budget was requested. With that budget the custom scheduler still runs,
 using equal 1 ms test estimates. It supports only `load` and `worksteal`.
 
+## Saved-run comparisons
+
+Saved-run comparisons use a separate policy from scheduling estimates: compare
+the median of final attempts, preserving missing metrics and rejecting incomplete
+budget checks. Scheduling may choose uncontended historical observations instead.
+Keeping these policies separate avoids treating a failed or missing observation as
+a successful CI check merely because it was usable for queue planning.
+
 ## Cost model
 
 A shared fixture makes the cost of a test depend on the worker. Every test has a
@@ -118,10 +127,16 @@ CPU work, peak demand and fixture transition. `Costs.project` folds these into a
 
 The scheduler keeps one `Charge` per dispatched item. It records the holds already
 included in the peak, so actual live holds supplement it without double counting.
+It also records each fixture's contribution to CPU holds and retained memory.
+Runtime admission previews and commits the same reconciliation: a fixture already
+covered by history adds only a missing contribution, never its full cost again.
 Charged duration, CPU work and the pre-dispatch fixture checkpoint remain historical
 facts even when a runtime request changes the reservation. Stealing refunds the same
 record and restores that checkpoint.
 Ordinary transfers and steals compare finish times with the same CPU-work floor.
+After placement, a stable module/class grouping is also tried within each worker's
+assignment. It replaces exact-family ordering only when `Costs.project` predicts
+less lane time and no more CPU work, so parameter locality can still win.
 
 ## Planning
 
@@ -364,6 +379,11 @@ as `runtime_wait`. All open fixture clocks pause during that wait, and test esti
 subtract it along with shared setup. CPU `elapsed` and `work` exclude the wait and
 its process-tree CPU work, so the measured rate covers execution. Older records
 without `runtime_wait` default to zero; pre-start waits are not subtracted again.
+The controller also attaches each wait's end epoch and monotonic duration as an
+optional `TestSpan.admission_waits` interval on the run axis. Model rebasing moves
+these intervals with phases and test spans. The HTML report displays them once
+even when both gates refused the same interval. Older totals without intervals stay
+unlocated; parked workers without an executed item are only in gate summaries.
 Event handlers are registered before each worker starts, even when another plugin
 selects the scheduler: without an admission gate a request is granted immediately.
 The cgroup quota behind `auto` is read from the process's own group, walking up to

{pytest_timing-0.3.1 → pytest_timing-0.4.0}/CHANGELOG.md

@@ -1,5 +1,38 @@
 # Changelog
 
+## 0.4.0
+
+- Keep slowest-test labels visible at the end of the timeline and wrap long labels
+  below their bars when neither side has room.
+- Add `pytest-timing compare` with per-test duration, phase, CPU and memory changes,
+  optional CI regression budgets, machine-readable JSON and explicit unavailable checks.
+- Add shared fixture costs by worker and linked time-range selection in HTML reports.
+  Zooming and scrolling resample the visible interval for finer detail.
+- Record admission wait intervals on their test attempts, preserve them through
+  merging and show them in lanes, a wait graph and range statistics.
+- Add `--timing-capture=light` to skip optional CPU and memory measurements while
+  retaining durations and resource admission; the default remains full capture.
+- Bound HTML chart samples and ticks independently of zoom, retaining peak values
+  over each bucket; cache merged-test totals and top-ten hover details.
+- Try module/class-contiguous ordering across overlapping fixture families without
+  changing worker assignments or increasing modeled CPU work.
+- Trace-only output no longer builds an unused JSON report document.
+- Runtime fixture admission no longer reserves a dynamic fixture's held CPU slots
+  or retained memory twice when it was already included in scheduling history.
+- The HTML CPU graph spreads recorded CPU time over the full test span, including
+  runtime admission waits, so the graph preserves the recorded total. Test details
+  continue to show average CPU use during execution, with those waits excluded.
+
+## 0.3.2
+
+- The HTML report shows CPU and memory usage. The table gains "CPU time", "CPUs"
+  (the CPUs a test kept busy on average, beside the slots it declared) and "Memory"
+  (what it needed on top of its worker's footprint) columns, the hover details say
+  the same with what the measurement covered, the header totals the run and names
+  each host's budget, and two graphs show CPUs busy and resident memory over time,
+  estimated from the per-test records. All of it appears only where there was a
+  reading.
+
 ## 0.3.1
 
 - Memory admission no longer charges what session- and package-scoped fixtures keep

{pytest_timing-0.3.1 → pytest_timing-0.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: pytest-timing
-Version: 0.3.1
+Version: 0.4.0
 Summary: Record test timings under pytest-xdist and render cargo-style timing reports (ASCII Gantt and HTML).
 Project-URL: Homepage, https://github.com/messense/pytest-timing
 Author-email: messense <messense@icloud.com>
@@ -89,6 +89,7 @@ waited for. Each record identifies its measurement coverage.
 | Option | Effect |
 |---|---|
 | `--timing` | Record timings and print the ASCII chart in the terminal summary. |
+| `--timing-capture=full\|light` | Full metrics (default), or timings without CPU/memory measurement. Implies timing. |
 | `--timing-html` | Write a self-contained HTML report to `pytest-timing.html`. |
 | `--timing-json` | Write the recorded run to `pytest-timing.json`. |
 | `--timing-trace` | Write a Chrome trace file for [Perfetto](https://ui.perfetto.dev) to `pytest-timing.trace.json`. |
@@ -109,7 +110,7 @@ and schedule paths are resolved from pytest's root directory.
 
 The ini key `timing` is a boolean; `timing_html`, `timing_json` and `timing_trace`
 accept paths or `true` for the default filenames. Other ini keys are
-`timing_schedule`, `timing_cpus`, `timing_memory`, `timing_top`, `timing_min` and
+`timing_capture`, `timing_schedule`, `timing_cpus`, `timing_memory`, `timing_top`, `timing_min` and
 `timing_ascii_style`.
 Environment variables use the uppercase key prefixed by `PYTEST_`, for example
 `PYTEST_TIMING=1`, `PYTEST_TIMING_HTML=path` or `PYTEST_TIMING_CPUS=auto`.
@@ -119,6 +120,16 @@ For output paths, scheduling and formatting, the command line wins over the
 environment, which wins over ini. Timing is enabled if any source requests it:
 `PYTEST_TIMING=0` does not disable `timing=true` in ini or an enabled output.
 
+Use `--timing-capture=light` (or `PYTEST_TIMING_CAPTURE=light`, or
+`timing_capture = light` in ini) for timing-only collection. It keeps phase and
+shared-fixture durations, worker lifecycle and admission waits, but does not create
+a process-tree CPU reader or memory sampler. CPU work/rate, pressure and resident
+memory measurements are unavailable; wait-only records have unknown measurement
+coverage, rather than a measured zero. Declared CPU admission and memory admission
+from an existing full report still apply. Capture stays `full` unless explicitly
+changed; use full capture when refreshing resource estimates for future scheduling.
+Capture mode names are case-insensitive in CLI, environment and ini settings.
+
 ## Outputs
 
 **JSON** is the plugin's own record of the run: tests with their worker, phases and
@@ -138,7 +149,17 @@ lifecycles, detected host CPU environments, and how the session ended (`finished
 the input to the CLI below and to `--timing-schedule`.
 
 **HTML** is a single file with no external dependencies, so it can be attached to a CI
-job as an artifact and opened anywhere.
+job as an artifact and opened anywhere. Where CPU and memory were measured it shows
+them: each test's CPU time, the CPUs it kept busy on average (beside the slots it
+declared) and the memory it needed on top of its worker's footprint, in the table and
+the hover details; the run's totals and budgets in the header; and CPU and memory
+usage over time. The two graphs are drawn from the per-test records, not sampled: a
+test's CPU time is spread evenly over its run, and a worker counts at its running
+test's peak, so they show where the load was rather than its exact shape.
+Each curve uses at most 4096 display samples, even at deep zoom. A sample keeps
+the maximum over its time bucket; hover reports that bucket's time range. This
+bounds display memory without dropping short peaks. Merged tiny-test bars cache
+their total duration and ten longest tests for repeated hover.
 
 **Trace** is a Chrome Trace Event file. Open it in
 [Perfetto UI](https://ui.perfetto.dev) with "Open trace file", or in `chrome://tracing`.
@@ -349,6 +370,58 @@ pytest-timing merge shard1.json shard2.json -o all.json
 `render` produces any of the outputs from a saved JSON run. `merge` places several runs
 (for example CI shards) on one shared time axis using their absolute start times.
 
+## Compare runs and enforce CI budgets
+
+```
+pytest-timing compare before.json after.json
+pytest-timing compare before.json after.json --budget duration=10% --budget memory=64MiB --json comparison.json
+```
+
+Comparisons match tests by node ID and show total, setup, call and teardown time,
+CPU time, and peak memory rise above the worker's baseline. For each run, only the
+final retry of each worker/occurrence is used, then repeated observations are
+combined with a median. The output includes retry counts and added/removed tests.
+Missing measurements stay unavailable, including partial coverage across repeats.
+
+A budget sets the maximum increase **per common test**. Percentages are relative
+to the baseline; a zero baseline permits no increase under a percentage budget.
+Time limits use seconds or `ms`; memory limits use bytes, `KiB`, `MiB` or `GiB`.
+Each metric accepts one budget. Added and removed tests are listed but not budgeted.
+Exit codes are `0` for success (or a comparison without budgets), `1` for exceeded
+budgets, and `2` for invalid input or unavailable checks. Budget checks need complete
+runs, at least one common test, successful final attempts and every requested metric.
+Differences in Python, pytest, worker count or distribution mode produce warnings;
+use comparable environments and repeat noisy workloads before choosing thresholds.
+
+## Explore the HTML report
+
+The scale control zooms all timelines together. Scrolling any timeline moves the
+others to the same time. Curves are resampled from the visible window, so a narrow
+peak gains detail as you zoom in, with at most 4,096 samples per curve. Hover values
+are the maximum within the displayed bucket.
+
+Enter start and end times in seconds and select **Select range** to fit that interval
+across lanes, concurrency, admission waits, CPU and memory. The test and fixture
+tables follow the selection and existing filters. **Clear range** returns to the
+whole run. The range summary covers all tests, independently of text/outcome/fixture
+filters; CPU totals integrate the original estimated curve, never its peak buckets.
+CPU time is spread evenly over each test span, while the memory curve places the
+recorded peak over that span; neither is a sampled execution trace.
+
+The shared fixture table groups recorded setup costs by fixture key and worker,
+showing repetition across tests and workers. Select a fixture to see its associated
+tests and use **Clear fixture filter** to restore them. Costs belong to overlapping
+tests: fixture setup timestamps are not recorded, so costs cannot be clipped to a
+selected interval. A `None` fixture value means reuse and contributes no setup.
+
+New reports include controller-observed `admission_waits` intervals on the exact
+test attempt that waited. Lane strips and the wait graph display those intervals;
+one interval refused by both CPU and memory counts once in the graph, summary and
+Held column (its hover still lists both gates).
+Older JSON files remain readable, but their wait totals cannot be placed on a
+timeline. Workers parked without ever running a test remain in the run-level gate
+summary rather than being assigned to an unrelated test.
+
 ## Overhead
 
 Timing collection also measures fixtures and CPU work in each worker. Its overhead

{pytest_timing-0.3.1 → pytest_timing-0.4.0}/README.md

@@ -63,6 +63,7 @@ waited for. Each record identifies its measurement coverage.
 | Option | Effect |
 |---|---|
 | `--timing` | Record timings and print the ASCII chart in the terminal summary. |
+| `--timing-capture=full\|light` | Full metrics (default), or timings without CPU/memory measurement. Implies timing. |
 | `--timing-html` | Write a self-contained HTML report to `pytest-timing.html`. |
 | `--timing-json` | Write the recorded run to `pytest-timing.json`. |
 | `--timing-trace` | Write a Chrome trace file for [Perfetto](https://ui.perfetto.dev) to `pytest-timing.trace.json`. |
@@ -83,7 +84,7 @@ and schedule paths are resolved from pytest's root directory.
 
 The ini key `timing` is a boolean; `timing_html`, `timing_json` and `timing_trace`
 accept paths or `true` for the default filenames. Other ini keys are
-`timing_schedule`, `timing_cpus`, `timing_memory`, `timing_top`, `timing_min` and
+`timing_capture`, `timing_schedule`, `timing_cpus`, `timing_memory`, `timing_top`, `timing_min` and
 `timing_ascii_style`.
 Environment variables use the uppercase key prefixed by `PYTEST_`, for example
 `PYTEST_TIMING=1`, `PYTEST_TIMING_HTML=path` or `PYTEST_TIMING_CPUS=auto`.
@@ -93,6 +94,16 @@ For output paths, scheduling and formatting, the command line wins over the
 environment, which wins over ini. Timing is enabled if any source requests it:
 `PYTEST_TIMING=0` does not disable `timing=true` in ini or an enabled output.
 
+Use `--timing-capture=light` (or `PYTEST_TIMING_CAPTURE=light`, or
+`timing_capture = light` in ini) for timing-only collection. It keeps phase and
+shared-fixture durations, worker lifecycle and admission waits, but does not create
+a process-tree CPU reader or memory sampler. CPU work/rate, pressure and resident
+memory measurements are unavailable; wait-only records have unknown measurement
+coverage, rather than a measured zero. Declared CPU admission and memory admission
+from an existing full report still apply. Capture stays `full` unless explicitly
+changed; use full capture when refreshing resource estimates for future scheduling.
+Capture mode names are case-insensitive in CLI, environment and ini settings.
+
 ## Outputs
 
 **JSON** is the plugin's own record of the run: tests with their worker, phases and
@@ -112,7 +123,17 @@ lifecycles, detected host CPU environments, and how the session ended (`finished
 the input to the CLI below and to `--timing-schedule`.
 
 **HTML** is a single file with no external dependencies, so it can be attached to a CI
-job as an artifact and opened anywhere.
+job as an artifact and opened anywhere. Where CPU and memory were measured it shows
+them: each test's CPU time, the CPUs it kept busy on average (beside the slots it
+declared) and the memory it needed on top of its worker's footprint, in the table and
+the hover details; the run's totals and budgets in the header; and CPU and memory
+usage over time. The two graphs are drawn from the per-test records, not sampled: a
+test's CPU time is spread evenly over its run, and a worker counts at its running
+test's peak, so they show where the load was rather than its exact shape.
+Each curve uses at most 4096 display samples, even at deep zoom. A sample keeps
+the maximum over its time bucket; hover reports that bucket's time range. This
+bounds display memory without dropping short peaks. Merged tiny-test bars cache
+their total duration and ten longest tests for repeated hover.
 
 **Trace** is a Chrome Trace Event file. Open it in
 [Perfetto UI](https://ui.perfetto.dev) with "Open trace file", or in `chrome://tracing`.
@@ -323,6 +344,58 @@ pytest-timing merge shard1.json shard2.json -o all.json
 `render` produces any of the outputs from a saved JSON run. `merge` places several runs
 (for example CI shards) on one shared time axis using their absolute start times.
 
+## Compare runs and enforce CI budgets
+
+```
+pytest-timing compare before.json after.json
+pytest-timing compare before.json after.json --budget duration=10% --budget memory=64MiB --json comparison.json
+```
+
+Comparisons match tests by node ID and show total, setup, call and teardown time,
+CPU time, and peak memory rise above the worker's baseline. For each run, only the
+final retry of each worker/occurrence is used, then repeated observations are
+combined with a median. The output includes retry counts and added/removed tests.
+Missing measurements stay unavailable, including partial coverage across repeats.
+
+A budget sets the maximum increase **per common test**. Percentages are relative
+to the baseline; a zero baseline permits no increase under a percentage budget.
+Time limits use seconds or `ms`; memory limits use bytes, `KiB`, `MiB` or `GiB`.
+Each metric accepts one budget. Added and removed tests are listed but not budgeted.
+Exit codes are `0` for success (or a comparison without budgets), `1` for exceeded
+budgets, and `2` for invalid input or unavailable checks. Budget checks need complete
+runs, at least one common test, successful final attempts and every requested metric.
+Differences in Python, pytest, worker count or distribution mode produce warnings;
+use comparable environments and repeat noisy workloads before choosing thresholds.
+
+## Explore the HTML report
+
+The scale control zooms all timelines together. Scrolling any timeline moves the
+others to the same time. Curves are resampled from the visible window, so a narrow
+peak gains detail as you zoom in, with at most 4,096 samples per curve. Hover values
+are the maximum within the displayed bucket.
+
+Enter start and end times in seconds and select **Select range** to fit that interval
+across lanes, concurrency, admission waits, CPU and memory. The test and fixture
+tables follow the selection and existing filters. **Clear range** returns to the
+whole run. The range summary covers all tests, independently of text/outcome/fixture
+filters; CPU totals integrate the original estimated curve, never its peak buckets.
+CPU time is spread evenly over each test span, while the memory curve places the
+recorded peak over that span; neither is a sampled execution trace.
+
+The shared fixture table groups recorded setup costs by fixture key and worker,
+showing repetition across tests and workers. Select a fixture to see its associated
+tests and use **Clear fixture filter** to restore them. Costs belong to overlapping
+tests: fixture setup timestamps are not recorded, so costs cannot be clipped to a
+selected interval. A `None` fixture value means reuse and contributes no setup.
+
+New reports include controller-observed `admission_waits` intervals on the exact
+test attempt that waited. Lane strips and the wait graph display those intervals;
+one interval refused by both CPU and memory counts once in the graph, summary and
+Held column (its hover still lists both gates).
+Older JSON files remain readable, but their wait totals cannot be placed on a
+timeline. Workers parked without ever running a test remain in the run-level gate
+summary rather than being assigned to an unrelated test.
+
 ## Overhead
 
 Timing collection also measures fixtures and CPU work in each worker. Its overhead

{pytest_timing-0.3.1 → pytest_timing-0.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "pytest-timing"
-version = "0.3.1"
+version = "0.4.0"
 description = "Record test timings under pytest-xdist and render cargo-style timing reports (ASCII Gantt and HTML)."
 readme = "README.md"
 license = "MIT"

{pytest_timing-0.3.1 → pytest_timing-0.4.0}/src/pytest_timing/__init__.py

@@ -4,6 +4,6 @@ from __future__ import annotations
 
 from pytest_timing.demand import cpu
 
-__version__ = "0.3.1"
+__version__ = "0.4.0"
 
 __all__ = ["__version__", "cpu"]

{pytest_timing-0.3.1 → pytest_timing-0.4.0}/src/pytest_timing/cli.py

@@ -1,18 +1,106 @@
-"""``pytest-timing`` command line: re-render or merge saved JSON runs."""
+"""``pytest-timing`` command line: render, merge or compare saved JSON runs."""
 
 from __future__ import annotations
 
 import argparse
+import json
+import math
 import sys
 from pathlib import Path
+from typing import Any
 
+from pytest_timing.compare import compare_runs, format_comparison, parse_budget
 from pytest_timing.model import Run, RunInfo, TestSpan, Worker
 from pytest_timing.outputs import OUTPUTS, write_output
 from pytest_timing.render.ascii import render_ascii
 
 
+def _number(value: Any, where: str) -> None:
+    if not isinstance(value, (int, float, str)):
+        raise ValueError(f"{where} must be a number")
+    try:
+        finite = math.isfinite(float(value))
+    except OverflowError:
+        finite = False
+    if not finite:
+        raise ValueError(f"{where} must be finite")
+
+
+def _record(
+    value: Any, where: str, *, required: str = "", numbers: str = "", optional: str = ""
+) -> dict[str, Any]:
+    if not isinstance(value, dict):
+        raise ValueError(f"{where} must be an object")
+    for key in required.split():
+        if key not in value:
+            raise ValueError(f"{where}.{key} is required")
+    for key in numbers.split():
+        if key in value:
+            _number(value[key], f"{where}.{key}")
+    for key in optional.split():
+        if value.get(key) is not None:
+            _number(value[key], f"{where}.{key}")
+    return value
+
+
+def _array(value: Any, where: str) -> list[Any]:
+    if not isinstance(value, list):
+        raise ValueError(f"{where} must be an array")
+    return value
+
+
 def _load(path: str) -> Run:
-    return Run.from_json(Path(path).read_text("utf-8"))
+    # Check external input before model conversion. Programming errors in the
+    # model, comparison or formatter must propagate rather than becoming exit 2.
+    doc = _record(
+        json.loads(Path(path).read_text("utf-8")), "report", required="run", numbers="schema"
+    )
+    info = _record(
+        doc["run"],
+        "run",
+        required="start stop",
+        numbers="start stop",
+        optional="numprocesses exit_status",
+    )
+    if info.get("termination") is not None and not isinstance(info["termination"], str):
+        raise ValueError("run.termination must be a string")
+    _array(info.get("argv", []), "run.argv")
+    for i, value in enumerate(_array(doc.get("workers", []), "workers")):
+        _record(value, f"workers[{i}]", required="id", optional="start ready collected down items")
+    for i, value in enumerate(_array(doc.get("tests", []), "tests")):
+        where = f"tests[{i}]"
+        test = _record(
+            value,
+            where,
+            required="nodeid worker outcome start stop",
+            numbers="start stop attempt occurrence",
+        )
+        for name, phase in _record(test.get("phases", {}), f"{where}.phases").items():
+            at = f"{where}.phases.{name}"
+            if len(_array(phase, at)) != 3:
+                raise ValueError(f"{at} must contain start, stop and duration")
+            for number in phase:
+                _number(number, at)
+        for key, seconds in _record(test.get("fixtures") or {}, f"{where}.fixtures").items():
+            if seconds is not None:
+                _number(seconds, f"{where}.fixtures.{key}")
+        if test.get("cpu") is not None:
+            _record(
+                test["cpu"],
+                f"{where}.cpu",
+                numbers="elapsed work setup_work demand wait runtime_wait",
+                optional="pressure",
+            )
+        if test.get("memory") is not None:
+            _record(test["memory"], f"{where}.memory", numbers="base peak after wait")
+        for j, value in enumerate(
+            _array(test.get("admission_waits", []), f"{where}.admission_waits")
+        ):
+            at = f"{where}.admission_waits[{j}]"
+            wait = _record(value, at, required="start stop", numbers="start stop")
+            if not all(isinstance(g, str) for g in _array(wait.get("gates", []), f"{at}.gates")):
+                raise ValueError(f"{at}.gates must contain strings")
+    return Run.from_dict(doc)
 
 
 def cmd_render(args: argparse.Namespace) -> int:
@@ -22,7 +110,8 @@ def cmd_render(args: argparse.Namespace) -> int:
     for kind, output in OUTPUTS.items():
         path = getattr(args, kind)
         if path:
-            doc = doc or run.to_dict()
+            if doc is None and output.needs_doc:
+                doc = run.to_dict()
             print(write_output(output, Path(path), run, doc))
             wrote = True
     if args.ascii or not wrote:
@@ -39,6 +128,20 @@ def cmd_render(args: argparse.Namespace) -> int:
     return 0
 
 
+def cmd_compare(args: argparse.Namespace) -> int:
+    try:
+        result = compare_runs(_load(args.before), _load(args.after), args.budget)
+        if args.json:
+            Path(args.json).write_text(
+                json.dumps(result, indent=2, allow_nan=False), encoding="utf-8"
+            )
+        print(format_comparison(result, args.top))
+        return int(result["exit_code"])
+    except (OSError, ValueError) as exc:
+        print(f"pytest-timing compare: {exc}", file=sys.stderr)
+        return 2
+
+
 def _uniquify(candidate: str, used: set[str]) -> str:
     """``candidate``, or ``candidate#2``, ``#3``... until it is not in ``used``."""
     name, n = candidate, 1
@@ -137,6 +240,22 @@ def build_parser() -> argparse.ArgumentParser:
     merge.add_argument("runs", nargs="+", help="pytest-timing JSON files")
     merge.add_argument("-o", "--output", required=True, metavar="PATH")
     merge.set_defaults(func=cmd_merge)
+    compare = sub.add_parser(
+        "compare", help="compare two saved runs and check per-test regression budgets"
+    )
+    compare.add_argument("before", help="baseline JSON run")
+    compare.add_argument("after", help="current JSON run")
+    compare.add_argument(
+        "--budget",
+        action="append",
+        type=parse_budget,
+        default=[],
+        metavar="METRIC=LIMIT",
+        help="maximum increase per common test, e.g. duration=10%%, cpu=50ms, memory=64MiB",
+    )
+    compare.add_argument("--json", metavar="PATH", help="write all differences and budget results")
+    compare.add_argument("--top", type=int, default=20, help="test details to print (default: 20)")
+    compare.set_defaults(func=cmd_compare)
     return parser
 
 

{pytest_timing-0.3.1 → pytest_timing-0.4.0}/src/pytest_timing/collector.py

@@ -26,6 +26,7 @@ from pytest_timing.model import (
     Run,
     RunInfo,
     TestSpan,
+    WaitInterval,
     Worker,
 )
 
@@ -91,6 +92,7 @@ class Collector:
         attempt: int,
         seconds: float,
         gates: Iterable[str] = (),
+        ended: float | None = None,
     ) -> None:
         """Add an admission delay to the exact execution that waited.
 
@@ -107,6 +109,11 @@ class Collector:
             if span is None or span.outcome != "crashed" or span.phases or span.attempt != attempt:
                 return
         gates = set(gates)
+        if ended is not None:
+            stop = self._rel(ended)
+            span.admission_waits.append(
+                WaitInterval(stop - seconds, stop, tuple(sorted(gates or {"cpu"})))
+            )
         if "cpu" in gates or not gates:
             if span.cpu is None:
                 span.cpu = CpuRecord(elapsed=span.duration)