pytest-abort 0.1.0__tar.gz

pytest_abort-0.1.0/LICENSE

@@ -0,0 +1,152 @@
+Apache License
+Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1. Definitions.
+
+   "License" shall mean the terms and conditions for use, reproduction,
+   and distribution as defined by Sections 1 through 9 of this document.
+
+   "Licensor" shall mean the copyright owner or entity authorized by
+   the copyright owner that is granting the License.
+
+   "Legal Entity" shall mean the union of the acting entity and all
+   other entities that control, are controlled by, or are under common
+   control with that entity. For the purposes of this definition,
+   "control" means (i) the power, direct or indirect, to cause the
+   direction or management of such entity, whether by contract or
+   otherwise, or (ii) ownership of fifty percent (50%) or more of the
+   outstanding shares, or (iii) beneficial ownership of such entity.
+
+   "You" (or "Your") shall mean an individual or Legal Entity
+   exercising permissions granted by this License.
+
+   "Source" form shall mean the preferred form for making modifications,
+   including but not limited to software source code, documentation
+   source, and configuration files.
+
+   "Object" form shall mean any form resulting from mechanical
+   transformation or translation of a Source form, including but
+   not limited to compiled object code, generated documentation,
+   and conversions to other media types.
+
+   "Work" shall mean the work of authorship, whether in Source or
+   Object form, made available under the License, as indicated by a
+   copyright notice that is included in or attached to the work
+   (an example is provided in the Appendix below).
+
+   "Derivative Works" shall mean any work, whether in Source or Object
+   form, that is based on (or derived from) the Work and for which the
+   editorial revisions, annotations, elaborations, or other modifications
+   represent, as a whole, an original work of authorship. For the purposes
+   of this License, Derivative Works shall not include works that remain
+   separable from, or merely link (or bind by name) to the interfaces of,
+   the Work and Derivative Works thereof.
+
+   "Contribution" shall mean any work of authorship, including
+   the original version of the Work and any modifications or additions
+   to that Work or Derivative Works thereof, that is intentionally
+   submitted to Licensor for inclusion in the Work by the copyright owner
+   or by an individual or Legal Entity authorized to submit on behalf of
+   the copyright owner. For the purposes of this definition, "submitted"
+   means any form of electronic, verbal, or written communication sent
+   to the Licensor or its representatives, including but not limited to
+   communication on electronic mailing lists, source code control systems,
+   and issue tracking systems that are managed by, or on behalf of, the
+   Licensor for the purpose of discussing and improving the Work, but
+   excluding communication that is conspicuously marked or otherwise
+   designated in writing by the copyright owner as "Not a Contribution."
+
+   "Contributor" shall mean Licensor and any individual or Legal Entity
+   on behalf of whom a Contribution has been received by Licensor and
+   subsequently incorporated within the Work.
+
+2. Grant of Copyright License. Subject to the terms and conditions of
+   this License, each Contributor hereby grants to You a perpetual,
+   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+   copyright license to reproduce, prepare Derivative Works of,
+   publicly display, publicly perform, sublicense, and distribute the
+   Work and such Derivative Works in Source or Object form.
+
+3. Grant of Patent License. Subject to the terms and conditions of
+   this License, each Contributor hereby grants to You a perpetual,
+   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+   (except as stated in this section) patent license to make, have made,
+   use, offer to sell, sell, import, and otherwise transfer the Work,
+   where such license applies only to those patent claims licensable
+   by such Contributor that are necessarily infringed by their
+   Contribution(s) alone or by combination of their Contribution(s)
+   with the Work to which such Contribution(s) was submitted. If You
+   institute patent litigation against any entity (including a
+   cross-claim or counterclaim in a lawsuit) alleging that the Work
+   or a Contribution incorporated within the Work constitutes direct
+   or contributory patent infringement, then any patent licenses
+   granted to You under this License for that Work shall terminate
+   as of the date such litigation is filed.
+
+4. Redistribution. You may reproduce and distribute copies of the
+   Work or Derivative Works thereof in any medium, with or without
+   modifications, and in Source or Object form, provided that You
+   meet the following conditions:
+
+   (a) You must give any other recipients of the Work or
+       Derivative Works a copy of this License; and
+
+   (b) You must cause any modified files to carry prominent notices
+       stating that You changed the files; and
+
+   (c) You must retain, in the Source form of any Derivative Works
+       that You distribute, all copyright, patent, trademark, and
+       attribution notices from the Source form of the Work,
+       excluding those notices that do not pertain to any part of
+       the Derivative Works; and
+
+   (d) If the Work includes a "NOTICE" text file as part of its
+       distribution, then any Derivative Works that You distribute must
+       include a readable copy of the attribution notices contained
+       within such NOTICE file, excluding those notices that do not
+       pertain to any part of the Derivative Works, in at least one
+       of the following places: within a NOTICE text file distributed
+       as part of the Derivative Works; within the Source form or
+       documentation, if provided along with the Derivative Works; or,
+       within a display generated by the Derivative Works, if and
+       wherever such third-party notices normally appear. The contents
+       of the NOTICE file are for informational purposes only and
+       do not modify the License. You may add Your own attribution
+       notices within Derivative Works that You distribute, alongside
+       or as an addendum to the NOTICE text from the Work, provided
+       that such additional attribution notices cannot be construed
+       as modifying the License.
+
+   You may add Your own copyright statement to Your modifications and
+   may provide additional or different license terms and conditions
+   for use, reproduction, or distribution of Your modifications, or
+   for any such Derivative Works as a whole, provided Your use,
+   reproduction, and distribution of the Work otherwise complies with
+   the conditions stated in this License.
+
+5. Submission of Contributions. Unless You explicitly state otherwise,
+   any Contribution intentionally submitted for inclusion in the Work
+   by You to the Licensor shall be under the terms and conditions of
+   this License, without any additional terms or conditions.
+
+6. Trademarks. This License does not grant permission to use the trade
+   names, trademarks, service marks, or product names of the Licensor.
+
+7. Disclaimer of Warranty. Unless required by applicable law or
+   agreed to in writing, Licensor provides the Work on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND.
+
+8. Limitation of Liability. In no event and under no legal theory,
+   whether in tort, contract, or otherwise, shall any Contributor be
+   liable for any damages arising as a result of this License or use
+   of the Work.
+
+9. Accepting Warranty or Additional Liability. While redistributing
+   the Work, You may choose to offer support, warranty, indemnity, or
+   other liability obligations and/or rights consistent with this License.
+
+END OF TERMS AND CONDITIONS
+

pytest_abort-0.1.0/PKG-INFO

@@ -0,0 +1,384 @@
+Metadata-Version: 2.4
+Name: pytest-abort
+Version: 0.1.0
+Summary: Pytest plugin + helpers for attributing hard crashes (SIGSEGV/SIGABRT) to the last running test and keeping pytest-html reports mergeable.
+License: Apache License
+        Version 2.0, January 2004
+        http://www.apache.org/licenses/
+        
+        TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+        
+        1. Definitions.
+        
+           "License" shall mean the terms and conditions for use, reproduction,
+           and distribution as defined by Sections 1 through 9 of this document.
+        
+           "Licensor" shall mean the copyright owner or entity authorized by
+           the copyright owner that is granting the License.
+        
+           "Legal Entity" shall mean the union of the acting entity and all
+           other entities that control, are controlled by, or are under common
+           control with that entity. For the purposes of this definition,
+           "control" means (i) the power, direct or indirect, to cause the
+           direction or management of such entity, whether by contract or
+           otherwise, or (ii) ownership of fifty percent (50%) or more of the
+           outstanding shares, or (iii) beneficial ownership of such entity.
+        
+           "You" (or "Your") shall mean an individual or Legal Entity
+           exercising permissions granted by this License.
+        
+           "Source" form shall mean the preferred form for making modifications,
+           including but not limited to software source code, documentation
+           source, and configuration files.
+        
+           "Object" form shall mean any form resulting from mechanical
+           transformation or translation of a Source form, including but
+           not limited to compiled object code, generated documentation,
+           and conversions to other media types.
+        
+           "Work" shall mean the work of authorship, whether in Source or
+           Object form, made available under the License, as indicated by a
+           copyright notice that is included in or attached to the work
+           (an example is provided in the Appendix below).
+        
+           "Derivative Works" shall mean any work, whether in Source or Object
+           form, that is based on (or derived from) the Work and for which the
+           editorial revisions, annotations, elaborations, or other modifications
+           represent, as a whole, an original work of authorship. For the purposes
+           of this License, Derivative Works shall not include works that remain
+           separable from, or merely link (or bind by name) to the interfaces of,
+           the Work and Derivative Works thereof.
+        
+           "Contribution" shall mean any work of authorship, including
+           the original version of the Work and any modifications or additions
+           to that Work or Derivative Works thereof, that is intentionally
+           submitted to Licensor for inclusion in the Work by the copyright owner
+           or by an individual or Legal Entity authorized to submit on behalf of
+           the copyright owner. For the purposes of this definition, "submitted"
+           means any form of electronic, verbal, or written communication sent
+           to the Licensor or its representatives, including but not limited to
+           communication on electronic mailing lists, source code control systems,
+           and issue tracking systems that are managed by, or on behalf of, the
+           Licensor for the purpose of discussing and improving the Work, but
+           excluding communication that is conspicuously marked or otherwise
+           designated in writing by the copyright owner as "Not a Contribution."
+        
+           "Contributor" shall mean Licensor and any individual or Legal Entity
+           on behalf of whom a Contribution has been received by Licensor and
+           subsequently incorporated within the Work.
+        
+        2. Grant of Copyright License. Subject to the terms and conditions of
+           this License, each Contributor hereby grants to You a perpetual,
+           worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+           copyright license to reproduce, prepare Derivative Works of,
+           publicly display, publicly perform, sublicense, and distribute the
+           Work and such Derivative Works in Source or Object form.
+        
+        3. Grant of Patent License. Subject to the terms and conditions of
+           this License, each Contributor hereby grants to You a perpetual,
+           worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+           (except as stated in this section) patent license to make, have made,
+           use, offer to sell, sell, import, and otherwise transfer the Work,
+           where such license applies only to those patent claims licensable
+           by such Contributor that are necessarily infringed by their
+           Contribution(s) alone or by combination of their Contribution(s)
+           with the Work to which such Contribution(s) was submitted. If You
+           institute patent litigation against any entity (including a
+           cross-claim or counterclaim in a lawsuit) alleging that the Work
+           or a Contribution incorporated within the Work constitutes direct
+           or contributory patent infringement, then any patent licenses
+           granted to You under this License for that Work shall terminate
+           as of the date such litigation is filed.
+        
+        4. Redistribution. You may reproduce and distribute copies of the
+           Work or Derivative Works thereof in any medium, with or without
+           modifications, and in Source or Object form, provided that You
+           meet the following conditions:
+        
+           (a) You must give any other recipients of the Work or
+               Derivative Works a copy of this License; and
+        
+           (b) You must cause any modified files to carry prominent notices
+               stating that You changed the files; and
+        
+           (c) You must retain, in the Source form of any Derivative Works
+               that You distribute, all copyright, patent, trademark, and
+               attribution notices from the Source form of the Work,
+               excluding those notices that do not pertain to any part of
+               the Derivative Works; and
+        
+           (d) If the Work includes a "NOTICE" text file as part of its
+               distribution, then any Derivative Works that You distribute must
+               include a readable copy of the attribution notices contained
+               within such NOTICE file, excluding those notices that do not
+               pertain to any part of the Derivative Works, in at least one
+               of the following places: within a NOTICE text file distributed
+               as part of the Derivative Works; within the Source form or
+               documentation, if provided along with the Derivative Works; or,
+               within a display generated by the Derivative Works, if and
+               wherever such third-party notices normally appear. The contents
+               of the NOTICE file are for informational purposes only and
+               do not modify the License. You may add Your own attribution
+               notices within Derivative Works that You distribute, alongside
+               or as an addendum to the NOTICE text from the Work, provided
+               that such additional attribution notices cannot be construed
+               as modifying the License.
+        
+           You may add Your own copyright statement to Your modifications and
+           may provide additional or different license terms and conditions
+           for use, reproduction, or distribution of Your modifications, or
+           for any such Derivative Works as a whole, provided Your use,
+           reproduction, and distribution of the Work otherwise complies with
+           the conditions stated in this License.
+        
+        5. Submission of Contributions. Unless You explicitly state otherwise,
+           any Contribution intentionally submitted for inclusion in the Work
+           by You to the Licensor shall be under the terms and conditions of
+           this License, without any additional terms or conditions.
+        
+        6. Trademarks. This License does not grant permission to use the trade
+           names, trademarks, service marks, or product names of the Licensor.
+        
+        7. Disclaimer of Warranty. Unless required by applicable law or
+           agreed to in writing, Licensor provides the Work on an "AS IS" BASIS,
+           WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND.
+        
+        8. Limitation of Liability. In no event and under no legal theory,
+           whether in tort, contract, or otherwise, shall any Contributor be
+           liable for any damages arising as a result of this License or use
+           of the Work.
+        
+        9. Accepting Warranty or Additional Liability. While redistributing
+           the Work, You may choose to offer support, warranty, indemnity, or
+           other liability obligations and/or rights consistent with this License.
+        
+        END OF TERMS AND CONDITIONS
+        
+        
+Keywords: pytest,plugin,crash,segfault,sigabrt,reporting
+Classifier: Development Status :: 3 - Alpha
+Classifier: Framework :: Pytest
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pytest>=7.0
+Dynamic: license-file
+
+# pytest-abort
+
+`pytest-abort` is a pytest plugin + helper library to **attribute hard crashes (SIGSEGV/SIGABRT) to the last running test** and to keep pytest-html reports **mergeable and renderable** by sanitizing `data-jsonblob` payloads before `pytest_html_merger` runs.
+
+## What this repo contains
+
+- **Pytest plugin**: `pytest_abort.plugin`
+  - Writes a JSON “last-running test” marker file before each test starts.
+  - Deletes the marker file on normal test completion.
+  - If pytest hard-crashes, cleanup never runs and the marker file remains with `status="running"`.
+
+- **Crash marker parser**: `pytest_abort.crash_file`
+  - `check_for_crash_file(path) -> dict | None`
+
+- **Runner helpers (library module)**: `pytest_abort.abort_handling`
+  - `handle_abort(json_file, html_file, last_running_file, testfile, crash_info=None) -> bool`
+  - `append_abort_to_json(...)`, `append_abort_to_html(...)`
+  - `sanitize_html_file_jsonblob(path)`, `sanitize_all_html_jsonblobs(log_dir)`
+
+## Key concept: the last-running marker file
+
+The plugin writes a small JSON file (path supplied by the runner) like:
+
+```json
+{
+  "test_name": "test_foo",
+  "nodeid": "tests/test_bar.py::TestBar::test_foo",
+  "start_time": "2026-01-14T23:59:59.123456",
+  "status": "running",
+  "pid": 12345,
+  "gpu_id": "0"
+}
+```
+
+If pytest exits normally, the file is deleted. If pytest is killed by a segfault/abort, the file remains and the outer runner can attribute the crash.
+
+## Installation
+
+Editable install:
+
+```bash
+python3 -m pip install -e .
+```
+
+If you plan to use `-n` (parallel execution) and/or per-test timeouts, you likely also want:
+
+```bash
+python3 -m pip install -U pytest-xdist pytest-timeout
+```
+
+## Build a wheel
+
+Build the wheel from the repo root:
+
+```bash
+python3 -m pip install --upgrade build
+python3 -m build
+```
+
+The wheel is produced under:
+
+- `./dist/` (for example: `./dist/pytest_abort-0.1.0-py3-none-any.whl`)
+
+Install the wheel:
+
+```bash
+python3 -m pip install ./dist/*.whl
+```
+
+## Using the plugin directly (debugging)
+
+Recommended (env var):
+
+```bash
+PYTEST_ABORT_LAST_RUNNING_FILE=/tmp/last_running.json \
+python3 -m pytest -q tests/test_something.py
+```
+
+Note:
+- If `pytest-abort` is **installed** (editable or wheel), pytest auto-loads it via the `pytest11` entry point, so you **do not** need `-p ...`.
+- If the plugin is **installed** and you want to load it explicitly anyway, use `-p pytest_abort`.
+- If the plugin is **not installed** and you’re loading it via `PYTHONPATH` only, use `-p pytest_abort.plugin`.
+- Don’t use `-p ...` when it’s already auto-loaded, or pytest will error with “Plugin already registered”.
+
+Optional CLI override:
+
+```bash
+python3 -m pytest \
+  --last-running-file /tmp/last_running.json \
+  -q tests/test_something.py
+```
+
+## Crashed-tests log (optional)
+
+You can also write a shared **crashed tests log** (JSONL: one JSON object per line).
+
+- Set `PYTEST_ABORT_CRASHED_TESTS_LOG=/path/to/crashed_tests.jsonl`
+- For `pytest -n` (xdist), also set `PYTEST_ABORT_LAST_RUNNING_DIR=/path/to/dir` (so each worker writes its own marker file).
+
+Example:
+
+```bash
+export PYTEST_ABORT_LAST_RUNNING_DIR=/tmp/last_running
+export PYTEST_ABORT_CRASHED_TESTS_LOG=/tmp/crashed_tests.jsonl
+
+pytest -n 8 tests
+```
+
+Notes:
+- In xdist, the **master process** appends to the crashed-tests log when a worker goes down.
+- In runner flows (like `run_single_gpu.py`), `handle_abort(...)` appends to the crashed-tests log if `PYTEST_ABORT_CRASHED_TESTS_LOG` is set.
+
+## Crash recovery for xdist runs (optional helper)
+
+If you want a “crash recovery” loop for `pytest -n ...`, you can use the included outer-process wrapper:
+
+```bash
+pytest-abort-retry --max-runs 5 --clear-crash-log -- \
+  pytest -n 8 --max-worker-restart=50 --tb=short --maxfail=20 tests examples
+```
+
+Per-test timeout example (requires `pytest-timeout`):
+
+```bash
+pytest-abort-retry --max-runs 5 --clear-crash-log -- \
+  pytest -n 8 --max-worker-restart=50 --tb=short --maxfail=20 \
+    --timeout=600 --timeout-method=thread \
+    tests examples
+```
+
+This will:
+- run pytest
+- read `PYTEST_ABORT_CRASHED_TESTS_LOG`
+- re-run pytest with `--deselect=<nodeid>` for crashed nodeids until stable (or `--max-runs`).
+
+Note:
+- The wrapper supports the standard `... -- pytest ...` form.
+- To avoid “wrong pytest binary / wrong environment” problems (missing `-n`, missing `--timeout`, etc.), the wrapper rewrites a leading `pytest ...` to run as `python -m pytest ...` using the wrapper’s interpreter.
+
+
+## Integration notes (outer runner)
+
+If you run pytest from an outer process (CI wrapper, custom runner, etc.), a common pattern is:
+
+- Add this package to the environment (editable install or wheel)
+- Set `PYTEST_ABORT_LAST_RUNNING_FILE` (or `PYTEST_ABORT_LAST_RUNNING_DIR` for xdist)
+- Produce per-run JSON/HTML artifacts with `pytest-json-report` and `pytest-html` (best-effort)
+- If a hard crash occurred, patch/repair the per-run artifacts from the outer process via `handle_abort(...)`
+- Before merging many HTML reports, sanitize `data-jsonblob` payloads so merge/HTML rendering stays robust
+
+## How rocm-jax uses it
+
+The [`rocm-jax`](https://github.com/ROCm/rocm-jax) test runners use `pytest-abort` to attribute hard crashes to the last-running test and to keep pytest-html reports mergeable.
+
+### `run_single_gpu.py`
+
+- Ensures `pytest_abort` is importable by the pytest subprocess (either by installing `pytest-abort` into the environment, or by adding the repo checkout to `PYTHONPATH`)
+- Enables the plugin via the installed `pytest11` entry point (no `-p` needed when installed). If using `PYTHONPATH` only, load explicitly with `-p pytest_abort.plugin`.
+- Sets `PYTEST_ABORT_LAST_RUNNING_FILE` per test-file run (`logs/*_last_running.json`)
+- On crash: re-runs remaining tests in the same file using `--deselect <crashed-nodeid>`
+- Appends crash info into `*_log.json` + `*_log.html` using `pytest_abort.abort_handling.handle_abort(...)`
+- Sanitizes per-file HTML jsonblobs before merging:
+  - `sanitize_all_html_jsonblobs("./logs")` then `pytest_html_merger`
+
+### `run_multi_gpu.py`
+
+- Uses the same plugin and env-var mechanism for its pytest subprocesses
+- Uses the same `./logs` directory and does **not** archive logs
+
+## Using the helper library from a runner (example)
+
+Hard crashes can prevent `pytest-html` / `pytest-json-report` from finishing their output files. The pattern used by the ROCm runners is:
+
+- Run pytest with `--html=...` and `--json-report-file=...` (best-effort)
+- Detect a hard crash via the last-running marker file
+- Call `pytest_abort.abort_handling.handle_abort(...)` **from the runner process** to ensure the per-testfile `*_log.json` and `*_log.html` exist and contain a synthetic “crashed” test entry
+
+Minimal example:
+
+```python
+import os
+import subprocess
+
+from pytest_abort.abort_handling import handle_abort, sanitize_all_html_jsonblobs
+
+json_log = "logs/example_log.json"
+html_log = "logs/example_log.html"
+last_running = "logs/example_last_running.json"
+
+env = os.environ.copy()
+env["PYTEST_ABORT_LAST_RUNNING_FILE"] = os.path.abspath(last_running)
+
+subprocess.run(
+    [
+        "python3",
+        "-m",
+        "pytest",
+        # Plugin is auto-loaded via pytest11 entry point when installed.
+        "--json-report",
+        f"--json-report-file={json_log}",
+        f"--html={html_log}",
+        "tests/some_test_file.py",
+    ],
+    env=env,
+    check=False,
+)
+
+# If a crash happened, this ensures JSON/HTML logs exist and are patched:
+handle_abort(json_log, html_log, last_running, testfile="some_test_file")
+
+# Before merging many per-file HTML reports:
+sanitize_all_html_jsonblobs("logs")
+```