testrisk 1.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- testrisk-1.0.0/.gitignore +46 -0
- testrisk-1.0.0/CHANGELOG.md +23 -0
- testrisk-1.0.0/CONTRIBUTING.md +42 -0
- testrisk-1.0.0/LICENSE +21 -0
- testrisk-1.0.0/PKG-INFO +266 -0
- testrisk-1.0.0/README.md +237 -0
- testrisk-1.0.0/RELEASING.md +26 -0
- testrisk-1.0.0/SECURITY.md +21 -0
- testrisk-1.0.0/pyproject.toml +114 -0
- testrisk-1.0.0/testrisk/__init__.py +32 -0
- testrisk-1.0.0/testrisk/__main__.py +22 -0
- testrisk-1.0.0/testrisk/ast_index.py +108 -0
- testrisk-1.0.0/testrisk/cli.py +247 -0
- testrisk-1.0.0/testrisk/coverage_data.py +303 -0
- testrisk-1.0.0/testrisk/discover.py +117 -0
- testrisk-1.0.0/testrisk/doctor.py +93 -0
- testrisk-1.0.0/testrisk/engine.py +279 -0
- testrisk-1.0.0/testrisk/errors.py +5 -0
- testrisk-1.0.0/testrisk/gitdiff.py +135 -0
- testrisk-1.0.0/testrisk/models.py +189 -0
- testrisk-1.0.0/testrisk/py.typed +0 -0
- testrisk-1.0.0/testrisk/rank.py +67 -0
- testrisk-1.0.0/testrisk/report.py +123 -0
- testrisk-1.0.0/testrisk/tests_map.py +91 -0
- testrisk-1.0.0/testrisk/ui.py +133 -0
- testrisk-1.0.0/testrisk/version.py +13 -0
- testrisk-1.0.0/tests/conftest.py +12 -0
- testrisk-1.0.0/tests/helpers.py +79 -0
- testrisk-1.0.0/tests/test_analyze.py +100 -0
- testrisk-1.0.0/tests/test_api.py +28 -0
- testrisk-1.0.0/tests/test_ast_index.py +35 -0
- testrisk-1.0.0/tests/test_cli.py +134 -0
- testrisk-1.0.0/tests/test_coverage_data.py +90 -0
- testrisk-1.0.0/tests/test_discover.py +77 -0
- testrisk-1.0.0/tests/test_doctor.py +21 -0
- testrisk-1.0.0/tests/test_gitdiff.py +64 -0
- testrisk-1.0.0/tests/test_rank.py +77 -0
- testrisk-1.0.0/tests/test_report.py +104 -0
- testrisk-1.0.0/tests/test_tests_map.py +35 -0
- testrisk-1.0.0/tests/test_ui.py +38 -0
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Bytecode
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*$py.class
|
|
5
|
+
|
|
6
|
+
# Virtual environments
|
|
7
|
+
.venv/
|
|
8
|
+
venv/
|
|
9
|
+
|
|
10
|
+
# Packaging / install
|
|
11
|
+
build/
|
|
12
|
+
dist/
|
|
13
|
+
wheels/
|
|
14
|
+
*.egg-info/
|
|
15
|
+
*.egg
|
|
16
|
+
.eggs/
|
|
17
|
+
__pypackages__/
|
|
18
|
+
|
|
19
|
+
# Secrets and local overrides
|
|
20
|
+
.env
|
|
21
|
+
.env.*
|
|
22
|
+
!.env.example
|
|
23
|
+
*.local
|
|
24
|
+
|
|
25
|
+
# Test, coverage, lint, type-check
|
|
26
|
+
.pytest_cache/
|
|
27
|
+
.ruff_cache/
|
|
28
|
+
.ty_cache/
|
|
29
|
+
.coverage
|
|
30
|
+
.coverage.*
|
|
31
|
+
htmlcov/
|
|
32
|
+
coverage.xml
|
|
33
|
+
coverage.json
|
|
34
|
+
|
|
35
|
+
# Editor / OS
|
|
36
|
+
.idea/
|
|
37
|
+
.vscode/
|
|
38
|
+
*.swp
|
|
39
|
+
*~
|
|
40
|
+
.DS_Store
|
|
41
|
+
Thumbs.db
|
|
42
|
+
|
|
43
|
+
# Logs and scratch files
|
|
44
|
+
*.log
|
|
45
|
+
*.tmp
|
|
46
|
+
*.bak
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here. This project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
4
|
+
|
|
5
|
+
|
|
6
|
+
## [1.0.0] — 2026-09-12
|
|
7
|
+
|
|
8
|
+
### Added
|
|
9
|
+
|
|
10
|
+
- First release: rank uncovered Python functions from `coverage.json`, `coverage.xml`, or `.coverage`.
|
|
11
|
+
- Scoring combines missing lines, uncovered branches, cyclomatic-ish complexity, git-changed lines, and whether related tests exist.
|
|
12
|
+
- CLI: `--changed`, `--base`, `--top`, `--file`, `--json`, `--prompt`, `--fail-under-changed`, `--doctor`, `--quiet`, `--color`.
|
|
13
|
+
- Walk-up discovery of a repo root (`pyproject.toml` / `.git`) when `--repo` is `.`.
|
|
14
|
+
- Human report, JSON report, and an evidence-based `--prompt` for coding agents.
|
|
15
|
+
- Library export: `analyze`, `Gap`, `GapError`, `GapReport`, `Options`, `Risk`, `find_repo_root`.
|
|
16
|
+
- Type-check `testrisk` with [ty](https://docs.astral.sh/ty/) (`uv run ty check`) in the `dev` group and CI.
|
|
17
|
+
- Ship `py.typed` so installers and type checkers treat testrisk as typed.
|
|
18
|
+
|
|
19
|
+
### Changed
|
|
20
|
+
|
|
21
|
+
- Published as **testrisk** (PyPI, import, and CLI). `test-gap` / `testgap` were unavailable.
|
|
22
|
+
- Library imports no longer pull in the CLI. Catch `GapError` from `testrisk`.
|
|
23
|
+
- Git diffs honor `--repo` even if `GIT_DIR` is set, and decode as UTF-8.
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
# Contributing
|
|
2
|
+
|
|
3
|
+
Thanks for helping improve testrisk.
|
|
4
|
+
|
|
5
|
+
## Development setup
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
git clone https://github.com/karlhillx/testrisk.git
|
|
9
|
+
cd testrisk
|
|
10
|
+
uv sync
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
`uv` creates `.venv` and installs the `dev` group (pytest, pytest-cov, ruff, ty).
|
|
14
|
+
|
|
15
|
+
## Checks before a PR
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
uv run pytest tests/
|
|
19
|
+
uv run ruff check testrisk tests
|
|
20
|
+
uv run ruff format --check testrisk tests
|
|
21
|
+
uv run ty check
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Optional coverage:
|
|
25
|
+
|
|
26
|
+
```bash
|
|
27
|
+
uv run pytest --cov=testrisk --cov-report=term-missing tests/
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
CLI smoke test (same as CI):
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
uv run python -m testrisk --version
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
## Releases
|
|
37
|
+
|
|
38
|
+
Maintainers: follow [RELEASING.md](RELEASING.md) and keep [CHANGELOG.md](CHANGELOG.md) in sync with user-visible changes.
|
|
39
|
+
|
|
40
|
+
## Security
|
|
41
|
+
|
|
42
|
+
See [SECURITY.md](SECURITY.md) for how to report vulnerabilities.
|
testrisk-1.0.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Karl Hill
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
testrisk-1.0.0/PKG-INFO
ADDED
|
@@ -0,0 +1,266 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: testrisk
|
|
3
|
+
Version: 1.0.0
|
|
4
|
+
Summary: Find the code most likely to need better tests
|
|
5
|
+
Project-URL: Homepage, https://github.com/karlhillx/testrisk
|
|
6
|
+
Project-URL: Repository, https://github.com/karlhillx/testrisk
|
|
7
|
+
Project-URL: Issues, https://github.com/karlhillx/testrisk/issues
|
|
8
|
+
Project-URL: Changelog, https://github.com/karlhillx/testrisk/blob/main/CHANGELOG.md
|
|
9
|
+
Author-email: Karl Hill <karlhillx@gmail.com>
|
|
10
|
+
License-Expression: MIT
|
|
11
|
+
License-File: LICENSE
|
|
12
|
+
Keywords: cli,coverage,pytest,quality,test-risk,testing
|
|
13
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
14
|
+
Classifier: Environment :: Console
|
|
15
|
+
Classifier: Intended Audience :: Developers
|
|
16
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
17
|
+
Classifier: Operating System :: OS Independent
|
|
18
|
+
Classifier: Programming Language :: Python :: 3
|
|
19
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
22
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
23
|
+
Classifier: Topic :: Software Development :: Quality Assurance
|
|
24
|
+
Classifier: Topic :: Software Development :: Testing
|
|
25
|
+
Classifier: Typing :: Typed
|
|
26
|
+
Requires-Python: >=3.12
|
|
27
|
+
Requires-Dist: coverage<8,>=7.6
|
|
28
|
+
Description-Content-Type: text/markdown
|
|
29
|
+
|
|
30
|
+
# testrisk
|
|
31
|
+
|
|
32
|
+
[](https://pypi.org/project/testrisk/)
|
|
33
|
+
[](https://opensource.org/licenses/MIT)
|
|
34
|
+
[](https://pypi.org/project/testrisk/)
|
|
35
|
+
[](https://github.com/karlhillx/testrisk/actions/workflows/test.yml)
|
|
36
|
+
|
|
37
|
+
**Find the code most likely to need better tests.** testrisk reads `coverage.py` data, maps uncovered lines onto functions, and ranks them by test risk — not another coverage percentage.
|
|
38
|
+
|
|
39
|
+
## Why testrisk?
|
|
40
|
+
|
|
41
|
+
- **Rank gaps, do not just list them** — uncovered branches, cyclomatic-ish complexity, git-changed lines, and missing tests all feed one score
|
|
42
|
+
- **Human by default, JSON when you need it** — `--prompt` turns the same evidence into a concise agent task
|
|
43
|
+
- **Optional changed-line gate** — `--fail-under-changed 95` for CI; the default command still just advises
|
|
44
|
+
- **Small install** — one runtime dependency: **coverage** (see `pyproject.toml`)
|
|
45
|
+
|
|
46
|
+
```text
|
|
47
|
+
testrisk 1.0.0
|
|
48
|
+
|
|
49
|
+
Coverage 91.7%
|
|
50
|
+
Branch 84.2%
|
|
51
|
+
|
|
52
|
+
Highest-value test gaps
|
|
53
|
+
|
|
54
|
+
1. demo/services.py::ServiceManager.start
|
|
55
|
+
Missing: 15-18
|
|
56
|
+
Branches: 2 uncovered
|
|
57
|
+
Complexity: 3
|
|
58
|
+
Risk: HIGH
|
|
59
|
+
|
|
60
|
+
Changed code
|
|
61
|
+
94.1% covered
|
|
62
|
+
2 uncovered executable lines
|
|
63
|
+
|
|
64
|
+
Suggested next target:
|
|
65
|
+
tests/test_services.py
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## Installation
|
|
69
|
+
|
|
70
|
+
### uvx (recommended — zero install)
|
|
71
|
+
|
|
72
|
+
```bash
|
|
73
|
+
cd /path/to/your/repo
|
|
74
|
+
uvx testrisk
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
Persistent install on your `PATH`:
|
|
78
|
+
|
|
79
|
+
```bash
|
|
80
|
+
uv tool install testrisk
|
|
81
|
+
testrisk --doctor
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
### via pipx (isolated CLI)
|
|
85
|
+
|
|
86
|
+
```bash
|
|
87
|
+
pipx install testrisk
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
### via pip
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
pip install testrisk
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
### If `testrisk` is not on your `PATH`
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
python -m testrisk --version
|
|
100
|
+
python -m testrisk --doctor
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
### from source
|
|
104
|
+
|
|
105
|
+
```bash
|
|
106
|
+
git clone https://github.com/karlhillx/testrisk.git
|
|
107
|
+
cd testrisk
|
|
108
|
+
uv sync
|
|
109
|
+
uv run testrisk --version
|
|
110
|
+
uv run python -m testrisk --version
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
## Quick start
|
|
114
|
+
|
|
115
|
+
Generate coverage first (pytest-cov or coverage.py), then rank:
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
pytest --cov --cov-report=json
|
|
119
|
+
testrisk
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
testrisk looks for `coverage.json`, then `coverage.xml`, then `.coverage`, walking up from `.` to the nearest `pyproject.toml` / `.git` when `--repo` is omitted.
|
|
123
|
+
|
|
124
|
+
```bash
|
|
125
|
+
testrisk --changed # only gaps on git-changed lines
|
|
126
|
+
testrisk --top 10 # default
|
|
127
|
+
testrisk --file src/foo.py # one file (repeatable)
|
|
128
|
+
testrisk --json # machine-readable report
|
|
129
|
+
testrisk --prompt # evidence-based agent task
|
|
130
|
+
testrisk --fail-under-changed 95 # exit 1 if changed-line coverage is low
|
|
131
|
+
testrisk --doctor # coverage file, git, and Python
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
`--quiet` prints only the suggested next test file.
|
|
135
|
+
|
|
136
|
+
### `--prompt`
|
|
137
|
+
|
|
138
|
+
```text
|
|
139
|
+
Write focused unit tests for these uncovered behaviors.
|
|
140
|
+
|
|
141
|
+
Do not modify production code unless required to expose a testable seam.
|
|
142
|
+
|
|
143
|
+
Target:
|
|
144
|
+
src/foo.py::parse_config
|
|
145
|
+
|
|
146
|
+
Uncovered:
|
|
147
|
+
44-48, 57
|
|
148
|
+
|
|
149
|
+
Untested branches:
|
|
150
|
+
line 46: false branch
|
|
151
|
+
line 57: exception/exit branch
|
|
152
|
+
|
|
153
|
+
Complexity: 6
|
|
154
|
+
Risk: MEDIUM
|
|
155
|
+
|
|
156
|
+
Existing tests:
|
|
157
|
+
tests/test_foo.py
|
|
158
|
+
```
|
|
159
|
+
|
|
160
|
+
## How ranking works
|
|
161
|
+
|
|
162
|
+
Each function or method with uncovered statements or branches gets a score:
|
|
163
|
+
|
|
164
|
+
| Signal | Weight |
|
|
165
|
+
|--------|--------|
|
|
166
|
+
| Uncovered executable lines | × 1 |
|
|
167
|
+
| Uncovered branches | × 2 |
|
|
168
|
+
| Complexity above 1 | × 0.75 |
|
|
169
|
+
| Changed uncovered lines | × 3 |
|
|
170
|
+
| No related test file | + 4 |
|
|
171
|
+
|
|
172
|
+
Risk is `HIGH` / `MEDIUM` / `LOW` from that score plus a few hard rules (for example: five or more changed uncovered lines, or complexity ≥ 10 with two uncovered branches).
|
|
173
|
+
|
|
174
|
+
Related tests are existing `test_*.py` / `*_test.py` files that mention the function or module stem. If none exist, the suggestion is `tests/test_<stem>.py`.
|
|
175
|
+
|
|
176
|
+
`--base` defaults to `origin/main`, then `main`, then `origin/master`, then `master`, then `HEAD`. Changed lines are `git diff` against the merge-base of that ref (plus your working tree).
|
|
177
|
+
|
|
178
|
+
## Exit codes
|
|
179
|
+
|
|
180
|
+
| Code | Meaning |
|
|
181
|
+
|------|---------|
|
|
182
|
+
| `0` | Success (or changed-line coverage meets `--fail-under-changed`) |
|
|
183
|
+
| `1` | Runtime failure (no coverage data, changed-line coverage below the floor) |
|
|
184
|
+
| `2` | Usage error |
|
|
185
|
+
| `130` | Interrupted with `Ctrl-C` |
|
|
186
|
+
|
|
187
|
+
## Use as a library
|
|
188
|
+
|
|
189
|
+
testrisk ships type hints (`py.typed`) and a small public API:
|
|
190
|
+
|
|
191
|
+
```python
|
|
192
|
+
from testrisk import GapError, analyze
|
|
193
|
+
|
|
194
|
+
try:
|
|
195
|
+
report = analyze(".", changed_only=True, top=5)
|
|
196
|
+
except GapError as exc:
|
|
197
|
+
raise SystemExit(exc) from exc
|
|
198
|
+
|
|
199
|
+
for gap in report.gaps:
|
|
200
|
+
print(gap.qualname, gap.risk, gap.score)
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
`analyze(...)` returns a `GapReport`. Optional kwargs: `coverage`, `changed_only`, `base`, `files`, `top`. You can also pass an `Options` instance. Only names in `testrisk.__all__` are public; import the CLI via `python -m testrisk` or the `testrisk` console script.
|
|
204
|
+
|
|
205
|
+
## Requirements
|
|
206
|
+
|
|
207
|
+
- **Python** 3.12+ (`requires-python` in `pyproject.toml`)
|
|
208
|
+
- **OS** Linux, macOS, and Windows
|
|
209
|
+
- **coverage** 7.x (installed automatically with `testrisk`)
|
|
210
|
+
- **git** (optional; needed for `--changed`, the changed-code section, and `--fail-under-changed`)
|
|
211
|
+
|
|
212
|
+
### Local development
|
|
213
|
+
|
|
214
|
+
```bash
|
|
215
|
+
uv sync
|
|
216
|
+
uv run pytest
|
|
217
|
+
uv run pytest --cov=testrisk --cov-report=xml tests/
|
|
218
|
+
uv run ruff check testrisk tests
|
|
219
|
+
uv run ty check
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
## Environment variables
|
|
223
|
+
|
|
224
|
+
| Variable | Description |
|
|
225
|
+
|----------|-------------|
|
|
226
|
+
| `NO_COLOR` | Disable color when `--color auto` |
|
|
227
|
+
| `FORCE_COLOR` | Enable color when `--color auto` even if stdout is not a TTY |
|
|
228
|
+
| `TESTRISK_DEBUG` | Print a traceback on unexpected errors (same as `--verbose` for crashes) |
|
|
229
|
+
|
|
230
|
+
testrisk **does not run your test suite**. It only reads coverage artifacts, source files, and git diffs.
|
|
231
|
+
|
|
232
|
+
## Troubleshooting
|
|
233
|
+
|
|
234
|
+
### "No coverage data found"
|
|
235
|
+
|
|
236
|
+
Run tests with coverage and write a report in the repo root:
|
|
237
|
+
|
|
238
|
+
```bash
|
|
239
|
+
pytest --cov --cov-report=json
|
|
240
|
+
# or
|
|
241
|
+
coverage run -m pytest && coverage json
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
Then pass `--coverage PATH` if the file is not in the usual place.
|
|
245
|
+
|
|
246
|
+
### Changed-code section is missing
|
|
247
|
+
|
|
248
|
+
The checkout is not a git repo, or git is not on `PATH`. `--fail-under-changed` needs git.
|
|
249
|
+
|
|
250
|
+
### `uvx: command not found`
|
|
251
|
+
|
|
252
|
+
Install [uv](https://docs.astral.sh/uv/) (`curl -LsSf https://astral.sh/uv/install.sh | sh` or `brew install uv`), then retry `uvx testrisk`.
|
|
253
|
+
|
|
254
|
+
## License
|
|
255
|
+
|
|
256
|
+
MIT License - see [LICENSE](LICENSE) for details.
|
|
257
|
+
|
|
258
|
+
## Contributing
|
|
259
|
+
|
|
260
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md). User-facing changes should be noted in [CHANGELOG.md](CHANGELOG.md). Security reports: [SECURITY.md](SECURITY.md).
|
|
261
|
+
|
|
262
|
+
## Links
|
|
263
|
+
|
|
264
|
+
- [PyPI](https://pypi.org/project/testrisk/)
|
|
265
|
+
- [GitHub Repository](https://github.com/karlhillx/testrisk)
|
|
266
|
+
- [Issue Tracker](https://github.com/karlhillx/testrisk/issues)
|
testrisk-1.0.0/README.md
ADDED
|
@@ -0,0 +1,237 @@
|
|
|
1
|
+
# testrisk
|
|
2
|
+
|
|
3
|
+
[](https://pypi.org/project/testrisk/)
|
|
4
|
+
[](https://opensource.org/licenses/MIT)
|
|
5
|
+
[](https://pypi.org/project/testrisk/)
|
|
6
|
+
[](https://github.com/karlhillx/testrisk/actions/workflows/test.yml)
|
|
7
|
+
|
|
8
|
+
**Find the code most likely to need better tests.** testrisk reads `coverage.py` data, maps uncovered lines onto functions, and ranks them by test risk — not another coverage percentage.
|
|
9
|
+
|
|
10
|
+
## Why testrisk?
|
|
11
|
+
|
|
12
|
+
- **Rank gaps, do not just list them** — uncovered branches, cyclomatic-ish complexity, git-changed lines, and missing tests all feed one score
|
|
13
|
+
- **Human by default, JSON when you need it** — `--prompt` turns the same evidence into a concise agent task
|
|
14
|
+
- **Optional changed-line gate** — `--fail-under-changed 95` for CI; the default command still just advises
|
|
15
|
+
- **Small install** — one runtime dependency: **coverage** (see `pyproject.toml`)
|
|
16
|
+
|
|
17
|
+
```text
|
|
18
|
+
testrisk 1.0.0
|
|
19
|
+
|
|
20
|
+
Coverage 91.7%
|
|
21
|
+
Branch 84.2%
|
|
22
|
+
|
|
23
|
+
Highest-value test gaps
|
|
24
|
+
|
|
25
|
+
1. demo/services.py::ServiceManager.start
|
|
26
|
+
Missing: 15-18
|
|
27
|
+
Branches: 2 uncovered
|
|
28
|
+
Complexity: 3
|
|
29
|
+
Risk: HIGH
|
|
30
|
+
|
|
31
|
+
Changed code
|
|
32
|
+
94.1% covered
|
|
33
|
+
2 uncovered executable lines
|
|
34
|
+
|
|
35
|
+
Suggested next target:
|
|
36
|
+
tests/test_services.py
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
## Installation
|
|
40
|
+
|
|
41
|
+
### uvx (recommended — zero install)
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
cd /path/to/your/repo
|
|
45
|
+
uvx testrisk
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
Persistent install on your `PATH`:
|
|
49
|
+
|
|
50
|
+
```bash
|
|
51
|
+
uv tool install testrisk
|
|
52
|
+
testrisk --doctor
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
### via pipx (isolated CLI)
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
pipx install testrisk
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
### via pip
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
pip install testrisk
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
### If `testrisk` is not on your `PATH`
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
python -m testrisk --version
|
|
71
|
+
python -m testrisk --doctor
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
### from source
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
git clone https://github.com/karlhillx/testrisk.git
|
|
78
|
+
cd testrisk
|
|
79
|
+
uv sync
|
|
80
|
+
uv run testrisk --version
|
|
81
|
+
uv run python -m testrisk --version
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
## Quick start
|
|
85
|
+
|
|
86
|
+
Generate coverage first (pytest-cov or coverage.py), then rank:
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
pytest --cov --cov-report=json
|
|
90
|
+
testrisk
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
testrisk looks for `coverage.json`, then `coverage.xml`, then `.coverage`, walking up from `.` to the nearest `pyproject.toml` / `.git` when `--repo` is omitted.
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
testrisk --changed # only gaps on git-changed lines
|
|
97
|
+
testrisk --top 10 # default
|
|
98
|
+
testrisk --file src/foo.py # one file (repeatable)
|
|
99
|
+
testrisk --json # machine-readable report
|
|
100
|
+
testrisk --prompt # evidence-based agent task
|
|
101
|
+
testrisk --fail-under-changed 95 # exit 1 if changed-line coverage is low
|
|
102
|
+
testrisk --doctor # coverage file, git, and Python
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
`--quiet` prints only the suggested next test file.
|
|
106
|
+
|
|
107
|
+
### `--prompt`
|
|
108
|
+
|
|
109
|
+
```text
|
|
110
|
+
Write focused unit tests for these uncovered behaviors.
|
|
111
|
+
|
|
112
|
+
Do not modify production code unless required to expose a testable seam.
|
|
113
|
+
|
|
114
|
+
Target:
|
|
115
|
+
src/foo.py::parse_config
|
|
116
|
+
|
|
117
|
+
Uncovered:
|
|
118
|
+
44-48, 57
|
|
119
|
+
|
|
120
|
+
Untested branches:
|
|
121
|
+
line 46: false branch
|
|
122
|
+
line 57: exception/exit branch
|
|
123
|
+
|
|
124
|
+
Complexity: 6
|
|
125
|
+
Risk: MEDIUM
|
|
126
|
+
|
|
127
|
+
Existing tests:
|
|
128
|
+
tests/test_foo.py
|
|
129
|
+
```
|
|
130
|
+
|
|
131
|
+
## How ranking works
|
|
132
|
+
|
|
133
|
+
Each function or method with uncovered statements or branches gets a score:
|
|
134
|
+
|
|
135
|
+
| Signal | Weight |
|
|
136
|
+
|--------|--------|
|
|
137
|
+
| Uncovered executable lines | × 1 |
|
|
138
|
+
| Uncovered branches | × 2 |
|
|
139
|
+
| Complexity above 1 | × 0.75 |
|
|
140
|
+
| Changed uncovered lines | × 3 |
|
|
141
|
+
| No related test file | + 4 |
|
|
142
|
+
|
|
143
|
+
Risk is `HIGH` / `MEDIUM` / `LOW` from that score plus a few hard rules (for example: five or more changed uncovered lines, or complexity ≥ 10 with two uncovered branches).
|
|
144
|
+
|
|
145
|
+
Related tests are existing `test_*.py` / `*_test.py` files that mention the function or module stem. If none exist, the suggestion is `tests/test_<stem>.py`.
|
|
146
|
+
|
|
147
|
+
`--base` defaults to `origin/main`, then `main`, then `origin/master`, then `master`, then `HEAD`. Changed lines are `git diff` against the merge-base of that ref (plus your working tree).
|
|
148
|
+
|
|
149
|
+
## Exit codes
|
|
150
|
+
|
|
151
|
+
| Code | Meaning |
|
|
152
|
+
|------|---------|
|
|
153
|
+
| `0` | Success (or changed-line coverage meets `--fail-under-changed`) |
|
|
154
|
+
| `1` | Runtime failure (no coverage data, changed-line coverage below the floor) |
|
|
155
|
+
| `2` | Usage error |
|
|
156
|
+
| `130` | Interrupted with `Ctrl-C` |
|
|
157
|
+
|
|
158
|
+
## Use as a library
|
|
159
|
+
|
|
160
|
+
testrisk ships type hints (`py.typed`) and a small public API:
|
|
161
|
+
|
|
162
|
+
```python
|
|
163
|
+
from testrisk import GapError, analyze
|
|
164
|
+
|
|
165
|
+
try:
|
|
166
|
+
report = analyze(".", changed_only=True, top=5)
|
|
167
|
+
except GapError as exc:
|
|
168
|
+
raise SystemExit(exc) from exc
|
|
169
|
+
|
|
170
|
+
for gap in report.gaps:
|
|
171
|
+
print(gap.qualname, gap.risk, gap.score)
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
`analyze(...)` returns a `GapReport`. Optional kwargs: `coverage`, `changed_only`, `base`, `files`, `top`. You can also pass an `Options` instance. Only names in `testrisk.__all__` are public; import the CLI via `python -m testrisk` or the `testrisk` console script.
|
|
175
|
+
|
|
176
|
+
## Requirements
|
|
177
|
+
|
|
178
|
+
- **Python** 3.12+ (`requires-python` in `pyproject.toml`)
|
|
179
|
+
- **OS** Linux, macOS, and Windows
|
|
180
|
+
- **coverage** 7.x (installed automatically with `testrisk`)
|
|
181
|
+
- **git** (optional; needed for `--changed`, the changed-code section, and `--fail-under-changed`)
|
|
182
|
+
|
|
183
|
+
### Local development
|
|
184
|
+
|
|
185
|
+
```bash
|
|
186
|
+
uv sync
|
|
187
|
+
uv run pytest
|
|
188
|
+
uv run pytest --cov=testrisk --cov-report=xml tests/
|
|
189
|
+
uv run ruff check testrisk tests
|
|
190
|
+
uv run ty check
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
## Environment variables
|
|
194
|
+
|
|
195
|
+
| Variable | Description |
|
|
196
|
+
|----------|-------------|
|
|
197
|
+
| `NO_COLOR` | Disable color when `--color auto` |
|
|
198
|
+
| `FORCE_COLOR` | Enable color when `--color auto` even if stdout is not a TTY |
|
|
199
|
+
| `TESTRISK_DEBUG` | Print a traceback on unexpected errors (same as `--verbose` for crashes) |
|
|
200
|
+
|
|
201
|
+
testrisk **does not run your test suite**. It only reads coverage artifacts, source files, and git diffs.
|
|
202
|
+
|
|
203
|
+
## Troubleshooting
|
|
204
|
+
|
|
205
|
+
### "No coverage data found"
|
|
206
|
+
|
|
207
|
+
Run tests with coverage and write a report in the repo root:
|
|
208
|
+
|
|
209
|
+
```bash
|
|
210
|
+
pytest --cov --cov-report=json
|
|
211
|
+
# or
|
|
212
|
+
coverage run -m pytest && coverage json
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
Then pass `--coverage PATH` if the file is not in the usual place.
|
|
216
|
+
|
|
217
|
+
### Changed-code section is missing
|
|
218
|
+
|
|
219
|
+
The checkout is not a git repo, or git is not on `PATH`. `--fail-under-changed` needs git.
|
|
220
|
+
|
|
221
|
+
### `uvx: command not found`
|
|
222
|
+
|
|
223
|
+
Install [uv](https://docs.astral.sh/uv/) (`curl -LsSf https://astral.sh/uv/install.sh | sh` or `brew install uv`), then retry `uvx testrisk`.
|
|
224
|
+
|
|
225
|
+
## License
|
|
226
|
+
|
|
227
|
+
MIT License - see [LICENSE](LICENSE) for details.
|
|
228
|
+
|
|
229
|
+
## Contributing
|
|
230
|
+
|
|
231
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md). User-facing changes should be noted in [CHANGELOG.md](CHANGELOG.md). Security reports: [SECURITY.md](SECURITY.md).
|
|
232
|
+
|
|
233
|
+
## Links
|
|
234
|
+
|
|
235
|
+
- [PyPI](https://pypi.org/project/testrisk/)
|
|
236
|
+
- [GitHub Repository](https://github.com/karlhillx/testrisk)
|
|
237
|
+
- [Issue Tracker](https://github.com/karlhillx/testrisk/issues)
|
|
@@ -0,0 +1,26 @@
|
|
|
1
|
+
# Releasing testrisk
|
|
2
|
+
|
|
3
|
+
PyPI uploads use **trusted publishing** from GitHub Actions (see [.github/workflows/publish.yml](.github/workflows/publish.yml) and [PyPI: adding a publisher](https://docs.pypi.org/trusted-publishers/adding-a-publisher/)).
|
|
4
|
+
|
|
5
|
+
## Checklist
|
|
6
|
+
|
|
7
|
+
1. **Changelog** — Move items from `CHANGELOG.md` **Unreleased** into a dated section for the new version (e.g. `## [1.1.0] — YYYY-MM-DD`).
|
|
8
|
+
2. **Version** — Set `version` in `pyproject.toml` to match the tag you will publish.
|
|
9
|
+
3. **Lockfile** — Run `uv lock` if `pyproject.toml` dependencies or `requires-python` changed.
|
|
10
|
+
4. **Commit** — Push to `main` (or your release branch) with the changelog and version bump.
|
|
11
|
+
5. **Tag** — Create an annotated tag: `git tag -a v1.1.0 -m "Release 1.1.0"` then `git push origin v1.1.0`.
|
|
12
|
+
6. **GitHub Release** — In the repo, **Releases → Draft a new release**, choose the tag, publish. That triggers the **Publish to PyPI** workflow.
|
|
13
|
+
|
|
14
|
+
Use **workflow_dispatch** on that workflow only if you need a manual retry after fixing PyPI/GitHub configuration.
|
|
15
|
+
|
|
16
|
+
## Pre-release sanity
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
uv sync --frozen
|
|
20
|
+
uv run pytest tests/
|
|
21
|
+
uv run ruff check testrisk tests
|
|
22
|
+
uv run ruff format --check testrisk tests
|
|
23
|
+
uv run ty check
|
|
24
|
+
uv run python -m testrisk --version
|
|
25
|
+
uv run python -m testrisk --doctor
|
|
26
|
+
```
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# Security policy
|
|
2
|
+
|
|
3
|
+
## Supported versions
|
|
4
|
+
|
|
5
|
+
We aim to support the latest release on [PyPI](https://pypi.org/project/testrisk/). Security fixes are applied to the current minor line when practical.
|
|
6
|
+
|
|
7
|
+
## Reporting a vulnerability
|
|
8
|
+
|
|
9
|
+
**Please do not open a public GitHub issue for undisclosed security problems.**
|
|
10
|
+
|
|
11
|
+
Instead, email **karlhillx@gmail.com** with:
|
|
12
|
+
|
|
13
|
+
- A short description of the issue
|
|
14
|
+
- Steps to reproduce (if possible)
|
|
15
|
+
- Affected testrisk version and environment (OS, Python version)
|
|
16
|
+
|
|
17
|
+
We will acknowledge receipt and work on a fix and release timeline.
|
|
18
|
+
|
|
19
|
+
## Scope notes
|
|
20
|
+
|
|
21
|
+
testrisk **reads local coverage artifacts, source files, and git diffs**. It does not execute your test suite or production code. Reports about “arbitrary code execution” because a malicious repo contains a `.coverage` file or Python source are generally out of scope unless they involve a bug in testrisk itself (for example unsafe subprocess arguments or path traversal outside the requested repo).
|