benchmatrix 0.2.1__tar.gz → 0.2.4__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/CHANGELOG.md +49 -17
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/CITATION.cff +2 -2
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/PKG-INFO +9 -4
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/README.md +8 -3
- benchmatrix-0.2.4/RELEASING.md +28 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/architecture.md +1 -1
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/deprecations.md +2 -2
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/first-release.md +2 -2
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/github-actions-security.md +8 -8
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/lifecycle.md +4 -4
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/performance.md +7 -7
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/publishing.md +46 -18
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/repository-settings.md +5 -5
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/how-to/run-with-docker.md +4 -4
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/index.md +4 -4
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/reference/compatibility.md +1 -1
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/reference/configuration.md +23 -17
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/reference/index.md +3 -3
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/container-release.md +26 -26
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/dependency-update.md +25 -25
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/failed-ci.md +6 -6
- benchmatrix-0.2.4/docs/runbooks/release.md +177 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/repository-setup.md +35 -35
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/security-report.md +5 -5
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/mkdocs.yml +29 -1
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/pyproject.toml +2 -1
- benchmatrix-0.2.4/tests/test_create_release_pr.py +78 -0
- benchmatrix-0.2.4/tests/test_create_release_tag.py +79 -0
- benchmatrix-0.2.4/tests/test_prepare_release.py +143 -0
- benchmatrix-0.2.4/tests/test_release_preflight.py +96 -0
- benchmatrix-0.2.1/RELEASING.md +0 -61
- benchmatrix-0.2.1/docs/runbooks/release.md +0 -147
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/.gitignore +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/CODE_OF_CONDUCT.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/CONTRIBUTING.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/LICENSE +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/SECURITY.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/_scripts/gen_api_reference.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/changelog.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/contributing.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/development.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/ai.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/dependency-policy.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/deployment-environments.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/license-policy.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/repository-health.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/secrets.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/threat-model.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/how-to/create-benchmark-matrix.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/how-to/parse-results.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/how-to/troubleshooting.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/incident-response.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/index.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/ownership-transfer.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/tutorials/first-benchmark.md +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/examples/test_factorial_benchmarks.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/examples/test_factorial_benchmarks_simplified.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/noxfile.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/__init__.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/_schema.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/bench_harness.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/bench_results.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/exceptions.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/py.typed +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/fixtures/benchmark_results/invalid_tail_missing_data.json +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/fixtures/benchmark_results/mixed_metrics.json +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/test_bench_harness.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/test_bench_results.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/test_integration.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/test_properties.py +0 -0
- {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/test_public_api.py +0 -0
|
@@ -12,12 +12,44 @@ additional pre-1.0 compatibility expectations described in
|
|
|
12
12
|
|
|
13
13
|
### Changed
|
|
14
14
|
|
|
15
|
+
### Deprecated
|
|
16
|
+
|
|
17
|
+
### Removed
|
|
18
|
+
|
|
19
|
+
### Fixed
|
|
20
|
+
|
|
21
|
+
### Security
|
|
22
|
+
|
|
23
|
+
## 0.2.4 - 2026-06-24
|
|
24
|
+
|
|
25
|
+
### Fixed
|
|
26
|
+
|
|
27
|
+
* Check out repository for context of `release` workflow
|
|
28
|
+
|
|
29
|
+
## 0.2.3 - 2026-06-23
|
|
30
|
+
|
|
31
|
+
### Fixed
|
|
32
|
+
|
|
33
|
+
* Ensure all assets uploaded to draft GitHub release
|
|
34
|
+
|
|
35
|
+
## 0.2.2 - 2026-06-23
|
|
36
|
+
|
|
37
|
+
### Changed
|
|
38
|
+
|
|
39
|
+
* Change documentation site from built-in MkDocs theme to Material theme
|
|
40
|
+
* Automate several steps of the release process
|
|
41
|
+
|
|
42
|
+
### Fixed
|
|
43
|
+
|
|
44
|
+
* Ensure all assets, including wheels and SBOMs attached to GitHub releases
|
|
45
|
+
* Standardize Markdown formatting on four spaces for indentation
|
|
46
|
+
|
|
15
47
|
## 0.2.1 - 2026-06-22
|
|
16
48
|
|
|
17
49
|
### Changed
|
|
18
50
|
|
|
19
51
|
* Fix release and release-verify workflow bugs preventing creation of PyPI
|
|
20
|
-
|
|
52
|
+
releases.
|
|
21
53
|
|
|
22
54
|
## 0.2.0 - 2026-06-22
|
|
23
55
|
|
|
@@ -26,31 +58,31 @@ additional pre-1.0 compatibility expectations described in
|
|
|
26
58
|
* Initial benchmark matrix utilities.
|
|
27
59
|
* pytest-benchmark JSON parsing and display utilities.
|
|
28
60
|
* Automated linting, typing, security, dependency, test, coverage, and package
|
|
29
|
-
|
|
30
|
-
|
|
61
|
+
validation for local development and pull requests, including scheduled
|
|
62
|
+
audits of locked dependencies for known vulnerabilities.
|
|
31
63
|
* Pre-commit automation, secret scanning, Markdown linting, GitHub Actions
|
|
32
|
-
|
|
64
|
+
workflow linting, and repository text and binary file attributes.
|
|
33
65
|
* MkDocs documentation site with strict builds, generated API reference pages,
|
|
34
|
-
|
|
66
|
+
and operational maintainer runbooks.
|
|
35
67
|
* Reproducible CycloneDX SBOM generation for locked runtime dependencies.
|
|
36
68
|
* uv-backed nox automation for supported-Python tests, quality checks, and
|
|
37
|
-
|
|
69
|
+
release artifact smoke testing.
|
|
38
70
|
* GitHub pull request auto-labeling and labels-as-code configuration for
|
|
39
|
-
|
|
71
|
+
maintainers.
|
|
40
72
|
* Dependabot automation for Python, Node, pre-commit, GitHub Actions, Docker,
|
|
41
|
-
|
|
73
|
+
and devcontainer dependency updates.
|
|
42
74
|
* Repository settings-as-code plus external setup checklists for branch
|
|
43
|
-
|
|
75
|
+
protection, security features, Pages, environments, and PyPI publishing.
|
|
44
76
|
* Focused Python dependency groups for test, lint, type, docs, security,
|
|
45
|
-
|
|
77
|
+
release, and automation tooling, with `dev` as the aggregate group.
|
|
46
78
|
* Compatibility, lifecycle, security-fix, release-branch, and deprecation
|
|
47
|
-
|
|
79
|
+
policy documentation.
|
|
48
80
|
* GitHub Actions CI/CD workflows for quality checks, multi-version and
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
81
|
+
cross-OS tests, docs deployment, PyPI Trusted Publishing, artifact
|
|
82
|
+
attestations, CodeQL, dependency review, OpenSSF Scorecard, and workflow
|
|
83
|
+
linting.
|
|
52
84
|
* CI test and coverage report artifacts, documentation link checking, minimum-
|
|
53
|
-
|
|
85
|
+
dependency tests, and post-release PyPI installation verification.
|
|
54
86
|
* Docker runtime and test images, local Docker targets, Docker-outside-of-Docker
|
|
55
|
-
|
|
56
|
-
|
|
87
|
+
devcontainer support, Dockerfile linting, GHCR publishing, image
|
|
88
|
+
SBOM/provenance, and critical-vulnerability image scanning.
|
|
@@ -10,8 +10,8 @@ authors:
|
|
|
10
10
|
given-names: "Ryan"
|
|
11
11
|
email: "ryancswallace@gmail.com"
|
|
12
12
|
alias: "ryancswallace"
|
|
13
|
-
version: 0.2.
|
|
14
|
-
date-released: "2026-06-
|
|
13
|
+
version: 0.2.4
|
|
14
|
+
date-released: "2026-06-24"
|
|
15
15
|
license: MIT
|
|
16
16
|
repository-code: "https://github.com/ryancswallace/benchmatrix"
|
|
17
17
|
url: "https://github.com/ryancswallace/benchmatrix"
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: benchmatrix
|
|
3
|
-
Version: 0.2.
|
|
3
|
+
Version: 0.2.4
|
|
4
4
|
Summary: A pytest-benchmark matrix, metadata, and JSON results layer.
|
|
5
5
|
Project-URL: Homepage, https://github.com/ryancswallace/benchmatrix
|
|
6
6
|
Project-URL: Changelog, https://github.com/ryancswallace/benchmatrix/blob/main/CHANGELOG.md
|
|
@@ -119,11 +119,16 @@ For local development from this repository:
|
|
|
119
119
|
make ready
|
|
120
120
|
```
|
|
121
121
|
|
|
122
|
-
To
|
|
123
|
-
from GitHub:
|
|
122
|
+
To install the latest release from PyPI:
|
|
124
123
|
|
|
125
124
|
```bash
|
|
126
|
-
uv add
|
|
125
|
+
uv add benchmatrix
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
or
|
|
129
|
+
|
|
130
|
+
```bash
|
|
131
|
+
pip install benchmatrix
|
|
127
132
|
```
|
|
128
133
|
|
|
129
134
|
## Documentation
|
|
@@ -88,11 +88,16 @@ For local development from this repository:
|
|
|
88
88
|
make ready
|
|
89
89
|
```
|
|
90
90
|
|
|
91
|
-
To
|
|
92
|
-
from GitHub:
|
|
91
|
+
To install the latest release from PyPI:
|
|
93
92
|
|
|
94
93
|
```bash
|
|
95
|
-
uv add
|
|
94
|
+
uv add benchmatrix
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
or
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
pip install benchmatrix
|
|
96
101
|
```
|
|
97
102
|
|
|
98
103
|
## Documentation
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Release policy
|
|
2
|
+
|
|
3
|
+
benchmatrix uses [Semantic Versioning](https://semver.org/).
|
|
4
|
+
|
|
5
|
+
## Compatibility
|
|
6
|
+
|
|
7
|
+
While the project is pre-1.0:
|
|
8
|
+
|
|
9
|
+
* patch releases contain compatible fixes and documentation improvements;
|
|
10
|
+
* minor releases may introduce breaking API changes;
|
|
11
|
+
* breaking changes should be called out clearly in the changelog.
|
|
12
|
+
|
|
13
|
+
Starting with 1.0, incompatible public API changes require a major release.
|
|
14
|
+
The supported public API is the set of names exported from
|
|
15
|
+
`benchmatrix.__init__`. Private modules and names beginning with an underscore
|
|
16
|
+
are not covered by the compatibility guarantee.
|
|
17
|
+
|
|
18
|
+
Python version support may change in a minor release before 1.0 and in a major
|
|
19
|
+
release after 1.0. Dropping a Python version will be documented in advance when
|
|
20
|
+
practical.
|
|
21
|
+
|
|
22
|
+
Release branches are not maintained as standing support branches. The active
|
|
23
|
+
support branch is `main`; temporary release or security branches may be used for
|
|
24
|
+
coordination and retired after release.
|
|
25
|
+
|
|
26
|
+
## Release process
|
|
27
|
+
|
|
28
|
+
See the detailed operational checklist in `docs/runbooks/release.md`.
|
|
@@ -14,7 +14,7 @@ replace its calibration, timing, statistics, terminal reporting, or JSON export.
|
|
|
14
14
|
* `bench_results.py` parses and displays saved benchmark results.
|
|
15
15
|
* `exceptions.py` contains package-specific exceptions.
|
|
16
16
|
* `_schema.py` is a private schema constant module shared by implementation
|
|
17
|
-
|
|
17
|
+
code.
|
|
18
18
|
* `__init__.py` defines the public package exports.
|
|
19
19
|
|
|
20
20
|
## Design constraints
|
|
@@ -15,7 +15,7 @@ Deprecation is not required for:
|
|
|
15
15
|
* private names beginning with `_`;
|
|
16
16
|
* undocumented implementation details;
|
|
17
17
|
* behavior that is removed immediately for security or serious correctness
|
|
18
|
-
|
|
18
|
+
reasons.
|
|
19
19
|
|
|
20
20
|
## Timing
|
|
21
21
|
|
|
@@ -39,7 +39,7 @@ major release unless release notes document a narrower exception.
|
|
|
39
39
|
|
|
40
40
|
1. confirm the changelog mentioned the deprecation;
|
|
41
41
|
2. confirm the removal timing has arrived or the security/correctness exception
|
|
42
|
-
|
|
42
|
+
applies;
|
|
43
43
|
3. remove the behavior and tests together;
|
|
44
44
|
4. update API docs, tutorials, examples, and compatibility notes;
|
|
45
45
|
5. run `make check` and `make test-matrix`.
|
|
@@ -14,11 +14,11 @@ structure to support users after publication.
|
|
|
14
14
|
* the built wheel can be installed and imported in a clean environment;
|
|
15
15
|
* changelog has a dated version section;
|
|
16
16
|
* `pyproject.toml`, `CITATION.cff`, the Git tag, and release notes use the same
|
|
17
|
-
|
|
17
|
+
version;
|
|
18
18
|
* security and support policies are present;
|
|
19
19
|
* the GitHub `pypi` environment exists with trusted maintainer reviewers;
|
|
20
20
|
* PyPI Trusted Publishing is configured for `release.yml` and the `pypi`
|
|
21
|
-
|
|
21
|
+
environment;
|
|
22
22
|
* a maintainer knows how to verify the package after PyPI publication.
|
|
23
23
|
|
|
24
24
|
If any item is missing, fix it before uploading artifacts. A delayed first
|
|
@@ -7,21 +7,21 @@ CI should verify project health without granting unnecessary write access.
|
|
|
7
7
|
* Default workflow permissions are read-only.
|
|
8
8
|
* Checkout does not persist credentials unless a job needs to push.
|
|
9
9
|
* The pull request labeler uses `pull_request_target` without checkout and only
|
|
10
|
-
|
|
10
|
+
receives the permission needed to add existing pull request labels.
|
|
11
11
|
* Workflow actions use carefully versioned refs and Dependabot updates. The
|
|
12
|
-
|
|
13
|
-
|
|
12
|
+
`.github/zizmor.yml` policy records this choice and disables only zizmor's
|
|
13
|
+
SHA-pinning audit until the project adopts full action SHA pinning.
|
|
14
14
|
* Scheduled audits are separated from normal pull request validation.
|
|
15
15
|
* Documentation deployment uses the `github-pages` environment with `pages:
|
|
16
|
-
|
|
16
|
+
write` and OIDC only in the deployment job.
|
|
17
17
|
* Package publishing uses the `pypi` environment and PyPI Trusted Publishing
|
|
18
|
-
|
|
18
|
+
instead of long-lived publishing tokens.
|
|
19
19
|
* Release artifacts are attested with GitHub artifact attestations before they
|
|
20
|
-
|
|
20
|
+
are published.
|
|
21
21
|
* Docker images are built in a dedicated workflow, scanned with Trivy, and only
|
|
22
|
-
|
|
22
|
+
pushed from non-pull-request events with `packages: write`.
|
|
23
23
|
* Artifacts should be limited to build outputs such as distributions, container
|
|
24
|
-
|
|
24
|
+
images, SBOM files, and attestation metadata.
|
|
25
25
|
|
|
26
26
|
## Pull requests
|
|
27
27
|
|
|
@@ -27,14 +27,14 @@ of truth.
|
|
|
27
27
|
## Change stages
|
|
28
28
|
|
|
29
29
|
* **Experimental**: behavior may change without deprecation. This includes
|
|
30
|
-
|
|
30
|
+
undocumented internals, private modules, and private names.
|
|
31
31
|
* **Documented**: behavior appears in docs or examples and should receive a
|
|
32
|
-
|
|
32
|
+
changelog note when changed.
|
|
33
33
|
* **Stable public API**: exported from `benchmatrix.__init__`, documented in the
|
|
34
|
-
|
|
34
|
+
API reference, and covered by tests.
|
|
35
35
|
* **Deprecated**: retained temporarily with migration guidance.
|
|
36
36
|
* **Removed**: no longer available after the documented removal release or after
|
|
37
|
-
|
|
37
|
+
an urgent security/correctness removal.
|
|
38
38
|
|
|
39
39
|
## Public API stability
|
|
40
40
|
|
|
@@ -39,14 +39,14 @@ metadata with saved runs.
|
|
|
39
39
|
Common pitfalls:
|
|
40
40
|
|
|
41
41
|
* A target returning a generator, coroutine, future, query plan, or other lazy
|
|
42
|
-
|
|
43
|
-
|
|
42
|
+
object may only measure object creation. Resolve lazy work inside the
|
|
43
|
+
synchronous target wrapper.
|
|
44
44
|
* Fresh-input factories and copying run outside the timed target body. Put
|
|
45
|
-
|
|
46
|
-
|
|
45
|
+
construction inside the target only when setup cost is part of the operation
|
|
46
|
+
being measured.
|
|
47
47
|
* Reused mutable inputs can drift across invocations. Use `fresh_inputs=True`,
|
|
48
|
-
|
|
48
|
+
`deep_copy`, or a domain-specific copier when targets mutate their inputs.
|
|
49
49
|
* Small differences need repeated runs, distribution checks, and controlled
|
|
50
|
-
|
|
50
|
+
environments before they become conclusions.
|
|
51
51
|
* A faster implementation still needs ordinary correctness tests. Benchmark
|
|
52
|
-
|
|
52
|
+
tests should not be the only proof that two implementations do the same work.
|
|
@@ -13,22 +13,39 @@ make build
|
|
|
13
13
|
|
|
14
14
|
This command builds the source distribution and wheel, checks metadata with
|
|
15
15
|
Twine, installs the built wheel into a temporary environment, imports the public
|
|
16
|
-
package,
|
|
16
|
+
package, generates the runtime CycloneDX SBOM at `dist/benchmatrix.cdx.json`,
|
|
17
|
+
and fails unless the release artifact files in `dist/` are the expected
|
|
18
|
+
wheel, source distribution, and SBOM for the package version in `pyproject.toml`.
|
|
17
19
|
|
|
18
20
|
The release workflow runs the same command before publishing, so local release
|
|
19
21
|
checks and CI release checks exercise the same path.
|
|
20
22
|
|
|
21
23
|
## Version metadata
|
|
22
24
|
|
|
23
|
-
The project
|
|
24
|
-
|
|
25
|
+
The project uses static package metadata, but release preparation is automated.
|
|
26
|
+
After the `CHANGELOG.md` entries under `## Unreleased` are ready, export the
|
|
27
|
+
release version without a leading `v`, then run the release pull request
|
|
28
|
+
preparation target:
|
|
25
29
|
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
+
```bash
|
|
31
|
+
export BENCHMATRIX_RELEASE_VERSION=X.Y.Z
|
|
32
|
+
make release-pr-ready
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
This command checks release prerequisites, updates `pyproject.toml`,
|
|
36
|
+
`CITATION.cff`, and `CHANGELOG.md`, sets `date-released` in `CITATION.cff`,
|
|
37
|
+
creates the dated changelog section, runs `uv lock` after the `pyproject.toml`
|
|
38
|
+
version changes, runs the full local `make check` suite, and creates or reuses
|
|
39
|
+
the standardized release branch, commit, and GitHub pull request. Use
|
|
40
|
+
`RELEASE_DATE=YYYY-MM-DD` when the release date should be explicit.
|
|
30
41
|
|
|
31
|
-
|
|
42
|
+
After the release pull request is merged, create the Git tag with:
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
make release-tag
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
The tag must be named `vX.Y.Z` and must match the package version `X.Y.Z`.
|
|
32
49
|
Do not publish if the tag and metadata disagree.
|
|
33
50
|
|
|
34
51
|
## Changelog and release notes
|
|
@@ -44,6 +61,14 @@ changes before routine maintenance items.
|
|
|
44
61
|
|
|
45
62
|
## GitHub Release publishing flow
|
|
46
63
|
|
|
64
|
+
Pushing an annotated tag named `vX.Y.Z` starts `.github/workflows/draft-release.yml`.
|
|
65
|
+
That workflow validates release metadata, extracts notes from `CHANGELOG.md`,
|
|
66
|
+
builds the source distribution, wheel, and SBOM, creates or updates a draft
|
|
67
|
+
GitHub Release for the pushed tag, attaches those files to the draft, and
|
|
68
|
+
verifies that the package assets are present before the workflow succeeds. The
|
|
69
|
+
workflow also supports manual dispatch for rerunning draft creation and asset
|
|
70
|
+
attachment for an existing tag.
|
|
71
|
+
|
|
47
72
|
`.github/workflows/release.yml` publishes when a GitHub Release is published.
|
|
48
73
|
The workflow listens for:
|
|
49
74
|
|
|
@@ -54,11 +79,15 @@ release:
|
|
|
54
79
|
```
|
|
55
80
|
|
|
56
81
|
That means creating or editing a draft release is safe. Publishing the release is
|
|
57
|
-
the deployment action. The
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
82
|
+
the deployment action. The draft-release workflow uses `GITHUB_TOKEN`, so it
|
|
83
|
+
intentionally stops before publication; GitHub suppresses most follow-on
|
|
84
|
+
workflow runs caused by that token to avoid recursive automation.
|
|
85
|
+
|
|
86
|
+
The publish job is guarded with `startsWith(github.ref, 'refs/tags/v')`, so PyPI
|
|
87
|
+
publication only runs from version tags such as `v0.2.0`. The workflow rebuilds
|
|
88
|
+
from the tagged source, uploads package distributions and the SBOM as separate
|
|
89
|
+
Actions artifacts, refreshes those files on the GitHub Release, attests them,
|
|
90
|
+
and publishes only the distributions to PyPI from the `pypi` environment.
|
|
62
91
|
|
|
63
92
|
The workflow also has manual dispatch. Treat `publish=false` as a build smoke
|
|
64
93
|
check. Treat `publish=true` as a real publication and only run it from an exact
|
|
@@ -89,10 +118,10 @@ GitHub repository settings:
|
|
|
89
118
|
1. Create an environment named `pypi`.
|
|
90
119
|
2. Add trusted maintainer reviewers.
|
|
91
120
|
3. Restrict deployment branches or tags to release refs if the repository plan
|
|
92
|
-
|
|
121
|
+
supports that control.
|
|
93
122
|
4. Do not add PyPI secrets when Trusted Publishing works.
|
|
94
123
|
5. If a temporary token is unavoidable, use a project-scoped token, store it only
|
|
95
|
-
|
|
124
|
+
in the `pypi` environment, and remove it after Trusted Publishing succeeds.
|
|
96
125
|
|
|
97
126
|
## Documentation artifacts
|
|
98
127
|
|
|
@@ -120,6 +149,5 @@ rm -rf "$verify_env"
|
|
|
120
149
|
|
|
121
150
|
The command should print `BenchmarkCase`. The release workflow's verification
|
|
122
151
|
job runs that same post-publication smoke check after a GitHub Release is
|
|
123
|
-
published. Also
|
|
124
|
-
|
|
125
|
-
and attestations.
|
|
152
|
+
published. Also inspect the PyPI project page, release workflow logs, GitHub
|
|
153
|
+
Release assets, Actions artifacts, SBOM, and attestations.
|
|
@@ -10,12 +10,12 @@ The repository includes `.github/settings.yml` for the GitHub Settings app. If
|
|
|
10
10
|
the app is installed, changes merged to the default branch sync these settings:
|
|
11
11
|
|
|
12
12
|
* repository description, homepage, topics, issues/projects/wiki flags, and
|
|
13
|
-
|
|
13
|
+
branch deletion after merge;
|
|
14
14
|
* allowed merge strategies: squash and rebase enabled, merge commits disabled;
|
|
15
15
|
* Dependabot alerts and automated security fixes;
|
|
16
16
|
* `main` branch protection, including required reviews, code owner review,
|
|
17
|
-
|
|
18
|
-
|
|
17
|
+
stale review dismissal, last-push approval, required status checks, linear
|
|
18
|
+
history, blocked force pushes, blocked deletions, and conversation resolution;
|
|
19
19
|
* deployment environments and their branch or tag deployment policies.
|
|
20
20
|
|
|
21
21
|
Install the Settings app only after branch protection is active enough that
|
|
@@ -106,7 +106,7 @@ The GitHub Settings app syncs these environments from `.github/settings.yml`:
|
|
|
106
106
|
|
|
107
107
|
* `github-pages` for documentation deployment, restricted to `main`;
|
|
108
108
|
* `pypi` for package publishing, restricted to `v*` tags and requiring
|
|
109
|
-
|
|
109
|
+
approval from project maintainer(s).
|
|
110
110
|
|
|
111
111
|
Keep secrets out of the environment when using PyPI Trusted Publishing because
|
|
112
112
|
OIDC should provide a short-lived publishing token.
|
|
@@ -138,7 +138,7 @@ Set these repository profile fields:
|
|
|
138
138
|
* description: `Build pytest-benchmark matrices and parse benchmark results with lightweight Python utilities.`;
|
|
139
139
|
* homepage: `https://ryancswallace.github.io/benchmatrix/` once Pages is live;
|
|
140
140
|
* topics: `benchmark`, `benchmarking`, `performance`, `pytest`,
|
|
141
|
-
|
|
141
|
+
`pytest-benchmark`, `python`.
|
|
142
142
|
|
|
143
143
|
Disable the wiki and projects unless the project starts using them deliberately.
|
|
144
144
|
Keep issues enabled.
|
|
@@ -8,10 +8,10 @@ convenience images for reproducible local use, CI smoke checks, and examples.
|
|
|
8
8
|
The repository builds two images:
|
|
9
9
|
|
|
10
10
|
* `ghcr.io/ryancswallace/benchmatrix` is the runtime image. It installs the
|
|
11
|
-
|
|
12
|
-
|
|
11
|
+
package with runtime dependencies only and defaults to a safe import smoke
|
|
12
|
+
test.
|
|
13
13
|
* `ghcr.io/ryancswallace/benchmatrix-test` is the test image. It includes the
|
|
14
|
-
|
|
14
|
+
project test and release tooling and defaults to `python -m pytest -q`.
|
|
15
15
|
|
|
16
16
|
Both images run as a non-root user.
|
|
17
17
|
|
|
@@ -65,7 +65,7 @@ The default command should print `BenchmarkCase`. To run a custom command:
|
|
|
65
65
|
|
|
66
66
|
```bash
|
|
67
67
|
docker run --rm ghcr.io/ryancswallace/benchmatrix:latest \
|
|
68
|
-
|
|
68
|
+
python -c "from benchmatrix import BenchmarkCase; print(BenchmarkCase.__name__)"
|
|
69
69
|
```
|
|
70
70
|
|
|
71
71
|
## Tag policy
|
|
@@ -9,13 +9,13 @@ Use this site by task type:
|
|
|
9
9
|
|
|
10
10
|
* **Tutorials** teach the first complete workflow from zero to a benchmark run.
|
|
11
11
|
* **How-to guides** solve focused tasks such as creating a benchmark matrix or
|
|
12
|
-
|
|
12
|
+
parsing saved JSON results.
|
|
13
13
|
* **Reference** records exact APIs, compatibility commitments, and automation
|
|
14
|
-
|
|
14
|
+
commands.
|
|
15
15
|
* **Explanation** describes architecture, policy, security posture, and design
|
|
16
|
-
|
|
16
|
+
tradeoffs.
|
|
17
17
|
* **Maintainer runbooks** are operational checklists for releases, incidents,
|
|
18
|
-
|
|
18
|
+
dependency updates, and repository administration.
|
|
19
19
|
|
|
20
20
|
## Quick start
|
|
21
21
|
|
|
@@ -90,7 +90,7 @@ true:
|
|
|
90
90
|
|
|
91
91
|
* the issue is high impact for installed users;
|
|
92
92
|
* a safe, minimal patch can be prepared without carrying substantial branch
|
|
93
|
-
|
|
93
|
+
maintenance cost;
|
|
94
94
|
* the affected release still has meaningful user adoption;
|
|
95
95
|
* the maintainer has capacity to validate and publish the backport.
|
|
96
96
|
|
|
@@ -60,31 +60,37 @@ The repository uses focused workflows that call the same Make targets used
|
|
|
60
60
|
locally:
|
|
61
61
|
|
|
62
62
|
* `.github/workflows/ci.yml` runs quality checks, supported-Python tests,
|
|
63
|
-
|
|
64
|
-
|
|
63
|
+
coverage and JUnit report artifact upload, minimum-dependency tests,
|
|
64
|
+
packaging smoke checks, SBOM generation, and scheduled dependency audits.
|
|
65
65
|
* `.github/workflows/docs.yml` builds docs, checks generated-site links, uploads
|
|
66
|
-
|
|
67
|
-
|
|
66
|
+
the link-check report, and deploys the MkDocs site from `main` through GitHub
|
|
67
|
+
Pages.
|
|
68
|
+
* `.github/workflows/draft-release.yml` validates release metadata when a `v*`
|
|
69
|
+
tag is pushed, extracts release notes from `CHANGELOG.md`, builds and
|
|
70
|
+
verifies release assets, and creates or updates the draft GitHub Release
|
|
71
|
+
with the package source distribution, wheel, and SBOM attached; it also
|
|
72
|
+
supports manual dispatch for retrying an existing tag.
|
|
68
73
|
* `.github/workflows/release.yml` builds release artifacts, uploads package
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
74
|
+
distributions and the SBOM as separate Actions artifacts, attaches them to
|
|
75
|
+
the GitHub Release, attests them, publishes the distributions to PyPI through
|
|
76
|
+
Trusted Publishing, and verifies installation from PyPI after publication.
|
|
72
77
|
* `.github/workflows/release-verify.yml` manually re-runs post-release
|
|
73
|
-
|
|
78
|
+
installation verification from PyPI.
|
|
74
79
|
* `.github/workflows/docker.yml` builds runtime and test Docker images, scans
|
|
75
|
-
|
|
76
|
-
|
|
80
|
+
them for critical vulnerabilities, and publishes them to GHCR from `main` and
|
|
81
|
+
`v*` tags.
|
|
77
82
|
* `.github/workflows/workflow-lint.yml` runs actionlint and zizmor for workflow
|
|
78
|
-
|
|
79
|
-
|
|
83
|
+
configuration changes, using `.github/zizmor.yml` for project-specific audit
|
|
84
|
+
policy.
|
|
80
85
|
* `.github/workflows/codeql.yml`, `.github/workflows/dependency-review.yml`,
|
|
81
|
-
|
|
82
|
-
|
|
86
|
+
and `.github/workflows/scorecard.yml` provide GitHub-native security and
|
|
87
|
+
supply-chain checks.
|
|
83
88
|
|
|
84
89
|
Standalone SBOM and artifact-attestation workflows are intentionally not added:
|
|
85
90
|
`make build`, CI packaging, and the release workflow already generate SBOMs and
|
|
86
|
-
release provenance without an extra scheduler. Release drafting is
|
|
87
|
-
|
|
91
|
+
release provenance without an extra scheduler. Release drafting is limited to a
|
|
92
|
+
small tag-triggered workflow so the deployment action remains publishing the
|
|
93
|
+
reviewed GitHub Release.
|
|
88
94
|
|
|
89
95
|
## Pull request labels
|
|
90
96
|
|
|
@@ -93,7 +99,7 @@ Pull request labels are applied automatically by GitHub Actions:
|
|
|
93
99
|
* `.github/labeler.yml` maps changed files and branch names to labels.
|
|
94
100
|
* `.github/workflows/labeler.yml` runs `actions/labeler` on pull requests.
|
|
95
101
|
* `.github/labels.yml` documents the repository's recommended label set for
|
|
96
|
-
|
|
102
|
+
maintainers.
|
|
97
103
|
|
|
98
104
|
Labels are not synced automatically. Create or update labels manually in GitHub
|
|
99
105
|
when the recommended label set changes.
|
|
@@ -6,11 +6,11 @@ on stable facts.
|
|
|
6
6
|
## Available reference material
|
|
7
7
|
|
|
8
8
|
* [API reference](api/index.md), generated from package docstrings by
|
|
9
|
-
|
|
9
|
+
mkdocstrings during the MkDocs build.
|
|
10
10
|
* [Compatibility](compatibility.md), covering supported Python versions and API
|
|
11
|
-
|
|
11
|
+
compatibility expectations.
|
|
12
12
|
* [Configuration and automation](configuration.md), covering local development
|
|
13
|
-
|
|
13
|
+
commands and generated artifacts.
|
|
14
14
|
|
|
15
15
|
Use tutorials and how-to guides when learning a workflow. Use this section when
|
|
16
16
|
checking names, options, and support boundaries.
|