benchmatrix 0.2.1__tar.gz → 0.2.4__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (71) hide show
  1. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/CHANGELOG.md +49 -17
  2. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/CITATION.cff +2 -2
  3. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/PKG-INFO +9 -4
  4. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/README.md +8 -3
  5. benchmatrix-0.2.4/RELEASING.md +28 -0
  6. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/architecture.md +1 -1
  7. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/deprecations.md +2 -2
  8. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/first-release.md +2 -2
  9. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/github-actions-security.md +8 -8
  10. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/lifecycle.md +4 -4
  11. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/performance.md +7 -7
  12. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/publishing.md +46 -18
  13. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/repository-settings.md +5 -5
  14. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/how-to/run-with-docker.md +4 -4
  15. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/index.md +4 -4
  16. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/reference/compatibility.md +1 -1
  17. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/reference/configuration.md +23 -17
  18. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/reference/index.md +3 -3
  19. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/container-release.md +26 -26
  20. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/dependency-update.md +25 -25
  21. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/failed-ci.md +6 -6
  22. benchmatrix-0.2.4/docs/runbooks/release.md +177 -0
  23. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/repository-setup.md +35 -35
  24. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/security-report.md +5 -5
  25. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/mkdocs.yml +29 -1
  26. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/pyproject.toml +2 -1
  27. benchmatrix-0.2.4/tests/test_create_release_pr.py +78 -0
  28. benchmatrix-0.2.4/tests/test_create_release_tag.py +79 -0
  29. benchmatrix-0.2.4/tests/test_prepare_release.py +143 -0
  30. benchmatrix-0.2.4/tests/test_release_preflight.py +96 -0
  31. benchmatrix-0.2.1/RELEASING.md +0 -61
  32. benchmatrix-0.2.1/docs/runbooks/release.md +0 -147
  33. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/.gitignore +0 -0
  34. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/CODE_OF_CONDUCT.md +0 -0
  35. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/CONTRIBUTING.md +0 -0
  36. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/LICENSE +0 -0
  37. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/SECURITY.md +0 -0
  38. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/_scripts/gen_api_reference.py +0 -0
  39. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/changelog.md +0 -0
  40. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/contributing.md +0 -0
  41. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/development.md +0 -0
  42. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/ai.md +0 -0
  43. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/dependency-policy.md +0 -0
  44. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/deployment-environments.md +0 -0
  45. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/license-policy.md +0 -0
  46. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/repository-health.md +0 -0
  47. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/secrets.md +0 -0
  48. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/explanation/threat-model.md +0 -0
  49. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/how-to/create-benchmark-matrix.md +0 -0
  50. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/how-to/parse-results.md +0 -0
  51. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/how-to/troubleshooting.md +0 -0
  52. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/incident-response.md +0 -0
  53. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/index.md +0 -0
  54. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/runbooks/ownership-transfer.md +0 -0
  55. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/docs/tutorials/first-benchmark.md +0 -0
  56. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/examples/test_factorial_benchmarks.py +0 -0
  57. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/examples/test_factorial_benchmarks_simplified.py +0 -0
  58. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/noxfile.py +0 -0
  59. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/__init__.py +0 -0
  60. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/_schema.py +0 -0
  61. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/bench_harness.py +0 -0
  62. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/bench_results.py +0 -0
  63. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/exceptions.py +0 -0
  64. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/src/benchmatrix/py.typed +0 -0
  65. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/fixtures/benchmark_results/invalid_tail_missing_data.json +0 -0
  66. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/fixtures/benchmark_results/mixed_metrics.json +0 -0
  67. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/test_bench_harness.py +0 -0
  68. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/test_bench_results.py +0 -0
  69. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/test_integration.py +0 -0
  70. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/test_properties.py +0 -0
  71. {benchmatrix-0.2.1 → benchmatrix-0.2.4}/tests/test_public_api.py +0 -0
@@ -12,12 +12,44 @@ additional pre-1.0 compatibility expectations described in
12
12
 
13
13
  ### Changed
14
14
 
15
+ ### Deprecated
16
+
17
+ ### Removed
18
+
19
+ ### Fixed
20
+
21
+ ### Security
22
+
23
+ ## 0.2.4 - 2026-06-24
24
+
25
+ ### Fixed
26
+
27
+ * Check out repository for context of `release` workflow
28
+
29
+ ## 0.2.3 - 2026-06-23
30
+
31
+ ### Fixed
32
+
33
+ * Ensure all assets uploaded to draft GitHub release
34
+
35
+ ## 0.2.2 - 2026-06-23
36
+
37
+ ### Changed
38
+
39
+ * Change documentation site from built-in MkDocs theme to Material theme
40
+ * Automate several steps of the release process
41
+
42
+ ### Fixed
43
+
44
+ * Ensure all assets, including wheels and SBOMs attached to GitHub releases
45
+ * Standardize Markdown formatting on four spaces for indentation
46
+
15
47
  ## 0.2.1 - 2026-06-22
16
48
 
17
49
  ### Changed
18
50
 
19
51
  * Fix release and release-verify workflow bugs preventing creation of PyPI
20
- releases.
52
+ releases.
21
53
 
22
54
  ## 0.2.0 - 2026-06-22
23
55
 
@@ -26,31 +58,31 @@ additional pre-1.0 compatibility expectations described in
26
58
  * Initial benchmark matrix utilities.
27
59
  * pytest-benchmark JSON parsing and display utilities.
28
60
  * Automated linting, typing, security, dependency, test, coverage, and package
29
- validation for local development and pull requests, including scheduled
30
- audits of locked dependencies for known vulnerabilities.
61
+ validation for local development and pull requests, including scheduled
62
+ audits of locked dependencies for known vulnerabilities.
31
63
  * Pre-commit automation, secret scanning, Markdown linting, GitHub Actions
32
- workflow linting, and repository text and binary file attributes.
64
+ workflow linting, and repository text and binary file attributes.
33
65
  * MkDocs documentation site with strict builds, generated API reference pages,
34
- and operational maintainer runbooks.
66
+ and operational maintainer runbooks.
35
67
  * Reproducible CycloneDX SBOM generation for locked runtime dependencies.
36
68
  * uv-backed nox automation for supported-Python tests, quality checks, and
37
- release artifact smoke testing.
69
+ release artifact smoke testing.
38
70
  * GitHub pull request auto-labeling and labels-as-code configuration for
39
- maintainers.
71
+ maintainers.
40
72
  * Dependabot automation for Python, Node, pre-commit, GitHub Actions, Docker,
41
- and devcontainer dependency updates.
73
+ and devcontainer dependency updates.
42
74
  * Repository settings-as-code plus external setup checklists for branch
43
- protection, security features, Pages, environments, and PyPI publishing.
75
+ protection, security features, Pages, environments, and PyPI publishing.
44
76
  * Focused Python dependency groups for test, lint, type, docs, security,
45
- release, and automation tooling, with `dev` as the aggregate group.
77
+ release, and automation tooling, with `dev` as the aggregate group.
46
78
  * Compatibility, lifecycle, security-fix, release-branch, and deprecation
47
- policy documentation.
79
+ policy documentation.
48
80
  * GitHub Actions CI/CD workflows for quality checks, multi-version and
49
- cross-OS tests, docs deployment, PyPI Trusted Publishing, artifact
50
- attestations, CodeQL, dependency review, OpenSSF Scorecard, and workflow
51
- linting.
81
+ cross-OS tests, docs deployment, PyPI Trusted Publishing, artifact
82
+ attestations, CodeQL, dependency review, OpenSSF Scorecard, and workflow
83
+ linting.
52
84
  * CI test and coverage report artifacts, documentation link checking, minimum-
53
- dependency tests, and post-release PyPI installation verification.
85
+ dependency tests, and post-release PyPI installation verification.
54
86
  * Docker runtime and test images, local Docker targets, Docker-outside-of-Docker
55
- devcontainer support, Dockerfile linting, GHCR publishing, image
56
- SBOM/provenance, and critical-vulnerability image scanning.
87
+ devcontainer support, Dockerfile linting, GHCR publishing, image
88
+ SBOM/provenance, and critical-vulnerability image scanning.
@@ -10,8 +10,8 @@ authors:
10
10
  given-names: "Ryan"
11
11
  email: "ryancswallace@gmail.com"
12
12
  alias: "ryancswallace"
13
- version: 0.2.1
14
- date-released: "2026-06-22"
13
+ version: 0.2.4
14
+ date-released: "2026-06-24"
15
15
  license: MIT
16
16
  repository-code: "https://github.com/ryancswallace/benchmatrix"
17
17
  url: "https://github.com/ryancswallace/benchmatrix"
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: benchmatrix
3
- Version: 0.2.1
3
+ Version: 0.2.4
4
4
  Summary: A pytest-benchmark matrix, metadata, and JSON results layer.
5
5
  Project-URL: Homepage, https://github.com/ryancswallace/benchmatrix
6
6
  Project-URL: Changelog, https://github.com/ryancswallace/benchmatrix/blob/main/CHANGELOG.md
@@ -119,11 +119,16 @@ For local development from this repository:
119
119
  make ready
120
120
  ```
121
121
 
122
- To try the package from another project before the first PyPI release, install
123
- from GitHub:
122
+ To install the latest release from PyPI:
124
123
 
125
124
  ```bash
126
- uv add "benchmatrix @ git+https://github.com/ryancswallace/benchmatrix"
125
+ uv add benchmatrix
126
+ ```
127
+
128
+ or
129
+
130
+ ```bash
131
+ pip install benchmatrix
127
132
  ```
128
133
 
129
134
  ## Documentation
@@ -88,11 +88,16 @@ For local development from this repository:
88
88
  make ready
89
89
  ```
90
90
 
91
- To try the package from another project before the first PyPI release, install
92
- from GitHub:
91
+ To install the latest release from PyPI:
93
92
 
94
93
  ```bash
95
- uv add "benchmatrix @ git+https://github.com/ryancswallace/benchmatrix"
94
+ uv add benchmatrix
95
+ ```
96
+
97
+ or
98
+
99
+ ```bash
100
+ pip install benchmatrix
96
101
  ```
97
102
 
98
103
  ## Documentation
@@ -0,0 +1,28 @@
1
+ # Release policy
2
+
3
+ benchmatrix uses [Semantic Versioning](https://semver.org/).
4
+
5
+ ## Compatibility
6
+
7
+ While the project is pre-1.0:
8
+
9
+ * patch releases contain compatible fixes and documentation improvements;
10
+ * minor releases may introduce breaking API changes;
11
+ * breaking changes should be called out clearly in the changelog.
12
+
13
+ Starting with 1.0, incompatible public API changes require a major release.
14
+ The supported public API is the set of names exported from
15
+ `benchmatrix.__init__`. Private modules and names beginning with an underscore
16
+ are not covered by the compatibility guarantee.
17
+
18
+ Python version support may change in a minor release before 1.0 and in a major
19
+ release after 1.0. Dropping a Python version will be documented in advance when
20
+ practical.
21
+
22
+ Release branches are not maintained as standing support branches. The active
23
+ support branch is `main`; temporary release or security branches may be used for
24
+ coordination and retired after release.
25
+
26
+ ## Release process
27
+
28
+ See the detailed operational checklist in `docs/runbooks/release.md`.
@@ -14,7 +14,7 @@ replace its calibration, timing, statistics, terminal reporting, or JSON export.
14
14
  * `bench_results.py` parses and displays saved benchmark results.
15
15
  * `exceptions.py` contains package-specific exceptions.
16
16
  * `_schema.py` is a private schema constant module shared by implementation
17
- code.
17
+ code.
18
18
  * `__init__.py` defines the public package exports.
19
19
 
20
20
  ## Design constraints
@@ -15,7 +15,7 @@ Deprecation is not required for:
15
15
  * private names beginning with `_`;
16
16
  * undocumented implementation details;
17
17
  * behavior that is removed immediately for security or serious correctness
18
- reasons.
18
+ reasons.
19
19
 
20
20
  ## Timing
21
21
 
@@ -39,7 +39,7 @@ major release unless release notes document a narrower exception.
39
39
 
40
40
  1. confirm the changelog mentioned the deprecation;
41
41
  2. confirm the removal timing has arrived or the security/correctness exception
42
- applies;
42
+ applies;
43
43
  3. remove the behavior and tests together;
44
44
  4. update API docs, tutorials, examples, and compatibility notes;
45
45
  5. run `make check` and `make test-matrix`.
@@ -14,11 +14,11 @@ structure to support users after publication.
14
14
  * the built wheel can be installed and imported in a clean environment;
15
15
  * changelog has a dated version section;
16
16
  * `pyproject.toml`, `CITATION.cff`, the Git tag, and release notes use the same
17
- version;
17
+ version;
18
18
  * security and support policies are present;
19
19
  * the GitHub `pypi` environment exists with trusted maintainer reviewers;
20
20
  * PyPI Trusted Publishing is configured for `release.yml` and the `pypi`
21
- environment;
21
+ environment;
22
22
  * a maintainer knows how to verify the package after PyPI publication.
23
23
 
24
24
  If any item is missing, fix it before uploading artifacts. A delayed first
@@ -7,21 +7,21 @@ CI should verify project health without granting unnecessary write access.
7
7
  * Default workflow permissions are read-only.
8
8
  * Checkout does not persist credentials unless a job needs to push.
9
9
  * The pull request labeler uses `pull_request_target` without checkout and only
10
- receives the permission needed to add existing pull request labels.
10
+ receives the permission needed to add existing pull request labels.
11
11
  * Workflow actions use carefully versioned refs and Dependabot updates. The
12
- `.github/zizmor.yml` policy records this choice and disables only zizmor's
13
- SHA-pinning audit until the project adopts full action SHA pinning.
12
+ `.github/zizmor.yml` policy records this choice and disables only zizmor's
13
+ SHA-pinning audit until the project adopts full action SHA pinning.
14
14
  * Scheduled audits are separated from normal pull request validation.
15
15
  * Documentation deployment uses the `github-pages` environment with `pages:
16
- write` and OIDC only in the deployment job.
16
+ write` and OIDC only in the deployment job.
17
17
  * Package publishing uses the `pypi` environment and PyPI Trusted Publishing
18
- instead of long-lived publishing tokens.
18
+ instead of long-lived publishing tokens.
19
19
  * Release artifacts are attested with GitHub artifact attestations before they
20
- are published.
20
+ are published.
21
21
  * Docker images are built in a dedicated workflow, scanned with Trivy, and only
22
- pushed from non-pull-request events with `packages: write`.
22
+ pushed from non-pull-request events with `packages: write`.
23
23
  * Artifacts should be limited to build outputs such as distributions, container
24
- images, SBOM files, and attestation metadata.
24
+ images, SBOM files, and attestation metadata.
25
25
 
26
26
  ## Pull requests
27
27
 
@@ -27,14 +27,14 @@ of truth.
27
27
  ## Change stages
28
28
 
29
29
  * **Experimental**: behavior may change without deprecation. This includes
30
- undocumented internals, private modules, and private names.
30
+ undocumented internals, private modules, and private names.
31
31
  * **Documented**: behavior appears in docs or examples and should receive a
32
- changelog note when changed.
32
+ changelog note when changed.
33
33
  * **Stable public API**: exported from `benchmatrix.__init__`, documented in the
34
- API reference, and covered by tests.
34
+ API reference, and covered by tests.
35
35
  * **Deprecated**: retained temporarily with migration guidance.
36
36
  * **Removed**: no longer available after the documented removal release or after
37
- an urgent security/correctness removal.
37
+ an urgent security/correctness removal.
38
38
 
39
39
  ## Public API stability
40
40
 
@@ -39,14 +39,14 @@ metadata with saved runs.
39
39
  Common pitfalls:
40
40
 
41
41
  * A target returning a generator, coroutine, future, query plan, or other lazy
42
- object may only measure object creation. Resolve lazy work inside the
43
- synchronous target wrapper.
42
+ object may only measure object creation. Resolve lazy work inside the
43
+ synchronous target wrapper.
44
44
  * Fresh-input factories and copying run outside the timed target body. Put
45
- construction inside the target only when setup cost is part of the operation
46
- being measured.
45
+ construction inside the target only when setup cost is part of the operation
46
+ being measured.
47
47
  * Reused mutable inputs can drift across invocations. Use `fresh_inputs=True`,
48
- `deep_copy`, or a domain-specific copier when targets mutate their inputs.
48
+ `deep_copy`, or a domain-specific copier when targets mutate their inputs.
49
49
  * Small differences need repeated runs, distribution checks, and controlled
50
- environments before they become conclusions.
50
+ environments before they become conclusions.
51
51
  * A faster implementation still needs ordinary correctness tests. Benchmark
52
- tests should not be the only proof that two implementations do the same work.
52
+ tests should not be the only proof that two implementations do the same work.
@@ -13,22 +13,39 @@ make build
13
13
 
14
14
  This command builds the source distribution and wheel, checks metadata with
15
15
  Twine, installs the built wheel into a temporary environment, imports the public
16
- package, and generates the runtime CycloneDX SBOM at `dist/benchmatrix.cdx.json`.
16
+ package, generates the runtime CycloneDX SBOM at `dist/benchmatrix.cdx.json`,
17
+ and fails unless the release artifact files in `dist/` are the expected
18
+ wheel, source distribution, and SBOM for the package version in `pyproject.toml`.
17
19
 
18
20
  The release workflow runs the same command before publishing, so local release
19
21
  checks and CI release checks exercise the same path.
20
22
 
21
23
  ## Version metadata
22
24
 
23
- The project currently uses static package metadata. A release version must be
24
- updated in these files:
25
+ The project uses static package metadata, but release preparation is automated.
26
+ After the `CHANGELOG.md` entries under `## Unreleased` are ready, export the
27
+ release version without a leading `v`, then run the release pull request
28
+ preparation target:
25
29
 
26
- * `pyproject.toml`, `[project] version`;
27
- * `CITATION.cff`, `version`;
28
- * `CITATION.cff`, `date-released` once the release date is known;
29
- * `CHANGELOG.md`, a versioned section with the release date.
30
+ ```bash
31
+ export BENCHMATRIX_RELEASE_VERSION=X.Y.Z
32
+ make release-pr-ready
33
+ ```
34
+
35
+ This command checks release prerequisites, updates `pyproject.toml`,
36
+ `CITATION.cff`, and `CHANGELOG.md`, sets `date-released` in `CITATION.cff`,
37
+ creates the dated changelog section, runs `uv lock` after the `pyproject.toml`
38
+ version changes, runs the full local `make check` suite, and creates or reuses
39
+ the standardized release branch, commit, and GitHub pull request. Use
40
+ `RELEASE_DATE=YYYY-MM-DD` when the release date should be explicit.
30
41
 
31
- The Git tag must be named `vX.Y.Z` and must match the package version `X.Y.Z`.
42
+ After the release pull request is merged, create the Git tag with:
43
+
44
+ ```bash
45
+ make release-tag
46
+ ```
47
+
48
+ The tag must be named `vX.Y.Z` and must match the package version `X.Y.Z`.
32
49
  Do not publish if the tag and metadata disagree.
33
50
 
34
51
  ## Changelog and release notes
@@ -44,6 +61,14 @@ changes before routine maintenance items.
44
61
 
45
62
  ## GitHub Release publishing flow
46
63
 
64
+ Pushing an annotated tag named `vX.Y.Z` starts `.github/workflows/draft-release.yml`.
65
+ That workflow validates release metadata, extracts notes from `CHANGELOG.md`,
66
+ builds the source distribution, wheel, and SBOM, creates or updates a draft
67
+ GitHub Release for the pushed tag, attaches those files to the draft, and
68
+ verifies that the package assets are present before the workflow succeeds. The
69
+ workflow also supports manual dispatch for rerunning draft creation and asset
70
+ attachment for an existing tag.
71
+
47
72
  `.github/workflows/release.yml` publishes when a GitHub Release is published.
48
73
  The workflow listens for:
49
74
 
@@ -54,11 +79,15 @@ release:
54
79
  ```
55
80
 
56
81
  That means creating or editing a draft release is safe. Publishing the release is
57
- the deployment action. The publish job is guarded with `startsWith(github.ref,
58
- 'refs/tags/v')`, so PyPI publication only runs from version tags such as
59
- `v0.2.0`. The workflow builds from the tagged source, uploads package
60
- distributions and the SBOM as separate release artifacts, attests them, and
61
- publishes only the distributions to PyPI from the `pypi` environment.
82
+ the deployment action. The draft-release workflow uses `GITHUB_TOKEN`, so it
83
+ intentionally stops before publication; GitHub suppresses most follow-on
84
+ workflow runs caused by that token to avoid recursive automation.
85
+
86
+ The publish job is guarded with `startsWith(github.ref, 'refs/tags/v')`, so PyPI
87
+ publication only runs from version tags such as `v0.2.0`. The workflow rebuilds
88
+ from the tagged source, uploads package distributions and the SBOM as separate
89
+ Actions artifacts, refreshes those files on the GitHub Release, attests them,
90
+ and publishes only the distributions to PyPI from the `pypi` environment.
62
91
 
63
92
  The workflow also has manual dispatch. Treat `publish=false` as a build smoke
64
93
  check. Treat `publish=true` as a real publication and only run it from an exact
@@ -89,10 +118,10 @@ GitHub repository settings:
89
118
  1. Create an environment named `pypi`.
90
119
  2. Add trusted maintainer reviewers.
91
120
  3. Restrict deployment branches or tags to release refs if the repository plan
92
- supports that control.
121
+ supports that control.
93
122
  4. Do not add PyPI secrets when Trusted Publishing works.
94
123
  5. If a temporary token is unavoidable, use a project-scoped token, store it only
95
- in the `pypi` environment, and remove it after Trusted Publishing succeeds.
124
+ in the `pypi` environment, and remove it after Trusted Publishing succeeds.
96
125
 
97
126
  ## Documentation artifacts
98
127
 
@@ -120,6 +149,5 @@ rm -rf "$verify_env"
120
149
 
121
150
  The command should print `BenchmarkCase`. The release workflow's verification
122
151
  job runs that same post-publication smoke check after a GitHub Release is
123
- published. Also
124
- inspect the PyPI project page, release workflow logs, uploaded artifacts, SBOM,
125
- and attestations.
152
+ published. Also inspect the PyPI project page, release workflow logs, GitHub
153
+ Release assets, Actions artifacts, SBOM, and attestations.
@@ -10,12 +10,12 @@ The repository includes `.github/settings.yml` for the GitHub Settings app. If
10
10
  the app is installed, changes merged to the default branch sync these settings:
11
11
 
12
12
  * repository description, homepage, topics, issues/projects/wiki flags, and
13
- branch deletion after merge;
13
+ branch deletion after merge;
14
14
  * allowed merge strategies: squash and rebase enabled, merge commits disabled;
15
15
  * Dependabot alerts and automated security fixes;
16
16
  * `main` branch protection, including required reviews, code owner review,
17
- stale review dismissal, last-push approval, required status checks, linear
18
- history, blocked force pushes, blocked deletions, and conversation resolution;
17
+ stale review dismissal, last-push approval, required status checks, linear
18
+ history, blocked force pushes, blocked deletions, and conversation resolution;
19
19
  * deployment environments and their branch or tag deployment policies.
20
20
 
21
21
  Install the Settings app only after branch protection is active enough that
@@ -106,7 +106,7 @@ The GitHub Settings app syncs these environments from `.github/settings.yml`:
106
106
 
107
107
  * `github-pages` for documentation deployment, restricted to `main`;
108
108
  * `pypi` for package publishing, restricted to `v*` tags and requiring
109
- approval from project maintainer(s).
109
+ approval from project maintainer(s).
110
110
 
111
111
  Keep secrets out of the environment when using PyPI Trusted Publishing because
112
112
  OIDC should provide a short-lived publishing token.
@@ -138,7 +138,7 @@ Set these repository profile fields:
138
138
  * description: `Build pytest-benchmark matrices and parse benchmark results with lightweight Python utilities.`;
139
139
  * homepage: `https://ryancswallace.github.io/benchmatrix/` once Pages is live;
140
140
  * topics: `benchmark`, `benchmarking`, `performance`, `pytest`,
141
- `pytest-benchmark`, `python`.
141
+ `pytest-benchmark`, `python`.
142
142
 
143
143
  Disable the wiki and projects unless the project starts using them deliberately.
144
144
  Keep issues enabled.
@@ -8,10 +8,10 @@ convenience images for reproducible local use, CI smoke checks, and examples.
8
8
  The repository builds two images:
9
9
 
10
10
  * `ghcr.io/ryancswallace/benchmatrix` is the runtime image. It installs the
11
- package with runtime dependencies only and defaults to a safe import smoke
12
- test.
11
+ package with runtime dependencies only and defaults to a safe import smoke
12
+ test.
13
13
  * `ghcr.io/ryancswallace/benchmatrix-test` is the test image. It includes the
14
- project test and release tooling and defaults to `python -m pytest -q`.
14
+ project test and release tooling and defaults to `python -m pytest -q`.
15
15
 
16
16
  Both images run as a non-root user.
17
17
 
@@ -65,7 +65,7 @@ The default command should print `BenchmarkCase`. To run a custom command:
65
65
 
66
66
  ```bash
67
67
  docker run --rm ghcr.io/ryancswallace/benchmatrix:latest \
68
- python -c "from benchmatrix import BenchmarkCase; print(BenchmarkCase.__name__)"
68
+ python -c "from benchmatrix import BenchmarkCase; print(BenchmarkCase.__name__)"
69
69
  ```
70
70
 
71
71
  ## Tag policy
@@ -9,13 +9,13 @@ Use this site by task type:
9
9
 
10
10
  * **Tutorials** teach the first complete workflow from zero to a benchmark run.
11
11
  * **How-to guides** solve focused tasks such as creating a benchmark matrix or
12
- parsing saved JSON results.
12
+ parsing saved JSON results.
13
13
  * **Reference** records exact APIs, compatibility commitments, and automation
14
- commands.
14
+ commands.
15
15
  * **Explanation** describes architecture, policy, security posture, and design
16
- tradeoffs.
16
+ tradeoffs.
17
17
  * **Maintainer runbooks** are operational checklists for releases, incidents,
18
- dependency updates, and repository administration.
18
+ dependency updates, and repository administration.
19
19
 
20
20
  ## Quick start
21
21
 
@@ -90,7 +90,7 @@ true:
90
90
 
91
91
  * the issue is high impact for installed users;
92
92
  * a safe, minimal patch can be prepared without carrying substantial branch
93
- maintenance cost;
93
+ maintenance cost;
94
94
  * the affected release still has meaningful user adoption;
95
95
  * the maintainer has capacity to validate and publish the backport.
96
96
 
@@ -60,31 +60,37 @@ The repository uses focused workflows that call the same Make targets used
60
60
  locally:
61
61
 
62
62
  * `.github/workflows/ci.yml` runs quality checks, supported-Python tests,
63
- coverage and JUnit report artifact upload, minimum-dependency tests,
64
- packaging smoke checks, SBOM generation, and scheduled dependency audits.
63
+ coverage and JUnit report artifact upload, minimum-dependency tests,
64
+ packaging smoke checks, SBOM generation, and scheduled dependency audits.
65
65
  * `.github/workflows/docs.yml` builds docs, checks generated-site links, uploads
66
- the link-check report, and deploys the MkDocs site from `main` through GitHub
67
- Pages.
66
+ the link-check report, and deploys the MkDocs site from `main` through GitHub
67
+ Pages.
68
+ * `.github/workflows/draft-release.yml` validates release metadata when a `v*`
69
+ tag is pushed, extracts release notes from `CHANGELOG.md`, builds and
70
+ verifies release assets, and creates or updates the draft GitHub Release
71
+ with the package source distribution, wheel, and SBOM attached; it also
72
+ supports manual dispatch for retrying an existing tag.
68
73
  * `.github/workflows/release.yml` builds release artifacts, uploads package
69
- distributions and the SBOM as separate artifacts, attests them, publishes the
70
- distributions to PyPI through Trusted Publishing, and verifies installation
71
- from PyPI after publication.
74
+ distributions and the SBOM as separate Actions artifacts, attaches them to
75
+ the GitHub Release, attests them, publishes the distributions to PyPI through
76
+ Trusted Publishing, and verifies installation from PyPI after publication.
72
77
  * `.github/workflows/release-verify.yml` manually re-runs post-release
73
- installation verification from PyPI.
78
+ installation verification from PyPI.
74
79
  * `.github/workflows/docker.yml` builds runtime and test Docker images, scans
75
- them for critical vulnerabilities, and publishes them to GHCR from `main` and
76
- `v*` tags.
80
+ them for critical vulnerabilities, and publishes them to GHCR from `main` and
81
+ `v*` tags.
77
82
  * `.github/workflows/workflow-lint.yml` runs actionlint and zizmor for workflow
78
- configuration changes, using `.github/zizmor.yml` for project-specific audit
79
- policy.
83
+ configuration changes, using `.github/zizmor.yml` for project-specific audit
84
+ policy.
80
85
  * `.github/workflows/codeql.yml`, `.github/workflows/dependency-review.yml`,
81
- and `.github/workflows/scorecard.yml` provide GitHub-native security and
82
- supply-chain checks.
86
+ and `.github/workflows/scorecard.yml` provide GitHub-native security and
87
+ supply-chain checks.
83
88
 
84
89
  Standalone SBOM and artifact-attestation workflows are intentionally not added:
85
90
  `make build`, CI packaging, and the release workflow already generate SBOMs and
86
- release provenance without an extra scheduler. Release drafting is also left out
87
- until the project has enough release volume to justify another automation path.
91
+ release provenance without an extra scheduler. Release drafting is limited to a
92
+ small tag-triggered workflow so the deployment action remains publishing the
93
+ reviewed GitHub Release.
88
94
 
89
95
  ## Pull request labels
90
96
 
@@ -93,7 +99,7 @@ Pull request labels are applied automatically by GitHub Actions:
93
99
  * `.github/labeler.yml` maps changed files and branch names to labels.
94
100
  * `.github/workflows/labeler.yml` runs `actions/labeler` on pull requests.
95
101
  * `.github/labels.yml` documents the repository's recommended label set for
96
- maintainers.
102
+ maintainers.
97
103
 
98
104
  Labels are not synced automatically. Create or update labels manually in GitHub
99
105
  when the recommended label set changes.
@@ -6,11 +6,11 @@ on stable facts.
6
6
  ## Available reference material
7
7
 
8
8
  * [API reference](api/index.md), generated from package docstrings by
9
- mkdocstrings during the MkDocs build.
9
+ mkdocstrings during the MkDocs build.
10
10
  * [Compatibility](compatibility.md), covering supported Python versions and API
11
- compatibility expectations.
11
+ compatibility expectations.
12
12
  * [Configuration and automation](configuration.md), covering local development
13
- commands and generated artifacts.
13
+ commands and generated artifacts.
14
14
 
15
15
  Use tutorials and how-to guides when learning a workflow. Use this section when
16
16
  checking names, options, and support boundaries.