testguard-cli 0.8.1__tar.gz → 0.9.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- testguard_cli-0.9.1/.github/scripts/build-one-pager.mjs +95 -0
- testguard_cli-0.9.1/.github/scripts/check-cost-budget.mjs +79 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/scripts/sync-release-version.mjs +7 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/workflows/auto-merge.yml +1 -1
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/workflows/release.yml +33 -5
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/workflows/scheduled-release.yml +88 -8
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/CHANGELOG.md +243 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/PKG-INFO +39 -7
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/README.md +38 -6
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/action.yml +1 -1
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/docs/testguard-explained.html +80 -7
- testguard_cli-0.9.1/docs/testguard-explained.pdf +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/docs-canonical/REQUIREMENTS.md +7 -7
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/package-lock.json +169 -168
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/package.json +2 -2
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/packaging/gitlab/testguard.gitlab-ci.yml +2 -2
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/packaging/homebrew/testguard.rb +2 -2
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/pyproject.toml +1 -1
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/cli.mjs +13 -4
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/commands/baseline.mjs +24 -3
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/init/init.mjs +15 -2
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/contention.mjs +20 -1
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/cost.mjs +43 -6
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/replay/replay.mjs +88 -7
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/scaffold/producers.mjs +16 -5
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/scaffold/producers.python.mjs +8 -3
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/scaffold/scaffold.mjs +25 -3
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/baseline.test.mjs +23 -0
- testguard_cli-0.9.1/test/calibration.test.mjs +158 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/classify.test.mjs +35 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/contention.test.mjs +35 -5
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/cost.test.mjs +65 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/init.test.mjs +55 -0
- testguard_cli-0.9.1/test/label.test.mjs +35 -0
- testguard_cli-0.9.1/test/one-pager.test.mjs +88 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/probe-error.test.mjs +37 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/release-sync.test.mjs +6 -2
- testguard_cli-0.9.1/test/replay-verdict.test.mjs +43 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/replay.test.mjs +4 -59
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/scaffold.python.test.mjs +25 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/scaffold.test.mjs +91 -1
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/testguard.claims.json +441 -9
- testguard_cli-0.9.1/testguard.cost-budget.json +96 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/testguard.ignore.json +1 -1
- testguard_cli-0.8.1/.github/scripts/build-one-pager.mjs +0 -49
- testguard_cli-0.8.1/.github/scripts/check-cost-budget.mjs +0 -43
- testguard_cli-0.8.1/docs/testguard-explained.pdf +0 -0
- testguard_cli-0.8.1/test/calibration.test.mjs +0 -73
- testguard_cli-0.8.1/testguard.cost-budget.json +0 -37
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.agents/skills/testguard/SKILL.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.codex/hooks.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.docguard.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.docguardignore +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.gitattributes +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/ISSUE_TEMPLATE/bug_report.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/ISSUE_TEMPLATE/feature_request.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/PULL_REQUEST_TEMPLATE.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/dependabot.yml +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/scripts/check-python-fixture.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/scripts/install-smoke.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/workflows/ci.yml +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.github/workflows/supply-chain.yml +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.gitignore +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.npmignore +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.npmrc +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.pre-commit-config.yaml +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/.pre-commit-hooks.yaml +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/AGENTS.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/CLAUDE.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/CODE_OF_CONDUCT.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/CONTRIBUTING.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/DRIFT-LOG.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/LICENSE +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/PRIVACY.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/SECURITY.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/SUPPORT.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/bench/README.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/cli/testguard.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/docs-canonical/ARCHITECTURE.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/docs-canonical/DATA-MODEL.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/docs-canonical/ENVIRONMENT.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/docs-canonical/SECURITY.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/docs-canonical/TEST-SPEC.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer/.gitignore +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer/README.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer/expected.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer/package.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer/src/export.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer/src/redact.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer/test/export-mocked.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer/test/flaky.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer/test/redact.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer/testguard.claims.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer/vitest.config.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-jest/.gitignore +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-jest/README.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-jest/expected.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-jest/jest.config.js +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-jest/package.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-jest/src/export.js +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-jest/src/redact.js +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-jest/test/export-mocked.test.js +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-jest/test/flaky.test.js +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-jest/test/redact.test.js +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-jest/testguard.claims.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/.gitignore +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/README.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/e2e/flaky.spec.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/e2e/toggle.spec.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/expected.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/package-lock.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/package.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/playwright.config.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/src/toggle.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/test/toggle.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/test-results/.last-run.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/testguard.claims.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-playwright/vitest.config.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/.gitignore +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/README.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/demo/__init__.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/demo/export.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/demo/orphan.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/demo/pipeline.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/demo/redact.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/expected.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/testguard.claims.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/tests/test_export_mocked.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/tests/test_flaky.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/tests/test_patched.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/fixtures/known-answer-python/tests/test_redact.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/GATE-SEMANTICS.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/README.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/baseline.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/brief.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/calibration-websec.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/calibration.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/claims-methods.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/claims.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/evidence-disturbed-run.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/evidence-negative-control.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/evidence-provisional.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/evidence-python.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/evidence-scan-method.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/evidence.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/gate.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/ignore.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/replay.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/examples/status.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/baseline.bad-fingerprint-key.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/baseline.zero-count.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/brief.text-without-heading.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/brief.unknown-verdict-key.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/calibration.backoff-cell-not-wilson.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/calibration.breakdown-does-not-sum-to-n.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/calibration.ci-not-wilson.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/calibration.compound-key-arity-mismatch.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/calibration.fallback-without-basis.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/calibration.p-not-positives-over-n.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/calibration.p-outside-ci.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/calibration.positives-exceed-n.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/claims.absolute-path.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/claims.duplicate-fault-id.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/claims.fault-injection-without-a-substitution.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/claims.method-detail-on-a-specified-method.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/claims.missing-provenance.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/claims.noop-fault.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/claims.occurrence-exceeds-hits.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/claims.parent-traversal.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.annotated-signal-without-reason.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.attribute-signal-not-a-defender.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.attribute-signal-without-attributes.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.byrunner-file-not-resolved.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.contention-without-runners.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.discovered-defender-mocks-subject.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.fault-injection-without-its-evidence.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.flaky-defender-without-flakiness.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.ignored-dirty-on-snapshot.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.killed-by-non-assertion.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.killed-on-red-baseline.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.killed-without-n-runs.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.method-detail-on-an-injecting-run.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.negative-control-not-reached-but-survived.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.negative-control-reached-on-a-kill.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.probe-error-without-message.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.provisional-flag-on-confirmed-run.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.rank-extra-property.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.restore-skipped-in-place.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.single-run-without-provisional.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.target-not-imported-but-killed.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.undeclared-killers-without-reason.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.unknown-verdict.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.unverifiable-without-reason.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/evidence.wrong-fingerprint.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/gate.counts-disagree.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/gate.exit-zero-with-uncovered.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/gate.ignore-cover-without-pattern.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/ignore.missing-reason.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/ignore.short-reason.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/replay.blind-with-a-failing-run.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/replay.duplicate-patch-id.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/status.claim-without-file.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/status.clean-with-new-findings.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/status.unclaimed-hidden-behind-unproven.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/invalid/status.write-test-without-target.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/conformance/schemas.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/lib/fingerprint.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/lib/validate.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/lib/wilson.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/schemas/baseline.schema.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/schemas/brief.schema.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/schemas/calibration.schema.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/schemas/claims.schema.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/schemas/common.schema.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/schemas/evidence.schema.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/schemas/gate.schema.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/schemas/ignore.schema.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/schemas/replay.schema.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/spec/schemas/status.schema.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/admit/admit.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/baseline/baseline.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/brief/brief.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/claims/annotations.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/claims/load.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/claims/removed.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/commands/admit.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/commands/brief.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/commands/claims.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/commands/gate.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/commands/init.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/commands/mcp.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/commands/probe.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/commands/replay.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/commands/scaffold.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/commands/status.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/evidence/writer.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/gate/changed.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/git.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/init/templates/SKILL.md +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/mcp/server.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/mcp/tools.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/attribution.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/classify.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/discover.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/independence.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/inject.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/mocks.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/probe.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/progress.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/pyimports.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/rank.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/runners/index.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/runners/jest.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/runners/playwright.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/runners/python/_testguard_pytest_plugin.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/runners/python/_testguard_report.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/runners/python/_testguard_unittest_main.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/runners/python.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/runners/shared.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/runners/vitest.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/probe/worktree.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/render.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/replay/label.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/status/status.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/util/glob.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/src/util/hash.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/admit.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/annotations.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/attribution.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/brief.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/claims-since.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/discover.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/gate.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/gitlab-template.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/glob.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/independence.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/inject.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/mcp.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/mocks.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/negative-control.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/probe-preconditions.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/probe.fixture.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/probe.jest.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/probe.playwright.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/probe.python.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/progress.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/pyimports.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/rank-aliases.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/runner-argv.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/runner-command.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/runner-playwright.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/runner-python.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/runner-vitest.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/samples/playwright-report.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/samples/python-report-collect-error.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/samples/python-report.json +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/snapshot.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/status.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/worktree.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/test/writer.test.mjs +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/testguard_cli/__init__.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/testguard_cli/wrapper.py +0 -0
- {testguard_cli-0.8.1 → testguard_cli-0.9.1}/vitest.config.mjs +0 -0
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Render docs/testguard-explained.html to PDF with headless Chrome.
|
|
4
|
+
*
|
|
5
|
+
* Chrome rather than a library because this repository has one pinned runtime
|
|
6
|
+
* dependency and keeps it that way: a PDF renderer would be the second, and it
|
|
7
|
+
* would be needed only to build a document nobody imports.
|
|
8
|
+
*
|
|
9
|
+
* Set CHROME to override the binary.
|
|
10
|
+
*
|
|
11
|
+
* (no flags) render; exit 0 with a notice when no Chrome is present,
|
|
12
|
+
* so a checkout without one can still run every other
|
|
13
|
+
* script.
|
|
14
|
+
* --require-chrome a missing Chrome is a failure. The release uses this:
|
|
15
|
+
* there, "skipped the render" means shipping a PDF that
|
|
16
|
+
* disagrees with the HTML beside it, which is the exact
|
|
17
|
+
* drift this script exists to prevent — and it would pass
|
|
18
|
+
* silently, which is worse than failing.
|
|
19
|
+
*
|
|
20
|
+
* After a successful render the page count is checked against the number of
|
|
21
|
+
* `.page` divs in the source. That is not a nicety: `.page` sets
|
|
22
|
+
* `min-height: 279.4mm` and `page-break-after: always`, which is a request,
|
|
23
|
+
* not a constraint. A section a few pixels too tall spills onto a second
|
|
24
|
+
* physical sheet with no warning from Chrome, renumbering nothing and
|
|
25
|
+
* contradicting the "Page N of M" footers the document prints itself. It
|
|
26
|
+
* happened while updating the brief for 0.8.1 and cost two extra pages.
|
|
27
|
+
*/
|
|
28
|
+
import { existsSync, readFileSync } from 'node:fs';
|
|
29
|
+
import { spawnSync } from 'node:child_process';
|
|
30
|
+
import { dirname, join, resolve } from 'node:path';
|
|
31
|
+
import { fileURLToPath } from 'node:url';
|
|
32
|
+
|
|
33
|
+
const root = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
|
34
|
+
const src = join(root, 'docs', 'testguard-explained.html');
|
|
35
|
+
const out = join(root, 'docs', 'testguard-explained.pdf');
|
|
36
|
+
const requireChrome = process.argv.slice(2).includes('--require-chrome');
|
|
37
|
+
|
|
38
|
+
// CHROME overrides rather than merely leads: a binary named explicitly and not
|
|
39
|
+
// found is a mistake worth hearing about, not a reason to quietly render with
|
|
40
|
+
// some other browser the machine happens to have.
|
|
41
|
+
const candidates = process.env.CHROME ? [process.env.CHROME] : [
|
|
42
|
+
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
|
|
43
|
+
'/Applications/Chromium.app/Contents/MacOS/Chromium',
|
|
44
|
+
'/usr/bin/google-chrome',
|
|
45
|
+
'/usr/bin/chromium',
|
|
46
|
+
'/usr/bin/chromium-browser',
|
|
47
|
+
];
|
|
48
|
+
|
|
49
|
+
const chrome = candidates.find((c) => existsSync(c));
|
|
50
|
+
if (!chrome) {
|
|
51
|
+
const how = process.env.CHROME
|
|
52
|
+
? `CHROME=${process.env.CHROME} does not exist`
|
|
53
|
+
: 'set CHROME=<path> to force';
|
|
54
|
+
if (requireChrome) {
|
|
55
|
+
console.error(`no Chrome found and --require-chrome was given; refusing to leave a stale PDF (${how})`);
|
|
56
|
+
process.exit(1);
|
|
57
|
+
}
|
|
58
|
+
console.log(`no Chrome found; skipping PDF build (${how})`);
|
|
59
|
+
process.exit(0);
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
const r = spawnSync(chrome, [
|
|
63
|
+
'--headless',
|
|
64
|
+
'--disable-gpu',
|
|
65
|
+
'--no-sandbox',
|
|
66
|
+
'--no-pdf-header-footer',
|
|
67
|
+
`--print-to-pdf=${out}`,
|
|
68
|
+
`file://${src}`,
|
|
69
|
+
], { encoding: 'utf8' });
|
|
70
|
+
|
|
71
|
+
if (r.status !== 0 || !existsSync(out)) {
|
|
72
|
+
console.error(r.stderr || 'chrome produced no PDF');
|
|
73
|
+
process.exit(1);
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
/** The sheets Chrome actually produced. `/Type /Pages` is the tree node, not a sheet. */
|
|
77
|
+
const sheets = (readFileSync(out).toString('latin1').match(/\/Type\s*\/Page(?![s/\w])/g) ?? []).length;
|
|
78
|
+
/** The sheets the document intends: one `.page` div each. */
|
|
79
|
+
const intended = (readFileSync(src, 'utf8').match(/<div class="page"[\s>]/g) ?? []).length;
|
|
80
|
+
|
|
81
|
+
if (intended === 0) {
|
|
82
|
+
console.error('docs/testguard-explained.html: no `.page` divs found — the page-count guard cannot run');
|
|
83
|
+
process.exit(1);
|
|
84
|
+
}
|
|
85
|
+
if (sheets !== intended) {
|
|
86
|
+
console.error(
|
|
87
|
+
`docs/testguard-explained.pdf: ${sheets} sheets rendered from ${intended} \`.page\` divs.\n` +
|
|
88
|
+
'A .page taller than its sheet spills silently. Letter at margin 0 is 816x1056 CSS px: load the\n' +
|
|
89
|
+
'HTML at that viewport and look for a .page whose getBoundingClientRect().height exceeds 1056.\n' +
|
|
90
|
+
'Measuring at any other width is meaningless, because the text reflows.',
|
|
91
|
+
);
|
|
92
|
+
process.exit(1);
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
console.log(`docs/testguard-explained.pdf written with ${chrome.split('/').pop()} — ${sheets} pages, one per .page div`);
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Fail CI when the self-probe gate outgrows its budget.
|
|
4
|
+
*
|
|
5
|
+
* The decision lives in `src/probe/cost.mjs` as a pure function, unit-tested
|
|
6
|
+
* and covered by a self-claim; this script is only wiring — read the evidence,
|
|
7
|
+
* read the budget, print, choose an exit code.
|
|
8
|
+
*
|
|
9
|
+
* usage: check-cost-budget.mjs <evidence.json> [previous-evidence.json]
|
|
10
|
+
*/
|
|
11
|
+
import { appendFileSync, readFileSync } from 'node:fs';
|
|
12
|
+
import { dirname, join, resolve } from 'node:path';
|
|
13
|
+
import { fileURLToPath } from 'node:url';
|
|
14
|
+
import { costReport, checkCostBudget, renderCostBudget } from '../../src/probe/cost.mjs';
|
|
15
|
+
|
|
16
|
+
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
|
17
|
+
const [evidencePath, previousPath] = process.argv.slice(2);
|
|
18
|
+
if (!evidencePath) {
|
|
19
|
+
console.error('usage: check-cost-budget.mjs <evidence.json> [previous-evidence.json]');
|
|
20
|
+
process.exit(2);
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
const read = (p) => JSON.parse(readFileSync(resolve(p), 'utf8'));
|
|
24
|
+
const budget = read(join(ROOT, 'testguard.cost-budget.json'));
|
|
25
|
+
|
|
26
|
+
// A previous run is informational only. Its absence must never fail the check:
|
|
27
|
+
// the first run on a branch, or a cold cache, has nothing to compare against.
|
|
28
|
+
let previousMs;
|
|
29
|
+
if (previousPath) {
|
|
30
|
+
try {
|
|
31
|
+
previousMs = costReport(read(previousPath).records).totalMs;
|
|
32
|
+
} catch {
|
|
33
|
+
previousMs = undefined;
|
|
34
|
+
}
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
const report = costReport(read(evidencePath).records);
|
|
38
|
+
const decision = checkCostBudget(report, {
|
|
39
|
+
budgetSeconds: budget.seconds,
|
|
40
|
+
perClaimSeconds: budget.perClaimSeconds,
|
|
41
|
+
previousMs,
|
|
42
|
+
});
|
|
43
|
+
const text = renderCostBudget(decision);
|
|
44
|
+
console.log(text);
|
|
45
|
+
|
|
46
|
+
// The gate's own history is the argument for writing this where it is seen:
|
|
47
|
+
// the run went 9.6 -> 23.6 minutes across one merged pull request and nothing
|
|
48
|
+
// said a word, because --cost only ever printed into a log nobody opens while
|
|
49
|
+
// it is passing. A new claim that quietly costs 51 s is the same failure one
|
|
50
|
+
// size down. A job summary is read without opening anything, so the number is
|
|
51
|
+
// in front of a reviewer while the change is still a change.
|
|
52
|
+
if (process.env.GITHUB_STEP_SUMMARY) {
|
|
53
|
+
const rows = (report.claims ?? []).slice(0, 10)
|
|
54
|
+
.map((c) => `| \`${c.claimId}\` | ${(c.ms / 1000).toFixed(1)}s | ${c.runs} |`)
|
|
55
|
+
.join('\n');
|
|
56
|
+
appendFileSync(process.env.GITHUB_STEP_SUMMARY, [
|
|
57
|
+
`## Self-probe cost \u2014 ${decision.ok ? 'within budget' : 'OVER BUDGET'}`,
|
|
58
|
+
'',
|
|
59
|
+
'```',
|
|
60
|
+
text,
|
|
61
|
+
'```',
|
|
62
|
+
'',
|
|
63
|
+
'<details><summary>Most expensive claims</summary>',
|
|
64
|
+
'',
|
|
65
|
+
'| claim | cost | runs |',
|
|
66
|
+
'| --- | ---: | ---: |',
|
|
67
|
+
rows,
|
|
68
|
+
'',
|
|
69
|
+
'A claim pays for every test in the file it names, so the fix for an expensive',
|
|
70
|
+
'claim is usually a cheaper defender rather than a bigger budget.',
|
|
71
|
+
'</details>',
|
|
72
|
+
'',
|
|
73
|
+
].join('\n'));
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
if (!decision.ok) {
|
|
77
|
+
console.log(`::error::${text.split('\n')[0]}`);
|
|
78
|
+
process.exit(1);
|
|
79
|
+
}
|
|
@@ -95,6 +95,13 @@ const surfaces = [
|
|
|
95
95
|
['README.md', /testguard\/v\d+\.\d+\.\d+\/packaging/g, `testguard/v${version}/packaging`],
|
|
96
96
|
['packaging/gitlab/testguard.gitlab-ci.yml', /testguard\/v\d+\.\d+\.\d+\/packaging/g, `testguard/v${version}/packaging`],
|
|
97
97
|
['packaging/gitlab/testguard.gitlab-ci.yml', /(\n version:\n description: [^\n]*\n default: ")\d+\.\d+\.\d+(")/, `$1${version}$2`],
|
|
98
|
+
// The technical brief's masthead. Nothing rendered it for three releases, so
|
|
99
|
+
// it still read v0.6 while the tool shipped 0.8.1. The HTML is the surface;
|
|
100
|
+
// the PDF beside it is a build artifact of this file, which is why the
|
|
101
|
+
// release runs build-one-pager.mjs --require-chrome straight after the sync.
|
|
102
|
+
// Sync the two separately and they disagree — the failure that is worse than
|
|
103
|
+
// being stale, because the document then contradicts itself.
|
|
104
|
+
['docs/testguard-explained.html', /(Technical Brief · v)\d+\.\d+\.\d+( · MIT)/, `$1${version}$2`],
|
|
98
105
|
];
|
|
99
106
|
|
|
100
107
|
if (listSurfaces) {
|
|
@@ -53,7 +53,7 @@ jobs:
|
|
|
53
53
|
// three files the bump writes — test/release-sync.test.mjs fails if
|
|
54
54
|
// a surface is missing, which is how the GitLab template's absence
|
|
55
55
|
// (it would have held every release PR for review) was found.
|
|
56
|
-
const RELEASE_SURFACES = new Set(['package.json', 'package-lock.json', 'pyproject.toml', 'CHANGELOG.md', 'action.yml', 'packaging/homebrew/testguard.rb', 'README.md', 'packaging/gitlab/testguard.gitlab-ci.yml']);
|
|
56
|
+
const RELEASE_SURFACES = new Set(['package.json', 'package-lock.json', 'pyproject.toml', 'CHANGELOG.md', 'action.yml', 'packaging/homebrew/testguard.rb', 'README.md', 'packaging/gitlab/testguard.gitlab-ci.yml', 'docs/testguard-explained.html', 'docs/testguard-explained.pdf']);
|
|
57
57
|
|
|
58
58
|
if (author === 'dependabot[bot]') {
|
|
59
59
|
const m = /from (\d+)\.(\d+)\.\S+ to (\d+)\.(\d+)\./.exec(pr.title || '');
|
|
@@ -118,15 +118,43 @@ jobs:
|
|
|
118
118
|
env:
|
|
119
119
|
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
|
120
120
|
run: |
|
|
121
|
+
set -o pipefail
|
|
121
122
|
TAG="v${{ needs.detect-version.outputs.version }}"
|
|
122
123
|
if git ls-remote --tags origin | grep -q "refs/tags/$TAG$"; then
|
|
123
124
|
echo "✅ Tag $TAG already exists on remote"
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
125
|
+
exit 0
|
|
126
|
+
fi
|
|
127
|
+
git config user.name "github-actions[bot]"
|
|
128
|
+
git config user.email "github-actions[bot]@users.noreply.github.com"
|
|
129
|
+
git tag -a "$TAG" -m "Release $TAG"
|
|
130
|
+
if git push origin "$TAG" 2>/tmp/tag-err; then
|
|
131
|
+
echo "✅ Pushed $TAG"
|
|
132
|
+
exit 0
|
|
133
|
+
fi
|
|
134
|
+
cat /tmp/tag-err
|
|
135
|
+
# A GitHub App may not PUSH a ref whose .github/workflows differ from
|
|
136
|
+
# the CURRENT default branch. That is not a property of this release:
|
|
137
|
+
# it becomes true the moment any workflow change merges to main while
|
|
138
|
+
# the release is in flight, which is exactly how v0.8.1 lost its tag
|
|
139
|
+
# ("refusing to allow a GitHub App to create or update workflow
|
|
140
|
+
# .github/workflows/release.yml without `workflows` permission")
|
|
141
|
+
# seven minutes after an unrelated workflow PR merged.
|
|
142
|
+
#
|
|
143
|
+
# So fall back to the API. It adds a ref to a commit that is ALREADY
|
|
144
|
+
# in the repository and creates no file content, so the restriction
|
|
145
|
+
# does not apply. The push is tried first on purpose: it is the path
|
|
146
|
+
# every previous release took, and this fallback only has to cover
|
|
147
|
+
# the case that currently fails outright.
|
|
148
|
+
if ! grep -q 'workflows. permission' /tmp/tag-err; then
|
|
149
|
+
echo "::error::git push of $TAG failed for a reason other than the workflow-permission restriction — see above."
|
|
150
|
+
exit 1
|
|
129
151
|
fi
|
|
152
|
+
echo "::warning::Pushing $TAG was refused because main's workflows moved during this release; creating the tag through the API instead."
|
|
153
|
+
TAG_SHA=$(gh api -X POST "repos/${GITHUB_REPOSITORY}/git/tags" -f tag="$TAG" -f message="Release $TAG" -f object="$GITHUB_SHA" -f type=commit --jq .sha) || {
|
|
154
|
+
echo "::error::Could not create the tag object for $TAG. Create it by hand: git tag -a $TAG $GITHUB_SHA -m 'Release $TAG' && git push origin $TAG"; exit 1; }
|
|
155
|
+
gh api -X POST "repos/${GITHUB_REPOSITORY}/git/refs" -f ref="refs/tags/$TAG" -f sha="$TAG_SHA" --jq .ref || {
|
|
156
|
+
echo "::error::Could not create refs/tags/$TAG from tag object $TAG_SHA. Create it by hand: git tag -a $TAG $GITHUB_SHA -m 'Release $TAG' && git push origin $TAG"; exit 1; }
|
|
157
|
+
echo "✅ Created $TAG through the API at $GITHUB_SHA"
|
|
130
158
|
- name: Release notes from CHANGELOG
|
|
131
159
|
run: |
|
|
132
160
|
VERSION="${{ needs.detect-version.outputs.version }}"
|
|
@@ -5,10 +5,18 @@ name: Scheduled Release
|
|
|
5
5
|
# tree is green, and open a release PR with auto-merge armed. Once merged, the
|
|
6
6
|
# hourly sweep in release.yml tags and publishes.
|
|
7
7
|
#
|
|
8
|
-
# GITHUB_TOKEN-created PRs do not trigger `pull_request` workflows, so
|
|
9
|
-
# the
|
|
10
|
-
#
|
|
11
|
-
#
|
|
8
|
+
# GITHUB_TOKEN-created PRs do not trigger `pull_request` workflows, so nothing
|
|
9
|
+
# would ever start the required checks and auto-merge would wait forever. This
|
|
10
|
+
# workflow therefore dispatches ci.yml on the release branch itself:
|
|
11
|
+
# `workflow_dispatch` is the one event GITHUB_TOKEN may trigger.
|
|
12
|
+
#
|
|
13
|
+
# That is not the whole story, and v0.9.0 proved it. The `pull_request` runs
|
|
14
|
+
# the PR creates anyway are held at `action_required`, because the repository
|
|
15
|
+
# requires approval from first-time contributors and `github-actions[bot]` is
|
|
16
|
+
# permanently one. Held runs keep check suites QUEUED on the head sha, so the
|
|
17
|
+
# rollup never becomes terminal and auto-merge waits — with every required
|
|
18
|
+
# check already green. This workflow now approves those runs itself, which
|
|
19
|
+
# GITHUB_TOKEN is permitted to do. Only then does no human step remain.
|
|
12
20
|
|
|
13
21
|
on:
|
|
14
22
|
schedule:
|
|
@@ -84,6 +92,11 @@ jobs:
|
|
|
84
92
|
} > /tmp/entry.md
|
|
85
93
|
awk '/^## \[Unreleased\]/ { print; print ""; while ((getline line < "/tmp/entry.md") > 0) print line; next } { print }' CHANGELOG.md > /tmp/CHANGELOG.new && mv /tmp/CHANGELOG.new CHANGELOG.md
|
|
86
94
|
node .github/scripts/sync-release-version.mjs
|
|
95
|
+
# The sync just wrote the brief's masthead; the committed PDF still
|
|
96
|
+
# shows the old one. --require-chrome because a skipped render here
|
|
97
|
+
# would ship an HTML and a PDF that disagree, silently and at exit 0.
|
|
98
|
+
# ubuntu-latest carries /usr/bin/google-chrome.
|
|
99
|
+
node .github/scripts/build-one-pager.mjs --require-chrome
|
|
87
100
|
echo "NEW_VERSION=$NEW" >> "$GITHUB_ENV"
|
|
88
101
|
|
|
89
102
|
- name: Verify the tree is still green
|
|
@@ -116,9 +129,11 @@ jobs:
|
|
|
116
129
|
# DERIVED from the sync script, never retyped here: when the GitLab
|
|
117
130
|
# template became a surface it was left out of the hand-written list,
|
|
118
131
|
# so the first release to reach this guard would have been rejected
|
|
119
|
-
# for writing a file the release is supposed to write. The
|
|
120
|
-
# extras are the ones the bump step writes directly
|
|
121
|
-
|
|
132
|
+
# for writing a file the release is supposed to write. The four
|
|
133
|
+
# extras are the ones the bump step writes directly: three from the
|
|
134
|
+
# bump itself, plus the brief's PDF, which is rendered from a surface
|
|
135
|
+
# rather than being one.
|
|
136
|
+
ALLOWED=$(node .github/scripts/sync-release-version.mjs --list-surfaces; printf 'package.json\npackage-lock.json\nCHANGELOG.md\ndocs/testguard-explained.pdf\n')
|
|
122
137
|
echo "Release surfaces allowed in this diff:"; printf '%s\n' "$ALLOWED" | sed 's/^/ /'
|
|
123
138
|
CHANGED=$(git diff --name-only HEAD --)
|
|
124
139
|
for f in $CHANGED; do
|
|
@@ -133,13 +148,78 @@ jobs:
|
|
|
133
148
|
git add -A
|
|
134
149
|
git commit -m "release: v${NEW_VERSION} — automated weekly batch"
|
|
135
150
|
git push origin "$BRANCH"
|
|
136
|
-
printf 'Automated weekly release. Version bump + CHANGELOG entry + synced version surfaces. Merging tags and publishes to npm and PyPI via release.yml.\n\
|
|
151
|
+
printf 'Automated weekly release. Version bump + CHANGELOG entry + synced version surfaces. Merging tags and publishes to npm and PyPI via release.yml.\n\nTwo things a GITHUB_TOKEN PR needs done for it, both handled by this workflow: it raises no `pull_request` event, so ci.yml was dispatched on `%s` itself; and the `pull_request` runs it does create are held at `action_required` (github-actions[bot] is permanently a first-time contributor), so those were approved too. Auto-merge is armed and completes once the three Node legs pass.\n\nIf it sits anyway: `gh workflow run ci.yml --ref %s` for missing checks, or check for runs still awaiting approval on the head sha.\n' "$BRANCH" "$BRANCH" > /tmp/pr-body.md
|
|
137
152
|
# A bare `gh pr create` failure here reads as a GraphQL error with no
|
|
138
153
|
# hint of the cause, which is how v0.8.1 lost its release PR after
|
|
139
154
|
# everything else in the release had succeeded.
|
|
140
155
|
if PR=$(gh pr create --base main --head "$BRANCH" --title "release: v${NEW_VERSION} — automated weekly batch" --body-file /tmp/pr-body.md 2>/tmp/pr-err); then
|
|
141
156
|
gh pr merge --auto --squash "$PR"
|
|
142
157
|
echo "Release PR: $PR"
|
|
158
|
+
# Nothing else will start the required checks: a PR opened by
|
|
159
|
+
# GITHUB_TOKEN raises no `pull_request` event, so auto-merge would
|
|
160
|
+
# sit on three checks that never run. `workflow_dispatch` is the
|
|
161
|
+
# one event GITHUB_TOKEN may trigger, and this job already holds
|
|
162
|
+
# the `actions: write` it needs. Guarded so a re-run of an
|
|
163
|
+
# already-tested branch does not queue a second identical run.
|
|
164
|
+
HEAD_SHA=$(git rev-parse HEAD)
|
|
165
|
+
RUNS=$(gh api "repos/${GITHUB_REPOSITORY}/commits/${HEAD_SHA}/check-runs" --jq '.total_count')
|
|
166
|
+
if [ "${RUNS:-0}" = "0" ]; then
|
|
167
|
+
gh workflow run ci.yml --ref "$BRANCH"
|
|
168
|
+
echo "Dispatched ci.yml on $BRANCH ($HEAD_SHA)"
|
|
169
|
+
else
|
|
170
|
+
echo "✅ $HEAD_SHA already has $RUNS check run(s)"
|
|
171
|
+
fi
|
|
172
|
+
|
|
173
|
+
# ── The third GITHUB_TOKEN wall, and the last one. ──
|
|
174
|
+
# The `pull_request` runs this PR creates are held at
|
|
175
|
+
# `action_required`: the repository requires approval from
|
|
176
|
+
# first-time contributors, and `github-actions[bot]` is
|
|
177
|
+
# PERMANENTLY one — it never authors a merged commit, so it can
|
|
178
|
+
# never graduate. Those held runs leave check suites QUEUED on the
|
|
179
|
+
# head sha forever, the status rollup never reaches a terminal
|
|
180
|
+
# state, and auto-merge waits on it even though the required
|
|
181
|
+
# contexts (test 20/22/24) are already green from the dispatched
|
|
182
|
+
# run above. v0.9.0 sat here until a human clicked Approve.
|
|
183
|
+
#
|
|
184
|
+
# GITHUB_TOKEN may clear it: `actions: write` carries the approve
|
|
185
|
+
# permission, measured 201 on a bot PR's own held run. It may not
|
|
186
|
+
# approve a REVIEW, which is a different gate and still ours alone.
|
|
187
|
+
#
|
|
188
|
+
# Runs appear asynchronously, so poll. Approving an already-running
|
|
189
|
+
# run answers 403 "not waiting for approval" — success, not failure.
|
|
190
|
+
echo "Clearing held workflow runs on $HEAD_SHA"
|
|
191
|
+
approved=0
|
|
192
|
+
for attempt in 1 2 3 4 5 6; do
|
|
193
|
+
HELD=$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs?head_sha=${HEAD_SHA}" \
|
|
194
|
+
--jq '[.workflow_runs[] | select(.status == "action_required" or .conclusion == "action_required") | .id] | .[]' 2>/dev/null || true)
|
|
195
|
+
for id in $HELD; do
|
|
196
|
+
if gh api -X POST "repos/${GITHUB_REPOSITORY}/actions/runs/${id}/approve" >/dev/null 2>&1; then
|
|
197
|
+
echo " approved run $id"
|
|
198
|
+
approved=$((approved + 1))
|
|
199
|
+
else
|
|
200
|
+
echo " run $id needed no approval"
|
|
201
|
+
fi
|
|
202
|
+
done
|
|
203
|
+
# Stop as soon as the picture is settled, rather than always
|
|
204
|
+
# paying the full poll: nothing held after something was
|
|
205
|
+
# approved means we are done, and nothing held by the third
|
|
206
|
+
# look means the gate is not applying to this run at all.
|
|
207
|
+
if [ -z "$HELD" ] && { [ "$approved" -gt 0 ] || [ "$attempt" -ge 3 ]; }; then
|
|
208
|
+
break
|
|
209
|
+
fi
|
|
210
|
+
sleep 10
|
|
211
|
+
done
|
|
212
|
+
|
|
213
|
+
# Never leave this silent: a run still held here means auto-merge
|
|
214
|
+
# will wait forever, which is the failure this whole block exists
|
|
215
|
+
# to remove. Say so, and name what to click.
|
|
216
|
+
STILL=$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs?head_sha=${HEAD_SHA}" \
|
|
217
|
+
--jq '[.workflow_runs[] | select(.status == "action_required" or .conclusion == "action_required") | .html_url] | .[]' 2>/dev/null || true)
|
|
218
|
+
if [ -n "$STILL" ]; then
|
|
219
|
+
echo "::error::Workflow runs are still held for approval, so auto-merge on $PR will wait forever. Approve them, or re-run this workflow: $STILL"
|
|
220
|
+
exit 1
|
|
221
|
+
fi
|
|
222
|
+
echo "✅ no run is held for approval; auto-merge can complete"
|
|
143
223
|
else
|
|
144
224
|
cat /tmp/pr-err
|
|
145
225
|
if grep -q 'not permitted to create or approve pull requests' /tmp/pr-err; then
|
|
@@ -7,7 +7,177 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.9.1] - 2026-09-19
|
|
11
|
+
|
|
12
|
+
Automated weekly release — everything merged since `v0.9.0`.
|
|
13
|
+
|
|
14
|
+
### Changed
|
|
15
|
+
|
|
16
|
+
- docs: the brief catches up with the tool, and stops falling behind (#112)
|
|
17
|
+
- chore(homebrew): sha256 for v0.9.0 (#110)
|
|
18
|
+
- feat(cost): two ceilings, the report where it is read, and three claims off expensive defenders (#113)
|
|
19
|
+
- fix(release): approve the runs GITHUB_TOKEN's own PR leaves held (#111)
|
|
20
|
+
|
|
21
|
+
|
|
22
|
+
### Added
|
|
23
|
+
- **A second cost ceiling, on the claim.** The total alone cannot see the
|
|
24
|
+
failure it was meant to catch: a new calibration claim cost 51 s for two
|
|
25
|
+
faults purely by being written into `replay.test.mjs` rather than
|
|
26
|
+
`calibration.test.mjs`, and re-pointing it took it to 4.9 s while the total
|
|
27
|
+
never moved. A per-fault average is worse — the regression that prompted
|
|
28
|
+
this budget raised the fault count 59 to 91 while per-fault cost barely
|
|
29
|
+
moved, so an average would have reported everything fine.
|
|
30
|
+
|
|
31
|
+
`perClaimSeconds` is 100 s, just above today's worst claim
|
|
32
|
+
(`TG-ADMIT-NEEDS-ALL-KILLED`, **87 s in CI** for one fault), so the rule
|
|
33
|
+
reads: no new claim may be worse than the worst we already have. Calibrated
|
|
34
|
+
against CI and not the local figure — 75 s, set from that claim's local
|
|
35
|
+
66.5 s, tripped on the first CI run. This file already warns about exactly
|
|
36
|
+
that for the total (950 s was 18% over local but 4.6% over CI, and the first
|
|
37
|
+
PR adding a claim tripped it); the same error applies one level down. A breach names the claim
|
|
38
|
+
and its remedy — a cheaper defender, not a bigger budget. The field is
|
|
39
|
+
optional and additive: a budget without one behaves exactly as before.
|
|
40
|
+
- **The cost report goes to the CI job summary, not only the log.** The gate
|
|
41
|
+
went from 9.6 to 23.6 minutes across one merged pull request and nothing
|
|
42
|
+
said a word, because `--cost` only ever printed into a log nobody opens
|
|
43
|
+
while it is passing. The summary is read without opening anything, so the
|
|
44
|
+
number is in front of a reviewer while the change is still a change.
|
|
45
|
+
|
|
46
|
+
### Performance
|
|
47
|
+
- **Three claims moved off expensive shared defenders: 65 s, ~30 s and ~30 s
|
|
48
|
+
down to under 2 s each.** All three faulted a *pure* function while naming
|
|
49
|
+
an integration test file, and a claim pays for every test in the file it
|
|
50
|
+
names. `TG-SURVIVOR-PROVES-THE-SUBJECT-RUNS` faults `classify()` but was
|
|
51
|
+
defended by `negative-control.test.mjs`, which spawns real runners against
|
|
52
|
+
fixture repositories; `TG-LABEL-NEVER-GUESSES` faults `labelDiff()` and
|
|
53
|
+
`TG-REPLAY-FLAKY-IS-NEVER-CAUGHT` faults `classifyReplay()`, both defended
|
|
54
|
+
by `replay.test.mjs`, which builds a scripted git corpus.
|
|
55
|
+
|
|
56
|
+
Two were a straight move of an already-isolated `describe` block into
|
|
57
|
+
`test/label.test.mjs` and `test/replay-verdict.test.mjs`; only the `classify`
|
|
58
|
+
case needed a test written. Each was proved with `admit` — passes on HEAD and
|
|
59
|
+
fails on every fault, 3/3 — *before* the expensive defender was dropped, and
|
|
60
|
+
never the other way round. The integration files keep their remaining claims,
|
|
61
|
+
where the fixture is the point.
|
|
62
|
+
|
|
63
|
+
### Changed
|
|
64
|
+
- **The cost budget is 1230 s, from 1350.** 1350 was set defensively against a
|
|
65
|
+
reuse-affected 1123 s, because this budget is defined against a from-scratch
|
|
66
|
+
probe and none had run yet. `release.yml` then measured **1081 s** at 129
|
|
67
|
+
faults on the v0.9.0 release commit with no reuse — so the six field-report
|
|
68
|
+
claims cost +82 s, not the +120 s a per-fault average predicted. 1230 is
|
|
69
|
+
~14% over 1081, the same headroom rule that set 1100 against 962. Leaving it
|
|
70
|
+
at 1350 would have let a 25% regression pass unnoticed.
|
|
71
|
+
|
|
10
72
|
### Fixed
|
|
73
|
+
- **The release no longer waits for a human to click Approve.** v0.9.0 sat with
|
|
74
|
+
every required check green and merged nothing. The `pull_request` runs a
|
|
75
|
+
release PR creates are held at `action_required`, because the repository
|
|
76
|
+
requires approval from first-time contributors and `github-actions[bot]` is
|
|
77
|
+
permanently one — it never authors a merged commit, so it can never
|
|
78
|
+
graduate. Held runs leave check suites QUEUED on the head sha, the status
|
|
79
|
+
rollup never reaches a terminal state, and auto-merge waits on it even though
|
|
80
|
+
`test (20/22/24)` are already green from the dispatched run.
|
|
81
|
+
|
|
82
|
+
`scheduled-release.yml` now approves those runs itself. `GITHUB_TOKEN` is
|
|
83
|
+
permitted to do this — `actions: write` carries the approve permission,
|
|
84
|
+
measured 201 against a bot PR's own held run — and the workflow already held
|
|
85
|
+
that permission, so nothing was granted and no secret was added. It may still
|
|
86
|
+
not approve a *review*, which is a different gate and remains ours alone.
|
|
87
|
+
A run left held after the poll is an `::error::` naming the run, never a
|
|
88
|
+
silent wait.
|
|
89
|
+
|
|
90
|
+
This is the third `GITHUB_TOKEN` wall in the same chain, after "raises no
|
|
91
|
+
`pull_request` event" (#105) and "its push raises no `push` event" (the
|
|
92
|
+
hourly sweep in release.yml). All three have the same root: an action taken
|
|
93
|
+
by `GITHUB_TOKEN` does not trigger the event that would normally follow it.
|
|
94
|
+
|
|
95
|
+
### Documentation
|
|
96
|
+
- **The technical brief describes the tool that exists.** It was last rendered
|
|
97
|
+
when it covered one verb, `probe`, and said nothing about `replay` and its
|
|
98
|
+
calibration — the empirical answer to whether an injected fault resembles a
|
|
99
|
+
real bug, which is the question a document arguing "evidence, not output"
|
|
100
|
+
can least afford to omit. A new page covers the eleven verbs and the
|
|
101
|
+
calibration, carrying the README's own framing: the greenfield transfer is
|
|
102
|
+
unproven, and `replay` is the instrument rather than the answer. The verdict
|
|
103
|
+
table, the cost formula and the prior-art section were checked against the
|
|
104
|
+
code and left alone.
|
|
105
|
+
- **The brief's PDF is rendered by the release instead of by memory.** Its
|
|
106
|
+
masthead read `v0.6` through three releases because nothing ever ran
|
|
107
|
+
`build-one-pager.mjs`. The HTML masthead is now a version surface and
|
|
108
|
+
`scheduled-release.yml` re-renders the PDF in the same step as the sync —
|
|
109
|
+
the same step, because an HTML and a PDF that disagree are worse than a
|
|
110
|
+
document uniformly out of date. `--require-chrome` makes a missing browser
|
|
111
|
+
fail the release rather than silently commit a stale PDF, and every render
|
|
112
|
+
must produce exactly one sheet per `.page` div: a section a few pixels too
|
|
113
|
+
tall spills onto a second sheet with no warning from Chrome, contradicting
|
|
114
|
+
the "Page N of M" footers the document prints about itself.
|
|
115
|
+
|
|
116
|
+
## [0.9.0] - 2026-09-18
|
|
117
|
+
|
|
118
|
+
Automated weekly release — everything merged since `v0.8.1`.
|
|
119
|
+
|
|
120
|
+
### Changed
|
|
121
|
+
|
|
122
|
+
- fix: the five findings from the 0.8.1 pocket-archive field report (#107)
|
|
123
|
+
- chore(deps): jest 30.5.1 -> 30.5.2 (#106)
|
|
124
|
+
- feat(release): the release PR starts its own CI (#105)
|
|
125
|
+
- fix(release): tag through the API when the push is refused (#104)
|
|
126
|
+
- chore(homebrew): sha256 for v0.8.1, from the published tarball (#103)
|
|
127
|
+
- fix(release): a blocked pull request says which setting blocks it (#102)
|
|
128
|
+
|
|
129
|
+
|
|
130
|
+
### Fixed
|
|
131
|
+
- **`scaffold` no longer dies on a zero-valued window literal, and proposes a
|
|
132
|
+
real fault for one.** The window producer weakened a literal by ×1000, and
|
|
133
|
+
×1000 of `0` is `0` — the replacement equalled its find, which the schema
|
|
134
|
+
rejects outright, so a single `expired: 0` aborted the whole file and printed
|
|
135
|
+
a raw Node stack. A 725-line source was unscaffoldable for this reason. The
|
|
136
|
+
same defect sat in the work-factor producer, where `rounds = 1` collapsed to
|
|
137
|
+
`1`, and in both Python producers, where `timeout=0` is PEP 8 spacing and so
|
|
138
|
+
the common form. The two cases are not the same, and are not fixed the same
|
|
139
|
+
way: a zero window is exactly the window worth widening, so it now widens to
|
|
140
|
+
a real value; a work factor already at 1 has nothing weaker to become, so it
|
|
141
|
+
proposes nothing rather than proposing itself. `usableProposal()` refuses a
|
|
142
|
+
no-op whatever a producer hands it, which makes the rule structural instead
|
|
143
|
+
of remembered.
|
|
144
|
+
- **`if (…) continue;` and `if (…) break;` are guards.** Only `return` and
|
|
145
|
+
`throw` were read as guard bodies, so pure selection-over-rows — the shape
|
|
146
|
+
the "pure logic, caller does the IO" advice produces — got no proposals at
|
|
147
|
+
all. Two fully-claimed files of seven such guards each returned nothing,
|
|
148
|
+
while the hand-written faults on those same lines all killed: the shape
|
|
149
|
+
`scaffold` skipped was the one demonstrably worth proposing. Both the
|
|
150
|
+
single-line and the block form, in JavaScript and Python.
|
|
151
|
+
- **An unexpected exception is named as our bug and keeps its trace.** Four
|
|
152
|
+
error classes became `error: …`; everything else reached the user as a Node
|
|
153
|
+
stack trace ending in `Node.js v24.18.0`, which reads as "your repository
|
|
154
|
+
broke the tool" and gives the operator nothing to do. It now says whose
|
|
155
|
+
defect it is and where to report it, and still prints the trace — a framed
|
|
156
|
+
error without one cannot be reported.
|
|
157
|
+
- **Every surface that describes the session-start hook is held to the code,
|
|
158
|
+
not just the README.** 0.6.0 banned "falls back to npx" in README.md and
|
|
159
|
+
added a mechanical check there. The same sentence lived in the line `init`
|
|
160
|
+
prints as it installs the hook and in the `--help` entry, and both went on
|
|
161
|
+
promising an `npx --no-install` fallback that `hookCommand()` had stopped
|
|
162
|
+
emitting — the sentence the user reads at the moment the hook is installed
|
|
163
|
+
described the network-reaching mechanism that was removed for reaching the
|
|
164
|
+
network. `TG-README-HOOK-MATCHES-THE-CODE` is now a claim about the
|
|
165
|
+
property, over all four surfaces. The fourth, `docs-canonical/SECURITY.md`,
|
|
166
|
+
was found by DocGuard while this claim was being widened from one to
|
|
167
|
+
three; it is correct today, and is checked because that is where the
|
|
168
|
+
sentence lives, not because a defect had been found there.
|
|
169
|
+
- **A test runner that has already exited is not reported as contention.**
|
|
170
|
+
`ps` is a snapshot, and a suite that finished between the snapshot and the
|
|
171
|
+
warning produced "1 test runner is already running (pid 65751)" for a process
|
|
172
|
+
the operator could not find. Liveness is re-checked immediately before
|
|
173
|
+
reporting. Detection stays best-effort: `EPERM` means running, not gone.
|
|
174
|
+
- **A release can tag itself even when `main` moves underneath it.** A GitHub
|
|
175
|
+
App may not *push* a ref whose `.github/workflows` differ from the current
|
|
176
|
+
default branch, so v0.8.1's tag was refused seven minutes after an unrelated
|
|
177
|
+
workflow PR merged — a release that was otherwise complete. `create-release`
|
|
178
|
+
still pushes first, and now falls back to creating the tag through the API,
|
|
179
|
+
which adds a ref to a commit already in the repository and so is not subject
|
|
180
|
+
to that restriction. Any other push failure is still fatal and says so.
|
|
11
181
|
- **A release that cannot open its pull request says why.** `GITHUB_TOKEN`
|
|
12
182
|
cannot call `createPullRequest` unless "Allow GitHub Actions to create and
|
|
13
183
|
approve pull requests" is enabled, and v0.8.1 hit exactly that: the branch,
|
|
@@ -19,6 +189,79 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
19
189
|
The failure is still a failure: the branch exists without a PR, and for the
|
|
20
190
|
formula `detect-version` keeps reporting the hash owed on every sweep.
|
|
21
191
|
|
|
192
|
+
### Changed
|
|
193
|
+
- **The calibration counts escaped bugs, not every commit that changed source
|
|
194
|
+
and a test together.** `p` is P(the suite misses it | a *bug* of this class
|
|
195
|
+
escapes), but the corpus was every commit with a source-and-test pair — which
|
|
196
|
+
in a conventional-commits repository is every feature, since a feature ships
|
|
197
|
+
with its tests as a matter of course. One run reported "12 of 12 measurable
|
|
198
|
+
bugs invisible — 100%" over five `feat:`, a `refactor:`, a `style:` and a
|
|
199
|
+
commit whose entire purpose was adding a test; only four were `fix:`. The
|
|
200
|
+
finding survived the correction — all four real fixes came back `blind` — and
|
|
201
|
+
is stronger stated over the four honestly than over the twelve loosely.
|
|
202
|
+
|
|
203
|
+
Selection is unchanged: the source-and-test pairing is still all that can be
|
|
204
|
+
judged without reading a subject line, and replaying a feature still measures
|
|
205
|
+
something. What narrows is the estimator. Where a majority of replayed
|
|
206
|
+
subjects carry a conventional type, `p` is computed over the `fix:`/`revert:`
|
|
207
|
+
records; `replay` prints both rates and the type breakdown so the narrowing
|
|
208
|
+
can be checked; and `calibration.json` records which rule produced it in
|
|
209
|
+
`source.detail`, so two calibrations from two populations can never be
|
|
210
|
+
compared by accident. A conventional repository with no `fix:` in range
|
|
211
|
+
calibrates nothing and says to widen the range, rather than silently counting
|
|
212
|
+
features as bugs.
|
|
213
|
+
- **`baseline` stops printing advice that rots, and stops repeating it.** It
|
|
214
|
+
named `.testguard/evidence.json` and `.testguard/brief.json` long after the
|
|
215
|
+
tool had grown to ten regenerated outputs, and printed them every run whether
|
|
216
|
+
or not the project already ignored them. The list now lives in one place,
|
|
217
|
+
owned by `init`, and only the outputs a project does not yet ignore are
|
|
218
|
+
printed — asked of git, so `.testguard/*` with negated exceptions, a global
|
|
219
|
+
ignore file and `.git/info/exclude` all answer correctly. `replay.json`,
|
|
220
|
+
`calibration.json`, `ci-self-evidence.json` and `.testguard/ci/` were missing
|
|
221
|
+
from the list `init` writes and have been added.
|
|
222
|
+
- **The weekly release starts its own CI, so no human step is left.** A pull
|
|
223
|
+
request opened by `GITHUB_TOKEN` raises no `pull_request` event, so the three
|
|
224
|
+
required Node legs never started and auto-merge sat on checks that would
|
|
225
|
+
never run — the workflow simply printed `gh workflow run ci.yml --ref
|
|
226
|
+
release/vX.Y.Z` and waited for someone to read it. It now dispatches that
|
|
227
|
+
run itself, as `release.yml`'s `homebrew-sha` job already did, guarded so a
|
|
228
|
+
re-run does not queue a second identical run. With
|
|
229
|
+
"Allow GitHub Actions to create and approve pull requests" enabled, a
|
|
230
|
+
release now needs no manual step at all.
|
|
231
|
+
- **The cost budget records a from-scratch release measurement.** 999 s for
|
|
232
|
+
115 faults on the v0.8.1 release commit, corroborated at 988 s on the
|
|
233
|
+
commit before it, against the unchanged 1100 s ceiling. The previous record
|
|
234
|
+
was 962 s at 96 faults.
|
|
235
|
+
|
|
236
|
+
### Performance
|
|
237
|
+
- **The self-probe cost budget is 1350 s, from 1100.** Six new claims and 14
|
|
238
|
+
new faults took the gate from 115 to 129 faults and 690 to 774 runs, and CI
|
|
239
|
+
measured 1123 s against the 1100 ceiling. The budget exists to make exactly
|
|
240
|
+
this visible rather than let it pass, so the raise is recorded with its
|
|
241
|
+
measurement and its reason in `testguard.cost-budget.json`. Two caveats are
|
|
242
|
+
written down there: the 1123 s run restored a cache under the fallback key
|
|
243
|
+
and so reused some verdicts, which makes it a floor rather than the
|
|
244
|
+
from-scratch figure this budget is defined against; and the number is
|
|
245
|
+
expected to come back down, because 44% of the gate is two shared defender
|
|
246
|
+
files re-run once per claim that names them. A third is recorded too: the
|
|
247
|
+
reported total is modelled rather than wall clock, because a reused record
|
|
248
|
+
contributes what it cost when it was last probed, so the same commit
|
|
249
|
+
measured 1123 s and then 1025 s depending on what the cache restored.
|
|
250
|
+
- **The new calibration claim costs 4.9 s instead of 51 s.** Its tests are
|
|
251
|
+
pure — no git, no fixture corpus, no scratch worktree — but they were added
|
|
252
|
+
to `replay.test.mjs`, whose scripted-corpus setup costs tens of seconds a
|
|
253
|
+
run, and a claim pays for every test in the file it names. Moved to
|
|
254
|
+
`calibration.test.mjs`, which exists for exactly this, and the claim
|
|
255
|
+
re-pointed. Same two faults, still killed 2/2.
|
|
256
|
+
|
|
257
|
+
### Documentation
|
|
258
|
+
- The README shows an ignore-file entry. It was mentioned twice without one,
|
|
259
|
+
so the field name (`pattern`, not `value`) and the fact that `expires` is a
|
|
260
|
+
full timestamp rather than a date had to be discovered from a validator
|
|
261
|
+
refusal.
|
|
262
|
+
- `--help` shows `gate --changed HEAD --include-dirty`. `--include-dirty` needs
|
|
263
|
+
a reference, and that combination is the one form a pre-commit hook wants.
|
|
264
|
+
|
|
22
265
|
## [0.8.1] - 2026-09-18
|
|
23
266
|
|
|
24
267
|
Automated weekly release — everything merged since `v0.8.0`.
|