testguard-cli 0.9.0__tar.gz → 0.9.2__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.agents/skills/testguard/SKILL.md +3 -0
- testguard_cli-0.9.2/.github/scripts/build-one-pager.mjs +95 -0
- testguard_cli-0.9.2/.github/scripts/check-cost-budget.mjs +79 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/scripts/sync-release-version.mjs +7 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/workflows/auto-merge.yml +1 -1
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/workflows/ci.yml +7 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/workflows/scheduled-release.yml +71 -6
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/CHANGELOG.md +125 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/PKG-INFO +20 -9
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/README.md +19 -8
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/action.yml +1 -1
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/docs/testguard-explained.html +80 -7
- testguard_cli-0.9.2/docs/testguard-explained.pdf +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/docs-canonical/ARCHITECTURE.md +6 -3
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/docs-canonical/DATA-MODEL.md +6 -1
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/docs-canonical/REQUIREMENTS.md +5 -3
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/docs-canonical/TEST-SPEC.md +6 -1
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/package-lock.json +2 -2
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/package.json +1 -1
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/packaging/gitlab/testguard.gitlab-ci.yml +2 -2
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/packaging/homebrew/testguard.rb +2 -2
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/pyproject.toml +1 -1
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/GATE-SEMANTICS.md +20 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/status.json +1 -0
- testguard_cli-0.9.2/spec/conformance/invalid/status.invalid-anchor-hidden.json +27 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/lib/validate.mjs +4 -1
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/schemas/brief.schema.json +1 -1
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/schemas/status.schema.json +20 -1
- testguard_cli-0.9.2/src/claims/anchors.mjs +218 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/cli.mjs +5 -2
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/commands/claims.mjs +10 -2
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/init/templates/SKILL.md +5 -1
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/cost.mjs +43 -6
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/status/status.mjs +19 -0
- testguard_cli-0.9.2/test/claims-anchors.test.mjs +90 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/classify.test.mjs +35 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/cost.test.mjs +65 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/gate.test.mjs +11 -0
- testguard_cli-0.9.2/test/label.test.mjs +35 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/mcp.test.mjs +11 -0
- testguard_cli-0.9.2/test/one-pager.test.mjs +88 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/probe.fixture.test.mjs +1 -1
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/release-sync.test.mjs +6 -2
- testguard_cli-0.9.2/test/replay-verdict.test.mjs +43 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/replay.test.mjs +0 -58
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/status.test.mjs +26 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/testguard.claims.json +177 -5
- testguard_cli-0.9.2/testguard.cost-budget.json +96 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/testguard.ignore.json +8 -1
- testguard_cli-0.9.0/.github/scripts/build-one-pager.mjs +0 -49
- testguard_cli-0.9.0/.github/scripts/check-cost-budget.mjs +0 -43
- testguard_cli-0.9.0/docs/testguard-explained.pdf +0 -0
- testguard_cli-0.9.0/testguard.cost-budget.json +0 -82
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.codex/hooks.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.docguard.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.docguardignore +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.gitattributes +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/ISSUE_TEMPLATE/bug_report.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/ISSUE_TEMPLATE/feature_request.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/PULL_REQUEST_TEMPLATE.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/dependabot.yml +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/scripts/check-python-fixture.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/scripts/install-smoke.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/workflows/release.yml +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.github/workflows/supply-chain.yml +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.gitignore +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.npmignore +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.npmrc +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.pre-commit-config.yaml +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/.pre-commit-hooks.yaml +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/AGENTS.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/CLAUDE.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/CODE_OF_CONDUCT.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/CONTRIBUTING.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/DRIFT-LOG.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/LICENSE +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/PRIVACY.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/SECURITY.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/SUPPORT.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/bench/README.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/cli/testguard.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/docs-canonical/ENVIRONMENT.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/docs-canonical/SECURITY.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer/.gitignore +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer/README.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer/expected.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer/package.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer/src/export.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer/src/redact.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer/test/export-mocked.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer/test/flaky.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer/test/redact.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer/testguard.claims.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer/vitest.config.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-jest/.gitignore +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-jest/README.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-jest/expected.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-jest/jest.config.js +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-jest/package.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-jest/src/export.js +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-jest/src/redact.js +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-jest/test/export-mocked.test.js +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-jest/test/flaky.test.js +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-jest/test/redact.test.js +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-jest/testguard.claims.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/.gitignore +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/README.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/e2e/flaky.spec.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/e2e/toggle.spec.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/expected.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/package-lock.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/package.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/playwright.config.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/src/toggle.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/test/toggle.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/test-results/.last-run.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/testguard.claims.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-playwright/vitest.config.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/.gitignore +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/README.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/demo/__init__.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/demo/export.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/demo/orphan.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/demo/pipeline.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/demo/redact.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/expected.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/testguard.claims.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/tests/test_export_mocked.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/tests/test_flaky.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/tests/test_patched.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/fixtures/known-answer-python/tests/test_redact.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/README.md +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/baseline.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/brief.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/calibration-websec.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/calibration.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/claims-methods.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/claims.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/evidence-disturbed-run.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/evidence-negative-control.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/evidence-provisional.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/evidence-python.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/evidence-scan-method.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/evidence.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/gate.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/ignore.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/examples/replay.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/baseline.bad-fingerprint-key.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/baseline.zero-count.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/brief.text-without-heading.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/brief.unknown-verdict-key.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/calibration.backoff-cell-not-wilson.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/calibration.breakdown-does-not-sum-to-n.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/calibration.ci-not-wilson.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/calibration.compound-key-arity-mismatch.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/calibration.fallback-without-basis.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/calibration.p-not-positives-over-n.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/calibration.p-outside-ci.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/calibration.positives-exceed-n.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/claims.absolute-path.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/claims.duplicate-fault-id.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/claims.fault-injection-without-a-substitution.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/claims.method-detail-on-a-specified-method.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/claims.missing-provenance.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/claims.noop-fault.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/claims.occurrence-exceeds-hits.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/claims.parent-traversal.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.annotated-signal-without-reason.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.attribute-signal-not-a-defender.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.attribute-signal-without-attributes.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.byrunner-file-not-resolved.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.contention-without-runners.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.discovered-defender-mocks-subject.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.fault-injection-without-its-evidence.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.flaky-defender-without-flakiness.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.ignored-dirty-on-snapshot.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.killed-by-non-assertion.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.killed-on-red-baseline.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.killed-without-n-runs.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.method-detail-on-an-injecting-run.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.negative-control-not-reached-but-survived.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.negative-control-reached-on-a-kill.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.probe-error-without-message.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.provisional-flag-on-confirmed-run.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.rank-extra-property.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.restore-skipped-in-place.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.single-run-without-provisional.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.target-not-imported-but-killed.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.undeclared-killers-without-reason.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.unknown-verdict.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.unverifiable-without-reason.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/evidence.wrong-fingerprint.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/gate.counts-disagree.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/gate.exit-zero-with-uncovered.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/gate.ignore-cover-without-pattern.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/ignore.missing-reason.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/ignore.short-reason.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/replay.blind-with-a-failing-run.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/replay.duplicate-patch-id.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/status.claim-without-file.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/status.clean-with-new-findings.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/status.unclaimed-hidden-behind-unproven.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/invalid/status.write-test-without-target.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/conformance/schemas.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/lib/fingerprint.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/lib/wilson.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/schemas/baseline.schema.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/schemas/calibration.schema.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/schemas/claims.schema.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/schemas/common.schema.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/schemas/evidence.schema.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/schemas/gate.schema.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/schemas/ignore.schema.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/spec/schemas/replay.schema.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/admit/admit.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/baseline/baseline.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/brief/brief.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/claims/annotations.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/claims/load.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/claims/removed.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/commands/admit.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/commands/baseline.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/commands/brief.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/commands/gate.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/commands/init.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/commands/mcp.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/commands/probe.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/commands/replay.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/commands/scaffold.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/commands/status.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/evidence/writer.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/gate/changed.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/git.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/init/init.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/mcp/server.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/mcp/tools.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/attribution.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/classify.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/contention.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/discover.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/independence.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/inject.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/mocks.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/probe.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/progress.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/pyimports.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/rank.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/runners/index.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/runners/jest.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/runners/playwright.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/runners/python/_testguard_pytest_plugin.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/runners/python/_testguard_report.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/runners/python/_testguard_unittest_main.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/runners/python.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/runners/shared.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/runners/vitest.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/probe/worktree.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/render.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/replay/label.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/replay/replay.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/scaffold/producers.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/scaffold/producers.python.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/scaffold/scaffold.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/util/glob.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/src/util/hash.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/admit.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/annotations.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/attribution.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/baseline.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/brief.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/calibration.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/claims-since.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/contention.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/discover.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/gitlab-template.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/glob.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/independence.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/init.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/inject.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/mocks.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/negative-control.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/probe-error.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/probe-preconditions.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/probe.jest.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/probe.playwright.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/probe.python.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/progress.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/pyimports.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/rank-aliases.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/runner-argv.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/runner-command.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/runner-playwright.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/runner-python.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/runner-vitest.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/samples/playwright-report.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/samples/python-report-collect-error.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/samples/python-report.json +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/scaffold.python.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/scaffold.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/snapshot.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/worktree.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/test/writer.test.mjs +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/testguard_cli/__init__.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/testguard_cli/wrapper.py +0 -0
- {testguard_cli-0.9.0 → testguard_cli-0.9.2}/vitest.config.mjs +0 -0
|
@@ -21,6 +21,7 @@ which files exist. `state` is one of:
|
|
|
21
21
|
| state | meaning | you do |
|
|
22
22
|
|---|---|---|
|
|
23
23
|
| `no-claims` | no `testguard.claims.json` | `testguard scaffold <file>` for a file with guards; replace every `TODO:` statement with what the code guarantees; keep or drop each proposal; move the claims into `testguard.claims.json` |
|
|
24
|
+
| `invalid-anchors` | an exact fault anchor moved or became ambiguous | run `testguard claims --check-anchors`; repair `next.target` in the claims file without changing what the fault means, then re-probe. Never guess or auto-fix an anchor. |
|
|
24
25
|
| `unprobed` | claims never probed | `testguard probe` |
|
|
25
26
|
| `evidence-stale` | code, tests or claims changed since the evidence | `testguard probe --include-dirty` (or `--claim <ID>` for one) |
|
|
26
27
|
| `provisional-only` | only `--confirm 1` evidence exists | `testguard probe --confirm 3` |
|
|
@@ -44,6 +45,8 @@ Exit codes: `0` clean · `1` unproven claims (or drift) · `2` precondition fail
|
|
|
44
45
|
## The fix loop
|
|
45
46
|
|
|
46
47
|
1. `testguard status --json` → take `next.target`.
|
|
48
|
+
After editing guarded code, run `testguard claims --check-anchors` before
|
|
49
|
+
any probe; repair a named fault definition without weakening its meaning.
|
|
47
50
|
2. Write the test. It must **fail when the fault is applied and pass on HEAD**. To check the first half by hand: apply `find` → `replace` in the target file, run the defender, restore.
|
|
48
51
|
3. `testguard probe --claim <ID> --include-dirty` — probes your uncommitted test without touching the tree. `--confirm 1` gives a fast **provisional** signal; a `?` on a verdict means unconfirmed.
|
|
49
52
|
4. When it is `killed` at `--confirm 3`, commit the test.
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Render docs/testguard-explained.html to PDF with headless Chrome.
|
|
4
|
+
*
|
|
5
|
+
* Chrome rather than a library because this repository has one pinned runtime
|
|
6
|
+
* dependency and keeps it that way: a PDF renderer would be the second, and it
|
|
7
|
+
* would be needed only to build a document nobody imports.
|
|
8
|
+
*
|
|
9
|
+
* Set CHROME to override the binary.
|
|
10
|
+
*
|
|
11
|
+
* (no flags) render; exit 0 with a notice when no Chrome is present,
|
|
12
|
+
* so a checkout without one can still run every other
|
|
13
|
+
* script.
|
|
14
|
+
* --require-chrome a missing Chrome is a failure. The release uses this:
|
|
15
|
+
* there, "skipped the render" means shipping a PDF that
|
|
16
|
+
* disagrees with the HTML beside it, which is the exact
|
|
17
|
+
* drift this script exists to prevent — and it would pass
|
|
18
|
+
* silently, which is worse than failing.
|
|
19
|
+
*
|
|
20
|
+
* After a successful render the page count is checked against the number of
|
|
21
|
+
* `.page` divs in the source. That is not a nicety: `.page` sets
|
|
22
|
+
* `min-height: 279.4mm` and `page-break-after: always`, which is a request,
|
|
23
|
+
* not a constraint. A section a few pixels too tall spills onto a second
|
|
24
|
+
* physical sheet with no warning from Chrome, renumbering nothing and
|
|
25
|
+
* contradicting the "Page N of M" footers the document prints itself. It
|
|
26
|
+
* happened while updating the brief for 0.8.1 and cost two extra pages.
|
|
27
|
+
*/
|
|
28
|
+
import { existsSync, readFileSync } from 'node:fs';
|
|
29
|
+
import { spawnSync } from 'node:child_process';
|
|
30
|
+
import { dirname, join, resolve } from 'node:path';
|
|
31
|
+
import { fileURLToPath } from 'node:url';
|
|
32
|
+
|
|
33
|
+
const root = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
|
34
|
+
const src = join(root, 'docs', 'testguard-explained.html');
|
|
35
|
+
const out = join(root, 'docs', 'testguard-explained.pdf');
|
|
36
|
+
const requireChrome = process.argv.slice(2).includes('--require-chrome');
|
|
37
|
+
|
|
38
|
+
// CHROME overrides rather than merely leads: a binary named explicitly and not
|
|
39
|
+
// found is a mistake worth hearing about, not a reason to quietly render with
|
|
40
|
+
// some other browser the machine happens to have.
|
|
41
|
+
const candidates = process.env.CHROME ? [process.env.CHROME] : [
|
|
42
|
+
'/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
|
|
43
|
+
'/Applications/Chromium.app/Contents/MacOS/Chromium',
|
|
44
|
+
'/usr/bin/google-chrome',
|
|
45
|
+
'/usr/bin/chromium',
|
|
46
|
+
'/usr/bin/chromium-browser',
|
|
47
|
+
];
|
|
48
|
+
|
|
49
|
+
const chrome = candidates.find((c) => existsSync(c));
|
|
50
|
+
if (!chrome) {
|
|
51
|
+
const how = process.env.CHROME
|
|
52
|
+
? `CHROME=${process.env.CHROME} does not exist`
|
|
53
|
+
: 'set CHROME=<path> to force';
|
|
54
|
+
if (requireChrome) {
|
|
55
|
+
console.error(`no Chrome found and --require-chrome was given; refusing to leave a stale PDF (${how})`);
|
|
56
|
+
process.exit(1);
|
|
57
|
+
}
|
|
58
|
+
console.log(`no Chrome found; skipping PDF build (${how})`);
|
|
59
|
+
process.exit(0);
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
const r = spawnSync(chrome, [
|
|
63
|
+
'--headless',
|
|
64
|
+
'--disable-gpu',
|
|
65
|
+
'--no-sandbox',
|
|
66
|
+
'--no-pdf-header-footer',
|
|
67
|
+
`--print-to-pdf=${out}`,
|
|
68
|
+
`file://${src}`,
|
|
69
|
+
], { encoding: 'utf8' });
|
|
70
|
+
|
|
71
|
+
if (r.status !== 0 || !existsSync(out)) {
|
|
72
|
+
console.error(r.stderr || 'chrome produced no PDF');
|
|
73
|
+
process.exit(1);
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
/** The sheets Chrome actually produced. `/Type /Pages` is the tree node, not a sheet. */
|
|
77
|
+
const sheets = (readFileSync(out).toString('latin1').match(/\/Type\s*\/Page(?![s/\w])/g) ?? []).length;
|
|
78
|
+
/** The sheets the document intends: one `.page` div each. */
|
|
79
|
+
const intended = (readFileSync(src, 'utf8').match(/<div class="page"[\s>]/g) ?? []).length;
|
|
80
|
+
|
|
81
|
+
if (intended === 0) {
|
|
82
|
+
console.error('docs/testguard-explained.html: no `.page` divs found — the page-count guard cannot run');
|
|
83
|
+
process.exit(1);
|
|
84
|
+
}
|
|
85
|
+
if (sheets !== intended) {
|
|
86
|
+
console.error(
|
|
87
|
+
`docs/testguard-explained.pdf: ${sheets} sheets rendered from ${intended} \`.page\` divs.\n` +
|
|
88
|
+
'A .page taller than its sheet spills silently. Letter at margin 0 is 816x1056 CSS px: load the\n' +
|
|
89
|
+
'HTML at that viewport and look for a .page whose getBoundingClientRect().height exceeds 1056.\n' +
|
|
90
|
+
'Measuring at any other width is meaningless, because the text reflows.',
|
|
91
|
+
);
|
|
92
|
+
process.exit(1);
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
console.log(`docs/testguard-explained.pdf written with ${chrome.split('/').pop()} — ${sheets} pages, one per .page div`);
|
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
/**
|
|
3
|
+
* Fail CI when the self-probe gate outgrows its budget.
|
|
4
|
+
*
|
|
5
|
+
* The decision lives in `src/probe/cost.mjs` as a pure function, unit-tested
|
|
6
|
+
* and covered by a self-claim; this script is only wiring — read the evidence,
|
|
7
|
+
* read the budget, print, choose an exit code.
|
|
8
|
+
*
|
|
9
|
+
* usage: check-cost-budget.mjs <evidence.json> [previous-evidence.json]
|
|
10
|
+
*/
|
|
11
|
+
import { appendFileSync, readFileSync } from 'node:fs';
|
|
12
|
+
import { dirname, join, resolve } from 'node:path';
|
|
13
|
+
import { fileURLToPath } from 'node:url';
|
|
14
|
+
import { costReport, checkCostBudget, renderCostBudget } from '../../src/probe/cost.mjs';
|
|
15
|
+
|
|
16
|
+
const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
|
|
17
|
+
const [evidencePath, previousPath] = process.argv.slice(2);
|
|
18
|
+
if (!evidencePath) {
|
|
19
|
+
console.error('usage: check-cost-budget.mjs <evidence.json> [previous-evidence.json]');
|
|
20
|
+
process.exit(2);
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
const read = (p) => JSON.parse(readFileSync(resolve(p), 'utf8'));
|
|
24
|
+
const budget = read(join(ROOT, 'testguard.cost-budget.json'));
|
|
25
|
+
|
|
26
|
+
// A previous run is informational only. Its absence must never fail the check:
|
|
27
|
+
// the first run on a branch, or a cold cache, has nothing to compare against.
|
|
28
|
+
let previousMs;
|
|
29
|
+
if (previousPath) {
|
|
30
|
+
try {
|
|
31
|
+
previousMs = costReport(read(previousPath).records).totalMs;
|
|
32
|
+
} catch {
|
|
33
|
+
previousMs = undefined;
|
|
34
|
+
}
|
|
35
|
+
}
|
|
36
|
+
|
|
37
|
+
const report = costReport(read(evidencePath).records);
|
|
38
|
+
const decision = checkCostBudget(report, {
|
|
39
|
+
budgetSeconds: budget.seconds,
|
|
40
|
+
perClaimSeconds: budget.perClaimSeconds,
|
|
41
|
+
previousMs,
|
|
42
|
+
});
|
|
43
|
+
const text = renderCostBudget(decision);
|
|
44
|
+
console.log(text);
|
|
45
|
+
|
|
46
|
+
// The gate's own history is the argument for writing this where it is seen:
|
|
47
|
+
// the run went 9.6 -> 23.6 minutes across one merged pull request and nothing
|
|
48
|
+
// said a word, because --cost only ever printed into a log nobody opens while
|
|
49
|
+
// it is passing. A new claim that quietly costs 51 s is the same failure one
|
|
50
|
+
// size down. A job summary is read without opening anything, so the number is
|
|
51
|
+
// in front of a reviewer while the change is still a change.
|
|
52
|
+
if (process.env.GITHUB_STEP_SUMMARY) {
|
|
53
|
+
const rows = (report.claims ?? []).slice(0, 10)
|
|
54
|
+
.map((c) => `| \`${c.claimId}\` | ${(c.ms / 1000).toFixed(1)}s | ${c.runs} |`)
|
|
55
|
+
.join('\n');
|
|
56
|
+
appendFileSync(process.env.GITHUB_STEP_SUMMARY, [
|
|
57
|
+
`## Self-probe cost \u2014 ${decision.ok ? 'within budget' : 'OVER BUDGET'}`,
|
|
58
|
+
'',
|
|
59
|
+
'```',
|
|
60
|
+
text,
|
|
61
|
+
'```',
|
|
62
|
+
'',
|
|
63
|
+
'<details><summary>Most expensive claims</summary>',
|
|
64
|
+
'',
|
|
65
|
+
'| claim | cost | runs |',
|
|
66
|
+
'| --- | ---: | ---: |',
|
|
67
|
+
rows,
|
|
68
|
+
'',
|
|
69
|
+
'A claim pays for every test in the file it names, so the fix for an expensive',
|
|
70
|
+
'claim is usually a cheaper defender rather than a bigger budget.',
|
|
71
|
+
'</details>',
|
|
72
|
+
'',
|
|
73
|
+
].join('\n'));
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
if (!decision.ok) {
|
|
77
|
+
console.log(`::error::${text.split('\n')[0]}`);
|
|
78
|
+
process.exit(1);
|
|
79
|
+
}
|
|
@@ -95,6 +95,13 @@ const surfaces = [
|
|
|
95
95
|
['README.md', /testguard\/v\d+\.\d+\.\d+\/packaging/g, `testguard/v${version}/packaging`],
|
|
96
96
|
['packaging/gitlab/testguard.gitlab-ci.yml', /testguard\/v\d+\.\d+\.\d+\/packaging/g, `testguard/v${version}/packaging`],
|
|
97
97
|
['packaging/gitlab/testguard.gitlab-ci.yml', /(\n version:\n description: [^\n]*\n default: ")\d+\.\d+\.\d+(")/, `$1${version}$2`],
|
|
98
|
+
// The technical brief's masthead. Nothing rendered it for three releases, so
|
|
99
|
+
// it still read v0.6 while the tool shipped 0.8.1. The HTML is the surface;
|
|
100
|
+
// the PDF beside it is a build artifact of this file, which is why the
|
|
101
|
+
// release runs build-one-pager.mjs --require-chrome straight after the sync.
|
|
102
|
+
// Sync the two separately and they disagree — the failure that is worse than
|
|
103
|
+
// being stale, because the document then contradicts itself.
|
|
104
|
+
['docs/testguard-explained.html', /(Technical Brief · v)\d+\.\d+\.\d+( · MIT)/, `$1${version}$2`],
|
|
98
105
|
];
|
|
99
106
|
|
|
100
107
|
if (listSurfaces) {
|
|
@@ -53,7 +53,7 @@ jobs:
|
|
|
53
53
|
// three files the bump writes — test/release-sync.test.mjs fails if
|
|
54
54
|
// a surface is missing, which is how the GitLab template's absence
|
|
55
55
|
// (it would have held every release PR for review) was found.
|
|
56
|
-
const RELEASE_SURFACES = new Set(['package.json', 'package-lock.json', 'pyproject.toml', 'CHANGELOG.md', 'action.yml', 'packaging/homebrew/testguard.rb', 'README.md', 'packaging/gitlab/testguard.gitlab-ci.yml']);
|
|
56
|
+
const RELEASE_SURFACES = new Set(['package.json', 'package-lock.json', 'pyproject.toml', 'CHANGELOG.md', 'action.yml', 'packaging/homebrew/testguard.rb', 'README.md', 'packaging/gitlab/testguard.gitlab-ci.yml', 'docs/testguard-explained.html', 'docs/testguard-explained.pdf']);
|
|
57
57
|
|
|
58
58
|
if (author === 'dependabot[bot]') {
|
|
59
59
|
const m = /from (\d+)\.(\d+)\.\S+ to (\d+)\.(\d+)\./.exec(pr.title || '');
|
|
@@ -185,6 +185,13 @@ jobs:
|
|
|
185
185
|
if: github.event_name == 'pull_request'
|
|
186
186
|
run: node cli/testguard.mjs gate . --changed "origin/${{ github.base_ref }}"
|
|
187
187
|
|
|
188
|
+
# Exact anchors and parseable replacements are knowable without a test
|
|
189
|
+
# runner. Fail here instead of spending the long self-probe merely to
|
|
190
|
+
# discover that a fault could never be applied.
|
|
191
|
+
- name: claims — fault anchors and replacements are valid
|
|
192
|
+
if: matrix.node-version == 22
|
|
193
|
+
run: node cli/testguard.mjs claims . --check-anchors
|
|
194
|
+
|
|
188
195
|
# ── Self-verification: TestGuard is governed by the standard it defines.
|
|
189
196
|
# testguard.claims.json states invariants of the tool itself; every
|
|
190
197
|
# fault must be killed by the tool's own tests. Exit 0 = all defended. ──
|
|
@@ -8,8 +8,15 @@ name: Scheduled Release
|
|
|
8
8
|
# GITHUB_TOKEN-created PRs do not trigger `pull_request` workflows, so nothing
|
|
9
9
|
# would ever start the required checks and auto-merge would wait forever. This
|
|
10
10
|
# workflow therefore dispatches ci.yml on the release branch itself:
|
|
11
|
-
# `workflow_dispatch` is the one event GITHUB_TOKEN may trigger.
|
|
12
|
-
#
|
|
11
|
+
# `workflow_dispatch` is the one event GITHUB_TOKEN may trigger.
|
|
12
|
+
#
|
|
13
|
+
# That is not the whole story, and v0.9.0 proved it. The `pull_request` runs
|
|
14
|
+
# the PR creates anyway are held at `action_required`, because the repository
|
|
15
|
+
# requires approval from first-time contributors and `github-actions[bot]` is
|
|
16
|
+
# permanently one. Held runs keep check suites QUEUED on the head sha, so the
|
|
17
|
+
# rollup never becomes terminal and auto-merge waits — with every required
|
|
18
|
+
# check already green. This workflow now approves those runs itself, which
|
|
19
|
+
# GITHUB_TOKEN is permitted to do. Only then does no human step remain.
|
|
13
20
|
|
|
14
21
|
on:
|
|
15
22
|
schedule:
|
|
@@ -85,6 +92,11 @@ jobs:
|
|
|
85
92
|
} > /tmp/entry.md
|
|
86
93
|
awk '/^## \[Unreleased\]/ { print; print ""; while ((getline line < "/tmp/entry.md") > 0) print line; next } { print }' CHANGELOG.md > /tmp/CHANGELOG.new && mv /tmp/CHANGELOG.new CHANGELOG.md
|
|
87
94
|
node .github/scripts/sync-release-version.mjs
|
|
95
|
+
# The sync just wrote the brief's masthead; the committed PDF still
|
|
96
|
+
# shows the old one. --require-chrome because a skipped render here
|
|
97
|
+
# would ship an HTML and a PDF that disagree, silently and at exit 0.
|
|
98
|
+
# ubuntu-latest carries /usr/bin/google-chrome.
|
|
99
|
+
node .github/scripts/build-one-pager.mjs --require-chrome
|
|
88
100
|
echo "NEW_VERSION=$NEW" >> "$GITHUB_ENV"
|
|
89
101
|
|
|
90
102
|
- name: Verify the tree is still green
|
|
@@ -117,9 +129,11 @@ jobs:
|
|
|
117
129
|
# DERIVED from the sync script, never retyped here: when the GitLab
|
|
118
130
|
# template became a surface it was left out of the hand-written list,
|
|
119
131
|
# so the first release to reach this guard would have been rejected
|
|
120
|
-
# for writing a file the release is supposed to write. The
|
|
121
|
-
# extras are the ones the bump step writes directly
|
|
122
|
-
|
|
132
|
+
# for writing a file the release is supposed to write. The four
|
|
133
|
+
# extras are the ones the bump step writes directly: three from the
|
|
134
|
+
# bump itself, plus the brief's PDF, which is rendered from a surface
|
|
135
|
+
# rather than being one.
|
|
136
|
+
ALLOWED=$(node .github/scripts/sync-release-version.mjs --list-surfaces; printf 'package.json\npackage-lock.json\nCHANGELOG.md\ndocs/testguard-explained.pdf\n')
|
|
123
137
|
echo "Release surfaces allowed in this diff:"; printf '%s\n' "$ALLOWED" | sed 's/^/ /'
|
|
124
138
|
CHANGED=$(git diff --name-only HEAD --)
|
|
125
139
|
for f in $CHANGED; do
|
|
@@ -134,7 +148,7 @@ jobs:
|
|
|
134
148
|
git add -A
|
|
135
149
|
git commit -m "release: v${NEW_VERSION} — automated weekly batch"
|
|
136
150
|
git push origin "$BRANCH"
|
|
137
|
-
printf 'Automated weekly release. Version bump + CHANGELOG entry + synced version surfaces. Merging tags and publishes to npm and PyPI via release.yml.\n\
|
|
151
|
+
printf 'Automated weekly release. Version bump + CHANGELOG entry + synced version surfaces. Merging tags and publishes to npm and PyPI via release.yml.\n\nTwo things a GITHUB_TOKEN PR needs done for it, both handled by this workflow: it raises no `pull_request` event, so ci.yml was dispatched on `%s` itself; and the `pull_request` runs it does create are held at `action_required` (github-actions[bot] is permanently a first-time contributor), so those were approved too. Auto-merge is armed and completes once the three Node legs pass.\n\nIf it sits anyway: `gh workflow run ci.yml --ref %s` for missing checks, or check for runs still awaiting approval on the head sha.\n' "$BRANCH" "$BRANCH" > /tmp/pr-body.md
|
|
138
152
|
# A bare `gh pr create` failure here reads as a GraphQL error with no
|
|
139
153
|
# hint of the cause, which is how v0.8.1 lost its release PR after
|
|
140
154
|
# everything else in the release had succeeded.
|
|
@@ -155,6 +169,57 @@ jobs:
|
|
|
155
169
|
else
|
|
156
170
|
echo "✅ $HEAD_SHA already has $RUNS check run(s)"
|
|
157
171
|
fi
|
|
172
|
+
|
|
173
|
+
# ── The third GITHUB_TOKEN wall, and the last one. ──
|
|
174
|
+
# The `pull_request` runs this PR creates are held at
|
|
175
|
+
# `action_required`: the repository requires approval from
|
|
176
|
+
# first-time contributors, and `github-actions[bot]` is
|
|
177
|
+
# PERMANENTLY one — it never authors a merged commit, so it can
|
|
178
|
+
# never graduate. Those held runs leave check suites QUEUED on the
|
|
179
|
+
# head sha forever, the status rollup never reaches a terminal
|
|
180
|
+
# state, and auto-merge waits on it even though the required
|
|
181
|
+
# contexts (test 20/22/24) are already green from the dispatched
|
|
182
|
+
# run above. v0.9.0 sat here until a human clicked Approve.
|
|
183
|
+
#
|
|
184
|
+
# GITHUB_TOKEN may clear it: `actions: write` carries the approve
|
|
185
|
+
# permission, measured 201 on a bot PR's own held run. It may not
|
|
186
|
+
# approve a REVIEW, which is a different gate and still ours alone.
|
|
187
|
+
#
|
|
188
|
+
# Runs appear asynchronously, so poll. Approving an already-running
|
|
189
|
+
# run answers 403 "not waiting for approval" — success, not failure.
|
|
190
|
+
echo "Clearing held workflow runs on $HEAD_SHA"
|
|
191
|
+
approved=0
|
|
192
|
+
for attempt in 1 2 3 4 5 6; do
|
|
193
|
+
HELD=$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs?head_sha=${HEAD_SHA}" \
|
|
194
|
+
--jq '[.workflow_runs[] | select(.status == "action_required" or .conclusion == "action_required") | .id] | .[]' 2>/dev/null || true)
|
|
195
|
+
for id in $HELD; do
|
|
196
|
+
if gh api -X POST "repos/${GITHUB_REPOSITORY}/actions/runs/${id}/approve" >/dev/null 2>&1; then
|
|
197
|
+
echo " approved run $id"
|
|
198
|
+
approved=$((approved + 1))
|
|
199
|
+
else
|
|
200
|
+
echo " run $id needed no approval"
|
|
201
|
+
fi
|
|
202
|
+
done
|
|
203
|
+
# Stop as soon as the picture is settled, rather than always
|
|
204
|
+
# paying the full poll: nothing held after something was
|
|
205
|
+
# approved means we are done, and nothing held by the third
|
|
206
|
+
# look means the gate is not applying to this run at all.
|
|
207
|
+
if [ -z "$HELD" ] && { [ "$approved" -gt 0 ] || [ "$attempt" -ge 3 ]; }; then
|
|
208
|
+
break
|
|
209
|
+
fi
|
|
210
|
+
sleep 10
|
|
211
|
+
done
|
|
212
|
+
|
|
213
|
+
# Never leave this silent: a run still held here means auto-merge
|
|
214
|
+
# will wait forever, which is the failure this whole block exists
|
|
215
|
+
# to remove. Say so, and name what to click.
|
|
216
|
+
STILL=$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs?head_sha=${HEAD_SHA}" \
|
|
217
|
+
--jq '[.workflow_runs[] | select(.status == "action_required" or .conclusion == "action_required") | .html_url] | .[]' 2>/dev/null || true)
|
|
218
|
+
if [ -n "$STILL" ]; then
|
|
219
|
+
echo "::error::Workflow runs are still held for approval, so auto-merge on $PR will wait forever. Approve them, or re-run this workflow: $STILL"
|
|
220
|
+
exit 1
|
|
221
|
+
fi
|
|
222
|
+
echo "✅ no run is held for approval; auto-merge can complete"
|
|
158
223
|
else
|
|
159
224
|
cat /tmp/pr-err
|
|
160
225
|
if grep -q 'not permitted to create or approve pull requests' /tmp/pr-err; then
|
|
@@ -7,6 +7,131 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.9.2] - 2026-09-19
|
|
11
|
+
|
|
12
|
+
Automated weekly release — everything merged since `v0.9.1`.
|
|
13
|
+
|
|
14
|
+
### Changed
|
|
15
|
+
|
|
16
|
+
- feat(claims): preflight fault anchors (#126)
|
|
17
|
+
- chore(homebrew): sha256 for v0.9.1 (#125)
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
### Added
|
|
21
|
+
|
|
22
|
+
- `claims --check-anchors` locates every exact fault anchor and syntax-checks
|
|
23
|
+
supported in-memory JavaScript and Python replacements without starting a
|
|
24
|
+
runner or touching the tree. CI runs the sub-second preflight before the
|
|
25
|
+
self-probe, while `status` reports rotted anchors as `invalid-anchors` with
|
|
26
|
+
the precise fault to repair. Repairs remain a reviewed judgement call:
|
|
27
|
+
there is deliberately no fuzzy match or `--fix`.
|
|
28
|
+
|
|
29
|
+
## [0.9.1] - 2026-09-19
|
|
30
|
+
|
|
31
|
+
Automated weekly release — everything merged since `v0.9.0`.
|
|
32
|
+
|
|
33
|
+
### Changed
|
|
34
|
+
|
|
35
|
+
- docs: the brief catches up with the tool, and stops falling behind (#112)
|
|
36
|
+
- chore(homebrew): sha256 for v0.9.0 (#110)
|
|
37
|
+
- feat(cost): two ceilings, the report where it is read, and three claims off expensive defenders (#113)
|
|
38
|
+
- fix(release): approve the runs GITHUB_TOKEN's own PR leaves held (#111)
|
|
39
|
+
|
|
40
|
+
|
|
41
|
+
### Added
|
|
42
|
+
- **A second cost ceiling, on the claim.** The total alone cannot see the
|
|
43
|
+
failure it was meant to catch: a new calibration claim cost 51 s for two
|
|
44
|
+
faults purely by being written into `replay.test.mjs` rather than
|
|
45
|
+
`calibration.test.mjs`, and re-pointing it took it to 4.9 s while the total
|
|
46
|
+
never moved. A per-fault average is worse — the regression that prompted
|
|
47
|
+
this budget raised the fault count 59 to 91 while per-fault cost barely
|
|
48
|
+
moved, so an average would have reported everything fine.
|
|
49
|
+
|
|
50
|
+
`perClaimSeconds` is 100 s, just above today's worst claim
|
|
51
|
+
(`TG-ADMIT-NEEDS-ALL-KILLED`, **87 s in CI** for one fault), so the rule
|
|
52
|
+
reads: no new claim may be worse than the worst we already have. Calibrated
|
|
53
|
+
against CI and not the local figure — 75 s, set from that claim's local
|
|
54
|
+
66.5 s, tripped on the first CI run. This file already warns about exactly
|
|
55
|
+
that for the total (950 s was 18% over local but 4.6% over CI, and the first
|
|
56
|
+
PR adding a claim tripped it); the same error applies one level down. A breach names the claim
|
|
57
|
+
and its remedy — a cheaper defender, not a bigger budget. The field is
|
|
58
|
+
optional and additive: a budget without one behaves exactly as before.
|
|
59
|
+
- **The cost report goes to the CI job summary, not only the log.** The gate
|
|
60
|
+
went from 9.6 to 23.6 minutes across one merged pull request and nothing
|
|
61
|
+
said a word, because `--cost` only ever printed into a log nobody opens
|
|
62
|
+
while it is passing. The summary is read without opening anything, so the
|
|
63
|
+
number is in front of a reviewer while the change is still a change.
|
|
64
|
+
|
|
65
|
+
### Performance
|
|
66
|
+
- **Three claims moved off expensive shared defenders: 65 s, ~30 s and ~30 s
|
|
67
|
+
down to under 2 s each.** All three faulted a *pure* function while naming
|
|
68
|
+
an integration test file, and a claim pays for every test in the file it
|
|
69
|
+
names. `TG-SURVIVOR-PROVES-THE-SUBJECT-RUNS` faults `classify()` but was
|
|
70
|
+
defended by `negative-control.test.mjs`, which spawns real runners against
|
|
71
|
+
fixture repositories; `TG-LABEL-NEVER-GUESSES` faults `labelDiff()` and
|
|
72
|
+
`TG-REPLAY-FLAKY-IS-NEVER-CAUGHT` faults `classifyReplay()`, both defended
|
|
73
|
+
by `replay.test.mjs`, which builds a scripted git corpus.
|
|
74
|
+
|
|
75
|
+
Two were a straight move of an already-isolated `describe` block into
|
|
76
|
+
`test/label.test.mjs` and `test/replay-verdict.test.mjs`; only the `classify`
|
|
77
|
+
case needed a test written. Each was proved with `admit` — passes on HEAD and
|
|
78
|
+
fails on every fault, 3/3 — *before* the expensive defender was dropped, and
|
|
79
|
+
never the other way round. The integration files keep their remaining claims,
|
|
80
|
+
where the fixture is the point.
|
|
81
|
+
|
|
82
|
+
### Changed
|
|
83
|
+
- **The cost budget is 1230 s, from 1350.** 1350 was set defensively against a
|
|
84
|
+
reuse-affected 1123 s, because this budget is defined against a from-scratch
|
|
85
|
+
probe and none had run yet. `release.yml` then measured **1081 s** at 129
|
|
86
|
+
faults on the v0.9.0 release commit with no reuse — so the six field-report
|
|
87
|
+
claims cost +82 s, not the +120 s a per-fault average predicted. 1230 is
|
|
88
|
+
~14% over 1081, the same headroom rule that set 1100 against 962. Leaving it
|
|
89
|
+
at 1350 would have let a 25% regression pass unnoticed.
|
|
90
|
+
|
|
91
|
+
### Fixed
|
|
92
|
+
- **The release no longer waits for a human to click Approve.** v0.9.0 sat with
|
|
93
|
+
every required check green and merged nothing. The `pull_request` runs a
|
|
94
|
+
release PR creates are held at `action_required`, because the repository
|
|
95
|
+
requires approval from first-time contributors and `github-actions[bot]` is
|
|
96
|
+
permanently one — it never authors a merged commit, so it can never
|
|
97
|
+
graduate. Held runs leave check suites QUEUED on the head sha, the status
|
|
98
|
+
rollup never reaches a terminal state, and auto-merge waits on it even though
|
|
99
|
+
`test (20/22/24)` are already green from the dispatched run.
|
|
100
|
+
|
|
101
|
+
`scheduled-release.yml` now approves those runs itself. `GITHUB_TOKEN` is
|
|
102
|
+
permitted to do this — `actions: write` carries the approve permission,
|
|
103
|
+
measured 201 against a bot PR's own held run — and the workflow already held
|
|
104
|
+
that permission, so nothing was granted and no secret was added. It may still
|
|
105
|
+
not approve a *review*, which is a different gate and remains ours alone.
|
|
106
|
+
A run left held after the poll is an `::error::` naming the run, never a
|
|
107
|
+
silent wait.
|
|
108
|
+
|
|
109
|
+
This is the third `GITHUB_TOKEN` wall in the same chain, after "raises no
|
|
110
|
+
`pull_request` event" (#105) and "its push raises no `push` event" (the
|
|
111
|
+
hourly sweep in release.yml). All three have the same root: an action taken
|
|
112
|
+
by `GITHUB_TOKEN` does not trigger the event that would normally follow it.
|
|
113
|
+
|
|
114
|
+
### Documentation
|
|
115
|
+
- **The technical brief describes the tool that exists.** It was last rendered
|
|
116
|
+
when it covered one verb, `probe`, and said nothing about `replay` and its
|
|
117
|
+
calibration — the empirical answer to whether an injected fault resembles a
|
|
118
|
+
real bug, which is the question a document arguing "evidence, not output"
|
|
119
|
+
can least afford to omit. A new page covers the eleven verbs and the
|
|
120
|
+
calibration, carrying the README's own framing: the greenfield transfer is
|
|
121
|
+
unproven, and `replay` is the instrument rather than the answer. The verdict
|
|
122
|
+
table, the cost formula and the prior-art section were checked against the
|
|
123
|
+
code and left alone.
|
|
124
|
+
- **The brief's PDF is rendered by the release instead of by memory.** Its
|
|
125
|
+
masthead read `v0.6` through three releases because nothing ever ran
|
|
126
|
+
`build-one-pager.mjs`. The HTML masthead is now a version surface and
|
|
127
|
+
`scheduled-release.yml` re-renders the PDF in the same step as the sync —
|
|
128
|
+
the same step, because an HTML and a PDF that disagree are worse than a
|
|
129
|
+
document uniformly out of date. `--require-chrome` makes a missing browser
|
|
130
|
+
fail the release rather than silently commit a stale PDF, and every render
|
|
131
|
+
must produce exactly one sheet per `.page` div: a section a few pixels too
|
|
132
|
+
tall spills onto a second sheet with no warning from Chrome, contradicting
|
|
133
|
+
the "Page N of M" footers the document prints about itself.
|
|
134
|
+
|
|
10
135
|
## [0.9.0] - 2026-09-18
|
|
11
136
|
|
|
12
137
|
Automated weekly release — everything merged since `v0.8.1`.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: testguard-cli
|
|
3
|
-
Version: 0.9.
|
|
3
|
+
Version: 0.9.2
|
|
4
4
|
Summary: Breaks your code on purpose and reports every promise your tests did not notice breaking. Mutation testing bound to the claims a project makes — vitest, jest, Playwright, pytest and unittest. Ships the Node.js CLI (requires Node.js 20+).
|
|
5
5
|
Project-URL: Homepage, https://github.com/raccioly/testguard
|
|
6
6
|
Project-URL: Documentation, https://github.com/raccioly/testguard#readme
|
|
@@ -104,8 +104,9 @@ risk. TestGuard binds every fault to a **stated claim**, so the output is not
|
|
|
104
104
|
a score but a finding: *"Your project says a missing scope fails closed.
|
|
105
105
|
Nothing checks that."* That is the difference between a metric and an audit.
|
|
106
106
|
|
|
107
|
-
📄 **[Read the
|
|
108
|
-
— the idea on page one,
|
|
107
|
+
📄 **[Read the six-page technical brief (PDF)](docs/testguard-explained.pdf)**
|
|
108
|
+
— the idea on page one, then the field evidence, the anatomy of a run, the
|
|
109
|
+
verdicts, the loop and its calibration, and the prior art.
|
|
109
110
|
|
|
110
111
|
## Why
|
|
111
112
|
|
|
@@ -150,9 +151,9 @@ raises coverage. TestGuard admits it because it fails when the claim is false.
|
|
|
150
151
|
| npm | `npm i -D testguard-cli` then `npx testguard probe` |
|
|
151
152
|
| pip | `pip install testguard-cli` then `testguard probe` (needs Node ≥ 20) |
|
|
152
153
|
| Homebrew | `brew tap raccioly/tap && brew install testguard` |
|
|
153
|
-
| GitHub Action | `uses: raccioly/testguard@v0.9.
|
|
154
|
+
| GitHub Action | `uses: raccioly/testguard@v0.9.2` — see [`action.yml`](./action.yml) |
|
|
154
155
|
| pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-probe` |
|
|
155
|
-
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.9.
|
|
156
|
+
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.9.2/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
|
|
156
157
|
|
|
157
158
|
Projects that set `min-release-age` in `.npmrc` cannot see a version published
|
|
158
159
|
less than that many days ago (`ENOVERSIONS`); install that one with
|
|
@@ -164,6 +165,7 @@ less than that many days ago (`ENOVERSIONS`); install that one with
|
|
|
164
165
|
npx testguard-cli init # install the agent layer at the git root
|
|
165
166
|
npx testguard-cli status --json # where the project is, and the ONE next action
|
|
166
167
|
npx testguard-cli claims # what does this project claim, and is each claim probeable?
|
|
168
|
+
npx testguard-cli claims --check-anchors # fail fast if an exact anchor moved or a replacement no longer parses
|
|
167
169
|
npx testguard-cli scaffold src/auth.ts # propose faults for a file, as a draft to keep or drop
|
|
168
170
|
npx testguard-cli probe # try to falsify each claim; report what the tests missed
|
|
169
171
|
npx testguard-cli admit test/auth.test.ts --claim AUTH-ADMIN # does this test satisfy the two-gate rule?
|
|
@@ -174,8 +176,9 @@ npx testguard-cli replay --since v1.0..HEAD # would this suite have caught the b
|
|
|
174
176
|
npx testguard-cli mcp # serve the read-only loop over MCP, on stdio
|
|
175
177
|
```
|
|
176
178
|
|
|
177
|
-
Exit codes are the contract: `0` nothing new to prove, `1` unproven claims
|
|
178
|
-
unclaimed changes, `2` a precondition failed and
|
|
179
|
+
Exit codes are the contract: `0` nothing new to prove, `1` unproven claims,
|
|
180
|
+
unclaimed changes or invalid fault anchors, `2` a precondition failed and
|
|
181
|
+
nothing was probed, `3` usage.
|
|
179
182
|
Every command accepts `--json`.
|
|
180
183
|
|
|
181
184
|
## How it works
|
|
@@ -200,6 +203,10 @@ npx testguard-cli admit test/x.test.ts --claim X # is this test green on HEAD
|
|
|
200
203
|
claim and every fault records who produced it. `testguard claims`
|
|
201
204
|
validates the file and reports drift against `@claim <ID>` annotations in
|
|
202
205
|
source. Test files are deliberately not scanned — a claim asserted by a test is the authorship trap the tool exists for — and annotation ids must contain a hyphen so prose is never mistaken for one.
|
|
206
|
+
`claims --check-anchors` additionally locates every exact anchor and parses
|
|
207
|
+
each in-memory JavaScript or Python replacement. It runs no tests, creates
|
|
208
|
+
no worktree and changes no file, so a moved anchor is reported before an
|
|
209
|
+
expensive probe. It reports and stops; it never guesses a new anchor.
|
|
203
210
|
2. **Probe** confirms the defenders are green N times unmodified, applies
|
|
204
211
|
each fault in a scratch git worktree (your tree is never touched), runs
|
|
205
212
|
the defenders N times, re-runs survivors against the whole suite with
|
|
@@ -468,6 +475,10 @@ is reported as `EXPIRED`.
|
|
|
468
475
|
With a reference known, `status --changed <ref>` reports `unclaimed-changes`
|
|
469
476
|
**before** any evidence state and makes the claim the next action; the brief
|
|
470
477
|
lists the unclaimed files first. The claim is written before more code.
|
|
478
|
+
After claim coverage, `status` reports `invalid-anchors` before any evidence
|
|
479
|
+
state when an exact fault anchor is missing or ambiguous, and points at the
|
|
480
|
+
fault definition to repair. Repair preserves the fault's meaning and requires
|
|
481
|
+
a re-probe; there is deliberately no `--fix`.
|
|
471
482
|
|
|
472
483
|
**In CI the base is detected** — GitHub Actions (`GITHUB_BASE_REF`) and GitLab
|
|
473
484
|
merge request pipelines (`CI_MERGE_REQUEST_DIFF_BASE_SHA`, then
|
|
@@ -482,12 +493,12 @@ detected base that does not resolve is a warning for `status` and `brief`
|
|
|
482
493
|
# GitHub Actions
|
|
483
494
|
- uses: actions/checkout@v4
|
|
484
495
|
with: { fetch-depth: 0 }
|
|
485
|
-
- uses: raccioly/testguard@v0.9.
|
|
496
|
+
- uses: raccioly/testguard@v0.9.2
|
|
486
497
|
with: { command: gate }
|
|
487
498
|
|
|
488
499
|
# GitLab CI — the component-shaped template: gate + probe, brief as an artifact and, opted in, as a merge-request note
|
|
489
500
|
include:
|
|
490
|
-
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.9.
|
|
501
|
+
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.9.2/packaging/gitlab/testguard.gitlab-ci.yml'
|
|
491
502
|
inputs: { dir: backend, post_note: true } # post_note needs TESTGUARD_GITLAB_TOKEN (api scope); one note, updated in place
|
|
492
503
|
```
|
|
493
504
|
|
|
@@ -80,8 +80,9 @@ risk. TestGuard binds every fault to a **stated claim**, so the output is not
|
|
|
80
80
|
a score but a finding: *"Your project says a missing scope fails closed.
|
|
81
81
|
Nothing checks that."* That is the difference between a metric and an audit.
|
|
82
82
|
|
|
83
|
-
📄 **[Read the
|
|
84
|
-
— the idea on page one,
|
|
83
|
+
📄 **[Read the six-page technical brief (PDF)](docs/testguard-explained.pdf)**
|
|
84
|
+
— the idea on page one, then the field evidence, the anatomy of a run, the
|
|
85
|
+
verdicts, the loop and its calibration, and the prior art.
|
|
85
86
|
|
|
86
87
|
## Why
|
|
87
88
|
|
|
@@ -126,9 +127,9 @@ raises coverage. TestGuard admits it because it fails when the claim is false.
|
|
|
126
127
|
| npm | `npm i -D testguard-cli` then `npx testguard probe` |
|
|
127
128
|
| pip | `pip install testguard-cli` then `testguard probe` (needs Node ≥ 20) |
|
|
128
129
|
| Homebrew | `brew tap raccioly/tap && brew install testguard` |
|
|
129
|
-
| GitHub Action | `uses: raccioly/testguard@v0.9.
|
|
130
|
+
| GitHub Action | `uses: raccioly/testguard@v0.9.2` — see [`action.yml`](./action.yml) |
|
|
130
131
|
| pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-probe` |
|
|
131
|
-
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.9.
|
|
132
|
+
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.9.2/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
|
|
132
133
|
|
|
133
134
|
Projects that set `min-release-age` in `.npmrc` cannot see a version published
|
|
134
135
|
less than that many days ago (`ENOVERSIONS`); install that one with
|
|
@@ -140,6 +141,7 @@ less than that many days ago (`ENOVERSIONS`); install that one with
|
|
|
140
141
|
npx testguard-cli init # install the agent layer at the git root
|
|
141
142
|
npx testguard-cli status --json # where the project is, and the ONE next action
|
|
142
143
|
npx testguard-cli claims # what does this project claim, and is each claim probeable?
|
|
144
|
+
npx testguard-cli claims --check-anchors # fail fast if an exact anchor moved or a replacement no longer parses
|
|
143
145
|
npx testguard-cli scaffold src/auth.ts # propose faults for a file, as a draft to keep or drop
|
|
144
146
|
npx testguard-cli probe # try to falsify each claim; report what the tests missed
|
|
145
147
|
npx testguard-cli admit test/auth.test.ts --claim AUTH-ADMIN # does this test satisfy the two-gate rule?
|
|
@@ -150,8 +152,9 @@ npx testguard-cli replay --since v1.0..HEAD # would this suite have caught the b
|
|
|
150
152
|
npx testguard-cli mcp # serve the read-only loop over MCP, on stdio
|
|
151
153
|
```
|
|
152
154
|
|
|
153
|
-
Exit codes are the contract: `0` nothing new to prove, `1` unproven claims
|
|
154
|
-
unclaimed changes, `2` a precondition failed and
|
|
155
|
+
Exit codes are the contract: `0` nothing new to prove, `1` unproven claims,
|
|
156
|
+
unclaimed changes or invalid fault anchors, `2` a precondition failed and
|
|
157
|
+
nothing was probed, `3` usage.
|
|
155
158
|
Every command accepts `--json`.
|
|
156
159
|
|
|
157
160
|
## How it works
|
|
@@ -176,6 +179,10 @@ npx testguard-cli admit test/x.test.ts --claim X # is this test green on HEAD
|
|
|
176
179
|
claim and every fault records who produced it. `testguard claims`
|
|
177
180
|
validates the file and reports drift against `@claim <ID>` annotations in
|
|
178
181
|
source. Test files are deliberately not scanned — a claim asserted by a test is the authorship trap the tool exists for — and annotation ids must contain a hyphen so prose is never mistaken for one.
|
|
182
|
+
`claims --check-anchors` additionally locates every exact anchor and parses
|
|
183
|
+
each in-memory JavaScript or Python replacement. It runs no tests, creates
|
|
184
|
+
no worktree and changes no file, so a moved anchor is reported before an
|
|
185
|
+
expensive probe. It reports and stops; it never guesses a new anchor.
|
|
179
186
|
2. **Probe** confirms the defenders are green N times unmodified, applies
|
|
180
187
|
each fault in a scratch git worktree (your tree is never touched), runs
|
|
181
188
|
the defenders N times, re-runs survivors against the whole suite with
|
|
@@ -444,6 +451,10 @@ is reported as `EXPIRED`.
|
|
|
444
451
|
With a reference known, `status --changed <ref>` reports `unclaimed-changes`
|
|
445
452
|
**before** any evidence state and makes the claim the next action; the brief
|
|
446
453
|
lists the unclaimed files first. The claim is written before more code.
|
|
454
|
+
After claim coverage, `status` reports `invalid-anchors` before any evidence
|
|
455
|
+
state when an exact fault anchor is missing or ambiguous, and points at the
|
|
456
|
+
fault definition to repair. Repair preserves the fault's meaning and requires
|
|
457
|
+
a re-probe; there is deliberately no `--fix`.
|
|
447
458
|
|
|
448
459
|
**In CI the base is detected** — GitHub Actions (`GITHUB_BASE_REF`) and GitLab
|
|
449
460
|
merge request pipelines (`CI_MERGE_REQUEST_DIFF_BASE_SHA`, then
|
|
@@ -458,12 +469,12 @@ detected base that does not resolve is a warning for `status` and `brief`
|
|
|
458
469
|
# GitHub Actions
|
|
459
470
|
- uses: actions/checkout@v4
|
|
460
471
|
with: { fetch-depth: 0 }
|
|
461
|
-
- uses: raccioly/testguard@v0.9.
|
|
472
|
+
- uses: raccioly/testguard@v0.9.2
|
|
462
473
|
with: { command: gate }
|
|
463
474
|
|
|
464
475
|
# GitLab CI — the component-shaped template: gate + probe, brief as an artifact and, opted in, as a merge-request note
|
|
465
476
|
include:
|
|
466
|
-
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.9.
|
|
477
|
+
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.9.2/packaging/gitlab/testguard.gitlab-ci.yml'
|
|
467
478
|
inputs: { dir: backend, post_note: true } # post_note needs TESTGUARD_GITLAB_TOKEN (api scope); one note, updated in place
|
|
468
479
|
```
|
|
469
480
|
|