testguard-cli 0.9.1__tar.gz → 0.9.3__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.agents/skills/testguard/SKILL.md +3 -0
- testguard_cli-0.9.3/.github/scripts/approve-held-runs.sh +64 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/workflows/ci.yml +7 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/workflows/release.yml +5 -2
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/workflows/scheduled-release.yml +4 -50
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/CHANGELOG.md +36 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/PKG-INFO +17 -7
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/README.md +16 -6
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/action.yml +1 -1
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/docs/testguard-explained.html +1 -1
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/docs/testguard-explained.pdf +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/docs-canonical/ARCHITECTURE.md +6 -3
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/docs-canonical/DATA-MODEL.md +6 -1
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/docs-canonical/REQUIREMENTS.md +3 -1
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/docs-canonical/TEST-SPEC.md +6 -1
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/package-lock.json +2 -2
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/package.json +1 -1
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/packaging/gitlab/testguard.gitlab-ci.yml +2 -2
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/packaging/homebrew/testguard.rb +2 -2
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/pyproject.toml +1 -1
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/GATE-SEMANTICS.md +20 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/status.json +1 -0
- testguard_cli-0.9.3/spec/conformance/invalid/status.invalid-anchor-hidden.json +27 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/lib/validate.mjs +4 -1
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/schemas/brief.schema.json +1 -1
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/schemas/status.schema.json +20 -1
- testguard_cli-0.9.3/src/claims/anchors.mjs +218 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/cli.mjs +5 -2
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/commands/claims.mjs +10 -2
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/init/templates/SKILL.md +5 -1
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/status/status.mjs +19 -0
- testguard_cli-0.9.3/test/claims-anchors.test.mjs +90 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/gate.test.mjs +11 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/mcp.test.mjs +11 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/probe.fixture.test.mjs +1 -1
- testguard_cli-0.9.3/test/release-approval.test.mjs +78 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/status.test.mjs +26 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/testguard.claims.json +127 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/testguard.ignore.json +7 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.codex/hooks.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.docguard.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.docguardignore +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.gitattributes +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/ISSUE_TEMPLATE/bug_report.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/ISSUE_TEMPLATE/feature_request.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/PULL_REQUEST_TEMPLATE.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/dependabot.yml +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/scripts/build-one-pager.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/scripts/check-cost-budget.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/scripts/check-python-fixture.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/scripts/install-smoke.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/scripts/sync-release-version.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/workflows/auto-merge.yml +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.github/workflows/supply-chain.yml +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.gitignore +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.npmignore +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.npmrc +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.pre-commit-config.yaml +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/.pre-commit-hooks.yaml +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/AGENTS.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/CLAUDE.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/CODE_OF_CONDUCT.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/CONTRIBUTING.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/DRIFT-LOG.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/LICENSE +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/PRIVACY.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/SECURITY.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/SUPPORT.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/bench/README.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/cli/testguard.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/docs-canonical/ENVIRONMENT.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/docs-canonical/SECURITY.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer/.gitignore +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer/README.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer/expected.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer/package.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer/src/export.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer/src/redact.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer/test/export-mocked.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer/test/flaky.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer/test/redact.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer/testguard.claims.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer/vitest.config.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-jest/.gitignore +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-jest/README.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-jest/expected.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-jest/jest.config.js +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-jest/package.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-jest/src/export.js +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-jest/src/redact.js +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-jest/test/export-mocked.test.js +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-jest/test/flaky.test.js +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-jest/test/redact.test.js +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-jest/testguard.claims.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/.gitignore +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/README.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/e2e/flaky.spec.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/e2e/toggle.spec.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/expected.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/package-lock.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/package.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/playwright.config.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/src/toggle.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/test/toggle.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/test-results/.last-run.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/testguard.claims.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-playwright/vitest.config.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/.gitignore +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/README.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/demo/__init__.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/demo/export.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/demo/orphan.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/demo/pipeline.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/demo/redact.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/expected.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/testguard.claims.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/tests/test_export_mocked.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/tests/test_flaky.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/tests/test_patched.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/fixtures/known-answer-python/tests/test_redact.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/README.md +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/baseline.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/brief.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/calibration-websec.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/calibration.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/claims-methods.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/claims.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/evidence-disturbed-run.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/evidence-negative-control.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/evidence-provisional.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/evidence-python.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/evidence-scan-method.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/evidence.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/gate.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/ignore.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/examples/replay.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/baseline.bad-fingerprint-key.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/baseline.zero-count.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/brief.text-without-heading.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/brief.unknown-verdict-key.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/calibration.backoff-cell-not-wilson.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/calibration.breakdown-does-not-sum-to-n.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/calibration.ci-not-wilson.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/calibration.compound-key-arity-mismatch.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/calibration.fallback-without-basis.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/calibration.p-not-positives-over-n.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/calibration.p-outside-ci.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/calibration.positives-exceed-n.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/claims.absolute-path.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/claims.duplicate-fault-id.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/claims.fault-injection-without-a-substitution.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/claims.method-detail-on-a-specified-method.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/claims.missing-provenance.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/claims.noop-fault.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/claims.occurrence-exceeds-hits.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/claims.parent-traversal.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.annotated-signal-without-reason.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.attribute-signal-not-a-defender.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.attribute-signal-without-attributes.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.byrunner-file-not-resolved.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.contention-without-runners.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.discovered-defender-mocks-subject.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.fault-injection-without-its-evidence.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.flaky-defender-without-flakiness.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.ignored-dirty-on-snapshot.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.killed-by-non-assertion.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.killed-on-red-baseline.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.killed-without-n-runs.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.method-detail-on-an-injecting-run.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.negative-control-not-reached-but-survived.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.negative-control-reached-on-a-kill.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.probe-error-without-message.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.provisional-flag-on-confirmed-run.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.rank-extra-property.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.restore-skipped-in-place.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.single-run-without-provisional.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.target-not-imported-but-killed.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.undeclared-killers-without-reason.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.unknown-verdict.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.unverifiable-without-reason.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/evidence.wrong-fingerprint.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/gate.counts-disagree.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/gate.exit-zero-with-uncovered.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/gate.ignore-cover-without-pattern.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/ignore.missing-reason.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/ignore.short-reason.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/replay.blind-with-a-failing-run.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/replay.duplicate-patch-id.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/status.claim-without-file.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/status.clean-with-new-findings.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/status.unclaimed-hidden-behind-unproven.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/invalid/status.write-test-without-target.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/conformance/schemas.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/lib/fingerprint.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/lib/wilson.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/schemas/baseline.schema.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/schemas/calibration.schema.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/schemas/claims.schema.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/schemas/common.schema.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/schemas/evidence.schema.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/schemas/gate.schema.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/schemas/ignore.schema.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/spec/schemas/replay.schema.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/admit/admit.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/baseline/baseline.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/brief/brief.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/claims/annotations.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/claims/load.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/claims/removed.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/commands/admit.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/commands/baseline.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/commands/brief.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/commands/gate.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/commands/init.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/commands/mcp.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/commands/probe.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/commands/replay.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/commands/scaffold.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/commands/status.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/evidence/writer.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/gate/changed.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/git.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/init/init.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/mcp/server.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/mcp/tools.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/attribution.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/classify.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/contention.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/cost.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/discover.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/independence.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/inject.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/mocks.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/probe.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/progress.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/pyimports.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/rank.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/runners/index.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/runners/jest.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/runners/playwright.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/runners/python/_testguard_pytest_plugin.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/runners/python/_testguard_report.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/runners/python/_testguard_unittest_main.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/runners/python.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/runners/shared.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/runners/vitest.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/probe/worktree.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/render.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/replay/label.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/replay/replay.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/scaffold/producers.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/scaffold/producers.python.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/scaffold/scaffold.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/util/glob.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/src/util/hash.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/admit.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/annotations.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/attribution.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/baseline.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/brief.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/calibration.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/claims-since.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/classify.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/contention.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/cost.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/discover.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/gitlab-template.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/glob.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/independence.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/init.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/inject.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/label.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/mocks.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/negative-control.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/one-pager.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/probe-error.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/probe-preconditions.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/probe.jest.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/probe.playwright.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/probe.python.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/progress.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/pyimports.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/rank-aliases.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/release-sync.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/replay-verdict.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/replay.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/runner-argv.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/runner-command.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/runner-playwright.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/runner-python.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/runner-vitest.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/samples/playwright-report.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/samples/python-report-collect-error.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/samples/python-report.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/scaffold.python.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/scaffold.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/snapshot.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/worktree.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/test/writer.test.mjs +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/testguard.cost-budget.json +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/testguard_cli/__init__.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/testguard_cli/wrapper.py +0 -0
- {testguard_cli-0.9.1 → testguard_cli-0.9.3}/vitest.config.mjs +0 -0
|
@@ -21,6 +21,7 @@ which files exist. `state` is one of:
|
|
|
21
21
|
| state | meaning | you do |
|
|
22
22
|
|---|---|---|
|
|
23
23
|
| `no-claims` | no `testguard.claims.json` | `testguard scaffold <file>` for a file with guards; replace every `TODO:` statement with what the code guarantees; keep or drop each proposal; move the claims into `testguard.claims.json` |
|
|
24
|
+
| `invalid-anchors` | an exact fault anchor moved or became ambiguous | run `testguard claims --check-anchors`; repair `next.target` in the claims file without changing what the fault means, then re-probe. Never guess or auto-fix an anchor. |
|
|
24
25
|
| `unprobed` | claims never probed | `testguard probe` |
|
|
25
26
|
| `evidence-stale` | code, tests or claims changed since the evidence | `testguard probe --include-dirty` (or `--claim <ID>` for one) |
|
|
26
27
|
| `provisional-only` | only `--confirm 1` evidence exists | `testguard probe --confirm 3` |
|
|
@@ -44,6 +45,8 @@ Exit codes: `0` clean · `1` unproven claims (or drift) · `2` precondition fail
|
|
|
44
45
|
## The fix loop
|
|
45
46
|
|
|
46
47
|
1. `testguard status --json` → take `next.target`.
|
|
48
|
+
After editing guarded code, run `testguard claims --check-anchors` before
|
|
49
|
+
any probe; repair a named fault definition without weakening its meaning.
|
|
47
50
|
2. Write the test. It must **fail when the fault is applied and pass on HEAD**. To check the first half by hand: apply `find` → `replace` in the target file, run the defender, restore.
|
|
48
51
|
3. `testguard probe --claim <ID> --include-dirty` — probes your uncommitted test without touching the tree. `--confirm 1` gives a fast **provisional** signal; a `?` on a verdict means unconfirmed.
|
|
49
52
|
4. When it is `killed` at `--confirm 3`, commit the test.
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
|
|
3
|
+
# Clear GitHub's first-time-contributor hold from workflow runs created for a
|
|
4
|
+
# GITHUB_TOKEN-authored pull request. The repository requires that approval,
|
|
5
|
+
# and github-actions[bot] can never graduate from it because it does not author
|
|
6
|
+
# the squash commit that lands on main.
|
|
7
|
+
|
|
8
|
+
set -euo pipefail
|
|
9
|
+
|
|
10
|
+
HEAD_SHA="${1:?usage: approve-held-runs.sh HEAD_SHA PR_URL}"
|
|
11
|
+
PR_URL="${2:?usage: approve-held-runs.sh HEAD_SHA PR_URL}"
|
|
12
|
+
: "${GITHUB_REPOSITORY:?GITHUB_REPOSITORY is required}"
|
|
13
|
+
|
|
14
|
+
POLL_SECONDS="${TESTGUARD_HELD_RUN_POLL_SECONDS:-10}"
|
|
15
|
+
approved=0
|
|
16
|
+
|
|
17
|
+
held_ids() {
|
|
18
|
+
gh api "repos/${GITHUB_REPOSITORY}/actions/runs?head_sha=${HEAD_SHA}" \
|
|
19
|
+
--jq '[.workflow_runs[] | select(.status == "action_required" or .conclusion == "action_required") | .id] | .[]'
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
held_urls() {
|
|
23
|
+
gh api "repos/${GITHUB_REPOSITORY}/actions/runs?head_sha=${HEAD_SHA}" \
|
|
24
|
+
--jq '[.workflow_runs[] | select(.status == "action_required" or .conclusion == "action_required") | .html_url] | .[]'
|
|
25
|
+
}
|
|
26
|
+
|
|
27
|
+
echo "Clearing held workflow runs on $HEAD_SHA"
|
|
28
|
+
for attempt in 1 2 3 4 5 6; do
|
|
29
|
+
# Fail closed. Treating an API failure as "no held runs" would report that
|
|
30
|
+
# auto-merge can proceed while leaving the PR queued forever.
|
|
31
|
+
if ! HELD=$(held_ids); then
|
|
32
|
+
echo "::error::Could not inspect workflow approval holds for $HEAD_SHA; auto-merge on $PR_URL may remain blocked."
|
|
33
|
+
exit 1
|
|
34
|
+
fi
|
|
35
|
+
|
|
36
|
+
for id in $HELD; do
|
|
37
|
+
if gh api -X POST "repos/${GITHUB_REPOSITORY}/actions/runs/${id}/approve" >/dev/null 2>&1; then
|
|
38
|
+
echo " approved run $id"
|
|
39
|
+
approved=$((approved + 1))
|
|
40
|
+
else
|
|
41
|
+
# A run can leave action_required between the query and the POST. The
|
|
42
|
+
# final query decides whether that race is harmless.
|
|
43
|
+
echo " run $id needed no approval"
|
|
44
|
+
fi
|
|
45
|
+
done
|
|
46
|
+
|
|
47
|
+
# Runs appear asynchronously. Once one was approved, the first empty query
|
|
48
|
+
# is terminal; when this repository gate does not apply, three empty polls
|
|
49
|
+
# are enough to establish that without paying the full minute.
|
|
50
|
+
if [ -z "$HELD" ] && { [ "$approved" -gt 0 ] || [ "$attempt" -ge 3 ]; }; then
|
|
51
|
+
break
|
|
52
|
+
fi
|
|
53
|
+
if [ "$attempt" -lt 6 ]; then sleep "$POLL_SECONDS"; fi
|
|
54
|
+
done
|
|
55
|
+
|
|
56
|
+
if ! STILL=$(held_urls); then
|
|
57
|
+
echo "::error::Could not verify workflow approval holds for $HEAD_SHA; auto-merge on $PR_URL may remain blocked."
|
|
58
|
+
exit 1
|
|
59
|
+
fi
|
|
60
|
+
if [ -n "$STILL" ]; then
|
|
61
|
+
echo "::error::Workflow runs are still held for approval, so auto-merge on $PR_URL will wait forever. Approve them, or re-run this workflow: $STILL"
|
|
62
|
+
exit 1
|
|
63
|
+
fi
|
|
64
|
+
echo "✅ no run is held for approval; auto-merge can complete"
|
|
@@ -185,6 +185,13 @@ jobs:
|
|
|
185
185
|
if: github.event_name == 'pull_request'
|
|
186
186
|
run: node cli/testguard.mjs gate . --changed "origin/${{ github.base_ref }}"
|
|
187
187
|
|
|
188
|
+
# Exact anchors and parseable replacements are knowable without a test
|
|
189
|
+
# runner. Fail here instead of spending the long self-probe merely to
|
|
190
|
+
# discover that a fault could never be applied.
|
|
191
|
+
- name: claims — fault anchors and replacements are valid
|
|
192
|
+
if: matrix.node-version == 22
|
|
193
|
+
run: node cli/testguard.mjs claims . --check-anchors
|
|
194
|
+
|
|
188
195
|
# ── Self-verification: TestGuard is governed by the standard it defines.
|
|
189
196
|
# testguard.claims.json states invariants of the tool itself; every
|
|
190
197
|
# fault must be killed by the tool's own tests. Exit 0 = all defended. ──
|
|
@@ -236,8 +236,10 @@ jobs:
|
|
|
236
236
|
# change lands as a bot PR with native auto-merge armed, and this job
|
|
237
237
|
# dispatches ci.yml on the branch itself — a PR opened with GITHUB_TOKEN
|
|
238
238
|
# gets no pull_request run, but workflow_dispatch is the documented
|
|
239
|
-
# exception.
|
|
240
|
-
#
|
|
239
|
+
# exception. The PR's own runs may still appear held for first-time-
|
|
240
|
+
# contributor approval, so this job clears those holds before returning.
|
|
241
|
+
# That also means the merge triggers nothing here; the hourly sweep
|
|
242
|
+
# observes the merged formula and finds brew_missing=false.
|
|
241
243
|
#
|
|
242
244
|
# Idempotent: an existing branch is reused, an existing open PR is left
|
|
243
245
|
# alone (CI re-dispatched only if its head has no check runs), a closed
|
|
@@ -334,3 +336,4 @@ jobs:
|
|
|
334
336
|
else
|
|
335
337
|
echo "✅ $HEAD_SHA already has $RUNS check run(s)"
|
|
336
338
|
fi
|
|
339
|
+
.github/scripts/approve-held-runs.sh "$HEAD_SHA" "$PR"
|
|
@@ -170,56 +170,10 @@ jobs:
|
|
|
170
170
|
echo "✅ $HEAD_SHA already has $RUNS check run(s)"
|
|
171
171
|
fi
|
|
172
172
|
|
|
173
|
-
#
|
|
174
|
-
#
|
|
175
|
-
#
|
|
176
|
-
|
|
177
|
-
# PERMANENTLY one — it never authors a merged commit, so it can
|
|
178
|
-
# never graduate. Those held runs leave check suites QUEUED on the
|
|
179
|
-
# head sha forever, the status rollup never reaches a terminal
|
|
180
|
-
# state, and auto-merge waits on it even though the required
|
|
181
|
-
# contexts (test 20/22/24) are already green from the dispatched
|
|
182
|
-
# run above. v0.9.0 sat here until a human clicked Approve.
|
|
183
|
-
#
|
|
184
|
-
# GITHUB_TOKEN may clear it: `actions: write` carries the approve
|
|
185
|
-
# permission, measured 201 on a bot PR's own held run. It may not
|
|
186
|
-
# approve a REVIEW, which is a different gate and still ours alone.
|
|
187
|
-
#
|
|
188
|
-
# Runs appear asynchronously, so poll. Approving an already-running
|
|
189
|
-
# run answers 403 "not waiting for approval" — success, not failure.
|
|
190
|
-
echo "Clearing held workflow runs on $HEAD_SHA"
|
|
191
|
-
approved=0
|
|
192
|
-
for attempt in 1 2 3 4 5 6; do
|
|
193
|
-
HELD=$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs?head_sha=${HEAD_SHA}" \
|
|
194
|
-
--jq '[.workflow_runs[] | select(.status == "action_required" or .conclusion == "action_required") | .id] | .[]' 2>/dev/null || true)
|
|
195
|
-
for id in $HELD; do
|
|
196
|
-
if gh api -X POST "repos/${GITHUB_REPOSITORY}/actions/runs/${id}/approve" >/dev/null 2>&1; then
|
|
197
|
-
echo " approved run $id"
|
|
198
|
-
approved=$((approved + 1))
|
|
199
|
-
else
|
|
200
|
-
echo " run $id needed no approval"
|
|
201
|
-
fi
|
|
202
|
-
done
|
|
203
|
-
# Stop as soon as the picture is settled, rather than always
|
|
204
|
-
# paying the full poll: nothing held after something was
|
|
205
|
-
# approved means we are done, and nothing held by the third
|
|
206
|
-
# look means the gate is not applying to this run at all.
|
|
207
|
-
if [ -z "$HELD" ] && { [ "$approved" -gt 0 ] || [ "$attempt" -ge 3 ]; }; then
|
|
208
|
-
break
|
|
209
|
-
fi
|
|
210
|
-
sleep 10
|
|
211
|
-
done
|
|
212
|
-
|
|
213
|
-
# Never leave this silent: a run still held here means auto-merge
|
|
214
|
-
# will wait forever, which is the failure this whole block exists
|
|
215
|
-
# to remove. Say so, and name what to click.
|
|
216
|
-
STILL=$(gh api "repos/${GITHUB_REPOSITORY}/actions/runs?head_sha=${HEAD_SHA}" \
|
|
217
|
-
--jq '[.workflow_runs[] | select(.status == "action_required" or .conclusion == "action_required") | .html_url] | .[]' 2>/dev/null || true)
|
|
218
|
-
if [ -n "$STILL" ]; then
|
|
219
|
-
echo "::error::Workflow runs are still held for approval, so auto-merge on $PR will wait forever. Approve them, or re-run this workflow: $STILL"
|
|
220
|
-
exit 1
|
|
221
|
-
fi
|
|
222
|
-
echo "✅ no run is held for approval; auto-merge can complete"
|
|
173
|
+
# Clear the first-time-contributor holds on this bot-created PR's
|
|
174
|
+
# workflow runs. The Homebrew PR path uses the same helper so the
|
|
175
|
+
# two release automations cannot drift again.
|
|
176
|
+
.github/scripts/approve-held-runs.sh "$HEAD_SHA" "$PR"
|
|
223
177
|
else
|
|
224
178
|
cat /tmp/pr-err
|
|
225
179
|
if grep -q 'not permitted to create or approve pull requests' /tmp/pr-err; then
|
|
@@ -7,6 +7,42 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.9.3] - 2026-09-19
|
|
11
|
+
|
|
12
|
+
Automated weekly release — everything merged since `v0.9.2`.
|
|
13
|
+
|
|
14
|
+
### Changed
|
|
15
|
+
|
|
16
|
+
- fix(release): clear held Homebrew PR runs (#129)
|
|
17
|
+
- chore(homebrew): sha256 for v0.9.2 (#128)
|
|
18
|
+
|
|
19
|
+
|
|
20
|
+
### Fixed
|
|
21
|
+
|
|
22
|
+
- Bot-created Homebrew checksum PRs now clear GitHub's first-time-contributor
|
|
23
|
+
workflow holds themselves, using the same fail-closed helper as scheduled
|
|
24
|
+
release PRs. A release no longer finishes publishing while leaving its final
|
|
25
|
+
formula update waiting for a manual Approve click.
|
|
26
|
+
|
|
27
|
+
## [0.9.2] - 2026-09-19
|
|
28
|
+
|
|
29
|
+
Automated weekly release — everything merged since `v0.9.1`.
|
|
30
|
+
|
|
31
|
+
### Changed
|
|
32
|
+
|
|
33
|
+
- feat(claims): preflight fault anchors (#126)
|
|
34
|
+
- chore(homebrew): sha256 for v0.9.1 (#125)
|
|
35
|
+
|
|
36
|
+
|
|
37
|
+
### Added
|
|
38
|
+
|
|
39
|
+
- `claims --check-anchors` locates every exact fault anchor and syntax-checks
|
|
40
|
+
supported in-memory JavaScript and Python replacements without starting a
|
|
41
|
+
runner or touching the tree. CI runs the sub-second preflight before the
|
|
42
|
+
self-probe, while `status` reports rotted anchors as `invalid-anchors` with
|
|
43
|
+
the precise fault to repair. Repairs remain a reviewed judgement call:
|
|
44
|
+
there is deliberately no fuzzy match or `--fix`.
|
|
45
|
+
|
|
10
46
|
## [0.9.1] - 2026-09-19
|
|
11
47
|
|
|
12
48
|
Automated weekly release — everything merged since `v0.9.0`.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: testguard-cli
|
|
3
|
-
Version: 0.9.
|
|
3
|
+
Version: 0.9.3
|
|
4
4
|
Summary: Breaks your code on purpose and reports every promise your tests did not notice breaking. Mutation testing bound to the claims a project makes — vitest, jest, Playwright, pytest and unittest. Ships the Node.js CLI (requires Node.js 20+).
|
|
5
5
|
Project-URL: Homepage, https://github.com/raccioly/testguard
|
|
6
6
|
Project-URL: Documentation, https://github.com/raccioly/testguard#readme
|
|
@@ -151,9 +151,9 @@ raises coverage. TestGuard admits it because it fails when the claim is false.
|
|
|
151
151
|
| npm | `npm i -D testguard-cli` then `npx testguard probe` |
|
|
152
152
|
| pip | `pip install testguard-cli` then `testguard probe` (needs Node ≥ 20) |
|
|
153
153
|
| Homebrew | `brew tap raccioly/tap && brew install testguard` |
|
|
154
|
-
| GitHub Action | `uses: raccioly/testguard@v0.9.
|
|
154
|
+
| GitHub Action | `uses: raccioly/testguard@v0.9.3` — see [`action.yml`](./action.yml) |
|
|
155
155
|
| pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-probe` |
|
|
156
|
-
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.9.
|
|
156
|
+
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.9.3/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
|
|
157
157
|
|
|
158
158
|
Projects that set `min-release-age` in `.npmrc` cannot see a version published
|
|
159
159
|
less than that many days ago (`ENOVERSIONS`); install that one with
|
|
@@ -165,6 +165,7 @@ less than that many days ago (`ENOVERSIONS`); install that one with
|
|
|
165
165
|
npx testguard-cli init # install the agent layer at the git root
|
|
166
166
|
npx testguard-cli status --json # where the project is, and the ONE next action
|
|
167
167
|
npx testguard-cli claims # what does this project claim, and is each claim probeable?
|
|
168
|
+
npx testguard-cli claims --check-anchors # fail fast if an exact anchor moved or a replacement no longer parses
|
|
168
169
|
npx testguard-cli scaffold src/auth.ts # propose faults for a file, as a draft to keep or drop
|
|
169
170
|
npx testguard-cli probe # try to falsify each claim; report what the tests missed
|
|
170
171
|
npx testguard-cli admit test/auth.test.ts --claim AUTH-ADMIN # does this test satisfy the two-gate rule?
|
|
@@ -175,8 +176,9 @@ npx testguard-cli replay --since v1.0..HEAD # would this suite have caught the b
|
|
|
175
176
|
npx testguard-cli mcp # serve the read-only loop over MCP, on stdio
|
|
176
177
|
```
|
|
177
178
|
|
|
178
|
-
Exit codes are the contract: `0` nothing new to prove, `1` unproven claims
|
|
179
|
-
unclaimed changes, `2` a precondition failed and
|
|
179
|
+
Exit codes are the contract: `0` nothing new to prove, `1` unproven claims,
|
|
180
|
+
unclaimed changes or invalid fault anchors, `2` a precondition failed and
|
|
181
|
+
nothing was probed, `3` usage.
|
|
180
182
|
Every command accepts `--json`.
|
|
181
183
|
|
|
182
184
|
## How it works
|
|
@@ -201,6 +203,10 @@ npx testguard-cli admit test/x.test.ts --claim X # is this test green on HEAD
|
|
|
201
203
|
claim and every fault records who produced it. `testguard claims`
|
|
202
204
|
validates the file and reports drift against `@claim <ID>` annotations in
|
|
203
205
|
source. Test files are deliberately not scanned — a claim asserted by a test is the authorship trap the tool exists for — and annotation ids must contain a hyphen so prose is never mistaken for one.
|
|
206
|
+
`claims --check-anchors` additionally locates every exact anchor and parses
|
|
207
|
+
each in-memory JavaScript or Python replacement. It runs no tests, creates
|
|
208
|
+
no worktree and changes no file, so a moved anchor is reported before an
|
|
209
|
+
expensive probe. It reports and stops; it never guesses a new anchor.
|
|
204
210
|
2. **Probe** confirms the defenders are green N times unmodified, applies
|
|
205
211
|
each fault in a scratch git worktree (your tree is never touched), runs
|
|
206
212
|
the defenders N times, re-runs survivors against the whole suite with
|
|
@@ -469,6 +475,10 @@ is reported as `EXPIRED`.
|
|
|
469
475
|
With a reference known, `status --changed <ref>` reports `unclaimed-changes`
|
|
470
476
|
**before** any evidence state and makes the claim the next action; the brief
|
|
471
477
|
lists the unclaimed files first. The claim is written before more code.
|
|
478
|
+
After claim coverage, `status` reports `invalid-anchors` before any evidence
|
|
479
|
+
state when an exact fault anchor is missing or ambiguous, and points at the
|
|
480
|
+
fault definition to repair. Repair preserves the fault's meaning and requires
|
|
481
|
+
a re-probe; there is deliberately no `--fix`.
|
|
472
482
|
|
|
473
483
|
**In CI the base is detected** — GitHub Actions (`GITHUB_BASE_REF`) and GitLab
|
|
474
484
|
merge request pipelines (`CI_MERGE_REQUEST_DIFF_BASE_SHA`, then
|
|
@@ -483,12 +493,12 @@ detected base that does not resolve is a warning for `status` and `brief`
|
|
|
483
493
|
# GitHub Actions
|
|
484
494
|
- uses: actions/checkout@v4
|
|
485
495
|
with: { fetch-depth: 0 }
|
|
486
|
-
- uses: raccioly/testguard@v0.9.
|
|
496
|
+
- uses: raccioly/testguard@v0.9.3
|
|
487
497
|
with: { command: gate }
|
|
488
498
|
|
|
489
499
|
# GitLab CI — the component-shaped template: gate + probe, brief as an artifact and, opted in, as a merge-request note
|
|
490
500
|
include:
|
|
491
|
-
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.9.
|
|
501
|
+
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.9.3/packaging/gitlab/testguard.gitlab-ci.yml'
|
|
492
502
|
inputs: { dir: backend, post_note: true } # post_note needs TESTGUARD_GITLAB_TOKEN (api scope); one note, updated in place
|
|
493
503
|
```
|
|
494
504
|
|
|
@@ -127,9 +127,9 @@ raises coverage. TestGuard admits it because it fails when the claim is false.
|
|
|
127
127
|
| npm | `npm i -D testguard-cli` then `npx testguard probe` |
|
|
128
128
|
| pip | `pip install testguard-cli` then `testguard probe` (needs Node ≥ 20) |
|
|
129
129
|
| Homebrew | `brew tap raccioly/tap && brew install testguard` |
|
|
130
|
-
| GitHub Action | `uses: raccioly/testguard@v0.9.
|
|
130
|
+
| GitHub Action | `uses: raccioly/testguard@v0.9.3` — see [`action.yml`](./action.yml) |
|
|
131
131
|
| pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-probe` |
|
|
132
|
-
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.9.
|
|
132
|
+
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.9.3/packaging/gitlab/testguard.gitlab-ci.yml` with `inputs:` — see [`packaging/gitlab/`](./packaging/gitlab/testguard.gitlab-ci.yml) |
|
|
133
133
|
|
|
134
134
|
Projects that set `min-release-age` in `.npmrc` cannot see a version published
|
|
135
135
|
less than that many days ago (`ENOVERSIONS`); install that one with
|
|
@@ -141,6 +141,7 @@ less than that many days ago (`ENOVERSIONS`); install that one with
|
|
|
141
141
|
npx testguard-cli init # install the agent layer at the git root
|
|
142
142
|
npx testguard-cli status --json # where the project is, and the ONE next action
|
|
143
143
|
npx testguard-cli claims # what does this project claim, and is each claim probeable?
|
|
144
|
+
npx testguard-cli claims --check-anchors # fail fast if an exact anchor moved or a replacement no longer parses
|
|
144
145
|
npx testguard-cli scaffold src/auth.ts # propose faults for a file, as a draft to keep or drop
|
|
145
146
|
npx testguard-cli probe # try to falsify each claim; report what the tests missed
|
|
146
147
|
npx testguard-cli admit test/auth.test.ts --claim AUTH-ADMIN # does this test satisfy the two-gate rule?
|
|
@@ -151,8 +152,9 @@ npx testguard-cli replay --since v1.0..HEAD # would this suite have caught the b
|
|
|
151
152
|
npx testguard-cli mcp # serve the read-only loop over MCP, on stdio
|
|
152
153
|
```
|
|
153
154
|
|
|
154
|
-
Exit codes are the contract: `0` nothing new to prove, `1` unproven claims
|
|
155
|
-
unclaimed changes, `2` a precondition failed and
|
|
155
|
+
Exit codes are the contract: `0` nothing new to prove, `1` unproven claims,
|
|
156
|
+
unclaimed changes or invalid fault anchors, `2` a precondition failed and
|
|
157
|
+
nothing was probed, `3` usage.
|
|
156
158
|
Every command accepts `--json`.
|
|
157
159
|
|
|
158
160
|
## How it works
|
|
@@ -177,6 +179,10 @@ npx testguard-cli admit test/x.test.ts --claim X # is this test green on HEAD
|
|
|
177
179
|
claim and every fault records who produced it. `testguard claims`
|
|
178
180
|
validates the file and reports drift against `@claim <ID>` annotations in
|
|
179
181
|
source. Test files are deliberately not scanned — a claim asserted by a test is the authorship trap the tool exists for — and annotation ids must contain a hyphen so prose is never mistaken for one.
|
|
182
|
+
`claims --check-anchors` additionally locates every exact anchor and parses
|
|
183
|
+
each in-memory JavaScript or Python replacement. It runs no tests, creates
|
|
184
|
+
no worktree and changes no file, so a moved anchor is reported before an
|
|
185
|
+
expensive probe. It reports and stops; it never guesses a new anchor.
|
|
180
186
|
2. **Probe** confirms the defenders are green N times unmodified, applies
|
|
181
187
|
each fault in a scratch git worktree (your tree is never touched), runs
|
|
182
188
|
the defenders N times, re-runs survivors against the whole suite with
|
|
@@ -445,6 +451,10 @@ is reported as `EXPIRED`.
|
|
|
445
451
|
With a reference known, `status --changed <ref>` reports `unclaimed-changes`
|
|
446
452
|
**before** any evidence state and makes the claim the next action; the brief
|
|
447
453
|
lists the unclaimed files first. The claim is written before more code.
|
|
454
|
+
After claim coverage, `status` reports `invalid-anchors` before any evidence
|
|
455
|
+
state when an exact fault anchor is missing or ambiguous, and points at the
|
|
456
|
+
fault definition to repair. Repair preserves the fault's meaning and requires
|
|
457
|
+
a re-probe; there is deliberately no `--fix`.
|
|
448
458
|
|
|
449
459
|
**In CI the base is detected** — GitHub Actions (`GITHUB_BASE_REF`) and GitLab
|
|
450
460
|
merge request pipelines (`CI_MERGE_REQUEST_DIFF_BASE_SHA`, then
|
|
@@ -459,12 +469,12 @@ detected base that does not resolve is a warning for `status` and `brief`
|
|
|
459
469
|
# GitHub Actions
|
|
460
470
|
- uses: actions/checkout@v4
|
|
461
471
|
with: { fetch-depth: 0 }
|
|
462
|
-
- uses: raccioly/testguard@v0.9.
|
|
472
|
+
- uses: raccioly/testguard@v0.9.3
|
|
463
473
|
with: { command: gate }
|
|
464
474
|
|
|
465
475
|
# GitLab CI — the component-shaped template: gate + probe, brief as an artifact and, opted in, as a merge-request note
|
|
466
476
|
include:
|
|
467
|
-
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.9.
|
|
477
|
+
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.9.3/packaging/gitlab/testguard.gitlab-ci.yml'
|
|
468
478
|
inputs: { dir: backend, post_note: true } # post_note needs TESTGUARD_GITLAB_TOKEN (api scope); one note, updated in place
|
|
469
479
|
```
|
|
470
480
|
|
|
@@ -144,7 +144,7 @@
|
|
|
144
144
|
<div class="page">
|
|
145
145
|
<div class="masthead">
|
|
146
146
|
<div class="wordmark">Test<span class="mark">Guard</span></div>
|
|
147
|
-
<div class="meta">Technical Brief · v0.9.
|
|
147
|
+
<div class="meta">Technical Brief · v0.9.3 · MIT</div>
|
|
148
148
|
</div>
|
|
149
149
|
|
|
150
150
|
<h1>Your tests pass. That is not the same as your tests working.</h1>
|
|
Binary file
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
<!-- docguard:version 1.0.0 -->
|
|
4
4
|
<!-- docguard:status approved -->
|
|
5
|
-
<!-- docguard:last-reviewed 2026-09-
|
|
5
|
+
<!-- docguard:last-reviewed 2026-09-19 -->
|
|
6
6
|
<!-- docguard:owner @raccioly -->
|
|
7
7
|
<!-- docguard:quality negation-load off — this tool is defined by what must not happen: no network, no telemetry, never optimistic under uncertainty. Stating those positively would misdescribe them. -->
|
|
8
8
|
|
|
@@ -35,7 +35,7 @@ code were wrong, would anything fail?*
|
|
|
35
35
|
|---|---|---|
|
|
36
36
|
| CLI dispatch | Parse argv, select a command, map errors to exit codes | `src/cli.mjs`, `cli/testguard.mjs` |
|
|
37
37
|
| Commands | Thin wrappers: read flags, call a module, render output | `src/commands/*.mjs` |
|
|
38
|
-
| Claims | Load and validate the claims file; reconcile `@claim` annotations; detect removed claims | `src/claims/` |
|
|
38
|
+
| Claims | Load and validate the claims file; reconcile `@claim` annotations; detect removed claims; preflight exact anchors and supported replacement syntax | `src/claims/` |
|
|
39
39
|
| Probe engine | Isolate, inject, run, escalate, restore, classify, rank | `src/probe/probe.mjs` |
|
|
40
40
|
| Verdict function | The pure decision: runs in, verdict out | `src/probe/classify.mjs` |
|
|
41
41
|
| Injector | Apply and restore a fault; anchor location | `src/probe/inject.mjs` |
|
|
@@ -131,7 +131,7 @@ to every other surface and `--check` fails the build on drift.
|
|
|
131
131
|
|
|
132
132
|
| Stage | Where | Gate |
|
|
133
133
|
|---|---|---|
|
|
134
|
-
| Pull request | `ci.yml` | Suite on Node 20/22/24, syntax check of every source, install-from-tarball smoke, Python wrapper import, fixture oracles for vitest and jest, Playwright job, change gate, removed-claim check |
|
|
134
|
+
| Pull request | `ci.yml` | Suite on Node 20/22/24, syntax check of every source, install-from-tarball smoke, Python wrapper import, fixture oracles for vitest and jest, Playwright job, change gate, removed-claim check, anchor/replacement preflight before self-probe |
|
|
135
135
|
| Release gate | `ci.yml` | TestGuard probes its own claims; every fault must be killed |
|
|
136
136
|
| Merge to main | `release.yml` | A version with no tag is tagged, released, and published to npm and PyPI over OIDC |
|
|
137
137
|
| Recovery | `release.yml` | Hourly sweep; each registry is gated on its own state, so a partial publish is fixed by re-running |
|
|
@@ -141,6 +141,9 @@ to every other surface and `--check` fails the build on drift.
|
|
|
141
141
|
```
|
|
142
142
|
testguard.claims.json (what must be true)
|
|
143
143
|
│
|
|
144
|
+
claims --check-anchors
|
|
145
|
+
(exact anchors + syntax, no tests)
|
|
146
|
+
│
|
|
144
147
|
▼
|
|
145
148
|
┌─────────── probe ────────────────────────────────────────────┐
|
|
146
149
|
│ scratch git worktree ←── isolation, never your tree │
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
<!-- docguard:version 1.0.0 -->
|
|
4
4
|
<!-- docguard:status approved -->
|
|
5
|
-
<!-- docguard:last-reviewed 2026-09-
|
|
5
|
+
<!-- docguard:last-reviewed 2026-09-19 -->
|
|
6
6
|
<!-- docguard:owner @raccioly -->
|
|
7
7
|
<!-- docguard:quality negation-load off — the data model is largely a set of prohibitions (no database, no server state, fingerprints never derived from fault text). -->
|
|
8
8
|
<!-- docguard:quality passive-voice off — this document describes what happens to documents as they flow through validation, where the artifact is the subject and the actor is irrelevant. -->
|
|
@@ -86,6 +86,11 @@ status.json ◀── derived from claims + evidence + baseline + working tree
|
|
|
86
86
|
brief.json ◀── derived from evidence + baseline + status
|
|
87
87
|
```
|
|
88
88
|
|
|
89
|
+
Status also carries `invalidFaults`, the current exact-anchor failures with
|
|
90
|
+
their observed and expected hit counts. When it is non-empty, the state is
|
|
91
|
+
`invalid-anchors` and the next action is `repair-fault`; evidence cannot make a
|
|
92
|
+
fault that no longer applies trustworthy.
|
|
93
|
+
|
|
89
94
|
A fingerprint is `sha256(claimId \n subjectId \n file \n verdict)`. It is
|
|
90
95
|
derived from identity and outcome, never from the fault's text, so repairing a
|
|
91
96
|
rotted anchor does not churn the baseline, while a change of verdict does
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
<!-- docguard:version 1.0.0 -->
|
|
4
4
|
<!-- docguard:status approved -->
|
|
5
|
-
<!-- docguard:last-reviewed 2026-09-
|
|
5
|
+
<!-- docguard:last-reviewed 2026-09-19 -->
|
|
6
6
|
<!-- docguard:owner @raccioly -->
|
|
7
7
|
<!-- docguard:quality passive-voice off — requirements are written in the standard "the system shall ..." register, where the agent is the system throughout. -->
|
|
8
8
|
|
|
@@ -26,6 +26,7 @@
|
|
|
26
26
|
| FR-12 | Answer whether one test satisfies the two-gate rule for a claim (`admit`) | P2 |
|
|
27
27
|
| FR-13 | Replay historical fix commits to measure whether the suite would have caught them, and calibrate by fault class | P3 |
|
|
28
28
|
| FR-14 | Serve the read-only operating loop over MCP so the loop survives a change of agent harness | P3 |
|
|
29
|
+
| FR-15 | Validate every exact fault anchor and the syntax of supported in-memory replacements before starting a test runner (`claims --check-anchors`), without guessing repairs | P1 |
|
|
29
30
|
|
|
30
31
|
## Non-Functional Requirements
|
|
31
32
|
|
|
@@ -99,6 +100,7 @@ fails the pull request naming the unclaimed file, and `status` reports
|
|
|
99
100
|
| FR-12 | `src/admit/admit.mjs` | `test/admit.test.mjs`, `TG-ADMIT-NEEDS-ALL-KILLED` |
|
|
100
101
|
| FR-13 | `src/replay/` | `test/replay.test.mjs`, `test/calibration.test.mjs`, `TG-REPLAY-FLAKY-IS-NEVER-CAUGHT`, `TG-CALIBRATION-COUNTS-NOCOVER-AS-A-MISS`, `TG-CALIBRATION-COUNTS-ONLY-BUGS` |
|
|
101
102
|
| FR-14 | `src/mcp/` | `test/mcp.test.mjs` |
|
|
103
|
+
| FR-15 | `src/claims/anchors.mjs`, `src/commands/claims.mjs`, `src/status/status.mjs` | `test/claims-anchors.test.mjs`, `test/status.test.mjs`, `TG-ANCHOR-PREFLIGHT-FAILS-FAST` |
|
|
102
104
|
| NFR-01 | `src/init/init.mjs`, runner resolution | `TG-INIT-HOOK-NO-NETWORK`, `TG-README-HOOK-MATCHES-THE-CODE` (every surface, not only the README), `TG-GITIGNORE-ADVICE-IS-ONE-LIST` |
|
|
103
105
|
| NFR-02 | `package.json` | `npm run test:install` in CI |
|
|
104
106
|
| NFR-03 | `src/probe/inject.mjs`, `src/probe/probe.mjs` | `TG-DIRTY-DEFENDERS-REFUSED`, `TG-IGNORED-DIRTY-RECORDED`, `TG-IN-PLACE-STILL-RAISES-A-MISSING-TARGET`, `TG-CONTROL-RESTORES-THE-SUBJECT` |
|
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
<!-- docguard:version 1.0.0 -->
|
|
4
4
|
<!-- docguard:status approved -->
|
|
5
|
-
<!-- docguard:last-reviewed 2026-09-
|
|
5
|
+
<!-- docguard:last-reviewed 2026-09-19 -->
|
|
6
6
|
<!-- docguard:owner @raccioly -->
|
|
7
7
|
<!-- docguard:quality negation-load off — the governing rule is a prohibition (a test that would pass with the behaviour broken is not a test), and the verdict rules are defined by what does not count as detection. -->
|
|
8
8
|
|
|
@@ -19,6 +19,7 @@ the test that defends it, and CI fails if any injected fault survives.
|
|
|
19
19
|
|---|---|---|---|
|
|
20
20
|
| Unit | `test/*.test.mjs` | One pure function's branches, without a runner | milliseconds |
|
|
21
21
|
| Conformance | `spec/conformance/` | Every valid example validates; every must-reject document is rejected for the defect its filename names | ~1 s |
|
|
22
|
+
| Anchor preflight | `test/claims-anchors.test.mjs` | Exact anchors still locate, supported replacements parse, the check stays below one second, and CI reaches it before self-probe | <1 s |
|
|
22
23
|
| Fixture acceptance | `test/probe.fixture.test.mjs`, `test/probe.jest.test.mjs`, `test/probe.playwright.test.mjs` | The whole pipeline reproduces a known-answer oracle, one per runner | 10–50 s each |
|
|
23
24
|
| Self-verification | `testguard.claims.json`, run by `npm run self:probe` | The tool's own invariants survive fault injection | **13.4 min** cold (measured 2026-09-18 at `cf5a083`: 96 faults, 576 defender runs, 804 s, `--serial`, idle machine); seconds when verdicts are reused, which is why CI restores the previous evidence from cache. Budgeted at 950 s in `testguard.cost-budget.json` and **gated** in CI — the figure above went 9.6 → 23.6 min across one merged change while `--cost` printed it into a log nobody read |
|
|
24
25
|
| Install smoke | `.github/scripts/install-smoke.mjs` | The **packed tarball** runs with production dependencies only | ~20 s |
|
|
@@ -57,6 +58,9 @@ What is gated instead:
|
|
|
57
58
|
4. **Every spec change ships its conformance case** — schema, semantic rule,
|
|
58
59
|
valid example and must-reject document, in the same pull request.
|
|
59
60
|
5. **A new fault shape needs a synthetic-file test and a README row.**
|
|
61
|
+
6. **Every fault must pass the cheap preflight before self-verification.** A
|
|
62
|
+
missing or ambiguous anchor, or a supported replacement that does not
|
|
63
|
+
parse, fails CI before the long self-probe starts.
|
|
60
64
|
|
|
61
65
|
## Service-to-Test Map
|
|
62
66
|
|
|
@@ -64,6 +68,7 @@ What is gated instead:
|
|
|
64
68
|
|---|---|
|
|
65
69
|
| `src/probe/classify.mjs` | `test/classify.test.mjs`, `test/negative-control.test.mjs` |
|
|
66
70
|
| `src/probe/inject.mjs` | `test/inject.test.mjs` |
|
|
71
|
+
| `src/claims/anchors.mjs` | `test/claims-anchors.test.mjs` |
|
|
67
72
|
| `src/probe/probe.mjs` | `test/probe.fixture.test.mjs`, `test/probe-preconditions.test.mjs`, `test/probe-error.test.mjs` |
|
|
68
73
|
| `src/probe/discover.mjs`, `src/probe/mocks.mjs` | `test/discover.test.mjs`, `test/mocks.test.mjs` |
|
|
69
74
|
| `src/probe/rank.mjs` | `test/rank-aliases.test.mjs`, `test/discover.test.mjs` |
|
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "testguard-cli",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.3",
|
|
4
4
|
"lockfileVersion": 3,
|
|
5
5
|
"requires": true,
|
|
6
6
|
"packages": {
|
|
7
7
|
"": {
|
|
8
8
|
"name": "testguard-cli",
|
|
9
|
-
"version": "0.9.
|
|
9
|
+
"version": "0.9.3",
|
|
10
10
|
"license": "MIT",
|
|
11
11
|
"dependencies": {
|
|
12
12
|
"ajv": "8.20.0"
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "testguard-cli",
|
|
3
|
-
"version": "0.9.
|
|
3
|
+
"version": "0.9.3",
|
|
4
4
|
"description": "Breaks your code on purpose and reports every promise your tests did not notice breaking. Mutation testing bound to the claims a project makes — vitest, jest, Playwright, pytest and unittest.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
# Include it from a project's .gitlab-ci.yml, pinned to a release:
|
|
4
4
|
#
|
|
5
5
|
# include:
|
|
6
|
-
# - remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.9.
|
|
6
|
+
# - remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.9.3/packaging/gitlab/testguard.gitlab-ci.yml'
|
|
7
7
|
# inputs:
|
|
8
8
|
# dir: backend # where testguard.claims.json lives (default ".")
|
|
9
9
|
# post_note: true # post the brief as a merge-request note (needs TESTGUARD_GITLAB_TOKEN, api scope)
|
|
@@ -29,7 +29,7 @@ spec:
|
|
|
29
29
|
inputs:
|
|
30
30
|
version:
|
|
31
31
|
description: testguard-cli version to run
|
|
32
|
-
default: "0.9.
|
|
32
|
+
default: "0.9.3"
|
|
33
33
|
dir:
|
|
34
34
|
description: directory whose test runner is invoked (where testguard.claims.json lives)
|
|
35
35
|
default: "."
|
|
@@ -15,8 +15,8 @@
|
|
|
15
15
|
class Testguard < Formula
|
|
16
16
|
desc "Proves a test suite defends the claims a project makes"
|
|
17
17
|
homepage "https://github.com/raccioly/testguard"
|
|
18
|
-
url "https://registry.npmjs.org/testguard-cli/-/testguard-cli-0.9.
|
|
19
|
-
sha256 "
|
|
18
|
+
url "https://registry.npmjs.org/testguard-cli/-/testguard-cli-0.9.3.tgz"
|
|
19
|
+
sha256 "33e88eaa37c34744c08c581a64b4939472e22e898ce1dabf15507fd77229d4e0"
|
|
20
20
|
license "MIT"
|
|
21
21
|
|
|
22
22
|
depends_on "node"
|
|
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "testguard-cli"
|
|
7
|
-
version = "0.9.
|
|
7
|
+
version = "0.9.3"
|
|
8
8
|
description = "Breaks your code on purpose and reports every promise your tests did not notice breaking. Mutation testing bound to the claims a project makes — vitest, jest, Playwright, pytest and unittest. Ships the Node.js CLI (requires Node.js 20+)."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
license = { text = "MIT" }
|
|
@@ -395,6 +395,26 @@ changed); `1` at least one uncovered file (or strict mode over an
|
|
|
395
395
|
all-excluded change); `2` the change cannot be evaluated (unresolvable
|
|
396
396
|
reference, invalid claims or ignore file, no repository); `3` no reference.
|
|
397
397
|
|
|
398
|
+
## Anchor preflight
|
|
399
|
+
|
|
400
|
+
`claims --check-anchors` answers what is knowable before a runner starts. For
|
|
401
|
+
every fault-injection fault it reads the target, applies the same exact
|
|
402
|
+
`locate()` arithmetic as a probe, and reports `ok`, `anchor-missing` or
|
|
403
|
+
`anchor-ambiguous` with the observed and expected hit counts. When a language
|
|
404
|
+
has a cheap parser, it also parses the in-memory replacement (`node --check`
|
|
405
|
+
semantics for JavaScript modules, `compile()` for Python). Unsupported
|
|
406
|
+
languages are anchor-checked and otherwise skipped without inference.
|
|
407
|
+
|
|
408
|
+
One non-`ok` anchor or non-parsing supported replacement exits `1`. The check
|
|
409
|
+
runs no tests, creates no worktree, reads no git state and writes no source.
|
|
410
|
+
It is not evidence that a defender detects the fault; the self-probe still
|
|
411
|
+
runs after it. There is no automatic repair or fuzzy relocation: choosing a
|
|
412
|
+
new anchor is a semantic edit to the claim and must be reviewed and re-probed.
|
|
413
|
+
|
|
414
|
+
`invalid-anchors` precedes every evidence state in the status document (after
|
|
415
|
+
`unclaimed-changes`, when a change reference is known). Its next action is
|
|
416
|
+
`repair-fault`, naming the first fault and `testguard claims --check-anchors`.
|
|
417
|
+
|
|
398
418
|
## Severity floor
|
|
399
419
|
|
|
400
420
|
`--severity <level>` gates only findings whose claim severity is at or above
|