testguard-cli 0.20.0__tar.gz → 0.21.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.agents/skills/testguard/SKILL.md +1 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/benchmark-node-direct.mjs +5 -3
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/benchmark-node-runner.mjs +5 -3
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/install-smoke.mjs +1 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.pre-commit-hooks.yaml +1 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/CHANGELOG.md +74 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/PKG-INFO +3 -3
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/README.md +2 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/action.yml +1 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/guides/ci/github-actions.md +6 -6
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/guides/ci/gitlab.md +1 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/guides/ci/pre-commit.md +1 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/guides/monorepo.md +2 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/guides/performance.md +22 -4
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/i18n/es/README.md +2 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/i18n/pt-BR/README.md +2 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/i18n/zh-CN/README.md +2 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/installation.md +5 -5
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/reference/artifacts.md +21 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/reference/cli.md +4 -6
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/reference/configuration.md +1 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/reference/verdicts.md +25 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/testguard-explained.html +1 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/testguard-explained.pdf +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/package-lock.json +80 -80
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/package-lock.json +135 -135
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/package.json +1 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/packaging/gitlab/testguard.gitlab-ci.yml +2 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/packaging/homebrew/testguard.rb +2 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/pyproject.toml +1 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/GATE-SEMANTICS.md +136 -9
- testguard_cli-0.21.0/spec/conformance/dependency-closure.test.mjs +41 -0
- testguard_cli-0.21.0/spec/conformance/examples/evidence-dependency-closure.json +318 -0
- testguard_cli-0.21.0/spec/conformance/examples/evidence-kill-kinds.json +418 -0
- testguard_cli-0.21.0/spec/conformance/invalid/evidence.assertion-policy-accuses-an-assertion-kill.json +422 -0
- testguard_cli-0.21.0/spec/conformance/invalid/evidence.assertion-policy-hides-exception-only-kill.json +413 -0
- testguard_cli-0.21.0/spec/conformance/invalid/evidence.assertion-policy-state-contradicts-list.json +418 -0
- testguard_cli-0.21.0/spec/conformance/invalid/evidence.dependency-hashes-without-closure-state.json +317 -0
- testguard_cli-0.21.0/spec/conformance/invalid/evidence.indeterminate-dependency-closure-without-reason.json +312 -0
- testguard_cli-0.21.0/spec/conformance/invalid/evidence.instrument-names-another-tool-version.json +318 -0
- testguard_cli-0.21.0/spec/conformance/invalid/evidence.killkind-contradicts-probe-runs.json +413 -0
- testguard_cli-0.21.0/spec/conformance/invalid/evidence.killkind-on-a-survivor.json +564 -0
- testguard_cli-0.21.0/spec/conformance/invalid/evidence.reuse-tally-disagrees-with-records.json +318 -0
- testguard_cli-0.21.0/spec/conformance/invalid/evidence.reused-with-indeterminate-dependency-closure.json +319 -0
- testguard_cli-0.21.0/spec/conformance/kill-kind.test.mjs +99 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/lib/validate.mjs +61 -4
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/brief.schema.json +1 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/common.schema.json +17 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/evidence.schema.json +80 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/status.schema.json +10 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/brief/brief.mjs +6 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/cli.mjs +6 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/admit.mjs +14 -5
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/probe.mjs +11 -3
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/evidence/writer.mjs +21 -3
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/init/templates/SKILL.md +1 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/mcp/tools.mjs +1 -1
- testguard_cli-0.21.0/src/probe/assertion-policy.mjs +32 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/attribution.mjs +23 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/classify.mjs +4 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/cost.mjs +1 -1
- testguard_cli-0.21.0/src/probe/dependencies.mjs +153 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/imports.mjs +200 -10
- testguard_cli-0.21.0/src/probe/kill-kind.mjs +45 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/policy-freshness.mjs +6 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/probe.mjs +94 -37
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/pyimports.mjs +84 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/rank.mjs +11 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/index.mjs +2 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/node/report.mjs +4 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/node-test.mjs +7 -3
- testguard_cli-0.21.0/src/probe/runners/playwright/kinds-reporter.cjs +42 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/playwright.mjs +36 -11
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/python/_testguard_pytest_plugin.py +15 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/python/_testguard_report.py +21 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/python/_testguard_unittest_main.py +5 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/python.mjs +7 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/shared.mjs +62 -10
- testguard_cli-0.21.0/src/probe/runners/vitest/kinds-reporter.mjs +56 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/vitest.mjs +24 -3
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/render.mjs +6 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/replay/replay.mjs +2 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/status/status.mjs +25 -3
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/admit.test.mjs +1 -1
- testguard_cli-0.21.0/test/assertion-policy.cli.test.mjs +144 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/attribution.test.mjs +32 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/classify.test.mjs +1 -1
- testguard_cli-0.21.0/test/dependency-closure.test.mjs +174 -0
- testguard_cli-0.21.0/test/evidence-atomic-write.test.mjs +51 -0
- testguard_cli-0.21.0/test/helpers/reuse-project.mjs +55 -0
- testguard_cli-0.21.0/test/kill-kind.acceptance.test.mjs +168 -0
- testguard_cli-0.21.0/test/kill-kind.test.mjs +97 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/negative-control.test.mjs +1 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/policy-freshness.test.mjs +24 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe.fixture.test.mjs +21 -4
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe.node.test.mjs +1 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/resource-timing.test.mjs +4 -2
- testguard_cli-0.21.0/test/reuse-instrument.test.mjs +122 -0
- testguard_cli-0.21.0/test/reuse-live-baseline.test.mjs +48 -0
- testguard_cli-0.21.0/test/reuse-soundness.test.mjs +89 -0
- testguard_cli-0.21.0/test/runner-failure-kind.test.mjs +204 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-node-report.test.mjs +23 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-playwright.test.mjs +29 -5
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-python.test.mjs +14 -2
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-report.test.mjs +11 -6
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-vitest.test.mjs +11 -6
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/samples/python-report.json +2 -2
- testguard_cli-0.21.0/test/status-dependencies.test.mjs +50 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/status.test.mjs +2 -1
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/testguard.claims.json +601 -16
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/testguard.ignore.json +14 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.docguard.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.docguardignore +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.gitattributes +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/ISSUE_TEMPLATE/bug_report.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/ISSUE_TEMPLATE/config.yml +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/ISSUE_TEMPLATE/documentation.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/ISSUE_TEMPLATE/feature_request.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/PULL_REQUEST_TEMPLATE.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/dependabot.yml +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/approve-held-runs.sh +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/build-one-pager.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/check-cost-budget.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/check-node-fixture.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/check-python-fixture.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/jules-policy.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/jules-triage.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/publish-homebrew-tap.sh +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/scripts/sync-release-version.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/workflows/auto-merge.yml +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/workflows/ci.yml +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/workflows/jules-triage.yml +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/workflows/release.yml +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/workflows/scheduled-release.yml +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.github/workflows/supply-chain.yml +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.gitignore +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.jules/README.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.jules/RULES.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.jules/preflight.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.jules/routines/docs-drift.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.jules/routines/runner-scout.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.jules/routines/scaffold-hunt.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.jules/setup.sh +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.npmignore +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.npmrc +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/.pre-commit-config.yaml +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/AGENTS.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/CLAUDE.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/CODE_OF_CONDUCT.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/CONTRIBUTING.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/DRIFT-LOG.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/LICENSE +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/PRIVACY.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/SECURITY.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/SUPPORT.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/bench/README.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/cli/testguard.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/README.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/concepts/how-it-works.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/faq.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/glossary.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/guides/ai-agents.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/guides/existing-projects.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/guides/new-projects.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/guides/python.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/guides/replay.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/guides/writing-claims.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/i18n/es/quickstart.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/i18n/pt-BR/quickstart.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/i18n/zh-CN/quickstart.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/quickstart.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/reference/languages-and-runners.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/reference/mcp.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/troubleshooting.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs/upgrade.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs-canonical/ARCHITECTURE.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs-canonical/DATA-MODEL.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs-canonical/ENVIRONMENT.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs-canonical/PRIOR-ART.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs-canonical/REQUIREMENTS.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs-canonical/SECURITY.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/docs-canonical/TEST-SPEC.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer/.gitignore +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer/README.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer/expected.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer/package.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer/src/export.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer/src/redact.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer/test/export-mocked.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer/test/flaky.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer/test/redact.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer/testguard.claims.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer/vitest.config.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-jest/.gitignore +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-jest/README.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-jest/expected.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-jest/jest.config.js +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-jest/package.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-jest/src/export.js +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-jest/src/redact.js +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-jest/test/export-mocked.test.js +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-jest/test/flaky.test.js +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-jest/test/redact.test.js +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-jest/testguard.claims.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-node/.gitignore +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-node/README.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-node/expected.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-node/package.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-node/src/policy.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-node/src/unused.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-node/test/flaky.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-node/test/mixed.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-node/test/policy.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-node/testguard.claims.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/.gitignore +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/README.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/e2e/flaky.spec.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/e2e/toggle.spec.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/expected.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/package.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/playwright.config.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/src/toggle.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/test/toggle.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/testguard.claims.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-playwright/vitest.config.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/.gitignore +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/README.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/demo/__init__.py +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/demo/export.py +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/demo/orphan.py +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/demo/pipeline.py +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/demo/redact.py +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/expected.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/testguard.claims.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/tests/test_export_mocked.py +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/tests/test_flaky.py +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/tests/test_patched.py +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/fixtures/known-answer-python/tests/test_redact.py +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/README.md +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/annotations.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/authoring-input.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/draft-append.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/annotations.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/authoring-input.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/baseline.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/brief.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/calibration-websec.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/calibration.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/claims-methods.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/claims.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/evidence-disturbed-run.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/evidence-negative-control.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/evidence-persistence-signal.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/evidence-provisional.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/evidence-python.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/evidence-resources.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/evidence-scan-method.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/evidence.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/gate.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/ignore.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/replay.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/status-baseline.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/status-resource-cost.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/examples/status.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/baseline.bad-fingerprint-key.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/baseline.zero-count.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/brief.text-without-heading.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/brief.unknown-verdict-key.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/calibration.backoff-cell-not-wilson.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/calibration.breakdown-does-not-sum-to-n.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/calibration.ci-not-wilson.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/calibration.compound-key-arity-mismatch.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/calibration.fallback-without-basis.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/calibration.p-not-positives-over-n.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/calibration.p-outside-ci.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/calibration.positives-exceed-n.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/claims.absolute-path.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/claims.duplicate-fault-id.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/claims.fault-injection-without-a-substitution.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/claims.method-detail-on-a-specified-method.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/claims.missing-provenance.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/claims.noop-fault.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/claims.occurrence-exceeds-hits.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/claims.parent-traversal.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.annotated-signal-without-reason.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.attribute-signal-not-a-defender.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.attribute-signal-without-attributes.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.byrunner-file-not-resolved.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.contention-without-runners.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.discovered-defender-mocks-subject.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.fault-injection-without-its-evidence.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.flaky-defender-without-flakiness.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.ignored-dirty-on-snapshot.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.killed-by-non-assertion.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.killed-on-red-baseline.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.killed-without-n-runs.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.method-detail-on-an-injecting-run.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.negative-control-not-reached-but-survived.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.negative-control-reached-on-a-kill.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.persistence-signal-not-a-defender.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.persistence-signal-on-a-non-survivor.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.persistence-signal-without-reason.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.probe-error-without-message.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.provisional-flag-on-confirmed-run.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.rank-extra-property.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.resource-accounting.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.restore-skipped-in-place.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.serial-worker-conflict.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.single-run-without-provisional.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.target-not-imported-but-killed.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.undeclared-killers-without-reason.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.unknown-verdict.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.unverifiable-without-reason.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/evidence.wrong-fingerprint.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/gate.counts-disagree.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/gate.exit-zero-with-uncovered.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/gate.ignore-cover-without-pattern.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/ignore.claim-pattern-not-an-identifier.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/ignore.fault-pattern-without-fault-id.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/ignore.fingerprint-kind.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/ignore.missing-reason.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/ignore.short-reason.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/replay.blind-with-a-failing-run.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/replay.duplicate-patch-id.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/replay.nocover-when-no-test-file-remained.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/status.baseline-and-run.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/status.baseline-frozen-and-restamped.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/status.claim-without-file.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/status.clean-with-new-findings.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/status.invalid-anchor-hidden.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/status.partial-scope-counts-disagree.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/status.resource-accounting.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/status.surface-counts-disagree.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/status.unclaimed-hidden-behind-unproven.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/invalid/status.write-test-without-target.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/origin-policy.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/origin-summaries.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/origins.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/conformance/schemas.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/lib/authoring-input.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/lib/fingerprint.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/lib/origins.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/lib/wilson.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/annotations.schema.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/authoring-input.schema.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/baseline.schema.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/calibration.schema.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/claims.schema.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/concerns.schema.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/gate.schema.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/ignore.schema.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/replay.schema.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/spec/schemas/sweep.schema.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/admit/admit.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/baseline/baseline.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/claims/anchors.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/claims/annotation-placement.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/claims/annotations.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/claims/load.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/claims/removed.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/command-budget.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/annotations.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/baseline.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/brief.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/claims.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/concerns.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/gate.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/init.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/mcp.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/replay.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/runner-options.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/scaffold.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/status.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/commands/sweep.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/gate/changed.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/git.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/init/init.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/mcp/server.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/contention.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/discover.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/independence.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/inject.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/mocks.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/origin-policy.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/progress.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/discovery.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/jest.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/lifecycle.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/node/direct.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/node/entry.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/runners/node/runner.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/universe.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/probe/worktree.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/replay/label.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/admission.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/append.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/apply-append.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/fix-input.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/fix-paths.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/intent.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/open-append.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/plan-append.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/plan-intent-input.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/prepare-append.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/producers.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/producers.python.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/request.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/scaffold/scaffold.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/status/surface.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/supply/concerns.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/supply/feedback.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/supply/persistence.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/supply/savepath.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/supply/select.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/sweep/pool.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/sweep/sweep.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/util/glob.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/src/util/hash.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/action.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/admit-decision.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/anchor-admission.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/annotation-advisory-limits.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/annotation-advisory.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/annotation-apply.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/annotation-cli.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/annotation-id-bounds.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/annotation-placement.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/annotations.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/baseline.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/brief.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/calibration.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/ci-evidence-helper.github.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/ci-evidence-helper.gitlab.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/claim-options.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/claim-selection.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/claims-anchors.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/claims-since.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/cli-surface.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/cli-unexpected-error.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/command-budget-write-boundaries.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/command-budget.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/contention.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/cost.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/discover.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/discovery-config.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/docs.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/draft-admission.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/draft-append-apply.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/draft-append-cli.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/draft-append-open.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/draft-append-plan.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/draft-append-prepare.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/draft-append.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/evidence-persistence-signal.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/fault-defender-orchestration.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/fault-defenders.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/fix-input.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/fix-inventory.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/fix-nested.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/fix-paths.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/fix-projects.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/fixture-git.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/gate-coverage.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/gate-ref.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/gate.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/gitlab-template.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/glob.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/helpers/gate-project.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/helpers/git.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/helpers/probe-assertions.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/helpers/release-project.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/helpers/yaml.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/imports.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/independence.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/init-hook-replace.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/init-ignores.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/init-preflight.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/init.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/inject.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/intent-authoring.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/intent-document.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/intent-input-cli.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/intent-input-plan.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/intent-input-request.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/intent-project.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/jules-policy.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/label.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/mcp-next.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/mcp-protocol.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/mcp.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/mocks.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/native-universe-hash.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/native-universe.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/negative-control-capabilities.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/one-pager.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/origin-policy-admission.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/origin-policy-cli.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/origin-policy-projections.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/origin-policy-request.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/origin-policy.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/origin-reporting.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/origin-summary.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe-control.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe-dirty-admission.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe-dirty-paths.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe-error-boundary.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe-error.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe-isolation.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe-native-manifest.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe-persistence-signal.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe-preconditions.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe-records.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe-runner-identity.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe.jest.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe.playwright.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/probe.python.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/progress.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/pyimports.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/python-interpreter.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/python-option-commands.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/python-wrapper.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/rank-aliases.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/release-approval.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/release-sync.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/release-tap.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/release-version.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/replay-mixed-runners.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/replay-no-test-file-left.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/replay-python-defenders.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/replay-selection.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/replay-verdict.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/replay.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/resource-safety.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-argv.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-command.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-discovery.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-node-admission.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-node-controller.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-node-direct.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-options.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-output-bounds.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-project-resolution.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-report-allocation.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-selection-blind.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/runner-worker-argv.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/samples/playwright-report.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/samples/python-report-collect-error.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/scaffold-admitted-source.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/scaffold-placeholder.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/scaffold.python.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/scaffold.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/snapshot.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/supply-concerns.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/supply-feedback.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/supply-persistence.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/supply-savepath.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/supply-select.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/surface.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/sweep-pool.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/sweep.fixture.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/sweep.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/vitest-config.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/worker-policy.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/worktree.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/test/writer.test.mjs +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/testguard.cost-budget.json +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/testguard_cli/__init__.py +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/testguard_cli/wrapper.py +0 -0
- {testguard_cli-0.20.0 → testguard_cli-0.21.0}/vitest.config.mjs +0 -0
|
@@ -132,6 +132,7 @@ code by construction. So, for every source file you create or change:
|
|
|
132
132
|
| `FAULT-INVALID` | the replacement does not compile | fix the fault definition |
|
|
133
133
|
| `TIMEOUT` | the fault makes the code hang | write an assertion that fails on it; a hang is not a detection |
|
|
134
134
|
| `FLAKY-DEFENDER` | defenders not green 3/3, or disagreed across runs | fix the flake first; no verdict is trustworthy until then |
|
|
135
|
+
| `killed` with `[killed by exception only: no assertion failed]`, or "N kills are by crash, not by assertion" | the fault made the code throw and a test went red, but nothing checked a value | add an assertion on the behavior the claim states; a test that merely calls the code is reach, not proof. `probe --require-assertion-kill` fails on these |
|
|
135
136
|
|
|
136
137
|
## The fix loop
|
|
137
138
|
|
|
@@ -24,6 +24,8 @@ mkdirSync(join(out, 'workers'), { mode: 0o700 }); // Refuse to overwrite a previ
|
|
|
24
24
|
const tool = mkdtempSync(join(tmpdir(), 'tg-direct-reference-'));
|
|
25
25
|
const dir = mkdtempSync(join(tmpdir(), 'tg-direct-bench-'));
|
|
26
26
|
const git = (...args) => execFileSync('git', args, { cwd: dir, encoding: 'utf8', timeout: 10000 }).trim();
|
|
27
|
+
// The base commit predates `exceptionFailures`; a kill there was counted wholly in `assertionFailures`. Compare the sum.
|
|
28
|
+
const comparable = ({ durationMs, exceptionFailures, kindNotReported, ...run }) => ({ ...run, assertionFailures: run.assertionFailures + (exceptionFailures ?? 0) });
|
|
27
29
|
const projection = (record) => ({
|
|
28
30
|
id: record.claim.id + '/' + record.subject.id,
|
|
29
31
|
subject: record.subject,
|
|
@@ -33,8 +35,8 @@ const projection = (record) => ({
|
|
|
33
35
|
targetHash: record.inputs?.targetHash,
|
|
34
36
|
defenderHashes: record.inputs?.defenderHashes,
|
|
35
37
|
discoveryHashes: record.inputs?.discoveryHashes,
|
|
36
|
-
baseline: (record.detail.baselineRuns ?? []).map(
|
|
37
|
-
probe: (record.detail.probeRuns ?? []).map(
|
|
38
|
+
baseline: (record.detail.baselineRuns ?? []).map(comparable),
|
|
39
|
+
probe: (record.detail.probeRuns ?? []).map(comparable),
|
|
38
40
|
negativeControl: record.detail.negativeControl ?? null,
|
|
39
41
|
});
|
|
40
42
|
// testUniverseHash intentionally changes: it binds the changed adapter itself.
|
|
@@ -76,7 +78,7 @@ try {
|
|
|
76
78
|
assert.equal(record.detail.probeRuns.length, 3);
|
|
77
79
|
assert.equal(record.detail.baselineRuns.length, 3);
|
|
78
80
|
assert(record.detail.baselineRuns.every((run) => run.outcome === 'pass'));
|
|
79
|
-
assert(record.detail.probeRuns.every((run) => run.outcome === 'fail' && run.assertionFailures > 0));
|
|
81
|
+
assert(record.detail.probeRuns.every((run) => run.outcome === 'fail' && run.assertionFailures + (run.exceptionFailures ?? 0) > 0));
|
|
80
82
|
}
|
|
81
83
|
}
|
|
82
84
|
const projected = evidence.records.map(projection);
|
|
@@ -59,10 +59,12 @@ const corpus = rows.map(([name, changes, expected]) => [name, { ...healthy, ...c
|
|
|
59
59
|
const sha256 = (text) => createHash('sha256').update(text).digest('hex');
|
|
60
60
|
const toolSources = Object.fromEntries(['node-test.mjs', 'node/runner.mjs', 'node/entry.mjs', 'node/report.mjs', 'shared.mjs', 'discovery.mjs', 'lifecycle.mjs', 'vitest.mjs'].map((file) => [file, sha256(readFileSync(join(ROOT, 'src/probe/runners', file)))]));
|
|
61
61
|
const vitestVersion = JSON.parse(readFileSync(join(ROOT, 'node_modules/vitest/package.json'), 'utf8')).version;
|
|
62
|
+
// The base commit predates `exceptionFailures`; a kill there was counted wholly in `assertionFailures`. Compare the sum.
|
|
63
|
+
const comparable = ({ durationMs, exceptionFailures, kindNotReported, ...run }) => ({ ...run, assertionFailures: run.assertionFailures + (exceptionFailures ?? 0) });
|
|
62
64
|
const proofProjection = (evidence) => evidence.records.map((r) => ({
|
|
63
65
|
id: `${r.claim.id}/${r.subject.id}`, verdict: r.verdict, reason: r.detail.reason ?? null,
|
|
64
|
-
baseline: r.detail.baselineRuns.map(
|
|
65
|
-
probe: r.detail.probeRuns.map(
|
|
66
|
+
baseline: r.detail.baselineRuns.map(comparable),
|
|
67
|
+
probe: r.detail.probeRuns.map(comparable),
|
|
66
68
|
}));
|
|
67
69
|
const results = [];
|
|
68
70
|
async function measure(engine, label) {
|
|
@@ -96,7 +98,7 @@ for(const [name,input,expected] of ${JSON.stringify(corpus)}) it(name,()=>{try{$
|
|
|
96
98
|
assert.equal(r.verdict, 'killed', `${r.claim.id} must be caught by both frameworks`);
|
|
97
99
|
assert.equal(r.detail.baselineRuns.length, 3); assert.equal(r.detail.probeRuns.length, 3);
|
|
98
100
|
for (const run of r.detail.baselineRuns) { assert.equal(run.outcome, 'pass'); assert.equal(run.tests.total, corpus.length); }
|
|
99
|
-
for (const run of r.detail.probeRuns) { assert.equal(run.outcome, 'fail'); assert(run.assertionFailures > 0); assert.equal(run.tests.total, corpus.length); }
|
|
101
|
+
for (const run of r.detail.probeRuns) { assert.equal(run.outcome, 'fail'); assert(run.assertionFailures + (run.exceptionFailures ?? 0) > 0); assert.equal(run.tests.total, corpus.length); }
|
|
100
102
|
}
|
|
101
103
|
const workers = readFileSync(receipts, 'utf8').trim().split('\n').map(JSON.parse).filter((r) => !r.collect);
|
|
102
104
|
assert.equal(workers.length, 15, 'three shared baseline runs plus four faults times three confirmations');
|
|
@@ -63,7 +63,7 @@ try {
|
|
|
63
63
|
{ status: 'failed', failureMessages: ['AssertionError: expected /specify timed out/'] },
|
|
64
64
|
{ status: 'failed', failureMessages: ['Error: Test timed out in 1000ms.'] },
|
|
65
65
|
] }] }, 1);
|
|
66
|
-
if (result.timeouts !== 1 || result.run.assertionFailures !== 1) throw new Error('installed runner confuses assertion text with timeout identity');`,
|
|
66
|
+
if (result.timeouts !== 1 || result.run.assertionFailures + (result.run.exceptionFailures ?? 0) !== 1) throw new Error('installed runner confuses assertion text with timeout identity');`,
|
|
67
67
|
installedRunner,
|
|
68
68
|
], consumer);
|
|
69
69
|
const claims = run(bin, ['claims', '.'], consumer);
|
|
@@ -7,6 +7,80 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
## [Unreleased]
|
|
9
9
|
|
|
10
|
+
## [0.21.0] - 2026-11-10
|
|
11
|
+
|
|
12
|
+
Automated weekly release — everything merged since `v0.20.1`.
|
|
13
|
+
|
|
14
|
+
### Changed
|
|
15
|
+
|
|
16
|
+
- feat(probe): tell a kill by assertion from a kill by crash; --require-assertion-kill (#233) (#241)
|
|
17
|
+
- fix(probe): bind verdict reuse to the import closure, a live baseline and the instrument (#232) (#240)
|
|
18
|
+
- chore(deps-dev): bump vitest from 5.0.2 to 5.0.5 in the all-npm group (#229)
|
|
19
|
+
- chore(deps): bump fast-uri from 3.1.8 to 3.1.9 (#228)
|
|
20
|
+
- [jules:docs-drift] brief — remove unsupported flags from docs (#227)
|
|
21
|
+
- chore(deps-dev): bump vite from 8.3.1 to 8.3.4 (#224)
|
|
22
|
+
- chore(deps-dev): bump vite from 8.3.0 to 8.3.4 in /fixtures/known-answer-playwright (#223)
|
|
23
|
+
- [jules:docs-drift] probe — remove removed --changed flag (#220)
|
|
24
|
+
- chore(homebrew): sha256 for v0.20.1 (#218)
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
### Added
|
|
28
|
+
|
|
29
|
+
- `killKind` on every kill: each failed test is classified `assertion` or `exception` from structured error identity (never message wording), `testRun.exceptionFailures` sits beside `assertionFailures`, and a `killed` record carries `detail.killKind` (`assertion`, `exception-only`, `mixed`). The default verdict is unchanged: an exception the fault provokes still kills.
|
|
30
|
+
- `probe --require-assertion-kill` (and `admit`): an exception-only kill fails the policy, exit 1, verdict still `killed`. `status` and `brief` count exception-only kills as a named weakness.
|
|
31
|
+
|
|
32
|
+
### Changed
|
|
33
|
+
|
|
34
|
+
- `assertionFailures` is now assertion-only; a kill is `assertionFailures + exceptionFailures > 0`. Evidence written earlier has no `exceptionFailures` or `killKind` and reads as before; under the policy a kill with no recorded kind fails (unknown rounds toward unproven).
|
|
35
|
+
- Under `--require-assertion-kill`, an exception-only kill whose runner reported no error identity (a custom `--runner-cmd` without `failureDetails`) says so per kill: `reason: "kind-not-reported-by-runner"` (`testRun.kindNotReported`, `detail.killKindReason`).
|
|
36
|
+
- vitest and Playwright runs add an injected reporter beside the JSON one (a sidecar file in the temp dir) to read error identity; a custom `--runner-cmd` without `failureDetails` classifies every failure as an exception.
|
|
37
|
+
|
|
38
|
+
### Fixed
|
|
39
|
+
|
|
40
|
+
- **Verdict reuse no longer carries a kill over a red suite (#232).** Reuse
|
|
41
|
+
compared only the target and defender files, so breaking a helper the target
|
|
42
|
+
imports reused the old `killed`. Evidence now records
|
|
43
|
+
`inputs.dependencyHashes` (the transitive project import closure of the
|
|
44
|
+
target and every defender, plus root lockfiles, manifests and runner config;
|
|
45
|
+
Python by module name, including ancestor `conftest.py`) and
|
|
46
|
+
`inputs.dependencyClosure`. Reuse requires equal hashes and a `complete`
|
|
47
|
+
closure; a computed `import()`/`require()`/`import_module()`, an unresolvable
|
|
48
|
+
relative import or an over-limit graph makes it `indeterminate` (reasons in
|
|
49
|
+
`inputs.dependencyIssues`) and that fault is probed every time. Before any
|
|
50
|
+
record is carried forward its defender set runs once unmodified, shared with
|
|
51
|
+
the baseline, so a red defender is probed in full; `run.measurements` gains
|
|
52
|
+
`reuseBaselineRuns`. `status` staleness and the origin-policy freshness check
|
|
53
|
+
apply the same closure and predicate. Evidence written before this change is
|
|
54
|
+
re-probed once, and until then `status` reports each of its records as
|
|
55
|
+
stale ("predates dependency hashing"). An indeterminate closure makes `--require-origin` policy
|
|
56
|
+
`unavailable`.
|
|
57
|
+
- **Reuse names its instrument, and says how much it reused (#232).** Evidence
|
|
58
|
+
records `run.instrument` (TestGuard version plus a digest of resolver alias
|
|
59
|
+
sources and root runner config/lockfiles); evidence from another version or
|
|
60
|
+
digest is never reused. `run.reuse: { reused, measured }` is in the evidence
|
|
61
|
+
and in `probe --json`. `GATE-SEMANTICS.md` now states that a gate over
|
|
62
|
+
authors who can write the evidence file must run `--no-reuse`.
|
|
63
|
+
- A `timeout` or `flaky-defender` record is never reused: like `probe-error`
|
|
64
|
+
it describes the run's environment, and a green live baseline today does not
|
|
65
|
+
make an old timeout the current answer. `status` reports evidence from
|
|
66
|
+
another TestGuard version or configuration digest as stale, in one line.
|
|
67
|
+
- Spec documents are written atomically (exclusive temporary file, then
|
|
68
|
+
rename): a run killed mid-write no longer leaves a truncated evidence file
|
|
69
|
+
the next probe reads as prior evidence.
|
|
70
|
+
- The module lexer behind defender discovery now reads regex literals,
|
|
71
|
+
numeric literals and template expressions containing quotes or nested
|
|
72
|
+
templates; before, a module with `/'/` in it was unreadable and discovery
|
|
73
|
+
through it was indeterminate.
|
|
74
|
+
|
|
75
|
+
## [0.20.1] - 2026-10-12
|
|
76
|
+
|
|
77
|
+
Automated weekly release — everything merged since `v0.20.0`.
|
|
78
|
+
|
|
79
|
+
### Changed
|
|
80
|
+
|
|
81
|
+
- chore(homebrew): sha256 for v0.20.0 (#216)
|
|
82
|
+
|
|
83
|
+
|
|
10
84
|
## [0.20.0] - 2026-10-11
|
|
11
85
|
|
|
12
86
|
Automated weekly release — everything merged since `v0.19.0`.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: testguard-cli
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.21.0
|
|
4
4
|
Summary: Breaks your code on purpose and reports every promise your tests did not notice breaking. Mutation testing bound to the claims a project makes — vitest, jest, Playwright, pytest and unittest. A wrapper that runs the Node.js CLI (requires Node.js 20+): a project-local node_modules/testguard-cli when present, otherwise the npm release of this same version through npx.
|
|
5
5
|
Project-URL: Homepage, https://github.com/raccioly/testguard
|
|
6
6
|
Project-URL: Documentation, https://github.com/raccioly/testguard#readme
|
|
@@ -153,9 +153,9 @@ raises coverage. TestGuard admits it because it fails when the claim is false.
|
|
|
153
153
|
| npm | `npm i -D testguard-cli` then `npx testguard probe` |
|
|
154
154
|
| pip | `pip install testguard-cli` then `testguard probe` (needs Node ≥ 20) |
|
|
155
155
|
| Homebrew | `brew tap raccioly/tap && brew install testguard` |
|
|
156
|
-
| GitHub Action | `uses: raccioly/testguard@v0.
|
|
156
|
+
| GitHub Action | `uses: raccioly/testguard@v0.21.0` |
|
|
157
157
|
| pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-gate`, `testguard-probe` |
|
|
158
|
-
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.
|
|
158
|
+
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.21.0/packaging/gitlab/testguard.gitlab-ci.yml` |
|
|
159
159
|
|
|
160
160
|
Requirements, offline installs, `ENOVERSIONS` and every CI option are in the
|
|
161
161
|
[installation guide](https://github.com/raccioly/testguard/blob/main/docs/installation.md).
|
|
@@ -129,9 +129,9 @@ raises coverage. TestGuard admits it because it fails when the claim is false.
|
|
|
129
129
|
| npm | `npm i -D testguard-cli` then `npx testguard probe` |
|
|
130
130
|
| pip | `pip install testguard-cli` then `testguard probe` (needs Node ≥ 20) |
|
|
131
131
|
| Homebrew | `brew tap raccioly/tap && brew install testguard` |
|
|
132
|
-
| GitHub Action | `uses: raccioly/testguard@v0.
|
|
132
|
+
| GitHub Action | `uses: raccioly/testguard@v0.21.0` |
|
|
133
133
|
| pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-gate`, `testguard-probe` |
|
|
134
|
-
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.
|
|
134
|
+
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.21.0/packaging/gitlab/testguard.gitlab-ci.yml` |
|
|
135
135
|
|
|
136
136
|
Requirements, offline installs, `ENOVERSIONS` and every CI option are in the
|
|
137
137
|
[installation guide](https://github.com/raccioly/testguard/blob/main/docs/installation.md).
|
|
@@ -100,7 +100,7 @@ jobs:
|
|
|
100
100
|
- uses: actions/checkout@v7
|
|
101
101
|
with:
|
|
102
102
|
fetch-depth: 0 # the base branch must exist locally
|
|
103
|
-
- uses: raccioly/testguard@v0.
|
|
103
|
+
- uses: raccioly/testguard@v0.21.0
|
|
104
104
|
with:
|
|
105
105
|
command: gate # every changed source file needs a claim
|
|
106
106
|
|
|
@@ -114,7 +114,7 @@ jobs:
|
|
|
114
114
|
with:
|
|
115
115
|
node-version: 22
|
|
116
116
|
- run: npm ci # the probe runs the project's own test runner
|
|
117
|
-
- uses: raccioly/testguard@v0.
|
|
117
|
+
- uses: raccioly/testguard@v0.21.0
|
|
118
118
|
with:
|
|
119
119
|
command: probe
|
|
120
120
|
severity: high # fail only on new high or critical findings
|
|
@@ -162,7 +162,7 @@ jobs:
|
|
|
162
162
|
testguard-evidence-
|
|
163
163
|
|
|
164
164
|
- id: testguard
|
|
165
|
-
uses: raccioly/testguard@v0.
|
|
165
|
+
uses: raccioly/testguard@v0.21.0
|
|
166
166
|
with:
|
|
167
167
|
command: probe
|
|
168
168
|
|
|
@@ -211,7 +211,7 @@ the probe at the interpreter if it is not one TestGuard finds on its own
|
|
|
211
211
|
with:
|
|
212
212
|
python-version: '3.12'
|
|
213
213
|
- run: python -m venv .venv && .venv/bin/pip install -r requirements.txt
|
|
214
|
-
- uses: raccioly/testguard@v0.
|
|
214
|
+
- uses: raccioly/testguard@v0.21.0
|
|
215
215
|
with:
|
|
216
216
|
command: probe
|
|
217
217
|
runner: python # pytest if .venv can import it, else unittest
|
|
@@ -227,7 +227,7 @@ Each project in a monorepo gets its own step, because a passing parent gate
|
|
|
227
227
|
says nothing about a child project's coverage:
|
|
228
228
|
|
|
229
229
|
```yaml
|
|
230
|
-
- uses: raccioly/testguard@v0.
|
|
230
|
+
- uses: raccioly/testguard@v0.21.0
|
|
231
231
|
with:
|
|
232
232
|
command: gate
|
|
233
233
|
working-directory: services/billing
|
|
@@ -245,7 +245,7 @@ run that verified nothing must not look like a pass. During adoption set
|
|
|
245
245
|
`allow-empty`:
|
|
246
246
|
|
|
247
247
|
```yaml
|
|
248
|
-
- uses: raccioly/testguard@v0.
|
|
248
|
+
- uses: raccioly/testguard@v0.21.0
|
|
249
249
|
with:
|
|
250
250
|
command: probe
|
|
251
251
|
allow-empty: 'true'
|
|
@@ -17,7 +17,7 @@ move together:
|
|
|
17
17
|
|
|
18
18
|
```yaml
|
|
19
19
|
include:
|
|
20
|
-
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.
|
|
20
|
+
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.21.0/packaging/gitlab/testguard.gitlab-ci.yml'
|
|
21
21
|
inputs:
|
|
22
22
|
dir: backend # where testguard.claims.json lives (default ".")
|
|
23
23
|
post_note: true # post the brief on the merge request; needs TESTGUARD_GITLAB_TOKEN
|
|
@@ -215,12 +215,12 @@ jobs:
|
|
|
215
215
|
node-version: 22
|
|
216
216
|
- run: npm ci # every workspace's dependencies, from the root
|
|
217
217
|
- name: gate
|
|
218
|
-
uses: raccioly/testguard@v0.
|
|
218
|
+
uses: raccioly/testguard@v0.21.0
|
|
219
219
|
with:
|
|
220
220
|
command: gate
|
|
221
221
|
working-directory: ${{ matrix.project }}
|
|
222
222
|
- name: probe
|
|
223
|
-
uses: raccioly/testguard@v0.
|
|
223
|
+
uses: raccioly/testguard@v0.21.0
|
|
224
224
|
with:
|
|
225
225
|
command: probe
|
|
226
226
|
working-directory: ${{ matrix.project }}
|
|
@@ -141,7 +141,8 @@ npx testguard-cli probe . --progress ndjson 2>events.ndjson # machine-readable
|
|
|
141
141
|
{"event":"verdict","claim":"REDACT-001","fault":"F1","verdict":"survived","severity":"critical","file":"demo/redact.py","at":"…"}
|
|
142
142
|
```
|
|
143
143
|
|
|
144
|
-
The stages are `baseline
|
|
144
|
+
The stages are `reuse-baseline` (the one live run that licenses carrying a
|
|
145
|
+
verdict forward), `baseline`, `baseline-flake` (finishing the baseline runs after
|
|
145
146
|
a red one, so "failed once in three" can be told from "fails every time"),
|
|
146
147
|
`probe`, `negative-control` and `escalation`. A verdict reused from earlier
|
|
147
148
|
evidence carries `"reused": true`.
|
|
@@ -203,6 +204,13 @@ changed. All of these must match the evidence at the output path:
|
|
|
203
204
|
|
|
204
205
|
- the fault itself (its content hash) and the claim's metadata;
|
|
205
206
|
- the target file's hash and every selected defender's hash;
|
|
207
|
+
- everything the target and the defenders import, transitively, inside the
|
|
208
|
+
project (`inputs.dependencyHashes`), plus the lockfiles, `package.json`,
|
|
209
|
+
`pyproject.toml` and runner/compiler config at the project root. Python
|
|
210
|
+
closures include every `conftest.py` above each file. When an import cannot
|
|
211
|
+
be followed — a computed `import(name)`, `require(x)` or
|
|
212
|
+
`importlib.import_module(name)`, a relative import that resolves to no file —
|
|
213
|
+
the closure is `indeterminate` and that fault is probed every time;
|
|
206
214
|
- the requested and resolved defender sets and where they came from;
|
|
207
215
|
- the test universe and the discovery inputs. For built-in runners this
|
|
208
216
|
binds the runner, its version and the hashes of its config files,
|
|
@@ -213,6 +221,13 @@ changed. All of these must match the evidence at the output path:
|
|
|
213
221
|
version, and where it was resolved from (`project`, `path`, `builtin`);
|
|
214
222
|
- `--confirm` and the worker policy (`--workers`, `--serial`).
|
|
215
223
|
|
|
224
|
+
Then one more check that no hash can make: the defender set runs once on
|
|
225
|
+
unmodified source. Green, and every fault it defends is reused; red, and each
|
|
226
|
+
is probed in full. That run is the first run of the shared baseline, so it
|
|
227
|
+
costs one run per defender set, never one per fault, and
|
|
228
|
+
`run.measurements.reuseBaselineRuns` counts it. Evidence written before this
|
|
229
|
+
rule has no `dependencyHashes` and is probed again once.
|
|
230
|
+
|
|
216
231
|
A record produced by a probe error is never reused, and neither is one whose
|
|
217
232
|
defenders failed to load (`defenders-failed-to-load`): that is a statement
|
|
218
233
|
about the environment — a broken command, a missing dependency, a virtualenv
|
|
@@ -226,9 +241,12 @@ names. A cold run, a changed runner config or a changed worker policy probes
|
|
|
226
241
|
from scratch even when your source has not changed.
|
|
227
242
|
|
|
228
243
|
One kind of change reuse cannot see: anything outside the hashed inputs that
|
|
229
|
-
changes a verdict
|
|
230
|
-
a package upgraded inside the same virtualenv, or a service your
|
|
231
|
-
|
|
244
|
+
changes a verdict while the defenders stay green, such as an environment
|
|
245
|
+
variable, a package upgraded inside the same virtualenv, or a service your
|
|
246
|
+
tests read, a `setupFiles`/`globalSetup` file the runner config names by
|
|
247
|
+
string, or a lockfile above the project directory in a monorepo. A change that
|
|
248
|
+
turns the defenders red is caught by the live run.
|
|
249
|
+
Pass `--no-reuse` after one that does not.
|
|
232
250
|
|
|
233
251
|
In CI, restore the previous run's evidence from a cache and write to the same
|
|
234
252
|
path, so only claims whose inputs changed are probed again. TestGuard's own CI
|
|
@@ -147,9 +147,9 @@ falsa.
|
|
|
147
147
|
| npm | `npm i -D testguard-cli` y luego `npx testguard probe` |
|
|
148
148
|
| pip | `pip install testguard-cli` y luego `testguard probe` (requiere Node ≥ 20) |
|
|
149
149
|
| Homebrew | `brew tap raccioly/tap && brew install testguard` |
|
|
150
|
-
| GitHub Action | `uses: raccioly/testguard@v0.
|
|
150
|
+
| GitHub Action | `uses: raccioly/testguard@v0.21.0` |
|
|
151
151
|
| pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-gate`, `testguard-probe` |
|
|
152
|
-
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.
|
|
152
|
+
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.21.0/packaging/gitlab/testguard.gitlab-ci.yml` |
|
|
153
153
|
|
|
154
154
|
Los requisitos, las instalaciones sin conexión, `ENOVERSIONS` y todas las
|
|
155
155
|
opciones de CI están en la
|
|
@@ -145,9 +145,9 @@ a cobertura. O TestGuard o admite porque ele falha quando a alegação é falsa.
|
|
|
145
145
|
| npm | `npm i -D testguard-cli` e depois `npx testguard probe` |
|
|
146
146
|
| pip | `pip install testguard-cli` e depois `testguard probe` (requer Node ≥ 20) |
|
|
147
147
|
| Homebrew | `brew tap raccioly/tap && brew install testguard` |
|
|
148
|
-
| GitHub Action | `uses: raccioly/testguard@v0.
|
|
148
|
+
| GitHub Action | `uses: raccioly/testguard@v0.21.0` |
|
|
149
149
|
| pre-commit | `repo: https://github.com/raccioly/testguard`, hooks `testguard-claims`, `testguard-gate`, `testguard-probe` |
|
|
150
|
-
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.
|
|
150
|
+
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.21.0/packaging/gitlab/testguard.gitlab-ci.yml` |
|
|
151
151
|
|
|
152
152
|
Requisitos, instalações offline, `ENOVERSIONS` e todas as opções de CI estão
|
|
153
153
|
no [guia de instalação](https://github.com/raccioly/testguard/blob/main/docs/installation.md).
|
|
@@ -85,9 +85,9 @@ expect(store.writeAudit).toHaveBeenCalledWith(
|
|
|
85
85
|
| npm | `npm i -D testguard-cli`,然后 `npx testguard probe` |
|
|
86
86
|
| pip | `pip install testguard-cli`,然后 `testguard probe`(需要 Node ≥ 20) |
|
|
87
87
|
| Homebrew | `brew tap raccioly/tap && brew install testguard` |
|
|
88
|
-
| GitHub Action | `uses: raccioly/testguard@v0.
|
|
88
|
+
| GitHub Action | `uses: raccioly/testguard@v0.21.0` |
|
|
89
89
|
| pre-commit | `repo: https://github.com/raccioly/testguard`,钩子 `testguard-claims`、`testguard-gate`、`testguard-probe` |
|
|
90
|
-
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.
|
|
90
|
+
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.21.0/packaging/gitlab/testguard.gitlab-ci.yml` |
|
|
91
91
|
|
|
92
92
|
环境要求、离线安装、`ENOVERSIONS` 以及所有 CI 选项,见
|
|
93
93
|
[安装指南](https://github.com/raccioly/testguard/blob/main/docs/installation.md)。
|
|
@@ -31,9 +31,9 @@ packages in total.
|
|
|
31
31
|
| npm devDependency | `npm i -D testguard-cli` then `npx testguard probe` | a JavaScript or TypeScript project: version pinned in the lockfile, and the local install the agent layer prefers |
|
|
32
32
|
| pip | `pip install testguard-cli` then `testguard probe` | a Python team that does not want to touch npm directly (still needs Node ≥ 20) |
|
|
33
33
|
| Homebrew | `brew tap raccioly/tap && brew install testguard` | a machine-wide `testguard` on `PATH` |
|
|
34
|
-
| GitHub Action | `uses: raccioly/testguard@v0.
|
|
34
|
+
| GitHub Action | `uses: raccioly/testguard@v0.21.0` | CI on GitHub |
|
|
35
35
|
| pre-commit | `repo: https://github.com/raccioly/testguard` with a release tag as `rev:` ([below](#pre-commit)) | local hooks on commit and push |
|
|
36
|
-
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.
|
|
36
|
+
| GitLab CI | `include: - remote: https://raw.githubusercontent.com/raccioly/testguard/v0.21.0/packaging/gitlab/testguard.gitlab-ci.yml` | CI on GitLab |
|
|
37
37
|
|
|
38
38
|
### npx
|
|
39
39
|
|
|
@@ -114,7 +114,7 @@ tarball, checked against a `sha256` recorded at release time.
|
|
|
114
114
|
```yaml
|
|
115
115
|
- uses: actions/checkout@v7
|
|
116
116
|
with: { fetch-depth: 0 }
|
|
117
|
-
- uses: raccioly/testguard@v0.
|
|
117
|
+
- uses: raccioly/testguard@v0.21.0
|
|
118
118
|
with: { command: gate }
|
|
119
119
|
```
|
|
120
120
|
|
|
@@ -128,7 +128,7 @@ and complete pull-request and main-branch workflows, are in
|
|
|
128
128
|
```yaml
|
|
129
129
|
repos:
|
|
130
130
|
- repo: https://github.com/raccioly/testguard
|
|
131
|
-
rev: v0.
|
|
131
|
+
rev: v0.21.0
|
|
132
132
|
hooks:
|
|
133
133
|
- id: testguard-claims
|
|
134
134
|
- id: testguard-gate
|
|
@@ -144,7 +144,7 @@ three hooks, their stages and their arguments are in
|
|
|
144
144
|
|
|
145
145
|
```yaml
|
|
146
146
|
include:
|
|
147
|
-
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.
|
|
147
|
+
- remote: 'https://raw.githubusercontent.com/raccioly/testguard/v0.21.0/packaging/gitlab/testguard.gitlab-ci.yml'
|
|
148
148
|
inputs: { dir: . }
|
|
149
149
|
```
|
|
150
150
|
|
|
@@ -85,7 +85,27 @@ complete, confirmed run writes it, which is why a provisional or partial run
|
|
|
85
85
|
goes to its own file and can never be frozen into a baseline by accident.
|
|
86
86
|
|
|
87
87
|
The next `probe` reads its own output file to reuse verdicts whose recorded
|
|
88
|
-
inputs have not changed (`--no-reuse` turns that off).
|
|
88
|
+
inputs have not changed (`--no-reuse` turns that off). Those inputs include
|
|
89
|
+
`inputs.dependencyHashes`, the hashes of everything the target and its
|
|
90
|
+
defenders import plus the root lockfiles and runner config, and
|
|
91
|
+
`inputs.dependencyClosure`: `complete`, or `indeterminate` with the reasons in
|
|
92
|
+
`inputs.dependencyIssues` when an import could not be followed. An
|
|
93
|
+
indeterminate record, or one written before these fields existed, is never
|
|
94
|
+
reused. Before any record is reused its defender set runs once, unmodified;
|
|
95
|
+
`run.measurements.reuseBaselineRuns` counts those runs. `run.instrument` names
|
|
96
|
+
the TestGuard version and a digest of the resolver and runner configuration;
|
|
97
|
+
evidence from another version or digest is never reused. `run.reuse` counts the
|
|
98
|
+
records carried forward and the records measured. Reuse trusts this file: if
|
|
99
|
+
the people a gate judges can write it, run that gate with `--no-reuse`.
|
|
100
|
+
|
|
101
|
+
The file is written atomically (a temporary file, then a rename), so an
|
|
102
|
+
interrupted run never leaves a truncated document behind.
|
|
103
|
+
|
|
104
|
+
**For tools that read this file.** `inputs.targetHash` and
|
|
105
|
+
`inputs.defenderHashes` are unchanged; `inputs.dependencyHashes` was added beside
|
|
106
|
+
them. A consumer judging whether a record is still fresh must compare all three
|
|
107
|
+
against the current tree: the first two alone miss a changed helper that the
|
|
108
|
+
target or a defender imports.
|
|
89
109
|
|
|
90
110
|
### evidence-provisional.json and evidence-partial.json
|
|
91
111
|
|
|
@@ -204,12 +204,12 @@ spec before it is written.
|
|
|
204
204
|
| `--no-escalate` | off | Do not re-run survivors against the whole suite. Escalation runs the whole suite up to N times per survivor. |
|
|
205
205
|
| `--no-reuse` | off | Re-probe claims whose recorded inputs have not changed. |
|
|
206
206
|
| `--allow-empty` | off | Adoption only: a valid claims file with zero claims exits `0` instead of `2`, writes no evidence, and says verification was skipped. Has no effect with `--claim`. |
|
|
207
|
+
| `--require-assertion-kill` | off | Fail (exit 1) when any kill was made only by an uncaught exception and never by an assertion, or comes from evidence that records no kind. The verdict stays `killed`; the policy names the faults and is stored as `assertionPolicy`. Also valid on `admit`, where it refuses a test whose kills are all exception-only. A kill whose runner reported no error identity fails with `reason: "kind-not-reported-by-runner"`, per kill, in the text line and in `--json`: only a runner that reports identity can satisfy this policy. The built-in runners do. A custom `--runner-cmd` can if it writes a jest-shaped report (`testResults[].assertionResults[]`) with `failureDetails` per failed test: `matcherResult`, or `name: "AssertionError"`, or `code: "ERR_ASSERTION"`, or an `operator`. Without `failureDetails` every failure is an exception and the policy cannot pass. |
|
|
207
208
|
| `--require-origin <kinds>` | none | Require every claim's declared source kind to be in this set (repeatable, comma-separated). Needs a complete confirmed run: refused with `--claim`, `--allow-empty`, `--ref`, `--ignore-dirty` or `--confirm` below 3. Labels do not authenticate independence. |
|
|
208
209
|
| `--progress <mode>` | `auto` | `auto`, `tty`, `plain`, `ndjson` or `none`. Always on stderr. `--quiet` and `--json` imply `none` unless this is given explicitly. |
|
|
209
210
|
| `--verbose` | off | Also print every killed fault. |
|
|
210
211
|
| `--quiet` | off | Print only the summary and the evidence path. |
|
|
211
212
|
| `--cost` | off | Also report what this run's defenders cost. |
|
|
212
|
-
| `--changed <ref>` | detected | With `--json`: include claim coverage of the change in the status document. |
|
|
213
213
|
| `--json` | off | Print the status document plus this run's result. |
|
|
214
214
|
|
|
215
215
|
Where the evidence goes, unless `--out` says otherwise:
|
|
@@ -238,8 +238,9 @@ Detection, limits and per-runner behaviour are in
|
|
|
238
238
|
[languages and runners](languages-and-runners.md).
|
|
239
239
|
|
|
240
240
|
**Exit:** `0` no new gating finding at or above the severity floor; `1` at
|
|
241
|
-
least one new finding,
|
|
242
|
-
debt and severity floors
|
|
241
|
+
least one new finding, a failed `--require-origin` policy, or a failed
|
|
242
|
+
`--require-assertion-kill` policy (which baseline debt and severity floors
|
|
243
|
+
cannot hide); `2` precondition (an empty claims file
|
|
243
244
|
without `--allow-empty`, a runner that does not resolve, a dirty defender or
|
|
244
245
|
target with the implicit HEAD, an exhausted command budget, an unavailable
|
|
245
246
|
origin policy); `3` usage (`--confirm` not a positive integer, `--budget` or
|
|
@@ -298,9 +299,6 @@ says which install answered (`local install`, `npx cache`, `pnpm dlx cache`,
|
|
|
298
299
|
| `--max <n>` | `20` | Items in the brief, 1 to 50. |
|
|
299
300
|
| `--evidence <path>` | `<dir>/.testguard/evidence.json` | Evidence to brief from. |
|
|
300
301
|
| `--baseline <path>` | `<dir>/.testguard/baseline.json` if present | Baseline that marks findings as known. |
|
|
301
|
-
| `--changed <ref>` | detected | Reference for the unclaimed-changes section. A detected reference that does not resolve is silent here. |
|
|
302
|
-
| `--include-dirty` | off | With a reference, measure the working tree. |
|
|
303
|
-
| `--out <path>` | `<dir>/.testguard/brief.json` | Where the brief document is written (ignored with `--text`/`--markdown`). |
|
|
304
302
|
| `--json` | off | Print the brief document. |
|
|
305
303
|
|
|
306
304
|
**Exit:** `0`; `2` when there is no evidence and neither `--text` nor
|
|
@@ -287,7 +287,7 @@ probe costs, use `probe --cost` or `claims --cost`; see the
|
|
|
287
287
|
|
|
288
288
|
| Variable | Read by | Meaning |
|
|
289
289
|
|---|---|---|
|
|
290
|
-
| `TESTGUARD_CHANGED_REF` | `gate`, `sweep`, `status`, `brief`,
|
|
290
|
+
| `TESTGUARD_CHANGED_REF` | `gate`, `sweep`, `status`, `brief`, the MCP tools | The reference to measure a change against when `--changed` is absent. Checked before the CI variables below. |
|
|
291
291
|
| `TESTGUARD_NODE_MODULES` | `probe`, `admit`, `sweep`, `replay` | A `node_modules` directory to link into the scratch worktree. `--node-modules` wins. |
|
|
292
292
|
| `TESTGUARD_PYTHON` | `probe`, `claims --check-anchors`, `admit`, `sweep`, `replay` | The Python interpreter for `.py` defenders. `--python` wins when both are given; the two mean the same thing: a bare name such as `python3` is looked up on `PATH`, anything containing `/` is a path. |
|
|
293
293
|
| `TESTGUARD_GITLAB_TOKEN` | the GitLab CI template only | A project or group access token with `api` scope, used when `post_note: true` to post the brief as a merge-request note. `CI_JOB_TOKEN` cannot write notes. The CLI never reads it. See the [GitLab guide](../guides/ci/gitlab.md). |
|
|
@@ -19,7 +19,7 @@ is a pass. Text output prints every verdict but `killed` in capitals.
|
|
|
19
19
|
|
|
20
20
|
| Verdict | Printed | Meaning | Gates |
|
|
21
21
|
|---|---|---|---|
|
|
22
|
-
| `killed` | `killed` | Every one of N probe runs failed
|
|
22
|
+
| `killed` | `killed` | Every one of N probe runs failed in a test body, by an assertion or by an exception the fault provoked, on defenders that were green N/N on unmodified code. The claim is defended; `detail.killKind` says how (below). | no |
|
|
23
23
|
| `survived` | `SURVIVED` | Every one of N probe runs passed with the fault applied. A real blind spot. | yes |
|
|
24
24
|
| `nocover` | `NOCOVER` | No test defends the claim: the declared globs resolve to nothing, or no test file imports the target without mocking it. Nothing was even tried. | yes |
|
|
25
25
|
| `unverifiable` | `UNVERIFIABLE` | The fault could not be probed: its anchor moved or matches more than once, the defenders failed to load, discovery was indeterminate, or the defenders never execute the target. Nothing was learned. | yes |
|
|
@@ -27,6 +27,30 @@ is a pass. Text output prints every verdict but `killed` in capitals.
|
|
|
27
27
|
| `timeout` | `TIMEOUT` | The defenders exceeded `--budget` with the fault applied. A hang is not a detection. | yes |
|
|
28
28
|
| `flaky-defender` | `FLAKY-DEFENDER` | The defenders were not green N/N on unmodified code, or the N probe runs disagreed. No verdict about the fault can be trusted until the defenders are fixed. | yes |
|
|
29
29
|
|
|
30
|
+
### How a fault was killed
|
|
31
|
+
|
|
32
|
+
A kill by an assertion proves the suite checks the behavior. A kill only by an
|
|
33
|
+
exception proves the suite **reaches** the code: the fault made it throw and a
|
|
34
|
+
test went red, but nothing rejected a value. Where tests are written by an
|
|
35
|
+
agent, that second kind is exactly the hollow test that should not count as
|
|
36
|
+
proof, so every `killed` record says which it was. The verdict is unchanged.
|
|
37
|
+
|
|
38
|
+
| `detail.killKind` | Meaning |
|
|
39
|
+
|---|---|
|
|
40
|
+
| `assertion` | Every probe run had at least one assertion-classified failing test. |
|
|
41
|
+
| `exception-only` | No run had one. A test with no assertion went red because the code threw. |
|
|
42
|
+
| `mixed` | Some runs had one and some did not. |
|
|
43
|
+
|
|
44
|
+
The runner decides from structured error identity (an `AssertionError`, a
|
|
45
|
+
matcher result, a failed `expect` step, a Python exception type), never from
|
|
46
|
+
message text; an error it cannot read is an exception. `expect(fn).not.toThrow()`
|
|
47
|
+
and `assert.doesNotThrow` are assertions. `probe --require-assertion-kill`
|
|
48
|
+
(and `admit`) turns the distinction into a gate: any `exception-only` kill, or a
|
|
49
|
+
kill from evidence that predates the field, fails the run with exit 1, and the
|
|
50
|
+
verdict stays `killed`. `status` and `brief` count exception-only kills as a
|
|
51
|
+
weakness next to the co-authored count. The rule and its known limits are
|
|
52
|
+
rule 3 of the [gate semantics](../../spec/GATE-SEMANTICS.md).
|
|
53
|
+
|
|
30
54
|
"Gates" means the verdict fails `probe` (exit `1`) unless it is in the
|
|
31
55
|
baseline or its claim is below `--severity`. When a baseline exists, `probe`
|
|
32
56
|
tags each unproven record `[NEW]`, `[baseline]` or `[below floor]`.
|
|
@@ -144,7 +144,7 @@
|
|
|
144
144
|
<div class="page">
|
|
145
145
|
<div class="masthead">
|
|
146
146
|
<div class="wordmark">Test<span class="mark">Guard</span></div>
|
|
147
|
-
<div class="meta">Technical Brief · v0.
|
|
147
|
+
<div class="meta">Technical Brief · v0.21.0 · MIT</div>
|
|
148
148
|
</div>
|
|
149
149
|
|
|
150
150
|
<h1>Your tests pass. That is not the same as your tests working.</h1>
|
|
Binary file
|