testguard-cli 0.6.0__tar.gz → 0.7.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (289) hide show
  1. testguard_cli-0.7.0/.agents/skills/testguard/SKILL.md +63 -0
  2. testguard_cli-0.7.0/.docguard.json +36 -0
  3. testguard_cli-0.7.0/.docguardignore +30 -0
  4. testguard_cli-0.7.0/.github/scripts/build-one-pager.mjs +49 -0
  5. testguard_cli-0.7.0/.github/scripts/check-python-fixture.mjs +53 -0
  6. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.github/scripts/sync-release-version.mjs +9 -0
  7. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.github/workflows/ci.yml +102 -2
  8. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.github/workflows/release.yml +1 -1
  9. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.gitignore +6 -0
  10. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.npmignore +4 -0
  11. testguard_cli-0.7.0/.pre-commit-config.yaml +40 -0
  12. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/AGENTS.md +6 -1
  13. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/CHANGELOG.md +373 -0
  14. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/CLAUDE.md +2 -0
  15. testguard_cli-0.7.0/DRIFT-LOG.md +32 -0
  16. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/PKG-INFO +280 -30
  17. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/README.md +277 -27
  18. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/action.yml +12 -2
  19. testguard_cli-0.7.0/docs/testguard-explained.html +774 -0
  20. testguard_cli-0.7.0/docs/testguard-explained.pdf +0 -0
  21. testguard_cli-0.7.0/docs-canonical/ARCHITECTURE.md +184 -0
  22. testguard_cli-0.7.0/docs-canonical/DATA-MODEL.md +121 -0
  23. testguard_cli-0.7.0/docs-canonical/ENVIRONMENT.md +72 -0
  24. testguard_cli-0.7.0/docs-canonical/REQUIREMENTS.md +115 -0
  25. testguard_cli-0.7.0/docs-canonical/SECURITY.md +103 -0
  26. testguard_cli-0.7.0/docs-canonical/TEST-SPEC.md +126 -0
  27. testguard_cli-0.7.0/fixtures/known-answer-python/.gitignore +4 -0
  28. testguard_cli-0.7.0/fixtures/known-answer-python/README.md +91 -0
  29. testguard_cli-0.7.0/fixtures/known-answer-python/demo/__init__.py +0 -0
  30. testguard_cli-0.7.0/fixtures/known-answer-python/demo/export.py +5 -0
  31. testguard_cli-0.7.0/fixtures/known-answer-python/demo/orphan.py +5 -0
  32. testguard_cli-0.7.0/fixtures/known-answer-python/demo/pipeline.py +7 -0
  33. testguard_cli-0.7.0/fixtures/known-answer-python/demo/redact.py +50 -0
  34. testguard_cli-0.7.0/fixtures/known-answer-python/expected.json +80 -0
  35. testguard_cli-0.7.0/fixtures/known-answer-python/testguard.claims.json +425 -0
  36. testguard_cli-0.7.0/fixtures/known-answer-python/tests/test_export_mocked.py +17 -0
  37. testguard_cli-0.7.0/fixtures/known-answer-python/tests/test_flaky.py +27 -0
  38. testguard_cli-0.7.0/fixtures/known-answer-python/tests/test_patched.py +18 -0
  39. testguard_cli-0.7.0/fixtures/known-answer-python/tests/test_redact.py +40 -0
  40. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/package.json +2 -2
  41. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/packaging/gitlab/testguard.gitlab-ci.yml +9 -2
  42. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/packaging/homebrew/testguard.rb +1 -1
  43. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/pyproject.toml +3 -3
  44. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/GATE-SEMANTICS.md +112 -5
  45. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/README.md +28 -10
  46. testguard_cli-0.7.0/spec/conformance/examples/claims-methods.json +96 -0
  47. testguard_cli-0.7.0/spec/conformance/examples/evidence-disturbed-run.json +244 -0
  48. testguard_cli-0.7.0/spec/conformance/examples/evidence-negative-control.json +335 -0
  49. testguard_cli-0.7.0/spec/conformance/examples/evidence-python.json +189 -0
  50. testguard_cli-0.7.0/spec/conformance/examples/evidence-scan-method.json +52 -0
  51. testguard_cli-0.7.0/spec/conformance/invalid/claims.fault-injection-without-a-substitution.json +33 -0
  52. testguard_cli-0.7.0/spec/conformance/invalid/evidence.attribute-signal-not-a-defender.json +169 -0
  53. testguard_cli-0.7.0/spec/conformance/invalid/evidence.attribute-signal-without-attributes.json +168 -0
  54. testguard_cli-0.7.0/spec/conformance/invalid/evidence.fault-injection-without-its-evidence.json +54 -0
  55. testguard_cli-0.7.0/spec/conformance/invalid/evidence.negative-control-not-reached-but-survived.json +190 -0
  56. testguard_cli-0.7.0/spec/conformance/invalid/evidence.negative-control-reached-on-a-kill.json +190 -0
  57. testguard_cli-0.7.0/spec/conformance/invalid/evidence.probe-error-without-message.json +243 -0
  58. testguard_cli-0.7.0/spec/conformance/invalid/evidence.restore-skipped-in-place.json +234 -0
  59. testguard_cli-0.7.0/spec/conformance/invalid/evidence.target-not-imported-but-killed.json +163 -0
  60. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/schemas.test.mjs +4 -1
  61. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/lib/validate.mjs +84 -8
  62. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/schemas/baseline.schema.json +7 -7
  63. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/schemas/brief.schema.json +12 -12
  64. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/schemas/calibration.schema.json +5 -5
  65. testguard_cli-0.7.0/spec/schemas/claims.schema.json +200 -0
  66. testguard_cli-0.7.0/spec/schemas/common.schema.json +223 -0
  67. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/schemas/evidence.schema.json +169 -73
  68. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/schemas/gate.schema.json +12 -12
  69. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/schemas/ignore.schema.json +4 -4
  70. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/schemas/replay.schema.json +10 -10
  71. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/schemas/status.schema.json +25 -25
  72. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/cli.mjs +20 -5
  73. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/commands/claims.mjs +22 -2
  74. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/commands/probe.mjs +24 -4
  75. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/gate/changed.mjs +9 -2
  76. testguard_cli-0.7.0/src/probe/attribution.mjs +123 -0
  77. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/probe/classify.mjs +18 -2
  78. testguard_cli-0.7.0/src/probe/cost.mjs +145 -0
  79. testguard_cli-0.7.0/src/probe/discover.mjs +29 -0
  80. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/probe/inject.mjs +41 -9
  81. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/probe/mocks.mjs +2 -0
  82. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/probe/probe.mjs +235 -52
  83. testguard_cli-0.7.0/src/probe/progress.mjs +87 -0
  84. testguard_cli-0.7.0/src/probe/pyimports.mjs +287 -0
  85. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/probe/rank.mjs +2 -0
  86. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/probe/runners/index.mjs +38 -11
  87. testguard_cli-0.7.0/src/probe/runners/jest.mjs +21 -0
  88. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/probe/runners/playwright.mjs +9 -0
  89. testguard_cli-0.7.0/src/probe/runners/python/_testguard_pytest_plugin.py +132 -0
  90. testguard_cli-0.7.0/src/probe/runners/python/_testguard_report.py +141 -0
  91. testguard_cli-0.7.0/src/probe/runners/python/_testguard_unittest_main.py +168 -0
  92. testguard_cli-0.7.0/src/probe/runners/python.mjs +316 -0
  93. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/probe/runners/shared.mjs +10 -1
  94. testguard_cli-0.7.0/src/probe/runners/vitest.mjs +20 -0
  95. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/replay/replay.mjs +3 -2
  96. testguard_cli-0.7.0/src/scaffold/producers.python.mjs +209 -0
  97. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/scaffold/scaffold.mjs +86 -32
  98. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/util/glob.mjs +7 -1
  99. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/admit.test.mjs +2 -0
  100. testguard_cli-0.7.0/test/attribution.test.mjs +198 -0
  101. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/baseline.test.mjs +2 -0
  102. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/brief.test.mjs +2 -0
  103. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/classify.test.mjs +4 -0
  104. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/contention.test.mjs +2 -0
  105. testguard_cli-0.7.0/test/cost.test.mjs +127 -0
  106. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/discover.test.mjs +2 -0
  107. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/gate.test.mjs +2 -0
  108. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/gitlab-template.test.mjs +1 -1
  109. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/init.test.mjs +2 -0
  110. testguard_cli-0.7.0/test/inject.test.mjs +93 -0
  111. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/mcp.test.mjs +3 -0
  112. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/mocks.test.mjs +2 -0
  113. testguard_cli-0.7.0/test/negative-control.test.mjs +184 -0
  114. testguard_cli-0.7.0/test/probe-error.test.mjs +182 -0
  115. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/probe-preconditions.test.mjs +2 -0
  116. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/probe.fixture.test.mjs +35 -0
  117. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/probe.jest.test.mjs +2 -0
  118. testguard_cli-0.7.0/test/probe.python.test.mjs +223 -0
  119. testguard_cli-0.7.0/test/progress.test.mjs +96 -0
  120. testguard_cli-0.7.0/test/pyimports.test.mjs +210 -0
  121. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/release-sync.test.mjs +61 -0
  122. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/replay.test.mjs +2 -0
  123. testguard_cli-0.7.0/test/runner-argv.test.mjs +129 -0
  124. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/runner-command.test.mjs +17 -1
  125. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/runner-playwright.test.mjs +6 -1
  126. testguard_cli-0.7.0/test/runner-python.test.mjs +202 -0
  127. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/runner-vitest.test.mjs +27 -0
  128. testguard_cli-0.7.0/test/samples/python-report-collect-error.json +10 -0
  129. testguard_cli-0.7.0/test/samples/python-report.json +16 -0
  130. testguard_cli-0.7.0/test/scaffold.python.test.mjs +153 -0
  131. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/scaffold.test.mjs +2 -0
  132. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/status.test.mjs +2 -0
  133. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/worktree.test.mjs +51 -0
  134. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/writer.test.mjs +3 -0
  135. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/testguard.claims.json +974 -38
  136. testguard_cli-0.7.0/testguard.ignore.json +62 -0
  137. testguard_cli-0.6.0/spec/schemas/claims.schema.json +0 -103
  138. testguard_cli-0.6.0/spec/schemas/common.schema.json +0 -125
  139. testguard_cli-0.6.0/src/probe/discover.mjs +0 -21
  140. testguard_cli-0.6.0/src/probe/runners/jest.mjs +0 -9
  141. testguard_cli-0.6.0/src/probe/runners/vitest.mjs +0 -8
  142. testguard_cli-0.6.0/test/inject.test.mjs +0 -46
  143. testguard_cli-0.6.0/testguard.ignore.json +0 -41
  144. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.codex/hooks.json +0 -0
  145. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.gitattributes +0 -0
  146. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.github/ISSUE_TEMPLATE/bug_report.md +0 -0
  147. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.github/ISSUE_TEMPLATE/feature_request.md +0 -0
  148. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.github/PULL_REQUEST_TEMPLATE.md +0 -0
  149. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.github/dependabot.yml +0 -0
  150. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.github/scripts/install-smoke.mjs +0 -0
  151. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.github/workflows/auto-merge.yml +0 -0
  152. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.github/workflows/scheduled-release.yml +0 -0
  153. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.github/workflows/supply-chain.yml +0 -0
  154. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.npmrc +0 -0
  155. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/.pre-commit-hooks.yaml +0 -0
  156. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/CODE_OF_CONDUCT.md +0 -0
  157. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/CONTRIBUTING.md +0 -0
  158. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/LICENSE +0 -0
  159. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/PRIVACY.md +0 -0
  160. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/SECURITY.md +0 -0
  161. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/SUPPORT.md +0 -0
  162. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/bench/README.md +0 -0
  163. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/cli/testguard.mjs +0 -0
  164. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer/.gitignore +0 -0
  165. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer/README.md +0 -0
  166. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer/expected.json +0 -0
  167. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer/package.json +0 -0
  168. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer/src/export.mjs +0 -0
  169. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer/src/redact.mjs +0 -0
  170. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer/test/export-mocked.test.mjs +0 -0
  171. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer/test/flaky.test.mjs +0 -0
  172. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer/test/redact.test.mjs +0 -0
  173. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer/testguard.claims.json +0 -0
  174. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer/vitest.config.mjs +0 -0
  175. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-jest/.gitignore +0 -0
  176. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-jest/README.md +0 -0
  177. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-jest/expected.json +0 -0
  178. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-jest/jest.config.js +0 -0
  179. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-jest/package.json +0 -0
  180. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-jest/src/export.js +0 -0
  181. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-jest/src/redact.js +0 -0
  182. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-jest/test/export-mocked.test.js +0 -0
  183. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-jest/test/flaky.test.js +0 -0
  184. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-jest/test/redact.test.js +0 -0
  185. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-jest/testguard.claims.json +0 -0
  186. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/.gitignore +0 -0
  187. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/README.md +0 -0
  188. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/e2e/flaky.spec.mjs +0 -0
  189. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/e2e/toggle.spec.mjs +0 -0
  190. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/expected.json +0 -0
  191. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/package-lock.json +0 -0
  192. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/package.json +0 -0
  193. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/playwright.config.mjs +0 -0
  194. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/src/toggle.mjs +0 -0
  195. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/test/toggle.test.mjs +0 -0
  196. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/test-results/.last-run.json +0 -0
  197. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/testguard.claims.json +0 -0
  198. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/fixtures/known-answer-playwright/vitest.config.mjs +0 -0
  199. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/package-lock.json +0 -0
  200. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/examples/baseline.json +0 -0
  201. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/examples/brief.json +0 -0
  202. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/examples/calibration.json +0 -0
  203. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/examples/claims.json +0 -0
  204. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/examples/evidence-provisional.json +0 -0
  205. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/examples/evidence.json +0 -0
  206. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/examples/gate.json +0 -0
  207. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/examples/ignore.json +0 -0
  208. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/examples/replay.json +0 -0
  209. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/examples/status.json +0 -0
  210. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/baseline.bad-fingerprint-key.json +0 -0
  211. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/baseline.zero-count.json +0 -0
  212. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/brief.text-without-heading.json +0 -0
  213. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/brief.unknown-verdict-key.json +0 -0
  214. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/calibration.p-outside-ci.json +0 -0
  215. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/calibration.positives-exceed-n.json +0 -0
  216. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/claims.absolute-path.json +0 -0
  217. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/claims.duplicate-fault-id.json +0 -0
  218. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/claims.missing-provenance.json +0 -0
  219. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/claims.noop-fault.json +0 -0
  220. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/claims.occurrence-exceeds-hits.json +0 -0
  221. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/claims.parent-traversal.json +0 -0
  222. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.annotated-signal-without-reason.json +0 -0
  223. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.byrunner-file-not-resolved.json +0 -0
  224. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.contention-without-runners.json +0 -0
  225. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.discovered-defender-mocks-subject.json +0 -0
  226. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.flaky-defender-without-flakiness.json +0 -0
  227. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.ignored-dirty-on-snapshot.json +0 -0
  228. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.killed-by-non-assertion.json +0 -0
  229. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.killed-on-red-baseline.json +0 -0
  230. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.killed-without-n-runs.json +0 -0
  231. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.provisional-flag-on-confirmed-run.json +0 -0
  232. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.rank-extra-property.json +0 -0
  233. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.single-run-without-provisional.json +0 -0
  234. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.undeclared-killers-without-reason.json +0 -0
  235. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.unknown-verdict.json +0 -0
  236. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.unverifiable-without-reason.json +0 -0
  237. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/evidence.wrong-fingerprint.json +0 -0
  238. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/gate.counts-disagree.json +0 -0
  239. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/gate.exit-zero-with-uncovered.json +0 -0
  240. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/gate.ignore-cover-without-pattern.json +0 -0
  241. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/ignore.missing-reason.json +0 -0
  242. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/ignore.short-reason.json +0 -0
  243. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/replay.blind-with-a-failing-run.json +0 -0
  244. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/replay.duplicate-patch-id.json +0 -0
  245. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/status.claim-without-file.json +0 -0
  246. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/status.clean-with-new-findings.json +0 -0
  247. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/status.unclaimed-hidden-behind-unproven.json +0 -0
  248. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/conformance/invalid/status.write-test-without-target.json +0 -0
  249. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/spec/lib/fingerprint.mjs +0 -0
  250. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/admit/admit.mjs +0 -0
  251. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/baseline/baseline.mjs +0 -0
  252. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/brief/brief.mjs +0 -0
  253. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/claims/annotations.mjs +0 -0
  254. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/claims/load.mjs +0 -0
  255. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/claims/removed.mjs +0 -0
  256. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/commands/admit.mjs +0 -0
  257. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/commands/baseline.mjs +0 -0
  258. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/commands/brief.mjs +0 -0
  259. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/commands/gate.mjs +0 -0
  260. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/commands/init.mjs +0 -0
  261. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/commands/mcp.mjs +0 -0
  262. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/commands/replay.mjs +0 -0
  263. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/commands/scaffold.mjs +0 -0
  264. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/commands/status.mjs +0 -0
  265. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/evidence/writer.mjs +0 -0
  266. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/git.mjs +0 -0
  267. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/init/init.mjs +0 -0
  268. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/init/templates/SKILL.md +0 -0
  269. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/mcp/server.mjs +0 -0
  270. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/mcp/tools.mjs +0 -0
  271. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/probe/contention.mjs +0 -0
  272. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/probe/independence.mjs +0 -0
  273. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/probe/worktree.mjs +0 -0
  274. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/render.mjs +0 -0
  275. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/replay/label.mjs +0 -0
  276. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/scaffold/producers.mjs +0 -0
  277. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/status/status.mjs +0 -0
  278. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/src/util/hash.mjs +0 -0
  279. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/annotations.test.mjs +0 -0
  280. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/claims-since.test.mjs +0 -0
  281. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/glob.test.mjs +0 -0
  282. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/independence.test.mjs +0 -0
  283. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/probe.playwright.test.mjs +0 -0
  284. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/rank-aliases.test.mjs +0 -0
  285. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/samples/playwright-report.json +0 -0
  286. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/test/snapshot.test.mjs +0 -0
  287. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/testguard_cli/__init__.py +0 -0
  288. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/testguard_cli/wrapper.py +0 -0
  289. {testguard_cli-0.6.0 → testguard_cli-0.7.0}/vitest.config.mjs +0 -0
@@ -0,0 +1,63 @@
1
+ ---
2
+ name: testguard
3
+ description: Prove the test suite defends this project's claims. Use before writing or changing tests, after changing guarded code, and whenever a session starts with a TEST BLINDSPOT CONTEXT block.
4
+ ---
5
+
6
+ # TestGuard — how an agent runs it
7
+
8
+ TestGuard injects the faults a project's claims forbid and reports every one
9
+ the tests fail to detect. It is a **claim verifier**, not a test generator.
10
+ You operate it through one loop and one source of truth.
11
+
12
+ ## The one source of truth
13
+
14
+ ```bash
15
+ testguard status --json
16
+ ```
17
+
18
+ Read `state` and `next`. Do what `next.command` says. Never infer state from
19
+ which files exist. `state` is one of:
20
+
21
+ | state | meaning | you do |
22
+ |---|---|---|
23
+ | `no-claims` | no `testguard.claims.json` | `testguard scaffold <file>` for a file with guards; replace every `TODO:` statement with what the code guarantees; keep or drop each proposal; move the claims into `testguard.claims.json` |
24
+ | `unprobed` | claims never probed | `testguard probe` |
25
+ | `evidence-stale` | code, tests or claims changed since the evidence | `testguard probe --include-dirty` (or `--claim <ID>` for one) |
26
+ | `provisional-only` | only `--confirm 1` evidence exists | `testguard probe --confirm 3` |
27
+ | `unproven` | a finding is not covered by the baseline | `next.target` names it; see the verdict table |
28
+ | `clean` | everything killed or baselined | `testguard baseline` if `next` says so; otherwise nothing |
29
+
30
+ Exit codes: `0` clean · `1` unproven claims (or drift) · `2` precondition failed / nothing to do yet · `3` usage.
31
+
32
+ ## Verdict → action
33
+
34
+ | verdict | what it means | the only acceptable fix |
35
+ |---|---|---|
36
+ | `SURVIVED` | the defenders stayed green while the claim was false | **write a test** in the defender file that fails with the fault applied and passes on HEAD |
37
+ | `SURVIVED [killed-by-undeclared-tests: …]` | other tests catch it | add those files to the claim's `defendedBy` |
38
+ | `NOCOVER` | no test file imports the target | write a new test file that imports it and asserts the claim |
39
+ | `UNVERIFIABLE` | the fault's anchor no longer matches, or the defenders fail to load | fix the **fault definition** (or the defenders' import); never the code |
40
+ | `FAULT-INVALID` | the replacement does not compile | fix the fault definition |
41
+ | `TIMEOUT` | the fault makes the code hang | write an assertion that fails on it; a hang is not a detection |
42
+ | `FLAKY-DEFENDER` | defenders not green 3/3, or disagreed across runs | fix the flake first; no verdict is trustworthy until then |
43
+
44
+ ## The fix loop
45
+
46
+ 1. `testguard status --json` → take `next.target`.
47
+ 2. Write the test. It must **fail when the fault is applied and pass on HEAD**. To check the first half by hand: apply `find` → `replace` in the target file, run the defender, restore.
48
+ 3. `testguard probe --claim <ID> --include-dirty` — probes your uncommitted test without touching the tree. `--confirm 1` gives a fast **provisional** signal; a `?` on a verdict means unconfirmed.
49
+ 4. When it is `killed` at `--confirm 3`, commit the test.
50
+ 5. When everything is killed or baselined, `testguard baseline` if `next` says so, and commit `.testguard/baseline.json`.
51
+
52
+ ## Rules that are not yours to bend
53
+
54
+ - **Never make a fault die by editing the claims file.** Editing a claim is allowed when the claim was wrong — but the evidence records every fault's content hash, and `status` lists any fault edited after it survived, with its previous verdict, as `changedFaults`. The change is permitted; it is never invisible. Say why in the commit.
55
+ - **A test that asserts current behaviour is not a fix.** If the test would pass with the fault applied, it defends nothing.
56
+ - **Never re-implement the code under test inside the test.** Drive the real function; assert the outcome, not that a function was called.
57
+ - `--confirm 1` results are provisional: they print with `?`, go to `evidence-provisional.json`, and cannot be baselined.
58
+ - Worktree mode probes a **commit**. If `probe` refuses because defenders are dirty, use `--include-dirty`; do not commit half-written tests to satisfy it.
59
+ - Do not add `.testguard/evidence.json` or `brief.json` to git; do commit `baseline.json`.
60
+
61
+ ## Reading the numbers
62
+
63
+ `N unproven faults across M claims` counts faults and the claims they belong to. There is no single score on purpose: blindness is concentrated, and one number hides where.
@@ -0,0 +1,36 @@
1
+ {
2
+ "$schema": "https://raccioly.github.io/docguard/schemas/docguard-config.schema.json",
3
+ "projectName": "testguard-cli",
4
+ "version": "0.6",
5
+ "profile": "enterprise",
6
+ "projectType": "cli",
7
+ "projectTypeConfig": {
8
+ "needsEnvVars": false,
9
+ "needsEnvExample": false,
10
+ "needsE2E": false,
11
+ "needsDatabase": false
12
+ },
13
+ "requiredFiles": {
14
+ "canonical": [
15
+ "docs-canonical/ARCHITECTURE.md",
16
+ "docs-canonical/DATA-MODEL.md",
17
+ "docs-canonical/SECURITY.md",
18
+ "docs-canonical/TEST-SPEC.md",
19
+ "docs-canonical/ENVIRONMENT.md",
20
+ "docs-canonical/REQUIREMENTS.md"
21
+ ]
22
+ },
23
+ "validators": {
24
+ "structure": true,
25
+ "docsSync": true,
26
+ "drift": true,
27
+ "changelog": true,
28
+ "architecture": true,
29
+ "testSpec": true,
30
+ "security": true,
31
+ "environment": true,
32
+ "freshness": true
33
+ },
34
+ "severity": {},
35
+ "findingSeverity": {}
36
+ }
@@ -0,0 +1,30 @@
1
+ # .docguardignore — paths to exclude from DocGuard validation.
2
+ # Gitignore-style syntax: one pattern per line, # for comments.
3
+ # Merged into config.ignore (in .docguard.json) at runtime.
4
+ #
5
+ # Common examples:
6
+ # build/ # exclude a directory
7
+ # **/__generated__/** # exclude anything in any __generated__ dir
8
+ # vendor/legacy.ts # exclude a single file
9
+ # **/*.snap # exclude all files matching a glob
10
+ #
11
+ # Build outputs, vendored libs, generated code are good candidates here.
12
+
13
+ # Vendored / generated code that's not yours to document
14
+ **/__generated__/**
15
+ **/generated/**
16
+ **/*.generated.*
17
+
18
+ # Migrations and lock files
19
+ **/migrations/**
20
+ **/*.lock
21
+ package-lock.json
22
+ yarn.lock
23
+ pnpm-lock.yaml
24
+ Cargo.lock
25
+ poetry.lock
26
+
27
+ # Common build artifacts (defaults also cover these, but listing here is clearer)
28
+ # dist/
29
+ # build/
30
+ # coverage/
@@ -0,0 +1,49 @@
1
+ #!/usr/bin/env node
2
+ /**
3
+ * Render docs/testguard-explained.html to PDF with headless Chrome.
4
+ *
5
+ * Chrome rather than a library because this repository has one pinned runtime
6
+ * dependency and keeps it that way: a PDF renderer would be the second, and it
7
+ * would be needed only to build a document nobody imports.
8
+ *
9
+ * Set CHROME to override the binary. Exits 0 with a notice when no Chrome is
10
+ * present, so a checkout without one can still run every other script.
11
+ */
12
+ import { existsSync } from 'node:fs';
13
+ import { spawnSync } from 'node:child_process';
14
+ import { dirname, join, resolve } from 'node:path';
15
+ import { fileURLToPath } from 'node:url';
16
+
17
+ const root = resolve(dirname(fileURLToPath(import.meta.url)), '..', '..');
18
+ const src = join(root, 'docs', 'testguard-explained.html');
19
+ const out = join(root, 'docs', 'testguard-explained.pdf');
20
+
21
+ const candidates = [
22
+ process.env.CHROME,
23
+ '/Applications/Google Chrome.app/Contents/MacOS/Google Chrome',
24
+ '/Applications/Chromium.app/Contents/MacOS/Chromium',
25
+ '/usr/bin/google-chrome',
26
+ '/usr/bin/chromium',
27
+ '/usr/bin/chromium-browser',
28
+ ].filter(Boolean);
29
+
30
+ const chrome = candidates.find((c) => existsSync(c));
31
+ if (!chrome) {
32
+ console.log('no Chrome found; skipping PDF build (set CHROME=<path> to force)');
33
+ process.exit(0);
34
+ }
35
+
36
+ const r = spawnSync(chrome, [
37
+ '--headless',
38
+ '--disable-gpu',
39
+ '--no-sandbox',
40
+ '--no-pdf-header-footer',
41
+ `--print-to-pdf=${out}`,
42
+ `file://${src}`,
43
+ ], { encoding: 'utf8' });
44
+
45
+ if (r.status !== 0 || !existsSync(out)) {
46
+ console.error(r.stderr || 'chrome produced no PDF');
47
+ process.exit(1);
48
+ }
49
+ console.log(`docs/testguard-explained.pdf written with ${chrome.split('/').pop()}`);
@@ -0,0 +1,53 @@
1
+ #!/usr/bin/env node
2
+ /**
3
+ * Compare a probe of `fixtures/known-answer-python` against its oracle.
4
+ *
5
+ * Shared by both CI legs so the stdlib engine and pytest are held to exactly
6
+ * the same expectations — the point of the fixture is that two engines with
7
+ * different report shapes must not disagree about a single verdict.
8
+ */
9
+ import { readFileSync } from 'node:fs';
10
+ import { dirname, join } from 'node:path';
11
+ import { fileURLToPath } from 'node:url';
12
+
13
+ const [evidencePath, expectedEngine] = process.argv.slice(2);
14
+ if (!evidencePath) {
15
+ console.error('usage: check-python-fixture.mjs <evidence.json> [engine]');
16
+ process.exit(2);
17
+ }
18
+
19
+ const ROOT = join(dirname(fileURLToPath(import.meta.url)), '..', '..');
20
+ const oracle = JSON.parse(readFileSync(join(ROOT, 'fixtures', 'known-answer-python', 'expected.json'), 'utf8')).expected;
21
+ const evidence = JSON.parse(readFileSync(evidencePath, 'utf8'));
22
+
23
+ const problems = [];
24
+ if (expectedEngine && evidence.run.runner.name !== expectedEngine) {
25
+ problems.push(`runner was ${evidence.run.runner.name}, expected ${expectedEngine}`);
26
+ }
27
+
28
+ const seen = new Set();
29
+ for (const r of evidence.records) {
30
+ const key = `${r.claim.id}/${r.subject.id}`;
31
+ seen.add(key);
32
+ const want = oracle[key];
33
+ if (!want) { problems.push(`${key}: not in expected.json`); continue; }
34
+ if (want.verdict !== r.verdict) problems.push(`${key}: ${r.verdict}, expected ${want.verdict}`);
35
+ if (want.reason && want.reason !== r.detail.reason) problems.push(`${key}: reason ${r.detail.reason}, expected ${want.reason}`);
36
+ if (Boolean(want.targetNotImported) !== Boolean(r.detail.targetNotImported)) {
37
+ problems.push(`${key}: targetNotImported ${Boolean(r.detail.targetNotImported)}, expected ${Boolean(want.targetNotImported)}`);
38
+ }
39
+ // The negative control is pinned too, not merely tolerated: a fixture that
40
+ // accepted either answer would stop defending the one behaviour it exists
41
+ // to prove, and the verdict alone cannot tell "the defenders reached this
42
+ // file and said nothing" from "they never reached it".
43
+ if ((want.negativeControl ?? null) !== (r.detail.negativeControl ?? null)) {
44
+ problems.push(`${key}: negativeControl ${r.detail.negativeControl ?? 'absent'}, expected ${want.negativeControl ?? 'absent'}`);
45
+ }
46
+ }
47
+ for (const key of Object.keys(oracle)) if (!seen.has(key)) problems.push(`${key}: expected but never probed`);
48
+
49
+ if (problems.length) {
50
+ for (const p of problems) console.error(`::error::${p}`);
51
+ process.exit(1);
52
+ }
53
+ console.log(`python fixture verdicts match expected.json under ${evidence.run.runner.name}: ${evidence.records.length} records`);
@@ -6,6 +6,14 @@
6
6
  * - action.yml default of the `version` input
7
7
  * - packaging/homebrew/*.rb the tarball URL (sha256 is set after publish)
8
8
  * - packaging/gitlab/*.yml the include URL tag and the `version` input default
9
+ * - README.md the GitHub Action reference AND the GitLab include URL
10
+ *
11
+ * A file may carry more than one form, and each needs its own entry: README
12
+ * holds both the `raccioly/testguard@vX.Y.Z` action reference and the
13
+ * `testguard/vX.Y.Z/packaging` template URL. Only the first was listed, so
14
+ * v0.6.0 shipped with both README copies of the GitLab URL left at v0.5.0
15
+ * while `--check` reported "all version surfaces at 0.6.0" — a surface this
16
+ * script was never told to look at cannot drift in its eyes.
9
17
  *
10
18
  * `--check` verifies instead of writing (used by release.yml).
11
19
  */
@@ -22,6 +30,7 @@ const surfaces = [
22
30
  ['action.yml', /(\n version:\n description: [^\n]*\n required: false\n default: ')[^']*(')/, `$1${version}$2`],
23
31
  ['packaging/homebrew/testguard.rb', /testguard-cli-\d+\.\d+\.\d+\.tgz/g, `testguard-cli-${version}.tgz`],
24
32
  ['README.md', /raccioly\/testguard@v\d+\.\d+\.\d+/g, `raccioly/testguard@v${version}`],
33
+ ['README.md', /testguard\/v\d+\.\d+\.\d+\/packaging/g, `testguard/v${version}/packaging`],
25
34
  ['packaging/gitlab/testguard.gitlab-ci.yml', /testguard\/v\d+\.\d+\.\d+\/packaging/g, `testguard/v${version}/packaging`],
26
35
  ['packaging/gitlab/testguard.gitlab-ci.yml', /(\n version:\n description: [^\n]*\n default: ")\d+\.\d+\.\d+(")/, `$1${version}$2`],
27
36
  ];
@@ -147,6 +147,30 @@ jobs:
147
147
  console.log('mcp:', list.result.tools.map((t) => t.name).join(', '), '→ state', call.result.structuredContent.state);
148
148
  "
149
149
 
150
+ # ── Documentation is canon: DocGuard validates this repository against
151
+ # docs-canonical/. This is the authoritative check — a machine-local
152
+ # .git/hooks file is neither shared with contributors nor reviewable,
153
+ # so CI is where doc drift is actually caught. `guard` exits 2 for
154
+ # warnings, which we surface without failing; errors fail the build. ──
155
+ - name: DocGuard — documentation matches the code
156
+ run: |
157
+ # Capture the exit code explicitly. A pipeline under the default
158
+ # `bash -e` aborts the step before any branch on its status can run,
159
+ # and `|| code=$?` is the one form errexit does not trip on.
160
+ code=0
161
+ npx -y docguard-cli@0.41.5 guard > "$RUNNER_TEMP/docguard-guard.txt" 2>&1 || code=$?
162
+ cat "$RUNNER_TEMP/docguard-guard.txt"
163
+ if [ "$code" -eq 2 ]; then
164
+ echo "::warning::DocGuard reported warnings (exit 2); see the output above."
165
+ elif [ "$code" -ne 0 ]; then
166
+ echo "::error::DocGuard guard failed (exit $code) — documentation contradicts the code."
167
+ exit 1
168
+ fi
169
+
170
+ - name: DocGuard — maturity score (reported, not gated)
171
+ if: always()
172
+ run: npx -y docguard-cli@0.41.5 score || true
173
+
150
174
  # ── Every change needs a claim: the gate runs on this repository's own
151
175
  # pull requests. A new source file without a claim (or an excusing
152
176
  # ignore entry with a reason) turns the PR red. ──
@@ -165,7 +189,9 @@ jobs:
165
189
  # testguard.claims.json states invariants of the tool itself; every
166
190
  # fault must be killed by the tool's own tests. Exit 0 = all defended. ──
167
191
  # Runs on ONE Node leg: the verdicts do not depend on the Node minor,
168
- # and each from-scratch probe is ~14 minutes. The previous run's
192
+ # and a from-scratch probe is ~10 minutes (it was ~24 before the
193
+ # defender split in #67; `--cost` below prints where the time goes,
194
+ # so a regression in gate cost is visible in the log). The previous run's
169
195
  # evidence is restored from the cache so claims whose target and
170
196
  # defenders did not change reuse their verdict (the tool's own
171
197
  # reuse rule); only what changed is re-probed. No escalation in CI:
@@ -182,7 +208,7 @@ jobs:
182
208
 
183
209
  - name: probe TestGuard's own claims
184
210
  if: matrix.node-version == 22
185
- run: node cli/testguard.mjs probe . --budget 180000 --no-escalate --out .testguard/ci-self-evidence.json
211
+ run: node cli/testguard.mjs probe . --budget 180000 --no-escalate --cost --out .testguard/ci-self-evidence.json
186
212
 
187
213
  # A stable, branch-named artifact so a developer can fetch the evidence
188
214
  # CI produced without knowing a run id (README: read CI's evidence
@@ -203,6 +229,80 @@ jobs:
203
229
  path: .testguard/ci-self-evidence.json
204
230
  if-no-files-found: warn
205
231
 
232
+ # ── Python: two engines, one oracle. The stdlib leg runs FIRST and with no
233
+ # pip install of any kind, because "a project with no test dependencies
234
+ # still probes" is a claim about the tool and the only way to keep it true
235
+ # is to prove it in an environment that has none. The pytest leg then
236
+ # installs pytest into a venv and must reach the SAME sixteen verdicts
237
+ # through a different report shape. ──
238
+ python:
239
+ runs-on: ubuntu-latest
240
+ steps:
241
+ - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
242
+ with:
243
+ fetch-depth: 0
244
+ - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
245
+ with:
246
+ node-version: 22
247
+ cache: npm
248
+ - name: Install (tool only)
249
+ run: npm ci
250
+
251
+ # A bare virtualenv, not the system interpreter: the claim under test is
252
+ # "a project whose test dependencies are the standard library still
253
+ # probes", and asserting that the runner image happens to lack pytest
254
+ # would make this job fail for a reason that has nothing to do with the
255
+ # product the day that image changes. Guarantee the condition instead.
256
+ - name: The stdlib engine must need nothing installed
257
+ shell: bash
258
+ run: |
259
+ set -o pipefail
260
+ python3 -m venv "$RUNNER_TEMP/bare"
261
+ if "$RUNNER_TEMP/bare/bin/python" -c 'import pytest' 2>/dev/null; then
262
+ echo "::error::the bare venv can import pytest; it is not bare and this leg proves nothing"
263
+ exit 1
264
+ fi
265
+ TESTGUARD_PYTHON="$RUNNER_TEMP/bare/bin/python" \
266
+ npx vitest run test/runner-python.test.mjs test/pyimports.test.mjs test/scaffold.python.test.mjs test/probe.python.test.mjs
267
+
268
+ - name: probe the Python fixture through the bin, stdlib engine
269
+ shell: bash
270
+ run: |
271
+ set -o pipefail
272
+ S=$(mktemp -d)
273
+ cp -R fixtures/known-answer-python/. "$S/"
274
+ rm -rf "$S/.flake-counter" "$S/.testguard" "$S/.pytest_cache"
275
+ find "$S" -name __pycache__ -type d -exec rm -rf {} + 2>/dev/null || true
276
+ git -C "$S" init -q
277
+ git -C "$S" -c user.email=ci@example.invalid -c user.name=ci add -A
278
+ git -C "$S" -c user.email=ci@example.invalid -c user.name=ci commit -qm fixture
279
+ status=0; node cli/testguard.mjs probe "$S" --budget 20000 --quiet --no-escalate --python "$RUNNER_TEMP/bare/bin/python" || status=$?
280
+ [ "$status" -eq 1 ] || { echo "::error::expected exit 1, got $status"; exit 1; }
281
+ node .github/scripts/check-python-fixture.mjs "$S/.testguard/evidence.json" unittest
282
+
283
+ - name: install pytest into a venv, and reach the same verdicts through it
284
+ shell: bash
285
+ run: |
286
+ set -o pipefail
287
+ python3 -m venv "$RUNNER_TEMP/pyenv"
288
+ "$RUNNER_TEMP/pyenv/bin/pip" install -q pytest
289
+ S=$(mktemp -d)
290
+ cp -R fixtures/known-answer-python/. "$S/"
291
+ rm -rf "$S/.flake-counter" "$S/.testguard" "$S/.pytest_cache"
292
+ find "$S" -name __pycache__ -type d -exec rm -rf {} + 2>/dev/null || true
293
+ git -C "$S" init -q
294
+ git -C "$S" -c user.email=ci@example.invalid -c user.name=ci add -A
295
+ git -C "$S" -c user.email=ci@example.invalid -c user.name=ci commit -qm fixture
296
+ status=0; node cli/testguard.mjs probe "$S" --budget 20000 --quiet --no-escalate \
297
+ --runner pytest --python "$RUNNER_TEMP/pyenv/bin/python" || status=$?
298
+ [ "$status" -eq 1 ] || { echo "::error::expected exit 1, got $status"; exit 1; }
299
+ node .github/scripts/check-python-fixture.mjs "$S/.testguard/evidence.json" pytest
300
+
301
+ - name: the acceptance suite again, now against an interpreter that HAS pytest
302
+ shell: bash
303
+ run: |
304
+ TESTGUARD_PYTHON="$RUNNER_TEMP/pyenv/bin/python" npx vitest run test/probe.python.test.mjs
305
+
206
306
  # ── Playwright: the per-file runner is proven against its own browserless
207
307
  # fixture, whose dependencies live in the fixture (never in the tool's
208
308
  # package.json). Installed and run here only. ──
@@ -80,7 +80,7 @@ jobs:
80
80
  - name: Install smoke — the tarball must run with production deps only
81
81
  run: npm run test:install
82
82
  - name: Self-probe must be fully defended before anything ships
83
- run: node cli/testguard.mjs probe . --budget 180000 --out "$RUNNER_TEMP/self-evidence.json" --quiet
83
+ run: node cli/testguard.mjs probe . --budget 180000 --out "$RUNNER_TEMP/self-evidence.json" --quiet --cost
84
84
  - name: Version surfaces are in sync
85
85
  run: node .github/scripts/sync-release-version.mjs --check
86
86
 
@@ -14,6 +14,7 @@ coverage/
14
14
  # Local agent tooling — not part of the public project
15
15
  .wolf/
16
16
  .claude/
17
+ .agent/
17
18
  CLAUDE.local.md
18
19
  graphify-out/
19
20
  .vitest/
@@ -22,3 +23,8 @@ dist/
22
23
  build/
23
24
  *.egg-info/
24
25
  __pycache__/
26
+
27
+ # Never commit secrets
28
+ .env
29
+ .env.*
30
+ !.env.example
@@ -20,3 +20,7 @@ CLAUDE.md
20
20
  CLAUDE.local.md
21
21
  AGENTS.md
22
22
  .DS_Store
23
+ # Python bytecode: version-specific, never shipped. The runner sets
24
+ # PYTHONDONTWRITEBYTECODE so none is produced, but a stray one must not pack.
25
+ __pycache__/
26
+ *.pyc
@@ -0,0 +1,40 @@
1
+ # Local hooks for contributors. Opt in once per clone:
2
+ #
3
+ # pre-commit install # doc + claim checks on commit
4
+ # pre-commit install --hook-type pre-push # the probe, before you push
5
+ #
6
+ # These are the same checks CI runs. CI is authoritative; this is the fast
7
+ # feedback loop. Nothing here reaches the network.
8
+ repos:
9
+ - repo: local
10
+ hooks:
11
+ - id: docguard-guard
12
+ name: DocGuard — docs match code
13
+ description: Validates this repository against docs-canonical/. Fails on errors, passes on warnings (exit 2). Needs docguard on PATH.
14
+ entry: bash -c 'code=0; docguard guard || code=$?; if [ "$code" -eq 2 ]; then exit 0; fi; exit "$code"'
15
+ language: system
16
+ pass_filenames: false
17
+ always_run: true
18
+
19
+ - id: testguard-claims
20
+ name: TestGuard — claims file valid, no annotation drift
21
+ entry: node cli/testguard.mjs claims .
22
+ language: system
23
+ pass_filenames: false
24
+ always_run: true
25
+
26
+ - id: testguard-gate
27
+ name: TestGuard — every changed source file carries a claim
28
+ entry: node cli/testguard.mjs gate . --changed HEAD --include-dirty --quiet
29
+ language: system
30
+ pass_filenames: false
31
+ always_run: true
32
+
33
+ - id: testguard-probe
34
+ name: TestGuard — probe the claims (pre-push)
35
+ description: Slow by design. Runs the faults and fails on any the tests miss.
36
+ entry: node cli/testguard.mjs probe . --quiet
37
+ language: system
38
+ pass_filenames: false
39
+ always_run: true
40
+ stages: [pre-push]
@@ -1,5 +1,7 @@
1
1
  # AI Agent Instructions — TestGuard
2
2
 
3
+ <!-- docguard:last-reviewed 2026-09-18 -->
4
+
3
5
  Read this before changing anything. It is short on purpose.
4
6
 
5
7
  ## What this is
@@ -32,7 +34,10 @@ node cli/testguard.mjs scaffold src/probe/classify.mjs --json # what the produ
32
34
  - `fixtures/known-answer/expected.json` — the oracle. Never edit it to match output.
33
35
  - `testguard.claims.json` — claims about this codebase, probed in CI.
34
36
  - `src/scaffold/producers.mjs` — the nine fault shapes. Deterministic line heuristics; no AST, no LLM. A new shape needs a synthetic-file test and a README row.
35
- - `src/probe/runners/` — one module per runner (`name`, `testGlobs`, `check`, `tests`, `run`, optionally `owns` for a per-file runner) over `shared.mjs` (jest-compatible report parsing, budgeted process runner, pluggable `parse`/`env`). `playwright.mjs` is a per-file runner: it owns the files under its `testDir`; `index.mjs` partitions defenders by runner and merges their runs pessimistically. A new runner needs its own fixture copy with its own `expected.json`, verified by hand.
37
+ - `src/probe/runners/` — one module per runner (`name`, `testGlobs`, `check`, `tests`, `run`, optionally `owns` for a per-file runner) over `shared.mjs` (jest-compatible report parsing, budgeted process runner, pluggable `parse`/`env`). `check`/`run` also receive `sourceDir` (the real project directory, for a toolchain the scratch worktree does not contain) and `targets` (the fault's files); `check` may return `engine`, and the engine is what the evidence records as `runner.name`. `playwright.mjs` and `python.mjs` are per-file runners: Playwright owns the files under its `testDir`, Python owns every `.py`; `index.mjs` partitions defenders by runner and merges their runs pessimistically. A new runner needs its own fixture copy with its own `expected.json`, verified by hand.
38
+ - `src/probe/runners/python/` — the injected Python reporters (`_testguard_report.py`, a pytest plugin, a stdlib `unittest` main). They reach the interpreter through `PYTHONPATH`, never through an install into the project under test; changing their report shape is a change to `python.mjs`'s `parseReport` in the same PR. They also report **import provenance**, which `probe.mjs` checks once the fault is live — see `checkProvenance` and `GATE-SEMANTICS.md` rule 9.
39
+ - `src/probe/pyimports.mjs` — Python defender discovery, patch-awareness and blast radius, matched by module name. A `patch("pkg.mod.fn")` is NOT a `vi.mock`: it leaves the file a defender. Only a patch of the module itself removes one.
40
+ - `src/scaffold/producers.python.mjs` — the Python fault shapes. A statement is removed with `pass`, never by deleting the line, and a line that leaves a bracket open is never removed: a fault that cannot compile is a probe run that says nothing.
36
41
  - `src/gate/changed.mjs` — claim coverage of a change (`gate --changed`). File-level, delta-only; an uncovered file exits 1; every ignore reliance is reported. Its rules are a section of `GATE-SEMANTICS.md`.
37
42
  - `src/status/status.mjs` — the state machine every rendering derives from. A new state or action is a spec change (`status.schema.json`) and a skill-template change (`src/init/templates/SKILL.md`) in the same PR.
38
43