@harness-engineering/cli 1.13.0 → 1.13.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/agents/skills/claude-code/add-harness-component/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/align-documentation/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/check-mechanical-constraints/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/cleanup-dead-code/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/detect-doc-drift/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/enforce-architecture/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-accessibility/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-api-design/SKILL.md +304 -0
- package/dist/agents/skills/claude-code/harness-api-design/skill.yaml +74 -0
- package/dist/agents/skills/claude-code/harness-architecture-advisor/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-auth/SKILL.md +279 -0
- package/dist/agents/skills/claude-code/harness-auth/skill.yaml +81 -0
- package/dist/agents/skills/claude-code/harness-autopilot/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-brainstorming/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-caching/SKILL.md +309 -0
- package/dist/agents/skills/claude-code/harness-caching/skill.yaml +73 -0
- package/dist/agents/skills/claude-code/harness-chaos/SKILL.md +295 -0
- package/dist/agents/skills/claude-code/harness-chaos/skill.yaml +72 -0
- package/dist/agents/skills/claude-code/harness-code-review/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-codebase-cleanup/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-compliance/SKILL.md +303 -0
- package/dist/agents/skills/claude-code/harness-compliance/skill.yaml +78 -0
- package/dist/agents/skills/claude-code/harness-containerization/SKILL.md +284 -0
- package/dist/agents/skills/claude-code/harness-containerization/skill.yaml +80 -0
- package/dist/agents/skills/claude-code/harness-data-pipeline/SKILL.md +274 -0
- package/dist/agents/skills/claude-code/harness-data-pipeline/skill.yaml +81 -0
- package/dist/agents/skills/claude-code/harness-data-validation/SKILL.md +343 -0
- package/dist/agents/skills/claude-code/harness-data-validation/skill.yaml +75 -0
- package/dist/agents/skills/claude-code/harness-database/SKILL.md +258 -0
- package/dist/agents/skills/claude-code/harness-database/skill.yaml +80 -0
- package/dist/agents/skills/claude-code/harness-debugging/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-dependency-health/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-deployment/SKILL.md +255 -0
- package/dist/agents/skills/claude-code/harness-deployment/skill.yaml +77 -0
- package/dist/agents/skills/claude-code/harness-design/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-design-mobile/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-design-system/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-design-web/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-diagnostics/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-docs-pipeline/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-dx/SKILL.md +276 -0
- package/dist/agents/skills/claude-code/harness-dx/skill.yaml +76 -0
- package/dist/agents/skills/claude-code/harness-e2e/SKILL.md +245 -0
- package/dist/agents/skills/claude-code/harness-e2e/skill.yaml +78 -0
- package/dist/agents/skills/claude-code/harness-event-driven/SKILL.md +280 -0
- package/dist/agents/skills/claude-code/harness-event-driven/skill.yaml +77 -0
- package/dist/agents/skills/claude-code/harness-execution/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-feature-flags/SKILL.md +287 -0
- package/dist/agents/skills/claude-code/harness-feature-flags/skill.yaml +74 -0
- package/dist/agents/skills/claude-code/harness-git-workflow/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-hotspot-detector/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-i18n/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-i18n-process/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-i18n-workflow/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-impact-analysis/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-incident-response/SKILL.md +223 -0
- package/dist/agents/skills/claude-code/harness-incident-response/skill.yaml +78 -0
- package/dist/agents/skills/claude-code/harness-infrastructure-as-code/SKILL.md +279 -0
- package/dist/agents/skills/claude-code/harness-infrastructure-as-code/skill.yaml +80 -0
- package/dist/agents/skills/claude-code/harness-integration-test/SKILL.md +271 -0
- package/dist/agents/skills/claude-code/harness-integration-test/skill.yaml +73 -0
- package/dist/agents/skills/claude-code/harness-integrity/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-knowledge-mapper/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-load-testing/SKILL.md +274 -0
- package/dist/agents/skills/claude-code/harness-load-testing/skill.yaml +79 -0
- package/dist/agents/skills/claude-code/harness-ml-ops/SKILL.md +341 -0
- package/dist/agents/skills/claude-code/harness-ml-ops/skill.yaml +79 -0
- package/dist/agents/skills/claude-code/harness-mobile-patterns/SKILL.md +326 -0
- package/dist/agents/skills/claude-code/harness-mobile-patterns/skill.yaml +82 -0
- package/dist/agents/skills/claude-code/harness-mutation-test/SKILL.md +251 -0
- package/dist/agents/skills/claude-code/harness-mutation-test/skill.yaml +70 -0
- package/dist/agents/skills/claude-code/harness-observability/SKILL.md +283 -0
- package/dist/agents/skills/claude-code/harness-observability/skill.yaml +78 -0
- package/dist/agents/skills/claude-code/harness-onboarding/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-parallel-agents/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-perf/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-perf-tdd/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-planning/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-pre-commit-review/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-product-spec/SKILL.md +285 -0
- package/dist/agents/skills/claude-code/harness-product-spec/skill.yaml +72 -0
- package/dist/agents/skills/claude-code/harness-property-test/SKILL.md +281 -0
- package/dist/agents/skills/claude-code/harness-property-test/skill.yaml +71 -0
- package/dist/agents/skills/claude-code/harness-refactoring/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-release-readiness/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-resilience/SKILL.md +255 -0
- package/dist/agents/skills/claude-code/harness-resilience/skill.yaml +76 -0
- package/dist/agents/skills/claude-code/harness-roadmap/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-secrets/SKILL.md +293 -0
- package/dist/agents/skills/claude-code/harness-secrets/skill.yaml +76 -0
- package/dist/agents/skills/claude-code/harness-security-review/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-security-scan/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-skill-authoring/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-soundness-review/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-sql-review/SKILL.md +315 -0
- package/dist/agents/skills/claude-code/harness-sql-review/skill.yaml +74 -0
- package/dist/agents/skills/claude-code/harness-state-management/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-tdd/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-test-advisor/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-test-data/SKILL.md +268 -0
- package/dist/agents/skills/claude-code/harness-test-data/skill.yaml +74 -0
- package/dist/agents/skills/claude-code/harness-ux-copy/SKILL.md +271 -0
- package/dist/agents/skills/claude-code/harness-ux-copy/skill.yaml +77 -0
- package/dist/agents/skills/claude-code/harness-verification/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-verify/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/harness-visual-regression/SKILL.md +257 -0
- package/dist/agents/skills/claude-code/harness-visual-regression/skill.yaml +74 -0
- package/dist/agents/skills/claude-code/initialize-harness-project/skill.yaml +1 -0
- package/dist/agents/skills/claude-code/validate-context-engineering/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/add-harness-component/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/align-documentation/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/check-mechanical-constraints/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/cleanup-dead-code/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/detect-doc-drift/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/enforce-architecture/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-accessibility/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-api-design/SKILL.md +304 -0
- package/dist/agents/skills/gemini-cli/harness-api-design/skill.yaml +74 -0
- package/dist/agents/skills/gemini-cli/harness-architecture-advisor/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-auth/SKILL.md +279 -0
- package/dist/agents/skills/gemini-cli/harness-auth/skill.yaml +81 -0
- package/dist/agents/skills/gemini-cli/harness-autopilot/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-brainstorming/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-caching/SKILL.md +309 -0
- package/dist/agents/skills/gemini-cli/harness-caching/skill.yaml +73 -0
- package/dist/agents/skills/gemini-cli/harness-chaos/SKILL.md +295 -0
- package/dist/agents/skills/gemini-cli/harness-chaos/skill.yaml +72 -0
- package/dist/agents/skills/gemini-cli/harness-code-review/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-codebase-cleanup/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-compliance/SKILL.md +303 -0
- package/dist/agents/skills/gemini-cli/harness-compliance/skill.yaml +78 -0
- package/dist/agents/skills/gemini-cli/harness-containerization/SKILL.md +284 -0
- package/dist/agents/skills/gemini-cli/harness-containerization/skill.yaml +80 -0
- package/dist/agents/skills/gemini-cli/harness-data-pipeline/SKILL.md +274 -0
- package/dist/agents/skills/gemini-cli/harness-data-pipeline/skill.yaml +81 -0
- package/dist/agents/skills/gemini-cli/harness-data-validation/SKILL.md +343 -0
- package/dist/agents/skills/gemini-cli/harness-data-validation/skill.yaml +75 -0
- package/dist/agents/skills/gemini-cli/harness-database/SKILL.md +258 -0
- package/dist/agents/skills/gemini-cli/harness-database/skill.yaml +80 -0
- package/dist/agents/skills/gemini-cli/harness-debugging/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-dependency-health/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-deployment/SKILL.md +255 -0
- package/dist/agents/skills/gemini-cli/harness-deployment/skill.yaml +77 -0
- package/dist/agents/skills/gemini-cli/harness-design/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-design-mobile/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-design-system/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-design-web/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-diagnostics/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-docs-pipeline/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-dx/SKILL.md +276 -0
- package/dist/agents/skills/gemini-cli/harness-dx/skill.yaml +76 -0
- package/dist/agents/skills/gemini-cli/harness-e2e/SKILL.md +245 -0
- package/dist/agents/skills/gemini-cli/harness-e2e/skill.yaml +78 -0
- package/dist/agents/skills/gemini-cli/harness-event-driven/SKILL.md +280 -0
- package/dist/agents/skills/gemini-cli/harness-event-driven/skill.yaml +77 -0
- package/dist/agents/skills/gemini-cli/harness-execution/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-feature-flags/SKILL.md +287 -0
- package/dist/agents/skills/gemini-cli/harness-feature-flags/skill.yaml +74 -0
- package/dist/agents/skills/gemini-cli/harness-git-workflow/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-hotspot-detector/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-i18n/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-i18n-process/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-i18n-workflow/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-impact-analysis/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-incident-response/SKILL.md +223 -0
- package/dist/agents/skills/gemini-cli/harness-incident-response/skill.yaml +78 -0
- package/dist/agents/skills/gemini-cli/harness-infrastructure-as-code/SKILL.md +279 -0
- package/dist/agents/skills/gemini-cli/harness-infrastructure-as-code/skill.yaml +80 -0
- package/dist/agents/skills/gemini-cli/harness-integration-test/SKILL.md +271 -0
- package/dist/agents/skills/gemini-cli/harness-integration-test/skill.yaml +73 -0
- package/dist/agents/skills/gemini-cli/harness-integrity/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-knowledge-mapper/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-load-testing/SKILL.md +274 -0
- package/dist/agents/skills/gemini-cli/harness-load-testing/skill.yaml +79 -0
- package/dist/agents/skills/gemini-cli/harness-ml-ops/SKILL.md +341 -0
- package/dist/agents/skills/gemini-cli/harness-ml-ops/skill.yaml +79 -0
- package/dist/agents/skills/gemini-cli/harness-mobile-patterns/SKILL.md +326 -0
- package/dist/agents/skills/gemini-cli/harness-mobile-patterns/skill.yaml +82 -0
- package/dist/agents/skills/gemini-cli/harness-mutation-test/SKILL.md +251 -0
- package/dist/agents/skills/gemini-cli/harness-mutation-test/skill.yaml +70 -0
- package/dist/agents/skills/gemini-cli/harness-observability/SKILL.md +283 -0
- package/dist/agents/skills/gemini-cli/harness-observability/skill.yaml +78 -0
- package/dist/agents/skills/gemini-cli/harness-onboarding/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-parallel-agents/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-perf/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-perf-tdd/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-planning/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-pre-commit-review/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-product-spec/SKILL.md +285 -0
- package/dist/agents/skills/gemini-cli/harness-product-spec/skill.yaml +72 -0
- package/dist/agents/skills/gemini-cli/harness-property-test/SKILL.md +281 -0
- package/dist/agents/skills/gemini-cli/harness-property-test/skill.yaml +71 -0
- package/dist/agents/skills/gemini-cli/harness-refactoring/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-release-readiness/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-resilience/SKILL.md +255 -0
- package/dist/agents/skills/gemini-cli/harness-resilience/skill.yaml +76 -0
- package/dist/agents/skills/gemini-cli/harness-roadmap/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-secrets/SKILL.md +293 -0
- package/dist/agents/skills/gemini-cli/harness-secrets/skill.yaml +76 -0
- package/dist/agents/skills/gemini-cli/harness-security-review/SKILL.md +240 -0
- package/dist/agents/skills/gemini-cli/harness-security-review/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-security-scan/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-skill-authoring/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-soundness-review/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-sql-review/SKILL.md +315 -0
- package/dist/agents/skills/gemini-cli/harness-sql-review/skill.yaml +74 -0
- package/dist/agents/skills/gemini-cli/harness-state-management/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-tdd/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-test-advisor/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-test-data/SKILL.md +268 -0
- package/dist/agents/skills/gemini-cli/harness-test-data/skill.yaml +74 -0
- package/dist/agents/skills/gemini-cli/harness-ux-copy/SKILL.md +271 -0
- package/dist/agents/skills/gemini-cli/harness-ux-copy/skill.yaml +77 -0
- package/dist/agents/skills/gemini-cli/harness-verification/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-verify/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/harness-visual-regression/SKILL.md +257 -0
- package/dist/agents/skills/gemini-cli/harness-visual-regression/skill.yaml +74 -0
- package/dist/agents/skills/gemini-cli/initialize-harness-project/skill.yaml +1 -0
- package/dist/agents/skills/gemini-cli/validate-context-engineering/skill.yaml +1 -0
- package/dist/{agents-md-P2RHSUV7.js → agents-md-XU3BHE22.js} +1 -1
- package/dist/{architecture-ESOOE26S.js → architecture-2R5Z4ZAF.js} +2 -2
- package/dist/bin/harness-mcp.js +14 -13
- package/dist/bin/harness.js +22 -21
- package/dist/{check-phase-gate-S2MZKLFQ.js → check-phase-gate-2OFZ7OWW.js} +3 -2
- package/dist/{chunk-LD3DKUK5.js → chunk-4ZMOCPYO.js} +1 -1
- package/dist/{chunk-5VY23YK3.js → chunk-65FRIL4D.js} +2 -2
- package/dist/{chunk-L2KLU56K.js → chunk-AOZRDOIP.js} +2 -2
- package/dist/{chunk-MACVXDZK.js → chunk-DZS7CJKL.js} +4 -4
- package/dist/{chunk-7PZWR4LI.js → chunk-IM32EEDM.js} +9 -9
- package/dist/{chunk-2YPZKGAG.js → chunk-IMFVFNJE.js} +1 -1
- package/dist/{chunk-HD4IBGLA.js → chunk-N5G5QMS3.js} +24 -1
- package/dist/{chunk-MI5XJQDY.js → chunk-ND6PNADU.js} +23 -9
- package/dist/{chunk-7KQSUZVG.js → chunk-NERR4TAO.js} +729 -436
- package/dist/{chunk-PSNN4LWX.js → chunk-NOPU4RZ4.js} +2 -2
- package/dist/{chunk-KELT6K6M.js → chunk-PQ5YK4AY.js} +287 -258
- package/dist/{chunk-WPPDRIJL.js → chunk-QY4T6YAZ.js} +3 -3
- package/dist/{chunk-RZSUJBZZ.js → chunk-SSKDAOX5.js} +31 -28
- package/dist/{chunk-2VU4MFM3.js → chunk-TKJZKICB.js} +6 -6
- package/dist/{chunk-GNGELAXY.js → chunk-TS3XWPW5.js} +1 -1
- package/dist/chunk-UAX4I5ZE.js +217 -0
- package/dist/{chunk-VRFZWGMS.js → chunk-XYLGHKG6.js} +5 -1
- package/dist/{chunk-6N4R6FVX.js → chunk-YBJ262QL.js} +1 -1
- package/dist/{chunk-3KOLLWWE.js → chunk-Z77YQRQT.js} +11 -207
- package/dist/{ci-workflow-4NYBUG6R.js → ci-workflow-EHV65NQB.js} +1 -1
- package/dist/{create-skill-WPXHSLX2.js → create-skill-XSWHMSM5.js} +2 -2
- package/dist/{dist-WF4C7A4A.js → dist-2B363XUH.js} +1 -1
- package/dist/{dist-M6BQODWC.js → dist-HXHWB7SV.js} +2 -2
- package/dist/{docs-BPYCN2DR.js → docs-FZOPM4GK.js} +4 -2
- package/dist/{engine-LXLIWQQ3.js → engine-OL4T6NZS.js} +1 -1
- package/dist/{entropy-4VDVV5CR.js → entropy-LVHJMFGH.js} +2 -2
- package/dist/{feedback-63QB5RCA.js → feedback-IHLVLMRD.js} +1 -1
- package/dist/{generate-agent-definitions-QABOJG56.js → generate-agent-definitions-64S3CLEZ.js} +3 -3
- package/dist/{glob-helper-5OHBUQAI.js → glob-helper-R5FXNUPS.js} +1 -1
- package/dist/{graph-loader-KO4GJ5N2.js → graph-loader-GJZ4FN4Y.js} +1 -1
- package/dist/index.d.ts +35 -8
- package/dist/index.js +23 -21
- package/dist/{loader-Z2IT7QX3.js → loader-DPYFB6R6.js} +1 -1
- package/dist/{mcp-KQHEL5IF.js → mcp-JQUI7BVZ.js} +14 -13
- package/dist/{performance-26BH47O4.js → performance-ZTVSUANN.js} +2 -2
- package/dist/{review-pipeline-GHR3WFBI.js → review-pipeline-76JHKGSV.js} +1 -1
- package/dist/{runtime-PDWD7UIK.js → runtime-X7U6SC7K.js} +1 -1
- package/dist/{security-UQFUZXEN.js → security-FWQZF2IZ.js} +1 -1
- package/dist/skill-executor-XZLYZYAK.js +8 -0
- package/dist/{validate-N7QJOKFZ.js → validate-GCHZJIL7.js} +2 -2
- package/dist/{validate-cross-check-EDQ5QGTM.js → validate-cross-check-STFHYMAZ.js} +1 -1
- package/package.json +3 -3
- package/dist/skill-executor-RG45LUO5.js +0 -8
|
@@ -0,0 +1,285 @@
|
|
|
1
|
+
# Harness Product Spec
|
|
2
|
+
|
|
3
|
+
> Generate structured product specifications from feature requests, issues, or descriptions. Produces user stories with EARS acceptance criteria, Given-When-Then scenarios, and PRD documents with traceable requirements.
|
|
4
|
+
|
|
5
|
+
## When to Use
|
|
6
|
+
|
|
7
|
+
- When a new feature needs formal specification before implementation begins
|
|
8
|
+
- When GitHub issues or feature requests need to be translated into actionable user stories
|
|
9
|
+
- When acceptance criteria are missing, vague, or untestable for existing stories
|
|
10
|
+
- NOT for technical architecture decisions (use harness-architecture-advisor)
|
|
11
|
+
- NOT for implementation planning with task breakdown (use harness-planning)
|
|
12
|
+
- NOT for bug triage or root cause analysis (use harness-debugging)
|
|
13
|
+
|
|
14
|
+
## Process
|
|
15
|
+
|
|
16
|
+
### Phase 1: PARSE -- Extract Feature Intent
|
|
17
|
+
|
|
18
|
+
1. **Resolve input source.** Accept one of:
|
|
19
|
+
- GitHub issue URL: fetch via `gh issue view <number> --json title,body,labels,comments`
|
|
20
|
+
- Feature description file: read the provided file path
|
|
21
|
+
- Inline text: use the provided description directly
|
|
22
|
+
|
|
23
|
+
2. **Extract core elements.** From the input, identify:
|
|
24
|
+
- **Goal:** What is the user trying to accomplish?
|
|
25
|
+
- **Actor(s):** Who are the users or systems involved? (e.g., "admin user," "API consumer," "billing system")
|
|
26
|
+
- **Trigger:** What initiates the feature? (user action, system event, time-based)
|
|
27
|
+
- **Constraints:** What limitations exist? (performance, platform, backward compatibility)
|
|
28
|
+
- **Context:** What existing system components are involved?
|
|
29
|
+
|
|
30
|
+
3. **Identify ambiguities.** Flag any element that is missing or unclear:
|
|
31
|
+
- "This issue mentions 'notifications' but does not specify the channel (email, in-app, push)"
|
|
32
|
+
- "No success metric defined -- what does 'working correctly' mean?"
|
|
33
|
+
- "Edge case not addressed: what happens when the user has no payment method?"
|
|
34
|
+
|
|
35
|
+
4. **Resolve ambiguities.** Use `emit_interaction` to present questions when critical information is missing:
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
The feature request mentions "user notifications" but does not specify:
|
|
39
|
+
1. Notification channel (email, in-app, push, SMS)
|
|
40
|
+
2. Whether notifications are configurable by the user
|
|
41
|
+
3. Retry behavior for failed deliveries
|
|
42
|
+
Please clarify before proceeding.
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
5. **Load project context.** Scan the project for existing specs, user stories, or PRDs to maintain consistency in format and terminology:
|
|
46
|
+
- Check `docs/specs/`, `docs/requirements/`, `docs/prd/` for existing documents
|
|
47
|
+
- Check `.github/ISSUE_TEMPLATE/` for the project's preferred issue format
|
|
48
|
+
- Identify domain terminology used in existing specs
|
|
49
|
+
|
|
50
|
+
6. **Classify feature type.** Categorize the feature as:
|
|
51
|
+
- **New capability:** Something the system cannot do today
|
|
52
|
+
- **Enhancement:** Improvement to an existing capability
|
|
53
|
+
- **Integration:** Connecting with an external system
|
|
54
|
+
- **Migration:** Moving from one approach to another
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
### Phase 2: CRAFT -- Generate User Stories and Acceptance Criteria
|
|
59
|
+
|
|
60
|
+
1. **Write user stories.** For each actor-goal pair, produce a story in standard format:
|
|
61
|
+
|
|
62
|
+
```
|
|
63
|
+
As a [actor],
|
|
64
|
+
I want to [action],
|
|
65
|
+
so that [benefit].
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
Break large features into multiple stories. Each story must be independently deliverable and testable.
|
|
69
|
+
|
|
70
|
+
2. **Write EARS acceptance criteria.** Apply the EARS (Easy Approach to Requirements Syntax) patterns:
|
|
71
|
+
- **Ubiquitous:** "The [system] shall [behavior]" -- for unconditional requirements
|
|
72
|
+
- **Event-driven:** "When [trigger], the [system] shall [behavior]" -- for responses to events
|
|
73
|
+
- **State-driven:** "While [state], the [system] shall [behavior]" -- for ongoing conditions
|
|
74
|
+
- **Optional:** "Where [feature is enabled], the [system] shall [behavior]" -- for configurable behavior
|
|
75
|
+
- **Unwanted:** "If [condition], then the [system] shall [response]" -- for error handling and edge cases
|
|
76
|
+
|
|
77
|
+
3. **Write Given-When-Then scenarios.** For each acceptance criterion, produce at least one BDD scenario:
|
|
78
|
+
|
|
79
|
+
```
|
|
80
|
+
Given [precondition],
|
|
81
|
+
When [action],
|
|
82
|
+
Then [expected outcome].
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Include:
|
|
86
|
+
- Happy path scenario
|
|
87
|
+
- At least one error/edge case scenario
|
|
88
|
+
- Boundary condition scenarios where applicable
|
|
89
|
+
|
|
90
|
+
4. **Define edge cases.** For each story, enumerate:
|
|
91
|
+
- What happens with empty input?
|
|
92
|
+
- What happens with maximum input?
|
|
93
|
+
- What happens when the user lacks permission?
|
|
94
|
+
- What happens during concurrent access?
|
|
95
|
+
- What happens when a dependency is unavailable?
|
|
96
|
+
|
|
97
|
+
5. **Assign story metadata.** For each story:
|
|
98
|
+
- **Priority:** Must-have, Should-have, Could-have, Won't-have (MoSCoW)
|
|
99
|
+
- **Size estimate:** S, M, L, XL (relative to other stories)
|
|
100
|
+
- **Dependencies:** Other stories or systems this depends on
|
|
101
|
+
- **Risk:** Low, Medium, High (with risk description)
|
|
102
|
+
|
|
103
|
+
---
|
|
104
|
+
|
|
105
|
+
### Phase 3: GENERATE -- Produce PRD Document
|
|
106
|
+
|
|
107
|
+
1. **Structure the PRD.** Generate a document with these sections:
|
|
108
|
+
- **Title and version** (feature name, PRD version, date, author)
|
|
109
|
+
- **Problem statement** (what problem does this solve, who has it, how painful is it)
|
|
110
|
+
- **Goals and non-goals** (explicit scope boundaries)
|
|
111
|
+
- **User stories** (from Phase 2, organized by priority)
|
|
112
|
+
- **Acceptance criteria** (EARS format, traceable to stories)
|
|
113
|
+
- **Technical constraints** (performance requirements, platform constraints, backward compatibility)
|
|
114
|
+
- **Success metrics** (measurable outcomes that define "done")
|
|
115
|
+
- **Open questions** (unresolved ambiguities from Phase 1)
|
|
116
|
+
- **Out of scope** (explicitly excluded items)
|
|
117
|
+
|
|
118
|
+
2. **Write the problem statement.** Include:
|
|
119
|
+
- Who is affected (specific user segments)
|
|
120
|
+
- How the problem manifests today (current workaround or pain)
|
|
121
|
+
- Quantified impact if available (time lost, error rate, support tickets)
|
|
122
|
+
|
|
123
|
+
3. **Define success metrics.** Every metric must be:
|
|
124
|
+
- **Measurable:** Can be tracked with existing or planned instrumentation
|
|
125
|
+
- **Time-bound:** Has a target timeline for evaluation
|
|
126
|
+
- **Specific:** Not "improve user experience" but "reduce checkout abandonment by 15% within 30 days"
|
|
127
|
+
|
|
128
|
+
4. **Map requirements to stories.** Create a traceability matrix:
|
|
129
|
+
|
|
130
|
+
```
|
|
131
|
+
REQ-001 -> US-001, US-003 (must-have)
|
|
132
|
+
REQ-002 -> US-002 (should-have)
|
|
133
|
+
REQ-003 -> US-004, US-005 (could-have)
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
5. **Write the PRD to file.** Save to the project's spec directory (detected in Phase 1 or defaulting to `docs/specs/`). Use a filename pattern: `YYYY-MM-DD-feature-name-prd.md`.
|
|
137
|
+
|
|
138
|
+
---
|
|
139
|
+
|
|
140
|
+
### Phase 4: VALIDATE -- Verify Completeness and Testability
|
|
141
|
+
|
|
142
|
+
1. **Check story independence.** Verify each user story can be delivered independently:
|
|
143
|
+
- Does the story depend on another story being completed first?
|
|
144
|
+
- If yes, is the dependency documented?
|
|
145
|
+
- Can the story be tested in isolation?
|
|
146
|
+
|
|
147
|
+
2. **Check acceptance criteria testability.** Every EARS criterion must be verifiable:
|
|
148
|
+
- Can an automated test be written for this criterion?
|
|
149
|
+
- Is the expected behavior specific enough to distinguish pass from fail?
|
|
150
|
+
- Are boundary values defined (not "handles large files" but "handles files up to 100MB")?
|
|
151
|
+
Flag untestable criteria: "Criterion AC-003 says 'the system should be fast' -- this is not testable. Recommend: 'the system shall respond within 200ms for the 95th percentile.'"
|
|
152
|
+
|
|
153
|
+
3. **Check coverage completeness.** Verify all parsed elements from Phase 1 are addressed:
|
|
154
|
+
- Every actor has at least one story
|
|
155
|
+
- Every constraint has a corresponding acceptance criterion
|
|
156
|
+
- Every ambiguity is either resolved or listed in open questions
|
|
157
|
+
- Error handling is specified for every user-facing action
|
|
158
|
+
|
|
159
|
+
4. **Check format consistency.** Verify the output matches existing project conventions:
|
|
160
|
+
- Story format matches templates in `.github/ISSUE_TEMPLATE/` if present
|
|
161
|
+
- Terminology matches existing specs (do not introduce "user" when the project uses "member")
|
|
162
|
+
- Priority scheme matches existing stories
|
|
163
|
+
|
|
164
|
+
5. **Output validation summary:**
|
|
165
|
+
|
|
166
|
+
```
|
|
167
|
+
Product Spec Validation: [COMPLETE/INCOMPLETE]
|
|
168
|
+
Stories: N generated (M must-have, K should-have)
|
|
169
|
+
Acceptance criteria: N (all testable: YES/NO)
|
|
170
|
+
BDD scenarios: N (covering N criteria)
|
|
171
|
+
Coverage: all actors covered, all constraints addressed
|
|
172
|
+
Open questions: N remaining
|
|
173
|
+
|
|
174
|
+
Generated: docs/specs/2026-03-27-notifications-prd.md
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
---
|
|
178
|
+
|
|
179
|
+
## Harness Integration
|
|
180
|
+
|
|
181
|
+
- **`harness skill run harness-product-spec`** -- Primary command for generating product specifications.
|
|
182
|
+
- **`harness validate`** -- Run after generating specs to verify project health.
|
|
183
|
+
- **`Bash`** -- Used to fetch GitHub issues via `gh` CLI and check existing spec files.
|
|
184
|
+
- **`Read`** -- Used to read input feature descriptions, existing specs, and issue templates.
|
|
185
|
+
- **`Write`** -- Used to generate PRD documents and user story files.
|
|
186
|
+
- **`Glob`** -- Used to locate existing spec directories, issue templates, and requirement documents.
|
|
187
|
+
- **`Grep`** -- Used to extract domain terminology from existing specs and find related stories.
|
|
188
|
+
- **`emit_interaction`** -- Used to present ambiguities for clarification and confirm spec structure before writing.
|
|
189
|
+
|
|
190
|
+
## Success Criteria
|
|
191
|
+
|
|
192
|
+
- Every feature input produces at least one user story with EARS acceptance criteria
|
|
193
|
+
- All acceptance criteria are testable (specific, measurable, with defined boundaries)
|
|
194
|
+
- BDD scenarios cover happy path and at least one error/edge case per criterion
|
|
195
|
+
- PRD document includes all required sections with traceable requirements
|
|
196
|
+
- Ambiguities are surfaced and either resolved or tracked as open questions
|
|
197
|
+
- Output format matches existing project conventions when they exist
|
|
198
|
+
- Generated PRD is saved to the correct directory with consistent naming
|
|
199
|
+
|
|
200
|
+
## Examples
|
|
201
|
+
|
|
202
|
+
### Example: GitHub Issue to PRD for Team Notifications
|
|
203
|
+
|
|
204
|
+
```
|
|
205
|
+
Phase 1: PARSE
|
|
206
|
+
Source: gh issue view 234 (title: "Add team notification preferences")
|
|
207
|
+
Actor: team admin, team member
|
|
208
|
+
Goal: control which notifications team members receive
|
|
209
|
+
Ambiguities found:
|
|
210
|
+
- Channel not specified (resolved: email + in-app per comment #3)
|
|
211
|
+
- "Important notifications" undefined (flagged as open question)
|
|
212
|
+
|
|
213
|
+
Phase 2: CRAFT
|
|
214
|
+
US-001: As a team admin, I want to set default notification preferences for my team,
|
|
215
|
+
so that new members receive appropriate notifications without manual setup.
|
|
216
|
+
AC-001 (Ubiquitous): The system shall apply team-default preferences to new members on join.
|
|
217
|
+
AC-002 (Event-driven): When a team admin updates default preferences, the system shall
|
|
218
|
+
prompt whether to apply to existing members.
|
|
219
|
+
AC-003 (Unwanted): If a team member has custom preferences, then the system shall
|
|
220
|
+
preserve them when team defaults change.
|
|
221
|
+
|
|
222
|
+
US-002: As a team member, I want to override team notification defaults,
|
|
223
|
+
so that I receive only notifications relevant to my role.
|
|
224
|
+
Scenario: Given a team member with default preferences,
|
|
225
|
+
When they disable "deployment" notifications,
|
|
226
|
+
Then they shall not receive deployment notifications
|
|
227
|
+
And their other preferences remain unchanged.
|
|
228
|
+
|
|
229
|
+
Phase 3: GENERATE
|
|
230
|
+
Written: docs/specs/2026-03-27-team-notifications-prd.md
|
|
231
|
+
Sections: problem statement, 4 user stories, 12 acceptance criteria, 8 BDD scenarios
|
|
232
|
+
Traceability: REQ-001 -> US-001, US-002 | REQ-002 -> US-003, US-004
|
|
233
|
+
|
|
234
|
+
Phase 4: VALIDATE
|
|
235
|
+
Stories: 4 (2 must-have, 1 should-have, 1 could-have)
|
|
236
|
+
Acceptance criteria: 12 (all testable: YES)
|
|
237
|
+
Open questions: 1 ("important notifications" needs product definition)
|
|
238
|
+
Result: COMPLETE
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
### Example: Inline Feature Description for Stripe Webhook Integration
|
|
242
|
+
|
|
243
|
+
```
|
|
244
|
+
Phase 1: PARSE
|
|
245
|
+
Source: inline text "We need to handle Stripe webhooks for subscription changes"
|
|
246
|
+
Actor: billing system (automated), finance admin (human oversight)
|
|
247
|
+
Constraints: idempotency required, webhook signature verification, 5-second response SLA
|
|
248
|
+
Ambiguities:
|
|
249
|
+
- Which subscription events? (resolved via clarification: created, updated, canceled, past_due)
|
|
250
|
+
- Retry handling? (Stripe retries for 72 hours)
|
|
251
|
+
|
|
252
|
+
Phase 2: CRAFT
|
|
253
|
+
US-001: As the billing system, I want to process Stripe subscription.updated webhooks,
|
|
254
|
+
so that user plan changes are reflected within 60 seconds.
|
|
255
|
+
AC-001 (Event-driven): When a subscription.updated webhook arrives, the system shall
|
|
256
|
+
update the user's plan within 60 seconds.
|
|
257
|
+
AC-002 (Unwanted): If a duplicate webhook event ID is received, then the system shall
|
|
258
|
+
return 200 OK without reprocessing.
|
|
259
|
+
AC-003 (Unwanted): If webhook signature verification fails, then the system shall
|
|
260
|
+
return 400 and log a security warning.
|
|
261
|
+
|
|
262
|
+
Phase 3: GENERATE
|
|
263
|
+
Written: docs/specs/2026-03-27-stripe-webhooks-prd.md
|
|
264
|
+
Technical constraints section includes: idempotency keys, signature verification,
|
|
265
|
+
5-second response SLA, Stripe retry behavior documentation
|
|
266
|
+
|
|
267
|
+
Phase 4: VALIDATE
|
|
268
|
+
All 4 webhook event types have stories: YES
|
|
269
|
+
Idempotency criterion is testable: YES (duplicate event ID -> no side effects)
|
|
270
|
+
Result: COMPLETE
|
|
271
|
+
```
|
|
272
|
+
|
|
273
|
+
## Gates
|
|
274
|
+
|
|
275
|
+
- **No generating specs from ambiguous input without clarification.** If the input lacks a clear actor, goal, or trigger, pause and ask. Do not invent requirements that were not stated or implied.
|
|
276
|
+
- **No untestable acceptance criteria.** Every criterion must be verifiable by an automated test or a specific manual procedure. "The system should be user-friendly" is not an acceptance criterion.
|
|
277
|
+
- **No skipping edge cases for user-facing actions.** Every action that a user can trigger must have at least one unwanted-behavior criterion (EARS "If" pattern) covering the error case.
|
|
278
|
+
- **No overwriting existing specs.** If a PRD already exists for this feature, present the diff rather than replacing the file. Existing specs may have been reviewed and approved.
|
|
279
|
+
|
|
280
|
+
## Escalation
|
|
281
|
+
|
|
282
|
+
- **When the feature request is too vague to parse:** Present what was extracted and what is missing: "This issue contains a title but no description. I need at minimum: who is the user, what action they want to perform, and why. Please add details to the issue or provide them here."
|
|
283
|
+
- **When requirements conflict with existing system behavior:** Flag the conflict: "AC-003 requires real-time sync, but the existing event system uses eventual consistency with up to 30-second delay. This needs an architectural decision before the spec can be finalized."
|
|
284
|
+
- **When the feature scope is too large for a single PRD:** Recommend splitting: "This feature contains 3 independent capabilities (notifications, preferences, audit log). Recommend splitting into 3 PRDs that can be prioritized and delivered independently."
|
|
285
|
+
- **When acceptance criteria require metrics that do not exist yet:** Flag the instrumentation gap: "Success metric 'reduce checkout time by 20%' requires checkout timing instrumentation that does not currently exist. Add an instrumentation story as a prerequisite."
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
name: harness-product-spec
|
|
2
|
+
version: "1.0.0"
|
|
3
|
+
description: User story generation, EARS acceptance criteria, and PRD creation from issues
|
|
4
|
+
cognitive_mode: constructive-architect
|
|
5
|
+
triggers:
|
|
6
|
+
- manual
|
|
7
|
+
- on_new_feature
|
|
8
|
+
platforms:
|
|
9
|
+
- claude-code
|
|
10
|
+
- gemini-cli
|
|
11
|
+
tools:
|
|
12
|
+
- Bash
|
|
13
|
+
- Read
|
|
14
|
+
- Write
|
|
15
|
+
- Edit
|
|
16
|
+
- Glob
|
|
17
|
+
- Grep
|
|
18
|
+
- emit_interaction
|
|
19
|
+
cli:
|
|
20
|
+
command: harness skill run harness-product-spec
|
|
21
|
+
args:
|
|
22
|
+
- name: path
|
|
23
|
+
description: Project root path
|
|
24
|
+
required: false
|
|
25
|
+
- name: source
|
|
26
|
+
description: "Input source: issue URL, feature description file, or inline text"
|
|
27
|
+
required: true
|
|
28
|
+
- name: format
|
|
29
|
+
description: "Output format: prd, user-stories, acceptance-criteria, all. Defaults to all."
|
|
30
|
+
required: false
|
|
31
|
+
mcp:
|
|
32
|
+
tool: run_skill
|
|
33
|
+
input:
|
|
34
|
+
skill: harness-product-spec
|
|
35
|
+
path: string
|
|
36
|
+
type: rigid
|
|
37
|
+
tier: 3
|
|
38
|
+
internal: false
|
|
39
|
+
keywords:
|
|
40
|
+
- product spec
|
|
41
|
+
- user story
|
|
42
|
+
- acceptance criteria
|
|
43
|
+
- EARS
|
|
44
|
+
- PRD
|
|
45
|
+
- requirements
|
|
46
|
+
- product requirements
|
|
47
|
+
- BDD
|
|
48
|
+
- given-when-then
|
|
49
|
+
- feature specification
|
|
50
|
+
stack_signals:
|
|
51
|
+
- "docs/specs/"
|
|
52
|
+
- "docs/requirements/"
|
|
53
|
+
- "specs/"
|
|
54
|
+
- ".github/ISSUE_TEMPLATE/"
|
|
55
|
+
- "docs/prd/"
|
|
56
|
+
phases:
|
|
57
|
+
- name: parse
|
|
58
|
+
description: Extract feature intent, stakeholders, and constraints from input source
|
|
59
|
+
required: true
|
|
60
|
+
- name: craft
|
|
61
|
+
description: Generate user stories with EARS acceptance criteria and edge cases
|
|
62
|
+
required: true
|
|
63
|
+
- name: generate
|
|
64
|
+
description: Produce PRD document with scope, requirements, and success metrics
|
|
65
|
+
required: true
|
|
66
|
+
- name: validate
|
|
67
|
+
description: Verify completeness, testability of acceptance criteria, and traceability to source
|
|
68
|
+
required: true
|
|
69
|
+
state:
|
|
70
|
+
persistent: false
|
|
71
|
+
files: []
|
|
72
|
+
depends_on: []
|
|
@@ -0,0 +1,281 @@
|
|
|
1
|
+
# Harness Property Test
|
|
2
|
+
|
|
3
|
+
> Property-based and generative testing with fast-check, hypothesis, and automatic shrinking. Discovers edge cases that example-based tests miss by generating thousands of random inputs and verifying invariants hold for all of them.
|
|
4
|
+
|
|
5
|
+
## When to Use
|
|
6
|
+
|
|
7
|
+
- Testing functions with large input spaces (parsers, serializers, encoders, validators)
|
|
8
|
+
- Verifying mathematical or algebraic properties (commutativity, associativity, round-trip encoding)
|
|
9
|
+
- Finding edge cases in data transformation, sorting, or filtering logic
|
|
10
|
+
- NOT when testing UI rendering or visual output (use harness-visual-regression instead)
|
|
11
|
+
- NOT when testing simple CRUD operations with well-defined inputs (use harness-tdd instead)
|
|
12
|
+
- NOT when testing external service integrations (use harness-integration-test instead)
|
|
13
|
+
|
|
14
|
+
## Process
|
|
15
|
+
|
|
16
|
+
### Phase 1: IDENTIFY -- Discover Testable Properties and Invariants
|
|
17
|
+
|
|
18
|
+
1. **Catalog candidate functions.** Search for functions that exhibit testable properties:
|
|
19
|
+
- **Pure functions** with deterministic output for given input
|
|
20
|
+
- **Serializers/deserializers** where `decode(encode(x)) === x` (round-trip property)
|
|
21
|
+
- **Sorting/filtering** where output maintains invariants (sorted order, subset relationship)
|
|
22
|
+
- **Validators** where valid input always passes and specific invalid inputs always fail
|
|
23
|
+
- **Mathematical functions** with known algebraic properties
|
|
24
|
+
|
|
25
|
+
2. **Identify properties for each candidate.** Common property categories:
|
|
26
|
+
- **Round-trip:** `deserialize(serialize(x)) === x` for any valid `x`
|
|
27
|
+
- **Idempotence:** `f(f(x)) === f(x)` (applying the function twice gives the same result)
|
|
28
|
+
- **Invariant preservation:** output always satisfies a postcondition regardless of input
|
|
29
|
+
- **Commutativity:** `f(a, b) === f(b, a)` for operations where order should not matter
|
|
30
|
+
- **No-crash (robustness):** function does not throw for any input in the domain
|
|
31
|
+
- **Monotonicity:** if `a <= b`, then `f(a) <= f(b)` for order-preserving functions
|
|
32
|
+
- **Equivalence:** `fastImpl(x) === referenceImpl(x)` for optimized implementations
|
|
33
|
+
|
|
34
|
+
3. **Define input domains.** For each property, specify:
|
|
35
|
+
- The type and range of valid inputs
|
|
36
|
+
- Constraints that inputs must satisfy (e.g., non-empty arrays, positive integers)
|
|
37
|
+
- Edge cases that the generator should emphasize (empty strings, zero, max int, Unicode)
|
|
38
|
+
|
|
39
|
+
4. **Prioritize by risk.** Focus property tests on:
|
|
40
|
+
- Functions where bugs have high business impact
|
|
41
|
+
- Functions with complex branching logic
|
|
42
|
+
- Functions that have had historical bugs or regression issues
|
|
43
|
+
|
|
44
|
+
5. **Report findings.** List candidate functions, their properties, and the expected generator configuration.
|
|
45
|
+
|
|
46
|
+
### Phase 2: DEFINE -- Write Property Specifications and Custom Generators
|
|
47
|
+
|
|
48
|
+
1. **Select the property testing framework.** Based on the project's language:
|
|
49
|
+
- **TypeScript/JavaScript:** fast-check
|
|
50
|
+
- **Python:** hypothesis
|
|
51
|
+
- **Rust:** proptest or quickcheck
|
|
52
|
+
- **Scala:** ScalaCheck
|
|
53
|
+
- **Haskell:** QuickCheck
|
|
54
|
+
- **Java/Kotlin:** jqwik
|
|
55
|
+
|
|
56
|
+
2. **Define custom generators (arbitraries) for domain types.** For each domain model:
|
|
57
|
+
- Build a generator that produces valid instances with realistic field values
|
|
58
|
+
- Add constraints matching the model's validation rules
|
|
59
|
+
- Compose generators for nested structures using `map`, `flatMap`, and `filter`
|
|
60
|
+
|
|
61
|
+
3. **Write property test specifications.** For each property identified in Phase 1:
|
|
62
|
+
- State the property as a universally quantified assertion: "For all inputs X satisfying constraint C, property P holds"
|
|
63
|
+
- Use the framework's property definition syntax
|
|
64
|
+
- Configure iteration count (default: 100 iterations for fast properties, 1000 for critical properties)
|
|
65
|
+
|
|
66
|
+
4. **Configure shrinking.** Ensure the framework's automatic shrinking is enabled:
|
|
67
|
+
- Shrinking reduces failing inputs to the minimal counterexample
|
|
68
|
+
- Custom generators should support shrinking (use `map` over `filter` where possible, since `filter` breaks shrinking)
|
|
69
|
+
- Set a shrink limit to prevent infinite shrinking on complex inputs
|
|
70
|
+
|
|
71
|
+
5. **Write seed values for reproducibility.** Configure:
|
|
72
|
+
- A fixed seed for CI to ensure deterministic reruns
|
|
73
|
+
- Seed logging so that any failure can be reproduced exactly
|
|
74
|
+
- Replay capability: failed seeds are stored and replayed on subsequent runs
|
|
75
|
+
|
|
76
|
+
### Phase 3: EXECUTE -- Run Property Tests and Collect Counterexamples
|
|
77
|
+
|
|
78
|
+
1. **Run property tests with verbose output.** Execute the test suite and observe:
|
|
79
|
+
- Number of test cases generated per property
|
|
80
|
+
- Any counterexamples found (failing inputs)
|
|
81
|
+
- Shrinking progress (how the framework reduces counterexamples)
|
|
82
|
+
|
|
83
|
+
2. **Analyze counterexamples.** For each failing property:
|
|
84
|
+
- Read the shrunk counterexample -- this is the minimal input that violates the property
|
|
85
|
+
- Understand why this input causes a failure
|
|
86
|
+
- Classify: is this a real bug, or is the property specification too strict?
|
|
87
|
+
|
|
88
|
+
3. **Reproduce counterexamples deterministically.** For each counterexample:
|
|
89
|
+
- Record the failing seed value
|
|
90
|
+
- Write an explicit example-based test using the shrunk counterexample as a regression test
|
|
91
|
+
- This regression test serves as documentation and prevents the same bug from recurring
|
|
92
|
+
|
|
93
|
+
4. **Handle flaky property tests.** If a property test fails intermittently:
|
|
94
|
+
- Increase the iteration count to reproduce more reliably
|
|
95
|
+
- Check if the property is sensitive to floating-point precision
|
|
96
|
+
- Verify that the generator does not produce inputs outside the valid domain
|
|
97
|
+
|
|
98
|
+
5. **Iterate on generator quality.** If the generator frequently produces uninteresting inputs:
|
|
99
|
+
- Add bias toward edge cases (empty collections, boundary values)
|
|
100
|
+
- Use `filter` sparingly (it discards inputs, wasting iterations)
|
|
101
|
+
- Prefer `map` and `flatMap` to construct valid inputs directly
|
|
102
|
+
|
|
103
|
+
### Phase 4: ANALYZE -- Diagnose Root Causes and Harden Implementations
|
|
104
|
+
|
|
105
|
+
1. **Fix bugs exposed by counterexamples.** For each real bug found:
|
|
106
|
+
- Understand the root cause using the minimal counterexample
|
|
107
|
+
- Fix the implementation
|
|
108
|
+
- Verify the property now holds (rerun with the same seed)
|
|
109
|
+
- Keep the regression test with the explicit counterexample
|
|
110
|
+
|
|
111
|
+
2. **Strengthen property specifications.** After fixing bugs:
|
|
112
|
+
- Consider whether additional properties are now testable
|
|
113
|
+
- Tighten existing properties if the fix enables stricter invariants
|
|
114
|
+
- Add properties for edge cases revealed by the counterexamples
|
|
115
|
+
|
|
116
|
+
3. **Measure property test effectiveness.** Evaluate:
|
|
117
|
+
- Number of unique bugs found by property tests vs. example-based tests
|
|
118
|
+
- Types of bugs found (off-by-one, overflow, Unicode handling, null handling)
|
|
119
|
+
- Generator coverage: what percentage of the input domain is being explored
|
|
120
|
+
|
|
121
|
+
4. **Integrate property tests into CI.** Configure:
|
|
122
|
+
- Property tests run on every PR with a moderate iteration count (100)
|
|
123
|
+
- Nightly runs use a higher iteration count (10,000) for deeper exploration
|
|
124
|
+
- Failed seeds are stored as artifacts for reproduction
|
|
125
|
+
|
|
126
|
+
5. **Run `harness validate`.** Confirm the project passes all harness checks with property tests in place.
|
|
127
|
+
|
|
128
|
+
### Graph Refresh
|
|
129
|
+
|
|
130
|
+
If a knowledge graph exists at `.harness/graph/`, refresh it after code changes to keep graph queries accurate:
|
|
131
|
+
|
|
132
|
+
```
|
|
133
|
+
harness scan [path]
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
## Harness Integration
|
|
137
|
+
|
|
138
|
+
- **`harness validate`** -- Run in ANALYZE phase after property tests are written and bugs are fixed. Confirms project health.
|
|
139
|
+
- **`harness check-deps`** -- Run after DEFINE phase to verify property testing framework is in devDependencies.
|
|
140
|
+
- **`emit_interaction`** -- Used to present counterexample analysis and property specification decisions to the human.
|
|
141
|
+
- **Grep** -- Used in IDENTIFY phase to find pure functions, serializers, validators, and mathematical operations.
|
|
142
|
+
- **Glob** -- Used to catalog existing property test files and domain type definitions.
|
|
143
|
+
|
|
144
|
+
## Success Criteria
|
|
145
|
+
|
|
146
|
+
- Every function with large input space has at least one property test
|
|
147
|
+
- Custom generators produce valid domain objects without relying heavily on `filter`
|
|
148
|
+
- All counterexamples are investigated: real bugs are fixed, property specs are adjusted for false positives
|
|
149
|
+
- Shrunk counterexamples are preserved as explicit regression tests
|
|
150
|
+
- Property tests are deterministic in CI (fixed seed) while still exploring randomly in local development
|
|
151
|
+
- `harness validate` passes with property tests in place
|
|
152
|
+
|
|
153
|
+
## Examples
|
|
154
|
+
|
|
155
|
+
### Example: fast-check for a TypeScript URL Parser
|
|
156
|
+
|
|
157
|
+
**IDENTIFY -- Properties of a URL parser:**
|
|
158
|
+
|
|
159
|
+
```
|
|
160
|
+
Function: parseUrl(input: string): ParsedUrl
|
|
161
|
+
Properties:
|
|
162
|
+
1. Round-trip: formatUrl(parseUrl(url)) === url for any valid URL
|
|
163
|
+
2. No-crash: parseUrl(arbitrary_string) never throws (returns Result type)
|
|
164
|
+
3. Invariant: parsed.protocol is always lowercase
|
|
165
|
+
4. Invariant: parsed.host never contains a trailing slash
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
**DEFINE -- Custom generator and property tests:**
|
|
169
|
+
|
|
170
|
+
```typescript
|
|
171
|
+
// tests/property/url-parser.prop.test.ts
|
|
172
|
+
import fc from 'fast-check';
|
|
173
|
+
import { parseUrl, formatUrl } from '../../src/url-parser';
|
|
174
|
+
|
|
175
|
+
// Custom generator for valid URLs
|
|
176
|
+
const urlArb = fc
|
|
177
|
+
.record({
|
|
178
|
+
protocol: fc.constantFrom('http', 'https', 'ftp'),
|
|
179
|
+
host: fc.domain(),
|
|
180
|
+
port: fc.option(fc.integer({ min: 1, max: 65535 }), { nil: undefined }),
|
|
181
|
+
path: fc
|
|
182
|
+
.array(
|
|
183
|
+
fc.stringOf(fc.constantFrom(...'abcdefghijklmnopqrstuvwxyz0123456789-_'.split('')), {
|
|
184
|
+
minLength: 1,
|
|
185
|
+
})
|
|
186
|
+
)
|
|
187
|
+
.map((segments) => '/' + segments.join('/')),
|
|
188
|
+
})
|
|
189
|
+
.map(({ protocol, host, port, path }) => `${protocol}://${host}${port ? ':' + port : ''}${path}`);
|
|
190
|
+
|
|
191
|
+
describe('URL parser properties', () => {
|
|
192
|
+
it('round-trips valid URLs', () => {
|
|
193
|
+
fc.assert(
|
|
194
|
+
fc.property(urlArb, (url) => {
|
|
195
|
+
const parsed = parseUrl(url);
|
|
196
|
+
if (!parsed.ok) return false; // skip invalid (generator should not produce these)
|
|
197
|
+
return formatUrl(parsed.value) === url;
|
|
198
|
+
}),
|
|
199
|
+
{ numRuns: 1000, seed: 42 }
|
|
200
|
+
);
|
|
201
|
+
});
|
|
202
|
+
|
|
203
|
+
it('never throws on arbitrary string input', () => {
|
|
204
|
+
fc.assert(
|
|
205
|
+
fc.property(fc.string(), (input) => {
|
|
206
|
+
const result = parseUrl(input);
|
|
207
|
+
// Must return a Result, never throw
|
|
208
|
+
return result.ok === true || result.ok === false;
|
|
209
|
+
}),
|
|
210
|
+
{ numRuns: 5000 }
|
|
211
|
+
);
|
|
212
|
+
});
|
|
213
|
+
|
|
214
|
+
it('always produces lowercase protocol', () => {
|
|
215
|
+
fc.assert(
|
|
216
|
+
fc.property(urlArb, (url) => {
|
|
217
|
+
const parsed = parseUrl(url.toUpperCase());
|
|
218
|
+
if (!parsed.ok) return true; // skip failures
|
|
219
|
+
return parsed.value.protocol === parsed.value.protocol.toLowerCase();
|
|
220
|
+
})
|
|
221
|
+
);
|
|
222
|
+
});
|
|
223
|
+
});
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
### Example: hypothesis for a Python Sorting Algorithm
|
|
227
|
+
|
|
228
|
+
**DEFINE -- Property tests with hypothesis:**
|
|
229
|
+
|
|
230
|
+
```python
|
|
231
|
+
# tests/property/test_sort_properties.py
|
|
232
|
+
from hypothesis import given, settings, assume
|
|
233
|
+
from hypothesis import strategies as st
|
|
234
|
+
from myapp.sorting import merge_sort
|
|
235
|
+
|
|
236
|
+
@given(st.lists(st.integers()))
|
|
237
|
+
def test_sort_preserves_length(xs):
|
|
238
|
+
"""Sorted output has the same length as input."""
|
|
239
|
+
assert len(merge_sort(xs)) == len(xs)
|
|
240
|
+
|
|
241
|
+
@given(st.lists(st.integers()))
|
|
242
|
+
def test_sort_preserves_elements(xs):
|
|
243
|
+
"""Sorted output contains exactly the same elements as input."""
|
|
244
|
+
assert sorted(merge_sort(xs)) == sorted(xs)
|
|
245
|
+
|
|
246
|
+
@given(st.lists(st.integers(), min_size=1))
|
|
247
|
+
def test_sort_produces_ordered_output(xs):
|
|
248
|
+
"""Every element is less than or equal to the next."""
|
|
249
|
+
result = merge_sort(xs)
|
|
250
|
+
for i in range(len(result) - 1):
|
|
251
|
+
assert result[i] <= result[i + 1]
|
|
252
|
+
|
|
253
|
+
@given(st.lists(st.integers()))
|
|
254
|
+
def test_sort_is_idempotent(xs):
|
|
255
|
+
"""Sorting an already-sorted list produces the same result."""
|
|
256
|
+
once = merge_sort(xs)
|
|
257
|
+
twice = merge_sort(once)
|
|
258
|
+
assert once == twice
|
|
259
|
+
|
|
260
|
+
@settings(max_examples=5000)
|
|
261
|
+
@given(st.lists(st.floats(allow_nan=False, allow_infinity=False)))
|
|
262
|
+
def test_sort_handles_floats(xs):
|
|
263
|
+
"""Sort works correctly with floating-point numbers."""
|
|
264
|
+
result = merge_sort(xs)
|
|
265
|
+
for i in range(len(result) - 1):
|
|
266
|
+
assert result[i] <= result[i + 1]
|
|
267
|
+
```
|
|
268
|
+
|
|
269
|
+
## Gates
|
|
270
|
+
|
|
271
|
+
- **No property tests without shrinking.** If the framework's automatic shrinking is disabled or the generator uses patterns that break shrinking (excessive `filter`), counterexamples will be unhelpfully large. Fix the generator to support shrinking.
|
|
272
|
+
- **No ignoring counterexamples.** Every counterexample produced by a property test must be investigated. If it reveals a real bug, fix it. If it is a false positive, adjust the property specification or generator. Never just increase the iteration count to make it "less likely to fail."
|
|
273
|
+
- **No property tests that always pass trivially.** A property that returns `true` for every input is useless. Review that properties make substantive assertions. If a property has a `return true` fallback for most inputs, the generator is producing too many invalid inputs.
|
|
274
|
+
- **Regression tests are mandatory for counterexamples.** Every shrunk counterexample that revealed a bug must be preserved as an explicit example-based test, even after the property test passes. The explicit test serves as documentation and prevents regression.
|
|
275
|
+
|
|
276
|
+
## Escalation
|
|
277
|
+
|
|
278
|
+
- **When the generator cannot produce valid inputs efficiently (> 50% rejection rate):** Rewrite the generator to construct valid inputs directly rather than filtering. Use `flatMap` to build constrained structures incrementally. If the domain constraints are too complex for a generator, consider whether the function's API needs simplification.
|
|
279
|
+
- **When a counterexample is too complex to understand even after shrinking:** The shrinking strategy may be insufficient for the data type. Write a custom shrinker that targets the specific structure. Alternatively, add intermediate logging to the property to trace which sub-property fails.
|
|
280
|
+
- **When property tests are too slow for CI (> 5 minutes):** Reduce the iteration count for PR runs (100 iterations). Run high-iteration tests (10,000+) as a nightly job. Consider whether some properties can be tested with smaller input ranges without losing coverage.
|
|
281
|
+
- **When the team debates whether a property is correct:** The property may be encoding an assumption that does not hold. Review the specification or domain requirements. If the correct behavior is ambiguous, escalate to product/domain experts before encoding the property in a test.
|