synapse-hub 2.2.4 → 2.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/agents/failure-analyst.md +11 -0
- package/.agents/agents/live-explorer.md +28 -0
- package/.agents/agents/orchestrator.md +23 -4
- package/.agents/agents/pom-crafter.md +15 -0
- package/.agents/agents/quality-gatekeeper.md +20 -4
- package/.agents/agents/test-runner.md +12 -0
- package/.agents/rules/communication-style.md +40 -0
- package/.agents/rules/context-engineering.md +23 -0
- package/.agents/rules/git-push-protocol.md +10 -16
- package/.agents/rules/graphify.md +14 -14
- package/.agents/rules/mandatory-playwright-live.md +29 -29
- package/.agents/rules/mobile_appium_rules.md +132 -0
- package/.agents/rules/performance_testing_rules.md +74 -0
- package/.agents/rules/systematic-debugging.md +6 -6
- package/.agents/skills/agent-orchestration-multi-agent-optimize/SKILL.md +64 -215
- package/.agents/skills/agent-orchestration-multi-agent-optimize/examples/swarm_workload_config.json +12 -0
- package/.agents/skills/antigravity-test-sentinel/SKILL.md +3 -3
- package/.agents/skills/antigravity-test-sentinel/examples/dod_gate_policy.json +11 -0
- package/.agents/skills/api-network-mocking-interception/SKILL.md +1 -1
- package/.agents/skills/api-network-mocking-interception/examples/network_route_stub.json +9 -0
- package/.agents/skills/api-spec-to-e2e-matrix/SKILL.md +9 -9
- package/.agents/skills/api-spec-to-e2e-matrix/examples/sample_openapi.json +19 -0
- package/.agents/skills/architecture-decision-records/SKILL.md +35 -412
- package/.agents/skills/architecture-decision-records/examples/0001-record-architecture-decisions.md +12 -0
- package/.agents/skills/auth-session-and-cookie-manager/SKILL.md +1 -1
- package/.agents/skills/auth-session-and-cookie-manager/examples/storage_state_sample.json +11 -0
- package/.agents/skills/backend-load-and-stress-patterns/SKILL.md +80 -186
- package/.agents/skills/backend-load-and-stress-patterns/examples/k6_load_test_sample.js +13 -0
- package/.agents/skills/bats-testing-patterns/SKILL.md +1 -1
- package/.agents/skills/bats-testing-patterns/examples/cli_runner_test.bats +6 -0
- package/.agents/skills/bats-testing-patterns/resources/implementation-playbook.md +614 -614
- package/.agents/skills/clean-pom-crafter/SKILL.md +21 -3
- package/.agents/skills/clean-pom-crafter/examples/standard_test_case.groovy +28 -28
- package/.agents/skills/clean-pom-crafter/references/pom_patterns.groovy +138 -138
- package/.agents/skills/code-refactoring-refactor-clean/SKILL.md +2 -2
- package/.agents/skills/code-refactoring-refactor-clean/examples/refactored_page_sample.groovy +9 -0
- package/.agents/skills/db-state-verifier-and-seeder/SKILL.md +5 -5
- package/.agents/skills/db-state-verifier-and-seeder/examples/db_fixture_seed.json +12 -0
- package/.agents/skills/e2e-testing-patterns/SKILL.md +1 -1
- package/.agents/skills/e2e-testing-patterns/examples/playwright_sharded_config.ts +12 -0
- package/.agents/skills/e2e-testing-patterns/resources/implementation-playbook.md +522 -522
- package/.agents/skills/failure-autopsy-and-healing/SKILL.md +17 -3
- package/.agents/skills/failure-autopsy-and-healing/examples/sample_junit_failure.xml +8 -0
- package/.agents/skills/failure-autopsy-and-healing/references/healing_matrix.md +64 -64
- package/.agents/skills/failure-autopsy-and-healing/references/qa_investigation_protocol.md +56 -56
- package/.agents/skills/failure-autopsy-and-healing/scripts/triage_failures.sh +98 -98
- package/.agents/skills/file-and-report-validator/SKILL.md +1 -1
- package/.agents/skills/file-and-report-validator/examples/sample_export_schema.json +14 -0
- package/.agents/skills/flaky-test-sentinel-and-quarantine/SKILL.md +3 -3
- package/.agents/skills/flaky-test-sentinel-and-quarantine/examples/quarantine_registry.json +11 -0
- package/.agents/skills/gherkin-to-suite/SKILL.md +103 -0
- package/.agents/skills/gherkin-to-suite/examples/checkout.feature +39 -0
- package/.agents/skills/gherkin-to-suite/examples/suite_report_sample.json +334 -0
- package/.agents/skills/github-actions-templates/SKILL.md +47 -304
- package/.agents/skills/github-actions-templates/examples/matrix_test_pipeline.yml +8 -0
- package/.agents/skills/graphify-impact-analyzer/SKILL.md +6 -6
- package/.agents/skills/graphify-impact-analyzer/examples/dependency_blast_radius.json +8 -0
- package/.agents/skills/katalon-cli-runner/SKILL.md +2 -2
- package/.agents/skills/katalon-cli-runner/examples/headless_kre_exec.sh +3 -0
- package/.agents/skills/katalon-cli-runner/references/groovy-patterns.md +209 -209
- package/.agents/skills/katalon-cli-runner/references/katalon-api.md +155 -155
- package/.agents/skills/katalon-cli-runner/references/runner_commands.md +67 -67
- package/.agents/skills/lam-script-generator/SKILL.md +42 -1
- package/.agents/skills/lam-script-generator/examples/user_story_sample.md +8 -0
- package/.agents/skills/lean-code-ladder/SKILL.md +83 -0
- package/.agents/skills/lean-code-ladder/examples/ladder_walkthrough.md +37 -0
- package/.agents/skills/meta-prompt-crafter/SKILL.md +107 -0
- package/.agents/skills/meta-prompt-crafter/examples/qa_prompt_template.md +35 -0
- package/.agents/skills/persistent-planning-files/SKILL.md +108 -0
- package/.agents/skills/persistent-planning-files/examples/task_plan_example.md +56 -0
- package/.agents/skills/playwright-live-interaction/SKILL.md +49 -1
- package/.agents/skills/playwright-live-interaction/examples/live_probe_script.js +5 -0
- package/.agents/skills/playwright-live-interaction/references/advanced_superpowers.md +160 -160
- package/.agents/skills/playwright-live-interaction/references/compact_accessibility_snapshots.md +75 -75
- package/.agents/skills/playwright-live-interaction/references/recipes.md +388 -388
- package/.agents/skills/playwright-live-interaction/references/swarm_orchestration.md +134 -134
- package/.agents/skills/playwright-webapp-testing/SKILL.md +93 -0
- package/.agents/skills/playwright-webapp-testing/examples/server_lifecycle_playwright.py +63 -0
- package/.agents/skills/rbt-manual-testing/SKILL.md +150 -0
- package/.agents/skills/rbt-manual-testing/examples/checkout_rbt_manual_suite.md +25 -0
- package/.agents/skills/security-dast-sentinel/SKILL.md +54 -135
- package/.agents/skills/security-dast-sentinel/examples/dast_scan_config.json +10 -0
- package/.agents/skills/self-healing-locator/SKILL.md +24 -10
- package/.agents/skills/self-healing-locator/examples/aom_diff_sample.json +16 -0
- package/.agents/skills/skillspector-security-guard/SKILL.md +32 -90
- package/.agents/skills/skillspector-security-guard/examples/safe_mcp_config.json +12 -0
- package/.agents/skills/surgical-coding-principles/SKILL.md +89 -0
- package/.agents/skills/surgical-coding-principles/examples/surgical_vs_scope_creep.md +46 -0
- package/.agents/skills/synthetic-test-data-generator/SKILL.md +3 -3
- package/.agents/skills/synthetic-test-data-generator/examples/idempotent_data_recipe.json +9 -0
- package/.agents/skills/systematic-debugging/SKILL.md +100 -0
- package/.agents/skills/systematic-debugging/examples/four_phase_case.md +44 -0
- package/.agents/skills/test-driven-development/SKILL.md +114 -0
- package/.agents/skills/test-driven-development/examples/red_green_refactor_qa.md +78 -0
- package/.agents/skills/token-and-context-optimizer/SKILL.md +1 -1
- package/.agents/skills/token-and-context-optimizer/examples/distilled_log_sample.txt +3 -0
- package/.agents/skills/verification-before-completion/SKILL.md +100 -0
- package/.agents/skills/verification-before-completion/examples/evidence_log_example.md +46 -0
- package/.agents/skills/visual-regression-sentinel/SKILL.md +21 -1
- package/.agents/skills/visual-regression-sentinel/examples/visual_mask_rules.json +9 -0
- package/.agents/skills/wcag-audit-patterns/SKILL.md +3 -3
- package/.agents/skills/wcag-audit-patterns/examples/wcag_compliance_checklist.json +10 -0
- package/.agents/skills/wcag-audit-patterns/resources/implementation-playbook.md +541 -541
- package/.agents/skills/web-vitals-and-performance-profiler/SKILL.md +1 -1
- package/.agents/skills/web-vitals-and-performance-profiler/examples/cwv_budget_policy.json +6 -0
- package/.agents/templates/completion_report.md +57 -57
- package/.agents/workflows/analyze_requirement_document.md +18 -0
- package/.agents/workflows/export_testcases_excel.md +32 -0
- package/.agents/workflows/generate_combinatorial_test_data.md +28 -0
- package/.agents/workflows/generate_cross_module_test_plan.md +37 -0
- package/.agents/workflows/generate_manual_testcases_rbt.md +56 -0
- package/.agents/workflows/generate_requirements_from_website.md +25 -0
- package/.agents/workflows/generate_testcases_from_requirements.md +21 -0
- package/.agents/workflows/graphify.md +10 -10
- package/.agents/workflows/verify.md +2 -2
- package/.cursor/rules/language.mdc +7 -7
- package/.cursor/rules/synapse-qa.mdc +23 -0
- package/.cursorrules +7 -0
- package/.pre-commit-hooks.yaml +14 -0
- package/.windsurfrules +7 -0
- package/AGENTS.md +54 -15
- package/CLAUDE.md +35 -12
- package/LICENSE +30 -0
- package/NOTICE +180 -0
- package/README.en.md +331 -0
- package/README.md +252 -515
- package/action.yml +51 -0
- package/benchmarks/qa_eval/README.md +71 -0
- package/benchmarks/qa_eval/build_dataset.py +677 -0
- package/benchmarks/qa_eval/dataset/api_contract_drift/qa-challenge-06_api_contract_drift.json +22 -0
- package/benchmarks/qa_eval/dataset/api_contract_drift/qa-challenge-07_api_contract_drift.json +22 -0
- package/benchmarks/qa_eval/dataset/api_contract_drift/qa-challenge-08_api_contract_drift.json +22 -0
- package/benchmarks/qa_eval/dataset/api_contract_drift/qa-challenge-09_api_contract_drift.json +22 -0
- package/benchmarks/qa_eval/dataset/api_contract_drift/qa-challenge-10_api_contract_drift.json +22 -0
- package/benchmarks/qa_eval/dataset/fake_pass_tautology/qa-challenge-21_fake_pass_tautology.json +21 -0
- package/benchmarks/qa_eval/dataset/fake_pass_tautology/qa-challenge-22_fake_pass_tautology.json +21 -0
- package/benchmarks/qa_eval/dataset/fake_pass_tautology/qa-challenge-23_fake_pass_tautology.json +21 -0
- package/benchmarks/qa_eval/dataset/fake_pass_tautology/qa-challenge-24_fake_pass_tautology.json +21 -0
- package/benchmarks/qa_eval/dataset/fake_pass_tautology/qa-challenge-25_fake_pass_tautology.json +21 -0
- package/benchmarks/qa_eval/dataset/locator_drift/qa-challenge-01_locator_drift.json +23 -0
- package/benchmarks/qa_eval/dataset/locator_drift/qa-challenge-02_locator_drift.json +23 -0
- package/benchmarks/qa_eval/dataset/locator_drift/qa-challenge-03_locator_drift.json +23 -0
- package/benchmarks/qa_eval/dataset/locator_drift/qa-challenge-04_locator_drift.json +23 -0
- package/benchmarks/qa_eval/dataset/locator_drift/qa-challenge-05_locator_drift.json +23 -0
- package/benchmarks/qa_eval/dataset/stale_and_mutation/qa-challenge-16_stale_and_mutation.json +22 -0
- package/benchmarks/qa_eval/dataset/stale_and_mutation/qa-challenge-17_stale_and_mutation.json +22 -0
- package/benchmarks/qa_eval/dataset/stale_and_mutation/qa-challenge-18_stale_and_mutation.json +22 -0
- package/benchmarks/qa_eval/dataset/stale_and_mutation/qa-challenge-19_stale_and_mutation.json +22 -0
- package/benchmarks/qa_eval/dataset/stale_and_mutation/qa-challenge-20_stale_and_mutation.json +23 -0
- package/benchmarks/qa_eval/dataset/timing_and_race/qa-challenge-11_timing_and_race.json +22 -0
- package/benchmarks/qa_eval/dataset/timing_and_race/qa-challenge-12_timing_and_race.json +22 -0
- package/benchmarks/qa_eval/dataset/timing_and_race/qa-challenge-13_timing_and_race.json +22 -0
- package/benchmarks/qa_eval/dataset/timing_and_race/qa-challenge-14_timing_and_race.json +22 -0
- package/benchmarks/qa_eval/dataset/timing_and_race/qa-challenge-15_timing_and_race.json +22 -0
- package/benchmarks/qa_eval/eval_runner.py +418 -0
- package/benchmarks/qa_eval/reports/AB_METHODOLOGY_REPORT.md +83 -0
- package/benchmarks/qa_eval/reports/BENCHMARK_REPORT.md +69 -0
- package/benchmarks/qa_eval/reports/eval_results.json +105 -0
- package/bin/agent-qa.js +16 -4
- package/docs/ARCHITECTURE.md +107 -0
- package/docs/CI_CD_INTEGRATION.md +94 -0
- package/docs/CLI_REFERENCE.md +157 -0
- package/docs/CONTRIBUTING.md +91 -0
- package/docs/PLUGIN_DEVELOPMENT_GUIDE.md +126 -0
- package/docs/TOKEN_QUOTA_PLAYBOOK.md +41 -0
- package/docs/adopted-ecosystem-tools.md +87 -0
- package/docs/schemas/synapse.config.schema.json +64 -0
- package/examples/cypress-ecommerce-suite/README.md +45 -0
- package/examples/cypress-ecommerce-suite/cypress/e2e/checkout.cy.js +17 -0
- package/examples/cypress-ecommerce-suite/cypress/pages/BasePage.js +29 -0
- package/examples/cypress-ecommerce-suite/cypress/pages/CartPage.js +51 -0
- package/examples/cypress-ecommerce-suite/cypress.config.js +13 -0
- package/examples/cypress-ecommerce-suite/package.json +13 -0
- package/examples/katalon-suite/Keywords/pages/BasePage.groovy +19 -0
- package/examples/katalon-suite/Keywords/pages/DashboardPage.groovy +29 -0
- package/examples/katalon-suite/Keywords/pages/LoginPage.groovy +45 -0
- package/examples/katalon-suite/Profiles/default.glbl +12 -0
- package/examples/katalon-suite/README.md +18 -0
- package/examples/katalon-suite/Scripts/Auth/TC_LoginSuccess/Script1234567890.groovy +12 -0
- package/examples/katalon-suite/Scripts/Dashboard/TC_ViewSummary/Script1234567891.groovy +15 -0
- package/examples/katalon-suite/katalon_suite.prj +21 -0
- package/examples/playwright-crm-suite/README.md +40 -0
- package/examples/playwright-crm-suite/package.json +14 -0
- package/examples/playwright-crm-suite/pages/BasePage.ts +37 -0
- package/examples/playwright-crm-suite/pages/CustomerPage.ts +57 -0
- package/examples/playwright-crm-suite/pages/LoginPage.ts +32 -0
- package/examples/playwright-crm-suite/playwright.config.ts +27 -0
- package/examples/playwright-crm-suite/tests/auth.spec.ts +22 -0
- package/examples/playwright-crm-suite/tests/customers.spec.ts +33 -0
- package/examples/plugins/sample-custom-plugin/README.md +14 -0
- package/examples/plugins/sample-custom-plugin/plugin.json +16 -0
- package/examples/plugins/sample-custom-plugin/skills/custom-security-headers-audit/SKILL.md +29 -0
- package/examples/plugins/sample-custom-plugin/skills/custom-security-headers-audit/examples/audit_fixture.json +8 -0
- package/examples/plugins/sample-custom-plugin/tools/custom_security_headers.py +125 -0
- package/examples/practices/README.md +22 -0
- package/examples/practices/requirements/crm/requirements_contracts.md +292 -0
- package/examples/practices/requirements/crm/requirements_customers.md +462 -0
- package/examples/practices/requirements/crm/requirements_login.md +202 -0
- package/examples/practices/requirements/crm/requirements_projects.md +396 -0
- package/examples/practices/requirements/crm/requirements_tasks.md +343 -0
- package/examples/practices/requirements/ecommerce/requirements_category_brand.txt +402 -0
- package/examples/practices/requirements/jira/KAN-5/KAN-1/2026-04-03_04-28-27.png +0 -0
- package/examples/practices/requirements/jira/KAN-5/KAN-1/KAN-1_requirement.md +52 -0
- package/examples/practices/requirements/jira/KAN-5/KAN-4/2026-04-03_04-28-19.png +0 -0
- package/examples/practices/requirements/jira/KAN-5/KAN-4/2026-04-03_04-28-57.png +0 -0
- package/examples/practices/requirements/jira/KAN-5/KAN-4/KAN-4_requirement.md +119 -0
- package/examples/practices/requirements/jira/KAN-5/KAN-5_overview.md +9 -0
- package/examples/practices/requirements/jira/KAN-5/KAN-6/2026-04-03_05-58-08.png +0 -0
- package/examples/practices/requirements/jira/KAN-5/KAN-6/2026-04-03_05-58-23.png +0 -0
- package/examples/practices/requirements/jira/KAN-5/KAN-6/KAN-6_requirement.md +32 -0
- package/examples/practices/testcases/api/book_management_api_test_cases.md +144 -0
- package/examples/practices/testcases/api/user_management_api_test_cases.md +139 -0
- package/examples/practices/testcases/crm/test_cases_crm_login.md +148 -0
- package/examples/practices/testcases/crm/test_cases_customers.md +156 -0
- package/examples/pytest-api-suite/README.md +45 -0
- package/examples/pytest-api-suite/conftest.py +31 -0
- package/examples/pytest-api-suite/pages/user_api_client.py +36 -0
- package/examples/pytest-api-suite/requirements.txt +2 -0
- package/examples/pytest-api-suite/tests/test_user_api.py +44 -0
- package/examples/selenium-suite/README.md +18 -0
- package/examples/selenium-suite/pages/base_page.py +27 -0
- package/examples/selenium-suite/pages/login_page.py +29 -0
- package/examples/selenium-suite/pages/order_page.py +25 -0
- package/examples/selenium-suite/pytest.ini +6 -0
- package/examples/selenium-suite/requirements.txt +2 -0
- package/examples/selenium-suite/tests/test_login.py +10 -0
- package/examples/selenium-suite/tests/test_order.py +10 -0
- package/package.json +27 -12
- package/scripts/generate_skill_examples.py +244 -0
- package/scripts/install.ps1 +112 -0
- package/scripts/install_global_plugin.ps1 +94 -0
- package/scripts/install_global_plugin.sh +49 -0
- package/scripts/lint_skills.ps1 +57 -0
- package/scripts/package_dist.py +323 -0
- package/scripts/setup.sh +59 -0
- package/scripts/update_hub.ps1 +19 -0
- package/scripts/update_hub.sh +15 -0
- package/skills_manifest.json +209 -6
- package/synapse.config.json +30 -0
- package/templates/PROJECT_AI_GUIDE_TEMPLATE.md +7 -7
- package/templates/PROJECT_STATE_TEMPLATE.md +1 -1
- package/templates/ai_bridges/CLAUDE.md +33 -16
- package/templates/ai_bridges/copilot-instructions.md +14 -21
- package/templates/ai_bridges/cursor_synapse_qa.mdc +17 -20
- package/templates/ai_bridges/windsurfrules +21 -21
- package/templates/ci/github-actions-synapse.yml +71 -4
- package/templates/welcome.html.tmpl +1674 -0
- package/tools/.infer_baseline.json +1102 -0
- package/tools/TOOLS_RUNBOOK.md +98 -131
- package/tools/__init__.py +26 -0
- package/tools/agent_canary.py +501 -501
- package/tools/analyze_project.py +31 -17
- package/tools/api_fuzzer.py +278 -6
- package/tools/author_integrity.py +6 -6
- package/tools/auto_heal.py +1 -1
- package/tools/browser_helpers.js +105 -7
- package/tools/bug_reporter.py +31 -1
- package/tools/bump_version.py +30 -3
- package/tools/cache_resolver.py +248 -14
- package/tools/check_env.py +55 -14
- package/tools/check_version_sync.py +171 -0
- package/tools/ci_healing_hook.py +1 -1
- package/tools/cli.py +454 -42
- package/tools/count_tokens.py +1257 -0
- package/tools/dashboard_generator.py +2638 -0
- package/tools/db_sandbox.py +198 -20
- package/tools/e2e_flow_prober.ps1 +12 -64
- package/tools/e2e_flow_prober.py +99 -0
- package/tools/e2e_flow_prober.sh +4 -0
- package/tools/flaky_detector.py +4 -3
- package/tools/generate_allure_report.ps1 +12 -29
- package/tools/generate_allure_report.py +77 -0
- package/tools/generate_allure_report.sh +4 -0
- package/tools/generate_jmx_plan.py +257 -58
- package/tools/generate_test_reports.ps1 +12 -134
- package/tools/generate_test_reports.py +133 -0
- package/tools/generate_test_reports.sh +4 -0
- package/tools/gherkin_suite.py +758 -0
- package/tools/git_pre_commit_guard.py +12 -6
- package/tools/goldfish_review.py +366 -0
- package/tools/heal_pr_automator.py +1 -1
- package/tools/heuristic_locator_matcher.py +170 -21
- package/tools/infer_linter.py +612 -70
- package/tools/lint_ai_docs.py +64 -0
- package/tools/matrix_sharder.py +1 -1
- package/tools/mcp_excel_server.ps1 +12 -250
- package/tools/mcp_excel_server.py +249 -0
- package/tools/mcp_excel_server.sh +4 -0
- package/tools/mcp_server.py +368 -144
- package/tools/md_to_xlsx.py +307 -0
- package/tools/micro_mutator.py +302 -69
- package/tools/omni_transpiler.py +2 -2
- package/tools/output_formatter.py +128 -0
- package/tools/pinpoint_failure.py +88 -88
- package/tools/project_state.py +2 -2
- package/tools/repo_map.py +1 -1
- package/tools/safety_firewall.py +196 -45
- package/tools/setup_project.py +295 -31
- package/tools/skillspector.py +31 -0
- package/tools/spec_drift_sentinel.py +439 -0
- package/tools/sync_agent_rules.py +248 -0
- package/tools/telemetry_correlator.py +45 -0
- package/tools/test_impact.py +51 -12
- package/tools/tms_sync_bridge.py +21 -5
- package/tools/universal_ir.py +3 -0
- package/tools/update_sentinel.py +87 -15
- package/tools/verify_all.ps1 +13 -0
- package/tools/verify_all.py +479 -85
- package/tools/visual_diff_canvas.py +1 -0
- package/tools/visual_som_helper.py +1 -1
- package/tools/watch_mode.py +305 -0
- package/tools/welcome_page.py +291 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: synthetic-test-data-generator
|
|
3
|
-
description: "Generate idempotent, boundary-value synthetic test data with unique runtime timestamps."
|
|
3
|
+
description: "Generate idempotent, boundary-value synthetic test data with unique runtime timestamps. Use when generating idempotent, unique test data for creation tests."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Synthetic Test Data Generator & Edge-Case Fuzzing Skill
|
|
@@ -81,8 +81,8 @@ Transform functional tests into input sanitization and robustness validation by
|
|
|
81
81
|
|
|
82
82
|
```bash
|
|
83
83
|
# Query CLI catalog for specific field vectors
|
|
84
|
-
|
|
85
|
-
|
|
84
|
+
python3 ~/.zcode/synapse-hub/tools/fuzz_payloads.py --field username --type string
|
|
85
|
+
python3 ~/.zcode/synapse-hub/tools/fuzz_payloads.py --category sqli --json
|
|
86
86
|
```
|
|
87
87
|
|
|
88
88
|
### Recipe D: Harmless SQLi Polyglot Sanity
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: systematic-debugging
|
|
3
|
+
description: "Investigate root cause through four disciplined phases before proposing any fix for a test failure, bug, or unexpected behavior. Use when a test fails, a defect appears, or a fix attempt has already failed and guessing is tempting."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Systematic Debugging — Enterprise Root-Cause Standard
|
|
7
|
+
|
|
8
|
+
Specialized skill for failure investigation in test automation: always find root cause before attempting fixes. Symptom fixes are failure.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## 1. The Iron Law
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
If Phase 1 is not complete, proposing fixes is prohibited. This applies ESPECIALLY when: under time pressure (emergencies make guessing tempting), a "quick fix" seems obvious, or previous fixes did not work.
|
|
19
|
+
|
|
20
|
+
## 2. The Four Phases
|
|
21
|
+
|
|
22
|
+
```mermaid
|
|
23
|
+
flowchart TD
|
|
24
|
+
P1["<b>Phase 1: Root Cause Investigation</b><br/>Read errors, reproduce, check changes, gather evidence"] --> P2["<b>Phase 2: Pattern Analysis</b><br/>Compare against working examples"]
|
|
25
|
+
P2 --> P3["<b>Phase 3: Hypothesis & Testing</b><br/>One hypothesis, minimal change"]
|
|
26
|
+
P3 --> P4["<b>Phase 4: Implementation</b><br/>Failing test first, single fix, verify"]
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
### Phase 1: Root Cause Investigation (before ANY fix)
|
|
30
|
+
|
|
31
|
+
1. **Read error messages completely** — stack traces, line numbers, error codes often contain the exact cause.
|
|
32
|
+
2. **Reproduce consistently** — exact steps, every time. Not reproducible means gather more data (flaky scoring via `tools/flaky_detector.py`), do not guess.
|
|
33
|
+
3. **Check recent changes** — git diff, recent commits, new dependencies, config drift.
|
|
34
|
+
4. **Gather evidence at component boundaries** — for multi-layer systems (CI → build → run; API → service → DB), log what enters and exits each boundary once, to see WHICH layer breaks, then investigate that layer.
|
|
35
|
+
5. **Trace data flow backward** — where does the bad value originate? Fix at source, never at symptom.
|
|
36
|
+
|
|
37
|
+
### Phase 2: Pattern Analysis
|
|
38
|
+
|
|
39
|
+
- Find working examples of similar behavior in the same codebase; list every difference from the broken path, however small.
|
|
40
|
+
- Read reference implementations completely before applying a pattern — partial understanding guarantees bugs.
|
|
41
|
+
|
|
42
|
+
### Phase 3: Hypothesis and Testing
|
|
43
|
+
|
|
44
|
+
- State one specific hypothesis: "X is the root cause because Y."
|
|
45
|
+
- Test it with the SMALLEST possible change, one variable at a time. Do not stack fixes.
|
|
46
|
+
- Confirmed → Phase 4. Rejected → form a NEW hypothesis; do not pile fix onto fix.
|
|
47
|
+
|
|
48
|
+
### Phase 4: Implementation
|
|
49
|
+
|
|
50
|
+
1. **Create the failing test case first** — the reproduction is the RED test (see `test-driven-development`).
|
|
51
|
+
2. **Implement a single fix** at the identified root cause. No "while I'm here" improvements, no bundled refactoring.
|
|
52
|
+
3. **Verify the fix** — test passes, full suite still green, claim backed by output per `verification-before-completion`.
|
|
53
|
+
4. **If the fix does not work:** STOP. Count attempts. Fewer than 3 → return to Phase 1 with the new information.
|
|
54
|
+
5. **If 3+ fixes failed:** question the architecture, not the code — each fix revealing a new problem elsewhere is the signature of a wrong pattern. Discuss with the user before a fourth attempt.
|
|
55
|
+
|
|
56
|
+
## 3. Hub Tooling Chain
|
|
57
|
+
|
|
58
|
+
| Step | Tool | Role in investigation |
|
|
59
|
+
|---|---|---|
|
|
60
|
+
| Isolate | `python3 tools/pinpoint_failure.py` | Extract exact failing line, exception, stack trace from JUnit XML in ~0.1s |
|
|
61
|
+
| Diagnose | `python3 tools/auto_heal.py` | Closed-loop triage and healing plan; reflexion lessons after each successful heal |
|
|
62
|
+
| Reuse | `python3 tools/cache_resolver.py query "<error>"` | 0-token lookup of previously solved, similar defects before re-investigating |
|
|
63
|
+
| Classify | `python3 tools/flaky_detector.py` | Separate non-deterministic flakiness from genuine defects |
|
|
64
|
+
| Ticket | `python3 tools/bug_reporter.py` | Auto-generate the bug ticket with RCA once root cause is proven |
|
|
65
|
+
|
|
66
|
+
Escalation order: `failure-analyst` performs the autopsy; `quality-gatekeeper` re-verifies after the fix lands.
|
|
67
|
+
|
|
68
|
+
## 4. Red Flags — STOP and Return to Phase 1
|
|
69
|
+
|
|
70
|
+
- "Quick fix for now, investigate later"
|
|
71
|
+
- "Just try changing X and see if it works"
|
|
72
|
+
- Adding multiple changes and running tests together
|
|
73
|
+
- "Skip the test, I'll manually verify"
|
|
74
|
+
- Proposing solutions before tracing the data flow
|
|
75
|
+
- "One more fix attempt" after 2+ failures
|
|
76
|
+
- Each fix revealing a new problem in a different place (architecture question)
|
|
77
|
+
|
|
78
|
+
## 5. When Investigation Finds No Root Cause
|
|
79
|
+
|
|
80
|
+
If disciplined investigation proves the issue is genuinely environmental or timing-dependent: document what was investigated, implement appropriate handling (retry, timeout, error message), add monitoring for future investigation — and remember: most "no root cause" conclusions are incomplete investigation. Hand residual non-determinism to `flaky-test-sentinel-and-quarantine` for scoring and quarantine instead of declaring it unsolvable.
|
|
81
|
+
|
|
82
|
+
## 6. Boundary vs Sibling Skills (Do Not Confuse)
|
|
83
|
+
|
|
84
|
+
| Sibling | Owns | Hand-off |
|
|
85
|
+
|---|---|---|
|
|
86
|
+
| `failure-autopsy-and-healing` | Mechanical triage/healing mechanics from JUnit XML, screenshots, PR automation | This skill is the investigation DISCIPLINE; autopsy tooling executes its Phase 1/4 |
|
|
87
|
+
| `test-driven-development` | The RED-GREEN-REFACTOR cycle | Phase 4 hands the reproduced bug over as the failing test |
|
|
88
|
+
| `flaky-test-sentinel-and-quarantine` | Flakiness scoring and quarantine lifecycle | Receives non-reproducible findings after Phase 1 |
|
|
89
|
+
| `self-healing-locator` | Semantic locator healing | Consumes the verified root cause for locator-class defects |
|
|
90
|
+
|
|
91
|
+
## 7. Reference Documentation
|
|
92
|
+
|
|
93
|
+
- `python3 tools/pinpoint_failure.py <report>` — first command on any failed run.
|
|
94
|
+
- `python3 tools/auto_heal.py` — closed-loop triage after isolation.
|
|
95
|
+
- `python3 tools/cache_resolver.py query "<error message>"` — 0-token known-defect lookup.
|
|
96
|
+
- [Four-phase worked case (`four_phase_case.md`)](examples/four_phase_case.md): a timeout failure traced to root cause.
|
|
97
|
+
|
|
98
|
+
## 8. Attribution
|
|
99
|
+
|
|
100
|
+
Adapted from **superpowers** skill `systematic-debugging` (https://github.com/obra/superpowers) — MIT License, Copyright (c) 2025 Jesse Vincent. The Iron Law, four-phase process, 3-fix architecture rule, and rationalization guardrails are re-scoped for enterprise QA automation in Synapse Hub; wording, structure, and tooling chain modified.
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
# Example: Four-Phase Investigation of a Timeout Failure
|
|
2
|
+
|
|
3
|
+
Symptom: `TC-220_ExportReport` fails intermittently with
|
|
4
|
+
`TimeoutError: waiting for selector '.export-toast'` after 30s.
|
|
5
|
+
|
|
6
|
+
## Phase 1 — Root Cause Investigation (NO fixes yet)
|
|
7
|
+
|
|
8
|
+
- Read the full stack trace: timeout is on the toast selector, not the
|
|
9
|
+
export click itself.
|
|
10
|
+
- Reproduce: fails ~1 in 5 runs on large datasets; never on small ones.
|
|
11
|
+
Evidence-gathering, not guessing: the dataset-size correlation is recorded.
|
|
12
|
+
- Recent changes: export endpoint moved to a queued job last sprint
|
|
13
|
+
(git log).
|
|
14
|
+
- Component-boundary check (run once): UI click reaches the API (201 logged);
|
|
15
|
+
API enqueues the job (job row appears); job worker completes in 8-45s
|
|
16
|
+
depending on dataset size. The UI waits a fixed 30s. Which layer breaks?
|
|
17
|
+
The assumption "export finishes quickly" — not any single component.
|
|
18
|
+
|
|
19
|
+
## Phase 2 — Pattern Analysis
|
|
20
|
+
|
|
21
|
+
- Working example: the import flow already polls a job-status endpoint with
|
|
22
|
+
a completion condition instead of waiting on a fixed toast timeout.
|
|
23
|
+
- Difference list: export flow waits on a UI toast; import flow polls the
|
|
24
|
+
job resource with a condition. One difference, however small it looks.
|
|
25
|
+
|
|
26
|
+
## Phase 3 — Hypothesis
|
|
27
|
+
|
|
28
|
+
> "The failure is caused by waiting on a time-bounded UI toast instead of
|
|
29
|
+
> polling the export job status; large datasets exceed the toast window."
|
|
30
|
+
|
|
31
|
+
Minimal test: poll the job endpoint until `state == 'done'` (condition-based
|
|
32
|
+
wait) on the large dataset, 10 consecutive runs. No other change.
|
|
33
|
+
|
|
34
|
+
## Phase 4 — Implementation
|
|
35
|
+
|
|
36
|
+
1. RED test: large-dataset export asserts on the polled job state — fails
|
|
37
|
+
against the toast-wait implementation (reproduces the defect honestly).
|
|
38
|
+
2. Single fix: replace the toast wait with the condition-based job poll,
|
|
39
|
+
mirroring the import flow pattern. Toast assertion kept as a secondary
|
|
40
|
+
visible-feedback check (never cut existing validation).
|
|
41
|
+
3. Verify: RED test passes; full project suite re-run — output quoted in
|
|
42
|
+
the report; lessons recorded via `tools/cache_resolver.py record-lesson`.
|
|
43
|
+
4. Anti-pattern rejected: bumping the timeout to 90s would have hidden the
|
|
44
|
+
root cause and broken the small-dataset UX latency signal.
|
|
@@ -0,0 +1,114 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: test-driven-development
|
|
3
|
+
description: "Enforce the RED-GREEN-REFACTOR cycle: no production or automation code is written before a failing test exists and has been watched failing. Use when implementing new helpers, Page Class logic, bug fixes, or any behavior change in test codebases."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Test-Driven Development — Enterprise Red-Green-Refactor Standard
|
|
7
|
+
|
|
8
|
+
Specialized skill for writing test-automation and tool code test-first: write the test, watch it fail for the right reason, write minimal code to pass, then refactor while staying green.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## 1. The Iron Law
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
- Code written before the test is deleted — not kept "as reference", not "adapted while writing tests". Delete means delete; implement fresh from the tests.
|
|
19
|
+
- "Violating the letter of the rules is violating the spirit of the rules." Thinking "skip TDD just this once" is rationalization, not judgment.
|
|
20
|
+
|
|
21
|
+
## 2. The Cycle
|
|
22
|
+
|
|
23
|
+
```mermaid
|
|
24
|
+
flowchart LR
|
|
25
|
+
R["<b>RED</b><br/>Write failing test"] --> VR{"Fails<br/>correctly?"}
|
|
26
|
+
VR -- no --> R
|
|
27
|
+
VR -- yes --> G["<b>GREEN</b><br/>Minimal code"]
|
|
28
|
+
G --> VG{"Passes +<br/>suite green?"}
|
|
29
|
+
VG -- no --> G
|
|
30
|
+
VG -- yes --> F["<b>REFACTOR</b><br/>Clean up, stay green"]
|
|
31
|
+
F --> R
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
### RED — Write the Failing Test
|
|
35
|
+
|
|
36
|
+
- One behavior per test; a name that describes the behavior; real code (mocks only when unavoidable).
|
|
37
|
+
- **Verify RED is mandatory:** run the test and confirm it fails for the expected reason — feature missing, not a typo. A test that passes immediately is testing existing behavior; fix the test, not the code.
|
|
38
|
+
|
|
39
|
+
### GREEN — Minimal Code
|
|
40
|
+
|
|
41
|
+
- Simplest code that passes the test. No features, no refactoring of other code, no "improvements" beyond what the test demands.
|
|
42
|
+
- **Verify GREEN is mandatory:** the new test passes AND the project suite still passes. A green single file is not a green suite — run the project's test command before calling the change done. Any failure that run shows, including one you did not cause, goes into your report by name; a red test scrolled past unmentioned is a report falsified by omission.
|
|
43
|
+
|
|
44
|
+
### REFACTOR — Clean Up While Green
|
|
45
|
+
|
|
46
|
+
- Remove duplication, improve names, extract helpers. Keep tests green; add no behavior.
|
|
47
|
+
|
|
48
|
+
## 3. Good Tests
|
|
49
|
+
|
|
50
|
+
| Quality | Good | Bad |
|
|
51
|
+
|---|---|---|
|
|
52
|
+
| Minimal | One thing; "and" in the name means split it | `test('validates email and domain and whitespace')` |
|
|
53
|
+
| Clear | Name describes behavior | `test('test1')` |
|
|
54
|
+
| Intent-revealing | Demonstrates the desired API | Obscures what the code should do |
|
|
55
|
+
| Honest | Asserts real behavior; you can name the production change that would make it fail | Asserts on mock behavior only |
|
|
56
|
+
|
|
57
|
+
## 4. Hub Integration
|
|
58
|
+
|
|
59
|
+
- **Fake-pass synergy:** test-first pairs with `python3 tools/infer_linter.py` (flags 0-assert and tautological tests) and Gate 8 — `python3 tools/micro_mutator.py` requires Mutation Score >= 80%, which a test written AFTER the code rarely achieves honestly.
|
|
60
|
+
- **Bug fixes:** the bug reproduction IS the RED test. Never fix a bug without a test that fails before the fix and passes after.
|
|
61
|
+
- **Locator logic:** selector helpers and Page Class methods follow the same cycle; live verification (Stage 2 of `playwright-live-interaction`, 100% hit rate under 200ms) is the RED check for locator changes.
|
|
62
|
+
|
|
63
|
+
## 5. Common Rationalizations (All Rejected)
|
|
64
|
+
|
|
65
|
+
| Excuse | Reality |
|
|
66
|
+
|---|---|
|
|
67
|
+
| "Too simple to test" | Simple code breaks; the test takes 30 seconds. |
|
|
68
|
+
| "I'll test after" | Tests written after pass immediately — which proves nothing. You never watched it fail, so you never proved it can catch the bug. |
|
|
69
|
+
| "Already manually tested" | Ad-hoc checking has no record and no repeatability; "worked when I tried it" is not coverage. |
|
|
70
|
+
| "Keep as reference, write tests first" | You will adapt it. That is testing after. Delete means delete. |
|
|
71
|
+
| "Need to explore first" | Fine — explore, throw the exploration away, start with TDD. |
|
|
72
|
+
| "Test hard = skip" | Hard to test means hard to use; listen to the design signal. |
|
|
73
|
+
| "TDD will slow me down" | TDD catches bugs before commit and lets you refactor without fear; the pragmatic shortcut is debugging in production. |
|
|
74
|
+
|
|
75
|
+
## 6. Red Flags — Stop and Start Over
|
|
76
|
+
|
|
77
|
+
- Code before test; test written after implementation
|
|
78
|
+
- Test passes immediately on first run
|
|
79
|
+
- Cannot explain why the test failed
|
|
80
|
+
- Tests promised "later", or rationalizing "just this once"
|
|
81
|
+
|
|
82
|
+
All of these mean: delete the code and restart the cycle from RED.
|
|
83
|
+
|
|
84
|
+
## 7. Verification Checklist
|
|
85
|
+
|
|
86
|
+
Before marking any implementation complete:
|
|
87
|
+
|
|
88
|
+
- [ ] Every new function/method has a test
|
|
89
|
+
- [ ] Each test was watched failing, for the expected reason
|
|
90
|
+
- [ ] Minimal code was written to pass each test
|
|
91
|
+
- [ ] The project's full test command was run and its output quoted
|
|
92
|
+
- [ ] Output is pristine (no errors, warnings)
|
|
93
|
+
- [ ] Edge cases and error paths covered
|
|
94
|
+
|
|
95
|
+
Cannot check all boxes? The cycle was skipped — start over.
|
|
96
|
+
|
|
97
|
+
## 8. Boundary vs Sibling Skills (Do Not Confuse)
|
|
98
|
+
|
|
99
|
+
| Sibling | Owns | Hand-off |
|
|
100
|
+
|---|---|---|
|
|
101
|
+
| `systematic-debugging` | Root-cause investigation of failures | Phase 4 hands the reproduced bug to this skill as the RED test |
|
|
102
|
+
| `verification-before-completion` | Evidence protocol for completion claims | This skill's GREEN check feeds the evidence that protocol quotes |
|
|
103
|
+
| `antigravity-test-sentinel` | Hard completion blocking on gates | The sentinel enforces what this skill produces (fresh, watched, honest tests) |
|
|
104
|
+
| `surgical-coding-principles` | Diff discipline | Goal-Driven Execution (principle 4) is operationalized by this cycle |
|
|
105
|
+
|
|
106
|
+
## 9. Reference Documentation
|
|
107
|
+
|
|
108
|
+
- `python3 tools/infer_linter.py` — AST scan for fake passes and 0-assert tests.
|
|
109
|
+
- `python3 tools/micro_mutator.py` — Gate 8 mutation floor (>= 80%) validating test honesty.
|
|
110
|
+
- [Worked QA example (`red_green_refactor_qa.md`)](examples/red_green_refactor_qa.md): full cycle for a retry helper in a test utility.
|
|
111
|
+
|
|
112
|
+
## 10. Attribution
|
|
113
|
+
|
|
114
|
+
Adapted from **superpowers** skill `test-driven-development` (https://github.com/obra/superpowers) — MIT License, Copyright (c) 2025 Jesse Vincent. The Iron Law, RED-GREEN-REFACTOR cycle, and rationalization table are re-scoped for enterprise QA automation in Synapse Hub; wording, structure, and examples modified.
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# Example: RED-GREEN-REFACTOR for a Test Utility Retry Helper
|
|
2
|
+
|
|
3
|
+
Task: "Our Playwright API calls occasionally race the dev server; add retry
|
|
4
|
+
to the request helper."
|
|
5
|
+
|
|
6
|
+
## RED — write the failing test first
|
|
7
|
+
|
|
8
|
+
```typescript
|
|
9
|
+
test('retries a failing request up to 3 attempts then succeeds', async () => {
|
|
10
|
+
let attempts = 0;
|
|
11
|
+
const flakyCall = async () => {
|
|
12
|
+
attempts += 1;
|
|
13
|
+
if (attempts < 3) throw new Error('ECONNRESET');
|
|
14
|
+
return { status: 200 };
|
|
15
|
+
};
|
|
16
|
+
|
|
17
|
+
const result = await requestWithRetry(flakyCall);
|
|
18
|
+
|
|
19
|
+
expect(result.status).toBe(200);
|
|
20
|
+
expect(attempts).toBe(3);
|
|
21
|
+
});
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
## Verify RED — MANDATORY
|
|
25
|
+
|
|
26
|
+
```text
|
|
27
|
+
$ npx vitest run tests/unit/retry.spec.ts
|
|
28
|
+
FAIL: requestWithRetry is not a function
|
|
29
|
+
```
|
|
30
|
+
|
|
31
|
+
Fails for the right reason: the feature does not exist yet. (An error like
|
|
32
|
+
"cannot read properties of undefined" would mean a typo — fix the test
|
|
33
|
+
before writing code.)
|
|
34
|
+
|
|
35
|
+
## GREEN — minimal implementation
|
|
36
|
+
|
|
37
|
+
```typescript
|
|
38
|
+
export async function requestWithRetry<T>(
|
|
39
|
+
call: () => Promise<T>,
|
|
40
|
+
maxAttempts = 3,
|
|
41
|
+
): Promise<T> {
|
|
42
|
+
let lastError: unknown;
|
|
43
|
+
for (let attempt = 1; attempt <= maxAttempts; attempt += 1) {
|
|
44
|
+
try {
|
|
45
|
+
return await call();
|
|
46
|
+
} catch (error) {
|
|
47
|
+
lastError = error;
|
|
48
|
+
}
|
|
49
|
+
}
|
|
50
|
+
throw lastError;
|
|
51
|
+
}
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
## Verify GREEN — MANDATORY
|
|
55
|
+
|
|
56
|
+
```text
|
|
57
|
+
$ npx vitest run tests/unit/retry.spec.ts
|
|
58
|
+
PASS 1/1
|
|
59
|
+
$ npm test # the PROJECT suite, not just this file
|
|
60
|
+
PASS 214/214
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
A green single file is not a green suite — the project command is the
|
|
64
|
+
acceptance run, and its output is quoted in the report.
|
|
65
|
+
|
|
66
|
+
## REFACTOR — while green
|
|
67
|
+
|
|
68
|
+
No duplication to remove yet; the next failing test (e.g. "gives up after 3
|
|
69
|
+
attempts and rethrows") starts the next cycle. Adding exponential backoff NOW
|
|
70
|
+
would be speculative — nobody asked for it.
|
|
71
|
+
|
|
72
|
+
## Honesty check
|
|
73
|
+
|
|
74
|
+
- The test uses real behavior (a counting closure), not a mock asserting on
|
|
75
|
+
itself.
|
|
76
|
+
- Gate 8 wiring: `tools/micro_mutator.py` inverts the attempt boundary — the
|
|
77
|
+
test must survive the mutant (kill it), keeping Mutation Score >= 80%.
|
|
78
|
+
- `tools/infer_linter.py` finds both assertions present and meaningful.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: token-and-context-optimizer
|
|
3
|
-
description: "Optimize prompt context, distill verbose logs, and prevent context rot in Antigravity."
|
|
3
|
+
description: "Optimize prompt context, distill verbose logs, and prevent context rot in Antigravity. Use when optimizing prompts or reducing token and context overhead."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Token & Context Optimizer — Antigravity Engineering Standard
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: verification-before-completion
|
|
3
|
+
description: "Require fresh verification evidence — a command run in the current session with its real output — before any claim that work is complete, fixed, or passing. Use when about to report success, commit, open a PR, or close a task."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Verification Before Completion — Enterprise Evidence Protocol
|
|
7
|
+
|
|
8
|
+
Specialized skill for the evidence discipline behind every success claim in the hub: no completion statement exists without fresh verification output. This is the operational form of the Truth Anchor (No Fake Pass) rule in `AGENTS.md` Section 3.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## 1. The Iron Law
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
NO COMPLETION CLAIMS WITHOUT FRESH VERIFICATION EVIDENCE
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
If the verification command was not run in the current message turn, the claim cannot be made. "Evidence before claims, always."
|
|
19
|
+
|
|
20
|
+
## 2. The Gate Function
|
|
21
|
+
|
|
22
|
+
Run before claiming ANY status or expressing satisfaction:
|
|
23
|
+
|
|
24
|
+
1. **IDENTIFY** — which command proves this claim?
|
|
25
|
+
2. **RUN** — execute the full command, fresh and complete (not a partial rerun, not a memory of an earlier run).
|
|
26
|
+
3. **READ** — full output, exit code, failure count.
|
|
27
|
+
4. **VERIFY** — does the output actually confirm the claim? If no: state the real status with the evidence. If yes: state the claim WITH the evidence.
|
|
28
|
+
5. **ONLY THEN** — make the claim.
|
|
29
|
+
|
|
30
|
+
Skipping any step is lying, not verifying.
|
|
31
|
+
|
|
32
|
+
## 3. Claim → Required Evidence Matrix
|
|
33
|
+
|
|
34
|
+
| Claim | Requires | Not Sufficient |
|
|
35
|
+
|---|---|---|
|
|
36
|
+
| Tests pass | Test command output in this session: 0 failures | Previous run, "should pass" |
|
|
37
|
+
| 9 Quality Gates pass | `python3 tools/verify_all.py --mode hub` output: 9/9 | Gates passing last week |
|
|
38
|
+
| No fake passes | `python3 tools/infer_linter.py` output: 0 findings | "Assertions look fine" |
|
|
39
|
+
| Mutation floor met | `python3 tools/micro_mutator.py` score >= 80% | Tests exist, so "they must kill mutants" |
|
|
40
|
+
| Bug fixed | Reproduction test passes now | Code changed, assumed fixed |
|
|
41
|
+
| Regression test works | Red-green cycle verified (watched fail, then pass) | Test passes once |
|
|
42
|
+
| Subagent finished | Its diff inspected + its claims re-verified independently | Agent reports "success" |
|
|
43
|
+
| Requirements met | Line-by-line checklist against the original request | Tests passing |
|
|
44
|
+
|
|
45
|
+
## 4. Red Flags — STOP
|
|
46
|
+
|
|
47
|
+
- Using "should", "probably", "seems to", "definitely works"
|
|
48
|
+
- Expressing satisfaction before verification ("Perfect!", "Done!")
|
|
49
|
+
- About to commit, push, or open a PR without verification in this turn
|
|
50
|
+
- Trusting a subagent's success report without independent re-verification
|
|
51
|
+
- Relying on a partial check (linter passed, therefore build must pass)
|
|
52
|
+
- Thinking "just this once"
|
|
53
|
+
|
|
54
|
+
## 5. Rationalization Prevention
|
|
55
|
+
|
|
56
|
+
| Excuse | Reality |
|
|
57
|
+
|---|---|
|
|
58
|
+
| "Should work now" | RUN the verification. |
|
|
59
|
+
| "I'm confident" | Confidence is not evidence. |
|
|
60
|
+
| "Just this once" | No exceptions — one unverifiable claim poisons the whole report. |
|
|
61
|
+
| "Linter passed" | The linter is not the compiler; linter does not equal build. |
|
|
62
|
+
| "Agent said success" | Verify independently; subagent reports are claims, not evidence. |
|
|
63
|
+
| "Partial check is enough" | Partial proves nothing about the whole. |
|
|
64
|
+
| "Different words, so the rule doesn't apply" | Spirit over letter — paraphrases and implications of success are covered. |
|
|
65
|
+
|
|
66
|
+
## 6. Key Patterns
|
|
67
|
+
|
|
68
|
+
**Tests:** run command, quote `N/N pass`, then claim "all tests pass". Never "looks correct".
|
|
69
|
+
|
|
70
|
+
**Regression (TDD red-green):** write test, run (MUST FAIL), apply fix, run (passes), quote both runs. A regression test without a watched failure is unproven.
|
|
71
|
+
|
|
72
|
+
**Quality gates:** `python3 tools/verify_all.py --mode hub` fresh output with the 9/9 line quoted — required before any "task done" statement in swarm Stage 5.
|
|
73
|
+
|
|
74
|
+
**Subagent delegation:** agent reports success, orchestrator inspects the diff and re-runs the decisive command before relaying any success upward.
|
|
75
|
+
|
|
76
|
+
**Requirements:** re-read the original request, build the checklist, verify each item, report gaps or completion — "tests pass, phase complete" without the checklist is insufficient.
|
|
77
|
+
|
|
78
|
+
## 7. When To Apply
|
|
79
|
+
|
|
80
|
+
Always before: ANY success or completion claim, ANY positive statement about work state, committing or PR creation, task closure, moving to the next task, delegating to subagents.
|
|
81
|
+
|
|
82
|
+
Applies to exact phrases, paraphrases, synonyms, and any communication implying completion or correctness.
|
|
83
|
+
|
|
84
|
+
## 8. Boundary vs Sibling Skills (Do Not Confuse)
|
|
85
|
+
|
|
86
|
+
| Sibling | Owns | Hand-off |
|
|
87
|
+
|---|---|---|
|
|
88
|
+
| `antigravity-test-sentinel` | Hard completion blocking until tests and gates pass | The sentinel is the enforcement gate; this skill is the evidence protocol the agent follows to pass it honestly |
|
|
89
|
+
| `test-driven-development` | RED-GREEN-REFACTOR cycle | GREEN verification output feeds the evidence quoted here |
|
|
90
|
+
| `systematic-debugging` | Root-cause investigation | Phase 4 fix verification uses this protocol for its "fixed" claim |
|
|
91
|
+
|
|
92
|
+
## 9. Reference Documentation
|
|
93
|
+
|
|
94
|
+
- `python3 tools/verify_all.py --mode hub` — the 9-gate acceptance command for hub work.
|
|
95
|
+
- `python3 tools/infer_linter.py` — fake-pass scan required before any "tests are honest" claim.
|
|
96
|
+
- [Evidence log example (`evidence_log_example.md`)](examples/evidence_log_example.md): claim/evidence pairing for a swarm task closure.
|
|
97
|
+
|
|
98
|
+
## 10. Attribution
|
|
99
|
+
|
|
100
|
+
Adapted from **superpowers** skill `verification-before-completion` (https://github.com/obra/superpowers) — MIT License, Copyright (c) 2025 Jesse Vincent. The Iron Law, gate function, claim-evidence matrix, and rationalization table are re-scoped for enterprise QA automation in Synapse Hub; wording, structure, and hub tooling modified.
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Example: Claim/Evidence Log for a Swarm Task Closure
|
|
2
|
+
|
|
3
|
+
Task: "Fix the flaky export test and get the suite green."
|
|
4
|
+
Subagent: test-runner -> failure-analyst -> quality-gatekeeper (Stage 5).
|
|
5
|
+
|
|
6
|
+
## Claim 1: "The flaky root cause is fixed"
|
|
7
|
+
|
|
8
|
+
| Field | Value |
|
|
9
|
+
|---|---|
|
|
10
|
+
| Claim | Export timeout root cause fixed via condition-based job poll |
|
|
11
|
+
| Evidence command | `npx playwright test tests/e2e/TC-220_export.spec.ts --repeat-each=10` |
|
|
12
|
+
| Fresh output (this turn) | `10 passed (94.2s)` |
|
|
13
|
+
| Verdict | Claim SUPPORTED — quoted with the output |
|
|
14
|
+
|
|
15
|
+
## Claim 2: "The full suite is green"
|
|
16
|
+
|
|
17
|
+
| Field | Value |
|
|
18
|
+
|---|---|
|
|
19
|
+
| Claim | Project suite green after the fix |
|
|
20
|
+
| Evidence command | `npm test` (the project command, not one file) |
|
|
21
|
+
| Fresh output (this turn) | `Tests: 2 failed, 214 passed` |
|
|
22
|
+
| Verdict | Claim REFUTED — report states 2 failures in TC-118/TC-140 with
|
|
23
|
+
distilled failure blocks (tools/distill_log.py). Work is NOT done. |
|
|
24
|
+
|
|
25
|
+
The 2 failures were pre-existing and not caused by the fix — still reported
|
|
26
|
+
by name. A red test scrolled past unmentioned is a report falsified by
|
|
27
|
+
omission.
|
|
28
|
+
|
|
29
|
+
## Claim 3 (after the second fix cycle): "Task complete, 9/9 gates"
|
|
30
|
+
|
|
31
|
+
| Field | Value |
|
|
32
|
+
|---|---|
|
|
33
|
+
| Evidence commands | `npm test` -> `Tests: 216 passed`; then
|
|
34
|
+
`python3 tools/verify_all.py --mode hub` |
|
|
35
|
+
| Fresh output (this turn) | `Gate 0..8: PASS — 9/9 gates` and mutation score
|
|
36
|
+
`0.86 (>= 0.80)` |
|
|
37
|
+
| Requirement checklist | Original request re-read; both requested items
|
|
38
|
+
(stabilize export test; suite green) verified line by line |
|
|
39
|
+
| Verdict | "Task complete" claim MADE — with all evidence attached |
|
|
40
|
+
|
|
41
|
+
## Anti-pattern (rejected)
|
|
42
|
+
|
|
43
|
+
> "All good, the suite is green now, ready to merge."
|
|
44
|
+
|
|
45
|
+
No command, no output, no turn — this is a claim without evidence. Under the
|
|
46
|
+
Iron Law it is not verification, it is a guess wearing a checkmark.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: visual-regression-sentinel
|
|
3
|
-
description: "Run pixel and layout shift visual regression checks with anti-flake masking."
|
|
3
|
+
description: "Run pixel and layout shift visual regression checks with anti-flake masking. Use when comparing screenshots for visual regressions with anti-flake masking."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Visual Regression Sentinel Skill
|
|
@@ -60,3 +60,23 @@ When visual regression fails:
|
|
|
60
60
|
2. Determine if it is a **Real Regression** (CSS bug, broken layout) or **Intentional Redesign**.
|
|
61
61
|
3. If intentional redesign: update baseline snapshots via `npx playwright test --update-snapshots`.
|
|
62
62
|
4. If bug: file bug report with side-by-side artifact comparison.
|
|
63
|
+
|
|
64
|
+
---
|
|
65
|
+
|
|
66
|
+
## 4. Assert on Extracted Data, Not on the Model's Claim
|
|
67
|
+
|
|
68
|
+
> *Borrowed and adapted from the Midscene.js prompting model (MIT license, web-infra-dev/midscene).*
|
|
69
|
+
|
|
70
|
+
When an AI/vision agent reports a step as "done", that claim is **not evidence**:
|
|
71
|
+
|
|
72
|
+
1. **Extract the visible state first** — pull structured data from what is
|
|
73
|
+
actually rendered (accessibility tree text, OCR'd region, or `aiQuery`-style
|
|
74
|
+
structured extraction), never the model's self-report.
|
|
75
|
+
2. **Assert on the extracted values** — pair every extraction with a real
|
|
76
|
+
assertion (`expect(...)`/`verify...`) against the expected value; a vision
|
|
77
|
+
model can read a screen, but only an assertion can *prove* it.
|
|
78
|
+
3. **Vision sees pixels, not DOM internals** — assertions on tab counts,
|
|
79
|
+
network state, or hidden attributes are invisible to the model; assert on
|
|
80
|
+
what is rendered.
|
|
81
|
+
4. **Only then run the visual diff** — feed the §3 autopsy protocol; a green
|
|
82
|
+
extraction plus a clean diff is proven, everything else is owed.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: wcag-audit-patterns
|
|
3
|
-
description: "Audit WCAG 2.2 accessibility (A/AA), keyboard navigation, and ARIA roles."
|
|
3
|
+
description: "Audit WCAG 2.2 accessibility (A/AA), keyboard navigation, and ARIA roles. Use when auditing accessibility against WCAG 2.2 A/AA criteria."
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# WCAG 2.2 Accessibility Audit Patterns
|
|
@@ -70,8 +70,8 @@ Agent can automatically audit accessibility for any HTML file or live page URL v
|
|
|
70
70
|
|
|
71
71
|
```bash
|
|
72
72
|
# Audit a local HTML snapshot
|
|
73
|
-
|
|
73
|
+
python3 ~/.zcode/synapse-hub/tools/wcag_audit.py path/to/page.html -o docs/wcag_audit_report.md
|
|
74
74
|
|
|
75
75
|
# Audit a live application URL directly
|
|
76
|
-
|
|
76
|
+
python3 ~/.zcode/synapse-hub/tools/wcag_audit.py https://staging.example.com --url -o docs/wcag_audit_report.md
|
|
77
77
|
```
|