@teevee98/synapse-hub 0.0.0-stage → 3.8.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (369) hide show
  1. package/.agents/agents/api-engineer.md +67 -0
  2. package/.agents/agents/failure-analyst.md +75 -0
  3. package/.agents/agents/live-explorer.md +70 -0
  4. package/.agents/agents/orchestrator.md +104 -0
  5. package/.agents/agents/perf-engineer.md +74 -0
  6. package/.agents/agents/pom-crafter.md +112 -0
  7. package/.agents/agents/quality-gatekeeper.md +87 -0
  8. package/.agents/agents/test-designer.md +73 -0
  9. package/.agents/agents/test-runner.md +67 -0
  10. package/.agents/rules/author-integrity-guard.md +33 -0
  11. package/.agents/rules/communication-style.md +18 -0
  12. package/.agents/rules/context-engineering.md +37 -0
  13. package/.agents/rules/git-push-protocol.md +33 -0
  14. package/.agents/rules/graphify.md +14 -0
  15. package/.agents/rules/lessons-learned.md +25 -0
  16. package/.agents/rules/mandatory-playwright-live.md +25 -0
  17. package/.agents/rules/mobile_appium_rules.md +43 -0
  18. package/.agents/rules/performance_testing_rules.md +35 -0
  19. package/.agents/rules/skill-precedence.md +29 -0
  20. package/.agents/rules/systematic-debugging.md +36 -0
  21. package/.agents/skills/actor-qa-swarm/SKILL.md +77 -0
  22. package/.agents/skills/actor-qa-swarm/examples/broker_sharding_sample.json +12 -0
  23. package/.agents/skills/agent-orchestration-multi-agent-optimize/SKILL.md +88 -0
  24. package/.agents/skills/agent-orchestration-multi-agent-optimize/examples/swarm_workload_config.json +15 -0
  25. package/.agents/skills/agentless-qa-repair/SKILL.md +69 -0
  26. package/.agents/skills/agentless-qa-repair/examples/healing_report_sample.json +17 -0
  27. package/.agents/skills/agentless-qa-repair/examples/three_phase_repair_workflow.md +40 -0
  28. package/.agents/skills/antigravity-test-sentinel/SKILL.md +63 -0
  29. package/.agents/skills/antigravity-test-sentinel/examples/dod_gate_policy.json +11 -0
  30. package/.agents/skills/antigravity-test-sentinel/examples/sprint_dod_enforcement.md +47 -0
  31. package/.agents/skills/api-network-mocking-interception/SKILL.md +142 -0
  32. package/.agents/skills/api-network-mocking-interception/examples/network_route_stub.json +9 -0
  33. package/.agents/skills/api-spec-to-e2e-matrix/SKILL.md +80 -0
  34. package/.agents/skills/api-spec-to-e2e-matrix/examples/sample_openapi.json +19 -0
  35. package/.agents/skills/architecture-decision-records/SKILL.md +64 -0
  36. package/.agents/skills/architecture-decision-records/examples/0001-record-architecture-decisions.md +12 -0
  37. package/.agents/skills/auth-session-and-cookie-manager/SKILL.md +75 -0
  38. package/.agents/skills/auth-session-and-cookie-manager/examples/multi_role_auth_matrix.md +70 -0
  39. package/.agents/skills/auth-session-and-cookie-manager/examples/storage_state_sample.json +11 -0
  40. package/.agents/skills/backend-load-and-stress-patterns/SKILL.md +85 -0
  41. package/.agents/skills/backend-load-and-stress-patterns/examples/k6_load_test_sample.js +13 -0
  42. package/.agents/skills/backend-load-and-stress-patterns/references/k6_load_template.js +82 -0
  43. package/.agents/skills/bats-testing-patterns/SKILL.md +100 -0
  44. package/.agents/skills/bats-testing-patterns/examples/cli_runner_test.bats +6 -0
  45. package/.agents/skills/bats-testing-patterns/resources/implementation-playbook.md +614 -0
  46. package/.agents/skills/clean-pom-crafter/SKILL.md +110 -0
  47. package/.agents/skills/clean-pom-crafter/examples/standard_test_case.groovy +28 -0
  48. package/.agents/skills/clean-pom-crafter/references/pom_patterns.groovy +138 -0
  49. package/.agents/skills/code-refactoring-refactor-clean/SKILL.md +64 -0
  50. package/.agents/skills/code-refactoring-refactor-clean/examples/refactored_page_sample.groovy +9 -0
  51. package/.agents/skills/code-refactoring-refactor-clean/examples/refactored_playwright_pom.ts +52 -0
  52. package/.agents/skills/code-refactoring-refactor-clean/resources/implementation-playbook.md +881 -0
  53. package/.agents/skills/consumer-driven-contract-testing/SKILL.md +147 -0
  54. package/.agents/skills/consumer-driven-contract-testing/examples/order_service_pact.json +45 -0
  55. package/.agents/skills/db-state-verifier-and-seeder/SKILL.md +123 -0
  56. package/.agents/skills/db-state-verifier-and-seeder/examples/db_fixture_seed.json +12 -0
  57. package/.agents/skills/e2e-testing-patterns/SKILL.md +103 -0
  58. package/.agents/skills/e2e-testing-patterns/examples/playwright_sharded_config.ts +12 -0
  59. package/.agents/skills/e2e-testing-patterns/resources/implementation-playbook.md +522 -0
  60. package/.agents/skills/event-stream-qa-orchestrator/SKILL.md +79 -0
  61. package/.agents/skills/event-stream-qa-orchestrator/examples/timeline_sample.jsonl +2 -0
  62. package/.agents/skills/failure-autopsy-and-healing/SKILL.md +66 -0
  63. package/.agents/skills/failure-autopsy-and-healing/examples/closed_loop_healing_walkthrough.md +48 -0
  64. package/.agents/skills/failure-autopsy-and-healing/examples/sample_junit_failure.xml +8 -0
  65. package/.agents/skills/failure-autopsy-and-healing/references/healing_matrix.md +64 -0
  66. package/.agents/skills/failure-autopsy-and-healing/references/qa_investigation_protocol.md +56 -0
  67. package/.agents/skills/failure-autopsy-and-healing/scripts/triage_failures.sh +83 -0
  68. package/.agents/skills/file-and-report-validator/SKILL.md +90 -0
  69. package/.agents/skills/file-and-report-validator/examples/sample_export_schema.json +14 -0
  70. package/.agents/skills/flaky-test-sentinel-and-quarantine/SKILL.md +114 -0
  71. package/.agents/skills/flaky-test-sentinel-and-quarantine/examples/quarantine_registry.json +11 -0
  72. package/.agents/skills/gherkin-to-suite/SKILL.md +103 -0
  73. package/.agents/skills/gherkin-to-suite/examples/checkout.feature +39 -0
  74. package/.agents/skills/gherkin-to-suite/examples/suite_report_sample.json +334 -0
  75. package/.agents/skills/github-actions-templates/SKILL.md +77 -0
  76. package/.agents/skills/github-actions-templates/examples/matrix_test_pipeline.yml +8 -0
  77. package/.agents/skills/graphify-impact-analyzer/SKILL.md +76 -0
  78. package/.agents/skills/graphify-impact-analyzer/examples/dependency_blast_radius.json +8 -0
  79. package/.agents/skills/katalon-cli-runner/SKILL.md +76 -0
  80. package/.agents/skills/katalon-cli-runner/examples/headless_kre_exec.sh +3 -0
  81. package/.agents/skills/katalon-cli-runner/references/groovy-patterns.md +209 -0
  82. package/.agents/skills/katalon-cli-runner/references/katalon-api.md +155 -0
  83. package/.agents/skills/katalon-cli-runner/references/runner_commands.md +67 -0
  84. package/.agents/skills/lam-script-generator/SKILL.md +123 -0
  85. package/.agents/skills/lam-script-generator/examples/user_story_sample.md +8 -0
  86. package/.agents/skills/lean-code-ladder/SKILL.md +94 -0
  87. package/.agents/skills/lean-code-ladder/examples/ladder_walkthrough.md +37 -0
  88. package/.agents/skills/lean-impact-scoreboard/SKILL.md +48 -0
  89. package/.agents/skills/lean-impact-scoreboard/examples/sample_benchmark_gain.md +20 -0
  90. package/.agents/skills/lean-impact-scoreboard/examples/sample_scoreboard.md +15 -0
  91. package/.agents/skills/meta-prompt-crafter/SKILL.md +107 -0
  92. package/.agents/skills/meta-prompt-crafter/examples/qa_prompt_template.md +35 -0
  93. package/.agents/skills/mipro-prompt-compiler/SKILL.md +59 -0
  94. package/.agents/skills/mipro-prompt-compiler/examples/compiled_module_sample.json +22 -0
  95. package/.agents/skills/mipro-prompt-compiler/examples/dspy_assertion_optimizer_case.md +46 -0
  96. package/.agents/skills/mobile-appium-som/SKILL.md +88 -0
  97. package/.agents/skills/mobile-appium-som/examples/BaseScreen.java +88 -0
  98. package/.agents/skills/mobile-appium-som/examples/LoginScreen.java +46 -0
  99. package/.agents/skills/mobile-appium-som/examples/TC_MobileLogin.java +43 -0
  100. package/.agents/skills/persistent-planning-files/SKILL.md +108 -0
  101. package/.agents/skills/persistent-planning-files/examples/task_plan_example.md +56 -0
  102. package/.agents/skills/playwright-live-interaction/SKILL.md +131 -0
  103. package/.agents/skills/playwright-live-interaction/examples/live_probe_script.js +5 -0
  104. package/.agents/skills/playwright-live-interaction/references/advanced_superpowers.md +160 -0
  105. package/.agents/skills/playwright-live-interaction/references/compact_accessibility_snapshots.md +75 -0
  106. package/.agents/skills/playwright-live-interaction/references/recipes.md +388 -0
  107. package/.agents/skills/playwright-live-interaction/references/swarm_orchestration.md +134 -0
  108. package/.agents/skills/playwright-webapp-testing/SKILL.md +93 -0
  109. package/.agents/skills/playwright-webapp-testing/examples/server_lifecycle_playwright.py +63 -0
  110. package/.agents/skills/qa-aci-interface/SKILL.md +67 -0
  111. package/.agents/skills/qa-aci-interface/examples/surgical_code_window_walkthrough.md +40 -0
  112. package/.agents/skills/qa-aci-interface/examples/surgical_edit_sample.json +12 -0
  113. package/.agents/skills/qa-memory-os/SKILL.md +80 -0
  114. package/.agents/skills/qa-memory-os/examples/archival_memory_sample.json +21 -0
  115. package/.agents/skills/rbt-manual-testing/SKILL.md +137 -0
  116. package/.agents/skills/rbt-manual-testing/examples/checkout_rbt_manual_suite.md +25 -0
  117. package/.agents/skills/rbt-manual-testing/references/field_validation_checklist.md +29 -0
  118. package/.agents/skills/repo-quality-audit/SKILL.md +68 -0
  119. package/.agents/skills/repo-quality-audit/examples/sample_audit_walkthrough.md +15 -0
  120. package/.agents/skills/repo-quality-audit/examples/sample_repo_audit.md +33 -0
  121. package/.agents/skills/review-spec/SKILL.md +60 -0
  122. package/.agents/skills/review-spec/examples/pocock_grilling_decision_rubric.md +40 -0
  123. package/.agents/skills/review-spec/examples/sample_spec_review.md +33 -0
  124. package/.agents/skills/security-dast-sentinel/SKILL.md +119 -0
  125. package/.agents/skills/security-dast-sentinel/examples/dast_scan_config.json +10 -0
  126. package/.agents/skills/security-dast-sentinel/references/AI-AND-LLM.md +113 -0
  127. package/.agents/skills/security-dast-sentinel/references/ATTACK-CLASSES.md +89 -0
  128. package/.agents/skills/security-dast-sentinel/references/dast_recipes.ts +33 -0
  129. package/.agents/skills/self-healing-locator/SKILL.md +86 -0
  130. package/.agents/skills/self-healing-locator/examples/aom_diff_sample.json +16 -0
  131. package/.agents/skills/senior-code-review/SKILL.md +108 -0
  132. package/.agents/skills/senior-code-review/examples/sample_review_walkthrough.md +29 -0
  133. package/.agents/skills/skillspector-security-guard/SKILL.md +57 -0
  134. package/.agents/skills/skillspector-security-guard/examples/audit_threat_vectors_report.json +35 -0
  135. package/.agents/skills/skillspector-security-guard/examples/safe_mcp_config.json +12 -0
  136. package/.agents/skills/surgical-coding-principles/SKILL.md +89 -0
  137. package/.agents/skills/surgical-coding-principles/examples/surgical_vs_scope_creep.md +46 -0
  138. package/.agents/skills/synthetic-test-data-generator/SKILL.md +135 -0
  139. package/.agents/skills/synthetic-test-data-generator/examples/idempotent_data_recipe.json +9 -0
  140. package/.agents/skills/systematic-debugging/SKILL.md +100 -0
  141. package/.agents/skills/systematic-debugging/examples/four_phase_case.md +44 -0
  142. package/.agents/skills/technical-debt-harvester/SKILL.md +70 -0
  143. package/.agents/skills/technical-debt-harvester/examples/sample_debt_ledger.md +25 -0
  144. package/.agents/skills/technical-debt-harvester/examples/sample_ledger_output.md +20 -0
  145. package/.agents/skills/test-driven-development/SKILL.md +114 -0
  146. package/.agents/skills/test-driven-development/examples/red_green_refactor_qa.md +78 -0
  147. package/.agents/skills/token-and-context-optimizer/SKILL.md +70 -0
  148. package/.agents/skills/token-and-context-optimizer/examples/distilled_log_sample.txt +3 -0
  149. package/.agents/skills/token-and-context-optimizer/references/context_engineering_guide.md +49 -0
  150. package/.agents/skills/tot-exploratory-testing/SKILL.md +59 -0
  151. package/.agents/skills/tot-exploratory-testing/examples/payment_gateway_tot_tree.json +62 -0
  152. package/.agents/skills/tot-exploratory-testing/examples/tot_graph_sample.json +35 -0
  153. package/.agents/skills/verification-before-completion/SKILL.md +100 -0
  154. package/.agents/skills/verification-before-completion/examples/evidence_log_example.md +46 -0
  155. package/.agents/skills/visual-regression-sentinel/SKILL.md +82 -0
  156. package/.agents/skills/visual-regression-sentinel/examples/visual_mask_rules.json +9 -0
  157. package/.agents/skills/wcag-audit-patterns/SKILL.md +77 -0
  158. package/.agents/skills/wcag-audit-patterns/examples/wcag_compliance_checklist.json +10 -0
  159. package/.agents/skills/wcag-audit-patterns/resources/implementation-playbook.md +541 -0
  160. package/.agents/skills/web-agent-gym-benchmarking/SKILL.md +62 -0
  161. package/.agents/skills/web-agent-gym-benchmarking/examples/benchmark_report_sample.json +22 -0
  162. package/.agents/skills/web-agent-gym-benchmarking/examples/enterprise_tasks_suite.json +38 -0
  163. package/.agents/skills/web-vitals-and-performance-profiler/SKILL.md +65 -0
  164. package/.agents/skills/web-vitals-and-performance-profiler/examples/cwv_budget_policy.json +6 -0
  165. package/.agents/skills/web-vitals-and-performance-profiler/examples/playwright_cwv_profiler_fixture.ts +118 -0
  166. package/.agents/templates/completion_report.md +57 -0
  167. package/.agents/workflows/analyze_requirement_document.md +18 -0
  168. package/.agents/workflows/export_testcases_excel.md +32 -0
  169. package/.agents/workflows/generate_combinatorial_test_data.md +28 -0
  170. package/.agents/workflows/generate_cross_module_test_plan.md +37 -0
  171. package/.agents/workflows/generate_manual_testcases_rbt.md +56 -0
  172. package/.agents/workflows/generate_requirements_from_website.md +25 -0
  173. package/.agents/workflows/generate_testcases_from_requirements.md +21 -0
  174. package/.agents/workflows/graphify.md +10 -0
  175. package/.agents/workflows/test-impact.md +15 -0
  176. package/.agents/workflows/verify.md +34 -0
  177. package/.cursor/rules/agents.mdc +16 -0
  178. package/.cursor/rules/language.mdc +7 -0
  179. package/.cursor/rules/synapse-qa.mdc +24 -0
  180. package/.cursorrules +27 -0
  181. package/.dockerignore +53 -0
  182. package/.pre-commit-hooks.yaml +14 -0
  183. package/.windsurfrules +26 -0
  184. package/AGENTS.md +86 -0
  185. package/CLAUDE.md +54 -0
  186. package/Dockerfile +81 -0
  187. package/LICENSE +52 -0
  188. package/NOTICE +264 -0
  189. package/README.en.md +172 -0
  190. package/README.md +171 -2
  191. package/action.yml +51 -0
  192. package/agentskills.json +16 -0
  193. package/bin/agent-qa.js +138 -0
  194. package/docker-compose.yml +74 -0
  195. package/docs/ARCHITECTURE.en.md +197 -0
  196. package/docs/ARCHITECTURE.md +198 -0
  197. package/docs/CI_CD_INTEGRATION.en.md +170 -0
  198. package/docs/CI_CD_INTEGRATION.md +170 -0
  199. package/docs/CLI_REFERENCE.en.md +175 -0
  200. package/docs/CLI_REFERENCE.md +177 -0
  201. package/docs/CONTRIBUTING.en.md +49 -0
  202. package/docs/CONTRIBUTING.md +49 -0
  203. package/docs/PERFORMANCE_AUDIT_REPORT.md +784 -0
  204. package/docs/PLUGIN_DEVELOPMENT_GUIDE.md +127 -0
  205. package/docs/QUALITY_GATES.en.md +121 -0
  206. package/docs/QUALITY_GATES.md +121 -0
  207. package/docs/TOKEN_QUOTA_PLAYBOOK.md +41 -0
  208. package/docs/TOOL_CATALOG.md +129 -0
  209. package/docs/adopted-ecosystem-tools.md +96 -0
  210. package/docs/research/EXTERNAL_AGENT_SKILLS_SURVEY.md +1062 -0
  211. package/docs/research/agent_skill_training_analysis.md +1359 -0
  212. package/docs/schemas/synapse.config.schema.json +87 -0
  213. package/package.json +94 -4
  214. package/plugin.json +31 -0
  215. package/scripts/generate_skill_examples.py +244 -0
  216. package/scripts/install.ps1 +112 -0
  217. package/scripts/install_global_plugin.ps1 +94 -0
  218. package/scripts/install_global_plugin.sh +49 -0
  219. package/scripts/lint_skills.ps1 +57 -0
  220. package/scripts/package_dist.py +337 -0
  221. package/scripts/setup.sh +59 -0
  222. package/scripts/update_hub.ps1 +19 -0
  223. package/scripts/update_hub.sh +15 -0
  224. package/skills_manifest.json +854 -0
  225. package/synapse.config.json +32 -0
  226. package/templates/PROJECT_AI_GUIDE_TEMPLATE.md +111 -0
  227. package/templates/PROJECT_STATE_TEMPLATE.md +20 -0
  228. package/templates/ai_bridges/CLAUDE.md +54 -0
  229. package/templates/ai_bridges/copilot-instructions.md +20 -0
  230. package/templates/ai_bridges/cursor_synapse_qa.mdc +24 -0
  231. package/templates/ai_bridges/windsurfrules +26 -0
  232. package/templates/ci/github-actions-synapse.yml +95 -0
  233. package/templates/ci/gitlab-ci-synapse.yml +16 -0
  234. package/templates/ui/index.html +1069 -0
  235. package/templates/welcome.html.tmpl +2079 -0
  236. package/tools/.infer_baseline.json +30 -0
  237. package/tools/TOOLS_RUNBOOK.md +1553 -0
  238. package/tools/TOOL_CATALOG.json +1769 -0
  239. package/tools/__init__.py +43 -0
  240. package/tools/_openapi_parser.py +158 -0
  241. package/tools/agent_canary.py +525 -0
  242. package/tools/agentless_patcher.py +417 -0
  243. package/tools/analyze_project.py +258 -0
  244. package/tools/api_drift_sentinel.py +208 -0
  245. package/tools/api_fuzzer.py +762 -0
  246. package/tools/author_integrity.py +184 -0
  247. package/tools/auto_heal.py +271 -0
  248. package/tools/auto_heal_reflexion.py +532 -0
  249. package/tools/autogen_qa_broker.py +844 -0
  250. package/tools/benchmark_swarm_context.py +394 -0
  251. package/tools/browser_helpers.js +746 -0
  252. package/tools/browsergym_eval.py +236 -0
  253. package/tools/bug_reporter.py +252 -0
  254. package/tools/bump_version.py +423 -0
  255. package/tools/cache_resolver.py +1214 -0
  256. package/tools/check_env.py +426 -0
  257. package/tools/check_version_sync.py +256 -0
  258. package/tools/ci_healing_hook.py +126 -0
  259. package/tools/cli.py +1598 -0
  260. package/tools/config_decoupler.py +253 -0
  261. package/tools/contract_validator.py +141 -0
  262. package/tools/core/__init__.py +70 -0
  263. package/tools/core/adapters.py +431 -0
  264. package/tools/core/circuit_breaker.py +263 -0
  265. package/tools/core/cli_runtime.py +66 -0
  266. package/tools/core/file_lock.py +159 -0
  267. package/tools/core/i18n.py +394 -0
  268. package/tools/core/memory_bus.py +905 -0
  269. package/tools/core/process_supervisor.py +332 -0
  270. package/tools/core/telemetry.py +329 -0
  271. package/tools/count_tokens.py +1267 -0
  272. package/tools/create_mock_docx.py +104 -0
  273. package/tools/dashboard_generator.py +2419 -0
  274. package/tools/db_sandbox.py +675 -0
  275. package/tools/debt_harvester.py +237 -0
  276. package/tools/distill_log.py +161 -0
  277. package/tools/dspy_qa_signatures.py +356 -0
  278. package/tools/e2e_flow_prober.py +132 -0
  279. package/tools/export_postman_collection.py +945 -0
  280. package/tools/flaky_detector.py +389 -0
  281. package/tools/fsm_enforcer.py +450 -0
  282. package/tools/fsm_middleware.py +401 -0
  283. package/tools/fuzz_payloads.py +197 -0
  284. package/tools/generate_allure_report.py +88 -0
  285. package/tools/generate_api_matrix.py +193 -0
  286. package/tools/generate_jmx_plan.py +1322 -0
  287. package/tools/generate_sbom.py +252 -0
  288. package/tools/generate_tc_from_story.py +203 -0
  289. package/tools/generate_test_reports.py +133 -0
  290. package/tools/generate_tool_catalog.py +482 -0
  291. package/tools/gherkin_suite.py +758 -0
  292. package/tools/git_pre_commit_guard.py +298 -0
  293. package/tools/goldfish_review.py +366 -0
  294. package/tools/heal/__init__.py +28 -0
  295. package/tools/heal_pr_automator.py +220 -0
  296. package/tools/heuristic_locator_matcher.py +351 -0
  297. package/tools/hub_publisher.py +623 -0
  298. package/tools/humanize_linter.py +438 -0
  299. package/tools/infer_linter.py +977 -0
  300. package/tools/init_config.py +307 -0
  301. package/tools/interview_story.py +116 -0
  302. package/tools/jira_sync.py +566 -0
  303. package/tools/jtl_analyzer.py +474 -0
  304. package/tools/lint/__init__.py +30 -0
  305. package/tools/lint_ai_docs.py +374 -0
  306. package/tools/lookup_tc.py +165 -0
  307. package/tools/lookup_xpath.py +167 -0
  308. package/tools/matrix_sharder.py +148 -0
  309. package/tools/mcp/__init__.py +18 -0
  310. package/tools/mcp_docx_server.ps1 +13 -0
  311. package/tools/mcp_docx_server.py +851 -0
  312. package/tools/mcp_excel_server.ps1 +13 -0
  313. package/tools/mcp_excel_server.py +441 -0
  314. package/tools/mcp_server.py +724 -0
  315. package/tools/md_to_xlsx.py +327 -0
  316. package/tools/micro_mutator.py +887 -0
  317. package/tools/mipro_assertion_tuner.py +210 -0
  318. package/tools/network_mock_shield.py +311 -0
  319. package/tools/newman_analyzer.py +384 -0
  320. package/tools/omni_emitter.py +579 -0
  321. package/tools/omni_parser.py +450 -0
  322. package/tools/omni_transpiler.py +168 -0
  323. package/tools/openxml_guard.py +438 -0
  324. package/tools/output_formatter.py +182 -0
  325. package/tools/package_release.py +325 -0
  326. package/tools/perf/__init__.py +24 -0
  327. package/tools/pinpoint_failure.py +190 -0
  328. package/tools/project_state.py +171 -0
  329. package/tools/qa_event_stream.py +358 -0
  330. package/tools/repo_map.py +346 -0
  331. package/tools/report/__init__.py +32 -0
  332. package/tools/reversible_sandbox.py +344 -0
  333. package/tools/safety_firewall.py +437 -0
  334. package/tools/security_coverage_ledger.py +538 -0
  335. package/tools/security_scanner.py +229 -0
  336. package/tools/senior_code_review.py +393 -0
  337. package/tools/setup_project.py +770 -0
  338. package/tools/skill_collision_guard.py +304 -0
  339. package/tools/skill_profiles.py +217 -0
  340. package/tools/skill_vault.py +481 -0
  341. package/tools/skillspector.py +1278 -0
  342. package/tools/smart_visual_masker.py +132 -0
  343. package/tools/spec_drift_sentinel.py +438 -0
  344. package/tools/swe_qa_aci.py +369 -0
  345. package/tools/synapse_observer.py +485 -0
  346. package/tools/synapse_qa_bench.py +190 -0
  347. package/tools/sync_agent_rules.py +274 -0
  348. package/tools/telemetry_correlator.py +397 -0
  349. package/tools/template_excel_exporter.py +514 -0
  350. package/tools/templates/jmeter-distributed/README.md +54 -0
  351. package/tools/templates/jmeter-distributed/docker-compose.yml +83 -0
  352. package/tools/test_impact.py +419 -0
  353. package/tools/test_memory_os.py +462 -0
  354. package/tools/textgrad_mutation_optimizer.py +237 -0
  355. package/tools/tms_sync_bridge.py +834 -0
  356. package/tools/tot_exploratory_tester.py +239 -0
  357. package/tools/trace_data_flow.py +240 -0
  358. package/tools/traffic_session_converter.py +256 -0
  359. package/tools/trend_analytics.py +314 -0
  360. package/tools/ui_server.py +401 -0
  361. package/tools/universal_ir.py +488 -0
  362. package/tools/update_sentinel.py +383 -0
  363. package/tools/validate_groovy_scripts.py +173 -0
  364. package/tools/verify_all.py +1259 -0
  365. package/tools/visual_diff_canvas.py +143 -0
  366. package/tools/visual_som_helper.py +550 -0
  367. package/tools/watch_mode.py +308 -0
  368. package/tools/wcag_audit.py +239 -0
  369. package/tools/welcome_page.py +290 -0
@@ -0,0 +1,48 @@
1
+ # Closed-Loop Failure Autopsy & Autonomous Healing Walkthrough
2
+
3
+ > **Scenario:** End-to-end checkout test fails in CI runner due to locator drift on the order confirmation button.
4
+
5
+ ---
6
+
7
+ ## 1. Fast Failure Pinpoint (0.1s from JUnit XML)
8
+
9
+ ```bash
10
+ python tools/pinpoint_failure.py Reports/junit.xml
11
+ ```
12
+
13
+ **Diagnostic Output:**
14
+ ```text
15
+ ❌ Failing Test Case : Tests/E2E_Checkout_Order.ts:42
16
+ 📍 Failing Line : await page.locator("button#submit-v2").click()
17
+ ⚠️ Exception Caught : TimeoutError: locator.click: Timeout 5000ms exceeded waiting for locator("button#submit-v2")
18
+ ```
19
+
20
+ ---
21
+
22
+ ## 2. Reflexion Verbal Diagnosis & AOM Heuristic Match
23
+
24
+ The Failure Analyst initiates reflection:
25
+ - **Reflexion:** *"Static ID selector `#submit-v2` did not appear within timeout window. Adjustment: Fallback to Accessibility role/name or dynamic data-testid."*
26
+ - **Heuristic Matcher:** Layer 1 semantic matching finds live element matching role `button` with label `"Confirm Order"`.
27
+
28
+ ---
29
+
30
+ ## 3. Autonomous Patch Synthesis & Verification
31
+
32
+ `pom-crafter` applies the surgical patch to `CheckoutPage.ts`:
33
+ ```diff
34
+ - await page.locator("button#submit-v2").click();
35
+ + await page.getByRole("button", name: "Confirm Order").click();
36
+ + await page.waitForLoadState("networkidle");
37
+ ```
38
+
39
+ Targeted re-test:
40
+ ```bash
41
+ python tools/auto_heal.py --execute --target-test "Tests/E2E_Checkout_Order.ts"
42
+ ```
43
+
44
+ **Verdict:**
45
+ ```text
46
+ ✅ [TRIAL 1 SUCCESS] Patch verified passing in 1.4s!
47
+ 💾 Recorded 2-line practical lesson to SQLite episodic memory.
48
+ ```
@@ -0,0 +1,8 @@
1
+ <?xml version="1.0" encoding="UTF-8"?>
2
+ <testsuites>
3
+ <testsuite name="AuthenticationSuite" tests="1" failures="1">
4
+ <testcase name="testLoginFailure" classname="AuthTest">
5
+ <failure message="Timeout 30000ms exceeded waiting for locator('#submit-btn')" type="ElementNotFoundError"/>
6
+ </testcase>
7
+ </testsuite>
8
+ </testsuites>
@@ -0,0 +1,64 @@
1
+ # Enterprise Automation Failure Diagnostic & Healing Matrix
2
+
3
+ Reference documentation for the `failure-autopsy-and-healing` skill.
4
+
5
+ ---
6
+
7
+ ## 1. Diagnostic & Healing Matrix
8
+
9
+ | Visual / Exception Symptom | Root Cause | Groovy POM Healing Patch |
10
+ | :--- | :--- | :--- |
11
+ | **Save button `[ ✓ ]` disabled / dimmed** | Angular Reactive Form missed change detection or missing required field (`*`). | `WebElement el = findElement(By.cssSelector("input[formcontrolname='code']"))`<br>`jsSetInput(el, uniqueCode)` |
12
+ | **Alert: *"Phạm vi giữa các khu/cụm không hợp lệ"*** | OpenLayers polygon overlaps previously saved boundary data. | `Actions actions = new Actions(driver)`<br>`actions.moveToElement(viewport, -180, -80).click().perform()` |
13
+ | **`ElementClickInterceptedException`** | Modal overlay `.p-component-overlay` fading out but still blocking clicks. | `((JavascriptExecutor) driver).executeScript("arguments[0].click();", element)` |
14
+ | **Red Toast: *"Mã đã tồn tại"*** | Non-unique test data colliding with existing records. | `String code = BasePage.uniqueCode("CCN_")` |
15
+ | **`StaleElementReferenceException`** | Angular DOM re-rendered after API response, detaching element. | `for (int i = 0; i < 3; i++) { try { findElement(by).click(); break; } catch (StaleElementReferenceException e) { sleep(500) } }` |
16
+ | **`TimeoutException` (Waiting for table/button)** | Loading overlay `.p-blockui` or spinner `.p-progress-spinner` blocking UI. | `waitForElementNotVisible(By.cssSelector(".p-blockui, .p-progress-spinner"), 10)` |
17
+ | **PrimeNG Dropdown value not applied** | Clicked `p-dropdown` but overlay panel `.p-dropdown-panel` has not rendered. | `waitForElementVisible(By.cssSelector(".p-dropdown-panel"), 5)`<br>`findElement(By.xpath("//li[contains(., '" + text + "')]")).click()` |
18
+ | **Blank Screenshot** | Browser crashed before capturing UI (OOM or lost WebDriver session). | Inspect `console0.log`, kill zombie drivers (`pkill -f chromedriver`), restart session. |
19
+
20
+ ---
21
+
22
+ ## 2. Four Flaky Test Root Causes & Cures
23
+
24
+ | Flaky Root Cause | Symptom | Curing Strategy (Root Fix) |
25
+ |---|---|---|
26
+ | **1. Unstable Locator** | Random failures due to CSS refactoring or dynamic hashes | Enforce Master Priority Map: switch to `[formcontrolname]`, `[data-testid]`; forbid positional XPaths. |
27
+ | **2. Timing & Race Condition** | Fails under network latency; clicks before Angular finishes render | Replace all hard sleeps with Smart Wait: `waitForElementVisible`, wait for `.p-blockui` / `.p-progress-spinner` to hide. |
28
+ | **3. Test Data Conflict** | Fails on parallel or repeated runs due to duplicate codes | Enforce `BasePage.uniqueCode(prefix)` with epoch timestamps. |
29
+ | **4. Environment Dependency** | Fails due to expired session or stale browser cache | Use `LoginPage.ensureLoggedIn()` before every script. |
30
+
31
+ ---
32
+
33
+ ## 3. Fast-Fix Triad (Zero-Token Diagnostic & Targeted Healing Workflow)
34
+
35
+ To maximize fix speed and minimize token consumption without degrading test fidelity, follow this 3-step loop:
36
+
37
+ ### Step 1: Zero-Token Pinpoint (CLI Log Extraction)
38
+ Never read entire JUnit XML reports (>50KB). Extract failed test cases, exception reasons, and file line numbers in <50 tokens:
39
+ ```bash
40
+ python3 tools/pinpoint_failure.py Reports/<timestamp>/<suite>/JUnit_Report.xml
41
+ ```
42
+
43
+ ### Step 2: Instant Form & DOM Audit (Playwright MCP Recipe 15)
44
+ When a test fails because a button is disabled (e.g. Save `[ ✓ ]`), never guess which field is invalid. Execute Recipe 15 via `browser_run_code_unsafe` on the active live page:
45
+ ```javascript
46
+ () => {
47
+ const invalids = Array.from(document.querySelectorAll('form .ng-invalid, input.ng-invalid, p-dropdown.ng-invalid'));
48
+ return invalids.map(el => ({
49
+ tag: el.tagName,
50
+ name: el.getAttribute('name') || el.getAttribute('formcontrolname') || el.id,
51
+ classes: el.className
52
+ }));
53
+ }
54
+ ```
55
+ This instantly highlights missing required inputs, pattern mismatches, or invalid dropdowns.
56
+
57
+ ### Step 3: Targeted Single-TC Re-test (Zero Waste)
58
+ Never rerun an entire Test Suite to verify a 1-line locator or timing fix. Rerun only the targeted test case:
59
+ ```bash
60
+ # Via KRE CLI (License mode):
61
+ katalonc -noSplash -runMode=console -projectPath="." -retry=0 -testCasePath="Test Cases/TC-xxx" -browserType="Chrome (headless)"
62
+
63
+ # Or verify live flow in 2 seconds via Playwright MCP script.
64
+ ```
@@ -0,0 +1,56 @@
1
+ # QA Investigation & Root-Cause Analysis Protocol
2
+
3
+ *(Inherited from `fugazi/test-automation-skills-agents`)*
4
+
5
+ A file-backed, evidence-first investigation journal protocol for diagnosing failing and flaky automation test cases in enterprise automation projects.
6
+
7
+ ---
8
+
9
+ ## 1. Core Principles
10
+
11
+ 1. **Context Window is Volatile RAM; Filesystem is Persistent Disk:**
12
+ - Long triage sessions cause context drift. All goals, hypotheses, and evidence must be logged in markdown.
13
+ 2. **Discovered Classification, Not Presumed:**
14
+ - Never assume upfront whether a failure is a Flaky test, an AUT Defect, or a Test Infrastructure issue. The classification is a finding discovered through evidence.
15
+ 3. **No Guesswork / No Blind Fixes:**
16
+ - Every fix must address a proven root cause, not a symptom.
17
+
18
+ ---
19
+
20
+ ## 2. Four-Phase Investigation Cycle
21
+
22
+ ```mermaid
23
+ flowchart TD
24
+ P1["<b>Phase 1: Reproduction & Triage</b><br/>• Run single-TC isolated re-test<br/>• Classify: Intermittent (Flaky) vs Deterministic (Bug)"]
25
+ P2["<b>Phase 2: Evidence Collection</b><br/>• Extract CLI logs & stack trace<br/>• Inspect Screenshots/FAILED_*.png<br/>• Capture network & timing data"]
26
+ P3["<b>Phase 3: Hypothesis & Root Cause</b><br/>• Form testable hypothesis<br/>• Isolate: Timing vs Stale DOM vs AUT defect"]
27
+ P4["<b>Phase 4: Targeted Fix & Validation</b><br/>• Apply Clean POM Page Class fix<br/>• Validate 3x consecutive runs<br/>• Run verify_all.py DoD check"]
28
+
29
+ P1 --> P2 --> P3 --> P4
30
+ ```
31
+
32
+ ### Phase 1: Reproduction & Triage
33
+ - Run the failing test in isolation using the targeted single-TC runner.
34
+ - Determine failure pattern:
35
+ - **Deterministic Bug:** Fails 100% of the time with identical stack trace.
36
+ - **Intermittent Flaky:** Passes sometimes, fails under load/timing variations.
37
+ - **Environment Blocker:** Fails on 401/403 session expiration or network timeout.
38
+
39
+ ### Phase 2: Evidence Collection
40
+ - Collect ground-truth artifacts:
41
+ 1. CLI log excerpt around the failure point.
42
+ 2. Failure screenshot in `Screenshots/FAILED_<TestCaseId>_*.png`.
43
+ 3. Form audit state (e.g. disabled Save button, red validation borders).
44
+ 4. Redact any sensitive credentials or personal tokens.
45
+
46
+ ### Phase 3: Hypothesis Formulation
47
+ - Check the 4 most common failure archetypes:
48
+ - **Archetype A (PrimeNG Toast Collision):** Previous toast covers the button. Cure: `dismissToast()`.
49
+ - **Archetype B (Stale Element / Angular Re-render):** DOM re-created mid-click. Cure: Re-fetch locator with `BasePage.x()`.
50
+ - **Archetype C (Dialog Backdrop Overlay):** `cdk-overlay-backdrop` blocks click. Cure: Wait for backdrop fade-out.
51
+ - **Archetype D (Real AUT Defect):** Backend HTTP 500 or constraint violation. Cure: Log to `KNOWN_ISSUES.md`.
52
+
53
+ ### Phase 4: Targeted Fix & Validation
54
+ - Implement fix exclusively inside Page Classes in `Keywords/<namespace>/pages/` or `pages/` (never in `Scripts/`).
55
+ - Validate stability across multiple runs.
56
+ - Run `python3 tools/verify_all.py` to confirm zero regressions.
@@ -0,0 +1,83 @@
1
+ #!/usr/bin/env bash
2
+ # Triage Failures Script for Enterprise Automation
3
+ # Cross-platform: Compatible with macOS, Linux CI, and Windows Git Bash
4
+
5
+ WORKSPACE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)"
6
+ SCREENSHOT_DIR="$WORKSPACE_DIR/Screenshots"
7
+ REPORTS_DIR="$WORKSPACE_DIR/Reports"
8
+
9
+ echo "=========================================================="
10
+ echo " 🔬 ENTERPRISE FAILURE AUTOPSY & TRIAGE SCANNER (CROSS-PLATFORM)"
11
+ echo " Workspace: $WORKSPACE_DIR"
12
+ echo "=========================================================="
13
+
14
+ # 1. Quét báo cáo JUnit XML gần nhất
15
+ LATEST_JUNIT=$(find "$REPORTS_DIR" -type f -name "JUnit_Report.xml" 2>/dev/null | sort -r | head -n 1)
16
+
17
+ if [ -n "$LATEST_JUNIT" ]; then
18
+ echo "📊 Báo cáo JUnit gần nhất: $(basename "$(dirname "$LATEST_JUNIT")")/JUnit_Report.xml"
19
+ FAILED_JUNIT_TESTS=$(grep -B 1 '<failure' "$LATEST_JUNIT" 2>/dev/null | grep '<testcase' | sed -n 's/.*name="\([^"]*\)".*/\1/p')
20
+ if [ -n "$FAILED_JUNIT_TESTS" ]; then
21
+ echo "🚨 Các Test Cases báo FAILED trong JUnit XML:"
22
+ echo "$FAILED_JUNIT_TESTS" | while read -r tc; do
23
+ echo " ❌ $tc"
24
+ done
25
+ echo "----------------------------------------------------------"
26
+ else
27
+ echo "✅ Không phát hiện ca thất bại nào trong JUnit XML gần nhất."
28
+ echo "----------------------------------------------------------"
29
+ fi
30
+ fi
31
+
32
+ # 2. Quét ảnh chụp FAILED_*.png
33
+ if [ ! -d "$SCREENSHOT_DIR" ]; then
34
+ echo "ℹ️ Không tìm thấy thư mục Screenshots. Dự án đang sạch lỗi!"
35
+ exit 0
36
+ fi
37
+
38
+ FAILED_COUNT=$(find "$SCREENSHOT_DIR" -maxdepth 1 -name "FAILED_*.png" 2>/dev/null | wc -l | tr -d ' ')
39
+
40
+ if [ "$FAILED_COUNT" -eq "0" ]; then
41
+ echo "✅ Tuyệt vời! Không có ảnh lỗi nào trong $SCREENSHOT_DIR."
42
+ exit 0
43
+ fi
44
+
45
+ echo "⚠️ Phát hiện $FAILED_COUNT ảnh lỗi kịch bản gần nhất:"
46
+ echo "----------------------------------------------------------"
47
+
48
+ # Hàm lấy thông tin file đa nền tảng (hỗ trợ Linux GNU stat, macOS BSD stat và fallback ls)
49
+ list_recent_screenshots() {
50
+ if stat --version 2>&1 | grep -q "GNU"; then
51
+ # Linux (GNU stat)
52
+ find "$SCREENSHOT_DIR" -maxdepth 1 -name "FAILED_*.png" -type f -exec stat -c "%Y %y %n" {} + 2>/dev/null | \
53
+ sort -rn | head -n 10 | awk '{print $1, $2, $3, $NF}'
54
+ elif stat -f "%m" . >/dev/null 2>&1; then
55
+ # macOS / BSD stat
56
+ find "$SCREENSHOT_DIR" -maxdepth 1 -name "FAILED_*.png" -type f -exec stat -f "%m %Sm %N" -t "%Y-%m-%d %H:%M:%S" {} + 2>/dev/null | \
57
+ sort -rn | head -n 10
58
+ else
59
+ # Fallback (POSIX ls)
60
+ ls -lt "$SCREENSHOT_DIR"/FAILED_*.png 2>/dev/null | head -n 10 | awk '{print "0", $6, $7, $NF}'
61
+ fi
62
+ }
63
+
64
+ list_recent_screenshots | while read -r epoch date time filepath; do
65
+ filename=$(basename "$filepath")
66
+
67
+ # Dynamic Page Class mapping via repo_map lookup or fallback to BasePage
68
+ suggested_page="BasePage.groovy"
69
+ if [ -f "tools/repo_map.txt" ] && [[ "$filename" =~ TC-([0-9]+) ]]; then
70
+ tc_pattern="TC-${BASH_REMATCH[1]}"
71
+ found_page=$(grep -m 1 -i "$tc_pattern" tools/repo_map.txt | awk '{print $1}' 2>/dev/null)
72
+ if [ -n "$found_page" ]; then
73
+ suggested_page="$found_page"
74
+ fi
75
+ fi
76
+
77
+ echo "📸 [$date $time] $filename"
78
+ echo " -> Đường dẫn ảnh : $filepath"
79
+ echo " -> Page Class cần vá : Keywords/pages/$suggested_page"
80
+ done
81
+
82
+ echo "----------------------------------------------------------"
83
+ echo "💡 Gợi ý AI: Dùng tool read_file mở ảnh hoặc dùng Playwright Live giữ nguyên browser để xác thực và vá Page Class."
@@ -0,0 +1,90 @@
1
+ ---
2
+ name: file-and-report-validator
3
+ description: "Validate downloaded and uploaded files (Excel, CSV, PDF) schemas and tabular integrity. Use when validating exported files or reports against a schema."
4
+ ---
5
+
6
+ # File & Report Validator Skill
7
+
8
+ > **Target:** Eliminate superficial testing of file exports and imports by inspecting and asserting exact file contents, schemas, formulas, and values across Excel, CSV, and PDF formats.
9
+
10
+ ---
11
+
12
+ ## 1. Core Principles
13
+
14
+ 1. **Verify Binary Contents, Not Just Download Events:**
15
+ - Catching the browser `download` event is insufficient. The downloaded file must be opened, parsed, and its tabular records cross-referenced against expected database or UI table states.
16
+ 2. **Schema & Header Conformance:**
17
+ - Assert exact column headers, cell data formats (dates, currencies, numeric precision), and sheet names.
18
+ 3. **Idempotent File Lifecycle:**
19
+ - Store temporary test downloads in a sandboxed scratch folder and purge them during teardown.
20
+
21
+ ---
22
+
23
+ ## 2. Standard Playwright Download Handling
24
+
25
+ ```typescript
26
+ import { test, expect } from '@playwright/test';
27
+ import path from 'path';
28
+
29
+ test('export and verify financial report', async ({ page }) => {
30
+ const downloadPromise = page.waitForEvent('download');
31
+ await page.click('button#export-excel');
32
+ const download = await downloadPromise;
33
+
34
+ const savePath = path.join('temp_downloads', download.suggestedFilename());
35
+ await download.saveAs(savePath);
36
+
37
+ expect(download.suggestedFilename()).toMatch(/^Report_\d{8}\.xlsx$/);
38
+ });
39
+ ```
40
+
41
+ ---
42
+
43
+ ## 3. Deep Content Inspection Recipes (Python)
44
+
45
+ ### Pattern A: Excel (.xlsx / .xls) Schema & Data Assertion
46
+ ```python
47
+ import pandas as pd
48
+
49
+ def validate_excel_report(file_path: str, expected_min_rows: int = 1):
50
+ df = pd.read_excel(file_path)
51
+
52
+ # 1. Assert header schema
53
+ expected_headers = ["ID", "Customer Name", "Total Amount", "Status"]
54
+ assert list(df.columns) == expected_headers, f"Header mismatch: {df.columns}"
55
+
56
+ # 2. Assert row counts and non-empty values
57
+ assert len(df) >= expected_min_rows, f"Expected at least {expected_min_rows} rows, got {len(df)}"
58
+ assert not df["ID"].isnull().any(), "Found NULL values in ID column"
59
+ print(f"[PASS] Successfully verified {len(df)} rows in {file_path}")
60
+ ```
61
+
62
+ ### Pattern B: CSV Chunk Streaming (For Big Data Exports > 100k Rows)
63
+ ```python
64
+ import csv
65
+
66
+ def validate_large_csv_stream(file_path: str, expected_header: list):
67
+ row_count = 0
68
+ with open(file_path, mode='r', encoding='utf-8') as f:
69
+ reader = csv.reader(f)
70
+ header = next(reader)
71
+ assert header == expected_header, f"CSV Header mismatch: {header}"
72
+ for row in reader:
73
+ row_count += 1
74
+ assert len(row) == len(header), f"Malformed row at line {row_count}"
75
+ print(f"[PASS] Verified {row_count} streamed CSV rows without memory overflow.")
76
+ ```
77
+
78
+ ### Pattern C: PDF Document Text & Metadata Extraction
79
+ ```python
80
+ import pypdf
81
+
82
+ def assert_pdf_contains_text(pdf_path: str, expected_keywords: list):
83
+ reader = pypdf.PdfReader(pdf_path)
84
+ assert len(reader.pages) > 0, "PDF is empty (0 pages)!"
85
+
86
+ full_text = "\n".join([page.extract_text() or "" for page in reader.pages])
87
+ for kw in expected_keywords:
88
+ assert kw in full_text, f"PDF assertion failed: Keyword '{kw}' missing in {pdf_path}"
89
+ print(f"[PASS] Verified keywords {expected_keywords} across {len(reader.pages)} PDF pages.")
90
+ ```
@@ -0,0 +1,14 @@
1
+ {
2
+ "expectedHeaders": [
3
+ "Order ID",
4
+ "Customer Name",
5
+ "Amount",
6
+ "Date",
7
+ "Status"
8
+ ],
9
+ "rowCountRange": [
10
+ 1,
11
+ 5000
12
+ ],
13
+ "dateFormat": "YYYY-MM-DD"
14
+ }
@@ -0,0 +1,114 @@
1
+ ---
2
+ name: flaky-test-sentinel-and-quarantine
3
+ description: "Detect non-deterministic flaky tests, calculate flakiness scores, and manage quarantine. Use when detecting flaky tests and managing quarantine lifecycles."
4
+ ---
5
+
6
+ # Flaky Test Sentinel & Quarantine Protocol
7
+
8
+ > **Purpose:** Master flaky test governance, automated quarantine isolation, and non-deterministic test remediation. Eliminates false-positive CI/CD pipeline blockers by identifying unstable tests through repeated execution ($N=5$), moving them into isolated quarantine suites, and tracking remediation until graduation.
9
+
10
+ ---
11
+
12
+ ## 1. The Flaky Test Dilemma & The 5-Stage Quarantine Lifecycle
13
+
14
+ In large enterprise test suites (hundreds to thousands of test cases), non-deterministic tests ("flaky tests" that intermittently pass and fail without code changes) erode trust in CI/CD, slow down deployments, and cause developers to ignore test results.
15
+
16
+ ```mermaid
17
+ flowchart TD
18
+ A["<b>1. Detection Engine</b><br/>Run test N=5 iterations<br/>Identify Pass/Fail flips"] --> B["<b>2. Automated Quarantine</b><br/>Tag test with @quarantine<br/>Isolate from main suite"]
19
+ B --> C["<b>3. Unblocked PR Pipeline</b><br/>PR merge runs stable tests only<br/>CI/CD remains green"]
20
+ B --> D["<b>4. Quarantine Lab</b><br/>Runs in parallel/nightly<br/>Logs failure autopsy"]
21
+ D --> E["<b>5. Graduation DoD Gate</b><br/>Must achieve 10/10 consecutive passes<br/>Graduates back to main"]
22
+ ```
23
+
24
+ ---
25
+
26
+ ## 2. Flakiness Scoring Formula & Classification
27
+
28
+ $$P = \frac{\text{Passed Executions}}{N} \times 100\% \quad \text{where } N \ge 5$$
29
+
30
+ $$\text{Flip Rate} = \frac{\text{Status Transitions (Pass} \leftrightarrow \text{Fail)}}{N - 1} \times 100\%$$
31
+
32
+ | Health Tier | Pass Rate ($P$) | Flip Count ($N=5$) | Action Required | Pipeline Impact |
33
+ | :--- | :--- | :--- | :--- | :--- |
34
+ | **STABLE (Green)** | $100\%$ | 0 flips | Kept in standard regression suite | Blocks PR on real bug |
35
+ | **MILD FLAKE** | $80\% \le P < 100\%$ | 1 - 2 flips | Log warning; monitor over 24 hours | Kept in suite with watchpoint |
36
+ | **SEVERE FLAKE** | $P < 80\%$ | $\ge 2$ flips | **Immediate Quarantine Tagging (`@quarantine`)** | Excluded from PR merge blocker |
37
+ | **DEAD TEST / BROKEN**| $0\%$ | 0 flips | File critical AUT/Test bug defect | Regular test failure |
38
+
39
+ ---
40
+
41
+ ## 3. Quarantine Tagging & Execution Strategies
42
+
43
+ ### Playwright / TypeScript Quarantine Tagging
44
+ ```typescript
45
+ import { test, expect } from '@playwright/test';
46
+
47
+ // Before Quarantine: Unstable test intermittently blocking main PRs
48
+ // test('TC-104: Submit monthly reconciliation report', async ({ page }) => { ... });
49
+
50
+ // After Quarantine: Tagged with @quarantine metadata
51
+ test('TC-104: Submit monthly reconciliation report @quarantine', async ({ page }) => {
52
+ // Test code remains intact for debugging
53
+ await page.goto('/reports/reconciliation');
54
+ // ...
55
+ });
56
+ ```
57
+
58
+ ### CI/CD Filter Configuration
59
+ ```bash
60
+ # 1. Main PR Blocker Suite: Runs ONLY 100% stable tests (Zero Flaky Disruptions)
61
+ npx playwright test --grep-invert "@quarantine"
62
+
63
+ # 2. Nightly Quarantine Lab: Runs quarantined tests to collect diagnostics & heal
64
+ npx playwright test --grep "@quarantine" --reporter=json
65
+ ```
66
+
67
+ ---
68
+
69
+ ## 4. Automated Multi-Run Flakiness Detector Tool
70
+
71
+ This repository provides an automated flakiness analyzer in `tools/flaky_detector.py`.
72
+
73
+ ### Running 5-Iteration Flakiness Benchmark
74
+ ```bash
75
+ # Run a specific test script 5 consecutive times to benchmark stability
76
+ synapse flaky --repeat 5 --cmd "npm test -- tests/e2e/TC-104.spec.ts"
77
+
78
+ # Or parse existing test execution history
79
+ synapse flaky --reports "reports/**/junit*.xml"
80
+ ```
81
+
82
+ Output Report (`reports/quarantine_board.json`):
83
+ ```json
84
+ {
85
+ "test_id": "TC-104",
86
+ "iterations": 5,
87
+ "passes": 3,
88
+ "failures": 2,
89
+ "pass_rate": 60.0,
90
+ "flakiness_status": "SEVERE_FLAKE",
91
+ "quarantine_recommended": true,
92
+ "probable_cause": "Angular PrimeNG animation fade-out / missing smart wait"
93
+ }
94
+ ```
95
+
96
+ ---
97
+
98
+ ## 5. Root-Cause Remediation Guide (Heal & Graduate)
99
+
100
+ | Flakiness Root Cause | Anti-Pattern | Mandatory Remediation |
101
+ | :--- | :--- | :--- |
102
+ | **Race Condition** | Static `Thread.sleep(3000)` or blind delays | Replace with Smart Wait: `waitForLoadState('networkidle')` or Angular `zone.isStable()`. |
103
+ | **Database Collision** | Reusing static IDs (`"ORDER_001"`) | Use dynamic identifiers: `BasePage.uniqueCode("AUTO_")` paired with Idempotent Teardown. |
104
+ | **Animation Interference**| Clicking during backdrop fade | Use `BasePage.clickJS()` or await overlay invisibility (`.p-component-overlay`). |
105
+ | **DOM Re-render Drift** | Direct click after async API fetch | Encapsulate 3-attempt retry loop with AOM Role fallback in Page Class. |
106
+
107
+ ---
108
+
109
+ ## 6. Graduation Definition of Done (DoD) Gate
110
+
111
+ Before removing the `@quarantine` tag and restoring a test to the main PR suite:
112
+ 1. **Verification Run:** The healed test must execute **10 consecutive iterations** in headless mode with **100% Pass Rate** (`10/10 PASS`).
113
+ 2. **Code Standards:** 100% UTF-8 No BOM, Clean POM flat architecture, zero hardcoded XPaths.
114
+ 3. **Artifact Update:** Update `reports/quarantine_board.json` to mark status as `GRADUATED`.
@@ -0,0 +1,11 @@
1
+ {
2
+ "quarantinedTests": [
3
+ {
4
+ "testId": "TC-CHECKOUT-881",
5
+ "flakinessScore": 0.35,
6
+ "reason": "Intermittent 3rd-party payment gateway timeout",
7
+ "quarantineDate": "2026-10-30",
8
+ "ticket": "QA-1204"
9
+ }
10
+ ]
11
+ }
@@ -0,0 +1,103 @@
1
+ ---
2
+ name: gherkin-to-suite
3
+ description: "Compile Gherkin .feature files into an expanded test matrix, hub Universal IR and a proven-vs-owed coverage-gap report with zero LLM tokens. Use when the user supplies BDD .feature specs or wants a BDD suite audited before codification."
4
+ ---
5
+
6
+ # Gherkin to Suite Compiler — Natural-Language Test Ingestion (0-Token)
7
+
8
+ This skill turns business-readable Gherkin `.feature` files into deterministic
9
+ test artifacts: an expanded test-case matrix, hub-native Universal IR for the
10
+ transpile pipeline, and a coverage-gap audit — all without a single LLM token.
11
+
12
+ > **Provenance:** Concept inspired by the Gherkin-driven E2E ingestion workflow
13
+ > popularized by TestZeus Hercules (AGPL-3.0); independently re-implemented
14
+ > from scratch for this MIT repository (no upstream code). The "proven vs
15
+ > owed" coverage traceability idea is inspired by Kane CLI (Apache-2.0).
16
+
17
+ ---
18
+
19
+ ## 1. When to Activate
20
+
21
+ - User supplies `.feature` / BDD specs and wants an executable test suite plan.
22
+ - Need to audit a BDD suite for missing assertions, negative paths, or thin
23
+ example tables before codifying into Clean POM scripts.
24
+ - Need structured test data (JSON IR) to feed `omni_transpiler` / `universal_ir`
25
+ pipelines instead of hand-writing scripts.
26
+
27
+ ## 2. Pipeline
28
+
29
+ ```mermaid
30
+ flowchart TD
31
+ Feature["<b>.feature file(s)</b><br/>Feature / Rule / Background / Scenario Outline / Examples / Tags / Doc Strings / Data Tables"] --> Parser["<b>1. Gherkin Parser (stdlib state machine)</b><br/>• Tags attach to Feature, Rule, Scenario, Examples<br/>• And/But inherit the concrete Given/When/Then role<br/>• Doc strings dedent per Gherkin spec, \\| escapes honored"]
32
+ Parser --> Expand["<b>2. Outline Expansion</b><br/>• One concrete case per Examples row<br/>• &lt;param&gt; substitution across steps, tables, doc strings<br/>• Unresolved &lt;param&gt; = hard error (header typos caught)"]
33
+ Expand --> IR["<b>3. Universal IR Mapping (deterministic, 0-token)</b><br/>• navigate/open → GOTO, fill/enter → FILL, click/tap → CLICK<br/>• Then + quoted text → ASSERT contains_text<br/>• Unmapped phrasing → CUSTOM_CALL (text preserved)"]
34
+ Expand --> Gaps["<b>4. Proven-vs-Owed Gap Audit</b><br/>• G1 case without Then assertion<br/>• G2 no negative path in feature<br/>• G3 thin Examples table"]
35
+ IR --> Out["<b>Test Matrix + Markdown Plan + JSON report</b>"]
36
+ Gaps --> Out
37
+ ```
38
+
39
+ ## 3. Commands
40
+
41
+ ```bash
42
+ # Compile a single feature (human summary + Markdown plan to stdout)
43
+ python tools/gherkin_suite.py tests/bdd/checkout.feature
44
+
45
+ # Compile a whole directory of .feature files
46
+ python tools/gherkin_suite.py tests/bdd/
47
+
48
+ # Machine-readable report to stdout / hub Universal IR for the omni pipeline
49
+ python tools/gherkin_suite.py tests/bdd/ --json
50
+ python tools/gherkin_suite.py tests/bdd/ --emit-ir
51
+
52
+ # Via the hub CLI
53
+ agent-qa gherkin tests/bdd/
54
+ ```
55
+
56
+ Exit codes: `0` = compiled (warnings allowed) · `1` = parse/authoring errors
57
+ (report and plan are still emitted so the suite can be iterated on).
58
+
59
+ ## 4. Output Contract
60
+
61
+ | Artifact | Location | Content |
62
+ | :--- | :--- | :--- |
63
+ | JSON report | `reports/gherkin_suite_report.json` | Per-feature totals, expanded cases with mapped steps, errors, gaps, `health_score_pct` |
64
+ | Markdown plan | stdout | Background, per-scenario steps with mapped actions, Examples tables, gap section |
65
+ | Universal IR | `--emit-ir` stdout | `TestCaseIR` dicts (id, title, tags, `StepIR` steps) consumable by `universal_ir` / `omni_*` tooling |
66
+
67
+ ## 5. Coverage-Gap Rules (proven vs owed)
68
+
69
+ | Rule | Trigger | Suggested Remedy |
70
+ | :--- | :--- | :--- |
71
+ | `G1-MISSING_ASSERTION` | Case has no Then/assert step | Add an observable outcome assertion |
72
+ | `G2-MISSING_NEGATIVE_PATH` | Feature has no invalid/error/fail scenario | Add at least one negative scenario |
73
+ | `G3-THIN_EXAMPLES` | Examples table has < 2 rows | Add boundary + negative rows (empty, max length, special chars) |
74
+ | *(error)* Unresolved `<param>` | Example header does not cover a used parameter | Fix the header typo or remove the parameter |
75
+
76
+ ## 6. Limitations (Honest Scope)
77
+
78
+ - English Gherkin keywords only; `# language:` headers are ignored.
79
+ - NL→IR mapping is deterministic keyword heuristics — best-effort, not an LLM
80
+ semantic parser; unmapped phrasing degrades to `CUSTOM_CALL` with the original
81
+ text preserved so nothing is silently lost.
82
+ - Compilation produces the suite plan and IR; executing against a live AUT is
83
+ the job of the target framework (Playwright/Katalon) via `omni_transpiler`
84
+ and the `clean-pom-crafter` skill.
85
+
86
+ ## 7. Reference Examples
87
+
88
+ - `examples/checkout.feature` — full-featured input (Background, Outline, two
89
+ Examples blocks, tags, data tables).
90
+ - `examples/suite_report_sample.json` — trimmed JSON report shape emitted to
91
+ `reports/gherkin_suite_report.json`.
92
+
93
+ ---
94
+
95
+ ## 8. Boundary vs `lam-script-generator` and `clean-pom-crafter`
96
+
97
+ - **This skill** = deterministic `.feature` → test matrix/IR compiler (0 LLM
98
+ tokens). Prefer it whenever BDD specs already exist.
99
+ - **`lam-script-generator`** is the LLM-reasoning path for stories that are NOT
100
+ in Gherkin form; feed this skill's Universal IR to it when codification needs
101
+ story-level reasoning.
102
+ - **`clean-pom-crafter`** receives the matrix/IR from either path and owns all
103
+ Page Class + locator decisions — this skill never emits raw selectors.
@@ -0,0 +1,39 @@
1
+ @checkout @smoke
2
+ Feature: Checkout Flow
3
+ Customers buy items and receive an order confirmation.
4
+
5
+ Background:
6
+ Given the user is logged in as "member"
7
+
8
+ @happy
9
+ Scenario: Pay with credit card
10
+ When the user navigates to "https://shop.example.com/checkout"
11
+ And the user fills in "4111-1111-1111-1111" into "Card Number"
12
+ And the user clicks the "Pay Now" button
13
+ Then the user should see "Order Confirmed"
14
+ And the url should contain "/confirmation"
15
+
16
+ @data-driven
17
+ Scenario Outline: Apply voucher <voucher>
18
+ When the user enters "<code>" into "Voucher Input"
19
+ And the user clicks the "Apply" button
20
+ Then the total should show "<total>"
21
+
22
+ @valid
23
+ Examples:
24
+ | voucher | code | total |
25
+ | percent | SAVE10 | 90 |
26
+ | fixed | SHIP0 | 100 |
27
+
28
+ @edge
29
+ Examples: Boundary rows
30
+ | voucher | code | total |
31
+ | expired | OLD1 | 200 |
32
+
33
+ Rule: Guest checkout restrictions
34
+
35
+ @negative
36
+ Scenario: Invalid card is rejected
37
+ When the user fills in "4000-0000-0000-0002" into "Card Number"
38
+ And the user clicks the "Pay Now" button
39
+ Then the user should see "Payment declined"