@teevee98/synapse-hub 0.0.0-stage → 3.8.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/agents/api-engineer.md +67 -0
- package/.agents/agents/failure-analyst.md +75 -0
- package/.agents/agents/live-explorer.md +70 -0
- package/.agents/agents/orchestrator.md +104 -0
- package/.agents/agents/perf-engineer.md +74 -0
- package/.agents/agents/pom-crafter.md +112 -0
- package/.agents/agents/quality-gatekeeper.md +87 -0
- package/.agents/agents/test-designer.md +73 -0
- package/.agents/agents/test-runner.md +67 -0
- package/.agents/rules/author-integrity-guard.md +33 -0
- package/.agents/rules/communication-style.md +18 -0
- package/.agents/rules/context-engineering.md +37 -0
- package/.agents/rules/git-push-protocol.md +33 -0
- package/.agents/rules/graphify.md +14 -0
- package/.agents/rules/lessons-learned.md +25 -0
- package/.agents/rules/mandatory-playwright-live.md +25 -0
- package/.agents/rules/mobile_appium_rules.md +43 -0
- package/.agents/rules/performance_testing_rules.md +35 -0
- package/.agents/rules/skill-precedence.md +29 -0
- package/.agents/rules/systematic-debugging.md +36 -0
- package/.agents/skills/actor-qa-swarm/SKILL.md +77 -0
- package/.agents/skills/actor-qa-swarm/examples/broker_sharding_sample.json +12 -0
- package/.agents/skills/agent-orchestration-multi-agent-optimize/SKILL.md +88 -0
- package/.agents/skills/agent-orchestration-multi-agent-optimize/examples/swarm_workload_config.json +15 -0
- package/.agents/skills/agentless-qa-repair/SKILL.md +69 -0
- package/.agents/skills/agentless-qa-repair/examples/healing_report_sample.json +17 -0
- package/.agents/skills/agentless-qa-repair/examples/three_phase_repair_workflow.md +40 -0
- package/.agents/skills/antigravity-test-sentinel/SKILL.md +63 -0
- package/.agents/skills/antigravity-test-sentinel/examples/dod_gate_policy.json +11 -0
- package/.agents/skills/antigravity-test-sentinel/examples/sprint_dod_enforcement.md +47 -0
- package/.agents/skills/api-network-mocking-interception/SKILL.md +142 -0
- package/.agents/skills/api-network-mocking-interception/examples/network_route_stub.json +9 -0
- package/.agents/skills/api-spec-to-e2e-matrix/SKILL.md +80 -0
- package/.agents/skills/api-spec-to-e2e-matrix/examples/sample_openapi.json +19 -0
- package/.agents/skills/architecture-decision-records/SKILL.md +64 -0
- package/.agents/skills/architecture-decision-records/examples/0001-record-architecture-decisions.md +12 -0
- package/.agents/skills/auth-session-and-cookie-manager/SKILL.md +75 -0
- package/.agents/skills/auth-session-and-cookie-manager/examples/multi_role_auth_matrix.md +70 -0
- package/.agents/skills/auth-session-and-cookie-manager/examples/storage_state_sample.json +11 -0
- package/.agents/skills/backend-load-and-stress-patterns/SKILL.md +85 -0
- package/.agents/skills/backend-load-and-stress-patterns/examples/k6_load_test_sample.js +13 -0
- package/.agents/skills/backend-load-and-stress-patterns/references/k6_load_template.js +82 -0
- package/.agents/skills/bats-testing-patterns/SKILL.md +100 -0
- package/.agents/skills/bats-testing-patterns/examples/cli_runner_test.bats +6 -0
- package/.agents/skills/bats-testing-patterns/resources/implementation-playbook.md +614 -0
- package/.agents/skills/clean-pom-crafter/SKILL.md +110 -0
- package/.agents/skills/clean-pom-crafter/examples/standard_test_case.groovy +28 -0
- package/.agents/skills/clean-pom-crafter/references/pom_patterns.groovy +138 -0
- package/.agents/skills/code-refactoring-refactor-clean/SKILL.md +64 -0
- package/.agents/skills/code-refactoring-refactor-clean/examples/refactored_page_sample.groovy +9 -0
- package/.agents/skills/code-refactoring-refactor-clean/examples/refactored_playwright_pom.ts +52 -0
- package/.agents/skills/code-refactoring-refactor-clean/resources/implementation-playbook.md +881 -0
- package/.agents/skills/consumer-driven-contract-testing/SKILL.md +147 -0
- package/.agents/skills/consumer-driven-contract-testing/examples/order_service_pact.json +45 -0
- package/.agents/skills/db-state-verifier-and-seeder/SKILL.md +123 -0
- package/.agents/skills/db-state-verifier-and-seeder/examples/db_fixture_seed.json +12 -0
- package/.agents/skills/e2e-testing-patterns/SKILL.md +103 -0
- package/.agents/skills/e2e-testing-patterns/examples/playwright_sharded_config.ts +12 -0
- package/.agents/skills/e2e-testing-patterns/resources/implementation-playbook.md +522 -0
- package/.agents/skills/event-stream-qa-orchestrator/SKILL.md +79 -0
- package/.agents/skills/event-stream-qa-orchestrator/examples/timeline_sample.jsonl +2 -0
- package/.agents/skills/failure-autopsy-and-healing/SKILL.md +66 -0
- package/.agents/skills/failure-autopsy-and-healing/examples/closed_loop_healing_walkthrough.md +48 -0
- package/.agents/skills/failure-autopsy-and-healing/examples/sample_junit_failure.xml +8 -0
- package/.agents/skills/failure-autopsy-and-healing/references/healing_matrix.md +64 -0
- package/.agents/skills/failure-autopsy-and-healing/references/qa_investigation_protocol.md +56 -0
- package/.agents/skills/failure-autopsy-and-healing/scripts/triage_failures.sh +83 -0
- package/.agents/skills/file-and-report-validator/SKILL.md +90 -0
- package/.agents/skills/file-and-report-validator/examples/sample_export_schema.json +14 -0
- package/.agents/skills/flaky-test-sentinel-and-quarantine/SKILL.md +114 -0
- package/.agents/skills/flaky-test-sentinel-and-quarantine/examples/quarantine_registry.json +11 -0
- package/.agents/skills/gherkin-to-suite/SKILL.md +103 -0
- package/.agents/skills/gherkin-to-suite/examples/checkout.feature +39 -0
- package/.agents/skills/gherkin-to-suite/examples/suite_report_sample.json +334 -0
- package/.agents/skills/github-actions-templates/SKILL.md +77 -0
- package/.agents/skills/github-actions-templates/examples/matrix_test_pipeline.yml +8 -0
- package/.agents/skills/graphify-impact-analyzer/SKILL.md +76 -0
- package/.agents/skills/graphify-impact-analyzer/examples/dependency_blast_radius.json +8 -0
- package/.agents/skills/katalon-cli-runner/SKILL.md +76 -0
- package/.agents/skills/katalon-cli-runner/examples/headless_kre_exec.sh +3 -0
- package/.agents/skills/katalon-cli-runner/references/groovy-patterns.md +209 -0
- package/.agents/skills/katalon-cli-runner/references/katalon-api.md +155 -0
- package/.agents/skills/katalon-cli-runner/references/runner_commands.md +67 -0
- package/.agents/skills/lam-script-generator/SKILL.md +123 -0
- package/.agents/skills/lam-script-generator/examples/user_story_sample.md +8 -0
- package/.agents/skills/lean-code-ladder/SKILL.md +94 -0
- package/.agents/skills/lean-code-ladder/examples/ladder_walkthrough.md +37 -0
- package/.agents/skills/lean-impact-scoreboard/SKILL.md +48 -0
- package/.agents/skills/lean-impact-scoreboard/examples/sample_benchmark_gain.md +20 -0
- package/.agents/skills/lean-impact-scoreboard/examples/sample_scoreboard.md +15 -0
- package/.agents/skills/meta-prompt-crafter/SKILL.md +107 -0
- package/.agents/skills/meta-prompt-crafter/examples/qa_prompt_template.md +35 -0
- package/.agents/skills/mipro-prompt-compiler/SKILL.md +59 -0
- package/.agents/skills/mipro-prompt-compiler/examples/compiled_module_sample.json +22 -0
- package/.agents/skills/mipro-prompt-compiler/examples/dspy_assertion_optimizer_case.md +46 -0
- package/.agents/skills/mobile-appium-som/SKILL.md +88 -0
- package/.agents/skills/mobile-appium-som/examples/BaseScreen.java +88 -0
- package/.agents/skills/mobile-appium-som/examples/LoginScreen.java +46 -0
- package/.agents/skills/mobile-appium-som/examples/TC_MobileLogin.java +43 -0
- package/.agents/skills/persistent-planning-files/SKILL.md +108 -0
- package/.agents/skills/persistent-planning-files/examples/task_plan_example.md +56 -0
- package/.agents/skills/playwright-live-interaction/SKILL.md +131 -0
- package/.agents/skills/playwright-live-interaction/examples/live_probe_script.js +5 -0
- package/.agents/skills/playwright-live-interaction/references/advanced_superpowers.md +160 -0
- package/.agents/skills/playwright-live-interaction/references/compact_accessibility_snapshots.md +75 -0
- package/.agents/skills/playwright-live-interaction/references/recipes.md +388 -0
- package/.agents/skills/playwright-live-interaction/references/swarm_orchestration.md +134 -0
- package/.agents/skills/playwright-webapp-testing/SKILL.md +93 -0
- package/.agents/skills/playwright-webapp-testing/examples/server_lifecycle_playwright.py +63 -0
- package/.agents/skills/qa-aci-interface/SKILL.md +67 -0
- package/.agents/skills/qa-aci-interface/examples/surgical_code_window_walkthrough.md +40 -0
- package/.agents/skills/qa-aci-interface/examples/surgical_edit_sample.json +12 -0
- package/.agents/skills/qa-memory-os/SKILL.md +80 -0
- package/.agents/skills/qa-memory-os/examples/archival_memory_sample.json +21 -0
- package/.agents/skills/rbt-manual-testing/SKILL.md +137 -0
- package/.agents/skills/rbt-manual-testing/examples/checkout_rbt_manual_suite.md +25 -0
- package/.agents/skills/rbt-manual-testing/references/field_validation_checklist.md +29 -0
- package/.agents/skills/repo-quality-audit/SKILL.md +68 -0
- package/.agents/skills/repo-quality-audit/examples/sample_audit_walkthrough.md +15 -0
- package/.agents/skills/repo-quality-audit/examples/sample_repo_audit.md +33 -0
- package/.agents/skills/review-spec/SKILL.md +60 -0
- package/.agents/skills/review-spec/examples/pocock_grilling_decision_rubric.md +40 -0
- package/.agents/skills/review-spec/examples/sample_spec_review.md +33 -0
- package/.agents/skills/security-dast-sentinel/SKILL.md +119 -0
- package/.agents/skills/security-dast-sentinel/examples/dast_scan_config.json +10 -0
- package/.agents/skills/security-dast-sentinel/references/AI-AND-LLM.md +113 -0
- package/.agents/skills/security-dast-sentinel/references/ATTACK-CLASSES.md +89 -0
- package/.agents/skills/security-dast-sentinel/references/dast_recipes.ts +33 -0
- package/.agents/skills/self-healing-locator/SKILL.md +86 -0
- package/.agents/skills/self-healing-locator/examples/aom_diff_sample.json +16 -0
- package/.agents/skills/senior-code-review/SKILL.md +108 -0
- package/.agents/skills/senior-code-review/examples/sample_review_walkthrough.md +29 -0
- package/.agents/skills/skillspector-security-guard/SKILL.md +57 -0
- package/.agents/skills/skillspector-security-guard/examples/audit_threat_vectors_report.json +35 -0
- package/.agents/skills/skillspector-security-guard/examples/safe_mcp_config.json +12 -0
- package/.agents/skills/surgical-coding-principles/SKILL.md +89 -0
- package/.agents/skills/surgical-coding-principles/examples/surgical_vs_scope_creep.md +46 -0
- package/.agents/skills/synthetic-test-data-generator/SKILL.md +135 -0
- package/.agents/skills/synthetic-test-data-generator/examples/idempotent_data_recipe.json +9 -0
- package/.agents/skills/systematic-debugging/SKILL.md +100 -0
- package/.agents/skills/systematic-debugging/examples/four_phase_case.md +44 -0
- package/.agents/skills/technical-debt-harvester/SKILL.md +70 -0
- package/.agents/skills/technical-debt-harvester/examples/sample_debt_ledger.md +25 -0
- package/.agents/skills/technical-debt-harvester/examples/sample_ledger_output.md +20 -0
- package/.agents/skills/test-driven-development/SKILL.md +114 -0
- package/.agents/skills/test-driven-development/examples/red_green_refactor_qa.md +78 -0
- package/.agents/skills/token-and-context-optimizer/SKILL.md +70 -0
- package/.agents/skills/token-and-context-optimizer/examples/distilled_log_sample.txt +3 -0
- package/.agents/skills/token-and-context-optimizer/references/context_engineering_guide.md +49 -0
- package/.agents/skills/tot-exploratory-testing/SKILL.md +59 -0
- package/.agents/skills/tot-exploratory-testing/examples/payment_gateway_tot_tree.json +62 -0
- package/.agents/skills/tot-exploratory-testing/examples/tot_graph_sample.json +35 -0
- package/.agents/skills/verification-before-completion/SKILL.md +100 -0
- package/.agents/skills/verification-before-completion/examples/evidence_log_example.md +46 -0
- package/.agents/skills/visual-regression-sentinel/SKILL.md +82 -0
- package/.agents/skills/visual-regression-sentinel/examples/visual_mask_rules.json +9 -0
- package/.agents/skills/wcag-audit-patterns/SKILL.md +77 -0
- package/.agents/skills/wcag-audit-patterns/examples/wcag_compliance_checklist.json +10 -0
- package/.agents/skills/wcag-audit-patterns/resources/implementation-playbook.md +541 -0
- package/.agents/skills/web-agent-gym-benchmarking/SKILL.md +62 -0
- package/.agents/skills/web-agent-gym-benchmarking/examples/benchmark_report_sample.json +22 -0
- package/.agents/skills/web-agent-gym-benchmarking/examples/enterprise_tasks_suite.json +38 -0
- package/.agents/skills/web-vitals-and-performance-profiler/SKILL.md +65 -0
- package/.agents/skills/web-vitals-and-performance-profiler/examples/cwv_budget_policy.json +6 -0
- package/.agents/skills/web-vitals-and-performance-profiler/examples/playwright_cwv_profiler_fixture.ts +118 -0
- package/.agents/templates/completion_report.md +57 -0
- package/.agents/workflows/analyze_requirement_document.md +18 -0
- package/.agents/workflows/export_testcases_excel.md +32 -0
- package/.agents/workflows/generate_combinatorial_test_data.md +28 -0
- package/.agents/workflows/generate_cross_module_test_plan.md +37 -0
- package/.agents/workflows/generate_manual_testcases_rbt.md +56 -0
- package/.agents/workflows/generate_requirements_from_website.md +25 -0
- package/.agents/workflows/generate_testcases_from_requirements.md +21 -0
- package/.agents/workflows/graphify.md +10 -0
- package/.agents/workflows/test-impact.md +15 -0
- package/.agents/workflows/verify.md +34 -0
- package/.cursor/rules/agents.mdc +16 -0
- package/.cursor/rules/language.mdc +7 -0
- package/.cursor/rules/synapse-qa.mdc +24 -0
- package/.cursorrules +27 -0
- package/.dockerignore +53 -0
- package/.pre-commit-hooks.yaml +14 -0
- package/.windsurfrules +26 -0
- package/AGENTS.md +86 -0
- package/CLAUDE.md +54 -0
- package/Dockerfile +81 -0
- package/LICENSE +52 -0
- package/NOTICE +264 -0
- package/README.en.md +171 -0
- package/README.md +170 -2
- package/action.yml +51 -0
- package/agentskills.json +16 -0
- package/bin/agent-qa.js +138 -0
- package/docker-compose.yml +74 -0
- package/docs/ARCHITECTURE.en.md +198 -0
- package/docs/ARCHITECTURE.md +199 -0
- package/docs/CI_CD_INTEGRATION.en.md +170 -0
- package/docs/CI_CD_INTEGRATION.md +170 -0
- package/docs/CLI_REFERENCE.en.md +174 -0
- package/docs/CLI_REFERENCE.md +176 -0
- package/docs/CONTRIBUTING.en.md +49 -0
- package/docs/CONTRIBUTING.md +49 -0
- package/docs/PERFORMANCE_AUDIT_REPORT.md +783 -0
- package/docs/PLUGIN_DEVELOPMENT_GUIDE.md +127 -0
- package/docs/QUALITY_GATES.en.md +118 -0
- package/docs/QUALITY_GATES.md +118 -0
- package/docs/TOKEN_QUOTA_PLAYBOOK.md +41 -0
- package/docs/TOOL_CATALOG.md +129 -0
- package/docs/adopted-ecosystem-tools.md +96 -0
- package/docs/research/EXTERNAL_AGENT_SKILLS_SURVEY.md +1062 -0
- package/docs/research/agent_skill_training_analysis.md +1358 -0
- package/docs/schemas/synapse.config.schema.json +86 -0
- package/package.json +94 -4
- package/plugin.json +31 -0
- package/scripts/generate_skill_examples.py +244 -0
- package/scripts/install.ps1 +112 -0
- package/scripts/install_global_plugin.ps1 +94 -0
- package/scripts/install_global_plugin.sh +49 -0
- package/scripts/lint_skills.ps1 +57 -0
- package/scripts/package_dist.py +337 -0
- package/scripts/setup.sh +59 -0
- package/scripts/update_hub.ps1 +19 -0
- package/scripts/update_hub.sh +15 -0
- package/skills_manifest.json +854 -0
- package/synapse.config.json +31 -0
- package/templates/PROJECT_AI_GUIDE_TEMPLATE.md +111 -0
- package/templates/PROJECT_STATE_TEMPLATE.md +20 -0
- package/templates/ai_bridges/CLAUDE.md +54 -0
- package/templates/ai_bridges/copilot-instructions.md +20 -0
- package/templates/ai_bridges/cursor_synapse_qa.mdc +24 -0
- package/templates/ai_bridges/windsurfrules +26 -0
- package/templates/ci/github-actions-synapse.yml +95 -0
- package/templates/ci/gitlab-ci-synapse.yml +16 -0
- package/templates/ui/index.html +1069 -0
- package/templates/welcome.html.tmpl +2074 -0
- package/tools/.infer_baseline.json +30 -0
- package/tools/TOOLS_RUNBOOK.md +1553 -0
- package/tools/TOOL_CATALOG.json +1769 -0
- package/tools/__init__.py +43 -0
- package/tools/_openapi_parser.py +158 -0
- package/tools/agent_canary.py +525 -0
- package/tools/agentless_patcher.py +417 -0
- package/tools/analyze_project.py +262 -0
- package/tools/api_drift_sentinel.py +208 -0
- package/tools/api_fuzzer.py +762 -0
- package/tools/author_integrity.py +190 -0
- package/tools/auto_heal.py +271 -0
- package/tools/auto_heal_reflexion.py +532 -0
- package/tools/autogen_qa_broker.py +844 -0
- package/tools/benchmark_swarm_context.py +394 -0
- package/tools/browser_helpers.js +746 -0
- package/tools/browsergym_eval.py +236 -0
- package/tools/bug_reporter.py +252 -0
- package/tools/bump_version.py +423 -0
- package/tools/cache_resolver.py +1214 -0
- package/tools/check_env.py +426 -0
- package/tools/check_version_sync.py +256 -0
- package/tools/ci_healing_hook.py +126 -0
- package/tools/cli.py +1598 -0
- package/tools/config_decoupler.py +253 -0
- package/tools/contract_validator.py +141 -0
- package/tools/core/__init__.py +70 -0
- package/tools/core/adapters.py +431 -0
- package/tools/core/circuit_breaker.py +263 -0
- package/tools/core/cli_runtime.py +66 -0
- package/tools/core/file_lock.py +159 -0
- package/tools/core/i18n.py +394 -0
- package/tools/core/memory_bus.py +905 -0
- package/tools/core/process_supervisor.py +332 -0
- package/tools/core/telemetry.py +329 -0
- package/tools/count_tokens.py +1267 -0
- package/tools/create_mock_docx.py +104 -0
- package/tools/dashboard_generator.py +2403 -0
- package/tools/db_sandbox.py +675 -0
- package/tools/debt_harvester.py +237 -0
- package/tools/distill_log.py +161 -0
- package/tools/dspy_qa_signatures.py +356 -0
- package/tools/e2e_flow_prober.py +132 -0
- package/tools/export_postman_collection.py +945 -0
- package/tools/flaky_detector.py +389 -0
- package/tools/fsm_enforcer.py +450 -0
- package/tools/fsm_middleware.py +401 -0
- package/tools/fuzz_payloads.py +197 -0
- package/tools/generate_allure_report.py +88 -0
- package/tools/generate_api_matrix.py +193 -0
- package/tools/generate_jmx_plan.py +1322 -0
- package/tools/generate_sbom.py +252 -0
- package/tools/generate_tc_from_story.py +203 -0
- package/tools/generate_test_reports.py +133 -0
- package/tools/generate_tool_catalog.py +482 -0
- package/tools/gherkin_suite.py +758 -0
- package/tools/git_pre_commit_guard.py +298 -0
- package/tools/goldfish_review.py +366 -0
- package/tools/heal/__init__.py +28 -0
- package/tools/heal_pr_automator.py +220 -0
- package/tools/heuristic_locator_matcher.py +351 -0
- package/tools/hub_publisher.py +623 -0
- package/tools/humanize_linter.py +438 -0
- package/tools/infer_linter.py +977 -0
- package/tools/init_config.py +306 -0
- package/tools/interview_story.py +116 -0
- package/tools/jira_sync.py +566 -0
- package/tools/jtl_analyzer.py +474 -0
- package/tools/lint/__init__.py +30 -0
- package/tools/lint_ai_docs.py +374 -0
- package/tools/lookup_tc.py +165 -0
- package/tools/lookup_xpath.py +167 -0
- package/tools/matrix_sharder.py +148 -0
- package/tools/mcp/__init__.py +18 -0
- package/tools/mcp_docx_server.ps1 +13 -0
- package/tools/mcp_docx_server.py +851 -0
- package/tools/mcp_excel_server.ps1 +13 -0
- package/tools/mcp_excel_server.py +441 -0
- package/tools/mcp_server.py +725 -0
- package/tools/md_to_xlsx.py +327 -0
- package/tools/micro_mutator.py +887 -0
- package/tools/mipro_assertion_tuner.py +210 -0
- package/tools/network_mock_shield.py +311 -0
- package/tools/newman_analyzer.py +384 -0
- package/tools/omni_emitter.py +579 -0
- package/tools/omni_parser.py +450 -0
- package/tools/omni_transpiler.py +168 -0
- package/tools/openxml_guard.py +438 -0
- package/tools/output_formatter.py +182 -0
- package/tools/package_release.py +325 -0
- package/tools/perf/__init__.py +24 -0
- package/tools/pinpoint_failure.py +190 -0
- package/tools/project_state.py +171 -0
- package/tools/qa_event_stream.py +358 -0
- package/tools/repo_map.py +346 -0
- package/tools/report/__init__.py +32 -0
- package/tools/reversible_sandbox.py +344 -0
- package/tools/safety_firewall.py +437 -0
- package/tools/security_coverage_ledger.py +538 -0
- package/tools/security_scanner.py +229 -0
- package/tools/senior_code_review.py +393 -0
- package/tools/setup_project.py +770 -0
- package/tools/skill_collision_guard.py +304 -0
- package/tools/skill_profiles.py +217 -0
- package/tools/skill_vault.py +481 -0
- package/tools/skillspector.py +1278 -0
- package/tools/smart_visual_masker.py +132 -0
- package/tools/spec_drift_sentinel.py +438 -0
- package/tools/swe_qa_aci.py +369 -0
- package/tools/synapse_observer.py +485 -0
- package/tools/synapse_qa_bench.py +190 -0
- package/tools/sync_agent_rules.py +274 -0
- package/tools/telemetry_correlator.py +397 -0
- package/tools/template_excel_exporter.py +514 -0
- package/tools/templates/jmeter-distributed/README.md +54 -0
- package/tools/templates/jmeter-distributed/docker-compose.yml +83 -0
- package/tools/test_impact.py +419 -0
- package/tools/test_memory_os.py +462 -0
- package/tools/textgrad_mutation_optimizer.py +237 -0
- package/tools/tms_sync_bridge.py +834 -0
- package/tools/tot_exploratory_tester.py +239 -0
- package/tools/trace_data_flow.py +240 -0
- package/tools/traffic_session_converter.py +256 -0
- package/tools/trend_analytics.py +314 -0
- package/tools/ui_server.py +401 -0
- package/tools/universal_ir.py +488 -0
- package/tools/update_sentinel.py +383 -0
- package/tools/validate_groovy_scripts.py +173 -0
- package/tools/verify_all.py +1271 -0
- package/tools/visual_diff_canvas.py +143 -0
- package/tools/visual_som_helper.py +550 -0
- package/tools/watch_mode.py +308 -0
- package/tools/wcag_audit.py +239 -0
- package/tools/welcome_page.py +290 -0
package/.agents/skills/failure-autopsy-and-healing/examples/closed_loop_healing_walkthrough.md
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
# Closed-Loop Failure Autopsy & Autonomous Healing Walkthrough
|
|
2
|
+
|
|
3
|
+
> **Scenario:** End-to-end checkout test fails in CI runner due to locator drift on the order confirmation button.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. Fast Failure Pinpoint (0.1s from JUnit XML)
|
|
8
|
+
|
|
9
|
+
```bash
|
|
10
|
+
python tools/pinpoint_failure.py Reports/junit.xml
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
**Diagnostic Output:**
|
|
14
|
+
```text
|
|
15
|
+
❌ Failing Test Case : Tests/E2E_Checkout_Order.ts:42
|
|
16
|
+
📍 Failing Line : await page.locator("button#submit-v2").click()
|
|
17
|
+
⚠️ Exception Caught : TimeoutError: locator.click: Timeout 5000ms exceeded waiting for locator("button#submit-v2")
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## 2. Reflexion Verbal Diagnosis & AOM Heuristic Match
|
|
23
|
+
|
|
24
|
+
The Failure Analyst initiates reflection:
|
|
25
|
+
- **Reflexion:** *"Static ID selector `#submit-v2` did not appear within timeout window. Adjustment: Fallback to Accessibility role/name or dynamic data-testid."*
|
|
26
|
+
- **Heuristic Matcher:** Layer 1 semantic matching finds live element matching role `button` with label `"Confirm Order"`.
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## 3. Autonomous Patch Synthesis & Verification
|
|
31
|
+
|
|
32
|
+
`pom-crafter` applies the surgical patch to `CheckoutPage.ts`:
|
|
33
|
+
```diff
|
|
34
|
+
- await page.locator("button#submit-v2").click();
|
|
35
|
+
+ await page.getByRole("button", name: "Confirm Order").click();
|
|
36
|
+
+ await page.waitForLoadState("networkidle");
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
Targeted re-test:
|
|
40
|
+
```bash
|
|
41
|
+
python tools/auto_heal.py --execute --target-test "Tests/E2E_Checkout_Order.ts"
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
**Verdict:**
|
|
45
|
+
```text
|
|
46
|
+
✅ [TRIAL 1 SUCCESS] Patch verified passing in 1.4s!
|
|
47
|
+
💾 Recorded 2-line practical lesson to SQLite episodic memory.
|
|
48
|
+
```
|
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
<?xml version="1.0" encoding="UTF-8"?>
|
|
2
|
+
<testsuites>
|
|
3
|
+
<testsuite name="AuthenticationSuite" tests="1" failures="1">
|
|
4
|
+
<testcase name="testLoginFailure" classname="AuthTest">
|
|
5
|
+
<failure message="Timeout 30000ms exceeded waiting for locator('#submit-btn')" type="ElementNotFoundError"/>
|
|
6
|
+
</testcase>
|
|
7
|
+
</testsuite>
|
|
8
|
+
</testsuites>
|
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
# Enterprise Automation Failure Diagnostic & Healing Matrix
|
|
2
|
+
|
|
3
|
+
Reference documentation for the `failure-autopsy-and-healing` skill.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. Diagnostic & Healing Matrix
|
|
8
|
+
|
|
9
|
+
| Visual / Exception Symptom | Root Cause | Groovy POM Healing Patch |
|
|
10
|
+
| :--- | :--- | :--- |
|
|
11
|
+
| **Save button `[ ✓ ]` disabled / dimmed** | Angular Reactive Form missed change detection or missing required field (`*`). | `WebElement el = findElement(By.cssSelector("input[formcontrolname='code']"))`<br>`jsSetInput(el, uniqueCode)` |
|
|
12
|
+
| **Alert: *"Phạm vi giữa các khu/cụm không hợp lệ"*** | OpenLayers polygon overlaps previously saved boundary data. | `Actions actions = new Actions(driver)`<br>`actions.moveToElement(viewport, -180, -80).click().perform()` |
|
|
13
|
+
| **`ElementClickInterceptedException`** | Modal overlay `.p-component-overlay` fading out but still blocking clicks. | `((JavascriptExecutor) driver).executeScript("arguments[0].click();", element)` |
|
|
14
|
+
| **Red Toast: *"Mã đã tồn tại"*** | Non-unique test data colliding with existing records. | `String code = BasePage.uniqueCode("CCN_")` |
|
|
15
|
+
| **`StaleElementReferenceException`** | Angular DOM re-rendered after API response, detaching element. | `for (int i = 0; i < 3; i++) { try { findElement(by).click(); break; } catch (StaleElementReferenceException e) { sleep(500) } }` |
|
|
16
|
+
| **`TimeoutException` (Waiting for table/button)** | Loading overlay `.p-blockui` or spinner `.p-progress-spinner` blocking UI. | `waitForElementNotVisible(By.cssSelector(".p-blockui, .p-progress-spinner"), 10)` |
|
|
17
|
+
| **PrimeNG Dropdown value not applied** | Clicked `p-dropdown` but overlay panel `.p-dropdown-panel` has not rendered. | `waitForElementVisible(By.cssSelector(".p-dropdown-panel"), 5)`<br>`findElement(By.xpath("//li[contains(., '" + text + "')]")).click()` |
|
|
18
|
+
| **Blank Screenshot** | Browser crashed before capturing UI (OOM or lost WebDriver session). | Inspect `console0.log`, kill zombie drivers (`pkill -f chromedriver`), restart session. |
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## 2. Four Flaky Test Root Causes & Cures
|
|
23
|
+
|
|
24
|
+
| Flaky Root Cause | Symptom | Curing Strategy (Root Fix) |
|
|
25
|
+
|---|---|---|
|
|
26
|
+
| **1. Unstable Locator** | Random failures due to CSS refactoring or dynamic hashes | Enforce Master Priority Map: switch to `[formcontrolname]`, `[data-testid]`; forbid positional XPaths. |
|
|
27
|
+
| **2. Timing & Race Condition** | Fails under network latency; clicks before Angular finishes render | Replace all hard sleeps with Smart Wait: `waitForElementVisible`, wait for `.p-blockui` / `.p-progress-spinner` to hide. |
|
|
28
|
+
| **3. Test Data Conflict** | Fails on parallel or repeated runs due to duplicate codes | Enforce `BasePage.uniqueCode(prefix)` with epoch timestamps. |
|
|
29
|
+
| **4. Environment Dependency** | Fails due to expired session or stale browser cache | Use `LoginPage.ensureLoggedIn()` before every script. |
|
|
30
|
+
|
|
31
|
+
---
|
|
32
|
+
|
|
33
|
+
## 3. Fast-Fix Triad (Zero-Token Diagnostic & Targeted Healing Workflow)
|
|
34
|
+
|
|
35
|
+
To maximize fix speed and minimize token consumption without degrading test fidelity, follow this 3-step loop:
|
|
36
|
+
|
|
37
|
+
### Step 1: Zero-Token Pinpoint (CLI Log Extraction)
|
|
38
|
+
Never read entire JUnit XML reports (>50KB). Extract failed test cases, exception reasons, and file line numbers in <50 tokens:
|
|
39
|
+
```bash
|
|
40
|
+
python3 tools/pinpoint_failure.py Reports/<timestamp>/<suite>/JUnit_Report.xml
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
### Step 2: Instant Form & DOM Audit (Playwright MCP Recipe 15)
|
|
44
|
+
When a test fails because a button is disabled (e.g. Save `[ ✓ ]`), never guess which field is invalid. Execute Recipe 15 via `browser_run_code_unsafe` on the active live page:
|
|
45
|
+
```javascript
|
|
46
|
+
() => {
|
|
47
|
+
const invalids = Array.from(document.querySelectorAll('form .ng-invalid, input.ng-invalid, p-dropdown.ng-invalid'));
|
|
48
|
+
return invalids.map(el => ({
|
|
49
|
+
tag: el.tagName,
|
|
50
|
+
name: el.getAttribute('name') || el.getAttribute('formcontrolname') || el.id,
|
|
51
|
+
classes: el.className
|
|
52
|
+
}));
|
|
53
|
+
}
|
|
54
|
+
```
|
|
55
|
+
This instantly highlights missing required inputs, pattern mismatches, or invalid dropdowns.
|
|
56
|
+
|
|
57
|
+
### Step 3: Targeted Single-TC Re-test (Zero Waste)
|
|
58
|
+
Never rerun an entire Test Suite to verify a 1-line locator or timing fix. Rerun only the targeted test case:
|
|
59
|
+
```bash
|
|
60
|
+
# Via KRE CLI (License mode):
|
|
61
|
+
katalonc -noSplash -runMode=console -projectPath="." -retry=0 -testCasePath="Test Cases/TC-xxx" -browserType="Chrome (headless)"
|
|
62
|
+
|
|
63
|
+
# Or verify live flow in 2 seconds via Playwright MCP script.
|
|
64
|
+
```
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# QA Investigation & Root-Cause Analysis Protocol
|
|
2
|
+
|
|
3
|
+
*(Inherited from `fugazi/test-automation-skills-agents`)*
|
|
4
|
+
|
|
5
|
+
A file-backed, evidence-first investigation journal protocol for diagnosing failing and flaky automation test cases in enterprise automation projects.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## 1. Core Principles
|
|
10
|
+
|
|
11
|
+
1. **Context Window is Volatile RAM; Filesystem is Persistent Disk:**
|
|
12
|
+
- Long triage sessions cause context drift. All goals, hypotheses, and evidence must be logged in markdown.
|
|
13
|
+
2. **Discovered Classification, Not Presumed:**
|
|
14
|
+
- Never assume upfront whether a failure is a Flaky test, an AUT Defect, or a Test Infrastructure issue. The classification is a finding discovered through evidence.
|
|
15
|
+
3. **No Guesswork / No Blind Fixes:**
|
|
16
|
+
- Every fix must address a proven root cause, not a symptom.
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## 2. Four-Phase Investigation Cycle
|
|
21
|
+
|
|
22
|
+
```mermaid
|
|
23
|
+
flowchart TD
|
|
24
|
+
P1["<b>Phase 1: Reproduction & Triage</b><br/>• Run single-TC isolated re-test<br/>• Classify: Intermittent (Flaky) vs Deterministic (Bug)"]
|
|
25
|
+
P2["<b>Phase 2: Evidence Collection</b><br/>• Extract CLI logs & stack trace<br/>• Inspect Screenshots/FAILED_*.png<br/>• Capture network & timing data"]
|
|
26
|
+
P3["<b>Phase 3: Hypothesis & Root Cause</b><br/>• Form testable hypothesis<br/>• Isolate: Timing vs Stale DOM vs AUT defect"]
|
|
27
|
+
P4["<b>Phase 4: Targeted Fix & Validation</b><br/>• Apply Clean POM Page Class fix<br/>• Validate 3x consecutive runs<br/>• Run verify_all.py DoD check"]
|
|
28
|
+
|
|
29
|
+
P1 --> P2 --> P3 --> P4
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
### Phase 1: Reproduction & Triage
|
|
33
|
+
- Run the failing test in isolation using the targeted single-TC runner.
|
|
34
|
+
- Determine failure pattern:
|
|
35
|
+
- **Deterministic Bug:** Fails 100% of the time with identical stack trace.
|
|
36
|
+
- **Intermittent Flaky:** Passes sometimes, fails under load/timing variations.
|
|
37
|
+
- **Environment Blocker:** Fails on 401/403 session expiration or network timeout.
|
|
38
|
+
|
|
39
|
+
### Phase 2: Evidence Collection
|
|
40
|
+
- Collect ground-truth artifacts:
|
|
41
|
+
1. CLI log excerpt around the failure point.
|
|
42
|
+
2. Failure screenshot in `Screenshots/FAILED_<TestCaseId>_*.png`.
|
|
43
|
+
3. Form audit state (e.g. disabled Save button, red validation borders).
|
|
44
|
+
4. Redact any sensitive credentials or personal tokens.
|
|
45
|
+
|
|
46
|
+
### Phase 3: Hypothesis Formulation
|
|
47
|
+
- Check the 4 most common failure archetypes:
|
|
48
|
+
- **Archetype A (PrimeNG Toast Collision):** Previous toast covers the button. Cure: `dismissToast()`.
|
|
49
|
+
- **Archetype B (Stale Element / Angular Re-render):** DOM re-created mid-click. Cure: Re-fetch locator with `BasePage.x()`.
|
|
50
|
+
- **Archetype C (Dialog Backdrop Overlay):** `cdk-overlay-backdrop` blocks click. Cure: Wait for backdrop fade-out.
|
|
51
|
+
- **Archetype D (Real AUT Defect):** Backend HTTP 500 or constraint violation. Cure: Log to `KNOWN_ISSUES.md`.
|
|
52
|
+
|
|
53
|
+
### Phase 4: Targeted Fix & Validation
|
|
54
|
+
- Implement fix exclusively inside Page Classes in `Keywords/<namespace>/pages/` or `pages/` (never in `Scripts/`).
|
|
55
|
+
- Validate stability across multiple runs.
|
|
56
|
+
- Run `python3 tools/verify_all.py` to confirm zero regressions.
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
#!/usr/bin/env bash
|
|
2
|
+
# Triage Failures Script for Enterprise Automation
|
|
3
|
+
# Cross-platform: Compatible with macOS, Linux CI, and Windows Git Bash
|
|
4
|
+
|
|
5
|
+
WORKSPACE_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../../.." && pwd)"
|
|
6
|
+
SCREENSHOT_DIR="$WORKSPACE_DIR/Screenshots"
|
|
7
|
+
REPORTS_DIR="$WORKSPACE_DIR/Reports"
|
|
8
|
+
|
|
9
|
+
echo "=========================================================="
|
|
10
|
+
echo " 🔬 ENTERPRISE FAILURE AUTOPSY & TRIAGE SCANNER (CROSS-PLATFORM)"
|
|
11
|
+
echo " Workspace: $WORKSPACE_DIR"
|
|
12
|
+
echo "=========================================================="
|
|
13
|
+
|
|
14
|
+
# 1. Quét báo cáo JUnit XML gần nhất
|
|
15
|
+
LATEST_JUNIT=$(find "$REPORTS_DIR" -type f -name "JUnit_Report.xml" 2>/dev/null | sort -r | head -n 1)
|
|
16
|
+
|
|
17
|
+
if [ -n "$LATEST_JUNIT" ]; then
|
|
18
|
+
echo "📊 Báo cáo JUnit gần nhất: $(basename "$(dirname "$LATEST_JUNIT")")/JUnit_Report.xml"
|
|
19
|
+
FAILED_JUNIT_TESTS=$(grep -B 1 '<failure' "$LATEST_JUNIT" 2>/dev/null | grep '<testcase' | sed -n 's/.*name="\([^"]*\)".*/\1/p')
|
|
20
|
+
if [ -n "$FAILED_JUNIT_TESTS" ]; then
|
|
21
|
+
echo "🚨 Các Test Cases báo FAILED trong JUnit XML:"
|
|
22
|
+
echo "$FAILED_JUNIT_TESTS" | while read -r tc; do
|
|
23
|
+
echo " ❌ $tc"
|
|
24
|
+
done
|
|
25
|
+
echo "----------------------------------------------------------"
|
|
26
|
+
else
|
|
27
|
+
echo "✅ Không phát hiện ca thất bại nào trong JUnit XML gần nhất."
|
|
28
|
+
echo "----------------------------------------------------------"
|
|
29
|
+
fi
|
|
30
|
+
fi
|
|
31
|
+
|
|
32
|
+
# 2. Quét ảnh chụp FAILED_*.png
|
|
33
|
+
if [ ! -d "$SCREENSHOT_DIR" ]; then
|
|
34
|
+
echo "ℹ️ Không tìm thấy thư mục Screenshots. Dự án đang sạch lỗi!"
|
|
35
|
+
exit 0
|
|
36
|
+
fi
|
|
37
|
+
|
|
38
|
+
FAILED_COUNT=$(find "$SCREENSHOT_DIR" -maxdepth 1 -name "FAILED_*.png" 2>/dev/null | wc -l | tr -d ' ')
|
|
39
|
+
|
|
40
|
+
if [ "$FAILED_COUNT" -eq "0" ]; then
|
|
41
|
+
echo "✅ Tuyệt vời! Không có ảnh lỗi nào trong $SCREENSHOT_DIR."
|
|
42
|
+
exit 0
|
|
43
|
+
fi
|
|
44
|
+
|
|
45
|
+
echo "⚠️ Phát hiện $FAILED_COUNT ảnh lỗi kịch bản gần nhất:"
|
|
46
|
+
echo "----------------------------------------------------------"
|
|
47
|
+
|
|
48
|
+
# Hàm lấy thông tin file đa nền tảng (hỗ trợ Linux GNU stat, macOS BSD stat và fallback ls)
|
|
49
|
+
list_recent_screenshots() {
|
|
50
|
+
if stat --version 2>&1 | grep -q "GNU"; then
|
|
51
|
+
# Linux (GNU stat)
|
|
52
|
+
find "$SCREENSHOT_DIR" -maxdepth 1 -name "FAILED_*.png" -type f -exec stat -c "%Y %y %n" {} + 2>/dev/null | \
|
|
53
|
+
sort -rn | head -n 10 | awk '{print $1, $2, $3, $NF}'
|
|
54
|
+
elif stat -f "%m" . >/dev/null 2>&1; then
|
|
55
|
+
# macOS / BSD stat
|
|
56
|
+
find "$SCREENSHOT_DIR" -maxdepth 1 -name "FAILED_*.png" -type f -exec stat -f "%m %Sm %N" -t "%Y-%m-%d %H:%M:%S" {} + 2>/dev/null | \
|
|
57
|
+
sort -rn | head -n 10
|
|
58
|
+
else
|
|
59
|
+
# Fallback (POSIX ls)
|
|
60
|
+
ls -lt "$SCREENSHOT_DIR"/FAILED_*.png 2>/dev/null | head -n 10 | awk '{print "0", $6, $7, $NF}'
|
|
61
|
+
fi
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
list_recent_screenshots | while read -r epoch date time filepath; do
|
|
65
|
+
filename=$(basename "$filepath")
|
|
66
|
+
|
|
67
|
+
# Dynamic Page Class mapping via repo_map lookup or fallback to BasePage
|
|
68
|
+
suggested_page="BasePage.groovy"
|
|
69
|
+
if [ -f "tools/repo_map.txt" ] && [[ "$filename" =~ TC-([0-9]+) ]]; then
|
|
70
|
+
tc_pattern="TC-${BASH_REMATCH[1]}"
|
|
71
|
+
found_page=$(grep -m 1 -i "$tc_pattern" tools/repo_map.txt | awk '{print $1}' 2>/dev/null)
|
|
72
|
+
if [ -n "$found_page" ]; then
|
|
73
|
+
suggested_page="$found_page"
|
|
74
|
+
fi
|
|
75
|
+
fi
|
|
76
|
+
|
|
77
|
+
echo "📸 [$date $time] $filename"
|
|
78
|
+
echo " -> Đường dẫn ảnh : $filepath"
|
|
79
|
+
echo " -> Page Class cần vá : Keywords/pages/$suggested_page"
|
|
80
|
+
done
|
|
81
|
+
|
|
82
|
+
echo "----------------------------------------------------------"
|
|
83
|
+
echo "💡 Gợi ý AI: Dùng tool read_file mở ảnh hoặc dùng Playwright Live giữ nguyên browser để xác thực và vá Page Class."
|
|
@@ -0,0 +1,90 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: file-and-report-validator
|
|
3
|
+
description: "Validate downloaded and uploaded files (Excel, CSV, PDF) schemas and tabular integrity. Use when validating exported files or reports against a schema."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# File & Report Validator Skill
|
|
7
|
+
|
|
8
|
+
> **Target:** Eliminate superficial testing of file exports and imports by inspecting and asserting exact file contents, schemas, formulas, and values across Excel, CSV, and PDF formats.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## 1. Core Principles
|
|
13
|
+
|
|
14
|
+
1. **Verify Binary Contents, Not Just Download Events:**
|
|
15
|
+
- Catching the browser `download` event is insufficient. The downloaded file must be opened, parsed, and its tabular records cross-referenced against expected database or UI table states.
|
|
16
|
+
2. **Schema & Header Conformance:**
|
|
17
|
+
- Assert exact column headers, cell data formats (dates, currencies, numeric precision), and sheet names.
|
|
18
|
+
3. **Idempotent File Lifecycle:**
|
|
19
|
+
- Store temporary test downloads in a sandboxed scratch folder and purge them during teardown.
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## 2. Standard Playwright Download Handling
|
|
24
|
+
|
|
25
|
+
```typescript
|
|
26
|
+
import { test, expect } from '@playwright/test';
|
|
27
|
+
import path from 'path';
|
|
28
|
+
|
|
29
|
+
test('export and verify financial report', async ({ page }) => {
|
|
30
|
+
const downloadPromise = page.waitForEvent('download');
|
|
31
|
+
await page.click('button#export-excel');
|
|
32
|
+
const download = await downloadPromise;
|
|
33
|
+
|
|
34
|
+
const savePath = path.join('temp_downloads', download.suggestedFilename());
|
|
35
|
+
await download.saveAs(savePath);
|
|
36
|
+
|
|
37
|
+
expect(download.suggestedFilename()).toMatch(/^Report_\d{8}\.xlsx$/);
|
|
38
|
+
});
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
## 3. Deep Content Inspection Recipes (Python)
|
|
44
|
+
|
|
45
|
+
### Pattern A: Excel (.xlsx / .xls) Schema & Data Assertion
|
|
46
|
+
```python
|
|
47
|
+
import pandas as pd
|
|
48
|
+
|
|
49
|
+
def validate_excel_report(file_path: str, expected_min_rows: int = 1):
|
|
50
|
+
df = pd.read_excel(file_path)
|
|
51
|
+
|
|
52
|
+
# 1. Assert header schema
|
|
53
|
+
expected_headers = ["ID", "Customer Name", "Total Amount", "Status"]
|
|
54
|
+
assert list(df.columns) == expected_headers, f"Header mismatch: {df.columns}"
|
|
55
|
+
|
|
56
|
+
# 2. Assert row counts and non-empty values
|
|
57
|
+
assert len(df) >= expected_min_rows, f"Expected at least {expected_min_rows} rows, got {len(df)}"
|
|
58
|
+
assert not df["ID"].isnull().any(), "Found NULL values in ID column"
|
|
59
|
+
print(f"[PASS] Successfully verified {len(df)} rows in {file_path}")
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
### Pattern B: CSV Chunk Streaming (For Big Data Exports > 100k Rows)
|
|
63
|
+
```python
|
|
64
|
+
import csv
|
|
65
|
+
|
|
66
|
+
def validate_large_csv_stream(file_path: str, expected_header: list):
|
|
67
|
+
row_count = 0
|
|
68
|
+
with open(file_path, mode='r', encoding='utf-8') as f:
|
|
69
|
+
reader = csv.reader(f)
|
|
70
|
+
header = next(reader)
|
|
71
|
+
assert header == expected_header, f"CSV Header mismatch: {header}"
|
|
72
|
+
for row in reader:
|
|
73
|
+
row_count += 1
|
|
74
|
+
assert len(row) == len(header), f"Malformed row at line {row_count}"
|
|
75
|
+
print(f"[PASS] Verified {row_count} streamed CSV rows without memory overflow.")
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
### Pattern C: PDF Document Text & Metadata Extraction
|
|
79
|
+
```python
|
|
80
|
+
import pypdf
|
|
81
|
+
|
|
82
|
+
def assert_pdf_contains_text(pdf_path: str, expected_keywords: list):
|
|
83
|
+
reader = pypdf.PdfReader(pdf_path)
|
|
84
|
+
assert len(reader.pages) > 0, "PDF is empty (0 pages)!"
|
|
85
|
+
|
|
86
|
+
full_text = "\n".join([page.extract_text() or "" for page in reader.pages])
|
|
87
|
+
for kw in expected_keywords:
|
|
88
|
+
assert kw in full_text, f"PDF assertion failed: Keyword '{kw}' missing in {pdf_path}"
|
|
89
|
+
print(f"[PASS] Verified keywords {expected_keywords} across {len(reader.pages)} PDF pages.")
|
|
90
|
+
```
|
|
@@ -0,0 +1,114 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: flaky-test-sentinel-and-quarantine
|
|
3
|
+
description: "Detect non-deterministic flaky tests, calculate flakiness scores, and manage quarantine. Use when detecting flaky tests and managing quarantine lifecycles."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Flaky Test Sentinel & Quarantine Protocol
|
|
7
|
+
|
|
8
|
+
> **Purpose:** Master flaky test governance, automated quarantine isolation, and non-deterministic test remediation. Eliminates false-positive CI/CD pipeline blockers by identifying unstable tests through repeated execution ($N=5$), moving them into isolated quarantine suites, and tracking remediation until graduation.
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## 1. The Flaky Test Dilemma & The 5-Stage Quarantine Lifecycle
|
|
13
|
+
|
|
14
|
+
In large enterprise test suites (hundreds to thousands of test cases), non-deterministic tests ("flaky tests" that intermittently pass and fail without code changes) erode trust in CI/CD, slow down deployments, and cause developers to ignore test results.
|
|
15
|
+
|
|
16
|
+
```mermaid
|
|
17
|
+
flowchart TD
|
|
18
|
+
A["<b>1. Detection Engine</b><br/>Run test N=5 iterations<br/>Identify Pass/Fail flips"] --> B["<b>2. Automated Quarantine</b><br/>Tag test with @quarantine<br/>Isolate from main suite"]
|
|
19
|
+
B --> C["<b>3. Unblocked PR Pipeline</b><br/>PR merge runs stable tests only<br/>CI/CD remains green"]
|
|
20
|
+
B --> D["<b>4. Quarantine Lab</b><br/>Runs in parallel/nightly<br/>Logs failure autopsy"]
|
|
21
|
+
D --> E["<b>5. Graduation DoD Gate</b><br/>Must achieve 10/10 consecutive passes<br/>Graduates back to main"]
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
---
|
|
25
|
+
|
|
26
|
+
## 2. Flakiness Scoring Formula & Classification
|
|
27
|
+
|
|
28
|
+
$$P = \frac{\text{Passed Executions}}{N} \times 100\% \quad \text{where } N \ge 5$$
|
|
29
|
+
|
|
30
|
+
$$\text{Flip Rate} = \frac{\text{Status Transitions (Pass} \leftrightarrow \text{Fail)}}{N - 1} \times 100\%$$
|
|
31
|
+
|
|
32
|
+
| Health Tier | Pass Rate ($P$) | Flip Count ($N=5$) | Action Required | Pipeline Impact |
|
|
33
|
+
| :--- | :--- | :--- | :--- | :--- |
|
|
34
|
+
| **STABLE (Green)** | $100\%$ | 0 flips | Kept in standard regression suite | Blocks PR on real bug |
|
|
35
|
+
| **MILD FLAKE** | $80\% \le P < 100\%$ | 1 - 2 flips | Log warning; monitor over 24 hours | Kept in suite with watchpoint |
|
|
36
|
+
| **SEVERE FLAKE** | $P < 80\%$ | $\ge 2$ flips | **Immediate Quarantine Tagging (`@quarantine`)** | Excluded from PR merge blocker |
|
|
37
|
+
| **DEAD TEST / BROKEN**| $0\%$ | 0 flips | File critical AUT/Test bug defect | Regular test failure |
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## 3. Quarantine Tagging & Execution Strategies
|
|
42
|
+
|
|
43
|
+
### Playwright / TypeScript Quarantine Tagging
|
|
44
|
+
```typescript
|
|
45
|
+
import { test, expect } from '@playwright/test';
|
|
46
|
+
|
|
47
|
+
// Before Quarantine: Unstable test intermittently blocking main PRs
|
|
48
|
+
// test('TC-104: Submit monthly reconciliation report', async ({ page }) => { ... });
|
|
49
|
+
|
|
50
|
+
// After Quarantine: Tagged with @quarantine metadata
|
|
51
|
+
test('TC-104: Submit monthly reconciliation report @quarantine', async ({ page }) => {
|
|
52
|
+
// Test code remains intact for debugging
|
|
53
|
+
await page.goto('/reports/reconciliation');
|
|
54
|
+
// ...
|
|
55
|
+
});
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
### CI/CD Filter Configuration
|
|
59
|
+
```bash
|
|
60
|
+
# 1. Main PR Blocker Suite: Runs ONLY 100% stable tests (Zero Flaky Disruptions)
|
|
61
|
+
npx playwright test --grep-invert "@quarantine"
|
|
62
|
+
|
|
63
|
+
# 2. Nightly Quarantine Lab: Runs quarantined tests to collect diagnostics & heal
|
|
64
|
+
npx playwright test --grep "@quarantine" --reporter=json
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
---
|
|
68
|
+
|
|
69
|
+
## 4. Automated Multi-Run Flakiness Detector Tool
|
|
70
|
+
|
|
71
|
+
This repository provides an automated flakiness analyzer in `tools/flaky_detector.py`.
|
|
72
|
+
|
|
73
|
+
### Running 5-Iteration Flakiness Benchmark
|
|
74
|
+
```bash
|
|
75
|
+
# Run a specific test script 5 consecutive times to benchmark stability
|
|
76
|
+
synapse flaky --repeat 5 --cmd "npm test -- tests/e2e/TC-104.spec.ts"
|
|
77
|
+
|
|
78
|
+
# Or parse existing test execution history
|
|
79
|
+
synapse flaky --reports "reports/**/junit*.xml"
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
Output Report (`reports/quarantine_board.json`):
|
|
83
|
+
```json
|
|
84
|
+
{
|
|
85
|
+
"test_id": "TC-104",
|
|
86
|
+
"iterations": 5,
|
|
87
|
+
"passes": 3,
|
|
88
|
+
"failures": 2,
|
|
89
|
+
"pass_rate": 60.0,
|
|
90
|
+
"flakiness_status": "SEVERE_FLAKE",
|
|
91
|
+
"quarantine_recommended": true,
|
|
92
|
+
"probable_cause": "Angular PrimeNG animation fade-out / missing smart wait"
|
|
93
|
+
}
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
---
|
|
97
|
+
|
|
98
|
+
## 5. Root-Cause Remediation Guide (Heal & Graduate)
|
|
99
|
+
|
|
100
|
+
| Flakiness Root Cause | Anti-Pattern | Mandatory Remediation |
|
|
101
|
+
| :--- | :--- | :--- |
|
|
102
|
+
| **Race Condition** | Static `Thread.sleep(3000)` or blind delays | Replace with Smart Wait: `waitForLoadState('networkidle')` or Angular `zone.isStable()`. |
|
|
103
|
+
| **Database Collision** | Reusing static IDs (`"ORDER_001"`) | Use dynamic identifiers: `BasePage.uniqueCode("AUTO_")` paired with Idempotent Teardown. |
|
|
104
|
+
| **Animation Interference**| Clicking during backdrop fade | Use `BasePage.clickJS()` or await overlay invisibility (`.p-component-overlay`). |
|
|
105
|
+
| **DOM Re-render Drift** | Direct click after async API fetch | Encapsulate 3-attempt retry loop with AOM Role fallback in Page Class. |
|
|
106
|
+
|
|
107
|
+
---
|
|
108
|
+
|
|
109
|
+
## 6. Graduation Definition of Done (DoD) Gate
|
|
110
|
+
|
|
111
|
+
Before removing the `@quarantine` tag and restoring a test to the main PR suite:
|
|
112
|
+
1. **Verification Run:** The healed test must execute **10 consecutive iterations** in headless mode with **100% Pass Rate** (`10/10 PASS`).
|
|
113
|
+
2. **Code Standards:** 100% UTF-8 No BOM, Clean POM flat architecture, zero hardcoded XPaths.
|
|
114
|
+
3. **Artifact Update:** Update `reports/quarantine_board.json` to mark status as `GRADUATED`.
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: gherkin-to-suite
|
|
3
|
+
description: "Compile Gherkin .feature files into an expanded test matrix, hub Universal IR and a proven-vs-owed coverage-gap report with zero LLM tokens. Use when the user supplies BDD .feature specs or wants a BDD suite audited before codification."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Gherkin to Suite Compiler — Natural-Language Test Ingestion (0-Token)
|
|
7
|
+
|
|
8
|
+
This skill turns business-readable Gherkin `.feature` files into deterministic
|
|
9
|
+
test artifacts: an expanded test-case matrix, hub-native Universal IR for the
|
|
10
|
+
transpile pipeline, and a coverage-gap audit — all without a single LLM token.
|
|
11
|
+
|
|
12
|
+
> **Provenance:** Concept inspired by the Gherkin-driven E2E ingestion workflow
|
|
13
|
+
> popularized by TestZeus Hercules (AGPL-3.0); independently re-implemented
|
|
14
|
+
> from scratch for this MIT repository (no upstream code). The "proven vs
|
|
15
|
+
> owed" coverage traceability idea is inspired by Kane CLI (Apache-2.0).
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## 1. When to Activate
|
|
20
|
+
|
|
21
|
+
- User supplies `.feature` / BDD specs and wants an executable test suite plan.
|
|
22
|
+
- Need to audit a BDD suite for missing assertions, negative paths, or thin
|
|
23
|
+
example tables before codifying into Clean POM scripts.
|
|
24
|
+
- Need structured test data (JSON IR) to feed `omni_transpiler` / `universal_ir`
|
|
25
|
+
pipelines instead of hand-writing scripts.
|
|
26
|
+
|
|
27
|
+
## 2. Pipeline
|
|
28
|
+
|
|
29
|
+
```mermaid
|
|
30
|
+
flowchart TD
|
|
31
|
+
Feature["<b>.feature file(s)</b><br/>Feature / Rule / Background / Scenario Outline / Examples / Tags / Doc Strings / Data Tables"] --> Parser["<b>1. Gherkin Parser (stdlib state machine)</b><br/>• Tags attach to Feature, Rule, Scenario, Examples<br/>• And/But inherit the concrete Given/When/Then role<br/>• Doc strings dedent per Gherkin spec, \\| escapes honored"]
|
|
32
|
+
Parser --> Expand["<b>2. Outline Expansion</b><br/>• One concrete case per Examples row<br/>• <param> substitution across steps, tables, doc strings<br/>• Unresolved <param> = hard error (header typos caught)"]
|
|
33
|
+
Expand --> IR["<b>3. Universal IR Mapping (deterministic, 0-token)</b><br/>• navigate/open → GOTO, fill/enter → FILL, click/tap → CLICK<br/>• Then + quoted text → ASSERT contains_text<br/>• Unmapped phrasing → CUSTOM_CALL (text preserved)"]
|
|
34
|
+
Expand --> Gaps["<b>4. Proven-vs-Owed Gap Audit</b><br/>• G1 case without Then assertion<br/>• G2 no negative path in feature<br/>• G3 thin Examples table"]
|
|
35
|
+
IR --> Out["<b>Test Matrix + Markdown Plan + JSON report</b>"]
|
|
36
|
+
Gaps --> Out
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
## 3. Commands
|
|
40
|
+
|
|
41
|
+
```bash
|
|
42
|
+
# Compile a single feature (human summary + Markdown plan to stdout)
|
|
43
|
+
python tools/gherkin_suite.py tests/bdd/checkout.feature
|
|
44
|
+
|
|
45
|
+
# Compile a whole directory of .feature files
|
|
46
|
+
python tools/gherkin_suite.py tests/bdd/
|
|
47
|
+
|
|
48
|
+
# Machine-readable report to stdout / hub Universal IR for the omni pipeline
|
|
49
|
+
python tools/gherkin_suite.py tests/bdd/ --json
|
|
50
|
+
python tools/gherkin_suite.py tests/bdd/ --emit-ir
|
|
51
|
+
|
|
52
|
+
# Via the hub CLI
|
|
53
|
+
agent-qa gherkin tests/bdd/
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
Exit codes: `0` = compiled (warnings allowed) · `1` = parse/authoring errors
|
|
57
|
+
(report and plan are still emitted so the suite can be iterated on).
|
|
58
|
+
|
|
59
|
+
## 4. Output Contract
|
|
60
|
+
|
|
61
|
+
| Artifact | Location | Content |
|
|
62
|
+
| :--- | :--- | :--- |
|
|
63
|
+
| JSON report | `reports/gherkin_suite_report.json` | Per-feature totals, expanded cases with mapped steps, errors, gaps, `health_score_pct` |
|
|
64
|
+
| Markdown plan | stdout | Background, per-scenario steps with mapped actions, Examples tables, gap section |
|
|
65
|
+
| Universal IR | `--emit-ir` stdout | `TestCaseIR` dicts (id, title, tags, `StepIR` steps) consumable by `universal_ir` / `omni_*` tooling |
|
|
66
|
+
|
|
67
|
+
## 5. Coverage-Gap Rules (proven vs owed)
|
|
68
|
+
|
|
69
|
+
| Rule | Trigger | Suggested Remedy |
|
|
70
|
+
| :--- | :--- | :--- |
|
|
71
|
+
| `G1-MISSING_ASSERTION` | Case has no Then/assert step | Add an observable outcome assertion |
|
|
72
|
+
| `G2-MISSING_NEGATIVE_PATH` | Feature has no invalid/error/fail scenario | Add at least one negative scenario |
|
|
73
|
+
| `G3-THIN_EXAMPLES` | Examples table has < 2 rows | Add boundary + negative rows (empty, max length, special chars) |
|
|
74
|
+
| *(error)* Unresolved `<param>` | Example header does not cover a used parameter | Fix the header typo or remove the parameter |
|
|
75
|
+
|
|
76
|
+
## 6. Limitations (Honest Scope)
|
|
77
|
+
|
|
78
|
+
- English Gherkin keywords only; `# language:` headers are ignored.
|
|
79
|
+
- NL→IR mapping is deterministic keyword heuristics — best-effort, not an LLM
|
|
80
|
+
semantic parser; unmapped phrasing degrades to `CUSTOM_CALL` with the original
|
|
81
|
+
text preserved so nothing is silently lost.
|
|
82
|
+
- Compilation produces the suite plan and IR; executing against a live AUT is
|
|
83
|
+
the job of the target framework (Playwright/Katalon) via `omni_transpiler`
|
|
84
|
+
and the `clean-pom-crafter` skill.
|
|
85
|
+
|
|
86
|
+
## 7. Reference Examples
|
|
87
|
+
|
|
88
|
+
- `examples/checkout.feature` — full-featured input (Background, Outline, two
|
|
89
|
+
Examples blocks, tags, data tables).
|
|
90
|
+
- `examples/suite_report_sample.json` — trimmed JSON report shape emitted to
|
|
91
|
+
`reports/gherkin_suite_report.json`.
|
|
92
|
+
|
|
93
|
+
---
|
|
94
|
+
|
|
95
|
+
## 8. Boundary vs `lam-script-generator` and `clean-pom-crafter`
|
|
96
|
+
|
|
97
|
+
- **This skill** = deterministic `.feature` → test matrix/IR compiler (0 LLM
|
|
98
|
+
tokens). Prefer it whenever BDD specs already exist.
|
|
99
|
+
- **`lam-script-generator`** is the LLM-reasoning path for stories that are NOT
|
|
100
|
+
in Gherkin form; feed this skill's Universal IR to it when codification needs
|
|
101
|
+
story-level reasoning.
|
|
102
|
+
- **`clean-pom-crafter`** receives the matrix/IR from either path and owns all
|
|
103
|
+
Page Class + locator decisions — this skill never emits raw selectors.
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
@checkout @smoke
|
|
2
|
+
Feature: Checkout Flow
|
|
3
|
+
Customers buy items and receive an order confirmation.
|
|
4
|
+
|
|
5
|
+
Background:
|
|
6
|
+
Given the user is logged in as "member"
|
|
7
|
+
|
|
8
|
+
@happy
|
|
9
|
+
Scenario: Pay with credit card
|
|
10
|
+
When the user navigates to "https://shop.example.com/checkout"
|
|
11
|
+
And the user fills in "4111-1111-1111-1111" into "Card Number"
|
|
12
|
+
And the user clicks the "Pay Now" button
|
|
13
|
+
Then the user should see "Order Confirmed"
|
|
14
|
+
And the url should contain "/confirmation"
|
|
15
|
+
|
|
16
|
+
@data-driven
|
|
17
|
+
Scenario Outline: Apply voucher <voucher>
|
|
18
|
+
When the user enters "<code>" into "Voucher Input"
|
|
19
|
+
And the user clicks the "Apply" button
|
|
20
|
+
Then the total should show "<total>"
|
|
21
|
+
|
|
22
|
+
@valid
|
|
23
|
+
Examples:
|
|
24
|
+
| voucher | code | total |
|
|
25
|
+
| percent | SAVE10 | 90 |
|
|
26
|
+
| fixed | SHIP0 | 100 |
|
|
27
|
+
|
|
28
|
+
@edge
|
|
29
|
+
Examples: Boundary rows
|
|
30
|
+
| voucher | code | total |
|
|
31
|
+
| expired | OLD1 | 200 |
|
|
32
|
+
|
|
33
|
+
Rule: Guest checkout restrictions
|
|
34
|
+
|
|
35
|
+
@negative
|
|
36
|
+
Scenario: Invalid card is rejected
|
|
37
|
+
When the user fills in "4000-0000-0000-0002" into "Card Number"
|
|
38
|
+
And the user clicks the "Pay Now" button
|
|
39
|
+
Then the user should see "Payment declined"
|