@teevee98/synapse-hub 4.1.4 → 4.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/rules/git-push-protocol.md +2 -2
- package/.agents/skills/01-web-automation/auth-session-and-cookie-manager/SKILL.md +52 -3
- package/.agents/skills/01-web-automation/bats-testing-patterns/SKILL.md +55 -5
- package/.agents/skills/01-web-automation/clean-pom-crafter/SKILL.md +62 -67
- package/.agents/skills/01-web-automation/e2e-testing-patterns/SKILL.md +54 -4
- package/.agents/skills/01-web-automation/katalon-cli-runner/SKILL.md +62 -12
- package/.agents/skills/01-web-automation/lam-script-generator/SKILL.md +55 -8
- package/.agents/skills/01-web-automation/mobile-appium-som/SKILL.md +60 -12
- package/.agents/skills/01-web-automation/mobile-appium-som/examples/BaseScreen.java +2 -2
- package/.agents/skills/01-web-automation/mobile-appium-som/examples/LoginScreen.java +2 -2
- package/.agents/skills/01-web-automation/mobile-appium-som/examples/TC_MobileLogin.java +2 -2
- package/.agents/skills/01-web-automation/playwright-live-interaction/SKILL.md +54 -91
- package/.agents/skills/01-web-automation/playwright-webapp-testing/SKILL.md +59 -11
- package/.agents/skills/01-web-automation/web-agent-gym-benchmarking/SKILL.md +50 -4
- package/.agents/skills/02-test-design/architecture-decision-records/SKILL.md +55 -9
- package/.agents/skills/02-test-design/gherkin-to-suite/SKILL.md +55 -9
- package/.agents/skills/02-test-design/rbt-manual-testing/SKILL.md +55 -8
- package/.agents/skills/02-test-design/review-spec/SKILL.md +50 -6
- package/.agents/skills/02-test-design/tot-exploratory-testing/SKILL.md +50 -4
- package/.agents/skills/03-api-and-contracts/api-network-mocking-interception/SKILL.md +56 -5
- package/.agents/skills/03-api-and-contracts/api-spec-to-e2e-matrix/SKILL.md +53 -7
- package/.agents/skills/03-api-and-contracts/consumer-driven-contract-testing/SKILL.md +86 -36
- package/.agents/skills/04-performance-and-load/backend-load-and-stress-patterns/SKILL.md +54 -6
- package/.agents/skills/04-performance-and-load/lean-impact-scoreboard/SKILL.md +50 -6
- package/.agents/skills/04-performance-and-load/web-vitals-and-performance-profiler/SKILL.md +50 -3
- package/.agents/skills/05-security-and-compliance/security-dast-sentinel/SKILL.md +63 -14
- package/.agents/skills/05-security-and-compliance/skillspector-security-guard/SKILL.md +51 -5
- package/.agents/skills/05-security-and-compliance/wcag-audit-patterns/SKILL.md +54 -4
- package/.agents/skills/06-healing-and-resilience/agentless-qa-repair/SKILL.md +51 -5
- package/.agents/skills/06-healing-and-resilience/antigravity-test-sentinel/SKILL.md +53 -6
- package/.agents/skills/06-healing-and-resilience/failure-autopsy-and-healing/SKILL.md +54 -8
- package/.agents/skills/06-healing-and-resilience/flaky-test-sentinel-and-quarantine/SKILL.md +60 -24
- package/.agents/skills/06-healing-and-resilience/self-healing-locator/SKILL.md +52 -6
- package/.agents/skills/06-healing-and-resilience/systematic-debugging/SKILL.md +56 -10
- package/.agents/skills/06-healing-and-resilience/test-driven-development/SKILL.md +58 -12
- package/.agents/skills/06-healing-and-resilience/verification-before-completion/SKILL.md +58 -12
- package/.agents/skills/06-healing-and-resilience/visual-regression-sentinel/SKILL.md +53 -5
- package/.agents/skills/07-data-and-state/db-state-verifier-and-seeder/SKILL.md +56 -5
- package/.agents/skills/07-data-and-state/file-and-report-validator/SKILL.md +53 -4
- package/.agents/skills/07-data-and-state/synthetic-test-data-generator/SKILL.md +58 -5
- package/.agents/skills/08-code-quality/code-refactoring-refactor-clean/SKILL.md +57 -11
- package/.agents/skills/08-code-quality/graphify-impact-analyzer/SKILL.md +83 -34
- package/.agents/skills/08-code-quality/graphify-impact-analyzer/examples/dependency_blast_radius.json +3 -4
- package/.agents/skills/08-code-quality/lean-code-ladder/SKILL.md +55 -10
- package/.agents/skills/08-code-quality/repo-quality-audit/SKILL.md +49 -6
- package/.agents/skills/08-code-quality/senior-code-review/SKILL.md +52 -7
- package/.agents/skills/08-code-quality/surgical-coding-principles/SKILL.md +55 -10
- package/.agents/skills/08-code-quality/technical-debt-harvester/SKILL.md +51 -5
- package/.agents/skills/09-ci-and-devops/github-actions-templates/SKILL.md +54 -17
- package/.agents/skills/09-ci-and-devops/github-actions-templates/examples/matrix_test_pipeline.yml +2 -1
- package/.agents/skills/10-agent-orchestration/actor-qa-swarm/SKILL.md +50 -4
- package/.agents/skills/10-agent-orchestration/agent-orchestration-multi-agent-optimize/SKILL.md +53 -7
- package/.agents/skills/10-agent-orchestration/event-stream-qa-orchestrator/SKILL.md +50 -4
- package/.agents/skills/10-agent-orchestration/persistent-planning-files/SKILL.md +58 -13
- package/.agents/skills/10-agent-orchestration/qa-memory-os/SKILL.md +51 -5
- package/.agents/skills/11-ai-and-prompting/meta-prompt-crafter/SKILL.md +57 -10
- package/.agents/skills/11-ai-and-prompting/mipro-prompt-compiler/SKILL.md +50 -4
- package/.agents/skills/11-ai-and-prompting/qa-aci-interface/SKILL.md +50 -4
- package/.agents/skills/11-ai-and-prompting/token-and-context-optimizer/SKILL.md +55 -9
- package/.cursor/rules/synapse-qa.mdc +20 -7
- package/.cursorrules +19 -5
- package/.windsurfrules +18 -3
- package/AGENTS.md +51 -59
- package/CLAUDE.md +38 -24
- package/LICENSE +5 -5
- package/NOTICE +1 -1
- package/README.en.md +4 -4
- package/README.md +4 -4
- package/agentskills.json +1 -1
- package/docs/ARCHITECTURE.en.md +2 -2
- package/docs/ARCHITECTURE.md +11 -2
- package/docs/CLIENT_SUPPORT_MATRIX.md +44 -0
- package/docs/CLI_REFERENCE.en.md +6 -4
- package/docs/CLI_REFERENCE.md +1 -1
- package/docs/CONTRIBUTING.en.md +1 -1
- package/docs/CONTRIBUTING.md +2 -2
- package/docs/SIDE_EFFECTING_TOOLS_INVENTORY.md +233 -0
- package/docs/TOOL_CATALOG.md +6 -2
- package/docs/research/agent_skill_training_analysis.md +1 -1
- package/package.json +1 -1
- package/plugin.json +1 -1
- package/skills_manifest.json +320 -55
- package/synapse.config.json +1 -1
- package/templates/ai_bridges/CLAUDE.md +29 -26
- package/templates/ai_bridges/copilot-instructions.md +14 -5
- package/templates/ai_bridges/cursor_synapse_qa.mdc +16 -7
- package/templates/ai_bridges/windsurfrules +14 -3
- package/templates/ui/index.html +2 -2
- package/tools/TOOLS_RUNBOOK.md +1 -1
- package/tools/TOOL_CATALOG.json +63 -3
- package/tools/__init__.py +1 -1
- package/tools/api_fuzzer.py +447 -132
- package/tools/author_integrity.py +121 -11
- package/tools/benchmark_swarm_context.py +130 -22
- package/tools/break_ui_generator.py +233 -0
- package/tools/browser_helpers.js +2 -2
- package/tools/check_version_sync.py +18 -50
- package/tools/cli.py +110 -19
- package/tools/core/bm25_retriever.py +218 -0
- package/tools/core/i18n.py +5 -5
- package/tools/core/provider_usage_adapter.py +383 -0
- package/tools/core/token_estimator.py +81 -15
- package/tools/count_tokens.py +7 -2
- package/tools/coverage_reporter.py +245 -0
- package/tools/dashboard_generator.py +8 -8
- package/tools/db_sandbox.py +290 -9
- package/tools/distill_log.py +1 -1
- package/tools/distill_skill.py +206 -0
- package/tools/git_pre_commit_guard.py +202 -1
- package/tools/humanize_linter.py +3 -0
- package/tools/mcp_docx_server.py +1 -1
- package/tools/mcp_excel_server.py +1 -1
- package/tools/mcp_server.py +104 -16
- package/tools/micro_mutator.py +114 -15
- package/tools/pentest_scope_guard.py +674 -58
- package/tools/perf/benchmark_perf_runner.py +42 -0
- package/tools/project_state.py +29 -5
- package/tools/reversible_sandbox.py +58 -13
- package/tools/rules_router.py +473 -60
- package/tools/security_scanner.py +80 -0
- package/tools/senior_code_review.py +14 -0
- package/tools/skill_canonical_digest.py +302 -0
- package/tools/skill_profiles.py +77 -7
- package/tools/slide_deck_exporter.py +477 -0
- package/tools/sync_agent_rules.py +187 -61
- package/tools/sync_skills_manifest.py +166 -0
- package/tools/task_memory_manager.py +275 -0
- package/tools/ui_server.py +3 -1
- package/tools/update_sentinel.py +3 -3
- package/tools/validate_host_interop.py +485 -0
- package/tools/validate_skills_schema.py +575 -0
- package/tools/verify_all.py +668 -44
- package/tools/welcome_page.py +2 -2
|
@@ -1,16 +1,45 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: web-agent-gym-benchmarking
|
|
3
3
|
description: "Empirical benchmark harness evaluating web agents and Page Classes against standardized enterprise tasks with headless <200ms latency budgets and ground-truth DB state verification. Use when benchmarking web agents or certifying Page Classes."
|
|
4
|
+
category: test_automation_and_execution
|
|
5
|
+
version: "1.0.0"
|
|
6
|
+
policies:
|
|
7
|
+
network: localhost-only
|
|
8
|
+
shell: workspace-sandbox
|
|
9
|
+
filesystem: workspace-sandbox
|
|
4
10
|
---
|
|
5
|
-
|
|
6
11
|
# Web Agent Gym Benchmarking Skill — Empirical Interaction & PageClass Certification
|
|
7
12
|
|
|
13
|
+
> **Author Attribution & MIT License Guard:** Synapse Hub is open-source software created by **Thế Vị (@TeeVee98)** (<https://github.com/TeeVee98/Synapse-Hub>) under the MIT License.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Trigger & Activation
|
|
18
|
+
|
|
19
|
+
- **Trigger Clause:** Empirical benchmark harness evaluating web agents and Page Classes against standardized enterprise tasks with headless <200ms latency budgets and ground-truth DB state verification. Use when benchmarking web agents or certifying Page Classes.
|
|
20
|
+
- **Activation Scope:** Domain workflows for `01-web-automation` involving `web-agent-gym-benchmarking`.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Non-Goals & Boundary Constraints
|
|
25
|
+
|
|
26
|
+
- **What NOT to do:**
|
|
27
|
+
- Do NOT benchmark unconstrained web agents on live production environments without safety sandboxes.
|
|
28
|
+
- Do NOT report synthetic benchmark metrics without recording task provenance and execution metadata.
|
|
29
|
+
- Do NOT alter baseline benchmark scorecards without empirical run data.
|
|
30
|
+
- **Sibling Skill Boundaries:**
|
|
31
|
+
- lean-impact-scoreboard displays consolidated scorecards; web-agent-gym-benchmarking executes the live navigation evaluation benchmarks.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 3. Ordered Execution Workflow
|
|
36
|
+
|
|
8
37
|
> **Paradigm Attribution:** Inspired by **ServiceNow Research BrowserGym & CMU WebArena**.
|
|
9
38
|
> **Core Principle:** Move beyond static assertions and text-only tool tests. Measure real agent navigation efficiency, enforce < 200ms headless step latency, and confirm that UI actions trigger ground-truth database mutations (eliminating fake passes).
|
|
10
39
|
|
|
11
40
|
---
|
|
12
41
|
|
|
13
|
-
|
|
42
|
+
### The Benchmark Architecture
|
|
14
43
|
|
|
15
44
|
```mermaid
|
|
16
45
|
flowchart TD
|
|
@@ -36,10 +65,11 @@ flowchart TD
|
|
|
36
65
|
|
|
37
66
|
---
|
|
38
67
|
|
|
39
|
-
|
|
68
|
+
### CLI Tool Usage
|
|
40
69
|
|
|
41
70
|
Autonomous commands via `tools/browsergym_eval.py`:
|
|
42
71
|
|
|
72
|
+
<!-- classification: illustrative -->
|
|
43
73
|
```bash
|
|
44
74
|
# 1. Run full benchmark suite in headless mode
|
|
45
75
|
python tools/browsergym_eval.py run-suite \
|
|
@@ -55,8 +85,24 @@ python tools/browsergym_eval.py evaluate-pageclass \
|
|
|
55
85
|
|
|
56
86
|
---
|
|
57
87
|
|
|
58
|
-
|
|
88
|
+
### Strict Operating Invariants
|
|
59
89
|
|
|
60
90
|
- **Headless Latency Ceiling:** Average action step latency must remain below **200ms**.
|
|
61
91
|
- **No Static Sleep:** Any Page Class with `sleep()` calls is automatically flagged as `REJECTED`.
|
|
62
92
|
- **Clean POM:** Strictly prohibits hardcoded selectors outside Page Classes.
|
|
93
|
+
|
|
94
|
+
---
|
|
95
|
+
|
|
96
|
+
## 4. Verification Points & DoD
|
|
97
|
+
|
|
98
|
+
1. Benchmark tasks execute in isolated sandbox browser environments.
|
|
99
|
+
2. Run provenance (timestamp, model, task ID, steps taken, success rate) is recorded.
|
|
100
|
+
3. Output benchmark metrics parse into valid structured JSON.
|
|
101
|
+
|
|
102
|
+
---
|
|
103
|
+
|
|
104
|
+
## 5. Handoff Protocol
|
|
105
|
+
|
|
106
|
+
- **Deliverables:** Benchmark execution logs, trajectory recordings, and score summary reports.
|
|
107
|
+
- **Next Subagent:** perf-engineer / orchestrator
|
|
108
|
+
- **Output Format:** Clean POM code, structured JSON telemetry, or validated markdown handoff.
|
|
@@ -1,15 +1,44 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: architecture-decision-records
|
|
3
3
|
description: "Record, structure, and maintain technical Architecture Decision Records (ADRs). Use when documenting significant architecture decisions as ADRs."
|
|
4
|
+
category: architecture_and_analysis
|
|
5
|
+
version: "2.0.0"
|
|
6
|
+
policies:
|
|
7
|
+
network: none
|
|
8
|
+
shell: none
|
|
9
|
+
filesystem: workspace-sandbox
|
|
4
10
|
---
|
|
5
|
-
|
|
6
11
|
# Architecture Decision Records (ADR) Skill
|
|
7
12
|
|
|
13
|
+
> **Author Attribution & MIT License Guard:** Synapse Hub is open-source software created by **Thế Vị (@TeeVee98)** (<https://github.com/TeeVee98/Synapse-Hub>) under the MIT License.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Trigger & Activation
|
|
18
|
+
|
|
19
|
+
- **Trigger Clause:** Record, structure, and maintain technical Architecture Decision Records (ADRs). Use when documenting significant architecture decisions as ADRs.
|
|
20
|
+
- **Activation Scope:** Domain workflows for `02-test-design` involving `architecture-decision-records`.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Non-Goals & Boundary Constraints
|
|
25
|
+
|
|
26
|
+
- **What NOT to do:**
|
|
27
|
+
- Do NOT use ADRs for trivial code refactoring or minor variable renames; ADRs are for architectural decisions.
|
|
28
|
+
- Do NOT leave ADRs in ambiguous states; follow explicit lifecycle statuses (PROPOSED, ACCEPTED, DEPRECATED, SUPERSEDED).
|
|
29
|
+
- Do NOT omit negative consequences or tradeoffs in the decision record.
|
|
30
|
+
- **Sibling Skill Boundaries:**
|
|
31
|
+
- review-spec audits functional specifications; architecture-decision-records captures strategic technical and architectural choices.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 3. Ordered Execution Workflow
|
|
36
|
+
|
|
8
37
|
> **Target:** Standardize technical decision logs, architectural trade-offs, and design rationale across enterprise testing platforms.
|
|
9
38
|
|
|
10
39
|
---
|
|
11
40
|
|
|
12
|
-
|
|
41
|
+
### When to Use
|
|
13
42
|
|
|
14
43
|
- Documenting significant framework, toolchain, or database decisions.
|
|
15
44
|
- Recording architecture changes (e.g. Clean POM flat architecture vs BDD layer).
|
|
@@ -18,7 +47,7 @@ description: "Record, structure, and maintain technical Architecture Decision Re
|
|
|
18
47
|
|
|
19
48
|
---
|
|
20
49
|
|
|
21
|
-
|
|
50
|
+
### ADR Lifecycle State Machine
|
|
22
51
|
|
|
23
52
|
```mermaid
|
|
24
53
|
stateDiagram-v2
|
|
@@ -30,26 +59,27 @@ stateDiagram-v2
|
|
|
30
59
|
|
|
31
60
|
---
|
|
32
61
|
|
|
33
|
-
|
|
62
|
+
### Standard Lightweight ADR Format
|
|
34
63
|
|
|
35
64
|
Create decision records inside `docs/adr/ADR-XXXX-<title>.md`:
|
|
36
65
|
|
|
66
|
+
<!-- classification: illustrative -->
|
|
37
67
|
```markdown
|
|
38
68
|
# ADR-0001: Adopt Clean POM Flat Architecture
|
|
39
69
|
|
|
40
|
-
|
|
70
|
+
### Status
|
|
41
71
|
Accepted
|
|
42
72
|
|
|
43
|
-
|
|
73
|
+
### Context
|
|
44
74
|
E2E automated test suites suffered from high maintenance overhead, locator duplication, and flaky runs caused by deeply nested inheritance hierarchies and raw selectors scattered across test scripts.
|
|
45
75
|
|
|
46
|
-
|
|
76
|
+
### Decision
|
|
47
77
|
Enforce a 2-tier Clean Page Object Model:
|
|
48
78
|
1. All test scripts live in `tests/` and contain only business action calls.
|
|
49
79
|
2. All locators and DOM operations live in Page Classes inheriting directly from `BasePage`.
|
|
50
80
|
3. Raw XPaths and CSS selectors are strictly prohibited in test scripts.
|
|
51
81
|
|
|
52
|
-
|
|
82
|
+
### Consequences
|
|
53
83
|
- **Positive:** Zero locator duplication, modular self-healing, faster onboarding.
|
|
54
84
|
- **Negative:** Requires refactoring legacy tests to encapsulate DOM calls into Page Classes.
|
|
55
85
|
- **Compliance:** Enforced in CI via `tools/verify_all.py` (Gate 3).
|
|
@@ -57,8 +87,24 @@ Enforce a 2-tier Clean Page Object Model:
|
|
|
57
87
|
|
|
58
88
|
---
|
|
59
89
|
|
|
60
|
-
|
|
90
|
+
### Best Practices
|
|
61
91
|
|
|
62
92
|
- Keep ADRs concise (< 500 words per decision).
|
|
63
93
|
- Store ADRs as markdown versioned alongside code in `docs/adr/`.
|
|
64
94
|
- Never modify an accepted ADR to change history; author a new ADR that supersedes the previous one.
|
|
95
|
+
|
|
96
|
+
---
|
|
97
|
+
|
|
98
|
+
## 4. Verification Points & DoD
|
|
99
|
+
|
|
100
|
+
1. ADR file created in `docs/adr/` with standard lightweight format (Title, Status, Context, Decision, Consequences).
|
|
101
|
+
2. Status transitions adhere to ADR lifecycle state machine.
|
|
102
|
+
3. Consequences section explicitly lists tradeoffs and negative side effects.
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## 5. Handoff Protocol
|
|
107
|
+
|
|
108
|
+
- **Deliverables:** Structured ADR document (`docs/adr/NNNN-*.md`) and updated ADR index.
|
|
109
|
+
- **Next Subagent:** orchestrator
|
|
110
|
+
- **Output Format:** Clean POM code, structured JSON telemetry, or validated markdown handoff.
|
|
@@ -1,10 +1,39 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: gherkin-to-suite
|
|
3
3
|
description: "Compile Gherkin .feature files into an expanded test matrix, hub Universal IR and a proven-vs-owed coverage-gap report with zero LLM tokens. Use when the user supplies BDD .feature specs or wants a BDD suite audited before codification."
|
|
4
|
+
category: test_automation_and_execution
|
|
5
|
+
version: "1.0.0"
|
|
6
|
+
policies:
|
|
7
|
+
network: none
|
|
8
|
+
shell: none
|
|
9
|
+
filesystem: workspace-sandbox
|
|
4
10
|
---
|
|
5
|
-
|
|
6
11
|
# Gherkin to Suite Compiler — Natural-Language Test Ingestion (0-Token)
|
|
7
12
|
|
|
13
|
+
> **Author Attribution & MIT License Guard:** Synapse Hub is open-source software created by **Thế Vị (@TeeVee98)** (<https://github.com/TeeVee98/Synapse-Hub>) under the MIT License.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Trigger & Activation
|
|
18
|
+
|
|
19
|
+
- **Trigger Clause:** Compile Gherkin .feature files into an expanded test matrix, hub Universal IR and a proven-vs-owed coverage-gap report with zero LLM tokens. Use when the user supplies BDD .feature specs or wants a BDD suite audited before codification.
|
|
20
|
+
- **Activation Scope:** Domain workflows for `02-test-design` involving `gherkin-to-suite`.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Non-Goals & Boundary Constraints
|
|
25
|
+
|
|
26
|
+
- **What NOT to do:**
|
|
27
|
+
- Do NOT author implementation-coupled Gherkin steps with CSS/XPath selectors inside `.feature` files.
|
|
28
|
+
- Do NOT generate tests without reviewing acceptance criteria and boundary value matrices.
|
|
29
|
+
- Do NOT create ambiguous or untestable Given-When-Then statements.
|
|
30
|
+
- **Sibling Skill Boundaries:**
|
|
31
|
+
- review-spec audits user stories; gherkin-to-suite translates BDD specs into structured test matrices and script skeletons.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 3. Ordered Execution Workflow
|
|
36
|
+
|
|
8
37
|
This skill turns business-readable Gherkin `.feature` files into deterministic
|
|
9
38
|
test artifacts: an expanded test-case matrix, hub-native Universal IR for the
|
|
10
39
|
transpile pipeline, and a coverage-gap audit — all without a single LLM token.
|
|
@@ -16,7 +45,7 @@ transpile pipeline, and a coverage-gap audit — all without a single LLM token.
|
|
|
16
45
|
|
|
17
46
|
---
|
|
18
47
|
|
|
19
|
-
|
|
48
|
+
### When to Activate
|
|
20
49
|
|
|
21
50
|
- User supplies `.feature` / BDD specs and wants an executable test suite plan.
|
|
22
51
|
- Need to audit a BDD suite for missing assertions, negative paths, or thin
|
|
@@ -24,7 +53,7 @@ transpile pipeline, and a coverage-gap audit — all without a single LLM token.
|
|
|
24
53
|
- Need structured test data (JSON IR) to feed `omni_transpiler` / `universal_ir`
|
|
25
54
|
pipelines instead of hand-writing scripts.
|
|
26
55
|
|
|
27
|
-
|
|
56
|
+
### Pipeline
|
|
28
57
|
|
|
29
58
|
```mermaid
|
|
30
59
|
flowchart TD
|
|
@@ -36,8 +65,9 @@ flowchart TD
|
|
|
36
65
|
Gaps --> Out
|
|
37
66
|
```
|
|
38
67
|
|
|
39
|
-
|
|
68
|
+
### Commands
|
|
40
69
|
|
|
70
|
+
<!-- classification: illustrative -->
|
|
41
71
|
```bash
|
|
42
72
|
# Compile a single feature (human summary + Markdown plan to stdout)
|
|
43
73
|
python tools/gherkin_suite.py tests/bdd/checkout.feature
|
|
@@ -56,7 +86,7 @@ agent-qa gherkin tests/bdd/
|
|
|
56
86
|
Exit codes: `0` = compiled (warnings allowed) · `1` = parse/authoring errors
|
|
57
87
|
(report and plan are still emitted so the suite can be iterated on).
|
|
58
88
|
|
|
59
|
-
|
|
89
|
+
### Output Contract
|
|
60
90
|
|
|
61
91
|
| Artifact | Location | Content |
|
|
62
92
|
| :--- | :--- | :--- |
|
|
@@ -64,7 +94,7 @@ Exit codes: `0` = compiled (warnings allowed) · `1` = parse/authoring errors
|
|
|
64
94
|
| Markdown plan | stdout | Background, per-scenario steps with mapped actions, Examples tables, gap section |
|
|
65
95
|
| Universal IR | `--emit-ir` stdout | `TestCaseIR` dicts (id, title, tags, `StepIR` steps) consumable by `universal_ir` / `omni_*` tooling |
|
|
66
96
|
|
|
67
|
-
|
|
97
|
+
### Coverage-Gap Rules (proven vs owed)
|
|
68
98
|
|
|
69
99
|
| Rule | Trigger | Suggested Remedy |
|
|
70
100
|
| :--- | :--- | :--- |
|
|
@@ -73,7 +103,7 @@ Exit codes: `0` = compiled (warnings allowed) · `1` = parse/authoring errors
|
|
|
73
103
|
| `G3-THIN_EXAMPLES` | Examples table has < 2 rows | Add boundary + negative rows (empty, max length, special chars) |
|
|
74
104
|
| *(error)* Unresolved `<param>` | Example header does not cover a used parameter | Fix the header typo or remove the parameter |
|
|
75
105
|
|
|
76
|
-
|
|
106
|
+
### Limitations (Honest Scope)
|
|
77
107
|
|
|
78
108
|
- English Gherkin keywords only; `# language:` headers are ignored.
|
|
79
109
|
- NL→IR mapping is deterministic keyword heuristics — best-effort, not an LLM
|
|
@@ -83,7 +113,7 @@ Exit codes: `0` = compiled (warnings allowed) · `1` = parse/authoring errors
|
|
|
83
113
|
the job of the target framework (Playwright/Katalon) via `omni_transpiler`
|
|
84
114
|
and the `clean-pom-crafter` skill.
|
|
85
115
|
|
|
86
|
-
|
|
116
|
+
### Reference Examples
|
|
87
117
|
|
|
88
118
|
- `examples/checkout.feature` — full-featured input (Background, Outline, two
|
|
89
119
|
Examples blocks, tags, data tables).
|
|
@@ -92,7 +122,7 @@ Exit codes: `0` = compiled (warnings allowed) · `1` = parse/authoring errors
|
|
|
92
122
|
|
|
93
123
|
---
|
|
94
124
|
|
|
95
|
-
|
|
125
|
+
### Boundary vs `lam-script-generator` and `clean-pom-crafter`
|
|
96
126
|
|
|
97
127
|
- **This skill** = deterministic `.feature` → test matrix/IR compiler (0 LLM
|
|
98
128
|
tokens). Prefer it whenever BDD specs already exist.
|
|
@@ -101,3 +131,19 @@ Exit codes: `0` = compiled (warnings allowed) · `1` = parse/authoring errors
|
|
|
101
131
|
story-level reasoning.
|
|
102
132
|
- **`clean-pom-crafter`** receives the matrix/IR from either path and owns all
|
|
103
133
|
Page Class + locator decisions — this skill never emits raw selectors.
|
|
134
|
+
|
|
135
|
+
---
|
|
136
|
+
|
|
137
|
+
## 4. Verification Points & DoD
|
|
138
|
+
|
|
139
|
+
1. Gherkin scenarios follow declarative business language (zero technical locators).
|
|
140
|
+
2. Test matrix covers positive, negative, and edge-case boundary scenarios.
|
|
141
|
+
3. Generated test skeletons adhere to Clean POM method mapping.
|
|
142
|
+
|
|
143
|
+
---
|
|
144
|
+
|
|
145
|
+
## 5. Handoff Protocol
|
|
146
|
+
|
|
147
|
+
- **Deliverables:** BDD feature files (`.feature`), test scenario matrices, and Clean POM script skeletons.
|
|
148
|
+
- **Next Subagent:** test-designer / pom-crafter
|
|
149
|
+
- **Output Format:** Clean POM code, structured JSON telemetry, or validated markdown handoff.
|
|
@@ -1,11 +1,40 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: rbt-manual-testing
|
|
3
3
|
description: "Comprehensive Risk-Based Testing (AI-RBT) 6-stage manual test case generation, Ambiguity Q&A analysis, Traceability Matrix, Field-Level Validation Checklist (15 input field types), Non-Functional scenarios (Double-Submit, Race Conditions), and Excel export formatting. Use when generating risk-based manual test cases and exploratory charters."
|
|
4
|
+
category: test_automation_and_execution
|
|
5
|
+
version: "2.0.0"
|
|
6
|
+
policies:
|
|
7
|
+
network: none
|
|
8
|
+
shell: none
|
|
9
|
+
filesystem: workspace-sandbox
|
|
4
10
|
---
|
|
5
|
-
|
|
6
11
|
# RBT Manual Testing (AI-Driven Risk-Based Testing Engine)
|
|
7
12
|
|
|
8
|
-
|
|
13
|
+
> **Author Attribution & MIT License Guard:** Synapse Hub is open-source software created by **Thế Vị (@TeeVee98)** (<https://github.com/TeeVee98/Synapse-Hub>) under the MIT License.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Trigger & Activation
|
|
18
|
+
|
|
19
|
+
- **Trigger Clause:** Comprehensive Risk-Based Testing (AI-RBT) 6-stage manual test case generation, Ambiguity Q&A analysis, Traceability Matrix, Field-Level Validation Checklist (15 input field types), Non-Functional scenarios (Double-Submit, Race Conditions), and Excel export formatting. Use when generating risk-based manual test cases and exploratory charters.
|
|
20
|
+
- **Activation Scope:** Domain workflows for `02-test-design` involving `rbt-manual-testing`.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Non-Goals & Boundary Constraints
|
|
25
|
+
|
|
26
|
+
- **What NOT to do:**
|
|
27
|
+
- Do NOT assign high risk scores uniformly across all features; differentiate by business impact and likelihood.
|
|
28
|
+
- Do NOT generate manual test suites without explicit step-by-step instructions and expected results.
|
|
29
|
+
- Do NOT omit test data setup and prerequisite teardown procedures.
|
|
30
|
+
- **Sibling Skill Boundaries:**
|
|
31
|
+
- gherkin-to-suite compiles automated BDD suites; rbt-manual-testing designs risk-weighted manual test suites and Excel matrix exports.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 3. Ordered Execution Workflow
|
|
36
|
+
|
|
37
|
+
### Overview & Capabilities
|
|
9
38
|
|
|
10
39
|
**RBT Manual Testing** is an enterprise-grade skill for designing manual test suites using **Risk-Based Testing (RBT)** and coverage matrix analysis. It provides two operational modes:
|
|
11
40
|
|
|
@@ -21,7 +50,7 @@ description: "Comprehensive Risk-Based Testing (AI-RBT) 6-stage manual test case
|
|
|
21
50
|
|
|
22
51
|
---
|
|
23
52
|
|
|
24
|
-
|
|
53
|
+
### Mode Routing
|
|
25
54
|
|
|
26
55
|
- **Trigger QUICK Mode:**
|
|
27
56
|
- "quick test cases", "generate test cases for registration form", "generate test cases from requirement".
|
|
@@ -32,7 +61,7 @@ description: "Comprehensive Risk-Based Testing (AI-RBT) 6-stage manual test case
|
|
|
32
61
|
|
|
33
62
|
---
|
|
34
63
|
|
|
35
|
-
|
|
64
|
+
### Mode 1: QUICK Mode (Fast Single-Pass)
|
|
36
65
|
|
|
37
66
|
1. **Requirement Analysis:** Map Happy Paths, Negative Paths, Boundary Conditions, and Edge Cases.
|
|
38
67
|
2. **Test Design Techniques:**
|
|
@@ -45,7 +74,7 @@ description: "Comprehensive Risk-Based Testing (AI-RBT) 6-stage manual test case
|
|
|
45
74
|
|
|
46
75
|
---
|
|
47
76
|
|
|
48
|
-
|
|
77
|
+
### Mode 2: FULL RBT (6-Stage Sequential Pipeline)
|
|
49
78
|
|
|
50
79
|
```mermaid
|
|
51
80
|
flowchart TD
|
|
@@ -92,7 +121,7 @@ flowchart TD
|
|
|
92
121
|
|
|
93
122
|
---
|
|
94
123
|
|
|
95
|
-
|
|
124
|
+
### Field-Level Validation Checklist (15 Field Types)
|
|
96
125
|
|
|
97
126
|
Field-level validation rules cover positive, boundary/negative (BVA), and security vectors across 15 standard field types:
|
|
98
127
|
1. Text (Name, Title) • 2. Email • 3. Phone • 4. Password • 5. Number • 6. Date • 7. Date Range • 8. Dropdown • 9. Multi-Select • 10. Checkbox/Radio • 11. Textarea • 12. File Upload • 13. OTP/MFA • 14. Range Slider • 15. Rich Text Editor.
|
|
@@ -101,10 +130,11 @@ Field-level validation rules cover positive, boundary/negative (BVA), and securi
|
|
|
101
130
|
|
|
102
131
|
---
|
|
103
132
|
|
|
104
|
-
|
|
133
|
+
### Standard Table Output Specification
|
|
105
134
|
|
|
106
135
|
All generated test cases must adhere to the 12-column Markdown specification:
|
|
107
136
|
|
|
137
|
+
<!-- classification: illustrative -->
|
|
108
138
|
```markdown
|
|
109
139
|
| TC ID | Module | Risk Level | Test Title | Pre-Condition | Test Steps | Expected Result | Priority | Test Data | Automatable | Auto Type | Tags |
|
|
110
140
|
| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
|
|
@@ -118,10 +148,11 @@ All generated test cases must adhere to the 12-column Markdown specification:
|
|
|
118
148
|
|
|
119
149
|
---
|
|
120
150
|
|
|
121
|
-
|
|
151
|
+
### Export to Styled Excel (.xlsx)
|
|
122
152
|
|
|
123
153
|
Export Markdown test cases directly into styled Excel workbooks via the CLI tool:
|
|
124
154
|
|
|
155
|
+
<!-- classification: illustrative -->
|
|
125
156
|
```bash
|
|
126
157
|
# Convert Markdown table into styled Excel
|
|
127
158
|
synapse export-excel path/to/test_cases.md -o output.xlsx --sheet-name "Regression Suite"
|
|
@@ -135,3 +166,19 @@ Generated workbooks automatically include:
|
|
|
135
166
|
- Frozen header pane (`A2`) and enabled `AutoFilter`.
|
|
136
167
|
- Color-coded badges for Risk Level and Priority.
|
|
137
168
|
- Multi-line text wrapping and optimized column widths.
|
|
169
|
+
|
|
170
|
+
---
|
|
171
|
+
|
|
172
|
+
## 4. Verification Points & DoD
|
|
173
|
+
|
|
174
|
+
1. Every test case carries explicit Risk Priority Number (RPN = Impact x Likelihood).
|
|
175
|
+
2. Test steps include preconditions, concrete test data, and verifiable expected outcomes.
|
|
176
|
+
3. Exported test suite generates clean tabular data (Excel / CSV / Markdown).
|
|
177
|
+
|
|
178
|
+
---
|
|
179
|
+
|
|
180
|
+
## 5. Handoff Protocol
|
|
181
|
+
|
|
182
|
+
- **Deliverables:** Risk-based manual test suite documents and Excel-compatible test case matrices.
|
|
183
|
+
- **Next Subagent:** test-designer
|
|
184
|
+
- **Output Format:** Clean POM code, structured JSON telemetry, or validated markdown handoff.
|
|
@@ -3,31 +3,59 @@ name: review-spec
|
|
|
3
3
|
description: "Stress-test specifications, user stories, and BDD feature files with zero-context rigor. Use when reviewing requirements, user stories, or acceptance criteria before code generation."
|
|
4
4
|
category: architecture_and_analysis
|
|
5
5
|
version: "1.0.0"
|
|
6
|
+
policies:
|
|
7
|
+
network: none
|
|
8
|
+
shell: none
|
|
9
|
+
filesystem: read-only
|
|
6
10
|
---
|
|
7
|
-
|
|
8
11
|
# Review Spec — Zero-Context Specification Stress-Tester
|
|
9
12
|
|
|
13
|
+
> **Author Attribution & MIT License Guard:** Synapse Hub is open-source software created by **Thế Vị (@TeeVee98)** (<https://github.com/TeeVee98/Synapse-Hub>) under the MIT License.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Trigger & Activation
|
|
18
|
+
|
|
19
|
+
- **Trigger Clause:** Stress-test specifications, user stories, and BDD feature files with zero-context rigor. Use when reviewing requirements, user stories, or acceptance criteria before code generation.
|
|
20
|
+
- **Activation Scope:** Domain workflows for `02-test-design` involving `review-spec`.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Non-Goals & Boundary Constraints
|
|
25
|
+
|
|
26
|
+
- **What NOT to do:**
|
|
27
|
+
- Do NOT accept vague, contradictory, or untestable acceptance criteria without flagging defects.
|
|
28
|
+
- Do NOT invent speculative business logic outside stated requirements.
|
|
29
|
+
- Do NOT skip boundary condition and error flow analysis.
|
|
30
|
+
- **Sibling Skill Boundaries:**
|
|
31
|
+
- architecture-decision-records records architectural choices; review-spec inspects requirement specs against the 5 Goldfish verification vectors.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 3. Ordered Execution Workflow
|
|
36
|
+
|
|
10
37
|
Review test specifications, User Stories, and BDD `.feature` files with stateless, zero-context rigor (Elephant-Goldfish Model) before generating code.
|
|
11
38
|
|
|
12
39
|
Order of evaluation: **System Boundaries -> Dynamic Data & Teardown -> Assertion Rigor -> Boundary & Negative Cases -> Concrete Implementation Targets**.
|
|
13
40
|
|
|
14
41
|
---
|
|
15
42
|
|
|
16
|
-
|
|
43
|
+
### When to Use
|
|
17
44
|
|
|
18
45
|
- Triggered automatically before `pom-crafter` synthesizes Page Classes or test scripts from natural-language specs.
|
|
19
46
|
- Triggered when the user asks to review, audit, or refine requirements, user stories, or acceptance criteria.
|
|
20
47
|
- Integrated CLI invocation:
|
|
21
|
-
|
|
48
|
+
<!-- classification: illustrative -->
|
|
49
|
+
```bash
|
|
22
50
|
python tools/goldfish_review.py path/to/spec.md
|
|
23
51
|
python tools/goldfish_review.py path/to/feature.feature --json
|
|
24
52
|
# or via CLI shim:
|
|
25
53
|
agent-qa review-spec path/to/spec.md
|
|
26
|
-
|
|
54
|
+
```
|
|
27
55
|
|
|
28
56
|
---
|
|
29
57
|
|
|
30
|
-
|
|
58
|
+
### Five Verification Vectors (Goldfish Protocol)
|
|
31
59
|
|
|
32
60
|
1. **Comprehension & System Boundaries (20%):**
|
|
33
61
|
- Explicit Actor/Role definition.
|
|
@@ -52,9 +80,25 @@ Order of evaluation: **System Boundaries -> Dynamic Data & Teardown -> Assertion
|
|
|
52
80
|
|
|
53
81
|
---
|
|
54
82
|
|
|
55
|
-
|
|
83
|
+
### Autonomous Execution Workflow
|
|
56
84
|
|
|
57
85
|
1. Ingest specification file (`.md`, `.feature`, or story text).
|
|
58
86
|
2. Run `python tools/goldfish_review.py <spec_file>`.
|
|
59
87
|
3. If review score is `< 80/100` or blocking gaps are found, report specific missing vectors to the user or orchestrator.
|
|
60
88
|
4. Once verified ($\ge 80/100$), proceed with Clean POM test script codification.
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## 4. Verification Points & DoD
|
|
93
|
+
|
|
94
|
+
1. Requirements audited across all 5 vectors (Ambiguity, Testability, Completeness, Consistency, Boundary).
|
|
95
|
+
2. Ambiguities documented with specific clarifying questions and risk ratings.
|
|
96
|
+
3. Specification readiness verdict issued (APPROVED, BLOCKED, or REVISE).
|
|
97
|
+
|
|
98
|
+
---
|
|
99
|
+
|
|
100
|
+
## 5. Handoff Protocol
|
|
101
|
+
|
|
102
|
+
- **Deliverables:** Requirement review audit report and ambiguity remediation ledger.
|
|
103
|
+
- **Next Subagent:** test-designer / orchestrator
|
|
104
|
+
- **Output Format:** Clean POM code, structured JSON telemetry, or validated markdown handoff.
|
|
@@ -1,16 +1,45 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: tot-exploratory-testing
|
|
3
3
|
description: "Non-linear Tree/Graph-of-Thoughts exploratory testing engine exploring Happy Path, Boundary/Race, and Negative/Chaos branches with Beam Search risk scoring and multi-hypothesis RCA. Use when decomposing complex user stories into exploratory test trees or triaging complex failures."
|
|
4
|
+
category: test_automation_and_execution
|
|
5
|
+
version: "1.0.0"
|
|
6
|
+
policies:
|
|
7
|
+
network: none
|
|
8
|
+
shell: none
|
|
9
|
+
filesystem: read-only
|
|
4
10
|
---
|
|
5
|
-
|
|
6
11
|
# Tree-of-Thoughts Exploratory Testing Skill — Deliberative QA Reasoning
|
|
7
12
|
|
|
13
|
+
> **Author Attribution & MIT License Guard:** Synapse Hub is open-source software created by **Thế Vị (@TeeVee98)** (<https://github.com/TeeVee98/Synapse-Hub>) under the MIT License.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Trigger & Activation
|
|
18
|
+
|
|
19
|
+
- **Trigger Clause:** Non-linear Tree/Graph-of-Thoughts exploratory testing engine exploring Happy Path, Boundary/Race, and Negative/Chaos branches with Beam Search risk scoring and multi-hypothesis RCA. Use when decomposing complex user stories into exploratory test trees or triaging complex failures.
|
|
20
|
+
- **Activation Scope:** Domain workflows for `02-test-design` involving `tot-exploratory-testing`.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Non-Goals & Boundary Constraints
|
|
25
|
+
|
|
26
|
+
- **What NOT to do:**
|
|
27
|
+
- Do NOT perform unstructured ad-hoc clicking without following explicit exploratory charters.
|
|
28
|
+
- Do NOT explore without logging decision branches and state transitions.
|
|
29
|
+
- Do NOT declare exploration complete without filing defect autopsies for observed failures.
|
|
30
|
+
- **Sibling Skill Boundaries:**
|
|
31
|
+
- rbt-manual-testing handles scripted manual test matrices; tot-exploratory-testing handles heuristic 3-branch Tree-of-Thought exploratory probing.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 3. Ordered Execution Workflow
|
|
36
|
+
|
|
8
37
|
> **Paradigm Attribution:** Inspired by **Princeton NLP Tree-of-Thoughts (ToT) & ETH Zürich Graph-of-Thoughts (GoT)**.
|
|
9
38
|
> **Core Principle:** Overcome linear single-track test generation. Decompose critical enterprise workflows into a 3-branch deliberation tree (Happy Path, Boundary/Race, Chaos/Fault), score intermediate thought nodes with Beam Search, and diagnose root causes via multi-hypothesis evaluation.
|
|
10
39
|
|
|
11
40
|
---
|
|
12
41
|
|
|
13
|
-
|
|
42
|
+
### The Deliberative 3-Branch Reasoning Tree
|
|
14
43
|
|
|
15
44
|
```mermaid
|
|
16
45
|
flowchart TD
|
|
@@ -32,10 +61,11 @@ flowchart TD
|
|
|
32
61
|
|
|
33
62
|
---
|
|
34
63
|
|
|
35
|
-
|
|
64
|
+
### CLI Tool Usage
|
|
36
65
|
|
|
37
66
|
Autonomous commands via `tools/tot_exploratory_tester.py`:
|
|
38
67
|
|
|
68
|
+
<!-- classification: illustrative -->
|
|
39
69
|
```bash
|
|
40
70
|
# 1. Expand User Story into Tree of Thoughts
|
|
41
71
|
python tools/tot_exploratory_tester.py expand-story \
|
|
@@ -52,8 +82,24 @@ python tools/tot_exploratory_tester.py rca-hypotheses \
|
|
|
52
82
|
|
|
53
83
|
---
|
|
54
84
|
|
|
55
|
-
|
|
85
|
+
### Strict Operating Invariants
|
|
56
86
|
|
|
57
87
|
- **Critical Flow Prioritization:** Focus multi-branch deliberation on high-risk domains (Financial Checkout, Authentication, Data Mutation).
|
|
58
88
|
- **Clean POM Output:** Test scenarios derived from thought trees must map directly into Page Classes extending `BasePage`.
|
|
59
89
|
- **Zero Hallucination:** Hypotheses must be grounded directly in error log evidence and stacktraces.
|
|
90
|
+
|
|
91
|
+
---
|
|
92
|
+
|
|
93
|
+
## 4. Verification Points & DoD
|
|
94
|
+
|
|
95
|
+
1. Exploration branches follow structured Tree-of-Thought heuristics (Happy Path, Boundary Chaos, Edge Flow).
|
|
96
|
+
2. Observed defects logged with reproduction steps, network traces, and screenshots.
|
|
97
|
+
3. Exploration summary report details covered user journeys and risk hotspots.
|
|
98
|
+
|
|
99
|
+
---
|
|
100
|
+
|
|
101
|
+
## 5. Handoff Protocol
|
|
102
|
+
|
|
103
|
+
- **Deliverables:** Exploratory testing charters, decision branch logs, and defect autopsy reports.
|
|
104
|
+
- **Next Subagent:** test-designer / failure-analyst
|
|
105
|
+
- **Output Format:** Clean POM code, structured JSON telemetry, or validated markdown handoff.
|