@teevee98/synapse-hub 4.2.0 → 4.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/skills/01-web-automation/auth-session-and-cookie-manager/SKILL.md +46 -2
- package/.agents/skills/01-web-automation/bats-testing-patterns/SKILL.md +49 -4
- package/.agents/skills/01-web-automation/clean-pom-crafter/SKILL.md +56 -66
- package/.agents/skills/01-web-automation/e2e-testing-patterns/SKILL.md +48 -3
- package/.agents/skills/01-web-automation/katalon-cli-runner/SKILL.md +56 -11
- package/.agents/skills/01-web-automation/lam-script-generator/SKILL.md +49 -7
- package/.agents/skills/01-web-automation/mobile-appium-som/SKILL.md +54 -11
- package/.agents/skills/01-web-automation/playwright-live-interaction/SKILL.md +48 -90
- package/.agents/skills/01-web-automation/playwright-webapp-testing/SKILL.md +53 -10
- package/.agents/skills/01-web-automation/web-agent-gym-benchmarking/SKILL.md +44 -3
- package/.agents/skills/02-test-design/architecture-decision-records/SKILL.md +49 -8
- package/.agents/skills/02-test-design/gherkin-to-suite/SKILL.md +49 -8
- package/.agents/skills/02-test-design/rbt-manual-testing/SKILL.md +49 -7
- package/.agents/skills/02-test-design/review-spec/SKILL.md +46 -5
- package/.agents/skills/02-test-design/tot-exploratory-testing/SKILL.md +44 -3
- package/.agents/skills/03-api-and-contracts/api-network-mocking-interception/SKILL.md +50 -4
- package/.agents/skills/03-api-and-contracts/api-spec-to-e2e-matrix/SKILL.md +47 -6
- package/.agents/skills/03-api-and-contracts/consumer-driven-contract-testing/SKILL.md +79 -34
- package/.agents/skills/04-performance-and-load/backend-load-and-stress-patterns/SKILL.md +47 -4
- package/.agents/skills/04-performance-and-load/lean-impact-scoreboard/SKILL.md +46 -5
- package/.agents/skills/04-performance-and-load/web-vitals-and-performance-profiler/SKILL.md +44 -2
- package/.agents/skills/05-security-and-compliance/security-dast-sentinel/SKILL.md +57 -13
- package/.agents/skills/05-security-and-compliance/skillspector-security-guard/SKILL.md +45 -4
- package/.agents/skills/05-security-and-compliance/wcag-audit-patterns/SKILL.md +48 -3
- package/.agents/skills/06-healing-and-resilience/agentless-qa-repair/SKILL.md +45 -4
- package/.agents/skills/06-healing-and-resilience/antigravity-test-sentinel/SKILL.md +47 -5
- package/.agents/skills/06-healing-and-resilience/failure-autopsy-and-healing/SKILL.md +48 -7
- package/.agents/skills/06-healing-and-resilience/flaky-test-sentinel-and-quarantine/SKILL.md +54 -23
- package/.agents/skills/06-healing-and-resilience/self-healing-locator/SKILL.md +46 -5
- package/.agents/skills/06-healing-and-resilience/systematic-debugging/SKILL.md +50 -9
- package/.agents/skills/06-healing-and-resilience/test-driven-development/SKILL.md +52 -11
- package/.agents/skills/06-healing-and-resilience/verification-before-completion/SKILL.md +52 -11
- package/.agents/skills/06-healing-and-resilience/visual-regression-sentinel/SKILL.md +47 -4
- package/.agents/skills/07-data-and-state/db-state-verifier-and-seeder/SKILL.md +50 -4
- package/.agents/skills/07-data-and-state/file-and-report-validator/SKILL.md +47 -3
- package/.agents/skills/07-data-and-state/synthetic-test-data-generator/SKILL.md +52 -4
- package/.agents/skills/08-code-quality/code-refactoring-refactor-clean/SKILL.md +51 -10
- package/.agents/skills/08-code-quality/graphify-impact-analyzer/SKILL.md +78 -34
- package/.agents/skills/08-code-quality/graphify-impact-analyzer/examples/dependency_blast_radius.json +3 -4
- package/.agents/skills/08-code-quality/lean-code-ladder/SKILL.md +49 -9
- package/.agents/skills/08-code-quality/repo-quality-audit/SKILL.md +45 -5
- package/.agents/skills/08-code-quality/senior-code-review/SKILL.md +48 -6
- package/.agents/skills/08-code-quality/surgical-coding-principles/SKILL.md +49 -9
- package/.agents/skills/08-code-quality/technical-debt-harvester/SKILL.md +47 -4
- package/.agents/skills/09-ci-and-devops/github-actions-templates/SKILL.md +48 -16
- package/.agents/skills/09-ci-and-devops/github-actions-templates/examples/matrix_test_pipeline.yml +2 -1
- package/.agents/skills/10-agent-orchestration/actor-qa-swarm/SKILL.md +44 -3
- package/.agents/skills/10-agent-orchestration/agent-orchestration-multi-agent-optimize/SKILL.md +47 -6
- package/.agents/skills/10-agent-orchestration/event-stream-qa-orchestrator/SKILL.md +44 -3
- package/.agents/skills/10-agent-orchestration/persistent-planning-files/SKILL.md +52 -12
- package/.agents/skills/10-agent-orchestration/qa-memory-os/SKILL.md +45 -4
- package/.agents/skills/11-ai-and-prompting/meta-prompt-crafter/SKILL.md +51 -9
- package/.agents/skills/11-ai-and-prompting/mipro-prompt-compiler/SKILL.md +44 -3
- package/.agents/skills/11-ai-and-prompting/qa-aci-interface/SKILL.md +44 -3
- package/.agents/skills/11-ai-and-prompting/token-and-context-optimizer/SKILL.md +49 -8
- package/.cursor/rules/synapse-qa.mdc +20 -7
- package/.cursorrules +19 -5
- package/.windsurfrules +18 -3
- package/AGENTS.md +51 -65
- package/CLAUDE.md +38 -24
- package/README.en.md +2 -2
- package/README.md +2 -2
- package/agentskills.json +1 -1
- package/docs/ARCHITECTURE.en.md +1 -1
- package/docs/ARCHITECTURE.md +1 -1
- package/docs/CLIENT_SUPPORT_MATRIX.md +44 -0
- package/docs/SIDE_EFFECTING_TOOLS_INVENTORY.md +233 -0
- package/package.json +1 -1
- package/plugin.json +1 -1
- package/skills_manifest.json +1 -1
- package/synapse.config.json +1 -1
- package/templates/ai_bridges/CLAUDE.md +29 -26
- package/templates/ai_bridges/copilot-instructions.md +14 -5
- package/templates/ai_bridges/cursor_synapse_qa.mdc +16 -7
- package/templates/ai_bridges/windsurfrules +14 -3
- package/tools/TOOLS_RUNBOOK.md +1 -1
- package/tools/__init__.py +1 -1
- package/tools/api_fuzzer.py +295 -41
- package/tools/benchmark_swarm_context.py +11 -4
- package/tools/browser_helpers.js +2 -2
- package/tools/cli.py +4 -4
- package/tools/core/bm25_retriever.py +1 -1
- package/tools/core/i18n.py +1 -1
- package/tools/core/provider_usage_adapter.py +66 -0
- package/tools/core/token_estimator.py +39 -1
- package/tools/count_tokens.py +2 -2
- package/tools/dashboard_generator.py +8 -8
- package/tools/db_sandbox.py +182 -43
- package/tools/distill_log.py +1 -1
- package/tools/distill_skill.py +17 -2
- package/tools/git_pre_commit_guard.py +190 -1
- package/tools/mcp_docx_server.py +1 -1
- package/tools/mcp_excel_server.py +1 -1
- package/tools/mcp_server.py +33 -5
- package/tools/micro_mutator.py +30 -1
- package/tools/pentest_scope_guard.py +288 -15
- package/tools/project_state.py +29 -5
- package/tools/reversible_sandbox.py +58 -13
- package/tools/rules_router.py +80 -27
- package/tools/skill_canonical_digest.py +60 -2
- package/tools/sync_agent_rules.py +187 -61
- package/tools/task_memory_manager.py +16 -1
- package/tools/ui_server.py +3 -1
- package/tools/update_sentinel.py +3 -3
- package/tools/validate_skills_schema.py +349 -56
- package/tools/verify_all.py +96 -2
- package/tools/welcome_page.py +1 -1
package/.agents/skills/06-healing-and-resilience/flaky-test-sentinel-and-quarantine/SKILL.md
CHANGED
|
@@ -10,13 +10,38 @@ policies:
|
|
|
10
10
|
---
|
|
11
11
|
# Flaky Test Sentinel & Quarantine Protocol
|
|
12
12
|
|
|
13
|
-
> **
|
|
13
|
+
> **Author Attribution & MIT License Guard:** Synapse Hub is open-source software created by **Thế Vị (@TeeVee98)** (<https://github.com/TeeVee98/Synapse-Hub>) under the MIT License.
|
|
14
14
|
|
|
15
15
|
---
|
|
16
16
|
|
|
17
|
-
## 1.
|
|
17
|
+
## 1. Trigger & Activation
|
|
18
18
|
|
|
19
|
-
|
|
19
|
+
- **Trigger Clause:** Use when detecting intermittent or non-deterministic test failures, calculating flakiness scores across repeated test executions ($N \ge 5$), managing test quarantine isolation, or tracking test graduation.
|
|
20
|
+
- **Activation Scope:**
|
|
21
|
+
- Test suites exhibiting non-deterministic pass/fail outcomes without code modifications.
|
|
22
|
+
- CI/CD pipeline stabilization via quarantine tagging (`@quarantine`).
|
|
23
|
+
- Quarantine lab execution and root-cause analysis.
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## 2. Non-Goals & Boundary Constraints
|
|
28
|
+
|
|
29
|
+
- **What NOT to do:**
|
|
30
|
+
- Do NOT permanently silence or ignore flaky tests; quarantined tests must be monitored and scheduled for graduation.
|
|
31
|
+
- Do NOT use static delays (`Thread.sleep`, `time.sleep`) as a quick fix for flakiness.
|
|
32
|
+
- Do NOT make absolute stability guarantees; use empirical flakiness scoring and pass rate heuristics.
|
|
33
|
+
- **Sibling Skill Boundaries:**
|
|
34
|
+
- `failure-autopsy-and-healing`: Focuses on deep autopsy of deterministic single-run failures and locator self-healing.
|
|
35
|
+
- `systematic-debugging`: Governs root-cause investigation across general code and test defects.
|
|
36
|
+
- `flaky-test-sentinel-and-quarantine` (this skill): Manages multi-run detection, flakiness classification, quarantine isolation, and the graduation gate.
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## 3. Ordered Execution Workflow
|
|
41
|
+
|
|
42
|
+
### Phase 1: Detection Engine & Quarantine Lifecycle
|
|
43
|
+
|
|
44
|
+
In large enterprise test suites, non-deterministic tests erode trust in CI/CD and slow down deployments. The 5-stage quarantine lifecycle isolates unstable tests while preserving delivery velocity:
|
|
20
45
|
|
|
21
46
|
```mermaid
|
|
22
47
|
flowchart TD
|
|
@@ -26,9 +51,7 @@ flowchart TD
|
|
|
26
51
|
D --> E["<b>5. Graduation DoD Gate</b><br/>Must achieve 10/10 consecutive passes<br/>Graduates back to main"]
|
|
27
52
|
```
|
|
28
53
|
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
## 2. Flakiness Scoring Formula & Classification
|
|
54
|
+
### Phase 2: Flakiness Scoring Formula & Classification
|
|
32
55
|
|
|
33
56
|
$$P = \frac{\text{Passed Executions}}{N} \times 100\% \quad \text{where } N \ge 5$$
|
|
34
57
|
|
|
@@ -41,11 +64,11 @@ $$\text{Flip Rate} = \frac{\text{Status Transitions (Pass} \leftrightarrow \text
|
|
|
41
64
|
| **SEVERE FLAKE** | $P < 80\%$ | $\ge 2$ flips | **Immediate Quarantine Tagging (`@quarantine`)** | Excluded from PR merge blocker |
|
|
42
65
|
| **DEAD TEST / BROKEN**| $0\%$ | 0 flips | File critical AUT/Test bug defect | Regular test failure |
|
|
43
66
|
|
|
44
|
-
|
|
67
|
+
### Phase 3: Quarantine Tagging & Execution Strategies
|
|
45
68
|
|
|
46
|
-
|
|
69
|
+
#### Playwright / TypeScript Quarantine Tagging
|
|
47
70
|
|
|
48
|
-
|
|
71
|
+
<!-- classification: illustrative -->
|
|
49
72
|
```typescript
|
|
50
73
|
import { test, expect } from '@playwright/test';
|
|
51
74
|
|
|
@@ -56,26 +79,26 @@ import { test, expect } from '@playwright/test';
|
|
|
56
79
|
test('TC-104: Submit monthly reconciliation report @quarantine', async ({ page }) => {
|
|
57
80
|
// Test code remains intact for debugging
|
|
58
81
|
await page.goto('/reports/reconciliation');
|
|
59
|
-
|
|
82
|
+
await expect(page.locator('.reconciliation-panel')).toBeVisible();
|
|
60
83
|
});
|
|
61
84
|
```
|
|
62
85
|
|
|
63
|
-
|
|
86
|
+
#### CI/CD Filter Configuration
|
|
87
|
+
|
|
88
|
+
<!-- classification: illustrative -->
|
|
64
89
|
```bash
|
|
65
|
-
# 1. Main PR Blocker Suite: Runs
|
|
90
|
+
# 1. Main PR Blocker Suite: Runs stable tests to minimize flaky disruptions
|
|
66
91
|
npx playwright test --grep-invert "@quarantine"
|
|
67
92
|
|
|
68
93
|
# 2. Nightly Quarantine Lab: Runs quarantined tests to collect diagnostics & heal
|
|
69
94
|
npx playwright test --grep "@quarantine" --reporter=json
|
|
70
95
|
```
|
|
71
96
|
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
## 4. Automated Multi-Run Flakiness Detector Tool
|
|
97
|
+
### Phase 4: Automated Multi-Run Flakiness Detector Tool
|
|
75
98
|
|
|
76
99
|
This repository provides an automated flakiness analyzer in `tools/flaky_detector.py`.
|
|
77
100
|
|
|
78
|
-
|
|
101
|
+
<!-- classification: illustrative -->
|
|
79
102
|
```bash
|
|
80
103
|
# Run a specific test script 5 consecutive times to benchmark stability
|
|
81
104
|
synapse flaky --repeat 5 --cmd "npm test -- tests/e2e/TC-104.spec.ts"
|
|
@@ -84,7 +107,9 @@ synapse flaky --repeat 5 --cmd "npm test -- tests/e2e/TC-104.spec.ts"
|
|
|
84
107
|
synapse flaky --reports "reports/**/junit*.xml"
|
|
85
108
|
```
|
|
86
109
|
|
|
87
|
-
|
|
110
|
+
Reference quarantine registry schema (see `examples/quarantine_registry.json` and generated `reports/quarantine_board.json`):
|
|
111
|
+
|
|
112
|
+
<!-- classification: illustrative -->
|
|
88
113
|
```json
|
|
89
114
|
{
|
|
90
115
|
"test_id": "TC-104",
|
|
@@ -98,9 +123,7 @@ Output Report (`reports/quarantine_board.json`):
|
|
|
98
123
|
}
|
|
99
124
|
```
|
|
100
125
|
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
## 5. Root-Cause Remediation Guide (Heal & Graduate)
|
|
126
|
+
### Phase 5: Root-Cause Remediation (Heal & Graduate)
|
|
104
127
|
|
|
105
128
|
| Flakiness Root Cause | Anti-Pattern | Mandatory Remediation |
|
|
106
129
|
| :--- | :--- | :--- |
|
|
@@ -111,9 +134,17 @@ Output Report (`reports/quarantine_board.json`):
|
|
|
111
134
|
|
|
112
135
|
---
|
|
113
136
|
|
|
114
|
-
##
|
|
137
|
+
## 4. Verification Points & DoD
|
|
115
138
|
|
|
116
139
|
Before removing the `@quarantine` tag and restoring a test to the main PR suite:
|
|
117
140
|
1. **Verification Run:** The healed test must execute **10 consecutive iterations** in headless mode with **100% Pass Rate** (`10/10 PASS`).
|
|
118
|
-
2. **Code Standards:** 100% UTF-8 No BOM, Clean POM flat architecture, zero hardcoded XPaths.
|
|
119
|
-
3. **Artifact Update:** Update `reports/quarantine_board.json` to mark status as `GRADUATED`.
|
|
141
|
+
2. **Code Standards:** 100% UTF-8 No BOM, Clean POM flat architecture, zero hardcoded XPaths, dynamic smart waits.
|
|
142
|
+
3. **Artifact Update:** Update `reports/quarantine_board.json` (or `examples/quarantine_registry.json`) to mark status as `GRADUATED`.
|
|
143
|
+
|
|
144
|
+
---
|
|
145
|
+
|
|
146
|
+
## 5. Handoff Protocol
|
|
147
|
+
|
|
148
|
+
- **Deliverables:** Updated test file with `@quarantine` tag added or removed, flakiness detection log, and `reports/quarantine_board.json`.
|
|
149
|
+
- **Next Subagent:** Pass quarantined tests to `failure-analyst` for root-cause diagnosis or to `quality-gatekeeper` for graduation sign-off.
|
|
150
|
+
- **Output Format:** JSON flakiness report with pass rate and flip count metrics.
|
|
@@ -10,11 +10,35 @@ policies:
|
|
|
10
10
|
---
|
|
11
11
|
# Self-Healing Locator Skill — Token-Efficient AOM Diffing
|
|
12
12
|
|
|
13
|
+
> **Author Attribution & MIT License Guard:** Synapse Hub is open-source software created by **Thế Vị (@TeeVee98)** (<https://github.com/TeeVee98/Synapse-Hub>) under the MIT License.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Trigger & Activation
|
|
18
|
+
|
|
19
|
+
- **Trigger Clause:** Heal broken locators using semantic AOM diffing with Role, Label, and TestId hierarchy. Use when locators break and automated healing strategies are needed.
|
|
20
|
+
- **Activation Scope:** Domain workflows for `06-healing-and-resilience` involving `self-healing-locator`.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Non-Goals & Boundary Constraints
|
|
25
|
+
|
|
26
|
+
- **What NOT to do:**
|
|
27
|
+
- Do NOT fall back to brittle positional XPaths when primary locators fail.
|
|
28
|
+
- Do NOT hardcode healed locators in test scripts; encapsulate updates inside Page Classes.
|
|
29
|
+
- Do NOT silently update locators without logging the drift and confidence score.
|
|
30
|
+
- **Sibling Skill Boundaries:**
|
|
31
|
+
- failure-autopsy-and-healing handles full failure autopsies; self-healing-locator focuses specifically on runtime and static selector recovery using AOM diffing.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 3. Ordered Execution Workflow
|
|
36
|
+
|
|
13
37
|
This skill provides an automated, token-conservative mechanism to diagnose, diff, and repair broken web element locators across web applications without overwhelming the LLM context window with raw HTML dumps.
|
|
14
38
|
|
|
15
39
|
---
|
|
16
40
|
|
|
17
|
-
|
|
41
|
+
### The Token Conservation Principle
|
|
18
42
|
|
|
19
43
|
- **Raw HTML DOM Problem:** Dumping a complete HTML webpage into the prompt can consume 50,000 to 150,000 tokens, degrading AI reasoning and triggering context rot.
|
|
20
44
|
- **AOM Subtree Solution:** Extract exclusively the **Accessibility Object Model (AOM)** tree or a 2-hop semantic DOM subtree surrounding the target element, consuming under **500 tokens**.
|
|
@@ -29,7 +53,7 @@ flowchart LR
|
|
|
29
53
|
|
|
30
54
|
---
|
|
31
55
|
|
|
32
|
-
|
|
56
|
+
### Master Locator Ranking Hierarchy
|
|
33
57
|
|
|
34
58
|
When searching for a replacement locator, evaluate candidates in this strict priority order:
|
|
35
59
|
|
|
@@ -49,10 +73,11 @@ When searching for a replacement locator, evaluate candidates in this strict pri
|
|
|
49
73
|
|
|
50
74
|
---
|
|
51
75
|
|
|
52
|
-
|
|
76
|
+
### In-Browser AOM Probing Snippet
|
|
53
77
|
|
|
54
78
|
Execute this compact JavaScript probe via Playwright or Chrome DevTools to locate the target node without dumping HTML:
|
|
55
79
|
|
|
80
|
+
<!-- classification: illustrative -->
|
|
56
81
|
```javascript
|
|
57
82
|
// Extract interactive elements tree scoped to active modal or container
|
|
58
83
|
() => {
|
|
@@ -70,7 +95,7 @@ Execute this compact JavaScript probe via Playwright or Chrome DevTools to locat
|
|
|
70
95
|
|
|
71
96
|
---
|
|
72
97
|
|
|
73
|
-
|
|
98
|
+
### Healing Execution Checklist
|
|
74
99
|
|
|
75
100
|
1. **Verify Uniqueness:** Ensure the candidate locator matches exactly 1 element on the live DOM.
|
|
76
101
|
2. **Encapsulate in Page Class:** Update the locator constant or private selector inside the relevant Page Class (e.g. `BasePage` or domain Page Class).
|
|
@@ -78,7 +103,7 @@ Execute this compact JavaScript probe via Playwright or Chrome DevTools to locat
|
|
|
78
103
|
|
|
79
104
|
---
|
|
80
105
|
|
|
81
|
-
|
|
106
|
+
### Boundary vs `failure-autopsy-and-healing`
|
|
82
107
|
|
|
83
108
|
- **This skill** = a locator-repair cascade (cache → testid → text-semantic →
|
|
84
109
|
geometric relative match), used when the diagnosis "stale/broken locator"
|
|
@@ -89,3 +114,19 @@ Execute this compact JavaScript probe via Playwright or Chrome DevTools to locat
|
|
|
89
114
|
skill does not touch).
|
|
90
115
|
- Chain of custody: autopsy RCA → "locator drift" verdict → this skill repairs
|
|
91
116
|
→ autopsy re-runs the healed test to close the loop.
|
|
117
|
+
|
|
118
|
+
---
|
|
119
|
+
|
|
120
|
+
## 4. Verification Points & DoD
|
|
121
|
+
|
|
122
|
+
1. Healed selector matches target element with $\ge 80\%$ confidence.
|
|
123
|
+
2. Locator update strictly preserves Clean POM encapsulation in Page Classes.
|
|
124
|
+
3. Verification run confirms successful element interaction with healed selector.
|
|
125
|
+
|
|
126
|
+
---
|
|
127
|
+
|
|
128
|
+
## 5. Handoff Protocol
|
|
129
|
+
|
|
130
|
+
- **Deliverables:** Updated Page Class selector definitions, healing confidence log, and test verification output.
|
|
131
|
+
- **Next Subagent:** failure-analyst / pom-crafter
|
|
132
|
+
- **Output Format:** Clean POM code, structured JSON telemetry, or validated markdown handoff.
|
|
@@ -10,19 +10,44 @@ policies:
|
|
|
10
10
|
---
|
|
11
11
|
# Systematic Debugging — Enterprise Root-Cause Standard
|
|
12
12
|
|
|
13
|
+
> **Author Attribution & MIT License Guard:** Synapse Hub is open-source software created by **Thế Vị (@TeeVee98)** (<https://github.com/TeeVee98/Synapse-Hub>) under the MIT License.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Trigger & Activation
|
|
18
|
+
|
|
19
|
+
- **Trigger Clause:** Investigate root cause through four disciplined phases before proposing any fix for a test failure, bug, or unexpected behavior. Use when a test fails, a defect appears, or a fix attempt has already failed and guessing is tempting.
|
|
20
|
+
- **Activation Scope:** Domain workflows for `06-healing-and-resilience` involving `systematic-debugging`.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Non-Goals & Boundary Constraints
|
|
25
|
+
|
|
26
|
+
- **What NOT to do:**
|
|
27
|
+
- Do NOT propose fixes before completing root-cause investigation (The Iron Law).
|
|
28
|
+
- Do NOT make multiple speculative changes simultaneously; test one hypothesis at a time.
|
|
29
|
+
- Do NOT modify out-of-scope files or write dummy bypasses to force test passes.
|
|
30
|
+
- **Sibling Skill Boundaries:**
|
|
31
|
+
- agentless-qa-repair and failure-autopsy handle automated patch synthesis; systematic-debugging governs human-agent scientific diagnostic protocol.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 3. Ordered Execution Workflow
|
|
36
|
+
|
|
13
37
|
Specialized skill for failure investigation in test automation: always find root cause before attempting fixes. Symptom fixes are failure.
|
|
14
38
|
|
|
15
39
|
---
|
|
16
40
|
|
|
17
|
-
|
|
41
|
+
### The Iron Law
|
|
18
42
|
|
|
19
|
-
|
|
43
|
+
<!-- classification: illustrative -->
|
|
44
|
+
```text
|
|
20
45
|
NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST
|
|
21
46
|
```
|
|
22
47
|
|
|
23
48
|
If Phase 1 is not complete, proposing fixes is prohibited. This applies ESPECIALLY when: under time pressure (emergencies make guessing tempting), a "quick fix" seems obvious, or previous fixes did not work.
|
|
24
49
|
|
|
25
|
-
|
|
50
|
+
### The Four Phases
|
|
26
51
|
|
|
27
52
|
```mermaid
|
|
28
53
|
flowchart TD
|
|
@@ -58,7 +83,7 @@ flowchart TD
|
|
|
58
83
|
4. **If the fix does not work:** STOP. Count attempts. Fewer than 3 → return to Phase 1 with the new information.
|
|
59
84
|
5. **If 3+ fixes failed:** question the architecture, not the code — each fix revealing a new problem elsewhere is the signature of a wrong pattern. Discuss with the user before a fourth attempt.
|
|
60
85
|
|
|
61
|
-
|
|
86
|
+
### Hub Tooling Chain
|
|
62
87
|
|
|
63
88
|
| Step | Tool | Role in investigation |
|
|
64
89
|
|---|---|---|
|
|
@@ -70,7 +95,7 @@ flowchart TD
|
|
|
70
95
|
|
|
71
96
|
Escalation order: `failure-analyst` performs the autopsy; `quality-gatekeeper` re-verifies after the fix lands.
|
|
72
97
|
|
|
73
|
-
|
|
98
|
+
### Red Flags — STOP and Return to Phase 1
|
|
74
99
|
|
|
75
100
|
- "Quick fix for now, investigate later"
|
|
76
101
|
- "Just try changing X and see if it works"
|
|
@@ -80,11 +105,11 @@ Escalation order: `failure-analyst` performs the autopsy; `quality-gatekeeper` r
|
|
|
80
105
|
- "One more fix attempt" after 2+ failures
|
|
81
106
|
- Each fix revealing a new problem in a different place (architecture question)
|
|
82
107
|
|
|
83
|
-
|
|
108
|
+
### When Investigation Finds No Root Cause
|
|
84
109
|
|
|
85
110
|
If disciplined investigation proves the issue is genuinely environmental or timing-dependent: document what was investigated, implement appropriate handling (retry, timeout, error message), add monitoring for future investigation — and remember: most "no root cause" conclusions are incomplete investigation. Hand residual non-determinism to `flaky-test-sentinel-and-quarantine` for scoring and quarantine instead of declaring it unsolvable.
|
|
86
111
|
|
|
87
|
-
|
|
112
|
+
### Boundary vs Sibling Skills (Do Not Confuse)
|
|
88
113
|
|
|
89
114
|
| Sibling | Owns | Hand-off |
|
|
90
115
|
|---|---|---|
|
|
@@ -93,13 +118,29 @@ If disciplined investigation proves the issue is genuinely environmental or timi
|
|
|
93
118
|
| `flaky-test-sentinel-and-quarantine` | Flakiness scoring and quarantine lifecycle | Receives non-reproducible findings after Phase 1 |
|
|
94
119
|
| `self-healing-locator` | Semantic locator healing | Consumes the verified root cause for locator-class defects |
|
|
95
120
|
|
|
96
|
-
|
|
121
|
+
### Reference Documentation
|
|
97
122
|
|
|
98
123
|
- `python3 tools/pinpoint_failure.py <report>` — first command on any failed run.
|
|
99
124
|
- `python3 tools/auto_heal.py` — closed-loop triage after isolation.
|
|
100
125
|
- `python3 tools/cache_resolver.py query "<error message>"` — 0-token known-defect lookup.
|
|
101
126
|
- [Four-phase worked case (`four_phase_case.md`)](examples/four_phase_case.md): a timeout failure traced to root cause.
|
|
102
127
|
|
|
103
|
-
|
|
128
|
+
### Attribution
|
|
104
129
|
|
|
105
130
|
Adapted from **superpowers** skill `systematic-debugging` (https://github.com/obra/superpowers) — MIT License, Copyright (c) 2025 Jesse Vincent. The Iron Law, four-phase process, 3-fix architecture rule, and rationalization guardrails are re-scoped for enterprise QA automation in Synapse Hub; wording, structure, and tooling chain modified.
|
|
131
|
+
|
|
132
|
+
---
|
|
133
|
+
|
|
134
|
+
## 4. Verification Points & DoD
|
|
135
|
+
|
|
136
|
+
1. Phase 1 root-cause investigation identifies exact failure trigger and line span.
|
|
137
|
+
2. Single minimal hypothesis tested with isolated modification.
|
|
138
|
+
3. Reproduction test confirms defect is resolved without regression.
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## 5. Handoff Protocol
|
|
143
|
+
|
|
144
|
+
- **Deliverables:** Root-cause investigation log, hypothesis testing records, and minimal verified patch.
|
|
145
|
+
- **Next Subagent:** failure-analyst / pom-crafter
|
|
146
|
+
- **Output Format:** Clean POM code, structured JSON telemetry, or validated markdown handoff.
|
|
@@ -10,20 +10,45 @@ policies:
|
|
|
10
10
|
---
|
|
11
11
|
# Test-Driven Development — Enterprise Red-Green-Refactor Standard
|
|
12
12
|
|
|
13
|
+
> **Author Attribution & MIT License Guard:** Synapse Hub is open-source software created by **Thế Vị (@TeeVee98)** (<https://github.com/TeeVee98/Synapse-Hub>) under the MIT License.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Trigger & Activation
|
|
18
|
+
|
|
19
|
+
- **Trigger Clause:** Enforce the RED-GREEN-REFACTOR cycle: no production or automation code is written before a failing test exists and has been watched failing. Use when implementing new helpers, Page Class logic, bug fixes, or any behavior change in test codebases.
|
|
20
|
+
- **Activation Scope:** Domain workflows for `06-healing-and-resilience` involving `test-driven-development`.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Non-Goals & Boundary Constraints
|
|
25
|
+
|
|
26
|
+
- **What NOT to do:**
|
|
27
|
+
- Do NOT write production implementation code before authoring a failing test (The Iron Law).
|
|
28
|
+
- Do NOT write tests that assert trivialities or implementation details; test observable behavior.
|
|
29
|
+
- Do NOT skip the Refactor phase of the Red-Green-Refactor cycle.
|
|
30
|
+
- **Sibling Skill Boundaries:**
|
|
31
|
+
- lean-code-ladder governs code minimization; test-driven-development governs the Red-Green-Refactor authoring discipline.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 3. Ordered Execution Workflow
|
|
36
|
+
|
|
13
37
|
Specialized skill for writing test-automation and tool code test-first: write the test, watch it fail for the right reason, write minimal code to pass, then refactor while staying green.
|
|
14
38
|
|
|
15
39
|
---
|
|
16
40
|
|
|
17
|
-
|
|
41
|
+
### The Iron Law
|
|
18
42
|
|
|
19
|
-
|
|
43
|
+
<!-- classification: illustrative -->
|
|
44
|
+
```text
|
|
20
45
|
NO PRODUCTION CODE WITHOUT A FAILING TEST FIRST
|
|
21
46
|
```
|
|
22
47
|
|
|
23
48
|
- Code written before the test is deleted — not kept "as reference", not "adapted while writing tests". Delete means delete; implement fresh from the tests.
|
|
24
49
|
- "Violating the letter of the rules is violating the spirit of the rules." Thinking "skip TDD just this once" is rationalization, not judgment.
|
|
25
50
|
|
|
26
|
-
|
|
51
|
+
### The Cycle
|
|
27
52
|
|
|
28
53
|
```mermaid
|
|
29
54
|
flowchart LR
|
|
@@ -50,7 +75,7 @@ flowchart LR
|
|
|
50
75
|
|
|
51
76
|
- Remove duplication, improve names, extract helpers. Keep tests green; add no behavior.
|
|
52
77
|
|
|
53
|
-
|
|
78
|
+
### Good Tests
|
|
54
79
|
|
|
55
80
|
| Quality | Good | Bad |
|
|
56
81
|
|---|---|---|
|
|
@@ -59,13 +84,13 @@ flowchart LR
|
|
|
59
84
|
| Intent-revealing | Demonstrates the desired API | Obscures what the code should do |
|
|
60
85
|
| Honest | Asserts real behavior; you can name the production change that would make it fail | Asserts on mock behavior only |
|
|
61
86
|
|
|
62
|
-
|
|
87
|
+
### Hub Integration
|
|
63
88
|
|
|
64
89
|
- **Fake-pass synergy:** test-first pairs with `python3 tools/infer_linter.py` (flags 0-assert and tautological tests) and Gate 8 — `python3 tools/micro_mutator.py` requires Mutation Score >= 80%, which a test written AFTER the code rarely achieves honestly.
|
|
65
90
|
- **Bug fixes:** the bug reproduction IS the RED test. Never fix a bug without a test that fails before the fix and passes after.
|
|
66
91
|
- **Locator logic:** selector helpers and Page Class methods follow the same cycle; live verification (Stage 2 of `playwright-live-interaction`, 100% hit rate under 200ms) is the RED check for locator changes.
|
|
67
92
|
|
|
68
|
-
|
|
93
|
+
### Common Rationalizations (All Rejected)
|
|
69
94
|
|
|
70
95
|
| Excuse | Reality |
|
|
71
96
|
|---|---|
|
|
@@ -77,7 +102,7 @@ flowchart LR
|
|
|
77
102
|
| "Test hard = skip" | Hard to test means hard to use; listen to the design signal. |
|
|
78
103
|
| "TDD will slow me down" | TDD catches bugs before commit and lets you refactor without fear; the pragmatic shortcut is debugging in production. |
|
|
79
104
|
|
|
80
|
-
|
|
105
|
+
### Red Flags — Stop and Start Over
|
|
81
106
|
|
|
82
107
|
- Code before test; test written after implementation
|
|
83
108
|
- Test passes immediately on first run
|
|
@@ -86,7 +111,7 @@ flowchart LR
|
|
|
86
111
|
|
|
87
112
|
All of these mean: delete the code and restart the cycle from RED.
|
|
88
113
|
|
|
89
|
-
|
|
114
|
+
### Verification Checklist
|
|
90
115
|
|
|
91
116
|
Before marking any implementation complete:
|
|
92
117
|
|
|
@@ -99,7 +124,7 @@ Before marking any implementation complete:
|
|
|
99
124
|
|
|
100
125
|
Cannot check all boxes? The cycle was skipped — start over.
|
|
101
126
|
|
|
102
|
-
|
|
127
|
+
### Boundary vs Sibling Skills (Do Not Confuse)
|
|
103
128
|
|
|
104
129
|
| Sibling | Owns | Hand-off |
|
|
105
130
|
|---|---|---|
|
|
@@ -108,12 +133,28 @@ Cannot check all boxes? The cycle was skipped — start over.
|
|
|
108
133
|
| `antigravity-test-sentinel` | Hard completion blocking on gates | The sentinel enforces what this skill produces (fresh, watched, honest tests) |
|
|
109
134
|
| `surgical-coding-principles` | Diff discipline | Goal-Driven Execution (principle 4) is operationalized by this cycle |
|
|
110
135
|
|
|
111
|
-
|
|
136
|
+
### Reference Documentation
|
|
112
137
|
|
|
113
138
|
- `python3 tools/infer_linter.py` — AST scan for fake passes and 0-assert tests.
|
|
114
139
|
- `python3 tools/micro_mutator.py` — Gate 8 mutation floor (>= 80%) validating test honesty.
|
|
115
140
|
- [Worked QA example (`red_green_refactor_qa.md`)](examples/red_green_refactor_qa.md): full cycle for a retry helper in a test utility.
|
|
116
141
|
|
|
117
|
-
|
|
142
|
+
### Attribution
|
|
118
143
|
|
|
119
144
|
Adapted from **superpowers** skill `test-driven-development` (https://github.com/obra/superpowers) — MIT License, Copyright (c) 2025 Jesse Vincent. The Iron Law, RED-GREEN-REFACTOR cycle, and rationalization table are re-scoped for enterprise QA automation in Synapse Hub; wording, structure, and examples modified.
|
|
145
|
+
|
|
146
|
+
---
|
|
147
|
+
|
|
148
|
+
## 4. Verification Points & DoD
|
|
149
|
+
|
|
150
|
+
1. Failing test authored first, proving absence or defect in capability (RED).
|
|
151
|
+
2. Minimal implementation written to pass the test (GREEN).
|
|
152
|
+
3. Code cleaned and refactored while maintaining passing test status (REFACTOR).
|
|
153
|
+
|
|
154
|
+
---
|
|
155
|
+
|
|
156
|
+
## 5. Handoff Protocol
|
|
157
|
+
|
|
158
|
+
- **Deliverables:** Test suite authored prior to code, minimal implementation diff, and test pass evidence.
|
|
159
|
+
- **Next Subagent:** pom-crafter / quality-gatekeeper
|
|
160
|
+
- **Output Format:** Clean POM code, structured JSON telemetry, or validated markdown handoff.
|
|
@@ -10,19 +10,44 @@ policies:
|
|
|
10
10
|
---
|
|
11
11
|
# Verification Before Completion — Enterprise Evidence Protocol
|
|
12
12
|
|
|
13
|
+
> **Author Attribution & MIT License Guard:** Synapse Hub is open-source software created by **Thế Vị (@TeeVee98)** (<https://github.com/TeeVee98/Synapse-Hub>) under the MIT License.
|
|
14
|
+
|
|
15
|
+
---
|
|
16
|
+
|
|
17
|
+
## 1. Trigger & Activation
|
|
18
|
+
|
|
19
|
+
- **Trigger Clause:** Require fresh verification evidence — a command run in the current session with its real output — before any claim that work is complete, fixed, or passing. Use when about to report success, commit, open a PR, or close a task.
|
|
20
|
+
- **Activation Scope:** Domain workflows for `06-healing-and-resilience` involving `verification-before-completion`.
|
|
21
|
+
|
|
22
|
+
---
|
|
23
|
+
|
|
24
|
+
## 2. Non-Goals & Boundary Constraints
|
|
25
|
+
|
|
26
|
+
- **What NOT to do:**
|
|
27
|
+
- Do NOT declare any task complete without running fresh verification commands.
|
|
28
|
+
- Do NOT accept claims of correctness without terminal execution output and exit code 0.
|
|
29
|
+
- Do NOT relax assertions or disable test cases to achieve passing status.
|
|
30
|
+
- **Sibling Skill Boundaries:**
|
|
31
|
+
- antigravity-test-sentinel enforces gate locks; verification-before-completion establishes the evidence-based Claim -> Evidence verification matrix.
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## 3. Ordered Execution Workflow
|
|
36
|
+
|
|
13
37
|
Specialized skill for the evidence discipline behind every success claim in the hub: no completion statement exists without fresh verification output. This is the operational form of the Truth Anchor (No Fake Pass) rule in `AGENTS.md` Section 3.
|
|
14
38
|
|
|
15
39
|
---
|
|
16
40
|
|
|
17
|
-
|
|
41
|
+
### The Iron Law
|
|
18
42
|
|
|
19
|
-
|
|
43
|
+
<!-- classification: illustrative -->
|
|
44
|
+
```text
|
|
20
45
|
NO COMPLETION CLAIMS WITHOUT FRESH VERIFICATION EVIDENCE
|
|
21
46
|
```
|
|
22
47
|
|
|
23
48
|
If the verification command was not run in the current message turn, the claim cannot be made. "Evidence before claims, always."
|
|
24
49
|
|
|
25
|
-
|
|
50
|
+
### The Gate Function
|
|
26
51
|
|
|
27
52
|
Run before claiming ANY status or expressing satisfaction:
|
|
28
53
|
|
|
@@ -34,7 +59,7 @@ Run before claiming ANY status or expressing satisfaction:
|
|
|
34
59
|
|
|
35
60
|
Skipping any step is lying, not verifying.
|
|
36
61
|
|
|
37
|
-
|
|
62
|
+
### Claim → Required Evidence Matrix
|
|
38
63
|
|
|
39
64
|
| Claim | Requires | Not Sufficient |
|
|
40
65
|
|---|---|---|
|
|
@@ -47,7 +72,7 @@ Skipping any step is lying, not verifying.
|
|
|
47
72
|
| Subagent finished | Its diff inspected + its claims re-verified independently | Agent reports "success" |
|
|
48
73
|
| Requirements met | Line-by-line checklist against the original request | Tests passing |
|
|
49
74
|
|
|
50
|
-
|
|
75
|
+
### Red Flags — STOP
|
|
51
76
|
|
|
52
77
|
- Using "should", "probably", "seems to", "definitely works"
|
|
53
78
|
- Expressing satisfaction before verification ("Perfect!", "Done!")
|
|
@@ -56,7 +81,7 @@ Skipping any step is lying, not verifying.
|
|
|
56
81
|
- Relying on a partial check (linter passed, therefore build must pass)
|
|
57
82
|
- Thinking "just this once"
|
|
58
83
|
|
|
59
|
-
|
|
84
|
+
### Rationalization Prevention
|
|
60
85
|
|
|
61
86
|
| Excuse | Reality |
|
|
62
87
|
|---|---|
|
|
@@ -68,7 +93,7 @@ Skipping any step is lying, not verifying.
|
|
|
68
93
|
| "Partial check is enough" | Partial proves nothing about the whole. |
|
|
69
94
|
| "Different words, so the rule doesn't apply" | Spirit over letter — paraphrases and implications of success are covered. |
|
|
70
95
|
|
|
71
|
-
|
|
96
|
+
### Key Patterns
|
|
72
97
|
|
|
73
98
|
**Tests:** run command, quote `N/N pass`, then claim "all tests pass". Never "looks correct".
|
|
74
99
|
|
|
@@ -80,13 +105,13 @@ Skipping any step is lying, not verifying.
|
|
|
80
105
|
|
|
81
106
|
**Requirements:** re-read the original request, build the checklist, verify each item, report gaps or completion — "tests pass, phase complete" without the checklist is insufficient.
|
|
82
107
|
|
|
83
|
-
|
|
108
|
+
### When To Apply
|
|
84
109
|
|
|
85
110
|
Always before: ANY success or completion claim, ANY positive statement about work state, committing or PR creation, task closure, moving to the next task, delegating to subagents.
|
|
86
111
|
|
|
87
112
|
Applies to exact phrases, paraphrases, synonyms, and any communication implying completion or correctness.
|
|
88
113
|
|
|
89
|
-
|
|
114
|
+
### Boundary vs Sibling Skills (Do Not Confuse)
|
|
90
115
|
|
|
91
116
|
| Sibling | Owns | Hand-off |
|
|
92
117
|
|---|---|---|
|
|
@@ -94,12 +119,28 @@ Applies to exact phrases, paraphrases, synonyms, and any communication implying
|
|
|
94
119
|
| `test-driven-development` | RED-GREEN-REFACTOR cycle | GREEN verification output feeds the evidence quoted here |
|
|
95
120
|
| `systematic-debugging` | Root-cause investigation | Phase 4 fix verification uses this protocol for its "fixed" claim |
|
|
96
121
|
|
|
97
|
-
|
|
122
|
+
### Reference Documentation
|
|
98
123
|
|
|
99
124
|
- `python3 tools/verify_all.py --mode hub` — the 10-gate acceptance command for hub work.
|
|
100
125
|
- `python3 tools/infer_linter.py` — fake-pass scan required before any "tests are honest" claim.
|
|
101
126
|
- [Evidence log example (`evidence_log_example.md`)](examples/evidence_log_example.md): claim/evidence pairing for a swarm task closure.
|
|
102
127
|
|
|
103
|
-
|
|
128
|
+
### Attribution
|
|
104
129
|
|
|
105
130
|
Adapted from **superpowers** skill `verification-before-completion` (https://github.com/obra/superpowers) — MIT License, Copyright (c) 2025 Jesse Vincent. The Iron Law, gate function, claim-evidence matrix, and rationalization table are re-scoped for enterprise QA automation in Synapse Hub; wording, structure, and hub tooling modified.
|
|
131
|
+
|
|
132
|
+
---
|
|
133
|
+
|
|
134
|
+
## 4. Verification Points & DoD
|
|
135
|
+
|
|
136
|
+
1. Fresh execution evidence collected for every code modification.
|
|
137
|
+
2. Test runners execute to completion with 0 failures.
|
|
138
|
+
3. Working tree verified clean of unintended side-effects or temporary files.
|
|
139
|
+
|
|
140
|
+
---
|
|
141
|
+
|
|
142
|
+
## 5. Handoff Protocol
|
|
143
|
+
|
|
144
|
+
- **Deliverables:** Verification command output logs, evidence matrix, and Definition of Done sign-off.
|
|
145
|
+
- **Next Subagent:** quality-gatekeeper
|
|
146
|
+
- **Output Format:** Clean POM code, structured JSON telemetry, or validated markdown handoff.
|