synapse-hub 2.5.0 → 2.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.agents/agents/failure-analyst.md +13 -0
- package/.agents/agents/orchestrator.md +4 -2
- package/.agents/agents/pom-crafter.md +12 -0
- package/.agents/agents/quality-gatekeeper.md +15 -2
- package/.agents/agents/test-runner.md +11 -0
- package/.agents/rules/context-engineering.md +22 -102
- package/.agents/rules/mobile_appium_rules.md +19 -115
- package/.agents/rules/performance_testing_rules.md +18 -62
- package/.agents/rules/skill-precedence.md +22 -42
- package/.agents/rules/systematic-debugging.md +10 -29
- package/.agents/skills/token-and-context-optimizer/references/context_engineering_guide.md +49 -0
- package/AGENTS.md +6 -1
- package/NOTICE +40 -4
- package/README.en.md +24 -8
- package/README.md +27 -11
- package/docs/ARCHITECTURE.md +37 -9
- package/docs/adopted-ecosystem-tools.md +9 -0
- package/docs/research/agent_skill_training_analysis.md +1357 -0
- package/package.json +1 -1
- package/skills_manifest.json +2 -2
- package/synapse.config.json +3 -2
- package/templates/welcome.html.tmpl +34 -25
- package/tools/TOOLS_RUNBOOK.md +129 -0
- package/tools/__init__.py +1 -1
- package/tools/auto_heal.py +10 -0
- package/tools/auto_heal_reflexion.py +172 -0
- package/tools/browser_helpers.js +55 -0
- package/tools/cli.py +3 -3
- package/tools/dashboard_generator.py +26 -10
- package/tools/dspy_qa_signatures.py +357 -0
- package/tools/interview_story.py +117 -0
- package/tools/lint_ai_docs.py +1 -1
- package/tools/mcp_excel_server.py +1 -1
- package/tools/mcp_server.py +1 -1
- package/tools/reversible_sandbox.py +157 -0
- package/tools/safety_firewall.py +28 -0
- package/tools/skill_collision_guard.py +231 -0
- package/tools/skill_vault.py +343 -0
- package/tools/synapse_qa_bench.py +190 -0
- package/tools/textgrad_mutation_optimizer.py +234 -0
- package/tools/verify_all.py +35 -4
- package/tools/watch_mode.py +1 -1
- package/tools/welcome_page.py +2 -2
|
@@ -5,58 +5,38 @@ description: Skill Resolution Precedence, Inter-Skill Coordination, and Anti-Col
|
|
|
5
5
|
|
|
6
6
|
# Skill Resolution Precedence & Anti-Collision Protocol
|
|
7
7
|
|
|
8
|
-
> **Purpose:** Resolve overlapping trigger scopes between Project-Local Skills and External Plugins,
|
|
8
|
+
> **Purpose:** Resolve overlapping trigger scopes between Project-Local Skills and External Plugins, eliminating context contagion, token bloat, and conflicting instructions.
|
|
9
9
|
|
|
10
10
|
---
|
|
11
11
|
|
|
12
|
-
## 1. Non-Negotiable Precedence Order
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
```text
|
|
17
|
-
[Tier 1: Project-Local Skills] (.agents/skills/*)
|
|
18
|
-
v (Highest priority — Specialized, Context-Aware & Clean POM Enforced)
|
|
19
|
-
[Tier 2: Core Project Rules] (.agents/rules/*)
|
|
20
|
-
v (Mandatory operating invariants & behavioral guardrails)
|
|
21
|
-
[Tier 3: External Plugin Skills] (antigravity-testing-kit, chrome-devtools-plugin, generic plugins)
|
|
22
|
-
v (Suppressed whenever a Tier 1 skill exists)
|
|
23
|
-
```
|
|
12
|
+
## 1. Non-Negotiable 3-Tier Precedence Order
|
|
13
|
+
`[Tier 1: Project-Local Skills] (.agents/skills/*)` > `[Tier 2: Core Rules] (.agents/rules/*)` > `[Tier 3: External Plugins] (generic plugins)`.
|
|
14
|
+
Whenever a Tier 1 local skill exists, the external plugin skill is **strictly suppressed**.
|
|
24
15
|
|
|
25
16
|
---
|
|
26
17
|
|
|
27
|
-
## 2. External Plugin Suppression
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
| **Code Change Review** | `senior-code-review` | `Generic PR Reviewer`, `Code Reviewer` | Evaluates 6 pillars (Bugs, Security, Concurrency, Missing Tests, Speed, Bloat) specifically against Git Diff. |
|
|
35
|
-
| **Locator Healing & Diffing** | `self-healing-locator`<br/>`failure-autopsy-and-healing` | `Locator Healer Agent` | Uses token-efficient semantic AOM diffing (Role > Label > TestId) instead of dumping thousands of raw HTML lines. |
|
|
36
|
-
| **Flaky Test Triage & RCA** | `failure-autopsy-and-healing`<br/>`flaky-test-sentinel-and-quarantine` | `Flaky Test Analyzer` | Triage occurs via 0-token JUnit XML parsing, screenshot autopsy, and SQLite episodic memory cache (`global_cache.db`). |
|
|
37
|
-
| **Live Probing & Inspection** | `playwright-live-interaction` | `Smart Locator Agent` | Direct CDP network idle 0, dynamic overlay detachment, and in-browser helper integration under 200ms. |
|
|
38
|
-
| **Accessibility (WCAG)** | `wcag-audit-patterns` *(Guideline)* | `a11y-debugging` *(Tooling Only)* | `wcag-audit-patterns` enforces WCAG 2.2 A/AA criteria; Chrome DevTools MCP is used strictly for low-level node inspection. |
|
|
18
|
+
## 2. External Plugin Suppression Rules (Tier 1 overrides Tier 3)
|
|
19
|
+
- **POM & Test Authoring:** Use `clean-pom-crafter` / `lam-script-generator`. Suppress generic QA/test plugins. Flat Clean POM only; no raw XPaths in test scripts.
|
|
20
|
+
- **Code Review:** Use `senior-code-review` (6 pillars against git diff). Suppress generic PR reviewers.
|
|
21
|
+
- **Locator Healing:** Use `self-healing-locator` / `failure-autopsy-and-healing` (AOM diffing). Suppress generic locator healers.
|
|
22
|
+
- **Flaky Test Triage:** Use `flaky-test-sentinel-and-quarantine` (episodic SQLite memory + JUnit parsing).
|
|
23
|
+
- **Live Probing:** Use `playwright-live-interaction` (CDP idle 0 under 200ms). Suppress generic smart locators.
|
|
24
|
+
- **Accessibility:** Use `wcag-audit-patterns` for WCAG 2.2 A/AA criteria; use Chrome DevTools MCP strictly for low-level node inspection.
|
|
39
25
|
|
|
40
26
|
---
|
|
41
27
|
|
|
42
|
-
## 3. Internal Inter-Skill Coordination &
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
| **Quality Review** | `senior-code-review` | `review-spec` (`goldfish_review`) | • `senior-code-review` audits **Source Code Diffs** (uncommitted / staged git changes).<br/>• `review-spec` audits **Requirements & User Stories** in a 0-context stress-test (Elephant-Goldfish model). |
|
|
51
|
-
| **System Audit** | `repo-quality-audit` | `analyze_project` (`tools/`) | • `analyze_project.py` produces **deterministic facts** (AST map, env check, data flow, 9 gates DoD).<br/>• `repo-quality-audit` applies **senior engineering judgment** (evaluates real load, ranks *"Fix #1 first"*). |
|
|
52
|
-
| **API Testing** | `api-spec-to-e2e-matrix` | `api-network-mocking-interception` | • `api-spec-to-e2e-matrix` compiles **test matrices** from OpenAPI specs.<br/>• `api-network-mocking-interception` stubs routes and injects **chaos network faults** during UI/API test runs. |
|
|
53
|
-
| **Failure Analysis** | `self-healing-locator` | `failure-autopsy-and-healing` | • `self-healing-locator` fixes **individual broken Page Class selectors**.<br/>• `failure-autopsy-and-healing` triages **suite-wide run crashes**, JUnit XML reports, and logs. |
|
|
54
|
-
| **DoD Verification** | `verification-before-completion` | `antigravity-test-sentinel` | • `verification-before-completion` enforces the **evidence protocol** (fresh command output required in session).<br/>• `antigravity-test-sentinel` is the **hard gatekeeper** (blocks task completion until all 9 Quality Gates pass). |
|
|
28
|
+
## 3. Internal Inter-Skill Coordination & Boundaries
|
|
29
|
+
- **Code Authoring:** `lean-code-ladder` decides *what* to build (7-rung ladder); `surgical-coding-principles` restricts *where* to touch (surgical git diff).
|
|
30
|
+
- **Technical Debt:** `technical-debt-harvester` tracks `shortcut:` markers; `code-refactoring-refactor-clean` executes refactoring.
|
|
31
|
+
- **Quality Review:** `senior-code-review` audits source diffs; `review-spec` audits requirements/stories (0-context stress test).
|
|
32
|
+
- **System Audit:** `tools/analyze_project.py` gathers facts (AST, env, gates); `repo-quality-audit` prioritizes "Fix #1 first".
|
|
33
|
+
- **API Testing:** `api-spec-to-e2e-matrix` compiles test matrices; `api-network-mocking-interception` stubs routes and chaos faults.
|
|
34
|
+
- **Failure Analysis:** `self-healing-locator` heals individual locators; `failure-autopsy-and-healing` triages suite-wide crashes.
|
|
35
|
+
- **DoD Gatekeeper:** `verification-before-completion` checks fresh command evidence; `antigravity-test-sentinel` enforces all 10 gates.
|
|
55
36
|
|
|
56
37
|
---
|
|
57
38
|
|
|
58
39
|
## 4. Strict Prohibitions
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
3. **Zero Unverified Assertions:** Never complete a code or test task without executing fresh verification via `tools/verify_all.py`.
|
|
40
|
+
1. **Zero Generic Boilerplate:** Never generate legacy Selenium, Cucumber, or `findTestObject` code.
|
|
41
|
+
2. **Zero Raw HTML Dumps:** Extract compact AOM subtrees (< 500 tokens); never dump raw HTML trees.
|
|
42
|
+
3. **Zero Unverified Assertions:** Never complete tasks without running fresh verification via `tools/verify_all.py`.
|
|
@@ -31,7 +31,7 @@ This rule governs all AI Agents operating on the repository. It mandates strict
|
|
|
31
31
|
```bash
|
|
32
32
|
python3 tools/verify_all.py
|
|
33
33
|
```
|
|
34
|
-
- All
|
|
34
|
+
- All 10 Gates must return `PASS`. If any gate fails, the task is NOT complete.
|
|
35
35
|
2. **Targeted Re-Test:**
|
|
36
36
|
- After modifying Page Classes, check impacted Test Cases with:
|
|
37
37
|
```bash
|
|
@@ -46,34 +46,15 @@ This rule governs all AI Agents operating on the repository. It mandates strict
|
|
|
46
46
|
|
|
47
47
|
## 3. Situational Tool Dispatch Matrix & Zero-Command Autonomy
|
|
48
48
|
|
|
49
|
-
> 🚀 **Zero-Command Autonomous Invariant
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
| **New project / Missing `PROJECT_AI_GUIDE.md` / Fresh clone** | `python3 tools/setup_project.py [target]` | 100% Autonomous: Detects framework (Playwright/Katalon/Pytest), generates `PROJECT_AI_GUIDE.md`, installs `pre-push` hook, extracts `repo_map.txt`, runs initial audit. |
|
|
59
|
-
| **User requests: "analyze project" / "re-analyze" / "project audit"** | `python3 tools/analyze_project.py [target]` | Execute comprehensive 5-in-1 health audit, sync repo_map, infer_linter, data_flow, verify_all & generate PROJECT_HEALTH_REPORT.md.<br/>**Post-Action:** Stage, commit & push updated project files (`PROJECT_HEALTH_REPORT.md`, `repo_map.txt`) to git. |
|
|
60
|
-
| **User requests: "review code" / "review diff" / "senior review"** | `python3 tools/senior_code_review.py` | 6-pillar code change review (Bugs, Security, Concurrency, Missing Tests, Speed, Bloat). |
|
|
61
|
-
| **User requests: "scan debt" / "list shortcuts" / "harvest debt"** | `python3 tools/debt_harvester.py` | Harvest `shortcut:` technical debt markers across repo into tracked markdown ledger. |
|
|
62
|
-
| **Starting task / Querying Page Class methods** | `tools/repo_map.txt`<br/>*(run `python3 tools/repo_map.py` if classes changed)* | Read compressed 500-token map for signatures; avoid reading 4,000-line files. |
|
|
63
|
-
| **Dispatching Browser Actions (Live/CDP)** | `python3 tools/safety_firewall.py --dry-run` | Mandatory Pre-execution Hook: Block irreversible/destructive operations before browser dispatch. |
|
|
64
|
-
| **Creating or updating Test Scripts** | `python3 tools/infer_linter.py` | Run AST scan for Fake-Pass (0 asserts) and empty catch blocks. |
|
|
65
|
-
| **New/Modified Test Case Quality** | `python3 tools/micro_mutator.py --min-score 80` | AST Mutation Testing: Reject test suites with mutation score < 80%. |
|
|
66
|
-
| **Before Git Commit or Completing Task** | `python3 tools/verify_all.py` | Validate all 9 Quality Gates DoD. |
|
|
67
|
-
| **After modifying Page Classes** | `python3 tools/test_impact.py --git-diff` | Scope impacted test cases; avoid re-running entire test suites. |
|
|
68
|
-
| **When Test FAILS in CLI/KRE** | `python3 tools/pinpoint_failure.py` | Parse JUnit XML to identify exact failing line and exception in 0.1s. |
|
|
69
|
-
| **When CLI / Gradle logs are verbose** | `python3 tools/distill_log.py <log>` | Filter 95% noise; preserve only actionable stacktraces and pointers. |
|
|
70
|
-
| **Handling UI exceptions (Overlay, Stale)** | `python3 tools/cache_resolver.py query "<error>"` | Lookup 0-token solution from local SQLite memory cache before improvising. |
|
|
71
|
-
| **Pre-Push / Suite Acceptance** | `python3 tools/trace_data_flow.py` | Run Taint Analysis to verify no orphan database junk remains uncleaned. |
|
|
72
|
-
| **Canary Self-Healing Audit** | `python3 tools/agent_canary.py --run` | Inject 8 failure vectors to benchmark MTTR and autonomous recovery rate. |
|
|
73
|
-
| **Testing GIS Canvas / Chart without DOM** | `python3 tools/visual_diff_canvas.py` | Compare pixel / SSIM to detect visual rendering regression. |
|
|
74
|
-
| **Pulling code / New clone** | `python3 tools/cache_resolver.py init` | Automatically sync Central Agent Memory (`~/.agent_memory/global_cache.db`). |
|
|
75
|
-
| **Before authoring test for Page/Flow** | `python3 tools/cache_resolver.py get-lessons --context "<page>"` | Query top 2 episodic lessons learned to avoid repeating past failure patterns. |
|
|
76
|
-
| **After healing a bug / failure** | `python3 tools/cache_resolver.py record-lesson` | Record a concise 2-line lesson into SQLite memory for cross-session reflexion. |
|
|
49
|
+
> 🚀 **Zero-Command Autonomous Invariant:** The user **NEVER** needs to manually type tool commands. The AI Agent **MUST AUTONOMOUSLY recognize context and trigger** the corresponding tools on-demand. Never run tools blindly; invoke only when the concrete situation arises.
|
|
50
|
+
|
|
51
|
+
- **Setup & Audit:** New project -> `tools/setup_project.py`; Project audit -> `tools/analyze_project.py`; Code review -> `tools/senior_code_review.py`; Debt scan -> `tools/debt_harvester.py`.
|
|
52
|
+
- **Quality Gates & DoD:** Pre-commit/Handoff -> `tools/verify_all.py` (10 Gates DoD); Script lint -> `tools/infer_linter.py`; Mutation testing -> `tools/micro_mutator.py --min-score 80`.
|
|
53
|
+
- **Failure Autopsy & Healing:** Test failure -> `tools/pinpoint_failure.py` (parse JUnit XML in 0.1s); Iterative repair -> `tools/auto_heal_reflexion.py`; Reversible rollback -> `tools/reversible_sandbox.py`.
|
|
54
|
+
- **Context & Safety:** Verbose logs -> `tools/distill_log.py`; Destructive browser actions -> `tools/safety_firewall.py`; Method signatures -> `tools/repo_map.txt`.
|
|
55
|
+
- **Memory & Reflexion:** UI exceptions -> `tools/cache_resolver.py query "<error>"`; Query/Save proven patterns -> `tools/skill_vault.py`.
|
|
56
|
+
|
|
57
|
+
*(For the complete CLI parameters and full tools catalog, refer to [tools/TOOLS_RUNBOOK.md](file://tools/TOOLS_RUNBOOK.md)).*
|
|
77
58
|
|
|
78
59
|
---
|
|
79
60
|
|
|
@@ -0,0 +1,49 @@
|
|
|
1
|
+
# Context Engineering & Token Optimization Guide (Full Architecture)
|
|
2
|
+
|
|
3
|
+
*(Synthesized from `muratcankoylan/Agent-Skills-for-Context-Engineering`, `alexgreensh/token-optimizer`, and `obra/superpowers`)*
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. Pillar 1: Observation Masking
|
|
8
|
+
Raw tool outputs (CLI execution logs, Gradle builds, browser console traces) contain 90% repetitive noise and low-signal tokens.
|
|
9
|
+
- **Log Truncation & Summarization:** Never dump thousands of lines into the active conversation. Extract only: failing line number, exception name, test case identifier (`TC-xxx`), and status.
|
|
10
|
+
- **Pointer Masking:** Use structured references pointing to files on disk: `[Full raw log saved to file://<path>; 2,400 verbose lines omitted]`.
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
## 2. Pillar 2: Progressive Disclosure
|
|
15
|
+
Never load large codebases or giant classes (over 500 lines) into context at once. Load information in 3 progressive tiers:
|
|
16
|
+
- **Tier 1:** High-Level Index & Repo Map (< 150 lines) via `tools/repo_map.txt`.
|
|
17
|
+
- **Tier 2:** Graph Semantic Scoping via `graphify explain` or `test_impact.py`.
|
|
18
|
+
- **Tier 3:** Targeted Micro-Read (20-50 lines using `StartLine`/`EndLine`).
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## 3. Pillar 3: KV-Cache Friendly Ordering
|
|
23
|
+
LLM inference engines cache Key-Value (KV) attention states for identical prompt prefixes.
|
|
24
|
+
- **Static Prefix First:** Keep instructions, rules, and tool schemas immutable at the top.
|
|
25
|
+
- **Append-Only Dynamic State:** Only append new test results and tool observations at the bottom.
|
|
26
|
+
- **Predictable Format:** Use identical pre-flight status badges on every turn to maximize cache hits (> 85%).
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## 4. Pillar 4: Scoped Context & Compaction Anchoring
|
|
31
|
+
- **Compaction State Anchoring:** Critical state must be persisted to disk in `PROJECT_STATE.md`. When compaction fires, re-anchor immediately from disk with zero memory loss.
|
|
32
|
+
- **Workspace Isolation (`Workspace: 'share'`):** Subagents use isolated worktrees to prevent dirty tree collisions.
|
|
33
|
+
- **Zero Context Contagion:** Subagents return distilled 3-5 line summaries; stdout remains in subagent sandbox.
|
|
34
|
+
- **Data Partitioning:** Subagents append agent IDs to test data (`uniqueCode = "AUTO_" + time + "_SUB" + id`).
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## 5. Pillar 5: Semantic Context Distillation (Pure Signal, Zero Noise)
|
|
39
|
+
All sensory inputs must pass through deterministic filters:
|
|
40
|
+
- **CLI Logs (`tools/distill_log.py`):** Truncate 95% of verbose lines.
|
|
41
|
+
- **DOM Purifier (`qa.getCompactAOM`):** Strictly forbidden to send raw HTML. Use compact AOM (< 500 tokens).
|
|
42
|
+
- **Universal IR (`tools/universal_ir.py`):** Author clean universal IR before emitting framework-specific code.
|
|
43
|
+
|
|
44
|
+
---
|
|
45
|
+
|
|
46
|
+
## 6. Pillar 6: Elephant-Goldfish Memory Model
|
|
47
|
+
- **The Elephant (Stateful Orchestrator):** Narrows problem boundaries, gathers intent, drafts specs in `docs/specs/*.md`, maintains `PROJECT_STATE.md`. Never writes tests during design phase.
|
|
48
|
+
- **The Goldfish (Stateless Subagent Swarm):** Fresh subagent with ZERO conversation history; evaluates solely cold artifacts on disk via `tools/goldfish_review.py`.
|
|
49
|
+
- **Air-Gap Data Contract:** All inter-stage data exchanges pass via disk files (JSON/JUnit XML), never raw prompt inheritance.
|
package/AGENTS.md
CHANGED
|
@@ -61,7 +61,12 @@ You are a lazy senior developer. The best code is the code never written. Solve
|
|
|
61
61
|
- User asks to scan debt / shortcuts -> Autonomously run `tools/debt_harvester.py`.
|
|
62
62
|
- Test failure occurs in runner -> Autonomously run `tools/pinpoint_failure.py` / `tools/auto_heal.py`.
|
|
63
63
|
- Verifying test assertion quality -> Autonomously run `tools/micro_mutator.py` (reject if Mutation Score < 80%).
|
|
64
|
-
-
|
|
64
|
+
- Complex test failures / iterative self-healing -> Run `tools/auto_heal_reflexion.py` (Noah Shinn Reflexion K=3).
|
|
65
|
+
- Reversible sandbox mutations / rollback -> Run `tools/reversible_sandbox.py`.
|
|
66
|
+
- Querying / Storing verified PageClass library -> Run `tools/skill_vault.py`.
|
|
67
|
+
- Vague requirements / story ambiguity -> Autonomously run `tools/interview_story.py`.
|
|
68
|
+
- Optimizing assertions against surviving mutants -> Run `tools/textgrad_mutation_optimizer.py`.
|
|
69
|
+
- Before completing code tasks -> Autonomously run `tools/verify_all.py` (10 Quality Gates DoD).
|
|
65
70
|
|
|
66
71
|
---
|
|
67
72
|
|
package/NOTICE
CHANGED
|
@@ -107,10 +107,19 @@ Third-Party Borrowings (idea- and text-level)
|
|
|
107
107
|
modified.
|
|
108
108
|
|
|
109
109
|
17. ponytail (https://github.com/DietrichGebert/ponytail) - MIT License,
|
|
110
|
-
Copyright (c) 2026 DietrichGebert.
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
110
|
+
Copyright (c) 2026 DietrichGebert. Core patterns and concepts adapted
|
|
111
|
+
from the Ponytail v5.3 lazy-senior model include:
|
|
112
|
+
- The 7-rung reuse-first ladder and never-cut floor in
|
|
113
|
+
.agents/skills/lean-code-ladder/SKILL.md
|
|
114
|
+
- 6-pillar code change review in .agents/skills/senior-code-review/SKILL.md
|
|
115
|
+
and tools/senior_code_review.py
|
|
116
|
+
- Codebase quality audit in .agents/skills/repo-quality-audit/SKILL.md
|
|
117
|
+
- Shortcut technical debt harvesting in
|
|
118
|
+
.agents/skills/technical-debt-harvester/SKILL.md and tools/debt_harvester.py
|
|
119
|
+
- Benchmark gain reporting in .agents/skills/lean-impact-scoreboard/SKILL.md
|
|
120
|
+
- Claude Code slash commands in commands/ (synapse-* and ponytail-* aliases)
|
|
121
|
+
All components were re-engineered, adapted, and integrated for enterprise
|
|
122
|
+
test automation and Clean POM architecture.
|
|
114
123
|
|
|
115
124
|
18. planning-with-files (https://github.com/OthmanAdi/planning-with-files)
|
|
116
125
|
- MIT License, Copyright (c) 2026 Ahmad Adi. The 3-file planning pattern
|
|
@@ -178,3 +187,30 @@ Third-Party Borrowings (idea- and text-level)
|
|
|
178
187
|
and token throughput optimization in
|
|
179
188
|
.agents/skills/agent-orchestration-multi-agent-optimize/SKILL.md are
|
|
180
189
|
synthesized from autonomous multi-agent design patterns (crewAI, AutoGen).
|
|
190
|
+
|
|
191
|
+
29. Context Engineering & Token Optimization (muratcankoylan & alexgreensh)
|
|
192
|
+
- MIT License. Observation masking, progressive disclosure, KV-cache
|
|
193
|
+
ordering, and token budget management in .agents/rules/context-engineering.md
|
|
194
|
+
and .agents/skills/token-and-context-optimizer/SKILL.md are adapted from
|
|
195
|
+
muratcankoylan/Agent-Skills-for-Context-Engineering and alexgreensh/token-optimizer.
|
|
196
|
+
|
|
197
|
+
30. vercel-labs/agent-browser (https://github.com/vercel-labs/agent-browser)
|
|
198
|
+
- MIT License. Compact accessibility tree snapshots and semantic node
|
|
199
|
+
filtering referenced in .agents/skills/playwright-live-interaction/references/
|
|
200
|
+
compact_accessibility_snapshots.md are adapted from vercel-labs/agent-browser.
|
|
201
|
+
|
|
202
|
+
31. fugazi/test-automation-skills-agents (https://github.com/fugazi/test-automation-skills-agents)
|
|
203
|
+
- MIT License. The closed-loop failure triage and investigation steps in
|
|
204
|
+
.agents/skills/failure-autopsy-and-healing/references/qa_investigation_protocol.md
|
|
205
|
+
are adapted from fugazi/test-automation-skills-agents.
|
|
206
|
+
|
|
207
|
+
32. LaVague AI Large Action Model (https://github.com/lavague-ai/LaVague)
|
|
208
|
+
- Apache License 2.0. The concept of translating natural language User
|
|
209
|
+
Stories and acceptance criteria into deterministic web actions in
|
|
210
|
+
.agents/skills/lam-script-generator/SKILL.md is inspired by the LaVague
|
|
211
|
+
Large Action Model (LAM) framework.
|
|
212
|
+
|
|
213
|
+
33. Graphify (https://github.com/Graphify-Labs/graphify)
|
|
214
|
+
- Apache License 2.0. Knowledge graph generation and AST dependency blast
|
|
215
|
+
radius mapping in .agents/rules/graphify.md and
|
|
216
|
+
.agents/skills/graphify-impact-analyzer/SKILL.md are adapted from Graphify.
|
package/README.en.md
CHANGED
|
@@ -4,9 +4,9 @@
|
|
|
4
4
|
[🇻🇳 Tiếng Việt](README.md) | [🇬🇧 English Edition](README.en.md)
|
|
5
5
|
|
|
6
6
|
> **Core Mission:** A standalone, zero-dependency Linter & Guardrail engine (zero external dependencies for core engine & DoD gates) protecting test suite integrity in the era of AI code generation — Catching tautological "fake-pass" tests, eliminating hard sleeps, distilling 95% CI log noise, and enforcing Clean POM architecture. Specialized tools (Excel export, Canvas visual diff) leverage standard optional extras (`openpyxl`, `Pillow`).
|
|
7
|
-
> **Extended Ecosystem:** Bundled with 42 production QA skills, 6 specialized AI roles, and
|
|
7
|
+
> **Extended Ecosystem:** Bundled with 42 production QA skills, 6 specialized AI roles, and 10 DoD Quality Gates supporting Playwright, Cypress, Pytest, and Katalon.
|
|
8
8
|
> **Author:** Thế Vị ([@TeeVee998](https://github.com/TeeVee998))
|
|
9
|
-
> **Hub Version:** `v2.
|
|
9
|
+
> **Hub Version:** `v2.6.0` | **Scope:** 42 Standardized Skills, 6 Specialized AI Agents, 68 CLI Commands & 68+ Test Engineering Tools, 10 DoD Quality Gates.
|
|
10
10
|
|
|
11
11
|
[](https://www.npmjs.com/package/synapse-hub)
|
|
12
12
|
[](https://nodejs.org)
|
|
@@ -110,7 +110,7 @@ npx synapse-hub uninstall
|
|
|
110
110
|
|
|
111
111
|
### Run Quality DoD Gates:
|
|
112
112
|
```bash
|
|
113
|
-
npx synapse-hub verify # Execute
|
|
113
|
+
npx synapse-hub verify # Execute 10 Quality Gates DoD (Gate 0–9)
|
|
114
114
|
npx synapse-hub analyze # Comprehensive 5-in-1 project health audit
|
|
115
115
|
```
|
|
116
116
|
|
|
@@ -152,6 +152,7 @@ repos:
|
|
|
152
152
|
| **Gate 6** | **Credential Security** | Blocks hardcoded plaintext passwords, private keys, and secret tokens in code. |
|
|
153
153
|
| **Gate 7** | **Static Defect & Fake-Pass Sentinel** | Static heuristic analysis catching 0-assertion tests, empty catch blocks, hardcoded/variable sleeps (≥ 3000ms), and unawaited promises. |
|
|
154
154
|
| **Gate 8** | **Assertion Sensitivity Sentinel** | Injects syntactic micro-mutations linked via dataflow (Ultra-fast Heuristic Static mode ~0.1s; pass `--test-cmd` for runtime re-test; requires Score ≥ 80%). |
|
|
155
|
+
| **Gate 9** | **Tier-2 Skill Routing & Collision Guard** | Audits semantic descriptions and activation triggers between skills; prevents ambiguous overlap collisions (Threshold < 0.75). |
|
|
155
156
|
|
|
156
157
|
---
|
|
157
158
|
|
|
@@ -161,7 +162,7 @@ repos:
|
|
|
161
162
|
| :--- | :--- | :--- |
|
|
162
163
|
| **Fresher / Junior** | Learn fast, deliver immediately, understand weak asserts | • `npx synapse-hub export-excel`: Generate standard test cases.<br/>• `npx synapse-hub lint ./tests`: Understand why assertions are tautological.<br/>• Inspect Clean POM examples in `examples/practices/`. |
|
|
163
164
|
| **Mid-Level** | Reduce flakiness, cut locator maintenance | • `npx synapse-hub flaky scan ./tests`: Detect flaky tests and quarantine them.<br/>• `npx synapse-hub heal`: Semantic locator healing via AOM diffing.<br/>• Add linter to pre-commit git hooks. |
|
|
164
|
-
| **Senior / Lead** | Control, measurement, CI/CD safety gates | • `npx synapse-hub verify`:
|
|
165
|
+
| **Senior / Lead** | Control, measurement, CI/CD safety gates | • `npx synapse-hub verify`: 10 Automated DoD Gates in CI.<br/>• `npx synapse-hub mutate ./tests`: Heuristic assertion sensitivity analysis before merge.<br/>• Standard Markdown/JSON telemetry for PR review bots. |
|
|
165
166
|
|
|
166
167
|
> *Pro Tip: If installed globally (`npm i -g synapse-hub`), you can use the shorter syntax `synapse-hub <cmd>` or equivalent aliases `synapse`, `agent-qa`.*
|
|
167
168
|
|
|
@@ -250,7 +251,7 @@ To uphold our strict **Truth Anchor** standard, we openly disclose known system
|
|
|
250
251
|
| 1 | `clean-pom-crafter` | Clean POM & Flat Architecture page class design (zero raw selectors in test scripts). |
|
|
251
252
|
| 2 | `playwright-live-interaction` | Live DOM/AOM exploration and < 200ms selector verification via headless Playwright. |
|
|
252
253
|
| 3 | `failure-autopsy-and-healing` | Failure triage from JUnit XML/screenshots and automated broken locator repair. |
|
|
253
|
-
| 4 | `infer_linter` / `antigravity-test-sentinel` | Quality gatekeeper enforcing zero fake-passes and
|
|
254
|
+
| 4 | `infer_linter` / `antigravity-test-sentinel` | Quality gatekeeper enforcing zero fake-passes and 10 DoD quality gates. |
|
|
254
255
|
| 5 | `token-and-context-optimizer` | Prompt context optimization, log distillation, and Context Rot prevention. |
|
|
255
256
|
| 6 | `flaky-test-sentinel-and-quarantine` | Non-deterministic test detection, flakiness scoring, and quarantine management. |
|
|
256
257
|
| 7 | `api-spec-to-e2e-matrix` | Convert OpenAPI/Swagger specifications into E2E testing matrices. |
|
|
@@ -290,7 +291,7 @@ To uphold our strict **Truth Anchor** standard, we openly disclose known system
|
|
|
290
291
|
| 41 | `technical-debt-harvester` | Harvests `shortcut:` comments into an actionable debt ledger via `agent-qa debt`. |
|
|
291
292
|
| 42 | `lean-impact-scoreboard` | Displays empirical benchmark scoreboard of code reduction %, token savings, and execution speed. |
|
|
292
293
|
|
|
293
|
-
> **Open source & credits:** Skills
|
|
294
|
+
> **Open source & credits:** Skills and tools are adapted and specialized from open-source projects (NVIDIA SkillSpector, andrej-karpathy-skills, ponytail, planning-with-files, anthropics/skills, superpowers, wshobson/agents, vercel-labs/agent-browser, LaVague, graphify...) — full license and author attribution details in `NOTICE` (entries 1-33); companion external-tool guide in `docs/adopted-ecosystem-tools.md`.
|
|
294
295
|
|
|
295
296
|
</details>
|
|
296
297
|
|
|
@@ -298,13 +299,13 @@ To uphold our strict **Truth Anchor** standard, we openly disclose known system
|
|
|
298
299
|
|
|
299
300
|
## 🛠️ 8. CLI Tooling Arsenal (tools/)
|
|
300
301
|
|
|
301
|
-
Synapse Hub includes over
|
|
302
|
+
Synapse Hub includes over 68 utility tools implemented entirely in the **Python Standard Library** (zero external dependencies):
|
|
302
303
|
|
|
303
304
|
```bash
|
|
304
305
|
# 1. Diagnose local environment (Node, Python, Chrome, Git)
|
|
305
306
|
npx synapse-hub doctor
|
|
306
307
|
|
|
307
|
-
# 2. Run
|
|
308
|
+
# 2. Run 10 Quality Gates DoD Verification (Gate 0–9)
|
|
308
309
|
npx synapse-hub verify
|
|
309
310
|
|
|
310
311
|
# 3. Scan for fake-passes, tautologies, and empty catch blocks
|
|
@@ -321,6 +322,21 @@ npx synapse-hub mutate ./tests
|
|
|
321
322
|
|
|
322
323
|
# 7. Detect and quarantine flaky tests
|
|
323
324
|
npx synapse-hub flaky scan ./tests
|
|
325
|
+
|
|
326
|
+
# 8. Query lifelong verified PageClass library (Voyager SkillVault)
|
|
327
|
+
python tools/skill_vault.py --query "table_pagination"
|
|
328
|
+
|
|
329
|
+
# 9. Episodic reflection self-healing loop (Noah Shinn Reflexion K=3)
|
|
330
|
+
python tools/auto_heal_reflexion.py --test "TC_Checkout_01" --error "ElementClickInterceptedException"
|
|
331
|
+
|
|
332
|
+
# 10. Interrogate ambiguous requirements (Pocock Grilling Engine)
|
|
333
|
+
python tools/interview_story.py "As a user I want to checkout"
|
|
334
|
+
|
|
335
|
+
# 11. Optimize assertion prompts against mutants (TextGrad Loss Optimizer)
|
|
336
|
+
python tools/textgrad_mutation_optimizer.py --file tests/test_login.py
|
|
337
|
+
|
|
338
|
+
# 12. Benchmark agent tool-calling precision (Gorilla BFCL Benchmark)
|
|
339
|
+
python tools/synapse_qa_bench.py
|
|
324
340
|
```
|
|
325
341
|
|
|
326
342
|
*(See [docs/CLI_REFERENCE.md](docs/CLI_REFERENCE.md) and [tools/TOOLS_RUNBOOK.md](tools/TOOLS_RUNBOOK.md) for parameter details).*
|
package/README.md
CHANGED
|
@@ -4,9 +4,9 @@
|
|
|
4
4
|
[🇻🇳 Tiếng Việt](README.md) | [🇬🇧 English Edition](README.en.md)
|
|
5
5
|
|
|
6
6
|
> **Định vị cốt lõi:** Bộ Linter & Guardrails độc lập (zero external dependencies cho core engine & DoD gates) bảo vệ độ tin cậy của test suite khi AI viết code — Phát hiện & ngăn chặn test "pass ảo" (0 assert, tautology), loại bỏ hoàn toàn sleep cố định, nén 95% log CI/CD, và chuẩn hóa cấu trúc Clean POM. Các tính năng nâng cao (xuất bảng Excel chuẩn doanh nghiệp, so sánh ảnh Canvas) sử dụng thư viện mở rộng tùy chọn (`openpyxl`, `Pillow`).
|
|
7
|
-
> **Module mở rộng:** Sẵn sàng kết nối 42 kỹ năng QA chuyên sâu, 6 vai trò AI Swarm và
|
|
7
|
+
> **Module mở rộng:** Sẵn sàng kết nối 42 kỹ năng QA chuyên sâu, 6 vai trò AI Swarm và 10 Cổng kiểm định DoD hỗ trợ Playwright, Cypress, Pytest và Katalon.
|
|
8
8
|
> **Tác giả:** Thế Vị ([@TeeVee998](https://github.com/TeeVee998))
|
|
9
|
-
> **Phiên bản Hub:** `v2.
|
|
9
|
+
> **Phiên bản Hub:** `v2.6.0` | **Quy mô:** 42 Kỹ năng chuẩn hóa, 6 Vai trò AI Agent, 68 Lệnh CLI & 68+ Công cụ kiểm thử chuyên sâu, 10 Cổng kiểm định DoD.
|
|
10
10
|
|
|
11
11
|
[](https://www.npmjs.com/package/synapse-hub)
|
|
12
12
|
[](https://nodejs.org)
|
|
@@ -108,9 +108,9 @@ npx synapse-hub setup
|
|
|
108
108
|
npx synapse-hub uninstall
|
|
109
109
|
```
|
|
110
110
|
|
|
111
|
-
### Chạy kiểm định
|
|
111
|
+
### Chạy kiểm định 10 Cổng Chất Lượng (DoD Gates):
|
|
112
112
|
```bash
|
|
113
|
-
npx synapse-hub verify # Kiểm tra
|
|
113
|
+
npx synapse-hub verify # Kiểm tra 10 Cổng Chất Lượng DoD (Gate 0–9)
|
|
114
114
|
npx synapse-hub analyze # Báo cáo sức khỏe dự án (PROJECT_HEALTH_REPORT.md)
|
|
115
115
|
```
|
|
116
116
|
|
|
@@ -152,6 +152,7 @@ repos:
|
|
|
152
152
|
| **Gate 6** | **Bảo Vệ Bí Mật & Mật Khẩu** | Quét chặn plaintext password, private keys hoặc API token lưu cứng trong code. |
|
|
153
153
|
| **Gate 7** | **Quét Lỗi Tĩnh & Bắt Test Pass Ảo** | Tự động phát hiện test 0 assertion, try-catch bỏ qua ngoại lệ ngầm, sleep cố định (≥ 3000ms) và unawaited expect. |
|
|
154
154
|
| **Gate 8** | **Đo Độ Nhạy Assert (Micro-Mutation)** | Đảm bảo các câu lệnh assert bắt được lỗi thật khi code bị biến đổi (Chế độ Heuristic Static ~0.1s; truyền `--test-cmd` cho runtime re-test; yêu cầu điểm ≥ 80%). |
|
|
155
|
+
| **Gate 9** | **Điều Phối Kỹ Năng & Ngăn Xung Đột (Skill Collision Guard)** | Kiểm tra độ tương đồng ngữ nghĩa và trigger activation giữa các skill; chặn xung đột trùng lặp (Threshold < 0.75). |
|
|
155
156
|
|
|
156
157
|
---
|
|
157
158
|
|
|
@@ -161,7 +162,7 @@ repos:
|
|
|
161
162
|
| :--- | :--- | :--- |
|
|
162
163
|
| **Fresher / Junior** | Học nhanh, ra việc ngay, hiểu vì sao test yếu | • `npx synapse-hub export-excel`: Tạo test case chuẩn chỉnh.<br/>• `npx synapse-hub lint ./tests`: Giải thích vì sao assert yếu hoặc pass ảo.<br/>• Tham khảo mẫu Clean POM chuẩn trong `examples/practices/`. |
|
|
163
164
|
| **Mid-Level** | Bớt flaky, giảm chi phí bảo trì locator | • `npx synapse-hub flaky scan ./tests`: Dò tìm test chập chờn và phân nhóm cách ly.<br/>• `npx synapse-hub heal`: Gợi ý thay thế locator hỏng qua phân tích cây AOM.<br/>• Tích hợp linter vào pre-commit hook. |
|
|
164
|
-
| **Senior / Lead** | Đo lường, kiểm soát chất lượng, an toàn CI/CD | • `npx synapse-hub verify`:
|
|
165
|
+
| **Senior / Lead** | Đo lường, kiểm soát chất lượng, an toàn CI/CD | • `npx synapse-hub verify`: 10 Cổng chất lượng DoD tự động trong CI.<br/>• `npx synapse-hub mutate ./tests`: Kiểm tra độ nhạy của assertion trước khi merge PR.<br/>• Báo cáo chuẩn Markdown/JSON dễ dàng gắn vào GitHub Actions/GitLab CI. |
|
|
165
166
|
|
|
166
167
|
> *Mẹo tiện dụng: Nếu bạn đã cài đặt toàn cục (`npm i -g synapse-hub`), bạn có thể gõ ngắn gọn `synapse-hub <lệnh>` hoặc các bí danh `synapse`, `agent-qa`.*
|
|
167
168
|
|
|
@@ -249,8 +250,8 @@ flowchart TD
|
|
|
249
250
|
| **`live-explorer`** | Khảo sát trực tiếp DOM/AOM qua Playwright headless, tự động ngăn chặn thao tác phá hỏng hoặc nguy hiểm. | `playwright-live-interaction`, `self-healing-locator`, `visual-regression-sentinel` | `safety_firewall.py`, `browser_helpers.js` |
|
|
250
251
|
| **`pom-crafter`** | Kỹ sư Clean POM, kiểm tra breaking change API, sinh test từ User Story. | `clean-pom-crafter`, `lam-script-generator`, `synthetic-test-data-generator` | `generate_tc_from_story.py`, `api_drift_sentinel.py` |
|
|
251
252
|
| **`test-runner`** | Quản lý chạy test headless CLI, chia tải song song CI, lọc log thông minh. | `katalon-cli-runner`, `e2e-testing-patterns`, `flaky-test-sentinel-and-quarantine` | `matrix_sharder.py`, `distill_log.py`, `pinpoint_failure.py` |
|
|
252
|
-
| **`failure-analyst`** | Phân tích nguyên nhân gốc rễ (RCA) khi test thất bại, tự động đề xuất vá locator
|
|
253
|
-
| **`quality-gatekeeper`**| Chốt chặn kiểm soát chất lượng, kiểm định
|
|
253
|
+
| **`failure-analyst`** | Phân tích nguyên nhân gốc rễ (RCA) khi test thất bại, tự động đề xuất vá locator và vòng lặp Reflexion $K=3$. | `failure-autopsy-and-healing`, `token-and-context-optimizer` | `auto_heal.py`, `bug_reporter.py`, `cache_resolver.py`, `auto_heal_reflexion.py` |
|
|
254
|
+
| **`quality-gatekeeper`**| Chốt chặn kiểm soát chất lượng, kiểm định 10 Cổng DoD và độ nhạy assertion. | `antigravity-test-sentinel`, `wcag-audit-patterns`, `security-dast-sentinel` | `verify_all.py`, `micro_mutator.py`, `infer_linter.py`, `skill_collision_guard.py` |
|
|
254
255
|
|
|
255
256
|
---
|
|
256
257
|
|
|
@@ -264,7 +265,7 @@ flowchart TD
|
|
|
264
265
|
| 1 | `clean-pom-crafter` | Thiết kế Page Classes chuẩn Clean POM & Flat Architecture (zero locator trong test script). |
|
|
265
266
|
| 2 | `playwright-live-interaction` | Khảo sát trực tiếp DOM/AOM, kiểm định selector < 200ms qua Playwright Headless. |
|
|
266
267
|
| 3 | `failure-autopsy-and-healing` | Phân tích nguyên nhân lỗi từ JUnit XML/Screenshot, tự động vá locator hỏng. |
|
|
267
|
-
| 4 | `antigravity-test-sentinel` | Chốt chặn kiểm soát chất lượng, phát hiện pass ảo và bảo đảm tuân thủ
|
|
268
|
+
| 4 | `antigravity-test-sentinel` | Chốt chặn kiểm soát chất lượng, phát hiện pass ảo và bảo đảm tuân thủ 10 cổng DoD. |
|
|
268
269
|
| 5 | `token-and-context-optimizer` | Tối ưu hóa prompt context, cô đọng log và chống suy giảm ngữ cảnh (Context Rot). |
|
|
269
270
|
| 6 | `flaky-test-sentinel-and-quarantine` | Phát hiện test chập chờn, tính điểm flakiness và quản lý vòng đời cách ly. |
|
|
270
271
|
| 7 | `api-spec-to-e2e-matrix` | Chuyển đổi đặc tả OpenAPI/Swagger thành ma trận kiểm thử E2E (Happy, Boundary, Fault). |
|
|
@@ -304,7 +305,7 @@ flowchart TD
|
|
|
304
305
|
| 41 | `technical-debt-harvester` | Quét và gom toàn bộ comment `shortcut:` thành sổ nợ kỹ thuật có kiểm soát qua lệnh `agent-qa debt`. |
|
|
305
306
|
| 42 | `lean-impact-scoreboard` | Hiển thị bảng điểm đo lường hiệu quả tinh gọn code, token và thời gian phản hồi. |
|
|
306
307
|
|
|
307
|
-
> **Nguồn mở & credit:** Kỹ năng
|
|
308
|
+
> **Nguồn mở & credit:** Kỹ năng và công cụ được kế thừa và phát triển từ các dự án mã nguồn mở tiêu biểu (NVIDIA SkillSpector, andrej-karpathy-skills, ponytail, planning-with-files, anthropics/skills, superpowers, wshobson/agents, vercel-labs/agent-browser, LaVague, graphify...) — toàn bộ danh mục bản quyền, license và tác giả chi tiết tại file `NOTICE` (mục 1–33); hướng dẫn công cụ ngoài kèm theo tại `docs/adopted-ecosystem-tools.md`.
|
|
308
309
|
|
|
309
310
|
</details>
|
|
310
311
|
|
|
@@ -312,13 +313,13 @@ flowchart TD
|
|
|
312
313
|
|
|
313
314
|
## 🛠️ 8. Bộ Công Cụ Dòng Lệnh CLI (tools/)
|
|
314
315
|
|
|
315
|
-
Dự án đi kèm hơn
|
|
316
|
+
Dự án đi kèm hơn 68 công cụ tiện ích viết hoàn toàn bằng **thư viện chuẩn Python** (Standard Library), không phụ thuộc package ngoài:
|
|
316
317
|
|
|
317
318
|
```bash
|
|
318
319
|
# 1. Chẩn đoán môi trường hệ thống (Node, Python, Chrome, Git)
|
|
319
320
|
npx synapse-hub doctor
|
|
320
321
|
|
|
321
|
-
# 2. Kiểm tra
|
|
322
|
+
# 2. Kiểm tra 10 Cổng Chất Lượng DoD (Gate 0–9)
|
|
322
323
|
npx synapse-hub verify
|
|
323
324
|
|
|
324
325
|
# 3. Quét phát hiện test pass ảo và ngoại lệ bị bỏ qua ngầm
|
|
@@ -335,6 +336,21 @@ npx synapse-hub mutate ./tests
|
|
|
335
336
|
|
|
336
337
|
# 7. Dò tìm và cách ly test chập chờn (Flaky Tests)
|
|
337
338
|
npx synapse-hub flaky scan ./tests
|
|
339
|
+
|
|
340
|
+
# 8. Tra cứu thư viện PageClass lưu trữ vĩnh cửu (Voyager SkillVault)
|
|
341
|
+
python tools/skill_vault.py --query "table_pagination"
|
|
342
|
+
|
|
343
|
+
# 9. Tự động sửa lỗi kịch bản qua vòng lặp phản hồi Reflexion (K=3)
|
|
344
|
+
python tools/auto_heal_reflexion.py --test "TC_Checkout_01" --error "ElementClickInterceptedException"
|
|
345
|
+
|
|
346
|
+
# 10. Thẩm vấn và làm rõ yêu cầu nghiệp vụ mập mờ (Pocock Grilling)
|
|
347
|
+
python tools/interview_story.py "As a user I want to checkout"
|
|
348
|
+
|
|
349
|
+
# 11. Tối ưu câu lệnh assert chống mutant (TextGrad Optimizer)
|
|
350
|
+
python tools/textgrad_mutation_optimizer.py --file tests/test_login.py
|
|
351
|
+
|
|
352
|
+
# 12. Đo lường độ chính xác gọi tool của AI Agent (Gorilla BFCL Benchmark)
|
|
353
|
+
python tools/synapse_qa_bench.py
|
|
338
354
|
```
|
|
339
355
|
|
|
340
356
|
*(Xem toàn bộ hướng dẫn tham số chi tiết tại [docs/CLI_REFERENCE.md](docs/CLI_REFERENCE.md) và [tools/TOOLS_RUNBOOK.md](tools/TOOLS_RUNBOOK.md)).*
|
package/docs/ARCHITECTURE.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Agent-Skills Architecture & Design Principles
|
|
2
2
|
|
|
3
3
|
> **Scope:** Enterprise Multi-Agent QA Hub
|
|
4
|
-
> **Version:** `2.
|
|
4
|
+
> **Version:** `2.6.0` | **Status:** Active / Production Standard
|
|
5
5
|
|
|
6
6
|
---
|
|
7
7
|
|
|
@@ -69,9 +69,9 @@ Test automation scripts orchestrated through this Hub enforce strict Clean POM b
|
|
|
69
69
|
|
|
70
70
|
---
|
|
71
71
|
|
|
72
|
-
## 4. The
|
|
72
|
+
## 4. The 10 Quality Gates Definition-of-Done (DoD)
|
|
73
73
|
|
|
74
|
-
All automated test code authored, modified, or refactored by AI agents must pass through the deterministic
|
|
74
|
+
All automated test code authored, modified, or refactored by AI agents must pass through the deterministic 10 Quality Gates pipeline (`tools/verify_all.py`) prior to committing or merging:
|
|
75
75
|
|
|
76
76
|
| Gate | Title | Verifier Tool | Strict Acceptance Criteria |
|
|
77
77
|
| :---: | :--- | :--- | :--- |
|
|
@@ -84,6 +84,7 @@ All automated test code authored, modified, or refactored by AI agents must pass
|
|
|
84
84
|
| **Gate 6** | **Credential Security** | `verify_all.py` | AST regex scan blocking hardcoded passwords, private keys, and secret tokens in code. |
|
|
85
85
|
| **Gate 7** | **Static Defect & Fake-Pass Sentinel** | `infer_linter.py` | Detects 0-assertion tests, swallowed exception blocks (`except: pass`), hard sleeps, and unawaited promises. |
|
|
86
86
|
| **Gate 8** | **Assertion Sensitivity Sentinel** | `micro_mutator.py` | Syntactic micro-mutations linked via variable dataflow; requires Sensitivity Score $\ge 80\%$. |
|
|
87
|
+
| **Gate 9** | **Tier-2 Skill Routing & Collision Guard** | `skill_collision_guard.py` | Audits skill trigger descriptions and vectors; blocks ambiguous activation with semantic overlap $\ge 0.75$. |
|
|
87
88
|
|
|
88
89
|
---
|
|
89
90
|
|
|
@@ -99,12 +100,12 @@ The Hub equips 6 specialized subagents with scoped tool and skill permissions:
|
|
|
99
100
|
|
|
100
101
|
| Subagent Role | Primary Focus | Allowed Skills | Execution Tools |
|
|
101
102
|
| :--- | :--- | :--- | :--- |
|
|
102
|
-
| **`orchestrator`** | Task decomposition & governance | `agent-orchestration-multi-agent-optimize`, `graphify-impact-analyzer`, `architecture-decision-records`, `token-and-context-optimizer` | `verify_all.py`, `test_impact.py`, `setup_project.py` |
|
|
103
|
-
| **`pom-crafter`** | Clean POM design & test generation | `clean-pom-crafter`, `lam-script-generator`, `synthetic-test-data-generator`, `api-spec-to-e2e-matrix` | `lookup_xpath.py`, `repo_map.py`, `generate_tc_from_story.py` |
|
|
103
|
+
| **`orchestrator`** | Task decomposition & governance | `agent-orchestration-multi-agent-optimize`, `graphify-impact-analyzer`, `architecture-decision-records`, `token-and-context-optimizer` | `verify_all.py`, `test_impact.py`, `setup_project.py`, `interview_story.py`, `dspy_qa_signatures.py` |
|
|
104
|
+
| **`pom-crafter`** | Clean POM design & test generation | `clean-pom-crafter`, `lam-script-generator`, `synthetic-test-data-generator`, `api-spec-to-e2e-matrix` | `lookup_xpath.py`, `repo_map.py`, `generate_tc_from_story.py`, `skill_vault.py` |
|
|
104
105
|
| **`live-explorer`** | Live DOM & AOM exploration (<200ms) | `playwright-live-interaction`, `self-healing-locator`, `visual-regression-sentinel`, `api-network-mocking-interception` | `browser_helpers.js`, `e2e_flow_prober.ps1`, `safety_firewall.py` |
|
|
105
|
-
| **`test-runner`** | Headless CLI test execution & CI | `katalon-cli-runner`, `e2e-testing-patterns`, `flaky-test-sentinel-and-quarantine`, `backend-load-and-stress-patterns`, `github-actions-templates` | `matrix_sharder.py`, `distill_log.py`, `pinpoint_failure.py`, `flaky_detector.py` |
|
|
106
|
-
| **`failure-analyst`**| Visual & XML error autopsy | `failure-autopsy-and-healing`, `self-healing-locator`, `file-and-report-validator` | `auto_heal.py`, `heal_pr_automator.py`, `pinpoint_failure.py`, `cache_resolver.py` |
|
|
107
|
-
| **`quality-gatekeeper`**|
|
|
106
|
+
| **`test-runner`** | Headless CLI test execution & CI | `katalon-cli-runner`, `e2e-testing-patterns`, `flaky-test-sentinel-and-quarantine`, `backend-load-and-stress-patterns`, `github-actions-templates` | `matrix_sharder.py`, `distill_log.py`, `pinpoint_failure.py`, `flaky_detector.py`, `reversible_sandbox.py`, `synapse_qa_bench.py` |
|
|
107
|
+
| **`failure-analyst`**| Visual & XML error autopsy | `failure-autopsy-and-healing`, `self-healing-locator`, `file-and-report-validator` | `auto_heal.py`, `heal_pr_automator.py`, `pinpoint_failure.py`, `cache_resolver.py`, `auto_heal_reflexion.py` |
|
|
108
|
+
| **`quality-gatekeeper`**| 10 Gates DoD & Fake-Pass Sentinel | `antigravity-test-sentinel`, `code-refactoring-refactor-clean`, `wcag-audit-patterns`, `web-vitals-and-performance-profiler`, `security-dast-sentinel` | `verify_all.py`, `infer_linter.py`, `micro_mutator.py`, `trace_data_flow.py`, `skill_collision_guard.py`, `textgrad_mutation_optimizer.py` |
|
|
108
109
|
|
|
109
110
|
---
|
|
110
111
|
|
|
@@ -115,4 +116,31 @@ To ensure consistency across various distribution registries and developer inter
|
|
|
115
116
|
- **NPM Package & Antigravity Plugin:** `synapse-hub` (distributed on NPM Registry and Antigravity Marketplace)
|
|
116
117
|
- **Python Package & Universal CLI:** `agent-qa` (PEP 621 package on PyPI providing the `agent-qa` CLI binary)
|
|
117
118
|
- **MCP Server Protocol Services:** `enterprise-agent-qa` (Core QA MCP server) and `synapse-mcp-excel` (Excel tool server)
|
|
118
|
-
- **Skill Catalog Manifest:** `enterprise-agent-skills-hub` (internal catalog schema identifier in `skills_manifest.json`)
|
|
119
|
+
- **Skill Catalog Manifest:** `enterprise-agent-skills-hub` (internal catalog schema identifier in `skills_manifest.json`) and `agentskills.json` (AgentSkills v1 open standard)
|
|
120
|
+
|
|
121
|
+
---
|
|
122
|
+
|
|
123
|
+
## 7. Autonomous Agent Self-Evolution & Training Subsystems
|
|
124
|
+
|
|
125
|
+
To elevate AI testing beyond static prompt engineering, Synapse Hub incorporates 6 advanced agent training, compilation, and self-evolution mechanisms synthesized from modern agentic architectures:
|
|
126
|
+
|
|
127
|
+
```mermaid
|
|
128
|
+
flowchart TD
|
|
129
|
+
subgraph SelfEvolution["Autonomous Agent Self-Evolution Subsystem"]
|
|
130
|
+
Vault["<b>Voyager Lifelong Vault</b><br/>(skill_vault.py)<br/>• Verified PageClass SQLite memory<br/>• Zero repetitive prompt re-synthesis"]
|
|
131
|
+
Reflexion["<b>Episodic Reflexion Loop</b><br/>(auto_heal_reflexion.py)<br/>• K=3 trial-feedback learning<br/>• Self-healing without git pollution"]
|
|
132
|
+
DSPy["<b>DSPy Declarative QA Engine</b><br/>(dspy_qa_signatures.py)<br/>• Typed QA signatures<br/>• Eliminates brittle prompt hacking"]
|
|
133
|
+
TextGrad["<b>TextGrad Mutation Loss</b><br/>(textgrad_mutation_optimizer.py)<br/>• AST mutants as loss function<br/>• Optimizes assertions to score ≥ 80%"]
|
|
134
|
+
Reversible["<b>GoEx Reversible Sandbox</b><br/>(reversible_sandbox.py)<br/>• LIFO undo execution stack<br/>• Zero leftover test debris"]
|
|
135
|
+
Bench["<b>Gorilla BFCL Benchmark</b><br/>(synapse_qa_bench.py)<br/>• Evaluates function calling fidelity<br/>• Prevents agent regression"]
|
|
136
|
+
end
|
|
137
|
+
|
|
138
|
+
Vault --> DSPy --> TextGrad --> Reversible --> Reflexion --> Bench
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
1. **Lifelong Skill Memory (`skill_vault.py`):** Inspired by Voyager, stores verified POM methods and complex interaction patterns in permanent SQLite memory, growing the agent's capability with every run.
|
|
142
|
+
2. **Episodic Trial-Feedback (`auto_heal_reflexion.py`):** Implements Noah Shinn's Reflexion framework, enabling agents to evaluate test failure traces across $K=3$ iterations, generating textual self-reflection to correct locator strategies.
|
|
143
|
+
3. **Declarative Signature Compilation (`dspy_qa_signatures.py`):** Adheres to Stanford DSPy principles, compiling prompt templates into structured, typed input-output signatures.
|
|
144
|
+
4. **Mutant-Directed Loss Optimization (`textgrad_mutation_optimizer.py`):** Treats surviving AST mutants as a loss function (TextGrad), iteratively refining assertion logic until all mutants are killed.
|
|
145
|
+
5. **Reversible Execution Teardown (`reversible_sandbox.py`):** Implements GoEx-style reversible rollback stacks for filesystem and database mutations during automated exploratory runs.
|
|
146
|
+
6. **Tool Function Calling Benchmark (`synapse_qa_bench.py`):** Implements Berkeley Function-Calling Leaderboard (BFCL) standards for enterprise QA tools, benchmarking tool selection precision and multi-turn execution accuracy.
|
|
@@ -85,3 +85,12 @@ Local model gateway (default `127.0.0.1:3456`) with routing, retry, failover, an
|
|
|
85
85
|
|
|
86
86
|
- **Security note:** it is a man-in-the-middle for API keys — run strictly local, never expose the port, and audit its config before pointing production credentials at it.
|
|
87
87
|
- Hub use: route mechanical subagent work to cheaper models while keeping strong models for synthesis/review; its request logs match the hub telemetry fields (token in/out, cost).
|
|
88
|
+
|
|
89
|
+
## 9. ponytail (DietrichGebert/ponytail) — MIT
|
|
90
|
+
|
|
91
|
+
Lazy senior developer plugin for Claude Code and agentic coding workflows.
|
|
92
|
+
|
|
93
|
+
- **Provenance:** Created by DietrichGebert (<https://github.com/DietrichGebert/ponytail>) under the MIT License.
|
|
94
|
+
- **Hub Inspiration:** Core concepts adapted into Synapse-Hub include the 7-rung reuse ladder (`lean-code-ladder`), technical debt harvesting (`technical-debt-harvester` & `tools/debt_harvester.py`), 6-pillar code review (`senior-code-review` & `tools/senior_code_review.py`), and whole-repo audit (`repo-quality-audit`).
|
|
95
|
+
- **Hub Adaptations:** Re-scoped specifically for enterprise automated testing suites, Clean POM architecture, and 0-token AST verification.
|
|
96
|
+
|