synapse-hub 2.4.1 → 2.4.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (77) hide show
  1. package/.agents/rules/graphify.md +14 -14
  2. package/.agents/rules/mandatory-playwright-live.md +29 -29
  3. package/.agents/skills/bats-testing-patterns/resources/implementation-playbook.md +614 -614
  4. package/.agents/skills/clean-pom-crafter/examples/standard_test_case.groovy +28 -28
  5. package/.agents/skills/clean-pom-crafter/references/pom_patterns.groovy +138 -138
  6. package/.agents/skills/e2e-testing-patterns/resources/implementation-playbook.md +522 -522
  7. package/.agents/skills/failure-autopsy-and-healing/references/healing_matrix.md +64 -64
  8. package/.agents/skills/failure-autopsy-and-healing/references/qa_investigation_protocol.md +56 -56
  9. package/.agents/skills/failure-autopsy-and-healing/scripts/triage_failures.sh +98 -98
  10. package/.agents/skills/katalon-cli-runner/references/groovy-patterns.md +209 -209
  11. package/.agents/skills/katalon-cli-runner/references/katalon-api.md +155 -155
  12. package/.agents/skills/katalon-cli-runner/references/runner_commands.md +67 -67
  13. package/.agents/skills/playwright-live-interaction/references/advanced_superpowers.md +160 -160
  14. package/.agents/skills/playwright-live-interaction/references/compact_accessibility_snapshots.md +75 -75
  15. package/.agents/skills/playwright-live-interaction/references/recipes.md +388 -388
  16. package/.agents/skills/playwright-live-interaction/references/swarm_orchestration.md +134 -134
  17. package/.agents/skills/wcag-audit-patterns/resources/implementation-playbook.md +541 -541
  18. package/.agents/templates/completion_report.md +57 -57
  19. package/.agents/workflows/graphify.md +10 -10
  20. package/.cursor/rules/language.mdc +7 -7
  21. package/README.en.md +1 -1
  22. package/README.md +1 -1
  23. package/benchmarks/qa_eval/dataset/api_contract_drift/qa-challenge-06_api_contract_drift.json +21 -21
  24. package/benchmarks/qa_eval/dataset/api_contract_drift/qa-challenge-07_api_contract_drift.json +21 -21
  25. package/benchmarks/qa_eval/dataset/api_contract_drift/qa-challenge-08_api_contract_drift.json +21 -21
  26. package/benchmarks/qa_eval/dataset/api_contract_drift/qa-challenge-09_api_contract_drift.json +21 -21
  27. package/benchmarks/qa_eval/dataset/api_contract_drift/qa-challenge-10_api_contract_drift.json +21 -21
  28. package/benchmarks/qa_eval/dataset/fake_pass_tautology/qa-challenge-21_fake_pass_tautology.json +20 -20
  29. package/benchmarks/qa_eval/dataset/fake_pass_tautology/qa-challenge-22_fake_pass_tautology.json +20 -20
  30. package/benchmarks/qa_eval/dataset/fake_pass_tautology/qa-challenge-23_fake_pass_tautology.json +20 -20
  31. package/benchmarks/qa_eval/dataset/fake_pass_tautology/qa-challenge-24_fake_pass_tautology.json +20 -20
  32. package/benchmarks/qa_eval/dataset/fake_pass_tautology/qa-challenge-25_fake_pass_tautology.json +20 -20
  33. package/benchmarks/qa_eval/dataset/locator_drift/qa-challenge-01_locator_drift.json +22 -22
  34. package/benchmarks/qa_eval/dataset/locator_drift/qa-challenge-02_locator_drift.json +22 -22
  35. package/benchmarks/qa_eval/dataset/locator_drift/qa-challenge-03_locator_drift.json +22 -22
  36. package/benchmarks/qa_eval/dataset/locator_drift/qa-challenge-04_locator_drift.json +22 -22
  37. package/benchmarks/qa_eval/dataset/locator_drift/qa-challenge-05_locator_drift.json +22 -22
  38. package/benchmarks/qa_eval/dataset/stale_and_mutation/qa-challenge-16_stale_and_mutation.json +21 -21
  39. package/benchmarks/qa_eval/dataset/stale_and_mutation/qa-challenge-17_stale_and_mutation.json +21 -21
  40. package/benchmarks/qa_eval/dataset/stale_and_mutation/qa-challenge-18_stale_and_mutation.json +21 -21
  41. package/benchmarks/qa_eval/dataset/stale_and_mutation/qa-challenge-19_stale_and_mutation.json +21 -21
  42. package/benchmarks/qa_eval/dataset/stale_and_mutation/qa-challenge-20_stale_and_mutation.json +22 -22
  43. package/benchmarks/qa_eval/dataset/timing_and_race/qa-challenge-11_timing_and_race.json +21 -21
  44. package/benchmarks/qa_eval/dataset/timing_and_race/qa-challenge-12_timing_and_race.json +21 -21
  45. package/benchmarks/qa_eval/dataset/timing_and_race/qa-challenge-13_timing_and_race.json +21 -21
  46. package/benchmarks/qa_eval/dataset/timing_and_race/qa-challenge-14_timing_and_race.json +21 -21
  47. package/benchmarks/qa_eval/dataset/timing_and_race/qa-challenge-15_timing_and_race.json +21 -21
  48. package/bin/agent-qa.js +0 -0
  49. package/docs/ARCHITECTURE.md +12 -1
  50. package/package.json +1 -1
  51. package/scripts/install.ps1 +112 -112
  52. package/scripts/lint_skills.ps1 +57 -57
  53. package/skills_manifest.json +1 -1
  54. package/synapse.config.json +1 -1
  55. package/tools/__init__.py +1 -1
  56. package/tools/agent_canary.py +501 -501
  57. package/tools/api_fuzzer.py +11 -2
  58. package/tools/browser_helpers.js +2 -2
  59. package/tools/bump_version.py +1 -0
  60. package/tools/check_env.py +71 -13
  61. package/tools/cli.py +2 -2
  62. package/tools/dashboard_generator.py +2 -2
  63. package/tools/e2e_flow_prober.sh +0 -0
  64. package/tools/generate_allure_report.sh +0 -0
  65. package/tools/generate_test_reports.sh +0 -0
  66. package/tools/mcp_excel_server.py +3 -1
  67. package/tools/mcp_excel_server.sh +0 -0
  68. package/tools/mcp_server.py +1 -1
  69. package/tools/pinpoint_failure.py +88 -88
  70. package/tools/setup_project.py +8 -0
  71. package/tools/update_sentinel.py +10 -9
  72. package/tools/verify_all.sh +0 -0
  73. package/tools/visual_som_helper.py +14 -3
  74. package/tools/welcome_page.py +1 -1
  75. package/benchmarks/qa_eval/reports/AB_METHODOLOGY_REPORT.md +0 -83
  76. package/benchmarks/qa_eval/reports/BENCHMARK_REPORT.md +0 -69
  77. package/benchmarks/qa_eval/reports/eval_results.json +0 -105
@@ -115,12 +115,21 @@ class ApiFuzzer:
115
115
  parsed = urllib.parse.urlparse(self.base_url)
116
116
  host = (parsed.hostname or "").lower().strip()
117
117
  is_safe_env = self._is_safe_non_prod_host(host)
118
- if host and not is_safe_env and not os.environ.get("SYNAPSE_ALLOW_PROD_FUZZ"):
118
+ allowed_hosts = {
119
+ h.strip().lower() for h in os.environ.get("SYNAPSE_ALLOW_PROD_HOSTS", "").split(",") if h.strip()
120
+ }
121
+ is_host_specifically_allowed = host in allowed_hosts
122
+ global_override = os.environ.get("SYNAPSE_ALLOW_PROD_FUZZ") in ("1", "true", "TRUE")
123
+
124
+ if host and not is_safe_env and not (is_host_specifically_allowed or global_override):
119
125
  raise PermissionError(
120
126
  f"[SAFETY FIREWALL - REJECTED] Target host '{host}' is not in the allowed non-production environment allowlist "
121
127
  "(localhost, private IP, .local/.internal, or verified dev/staging/qa/test subdomain). "
122
- "Set SYNAPSE_ALLOW_PROD_FUZZ=1 only if authorized."
128
+ f"To authorize this host specifically, set SYNAPSE_ALLOW_PROD_HOSTS='{host}' or SYNAPSE_ALLOW_PROD_FUZZ=1 if authorized."
123
129
  )
130
+ if host and not is_safe_env and global_override and not is_host_specifically_allowed:
131
+ print(f" ⚠️ [SECURITY WARNING] SYNAPSE_ALLOW_PROD_FUZZ=1 global override active for host '{host}'. "
132
+ f"Prefer setting SYNAPSE_ALLOW_PROD_HOSTS='{host}' for host-scoped permission.")
124
133
 
125
134
  # Prepare SSL context (support self-signed staging certificates via explicit --insecure flag)
126
135
  self.ssl_context = ssl.create_default_context()
@@ -13,7 +13,7 @@
13
13
  (function () {
14
14
  'use strict';
15
15
 
16
- if (window.__qa && window.__qa.__version === '2.4.1') {
16
+ if (window.__qa && window.__qa.__version === '2.4.3') {
17
17
  return window.__qa;
18
18
  }
19
19
 
@@ -47,7 +47,7 @@
47
47
  }
48
48
 
49
49
  const qa = {
50
- __version: '2.4.1',
50
+ __version: '2.4.3',
51
51
 
52
52
  /**
53
53
  * 1. Smart Action Batching
@@ -196,6 +196,7 @@ def main():
196
196
  update_file("tools/mcp_excel_server.ps1", r'(version\s*=\s*")[^"]+(")', rf"\g<1>{new_ver}\g<2>", args.dry_run)
197
197
 
198
198
  # 7b. tools/mcp_excel_server.py
199
+ update_file("tools/mcp_excel_server.py", r'(SERVER_VERSION\s*=\s*")[^"]+(")', rf"\g<1>{new_ver}\g<2>", args.dry_run)
199
200
  update_file("tools/mcp_excel_server.py", r'("serverInfo":\s*\{[^}]*"version":\s*")[^"]+(")', rf"\g<1>{new_ver}\g<2>", args.dry_run)
200
201
 
201
202
  # 8. docs/ARCHITECTURE.md
@@ -75,6 +75,7 @@ class DiagnosticRunner:
75
75
  self.results = []
76
76
  self.warnings = []
77
77
  self.errors = []
78
+ self.is_standalone = False
78
79
 
79
80
  def log_check(self, name, status, message, fix_hint=""):
80
81
  badge = f"{GREEN}[PASS]{RESET}" if status == "PASS" else (f"{YELLOW}[WARN]{RESET}" if status == "WARN" else f"{RED}[FAIL]{RESET}")
@@ -206,8 +207,8 @@ class DiagnosticRunner:
206
207
  self.log_check("Python Virtual Environment", "PASS", f"Python {py_ver} (Optional: create .venv if using auxiliary runners)")
207
208
 
208
209
  def check_project_structure(self):
210
+ # 1. QA Hub Repository Check
209
211
  is_hub = (self.target_dir / "skills_manifest.json").exists() or (self.target_dir / ".agents" / "skills").is_dir()
210
-
211
212
  if is_hub:
212
213
  required_dirs = [".agents/skills", ".agents/rules", "tools", "tests"]
213
214
  missing = [rd for rd in required_dirs if not (self.target_dir / rd).exists()]
@@ -220,24 +221,78 @@ class DiagnosticRunner:
220
221
  "Restore from official agent-skills Git repository.")
221
222
  return
222
223
 
223
- is_playwright = (self.target_dir / "playwright.config.ts").exists() or (self.target_dir / "playwright.config.js").exists()
224
+ # 2. Standalone Host / User Home Detection
225
+ is_user_home = False
226
+ try:
227
+ is_user_home = self.target_dir.resolve() == Path.home().resolve()
228
+ except Exception:
229
+ pass
230
+
231
+ has_project_marker = any([
232
+ (self.target_dir / marker).exists() for marker in [
233
+ ".git", "package.json", "requirements.txt", "pom.xml", "build.gradle",
234
+ "pytest.ini", "cypress.config.ts", "cypress.config.js",
235
+ "playwright.config.ts", "playwright.config.js", "Keywords", "Scripts"
236
+ ]
237
+ ]) or bool(list(self.target_dir.glob("*.prj")))
238
+
239
+ if is_user_home or not has_project_marker:
240
+ self.is_standalone = True
241
+ self.log_check("Project Structure (Standalone)", "PASS",
242
+ "Host machine environment verified. (To diagnose a specific test project: agent-qa doctor <path>)")
243
+ return
244
+
245
+ # 3. Playwright Check
246
+ pkg_json = self.target_dir / "package.json"
247
+ has_playwright_pkg = pkg_json.exists() and "@playwright/test" in pkg_json.read_text(encoding="utf-8", errors="ignore")
248
+ is_playwright = (self.target_dir / "playwright.config.ts").exists() or (self.target_dir / "playwright.config.js").exists() or has_playwright_pkg
224
249
  if is_playwright:
225
- required_dirs = ["tests"]
250
+ has_tests = (self.target_dir / "tests").is_dir() or (self.target_dir / "e2e").is_dir() or (self.target_dir / "src").is_dir()
251
+ if has_tests:
252
+ self.log_check("Project Structure (Playwright)", "PASS", "Playwright test project structure verified")
253
+ else:
254
+ self.log_check("Project Structure (Playwright)", "WARN", "Missing 'tests/' or 'e2e/' directory for Playwright specs",
255
+ "Create a 'tests/' directory or run 'npx playwright test --init'")
256
+ return
257
+
258
+ # 4. Cypress Check
259
+ is_cypress = (self.target_dir / "cypress.config.ts").exists() or (self.target_dir / "cypress.config.js").exists() or (self.target_dir / "cypress").is_dir()
260
+ if is_cypress:
261
+ if (self.target_dir / "cypress").is_dir():
262
+ self.log_check("Project Structure (Cypress)", "PASS", "Cypress test project structure verified")
263
+ else:
264
+ self.log_check("Project Structure (Cypress)", "WARN", "Missing 'cypress/' directory",
265
+ "Create a 'cypress/' directory for E2E specs")
266
+ return
267
+
268
+ # 5. Pytest / Python Selenium Check
269
+ is_pytest = (self.target_dir / "pytest.ini").exists() or (self.target_dir / "conftest.py").exists()
270
+ has_py_tests = (self.target_dir / "tests").is_dir() or bool(list(self.target_dir.glob("test_*.py")))
271
+ if is_pytest or (has_py_tests and (self.target_dir / "requirements.txt").exists()):
272
+ self.log_check("Project Structure (Pytest)", "PASS", "Python/Pytest test project structure verified")
273
+ return
274
+
275
+ # 6. Katalon Studio (Enterprise Web Clean POM)
276
+ is_katalon = bool(list(self.target_dir.glob("*.prj"))) or (self.target_dir / "Keywords").is_dir() or (self.target_dir / "Scripts").is_dir()
277
+ if is_katalon:
278
+ required_dirs = ["Scripts", "Keywords", "Profiles"]
226
279
  missing = [rd for rd in required_dirs if not (self.target_dir / rd).exists()]
227
280
  if not missing:
228
- self.log_check("Project Structure (Playwright)", "PASS", "Playwright project structure verified")
281
+ self.log_check("Project Structure (Clean POM)", "PASS", "Clean POM structure verified (Keywords, Profiles, Scripts)")
229
282
  else:
230
- self.log_check("Project Structure (Playwright)", "FAIL", f"Missing directories: {', '.join(missing)}")
283
+ self.log_check("Project Structure (POM)", "WARN" if len(missing) < len(required_dirs) else "FAIL",
284
+ f"Missing directories: {', '.join(missing)}",
285
+ "Verify git clone command or select target test project directory.")
231
286
  return
232
287
 
233
- required_dirs = ["Scripts", "Keywords", "Profiles"]
234
- missing = [rd for rd in required_dirs if not (self.target_dir / rd).exists()]
235
- if not missing:
236
- self.log_check("Project Structure (Clean POM)", "PASS", "Clean POM structure verified (Keywords, Profiles, Scripts)")
237
- else:
238
- self.log_check("Project Structure (POM)", "WARN" if len(missing) < len(required_dirs) else "FAIL",
239
- f"Missing directories: {', '.join(missing)}",
240
- "Verify git clone command or select target test project directory.")
288
+ # 7. Generic Test Project (any project with tests/ or test/)
289
+ if (self.target_dir / "tests").is_dir() or (self.target_dir / "test").is_dir() or (self.target_dir / "e2e").is_dir():
290
+ self.log_check("Project Structure (Automated Tests)", "PASS", "Standard automated test directory detected")
291
+ return
292
+
293
+ # Fallback: General project structure missing test suite
294
+ self.log_check("Project Structure", "WARN", "No automated test directories detected (tests/, Scripts/, cypress/)",
295
+ "Run 'agent-qa setup .' to initialize Clean POM / Playwright configuration for this project.")
241
296
 
242
297
  def check_network(self):
243
298
  env_file = self.target_dir / ".env"
@@ -253,6 +308,9 @@ class DiagnosticRunner:
253
308
 
254
309
  base = os.environ.get("BASE_URL", "").rstrip("/")
255
310
  if not base:
311
+ if getattr(self, "is_standalone", False):
312
+ self.log_check("Network / Server Base URL", "PASS", "Host standalone check (configure BASE_URL in project .env when targeting an AUT)")
313
+ return
256
314
  self.log_check("Network / Server Base URL", "WARN", "BASE_URL not configured in .env",
257
315
  "Copy .env.example to .env and configure BASE_URL to verify connectivity.")
258
316
  return
package/tools/cli.py CHANGED
@@ -41,7 +41,7 @@ RESET = "\033[0m"
41
41
  def cmd_version(args):
42
42
  """Displays the current hub version, skill catalog count, and license information."""
43
43
  manifest_path = os.path.join(REPO_ROOT, "skills_manifest.json")
44
- version = "2.4.1"
44
+ version = "2.4.3"
45
45
  if os.path.exists(manifest_path):
46
46
  try:
47
47
  import json
@@ -516,7 +516,7 @@ def cmd_mcp_excel(args):
516
516
 
517
517
  def _hub_version():
518
518
  manifest_path = os.path.join(REPO_ROOT, "skills_manifest.json")
519
- version = "2.4.1"
519
+ version = "2.4.3"
520
520
  if os.path.exists(manifest_path):
521
521
  try:
522
522
  import json
@@ -1885,7 +1885,7 @@ def generate_hub_html(
1885
1885
 
1886
1886
  translations_vi = {
1887
1887
  "header_title": "Bảng Điều Hành Chất Lượng Doanh Nghiệp",
1888
- "header_subtitle": "Synapse-Hub v2.4.1 &bull; Hệ Sinh Thái Kiểm Thử AI Agent Tự Chủ",
1888
+ "header_subtitle": "Synapse-Hub v2.4.2 &bull; Hệ Sinh Thái Kiểm Thử AI Agent Tự Chủ",
1889
1889
  "nav_welcome": "Giới Thiệu",
1890
1890
  "meta_mode": "🏷️ Chế độ: <strong>HỆ SINH THÁI QA HUB</strong>",
1891
1891
  "meta_gen": f"🕒 Khởi tạo: <strong>{now_str}</strong>",
@@ -1963,7 +1963,7 @@ def generate_hub_html(
1963
1963
 
1964
1964
  translations_en = {
1965
1965
  "header_title": "Enterprise QA Executive Dashboard",
1966
- "header_subtitle": "Synapse-Hub v2.4.1 &bull; Autonomous AI Agent Testing Ecosystem",
1966
+ "header_subtitle": "Synapse-Hub v2.4.2 &bull; Autonomous AI Agent Testing Ecosystem",
1967
1967
  "nav_welcome": "Welcome",
1968
1968
  "meta_mode": "🏷️ Mode: <strong>QA HUB ECOSYSTEM</strong>",
1969
1969
  "meta_gen": f"🕒 Generated: <strong>{now_str}</strong>",
File without changes
File without changes
File without changes
@@ -23,6 +23,8 @@ if hasattr(sys.stdout, 'reconfigure'):
23
23
  except Exception:
24
24
  pass
25
25
 
26
+ SERVER_VERSION = "2.4.3"
27
+
26
28
  NS_MAIN = "{http://schemas.openxmlformats.org/spreadsheetml/2006/main}"
27
29
  NS_REL = "{http://schemas.openxmlformats.org/officeDocument/2006/relationships}"
28
30
 
@@ -194,7 +196,7 @@ def process_message(msg: Dict[str, Any]) -> Dict[str, Any]:
194
196
  "result": {
195
197
  "protocolVersion": "2024-11-05",
196
198
  "capabilities": {"tools": {}},
197
- "serverInfo": {"name": "synapse-mcp-excel", "version": "2.4.1"}
199
+ "serverInfo": {"name": "synapse-mcp-excel", "version": SERVER_VERSION}
198
200
  }
199
201
  }
200
202
  elif method == "tools/list":
File without changes
@@ -47,7 +47,7 @@ import gherkin_suite
47
47
  import api_fuzzer
48
48
 
49
49
  SERVER_NAME = "enterprise-agent-qa"
50
- SERVER_VERSION = "2.4.1"
50
+ SERVER_VERSION = "2.4.3"
51
51
 
52
52
  MCP_TOOLS = [
53
53
  {
@@ -1,88 +1,88 @@
1
- #!/usr/bin/env python3
2
- """
3
- Fast Zero-Token Failure Pinpointer for Katalon & Jenkins Reports.
4
- Scans the latest Reports/**/JUnit_Report.xml, extracts failed Test Cases,
5
- exact failed lines in Groovy/Script, and the root-cause exception.
6
- Outputs < 100 tokens.
7
-
8
- Usage:
9
- python3 tools/pinpoint_failure.py
10
- python3 tools/pinpoint_failure.py path/to/JUnit_Report.xml
11
- """
12
- import os
13
- import sys
14
- import glob
15
- import xml.etree.ElementTree as ET
16
- import re
17
-
18
- def find_latest_report():
19
- reports = glob.glob("Reports/**/JUnit_Report.xml", recursive=True)
20
- if not reports:
21
- return None
22
- reports.sort(key=lambda x: os.path.getmtime(x), reverse=True)
23
- return reports[0]
24
-
25
- def parse_failures(xml_path):
26
- if not os.path.exists(xml_path):
27
- print(f"Error: {xml_path} does not exist.")
28
- return []
29
-
30
- tree = ET.parse(xml_path)
31
- root = tree.getroot()
32
- failures = []
33
-
34
- suites = [root] if root.tag == "testsuite" else root.findall("testsuite")
35
-
36
- for suite in suites:
37
- for tc in suite.findall("testcase"):
38
- tc_name = tc.get("name", "Unknown")
39
- tc_time = tc.get("time", "0")
40
- fail = tc.find("failure")
41
- err = tc.find("error")
42
- target = fail if fail is not None else err
43
-
44
- if target is not None:
45
- msg = target.get("message", "")
46
- text = target.text or ""
47
- combined = (msg + "\n" + text).replace("&#xa;", "\n")
48
-
49
- # Extract Root Cause Reason
50
- reason_match = re.search(r"Reason:\s*(.+?)(?=\n\s*at|\Z)", combined, re.DOTALL)
51
- if reason_match:
52
- reason = reason_match.group(1).strip()
53
- else:
54
- lines = [l.strip() for l in combined.split("\n") if l.strip()]
55
- reason = lines[0] if lines else "Unknown Error"
56
-
57
- # Extract Failed Line in Groovy Page / Script
58
- Enterprise_lines = re.findall(r"(?:Enterprise\.pages\.[\w]+|TC-[\w\s\u00C0-\u1EF9]+|Script\d+)\.groovy:\d+|(?:TC-[\w\s\u00C0-\u1EF9]+:\d+)", combined)
59
-
60
- failures.append({
61
- "tc": tc_name.split("/")[-1],
62
- "time": f"{float(tc_time):.1f}s",
63
- "reason": reason.split("\n")[0][:180],
64
- "failed_at": Enterprise_lines[:2] if Enterprise_lines else ["See JUnit report"]
65
- })
66
-
67
- return failures
68
-
69
- def main():
70
- xml_path = sys.argv[1] if len(sys.argv) > 1 else find_latest_report()
71
- if not xml_path:
72
- print("No JUnit_Report.xml found in Reports/.")
73
- sys.exit(0)
74
-
75
- print(f"=== Report: {xml_path} ===")
76
- failures = parse_failures(xml_path)
77
- if not failures:
78
- print("✅ 100% PASSED: Zero failures found in report.")
79
- return
80
-
81
- print(f"❌ Found {len(failures)} Failed Test Case(s):")
82
- for i, f in enumerate(failures, 1):
83
- print(f"[{i}] {f['tc']} ({f['time']})")
84
- print(f" Cause: {f['reason']}")
85
- print(f" Line: {', '.join(f['failed_at'])}")
86
-
87
- if __name__ == "__main__":
88
- main()
1
+ #!/usr/bin/env python3
2
+ """
3
+ Fast Zero-Token Failure Pinpointer for Katalon & Jenkins Reports.
4
+ Scans the latest Reports/**/JUnit_Report.xml, extracts failed Test Cases,
5
+ exact failed lines in Groovy/Script, and the root-cause exception.
6
+ Outputs < 100 tokens.
7
+
8
+ Usage:
9
+ python3 tools/pinpoint_failure.py
10
+ python3 tools/pinpoint_failure.py path/to/JUnit_Report.xml
11
+ """
12
+ import os
13
+ import sys
14
+ import glob
15
+ import xml.etree.ElementTree as ET
16
+ import re
17
+
18
+ def find_latest_report():
19
+ reports = glob.glob("Reports/**/JUnit_Report.xml", recursive=True)
20
+ if not reports:
21
+ return None
22
+ reports.sort(key=lambda x: os.path.getmtime(x), reverse=True)
23
+ return reports[0]
24
+
25
+ def parse_failures(xml_path):
26
+ if not os.path.exists(xml_path):
27
+ print(f"Error: {xml_path} does not exist.")
28
+ return []
29
+
30
+ tree = ET.parse(xml_path)
31
+ root = tree.getroot()
32
+ failures = []
33
+
34
+ suites = [root] if root.tag == "testsuite" else root.findall("testsuite")
35
+
36
+ for suite in suites:
37
+ for tc in suite.findall("testcase"):
38
+ tc_name = tc.get("name", "Unknown")
39
+ tc_time = tc.get("time", "0")
40
+ fail = tc.find("failure")
41
+ err = tc.find("error")
42
+ target = fail if fail is not None else err
43
+
44
+ if target is not None:
45
+ msg = target.get("message", "")
46
+ text = target.text or ""
47
+ combined = (msg + "\n" + text).replace("&#xa;", "\n")
48
+
49
+ # Extract Root Cause Reason
50
+ reason_match = re.search(r"Reason:\s*(.+?)(?=\n\s*at|\Z)", combined, re.DOTALL)
51
+ if reason_match:
52
+ reason = reason_match.group(1).strip()
53
+ else:
54
+ lines = [l.strip() for l in combined.split("\n") if l.strip()]
55
+ reason = lines[0] if lines else "Unknown Error"
56
+
57
+ # Extract Failed Line in Groovy Page / Script
58
+ Enterprise_lines = re.findall(r"(?:Enterprise\.pages\.[\w]+|TC-[\w\s\u00C0-\u1EF9]+|Script\d+)\.groovy:\d+|(?:TC-[\w\s\u00C0-\u1EF9]+:\d+)", combined)
59
+
60
+ failures.append({
61
+ "tc": tc_name.split("/")[-1],
62
+ "time": f"{float(tc_time):.1f}s",
63
+ "reason": reason.split("\n")[0][:180],
64
+ "failed_at": Enterprise_lines[:2] if Enterprise_lines else ["See JUnit report"]
65
+ })
66
+
67
+ return failures
68
+
69
+ def main():
70
+ xml_path = sys.argv[1] if len(sys.argv) > 1 else find_latest_report()
71
+ if not xml_path:
72
+ print("No JUnit_Report.xml found in Reports/.")
73
+ sys.exit(0)
74
+
75
+ print(f"=== Report: {xml_path} ===")
76
+ failures = parse_failures(xml_path)
77
+ if not failures:
78
+ print("✅ 100% PASSED: Zero failures found in report.")
79
+ return
80
+
81
+ print(f"❌ Found {len(failures)} Failed Test Case(s):")
82
+ for i, f in enumerate(failures, 1):
83
+ print(f"[{i}] {f['tc']} ({f['time']})")
84
+ print(f" Cause: {f['reason']}")
85
+ print(f" Line: {', '.join(f['failed_at'])}")
86
+
87
+ if __name__ == "__main__":
88
+ main()
@@ -394,6 +394,14 @@ def inject_antigravity_global_plugin():
394
394
  if not os.path.exists(dst_readme) or not os.path.samefile(src_readme, dst_readme):
395
395
  shutil.copy2(src_readme, dst_readme)
396
396
 
397
+ # 3. LICENSE & NOTICE (MIT License Provenance & Author Attribution)
398
+ for meta_file in ("LICENSE", "NOTICE"):
399
+ src_meta = os.path.join(HUB_ROOT, meta_file)
400
+ dst_meta = os.path.join(target_plugin_dir, meta_file)
401
+ if os.path.exists(src_meta):
402
+ if not os.path.exists(dst_meta) or not os.path.samefile(src_meta, dst_meta):
403
+ shutil.copy2(src_meta, dst_meta)
404
+
397
405
  # Helper to sync directory (junction or copy)
398
406
  def sync_dir(src, dst):
399
407
  if not os.path.exists(src):
@@ -65,7 +65,7 @@ def get_local_version():
65
65
  try:
66
66
  with open(pkg_file, "r", encoding="utf-8") as f:
67
67
  data = json.load(f)
68
- return data.get("version", "2.4.1")
68
+ return data.get("version", "2.4.3")
69
69
  except (OSError, json.JSONDecodeError):
70
70
  pass
71
71
 
@@ -74,11 +74,11 @@ def get_local_version():
74
74
  try:
75
75
  with open(manifest_file, "r", encoding="utf-8") as f:
76
76
  data = json.load(f)
77
- return data.get("version", "2.4.1")
77
+ return data.get("version", "2.4.3")
78
78
  except (OSError, json.JSONDecodeError):
79
79
  pass
80
80
 
81
- return "2.4.1"
81
+ return "2.4.3"
82
82
 
83
83
 
84
84
  def fetch_remote_version():
@@ -321,14 +321,15 @@ def perform_update():
321
321
  except (subprocess.SubprocessError, OSError):
322
322
  pass
323
323
  else:
324
- # Refresh plugin.json in global plugin
324
+ # Refresh metadata files in global plugin
325
325
  try:
326
326
  import shutil
327
- src_pkg = os.path.join(HUB_ROOT, "plugin.json")
328
- dst_pkg = os.path.join(GLOBAL_PLUGIN_DIR, "plugin.json")
329
- if os.path.exists(src_pkg):
330
- shutil.copy2(src_pkg, dst_pkg)
331
- print(f" {GREEN}[OK] Refreshed plugin.json in: {GLOBAL_PLUGIN_DIR}{RESET}")
327
+ for meta_f in ("plugin.json", "LICENSE", "NOTICE", "README.md"):
328
+ src_f = os.path.join(HUB_ROOT, meta_f)
329
+ dst_f = os.path.join(GLOBAL_PLUGIN_DIR, meta_f)
330
+ if os.path.exists(src_f):
331
+ shutil.copy2(src_f, dst_f)
332
+ print(f" {GREEN}[OK] Refreshed metadata & LICENSE in: {GLOBAL_PLUGIN_DIR}{RESET}")
332
333
  except OSError:
333
334
  pass
334
335
 
File without changes
@@ -1,4 +1,5 @@
1
1
  #!/usr/bin/env python3
2
+ from __future__ import annotations
2
3
  """
3
4
  Enterprise Vision-Grounded Fallback & Set-of-Marks (SoM) Engine
4
5
  --------------------------------------------------------------
@@ -36,6 +37,11 @@ except ImportError:
36
37
  ImageFilter = None
37
38
  HAS_PIL = False
38
39
 
40
+ if HAS_PIL and Image is not None:
41
+ ImageSourceType = Union[Image.Image, str]
42
+ else:
43
+ ImageSourceType = Any
44
+
39
45
  # Enforce UTF-8 stdout encoding for Windows console compatibility
40
46
  if hasattr(sys.stdout, 'reconfigure'):
41
47
  try:
@@ -58,11 +64,13 @@ MARK_COLORS = [
58
64
  ]
59
65
 
60
66
 
61
- def compute_visual_hash(image_source: Union[Image.Image, str], bbox: Optional[List[int]] = None, hash_size: int = 8) -> str:
67
+ def compute_visual_hash(image_source: ImageSourceType, bbox: Optional[List[int]] = None, hash_size: int = 8) -> str:
62
68
  """
63
69
  Computes a 64-bit difference hash (dHash) for an image or cropped component.
64
70
  Fast, deterministic, and 100% offline.
65
71
  """
72
+ if not HAS_PIL or Image is None:
73
+ raise ImportError("Pillow is required for visual perceptual operations (pip install Pillow).")
66
74
  if isinstance(image_source, str):
67
75
  if not os.path.exists(image_source):
68
76
  raise FileNotFoundError(f"Image not found: {image_source}")
@@ -135,7 +143,7 @@ class VisualSoMHelper:
135
143
  def lookup_cached_coordinates(
136
144
  self,
137
145
  component_name: str,
138
- image_source: Union[Image.Image, str],
146
+ image_source: ImageSourceType,
139
147
  bbox: Optional[List[int]] = None,
140
148
  threshold: float = 0.98,
141
149
  db_path: Optional[str] = None
@@ -198,7 +206,7 @@ class VisualSoMHelper:
198
206
  def save_cached_coordinates(
199
207
  self,
200
208
  component_name: str,
201
- image_source: Union[Image.Image, str],
209
+ image_source: ImageSourceType,
202
210
  center_x: int,
203
211
  center_y: int,
204
212
  bbox: Optional[List[int]] = None,
@@ -324,6 +332,9 @@ class VisualSoMHelper:
324
332
  If elements is omitted, autonomously runs detect_canvas_interactive_regions.
325
333
  Returns (marked_image_path, marks_map).
326
334
  """
335
+ if not HAS_PIL or Image is None:
336
+ raise ImportError("Pillow is required for Set-of-Marks overlay generation (pip install Pillow).")
337
+
327
338
  if not os.path.exists(image_path):
328
339
  img = Image.new("RGB", (self.viewport_width, self.viewport_height), color="#1E1E2E")
329
340
  draw = ImageDraw.Draw(img)
@@ -48,7 +48,7 @@ def get_hub_metadata() -> Dict[str, Any]:
48
48
  """Retrieves version and catalog count from skills_manifest.json."""
49
49
  manifest_path = os.path.join(HUB_ROOT, "skills_manifest.json")
50
50
  meta = {
51
- "version": "2.4.1",
51
+ "version": "2.4.3",
52
52
  "totalSkills": 38,
53
53
  "author": "Thế Vị (@TeeVee998)",
54
54
  "repo": "https://github.com/TeeVee998/Synapse-Hub"
@@ -1,83 +0,0 @@
1
- # A/B Empirical Evaluation Methodology & Replication Report
2
-
3
- > **Dataset Scope:** 25 Production Challenges across 5 Dynamic QA Failure Vectors
4
- > **Replication Seed:** `42` | **Temperature:** `0.0` | **Deterministic Execution Mode**
5
- > **Evaluation Harness:** `benchmarks/qa_eval/eval_runner.py`
6
- > **Author & Provenance:** Thế Vị (@TeeVee998) — [Synapse Hub](https://github.com/TeeVee998/Synapse-Hub)
7
-
8
- ---
9
-
10
- ## 1. Executive Summary
11
-
12
- This report establishes the empirical A/B testing methodology comparing **Naive LLM Prompting (Control Group)** against **Synapse Hub Multi-Agent Swarm (Treatment Group)** across 25 real-world test automation failure scenarios.
13
-
14
- ```mermaid
15
- flowchart LR
16
- Dataset["25 Production Challenges<br/>(Seed=42, Temp=0.0)"] --> Control["<b>Control Group (Naive LLM)</b><br/>• Monolithic context read<br/>• Raw DOM dump (>15k tokens)<br/>• Unconstrained prompt repair"]
17
- Dataset --> Treatment["<b>Treatment Group (Synapse Hub)</b><br/>• Deterministic cascade tools<br/>• Compact AOM & repo_map<br/>• 8 Quality Gates DoD"]
18
-
19
- Control --> ControlResults["• Mean Tokens: ~18,400<br/>• MTTR: 8-15 min<br/>• Pass Rate: 56.0%<br/>• Fake-Pass Trap: 44.0%"]
20
- Treatment --> TreatmentResults["• Mean Tokens: 76.2<br/>• MTTR: < 0.1s<br/>• Pass Rate: 100.0%<br/>• Fake-Pass Trap: 0.0%"]
21
- ```
22
-
23
- ---
24
-
25
- ## 2. Experimental Setup & Protocol
26
-
27
- ### 2.1 Sample Distribution ($N = 25$)
28
- Challenges are partitioned evenly across 5 critical enterprise QA failure vectors:
29
- 1. **`locator_drift` (5 challenges):** Dynamic CSS hashes, nested shadow DOM piercing, semantic label shifts, legacy ID removals, HTML tag evolution.
30
- 2. **`api_contract_drift` (5 challenges):** Type mutations, field renames, object flattening, creation status code shifts (200 $\to$ 201), auth prefix upgrades.
31
- 3. **`timing_and_race` (5 challenges):** Variable latency spikes, PrimeNG backdrop fade intercepts, debounced autocomplete races, SVG animations, infinite scroll chunks.
32
- 4. **`stale_and_mutation` (5 challenges):** WebSocket auto-refresh stale elements, tab component unmounts, tbody pagination detachments, re-rendered modal nodes, locale swaps.
33
- 5. **`fake_pass_tautology` (5 challenges):** Swallowed 500 errors in empty catch blocks, zero-assertion smoke tests, `assert true == true`, unawaited async expectations, unbounded relational assertions (`status >= 200`).
34
-
35
- ### 2.2 Hyperparameters & Control Variables
36
- - **Random Seed:** `42` (pinned across sampling and mutation generators).
37
- - **Sampling Temperature:** `0.0` (greedy decoding for deterministic reproducibility).
38
- - **Environment:** Headless Chromium, Viewport `1920x1080`, pure CPU/Memory baseline.
39
-
40
- ---
41
-
42
- ## 3. Empirical Results Comparison
43
-
44
- | Evaluation Metric | Control Group (Naive LLM) | Treatment Group (Synapse Hub) | Relative Gain / Reduction |
45
- | :--- | :---: | :---: | :---: |
46
- | **Autonomous Resolution Rate** | 56.0% (14 / 25) | **100.0% (25 / 25)** | **+44.0% absolute gain** |
47
- | **Mean Token Cost / Bug Fix** | 18,450 tokens | **76.2 tokens** | **99.59% token reduction** |
48
- | **Mean Time-to-Heal (MTTR)** | ~8.4 minutes | **< 0.001s (Deterministic)** | **Instantaneous 0-token repair** |
49
- | **AST Mutation Score** | 32.4% (Mutants survived) | **100.0% (All mutants killed)** | **Zero tautological test escapes** |
50
- | **Fake-Pass Vulnerability Rate** | 44.0% accepted fake pass | **0.0% (Strictly blocked)** | **100% Truth Anchor enforcement** |
51
- | **Zero Side-Effect Compliance** | 68.0% (Unrelated edits) | **100.0% (Atomic patch)** | **Zero regression risk** |
52
-
53
- ---
54
-
55
- ## 4. Key Mechanisms Driving the Differential
56
-
57
- 1. **Deterministic 3-Layer Fallback (`HeuristicLocatorMatcher`):**
58
- - Naive LLMs consume 10k–40k tokens reading raw HTML to guess new selectors.
59
- - Synapse Hub parses compact DOM elements locally and resolves stable `data-testid`, ARIA labels, or Levenshtein text anchors in under 15ms with **0 LLM tokens**.
60
- 2. **Cognitive Forcing & Micro-Mutations (`micro_mutator.py`):**
61
- - Naive LLMs frequently emit tautological passes (`expect(true).toBe(true)` or empty catches) when under token pressure.
62
- - Synapse Hub injects AST mutations and requires $\ge 80\%$ mutation score before handoff, catching silent passes at Gate 7 and Gate 8.
63
- 3. **Log & Context Distillation (`distill_log.py` & `repo_map.py`):**
64
- - Filters 95% of execution noise, isolating exact failing stacktrace pointers.
65
-
66
- ---
67
-
68
- ## 5. Replication Guide
69
-
70
- To independently reproduce this benchmark evaluation:
71
-
72
- ```bash
73
- # 1. Clone repository
74
- git clone https://github.com/TeeVee998/Synapse-Hub.git
75
- cd Synapse-Hub
76
-
77
- # 2. Run full deterministic benchmark harness
78
- python benchmarks/qa_eval/eval_runner.py
79
-
80
- # 3. Verify anti-fake-pass and robustness test
81
- python -m unittest tests/test_benchmark_robustness.py -v
82
- ```
83
- All outputs are serialized to `benchmarks/qa_eval/reports/eval_results.json`.