synapse-hub 2.1.0 → 2.2.0

package/tools/TOOLS_RUNBOOK.md

@@ -35,7 +35,10 @@
 | **Generating test scripts from Natural Language Story** | `tools/generate_tc_from_story.py`| `python tools/generate_tc_from_story.py "<story>"` |
 | **Generating E2E matrix from OpenAPI/Swagger** | `tools/generate_api_matrix.py` | `python tools/generate_api_matrix.py --spec <swagger.json>` |
 | **Exporting API tests to Postman Collection v2.1** | `tools/export_postman_collection.py` | `agent-qa export-postman --spec <swagger.json> -o col.json` |
+| **Analyzing Newman Results & CI/CD Gatekeeper** | `tools/newman_analyzer.py` | `agent-qa analyze-newman report.json --max-failures 0` |
 | **Generating JMeter Performance Test Plan (.jmx)** | `tools/generate_jmx_plan.py` | `agent-qa generate-jmx --spec <swagger.json> --vus 100 -o plan.jmx` |
+| **Analyzing JMeter JTL & Enforcing CI/CD SLA Gate** | `tools/jtl_analyzer.py` | `agent-qa analyze-jtl results.jtl --sla-p95 500 --max-error-rate 1.0` |
+| **Full-Stack APM & Server Telemetry RCA Correlator** | `tools/telemetry_correlator.py` | `agent-qa correlate-telemetry --jtl results.jtl --server-metrics srv.csv` |
 | **Auditing Web Accessibility (WCAG 2.2)** | `tools/wcag_audit.py` | `python tools/wcag_audit.py <path_to_html_or_url>` |
 | **0-Token Polyglot Test Transpilation** | `tools/omni_transpiler.py` | `agent-qa transpile <file> --to <target>` (Playwright TS, Python, Java, Katalon, Cypress) |
 | **0-Token Repository Polyglot AST Scan** | `tools/omni_transpiler.py` | `agent-qa omni-scan [path]` (Instant inventory of Pages, Locators, Tests across languages) |
@@ -52,11 +55,28 @@
 | **Lookup XPath Locator in cached memory** | `tools/lookup_xpath.py` | `python tools/lookup_xpath.py "<keyword>"` |
 | **Lookup Test Case metadata** | `tools/lookup_tc.py` | `python tools/lookup_tc.py "<keyword_or_id>"` |
 | **Linting Markdown documentation quality** | `tools/lint_ai_docs.py` | `python tools/lint_ai_docs.py` |
+| **Auditing AI Agent Skills & MCP Security** | `tools/skillspector.py` | `python tools/skillspector.py [path]` (or `agent-qa skillspector [path]` / `npx synapse-hub skillspector`) |
+| **Upstream Version Check & 1-Click Self-Update** | `tools/update_sentinel.py` | `python tools/update_sentinel.py --check / --update` (or `agent-qa update` / `npx synapse-hub update`) |
 
 ---
 
 ## 2. DETAILED TOOL USAGE
 
+### 0. `tools/update_sentinel.py` (Upstream Update & Drift Sentinel)
+Checks upstream releases on NPM Registry and GitHub, displaying non-blocking update notifications when a new version is published. Supports 1-click self-updating across local git repos and global plugins.
+
+```bash
+# Check if update is available without applying
+python tools/update_sentinel.py --check
+# or via CLI
+agent-qa update --check
+
+# Perform 1-click self-update (pulls git, syncs memory cache, verifies DoD)
+python tools/update_sentinel.py --update
+# or via CLI
+agent-qa update
+```
+
 ### 1. `tools/generate_api_matrix.py`
 - **Purpose:** Ingests OpenAPI / Swagger JSON schemas and automatically computes comprehensive E2E test matrices: Positive, Boundary Value (min/max length, special characters, XSS payloads), and Negative (invalid types, missing required fields).
 - **CLI Syntax:**
@@ -319,31 +339,202 @@
 ---
 
 ### 21. `tools/export_postman_collection.py`
-- **Purpose:** Enterprise OpenAPI/Swagger & HAR to Postman Collection v2.1.0 Exporter. Generates fully structured Postman collections with dynamic pre-request scripts, SLA latency tests, response code assertions, and autonomous bearer token / created entity ID extraction for Newman and CI/CD pipelines.
+- **Purpose:** Enterprise OpenAPI/Swagger, HAR & Zero-Input Domain Postman Collection v2.1.0 Exporter (Postman Pinnacle Suite). Generates fully structured Postman collections with dynamic pre-request scripts, SLA latency tests, response code assertions, autonomous token/entity chaining, native **AJV JSON Schema Validation**, companion **Data-Driven Testing (DDT) CSV matrix**, and **Autonomous Zero-Input Domain Synthesis** (`--domain [ecommerce|banking|saas]`).
 - **CLI Syntax:**
   ```bash
-  # Ingest OpenAPI spec and export Postman collection + companion environment
+  # 1. Ingest OpenAPI spec and export Postman collection + environment + AJV Schema asserts
   agent-qa export-postman --spec swagger.json -o collection.json --env-output env.json
 
-  # Replay production HAR traffic as Postman requests
-  agent-qa export-postman --har session.har -o har_collection.json
+  # 2. Autonomous Zero-Input Synthesis (No file provided! Synthesizes complete Banking/Ecom suite)
+  agent-qa export-postman --domain banking -o banking_col.json --env-output env.json --ddt-output ddt_data.csv
+  agent-qa export-postman --domain ecommerce -o ecom_col.json --ddt-output ecom_data.csv
 
-  # Generate sample collection from built-in enterprise schema
-  agent-qa export-postman --sample -o sample_collection.json
+  # 3. Replay production HAR traffic as Postman requests
+  agent-qa export-postman --har session.har -o har_collection.json
   ```
 
 ---
 
 ### 22. `tools/generate_jmx_plan.py`
-- **Purpose:** Apache JMeter 5.x Performance Test Plan XML (.jmx) Generator. Ingests OpenAPI specifications, Postman Collections, or HAR recordings to output production-ready JMeter load, stress, and spike test plans with configurable VUs, ramp-up schedules, SLA assertions (< 500ms), and dynamic session token propagation.
+- **Purpose:** Apache JMeter 5.x Performance Test Plan XML (.jmx) Generator (JMeter 360° Pinnacle Suite). Ingests OpenAPI specifications, Postman Collections, or HAR recordings to output production-ready JMeter load, stress, spike, and soak test plans with HTTP Cookie & Cache Managers, CSV Data Set Config, Transaction Controllers, Pareto Traffic Shaping (`--weighted-mix`), Deep Data Correlation (CSRF regex & random dynamic selectors), JSR223 Groovy HMAC payload signers, InfluxDB live APM telemetry, PerfMon server agent monitoring, and Multi-Protocol support (HTTP, JDBC, WebSocket, gRPC, Kafka).
 - **CLI Syntax:**
   ```bash
-  # Generate JMeter plan from OpenAPI with 100 Virtual Users for 10 minutes
-  agent-qa generate-jmx --spec swagger.json -o load_plan.jmx --vus 100 --duration 600
+  # 1. Generate JMeter plan with Pareto traffic shaping (70% Browse, 20% Cart, 10% Checkout)
+  agent-qa generate-jmx --spec swagger.json -o load_plan.jmx --weighted-mix "70:Browse,20:Cart,10:Checkout"
+
+  # 2. Convert existing Postman collection with CSV data & Deep Correlation (CSRF + random items)
+  agent-qa generate-jmx --postman collection.json -o perf_plan.jmx --csv-data users.csv --deep-correlation
 
-  # Convert existing Postman collection into JMeter performance test plan
-  agent-qa generate-jmx --postman collection.json -o perf_plan.jmx --vus 50 --ramp-up 20
+  # 3. Generate multi-protocol test plans (WebSocket, gRPC, Kafka, JDBC)
+  agent-qa generate-jmx --ws-sample -o websocket_plan.jmx
+  agent-qa generate-jmx --grpc-sample -o grpc_plan.jmx
+  agent-qa generate-jmx --kafka-sample -o kafka_plan.jmx
+  agent-qa generate-jmx --jdbc-sample -o jdbc_plan.jmx
 
-  # Run headless JMeter CLI
+  # 4. Generate with live InfluxDB streaming & PerfMon Server Agent
+  agent-qa generate-jmx --spec swagger.json -o perf_plan.jmx --influxdb-url http://influxdb:8086/write?db=jmeter --perfmon-server 192.168.1.50:4444
+
+  # 5. Run headless JMeter CLI
   jmeter -n -t load_plan.jmx -l results.jtl -e -o ./html_report
   ```
+
+---
+
+### 23. `tools/jtl_analyzer.py`
+- **Purpose:** Apache JMeter JTL Results Analyzer & CI/CD SLA Quality Gatekeeper. Parses `.jtl` (CSV or XML), computes statistical percentiles ($P_{50}, P_{90}, P_{95}, P_{99}$, Throughput, Error Rate), evaluates user SLA thresholds, supports server telemetry correlation (`--telemetry`), and returns deterministic exit code `0` (Pass) or `1` (Fail) to enforce quality gates in automated pipelines.
+- **CLI Syntax:**
+  ```bash
+  # 1. Enforce CI/CD Quality Gate with P95 Latency SLA (< 500ms) and max error rate (< 1%)
+  agent-qa analyze-jtl results.jtl --sla-p95 500 --max-error-rate 1.0
+
+  # 2. Correlate with Server-Side APM Telemetry directly in Quality Gate
+  agent-qa analyze-jtl results.jtl --sla-p95 500 --telemetry server_metrics.csv --markdown pr_report.md
+
+  # 3. Dry-run sample benchmark verification
+  agent-qa analyze-jtl --sample --sla-p95 500
+  ```
+
+---
+
+### 24. `tools/telemetry_correlator.py`
+- **Purpose:** Full-Stack APM & Server Telemetry Performance Correlator. Performs time-aligned Pearson correlation between Client-Side perceived response times (`.jtl`) and Server-Side infrastructure metrics (CPU, RAM, Database Connection Pool, Disk I/O). Automatically isolates bottlenecks and produces Root Cause Analysis (RCA) diagnostic reports.
+- **CLI Syntax:**
+  ```bash
+  # Correlate client JTL latency with server infrastructure telemetry
+  agent-qa correlate-telemetry --jtl results.jtl --server-metrics server_metrics.csv --markdown rca_report.md
+
+  # Dry-run sample correlation verification
+  agent-qa correlate-telemetry --sample
+  ```
+
+---
+
+### 25. `tools/skillspector.py` (AI Agent Skill & MCP Security Sentinel)
+- **Purpose:** Autonomous security vulnerability and supply-chain auditor for AI Agent skills, prompt instructions, MCP tool configurations, and bundled automation scripts. Synthesized from **[NVIDIA/SkillSpector](https://github.com/NVIDIA/SkillSpector)** (Apache 2.0). Features 71 detection vectors across 17 categories, the NVIDIA Prohibition Context Engine to eliminate false positives in defensive rules, and multi-format outputs (Terminal, JSON, Markdown, SARIF 2.1.0).
+- **Threat Categories Covered:**
+  - Prompt Injection (`P1`, `P2`, `P3`, `P4`, `P9`)
+  - Data Exfiltration & Credential Harvesting (`E1`–`E5`)
+  - Privilege Escalation & Container Escapes (`PE1`–`PE5`)
+  - Agent Configuration Snooping (`AS1`–`AS3` targeting `~/.claude`, `~/.gemini`, `mcp.json`)
+  - System Prompt Leakage (`P6`–`P8`)
+  - Anti-Refusal Statements & Jailbreaks (`AR1`–`AR3`)
+  - MCP Tool Poisoning & Unicode Homoglyphs (`TP1`–`TP4`)
+  - MCP Least Privilege (`LP1`–`LP4`)
+  - Behavioral AST & Dangerous Deserialization (`AST1`–`AST10`)
+  - Supply Chain & SSRF (`SC1`–`SC2`, `SSRF1`–`SSRF2`)
+- **CLI Syntax:**
+  ```bash
+  # 1. Audit a skill directory or file with rich terminal output
+  python tools/skillspector.py .agents/skills/my-skill/
+  agent-qa skillspector .agents/skills/my-skill/
+  npx synapse-hub skillspector .agents/skills/my-skill/
+
+  # 2. Output SARIF 2.1.0 for GitHub Security tab integration
+  python tools/skillspector.py .agents/skills/ --format sarif --output reports/skillspector.sarif
+
+  # 3. Output JSON for CI/CD automated policy gate
+  python tools/skillspector.py .agents/skills/ --format json --output reports/skillspector.json
+
+  # 4. Strict CI gate: exit code 1 if any CRITICAL or DO_NOT_INSTALL findings exist
+  python tools/skillspector.py .agents/skills/ --strict
+  ```
+
+---
+
+### 26. `tools/newman_analyzer.py` (Newman CI/CD SLA Quality Gatekeeper)
+- **Purpose:** Postman / Newman Automated Test Results Analyzer & CI/CD Gatekeeper. Ingests Newman JSON reports (`newman run -r json`), parses total requests, triages failed assertions with root causes, checks average latency SLA, and enforces deterministic exit code `0` (Pass) vs `1` (Fail) to prevent broken API contracts from deploying.
+- **CLI Syntax:**
+  ```bash
+  # 1. Execute Newman with JSON report export
+  newman run collection.json -e env.json -r cli,json --reporter-json-export newman_report.json
+
+  # 2. Enforce CI/CD Quality Gate (Exit code 0 on Pass, 1 on Fail)
+  agent-qa analyze-newman newman_report.json --max-failures 0 --sla-avg 500
+
+  # 3. Export Pull Request Markdown report & JSON metrics
+  agent-qa analyze-newman newman_report.json --markdown pr_api_report.md --json api_metrics.json
+
+  # 4. Dry-run sample benchmark verification
+  agent-qa analyze-newman --sample
+  ```
+
+---
+
+### 27. `tools/auto_heal.py` & `tools/heuristic_locator_matcher.py` (Closed-Loop Test Failure Diagnosis & Locator Healing)
+- **Purpose:** In-place autonomous test failure healer powered by `tools/heuristic_locator_matcher.py`. Correlates JUnit XML failures and screenshots (`Screenshots/FAILED_*.png`) with live DOM state, computes semantic element diffs via Role/Label/TestId hierarchy, and patches broken Page Class locators without full rerun.
+- **CLI Syntax:**
+  ```bash
+  python tools/auto_heal.py
+  agent-qa auto-heal
+  ```
+
+---
+
+### 28. `tools/flaky_detector.py` (Flaky Test Sentinel & Quarantine Lifecycle)
+- **Purpose:** Detects non-deterministic test flakiness across test suites, calculates historical flakiness scores, and manages automated test quarantine lifecycles to protect CI/CD pipelines from false alarms.
+- **CLI Syntax:**
+  ```bash
+  python tools/flaky_detector.py --scan Reports/
+  agent-qa flaky --scan Reports/
+  ```
+
+---
+
+### 29. `tools/browser_helpers.js` (In-Browser Live AOM & Safe Interaction Prober)
+- **Purpose:** Client-side JavaScript helper injected into headless Playwright/Chrome sessions. Provides `window.__helper` and `window.qa` APIs: `qa.getCompactAOM()` (extracts interactive nodes sub-200ms under 500 tokens), safe click overlay bypassing for PrimeNG/AntD backdrops, and live selector hit-rate verification.
+- **Usage:** Injected automatically by `live-explorer` subagent and Playwright test fixtures.
+
+---
+
+### 30. `tools/project_state.py` (Compaction Anchoring & Persistent State Store)
+- **Purpose:** Persists mission-critical project state (active test cases, defect registers, test data tracking) into disk-anchored files (`PROJECT_STATE.md` and `.project_state.properties`). Prevents loss of context during Antigravity automated context compaction.
+- **Usage:** Autonomously invoked by the Lead Orchestrator agent across turn transitions.
+
+---
+
+### 31. `tools/universal_ir.py`, `tools/omni_parser.py`, `tools/omni_emitter.py`
+- **Purpose:** Universal Intermediate Representation (IR) engine powering 0-token polyglot test transpilation (`agent-qa transpile`). Parses test scripts from Groovy, TypeScript, Python, or Java into a framework-agnostic AST and emits clean, idiomatic code for target test frameworks.
+- **CLI Syntax:**
+  ```bash
+  agent-qa transpile Scripts/TC_Login.groovy --to playwright-ts
+  ```
+
+---
+
+### 32. `tools/check_env.py` (Prerequisites & Toolchain Diagnostic Checker)
+- **Purpose:** Rapid diagnostic tool that checks required local execution toolchains: Python 3.9+, Node.js 18+, Java JDK 11+, Playwright browser binaries, Git, and Docker.
+- **CLI Syntax:**
+  ```bash
+  python tools/check_env.py
+  ```
+
+---
+
+### 33. `tools/git_pre_commit_guard.py` (Zero-Leakage Git Pre-Commit Hook)
+- **Purpose:** Git hook gatekeeper preventing accidental commits of UTF-8 BOM characters, hardcoded secrets/API keys, absolute machine paths, or temporary dump files into git repositories.
+- **CLI Syntax:**
+  ```bash
+  python tools/git_pre_commit_guard.py
+  ```
+
+---
+
+### 34. `tools/validate_groovy_scripts.py` (Katalon / Groovy AST Syntax Guard)
+- **Purpose:** Static verification scanner for Groovy and Katalon test scripts. Ensures 100% adherence to Clean POM Flat Architecture, zero raw XPaths in test cases, and valid import signatures.
+- **CLI Syntax:**
+  ```bash
+  python tools/validate_groovy_scripts.py
+  ```
+
+---
+
+### 35. Platform Scripts & Automation Utilities
+- `tools/verify_all.sh`: macOS / Linux Bash equivalent for the 8 Quality Gates DoD verification.
+- `tools/generate_allure_report.ps1`: Automated generation and serving of Allure test execution dashboards.
+- `tools/generate_test_reports.ps1`: Automated consolidation of polyglot test reports into centralized Markdown summaries.
+- `tools/mcp_excel_server.ps1`: Specialized Model Context Protocol (MCP) server for structured Excel data interaction.
+- `tools/e2e_flow_prober.ps1`: Automated E2E smoke flow prober across target web environments.
+- `tools/bump_version.py`: Automated SemVer version bumper for Synapse-Hub releases.
+
+
+

package/tools/browser_helpers.js

@@ -13,7 +13,7 @@
 (function () {
   'use strict';
 
-  if (window.__qa && window.__qa.__version === '2.1.0') {
+  if (window.__qa && window.__qa.__version === '2.2.0') {
     return window.__qa;
   }
 
@@ -47,7 +47,7 @@
   }
 
   const qa = {
-    __version: '2.1.0',
+    __version: '2.2.0',
 
     /**
      * 1. Smart Action Batching

package/tools/bump_version.py

@@ -31,6 +31,14 @@ import json
 import argparse
 import subprocess
 
+# Enforce UTF-8 stdout encoding for Windows console compatibility
+if hasattr(sys.stdout, 'reconfigure'):
+    try:
+        sys.stdout.reconfigure(encoding='utf-8')
+        sys.stderr.reconfigure(encoding='utf-8')
+    except Exception:
+        pass
+
 REPO_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 
 def parse_semver(v_str: str):

package/tools/cli.py

@@ -25,7 +25,7 @@ sys.path.insert(0, SCRIPT_DIR)
 
 def cmd_version(args):
     manifest_path = os.path.join(REPO_ROOT, "skills_manifest.json")
-    version = "2.1.0"
+    version = "2.2.0"
     if os.path.exists(manifest_path):
         try:
             import json
@@ -281,6 +281,45 @@ def cmd_generate_jmx(args):
     sys.exit(jmx_main())
 
 
+def cmd_skillspector(args):
+    from skillspector import main as skillspector_main
+    sys.argv = [sys.argv[0]] + (sys.argv[2:] if len(sys.argv) > 2 else [])
+    sys.exit(skillspector_main())
+
+
+def cmd_analyze_jtl(args):
+    from jtl_analyzer import main as jtl_main
+    sys.argv = [sys.argv[0]] + (sys.argv[2:] if len(sys.argv) > 2 else [])
+    sys.exit(jtl_main())
+
+
+def cmd_correlate_telemetry(args):
+    from telemetry_correlator import main as telem_main
+    sys.argv = [sys.argv[0]] + (sys.argv[2:] if len(sys.argv) > 2 else [])
+    sys.exit(telem_main())
+
+
+def cmd_analyze_newman(args):
+    from newman_analyzer import main as newman_main
+    sys.argv = [sys.argv[0]] + (sys.argv[2:] if len(sys.argv) > 2 else [])
+    sys.exit(newman_main())
+
+
+def cmd_update(args):
+    import update_sentinel
+    if getattr(args, "check", False):
+        has_update, cur, rem = update_sentinel.check_for_updates(force=True, notify=False)
+        print(f"Current Hub Version : v{cur}")
+        print(f"Latest Upstream     : v{rem if rem else 'Unknown'}")
+        if has_update:
+            print(f"⚠️ An update is available: v{cur} -> v{rem}. Run 'agent-qa update' to upgrade.")
+            sys.exit(1)
+        else:
+            print("✅ Synapse-Hub is up to date.")
+            sys.exit(0)
+    update_sentinel.perform_update()
+
+
 def main():
     parser = argparse.ArgumentParser(
         prog="agent-qa",
@@ -422,9 +461,34 @@ def main():
     p_postman.set_defaults(func=cmd_export_postman)
 
     # generate-jmx
-    p_jmx = subparsers.add_parser("generate-jmx", help="Generate Apache JMeter 5.x Test Plan XML (.jmx) from OpenAPI / Postman / HAR")
+    p_jmx = subparsers.add_parser("generate-jmx", help="Generate Apache JMeter 5.x Test Plan XML (.jmx) with JMeter 360 Enterprise Capabilities")
     p_jmx.set_defaults(func=cmd_generate_jmx)
 
+    # analyze-jtl
+    p_jtl = subparsers.add_parser("analyze-jtl", help="Analyze JMeter .jtl results, evaluate P90/P95/P99 SLA, and enforce CI/CD Quality Gate")
+    p_jtl.set_defaults(func=cmd_analyze_jtl)
+
+    # correlate-telemetry
+    p_telemetry = subparsers.add_parser("correlate-telemetry", help="Correlate Client-Side JMeter latency with Server-Side APM CPU/RAM/DB telemetry")
+    p_telemetry.set_defaults(func=cmd_correlate_telemetry)
+
+    # analyze-newman
+    p_newman = subparsers.add_parser("analyze-newman", help="Analyze Postman / Newman test results, triage failed assertions, and enforce CI/CD Quality Gate")
+    p_newman.set_defaults(func=cmd_analyze_newman)
+
+    # update
+    p_update = subparsers.add_parser("update", help="1-Click update Synapse-Hub to latest version from GitHub/NPM")
+    p_update.add_argument("--check", action="store_true", help="Check for available updates without applying")
+    p_update.set_defaults(func=cmd_update)
+
+    # skillspector (AI Agent Skill & MCP Security Scanner)
+    p_skillspector = subparsers.add_parser("skillspector", help="AI Agent Skill & MCP Security Sentinel (71 vectors, NVIDIA SkillSpector spec)")
+    p_skillspector.add_argument("path", nargs="?", default=".", help="Skill directory or file to scan (default: current directory)")
+    p_skillspector.add_argument("--format", choices=["terminal", "json", "markdown", "sarif"], default="terminal", help="Output format")
+    p_skillspector.add_argument("--output", "-o", help="Write report to output file")
+    p_skillspector.add_argument("--strict", action="store_true", help="Fail with non-zero exit code on HIGH/CRITICAL finding")
+    p_skillspector.set_defaults(func=cmd_skillspector)
+
     args, unknown = parser.parse_known_args()
     if getattr(args, "offline", False):
         os.environ["AGENT_QA_OFFLINE"] = "1"
@@ -434,6 +498,14 @@ def main():
         parser.print_help()
         sys.exit(0)
 
+    # Non-intrusive cached update notification (24h cache)
+    if args.subcommand not in ("update", "mcp-serve", "version"):
+        try:
+            import update_sentinel
+            update_sentinel.check_for_updates(force=False, notify=True)
+        except Exception:
+            pass
+
     # Dispatch subcommand
     args.func(args)