synapse-hub 2.2.3 → 2.4.0

package/.agents/skills/backend-load-and-stress-patterns/SKILL.md

@@ -1,248 +1,142 @@
 ---
 name: backend-load-and-stress-patterns
-description: "Orchestrate k6 and Locust backend load, stress, and SLA endurance test suites."
+description: "Orchestrate k6, Locust, and JMeter backend load, stress, and SLA endurance test suites. Use when designing load, stress, or spike test patterns for backend systems."
 ---
 
-# Backend Load & Stress Testing Patterns (k6 & Locust Orchestrator)
+# Backend Load & Performance Testing Skill
 
-> **Purpose:** Master backend performance, throughput, and scalability engineering. Directs AI agents to model concurrent Virtual User (VU) traffic, execute stress/soak test profiles, and enforce automated SLA gates (P95/P99 latency, error rate < 0.1%) to detect bottlenecks before production release.
+> **Target:** Model concurrent Virtual User (VU) and arrival-rate traffic, execute stress/soak test profiles, and enforce automated SLA gates (P95/P99 latency, error rate < 1.0%) to detect bottlenecks before production.  
+> **Mandatory Rule:** Adhere strictly to [.agents/rules/performance_testing_rules.md](file://.agents/rules/performance_testing_rules.md) for safety approvals, hard caps, and truth-anchor reporting.
 
 ---
 
-## 1. Five Core Performance Test Archetypes
+## 1. Test Archetypes & SLA Targets
 
-```mermaid
-flowchart LR
-    A["<b>1. Smoke Test</b><br/>1-5 VUs / 1 min<br/>Sanity check"] --> B["<b>2. Load Test</b><br/>50-200 VUs / 15m<br/>Expected daily peak"]
-    B --> C["<b>3. Stress Test</b><br/>500-2000+ VUs<br/>Find breaking point"]
-    C --> D["<b>4. Spike Test</b><br/>Instant surge 10->1000<br/>Traffic shocks"]
-    D --> E["<b>5. Soak / Endurance</b><br/>Stable load / 2-12 hrs<br/>Memory leaks & OOM"]
-```
-
-| Test Archetype | Virtual Users (VUs) | Duration | Primary Objective | Failure Indicator |
+| Archetype | VUs / Rate | Duration | Objective | SLA Assertion |
 | :--- | :--- | :--- | :--- | :--- |
-| **Smoke Test** | 1 - 5 VUs | 1 - 2 mins | Script syntax & endpoint health sanity | Any HTTP 5xx or unhandled exceptions |
-| **Load Test** | 50 - 200 VUs | 15 - 30 mins | Verify system SLA under standard peak traffic | P95 latency > 500ms or error rate > 0.1% |
-| **Stress Test** | 200 -> 2,000+ VUs | 30 - 60 mins | Identify maximum throughput and degradation cliff | DB connection pool exhaustion, 503/504 errors |
-| **Spike Test** | 10 -> 1,000 VUs in 10s | 5 - 10 mins | Test autoscaling latency and rate limiters | Gateway timeout (504), service unresponsiveness |
-| **Soak Test** | 50 - 100 VUs | 2 - 12 hours | Detect slow memory leaks, disk fill, thread locks | Degraded throughput over time, JVM OOM |
+| **Smoke Test** | 1 - 5 VUs | 1 - 2 min | Health sanity check | Error rate = 0%, P95 < 300ms |
+| **Average Load** | 50 - 200 VUs | 10 - 30 min | Standard peak capacity | Error rate < 0.5%, P95 < 500ms |
+| **Stress Test** | 200 -> 2,000 VUs | 20 - 45 min | Find degradation cliff | Graceful 429/503 (no JVM OOM) |
+| **Spike Test** | 10 -> 1,000 VUs | 5 - 10 min | Autoscaling & rate limiters | Gateway recovery < 30s |
+| **Soak Test** | 50 - 100 VUs | 2 - 12 hr | Memory leaks & connection leaks | Stable heap & steady latency |
 
 ---
 
-## 2. Automated SLA Threshold Assertions
+## 2. Tooling Unified Runbook
 
-Every performance test execution must assert measurable SLA criteria. If thresholds fail, the CI/CD pipeline returns a non-zero exit code.
-
-```javascript
-// Enterprise SLA Gates (k6)
-export const options = {
-  thresholds: {
-    // 95% of requests must complete under 500ms
-    http_req_duration: ['p(95)<500', 'p(99)<1000'],
-    // Request failure rate must stay under 0.1%
-    http_req_failed: ['rate<0.001'],
-    // Functional checks must maintain > 99.9% success
-    checks: ['rate>0.999'],
-  },
-};
-```
+### Option A: Apache JMeter (Headless CLI + JTL Analyzer)
+```bash
+# 1. Generate XML test plan from OpenAPI / Postman
+synapse generate-jmx --spec api.json -o load_plan.jmx --vus 50 --duration 180 --sla-p95 500
 
----
+# 2. Run headless benchmark
+jmeter -n -t load_plan.jmx -l results.jtl -e -o ./html_report
 
-## 3. Production-Ready k6 Load Script (TypeScript / JavaScript)
+# 3. Automated CI/CD Quality Gatekeeper
+synapse analyze-jtl results.jtl --sla-p95 500 --max-error-rate 1.0
+```
 
+### Option B: k6 Production Load Testing (with Kill-Switch & Arrival Rate)
 ```javascript
 import http from 'k6/http';
 import { check, sleep } from 'k6';
-import { Counter, Rate, Trend } from 'k6/metrics';
-
-// Custom Enterprise Metrics
-const errorRate = new Rate('custom_error_rate');
-const searchLatency = new Trend('search_latency');
 
 export const options = {
+  discardResponseBodies: true,
   scenarios: {
-    enterprise_workload: {
-      executor: 'ramping-vus',
-      startVUs: 0,
+    steady_load: {
+      executor: 'ramping-arrival-rate',
+      startRate: 10,
+      timeUnit: '1s',
+      preAllocatedVUs: 50,
+      maxVUs: 200,
       stages: [
-        { duration: '2m', target: 50 },   // Ramp-up to 50 users
-        { duration: '5m', target: 50 },   // Steady state peak
-        { duration: '2m', target: 150 },  // Surge stress load
-        { duration: '3m', target: 150 },  // Sustained stress
-        { duration: '2m', target: 0 },    // Ramp-down to 0
+        { duration: '2m', target: 50 },  // Ramp-up
+        { duration: '10m', target: 50 }, // Steady state (SLA measured here)
+        { duration: '2m', target: 0 },   // Ramp-down
       ],
-      gracefulRampDown: '30s',
     },
   },
   thresholds: {
-    http_req_duration: ['p(95)<400', 'p(99)<800'],
-    custom_error_rate: ['rate<0.005'], // < 0.5% errors under stress
+    // 1. Steady state SLA thresholds
+    'http_req_failed{scenario:steady_load}': ['rate<0.01'],
+    'http_req_duration{scenario:steady_load,expected_response:true}': ['p(95)<500', 'p(99)<1000'],
+    'checks{scenario:steady_load}': ['rate>0.99'],
+    'dropped_iterations{scenario:steady_load}': ['count==0'],
+    // 2. Kill-switch: abort test if system fails catastrophically
+    http_req_failed: [{ threshold: 'rate<0.10', abortOnFail: true, delayAbortEval: '30s' }],
   },
 };
 
-const BASE_URL = __ENV.TARGET_URL || 'https://api.staging.enterprise.local';
-
+// 1. One-time setup: authenticate and obtain auth token
 export function setup() {
-  // Pre-test Authentication & Seeding (Executed once)
-  const authRes = http.post(`${BASE_URL}/api/v1/auth/login`, JSON.stringify({
-    username: __ENV.PERF_USER || 'perf_agent_01',
-    password: __ENV.PERF_PASSWORD || 'SecretVault#2026',
+  const loginRes = http.post(`${__ENV.BASE_URL}/api/v1/auth/login`, JSON.stringify({
+    username: __ENV.PERF_USER,
+    password: __ENV.PERF_PASSWORD,
   }), { headers: { 'Content-Type': 'application/json' } });
-
-  check(authRes, { 'Auth 200 OK': (r) => r.status === 200 });
-  const token = authRes.json('token');
-  return { token };
+  
+  return { token: loginRes.json('token') };
 }
 
+// 2. Virtual User journey
 export default function (data) {
   const params = {
     headers: {
       'Authorization': `Bearer ${data.token}`,
       'Content-Type': 'application/json',
-      'X-Correlation-ID': `PERF-${__VU}-${__ITER}-${Date.now()}`,
     },
+    tags: { name: 'GET /api/v1/items' },
   };
 
-  // 1. Transaction A: Fast Read API
-  const resGet = http.get(`${BASE_URL}/api/v1/catalogs?limit=20`, params);
-  const getOk = check(resGet, {
-    'Catalog GET 200': (r) => r.status === 200,
-    'Catalog Payload Valid': (r) => r.json('data.items') !== undefined,
+  const res = http.get(`${__ENV.BASE_URL}/api/v1/items?page=1&limit=20`, params);
+  check(res, {
+    'status is 200': (r) => r.status === 200,
   });
-  errorRate.add(!getOk);
-  searchLatency.add(resGet.timings.duration);
 
-  sleep(1);
-
-  // 2. Transaction B: Dynamic Mutation API
-  const payload = JSON.stringify({
-    code: `AUTO_PERF_${__VU}_${__ITER}_${Date.now()}`,
-    name: 'Dynamic Synthetic Benchmark Item',
-    active: true,
-  });
-
-  const resPost = http.post(`${BASE_URL}/api/v1/catalogs`, payload, params);
-  const postOk = check(resPost, {
-    'Catalog POST 201': (r) => r.status === 201 || r.status === 200,
-  });
-  errorRate.add(!postOk);
-
-  sleep(2);
+  // Realistic think time between actions (uniform 1s to 3s)
+  sleep(Math.random() * 2 + 1);
 }
 
+// 3. Ephemeral teardown cleanup
 export function teardown(data) {
-  // Clean up benchmark markers if required
+  // Execute teardown API calls for created PERF_ entities
 }
 ```
 
 ---
 
-## 4. Production-Ready Locust Load Script (Python)
-
-```python
-import os
-import time
-from locust import HttpUser, task, between, events
-
-class EnterpriseBackendUser(HttpUser):
-    wait_time = between(1.0, 2.5)  # Realistic user think-time
-
-    def on_start(self):
-        """Initial authentication handshake per Virtual User."""
-        res = self.client.post("/api/v1/auth/login", json={
-            "username": os.getenv("PERF_USER", "perf_tester"),
-            "password": os.getenv("PERF_PWD", "SecretPass2026")
-        })
-        if res.status_code == 200:
-            token = res.json().get("token")
-            self.client.headers.update({"Authorization": f"Bearer {token}"})
-
-    @task(3)
-    def query_search_index(self):
-        """High-frequency read query."""
-        with self.client.get("/api/v1/search?q=enterprise&page=1", catch_response=True) as response:
-            if response.status_code != 200:
-                response.failure(f"Search API failed with HTTP {response.status_code}")
-            elif response.elapsed.total_seconds() > 0.5:
-                response.failure(f"Search API SLA breached (>500ms): {response.elapsed.total_seconds():.3f}s")
-
-    @task(1)
-    def submit_order_transaction(self):
-        """Mutation transaction with dynamic unique identifier."""
-        order_id = f"ORDER_PERF_{time.time_ns()}"
-        with self.client.post("/api/v1/orders", json={
-            "orderCode": order_id,
-            "items": [{"sku": "SKU-990", "quantity": 1}],
-            "totalAmount": 150000
-        }, catch_response=True) as response:
-            if response.status_code not in (200, 201):
-                response.failure(f"Order submission failed: {response.text}")
-```
+## 3. Best Practices & Safety Invariants
 
----
+- **Isolated Test Data:** Always use dynamic runtime identifiers (`PERF_AUTO_${Date.now()}`) to eliminate primary key collisions.
+- **TearDown Hygiene:** Complement POST/creation load tests with automated teardown or ephemeral DB snapshots (`tools/db_sandbox.py`).
+- **Telemetry Correlation:** Pair JTL/k6 metrics with server CPU/Memory traces via `tools/telemetry_correlator.py`.
+- **Truth Anchor:** Never loosen thresholds to manufacture a false pass. A failing threshold indicates a system capacity limit to be investigated.
 
-## 5. Execution Runbook & CI/CD Integration
+---
 
-### Run k6 Headless with JSON Summary
-```bash
-# Execute 100-user load test and export raw metrics
-k6 run --out json=reports/k6_metrics.json tests/load/k6_catalog_load.js
-
-# Assert exit code in CI/CD pipeline
-if [ $? -ne 0 ]; then
-  echo "❌ Performance SLA threshold breached!"
-  exit 1
-fi
-```
+## 4. Code-as-Code Load Model (Locust)
 
-### Run Locust Headless in CI/CD
-```bash
-locust -f tests/load/locustfile.py --headless -u 100 -r 10 --run-time 5m \
-       --host https://api.staging.enterprise.local \
-       --html reports/locust_report.html \
-       --exit-code-on-error 1
-```
+> *Borrowed and adapted from the Locust load-testing model (MIT license, locustio/locust).*
 
-### Autonomous Apache JMeter 360° Pinnacle Suite (Full-Stack & Multi-Protocol)
-Hub provides complete enterprise lifecycle automation for Apache JMeter 5.x via `generate_jmx_plan.py`, `jtl_analyzer.py`, and `telemetry_correlator.py`:
+When version-controlled, reviewable load tests matter more than XML tooling,
+generate a runnable locustfile from the same API ingestion used for JMeter:
 
 ```bash
-# 1. Generate Full Enterprise Plan with Pareto Traffic Shaping (70/20/10) & Deep Correlation
-agent-qa generate-jmx --spec swagger.json -o perf_plan.jmx \
-  --workload spike --vus 200 --duration 600 \
-  --weighted-mix "70:Browse,20:Cart,10:Checkout" \
-  --csv-data users.csv --deep-correlation --jsr223-hmac
-
-# 2. Multi-Protocol Load Plans (WebSocket, gRPC, Kafka, JDBC)
-agent-qa generate-jmx --ws-sample -o websocket_plan.jmx      # Real-time WebSocket feed
-agent-qa generate-jmx --grpc-sample -o grpc_plan.jmx        # Microservices Unary RPC
-agent-qa generate-jmx --kafka-sample -o kafka_plan.jmx      # Event-driven message broker
-agent-qa generate-jmx --jdbc-sample -o jdbc_plan.jmx        # Direct Database SQL pool
-
-# 3. Live APM Telemetry & PerfMon Hardware Agent
-agent-qa generate-jmx --spec swagger.json -o perf_plan.jmx \
-  --influxdb-url http://influxdb:8086/write?db=jmeter \
-  --perfmon-server 192.168.1.50:4444
-
-# 4. Distributed Cluster Execution (Master + 3 Slaves)
-docker-compose -f tools/templates/jmeter-distributed/docker-compose.yml up -d
-jmeter -n -t perf_plan.jmx -R slave1,slave2,slave3 -l results.jtl
-
-# 5. CI/CD SLA Quality Gate Enforcement (Exit code 0 on Pass, 1 on Breach)
-agent-qa analyze-jtl results.jtl --sla-p95 500 --max-error-rate 1.0 \
-  --markdown pr_perf_report.md --json metrics.json
-
-# 6. Full-Stack APM & Server Infrastructure RCA Correlator
-agent-qa correlate-telemetry --jtl results.jtl --server-metrics server_metrics.csv \
-  --markdown rca_report.md
+synapse generate-jmx --spec swagger.json --emit-locust locustfile.py \
+  --weighted-mix "70,20,10" --think-time-min 500 --think-time-max 1500
+locust -f locustfile.py --host=https://api.staging.enterprise.local
 ```
 
----
-
-## 6. Performance Engineering Checklist
-- [ ] Authentication executed once per VU in `setup()` or `on_start()`, never in tight loops.
-- [ ] Synthetic transactions use dynamic unique codes to eliminate DB primary key collisions.
-- [ ] SLA thresholds include both latency (P95/P99) and availability (`error_rate < 0.1%`).
-- [ ] Soak tests executed against isolated staging environments with active APM/DB monitoring.
-- [ ] Teardown removes temporary benchmark data to prevent DB bloat.
+The generated file follows the Locust shape — one `HttpUser` class, one
+`@task(weight)` method per endpoint, `wait_time` from think-time — so QA
+engineers can hand-tune scenarios in pure Python and commit them next to the
+test suite.
+
+1. **Weights are the traffic mix** — the same `--weighted-mix` percentages that
+   drive JMeter `ThroughputController`s drive `@task` weights; keep both exports
+   in sync from one spec.
+2. **Prefer code-as-code for logic-heavy journeys** — conditional flows
+   (login → create → assert → delete) are Python, not XML; use the JMX export
+   for protocol breadth (JDBC/Kafka/gRPC/WS) and Locust for flow logic.
+3. **Same SLA gates** — run `agent-qa analyze-jtl` semantics on Locust stats:
+   fail CI when P95 breaches the SLA budget.

package/.agents/skills/backend-load-and-stress-patterns/examples/k6_load_test_sample.js

@@ -0,0 +1,13 @@
+import http from 'k6/http';
+import { check, sleep } from 'k6';
+
+export const options = {
+  vus: 10,
+  duration: '30s',
+};
+
+export default function () {
+  const res = http.get('https://test.k6.io');
+  check(res, { 'status was 200': (r) => r.status == 200 });
+  sleep(1);
+}

package/.agents/skills/bats-testing-patterns/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: bats-testing-patterns
-description: "Test bash/sh scripts, CLI tools, and CI/CD pipelines with Bats."
+description: "Test bash/sh scripts, CLI tools, and CI/CD pipelines with Bats. Use when writing Bash automation tests with the BATS framework."
 ---
 
 # Bats Testing Patterns (Bash Automated Testing System)

package/.agents/skills/bats-testing-patterns/examples/cli_runner_test.bats

@@ -0,0 +1,6 @@
+#!/usr/bin/env bats
+
+@test 'verify CLI returns version' {
+  run python3 tools/cli.py --version
+  [ "$status" -eq 0 ]
+}