@intentsolutionsio/performance-test-suite 1.0.0

package/skills/running-performance-tests/SKILL.md

@@ -0,0 +1,140 @@
+---
+name: running-performance-tests
+description: |
+  Execute load testing, stress testing, and performance benchmarking.
+  Use when performing specialized testing.
+  Trigger with phrases like "run load tests", "test performance", or "benchmark the system".
+
+allowed-tools: Read, Write, Edit, Grep, Glob, Bash(test:perf-*)
+version: 1.0.0
+author: Jeremy Longshore <jeremy@intentsolutions.io>
+license: MIT
+compatible-with: claude-code, codex, openclaw
+tags: [testing, performance, performance-tests]
+---
+# Performance Test Suite
+
+## Overview
+
+Execute load testing, stress testing, and performance benchmarking to identify bottlenecks, establish baseline metrics, and verify SLA compliance. Supports k6 (recommended), Artillery, Apache JMeter, Locust (Python), and autocannon (Node.js).
+
+## Prerequisites
+
+- Performance testing tool installed (`k6`, `artillery`, `locust`, `jmeter`, or `autocannon`)
+- Target application deployed in a production-like environment (not local dev)
+- Baseline performance metrics or SLA targets (e.g., p95 < 200ms, 99.9% availability)
+- Monitoring stack accessible (Grafana, CloudWatch, Datadog) for resource metrics during tests
+- Test data sufficient to avoid cache-only responses
+
+## Instructions
+
+1. Define performance test scenarios based on production traffic patterns:
+   - **Load test**: Simulate expected peak traffic (e.g., 500 concurrent users for 10 minutes).
+   - **Stress test**: Ramp beyond expected capacity to find the breaking point.
+   - **Spike test**: Sudden burst of traffic (0 to 1000 users in 10 seconds).
+   - **Soak test**: Sustained moderate load for extended duration (1-4 hours) to detect memory leaks.
+2. Create test scripts targeting critical endpoints:
+   - Identify the top 5-10 most-hit API endpoints from production access logs.
+   - Include both read (GET) and write (POST/PUT/DELETE) operations.
+   - Simulate realistic user behavior with think time between requests.
+   - Use parameterized data to avoid cache-only hits (randomize query parameters, user IDs).
+3. Configure load profiles:
+   - Define virtual user (VU) ramp-up stages (e.g., 10 VUs for 1 minute, then 50 VUs for 5 minutes).
+   - Set test duration appropriate to the scenario (load: 10-15 min, soak: 1-4 hours).
+   - Configure request timeouts matching production settings.
+4. Execute the performance test:
+   - Run from a machine with sufficient network bandwidth and CPU.
+   - Avoid running from the same host as the application under test.
+   - Monitor application metrics (CPU, memory, DB connections) during execution.
+5. Analyze results against SLA thresholds:
+   - p50, p90, p95, p99 response times.
+   - Requests per second (throughput).
+   - Error rate (target: < 0.1% for load test, higher tolerance for stress test).
+   - Resource utilization (CPU < 80%, memory < 85% at peak load).
+6. Identify and document bottlenecks:
+   - Slow database queries (check slow query logs).
+   - CPU-bound operations (profiling data).
+   - Memory leaks (growing RSS over soak test).
+   - Connection pool exhaustion (database or HTTP client).
+7. Generate a performance report with visualizations and recommendations.
+
+## Output
+
+- Performance test scripts (k6 `.js`, Artillery `.yml`, or Locust `.py` files)
+- Execution results with response time percentiles, throughput, and error rates
+- Performance report comparing results against SLA thresholds
+- Bottleneck analysis with specific recommendations
+- CI integration configuration for automated performance regression detection
+
+## Error Handling
+
+| Error | Cause | Solution |
+|-------|-------|---------|
+| Connection reset by peer | Server or load balancer dropping connections under load | Check max connections settings; increase connection pool size; verify keep-alive configuration |
+| Timeouts spike at certain VU count | Application thread pool or database connection pool exhausted | Profile connection usage; increase pool size; add connection queuing; optimize slow queries |
+| Inconsistent results between runs | Cache warming, garbage collection pauses, or noisy neighbor effects | Run a warm-up phase before measurement; use dedicated test infrastructure; average across 3 runs |
+| Load generator CPU maxed out | Single machine cannot generate sufficient load | Distribute load generation across multiple machines; use cloud-based load generation services |
+| All requests return cached responses | Test data not sufficiently varied | Randomize request parameters; use unique IDs per request; disable CDN caching for test environment |
+
+## Examples
+
+**k6 load test script:**
+```javascript
+import http from 'k6/http';
+import { check, sleep } from 'k6';
+
+export const options = {
+  stages: [
+    { duration: '2m', target: 50 },   // Ramp up
+    { duration: '5m', target: 50 },   // Sustained load
+    { duration: '2m', target: 200 },  // Stress  # HTTP 200 OK
+    { duration: '1m', target: 0 },    // Ramp down
+  ],
+  thresholds: {
+    http_req_duration: ['p(95)<200', 'p(99)<500'],  # 500: HTTP 200 OK
+    http_req_failed: ['rate<0.01'],
+  },
+};
+
+export default function () {
+  const res = http.get('https://api.test.com/products');
+  check(res, {
+    'status is 200': (r) => r.status === 200,  # HTTP 200 OK
+    'response time OK': (r) => r.timings.duration < 300,  # 300: timeout: 5 minutes
+  });
+  sleep(1); // Think time
+}
+```
+
+**Artillery test configuration:**
+```yaml
+config:
+  target: "https://api.test.com"
+  phases:
+    - duration: 120
+      arrivalRate: 10
+      name: "Warm up"
+    - duration: 300  # 300: timeout: 5 minutes
+      arrivalRate: 50
+      name: "Sustained load"
+  ensure:
+    p95: 200  # HTTP 200 OK
+    maxErrorRate: 1
+scenarios:
+  - flow:
+      - get:
+          url: "/api/products"
+      - think: 1
+      - post:
+          url: "/api/cart"
+          json: { productId: "{{ $randomString() }}" }
+```
+
+## Resources
+
+- k6 documentation: https://grafana.com/docs/k6/latest/
+- Artillery: https://www.artillery.io/docs
+- Locust (Python): https://docs.locust.io/
+- Apache JMeter: https://jmeter.apache.org/
+- autocannon (Node.js): https://github.com/mcollina/autocannon
+- Performance testing best practices: https://grafana.com/blog/2024/01/30/load-testing-best-practices/

package/skills/running-performance-tests/assets/README.md

@@ -0,0 +1,7 @@
+# Assets
+
+Bundled resources for performance-test-suite skill
+
+- [ ] test_template.js: Template file for creating k6 performance tests.
+- [ ] report_template.html: HTML template for generating performance test reports.
+- [ ] example_test_configurations.json: Example configurations for different types of performance tests.

package/skills/running-performance-tests/assets/example_test_configurations.json

@@ -0,0 +1,121 @@
+[
+  {
+    "_comment": "Example configuration for a basic load test with gradual ramp-up",
+    "test_name": "Basic Load Test - Ramp Up",
+    "test_type": "load",
+    "description": "Simulates a gradual increase in users to test system responsiveness under increasing load.",
+    "target_url": "https://example.com/api/users",
+    "http_method": "GET",
+    "num_users": 100,
+    "ramp_up_time": 60,
+    "duration": 300,
+    "request_body": null,
+    "headers": {
+      "Content-Type": "application/json",
+      "Accept": "application/json"
+    },
+    "assertions": [
+      {
+        "type": "status_code",
+        "value": 200
+      },
+      {
+        "type": "response_time",
+        "max_ms": 500
+      }
+    ],
+    "metrics": [
+      "response_time",
+      "requests_per_second",
+      "error_rate"
+    ]
+  },
+  {
+    "_comment": "Example configuration for a stress test to find the breaking point",
+    "test_name": "Stress Test - Breaking Point",
+    "test_type": "stress",
+    "description": "Gradually increases load until the system reaches its breaking point.",
+    "target_url": "https://example.com/api/products",
+    "http_method": "POST",
+    "num_users": 500,
+    "ramp_up_time": 120,
+    "duration": 600,
+    "request_body": {
+      "product_name": "Test Product",
+      "price": 99.99
+    },
+    "headers": {
+      "Content-Type": "application/json"
+    },
+    "assertions": [
+      {
+        "type": "status_code",
+        "value": 201
+      }
+    ],
+    "metrics": [
+      "cpu_usage",
+      "memory_usage",
+      "database_connections",
+      "error_rate"
+    ]
+  },
+  {
+    "_comment": "Example configuration for a spike test simulating a flash sale",
+    "test_name": "Spike Test - Flash Sale",
+    "test_type": "spike",
+    "description": "Simulates a sudden surge in traffic to test system resilience.",
+    "target_url": "https://example.com/api/orders",
+    "http_method": "POST",
+    "num_users": 2000,
+    "ramp_up_time": 10,
+    "duration": 60,
+    "request_body": {
+      "user_id": 123,
+      "product_id": 456,
+      "quantity": 1
+    },
+    "headers": {
+      "Content-Type": "application/json"
+    },
+    "assertions": [
+      {
+        "type": "status_code",
+        "value": 201
+      }
+    ],
+    "metrics": [
+      "response_time",
+      "requests_per_second",
+      "error_rate",
+       "database_queue_length"
+    ]
+  },
+  {
+    "_comment": "Example configuration for an endurance test to check for memory leaks",
+    "test_name": "Endurance Test - Memory Leak Check",
+    "test_type": "endurance",
+    "description": "Runs a sustained load for an extended period to detect memory leaks and performance degradation.",
+    "target_url": "https://example.com/api/data",
+    "http_method": "GET",
+    "num_users": 50,
+    "ramp_up_time": 30,
+    "duration": 3600,
+    "request_body": null,
+    "headers": {
+      "Authorization": "Bearer <your_api_key>"
+    },
+    "assertions": [
+      {
+        "type": "status_code",
+        "value": 200
+      }
+    ],
+    "metrics": [
+      "memory_usage",
+      "cpu_usage",
+      "response_time",
+      "garbage_collection_count"
+    ]
+  }
+]

package/skills/running-performance-tests/assets/report_template.html

@@ -0,0 +1,203 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>Performance Test Report</title>
+    <style>
+        /* Inline CSS for styling */
+        body {
+            font-family: sans-serif;
+            margin: 0;
+            padding: 0;
+            background-color: #f4f4f4;
+            color: #333;
+        }
+
+        .container {
+            width: 90%;
+            max-width: 1200px;
+            margin: 20px auto;
+            background-color: #fff;
+            padding: 20px;
+            border-radius: 8px;
+            box-shadow: 0 0 10px rgba(0, 0, 0, 0.1);
+        }
+
+        h1, h2, h3 {
+            color: #0056b3;
+            margin-bottom: 10px;
+        }
+
+        table {
+            width: 100%;
+            border-collapse: collapse;
+            margin-top: 20px;
+        }
+
+        th, td {
+            padding: 10px;
+            text-align: left;
+            border-bottom: 1px solid #ddd;
+        }
+
+        th {
+            background-color: #f2f2f2;
+        }
+
+        .summary {
+            margin-top: 20px;
+            padding: 15px;
+            background-color: #e9ecef;
+            border-radius: 5px;
+        }
+
+        .summary p {
+            margin: 5px 0;
+        }
+
+        .charts {
+            display: flex;
+            flex-wrap: wrap;
+            justify-content: space-around;
+            margin-top: 20px;
+        }
+
+        .chart {
+            width: 45%;
+            min-width: 300px;
+            margin-bottom: 20px;
+            border: 1px solid #ddd;
+            padding: 10px;
+            border-radius: 5px;
+            box-sizing: border-box; /* Important for padding */
+        }
+
+        .chart img {
+            max-width: 100%;
+            height: auto;
+        }
+
+        @media (max-width: 768px) {
+            .container {
+                width: 95%;
+            }
+
+            .chart {
+                width: 100%;
+            }
+        }
+
+        .bottlenecks {
+            margin-top: 20px;
+            padding: 15px;
+            background-color: #ffebee;
+            border-radius: 5px;
+            border: 1px solid #ef9a9a;
+        }
+
+        .bottlenecks h3 {
+            color: #d32f2f;
+        }
+
+        .recommendations {
+            margin-top: 20px;
+            padding: 15px;
+            background-color: #e8f5e9;
+            border-radius: 5px;
+            border: 1px solid #a5d6a7;
+        }
+
+        .recommendations h3 {
+            color: #388e3c;
+        }
+    </style>
+</head>
+<body>
+
+    <!-- Main Container -->
+    <div class="container">
+
+        <!-- Report Header -->
+        <h1>Performance Test Report</h1>
+        <p>Test Run: {{test_run_id}}</p>
+        <p>Date: {{report_date}}</p>
+
+        <!-- Test Summary -->
+        <div class="summary">
+            <h2>Test Summary</h2>
+            <p><strong>Test Type:</strong> {{test_type}}</p>
+            <p><strong>Target URL:</strong> {{target_url}}</p>
+            <p><strong>Number of Users:</strong> {{number_of_users}}</p>
+            <p><strong>Duration:</strong> {{duration}}</p>
+            <p><strong>Status:</strong> {{test_status}}</p>
+        </div>
+
+        <!-- Key Metrics Table -->
+        <h2>Key Metrics</h2>
+        <table>
+            <thead>
+                <tr>
+                    <th>Metric</th>
+                    <th>Value</th>
+                </tr>
+            </thead>
+            <tbody>
+                <tr>
+                    <td>Average Response Time</td>
+                    <td>{{average_response_time}} ms</td>
+                </tr>
+                <tr>
+                    <td>Throughput</td>
+                    <td>{{throughput}} requests/second</td>
+                </tr>
+                <tr>
+                    <td>Error Rate</td>
+                    <td>{{error_rate}} %</td>
+                </tr>
+                <tr>
+                    <td>95th Percentile Response Time</td>
+                    <td>{{percentile_95_response_time}} ms</td>
+                </tr>
+            </tbody>
+        </table>
+
+        <!-- Charts -->
+        <div class="charts">
+            <div class="chart">
+                <h3>Response Time Over Time</h3>
+                <img src="{{response_time_chart_url}}" alt="Response Time Chart">
+            </div>
+            <div class="chart">
+                <h3>Throughput Over Time</h3>
+                <img src="{{throughput_chart_url}}" alt="Throughput Chart">
+            </div>
+            <!-- Add more charts as needed -->
+        </div>
+
+        <!-- Bottleneck Identification -->
+        <div class="bottlenecks">
+            <h3>Bottleneck Identification</h3>
+            <p>{{bottlenecks_summary}}</p>
+            <ul>
+                {{#bottlenecks}}
+                <li>{{.}}</li>
+                {{/bottlenecks}}
+            </ul>
+        </div>
+
+        <!-- Recommendations -->
+        <div class="recommendations">
+            <h3>Recommendations</h3>
+            <p>{{recommendations_summary}}</p>
+            <ul>
+                {{#recommendations}}
+                <li>{{.}}</li>
+                {{/recommendations}}
+            </ul>
+        </div>
+
+    </div>
+
+</body>
+</html>

package/skills/running-performance-tests/assets/test_template.js

@@ -0,0 +1,65 @@
+// test_template.js - Template for K6 performance tests
+
+// Import necessary modules from K6
+import http from 'k6/http';
+import { check, sleep } from 'k6';
+
+// Configuration options
+export const options = {
+  // Stages define the load pattern
+  stages: [
+    // Example: Ramp-up to 10 virtual users (VUs) over 10 seconds
+    { duration: '10s', target: 10 },
+
+    // Example: Maintain 10 VUs for 30 seconds
+    { duration: '30s', target: 10 },
+
+    // Example: Ramp-down to 0 VUs over 10 seconds
+    { duration: '10s', target: 0 },
+  ],
+
+  // Thresholds define pass/fail criteria
+  thresholds: {
+    // Example: 95th percentile response time should be below 200ms
+    http_req_duration: ['p95<200'],
+
+    // Example: 99% of requests should be successful
+    http_req_failed: ['rate<0.01'], // <1% failure rate
+  },
+};
+
+// Define the virtual user (VU) function
+export default function () {
+  // Replace with your target URL
+  const url = 'YOUR_TARGET_URL_HERE';
+
+  // Replace with your request parameters (optional)
+  const params = {
+    headers: {
+      'Content-Type': 'application/json',
+    },
+  };
+
+  // Replace with your request body (optional)
+  const payload = JSON.stringify({
+    key1: 'value1',
+    key2: 'value2',
+  });
+
+  // Make an HTTP request
+  const res = http.get(url, params); // or use http.post, http.put, http.delete, etc.
+
+  // Check the response status code
+  check(res, {
+    'status is 200': (r) => r.status === 200,
+  });
+
+  // Add more checks as needed
+  // Example: check(res, { 'response time < 500ms': (r) => r.timings.duration < 500 });
+
+  // Introduce a delay between requests (optional)
+  sleep(1); // Sleep for 1 second
+
+  // Log response data for debugging (optional - REMOVE IN PRODUCTION!)
+  // console.log(res.body);
+}

package/skills/running-performance-tests/references/README.md

@@ -0,0 +1,4 @@
+# References
+
+Bundled resources for performance-test-suite skill
+

package/skills/running-performance-tests/scripts/README.md

@@ -0,0 +1,11 @@
+# Scripts
+
+Bundled resources for performance-test-suite skill
+
+- [x] init_test.py: Script to initialize a performance test based on user input.
+- [x] run_test.sh: Script to execute the performance test using k6 or other tools.
+- [x] analyze_results.py: Script to analyze the test results and generate a report.
+
+
+## Auto-Generated
+Scripts generated on 2025-12-10 03:48:17

package/skills/running-performance-tests/scripts/analyze_results.py

@@ -0,0 +1,134 @@
+#!/usr/bin/env python3
+"""
+performance-test-suite - Analysis Script
+Script to analyze the test results and generate a report.
+Generated: 2025-12-10 03:48:17
+"""
+
+import os
+import json
+import argparse
+from pathlib import Path
+from typing import Dict, List
+from datetime import datetime
+
+class Analyzer:
+    def __init__(self, target_path: str):
+        self.target_path = Path(target_path)
+        self.stats = {
+            'total_files': 0,
+            'total_size': 0,
+            'file_types': {},
+            'issues': [],
+            'recommendations': []
+        }
+
+    def analyze_directory(self) -> Dict:
+        """Analyze directory structure and contents."""
+        if not self.target_path.exists():
+            self.stats['issues'].append(f"Path does not exist: {self.target_path}")
+            return self.stats
+
+        for file_path in self.target_path.rglob('*'):
+            if file_path.is_file():
+                self.analyze_file(file_path)
+
+        return self.stats
+
+    def analyze_file(self, file_path: Path):
+        """Analyze individual file."""
+        self.stats['total_files'] += 1
+        self.stats['total_size'] += file_path.stat().st_size
+
+        # Track file types
+        ext = file_path.suffix.lower()
+        if ext:
+            self.stats['file_types'][ext] = self.stats['file_types'].get(ext, 0) + 1
+
+        # Check for potential issues
+        if file_path.stat().st_size > 100 * 1024 * 1024:  # 100MB
+            self.stats['issues'].append(f"Large file: {file_path} ({file_path.stat().st_size // 1024 // 1024}MB)")
+
+        if file_path.stat().st_size == 0:
+            self.stats['issues'].append(f"Empty file: {file_path}")
+
+    def generate_recommendations(self):
+        """Generate recommendations based on analysis."""
+        if self.stats['total_files'] == 0:
+            self.stats['recommendations'].append("No files found - check target path")
+
+        if len(self.stats['file_types']) > 20:
+            self.stats['recommendations'].append("Many file types detected - consider organizing")
+
+        if self.stats['total_size'] > 1024 * 1024 * 1024:  # 1GB
+            self.stats['recommendations'].append("Large total size - consider archiving old data")
+
+    def generate_report(self) -> str:
+        """Generate analysis report."""
+        report = []
+        report.append("\n" + "="*60)
+        report.append(f"ANALYSIS REPORT - performance-test-suite")
+        report.append("="*60)
+        report.append(f"Target: {self.target_path}")
+        report.append(f"Generated: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}")
+        report.append("")
+
+        # Statistics
+        report.append("📊 STATISTICS")
+        report.append(f"  Total Files: {self.stats['total_files']:,}")
+        report.append(f"  Total Size: {self.stats['total_size'] / 1024 / 1024:.2f} MB")
+        report.append(f"  File Types: {len(self.stats['file_types'])}")
+
+        # Top file types
+        if self.stats['file_types']:
+            report.append("\n📁 TOP FILE TYPES")
+            sorted_types = sorted(self.stats['file_types'].items(), key=lambda x: x[1], reverse=True)[:5]
+            for ext, count in sorted_types:
+                report.append(f"  {ext or 'no extension'}: {count} files")
+
+        # Issues
+        if self.stats['issues']:
+            report.append(f"\n⚠️  ISSUES ({len(self.stats['issues'])})")
+            for issue in self.stats['issues'][:10]:
+                report.append(f"  - {issue}")
+            if len(self.stats['issues']) > 10:
+                report.append(f"  ... and {len(self.stats['issues']) - 10} more")
+
+        # Recommendations
+        if self.stats['recommendations']:
+            report.append("\n💡 RECOMMENDATIONS")
+            for rec in self.stats['recommendations']:
+                report.append(f"  - {rec}")
+
+        report.append("")
+        return "\n".join(report)
+
+def main():
+    parser = argparse.ArgumentParser(description="Script to analyze the test results and generate a report.")
+    parser.add_argument('target', help='Target directory to analyze')
+    parser.add_argument('--output', '-o', help='Output report file')
+    parser.add_argument('--json', action='store_true', help='Output as JSON')
+
+    args = parser.parse_args()
+
+    print(f"🔍 Analyzing {args.target}...")
+    analyzer = Analyzer(args.target)
+    stats = analyzer.analyze_directory()
+    analyzer.generate_recommendations()
+
+    if args.json:
+        output = json.dumps(stats, indent=2)
+    else:
+        output = analyzer.generate_report()
+
+    if args.output:
+        Path(args.output).write_text(output)
+        print(f"✓ Report saved to {args.output}")
+    else:
+        print(output)
+
+    return 0 if len(stats['issues']) == 0 else 1
+
+if __name__ == "__main__":
+    import sys
+    sys.exit(main())