@assertrx/dino 1.0.10 → 1.0.12

package/skills/analyzing-test-results/reference/bottleneck-patterns.md

@@ -0,0 +1,130 @@
+# k6 Bottleneck Patterns Reference
+
+## Diagnostic Matrix
+
+| Symptom | Primary Metric | Root Cause | Action |
+|---------|---------------|------------|--------|
+| High blocked time | `http_req_blocked` > 100ms | Connection pool exhaustion, DNS issues | Increase connection pool, use HTTP keep-alive |
+| High connecting time | `http_req_connecting` > 50ms | Server distance, TCP overhead | Use closer servers, CDN, connection reuse |
+| High TLS time | `http_req_tls_handshaking` > 100ms | New TLS connection per request | Enable keep-alive, TLS session resumption |
+| High waiting time | `http_req_waiting` > 500ms | Slow server processing | Optimize backend, DB queries, caching |
+| High receiving time | `http_req_receiving` > 100ms | Large response body | Compress responses, paginate, reduce payload |
+| High error rate | `http_req_failed` > 1% | Server overload, bugs | Check error codes, server logs, capacity |
+| Low throughput | `http_reqs` rate declining | Resource saturation | Scale infrastructure, optimize code |
+| Dropped iterations | `dropped_iterations` > 0 | Not enough VUs for target rate | Increase `preAllocatedVUs` or `maxVUs` |
+
+## Pattern 1: Connection Pool Exhaustion
+
+**Symptoms:**
+- `http_req_blocked` p95 > 100ms (often > 1s)
+- `http_req_blocked` increases over time
+- `http_req_connecting` stays low
+
+**Diagnosis:** k6 cannot open new connections fast enough. Requests queue waiting for available connections.
+
+**Solutions:**
+- Check target server connection limits
+- Reduce concurrent VU count
+- Add think time (`sleep()`) between requests
+- Check if server supports HTTP/2 (multiplexing)
+
+## Pattern 2: TLS Overhead
+
+**Symptoms:**
+- `http_req_tls_handshaking` consistently > 50ms
+- First requests have much higher duration than subsequent ones
+- `http_req_blocked` correlates with TLS timing
+
+**Diagnosis:** New TLS connections being established for each request instead of reusing.
+
+**Solutions:**
+- k6 reuses connections by default within an iteration
+- Ensure server supports TLS session resumption
+- Check if `noConnectionReuse` option is accidentally set
+- Consider HTTP/2 for multiplexing over single connection
+
+## Pattern 3: Server-Side Bottleneck
+
+**Symptoms:**
+- `http_req_waiting` (TTFB) increases as VUs increase
+- `http_req_waiting` is the dominant component of duration
+- `http_req_sending` and `http_req_receiving` remain low
+
+**Diagnosis:** Server is struggling under load. Could be CPU, memory, database, or application bottleneck.
+
+**Solutions:**
+- Profile server-side code
+- Optimize database queries (add indexes, caching)
+- Scale horizontally (add servers)
+- Implement caching (Redis, CDN)
+- Check for thread/connection pool limits
+
+## Pattern 4: Response Size Bottleneck
+
+**Symptoms:**
+- `http_req_receiving` is high relative to total duration
+- `data_received` is very high
+- Throughput drops as response sizes increase
+
+**Diagnosis:** Large response payloads consuming bandwidth and processing time.
+
+**Solutions:**
+- Enable response compression (gzip, brotli)
+- Implement pagination for list endpoints
+- Return only necessary fields (field selection)
+- Use `discardResponseBodies: true` in k6 options if body isn't needed
+
+## Pattern 5: Gradual Degradation (Soak Test)
+
+**Symptoms:**
+- Metrics gradually worsen over time
+- `http_req_duration` slowly increases during soak test
+- Error rate slowly creeps up
+
+**Diagnosis:** Memory leak, connection leak, or resource exhaustion in server.
+
+**Solutions:**
+- Monitor server-side memory usage
+- Check for connection/handle leaks
+- Verify database connection pool sizing
+- Check log file growth
+- Monitor disk space
+
+## Pattern 6: Sudden Failure (Breakpoint)
+
+**Symptoms:**
+- Metrics suddenly spike at specific VU count or RPS
+- Error rate jumps from ~0% to high percentage
+- `http_req_duration` increases dramatically
+
+**Diagnosis:** System has hit a hard limit (connection limit, thread pool, memory).
+
+**Solutions:**
+- Identify the exact threshold where failure begins
+- Check server error logs at that timestamp
+- Common limits: max connections, max threads, memory, file descriptors
+- Tune the specific limit or scale infrastructure
+
+## Correlation Between Metrics
+
+**Healthy test:**
+- `http_req_duration` stable across all percentiles
+- `http_req_failed` near 0%
+- `http_reqs` rate matches expected throughput
+- `dropped_iterations` = 0
+
+**Signs of trouble:**
+- p99 >> p50 (high variance = inconsistent performance)
+- `http_req_failed` increasing over time
+- `http_reqs` rate declining while VUs are constant
+- `http_req_blocked` growing (connection issues)
+- `dropped_iterations` increasing (can't keep up with arrival rate)
+
+## Performance Ratio Guidelines
+
+| Ratio | Healthy | Warning | Critical |
+|-------|---------|---------|----------|
+| p95 / p50 | < 2x | 2-5x | > 5x |
+| p99 / p50 | < 3x | 3-10x | > 10x |
+| Error rate | < 0.1% | 0.1-1% | > 1% |
+| TTFB / Duration | < 80% | 80-95% | > 95% |

package/skills/analyzing-test-results/reference/metrics.md

@@ -0,0 +1,134 @@
+# k6 Metrics Reference
+
+## Built-in HTTP Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `http_reqs` | Counter | Total HTTP requests made |
+| `http_req_duration` | Trend | Total request time (sending + waiting + receiving) |
+| `http_req_blocked` | Trend | Time blocked before initiating request |
+| `http_req_connecting` | Trend | Time establishing TCP connection |
+| `http_req_tls_handshaking` | Trend | Time performing TLS handshake |
+| `http_req_sending` | Trend | Time sending request body |
+| `http_req_waiting` | Trend | Time waiting for server response (TTFB) |
+| `http_req_receiving` | Trend | Time receiving response body |
+| `http_req_failed` | Rate | Rate of failed requests (non-2xx/3xx or expected status) |
+
+## Execution Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `vus` | Gauge | Current number of active virtual users |
+| `vus_max` | Gauge | Maximum possible number of VUs |
+| `iterations` | Counter | Total number of completed iterations |
+| `iteration_duration` | Trend | Time to complete one full iteration |
+| `dropped_iterations` | Counter | Iterations that couldn't start (arrival-rate executors) |
+| `data_received` | Counter | Amount of data received (bytes) |
+| `data_sent` | Counter | Amount of data sent (bytes) |
+
+## Browser Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `browser_web_vital_lcp` | Trend | Largest Contentful Paint |
+| `browser_web_vital_fcp` | Trend | First Contentful Paint |
+| `browser_web_vital_cls` | Trend | Cumulative Layout Shift |
+| `browser_web_vital_inp` | Trend | Interaction to Next Paint |
+| `browser_web_vital_ttfb` | Trend | Time to First Byte |
+| `browser_web_vital_fid` | Trend | First Input Delay (deprecated) |
+| `browser_data_received` | Counter | Browser data received (bytes) |
+| `browser_data_sent` | Counter | Browser data sent (bytes) |
+| `browser_http_req_duration` | Trend | Browser HTTP request duration |
+| `browser_http_req_failed` | Rate | Browser failed request rate |
+
+## WebSocket Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `ws_connecting` | Trend | WebSocket connection time |
+| `ws_session_duration` | Trend | Total session duration |
+| `ws_msgs_sent` | Counter | Messages sent |
+| `ws_msgs_received` | Counter | Messages received |
+| `ws_ping` | Trend | Ping round-trip time |
+| `ws_sessions` | Counter | Sessions started |
+
+## gRPC Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `grpc_req_duration` | Trend | gRPC request duration |
+| `grpc_streams` | Counter | Streams initiated |
+| `grpc_streams_msgs_sent` | Counter | Stream messages sent |
+| `grpc_streams_msgs_received` | Counter | Stream messages received |
+
+## Check Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `checks` | Rate | Overall rate of successful checks |
+
+## Group Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `group_duration` | Trend | Time to complete a group |
+
+## Custom Metric Types
+
+### Counter — Cumulative sum
+
+```javascript
+import { Counter } from 'k6/metrics';
+const errors = new Counter('api_errors');
+
+errors.add(1);                    // Increment by 1
+errors.add(5);                    // Increment by 5
+// Threshold: count, rate
+// 'api_errors': ['count<100']
+```
+
+### Gauge — Instantaneous value
+
+```javascript
+import { Gauge } from 'k6/metrics';
+const queueSize = new Gauge('queue_size');
+
+queueSize.add(42);                // Set value
+// Threshold: value
+// 'queue_size': ['value<1000']
+```
+
+### Rate — Percentage of non-zero values
+
+```javascript
+import { Rate } from 'k6/metrics';
+const successRate = new Rate('success_rate');
+
+successRate.add(true);            // Success (1)
+successRate.add(false);           // Failure (0)
+successRate.add(1);               // Success
+successRate.add(0);               // Failure
+// Threshold: rate
+// 'success_rate': ['rate>0.95']
+```
+
+### Trend — Statistics over time
+
+```javascript
+import { Trend } from 'k6/metrics';
+const apiLatency = new Trend('api_latency');
+
+apiLatency.add(150);              // Add sample (ms)
+apiLatency.add(200);
+// Threshold: avg, min, max, med, p(N)
+// 'api_latency': ['p(95)<500', 'avg<200']
+```
+
+## Metric Type Aggregation Methods
+
+| Type | Available Methods | Example |
+|------|------------------|---------|
+| Counter | `count`, `rate` | `count<100`, `rate>10` |
+| Gauge | `value` | `value<1000` |
+| Rate | `rate` | `rate>0.95` |
+| Trend | `avg`, `min`, `max`, `med`, `p(N)` | `p(95)<500`, `avg<200` |

package/skills/analyzing-test-results/reference/visualization.md

@@ -0,0 +1,152 @@
+# k6 Results Visualization Reference
+
+## Output Options
+
+### Console Summary (Default)
+
+k6 prints an end-of-test summary to stdout by default. No configuration needed.
+
+### JSON Lines (Real-Time Streaming)
+
+```bash
+k6 run --out json=results.json script.js
+```
+
+Each line is a JSON object with metric data, timestamps, and tags. Useful for post-processing.
+
+### CSV Output
+
+```bash
+k6 run --out csv=results.csv script.js
+```
+
+### Web Dashboard (Built-in)
+
+```bash
+K6_WEB_DASHBOARD=true k6 run script.js
+# Access at http://localhost:5665
+```
+
+Real-time visualization in the browser. Shows charts for all metrics.
+
+### Multiple Outputs
+
+```bash
+k6 run --out json=results.json --out csv=results.csv script.js
+```
+
+## handleSummary() Customization
+
+Override the default end-of-test summary:
+
+```javascript
+import { textSummary } from 'https://jslib.k6.io/k6-summary/0.1.0/index.js';
+
+export function handleSummary(data) {
+  return {
+    // Console output
+    stdout: textSummary(data, { indent: ' ', enableColors: true }),
+    // JSON file
+    'summary.json': JSON.stringify(data, null, 2),
+    // Custom HTML report
+    'report.html': generateHtmlReport(data),
+  };
+}
+
+function generateHtmlReport(data) {
+  const metrics = data.metrics;
+  return `
+    <html>
+    <body>
+      <h1>k6 Test Report</h1>
+      <h2>HTTP Request Duration</h2>
+      <p>Average: ${metrics.http_req_duration.values.avg.toFixed(2)}ms</p>
+      <p>P95: ${metrics.http_req_duration.values['p(95)'].toFixed(2)}ms</p>
+      <p>P99: ${metrics.http_req_duration.values['p(99)'].toFixed(2)}ms</p>
+      <h2>Error Rate</h2>
+      <p>${(metrics.http_req_failed.values.rate * 100).toFixed(2)}%</p>
+      <h2>Throughput</h2>
+      <p>${metrics.http_reqs.values.rate.toFixed(2)} req/s</p>
+    </body>
+    </html>
+  `;
+}
+```
+
+### handleSummary Data Structure
+
+```javascript
+{
+  root_group: {
+    name: '',
+    path: '',
+    groups: [...],
+    checks: [...],
+  },
+  metrics: {
+    http_req_duration: {
+      type: 'trend',
+      contains: 'time',
+      values: {
+        avg: 131.01,
+        min: 116.4,
+        med: 127.6,
+        max: 196.38,
+        'p(90)': 146.21,
+        'p(95)': 176.91,
+      },
+      thresholds: {
+        'p(95)<500': { ok: true },
+      },
+    },
+    // ... other metrics
+  },
+}
+```
+
+## Grafana Cloud k6
+
+### Running with Cloud Results
+
+```bash
+# Set token
+export K6_CLOUD_TOKEN=your-token-here
+
+# Run locally, send results to cloud
+k6 cloud run --local-execution script.js
+
+# Run entirely in cloud
+k6 cloud run script.js
+```
+
+### Cloud Dashboard Features
+
+- Real-time metric visualization
+- Performance trend comparison across runs
+- Threshold pass/fail tracking
+- Team sharing and collaboration
+- Historical result storage
+
+## Post-Processing JSON Results
+
+### Using jq
+
+```bash
+# Extract all http_req_duration data points
+cat results.json | jq 'select(.type=="Point" and .metric=="http_req_duration")'
+
+# Get all failed requests
+cat results.json | jq 'select(.type=="Point" and .metric=="http_req_failed" and .data.value==1)'
+```
+
+### Using Python
+
+```python
+import json
+
+with open('results.json') as f:
+    for line in f:
+        data = json.loads(line)
+        if data.get('metric') == 'http_req_duration':
+            print(f"Duration: {data['data']['value']}ms")
+```

package/skills/designing-test-scenarios/SKILL.md

@@ -0,0 +1,247 @@
+---
+name: designing-test-scenarios
+description: Use when designing k6 load profiles, choosing executors, configuring VU patterns, setting thresholds, or planning test scenarios. Use when the user asks about stress test, spike test, soak test, breakpoint test, load model, executor selection, or pass/fail criteria.
+---
+
+# Designing k6 Test Scenarios
+
+Design load profiles, select executors, configure thresholds, and orchestrate multi-scenario tests. This skill helps choose the right approach for your performance testing goals.
+
+## Test Type Selection Guide
+
+Choose a test type based on your goal:
+
+| Goal | Test Type | VUs | Duration | Details |
+|------|-----------|-----|----------|---------|
+| Verify script works, get baseline | **Smoke** | 2-5 | 30s-3m | Run after every script change |
+| Assess typical load performance | **Load** | Average production | 5-60min | Gradual ramp-up, sustained plateau |
+| Check behavior under heavy load | **Stress** | 50-100%+ above avg | 10-60min | Higher than normal, expect degradation |
+| Detect long-running issues (leaks) | **Soak** | Average production | 3-72 hours | Same as load but extended duration |
+| Survive sudden traffic burst | **Spike** | Very high, sudden | 2-5min | Rapid ramp, minimal plateau |
+| Find system capacity limits | **Breakpoint** | Incremental to failure | Until break | Continuous ramp, no plateau |
+
+See [reference/test-types.md](reference/test-types.md) for detailed configurations and stage patterns per test type.
+
+## Executor Selection
+
+```
+What drives your test?
+│
+├─ Fixed number of total iterations → shared-iterations
+│   (e.g., "run exactly 1000 requests")
+│
+├─ Fixed iterations per VU → per-vu-iterations
+│   (e.g., "each VU runs exactly 50 requests")
+│
+├─ Constant VU count for duration → constant-vus
+│   (e.g., "keep 20 users active for 5 minutes")
+│
+├─ Variable VU count over time → ramping-vus
+│   (e.g., "ramp from 0 to 100 users over 10 minutes")
+│
+├─ Constant request rate (RPS) → constant-arrival-rate
+│   (e.g., "maintain 50 requests/second")
+│
+└─ Variable request rate over time → ramping-arrival-rate
+    (e.g., "ramp from 10 to 100 requests/second")
+```
+
+See [reference/executors.md](reference/executors.md) for complete configuration parameters and examples for each executor.
+
+## Open vs Closed Model
+
+**Closed Model** (VU-based executors): Next iteration starts when previous finishes. System slowdown reduces throughput.
+- Executors: `shared-iterations`, `per-vu-iterations`, `constant-vus`, `ramping-vus`
+- Use for: Simple tests, when iteration count matters more than rate
+
+**Open Model** (arrival-rate executors): Iterations start at fixed rate regardless of response time. Maintains throughput even under stress.
+- Executors: `constant-arrival-rate`, `ramping-arrival-rate`
+- Use for: Realistic RPS testing, stress testing, avoiding coordinated omission
+
+**Recommendation:** Prefer open model (arrival-rate) for production-like load testing. It maintains target throughput even when the system slows down, giving more accurate results.
+
+## Scenario Configuration
+
+### Basic Scenario
+
+```javascript
+export const options = {
+  scenarios: {
+    my_scenario: {
+      executor: 'constant-vus',
+      vus: 50,
+      duration: '5m',
+    },
+  },
+};
+```
+
+### Common Scenario Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `executor` | string | required | Executor type |
+| `startTime` | string | `"0s"` | Delay before scenario starts |
+| `gracefulStop` | string | `"30s"` | Wait time for in-flight iterations |
+| `exec` | string | `"default"` | Exported function to execute |
+| `env` | object | `{}` | Scenario-specific environment variables |
+| `tags` | object | `{}` | Scenario-specific tags |
+
+### Multi-Scenario Orchestration
+
+Run different load patterns simultaneously or sequentially:
+
+```javascript
+export const options = {
+  scenarios: {
+    // Scenario 1: Browse products (majority of traffic)
+    browse: {
+      executor: 'constant-arrival-rate',
+      rate: 100,
+      timeUnit: '1s',
+      duration: '10m',
+      preAllocatedVUs: 50,
+      exec: 'browseProducts',
+      tags: { scenario: 'browse' },
+    },
+    // Scenario 2: Purchase flow (smaller portion)
+    purchase: {
+      executor: 'constant-arrival-rate',
+      rate: 10,
+      timeUnit: '1s',
+      duration: '10m',
+      preAllocatedVUs: 20,
+      startTime: '2m',  // Start 2 minutes after browse
+      exec: 'purchaseItem',
+      tags: { scenario: 'purchase' },
+    },
+    // Scenario 3: Spike test (starts at 5 minutes)
+    spike: {
+      executor: 'ramping-vus',
+      startVUs: 0,
+      stages: [
+        { duration: '30s', target: 500 },
+        { duration: '1m', target: 500 },
+        { duration: '30s', target: 0 },
+      ],
+      startTime: '5m',
+      exec: 'browseProducts',
+      tags: { scenario: 'spike' },
+    },
+  },
+};
+
+export function browseProducts() { /* ... */ }
+export function purchaseItem() { /* ... */ }
+```
+
+## Threshold Design
+
+See [reference/thresholds.md](reference/thresholds.md) for complete threshold patterns.
+
+### Quick Reference
+
+```javascript
+export const options = {
+  thresholds: {
+    // Response time
+    http_req_duration: ['p(95)<500', 'p(99)<1000'],
+    // Error rate
+    http_req_failed: ['rate<0.01'],
+    // Throughput
+    http_reqs: ['rate>100'],
+    // Check pass rate
+    checks: ['rate>0.99'],
+    // Custom metric
+    my_custom_trend: ['avg<200', 'p(90)<400'],
+    // Tag-based (different thresholds per endpoint)
+    'http_req_duration{name:GetUser}': ['p(95)<300'],
+    'http_req_duration{name:CreateOrder}': ['p(95)<800'],
+  },
+};
+```
+
+### Abort on Failure
+
+```javascript
+export const options = {
+  thresholds: {
+    http_req_duration: [{
+      threshold: 'p(99)<1000',
+      abortOnFail: true,
+      delayAbortEval: '30s', // Wait 30s before evaluating
+    }],
+  },
+};
+```
+
+## Custom Metrics
+
+```javascript
+import { Trend, Counter, Rate, Gauge } from 'k6/metrics';
+
+const loginDuration = new Trend('login_duration');
+const failedLogins = new Counter('failed_logins');
+const successRate = new Rate('login_success_rate');
+const activeUsers = new Gauge('active_users');
+
+export const options = {
+  thresholds: {
+    login_duration: ['p(95)<2000'],
+    login_success_rate: ['rate>0.95'],
+  },
+};
+
+export default function () {
+  const start = Date.now();
+  const res = http.post(/* login request */);
+  loginDuration.add(Date.now() - start);
+  successRate.add(res.status === 200);
+  if (res.status !== 200) failedLogins.add(1);
+}
+```
+
+## Tags and Groups
+
+### Tags for Metric Filtering
+
+```javascript
+// Per-request tags
+http.get(url, { tags: { name: 'Homepage', type: 'page' } });
+
+// Per-scenario tags (in options)
+scenarios: {
+  api_test: {
+    executor: 'constant-vus',
+    tags: { team: 'backend' },
+  },
+}
+```
+
+### Groups for Code Organization
+
+```javascript
+import { group } from 'k6';
+
+export default function () {
+  group('Authentication', () => {
+    http.post('/login', credentials);
+  });
+
+  group('Browse Products', () => {
+    http.get('/products');
+    http.get('/products/1');
+  });
+
+  group('Checkout', () => {
+    http.post('/cart', item);
+    http.post('/checkout', payment);
+  });
+}
+```
+
+## Related Skills
+
+- For generating test scripts: `/k6:generating-api-load-tests`
+- For interpreting test results: `/k6:analyzing-test-results`
+- For browser-based testing scenarios: `/k6:generating-browser-tests`