@assertrx/dino 1.0.9 → 1.0.11

package/skills/analyzing-test-results/reference/metrics.md

@@ -0,0 +1,134 @@
+# k6 Metrics Reference
+
+## Built-in HTTP Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `http_reqs` | Counter | Total HTTP requests made |
+| `http_req_duration` | Trend | Total request time (sending + waiting + receiving) |
+| `http_req_blocked` | Trend | Time blocked before initiating request |
+| `http_req_connecting` | Trend | Time establishing TCP connection |
+| `http_req_tls_handshaking` | Trend | Time performing TLS handshake |
+| `http_req_sending` | Trend | Time sending request body |
+| `http_req_waiting` | Trend | Time waiting for server response (TTFB) |
+| `http_req_receiving` | Trend | Time receiving response body |
+| `http_req_failed` | Rate | Rate of failed requests (non-2xx/3xx or expected status) |
+
+## Execution Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `vus` | Gauge | Current number of active virtual users |
+| `vus_max` | Gauge | Maximum possible number of VUs |
+| `iterations` | Counter | Total number of completed iterations |
+| `iteration_duration` | Trend | Time to complete one full iteration |
+| `dropped_iterations` | Counter | Iterations that couldn't start (arrival-rate executors) |
+| `data_received` | Counter | Amount of data received (bytes) |
+| `data_sent` | Counter | Amount of data sent (bytes) |
+
+## Browser Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `browser_web_vital_lcp` | Trend | Largest Contentful Paint |
+| `browser_web_vital_fcp` | Trend | First Contentful Paint |
+| `browser_web_vital_cls` | Trend | Cumulative Layout Shift |
+| `browser_web_vital_inp` | Trend | Interaction to Next Paint |
+| `browser_web_vital_ttfb` | Trend | Time to First Byte |
+| `browser_web_vital_fid` | Trend | First Input Delay (deprecated) |
+| `browser_data_received` | Counter | Browser data received (bytes) |
+| `browser_data_sent` | Counter | Browser data sent (bytes) |
+| `browser_http_req_duration` | Trend | Browser HTTP request duration |
+| `browser_http_req_failed` | Rate | Browser failed request rate |
+
+## WebSocket Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `ws_connecting` | Trend | WebSocket connection time |
+| `ws_session_duration` | Trend | Total session duration |
+| `ws_msgs_sent` | Counter | Messages sent |
+| `ws_msgs_received` | Counter | Messages received |
+| `ws_ping` | Trend | Ping round-trip time |
+| `ws_sessions` | Counter | Sessions started |
+
+## gRPC Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `grpc_req_duration` | Trend | gRPC request duration |
+| `grpc_streams` | Counter | Streams initiated |
+| `grpc_streams_msgs_sent` | Counter | Stream messages sent |
+| `grpc_streams_msgs_received` | Counter | Stream messages received |
+
+## Check Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `checks` | Rate | Overall rate of successful checks |
+
+## Group Metrics
+
+| Metric | Type | Description |
+|--------|------|-------------|
+| `group_duration` | Trend | Time to complete a group |
+
+## Custom Metric Types
+
+### Counter — Cumulative sum
+
+```javascript
+import { Counter } from 'k6/metrics';
+const errors = new Counter('api_errors');
+
+errors.add(1);                    // Increment by 1
+errors.add(5);                    // Increment by 5
+// Threshold: count, rate
+// 'api_errors': ['count<100']
+```
+
+### Gauge — Instantaneous value
+
+```javascript
+import { Gauge } from 'k6/metrics';
+const queueSize = new Gauge('queue_size');
+
+queueSize.add(42);                // Set value
+// Threshold: value
+// 'queue_size': ['value<1000']
+```
+
+### Rate — Percentage of non-zero values
+
+```javascript
+import { Rate } from 'k6/metrics';
+const successRate = new Rate('success_rate');
+
+successRate.add(true);            // Success (1)
+successRate.add(false);           // Failure (0)
+successRate.add(1);               // Success
+successRate.add(0);               // Failure
+// Threshold: rate
+// 'success_rate': ['rate>0.95']
+```
+
+### Trend — Statistics over time
+
+```javascript
+import { Trend } from 'k6/metrics';
+const apiLatency = new Trend('api_latency');
+
+apiLatency.add(150);              // Add sample (ms)
+apiLatency.add(200);
+// Threshold: avg, min, max, med, p(N)
+// 'api_latency': ['p(95)<500', 'avg<200']
+```
+
+## Metric Type Aggregation Methods
+
+| Type | Available Methods | Example |
+|------|------------------|---------|
+| Counter | `count`, `rate` | `count<100`, `rate>10` |
+| Gauge | `value` | `value<1000` |
+| Rate | `rate` | `rate>0.95` |
+| Trend | `avg`, `min`, `max`, `med`, `p(N)` | `p(95)<500`, `avg<200` |

package/skills/analyzing-test-results/reference/visualization.md

@@ -0,0 +1,152 @@
+# k6 Results Visualization Reference
+
+## Output Options
+
+### Console Summary (Default)
+
+k6 prints an end-of-test summary to stdout by default. No configuration needed.
+
+### JSON Lines (Real-Time Streaming)
+
+```bash
+k6 run --out json=results.json script.js
+```
+
+Each line is a JSON object with metric data, timestamps, and tags. Useful for post-processing.
+
+### CSV Output
+
+```bash
+k6 run --out csv=results.csv script.js
+```
+
+### Web Dashboard (Built-in)
+
+```bash
+K6_WEB_DASHBOARD=true k6 run script.js
+# Access at http://localhost:5665
+```
+
+Real-time visualization in the browser. Shows charts for all metrics.
+
+### Multiple Outputs
+
+```bash
+k6 run --out json=results.json --out csv=results.csv script.js
+```
+
+## handleSummary() Customization
+
+Override the default end-of-test summary:
+
+```javascript
+import { textSummary } from 'https://jslib.k6.io/k6-summary/0.1.0/index.js';
+
+export function handleSummary(data) {
+  return {
+    // Console output
+    stdout: textSummary(data, { indent: ' ', enableColors: true }),
+    // JSON file
+    'summary.json': JSON.stringify(data, null, 2),
+    // Custom HTML report
+    'report.html': generateHtmlReport(data),
+  };
+}
+
+function generateHtmlReport(data) {
+  const metrics = data.metrics;
+  return `
+    <html>
+    <body>
+      <h1>k6 Test Report</h1>
+      <h2>HTTP Request Duration</h2>
+      <p>Average: ${metrics.http_req_duration.values.avg.toFixed(2)}ms</p>
+      <p>P95: ${metrics.http_req_duration.values['p(95)'].toFixed(2)}ms</p>
+      <p>P99: ${metrics.http_req_duration.values['p(99)'].toFixed(2)}ms</p>
+      <h2>Error Rate</h2>
+      <p>${(metrics.http_req_failed.values.rate * 100).toFixed(2)}%</p>
+      <h2>Throughput</h2>
+      <p>${metrics.http_reqs.values.rate.toFixed(2)} req/s</p>
+    </body>
+    </html>
+  `;
+}
+```
+
+### handleSummary Data Structure
+
+```javascript
+{
+  root_group: {
+    name: '',
+    path: '',
+    groups: [...],
+    checks: [...],
+  },
+  metrics: {
+    http_req_duration: {
+      type: 'trend',
+      contains: 'time',
+      values: {
+        avg: 131.01,
+        min: 116.4,
+        med: 127.6,
+        max: 196.38,
+        'p(90)': 146.21,
+        'p(95)': 176.91,
+      },
+      thresholds: {
+        'p(95)<500': { ok: true },
+      },
+    },
+    // ... other metrics
+  },
+}
+```
+
+## Grafana Cloud k6
+
+### Running with Cloud Results
+
+```bash
+# Set token
+export K6_CLOUD_TOKEN=your-token-here
+
+# Run locally, send results to cloud
+k6 cloud run --local-execution script.js
+
+# Run entirely in cloud
+k6 cloud run script.js
+```
+
+### Cloud Dashboard Features
+
+- Real-time metric visualization
+- Performance trend comparison across runs
+- Threshold pass/fail tracking
+- Team sharing and collaboration
+- Historical result storage
+
+## Post-Processing JSON Results
+
+### Using jq
+
+```bash
+# Extract all http_req_duration data points
+cat results.json | jq 'select(.type=="Point" and .metric=="http_req_duration")'
+
+# Get all failed requests
+cat results.json | jq 'select(.type=="Point" and .metric=="http_req_failed" and .data.value==1)'
+```
+
+### Using Python
+
+```python
+import json
+
+with open('results.json') as f:
+    for line in f:
+        data = json.loads(line)
+        if data.get('metric') == 'http_req_duration':
+            print(f"Duration: {data['data']['value']}ms")
+```

package/skills/designing-test-scenarios/SKILL.md

@@ -0,0 +1,247 @@
+---
+name: designing-test-scenarios
+description: Use when designing k6 load profiles, choosing executors, configuring VU patterns, setting thresholds, or planning test scenarios. Use when the user asks about stress test, spike test, soak test, breakpoint test, load model, executor selection, or pass/fail criteria.
+---
+
+# Designing k6 Test Scenarios
+
+Design load profiles, select executors, configure thresholds, and orchestrate multi-scenario tests. This skill helps choose the right approach for your performance testing goals.
+
+## Test Type Selection Guide
+
+Choose a test type based on your goal:
+
+| Goal | Test Type | VUs | Duration | Details |
+|------|-----------|-----|----------|---------|
+| Verify script works, get baseline | **Smoke** | 2-5 | 30s-3m | Run after every script change |
+| Assess typical load performance | **Load** | Average production | 5-60min | Gradual ramp-up, sustained plateau |
+| Check behavior under heavy load | **Stress** | 50-100%+ above avg | 10-60min | Higher than normal, expect degradation |
+| Detect long-running issues (leaks) | **Soak** | Average production | 3-72 hours | Same as load but extended duration |
+| Survive sudden traffic burst | **Spike** | Very high, sudden | 2-5min | Rapid ramp, minimal plateau |
+| Find system capacity limits | **Breakpoint** | Incremental to failure | Until break | Continuous ramp, no plateau |
+
+See [reference/test-types.md](reference/test-types.md) for detailed configurations and stage patterns per test type.
+
+## Executor Selection
+
+```
+What drives your test?
+│
+├─ Fixed number of total iterations → shared-iterations
+│   (e.g., "run exactly 1000 requests")
+│
+├─ Fixed iterations per VU → per-vu-iterations
+│   (e.g., "each VU runs exactly 50 requests")
+│
+├─ Constant VU count for duration → constant-vus
+│   (e.g., "keep 20 users active for 5 minutes")
+│
+├─ Variable VU count over time → ramping-vus
+│   (e.g., "ramp from 0 to 100 users over 10 minutes")
+│
+├─ Constant request rate (RPS) → constant-arrival-rate
+│   (e.g., "maintain 50 requests/second")
+│
+└─ Variable request rate over time → ramping-arrival-rate
+    (e.g., "ramp from 10 to 100 requests/second")
+```
+
+See [reference/executors.md](reference/executors.md) for complete configuration parameters and examples for each executor.
+
+## Open vs Closed Model
+
+**Closed Model** (VU-based executors): Next iteration starts when previous finishes. System slowdown reduces throughput.
+- Executors: `shared-iterations`, `per-vu-iterations`, `constant-vus`, `ramping-vus`
+- Use for: Simple tests, when iteration count matters more than rate
+
+**Open Model** (arrival-rate executors): Iterations start at fixed rate regardless of response time. Maintains throughput even under stress.
+- Executors: `constant-arrival-rate`, `ramping-arrival-rate`
+- Use for: Realistic RPS testing, stress testing, avoiding coordinated omission
+
+**Recommendation:** Prefer open model (arrival-rate) for production-like load testing. It maintains target throughput even when the system slows down, giving more accurate results.
+
+## Scenario Configuration
+
+### Basic Scenario
+
+```javascript
+export const options = {
+  scenarios: {
+    my_scenario: {
+      executor: 'constant-vus',
+      vus: 50,
+      duration: '5m',
+    },
+  },
+};
+```
+
+### Common Scenario Options
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `executor` | string | required | Executor type |
+| `startTime` | string | `"0s"` | Delay before scenario starts |
+| `gracefulStop` | string | `"30s"` | Wait time for in-flight iterations |
+| `exec` | string | `"default"` | Exported function to execute |
+| `env` | object | `{}` | Scenario-specific environment variables |
+| `tags` | object | `{}` | Scenario-specific tags |
+
+### Multi-Scenario Orchestration
+
+Run different load patterns simultaneously or sequentially:
+
+```javascript
+export const options = {
+  scenarios: {
+    // Scenario 1: Browse products (majority of traffic)
+    browse: {
+      executor: 'constant-arrival-rate',
+      rate: 100,
+      timeUnit: '1s',
+      duration: '10m',
+      preAllocatedVUs: 50,
+      exec: 'browseProducts',
+      tags: { scenario: 'browse' },
+    },
+    // Scenario 2: Purchase flow (smaller portion)
+    purchase: {
+      executor: 'constant-arrival-rate',
+      rate: 10,
+      timeUnit: '1s',
+      duration: '10m',
+      preAllocatedVUs: 20,
+      startTime: '2m',  // Start 2 minutes after browse
+      exec: 'purchaseItem',
+      tags: { scenario: 'purchase' },
+    },
+    // Scenario 3: Spike test (starts at 5 minutes)
+    spike: {
+      executor: 'ramping-vus',
+      startVUs: 0,
+      stages: [
+        { duration: '30s', target: 500 },
+        { duration: '1m', target: 500 },
+        { duration: '30s', target: 0 },
+      ],
+      startTime: '5m',
+      exec: 'browseProducts',
+      tags: { scenario: 'spike' },
+    },
+  },
+};
+
+export function browseProducts() { /* ... */ }
+export function purchaseItem() { /* ... */ }
+```
+
+## Threshold Design
+
+See [reference/thresholds.md](reference/thresholds.md) for complete threshold patterns.
+
+### Quick Reference
+
+```javascript
+export const options = {
+  thresholds: {
+    // Response time
+    http_req_duration: ['p(95)<500', 'p(99)<1000'],
+    // Error rate
+    http_req_failed: ['rate<0.01'],
+    // Throughput
+    http_reqs: ['rate>100'],
+    // Check pass rate
+    checks: ['rate>0.99'],
+    // Custom metric
+    my_custom_trend: ['avg<200', 'p(90)<400'],
+    // Tag-based (different thresholds per endpoint)
+    'http_req_duration{name:GetUser}': ['p(95)<300'],
+    'http_req_duration{name:CreateOrder}': ['p(95)<800'],
+  },
+};
+```
+
+### Abort on Failure
+
+```javascript
+export const options = {
+  thresholds: {
+    http_req_duration: [{
+      threshold: 'p(99)<1000',
+      abortOnFail: true,
+      delayAbortEval: '30s', // Wait 30s before evaluating
+    }],
+  },
+};
+```
+
+## Custom Metrics
+
+```javascript
+import { Trend, Counter, Rate, Gauge } from 'k6/metrics';
+
+const loginDuration = new Trend('login_duration');
+const failedLogins = new Counter('failed_logins');
+const successRate = new Rate('login_success_rate');
+const activeUsers = new Gauge('active_users');
+
+export const options = {
+  thresholds: {
+    login_duration: ['p(95)<2000'],
+    login_success_rate: ['rate>0.95'],
+  },
+};
+
+export default function () {
+  const start = Date.now();
+  const res = http.post(/* login request */);
+  loginDuration.add(Date.now() - start);
+  successRate.add(res.status === 200);
+  if (res.status !== 200) failedLogins.add(1);
+}
+```
+
+## Tags and Groups
+
+### Tags for Metric Filtering
+
+```javascript
+// Per-request tags
+http.get(url, { tags: { name: 'Homepage', type: 'page' } });
+
+// Per-scenario tags (in options)
+scenarios: {
+  api_test: {
+    executor: 'constant-vus',
+    tags: { team: 'backend' },
+  },
+}
+```
+
+### Groups for Code Organization
+
+```javascript
+import { group } from 'k6';
+
+export default function () {
+  group('Authentication', () => {
+    http.post('/login', credentials);
+  });
+
+  group('Browse Products', () => {
+    http.get('/products');
+    http.get('/products/1');
+  });
+
+  group('Checkout', () => {
+    http.post('/cart', item);
+    http.post('/checkout', payment);
+  });
+}
+```
+
+## Related Skills
+
+- For generating test scripts: `/k6:generating-api-load-tests`
+- For interpreting test results: `/k6:analyzing-test-results`
+- For browser-based testing scenarios: `/k6:generating-browser-tests`