perf-skills 1.0.0

package/perf-skills/references/tools/k6.md

@@ -0,0 +1,269 @@
+# k6 Reference
+
+> Targets: k6 v0.50+, xk6 extensions ecosystem
+
+k6 is a developer-centric, open-source load testing tool built by Grafana Labs. Tests are written in JavaScript/TypeScript and executed with a Go runtime — making it fast, scriptable, and CI/CD-friendly.
+
+---
+
+## Core Concepts
+
+| Concept | Description |
+|---|---|
+| **Virtual User (VU)** | A single simulated user executing the script |
+| **Iteration** | One full execution of the default function |
+| **Scenario** | Named execution profile with its own executor and config |
+| **Executor** | Controls how VUs and iterations are scheduled |
+| **Metric** | Built-in or custom measurement (counters, gauges, rates, trends) |
+| **Check** | Inline assertion — does NOT fail the test; records pass/fail rate |
+| **Threshold** | SLA definition — FAILS the test if breached |
+| **Group** | Logical grouping of requests for metric aggregation |
+
+---
+
+## Script Structure
+
+```javascript
+import http from 'k6/http';
+import { check, sleep, group } from 'k6';
+import { Counter, Trend } from 'k6/metrics';
+
+// Custom metrics
+const loginErrors = new Counter('login_errors');
+const checkoutDuration = new Trend('checkout_duration_ms');
+
+// Test configuration
+export const options = {
+  scenarios: {
+    ramp_up: {
+      executor: 'ramping-vus',
+      startVUs: 0,
+      stages: [
+        { duration: '2m', target: 50 },
+        { duration: '5m', target: 50 },
+        { duration: '2m', target: 0 },
+      ],
+    },
+  },
+  thresholds: {
+    http_req_duration: ['p(95)<500', 'p(99)<1000'],
+    http_req_failed: ['rate<0.01'],
+    login_errors: ['count<5'],
+  },
+};
+
+// Setup (runs once before VUs start)
+export function setup() {
+  const res = http.post('https://api.example.com/auth/token', {
+    client_id: 'perf-test',
+    client_secret: __ENV.API_SECRET,
+  });
+  return { token: res.json('access_token') };
+}
+
+// Default VU function
+export default function (data) {
+  group('Login Flow', () => {
+    const res = http.post('https://api.example.com/login', JSON.stringify({
+      username: `user_${__VU}@example.com`,
+      password: 'testpass',
+    }), {
+      headers: { 'Content-Type': 'application/json', 'Authorization': `Bearer ${data.token}` },
+    });
+
+    const ok = check(res, {
+      'login status 200': (r) => r.status === 200,
+      'has userId': (r) => r.json('userId') !== undefined,
+    });
+    if (!ok) loginErrors.add(1);
+  });
+
+  sleep(Math.random() * 3 + 1); // Think time: 1–4 seconds
+}
+
+// Teardown (runs once after all VUs finish)
+export function teardown(data) {
+  // Cleanup, invalidate tokens, etc.
+}
+```
+
+---
+
+## Executors
+
+| Executor | Use Case | Key Options |
+|---|---|---|
+| `ramping-vus` | Ramp up/down | `stages: [{duration, target}]` |
+| `constant-vus` | Fixed concurrency | `vus`, `duration` |
+| `per-vu-iterations` | Each VU runs N times | `vus`, `iterations` |
+| `shared-iterations` | Total N iterations split across VUs | `vus`, `iterations` |
+| `constant-arrival-rate` | Fixed RPS (open model) | `rate`, `duration`, `preAllocatedVUs` |
+| `ramping-arrival-rate` | Ramping RPS (open model) | `stages`, `preAllocatedVUs` |
+| `externally-controlled` | Controlled via k6 REST API in real-time | — |
+
+> For closed vs open model guidance and when to use each, see `references/topics/workload-design.md`.
+
+---
+
+## Checks vs Thresholds
+
+```javascript
+// Check: logs pass/fail but does NOT fail the test
+check(res, { 'status 200': (r) => r.status === 200 });
+
+// Threshold: FAILS the test run (non-zero exit code) if breached
+// Put these in options.thresholds
+thresholds: {
+  http_req_duration: ['p(95)<500'],  // p95 must be under 500ms
+  checks: ['rate>0.99'],             // At least 99% of checks must pass
+}
+```
+
+Use thresholds for SLA enforcement in CI/CD pipelines.
+
+---
+
+## Parameterization
+
+### Environment variables
+```bash
+k6 run -e USERNAME=testuser -e PASSWORD=secret script.js
+```
+```javascript
+const username = __ENV.USERNAME;
+```
+
+### SharedArray (CSV / JSON data files)
+```javascript
+import { SharedArray } from 'k6/data';
+import papaparse from 'https://jslib.k6.io/papaparse/5.1.1/index.js';
+
+const users = new SharedArray('users', function () {
+  return papaparse.parse(open('./data/users.csv'), { header: true }).data;
+});
+
+export default function () {
+  const user = users[__VU % users.length]; // Round-robin by VU
+  // use user.username, user.password
+}
+```
+
+**SharedArray loads data once and shares it across all VUs** — critical for memory efficiency at high concurrency.
+
+---
+
+## Correlation (Session Handling)
+
+```javascript
+// Extract value from JSON response
+const token = res.json('data.access_token');
+
+// Extract from HTML (regex)
+const csrfToken = res.html().find('input[name="_token"]').attr('value');
+
+// Extract from headers
+const sessionId = res.headers['Set-Cookie'].match(/JSESSIONID=([^;]+)/)[1];
+
+// Use in next request
+http.get('https://api.example.com/profile', {
+  headers: { 'Authorization': `Bearer ${token}` },
+});
+```
+
+---
+
+## Custom Metrics
+
+```javascript
+import { Counter, Gauge, Rate, Trend } from 'k6/metrics';
+
+const checkoutErrors    = new Counter('checkout_errors');    // cumulative count
+const activeUsers       = new Gauge('active_users');         // current value
+const successRate       = new Rate('success_rate');          // ratio
+const transactionTime   = new Trend('transaction_time_ms'); // distribution (p50/p95/p99)
+
+// Usage inside default()
+transactionTime.add(res.timings.duration);
+successRate.add(res.status === 200);
+```
+
+---
+
+## Modules & Extensions
+
+### Built-in k6 modules
+- `k6/http` — HTTP/1.1 and HTTP/2
+- `k6/ws` — WebSocket
+- `k6/grpc` — gRPC
+- `k6/browser` — Browser-based testing (Chromium)
+- `k6/experimental/redis` — Redis client
+- `k6/crypto` — Hashing and HMAC
+- `k6/encoding` — Base64
+
+### Popular xk6 extensions (require custom build)
+- `xk6-kafka` — Kafka load testing
+- `xk6-sql` — SQL database testing
+- `xk6-output-prometheus-remote` — Push metrics to Prometheus
+
+Build custom k6: `xk6 build --with github.com/grafana/xk6-kafka`
+
+---
+
+## Running k6
+
+```bash
+# Local run
+k6 run script.js
+
+# With options override
+k6 run --vus 50 --duration 5m script.js
+
+# With env vars
+k6 run -e BASE_URL=https://staging.example.com script.js
+
+# Output to multiple sinks
+k6 run --out json=results.json --out influxdb=http://localhost:8086/k6 script.js
+
+# Cloud run (Grafana Cloud k6)
+k6 cloud script.js
+```
+
+---
+
+## Output and Observability
+
+| Output | Command |
+|---|---|
+| Terminal summary | Default |
+| JSON | `--out json=results.json` |
+| InfluxDB (Grafana stack) | `--out influxdb=http://influx:8086/k6` |
+| Prometheus | `--out experimental-prometheus-rw` |
+| Grafana Cloud | `k6 cloud` or `--out cloud` |
+| CSV | `--out csv=results.csv` |
+
+### InfluxDB + Grafana Dashboard
+The canonical observability stack: k6 → InfluxDB → Grafana with the official k6 Grafana dashboard (ID: 2587).
+
+---
+
+## TypeScript Support
+
+```bash
+# Use k6 template with TypeScript
+npx create-k6-app@latest --template typescript my-test
+```
+
+Benefits: Type safety, IDE autocomplete, better refactoring for large test suites.
+
+---
+
+## k6-Specific Tips
+
+- **Always use `SharedArray`** for data files — loading data inside `default()` causes massive per-iteration overhead.
+- **Use `for` loops only for sequential steps**, not concurrency — use VUs and executors for parallel load.
+- **Remove `console.log` from hot paths** — dramatically reduces throughput during load runs.
+- **Use `__ENV.BASE_URL`** for environment portability — never hardcode base URLs.
+- **Prefer thresholds over checks alone** — without thresholds, the test always "passes" in CI/CD.
+
+> For CI/CD integration (GitHub Actions, GitLab CI, Jenkins), see `references/topics/test-execution.md`.
+> For anti-patterns, assertions, think time, and parameterization principles, see **Key Principles** in `SKILL.md`.

package/perf-skills/references/tools/loadrunner.md

@@ -0,0 +1,116 @@
+# LoadRunner Reference
+
+> Targets: OpenText LoadRunner 2024+
+
+HP/Micro Focus LoadRunner (now OpenText LoadRunner) is the industry-standard enterprise load testing tool, especially dominant in financial services, telecom, and government.
+
+---
+
+## Core Components
+
+| Component | Description |
+|---|---|
+| **VuGen** | Script recorder and editor (C-based scripting) |
+| **Controller** | Orchestrates scenarios and load injection |
+| **Load Generator** | Hosts running VUs |
+| **Analysis** | Post-test results analysis and reporting |
+| **LoadRunner Cloud** | SaaS cloud execution platform |
+
+---
+
+## VuGen Script Structure
+
+```c
+// vuser_init: Runs once at startup
+vuser_init()
+{
+    lr_think_time(0);
+    web_set_user("testuser", "password", "realm");
+    return 0;
+}
+
+// Action: Main loop (repeated per iteration)
+Action()
+{
+    // Parameterize
+    lr_start_transaction("Login");
+    web_submit_form("login",
+        ITEMDATA,
+        "Name=username", "Value={P_USERNAME}", ENDITEM,
+        "Name=password", "Value={P_PASSWORD}", ENDITEM,
+        LAST);
+    lr_end_transaction("Login", LR_AUTO);
+
+    lr_think_time(3);
+
+    // Correlate session ID
+    web_reg_save_param_regexp(
+        "ParamName=SessionToken",
+        "RegExp=name=\"_token\" value=\"([^\"]+)\"",
+        "Ord=1",
+        LAST);
+    web_url("Dashboard", "URL=https://app/dashboard", LAST);
+
+    return 0;
+}
+
+// vuser_end: Runs once at teardown
+vuser_end()
+{
+    web_custom_request("Logout", "URL=https://app/logout", "Method=POST", LAST);
+    return 0;
+}
+```
+
+---
+
+## Key Protocols
+
+| Protocol | Use Case |
+|---|---|
+| Web (HTTP/HTML) | Standard web apps, REST APIs |
+| Web Services | SOAP/WSDL |
+| Citrix ICA | Virtual desktop (Citrix) |
+| SAP GUI | SAP ERP client |
+| Oracle NCA | Oracle Forms |
+| TruClient | AJAX-heavy SPAs (browser-based recording) |
+| JDBC | Database load testing |
+| JMS | Message queue testing |
+| Flex/AMF | Adobe Flex applications |
+
+---
+
+## Correlation in VuGen
+
+```c
+// Register extraction BEFORE the request it appears in
+web_reg_save_param_regexp(
+    "ParamName=AuthToken",
+    "RegExp=Bearer ([A-Za-z0-9._-]+)",
+    "Search=Headers",
+    "Ord=1",
+    LAST);
+
+// Use the parameter in subsequent requests
+web_add_header("Authorization", "Bearer {AuthToken}");
+```
+
+---
+
+## Controller Scenario Design
+
+1. **Manual Scenario** - specify VU count per script, ramp-up duration, load generator assignment.
+2. **Goal-Oriented Scenario** - Controller adjusts VUs automatically to hit a target (RPS, response time).
+3. **Percentage Mode** - define percentage of VU load for each script in a mixed workload.
+
+---
+
+## Analysis Tips
+
+- Use **Transaction Response Time** breakdown (DNS, connect, send, wait, receive).
+- Compare **Hits/sec** and **Throughput** against **Response Time** to find saturation points.
+- Use **Correlation graphs** to overlay server metrics (CPU, memory) from SiteScope/Diagnostics.
+- Export to Excel or integrate with LoadRunner Cloud for trend analysis across runs.
+
+> For anti-patterns, assertions, think time, and parameterization principles, see **Key Principles** in `SKILL.md`.
+> For CI/CD integration details, see `references/topics/test-execution.md`.

package/perf-skills/references/tools/locust.md

@@ -0,0 +1,133 @@
+# Locust Reference
+
+> Targets: Locust 2.20+, Python 3.9+
+
+Locust is a Python-based, open-source load testing tool. Tests are plain Python code — no DSL, no XML. It supports distributed testing and has a built-in web UI.
+
+---
+
+## Core Concepts
+
+| Concept | Description |
+|---|---|
+| **User class** | Defines behavior of a simulated user |
+| **TaskSet** | Group of tasks; can be nested |
+| **task decorator** | Marks a method as a task with optional weight |
+| **wait_time** | Think time between tasks |
+| **HttpUser** | Preconfigured User with HTTP client |
+| **FastHttpUser** | High-performance HTTP client (gevent-based) |
+| **events** | Hooks for setup, teardown, request success/failure |
+
+---
+
+## Basic Script
+
+```python
+from locust import HttpUser, task, between
+import json, random
+
+class ShopUser(HttpUser):
+    wait_time = between(1, 4)  # Think time: 1–4 seconds (uniform random)
+    host = "https://api.example.com"
+
+    def on_start(self):
+        """Runs once per VU at startup"""
+        res = self.client.post("/auth/login", json={
+            "username": f"user{random.randint(1, 1000)}@test.com",
+            "password": "testpass123"
+        })
+        self.token = res.json()["access_token"]
+        self.client.headers.update({"Authorization": f"Bearer {self.token}"})
+
+    @task(3)                   # Weight: called 3x more than weight-1 tasks
+    def browse_products(self):
+        self.client.get("/products", name="GET /products")
+
+    @task(1)
+    def add_to_cart(self):
+        product_id = random.randint(1, 100)
+        with self.client.post(
+            f"/cart/items",
+            json={"productId": product_id, "qty": 1},
+            catch_response=True,    # Manually handle response
+            name="POST /cart/items"
+        ) as response:
+            if response.status_code == 200:
+                response.success()
+            else:
+                response.failure(f"Expected 200, got {response.status_code}")
+
+    def on_stop(self):
+        self.client.post("/auth/logout")
+```
+
+---
+
+## Running Locust
+
+```bash
+# Web UI mode
+locust -f locustfile.py --host https://api.example.com
+
+# Headless (CLI) mode
+locust -f locustfile.py \
+  --headless \
+  --users 100 \
+  --spawn-rate 10 \
+  --run-time 5m \
+  --host https://api.example.com \
+  --html report.html \
+  --csv results
+
+# Distributed mode
+# On master:
+locust -f locustfile.py --master --users 1000 --spawn-rate 50
+# On each worker:
+locust -f locustfile.py --worker --master-host=<master-ip>
+```
+
+---
+
+## Custom Wait Times
+
+```python
+from locust import constant, constant_pacing, between, constant_throughput
+
+wait_time = between(1, 5)              # Uniform random 1–5s
+wait_time = constant(2)                # Fixed 2s
+wait_time = constant_pacing(10)        # Paces to 1 iteration per 10s (handles slow responses)
+wait_time = constant_throughput(0.1)   # Target 0.1 RPS per VU (10s average)
+```
+
+---
+
+## Custom Events and Hooks
+
+```python
+from locust import events
+
+@events.test_start.add_listener
+def on_test_start(environment, **kwargs):
+    print("Test starting — seeding test data")
+
+@events.request.add_listener
+def on_request(request_type, name, response_time, response_length, exception, **kwargs):
+    if exception:
+        print(f"FAILED: {name} — {exception}")
+
+@events.test_stop.add_listener
+def on_test_stop(environment, **kwargs):
+    print("Test finished — cleaning up data")
+```
+
+---
+
+## Locust-Specific Tips
+
+- Always use `name=` parameter to group parameterized URLs (e.g., `/products/123` → `name="GET /products/{id}"`).
+- Use `catch_response=True` for custom validation — HTTP 200 with error body passes silently otherwise.
+- Use `FastHttpUser` instead of `HttpUser` for CPU-bound or high-throughput scenarios.
+- Use distributed mode above 500–1000 VUs — a single worker process saturates one CPU core.
+
+> For anti-patterns, assertions, think time, and parameterization principles, see **Key Principles** in `SKILL.md`.
+> For CI/CD integration and distributed execution details, see `references/topics/test-execution.md`.

package/perf-skills/references/tools/neoload.md

@@ -0,0 +1,61 @@
+# NeoLoad Reference
+
+> Targets: NeoLoad 2024.x+
+
+NeoLoad is a commercial enterprise performance testing tool from Tricentis, designed for large-scale, protocol-rich testing including HTTP, SAP, Citrix, and Flex.
+
+---
+
+## Core Concepts
+
+| Concept | Description |
+|---|---|
+| **Virtual User Profile** | Defines a user type and its behavior |
+| **Population** | Group of VU profiles with percentage mix |
+| **Scenario** | Load injection profile (ramp, constant, peak) |
+| **Container** | Transaction group (like JMeter Transaction Controller) |
+| **Variable Extractor** | Correlation mechanism |
+| **SLA Profile** | KPI thresholds; alerts and pass/fail criteria |
+| **Controller** | Orchestrates test execution |
+| **Load Generator** | Injects simulated users |
+
+---
+
+## Workload Design in NeoLoad
+
+1. **Record** via NeoLoad browser proxy or VU recorder
+2. **Parameterize** using Variable Extractors (Regexp, XPath, JSONPath) and File Variables
+3. **Apply SLA Profile** with thresholds for response time, error rate, throughput
+4. **Design Population** mixing multiple VU profiles
+5. **Configure Scenario** with ramp-up policies and constant phase
+
+---
+
+## CLI / API Execution
+
+```bash
+# Run from command line
+NeoLoadCmd -project mytest.nlp \
+  -scenario "Regression_Load" \
+  -testResultName "CI_Run_${BUILD_NUMBER}" \
+  -leaseServer neoload-controller:7400
+
+# REST API trigger (for CI/CD)
+curl -X POST "https://neoload-api.tricentis.com/v3/tests/{testId}/start" \
+  -H "accountToken: $NEOLOAD_TOKEN" \
+  -d '{"scenario": "Load_Scenario"}'
+```
+
+---
+
+## NeoLoad-Specific Tips
+
+- Always set **Network Emulation** to match target environment (LAN, WAN, mobile).
+- Use **Shared Containers** for reusable sequences (e.g., authentication flow).
+- Configure **Cache and Cookie Policies** per application behavior.
+- Use **NeoLoad Web** for centralized result storage, trend analysis, and team collaboration.
+- Integrate with **Dynatrace/AppDynamics** via built-in APM connectors.
+- Use **as-code YAML definitions** (NeoLoad as Code) for version-controlled test configs.
+
+> For anti-patterns, assertions, think time, and parameterization principles, see **Key Principles** in `SKILL.md`.
+> For CI/CD integration details, see `references/topics/test-execution.md`.

package/perf-skills/references/tools/octoperf.md

@@ -0,0 +1,50 @@
+# OctoPerf Reference
+
+> Targets: OctoPerf SaaS (current)
+
+OctoPerf is a cloud-based SaaS performance testing platform built on top of JMeter. It provides a professional UI for test design, cloud execution, and centralized result management.
+
+---
+
+## Key Capabilities
+
+| Feature | Description |
+|---|---|
+| **JMeter import/export** | Import existing `.jmx` files; export OctoPerf designs as JMeter |
+| **Visual scenario builder** | GUI for scripting without raw JMeter XML editing |
+| **Cloud execution** | On-demand cloud load generators (AWS regions) |
+| **Real-time dashboards** | Live metrics during test run |
+| **Trend analysis** | Compare runs over time |
+| **SLA alerts** | Email/Slack notifications on KPI breach |
+
+---
+
+## Workflow
+
+1. **Import or Record** - upload existing JMX or use OctoPerf's HAR import (record browser -> export HAR -> import).
+2. **Parameterize** - use OctoPerf variable extractors (JSON, Regexp, Header) in the GUI.
+3. **Configure Load Profile** - ramp-up wizard or custom curve editor.
+4. **Select Regions** - pick cloud injection regions (US-East, EU-West, APAC, etc.).
+5. **Run and Monitor** - real-time dashboard with RPS, response time percentiles, error rate.
+6. **Analyze** - use built-in report builder or export to CSV/JSON.
+
+---
+
+## OctoPerf-Specific Tips
+
+- Use OctoPerf's **HAR importer** to quickly create scripts from browser recordings.
+- Leverage **Virtual User Groups** to model different user populations.
+- Store test plans in **Workspaces** with version history.
+- Use the **REST API** to trigger runs from CI/CD:
+
+```bash
+# Trigger run via API
+curl -X POST "https://api.octoperf.com/analysis/executions" \
+  -H "X-Api-Key: $OCTOPERF_API_KEY" \
+  -d '{"scenarioId": "abc123", "name": "CI Build #42"}'
+```
+
+- Always set **SLA thresholds** in the test profile so CI gates work reliably.
+
+> For anti-patterns, assertions, think time, and parameterization principles, see **Key Principles** in `SKILL.md`.
+> For CI/CD integration details, see `references/topics/test-execution.md`.