qa-skills 3.0.0 → 3.1.0

package/skills/qa-jmeter-writer/references/best-practices.md

@@ -1,157 +0,0 @@
-# JMeter Best Practices
-
-## Non-GUI Mode
-
-| Do | Don't |
-|----|-------|
-| Always use `-n` for load tests | Run load tests in GUI mode |
-| Disable View Results Tree in load plans | Keep View Results Tree enabled for 1000+ threads |
-| Use Aggregate Report, Summary Report | Use listeners that store full response data |
-| Set `jmeter.save.saveservice.response_data=false` | Save response data in JTL for load runs |
-
-**Why:** GUI mode consumes significant memory and is not designed for high load. View Results Tree stores every response and causes OOM.
-
----
-
-## Parameterization
-
-### CSV Data Set Config
-
-| Setting | Recommendation |
-|--------|----------------|
-| **Sharing mode** | `All threads` for shared data; `Current thread` for unique per user |
-| **Delimiter** | `,` (comma) or `\t` (tab) |
-| **Recycle on EOF** | `true` for repeated use; `false` to stop when exhausted |
-| **Stop thread on EOF** | `true` when recycle=false and you want graceful stop |
-| **File path** | Use `${__P(csvPath, default.csv)}` for CI flexibility |
-
-### Unique Data
-
-- Use enough rows for `threads × loops` when recycle=false
-- For user simulation: unique emails, usernames per row
-- Consider UUID or timestamp in JSR223 for dynamic uniqueness
-
-### Avoid
-
-- Hardcoded URLs, credentials, IDs
-- Same data for all users when testing concurrency (e.g., same login)
-
----
-
-## Correlation
-
-### Order of Execution
-
-1. **Sampler** (e.g., Login) → **PostProcessor** (extract token) → **Next Sampler** (use `${token}`)
-2. Extractors must be children of the sampler whose response they parse
-3. Use **JSON Extractor** or **Regular Expression Extractor** as PostProcessor
-
-### Variable Scope
-
-- `vars` (JMeterVariables): per-thread, cleared each iteration
-- `props` (JMeterProperties): global, shared across threads
-- Use `vars.put("name", value)` in JSR223; `${name}` in samplers
-
-### Regex Tips
-
-- Prefer non-greedy: `"token":"([^"]+)"` over `"token":"(.+)"`
-- Use `$1$` in Template for first capture group
-- Test regex in View Results Tree (debug only) or regex101.com
-
-### JSON Extraction
-
-- Use **JSON Extractor** or **JSR223 PostProcessor** with JsonSlurper
-- JSONPath: `$.data.token`, `$.items[0].id`
-
----
-
-## Result Analysis
-
-### JTL Columns
-
-| Column | Purpose |
-|--------|---------|
-| `timeStamp` | Request time |
-| `elapsed` | Total time (ms) |
-| `label` | Sampler name |
-| `responseCode` | HTTP status |
-| `success` | true/false |
-| `bytes` | Response size |
-| `latency` | Time to first byte |
-| `connect` | Connection time |
-
-### HTML Report
-
-- **Dashboard**: Overview, APDEX, response times, throughput
-- **APDEX**: Satisfied (fast), Tolerated (ok), Frustrated (slow)
-- **Throughput**: Requests per second
-- **Response Times**: Percentiles (90, 95, 99)
-
-### Interpreting Results
-
-| Metric | Good | Investigate |
-|--------|------|-------------|
-| Error rate | < 1% | > 1% |
-| p95 latency | Within SLA | Exceeds SLA |
-| Throughput | Meets target | Below target |
-| Connect time | Low, stable | High or spiking |
-
----
-
-## Test Design
-
-### Ramp-Up
-
-- **Rule of thumb**: Ramp-up = 10–20% of total test duration
-- Too short: artificial spike, connection storms
-- Too long: wastes time before steady state
-
-### Think Time
-
-- Add **Constant Timer** or **Uniform Random Timer** between requests
-- Typical: 1–3 seconds for web; 0–500 ms for API
-- Omit for throughput-only tests
-
-### Assertions
-
-- Use **Response Assertion** for status codes (200, 201)
-- Use **Duration Assertion** for SLA (e.g., 2000 ms)
-- Use **JSON Assertion** for API contract validation
-- Avoid overly strict assertions that fail on minor variations
-
-### Naming
-
-- Use descriptive **labels** for samplers (e.g., `GET /users`, `POST /login`)
-- Labels appear in reports and JTL; avoid generic names like "HTTP Request"
-
----
-
-## Performance of JMeter Itself
-
-| Tweak | Purpose |
-|-------|---------|
-| Increase JVM heap | `-Xmx4g` for 5k+ threads |
-| Disable unnecessary listeners | Reduce memory and I/O |
-| Use `jmeter.save.saveservice.*` to minimize JTL | Smaller files, faster writes |
-| Run workers on separate machines | Distribute load, avoid single-node bottleneck |
-| Use latest JMeter | Bug fixes, performance improvements |
-
----
-
-## Security
-
-- Never commit credentials; use `${__P()}` or environment variables
-- Use `user.properties` (gitignored) for local overrides
-- In CI: use secrets for API keys, tokens
-- Test against staging/QA, not production, unless explicitly approved
-
----
-
-## Integration with QA Skills
-
-| Skill | Use Case |
-|-------|----------|
-| **qa-plan-creator** | Performance plan → thread groups, duration, load profile |
-| **qa-nfr-analyst** | NFR specs → SLA assertions, duration thresholds |
-| **qa-api-contract-curator** | OpenAPI → HTTP samplers, JSON assertions |
-| **qa-requirements-generator** | Requirements → test scenarios, endpoints |

package/skills/qa-jmeter-writer/references/config.md

@@ -1,204 +0,0 @@
-# JMeter Configuration
-
-## CLI Mode (Non-GUI)
-
-### Basic Run
-
-```bash
-jmeter -n -t test.jmx -l results.jtl
-```
-
-| Option | Purpose |
-|--------|---------|
-| `-n` | Non-GUI mode (required for CI/headless) |
-| `-t test.jmx` | Test plan file |
-| `-l results.jtl` | Results file (JTL/CSV) |
-| `-j jmeter.log` | Log file |
-| `-q user.properties` | User properties file |
-
-### HTML Report Generation
-
-```bash
-jmeter -n -t test.jmx -l results.jtl -e -o report/
-```
-
-| Option | Purpose |
-|--------|---------|
-| `-e` | Generate HTML report after run |
-| `-o report/` | Output directory for HTML report |
-
-Report includes: dashboard, APDEX, response times, throughput, errors.
-
-### Property Overrides
-
-```bash
-jmeter -n -t test.jmx -l results.jtl -Jhost=api.staging.example.com -Jusers=50
-```
-
-| Option | Purpose |
-|--------|---------|
-| `-Jprop=value` | Set JMeter property (single) |
-| `-Gprop=value` | Global property (distributed mode) |
-| `-q user.properties` | Load from file |
-
-Use in test plan: `${__P(host, default)}`, `${__P(users, 10)}`.
-
----
-
-## Key Properties
-
-### JTL Output Format
-
-```properties
-# user.properties or -J
-jmeter.save.saveservice.output_format=csv
-jmeter.save.saveservice.response_data=false
-jmeter.save.saveservice.samplerData=false
-jmeter.save.saveservice.requestHeaders=false
-jmeter.save.saveservice.responseHeaders=false
-jmeter.save.saveservice.url=false
-jmeter.save.saveservice.filename=false
-jmeter.save.saveservice.hostname=false
-jmeter.save.saveservice.thread_counts=false
-jmeter.save.saveservice.latency=true
-jmeter.save.saveservice.timestamp_format=yyyy/MM/dd HH:mm:ss
-jmeter.save.saveservice.print_field_names=true
-```
-
-Minimal JTL for performance: `timeStamp`, `elapsed`, `label`, `responseCode`, `success`, `bytes`, `latency`.
-
-### HTML Report
-
-```properties
-jmeter.reportgenerator.overall_granularity=60000
-jmeter.reportgenerator.apdex_satisfied_threshold=500
-jmeter.reportgenerator.apdex_tolerated_threshold=1500
-```
-
-### JVM Heap
-
-```bash
-# Set in jmeter or jmeter.bat
-HEAP="-Xms1g -Xmx4g -XX:MaxMetaspaceSize=256m"
-```
-
-Increase for high thread counts (e.g., 10k+ threads).
-
----
-
-## Plugins
-
-### JMeter Plugins Manager
-
-Install via Plugins Manager (Options → Plugins Manager):
-
-| Plugin | Purpose |
-|--------|---------|
-| **Custom Thread Groups** | Ultimate Thread Group, Stepping Thread Group |
-| **3 Basic Graphs** | Response Times, Throughput, Active Threads |
-| **PerfMon** | Server metrics (CPU, memory) |
-| **Dummy Sampler** | Simulate responses for debugging |
-| **JSON Path Assertion** | JSONPath assertions (built-in 5.2+) |
-
-### Maven/Gradle
-
-```xml
-<!-- For programmatic JMeter execution -->
-<dependency>
-  <groupId>org.apache.jmeter</groupId>
-  <artifactId>ApacheJMeter_core</artifactId>
-  <version>5.6.3</version>
-</dependency>
-```
-
----
-
-## Distributed Testing
-
-### Architecture
-
-- **Controller**: Runs GUI/non-GUI, orchestrates workers
-- **Workers**: Run actual load, send results to controller
-
-### Setup
-
-1. **Same JMeter version** on all nodes
-2. **Same plugins** on all nodes
-3. **RMI ports**: 1099 (default), 4000 (server)
-4. **Firewall**: Allow controller ↔ workers
-
-### Start Workers
-
-```bash
-# On each worker machine
-jmeter-server
-# or
-jmeter -s -Jserver.rmi.localport=50000
-```
-
-### Run from Controller
-
-```bash
-jmeter -n -t test.jmx -l results.jtl -R worker1,worker2,worker3
-# or
-jmeter -n -t test.jmx -l results.jtl -Gremote_hosts=worker1,worker2
-```
-
-| Option | Purpose |
-|--------|---------|
-| `-R host1,host2` | Remote hosts (workers) |
-| `-Gprop=value` | Global property for all workers |
-| `-Jserver.rmi.ssl.disable=true` | Disable RMI SSL if needed |
-
-### Remote Hosts Property
-
-```properties
-# jmeter.properties
-remote_hosts=127.0.0.1:1099,worker2:1099
-```
-
----
-
-## Directory Layout
-
-```
-project/
-  jmeter/
-    plans/
-      api-load.jmx
-      api-stress.jmx
-    data/
-      users.csv
-    results/          # JTL output
-    report/           # HTML report
-  user.properties     # Overrides
-  jmeter.log
-```
-
----
-
-## CI Integration (GitHub Actions)
-
-```yaml
-- name: Run JMeter
-  run: |
-    jmeter -n -t jmeter/plans/api-load.jmx \
-      -l jmeter/results/results.jtl \
-      -e -o jmeter/report \
-      -Jhost=${{ env.API_HOST }} \
-      -Jusers=${{ env.LOAD_USERS }}
-```
-
----
-
-## User Defined Variables
-
-Define at Test Plan or Thread Group level:
-
-| Variable | Value | Usage |
-|----------|-------|-------|
-| `host` | api.example.com | `${host}` |
-| `basePath` | /v1 | `${basePath}` |
-| `protocol` | https | `${protocol}` |
-
-Prefer `${__P(var, default)}` for CI overrides.

package/skills/qa-jmeter-writer/references/patterns.md

@@ -1,242 +0,0 @@
-# JMeter Test Plan Patterns
-
-## Thread Groups
-
-### Basic Thread Group
-
-| Property | Purpose | Example |
-|----------|---------|---------|
-| `num_threads` | Number of virtual users | 10, 100, 1000 |
-| `ramp_time` | Ramp-up period (seconds) | 60, 300 |
-| `loops` | Iterations per user (-1 = forever) | 1, 10, -1 |
-| `duration` | Test duration (seconds) | Use with loops=-1 |
-| `scheduler` | Enable scheduler | true/false |
-| `delay` | Startup delay (seconds) | 0 |
-| `on_sample_error` | Action on failure | continue, stopthread, stoppage, stopnow |
-
-### Load Profile Mapping
-
-| Test Type | Threads | Ramp-up | Loops/Duration |
-|-----------|---------|---------|---------------|
-| Smoke | 1–5 | 10s | 1–5 |
-| Load | Target VUs | 10–20% of duration | Duration-based |
-| Stress | Ramp to breaking point | Gradual | Until failure |
-| Soak | Steady VUs | Ramp-up | Hours |
-| Spike | Sudden surge | Very short | Short burst |
-
-### Ultimate Thread Group (Plugin)
-
-For complex load profiles (stages), use Ultimate Thread Group or Stepping Thread Group plugins when available.
-
----
-
-## Samplers
-
-### HTTP Request Sampler
-
-```xml
-<HTTPSamplerProxy>
-  <stringProp name="HTTPSampler.domain">${__P(host,api.example.com)}</stringProp>
-  <stringProp name="HTTPSampler.path">/users</stringProp>
-  <stringProp name="HTTPSampler.method">GET</stringProp>
-  <boolProp name="HTTPSampler.follow_redirects">true</boolProp>
-  <boolProp name="HTTPSampler.auto_redirects">false</boolProp>
-  <boolProp name="HTTPSampler.use_keepalive">true</boolProp>
-  <boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
-  <elementProp name="HTTPsampler.Arguments">
-    <collectionProp name="Arguments.arguments">
-      <elementProp name="" elementType="HTTPArgument">
-        <boolProp name="HTTPArgument.always_encode">false</boolProp>
-        <stringProp name="Argument.value">{"email":"${email}"}</stringProp>
-        <stringProp name="Argument.metadata">=</stringProp>
-      </elementProp>
-    </collectionProp>
-  </elementProp>
-</HTTPSamplerProxy>
-```
-
-| Property | Purpose |
-|----------|---------|
-| `HTTPSampler.domain` | Host (or use full URL in path) |
-| `HTTPSampler.path` | Path + query |
-| `HTTPSampler.method` | GET, POST, PUT, DELETE, PATCH |
-| `HTTPSampler.protocol` | http or https |
-| `HTTPSampler.port` | Port (empty = default) |
-| `HTTPsampler.Arguments` | Request body (POST/PUT) |
-| `HTTPSampler.contentEncoding` | UTF-8 |
-
-### HTTP Header Manager
-
-Add as child of sampler or test plan:
-
-```xml
-<HeaderManager>
-  <collectionProp name="HeaderManager.headers">
-    <elementProp name="Content-Type" elementType="Header">
-      <stringProp name="Header.name">Content-Type</stringProp>
-      <stringProp name="Header.value">application/json</stringProp>
-    </elementProp>
-    <elementProp name="Authorization" elementType="Header">
-      <stringProp name="Header.name">Authorization</stringProp>
-      <stringProp name="Header.value">Bearer ${token}</stringProp>
-    </elementProp>
-  </collectionProp>
-</HeaderManager>
-```
-
-### JDBC Request Sampler
-
-For database load testing:
-
-| Property | Purpose |
-|----------|---------|
-| `dataSource` | JDBC connection pool variable |
-| `queryType` | Select, Update, Callable, Prepared |
-| `query` | SQL statement |
-
-### JMS Sampler
-
-For messaging (JMS Point-to-Point, Pub/Sub) when testing message brokers.
-
----
-
-## Assertions
-
-### Response Assertion
-
-| Field | Purpose |
-|-------|---------|
-| `Assertion.test_field` | Response Code, Response Message, Response Headers, Response Body |
-| `Assertion.pattern` | String or regex to match |
-| `Assertion.test_type` | Contains, Matches, Equals, Substring |
-| `Assertion.assume_success` | Treat as success if not applied |
-
-Common patterns:
-- Response Code: `200`, `201`, `2\d{2}`
-- Response Body: `"success":\s*true`, `"id":\s*\d+`
-
-### Duration Assertion
-
-| Property | Purpose |
-|----------|---------|
-| `DurationAssertion.duration` | Max allowed response time (ms) |
-
-Use for SLA validation (e.g., 2000 ms for p95).
-
-### Size Assertion
-
-| Property | Purpose |
-|----------|---------|
-| `SizeAssertion.size` | Expected size in bytes |
-| `SizeAssertion.assume_success` | Treat as success if not applied |
-
-### JSON Assertion (JMeter 5.2+)
-
-| Property | Purpose |
-|----------|---------|
-| `JSON_PATH` | JSONPath expression (e.g., `$.data.id`) |
-| `EXPECTED_VALUE` | Expected value or leave empty for existence |
-| `ASSERT_JSON_PATH_EXISTS` | true/false |
-
----
-
-## Extractors
-
-### Regular Expression Extractor
-
-| Property | Purpose |
-|----------|---------|
-| `Refname` | Variable name (e.g., `token`) |
-| `Regex` | Regex with capture group, e.g., `"token":"([^"]+)"` |
-| `Template` | `$1$` for first group |
-| `Match Nr` | 1 = first, -1 = random, 0 = all |
-| `Default` | Default value if no match |
-
-### JSON Extractor
-
-| Property | Purpose |
-|----------|---------|
-| `JSON_PATH` | JSONPath, e.g., `$.data.token` |
-| `VAR` | Variable name |
-| `DEFAULT` | Default if not found |
-
-### Boundary Extractor
-
-For simple left/right boundary extraction when regex is overkill.
-
----
-
-## Timers
-
-### Constant Timer
-
-| Property | Purpose |
-|----------|---------|
-| `ConstantTimer.delay` | Delay in ms (e.g., 1000 for 1s think time) |
-
-### Uniform Random Timer
-
-| Property | Purpose |
-|----------|---------|
-| `ConstantTimer.delay` | Minimum delay (ms) |
-| `RandomTimer.range` | Additional random range (ms) |
-
-Total delay = delay + random(0, range).
-
-### Gaussian Random Timer
-
-Similar to Uniform but with Gaussian distribution for more realistic think time.
-
----
-
-## Controllers
-
-### Loop Controller
-
-| Property | Purpose |
-|----------|---------|
-| `LoopController.loops` | Number of iterations |
-
-### If Controller
-
-| Property | Purpose |
-|----------|---------|
-| `IfController.condition` | JavaScript or `${__jexl3(...)}` expression |
-
-Example: `${__jexl3(${status} == 200)}`
-
-### Once Only Controller
-
-Execute child elements only on first iteration (e.g., login once).
-
-### Transaction Controller
-
-Group samplers into a transaction for aggregate timing.
-
----
-
-## Pre/Post Processors
-
-### JSR223 PreProcessor (Groovy)
-
-```groovy
-// Set variable from previous response
-vars.put("timestamp", String.valueOf(System.currentTimeMillis()));
-
-// Correlation: extract from prev sampler (use PostProcessor on prior sampler instead)
-// vars.put("token", prev.getResponseDataAsString());
-```
-
-Prefer Groovy over BeanShell (faster, maintained).
-
-### JSR223 PostProcessor
-
-```groovy
-import groovy.json.JsonSlurper
-def json = new JsonSlurper().parseText(prev.getResponseDataAsString())
-vars.put("userId", json.data.id.toString())
-```
-
-### User Parameters
-
-For per-iteration parameter overrides without CSV.