npm - perf-skills - Versions diffs - 1.0.2 → 1.0.4 - Mend

perf-skills 1.0.2 → 1.0.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (6) hide show

package/.claude-plugin/marketplace.json +2 -2
package/.claude-plugin/plugin.json +1 -1
package/README.md +35 -35
package/package.json +1 -1
package/skills/perf/SKILL.md +226 -71
package/skills/perf/evals/evals.json +149 -0

package/.claude-plugin/marketplace.json CHANGED Viewed

@@ -6,14 +6,14 @@
   },
   "metadata": {
     "description": "Expert AI skill for performance testing — covers the full lifecycle using JMeter, k6, Gatling, Locust, NeoLoad, LoadRunner, and OctoPerf.",
-    "version": "1.0.2"
+    "version": "1.0.4"
   },
   "plugins": [
     {
       "name": "perf",
       "source": "./",
       "description": "Expert guidance for performance testing: planning, scripting, executing, and analyzing load/stress/soak/spike tests across 7 tools and 10 topic areas.",
-      "version": "1.0.2",
+      "version": "1.0.4",
       "author": {
         "name": "NaveenKumar Namachivayam",
         "email": "contact@qainsights.com"

package/.claude-plugin/plugin.json CHANGED Viewed

@@ -1,6 +1,6 @@
 {
   "name": "perf",
   "description": "Expert guidance for performance testing: planning, scripting, executing, and analyzing load/stress/soak/spike tests using JMeter, k6, Gatling, Locust, NeoLoad, LoadRunner, OctoPerf.",
-  "version": "1.0.2",
+  "version": "1.0.4",
   "skills": "./skills/"
 }

package/README.md CHANGED Viewed

@@ -1,12 +1,12 @@
 # perf-skills
-> *"The more you sweat in training, the less you bleed in battle."* — Richard Marcinko
+> *"The more you sweat in training, the less you bleed in battle."* - Richard Marcinko
 >
 > The more relevant skills your AI loads during development, the fewer fires you fight in production.
 ## What Is This?
-`perf-skills` is a structured knowledge base that gives AI coding assistants deep, opinionated expertise in performance testing. It works with any AI tool that supports custom instructions, rules, or skill files — helping you plan, script, execute, and analyze load tests using any major tool.
+`perf-skills` is a structured knowledge base that gives AI coding assistants deep, opinionated expertise in performance testing. It works with any AI tool that supports custom instructions, rules, or skill files - helping you plan, script, execute, and analyze load tests using any major tool.
 ### Supported Tools
@@ -19,22 +19,22 @@
 ### Topics Covered
-- **Workload design** — concurrency models, Little's Law, transaction mix, SLA targets
-- **Test data** — parameterization, CSV feeds, synthetic data, data isolation
-- **Script generation** — correlation, assertions, error handling, session management
-- **Test execution** — local, distributed, CI/CD pipelines, cloud execution
-- **Results analysis** — percentiles, bottleneck identification, trend comparison, reporting
-- **Observability** — APM integration, Prometheus/Grafana, distributed tracing, log correlation
-- **Production testing** — canary, shadow traffic, chaos engineering, safety controls
-- **Protocol testing** — gRPC, GraphQL, WebSocket/SSE, Kafka/message queues
-- **Database testing** — JDBC load testing, connection pools, query concurrency, replication lag
-- **Modern architectures** — microservices, Kubernetes (HPA, service mesh), serverless (cold starts), frontend (Core Web Vitals)
+- **Workload design** - concurrency models, Little's Law, transaction mix, SLA targets
+- **Test data** - parameterization, CSV feeds, synthetic data, data isolation
+- **Script generation** - correlation, assertions, error handling, session management
+- **Test execution** - local, distributed, CI/CD pipelines, cloud execution
+- **Results analysis** - percentiles, bottleneck identification, trend comparison, reporting
+- **Observability** - APM integration, Prometheus/Grafana, distributed tracing, log correlation
+- **Production testing** - canary, shadow traffic, chaos engineering, safety controls
+- **Protocol testing** - gRPC, GraphQL, WebSocket/SSE, Kafka/message queues
+- **Database testing** - JDBC load testing, connection pools, query concurrency, replication lag
+- **Modern architectures** - microservices, Kubernetes (HPA, service mesh), serverless (cold starts), frontend (Core Web Vitals)
 ## Compatible AI Coding Assistants
 | Tool | Integration Method | Setup |
 |---|---|---|
-| **Windsurf (Cascade)** | Skills | Copy to skills directory — auto-triggers on perf questions |
+| **Windsurf (Cascade)** | Skills | Copy to skills directory - auto-triggers on perf questions |
 | **Cursor** | Rules / Docs | Add as project rules or index via `@Docs` |
 | **Claude Code** | CLAUDE.md / Custom instructions | Reference files in `CLAUDE.md` or feed as context |
 | **Cline** | Custom instructions / `.clinerules` | Add to `.clinerules` or workspace instructions |
@@ -88,12 +88,12 @@ cp -r perf-skills/skills/perf ~/.claude/skills/
 ### Cursor (Rules / Docs)
-**Option A — Project Rules:**
+**Option A - Project Rules:**
 1. Clone the repository: `git clone https://github.com/QAInsights/perf-skills.git`
 2. Create `.cursor/rules/perf.mdc` in your project root.
 3. Copy content from `perf-skills/skills/perf/SKILL.md` into the rule file.
-**Option B — @Docs indexing:**
+**Option B - @Docs indexing:**
 1. Clone the repository into your project or a known location.
 2. Open Cursor Settings → Features → Docs.
 3. Add the `perf-skills/skills/perf/` directory as a doc source.
@@ -154,7 +154,7 @@ The skill is plain markdown files. Any AI tool that can read files or accept cus
 ### As a Standalone Knowledge Base
-Browse the markdown files directly — they're self-contained references useful even without an AI assistant.
+Browse the markdown files directly - they're self-contained references useful even without an AI assistant.
 ## File Structure
@@ -167,16 +167,16 @@ perf-skills/                              # Repository root
 ├── LICENSE.md
 └── skills/
     └── perf/                             # Skill module
-        ├── SKILL.md                      # Entry point — tool selection, lifecycle, key principles
+        ├── SKILL.md                      # Entry point - tool selection, lifecycle, key principles
         └── references/
             ├── tools/                    # Tool-specific syntax and configuration
-            │   ├── jmeter.md             # JMeter 5.6+ — samplers, extractors, plugins, Groovy
-            │   ├── k6.md                 # k6 v0.50+ — executors, checks, thresholds, modules
-            │   ├── gatling.md            # Gatling 3.10+ — Scala/Java DSL, feeders, injection
-            │   ├── locust.md             # Locust 2.20+ — Python scripts, events, FastHttpUser
-            │   ├── neoload.md            # NeoLoad — GUI workflow, CLI, API execution
-            │   ├── loadrunner.md         # LoadRunner — VuGen, protocols, Controller scenarios
-            │   └── octoperf.md           # OctoPerf — JMeter-based SaaS, HAR import, cloud
+            │   ├── jmeter.md             # JMeter 5.6+ - samplers, extractors, plugins, Groovy
+            │   ├── k6.md                 # k6 v0.50+ - executors, checks, thresholds, modules
+            │   ├── gatling.md            # Gatling 3.10+ - Scala/Java DSL, feeders, injection
+            │   ├── locust.md             # Locust 2.20+ - Python scripts, events, FastHttpUser
+            │   ├── neoload.md            # NeoLoad - GUI workflow, CLI, API execution
+            │   ├── loadrunner.md         # LoadRunner - VuGen, protocols, Controller scenarios
+            │   └── octoperf.md           # OctoPerf - JMeter-based SaaS, HAR import, cloud
             └── topics/                   # Cross-cutting concepts (tool-agnostic)
                 ├── workload-design.md        # Concurrency models, load profiles, Little's Law
                 ├── test-data.md              # CSV, DB seeding, Faker, data isolation patterns
@@ -196,16 +196,16 @@ perf-skills/                              # Repository root
 `SKILL.md` acts as the entry point and router. It contains:
-1. **Loading Priority Rules** — tells the AI which file(s) to load based on the user's question (never all at once).
-2. **Reference Map** — maps user intent to the right file.
-3. **Protocol Routing Table** — maps protocols (gRPC, GraphQL, etc.) to recommended tools and reference files.
-4. **Tool Selection Matrix** — helps recommend a tool when the user hasn't chosen one.
-5. **Key Principles** — the single source of truth for cross-cutting best practices (assertions, think time, parameterization, correlation).
+1. **Loading Priority Rules** - tells the AI which file(s) to load based on the user's question (never all at once).
+2. **Reference Map** - maps user intent to the right file.
+3. **Protocol Routing Table** - maps protocols (gRPC, GraphQL, etc.) to recommended tools and reference files.
+4. **Tool Selection Matrix** - helps recommend a tool when the user hasn't chosen one.
+5. **Key Principles** - the single source of truth for cross-cutting best practices (assertions, think time, parameterization, correlation).
 ### Design Principles
 - **Token-efficient**: Tool files contain only tool-specific syntax. Cross-cutting concepts live in topic files. No duplication.
-- **Selective loading**: The AI loads 1–2 files per question, not the entire knowledge base.
+- **Selective loading**: The AI loads 1-2 files per question, not the entire knowledge base.
 - **Single source of truth**: Each concept is defined in exactly one place. Tool files cross-reference topic files for shared concepts.
 - **Opinionated**: The skill prescribes best practices, not just documentation. It tells you what to do, not just what's possible.
@@ -216,7 +216,7 @@ perf-skills/                              # Repository root
 | "Help me write a k6 load test for our REST API" | `k6.md` |
 | "How should I design the workload for our e-commerce app?" | `workload-design.md` |
 | "Set up JMeter in our GitHub Actions pipeline" | `test-execution.md` + `jmeter.md` |
-| "Our p95 latency is spiking at 500 VUs — how do I debug?" | `results-analysis.md` + `observability.md` |
+| "Our p95 latency is spiking at 500 VUs - how do I debug?" | `results-analysis.md` + `observability.md` |
 | "How do I load test a gRPC service?" | `protocol-testing.md` + `k6.md` |
 | "What tool should I use? We're a Python team." | `SKILL.md` (Tool Selection Matrix) |
 | "Test our Kafka consumer throughput" | `protocol-testing.md` |
@@ -227,10 +227,10 @@ perf-skills/                              # Repository root
 To add or update content:
-1. **Tool-specific content** goes in `references/tools/<tool>.md` — syntax, config, tool-unique tips only.
-2. **Cross-cutting concepts** go in `references/topics/<topic>.md` — patterns that apply across tools.
-3. **Never duplicate** — if a concept exists in a topic file, tool files should cross-reference it, not restate it.
-4. **Update SKILL.md** if you add a new file — add it to the Reference Map and Protocol Routing Table if applicable.
+1. **Tool-specific content** goes in `references/tools/<tool>.md` - syntax, config, tool-unique tips only.
+2. **Cross-cutting concepts** go in `references/topics/<topic>.md` - patterns that apply across tools.
+3. **Never duplicate** - if a concept exists in a topic file, tool files should cross-reference it, not restate it.
+4. **Update SKILL.md** if you add a new file - add it to the Reference Map and Protocol Routing Table if applicable.
 5. **Add a version indicator** (`> Targets: ...`) to new tool files.
 ## License

package/package.json CHANGED Viewed

@@ -1,6 +1,6 @@
 {
   "name": "perf-skills",
-  "version": "1.0.2",
+  "version": "1.0.4",
   "description": "Expert AI skill for performance testing — covers JMeter, k6, Gatling, Locust, NeoLoad, LoadRunner, OctoPerf across the full lifecycle",
   "main": "skills/perf/SKILL.md",
   "files": [

package/skills/perf/SKILL.md CHANGED Viewed

@@ -1,63 +1,80 @@
 ---
 name: perf
-description: Use when load testing with JMeter, k6, Gatling, Locust, NeoLoad, LoadRunner, or OctoPerf.
+description: Performance testing expert covering the full lifecycle for
+  JMeter, k6, Gatling, Locust, NeoLoad, and LoadRunner. Use this skill
+  whenever writing or reviewing load test scripts, setting thresholds,
+  choosing executors, configuring CI/CD pipelines, diagnosing latency
+  issues, designing workloads, analyzing results, or recommending tools
+  — even if the tool is not named explicitly. Always consult before
+  suggesting thresholds, executor types, or output configuration.
+  Prefer this skill over general knowledge for any performance testing
+  decision, debugging session, or tool comparison.
 ---
 # Performance Testing Skill
-This skill provides expert, opinionated guidance across the full performance testing lifecycle — from workload design through production observation. It covers both commercial tools (LoadRunner, NeoLoad, OctoPerf) and open-source tools (JMeter, k6, Gatling, Locust).
+This skill provides expert, opinionated guidance across the full
+performance testing lifecycle — from workload design through production
+observation. It covers both commercial tools (LoadRunner, NeoLoad,
+OctoPerf) and open-source tools (JMeter, k6, Gatling, Locust).
 ---
 ## How to Use This Skill
-Read the relevant reference files based on what the user needs. Multiple files may apply.
+Read the relevant reference files based on what the user needs.
+Multiple files may apply.
 ### Loading Priority Rules
 1. **Tool-specific syntax/config** → load the tool file only.
-2. **Strategy/concepts** (workload design, test data, analysis) → load the topic file only.
-3. **Both apply** (e.g., "JMeter CI/CD") → load the topic file first for patterns, then the tool file for syntax.
+2. **Strategy/concepts** (workload design, test data, analysis) → load
+   the topic file only.
+3. **Both apply** (e.g., "JMeter CI/CD") → load the topic file first
+   for patterns, then the tool file for syntax.
 4. **Never load all files at once** — select the 1–2 most relevant.
-5. **Cross-cutting principles** (assertions, think time, parameterization) → this file's Key Principles section is the single source of truth.
+5. **Cross-cutting principles** (assertions, think time,
+   parameterization) → this file's Key Principles section is the
+   single source of truth.
 ### Reference Map
-| User needs help with... | Read this file |
-|---|---|
-| Choosing the right tool | This file — see Tool Selection Matrix below |
-| JMeter scripts, plugins, config | `references/tools/jmeter.md` |
-| k6 scripting, extensions, cloud | `references/tools/k6.md` |
-| Gatling simulations, Scala/Java DSL | `references/tools/gatling.md` |
-| Locust Python tests, distributed | `references/tools/locust.md` |
-| NeoLoad projects, GUI, APIs | `references/tools/neoload.md` |
-| LoadRunner scripts, protocols, VuGen | `references/tools/loadrunner.md` |
-| OctoPerf cloud test management | `references/tools/octoperf.md` |
-| Designing workloads, concurrency, pacing | `references/topics/workload-design.md` |
-| Test data, parameterization, CSV feeds | `references/topics/test-data.md` |
-| Script patterns, correlation, best practices | `references/topics/script-generation.md` |
-| CI/CD, distributed execution, cloud runners | `references/topics/test-execution.md` |
-| Analyzing results, percentiles, SLAs | `references/topics/results-analysis.md` |
-| APM, metrics, tracing, dashboards | `references/topics/observability.md` |
-| Staging vs production testing strategies | `references/topics/production-testing.md` |
-| gRPC, GraphQL, WebSocket, messaging protocols | `references/topics/protocol-testing.md` |
-| Database load testing (JDBC, connection pools) | `references/topics/database-testing.md` |
-| Microservices, K8s, serverless performance | `references/topics/modern-architectures.md` |
+| User needs help with...                          | Read this file                                    |
+|--------------------------------------------------|---------------------------------------------------|
+| Choosing the right tool                          | This file — see Tool Selection Matrix below       |
+| JMeter scripts, plugins, config                  | `references/tools/jmeter.md`                      |
+| k6 scripting, extensions, cloud                  | `references/tools/k6.md`                          |
+| Gatling simulations, Scala/Java DSL              | `references/tools/gatling.md`                     |
+| Locust Python tests, distributed                 | `references/tools/locust.md`                      |
+| NeoLoad projects, GUI, APIs                      | `references/tools/neoload.md`                     |
+| LoadRunner scripts, protocols, VuGen             | `references/tools/loadrunner.md`                  |
+| OctoPerf cloud test management                   | `references/tools/octoperf.md`                    |
+| Designing workloads, concurrency, pacing         | `references/topics/workload-design.md`            |
+| Test data, parameterization, CSV feeds           | `references/topics/test-data.md`                  |
+| Script patterns, correlation, best practices     | `references/topics/script-generation.md`          |
+| CI/CD, distributed execution, cloud runners      | `references/topics/test-execution.md`             |
+| Analyzing results, percentiles, SLAs             | `references/topics/results-analysis.md`           |
+| APM, metrics, tracing, dashboards                | `references/topics/observability.md`              |
+| Staging vs production testing strategies         | `references/topics/production-testing.md`         |
+| gRPC, GraphQL, WebSocket, messaging protocols    | `references/topics/protocol-testing.md`           |
+| Database load testing (JDBC, connection pools)   | `references/topics/database-testing.md`           |
+| Microservices, K8s, serverless performance       | `references/topics/modern-architectures.md`       |
 ### Protocol Routing Table
-When the user's question is protocol-specific, use this to select the right tool and reference:
+When the user's question is protocol-specific, use this to select the
+right tool and reference:
-| Protocol | Recommended Tools | Reference |
-|---|---|---|
-| HTTP / REST | k6, Gatling, JMeter | Tool file |
-| gRPC | k6, Gatling, JMeter (plugin) | `references/topics/protocol-testing.md` + tool file |
-| GraphQL | k6, Gatling | `references/topics/protocol-testing.md` + tool file |
-| WebSocket / SSE | Gatling, k6 | `references/topics/protocol-testing.md` + tool file |
-| JDBC / Database | JMeter | `references/topics/database-testing.md` + `references/tools/jmeter.md` |
-| Kafka / Message Queues | k6 (xk6-kafka), JMeter | `references/topics/protocol-testing.md` |
-| SOAP / WSDL | LoadRunner, JMeter | Tool file |
-| SAP / Citrix | LoadRunner, NeoLoad | Tool file |
+| Protocol               | Recommended Tools            | Reference                                                     |
+|------------------------|------------------------------|---------------------------------------------------------------|
+| HTTP / REST            | k6, Gatling, JMeter          | Tool file                                                     |
+| gRPC                   | k6, Gatling, JMeter (plugin) | `references/topics/protocol-testing.md` + tool file          |
+| GraphQL                | k6, Gatling                  | `references/topics/protocol-testing.md` + tool file          |
+| WebSocket / SSE        | Gatling, k6                  | `references/topics/protocol-testing.md` + tool file          |
+| JDBC / Database        | JMeter                       | `references/topics/database-testing.md` + `references/tools/jmeter.md` |
+| Kafka / Message Queues | k6 (xk6-kafka), JMeter       | `references/topics/protocol-testing.md`                      |
+| SOAP / WSDL            | LoadRunner, JMeter           | Tool file                                                     |
+| SAP / Citrix           | LoadRunner, NeoLoad          | Tool file                                                     |
 ---
@@ -65,45 +82,173 @@ When the user's question is protocol-specific, use this to select the right tool
 Use this to recommend the right tool when the user hasn't decided yet.
-| Criteria | JMeter | k6 | Gatling | Locust | NeoLoad | LoadRunner | OctoPerf |
-|---|---|---|---|---|---|---|---|
-| **Language** | GUI/XML + Groovy | JavaScript/TypeScript | Scala/Java | Python | GUI + NeoLoad DSL | VuGen C-like | Web UI (JMeter-based) |
-| **Open source** | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ (SaaS) |
+| Criteria             | JMeter              | k6                    | Gatling              | Locust         | NeoLoad          | LoadRunner              | OctoPerf              |
+|----------------------|---------------------|-----------------------|----------------------|----------------|------------------|-------------------------|-----------------------|
+| **Language**         | GUI/XML + Groovy    | JavaScript/TypeScript | Scala/Java           | Python         | GUI + NeoLoad DSL| VuGen C-like            | Web UI (JMeter-based) |
+| **Open source**      | ✅                  | ✅                    | ✅                   | ✅             | ❌               | ❌                      | ❌ (SaaS)             |
 | **Protocol support** | HTTP, JDBC, JMS, MQTT, FTP, gRPC | HTTP, gRPC, WS | HTTP, JMS, gRPC | HTTP, gRPC | HTTP, gRPC, WS, SAP | HTTP, Citrix, SAP, Flex | HTTP (JMeter-backed) |
-| **Developer-friendly** | Medium | High | High | High | Low | Low | Medium |
-| **Enterprise support** | Community + BlazeMeter | Grafana Cloud | Gatling Enterprise | Limited | ✅ | ✅ | ✅ |
-| **CI/CD integration** | Good (Maven/Gradle) | Excellent | Excellent | Good | Good | Moderate | Good |
-| **Cloud execution** | BlazeMeter, OctoPerf | Grafana Cloud | Gatling Enterprise | Self-managed | NeoLoad Cloud | AWS/on-prem | OctoPerf Cloud |
-| **Best for** | Legacy systems, JDBC, protocols | Modern APIs, TypeScript devs | High-throughput HTTP | Python teams, flexible | SAP/Citrix enterprise | Mainframe, legacy enterprise | JMeter teams needing cloud UI |
+| **Developer-friendly** | Medium            | High                  | High                 | High           | Low              | Low                     | Medium                |
+| **Enterprise support** | Community + BlazeMeter | Grafana Cloud    | Gatling Enterprise   | Limited        | ✅               | ✅                      | ✅                    |
+| **CI/CD integration** | Good (Maven/Gradle) | Excellent            | Excellent            | Good           | Good             | Moderate                | Good                  |
+| **Cloud execution**  | BlazeMeter, OctoPerf | Grafana Cloud        | Gatling Enterprise   | Self-managed   | NeoLoad Cloud    | AWS/on-prem             | OctoPerf Cloud        |
+| **Best for**         | Legacy systems, JDBC, protocols | Modern APIs, TypeScript devs | High-throughput HTTP | Python teams, flexible | SAP/Citrix enterprise | Mainframe, legacy enterprise | JMeter teams needing cloud UI |
 ### Quick decision rules
 - **Team writes code** → k6 or Gatling
 - **Team uses GUI** → JMeter or NeoLoad
 - **Python shop** → Locust
 - **SAP / mainframe / Citrix** → LoadRunner or NeoLoad
-- **Need cloud SaaS with minimal setup** → OctoPerf (JMeter) or Grafana Cloud (k6)
+- **Need cloud SaaS with minimal setup** → OctoPerf (JMeter) or
+  Grafana Cloud (k6)
 - **Free + protocol variety** → JMeter
-- **Correlation needed for session-heavy flows** → JMeter (with Correlation Recorder) or LoadRunner
+- **Correlation needed for session-heavy flows** → JMeter (with
+  Correlation Recorder) or LoadRunner
 - **gRPC or GraphQL APIs** → k6 or Gatling
 - **Message queues (Kafka, RabbitMQ)** → k6 (xk6-kafka) or JMeter
 ---
+## Common Mistakes by Tool
+These are the mistakes that cause silent CI failures, misleading
+results, or test collapse at scale. Flag them proactively whenever
+reviewing scripts or diagnosing problems — users often don't know to
+ask about them.
+### k6
+- **`check()` without `thresholds`** — checks log pass/fail but do
+  NOT fail the test run. Without thresholds, CI always reports green
+  regardless of latency. Always add `thresholds` to `options`.
+- **Data loaded inside `default()`** — loading CSV or JSON inside the
+  VU function runs on every iteration, causing massive per-iteration
+  overhead and OOM at scale. Always use `SharedArray` in the init
+  scope.
+- **`shared-iterations` for user journeys** — VUs race to claim
+  iterations and may skip steps, producing incomplete journey metrics.
+  Use `per-vu-iterations` for any multi-step flow.
+- **No `sleep()` between steps** — 100 VUs with zero think time
+  generates the absolute maximum RPS for that iteration time, far
+  exceeding what 100 real concurrent users produce. Always add
+  realistic think time.
+- **`console.log` in `default()`** — causes 30–50% throughput drop
+  under load. Use custom metrics (`Counter`, `Trend`) instead.
+- **Hardcoded `BASE_URL`** — use `__ENV.BASE_URL` for environment
+  portability.
+### JMeter
+- **Listeners enabled in non-GUI runs** — View Results Tree, Aggregate
+  Report etc. buffer all samples in memory during headless runs,
+  causing memory leak and eventual crash. Disable all listeners before
+  CI execution; use `-l results.jtl` for output.
+- **Missing correlation on session-heavy apps** — JSESSIONID, CSRF
+  tokens, ViewState, OAuth codes must be extracted and reused. Without
+  correlation, the test fails for every user after the first.
+- **Hardcoded thread counts** — parameterize via JMeter properties
+  (`${__P(threads,10)}`) so CI can override without editing the JMX.
+- **Zero think time** — never reflects real user behavior; always add
+  at minimum a Constant Timer (300–500ms) between samplers.
+- **Throughput Controller % mode misuse** — percentage applies per
+  iteration of the parent controller, not globally. Most users expect
+  global percentage; use `Total Executions` mode instead or be
+  explicit.
+### Gatling
+- **Blocking calls inside `exec()`** — any blocking I/O inside an exec
+  block stalls the entire Akka actor, killing simulation concurrency.
+  Use Gatling's async feed/session API exclusively.
+- **Missing `.check()` on responses** — without checks, 4xx and 5xx
+  responses are silently counted as successful. Always add at minimum
+  `.check(status.is(200))`.
+- **Fixed `pause()` values** — use `uniformPaused(min, max)` or
+  `normalPausedWithPercentageDuration` for realistic think time
+  distribution.
+### Locust
+- **`self.client` without `catch_response=True`** — by default, Locust
+  marks any HTTP response as success regardless of status code. Use
+  `with self.client.get(..., catch_response=True) as r:` and call
+  `r.failure()` explicitly.
+- **Unequal task weights without intent** — tasks default to equal
+  weight; if your user journey has unequal step frequency, set weights
+  explicitly or the journey ratio will be wrong.
+- **Master + workers on same machine** — causes resource contention
+  that skews both throughput and latency measurements. Always run
+  workers on separate machines or containers for distributed tests.
+---
+## Cross-Tool Concept Mapping
+Use this when users are migrating between tools or asking how a
+concept from one tool maps to another. Claude should always provide
+the specific mapping rather than a generic explanation.
+| Concept          | JMeter                  | k6                      | Gatling                  | Locust                   | LoadRunner          |
+|------------------|-------------------------|-------------------------|--------------------------|--------------------------|---------------------|
+| Virtual user     | Thread                  | VU                      | User                     | User                     | Vuser               |
+| Test plan        | .jmx file               | .js / .ts script        | Simulation class         | .py file                 | VuGen script (.usr) |
+| User entrypoint  | Thread Group            | `default()` function    | `scenario()`             | task methods             | `Action()`          |
+| Concurrency ctrl | Thread Group settings   | executor                | `inject()`               | `spawn_rate`             | Vuser Group         |
+| Think time       | Constant/Uniform Timer  | `sleep()`               | `pause()`                | `time.sleep()`           | `lr_think_time()`   |
+| Inline assertion | Response Assertion      | `check()`               | `.check()`               | `catch_response`         | `lr_eval_string()`  |
+| SLA enforcement  | Duration Assertion      | `thresholds`            | Assertions (Enterprise)  | custom + exit code       | SLA definition      |
+| Correlation      | Regex / CSS Extractor   | `res.json()` / regex    | `.check()` + `saveAs()`  | `response.text` + regex  | `web_reg_save_param`|
+| Data feed        | CSV Data Set Config     | `SharedArray`           | `feeder`                 | CSV reader               | `lr_paramarr()`     |
+| Grouping         | Transaction Controller  | `group()`               | `group()`                | task sets                | Transaction         |
+| Distributed      | Controller + Agents     | k6 cloud / k6 operator  | Gatling Enterprise       | master + workers         | Load Generator      |
+| Results output   | .jtl (CSV/XML)          | JSON / InfluxDB / cloud | simulation.log           | CSV / Locust web UI      | .lrr file           |
+---
+## Threshold Starting Points
+These are community baselines — always tell the user to adjust these
+to their actual SLA requirements. Never present them as universal
+targets.
+| Endpoint type    | p95       | p99        | Error rate | Notes                              |
+|------------------|-----------|------------|------------|------------------------------------|
+| Web page (HTML)  | < 3000ms  | < 5000ms   | < 1%       | Aligns with Google CWV LCP < 2.5s  |
+| REST API         | < 500ms   | < 1000ms   | < 1%       | Common industry baseline           |
+| Auth / Login     | < 300ms   | < 500ms    | < 0.1%     | Stricter — security-sensitive path |
+| Search / Query   | < 800ms   | < 1500ms   | < 0.5%     | Varies heavily by dataset size     |
+| Write (POST/PUT) | < 800ms   | < 1500ms   | < 0.5%     | Includes DB write latency          |
+| Checkout/Payment | < 1000ms  | < 2000ms   | < 0.1%     | Stricter — revenue-critical path   |
+| Background/async | < 5000ms  | < 10000ms  | < 2%       | Batch jobs, async processors       |
+**k6 specific:** Always define both `http_req_duration` AND
+`http_req_waiting` as separate thresholds. `http_req_waiting`
+(TTFB) isolates server-side latency from network overhead — it is
+the first metric to check when diagnosing high p95. Always include
+`checks: ['rate>0.99']`.
+**JMeter specific:** Add both a Response Assertion and a Duration
+Assertion per sampler. Never rely on listener output alone for
+CI pass/fail; use the JMeter exit code driven by assertions.
+---
 ## Performance Testing Lifecycle Overview
-Always think through these phases when helping a user — they often ask about one phase but need context from others.
+Always think through these phases when helping a user — they often ask
+about one phase but need context from others.
 ```
 1. PLAN
-   └─ Workload design → concurrency model → SLA targets → test type selection
+   └─ Workload design → concurrency model → SLA targets → test type
        → references/topics/workload-design.md
 2. DATA
-   └─ Identify variables → parameterization strategy → test data generation
+   └─ Identify variables → parameterization strategy → data generation
        → references/topics/test-data.md
 3. SCRIPT
-   └─ Record or code → correlation → parameterization → assertions → script review
+   └─ Record or code → correlation → parameterization → assertions
        → references/topics/script-generation.md + tool-specific file
 4. EXECUTE
@@ -127,38 +272,48 @@ Always think through these phases when helping a user — they often ask about o
 ## Common Performance Test Types
-| Test Type | Goal | Key Metric |
-|---|---|---|
-| **Load** | Validate system at expected load | Response time, throughput, error rate |
-| **Stress** | Find the breaking point | Max VUs before degradation, error onset |
-| **Soak/Endurance** | Detect memory leaks, slow degradation | Resource trend over time (hours) |
-| **Spike** | Behavior under sudden traffic burst | Recovery time, error spike |
-| **Capacity** | Find max sustainable load | Throughput ceiling at SLA thresholds |
-| **Smoke** | Quick sanity check | Single VU — no errors |
-| **Breakpoint** | Incremental ramp until failure | Failure threshold VU count |
+| Test Type        | Goal                                  | Key Metric                              |
+|------------------|---------------------------------------|-----------------------------------------|
+| **Load**         | Validate system at expected load      | Response time, throughput, error rate   |
+| **Stress**       | Find the breaking point               | Max VUs before degradation, error onset |
+| **Soak/Endurance** | Detect memory leaks, slow degradation | Resource trend over time (hours)      |
+| **Spike**        | Behavior under sudden traffic burst   | Recovery time, error spike              |
+| **Capacity**     | Find max sustainable load             | Throughput ceiling at SLA thresholds    |
+| **Smoke**        | Quick sanity check                    | Single VU — no errors                   |
+| **Breakpoint**   | Incremental ramp until failure        | Failure threshold VU count              |
 ---
 ## Key Principles to Always Apply
-1. **Never test against production blindly** — always have a rollback plan and alerting in place.
-2. **Baseline first** — always establish a baseline before stress or soak runs.
-3. **Think time and pacing matter** — unrealistic zero-think-time tests produce misleading results.
-4. **Parameterize everything** — hardcoded credentials, tokens, and IDs will fail at scale.
-5. **Assertions are not optional** — tests without assertions are just generating traffic, not validating behavior.
-6. **Isolate the system under test** — shared environments invalidate results.
-7. **Correlate dynamic values** — session tokens, CSRF, ViewState, etc. must be extracted and reused.
+1. **Never test against production blindly** — always have a rollback
+   plan and alerting in place.
+2. **Baseline first** — always establish a baseline before stress or
+   soak runs.
+3. **Think time and pacing matter** — unrealistic zero-think-time tests
+   produce misleading results.
+4. **Parameterize everything** — hardcoded credentials, tokens, and IDs
+   will fail at scale.
+5. **Assertions are not optional** — tests without assertions are just
+   generating traffic, not validating behavior.
+6. **Isolate the system under test** — shared environments invalidate
+   results.
+7. **Correlate dynamic values** — session tokens, CSRF, ViewState etc.
+   must be extracted and reused.
 ---
 ## Asking the Right Questions
-When a user brings a performance problem, ask (or infer) these before prescribing a solution:
+When a user brings a performance problem, ask (or infer) these before
+prescribing a solution:
 - What is the **target concurrency** (VUs or RPS)?
 - What is the **SLA** (e.g., p95 < 500ms, error rate < 1%)?
 - What is the **protocol** (HTTP/REST, gRPC, JDBC, WebSocket)?
-- Is the app **stateful** (session-based) or **stateless** (token-based)?
+- Is the app **stateful** (session-based) or **stateless**
+  (token-based)?
 - Where will tests **run from** (local, CI, cloud)?
 - What **environment** is being tested (dev, staging, prod)?
-- Is there an **APM tool** in place (Datadog, Dynatrace, Grafana, New Relic)?
+- Is there an **APM tool** in place (Datadog, Dynatrace, Grafana,
+  New Relic)?

package/skills/perf/evals/evals.json ADDED Viewed

@@ -0,0 +1,149 @@
+{
+  "skill_name": "perf",
+  "evals": [
+    {
+      "id": 1,
+      "prompt": "I'm using k6 checks to validate all my API responses. My CI pipeline shows all tests passing but users are complaining about slowness. What am I missing?",
+      "expected_output": "Identifies that checks do not fail the test run — thresholds are required for CI enforcement. Provides a concrete threshold example.",
+      "files": [],
+      "assertions": [
+        {
+          "id": "checks-vs-thresholds-distinction",
+          "text": "Explicitly states that checks do NOT produce a non-zero exit code and cannot fail a CI pipeline"
+        },
+        {
+          "id": "thresholds-required",
+          "text": "States that thresholds in options.thresholds are required to enforce SLA in CI"
+        },
+        {
+          "id": "concrete-example",
+          "text": "Provides a concrete threshold code example, not just an explanation"
+        }
+      ]
+    },
+    {
+      "id": 2,
+      "prompt": "I need to load test a Kafka consumer that processes order events. What tool should I use and how do I get started?",
+      "expected_output": "Recommends xk6-kafka with the custom build command, or JMeter with Kafka plugin. Does not suggest a generic HTTP tool.",
+      "files": [],
+      "assertions": [
+        {
+          "id": "correct-tool",
+          "text": "Recommends xk6-kafka or JMeter Kafka plugin — not a generic HTTP tool like plain k6 or Gatling HTTP"
+        },
+        {
+          "id": "xk6-build-command",
+          "text": "Includes the xk6 custom build command or JMeter plugin installation reference"
+        },
+        {
+          "id": "not-http",
+          "text": "Does not suggest testing Kafka via HTTP endpoints as a primary approach"
+        }
+      ]
+    },
+    {
+      "id": 3,
+      "prompt": "My JMeter test runs fine locally but JMeter crashes after 20 minutes in CI with an OutOfMemoryError. The test plan works on my machine with 50 threads.",
+      "expected_output": "Identifies listeners enabled in non-GUI run as the primary cause. Recommends disabling listeners and using -l flag for output.",
+      "files": [],
+      "assertions": [
+        {
+          "id": "listener-root-cause",
+          "text": "Identifies enabled listeners (View Results Tree, Aggregate Report etc.) in non-GUI runs as the memory leak cause"
+        },
+        {
+          "id": "disable-listeners",
+          "text": "Recommends disabling listeners in the test plan for CI runs"
+        },
+        {
+          "id": "jtl-output",
+          "text": "Mentions using -l results.jtl flag as the correct output method for non-GUI runs"
+        }
+      ]
+    },
+    {
+      "id": 4,
+      "prompt": "We're migrating our load tests from LoadRunner to k6. What does a VuGen Action map to in k6, and what about think time and correlation?",
+      "expected_output": "Maps VuGen Action → k6 default() function. Maps lr_think_time() → sleep(). Maps web_reg_save_param → res.json() or regex extraction.",
+      "files": [],
+      "assertions": [
+        {
+          "id": "action-mapping",
+          "text": "Maps VuGen Action() specifically to k6 default() function"
+        },
+        {
+          "id": "think-time-mapping",
+          "text": "Maps lr_think_time() to k6 sleep()"
+        },
+        {
+          "id": "correlation-mapping",
+          "text": "Maps web_reg_save_param or LoadRunner correlation to k6 res.json() or regex extraction"
+        }
+      ]
+    },
+    {
+      "id": 5,
+      "prompt": "I'm writing a k6 script for an e-commerce checkout flow: login → browse → add to cart → checkout. Should I use shared-iterations or per-vu-iterations?",
+      "expected_output": "Recommends per-vu-iterations. Explains that shared-iterations causes VUs to skip steps in multi-step journeys, producing incorrect metrics.",
+      "files": [],
+      "assertions": [
+        {
+          "id": "correct-executor",
+          "text": "Recommends per-vu-iterations for the user journey"
+        },
+        {
+          "id": "shared-iterations-problem",
+          "text": "Explains why shared-iterations is wrong for journeys — VUs race for iterations and may skip steps"
+        },
+        {
+          "id": "not-ambiguous",
+          "text": "Gives a clear recommendation, not a 'it depends' answer without a conclusion"
+        }
+      ]
+    },
+    {
+      "id": 6,
+      "prompt": "My Locust test shows zero failures even though I can see 404 and 500 errors in the server logs during the test run.",
+      "expected_output": "Identifies missing catch_response=True as the cause. Without it, Locust counts all HTTP responses as success regardless of status code.",
+      "files": [],
+      "assertions": [
+        {
+          "id": "catch-response-cause",
+          "text": "Identifies missing catch_response=True as the root cause"
+        },
+        {
+          "id": "failure-call",
+          "text": "Explains that r.failure() must be called explicitly inside the catch_response context manager"
+        },
+        {
+          "id": "code-example",
+          "text": "Provides a corrected code snippet using catch_response=True"
+        }
+      ]
+    },
+    {
+      "id": 7,
+      "prompt": "What are good starting threshold values for a REST API and a checkout payment endpoint in k6?",
+      "expected_output": "Provides differentiated thresholds: REST API p95<500ms, checkout p95<1000ms with stricter error rate. Includes both http_req_duration and http_req_waiting. Notes these are baselines to adjust.",
+      "files": [],
+      "assertions": [
+        {
+          "id": "differentiated-thresholds",
+          "text": "Gives different threshold values for REST API vs checkout — not the same value for both"
+        },
+        {
+          "id": "http-req-waiting",
+          "text": "Includes http_req_waiting as a separate threshold alongside http_req_duration"
+        },
+        {
+          "id": "checks-threshold",
+          "text": "Includes checks: rate>0.99 threshold"
+        },
+        {
+          "id": "adjust-caveat",
+          "text": "Notes that values are baselines and should be adjusted to actual SLA requirements"
+        }
+      ]
+    }
+  ]
+}