@harness-engineering/cli 1.14.0 → 1.16.0

package/dist/agents/skills/cursor/harness-perf/SKILL.md

@@ -0,0 +1,260 @@
+# Harness Perf
+
+> Performance enforcement and benchmark management. Tier-based gates block commits and merges based on complexity, coupling, and runtime regression severity.
+
+## When to Use
+
+- After code changes to verify performance hasn't degraded
+- On PRs to enforce performance budgets
+- For periodic performance audits
+- NOT for initial development (use harness-tdd for that)
+- NOT for brainstorming performance improvements (use harness-brainstorming)
+
+## Process
+
+### Iron Law
+
+**No merge with Tier 1 performance violations. No commit with cyclomatic complexity exceeding the error threshold.**
+
+Tier 1 violations are non-negotiable blockers. If a Tier 1 violation is detected, execution halts and the violation must be resolved before any further progress. Do not attempt workarounds.
+
+---
+
+### Phase 1: ANALYZE — Structural and Coupling Checks
+
+1. **Run structural checks.** Execute `harness check-perf --structural` to compute complexity metrics for all changed files:
+   - Cyclomatic complexity per function
+   - Nesting depth per function
+   - File length (lines of code)
+   - Parameter count per function
+
+2. **Run coupling checks.** Execute `harness check-perf --coupling` to compute coupling metrics:
+   - Fan-in and fan-out per module
+   - Afferent and efferent coupling
+   - Transitive dependency depth
+   - Circular dependency detection
+
+3. **Classify violations by tier:**
+   - **Tier 1 (error, block commit):** Cyclomatic complexity > 15, circular dependencies, hotspot in top 5%
+   - **Tier 2 (warning, block merge):** Complexity > 10, nesting > 4, fan-out > 10, size budget exceeded
+   - **Tier 3 (info, no gate):** File length > 300, fan-in > 20, transitive depth > 30
+
+4. **If Tier 1 violations found,** report them immediately and STOP. Do not proceed to benchmarks. The violations must be fixed first.
+
+5. **If no violations found,** proceed to Phase 2.
+
+---
+
+### Graph Availability
+
+Hotspot scoring and coupling analysis benefit from the knowledge graph but work without it.
+
+**Staleness sensitivity:** Medium -- auto-refresh if >10 commits stale. Hotspot scoring uses churn data which does not change rapidly.
+
+| Feature                              | With Graph                                                   | Without Graph                                                                                                                    |
+| ------------------------------------ | ------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------- |
+| Hotspot scoring (churn x complexity) | `GraphComplexityAdapter` computes from graph nodes           | `git log --format="%H" -- <file>` for per-file commit count; complexity from `check-perf --structural` output; multiply manually |
+| Coupling ratio                       | `GraphCouplingAdapter` computes from graph edges             | Parse import statements, count fan-out/fan-in per file                                                                           |
+| Critical path resolution             | Graph inference (high fan-in) + `@perf-critical` annotations | `@perf-critical` annotations only; grep for decorator/comment                                                                    |
+| Transitive dep depth                 | Graph BFS depth                                              | Import chain follow, 2 levels deep                                                                                               |
+
+**Notice when running without graph:** "Running without graph (run `harness scan` to enable hotspot scoring and coupling analysis)"
+
+**Impact on tiers:** Without graph, Tier 1 hotspot detection is degraded. Hotspot scoring falls back to churn-only (no complexity multiplication). This limitation is documented in the performance report output.
+
+---
+
+### Phase 2: BENCHMARK — Runtime Performance
+
+This phase runs only when `.bench.ts` files exist in the project. If none are found, skip to Phase 3.
+
+1. **Check baseline lock-in.** Before running benchmarks, verify baselines are kept in sync:
+   - List all `.bench.ts` files changed in this PR: `git diff --name-only | grep '.bench.ts'`
+   - If any `.bench.ts` files are new or modified:
+     - Check if `.harness/perf/baselines.json` is also modified in this PR
+     - If NOT modified: flag as Tier 2 warning: "Benchmark files changed but baselines not updated. Run `harness perf baselines update` and commit the result."
+     - If modified: verify the updated baselines include entries for all changed benchmarks
+   - If no `.bench.ts` files changed: skip this check
+   - This check also runs standalone via `--check-baselines` flag
+
+2. **Check for benchmark files.** Scan the project for `*.bench.ts` files. If none exist, skip this phase entirely.
+
+3. **Verify clean working tree.** Run `git status --porcelain`. If there are uncommitted changes, STOP. Benchmarks on dirty trees produce unreliable results.
+
+4. **Run benchmarks.** Execute `harness perf bench` to run all benchmark suites.
+
+5. **Load baselines.** Read `.harness/perf/baselines.json` for previous benchmark results. If no baselines exist, treat this as a baseline-capture run.
+
+6. **Compare results against baselines** using the `RegressionDetector`:
+   - Calculate percentage change for each benchmark
+   - Apply noise margin (default: 3%) before flagging regressions
+   - Distinguish between critical-path and non-critical-path benchmarks
+
+7. **Resolve critical paths** via `CriticalPathResolver`:
+   - Check `@perf-critical` annotations in source files
+   - Check graph fan-in data (functions called by many consumers)
+   - Functions in the critical path set have stricter thresholds
+
+8. **Flag regressions by tier:**
+   - **Tier 1:** >5% regression on a critical path benchmark
+   - **Tier 2:** >10% regression on a non-critical-path benchmark
+   - **Tier 3:** >5% regression on a non-critical-path benchmark (within noise margin consideration)
+
+9. **If this is a baseline-capture run,** report results without regression comparison. Recommend running `harness perf baselines update` to persist.
+
+---
+
+### Phase 3: REPORT — Generate Performance Report
+
+1. **Format violations by tier.** Present Tier 1 violations first (most severe), then Tier 2, then Tier 3. Each violation entry includes:
+   - File path and function name
+   - Metric name and current value
+   - Threshold that was exceeded
+   - Tier classification and gate impact
+
+2. **Show hotspot scores** for top functions if knowledge graph data is available:
+   - Query the graph for functions with high churn + high fan-in
+   - Rank by composite hotspot score
+   - Flag any hotspots that also have performance violations
+
+3. **Show benchmark regression summary** if benchmarks ran:
+   - Table of benchmark name, baseline, current, delta percentage, tier
+   - Highlight critical-path benchmarks with a marker
+   - Show noise margin and whether the regression exceeds it
+
+4. **Recommend specific actions** for each Tier 1 and Tier 2 violation:
+   - For high complexity: suggest extract-method or strategy pattern refactoring
+   - For high coupling: suggest interface extraction or dependency inversion
+   - For benchmark regressions: suggest profiling the specific code path
+   - For size budget violations: suggest module decomposition
+
+5. **Output the report** in structured markdown format suitable for PR comments or CI output.
+
+---
+
+### Phase 4: ENFORCE — Apply Gate Decisions
+
+1. **Tier 1 violations present** — FAIL. Block commit and merge. List all Tier 1 violations with their locations and values. The developer must fix these before proceeding.
+
+2. **Tier 2 violations present, no Tier 1** — WARN. Allow commit but block merge until addressed. List all Tier 2 violations. These must be resolved before the PR can be merged.
+
+3. **Only Tier 3 or no violations** — PASS. Proceed normally. Log Tier 3 violations as informational notes.
+
+4. **Record gate decision** in `.harness/state.json` under a `perfGate` key:
+
+   ```json
+   {
+     "perfGate": {
+       "result": "pass|warn|fail",
+       "tier1Count": 0,
+       "tier2Count": 0,
+       "tier3Count": 0,
+       "timestamp": "ISO-8601"
+     }
+   }
+   ```
+
+5. **Exit with appropriate code:** 0 for pass, 1 for fail, 0 for warn (with warning output).
+
+---
+
+## Harness Integration
+
+- **`harness check-perf`** — Primary command for all performance checks. Runs structural and coupling analysis.
+- **`harness check-perf --structural`** — Run only structural complexity checks.
+- **`harness check-perf --coupling`** — Run only coupling analysis.
+- **`harness perf bench`** — Run benchmarks only. Requires clean working tree.
+- **`harness perf baselines show`** — View current benchmark baselines.
+- **`harness perf baselines update`** — Persist current benchmark results as new baselines.
+- **`harness perf --check-baselines`** -- Verify baseline file is updated when benchmarks change. Runs the baseline lock-in check standalone.
+- **`harness perf critical-paths`** — View the current critical path set and how it was determined.
+- **`harness validate`** — Run after enforcement to verify overall project health.
+- **`harness graph scan`** — Refresh knowledge graph for accurate hotspot scoring.
+
+## Tier Classification
+
+| Tier | Severity | Gate         | Examples                                                                                            |
+| ---- | -------- | ------------ | --------------------------------------------------------------------------------------------------- |
+| 1    | error    | Block commit | Cyclomatic complexity > 15, >5% regression on critical path, hotspot in top 5%, circular dependency |
+| 2    | warning  | Block merge  | Complexity > 10, nesting > 4, >10% regression elsewhere, fan-out > 10, size budget exceeded         |
+| 3    | info     | None         | File length > 300 lines, fan-in > 20, transitive depth > 30, >5% non-critical regression            |
+
+## Success Criteria
+
+- All Tier 1 violations are resolved before proceeding
+- Performance report follows structured format with tier classification
+- Benchmark regressions are compared against noise margin before flagging
+- Gate decision is recorded in state
+- `harness validate` passes after enforcement
+
+## Examples
+
+### Example: PR with High Complexity Function
+
+```
+Phase 1: ANALYZE
+  harness check-perf --structural
+  Result: processOrderBatch() in src/orders/processor.ts has cyclomatic complexity 18 (Tier 1, threshold: 15)
+
+Phase 2: BENCHMARK — skipped (Tier 1 violation found)
+
+Phase 3: REPORT
+  TIER 1 VIOLATIONS (1):
+    - src/orders/processor.ts:processOrderBatch — complexity 18 > 15
+      Recommendation: Extract validation and transformation into separate functions
+
+Phase 4: ENFORCE
+  Result: FAIL — 1 Tier 1 violation. Commit blocked.
+```
+
+### Example: Benchmark Regression on Critical Path
+
+```
+Phase 1: ANALYZE — no structural violations
+
+Phase 2: BENCHMARK
+  harness perf bench
+  Baseline: parseDocument 4.2ms, current: 4.8ms (+14.3%)
+  parseDocument is @perf-critical — Tier 1 threshold applies (>5%)
+
+Phase 3: REPORT
+  TIER 1 VIOLATIONS (1):
+    - parseDocument: 14.3% regression on critical path (threshold: 5%)
+      Recommendation: Profile parseDocument to identify the regression source
+
+Phase 4: ENFORCE
+  Result: FAIL — 1 Tier 1 violation. Merge blocked.
+```
+
+### Example: Clean PR with Minor Warnings
+
+```
+Phase 1: ANALYZE
+  harness check-perf --structural --coupling
+  Result: src/utils/formatter.ts has 320 lines (Tier 3, threshold: 300)
+
+Phase 2: BENCHMARK
+  harness perf bench — all within noise margin
+
+Phase 3: REPORT
+  TIER 3 INFO (1):
+    - src/utils/formatter.ts: 320 lines > 300 line threshold
+  No Tier 1 or Tier 2 violations.
+
+Phase 4: ENFORCE
+  Result: PASS — no blocking violations.
+```
+
+## Gates
+
+- **No ignoring Tier 1 violations.** They must be fixed or the threshold must be reconfigured (with documented justification).
+- **No running benchmarks with dirty working tree.** Uncommitted changes invalidate benchmark results.
+- **No updating baselines without running benchmarks.** Baselines must come from fresh runs against committed code.
+- **No suppressing violations without documentation.** If a threshold is relaxed, the rationale must be documented in the project configuration.
+
+## Escalation
+
+- **When Tier 1 violations cannot be fixed within the current task:** Propose refactoring the function into smaller units, or raising the threshold with a documented justification. Do not silently skip the violation.
+- **When benchmark results are noisy or inconsistent:** Increase warmup iterations, pin the runtime environment, or run benchmarks in isolation. Report the noise level so the developer can make an informed decision.
+- **When critical path detection seems wrong:** Check `@perf-critical` annotations in source files and verify graph fan-in thresholds. The critical path set can be overridden in `.harness/perf/critical-paths.json`.
+- **When a violation is a false positive:** Document it with a `// perf-ignore: <reason>` comment and add the exception to `.harness/perf/exceptions.json`.

package/dist/agents/skills/cursor/harness-perf/skill.yaml

@@ -0,0 +1,51 @@
+name: harness-perf
+version: "1.0.0"
+description: Performance enforcement and benchmark management
+cognitive_mode: meticulous-verifier
+triggers:
+  - manual
+  - on_pr
+platforms:
+  - claude-code
+  - gemini-cli
+tools:
+  - Bash
+  - Read
+  - Write
+  - Edit
+  - Glob
+  - Grep
+cli:
+  command: harness skill run harness-perf
+  args:
+    - name: path
+      description: Project root path
+      required: false
+    - name: check-baselines
+      description: Verify baseline file is updated when benchmarks change
+      required: false
+mcp:
+  tool: run_skill
+  input:
+    skill: harness-perf
+    path: string
+type: rigid
+tier: 2
+phases:
+  - name: analyze
+    description: Run structural complexity and coupling checks
+    required: true
+  - name: benchmark
+    description: Run benchmarks and detect regressions
+    required: false
+  - name: report
+    description: Generate perf report with violations and recommendations
+    required: true
+  - name: enforce
+    description: Apply tier-based gate decisions
+    required: true
+state:
+  persistent: false
+  files: []
+depends_on:
+  - harness-verify

package/dist/agents/skills/cursor/harness-perf-tdd/SKILL.md

@@ -0,0 +1,249 @@
+# Harness Perf TDD
+
+> Red-Green-Refactor with performance assertions. Every feature gets a correctness test AND a benchmark. No optimization without measurement.
+
+## When to Use
+
+- Implementing performance-critical features
+- When the spec includes performance requirements (e.g., "must respond in < 100ms")
+- When modifying `@perf-critical` annotated code
+- When adding hot-path logic (parsers, serializers, query resolvers, middleware)
+- NOT for non-performance-sensitive code (use harness-tdd instead)
+- NOT for refactoring existing code that already has benchmarks (use harness-refactoring + harness-perf)
+
+## Process
+
+### Iron Law
+
+**No production code exists without both a failing test AND a failing benchmark that demanded its creation.**
+
+If you find yourself writing production code before both the test and the benchmark exist, STOP. Write the test. Write the benchmark. Then implement.
+
+---
+
+### Phase 1: RED — Write Failing Test + Benchmark
+
+1. **Write the correctness test** following the same process as harness-tdd Phase 1 (RED):
+   - Identify the smallest behavior to test
+   - Write ONE minimal test with a clear assertion
+   - Follow the project's test conventions
+
+2. **Write a `.bench.ts` benchmark file** alongside the test file:
+   - Co-locate with source: `handler.ts` -> `handler.bench.ts`
+   - Use Vitest bench syntax for benchmark definitions
+   - Set a performance assertion if the spec includes one
+
+   ```typescript
+   import { bench, describe } from 'vitest';
+   import { processData } from './handler';
+
+   describe('processData benchmarks', () => {
+     bench('processData with small input', () => {
+       processData(smallInput);
+     });
+
+     bench('processData with large input', () => {
+       processData(largeInput);
+     });
+   });
+   ```
+
+3. **Run the test** — observe failure. The function is not implemented yet, so the test should fail with "not defined" or "not a function."
+
+4. **Run the benchmark** — observe failure or no baseline. This establishes that the benchmark exists and will track performance once the implementation lands.
+
+---
+
+### Phase 2: GREEN — Pass Test and Benchmark
+
+1. **Write the minimum implementation** to make the correctness test pass. Do not optimize yet. The goal is correctness first.
+
+2. **Run the test** — observe pass. If it fails, fix the implementation until it passes.
+
+3. **Run the benchmark** -- capture initial results and apply thresholds:
+
+   **When the spec defines a performance requirement** (e.g., "< 50ms"):
+   - Use the spec requirement as the benchmark assertion threshold
+   - Verify it passes; if not, see step 4
+
+   **When the spec is vague or silent on performance:**
+   - Fall back to harness-perf tier thresholds:
+     - Critical path functions (annotated `@perf-critical` or high fan-in): must not regress >5% from baseline (Tier 1)
+     - Non-critical functions: must not regress >10% from baseline (Tier 2)
+     - Structural complexity: must stay under Tier 2 thresholds (cyclomatic <=15, nesting <=4, function length <=50 lines, params <=5)
+   - These thresholds give developers concrete targets even when the spec does not specify performance requirements
+
+   **When no baseline exists (new code):**
+   - This run captures the initial baseline
+   - No regression comparison on first run
+   - VALIDATE phase (Phase 4) ensures the captured baseline is committed via `harness perf baselines update`
+
+4. **If the performance assertion fails,** you have two options:
+   - The implementation approach is fundamentally wrong (e.g., O(n^2) when O(n) is needed) — revise the algorithm
+   - The assertion is too aggressive for a first pass — note it and defer to REFACTOR phase
+
+---
+
+### Phase 3: REFACTOR — Optimize While Green
+
+This phase is optional. Enter it when:
+
+- The benchmark shows room for improvement against the performance requirement
+- Profiling reveals an obvious bottleneck
+- The code can be simplified while maintaining or improving performance
+
+1. **Profile the implementation** if the benchmark result is far from the requirement. Use the benchmark output to identify the bottleneck.
+
+2. **Refactor for performance** — consider:
+   - Algorithm improvements (sort, search, data structure choice)
+   - Caching or memoization for repeated computations
+   - Reducing allocations (object pooling, buffer reuse)
+   - Eliminating unnecessary work (early returns, lazy evaluation)
+
+3. **After each change,** run both checks:
+   - **Test:** Still passing? If not, the refactor broke correctness. Revert.
+   - **Benchmark:** Improved? If not, the refactor was not effective. Consider reverting.
+
+4. **Stop when** the benchmark meets the performance requirement, or when further optimization yields diminishing returns (< 1% improvement per change).
+
+5. **Do not gold-plate.** If the requirement is "< 100ms" and you are at 40ms, stop. Move on.
+
+---
+
+### Phase 4: VALIDATE — Harness Checks
+
+1. **Run `harness check-perf`** to verify no Tier 1 or Tier 2 violations were introduced by the implementation:
+   - Cyclomatic complexity within thresholds
+   - Coupling metrics acceptable
+   - No benchmark regressions in other modules
+
+2. **Run `harness validate`** to verify overall project health:
+   - All tests pass
+   - Linter clean
+   - Type checks pass
+
+3. **Update baselines** if this is a new benchmark:
+
+   ```bash
+   harness perf baselines update
+   ```
+
+   This persists the current benchmark results so future runs can detect regressions.
+
+4. **Commit with a descriptive message** that mentions both the feature and its performance characteristics:
+   ```
+   feat(parser): add streaming JSON parser (<50ms for 1MB payloads)
+   ```
+
+---
+
+## Benchmark File Convention
+
+Benchmark files are co-located with their source files, using the `.bench.ts` extension:
+
+| Source File                   | Benchmark File                      |
+| ----------------------------- | ----------------------------------- |
+| `src/parser/handler.ts`       | `src/parser/handler.bench.ts`       |
+| `src/api/resolver.ts`         | `src/api/resolver.bench.ts`         |
+| `packages/core/src/engine.ts` | `packages/core/src/engine.bench.ts` |
+
+Each benchmark file should:
+
+- Import only from the module under test
+- Define benchmarks in a `describe` block named after the module
+- Include both small-input and large-input cases when applicable
+- Use realistic data (not empty objects or trivial inputs)
+
+---
+
+## Harness Integration
+
+- **`harness check-perf`** — Run after implementation to check for violations
+- **`harness perf bench`** — Run benchmarks in isolation
+- **`harness perf baselines update`** — Persist benchmark results as new baselines
+- **`harness validate`** — Full project health check
+- **`harness perf critical-paths`** — View critical path set to understand which benchmarks have stricter thresholds
+
+## Success Criteria
+
+- Every new function has both a test file (`.test.ts`) and a bench file (`.bench.ts`)
+- Benchmarks run without errors
+- No Tier 1 performance violations after implementation
+- Baselines are updated for new benchmarks
+- Commit message includes performance context when relevant
+
+## Examples
+
+### Example: Implementing a Performance-Critical Parser
+
+**Phase 1: RED**
+
+```typescript
+// src/parser/json-stream.test.ts
+it('parses 1MB JSON in under 50ms', () => {
+  const result = parseStream(largeMbPayload);
+  expect(result).toEqual(expectedOutput);
+});
+
+// src/parser/json-stream.bench.ts
+bench('parseStream 1MB', () => {
+  parseStream(largeMbPayload);
+});
+```
+
+Run test: FAIL (parseStream not defined). Run benchmark: FAIL (no implementation).
+
+**Phase 2: GREEN**
+
+```typescript
+// src/parser/json-stream.ts
+export function parseStream(input: string): ParsedResult {
+  return JSON.parse(input); // simplest correct implementation
+}
+```
+
+Run test: PASS. Run benchmark: 38ms average (meets <50ms requirement).
+
+**Phase 3: REFACTOR** — skipped (38ms already meets 50ms target).
+
+**Phase 4: VALIDATE**
+
+```
+harness check-perf — no violations
+harness validate — passes
+harness perf baselines update — baseline saved
+git commit -m "feat(parser): add streaming JSON parser (<50ms for 1MB payloads)"
+```
+
+### Example: Optimizing an Existing Hot Path
+
+**Phase 1: RED** — test and benchmark already exist from initial implementation.
+
+**Phase 3: REFACTOR**
+
+```
+Before: resolveImports 12ms (requirement: <5ms)
+Change: switch from recursive descent to iterative with stack
+After:  resolveImports 3.8ms
+Test: still passing
+```
+
+**Phase 4: VALIDATE**
+
+```
+harness check-perf — complexity reduced from 12 to 8 (improvement)
+harness perf baselines update — new baseline saved
+```
+
+## Gates
+
+- **No code before test AND benchmark.** Both must exist before implementation begins.
+- **No optimization without measurement.** Run the benchmark before and after refactoring. Gut feelings are not measurements.
+- **No skipping VALIDATE.** `harness check-perf` and `harness validate` must pass after every cycle.
+- **No committing without updated baselines.** New benchmarks must have baselines persisted.
+
+## Escalation
+
+- **When the performance requirement cannot be met:** Report the best achieved result and propose either relaxing the requirement or redesigning the approach. Include benchmark data.
+- **When benchmarks are flaky:** Increase iteration count, add warmup, or isolate the benchmark from I/O. Report the variance so the team can decide on an acceptable noise margin.
+- **When the test and benchmark have conflicting needs:** Correctness always wins. If a correct implementation cannot meet the performance requirement, escalate to the team for a design decision.

package/dist/agents/skills/cursor/harness-perf-tdd/skill.yaml

@@ -0,0 +1,48 @@
+name: harness-perf-tdd
+version: "1.0.0"
+description: Performance-aware TDD with benchmark assertions in the red-green-refactor cycle
+cognitive_mode: meticulous-implementer
+triggers:
+  - manual
+platforms:
+  - claude-code
+  - gemini-cli
+tools:
+  - Bash
+  - Read
+  - Write
+  - Edit
+  - Glob
+  - Grep
+cli:
+  command: harness skill run harness-perf-tdd
+  args:
+    - name: path
+      description: Project root path
+      required: false
+mcp:
+  tool: run_skill
+  input:
+    skill: harness-perf-tdd
+    path: string
+type: rigid
+tier: 3
+phases:
+  - name: red
+    description: Write failing test and benchmark assertion
+    required: true
+  - name: green
+    description: Implement to pass test and benchmark
+    required: true
+  - name: refactor
+    description: Optimize while keeping both green
+    required: false
+  - name: validate
+    description: Run harness check-perf and harness validate
+    required: true
+state:
+  persistent: false
+  files: []
+depends_on:
+  - harness-tdd
+  - harness-perf