@harness-engineering/cli 1.13.0 → 1.14.0

package/dist/agents/skills/claude-code/harness-mutation-test/skill.yaml

@@ -0,0 +1,70 @@
+name: harness-mutation-test
+version: "1.0.0"
+description: Test quality validation through mutation testing with Stryker and mutation scoring
+cognitive_mode: adversarial-reviewer
+triggers:
+  - manual
+  - on_milestone
+platforms:
+  - claude-code
+  - gemini-cli
+tools:
+  - Bash
+  - Read
+  - Write
+  - Edit
+  - Glob
+  - Grep
+  - emit_interaction
+cli:
+  command: harness skill run harness-mutation-test
+  args:
+    - name: path
+      description: Project root path
+      required: false
+    - name: target
+      description: "Target file or directory for mutation testing. Runs project-wide when omitted."
+      required: false
+    - name: threshold
+      description: "Minimum mutation score (0-100). Defaults to 80."
+      required: false
+mcp:
+  tool: run_skill
+  input:
+    skill: harness-mutation-test
+    path: string
+type: rigid
+tier: 3
+internal: false
+keywords:
+  - mutation testing
+  - Stryker
+  - mutant
+  - mutation score
+  - test quality
+  - test effectiveness
+  - survived mutant
+  - killed mutant
+  - test strength
+stack_signals:
+  - "stryker.conf.*"
+  - ".stryker-tmp/"
+  - "mutation-report/"
+  - "stryker.config.*"
+phases:
+  - name: configure
+    description: Set up mutation testing framework, identify target modules, and define mutant operators
+    required: true
+  - name: generate
+    description: Generate mutants by applying code transformations to production source
+    required: true
+  - name: execute
+    description: Run test suite against each mutant and classify as killed or survived
+    required: true
+  - name: analyze
+    description: Calculate mutation score, identify weak tests, and recommend improvements
+    required: true
+state:
+  persistent: false
+  files: []
+depends_on: []

package/dist/agents/skills/claude-code/harness-observability/SKILL.md

@@ -0,0 +1,283 @@
+# Harness Observability
+
+> Structured logging, metrics, distributed tracing, and alerting strategy. The three pillars of observability, assessed and designed for production readiness.
+
+## When to Use
+
+- When designing or reviewing observability instrumentation for a service
+- When auditing logging, metrics, or tracing coverage gaps
+- When defining SLIs, SLOs, and alerting strategies for a new feature
+- NOT for application performance benchmarking (use harness-perf)
+- NOT for security-focused log analysis (use harness-security-review)
+- NOT for incident response procedures (use harness-incident-response)
+
+## Process
+
+### Phase 1: DETECT -- Identify Existing Instrumentation
+
+1. **Scan for observability libraries.** Check package manifests for instrumentation dependencies:
+   - **Logging:** winston, pino, bunyan, log4js, structlog, serilog, zap, logrus
+   - **Metrics:** prom-client, micrometer, statsd-client, @opentelemetry/sdk-metrics
+   - **Tracing:** @opentelemetry/sdk-trace-node, dd-trace, jaeger-client, zipkin
+   - **Platforms:** @datadog/browser-rum, newrelic, @sentry/node, elastic-apm-node
+
+2. **Locate instrumentation code.** Search for logger, metrics, and tracing initialization:
+   - Logger configuration files (log levels, formatters, transports)
+   - Metrics registry and custom metric definitions
+   - Tracer initialization and span creation patterns
+   - Middleware for automatic HTTP instrumentation
+
+3. **Detect collector and exporter configuration.** Look for:
+   - OpenTelemetry Collector config (`otel-collector-config.yaml`)
+   - Prometheus scrape configuration (`prometheus.yml`)
+   - Grafana dashboards (`grafana/dashboards/`)
+   - Datadog agent configuration (`datadog.yaml`)
+   - Fluentd or Fluent Bit configuration for log forwarding
+
+4. **Identify alerting configuration.** Search for:
+   - Prometheus alerting rules (`alert.rules.yml`)
+   - Grafana alert definitions
+   - PagerDuty, Opsgenie, or Slack integration configs
+   - SLO definitions in monitoring-as-code format
+
+5. **Present detection summary:**
+
+   ```
+   Observability Detection:
+     Logging: pino (structured JSON) -- 12 logger instances found
+     Metrics: prom-client -- 8 custom metrics defined
+     Tracing: @opentelemetry/sdk-trace-node -- initialized in src/tracing.ts
+     Collector: OpenTelemetry Collector -> Grafana Cloud
+     Alerting: 3 Prometheus alert rules, Slack integration
+   ```
+
+---
+
+### Phase 2: AUDIT -- Evaluate Coverage and Quality
+
+1. **Audit logging quality.** Evaluate each logger usage for:
+   - **Structured format:** JSON output with consistent field names, not string concatenation
+   - **Log levels:** Appropriate use of error, warn, info, debug (not everything at info)
+   - **Context fields:** Request ID, user ID, operation name included in log entries
+   - **Correlation IDs:** Trace ID propagated through log entries for cross-service correlation
+   - **Sensitive data:** No PII, credentials, or tokens logged
+   - **Error logging:** Stack traces included for errors, not just error messages
+
+2. **Audit metrics coverage.** Check for standard metrics:
+   - **RED metrics:** Request rate, error rate, duration for every HTTP endpoint
+   - **USE metrics:** Utilization, saturation, errors for system resources
+   - **Custom business metrics:** Domain-specific counters and gauges
+   - **Histogram buckets:** Appropriate bucket boundaries for latency histograms
+   - **Label cardinality:** No high-cardinality labels (user ID, request ID as metric labels)
+
+3. **Audit tracing implementation.** Verify:
+   - **Span creation:** Entry points create root spans, downstream calls create child spans
+   - **Context propagation:** Trace context flows across HTTP boundaries (W3C or B3 headers)
+   - **Span attributes:** Meaningful attributes set on spans (not empty spans)
+   - **Error recording:** Exceptions recorded on spans with status codes
+   - **Sampling strategy:** Configured and appropriate (not sampling 100% in production)
+
+4. **Audit alerting effectiveness.** Check each alert for:
+   - **Actionability:** Does the alert tell the on-call engineer what to do?
+   - **Signal quality:** Based on symptoms (error rate) not causes (CPU usage)
+   - **Thresholds:** Based on SLO burn rate, not arbitrary values
+   - **Runbook link:** Each alert has a link to a runbook or troubleshooting guide
+   - **Routing:** Alerts route to the correct team and escalation path
+
+5. **Score each pillar and identify gaps:**
+
+   ```
+   Observability Audit:
+     Logging:  7/10 -- structured, but missing correlation IDs in 4 services
+     Metrics:  5/10 -- RED metrics partial, no business metrics
+     Tracing:  8/10 -- good coverage, sampling needs tuning
+     Alerting: 3/10 -- only 3 rules, no SLO-based alerts, no runbooks
+   ```
+
+---
+
+### Phase 3: DESIGN -- Recommend Instrumentation Strategy
+
+1. **Design logging strategy.** Recommend:
+   - Standard log format with required fields (timestamp, level, service, traceId, message)
+   - Logger factory that injects correlation context automatically
+   - Log level configuration per environment (debug in dev, info in production)
+   - Log aggregation pipeline (application -> collector -> storage -> query)
+   - Retention policy per environment
+
+2. **Design metrics strategy.** Recommend:
+   - Standard RED metrics for every HTTP endpoint using middleware
+   - Custom business metrics aligned with product KPIs
+   - Histogram bucket configuration based on expected latency distribution
+   - Metric naming convention following Prometheus best practices (`http_request_duration_seconds`)
+   - Dashboard templates for standard service views
+
+3. **Design tracing strategy.** Recommend:
+   - Automatic instrumentation for HTTP, database, and message queue operations
+   - Manual span creation for business-critical code paths
+   - Sampling strategy (head-based for development, tail-based for production)
+   - Baggage propagation for cross-cutting concerns (tenant ID, feature flags)
+   - Trace-to-log correlation via trace ID injection
+
+4. **Define SLIs and SLOs.** For each service endpoint:
+   - **Availability SLI:** Proportion of successful requests (non-5xx)
+   - **Latency SLI:** Proportion of requests faster than threshold (p99 < 500ms)
+   - **SLO target:** 99.9% availability, 99% latency within budget
+   - **Error budget:** Calculation and burn rate alerting
+   - **SLO dashboard:** Visual budget remaining over rolling window
+
+5. **Design alerting strategy.** Recommend:
+   - Multi-window, multi-burn-rate alerts based on SLOs
+   - Page-worthy alerts (high burn rate) vs. ticket-worthy alerts (slow burn)
+   - Alert grouping to reduce noise (one alert per service, not per instance)
+   - Escalation paths with timeouts
+   - Regular alert review cadence (monthly) to retire noisy alerts
+
+---
+
+### Phase 4: VALIDATE -- Verify Instrumentation Correctness
+
+1. **Validate log output.** Check that logs from a test run:
+   - Parse as valid JSON (if structured logging is configured)
+   - Contain required fields (timestamp, level, service name)
+   - Include trace IDs when tracing is active
+   - Do not contain sensitive data (grep for common patterns: password, token, secret)
+   - Use appropriate log levels (no error-level logs for normal operations)
+
+2. **Validate metric exposition.** Verify:
+   - Metrics endpoint (`/metrics`) is accessible and returns Prometheus format
+   - All declared custom metrics appear in the output
+   - Histogram buckets are populated with reasonable values
+   - No high-cardinality label combinations (check label count)
+   - Counter values increment correctly (not resetting unexpectedly)
+
+3. **Validate trace propagation.** Verify end-to-end:
+   - A request to the entry service generates a trace visible in the tracing backend
+   - Child spans appear for downstream service calls
+   - Span attributes are populated with expected values
+   - Error spans have appropriate status and exception recording
+   - Sampling is working as configured (not dropping all traces or keeping all)
+
+4. **Validate alerting rules.** Check:
+   - Alert expressions are syntactically valid (PromQL, LogQL)
+   - Alert thresholds fire on historical data that represents real incidents
+   - Alert routing sends to the correct channel or pager
+   - Runbook links resolve to actual documentation
+
+5. **Generate observability report:**
+
+   ```
+   Observability Validation: [PASS/WARN/FAIL]
+
+   Logging: PASS (structured, correlated, no PII detected)
+   Metrics: WARN (RED metrics present, missing business metrics)
+   Tracing: PASS (propagation verified, sampling at 10%)
+   Alerting: FAIL (no SLO-based alerts, 2 of 3 rules missing runbooks)
+
+   Priority actions:
+     1. Define SLOs for /api/orders and /api/payments endpoints
+     2. Add multi-burn-rate alerts based on SLO error budget
+     3. Write runbooks for existing alerts
+     4. Add order_total and payment_success_rate business metrics
+   ```
+
+---
+
+## Harness Integration
+
+- **`harness skill run harness-observability`** -- Primary invocation for observability audit.
+- **`harness validate`** -- Run after instrumentation changes to verify project health.
+- **`harness check-deps`** -- Verify observability library dependencies are installed.
+- **`emit_interaction`** -- Present audit results and SLO design recommendations.
+
+## Success Criteria
+
+- All three observability pillars (logging, metrics, tracing) are assessed
+- Coverage gaps are identified with specific remediation recommendations
+- SLI/SLO definitions are provided for critical service endpoints
+- Alerting strategy is evaluated for actionability and signal quality
+- No sensitive data detected in log output
+- Correlation between logs and traces is verified
+
+## Examples
+
+### Example: Express.js API with OpenTelemetry
+
+```
+Phase 1: DETECT
+  Logging: pino with pino-http middleware
+  Metrics: @opentelemetry/sdk-metrics -> Prometheus
+  Tracing: @opentelemetry/sdk-trace-node -> Jaeger
+  Collector: OTel Collector (otel-collector-config.yaml)
+  Alerting: 5 Prometheus rules in monitoring/alerts.yml
+
+Phase 2: AUDIT
+  Logging: 8/10 -- structured JSON, trace IDs present, missing request body size
+  Metrics: 6/10 -- http_request_duration_seconds present, missing queue depth
+             and business metrics (orders_created_total)
+  Tracing: 9/10 -- auto-instrumented HTTP + pg + Redis, manual spans on
+             checkout flow
+  Alerting: 4/10 -- static thresholds, no SLO burn rate, 2 missing runbooks
+
+Phase 3: DESIGN
+  SLOs recommended:
+    - POST /api/orders: 99.9% availability, p99 < 800ms
+    - GET /api/products: 99.95% availability, p99 < 200ms
+  Alerting: Replace static "error rate > 5%" with multi-window burn rate
+  Metrics: Add orders_created_total, cart_abandonment_rate gauges
+  Logging: Add request/response body size for capacity planning
+
+Phase 4: VALIDATE
+  Log output: PASS (valid JSON, no PII)
+  Metrics endpoint: PASS (all custom metrics present)
+  Trace propagation: PASS (end-to-end verified)
+  Alert rules: WARN (valid PromQL, but thresholds not SLO-based)
+  Result: WARN -- alerting strategy needs SLO alignment
+```
+
+### Example: Go Microservices with Datadog
+
+```
+Phase 1: DETECT
+  Logging: zap (structured) across 4 services
+  Metrics: Datadog dogstatsd client
+  Tracing: dd-trace-go with automatic HTTP/gRPC instrumentation
+  Collector: Datadog Agent (datadog.yaml in k8s/)
+  Alerting: 12 monitors in Datadog (Terraform-managed)
+
+Phase 2: AUDIT
+  Logging: 9/10 -- consistent structured format, correlation IDs, no PII
+  Metrics: 7/10 -- RED metrics present, custom counters for business events,
+             but missing histogram for gRPC call duration
+  Tracing: 8/10 -- HTTP and gRPC instrumented, database spans present,
+             Redis spans missing
+  Alerting: 6/10 -- good coverage but static thresholds, no error budgets
+
+Phase 3: DESIGN
+  1. Add dd-trace-go Redis integration for complete trace picture
+  2. Add grpc_server_handling_seconds histogram
+  3. Define SLOs in Datadog for top 5 endpoints
+  4. Convert 4 highest-priority monitors to SLO burn rate alerts
+  5. Add Datadog SLO dashboard for team visibility
+
+Phase 4: VALIDATE
+  Log output: PASS
+  Metrics: WARN (missing gRPC histogram)
+  Traces: WARN (Redis spans missing)
+  Alerts: WARN (no SLO-based alerts)
+  Result: WARN -- 3 instrumentation gaps, alerting needs SLO alignment
+```
+
+## Gates
+
+- **No sensitive data in logs.** If PII, credentials, or tokens are detected in log output, it is a blocking finding. The logging configuration must sanitize or redact sensitive fields before any other improvements are made.
+- **No high-cardinality metric labels.** Metric labels with unbounded values (user IDs, request IDs, timestamps) cause storage explosion and query timeouts. This is a blocking finding.
+- **No alerting without runbooks.** Production alerts that lack a runbook link are incomplete. Every page-worthy alert must have actionable documentation.
+- **No tracing without context propagation.** Traces that do not propagate context across service boundaries provide incomplete pictures and mislead investigations. Broken propagation is a blocking finding.
+
+## Escalation
+
+- **When observability libraries conflict:** Some combinations (e.g., dd-trace and OTel auto-instrumentation) cause duplicate spans or metric conflicts. Recommend choosing one provider and removing the other. If migration is needed, present a phased approach.
+- **When sampling drops critical traces:** If tail-based sampling is misconfigured, critical error traces may be dropped. Recommend always-sample rules for error responses and critical code paths, with probabilistic sampling for normal traffic.
+- **When metric cardinality is already out of control:** Do not attempt to fix retroactively in the metrics backend. Recommend adding new metrics with correct labels and deprecating the high-cardinality ones. Set a timeline for removal.
+- **When no observability infrastructure exists:** This is a design-from-scratch scenario. Start with structured logging (lowest barrier), then add metrics (RED for HTTP endpoints), then tracing. Do not attempt all three at once.

package/dist/agents/skills/claude-code/harness-observability/skill.yaml

@@ -0,0 +1,78 @@
+name: harness-observability
+version: "1.0.0"
+description: Structured logging, metrics, distributed tracing, and alerting strategies
+cognitive_mode: advisory-guide
+tier: 3
+internal: false
+keywords:
+  - logging
+  - metrics
+  - tracing
+  - observability
+  - OpenTelemetry
+  - Prometheus
+  - Grafana
+  - Datadog
+  - alerting
+  - SLI
+  - SLO
+  - structured logging
+  - correlation ID
+  - spans
+stack_signals:
+  - "src/**/logger*"
+  - "src/**/metrics*"
+  - "src/**/tracing*"
+  - "otel-collector*"
+  - "prometheus.yml"
+  - "grafana/"
+  - "datadog.yaml"
+triggers:
+  - manual
+  - on_new_feature
+platforms:
+  - claude-code
+  - gemini-cli
+tools:
+  - Bash
+  - Read
+  - Write
+  - Edit
+  - Glob
+  - Grep
+  - emit_interaction
+cli:
+  command: harness skill run harness-observability
+  args:
+    - name: path
+      description: Project root path
+      required: false
+    - name: pillar
+      description: Observability pillar to focus on (logging, metrics, tracing, all)
+      required: false
+    - name: provider
+      description: Observability provider context (otel, datadog, prometheus)
+      required: false
+mcp:
+  tool: run_skill
+  input:
+    skill: harness-observability
+    path: string
+type: rigid
+phases:
+  - name: detect
+    description: Identify existing observability instrumentation and providers
+    required: true
+  - name: audit
+    description: Evaluate coverage gaps across logging, metrics, and tracing
+    required: true
+  - name: design
+    description: Recommend instrumentation strategy and SLI/SLO definitions
+    required: true
+  - name: validate
+    description: Verify instrumentation correctness and alerting configuration
+    required: true
+state:
+  persistent: false
+  files: []
+depends_on: []

package/dist/agents/skills/claude-code/harness-onboarding/skill.yaml

@@ -24,6 +24,7 @@ mcp:
     skill: harness-onboarding
     path: string
 type: flexible
+tier: 1
 state:
   persistent: false
   files: []

package/dist/agents/skills/claude-code/harness-parallel-agents/skill.yaml

@@ -27,6 +27,7 @@ mcp:
     skill: harness-parallel-agents
     path: string
 type: flexible
+internal: true
 state:
   persistent: false
   files: []

package/dist/agents/skills/claude-code/harness-perf/skill.yaml

@@ -30,6 +30,7 @@ mcp:
     skill: harness-perf
     path: string
 type: rigid
+tier: 2
 phases:
   - name: analyze
     description: Run structural complexity and coupling checks

package/dist/agents/skills/claude-code/harness-perf-tdd/skill.yaml

@@ -26,6 +26,7 @@ mcp:
     skill: harness-perf-tdd
     path: string
 type: rigid
+tier: 3
 phases:
   - name: red
     description: Write failing test and benchmark assertion

package/dist/agents/skills/claude-code/harness-planning/SKILL.md

@@ -312,6 +312,45 @@ One sentence.
 
 ````
 
+## Session State
+
+This skill reads and writes to the following session sections via `manage_state`:
+
+| Section | Read | Write | Purpose |
+|---------|------|-------|---------|
+| terminology | yes | no | Reads domain terms to use consistent language in plan |
+| decisions | yes | yes | Reads brainstorming decisions; records planning-phase decisions |
+| constraints | yes | yes | Reads existing constraints; adds constraints discovered during decomposition |
+| risks | yes | yes | Reads existing risks; adds implementation risks identified during task design |
+| openQuestions | yes | yes | Reads unresolved questions; adds new questions, resolves answered ones |
+| evidence | yes | yes | Reads prior evidence from brainstorming; writes file:line citations for task specifications |
+
+**When to write:** During Phase 1 (SCOPE) write newly discovered constraints and risks. During Phase 2 (DECOMPOSE) write decisions about task structure and sequencing. Mark resolved questions during Phase 4 (VALIDATE).
+
+**When to read:** At the start of Phase 1 (SCOPE), read all sections via `gather_context` with `include: ["sessions"]` to inherit context from brainstorming. Use terminology for consistent naming in task descriptions.
+
+## Evidence Requirements
+
+When this skill makes claims about existing code structure, file locations, or implementation patterns in task specifications, it MUST cite evidence using one of:
+
+1. **File reference:** `file:line` format (e.g., `src/services/index.ts:15` -- "barrel export exists, will add new export here")
+2. **Code pattern reference:** `file:line` format with pattern description (e.g., `src/services/user-service.ts:1-30` -- "existing service follows constructor injection pattern, new service will match")
+3. **Test output:** Include the command and its observed output when referencing current test state
+4. **Session evidence:** Write to the `evidence` session section:
+   ```json
+   manage_state({
+     action: "append_entry",
+     session: "<current-session>",
+     section: "evidence",
+     authorSkill: "harness-planning",
+     content: "src/services/index.ts:15 -- barrel export pattern confirmed for new service integration"
+   })
+   ```
+
+**When to cite:** During Phase 1 (SCOPE) when referencing existing files for observable truths. During Phase 2 (DECOMPOSE) when specifying exact file paths and code patterns in task instructions. When the file map references existing files for modification.
+
+**Uncited claims:** Technical assertions about existing code without citations MUST be prefixed with `[UNVERIFIED]`. Example: `[UNVERIFIED] The service barrel exports all services`. Uncited claims are flagged during review (Wave 2.2).
+
 ## Harness Integration
 
 - **`harness validate`** — Run during Phase 4 (before writing the plan) and included as a step in every task.

package/dist/agents/skills/claude-code/harness-planning/skill.yaml

@@ -28,6 +28,7 @@ mcp:
     skill: harness-planning
     path: string
 type: rigid
+tier: 1
 phases:
   - name: scope
     description: Define goals and constraints

package/dist/agents/skills/claude-code/harness-pre-commit-review/skill.yaml

@@ -16,6 +16,7 @@ mcp:
     skill: harness-pre-commit-review
     path: string
 type: rigid
+tier: 3
 phases:
   - name: mechanical-checks
     description: Run deterministic checks (lint, typecheck, tests)