@harness-engineering/cli 1.13.0 → 1.14.0

package/dist/agents/skills/gemini-cli/harness-feature-flags/SKILL.md

@@ -0,0 +1,287 @@
+# Harness Feature Flags
+
+> Flag lifecycle management, A/B testing infrastructure, and gradual rollout design. Ship features safely with controlled exposure and clean retirement.
+
+## When to Use
+
+- When designing feature flag architecture for a new project or feature
+- When auditing existing flags for staleness, test coverage, or hygiene
+- When planning gradual rollout or A/B testing strategies
+- NOT for deployment pipeline configuration (use harness-deployment)
+- NOT for environment variable management (use harness-secrets)
+- NOT for application performance testing under flag variants (use harness-perf)
+
+## Process
+
+### Phase 1: DETECT -- Discover Flag Definitions and Usage
+
+1. **Identify flag provider.** Scan the project for feature flag infrastructure:
+   - **Third-party SDKs:** LaunchDarkly (`launchdarkly-node-server-sdk`), Unleash (`unleash-client`), Flagsmith (`flagsmith-nodejs`), Split (`@splitsoftware/splitio`)
+   - **Custom implementations:** Files matching `*feature-flag*`, `*toggle*`, `*flags*`
+   - **Configuration-based:** `flags.json`, `features.json`, `feature-flags.yaml`
+   - **Environment-based:** Feature flags controlled via environment variables (`FEATURE_*`, `FF_*`)
+
+2. **Catalog all flag definitions.** For each flag found, record:
+   - Flag name/key
+   - Flag type (boolean, multivariate, percentage rollout)
+   - Default value (what happens when the flag service is unreachable)
+   - Where it is defined (provider dashboard, config file, code)
+   - Creation date or first appearance in git history
+
+3. **Map flag usage in code.** For each flag, find all evaluation points:
+   - Source files that check the flag value
+   - Conditional branches gated by the flag
+   - Components that render differently based on flag state
+   - API endpoints with flag-dependent behavior
+   - Test files that exercise flag variants
+
+4. **Detect flag categories.** Classify each flag:
+   - **Release flag:** Gates a new feature for gradual rollout (temporary)
+   - **Experiment flag:** Controls A/B test variants (temporary)
+   - **Ops flag:** Circuit breaker or kill switch (permanent)
+   - **Permission flag:** Controls access to premium features (permanent)
+
+5. **Present detection summary:**
+
+   ```
+   Feature Flag Detection:
+     Provider: LaunchDarkly (SDK v7.x)
+     Flags defined: 23
+     Flags evaluated in code: 19 (4 defined but unused)
+     Categories: 12 release, 5 experiment, 4 ops, 2 permission
+     Stale candidates: 6 (release flags older than 90 days)
+   ```
+
+---
+
+### Phase 2: DESIGN -- Recommend Flag Architecture and Rollout Strategy
+
+1. **Evaluate flag naming convention.** Check for consistency:
+   - Recommended format: `{team}.{feature}.{variant}` (e.g., `payments.stripe-v2.enabled`)
+   - Flags should have descriptive names (not `flag_123` or `test_flag`)
+   - Naming should indicate category (prefix with `ops.` for operational flags)
+   - Temporary flags should include a target removal date in metadata
+
+2. **Design flag evaluation architecture.** Recommend patterns for:
+   - **Server-side evaluation:** Flags evaluated on the backend, consistent behavior
+   - **Client-side evaluation:** Flags evaluated in the browser/app, real-time updates
+   - **Edge evaluation:** Flags evaluated at CDN layer for zero-latency decisions
+   - **Default values:** Every flag must have a safe default (feature off) for provider outages
+
+3. **Design rollout strategy.** For release flags:
+   - **Percentage rollout:** Start at 1%, monitor metrics, increase to 5%, 25%, 50%, 100%
+   - **User segment targeting:** Internal users first, then beta users, then general availability
+   - **Geographic rollout:** Single region first, expand after validation
+   - **Time-based gates:** Enable during low-traffic periods for initial validation
+   - Define success criteria for each rollout stage (error rate, latency, conversion)
+
+4. **Design A/B testing infrastructure.** For experiment flags:
+   - Consistent user assignment (same user always sees same variant)
+   - Statistical significance calculation before declaring a winner
+   - Event tracking integration for measuring experiment outcomes
+   - Guardrail metrics that auto-disable experiments if health degrades
+   - Experiment duration limits (no indefinite experiments)
+
+5. **Design fallback behavior.** Ensure resilience:
+   - Provider SDK timeout configuration (fail fast, not block)
+   - Local cache for flag values (stale-while-revalidate)
+   - Default values that are safe (feature disabled, not enabled)
+   - Circuit breaker on flag evaluation errors
+   - Monitoring on flag evaluation latency and error rate
+
+---
+
+### Phase 3: VALIDATE -- Verify Flag Hygiene and Test Coverage
+
+1. **Check test coverage per flag.** For each flag, verify:
+   - Tests exist for both the flag-on and flag-off code paths
+   - Integration tests cover the default value behavior (provider down scenario)
+   - No tests depend on a specific flag value being set in production
+   - Test utilities exist for overriding flag values in test context
+
+2. **Validate flag consistency.** Check for common issues:
+   - Flags checked in code but not defined in the provider (will use defaults)
+   - Flags defined in the provider but never checked in code (dead flags)
+   - Same flag checked with different keys in different files (typos)
+   - Nested flag checks that create combinatorial complexity
+
+3. **Check for flag coupling.** Identify problematic patterns:
+   - Flags that depend on other flags (if A and B, then C)
+   - Flags that must be enabled in a specific order
+   - Flags that share state or side effects
+   - Recommend decoupling or consolidating dependent flags
+
+4. **Validate rollout configuration.** For active rollouts:
+   - Percentage values are within expected ranges
+   - Targeting rules are correctly configured
+   - Kill switch is functional (can disable the feature instantly)
+   - Rollback plan is documented
+
+5. **Generate validation report:**
+
+   ```
+   Flag Validation: [PASS/WARN/FAIL]
+
+   Test coverage: WARN (3 flags missing flag-off tests)
+   Consistency: PASS (all code references match definitions)
+   Coupling: WARN (2 flags have interdependencies)
+   Rollout config: PASS (active rollouts correctly configured)
+
+   Issues:
+     1. payments.stripe-v2 -- no test for flag-off fallback path
+     2. search.new-algorithm + search.reranking -- coupled flags
+     3. checkout.express -- missing kill switch test
+   ```
+
+---
+
+### Phase 4: LIFECYCLE -- Audit Stale Flags and Plan Cleanup
+
+1. **Identify stale flags.** Flag candidates for removal:
+   - Release flags that have been at 100% for more than 30 days
+   - Experiment flags past their declared end date
+   - Flags with no evaluation in the last 90 days (from provider analytics)
+   - Flags whose feature has been fully adopted (no rollback expected)
+
+2. **Generate cleanup plan.** For each stale flag:
+   - List all code locations that reference the flag
+   - Identify which branch to keep (the flag-on path for graduated features)
+   - Estimate lines of code to remove (dead branch cleanup)
+   - Determine if any tests need updating after flag removal
+   - Assign a cleanup owner and deadline
+
+3. **Assess technical debt.** Calculate flag burden:
+   - Total active flags per service (recommend under 20)
+   - Ratio of temporary to permanent flags
+   - Average flag age for temporary flags
+   - Code complexity added by flag branching (conditional paths)
+   - Estimate of test matrix growth due to flags
+
+4. **Recommend lifecycle policies.** Design governance:
+   - Maximum lifetime for release flags (e.g., 90 days)
+   - Maximum lifetime for experiment flags (e.g., 30 days)
+   - Required metadata: owner, creation date, expiry date, cleanup ticket
+   - Automated alerting when flags exceed their lifetime
+   - Monthly flag review cadence with the team
+
+5. **Generate lifecycle report:**
+
+   ```
+   Flag Lifecycle Report:
+
+   Active flags: 23
+   Stale flags: 6 (candidates for removal)
+   Technical debt: ~340 lines of dead code across 6 stale flags
+
+   Cleanup plan:
+     1. payments.old-checkout -- at 100% for 45 days, 4 files, ~80 LOC to remove
+     2. search.v1-algorithm -- experiment ended 60 days ago, 2 files, ~40 LOC
+     3. onboarding.legacy-flow -- at 100% for 90 days, 6 files, ~120 LOC
+     4. notifications.email-v1 -- unused for 120 days, 3 files, ~50 LOC
+     5. dashboard.old-charts -- at 100% for 35 days, 2 files, ~30 LOC
+     6. auth.password-reset-v1 -- at 100% for 60 days, 1 file, ~20 LOC
+
+   Recommendations:
+     - Adopt 90-day maximum lifetime policy for release flags
+     - Add flag expiry dates to LaunchDarkly metadata
+     - Schedule monthly flag cleanup sprint
+     - Target: reduce active flag count from 23 to 17
+   ```
+
+---
+
+## Harness Integration
+
+- **`harness skill run harness-feature-flags`** -- Primary invocation for flag analysis and lifecycle management.
+- **`harness validate`** -- Run after flag cleanup to verify project health.
+- **`harness check-deps`** -- Verify flag provider SDK dependencies are installed.
+- **`emit_interaction`** -- Present lifecycle report and gather decisions on cleanup priorities.
+
+## Success Criteria
+
+- All feature flags in the project are discovered and cataloged
+- Flags are categorized by type (release, experiment, ops, permission)
+- Stale flags are identified with specific cleanup plans
+- Test coverage for both flag-on and flag-off paths is verified
+- Rollout configuration is validated for active flags
+- Lifecycle policies are recommended with enforcement mechanisms
+
+## Examples
+
+### Example: React SPA with LaunchDarkly
+
+```
+Phase 1: DETECT
+  Provider: LaunchDarkly (React SDK v3.x)
+  Flags defined: 15 (in LaunchDarkly dashboard)
+  Flags in code: 13 (2 unused in dashboard)
+  Categories: 8 release, 3 experiment, 2 ops
+
+Phase 2: DESIGN
+  Naming: WARN -- inconsistent (mix of camelCase and kebab-case)
+  Recommended convention: team.feature.variant (kebab-case)
+  Evaluation: Client-side (React SDK), 200ms init timeout
+  Default values: WARN -- 2 flags default to true (should default to false)
+  Rollout: checkout.express-pay at 25% (targeting premium users)
+
+Phase 3: VALIDATE
+  Test coverage: WARN -- 4 flags missing useFlags mock in component tests
+  Consistency: PASS
+  Coupling: PASS (no interdependent flags)
+  Kill switch: PASS (all release flags can be instantly disabled)
+
+Phase 4: LIFECYCLE
+  Stale: 3 flags at 100% for 30+ days
+  Cleanup estimate: 180 LOC across 8 files
+  Recommendation: Remove search.v2-results (at 100% for 65 days, 3 components)
+  Result: WARN -- 3 stale flags, 4 missing tests, 2 unsafe defaults
+```
+
+### Example: Spring Boot API with Custom Flag System
+
+```
+Phase 1: DETECT
+  Provider: Custom implementation (FeatureFlagService.java)
+  Flag store: PostgreSQL table (feature_flags)
+  Flags defined: 28
+  Flags in code: 24 (4 in database but unreferenced)
+  Categories: 15 release, 6 experiment, 5 ops, 2 permission
+
+Phase 2: DESIGN
+  Architecture: Server-side evaluation with 60s cache refresh
+  Concern: No SDK resilience -- database outage disables all flags
+  Recommend: Add in-memory cache with stale-while-revalidate pattern
+  Recommend: Add health check endpoint for flag service status
+  Rollout: Using percentage field in database -- no segment targeting
+  Recommend: Add user segment support for targeted rollouts
+
+Phase 3: VALIDATE
+  Test coverage: WARN -- @FeatureFlag annotation used but no test helper
+    to toggle flags in test context (tests rely on database state)
+  Consistency: WARN -- 4 database entries have no code references
+  Coupling: FAIL -- 3 flags form a dependency chain
+    (billing.new-pricing requires billing.tax-calc-v2 requires billing.currency-v3)
+  Recommend: Consolidate into single billing.pricing-v3 flag
+
+Phase 4: LIFECYCLE
+  Stale: 8 flags enabled for all users for 60+ days
+  Technical debt: ~520 LOC of dead code
+  Flag count: 28 (above recommended 20 per service)
+  Recommendation: Immediate cleanup sprint targeting 8 stale flags
+  Recommendation: Enforce 60-day lifetime policy going forward
+  Result: FAIL -- flag coupling detected, high stale flag count
+```
+
+## Gates
+
+- **No flags without default values.** Every flag evaluation must specify a default value for provider outage scenarios. Missing defaults are blocking findings.
+- **No stale release flags beyond policy limit.** Release flags that exceed their maximum lifetime (default 90 days at 100%) are blocking warnings. They must be cleaned up or have an explicit extension with documented justification.
+- **No coupled flag dependencies.** Flags that require other flags to be in a specific state create combinatorial complexity and are fragile. Flag coupling is a blocking finding that requires consolidation.
+- **No flags without test coverage for both paths.** Feature flags that lack tests for the off-path leave the fallback behavior unverified. Both paths must be tested.
+
+## Escalation
+
+- **When flag count exceeds 30 per service:** The flag system is becoming a maintenance burden. Recommend a flag audit sprint with a target of reducing to under 20. Prioritize removing the oldest release flags first.
+- **When a flag provider outage affects production:** If the application does not handle provider unavailability gracefully, this is an architectural issue. Recommend implementing local caching, safe defaults, and a circuit breaker pattern before adding more flags.
+- **When experiment results are inconclusive:** Do not extend the experiment indefinitely. Recommend increasing sample size (broader targeting), simplifying the variants, or declaring the experiment inconclusive and removing it.
+- **When flag cleanup requires database migration:** Removing flags that are stored in a database may require migration scripts. Coordinate cleanup with a release cycle and ensure rollback is possible if the migration fails.

package/dist/agents/skills/gemini-cli/harness-feature-flags/skill.yaml

@@ -0,0 +1,74 @@
+name: harness-feature-flags
+version: "1.0.0"
+description: Flag lifecycle management, A/B testing infrastructure, and gradual rollouts
+cognitive_mode: advisory-guide
+tier: 3
+internal: false
+keywords:
+  - feature flags
+  - feature toggles
+  - LaunchDarkly
+  - Unleash
+  - Flagsmith
+  - A/B testing
+  - canary
+  - gradual rollout
+  - kill switch
+  - experiment
+stack_signals:
+  - "src/**/flags/**"
+  - "src/**/features/**"
+  - "src/**/*feature-flag*"
+  - "src/**/*toggle*"
+  - "flags.json"
+  - "features.json"
+triggers:
+  - manual
+  - on_new_feature
+platforms:
+  - claude-code
+  - gemini-cli
+tools:
+  - Bash
+  - Read
+  - Write
+  - Edit
+  - Glob
+  - Grep
+  - emit_interaction
+cli:
+  command: harness skill run harness-feature-flags
+  args:
+    - name: path
+      description: Project root path
+      required: false
+    - name: audit
+      description: Run stale flag audit to find flags past their expiry
+      type: boolean
+      required: false
+    - name: provider
+      description: Flag provider context (launchdarkly, unleash, flagsmith, custom)
+      required: false
+mcp:
+  tool: run_skill
+  input:
+    skill: harness-feature-flags
+    path: string
+type: rigid
+phases:
+  - name: detect
+    description: Discover flag definitions, providers, and usage patterns
+    required: true
+  - name: design
+    description: Recommend flag architecture, naming, and rollout strategies
+    required: true
+  - name: validate
+    description: Verify flag hygiene, test coverage, and fallback behavior
+    required: true
+  - name: lifecycle
+    description: Audit stale flags and recommend cleanup or graduation
+    required: true
+state:
+  persistent: false
+  files: []
+depends_on: []

package/dist/agents/skills/gemini-cli/harness-git-workflow/skill.yaml

@@ -25,6 +25,7 @@ mcp:
     skill: harness-git-workflow
     path: string
 type: flexible
+tier: 3
 state:
   persistent: false
   files: []

package/dist/agents/skills/gemini-cli/harness-hotspot-detector/skill.yaml

@@ -25,6 +25,7 @@ mcp:
     skill: harness-hotspot-detector
     path: string
 type: rigid
+tier: 2
 phases:
   - name: co-change
     description: Analyze co-change patterns from git history

package/dist/agents/skills/gemini-cli/harness-i18n/skill.yaml

@@ -35,6 +35,7 @@ mcp:
     skill: harness-i18n
     path: string
 type: rigid
+tier: 3
 phases:
   - name: detect
     description: Identify project platform(s), i18n framework in use, existing translation files, and locale config

package/dist/agents/skills/gemini-cli/harness-i18n-process/skill.yaml

@@ -27,6 +27,7 @@ mcp:
     skill: harness-i18n-process
     path: string
 type: flexible
+tier: 3
 phases:
   - name: check-config
     description: Read harness.config.json to determine mode — gate mode when i18n.enabled is true, prompt mode when unconfigured or disabled

package/dist/agents/skills/gemini-cli/harness-i18n-workflow/skill.yaml

@@ -33,6 +33,7 @@ mcp:
     skill: harness-i18n-workflow
     path: string
 type: flexible
+tier: 3
 phases:
   - name: configure
     description: Set up i18n config in harness.config.json with opinionated defaults or guided customization

package/dist/agents/skills/gemini-cli/harness-impact-analysis/skill.yaml

@@ -28,6 +28,7 @@ mcp:
     skill: harness-impact-analysis
     path: string
 type: rigid
+tier: 2
 phases:
   - name: identify
     description: Identify changed files from diff or input

package/dist/agents/skills/gemini-cli/harness-incident-response/SKILL.md

@@ -0,0 +1,223 @@
+# Harness Incident Response
+
+> Runbook generation, postmortem analysis, and SLO/SLA tracking. Diagnoses incidents by tracing symptoms through services, produces structured postmortems, and maintains error budget accounting.
+
+## When to Use
+
+- After a production incident to generate a structured postmortem with timeline and action items
+- To create or audit runbooks for critical services and failure scenarios
+- To define, track, or adjust SLOs/SLAs and monitor error budget consumption
+- NOT for real-time incident coordination (use PagerDuty, OpsGenie, or incident.io for live response)
+- NOT for infrastructure provisioning or remediation (use harness-infrastructure-as-code)
+- NOT for performance benchmarking (use harness-load-testing for capacity planning)
+
+## Process
+
+### Phase 1: ASSESS -- Determine Scope and Severity
+
+1. **Identify the incident signal.** Scan available evidence to determine what triggered the investigation:
+   - Check for existing incident reports in `docs/incidents/` or `docs/postmortems/`
+   - Look for recent error spikes in log files, monitoring configs, or alerting rules
+   - Review recent deployments via `git log --oneline --since="48 hours ago"` for correlated changes
+
+2. **Map affected services.** Trace the blast radius from the incident signal:
+   - Identify the originating service from error messages or alert metadata
+   - Walk dependency chains using import graphs or service manifests (`docker-compose.yml`, `kubernetes/`, service mesh configs)
+   - List all downstream services that depend on the affected component
+
+3. **Classify severity.** Apply the project's severity matrix if one exists in `docs/runbooks/severity-matrix.md`. Otherwise, use standard classification:
+   - **SEV1:** Complete service outage, data loss, or security breach affecting all users
+   - **SEV2:** Major feature degradation affecting a significant subset of users
+   - **SEV3:** Minor feature degradation with workaround available
+   - **SEV4:** Cosmetic issue or internal tooling degradation
+
+4. **Establish timeline boundaries.** Determine:
+   - When the incident started (first error, first alert, or first user report)
+   - When it was detected (MTTD -- Mean Time to Detect)
+   - When mitigation began
+   - When the incident was resolved (MTTR -- Mean Time to Recover)
+
+5. **Check for existing runbooks.** Search `docs/runbooks/` and `runbooks/` for procedures matching the affected service or failure mode. If a runbook exists, evaluate whether it was followed and whether it was effective.
+
+---
+
+### Phase 2: INVESTIGATE -- Trace Root Cause
+
+1. **Correlate with recent changes.** Run `git log --oneline --since="7 days ago"` and cross-reference commits with the incident timeline. Flag commits that touched affected services or their dependencies.
+
+2. **Analyze error patterns.** Search the codebase for error handling related to the failure:
+   - Grep for error messages, exception types, or error codes mentioned in the incident
+   - Check retry logic, timeout configurations, and circuit breaker states
+   - Identify whether the failure was transient (timeout, network) or persistent (logic error, data corruption)
+
+3. **Trace data flow.** Map the request path from entry point to failure point:
+   - Identify API endpoints, message queues, or cron jobs involved
+   - Check database queries and external API calls along the path
+   - Look for missing validation, unhandled edge cases, or race conditions
+
+4. **Identify contributing factors.** Distinguish between root cause and contributing factors:
+   - Root cause: the single change or condition that directly caused the failure
+   - Contributing factors: conditions that allowed the failure to reach production (missing tests, inadequate monitoring, deployment without canary)
+
+5. **Validate the hypothesis.** Confirm the root cause by checking:
+   - Does reverting the identified change (or simulating the revert) resolve the issue?
+   - Does the failure reproduce under the identified conditions?
+   - Are there other incidents with the same root cause pattern?
+
+---
+
+### Phase 3: DOCUMENT -- Generate Artifacts
+
+1. **Generate the postmortem report.** Create a structured document in `docs/postmortems/YYYY-MM-DD-<slug>.md` with these sections:
+   - **Summary:** One-paragraph description of what happened, impact, and duration
+   - **Timeline:** Chronological list of events from first signal to resolution
+   - **Root Cause:** Clear statement of what went wrong and why
+   - **Contributing Factors:** Conditions that enabled the failure
+   - **Impact:** User-facing impact, data impact, SLO impact, revenue impact if applicable
+   - **Detection:** How the incident was detected and time to detection
+   - **Mitigation:** Steps taken to resolve the incident
+   - **Action Items:** Numbered list with owner, priority, and due date
+
+2. **Create or update runbooks.** For each failure mode identified:
+   - If no runbook exists, create one in `docs/runbooks/<service>-<failure-mode>.md`
+   - Structure: Symptoms, Diagnosis Steps, Mitigation Steps, Escalation Path, Recovery Verification
+   - Include concrete commands (kubectl, database queries, API calls) not just prose descriptions
+   - Reference monitoring dashboards and alert names
+
+3. **Update the incident log.** If `docs/incidents/index.md` exists, append the new incident with date, severity, MTTR, and link to the postmortem.
+
+4. **Tag related code.** Add or update `// INCIDENT-YYYY-MM-DD: <description>` comments at the code locations involved in the root cause. This creates a searchable history of incident-prone code.
+
+---
+
+### Phase 4: IMPROVE -- SLO Adjustments and Prevention
+
+1. **Calculate SLO impact.** If `slo.yaml` or equivalent SLO definitions exist:
+   - Determine how much error budget the incident consumed
+   - Calculate remaining error budget for the current window
+   - If error budget is exhausted, flag that feature development should pause for reliability work
+
+2. **Evaluate alerting effectiveness.** For each alert that fired (or should have fired):
+   - Was the alert timely? Compare alert time to incident start time
+   - Was the alert actionable? Did it point to the right service and include enough context?
+   - Were there false negatives? Identify monitoring gaps that should have caught the issue earlier
+
+3. **Propose SLO adjustments.** Based on the incident analysis:
+   - If the SLO was violated but the impact was acceptable, the SLO may be too tight
+   - If the SLO was not violated but users were impacted, the SLO may be too loose
+   - Recommend specific SLI (Service Level Indicator) thresholds with justification
+
+4. **Generate preventive action items.** Categorize actions by type:
+   - **Code fixes:** Specific bugs or missing validations to address
+   - **Testing gaps:** Missing integration tests, chaos tests, or load tests to add
+   - **Monitoring improvements:** New alerts, dashboards, or SLIs to implement
+   - **Process improvements:** Deployment safeguards, runbook updates, or on-call training
+   - **Architecture changes:** Circuit breakers, bulkheads, or redundancy to add
+
+5. **Produce the improvement summary.** Output a prioritized action list with effort estimates and expected impact on MTTD and MTTR.
+
+---
+
+## Harness Integration
+
+- **`harness skill run harness-incident-response`** -- Primary CLI entry point. Runs all four phases.
+- **`harness validate`** -- Run after generating documents to ensure project structure is intact.
+- **`harness check-deps`** -- Verify service dependency declarations match the incident trace.
+- **`emit_interaction`** -- Used at severity classification (checkpoint:decision) to confirm severity with the operator before proceeding.
+- **`Glob`** -- Discover existing runbooks, postmortems, and SLO definitions.
+- **`Grep`** -- Search for error patterns, alert configurations, and incident-related code comments.
+- **`Write`** -- Generate postmortem reports and runbook documents.
+- **`Edit`** -- Update existing runbooks and incident indexes.
+
+## Success Criteria
+
+- Postmortem document is complete with all required sections (summary, timeline, root cause, action items)
+- Timeline includes MTTD and MTTR calculations with specific timestamps
+- Root cause is a specific, falsifiable statement (not "the system failed")
+- Action items have owners, priorities, and due dates
+- Runbooks contain concrete commands, not just descriptive prose
+- SLO impact is quantified against the error budget when SLO definitions exist
+
+## Examples
+
+### Example: Node.js API Timeout Incident with Datadog Alerts
+
+```
+Phase 1: ASSESS
+  Signal: Datadog alert "api-gateway p99 latency > 2000ms" fired at 14:32 UTC
+  Affected: api-gateway -> user-service -> PostgreSQL
+  Severity: SEV2 (major degradation, 40% of requests timing out)
+  MTTD: 4 minutes (alert fired 4 min after first error)
+  MTTR: 47 minutes (resolved at 15:19 UTC)
+
+Phase 2: INVESTIGATE
+  Correlated change: commit abc123 "add user preferences join" deployed at 14:25 UTC
+  Root cause: N+1 query in GET /api/users/:id/preferences — new LEFT JOIN
+    on unindexed column `preferences.user_id` caused full table scan
+  Contributing factors:
+    - No query performance test for the preferences endpoint
+    - Missing database index on preferences.user_id
+    - No circuit breaker between api-gateway and user-service
+
+Phase 3: DOCUMENT
+  Created: docs/postmortems/2026-03-15-user-service-timeout.md
+  Created: docs/runbooks/user-service-database-slow-query.md
+  Updated: docs/incidents/index.md
+
+Phase 4: IMPROVE
+  SLO impact: Consumed 12% of monthly error budget (88% remaining)
+  Action items:
+    1. [P0] Add index on preferences.user_id (owner: @backend, due: 2026-03-16)
+    2. [P1] Add query execution time assertions to integration tests (owner: @backend, due: 2026-03-22)
+    3. [P1] Add circuit breaker on api-gateway -> user-service (owner: @platform, due: 2026-03-22)
+    4. [P2] Add Datadog query performance monitor for user-service (owner: @sre, due: 2026-03-29)
+```
+
+### Example: Kubernetes Pod CrashLoopBackOff with PagerDuty Escalation
+
+```
+Phase 1: ASSESS
+  Signal: PagerDuty incident #4521 — payment-service pods in CrashLoopBackOff
+  Affected: payment-service -> Stripe API -> order-service (downstream)
+  Severity: SEV1 (payment processing completely down)
+  MTTD: 2 minutes (PagerDuty auto-detected from Kubernetes health checks)
+  MTTR: 23 minutes
+
+Phase 2: INVESTIGATE
+  Root cause: Environment variable STRIPE_WEBHOOK_SECRET rotated in Vault
+    but payment-service pods were not restarted to pick up new value.
+    Stripe signature verification failed on all incoming webhooks, causing
+    panic in the webhook handler (no error recovery).
+  Contributing factors:
+    - Vault secret rotation did not trigger pod restart
+    - Webhook handler used panic instead of returning error
+    - No runbook for secret rotation procedures
+
+Phase 3: DOCUMENT
+  Created: docs/postmortems/2026-03-20-payment-service-crashloop.md
+  Created: docs/runbooks/payment-service-secret-rotation.md
+  Created: docs/runbooks/payment-service-stripe-webhook-failure.md
+  Updated: docs/incidents/index.md
+
+Phase 4: IMPROVE
+  SLO impact: Consumed 100% of weekly error budget. Feature freeze recommended.
+  Action items:
+    1. [P0] Add Vault agent sidecar with auto-restart on secret change (owner: @platform)
+    2. [P0] Replace panic with error return in webhook handler (owner: @payments)
+    3. [P1] Add synthetic Stripe webhook test to canary suite (owner: @payments)
+    4. [P2] Create secret rotation runbook for all services (owner: @sre)
+```
+
+## Gates
+
+- **No postmortem without a root cause statement.** A postmortem that says "cause unknown" is incomplete. If the root cause cannot be determined, the postmortem must document what was investigated, what was ruled out, and what additional data is needed. Do not close the investigation.
+- **No action items without owners.** Every action item must have an assigned owner and a due date. Unowned action items are never completed. If no owner can be identified, escalate to the team lead.
+- **No severity downgrade without justification.** If a severity is reclassified during investigation, the reason must be documented in the postmortem timeline. Severity downgrades without evidence indicate pressure to minimize, not genuine reassessment.
+- **No skipping the improvement phase.** Documentation without follow-through produces shelf-ware. The improvement phase must produce at least one concrete, actionable item per contributing factor identified.
+
+## Escalation
+
+- **When root cause cannot be determined from code alone:** The incident may require production logs, metrics, or traces that are not available in the codebase. Report: "Root cause analysis requires access to [specific observability data]. Recommend reviewing [Datadog/Grafana/CloudWatch] dashboards for the incident window."
+- **When the incident reveals a systemic architecture issue:** A single postmortem action item is insufficient. Report: "This incident pattern indicates a systemic issue with [description]. Recommend a dedicated architecture review using harness-architecture-advisor."
+- **When SLO definitions do not exist:** Error budget calculation is impossible without SLOs. Report: "No SLO definitions found. Recommend establishing baseline SLOs before the next incident review. See the SLO starter template in docs/runbooks/slo-template.yaml."
+- **When multiple teams are involved in the blast radius:** A single postmortem owner may not have visibility into all contributing factors. Report: "This incident spans [N] services owned by [teams]. Recommend a joint postmortem review with representatives from each team."