@cubis/foundry 0.3.67 → 0.3.69

package/workflows/workflows/agent-environment-setup/platforms/claude/skills/devops-engineer/references/incident-response.md

@@ -1,334 +0,0 @@
-# Incident Response
-
-> Reference for: DevOps Engineer
-> Load when: Production incidents, outages, on-call, MTTR, postmortems, runbooks, chaos engineering
-
-## Response Metrics
-
-- **MTTD** (Mean Time to Detect): Target < 5 minutes
-- **MTTA** (Mean Time to Acknowledge): Target < 5 minutes
-- **MTTR** (Mean Time to Resolve): Target < 30 minutes
-- **MTBF** (Mean Time Between Failures): Maximize
-
-### Severity Levels
-
-| Level | Impact | Response | Example |
-|-------|--------|----------|---------|
-| SEV1 | Complete outage | Immediate | Database down, payment failed |
-| SEV2 | Major degradation | 15 min | API latency >5s, 50% errors |
-| SEV3 | Minor degradation | 1 hour | Non-critical feature broken |
-| SEV4 | Low impact | Business hours | UI glitch, logging issues |
-
-## Runbook Template
-
-```markdown
-# Runbook: High API Error Rate
-
-## Symptoms
-- Alert: `api_error_rate > 0.05`
-- Dashboard: https://grafana.example.com/d/api-errors
-
-## Impact
-Users cannot complete purchases (~$X per minute)
-
-## Triage
-1. Check dashboard for affected endpoints
-2. Check recent deployments: `kubectl rollout history deployment/api`
-3. Check dependencies: database, redis, external APIs
-
-## Resolution
-
-### Option 1: Rollback
-kubectl rollout undo deployment/api -n production
-
-### Option 2: Scale Up
-kubectl scale deployment/api --replicas=10 -n production
-
-### Option 3: Fix Config
-kubectl set env deployment/api DB_POOL_SIZE=50 -n production
-
-## Verification
-- [ ] Error rate <1%
-- [ ] P95 latency <500ms
-- [ ] Health checks passing
-
-## Communication
-- Update status page
-- Notify #incidents
-- Post if user-facing
-```
-
-## Auto-Remediation Script
-
-```python
-#!/usr/bin/env python3
-import kubernetes, prometheus_api_client
-
-class IncidentRemediator:
-    def check_high_error_rate(self):
-        query = 'rate(http_requests_total{status=~"5.."}[5m]) > 0.05'
-        result = self.prometheus.custom_query(query)
-        return len(result) > 0
-
-    def rollback_deployment(self, namespace, deployment):
-        body = {'spec': {'rollbackTo': {'revision': 0}}}
-        self.k8s.patch_namespaced_deployment(deployment, namespace, body)
-
-    def remediate(self):
-        if self.check_high_error_rate():
-            if self.rollback_deployment('production', 'api'):
-                time.sleep(120)
-                if not self.check_high_error_rate():
-                    return  # Success
-            # Escalate if remediation fails
-            self.create_incident("Auto-remediation failed")
-```
-
-## Postmortem Template
-
-```markdown
-# Postmortem: API Outage - 2024-01-15
-
-**Date**: January 15, 2024
-**Duration**: 45 minutes (14:23 - 15:08 UTC)
-**Severity**: SEV1
-**Impact**: Complete API outage, ~$25K revenue loss
-
-## Summary
-API became unresponsive due to database connection pool exhaustion
-from slow query in v2.3.1.
-
-## Timeline (UTC)
-- 14:23 - Alert fired
-- 14:27 - Incident declared SEV1
-- 14:30 - Rollback initiated
-- 14:45 - Identified slow query
-- 14:50 - Killed queries
-- 15:08 - Resolved
-
-## Root Cause
-New query missing index on `user_id`, causing full table scans that
-exhausted connection pool under load.
-
-## Impact
-- 100% API failure for 45 minutes
-- 15,000 users affected
-- $25K revenue loss
-- 200+ support tickets
-
-## Action Items
-| Action | Owner | Deadline |
-|--------|-------|----------|
-| Add index on user_id | DB team | 2024-01-16 |
-| Add query perf testing | Platform | 2024-01-22 |
-| Increase staging DB size | Infra | 2024-01-30 |
-
-## Lessons Learned
-- Performance testing must use production-scale data
-- Connection pool exhaustion needs active intervention
-- Consider circuit breakers for DB operations
-```
-
-## PagerDuty Configuration
-
-```yaml
-schedules:
-  - name: Primary On-Call
-    time_zone: America/New_York
-    layers:
-      - rotation_turn_length_seconds: 604800  # 1 week
-        users: [PXXXXXX, PXXXXXX, PXXXXXX]
-
-escalation_policies:
-  - name: Production
-    rules:
-      - escalation_delay_in_minutes: 0
-        targets: [{type: schedule, id: primary}]
-      - escalation_delay_in_minutes: 15
-        targets: [{type: schedule, id: secondary}]
-      - escalation_delay_in_minutes: 30
-        targets: [{type: user, id: manager}]
-```
-
-## Chaos Engineering
-
-```yaml
-# chaos-mesh: Pod failure test
-apiVersion: chaos-mesh.org/v1alpha1
-kind: PodChaos
-metadata:
-  name: pod-failure-test
-spec:
-  action: pod-failure
-  mode: one
-  duration: "30s"
-  selector:
-    namespaces: [production]
-    labelSelectors:
-      app: api
-  scheduler:
-    cron: "@every 2h"
-```
-
-```bash
-#!/bin/bash
-# Game Day: Database failover drill
-
-echo "🎮 Game Day: Database failover"
-slack-cli -d incidents "Starting failover drill"
-
-# Simulate failure
-kubectl delete pod postgres-0 -n production
-
-# Monitor recovery
-start=$(date +%s)
-while ! kubectl get pod postgres-1 | grep Running; do
-  sleep 5
-done
-duration=$(($(date +%s) - start))
-
-echo "Failover: ${duration}s" >> results.md
-curl -f https://api.example.com/health || echo "FAIL"
-```
-
-## Evidence Collection & Forensics
-
-```bash
-#!/bin/bash
-# collect-evidence.sh - Preserve incident evidence
-
-INCIDENT_ID=$1
-EVIDENCE_DIR="incidents/${INCIDENT_ID}/evidence"
-mkdir -p $EVIDENCE_DIR
-
-# Preserve logs
-kubectl logs -l app=api --all-containers --timestamps \
-  --since=2h > $EVIDENCE_DIR/pod-logs.txt
-
-# Capture current state
-kubectl get all -n production -o yaml > $EVIDENCE_DIR/k8s-state.yaml
-kubectl describe pods -n production > $EVIDENCE_DIR/pod-details.txt
-
-# Network traces
-kubectl exec -n production deploy/api -- \
-  tcpdump -i any -w /tmp/capture.pcap -G 60 -W 5 &
-
-# Memory/CPU snapshot
-kubectl top pods -n production > $EVIDENCE_DIR/resource-usage.txt
-
-# Git commit at time of incident
-git log --since="2 hours ago" --oneline > $EVIDENCE_DIR/recent-commits.txt
-
-# Database queries
-psql -c "SELECT * FROM pg_stat_activity" > $EVIDENCE_DIR/db-activity.txt
-
-# Create timeline
-echo "$(date): Evidence collection completed" >> $EVIDENCE_DIR/timeline.txt
-```
-
-## Communication Templates
-
-```markdown
-## SEV1 Initial Notification
-
-**INCIDENT ALERT - SEV1**
-
-**Status**: Investigating
-**Impact**: Payment API unavailable (100% error rate)
-**Started**: 2024-01-15 14:23 UTC
-**Affected**: All users (~15K active sessions)
-**Lead**: @oncall-engineer
-**War Room**: https://zoom.us/incident-123
-
-Updates every 15 minutes or on major change.
-
----
-
-## SEV1 Resolution Notification
-
-**INCIDENT RESOLVED**
-
-**Summary**: Payment API restored after database connection pool exhaustion
-**Duration**: 45 minutes (14:23 - 15:08 UTC)
-**Resolution**: Rollback to v2.3.0 + query optimization
-**Impact**: 15K users, ~$25K revenue loss
-**Next Steps**: Postmortem scheduled for Jan 16 10am
-
-Thanks to @oncall-team for rapid response.
-```
-
-## Incident Classification
-
-| Type | Examples | Response Team | Escalation |
-|------|----------|---------------|------------|
-| **Security** | Breach, data leak, unauthorized access | Security + DevOps | CISO, Legal |
-| **Service** | Outage, degradation, errors | DevOps + SRE | Engineering VP |
-| **Data** | Corruption, loss, sync issues | DBA + DevOps | CTO |
-| **Compliance** | GDPR, SOC2, audit violations | Compliance + Legal | CEO |
-| **Third-party** | Provider outage, API failures | DevOps + Product | Account manager |
-
-## Security Incident Specifics
-
-```bash
-# Compromise investigation checklist
-□ Isolate affected systems
-□ Preserve logs and memory dumps
-□ Identify attack vector
-□ Check for lateral movement
-□ Scan for malware/backdoors
-□ Review access logs for 30 days
-□ Identify data accessed
-□ Assess exfiltration risk
-□ Check for persistence mechanisms
-□ Coordinate with security team
-□ Notify legal if PII involved
-□ Document chain of custody
-```
-
-## Compliance Requirements
-
-```yaml
-# Incident notification requirements
-gdpr:
-  notification_deadline: 72h
-  authority: Data Protection Officer
-  required_info:
-    - Nature of breach
-    - Data categories affected
-    - Number of individuals
-    - Consequences
-    - Remediation measures
-
-sox:
-  notification_deadline: immediate
-  authority: Audit Committee
-  documentation:
-    - Financial impact
-    - Control failures
-    - Remediation plan
-
-pci_dss:
-  notification_deadline: 24h
-  authority: Card brands + acquirer
-  required_info:
-    - Cardholder data affected
-    - Incident timeline
-    - Forensic investigation
-```
-
-## Best Practices
-
-- Maintain runbooks for critical services
-- Practice with game days monthly
-- Automate common remediation
-- Keep postmortems blameless
-- Track incident metrics
-- Test recovery procedures
-- Document all incidents
-- Improve detection continuously
-- Preserve evidence chain properly
-- Coordinate communication clearly
-- Escalate security incidents immediately
-- Understand compliance obligations
-- Train team on response procedures
-- Review and update playbooks quarterly

package/workflows/workflows/agent-environment-setup/platforms/claude/skills/devops-engineer/references/kubernetes.md

@@ -1,157 +0,0 @@
-# Kubernetes Manifests
-
-> Reference for: DevOps Engineer
-> Load when: K8s deployments, services, ingress, pods, configurations
-
-## Complete Deployment Stack
-
-```yaml
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  name: app
-  labels:
-    app: app
-spec:
-  replicas: 3
-  selector:
-    matchLabels:
-      app: app
-  template:
-    metadata:
-      labels:
-        app: app
-    spec:
-      containers:
-        - name: app
-          image: ghcr.io/org/app:latest
-          ports:
-            - containerPort: 3000
-          resources:
-            requests:
-              memory: "128Mi"
-              cpu: "100m"
-            limits:
-              memory: "256Mi"
-              cpu: "500m"
-          livenessProbe:
-            httpGet:
-              path: /health
-              port: 3000
-            initialDelaySeconds: 10
-            periodSeconds: 10
-          readinessProbe:
-            httpGet:
-              path: /ready
-              port: 3000
-            initialDelaySeconds: 5
-            periodSeconds: 5
-          env:
-            - name: DATABASE_URL
-              valueFrom:
-                secretKeyRef:
-                  name: app-secrets
-                  key: database-url
----
-apiVersion: v1
-kind: Service
-metadata:
-  name: app
-spec:
-  selector:
-    app: app
-  ports:
-    - port: 80
-      targetPort: 3000
-  type: ClusterIP
----
-apiVersion: networking.k8s.io/v1
-kind: Ingress
-metadata:
-  name: app
-  annotations:
-    cert-manager.io/cluster-issuer: letsencrypt-prod
-spec:
-  ingressClassName: nginx
-  tls:
-    - hosts: [app.example.com]
-      secretName: app-tls
-  rules:
-    - host: app.example.com
-      http:
-        paths:
-          - path: /
-            pathType: Prefix
-            backend:
-              service:
-                name: app
-                port:
-                  number: 80
-```
-
-## ConfigMap and Secrets
-
-```yaml
-apiVersion: v1
-kind: ConfigMap
-metadata:
-  name: app-config
-data:
-  LOG_LEVEL: "info"
-  API_TIMEOUT: "30s"
----
-apiVersion: v1
-kind: Secret
-metadata:
-  name: app-secrets
-type: Opaque
-stringData:
-  database-url: "postgres://user:pass@host:5432/db"
-```
-
-## Horizontal Pod Autoscaler
-
-```yaml
-apiVersion: autoscaling/v2
-kind: HorizontalPodAutoscaler
-metadata:
-  name: app-hpa
-spec:
-  scaleTargetRef:
-    apiVersion: apps/v1
-    kind: Deployment
-    name: app
-  minReplicas: 2
-  maxReplicas: 10
-  metrics:
-    - type: Resource
-      resource:
-        name: cpu
-        target:
-          type: Utilization
-          averageUtilization: 70
-```
-
-## Quick Reference
-
-| Resource | Purpose |
-|----------|---------|
-| Deployment | Manages ReplicaSets, rolling updates |
-| Service | Internal load balancing, DNS |
-| Ingress | External HTTP/HTTPS routing |
-| ConfigMap | Non-sensitive configuration |
-| Secret | Sensitive data (base64 encoded) |
-| HPA | Auto-scaling based on metrics |
-| PVC | Persistent storage claims |
-
-## Common kubectl Commands
-
-```bash
-kubectl apply -f deployment.yaml
-kubectl get pods -l app=app
-kubectl describe pod <pod-name>
-kubectl logs -f <pod-name>
-kubectl exec -it <pod-name> -- /bin/sh
-kubectl rollout status deployment/app
-kubectl rollout undo deployment/app
-```