@sylix/coworker 2.0.11 → 2.0.12

package/dist/skills/defaults/devops/secrets-management.md

@@ -0,0 +1,341 @@
+---
+name: secrets-management
+description: Implement secure secrets management for CI/CD pipelines using Vault, AWS Secrets Manager, or native platform solutions. Use when handling sensitive credentials, rotating secrets, or securing CI/CD environments.
+---
+
+# Secrets Management
+
+Secure secrets management practices for CI/CD pipelines using Vault, AWS Secrets Manager, and other tools.
+
+## Purpose
+
+Implement secure secrets management in CI/CD pipelines without hardcoding sensitive information.
+
+## When to Use
+
+- Store API keys and credentials
+- Manage database passwords
+- Handle TLS certificates
+- Rotate secrets automatically
+- Implement least-privilege access
+
+## Secrets Management Tools
+
+### HashiCorp Vault
+
+- Centralized secrets management
+- Dynamic secrets generation
+- Secret rotation
+- Audit logging
+- Fine-grained access control
+
+### AWS Secrets Manager
+
+- AWS-native solution
+- Automatic rotation
+- Integration with RDS
+- CloudFormation support
+
+### Azure Key Vault
+
+- Azure-native solution
+- HSM-backed keys
+- Certificate management
+- RBAC integration
+
+### Google Secret Manager
+
+- GCP-native solution
+- Versioning
+- IAM integration
+
+## HashiCorp Vault Integration
+
+### Setup Vault
+
+```bash
+# Start Vault dev server
+vault server -dev
+
+# Set environment
+export VAULT_ADDR='http://127.0.0.1:8200'
+export VAULT_TOKEN='root'
+
+# Enable secrets engine
+vault secrets enable -path=secret kv-v2
+
+# Store secret
+vault kv put secret/database/config username=admin password=secret
+```
+
+### GitHub Actions with Vault
+
+```yaml
+name: Deploy with Vault Secrets
+
+on: [push]
+
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Import Secrets from Vault
+        uses: hashicorp/vault-action@v2
+        with:
+          url: https://vault.example.com:8200
+          token: ${{ secrets.VAULT_TOKEN }}
+          secrets: |
+            secret/data/database username | DB_USERNAME ;
+            secret/data/database password | DB_PASSWORD ;
+            secret/data/api key | API_KEY
+
+      - name: Use secrets
+        run: |
+          echo "Connecting to database as $DB_USERNAME"
+          # Use $DB_PASSWORD, $API_KEY
+```
+
+### GitLab CI with Vault
+
+```yaml
+deploy:
+  image: vault:latest
+  before_script:
+    - export VAULT_ADDR=https://vault.example.com:8200
+    - export VAULT_TOKEN=$VAULT_TOKEN
+    - apk add curl jq
+  script: |
+    DB_PASSWORD=$(vault kv get -field=password secret/database/config)
+    API_KEY=$(vault kv get -field=key secret/api/credentials)
+    echo "Deploying with secrets..."
+    # Use $DB_PASSWORD, $API_KEY
+```
+
+## AWS Secrets Manager
+
+### Store Secret
+
+```bash
+aws secretsmanager create-secret \
+  --name production/database/password \
+  --secret-string "super-secret-password"
+```
+
+### Retrieve in GitHub Actions
+
+```yaml
+- name: Configure AWS credentials
+  uses: aws-actions/configure-aws-credentials@v4
+  with:
+    aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
+    aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
+    aws-region: us-west-2
+
+- name: Get secret from AWS
+  run: |
+    SECRET=$(aws secretsmanager get-secret-value \
+      --secret-id production/database/password \
+      --query SecretString \
+      --output text)
+    echo "::add-mask::$SECRET"
+    echo "DB_PASSWORD=$SECRET" >> $GITHUB_ENV
+
+- name: Use secret
+  run: |
+    # Use $DB_PASSWORD
+    ./deploy.sh
+```
+
+### Terraform with AWS Secrets Manager
+
+```hcl
+data "aws_secretsmanager_secret_version" "db_password" {
+  secret_id = "production/database/password"
+}
+
+resource "aws_db_instance" "main" {
+  allocated_storage    = 100
+  engine               = "postgres"
+  instance_class       = "db.t3.large"
+  username             = "admin"
+  password             = jsondecode(data.aws_secretsmanager_secret_version.db_password.secret_string)["password"]
+}
+```
+
+## GitHub Secrets
+
+### Organization/Repository Secrets
+
+```yaml
+- name: Use GitHub secret
+  run: |
+    echo "API Key: ${{ secrets.API_KEY }}"
+    echo "Database URL: ${{ secrets.DATABASE_URL }}"
+```
+
+### Environment Secrets
+
+```yaml
+deploy:
+  runs-on: ubuntu-latest
+  environment: production
+  steps:
+    - name: Deploy
+      run: |
+        echo "Deploying with ${{ secrets.PROD_API_KEY }}"
+```
+
+## GitLab CI/CD Variables
+
+### Project Variables
+
+```yaml
+deploy:
+  script:
+    - echo "Deploying with $API_KEY"
+    - echo "Database: $DATABASE_URL"
+```
+
+### Protected and Masked Variables
+
+- Protected: Only available in protected branches
+- Masked: Hidden in job logs
+- File type: Stored as file
+
+## Best Practices
+
+1. **Never commit secrets** to Git
+2. **Use different secrets** per environment
+3. **Rotate secrets regularly**
+4. **Implement least-privilege access**
+5. **Enable audit logging**
+6. **Use secret scanning** (GitGuardian, TruffleHog)
+7. **Mask secrets in logs**
+8. **Encrypt secrets at rest**
+9. **Use short-lived tokens** when possible
+10. **Document secret requirements**
+
+## Secret Rotation
+
+### Automated Rotation with AWS
+
+```python
+import boto3
+import json
+
+def lambda_handler(event, context):
+    client = boto3.client('secretsmanager')
+
+    # Get current secret
+    response = client.get_secret_value(SecretId='my-secret')
+    current_secret = json.loads(response['SecretString'])
+
+    # Generate new password
+    new_password = generate_strong_password()
+
+    # Update database password
+    update_database_password(new_password)
+
+    # Update secret
+    client.put_secret_value(
+        SecretId='my-secret',
+        SecretString=json.dumps({
+            'username': current_secret['username'],
+            'password': new_password
+        })
+    )
+
+    return {'statusCode': 200}
+```
+
+### Manual Rotation Process
+
+1. Generate new secret
+2. Update secret in secret store
+3. Update applications to use new secret
+4. Verify functionality
+5. Revoke old secret
+
+## External Secrets Operator
+
+### Kubernetes Integration
+
+```yaml
+apiVersion: external-secrets.io/v1beta1
+kind: SecretStore
+metadata:
+  name: vault-backend
+  namespace: production
+spec:
+  provider:
+    vault:
+      server: "https://vault.example.com:8200"
+      path: "secret"
+      version: "v2"
+      auth:
+        kubernetes:
+          mountPath: "kubernetes"
+          role: "production"
+
+---
+apiVersion: external-secrets.io/v1beta1
+kind: ExternalSecret
+metadata:
+  name: database-credentials
+  namespace: production
+spec:
+  refreshInterval: 1h
+  secretStoreRef:
+    name: vault-backend
+    kind: SecretStore
+  target:
+    name: database-credentials
+    creationPolicy: Owner
+  data:
+    - secretKey: username
+      remoteRef:
+        key: database/config
+        property: username
+    - secretKey: password
+      remoteRef:
+        key: database/config
+        property: password
+```
+
+## Secret Scanning
+
+### Pre-commit Hook
+
+```bash
+#!/bin/bash
+# .git/hooks/pre-commit
+
+# Check for secrets with TruffleHog
+docker run --rm -v "$(pwd):/repo" \
+  trufflesecurity/trufflehog:latest \
+  filesystem --directory=/repo
+
+if [ $? -ne 0 ]; then
+  echo "❌ Secret detected! Commit blocked."
+  exit 1
+fi
+```
+
+### CI/CD Secret Scanning
+
+```yaml
+secret-scan:
+  stage: security
+  image: trufflesecurity/trufflehog:latest
+  script:
+    - trufflehog filesystem .
+  allow_failure: false
+```
+
+## Related Skills
+
+- `github-actions-templates` - For GitHub Actions integration
+- `gitlab-ci-patterns` - For GitLab CI integration
+- `deployment-pipeline-design` - For pipeline architecture

package/dist/skills/defaults/devops/service-mesh-observability.md

@@ -0,0 +1,385 @@
+---
+name: service-mesh-observability
+description: Implement comprehensive observability for service meshes including distributed tracing, metrics, and visualization. Use when setting up mesh monitoring, debugging latency issues, or implementing SLOs for service communication.
+---
+
+# Service Mesh Observability
+
+Complete guide to observability patterns for Istio, Linkerd, and service mesh deployments.
+
+## When to Use This Skill
+
+- Setting up distributed tracing across services
+- Implementing service mesh metrics and dashboards
+- Debugging latency and error issues
+- Defining SLOs for service communication
+- Visualizing service dependencies
+- Troubleshooting mesh connectivity
+
+## Core Concepts
+
+### 1. Three Pillars of Observability
+
+```
+┌─────────────────────────────────────────────────────┐
+│                  Observability                       │
+├─────────────────┬─────────────────┬─────────────────┤
+│     Metrics     │     Traces      │      Logs       │
+│                 │                 │                 │
+│ • Request rate  │ • Span context  │ • Access logs   │
+│ • Error rate    │ • Latency       │ • Error details │
+│ • Latency P50   │ • Dependencies  │ • Debug info    │
+│ • Saturation    │ • Bottlenecks   │ • Audit trail   │
+└─────────────────┴─────────────────┴─────────────────┘
+```
+
+### 2. Golden Signals for Mesh
+
+| Signal         | Description               | Alert Threshold   |
+| -------------- | ------------------------- | ----------------- |
+| **Latency**    | Request duration P50, P99 | P99 > 500ms       |
+| **Traffic**    | Requests per second       | Anomaly detection |
+| **Errors**     | 5xx error rate            | > 1%              |
+| **Saturation** | Resource utilization      | > 80%             |
+
+## Templates
+
+### Template 1: Istio with Prometheus & Grafana
+
+```yaml
+# Install Prometheus
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: prometheus
+  namespace: istio-system
+data:
+  prometheus.yml: |
+    global:
+      scrape_interval: 15s
+    scrape_configs:
+      - job_name: 'istio-mesh'
+        kubernetes_sd_configs:
+          - role: endpoints
+            namespaces:
+              names:
+                - istio-system
+        relabel_configs:
+          - source_labels: [__meta_kubernetes_service_name]
+            action: keep
+            regex: istio-telemetry
+---
+# ServiceMonitor for Prometheus Operator
+apiVersion: monitoring.coreos.com/v1
+kind: ServiceMonitor
+metadata:
+  name: istio-mesh
+  namespace: istio-system
+spec:
+  selector:
+    matchLabels:
+      app: istiod
+  endpoints:
+    - port: http-monitoring
+      interval: 15s
+```
+
+### Template 2: Key Istio Metrics Queries
+
+```promql
+# Request rate by service
+sum(rate(istio_requests_total{reporter="destination"}[5m])) by (destination_service_name)
+
+# Error rate (5xx)
+sum(rate(istio_requests_total{reporter="destination", response_code=~"5.."}[5m]))
+  / sum(rate(istio_requests_total{reporter="destination"}[5m])) * 100
+
+# P99 latency
+histogram_quantile(0.99,
+  sum(rate(istio_request_duration_milliseconds_bucket{reporter="destination"}[5m]))
+  by (le, destination_service_name))
+
+# TCP connections
+sum(istio_tcp_connections_opened_total{reporter="destination"}) by (destination_service_name)
+
+# Request size
+histogram_quantile(0.99,
+  sum(rate(istio_request_bytes_bucket{reporter="destination"}[5m]))
+  by (le, destination_service_name))
+```
+
+### Template 3: Jaeger Distributed Tracing
+
+```yaml
+# Jaeger installation for Istio
+apiVersion: install.istio.io/v1alpha1
+kind: IstioOperator
+spec:
+  meshConfig:
+    enableTracing: true
+    defaultConfig:
+      tracing:
+        sampling: 100.0 # 100% in dev, lower in prod
+        zipkin:
+          address: jaeger-collector.istio-system:9411
+---
+# Jaeger deployment
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: jaeger
+  namespace: istio-system
+spec:
+  selector:
+    matchLabels:
+      app: jaeger
+  template:
+    metadata:
+      labels:
+        app: jaeger
+    spec:
+      containers:
+        - name: jaeger
+          image: jaegertracing/all-in-one:1.50
+          ports:
+            - containerPort: 5775 # UDP
+            - containerPort: 6831 # Thrift
+            - containerPort: 6832 # Thrift
+            - containerPort: 5778 # Config
+            - containerPort: 16686 # UI
+            - containerPort: 14268 # HTTP
+            - containerPort: 14250 # gRPC
+            - containerPort: 9411 # Zipkin
+          env:
+            - name: COLLECTOR_ZIPKIN_HOST_PORT
+              value: ":9411"
+```
+
+### Template 4: Linkerd Viz Dashboard
+
+```bash
+# Install Linkerd viz extension
+linkerd viz install | kubectl apply -f -
+
+# Access dashboard
+linkerd viz dashboard
+
+# CLI commands for observability
+# Top requests
+linkerd viz top deploy/my-app
+
+# Per-route metrics
+linkerd viz routes deploy/my-app --to deploy/backend
+
+# Live traffic inspection
+linkerd viz tap deploy/my-app --to deploy/backend
+
+# Service edges (dependencies)
+linkerd viz edges deployment -n my-namespace
+```
+
+### Template 5: Grafana Dashboard JSON
+
+```json
+{
+  "dashboard": {
+    "title": "Service Mesh Overview",
+    "panels": [
+      {
+        "title": "Request Rate",
+        "type": "graph",
+        "targets": [
+          {
+            "expr": "sum(rate(istio_requests_total{reporter=\"destination\"}[5m])) by (destination_service_name)",
+            "legendFormat": "{{destination_service_name}}"
+          }
+        ]
+      },
+      {
+        "title": "Error Rate",
+        "type": "gauge",
+        "targets": [
+          {
+            "expr": "sum(rate(istio_requests_total{response_code=~\"5..\"}[5m])) / sum(rate(istio_requests_total[5m])) * 100"
+          }
+        ],
+        "fieldConfig": {
+          "defaults": {
+            "thresholds": {
+              "steps": [
+                { "value": 0, "color": "green" },
+                { "value": 1, "color": "yellow" },
+                { "value": 5, "color": "red" }
+              ]
+            }
+          }
+        }
+      },
+      {
+        "title": "P99 Latency",
+        "type": "graph",
+        "targets": [
+          {
+            "expr": "histogram_quantile(0.99, sum(rate(istio_request_duration_milliseconds_bucket{reporter=\"destination\"}[5m])) by (le, destination_service_name))",
+            "legendFormat": "{{destination_service_name}}"
+          }
+        ]
+      },
+      {
+        "title": "Service Topology",
+        "type": "nodeGraph",
+        "targets": [
+          {
+            "expr": "sum(rate(istio_requests_total{reporter=\"destination\"}[5m])) by (source_workload, destination_service_name)"
+          }
+        ]
+      }
+    ]
+  }
+}
+```
+
+### Template 6: Kiali Service Mesh Visualization
+
+```yaml
+# Kiali installation
+apiVersion: kiali.io/v1alpha1
+kind: Kiali
+metadata:
+  name: kiali
+  namespace: istio-system
+spec:
+  auth:
+    strategy: anonymous # or openid, token
+  deployment:
+    accessible_namespaces:
+      - "**"
+  external_services:
+    prometheus:
+      url: http://prometheus.istio-system:9090
+    tracing:
+      url: http://jaeger-query.istio-system:16686
+    grafana:
+      url: http://grafana.istio-system:3000
+```
+
+### Template 7: OpenTelemetry Integration
+
+```yaml
+# OpenTelemetry Collector for mesh
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: otel-collector-config
+data:
+  config.yaml: |
+    receivers:
+      otlp:
+        protocols:
+          grpc:
+            endpoint: 0.0.0.0:4317
+          http:
+            endpoint: 0.0.0.0:4318
+      zipkin:
+        endpoint: 0.0.0.0:9411
+
+    processors:
+      batch:
+        timeout: 10s
+
+    exporters:
+      jaeger:
+        endpoint: jaeger-collector:14250
+        tls:
+          insecure: true
+      prometheus:
+        endpoint: 0.0.0.0:8889
+
+    service:
+      pipelines:
+        traces:
+          receivers: [otlp, zipkin]
+          processors: [batch]
+          exporters: [jaeger]
+        metrics:
+          receivers: [otlp]
+          processors: [batch]
+          exporters: [prometheus]
+---
+# Istio Telemetry v2 with OTel
+apiVersion: telemetry.istio.io/v1alpha1
+kind: Telemetry
+metadata:
+  name: mesh-default
+  namespace: istio-system
+spec:
+  tracing:
+    - providers:
+        - name: otel
+      randomSamplingPercentage: 10
+```
+
+## Alerting Rules
+
+```yaml
+apiVersion: monitoring.coreos.com/v1
+kind: PrometheusRule
+metadata:
+  name: mesh-alerts
+  namespace: istio-system
+spec:
+  groups:
+    - name: mesh.rules
+      rules:
+        - alert: HighErrorRate
+          expr: |
+            sum(rate(istio_requests_total{response_code=~"5.."}[5m])) by (destination_service_name)
+            / sum(rate(istio_requests_total[5m])) by (destination_service_name) > 0.05
+          for: 5m
+          labels:
+            severity: critical
+          annotations:
+            summary: "High error rate for {{ $labels.destination_service_name }}"
+
+        - alert: HighLatency
+          expr: |
+            histogram_quantile(0.99, sum(rate(istio_request_duration_milliseconds_bucket[5m]))
+            by (le, destination_service_name)) > 1000
+          for: 5m
+          labels:
+            severity: warning
+          annotations:
+            summary: "High P99 latency for {{ $labels.destination_service_name }}"
+
+        - alert: MeshCertExpiring
+          expr: |
+            (certmanager_certificate_expiration_timestamp_seconds - time()) / 86400 < 7
+          labels:
+            severity: warning
+          annotations:
+            summary: "Mesh certificate expiring in less than 7 days"
+```
+
+## Best Practices
+
+### Do's
+
+- **Sample appropriately** - 100% in dev, 1-10% in prod
+- **Use trace context** - Propagate headers consistently
+- **Set up alerts** - For golden signals
+- **Correlate metrics/traces** - Use exemplars
+- **Retain strategically** - Hot/cold storage tiers
+
+### Don'ts
+
+- **Don't over-sample** - Storage costs add up
+- **Don't ignore cardinality** - Limit label values
+- **Don't skip dashboards** - Visualize dependencies
+- **Don't forget costs** - Monitor observability costs
+
+## Resources
+
+- [Istio Observability](https://istio.io/latest/docs/tasks/observability/)
+- [Linkerd Observability](https://linkerd.io/2.14/features/dashboard/)
+- [OpenTelemetry](https://opentelemetry.io/)
+- [Kiali](https://kiali.io/)