@jetrabbits/agentic 0.0.1

package/areas/devops/database-ops/skills/postgres-operations/SKILL.md

@@ -0,0 +1,156 @@
+---
+name: postgres-operations
+type: skill
+description: PostgreSQL operational runbooks — health checks, vacuum, bloat, locks, PITR, connection pool management.
+related-rules:
+  - backup-policy.md
+  - access-control.md
+  - migration-runbook.md
+allowed-tools: Read, Bash
+---
+
+# Skill: PostgreSQL Operations
+
+> **Expertise:** PostgreSQL health, vacuuming, lock analysis, PITR, WAL archiving, PgBouncer, K8s-hosted PostgreSQL.
+
+## When to load
+
+When investigating a slow database, diagnosing lock waits, running PITR recovery, or managing a PostgreSQL instance.
+
+## Health Check Commands
+
+```sql
+-- Database size overview
+SELECT
+  datname,
+  pg_size_pretty(pg_database_size(datname)) AS size,
+  numbackends AS active_connections
+FROM pg_stat_database
+ORDER BY pg_database_size(datname) DESC;
+
+-- Table sizes (top 20)
+SELECT
+  schemaname || '.' || tablename AS table,
+  pg_size_pretty(pg_total_relation_size(schemaname || '.' || tablename)) AS total_size,
+  pg_size_pretty(pg_relation_size(schemaname || '.' || tablename)) AS table_size,
+  pg_size_pretty(pg_indexes_size(schemaname || '.' || tablename)) AS index_size
+FROM pg_tables
+ORDER BY pg_total_relation_size(schemaname || '.' || tablename) DESC
+LIMIT 20;
+
+-- Replication lag (primary)
+SELECT
+  client_addr,
+  state,
+  pg_size_pretty(pg_wal_lsn_diff(sent_lsn, replay_lsn)) AS replication_lag
+FROM pg_stat_replication;
+```
+
+## Lock Investigation
+
+```sql
+-- Active locks and blocking queries
+SELECT
+  blocking.pid AS blocking_pid,
+  blocking.query AS blocking_query,
+  blocked.pid AS blocked_pid,
+  blocked.query AS blocked_query,
+  blocked.wait_event_type,
+  blocked.wait_event
+FROM pg_stat_activity blocking
+JOIN pg_stat_activity blocked
+  ON blocked.wait_event_type = 'Lock'
+  AND blocking.pid != blocked.pid
+WHERE blocking.state = 'active';
+
+-- Kill blocking query (confirm before running!)
+SELECT pg_terminate_backend(<blocking_pid>);
+
+-- Long-running queries (> 5 min)
+SELECT pid, now() - pg_stat_activity.query_start AS duration, query, state
+FROM pg_stat_activity
+WHERE (now() - pg_stat_activity.query_start) > interval '5 minutes'
+  AND state = 'active';
+```
+
+## VACUUM and Bloat
+
+```sql
+-- Check autovacuum health
+SELECT
+  schemaname || '.' || relname AS table,
+  last_autovacuum,
+  last_autoanalyze,
+  n_dead_tup,
+  n_live_tup,
+  round(n_dead_tup::numeric / NULLIF(n_live_tup + n_dead_tup, 0) * 100, 2) AS dead_pct
+FROM pg_stat_user_tables
+ORDER BY n_dead_tup DESC
+LIMIT 20;
+
+-- Manual VACUUM ANALYZE (non-blocking)
+VACUUM ANALYZE VERBOSE orders;
+
+-- VACUUM FULL (rewrites table — locks! use with maintenance window)
+VACUUM FULL orders;
+```
+
+## Connection Pool (PgBouncer)
+
+```bash
+# Check PgBouncer stats
+psql -h pgbouncer -p 6432 pgbouncer -c "SHOW POOLS;"
+psql -h pgbouncer -p 6432 pgbouncer -c "SHOW STATS;"
+psql -h pgbouncer -p 6432 pgbouncer -c "SHOW CLIENTS;"
+
+# Reload config (no restart needed)
+psql -h pgbouncer -p 6432 pgbouncer -c "RELOAD;"
+
+# PgBouncer config for transaction mode (K8s apps)
+[databases]
+mydb = host=postgres-primary port=5432 dbname=mydb
+
+[pgbouncer]
+pool_mode = transaction
+max_client_conn = 1000
+default_pool_size = 20
+min_pool_size = 5
+server_idle_timeout = 600
+```
+
+## PITR (Point-in-Time Recovery) — pgBackRest
+
+```bash
+# Verify backup status
+pgbackrest --stanza=main info
+
+# Take full backup
+pgbackrest --stanza=main --type=full backup
+
+# PITR restore to specific time
+pgbackrest --stanza=main --delta restore \
+  --target="2024-11-15 03:40:00" \
+  --target-action=promote
+
+# After restore: promote replica to primary
+pg_ctl promote -D /var/lib/postgresql/data
+```
+
+## K8s PostgreSQL (CloudNativePG / Zalando Operator)
+
+```bash
+# Check cluster status (CloudNativePG)
+kubectl get cluster -n database
+kubectl describe cluster postgres-cluster -n database
+
+# Connect to primary
+kubectl exec -it -n database \
+  $(kubectl get pods -n database -l cnpg.io/cluster=postgres-cluster,role=primary -o name) \
+  -- psql -U postgres mydb
+
+# Manual failover
+kubectl cnpg promote postgres-cluster -n database
+
+# Check backup status
+kubectl get backup -n database
+```

package/areas/devops/database-ops/skills/redis-operations/SKILL.md

@@ -0,0 +1,174 @@
+---
+name: redis-operations
+type: skill
+description: Redis operational runbooks — memory management, eviction policy, persistence config, Sentinel/Cluster, K8s-hosted Redis ops.
+related-rules:
+  - backup-policy.md
+  - access-control.md
+allowed-tools: Read, Bash
+---
+
+# Skill: Redis Operations
+
+> **Expertise:** Redis memory management, eviction, persistence (RDB+AOF), Redis Sentinel, Redis Cluster, K8s Redis (Bitnami/Spotahome operator).
+
+## When to load
+
+When investigating Redis memory pressure, configuring persistence, debugging eviction, or setting up Redis HA.
+
+## Health Check Commands
+
+```bash
+# Connect to Redis
+redis-cli -h redis-master -p 6379 -a $REDIS_PASSWORD
+
+# Server info overview
+redis-cli INFO server | grep -E "redis_version|uptime|tcp_port"
+redis-cli INFO memory | grep -E "used_memory_human|used_memory_peak_human|mem_fragmentation_ratio|maxmemory"
+redis-cli INFO stats   | grep -E "total_commands_processed|rejected_connections|evicted_keys"
+redis-cli INFO keyspace  # databases with key counts + expires
+
+# Real-time monitoring (ops/sec per command)
+redis-cli --stat          # 1-second interval stats
+redis-cli MONITOR         # log every command (NEVER in production for long — high overhead)
+
+# Slow log (commands over threshold)
+redis-cli CONFIG SET slowlog-log-slower-than 10000   # 10ms threshold
+redis-cli SLOWLOG GET 20   # last 20 slow commands
+redis-cli SLOWLOG LEN
+redis-cli SLOWLOG RESET
+```
+
+## Memory Management
+
+```bash
+# Check memory usage breakdown
+redis-cli MEMORY DOCTOR        # health analysis + recommendations
+redis-cli MEMORY STATS         # detailed breakdown
+redis-cli MEMORY USAGE <key>   # bytes used by a specific key
+
+# Find big keys (scan, not KEYS — non-blocking)
+redis-cli --bigkeys            # sample-based big key finder
+redis-cli --memkeys            # memory usage per key (sample)
+
+# Memory fragmentation: ratio > 1.5 = fragmentation, < 1.0 = swap
+redis-cli INFO memory | grep mem_fragmentation_ratio
+
+# Defragment memory online (Redis 4+)
+redis-cli CONFIG SET activedefrag yes
+redis-cli CONFIG SET active-defrag-threshold-lower 10   # start at 10% frag
+```
+
+## Eviction Policy
+
+```bash
+# View current policy
+redis-cli CONFIG GET maxmemory-policy
+
+# Set eviction policy
+# allkeys-lru    — evict any key by LRU (general-purpose cache)
+# volatile-lru   — evict only keys with TTL by LRU (mixed TTL/no-TTL use)
+# allkeys-lfu    — evict by LFU (access frequency, Redis 4+) — best for skewed access
+# noeviction     — return OOM error when full (use for session store / queue — no silent data loss)
+redis-cli CONFIG SET maxmemory-policy allkeys-lru
+
+# Set memory limit
+redis-cli CONFIG SET maxmemory 4gb
+
+# Persist config to redis.conf
+redis-cli CONFIG REWRITE
+```
+
+## Persistence Configuration
+
+```bash
+# RDB snapshot (point-in-time)
+redis-cli CONFIG SET save "3600 1 300 100 60 10000"  # save if N changes in M seconds
+
+# AOF (append-only file — more durable)
+redis-cli CONFIG SET appendonly yes
+redis-cli CONFIG SET appendfsync everysec   # fsync every second (balance: perf vs durability)
+# appendfsync always   — safest (every write) — high IOPS
+# appendfsync everysec — recommended — max 1s data loss
+# appendfsync no       — fastest — OS decides when to flush
+
+# AOF rewrite (compaction)
+redis-cli BGREWRITEAOF
+
+# Manual RDB snapshot
+redis-cli BGSAVE
+redis-cli LASTSAVE   # Unix timestamp of last successful save
+
+# Best practice: enable both RDB + AOF
+# RDB for fast restarts, AOF for minimal data loss
+```
+
+## Redis Sentinel (HA)
+
+```bash
+# Check Sentinel status
+redis-cli -p 26379 SENTINEL masters
+redis-cli -p 26379 SENTINEL slaves mymaster
+redis-cli -p 26379 SENTINEL sentinels mymaster
+
+# Force failover (test)
+redis-cli -p 26379 SENTINEL failover mymaster
+
+# sentinel.conf (minimal)
+sentinel monitor mymaster redis-master 6379 2  # quorum=2
+sentinel down-after-milliseconds mymaster 5000
+sentinel failover-timeout mymaster 60000
+sentinel parallel-syncs mymaster 1
+```
+
+## K8s Redis (Bitnami Helm)
+
+```yaml
+# values.yaml (Redis Sentinel mode)
+architecture: replication
+auth:
+  enabled: true
+  existingSecret: redis-password
+  existingSecretPasswordKey: password
+
+master:
+  resources:
+    requests: { memory: 256Mi, cpu: 100m }
+    limits:   { memory: 1Gi,   cpu: 500m }
+  persistence:
+    enabled: true
+    size: 8Gi
+
+replica:
+  replicaCount: 2
+
+sentinel:
+  enabled: true
+  resources:
+    requests: { memory: 64Mi, cpu: 50m }
+    limits:   { memory: 128Mi, cpu: 100m }
+```
+
+```bash
+# K8s Redis health check
+kubectl exec -it redis-master-0 -n cache -- redis-cli ping
+kubectl exec -it redis-master-0 -n cache -- redis-cli INFO replication
+
+# Force failover in K8s
+kubectl exec -it redis-master-0 -n cache -- \
+  redis-cli -p 26379 SENTINEL failover mymaster
+
+# Check pod resources vs actual memory usage
+kubectl top pods -n cache -l app.kubernetes.io/name=redis
+```
+
+## Common Issues & Fixes
+
+| Symptom | Diagnosis | Fix |
+|:---|:---|:---|
+| `OOM command not allowed` | maxmemory reached + `noeviction` | Increase maxmemory or change eviction policy |
+| High eviction rate | Cache too small or no TTLs on keys | Increase maxmemory; audit keys without TTL |
+| `WRONGTYPE` errors | Key type mismatch in application | Flush specific key: `DEL <key>` |
+| Connection refused | maxclients reached | `CONFIG SET maxclients 10000` |
+| Slow KEYS command | Running `KEYS *` in production | Replace with SCAN; never use KEYS in prod |
+| AOF growing unbounded | Auto-rewrite threshold too high | Lower `auto-aof-rewrite-percentage` to 50 |

package/areas/devops/database-ops/workflows/backup-verify.md

@@ -0,0 +1,107 @@
+---
+name: backup-verify
+type: workflow
+trigger: /backup-verify
+description: Verify database backup integrity — automated restore test, row count validation, alert if backup fails.
+inputs:
+  - database_name
+  - backup_tool (pgbackrest|barman|custom)
+outputs:
+  - verification_report
+  - restore_test_passed (bool)
+roles:
+  - devops-engineer
+execution:
+  initiator: developer
+related-rules:
+  - backup-policy.md
+uses-skills:
+  - backup-restore
+  - postgres-operations
+quality-gates:
+  - restore test completes without error
+  - row counts match production within 1%
+  - backup age < 26 hours
+  - test environment destroyed after verification
+---
+
+## Steps
+
+### 1. Pre-Check: Backup Catalog — `@devops-engineer`
+```bash
+# pgBackRest
+pgbackrest --stanza=main info
+
+# Verify: last full backup age < 24h; WAL archiving current
+# Exit non-zero if no backup found → alert on-call immediately
+LAST_FULL=$(pgbackrest --stanza=main info --output=json | jq -r '.[] | .backup[-1].timestamp.stop')
+AGE_HOURS=$(( ( $(date +%s) - $LAST_FULL ) / 3600 ))
+if [ $AGE_HOURS -gt 26 ]; then
+  echo "ALERT: Last backup is ${AGE_HOURS}h old (> 26h threshold)" && exit 1
+fi
+```
+- **Done when:** backup catalog exists, last backup < 26h old
+
+### 2. Provision Test Environment — `@devops-engineer`
+```bash
+# Spin up isolated postgres pod for restore test
+kubectl run restore-test \
+  --image=postgres:16-alpine \
+  --env="POSTGRES_PASSWORD=testpass" \
+  --restart=Never \
+  -n database-ops
+kubectl wait pod/restore-test --for=condition=Ready --timeout=60s
+```
+
+### 3. Restore Latest Backup — `@devops-engineer`
+```bash
+# Restore to test pod (pgBackRest delta restore is faster if data dir pre-exists)
+pgbackrest --stanza=main restore \
+  --pg1-path=/var/lib/postgresql/data \
+  --target-action=promote \
+  --delta
+
+# Start postgres and confirm it accepts connections
+pg_ctl start -D /var/lib/postgresql/data
+psql -c "SELECT 1" postgres  # must succeed
+```
+- **Done when:** postgres starts cleanly; no recovery errors in log
+
+### 4. Row Count Validation — `@devops-engineer`
+```bash
+# Compare critical table row counts: restored vs production
+TABLES="orders payments users products"
+for table in $TABLES; do
+  PROD=$(psql -h postgres-primary -c "SELECT count(*) FROM $table" -t | tr -d ' ')
+  REST=$(psql -h restore-test-svc   -c "SELECT count(*) FROM $table" -t | tr -d ' ')
+  DIFF=$(echo "scale=4; ($PROD - $REST) / $PROD * 100" | bc)
+  echo "$table: prod=$PROD restored=$REST diff=$DIFF%"
+  # Alert if diff > 1%
+done
+```
+- **Done when:** all critical tables within 1% row count of production
+
+### 5. Report + Cleanup — `@devops-engineer`
+```bash
+# Destroy test pod immediately after verification
+kubectl delete pod restore-test -n database-ops
+
+# Write report
+cat > backup-verify-$(date +%Y%m%d).txt << EOF
+Date: $(date -u)
+Backup age: ${AGE_HOURS}h
+Restore: SUCCESS
+Tables validated: $TABLES
+Row count drift: all within 1%
+Test env: destroyed
+EOF
+
+# Post result to Slack
+curl -X POST $SLACK_WEBHOOK \
+  -H 'Content-type: application/json' \
+  --data "{\"text\":\"✅ DB backup verified $(date +%Y-%m-%d) — restore successful, all tables within 1%\"}"
+```
+- **If any step fails:** post failure to Slack + page on-call → P1 incident
+
+## Exit
+Restore successful + row counts validated + test env destroyed + report posted = backup verified.

package/areas/devops/database-ops/workflows/db-incident.md

@@ -0,0 +1,86 @@
+---
+name: db-incident
+type: workflow
+trigger: /db-incident
+description: Respond to a database incident — connection exhaustion, lock contention, replication lag, performance degradation.
+inputs:
+  - database_name
+  - symptom
+  - severity
+outputs:
+  - root_cause_summary
+  - remediation_applied
+roles:
+  - devops-engineer
+  - developer
+execution:
+  initiator: developer
+related-rules:
+  - backup-policy.md
+  - access-control.md
+uses-skills:
+  - postgres-operations
+  - db-performance
+quality-gates:
+  - backup verified before any destructive action
+  - connection pool not bypassed during incident
+---
+
+## Steps
+
+### 1. Triage — `@devops-engineer`
+- Check: connection count, active queries, lock waits, replication lag
+```sql
+SELECT count(*), state FROM pg_stat_activity GROUP BY state;
+SELECT * FROM pg_stat_replication;
+```
+- Check PgBouncer: `SHOW POOLS; SHOW STATS;`
+- **Done when:** failure mode classified (connection exhaustion / lock / slow query / replication)
+
+### 2. Immediate Mitigation by Type
+
+**Connection exhaustion (max_connections reached):**
+```sql
+-- Kill idle connections (not in transaction)
+SELECT pg_terminate_backend(pid)
+FROM pg_stat_activity
+WHERE state = 'idle' AND query_start < now() - interval '30 minutes';
+```
+- Check PgBouncer pool size — increase `default_pool_size` in pgbouncer.ini; `RELOAD`
+
+**Lock contention:**
+```sql
+-- Identify and kill blocking query (after confirming safe)
+SELECT pg_terminate_backend(<blocking_pid>);
+```
+
+**Slow query (high CPU, degraded performance):**
+```sql
+-- Find and kill runaway query
+SELECT pid, query_start, state, query FROM pg_stat_activity
+WHERE state = 'active' ORDER BY query_start ASC LIMIT 10;
+SELECT pg_cancel_backend(<pid>);   -- graceful
+SELECT pg_terminate_backend(<pid>); -- forceful
+```
+
+**Replication lag > RPO threshold:**
+- Check WAL receiver on replica: `SELECT * FROM pg_stat_wal_receiver;`
+- Check network between primary and replica
+- If lag growing: consider increasing `wal_sender_timeout`
+
+### 3. Root Cause — `@devops-engineer` + `@developer`
+- Check `pg_stat_statements` for query regressions (new slow query after deploy?)
+- Check recent schema migrations (new index missing? index not created concurrently?)
+- Review application logs for query pattern change
+
+### 4. Fix & Verify — `@devops-engineer`
+- Apply fix (create missing index, kill leaked connections, tune pgbouncer)
+- Watch metrics stabilize over 5 min
+- **Done when:** connection count normal, query latency normal, no lock waits
+
+### 5. Document — `@devops-engineer`
+- Root cause + fix in incident ticket
+- If query regression: create optimization ticket for development team
+
+## Exit
+Metrics normal + root cause documented = db incident resolved.

package/areas/devops/devsecops/AGENTS.md

@@ -0,0 +1,47 @@
+# DevSecOps — guidance index
+
+## What this area covers
+
+Shift-left security integration: container hardening, SBOM and supply-chain attestation, OPA / Kyverno policy enforcement, secret detection, and Sigstore artifact signing. Security controls are embedded in the delivery pipeline, not applied after the fact.
+
+## Guidance chain
+
+1. Project `.agent/` baseline
+2. `devsecops/rules/*` — load all
+3. `devsecops/skills/*/SKILL.md` — load only the skill matching the current task
+4. `devsecops/workflows/*` — load the workflow matching the triggered command
+
+## Cross-cutting constraints
+
+- **Shift left** — security checks run in CI, not in a post-deploy audit.
+- **Policy as code** — all security policies are version-controlled and machine-enforced; manual review is a supplement, not a substitute.
+- **Container images are immutable artifacts** — no shell access, no package installs at runtime.
+- **Every artifact is signed** — unsigned images and binaries are rejected at admission.
+
+## Spec map
+
+```text
+devsecops/
+├── rules/
+│   ├── shift-left-policy.md       ← required CI checks, fail-fast thresholds
+│   ├── container-security.md      ← base image standards, rootless, read-only FS
+│   └── policy-as-code.md          ← OPA/Kyverno enforcement points, violation handling
+├── skills/
+│   ├── container-hardening/SKILL.md    ← distroless, non-root, capability drops
+│   ├── sbom-supply-chain/SKILL.md      ← Syft, CycloneDX, SLSA provenance
+│   ├── opa-policies/SKILL.md           ← Rego authoring, Conftest, Gatekeeper
+│   ├── secret-detection/SKILL.md       ← Gitleaks, truffleHog, pre-commit integration
+│   └── sigstore-signing/SKILL.md       ← Cosign, keyless signing, Rekor transparency log
+├── workflows/
+│   ├── security-scan-pipeline.md   ← /security-scan-pipeline
+│   └── policy-onboard.md           ← /policy-onboard
+└── prompts/
+    └── *.md
+```
+
+## Discovery patterns
+
+- `rules/*.md`
+- `skills/*/SKILL.md`
+- `workflows/*.md`
+- `prompts/*.md`

package/areas/devops/devsecops/prompts/policy-onboard.md

@@ -0,0 +1,79 @@
+---
+workflow: policy-onboard
+---
+
+# Prompt: `/policy-onboard`
+
+Use when: adding OPA/Kyverno admission policies to a cluster or namespace.
+
+---
+
+## Example 1 — Full admission policy baseline for new cluster
+
+**EN:**
+```
+/policy-onboard
+
+Cluster: prod-cluster-eu
+Policy engine: OPA/Gatekeeper (already installed)
+Scope: all namespaces labelled environment=production or environment=staging
+Policies to deploy (enforcement: deny for prod, warn for staging):
+  - require-non-root (runAsNonRoot: true, UID != 0)
+  - disallow-privileged-containers
+  - require-resource-limits (CPU + memory both required)
+  - require-image-digest (no :latest or mutable tags)
+  - disallow-host-namespaces (no hostNetwork/hostPID/hostIPC)
+  - require-labels: app, team, environment on all Deployments
+Test each policy with: passing manifest + failing manifest before deploying
+```
+
+**RU:**
+```
+/policy-onboard
+
+Кластер: prod-cluster-eu
+Policy engine: OPA/Gatekeeper (уже установлен)
+Скоуп: все namespaces с лейблом environment=production или environment=staging
+Политики для деплоя (enforcement: deny для prod, warn для staging):
+  - require-non-root (runAsNonRoot: true, UID != 0)
+  - disallow-privileged-containers
+  - require-resource-limits (CPU + memory оба обязательны)
+  - require-image-digest (без :latest и mutable тегов)
+  - disallow-host-namespaces (без hostNetwork/hostPID/hostIPC)
+  - require-labels: app, team, environment на всех Deployment
+Протестировать каждую политику с: passing манифест + failing манифест перед деплоем
+```
+
+---
+
+## Example 2 — Fix policy-blocked deployment
+
+**EN:**
+```
+/policy-onboard
+
+Problem: Deployment of order-service v3.0.0 blocked by admission webhook
+Error: "admission webhook gatekeeper denied: Container 'app' must set runAsNonRoot: true"
+Current Dockerfile: no USER instruction; runs as root
+Fix needed:
+  1. Add USER 1000:1000 to Dockerfile
+  2. Add securityContext.runAsNonRoot: true + runAsUser: 1000 to Helm values
+  3. Verify readOnlyRootFilesystem: true won't break app (check for writes to / )
+  4. Rebuild image + re-deploy
+  5. Confirm no other policy violations in the same deployment
+```
+
+**RU:**
+```
+/policy-onboard
+
+Проблема: деплой order-service v3.0.0 заблокирован admission webhook
+Ошибка: "admission webhook gatekeeper denied: Container 'app' must set runAsNonRoot: true"
+Текущий Dockerfile: нет инструкции USER; запускается от root
+Необходимое исправление:
+  1. Добавить USER 1000:1000 в Dockerfile
+  2. Добавить securityContext.runAsNonRoot: true + runAsUser: 1000 в Helm values
+  3. Убедиться что readOnlyRootFilesystem: true не сломает приложение (проверить записи в /)
+  4. Пересобрать образ + повторить деплой
+  5. Подтвердить отсутствие других нарушений политик в том же deployment
+```