aigroup-workflow 2.1.1 → 2.2.0

package/skills/monitoring-expert/SKILL.md

@@ -1,176 +0,0 @@
----
-name: monitoring-expert
-description: Configures monitoring systems, implements structured logging pipelines, creates Prometheus/Grafana dashboards, defines alerting rules, and instruments distributed tracing. Implements Prometheus/Grafana stacks, conducts load testing, performs application profiling, and plans infrastructure capacity. Use when setting up application monitoring, adding observability to services, debugging production issues with logs/metrics/traces, running load tests with k6 or Artillery, profiling CPU/memory bottlenecks, or forecasting capacity needs.
-license: MIT
-metadata:
-  author: https://github.com/Jeffallan
-  version: "1.1.0"
-  domain: devops
-  triggers: monitoring, observability, logging, metrics, tracing, alerting, Prometheus, Grafana, DataDog, APM, performance testing, load testing, profiling, capacity planning, bottleneck
-  role: specialist
-  scope: implementation
-  output-format: code
-  related-skills: devops-engineer, debugging-wizard, architecture-designer
----
-
-# Monitoring Expert
-
-Observability and performance specialist implementing comprehensive monitoring, alerting, tracing, and performance testing systems.
-
-## Core Workflow
-
-1. **Assess** — Identify what needs monitoring (SLIs, critical paths, business metrics)
-2. **Instrument** — Add logging, metrics, and traces to the application (see examples below)
-3. **Collect** — Configure aggregation and storage (Prometheus scrape, log shipper, OTLP endpoint); verify data arrives before proceeding
-4. **Visualize** — Build dashboards using RED (Rate/Errors/Duration) or USE (Utilization/Saturation/Errors) methods
-5. **Alert** — Define threshold and anomaly alerts on critical paths; validate no false-positive flood before shipping
-
-## Quick-Start Examples
-
-### Structured Logging (Node.js / Pino)
-```js
-import pino from 'pino';
-
-const logger = pino({ level: 'info' });
-
-// Good — structured fields, includes correlation ID
-logger.info({ requestId: req.id, userId: req.user.id, durationMs: elapsed }, 'order.created');
-
-// Bad — string interpolation, no correlation
-console.log(`Order created for user ${userId}`);
-```
-
-### Prometheus Metrics (Node.js)
-```js
-import { Counter, Histogram, register } from 'prom-client';
-
-const httpRequests = new Counter({
-  name: 'http_requests_total',
-  help: 'Total HTTP requests',
-  labelNames: ['method', 'route', 'status'],
-});
-
-const httpDuration = new Histogram({
-  name: 'http_request_duration_seconds',
-  help: 'HTTP request latency',
-  labelNames: ['method', 'route'],
-  buckets: [0.05, 0.1, 0.3, 0.5, 1, 2, 5],
-});
-
-// Instrument a route
-app.use((req, res, next) => {
-  const end = httpDuration.startTimer({ method: req.method, route: req.path });
-  res.on('finish', () => {
-    httpRequests.inc({ method: req.method, route: req.path, status: res.statusCode });
-    end();
-  });
-  next();
-});
-
-// Expose scrape endpoint
-app.get('/metrics', async (req, res) => {
-  res.set('Content-Type', register.contentType);
-  res.end(await register.metrics());
-});
-```
-
-### OpenTelemetry Tracing (Node.js)
-```js
-import { NodeSDK } from '@opentelemetry/sdk-node';
-import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
-import { trace } from '@opentelemetry/api';
-
-const sdk = new NodeSDK({
-  traceExporter: new OTLPTraceExporter({ url: 'http://jaeger:4318/v1/traces' }),
-});
-sdk.start();
-
-// Manual span around a critical operation
-const tracer = trace.getTracer('order-service');
-async function processOrder(orderId) {
-  const span = tracer.startSpan('order.process');
-  span.setAttribute('order.id', orderId);
-  try {
-    const result = await db.saveOrder(orderId);
-    span.setStatus({ code: SpanStatusCode.OK });
-    return result;
-  } catch (err) {
-    span.recordException(err);
-    span.setStatus({ code: SpanStatusCode.ERROR });
-    throw err;
-  } finally {
-    span.end();
-  }
-}
-```
-
-### Prometheus Alerting Rule
-```yaml
-groups:
-  - name: api.rules
-    rules:
-      - alert: HighErrorRate
-        expr: |
-          rate(http_requests_total{status=~"5.."}[5m])
-          / rate(http_requests_total[5m]) > 0.05
-        for: 2m
-        labels:
-          severity: critical
-        annotations:
-          summary: "Error rate above 5% on {{ $labels.route }}"
-```
-
-### k6 Load Test
-```js
-import http from 'k6/http';
-import { check, sleep } from 'k6';
-
-export const options = {
-  stages: [
-    { duration: '1m', target: 50 },   // ramp up
-    { duration: '5m', target: 50 },   // sustained load
-    { duration: '1m', target: 0 },    // ramp down
-  ],
-  thresholds: {
-    http_req_duration: ['p(95)<500'],  // 95th percentile < 500 ms
-    http_req_failed:   ['rate<0.01'],  // error rate < 1%
-  },
-};
-
-export default function () {
-  const res = http.get('https://api.example.com/orders');
-  check(res, { 'status is 200': (r) => r.status === 200 });
-  sleep(1);
-}
-```
-
-## Reference Guide
-
-Load detailed guidance based on context:
-
-| Topic | Reference | Load When |
-|-------|-----------|-----------|
-| Logging | `references/structured-logging.md` | Pino, JSON logging |
-| Metrics | `references/prometheus-metrics.md` | Counter, Histogram, Gauge |
-| Tracing | `references/opentelemetry.md` | OpenTelemetry, spans |
-| Alerting | `references/alerting-rules.md` | Prometheus alerts |
-| Dashboards | `references/dashboards.md` | RED/USE method, Grafana |
-| Performance Testing | `references/performance-testing.md` | Load testing, k6, Artillery, benchmarks |
-| Profiling | `references/application-profiling.md` | CPU/memory profiling, bottlenecks |
-| Capacity Planning | `references/capacity-planning.md` | Scaling, forecasting, budgets |
-
-## Constraints
-
-### MUST DO
-- Use structured logging (JSON)
-- Include request IDs for correlation
-- Set up alerts for critical paths
-- Monitor business metrics, not just technical
-- Use appropriate metric types (counter/gauge/histogram)
-- Implement health check endpoints
-
-### MUST NOT DO
-- Log sensitive data (passwords, tokens, PII)
-- Alert on every error (alert fatigue)
-- Use string interpolation in logs (use structured fields)
-- Skip correlation IDs in distributed systems

package/skills/monitoring-expert/references/alerting-rules.md

@@ -1,141 +0,0 @@
-# Alerting Rules
-
-## Prometheus Alert Rules
-
-```yaml
-# alerts.yml
-groups:
-  - name: application
-    rules:
-      - alert: HighErrorRate
-        expr: |
-          sum(rate(http_requests_total{status=~"5.."}[5m]))
-          /
-          sum(rate(http_requests_total[5m])) > 0.05
-        for: 5m
-        labels:
-          severity: critical
-        annotations:
-          summary: High error rate detected
-          description: Error rate is {{ $value | humanizePercentage }}
-
-      - alert: HighLatency
-        expr: |
-          histogram_quantile(0.95, rate(http_request_duration_seconds_bucket[5m])) > 1
-        for: 5m
-        labels:
-          severity: warning
-        annotations:
-          summary: High latency detected
-          description: 95th percentile latency is {{ $value }}s
-
-      - alert: ServiceDown
-        expr: up == 0
-        for: 1m
-        labels:
-          severity: critical
-        annotations:
-          summary: Service {{ $labels.instance }} is down
-
-  - name: infrastructure
-    rules:
-      - alert: HighMemoryUsage
-        expr: |
-          (node_memory_MemTotal_bytes - node_memory_MemAvailable_bytes)
-          / node_memory_MemTotal_bytes > 0.9
-        for: 5m
-        labels:
-          severity: warning
-        annotations:
-          summary: High memory usage on {{ $labels.instance }}
-
-      - alert: HighCPUUsage
-        expr: |
-          100 - (avg by(instance) (rate(node_cpu_seconds_total{mode="idle"}[5m])) * 100) > 80
-        for: 10m
-        labels:
-          severity: warning
-        annotations:
-          summary: High CPU usage on {{ $labels.instance }}
-
-      - alert: DiskSpaceLow
-        expr: |
-          (node_filesystem_avail_bytes / node_filesystem_size_bytes) < 0.1
-        for: 5m
-        labels:
-          severity: critical
-        annotations:
-          summary: Disk space low on {{ $labels.instance }}
-```
-
-## Alert Design Principles
-
-```yaml
-# Good alert: Actionable, specific
-- alert: DatabaseConnectionPoolExhausted
-  expr: db_pool_available_connections == 0
-  for: 2m
-  annotations:
-    runbook_url: https://wiki.example.com/runbooks/db-pool
-
-# Bad alert: Too noisy, not actionable
-- alert: AnyError
-  expr: errors_total > 0  # Will always fire
-```
-
-## Severity Levels
-
-| Severity | Response | Example |
-|----------|----------|---------|
-| `critical` | Page immediately | Service down, data loss |
-| `warning` | Investigate soon | High latency, low disk |
-| `info` | Check in morning | Unusual traffic pattern |
-
-## Alertmanager Configuration
-
-```yaml
-# alertmanager.yml
-global:
-  slack_api_url: 'https://hooks.slack.com/...'
-
-route:
-  receiver: 'slack-notifications'
-  group_by: ['alertname', 'severity']
-  group_wait: 30s
-  group_interval: 5m
-  repeat_interval: 4h
-
-  routes:
-    - match:
-        severity: critical
-      receiver: 'pagerduty'
-    - match:
-        severity: warning
-      receiver: 'slack-notifications'
-
-receivers:
-  - name: 'slack-notifications'
-    slack_configs:
-      - channel: '#alerts'
-        send_resolved: true
-
-  - name: 'pagerduty'
-    pagerduty_configs:
-      - service_key: 'your-key'
-```
-
-## Quick Reference
-
-| Field | Purpose |
-|-------|---------|
-| `expr` | PromQL query |
-| `for` | Duration before firing |
-| `labels` | Classification (severity) |
-| `annotations` | Human-readable info |
-
-| Threshold | Use |
-|-----------|-----|
-| Error rate > 5% | Critical |
-| p95 latency > 1s | Warning |
-| Disk < 10% | Critical |
-| Memory > 90% | Warning |

package/skills/monitoring-expert/references/application-profiling.md

@@ -1,331 +0,0 @@
-# Application Profiling
-
-## Node.js Profiling
-
-### CPU Profiling with clinic.js
-
-```bash
-# Install
-npm install -g clinic
-
-# CPU profiling
-clinic doctor -- node app.js
-
-# Flame graph
-clinic flame -- node app.js
-
-# Bubble profiling
-clinic bubbleprof -- node app.js
-
-# Generate report
-clinic doctor --collect-only -- node app.js
-clinic doctor --visualize-only PID.clinic-doctor
-```
-
-### Built-in Node.js Profiler
-
-```javascript
-// Start profiling
-node --prof app.js
-
-# Process the output
-node --prof-process isolate-0x*.log > processed.txt
-
-# Chrome DevTools
-node --inspect app.js
-# Open chrome://inspect
-```
-
-### Memory Profiling
-
-```javascript
-import v8 from 'v8';
-import fs from 'fs';
-
-// Heap snapshot
-const snapshot = v8.writeHeapSnapshot();
-console.log('Snapshot written to:', snapshot);
-
-// Memory usage
-const usage = process.memoryUsage();
-console.log({
-  rss: `${Math.round(usage.rss / 1024 / 1024)}MB`,
-  heapTotal: `${Math.round(usage.heapTotal / 1024 / 1024)}MB`,
-  heapUsed: `${Math.round(usage.heapUsed / 1024 / 1024)}MB`,
-  external: `${Math.round(usage.external / 1024 / 1024)}MB`,
-});
-```
-
-### Custom Performance Marks
-
-```javascript
-import { performance, PerformanceObserver } from 'perf_hooks';
-
-// Mark start
-performance.mark('operation-start');
-
-// ... do work ...
-await processOrder(orderId);
-
-// Mark end
-performance.mark('operation-end');
-
-// Measure
-performance.measure('operation', 'operation-start', 'operation-end');
-
-// Observer
-const obs = new PerformanceObserver((items) => {
-  items.getEntries().forEach((entry) => {
-    console.log(`${entry.name}: ${entry.duration}ms`);
-  });
-});
-obs.observe({ entryTypes: ['measure'] });
-```
-
-## Python Profiling
-
-### cProfile
-
-```python
-import cProfile
-import pstats
-
-# Profile a function
-def main():
-    # Your code here
-    process_data()
-
-if __name__ == '__main__':
-    profiler = cProfile.Profile()
-    profiler.enable()
-
-    main()
-
-    profiler.disable()
-    stats = pstats.Stats(profiler)
-    stats.sort_stats('cumulative')
-    stats.print_stats(20)  # Top 20 functions
-```
-
-### Line Profiler
-
-```python
-from line_profiler import LineProfiler
-
-@profile
-def expensive_function():
-    # Code to profile
-    result = []
-    for i in range(10000):
-        result.append(i ** 2)
-    return result
-
-# Run with: kernprof -l -v script.py
-```
-
-### Memory Profiler
-
-```python
-from memory_profiler import profile
-
-@profile
-def process_large_data():
-    data = [i for i in range(1000000)]
-    result = [x * 2 for x in data]
-    return result
-
-# Run with: python -m memory_profiler script.py
-```
-
-### py-spy
-
-```bash
-# CPU sampling (live process)
-py-spy top --pid 12345
-
-# Generate flame graph
-py-spy record -o profile.svg --pid 12345
-
-# Record for duration
-py-spy record -o profile.svg --duration 60 -- python app.py
-```
-
-## Go Profiling
-
-### pprof
-
-```go
-import (
-    "net/http"
-    _ "net/http/pprof"
-    "runtime"
-)
-
-func main() {
-    // Enable profiling endpoint
-    go func() {
-        http.ListenAndServe("localhost:6060", nil)
-    }()
-
-    // Your application code
-}
-```
-
-```bash
-# CPU profile
-curl http://localhost:6060/debug/pprof/profile?seconds=30 > cpu.prof
-go tool pprof cpu.prof
-
-# Memory profile
-curl http://localhost:6060/debug/pprof/heap > heap.prof
-go tool pprof heap.prof
-
-# Goroutine profile
-curl http://localhost:6060/debug/pprof/goroutine > goroutine.prof
-go tool pprof goroutine.prof
-
-# Web interface
-go tool pprof -http=:8080 cpu.prof
-```
-
-## Java Profiling
-
-### VisualVM
-
-```bash
-# Start application with JMX
-java -Dcom.sun.management.jmxremote \
-     -Dcom.sun.management.jmxremote.port=9010 \
-     -Dcom.sun.management.jmxremote.authenticate=false \
-     -Dcom.sun.management.jmxremote.ssl=false \
-     -jar app.jar
-
-# Connect with VisualVM
-jvisualvm
-```
-
-### async-profiler
-
-```bash
-# CPU profiling
-./profiler.sh -d 30 -f cpu.html <pid>
-
-# Allocation profiling
-./profiler.sh -d 30 -e alloc -f alloc.html <pid>
-
-# Flame graph
-./profiler.sh -d 30 -f flamegraph.svg <pid>
-```
-
-## Database Query Profiling
-
-### PostgreSQL
-
-```sql
--- Enable query logging
-ALTER SYSTEM SET log_min_duration_statement = 100;  -- Log queries > 100ms
-SELECT pg_reload_conf();
-
--- Explain analyze
-EXPLAIN ANALYZE
-SELECT * FROM orders
-WHERE user_id = 123
-AND created_at > NOW() - INTERVAL '30 days';
-
--- Track slow queries
-SELECT query, calls, total_time, mean_time
-FROM pg_stat_statements
-ORDER BY mean_time DESC
-LIMIT 10;
-```
-
-### MySQL
-
-```sql
--- Enable slow query log
-SET GLOBAL slow_query_log = 'ON';
-SET GLOBAL long_query_time = 0.1;  -- 100ms
-
--- Explain query
-EXPLAIN ANALYZE
-SELECT * FROM orders
-WHERE user_id = 123;
-
--- Performance schema
-SELECT * FROM performance_schema.events_statements_summary_by_digest
-ORDER BY SUM_TIMER_WAIT DESC
-LIMIT 10;
-```
-
-## APM Integration
-
-### New Relic
-
-```javascript
-import newrelic from 'newrelic';
-
-// Custom transaction
-newrelic.startBackgroundTransaction('process-orders', async () => {
-  const orders = await getOrders();
-
-  // Custom segment
-  await newrelic.startSegment('validate-orders', true, async () => {
-    return validateOrders(orders);
-  });
-});
-
-// Custom metrics
-newrelic.recordMetric('Custom/OrderValue', orderTotal);
-```
-
-### DataDog APM
-
-```javascript
-import tracer from 'dd-trace';
-tracer.init();
-
-// Custom span
-const span = tracer.startSpan('process.order', {
-  resource: orderId,
-  tags: {
-    'order.total': orderTotal,
-    'user.id': userId,
-  },
-});
-
-try {
-  await processOrder(orderId);
-  span.setTag('status', 'success');
-} catch (err) {
-  span.setTag('error', err);
-} finally {
-  span.finish();
-}
-```
-
-## Quick Reference
-
-| Tool | Language | Type |
-|------|----------|------|
-| clinic.js | Node.js | CPU, Event loop |
-| Chrome DevTools | Node.js | CPU, Memory |
-| cProfile | Python | CPU |
-| py-spy | Python | CPU (sampling) |
-| pprof | Go | CPU, Memory, Goroutines |
-| VisualVM | Java | CPU, Memory, Threads |
-| async-profiler | Java | CPU, Allocation |
-
-| Metric | What to Look For |
-|--------|------------------|
-| CPU time | Hot functions, tight loops |
-| Memory | Large allocations, leaks |
-| I/O wait | Blocking operations |
-| GC time | Excessive collections |
-| Thread count | Thread pool saturation |
-
-| Problem | Symptom |
-|---------|---------|
-| CPU bound | High CPU usage, slow processing |
-| Memory leak | Growing memory, eventual crash |
-| I/O bound | Low CPU, high wait time |
-| Lock contention | Idle threads, poor scaling |