omgkit 2.13.0 → 2.15.0

package/plugin/workflows/ai-ml/monitoring-setup.md

@@ -0,0 +1,227 @@
+---
+description: Set up comprehensive ML model monitoring for drift and performance
+triggers:
+  - manual
+  - ml:monitoring
+agents:
+  - ml-engineer
+  - site-reliability-engineer
+---
+
+# ML Monitoring Setup Workflow
+
+Implement production monitoring for ML models.
+
+## Prerequisites
+- [ ] Model deployed to production
+- [ ] Baseline metrics established
+- [ ] Logging infrastructure available
+
+## Phase 1: Monitoring Strategy
+
+### Step 1.1: Define Monitoring Scope
+```yaml
+agent: ml-engineer
+action: define
+monitoring_types:
+  - Data monitoring: Input feature health
+  - Model monitoring: Prediction quality
+  - Operational monitoring: System health
+  - Business monitoring: Impact metrics
+```
+
+### Step 1.2: Establish Baselines
+```yaml
+agent: ml-engineer
+action: baseline
+metrics:
+  - Feature distributions
+  - Prediction distributions
+  - Model performance (with labels)
+  - Latency percentiles
+window: Training data period
+```
+
+## Phase 2: Data Monitoring
+
+### Step 2.1: Feature Monitoring
+```yaml
+agent: ml-engineer
+action: configure
+checks:
+  - Missing values: Count nulls per feature
+  - Type validation: Expected data types
+  - Range validation: Min/max bounds
+  - Distribution shift: Statistical tests
+methods:
+  - KS test (continuous)
+  - Chi-square (categorical)
+  - PSI (Population Stability Index)
+```
+
+### Step 2.2: Data Quality Alerts
+```yaml
+agent: ml-engineer
+action: configure
+alerts:
+  - schema_violation:
+      condition: Unexpected schema change
+      severity: critical
+  - missing_rate_high:
+      threshold: > 5% missing
+      severity: warning
+  - distribution_drift:
+      threshold: PSI > 0.2
+      severity: warning
+```
+
+## Phase 3: Model Monitoring
+
+### Step 3.1: Prediction Monitoring
+```yaml
+agent: ml-engineer
+action: configure
+metrics:
+  - Prediction distribution
+  - Confidence scores
+  - Prediction counts by class
+  - Output drift detection
+tools:
+  - Evidently
+  - Whylogs
+  - Arize
+  - Custom monitoring
+```
+
+### Step 3.2: Performance Monitoring
+```yaml
+agent: ml-engineer
+action: configure
+with_ground_truth:
+  - Accuracy over time
+  - Precision/Recall trends
+  - AUC trends
+  - Error rate by segment
+delayed_labels:
+  - Label collection pipeline
+  - Performance calculation job
+  - Alerting on degradation
+```
+
+### Step 3.3: Model Drift Detection
+```yaml
+agent: ml-engineer
+action: configure
+drift_types:
+  - Concept drift: Relationship change
+  - Data drift: Feature distribution change
+  - Prediction drift: Output change
+detection:
+  - Statistical tests
+  - Page-Hinkley test
+  - ADWIN algorithm
+  - Custom detectors
+```
+
+## Phase 4: Operational Monitoring
+
+### Step 4.1: System Metrics
+```yaml
+agent: site-reliability-engineer
+action: configure
+metrics:
+  - Request latency (p50, p95, p99)
+  - Throughput (requests/second)
+  - Error rate
+  - GPU/CPU utilization
+  - Memory usage
+  - Queue depth
+```
+
+### Step 4.2: SLI/SLO Definition
+```yaml
+agent: site-reliability-engineer
+action: define
+slis:
+  - Availability: Successful requests / Total requests
+  - Latency: % requests < threshold
+  - Freshness: Time since last successful prediction
+slos:
+  - availability: 99.9%
+  - latency_p99: < 100ms
+  - freshness: < 1 minute
+```
+
+## Phase 5: Alerting and Response
+
+### Step 5.1: Alert Configuration
+```yaml
+agent: site-reliability-engineer
+action: configure
+alert_rules:
+  - model_drift_detected:
+      severity: warning
+      action: notify_ml_team
+  - performance_degradation:
+      severity: critical
+      action: page_on_call
+  - latency_spike:
+      severity: warning
+      action: auto_scale
+  - error_rate_high:
+      severity: critical
+      action: rollback_ready
+```
+
+### Step 5.2: Response Playbooks
+```yaml
+agent: site-reliability-engineer
+action: document
+playbooks:
+  - drift_detected:
+      - Investigate data changes
+      - Check upstream systems
+      - Consider retraining
+  - performance_degraded:
+      - Check recent deployments
+      - Analyze error patterns
+      - Rollback if needed
+  - latency_high:
+      - Check resource usage
+      - Scale horizontally
+      - Optimize batch sizes
+```
+
+## Phase 6: Dashboards
+
+### Step 6.1: Create Dashboards
+```yaml
+agent: site-reliability-engineer
+action: create
+dashboards:
+  - model_health:
+      - Prediction volume
+      - Drift metrics
+      - Performance trends
+  - operational:
+      - Latency distribution
+      - Error rates
+      - Resource usage
+  - business:
+      - Model impact metrics
+      - A/B test results
+```
+
+## Outputs
+- [ ] Monitoring configuration
+- [ ] Alert rules
+- [ ] Dashboards
+- [ ] SLI/SLO definitions
+- [ ] Response playbooks
+
+## Quality Gates
+- All critical metrics monitored
+- Alerts tested and working
+- Dashboards accurate
+- Playbooks documented
+- Team trained on response

package/plugin/workflows/api/api-design.md

@@ -10,7 +10,7 @@ agents:
   - docs-manager
   - code-reviewer
 skills:
-  - api-architecture
+  - backend/api-architecture
 commands:
   - /planning:plan
   - /quality:api-gen

package/plugin/workflows/api/api-testing.md

@@ -8,8 +8,8 @@ agents:
   - tester
   - security-auditor
 skills:
-  - api-architecture
-  - owasp
+  - backend/api-architecture
+  - security/owasp
 commands:
   - /dev:test
   - /quality:security-scan

package/plugin/workflows/content/technical-docs.md

@@ -10,7 +10,7 @@ agents:
   - copywriter
   - code-reviewer
 skills:
-  - writing-plans
+  - methodology/writing-plans
 commands:
   - /planning:doc
   - /dev:review

package/plugin/workflows/database/migration.md

@@ -9,7 +9,7 @@ agents:
   - planner
   - tester
 skills:
-  - database-migration
+  - databases/database-migration
 commands:
   - /planning:plan
   - /dev:test

package/plugin/workflows/database/optimization.md

@@ -8,7 +8,7 @@ agents:
   - database-admin
   - tester
 skills:
-  - database-optimization
+  - databases/database-optimization
 commands:
   - /quality:optimize
   - /dev:test

package/plugin/workflows/database/schema-design.md

@@ -9,9 +9,9 @@ agents:
   - planner
   - tester
 skills:
-  - database-schema-design
-  - postgresql
-  - mongodb
+  - databases/database-schema-design
+  - databases/postgresql
+  - databases/mongodb
 commands:
   - /planning:plan
   - /dev:feature

package/plugin/workflows/development/bug-fix.md

@@ -10,9 +10,9 @@ agents:
   - tester
   - git-manager
 skills:
-  - systematic-debugging
-  - root-cause-tracing
-  - test-driven-development
+  - methodology/systematic-debugging
+  - methodology/root-cause-tracing
+  - methodology/test-driven-development
 commands:
   - /dev:fix
   - /dev:fix-hard

package/plugin/workflows/development/code-review.md

@@ -9,7 +9,8 @@ agents:
   - fullstack-developer
   - git-manager
 skills:
-  - verification-before-completion
+  - methodology/verification-before-completion
+  - security/owasp
 commands:
   - /dev:review
   - /dev:fix

package/plugin/workflows/development/feature.md

@@ -11,9 +11,9 @@ agents:
   - code-reviewer
   - git-manager
 skills:
-  - writing-plans
-  - executing-plans
-  - test-driven-development
+  - methodology/writing-plans
+  - methodology/executing-plans
+  - methodology/test-driven-development
 commands:
   - /planning:plan
   - /dev:feature

package/plugin/workflows/development/refactor.md

@@ -10,8 +10,8 @@ agents:
   - tester
   - git-manager
 skills:
-  - verification-before-completion
-  - test-driven-development
+  - methodology/verification-before-completion
+  - methodology/test-driven-development
 commands:
   - /quality:refactor
   - /dev:test

package/plugin/workflows/event-driven/consumer-groups.md

@@ -0,0 +1,190 @@
+---
+description: Manage Kafka consumer groups for scaling and reliability
+triggers:
+  - manual
+  - kafka:consumers
+agents:
+  - data-engineer
+  - site-reliability-engineer
+---
+
+# Consumer Group Management Workflow
+
+Configure and manage Kafka consumer groups effectively.
+
+## Prerequisites
+- [ ] Kafka cluster available
+- [ ] Topics created
+- [ ] Consumer application ready
+
+## Phase 1: Consumer Group Design
+
+### Step 1.1: Define Consumer Groups
+```yaml
+agent: data-engineer
+action: design
+considerations:
+  - One group per logical consumer
+  - Group ID naming convention
+  - Isolation between environments
+naming: "{app}-{env}-{purpose}"
+examples:
+  - order-service-prod-orders
+  - analytics-prod-clickstream
+```
+
+### Step 1.2: Partition Assignment
+```yaml
+agent: data-engineer
+action: configure
+strategies:
+  - RangeAssignor: Contiguous partitions per consumer
+  - RoundRobinAssignor: Even distribution
+  - StickyAssignor: Minimize rebalances
+  - CooperativeStickyAssignor: Incremental rebalances
+recommendation: CooperativeStickyAssignor
+```
+
+## Phase 2: Consumer Configuration
+
+### Step 2.1: Performance Settings
+```yaml
+agent: data-engineer
+action: configure
+settings:
+  fetch.min.bytes: 1
+  fetch.max.wait.ms: 500
+  max.poll.records: 500
+  max.poll.interval.ms: 300000
+  session.timeout.ms: 45000
+  heartbeat.interval.ms: 15000
+```
+
+### Step 2.2: Offset Management
+```yaml
+agent: data-engineer
+action: configure
+offset_settings:
+  enable.auto.commit: false  # Manual preferred
+  auto.offset.reset: earliest  # or latest
+  isolation.level: read_committed  # For transactions
+commit_strategy:
+  - After processing batch
+  - With idempotency
+  - Track last committed
+```
+
+## Phase 3: Scaling Configuration
+
+### Step 3.1: Consumer Scaling
+```yaml
+agent: site-reliability-engineer
+action: configure
+scaling:
+  min_consumers: 1
+  max_consumers: partition_count
+  scaling_metric: consumer_lag
+  scale_up_threshold: 10000
+  scale_down_threshold: 100
+```
+
+### Step 3.2: Rebalance Optimization
+```yaml
+agent: data-engineer
+action: configure
+rebalance_settings:
+  partition.assignment.strategy: CooperativeStickyAssignor
+  max.poll.interval.ms: 300000  # Allow slow processing
+  session.timeout.ms: 45000
+  heartbeat.interval.ms: 15000
+static_membership:
+  group.instance.id: unique_per_consumer
+  session.timeout.ms: 300000  # Longer for static
+```
+
+## Phase 4: Monitoring Setup
+
+### Step 4.1: Lag Monitoring
+```yaml
+agent: site-reliability-engineer
+action: configure
+metrics:
+  - consumer_lag_per_partition
+  - consumer_lag_per_group
+  - records_consumed_rate
+  - rebalances_per_hour
+tools:
+  - Burrow
+  - Kafka Exporter
+  - Built-in JMX metrics
+```
+
+### Step 4.2: Alerting
+```yaml
+agent: site-reliability-engineer
+action: configure
+alerts:
+  - consumer_lag_critical:
+      threshold: 100000
+      severity: critical
+  - consumer_lag_warning:
+      threshold: 10000
+      severity: warning
+  - rebalance_storm:
+      threshold: 5_per_minute
+      severity: warning
+  - consumer_group_empty:
+      condition: active_members == 0
+      severity: critical
+```
+
+## Phase 5: Operations
+
+### Step 5.1: Offset Management Operations
+```yaml
+agent: site-reliability-engineer
+action: document
+operations:
+  reset_to_earliest:
+    command: kafka-consumer-groups --reset-offsets --to-earliest
+    use_when: Reprocess all events
+  reset_to_timestamp:
+    command: kafka-consumer-groups --reset-offsets --to-datetime
+    use_when: Reprocess from specific time
+  skip_messages:
+    command: kafka-consumer-groups --reset-offsets --shift-by N
+    use_when: Skip poison messages
+```
+
+### Step 5.2: Troubleshooting Runbook
+```yaml
+agent: site-reliability-engineer
+action: document
+scenarios:
+  - lag_increasing:
+      check: Consumer processing speed
+      check: Partition count vs consumers
+      action: Scale consumers or optimize
+  - rebalance_storm:
+      check: Session timeout vs processing time
+      check: Network issues
+      action: Adjust timeouts or use static membership
+  - stuck_consumer:
+      check: max.poll.interval.ms
+      check: Processing exceptions
+      action: Restart or fix code
+```
+
+## Outputs
+- [ ] Consumer group configuration
+- [ ] Scaling configuration
+- [ ] Monitoring dashboards
+- [ ] Alert rules
+- [ ] Operations runbook
+
+## Quality Gates
+- Consumer lag within SLA
+- No rebalance storms
+- Monitoring in place
+- Alerts configured
+- Runbook documented

package/plugin/workflows/event-driven/event-storming.md

@@ -0,0 +1,172 @@
+---
+description: Facilitate event storming sessions to discover domain events and workflows
+triggers:
+  - manual
+  - discovery:events
+agents:
+  - domain-decomposer
+  - architect
+---
+
+# Event Storming Workflow
+
+Discover domain events and model business processes.
+
+## Prerequisites
+- [ ] Domain experts available
+- [ ] Whiteboard/digital collaboration tool
+- [ ] Business process documentation
+
+## Phase 1: Big Picture Event Storming
+
+### Step 1.1: Identify Domain Events
+```yaml
+agent: domain-decomposer
+action: discover
+focus: domain_events
+notation: orange_stickies
+guidelines:
+  - Past tense verbs (OrderPlaced, PaymentReceived)
+  - Business-meaningful names
+  - Temporal ordering
+```
+
+### Step 1.2: Timeline Organization
+```yaml
+agent: domain-decomposer
+action: organize
+steps:
+  - Place events chronologically
+  - Identify parallel flows
+  - Mark pivotal events
+  - Note questions and hotspots
+```
+
+## Phase 2: Process Modeling
+
+### Step 2.1: Add Commands
+```yaml
+agent: domain-decomposer
+action: model
+element: commands
+notation: blue_stickies
+pattern:
+  - Command triggers Event
+  - Name as imperative (PlaceOrder)
+  - Link to triggering event
+```
+
+### Step 2.2: Add Actors
+```yaml
+agent: domain-decomposer
+action: model
+element: actors
+notation: yellow_stickies
+types:
+  - Users (roles)
+  - External systems
+  - Time (scheduled events)
+```
+
+### Step 2.3: Add Policies
+```yaml
+agent: domain-decomposer
+action: model
+element: policies
+notation: purple_stickies
+pattern: "When [Event] then [Command]"
+examples:
+  - "When OrderPlaced then ReserveInventory"
+  - "When PaymentFailed then CancelOrder"
+```
+
+## Phase 3: Aggregate Discovery
+
+### Step 3.1: Group Related Concepts
+```yaml
+agent: domain-decomposer
+action: discover
+element: aggregates
+notation: yellow_boundary
+criteria:
+  - Commands that modify together
+  - Events from same source
+  - Consistency boundary
+```
+
+### Step 3.2: Define Aggregate Boundaries
+```yaml
+agent: architect
+action: define
+boundaries:
+  - Invariants protected
+  - Transaction scope
+  - Entity relationships
+outputs:
+  - aggregate_diagram
+  - invariant_rules
+```
+
+## Phase 4: Bounded Context Identification
+
+### Step 4.1: Identify Context Boundaries
+```yaml
+agent: domain-decomposer
+action: discover
+element: bounded_contexts
+indicators:
+  - Language changes
+  - Different stakeholders
+  - Different models for same concept
+  - Natural team boundaries
+```
+
+### Step 4.2: Map Context Relationships
+```yaml
+agent: architect
+action: map
+relationships:
+  - Upstream/Downstream
+  - Conformist/Anti-corruption Layer
+  - Partnership
+  - Customer/Supplier
+```
+
+## Phase 5: Documentation
+
+### Step 5.1: Create Event Catalog
+```yaml
+agent: domain-decomposer
+action: document
+outputs:
+  - event_catalog:
+      - name
+      - description
+      - payload_schema
+      - producers
+      - consumers
+```
+
+### Step 5.2: Create Process Diagrams
+```yaml
+agent: architect
+action: generate
+outputs:
+  - process_flow_diagrams
+  - aggregate_diagrams
+  - context_maps
+```
+
+## Outputs
+- [ ] Domain event catalog
+- [ ] Command list
+- [ ] Aggregate definitions
+- [ ] Bounded context map
+- [ ] Process flow diagrams
+
+## Quality Gates
+- All events have clear ownership
+- Aggregates have defined invariants
+- Context boundaries are explicit
+- Ubiquitous language documented
+- Stakeholders validated model