sdtk-ops-kit 0.2.0

package/assets/toolkit/toolkit/skills/ops-incident/SKILL.md

@@ -0,0 +1,113 @@
+<!-- Based on agency-agents by AgentLand Contributors (MIT License, 2025). Adapted for SDTK-OPS. -->
+---
+name: ops-incident
+description: Incident response and management. Use when a production incident occurs or when establishing incident response procedures -- covers severity classification, response coordination, post-mortem facilitation, and on-call design.
+---
+
+# Ops Incident
+
+## Overview
+
+Incident response must turn chaos into a structured workflow: classify severity, assign roles, build a timeline, test one hypothesis path at a time, stabilize the system, and capture systemic follow-up before memory fades.
+
+## The Iron Law
+
+```
+NO INCIDENT RESOLVED WITHOUT A TIMELINE, IMPACT ASSESSMENT, AND ACTION ITEMS WITHIN 48 HOURS
+```
+
+## Severity Classification Matrix
+
+| Level | Name | Criteria | Response Time | Update Cadence | Escalation |
+|-------|------|----------|---------------|----------------|------------|
+| SEV1 | Critical | full service outage, data loss risk, security breach | under 5 min | every 15 min | leadership immediately |
+| SEV2 | Major | degraded service for more than 25% of users, key feature down | under 15 min | every 30 min | engineering manager within 15 min |
+| SEV3 | Moderate | minor feature broken, workaround available | under 1 hour | every 2 hours | team lead next standup |
+| SEV4 | Low | cosmetic issue, no user impact, tech debt trigger | next business day | daily | backlog triage |
+
+Escalate severity when:
+- impact scope doubles
+- no root cause is identified after 30 minutes for SEV1 or 2 hours for SEV2
+- a paying customer is blocked, minimum SEV2
+- any data integrity concern appears, immediate SEV1
+
+## The Four-Phase Process
+
+### 1. Detection And Declaration
+
+- acknowledge the page or signal
+- classify severity using the matrix
+- declare the incident and open a timeline immediately
+- state current impact, suspected scope, and next update time
+
+### 2. Structured Response
+
+Assign roles:
+- **Incident Commander**
+  - owns timeline, severity, decisions, and cadence
+- **Technical Lead**
+  - drives diagnosis and remediation with `ops-debug`
+- **Scribe**
+  - records timestamps, evidence, commands, and decisions
+- **Communications Lead**
+  - sends stakeholder updates on schedule
+
+Response rules:
+- timebox each hypothesis path to 15 minutes before re-evaluating
+- fix the bleeding first, then optimize
+- keep one source of truth for status and impact
+
+### 3. Resolution And Stabilization
+
+- apply the lowest-risk effective mitigation
+- verify recovery through metrics, not by visual guesswork
+- monitor for 15 to 30 minutes after recovery
+- do not close the incident until the service is stable
+
+### 4. Post-Mortem
+
+- schedule the review within 48 hours
+- document impact, timeline, root cause, contributing factors, and action items
+- convert learning into runbook, alert, test, or design changes
+
+## <HARD-GATE>
+
+Incident review and post-mortem must stay blameless:
+- say "the system allowed this failure mode"
+- do not frame the review as "which person caused it"
+- protect psychological safety so the real causes are visible
+
+## Operational Metrics
+
+| Metric | Target |
+|--------|--------|
+| MTTD | under 5 minutes |
+| MTTR for SEV1 | under 30 minutes |
+| Post-mortems within 48 hours | 100% |
+| Action item completion | 90% |
+| Repeat incidents | 0 for the same unresolved cause |
+
+## Reference Files
+
+Use:
+- `./references/runbook-template.md`
+- `./references/postmortem-template.md`
+- `./references/communication-templates.md`
+
+## Common Mistakes
+
+| Mistake | Why it fails |
+|---------|--------------|
+| Start fixing before declaring severity and roles | Communication and triage fragment immediately |
+| Chase multiple hypotheses at once | The team loses evidence and ownership |
+| Close the incident when errors drop briefly | Latent instability returns and the timeline becomes unclear |
+| Write a blame document instead of a post-mortem | Systemic fixes are replaced by defensiveness |
+| Skip action item ownership and due dates | The same incident class repeats |
+
+## Execution Handoff
+
+During incident response:
+- use `ops-debug` for diagnosis
+- use `ops-monitor` to confirm recovery against live signals
+- use `ops-verify` before declaring the system stable
+

package/assets/toolkit/toolkit/skills/ops-incident/references/communication-templates.md

@@ -0,0 +1,34 @@
+<!-- Based on agency-agents by AgentLand Contributors (MIT License, 2025). Adapted for SDTK-OPS. -->
+
+# Communication Templates
+
+```markdown
+# SEV1 Initial Notification
+**Subject**: [SEV1] [Service Name] - [Brief Impact Description]
+
+**Current Status**: We are investigating an issue affecting [service or feature].
+**Impact**: [X]% of users are experiencing [symptom].
+**Next Update**: In 15 minutes or sooner if the situation changes materially.
+
+---
+
+# SEV1 Status Update
+**Subject**: [SEV1 UPDATE] [Service Name] - [Current State]
+
+**Status**: [Investigating / Identified / Mitigating / Resolved]
+**Current Understanding**: [what we know about the cause]
+**Actions Taken**: [what has been done so far]
+**Next Steps**: [what happens next]
+**Next Update**: In 15 minutes.
+
+---
+
+# Incident Resolved
+**Subject**: [RESOLVED] [Service Name] - [Brief Description]
+
+**Resolution**: [what fixed the issue]
+**Duration**: [start time] to [end time] ([total duration])
+**Impact Summary**: [who was affected and how]
+**Follow-up**: Post-mortem scheduled for [date]. Action items will be tracked in [link].
+```
+

package/assets/toolkit/toolkit/skills/ops-incident/references/postmortem-template.md

@@ -0,0 +1,69 @@
+<!-- Based on agency-agents by AgentLand Contributors (MIT License, 2025). Adapted for SDTK-OPS. -->
+
+# Postmortem Template
+
+```markdown
+# Post-Mortem: [Incident Title]
+
+**Date**: YYYY-MM-DD
+**Severity**: SEV[1-4]
+**Duration**: [start time] to [end time] ([total duration])
+**Author**: [name]
+**Status**: [Draft / Review / Final]
+
+## Executive Summary
+[2-3 sentences on what happened, who was affected, and how it was resolved]
+
+## Impact
+- **Users affected**: [number or percentage]
+- **Revenue impact**: [estimated or N/A]
+- **SLO budget consumed**: [X% of monthly error budget]
+- **Support tickets created**: [count]
+
+## Timeline (UTC)
+| Time | Event |
+|------|-------|
+| 14:02 | Monitoring alert fires |
+| 14:05 | On-call engineer acknowledges page |
+| 14:08 | Incident declared and roles assigned |
+| 14:12 | Root-cause hypothesis recorded |
+| 14:18 | Mitigation started |
+| 14:23 | Metrics begin returning to baseline |
+| 14:30 | Incident resolved |
+| 14:45 | All-clear communicated |
+
+## Root Cause Analysis
+### What Happened
+[Detailed technical explanation of the failure chain]
+
+### Contributing Factors
+1. **Immediate cause**: [the direct trigger]
+2. **Underlying cause**: [why the trigger was possible]
+3. **Systemic cause**: [what process or design gap allowed it]
+
+### 5 Whys
+1. Why did the service fail? [answer]
+2. Why did that happen? [answer]
+3. Why was that possible? [answer]
+4. Why was the guardrail missing? [answer]
+5. Why did the system permit the failure mode? [root issue]
+
+## What Went Well
+- [things that helped detection or response]
+- [tools or practices that reduced impact]
+
+## What Went Poorly
+- [gaps that slowed detection or resolution]
+- [runbooks, alerts, or ownership issues]
+
+## Action Items
+| ID | Action | Owner | Priority | Due Date | Status |
+|----|--------|-------|----------|----------|--------|
+| 1 | [action] | [owner] | P1 | YYYY-MM-DD | Not Started |
+| 2 | [action] | [owner] | P1 | YYYY-MM-DD | Not Started |
+| 3 | [action] | [owner] | P2 | YYYY-MM-DD | Not Started |
+
+## Lessons Learned
+[Key takeaways that should change design, monitoring, or procedure]
+```
+

package/assets/toolkit/toolkit/skills/ops-incident/references/runbook-template.md

@@ -0,0 +1,69 @@
+<!-- Based on agency-agents by AgentLand Contributors (MIT License, 2025). Adapted for SDTK-OPS. -->
+
+# Incident Runbook Template
+
+```markdown
+# Runbook: [Service Or Failure Scenario]
+
+## Quick Reference
+- **Service**: [service name and repo link]
+- **Owner Team**: [team name and contact channel]
+- **On-Call Rotation**: [schedule link or contact path]
+- **Dashboards**: [monitoring links]
+- **Last Tested**: [date of drill or last validation]
+
+## Detection
+- **Alert**: [alert name and system]
+- **Symptoms**: [what users and metrics look like]
+- **False Positive Check**: [how to confirm it is real]
+
+## Diagnosis
+1. Check service health: `[command]`
+2. Review error rate and latency dashboards: `[links]`
+3. Check recent deployments or config changes: `[command or link]`
+4. Review dependency health: `[links or commands]`
+
+## Remediation
+
+### Option A: Rollback
+```bash
+# Identify the last known good revision
+[rollback-history-command]
+
+# Roll back to the prior revision
+[rollback-command]
+
+# Verify the rollback finished
+[status-command]
+```
+
+### Option B: Restart
+```bash
+# Use a rolling restart where possible
+[restart-command]
+
+# Monitor progress
+[status-command]
+```
+
+### Option C: Scale Up
+```bash
+# Increase capacity if the issue is load-related
+[scale-command]
+
+# Verify capacity and error rate
+[verify-command]
+```
+
+## Verification
+- [ ] Error rate returned to baseline
+- [ ] Latency is within SLO
+- [ ] No new alerts firing for 10 minutes
+- [ ] User-facing functionality manually verified
+
+## Communication
+- Internal: [incident channel or update path]
+- External: [status page or customer update path]
+- Follow-up: Create post-mortem within 48 hours
+```
+

package/assets/toolkit/toolkit/skills/ops-infra-plan/SKILL.md

@@ -0,0 +1,123 @@
+<!-- Based on agency-agents by AgentLand Contributors (MIT License, 2025) and gstack by Garry Tan (MIT License, 2026). Adapted for SDTK-OPS. -->
+---
+name: ops-infra-plan
+description: Infrastructure architecture planning. Use when designing cloud resources, networking, security groups, IAM policies, or IaC modules -- produces a reviewable infrastructure plan before provisioning.
+---
+
+# Ops Infra Plan
+
+## Overview
+
+Provisioning without a reviewed design creates hidden coupling, rollback gaps, and security drift. Write the infrastructure plan first, make the dependencies explicit, then provision only what the approved plan covers.
+
+## The Iron Law
+
+```
+NO INFRASTRUCTURE PROVISIONING WITHOUT A REVIEWED PLAN
+```
+
+## When to Use
+
+Use for:
+- new service or environment setup
+- network topology changes
+- security group, firewall, or IAM changes
+- database, cache, queue, or storage provisioning
+- scaling architecture updates
+- disaster recovery design
+
+## Required Plan Sections
+
+Every infrastructure plan must include:
+- **Architecture Diagram (ASCII)**
+  - show traffic entry, workload tier, stateful dependencies, and operator control points
+- **Resource Inventory**
+  - resource name, purpose, environment, owner, critical dependency
+- **Networking Design**
+  - ingress and egress paths, DNS, load balancing, private connectivity, access boundaries
+- **Security Design**
+  - identities, secrets flow, encryption posture, least-privilege access, audit boundaries
+- **Capacity Estimates**
+  - CPU, memory, storage, throughput, and auto-scaling limits
+- **Multi-Environment Strategy**
+  - dev, staging, and production separation with configuration and data isolation notes
+- **Rollback Plan**
+  - rollback trigger, exact rollback action, verification evidence, and decision owner
+- **Dependency Order**
+  - what must exist before provisioning the next layer
+
+## ASCII Diagram Pattern
+
+Use an explicit diagram even for small changes:
+
+```text
+Internet
+  |
+DNS -> Load Balancer
+          |
+       App Service
+          |
+   Database / Cache / Queue
+          |
+  Backups / Logs / Metrics
+```
+
+If a system boundary or dependency matters during rollout, it belongs in the diagram.
+
+## Resource Inventory Template
+
+Use a table like this:
+
+| Resource | Purpose | Environment | Owner | Depends On |
+|----------|---------|-------------|-------|------------|
+| app-lb | Public traffic entry | production | ops | DNS, TLS cert |
+| app-service | Stateless workload | production | ops | image, secrets, DB |
+| app-db | Primary relational store | production | ops | subnet, backup policy |
+
+## IaC Patterns
+
+Prefer reusable modules and explicit inputs over hand-built resources. Common patterns:
+- network foundation first: VPC or equivalent, subnets, routes, security boundaries
+- stateless compute behind a load balancer with health checks and auto-scaling
+- stateful services with backup, encryption, and maintenance windows defined up front
+- alarms and dashboards defined in the same plan as the resource they guard
+- environment-specific values separated from reusable module logic
+
+Use `./references/iac-patterns.md` for concrete Terraform-style examples adapted from the source material.
+
+## Review Checklist
+
+Review the plan for:
+- naming consistency across resources, modules, and environments
+- security groups or firewall rules scoped to least privilege
+- encryption at rest and in transit
+- backup schedule and restore ownership
+- observability coverage for each critical resource
+- cost estimate, scaling bounds, and obvious waste
+- rollback path that works after partial apply
+
+## <HARD-GATE>
+
+Do not provision until all four are true:
+- written plan exists and matches the intended scope
+- resource inventory is complete
+- rollback procedure exists for the first failed step and for partial apply
+- security review covers network access, identities, secrets, and encryption
+
+## Common Mistakes
+
+| Mistake | Why it fails |
+|---------|--------------|
+| Provision first, document later | The live system becomes the only source of truth |
+| Copy production config into dev | Cost, permissions, and blast radius drift immediately |
+| Hardcode IPs, hostnames, or secrets | Reuse fails and rotation becomes dangerous |
+| Plan resources without capacity notes | Scaling and cost failures surface during rollout |
+| Skip rollback because IaC is "reversible" | Partial apply and data changes still need explicit recovery |
+
+## Execution Handoff
+
+After the plan is approved:
+- provision in dependency order
+- verify each applied layer before moving forward
+- use `ops-verify` before marking any provisioning step complete
+

package/assets/toolkit/toolkit/skills/ops-infra-plan/references/iac-patterns.md

@@ -0,0 +1,141 @@
+<!-- Based on agency-agents by AgentLand Contributors (MIT License, 2025) and gstack by Garry Tan (MIT License, 2026). Adapted for SDTK-OPS. -->
+
+# IaC Patterns
+
+## Overview
+
+These examples capture common infrastructure-as-code patterns for a networked service with autoscaling, load balancing, and operational safeguards. Treat provider-specific syntax as illustrative. The planning pattern is portable even when the resource names differ.
+
+## Network And Compute Pattern
+
+```hcl
+module "network" {
+  source = "./modules/network"
+
+  name               = "app-prod"
+  cidr_block         = "10.20.0.0/16"
+  public_subnets     = ["10.20.1.0/24", "10.20.2.0/24"]
+  private_subnets    = ["10.20.11.0/24", "10.20.12.0/24"]
+  enable_nat_gateway = true
+}
+
+resource "aws_launch_template" "app" {
+  name_prefix   = "app-prod-"
+  image_id      = var.image_id
+  instance_type = var.instance_type
+
+  user_data = base64encode(templatefile("${path.module}/bootstrap.sh", {
+    env = "production"
+  }))
+}
+
+resource "aws_autoscaling_group" "app" {
+  name                = "app-prod"
+  desired_capacity    = 3
+  min_size            = 3
+  max_size            = 6
+  vpc_zone_identifier = module.network.private_subnet_ids
+  target_group_arns   = [aws_lb_target_group.app.arn]
+  health_check_type   = "ELB"
+  health_check_grace_period = 300
+
+  launch_template {
+    id      = aws_launch_template.app.id
+    version = "$Latest"
+  }
+}
+```
+
+Pattern notes:
+- keep shared network concerns in a module
+- treat image version and instance size as explicit inputs
+- set min and max capacity in the same change that introduces autoscaling
+- attach compute to health-checked traffic entry, not direct public access
+
+## Load Balancer Pattern
+
+```hcl
+resource "aws_lb" "app" {
+  name               = "app-prod"
+  internal           = false
+  load_balancer_type = "application"
+  subnets            = module.network.public_subnet_ids
+  security_groups    = [aws_security_group.lb.id]
+}
+
+resource "aws_lb_target_group" "app" {
+  name     = "app-prod"
+  port     = 8080
+  protocol = "HTTP"
+  vpc_id   = module.network.vpc_id
+
+  health_check {
+    path                = "/health"
+    healthy_threshold   = 2
+    unhealthy_threshold = 3
+    interval            = 30
+    timeout             = 5
+  }
+}
+```
+
+Pattern notes:
+- define health checks next to the traffic entry resource
+- use explicit thresholds instead of provider defaults
+- keep listener, target, and security policy changes in the same reviewed plan
+
+## Database Pattern
+
+```hcl
+resource "aws_db_subnet_group" "app" {
+  name       = "app-prod"
+  subnet_ids = module.network.private_subnet_ids
+}
+
+resource "aws_db_instance" "app" {
+  identifier              = "app-prod"
+  engine                  = "postgres"
+  instance_class          = "db.t3.medium"
+  allocated_storage       = 100
+  storage_encrypted       = true
+  backup_retention_period = 7
+  multi_az                = true
+  db_subnet_group_name    = aws_db_subnet_group.app.name
+  vpc_security_group_ids  = [aws_security_group.db.id]
+}
+```
+
+Pattern notes:
+- keep stateful resources private by default
+- define backup retention and encryption in the first version
+- make high availability an explicit decision, not an accident
+
+## Alarm Pattern
+
+```hcl
+resource "aws_cloudwatch_metric_alarm" "cpu_high" {
+  alarm_name          = "app-prod-cpu-high"
+  comparison_operator = "GreaterThanThreshold"
+  evaluation_periods  = 2
+  metric_name         = "CPUUtilization"
+  namespace           = "AWS/EC2"
+  period              = 300
+  statistic           = "Average"
+  threshold           = 75
+  alarm_description   = "CPU high for two consecutive periods"
+}
+```
+
+Pattern notes:
+- define an alarm when you define the critical resource
+- use thresholds the team can explain and review
+- connect alerting design to rollback and incident response decisions
+
+## Planning Checklist
+
+Before applying an IaC change, confirm:
+- all provider-specific examples have a cloud-agnostic rationale
+- network, identity, and secret dependencies are listed
+- rollback covers both failed create and partial update
+- health checks, alarms, and backups are defined with the resource
+

package/assets/toolkit/toolkit/skills/ops-monitor/SKILL.md

@@ -0,0 +1,110 @@
+<!-- Based on agency-agents by AgentLand Contributors (MIT License, 2025). Adapted for SDTK-OPS. -->
+---
+name: ops-monitor
+description: Observability and monitoring setup. Use when designing monitoring, defining SLOs/SLIs, configuring alerts, or building dashboards -- covers the three pillars of observability and Golden Signals framework.
+---
+
+# Ops Monitor
+
+## Overview
+
+Monitoring exists to detect user-impacting problems early, explain what is happening, and guide a safe response. A service is not production-ready until metrics, logs, traces, alerts, and response expectations are all defined together.
+
+## The Iron Law
+
+```
+NO SERVICE IN PRODUCTION WITHOUT MONITORING AND ALERTING
+```
+
+## When to Use
+
+Use for:
+- new production services
+- dashboard and alert design
+- SLO and SLI definition
+- noisy alert cleanup
+- monitoring gaps discovered during incidents
+
+## The Three Pillars
+
+| Pillar | Purpose | Key Question |
+|--------|---------|--------------|
+| Metrics | trends, alerting, SLO tracking | Is the system healthy over time? |
+| Logs | event details, audit trail, debugging context | What happened at a specific time? |
+| Traces | request flow across services | Where is the latency or failure path? |
+
+## Golden Signals
+
+| Signal | What To Measure | Example Metrics |
+|--------|-----------------|-----------------|
+| Latency | request duration for successful and failed work | `http_request_duration_seconds_bucket`, `grpc_server_handling_seconds_bucket` |
+| Traffic | request volume or work rate | `http_requests_total`, `rpc_requests_total`, queue throughput |
+| Errors | failure rate by type and severity | `http_requests_total{status=~"5.."}`, timeout counters, business error counters |
+| Saturation | how close the system is to a limit | CPU, memory, queue depth, disk, connection pool usage |
+
+## SLO And SLI Framework
+
+Define:
+- **SLI**
+  - what you measure for user-visible behavior
+  - examples: availability, latency, correctness
+- **SLO**
+  - the target and window for that SLI
+  - examples: `99.95% availability over 30d`, `99% of requests under 300ms over 30d`
+
+Use `./references/slo-templates.md` for YAML templates that combine availability, latency, correctness, burn-rate alerts, and error-budget policy.
+
+## Error Budget Policy
+
+Use a simple decision policy:
+- budget remaining above 50%: normal development
+- budget remaining 25% to 50%: reliability review before risky changes
+- budget remaining below 25%: prioritize reliability work
+- budget exhausted: freeze non-critical deploys until reviewed
+
+## Alert Design
+
+Alerts should:
+- page on symptoms, not on every possible internal cause
+- link to a runbook for first response
+- define at least two severities where appropriate
+- be tuned so false positives stay below 15%
+- point to the dashboard or query needed for triage
+
+An alert that does not change operator behavior should be removed or rewritten.
+
+## Dashboard Design
+
+| Dashboard | Audience | Required Content |
+|-----------|----------|------------------|
+| Executive | incident leads, managers | SLO status, error budget, active incidents, high-level trends |
+| Service | on-call engineers | Golden Signals, dependency health, deploy markers, recent alerts |
+| Debug | responders and service owners | detailed metrics, logs, traces, and breakdowns by instance, region, or endpoint |
+
+## <HARD-GATE>
+
+Before a service is treated as production-ready, all must be true:
+- SLIs are defined for user-visible behavior
+- SLOs are set with explicit targets and windows
+- burn-rate alerts are configured
+- a Golden Signals dashboard exists
+- the on-call team knows which alerts fire and what they mean
+
+## Common Mistakes
+
+| Mistake | Why it fails |
+|---------|--------------|
+| Alert on every low-level cause | Operators get noise instead of signal |
+| Track uptime only | User-visible latency and correctness failures are missed |
+| Dashboard with no SLO context | Teams see metrics but not reliability risk |
+| No runbook links in alerts | Response time expands during real incidents |
+| Collect logs and traces without correlation keys | Cross-system debugging stays slow and manual |
+
+## Execution Handoff
+
+After monitoring is defined:
+- validate alerts with a controlled test or drill
+- route incident handling through `ops-incident`
+- use `ops-debug` for root-cause investigation
+- use `ops-verify` before claiming monitoring coverage is complete
+