@patricio0312rev/skillset 0.1.0

package/templates/architecture/scalability-playbook/ SKILL.md

@@ -0,0 +1,279 @@
+---
+name: scalability-playbook
+description: Identifies performance bottlenecks and provides ordered scaling strategies with triggers, phased plans, and cost implications. Use for "scalability planning", "performance bottlenecks", "capacity planning", or "growth strategy".
+---
+
+# Scalability Playbook
+
+Systematic approach to identifying and resolving scalability bottlenecks.
+
+## Bottleneck Analysis
+
+### Current System Profile
+
+```
+Traffic: 1,000 req/min
+Users: 10,000 active
+Data: 100GB database
+Response time: p95 = 500ms
+```
+
+### Identified Bottlenecks
+
+#### 1. Database Queries
+
+**Symptom:** Slow page loads (2-3s)
+**Measurement:** Query time p95 = 800ms
+**Impact:** HIGH - affects all reads
+**Trigger:** When p95 >500ms
+
+#### 2. Single Server
+
+**Symptom:** High CPU (>80%)
+**Measurement:** Load average >4
+**Impact:** MEDIUM - intermittent slowdowns
+**Trigger:** When CPU >70%
+
+#### 3. No Caching
+
+**Symptom:** Repeated DB queries
+**Measurement:** Cache hit rate = 0%
+**Impact:** MEDIUM - unnecessary load
+**Trigger:** When query volume >10k/min
+
+## Scaling Strategies (Ordered)
+
+### Level 1: Quick Wins (Days)
+
+#### 1.1 Add Database Indexes
+
+**Problem:** Slow queries
+**Solution:**
+
+```sql
+CREATE INDEX idx_users_email ON users(email);
+CREATE INDEX idx_orders_user_created ON orders(user_id, created_at);
+```
+
+**Expected Impact:** 80% faster queries
+**Cost:** $0
+**Effort:** 1 day
+
+#### 1.2 Enable Query Caching
+
+**Problem:** Repeated queries
+**Solution:** Redis cache layer
+
+```typescript
+const cached = await redis.get(`user:${userId}`);
+if (cached) return JSON.parse(cached);
+
+const user = await db.users.findById(userId);
+await redis.setex(`user:${userId}`, 3600, JSON.stringify(user));
+```
+
+**Expected Impact:** 60% reduction in DB load
+**Cost:** $50/month
+**Effort:** 2 days
+
+### Level 2: Horizontal Scaling (Weeks)
+
+#### 2.1 Add Read Replicas
+
+**Problem:** Read-heavy workload
+**Solution:** Route reads to replicas
+
+```
+Write Load: Primary DB
+Read Load: 3x Read Replicas
+```
+
+**Expected Impact:** 3x read capacity
+**Cost:** $300/month
+**Effort:** 1 week
+
+#### 2.2 Load Balancer + Multiple Servers
+
+**Problem:** Single point of failure
+**Solution:**
+
+```
+ALB
+ ├── Server 1
+ ├── Server 2
+ └── Server 3
+```
+
+**Expected Impact:** 3x throughput
+**Cost:** $400/month
+**Effort:** 1 week
+
+### Level 3: Architecture Changes (Months)
+
+#### 3.1 CDN for Static Assets
+
+**Problem:** Slow asset delivery
+**Solution:** CloudFront CDN
+**Expected Impact:** 90% faster asset loads
+**Cost:** $100/month
+**Effort:** 1 week
+
+#### 3.2 Async Processing
+
+**Problem:** Slow sync operations
+**Solution:** Background job queues
+
+```typescript
+// Before: Sync
+await sendEmail(user);
+await processPayment(order);
+await updateAnalytics(event);
+return response; // Waits 5+ seconds
+
+// After: Async
+await queue.add("send-email", { userId });
+await queue.add("process-payment", { orderId });
+await queue.add("update-analytics", { event });
+return response; // Returns immediately
+```
+
+**Expected Impact:** 80% faster responses
+**Cost:** $50/month (SQS)
+**Effort:** 2 weeks
+
+### Level 4: Data Layer Optimization (Months)
+
+#### 4.1 Database Sharding
+
+**Problem:** Single DB too large
+**Solution:** Shard by user_id
+
+```
+Shard 1: user_id 0-24999
+Shard 2: user_id 25000-49999
+Shard 3: user_id 50000-74999
+Shard 4: user_id 75000-99999
+```
+
+**Expected Impact:** 4x capacity
+**Cost:** $1,200/month
+**Effort:** 2 months
+
+#### 4.2 Event-Driven Architecture
+
+**Problem:** Tight coupling, cascading failures
+**Solution:** Message broker (Kafka)
+
+```
+Service A → Kafka → Service B
+          ↘        ↗ Service C
+```
+
+**Expected Impact:** Better isolation, resilience
+**Cost:** $500/month
+**Effort:** 3 months
+
+## Scaling Triggers
+
+```markdown
+| Metric           | Current | Warning | Critical | Action                  |
+| ---------------- | ------- | ------- | -------- | ----------------------- |
+| CPU              | 40%     | 70%     | 85%      | Add servers             |
+| Memory           | 50%     | 75%     | 90%      | Upgrade instances       |
+| DB Connections   | 20      | 40      | 50       | Add read replicas       |
+| Query Time (p95) | 200ms   | 500ms   | 1000ms   | Add indexes             |
+| Queue Depth      | 100     | 1000    | 5000     | Add workers             |
+| Error Rate       | 0.1%    | 1%      | 5%       | Investigate immediately |
+```
+
+## Phased Scaling Plan
+
+### Phase 1: Current → 10x (0-3 months)
+
+**Target:** 10,000 req/min, 100K users
+
+**Actions:**
+
+1. Add database indexes (Week 1)
+2. Implement Redis caching (Week 2)
+3. Add 3x read replicas (Week 4)
+4. Horizontal scale app servers (Week 6)
+5. CDN for static assets (Week 8)
+
+**Cost:** $500 → $1,000/month
+
+### Phase 2: 10x → 100x (3-12 months)
+
+**Target:** 100,000 req/min, 1M users
+
+**Actions:**
+
+1. Database sharding (Month 4-6)
+2. Multi-region deployment (Month 6-8)
+3. Microservices extraction (Month 8-12)
+4. Event-driven architecture (Month 10-12)
+
+**Cost:** $1,000 → $10,000/month
+
+### Phase 3: 100x → 1000x (12-24 months)
+
+**Target:** 1M req/min, 10M users
+
+**Actions:**
+
+1. Global CDN (Month 13)
+2. Advanced caching (L1/L2) (Month 14-15)
+3. Custom DB solutions (Month 16-18)
+4. Edge computing (Month 18-20)
+
+**Cost:** $10,000 → $100,000/month
+
+## Load Testing Plan
+
+```bash
+# Current baseline
+hey -n 10000 -c 100 https://api.example.com/users
+
+# Target 10x
+hey -n 100000 -c 1000 https://api.example.com/users
+
+# Measure:
+# - Requests/sec
+# - p50, p95, p99 latency
+# - Error rate
+# - Resource utilization
+```
+
+## Cost-Benefit Analysis
+
+```markdown
+| Strategy      | Cost/Month | Expected Impact    | ROI | Priority |
+| ------------- | ---------- | ------------------ | --- | -------- |
+| DB Indexes    | $0         | 80% faster queries | ∞   | HIGH     |
+| Redis Cache   | $50        | 60% less DB load   | 12x | HIGH     |
+| Read Replicas | $300       | 3x capacity        | 10x | MEDIUM   |
+| Load Balancer | $400       | 3x throughput      | 7x  | MEDIUM   |
+| DB Sharding   | $1,200     | 4x capacity        | 3x  | LOW      |
+```
+
+## Best Practices
+
+1. **Measure first**: Don't optimize blindly
+2. **Low-hanging fruit**: Start with easy wins
+3. **Load test**: Validate before production
+4. **Monitor continuously**: Set up alerts
+5. **Plan ahead**: Scale before hitting limits
+6. **Cost-conscious**: ROI-driven decisions
+7. **Incremental**: Small, safe changes
+
+## Output Checklist
+
+- [ ] Current system profile
+- [ ] Bottlenecks identified and measured
+- [ ] Scaling strategies ordered by effort
+- [ ] Triggers defined for each action
+- [ ] Phased plan (1x → 10x → 100x)
+- [ ] Cost estimates per phase
+- [ ] Load testing plan
+- [ ] Monitoring dashboard
+- [ ] Rollback procedures

package/templates/architecture/system-design-generator/ SKILL.md

@@ -0,0 +1,339 @@
+---
+name: system-design-generator
+description: Produces comprehensive system architecture plans for features and products including component breakdown, data flow diagrams, system boundaries, API contracts, and scaling considerations. Use for "system design", "architecture planning", "feature design", or "technical specs".
+---
+
+# System Design Generator
+
+Create comprehensive system architecture plans from requirements.
+
+## System Design Document Template
+
+```markdown
+# System Design: [Feature/Product Name]
+
+## Overview
+
+Brief description of what we're building and why.
+
+## Requirements
+
+### Functional
+
+- User can upload videos (max 1GB)
+- System processes video within 5 minutes
+- User receives notification when complete
+
+### Non-Functional
+
+- Handle 1000 uploads/day
+- 99.9% uptime
+- Process videos in <5 minutes (p95)
+- Cost: <$0.50 per video
+
+## High-Level Architecture
+```
+
+┌─────────┐ ┌──────────┐ ┌─────────────┐
+│ Client │─────▶│ API │─────▶│ Upload │
+│ │ │ Gateway │ │ Service │
+└─────────┘ └──────────┘ └─────────────┘
+│
+▼
+┌─────────────┐
+│ Storage │
+│ (S3) │
+└─────────────┘
+│
+▼
+┌─────────────┐
+│ Processing │◀─┐
+│ Queue │ │
+└─────────────┘ │
+│ │
+▼ │
+┌─────────────┐ │
+│ Processor │─┘
+│ Workers │
+└─────────────┘
+│
+▼
+┌─────────────┐
+│Notification │
+│ Service │
+└─────────────┘
+
+```
+
+## Components
+
+### 1. API Gateway
+**Responsibilities:**
+- Authentication
+- Rate limiting
+- Request routing
+
+**Technology:** Kong/AWS API Gateway
+**Scaling:** Auto-scale based on requests/sec
+
+### 2. Upload Service
+**Responsibilities:**
+- Generate pre-signed S3 URLs
+- Validate file metadata
+- Enqueue processing jobs
+
+**API:**
+```
+
+POST /uploads
+Request: { filename, size, content_type }
+Response: { upload_url, upload_id }
+
+```
+
+**Technology:** Node.js + Express
+**Scaling:** Horizontal (stateless)
+
+### 3. Storage (S3)
+**Responsibilities:**
+- Store raw videos
+- Store processed outputs
+- Serve content via CDN
+
+**Structure:**
+```
+
+/uploads/{user_id}/{upload_id}/original.mp4
+/processed/{user_id}/{upload_id}/output.mp4
+
+````
+
+### 4. Processing Queue
+**Responsibilities:**
+- Buffer processing jobs
+- Ensure at-least-once delivery
+- DLQ for failed jobs
+
+**Technology:** AWS SQS
+**Configuration:**
+- Visibility timeout: 15 minutes
+- DLQ after 3 retries
+
+### 5. Processor Workers
+**Responsibilities:**
+- Transcode videos
+- Generate thumbnails
+- Update database
+
+**Technology:** Python + FFmpeg
+**Scaling:** Auto-scale on queue depth
+
+## Data Flow
+
+### Upload Flow
+1. Client requests upload URL from Upload Service
+2. Upload Service generates pre-signed S3 URL
+3. Client uploads directly to S3
+4. Client notifies Upload Service of completion
+5. Upload Service enqueues processing job
+6. Returns upload_id to client
+
+### Processing Flow
+1. Worker polls queue for jobs
+2. Downloads video from S3
+3. Processes video (transcode, thumbnail)
+4. Uploads results to S3
+5. Updates database status
+6. Sends notification
+7. Deletes message from queue
+
+## Data Model
+
+```typescript
+interface Upload {
+  id: string;
+  user_id: string;
+  filename: string;
+  size: number;
+  status: 'pending' | 'processing' | 'complete' | 'failed';
+  original_url: string;
+  processed_url?: string;
+  created_at: Date;
+  processed_at?: Date;
+}
+
+interface ProcessingJob {
+  upload_id: string;
+  attempts: number;
+  error?: string;
+}
+````
+
+## API Contract
+
+### Upload Endpoints
+
+```
+POST   /uploads           - Request upload URL
+GET    /uploads/:id       - Get upload status
+DELETE /uploads/:id       - Cancel upload
+GET    /uploads           - List user uploads
+```
+
+### Webhooks
+
+```
+POST {webhook_url}
+{
+  "event": "upload.completed",
+  "upload_id": "...",
+  "status": "complete",
+  "processed_url": "..."
+}
+```
+
+## Scaling Considerations
+
+### Current Capacity
+
+- 1000 uploads/day = ~1 per minute
+- Single worker can process 1 video every 5 minutes
+- Need 5 workers for current load
+
+### 10x Scale (10,000/day)
+
+- ~10 uploads per minute
+- Need 50 workers
+- Use spot instances for cost savings
+- Add Redis cache for status checks
+
+### 100x Scale (100,000/day)
+
+- ~100 uploads per minute
+- Partition by region
+- Use Kafka instead of SQS
+- Database sharding by user_id
+
+## Failure Modes
+
+### S3 Unavailable
+
+- Impact: Uploads fail
+- Mitigation: Multi-region S3 replication
+
+### Queue Backed Up
+
+- Impact: Processing delays
+- Mitigation: Auto-scale workers faster
+
+### Worker Crash During Processing
+
+- Impact: Job retried
+- Mitigation: Idempotent processing
+
+## Cost Estimate
+
+**Monthly (1000 uploads/day):**
+
+- S3 Storage: $50
+- S3 Transfer: $100
+- SQS: $10
+- Workers (EC2): $300
+- Database: $100
+  **Total: ~$560/month**
+
+## Security
+
+- Pre-signed URLs expire in 1 hour
+- Videos in private S3 buckets
+- CloudFront signed URLs for delivery
+- Rate limiting per user
+
+## Monitoring
+
+**Metrics:**
+
+- Upload success rate
+- Processing time (p50, p95, p99)
+- Queue depth
+- Worker CPU/memory
+- Error rate by type
+
+**Alerts:**
+
+- Queue depth >1000
+- Processing time p95 >10 minutes
+- Error rate >5%
+
+## Open Questions
+
+- [ ] Video retention policy? (30 days? 1 year?)
+- [ ] Maximum video duration? (affects processing time)
+- [ ] Regional data residency requirements?
+
+````
+
+## Component Template
+
+```markdown
+### Component Name
+
+**Responsibilities:**
+- Primary responsibility
+- Secondary responsibility
+
+**Technology Stack:**
+- Language: [Python/Node/Go]
+- Framework: [Express/FastAPI/Gin]
+- Database: [PostgreSQL/MongoDB]
+
+**API/Interface:**
+```typescript
+interface ComponentAPI {
+  method(params): ReturnType;
+}
+````
+
+**Scaling Strategy:**
+
+- Horizontal: Stateless, load balanced
+- Vertical: Cache layer, connection pooling
+
+**Dependencies:**
+
+- Service A (for X)
+- Database B (for persistence)
+
+**Failure Handling:**
+
+- Retry with exponential backoff
+- Circuit breaker for downstream services
+- Fallback to cached data
+
+```
+
+## Best Practices
+
+1. **Start with requirements**: Functional + non-functional
+2. **Draw diagrams first**: Visual clarity
+3. **Define boundaries**: What's in scope vs out
+4. **Document tradeoffs**: Every choice has costs
+5. **Plan for failure**: What breaks and how to handle
+6. **Consider scale**: Current, 10x, 100x
+7. **Estimate costs**: Build vs buy decisions
+8. **Leave open questions**: Don't pretend to know everything
+
+## Output Checklist
+
+- [ ] Requirements documented (functional + non-functional)
+- [ ] High-level architecture diagram
+- [ ] Component breakdown (3-7 components)
+- [ ] Data flow documented
+- [ ] Data model defined
+- [ ] API contracts specified
+- [ ] Scaling considerations (1x, 10x, 100x)
+- [ ] Failure modes identified
+- [ ] Cost estimate provided
+- [ ] Security considerations
+- [ ] Monitoring plan
+```