anchi-toolkit 1.2.0 → 1.2.2

package/dist/skills/backend-development/references/backend-performance.md

@@ -0,0 +1,397 @@
+# Backend Performance & Scalability
+
+Performance optimization strategies, caching patterns, and scalability best practices (2025).
+
+## Database Performance
+
+### Query Optimization
+
+#### Indexing Strategies
+
+**Impact:** 30% disk I/O reduction, 10-100x query speedup
+
+```sql
+-- Create index on frequently queried columns
+CREATE INDEX idx_users_email ON users(email);
+CREATE INDEX idx_orders_user_id ON orders(user_id);
+
+-- Composite index for multi-column queries
+CREATE INDEX idx_orders_user_date ON orders(user_id, created_at DESC);
+
+-- Partial index for filtered queries
+CREATE INDEX idx_active_users ON users(email) WHERE active = true;
+
+-- Analyze query performance
+EXPLAIN ANALYZE SELECT * FROM orders
+WHERE user_id = 123 AND created_at > '2025-01-01';
+```
+
+**Index Types:**
+- **B-tree** - Default, general-purpose (equality, range queries)
+- **Hash** - Fast equality lookups, no range queries
+- **GIN** - Full-text search, JSONB queries
+- **GiST** - Geospatial queries, range types
+
+**When NOT to Index:**
+- Small tables (<1000 rows)
+- Frequently updated columns
+- Low-cardinality columns (e.g., boolean with 2 values)
+
+### Connection Pooling
+
+**Impact:** 5-10x performance improvement
+
+```typescript
+// PostgreSQL with pg-pool
+import { Pool } from 'pg';
+
+const pool = new Pool({
+  host: process.env.DB_HOST,
+  database: process.env.DB_NAME,
+  user: process.env.DB_USER,
+  password: process.env.DB_PASSWORD,
+  max: 20, // Maximum connections
+  min: 5, // Minimum connections
+  idleTimeoutMillis: 30000, // Close idle connections after 30s
+  connectionTimeoutMillis: 2000, // Error if can't connect in 2s
+});
+
+// Use pool for queries
+const result = await pool.query('SELECT * FROM users WHERE id = $1', [userId]);
+```
+
+**Recommended Pool Sizes:**
+- **Web servers:** `connections = (core_count * 2) + effective_spindle_count`
+- **Typical:** 20-30 connections per app instance
+- **Monitor:** Connection saturation in production
+
+### N+1 Query Problem
+
+**Bad: N+1 queries**
+```typescript
+// Fetches 1 query for posts, then N queries for authors
+const posts = await Post.findAll();
+for (const post of posts) {
+  post.author = await User.findById(post.authorId); // N queries!
+}
+```
+
+**Good: Join or eager loading**
+```typescript
+// Single query with JOIN
+const posts = await Post.findAll({
+  include: [{ model: User, as: 'author' }],
+});
+```
+
+## Caching Strategies
+
+### Redis Caching
+
+**Impact:** 90% DB load reduction, 10-100x faster response
+
+#### Cache-Aside Pattern (Lazy Loading)
+
+```typescript
+async function getUser(userId: string) {
+  // Try cache first
+  const cached = await redis.get(`user:${userId}`);
+  if (cached) return JSON.parse(cached);
+
+  // Cache miss - fetch from DB
+  const user = await db.users.findById(userId);
+
+  // Store in cache (TTL: 1 hour)
+  await redis.setex(`user:${userId}`, 3600, JSON.stringify(user));
+
+  return user;
+}
+```
+
+#### Write-Through Pattern
+
+```typescript
+async function updateUser(userId: string, data: UpdateUserDto) {
+  // Update database
+  const user = await db.users.update(userId, data);
+
+  // Update cache immediately
+  await redis.setex(`user:${userId}`, 3600, JSON.stringify(user));
+
+  return user;
+}
+```
+
+#### Cache Invalidation
+
+```typescript
+// Invalidate on update
+async function deleteUser(userId: string) {
+  await db.users.delete(userId);
+  await redis.del(`user:${userId}`);
+  await redis.del(`user:${userId}:posts`); // Invalidate related caches
+}
+
+// Pattern-based invalidation
+await redis.keys('user:*').then(keys => redis.del(...keys));
+```
+
+### Cache Layers
+
+```
+Client
+  → CDN Cache (static assets, 50%+ latency reduction)
+  → API Gateway Cache (public endpoints)
+  → Application Cache (Redis)
+  → Database Query Cache
+  → Database
+```
+
+### Cache Best Practices
+
+1. **Cache frequently accessed data** - User profiles, config, product catalogs
+2. **Set appropriate TTL** - Balance freshness vs performance
+3. **Invalidate on write** - Keep cache consistent
+4. **Use cache keys wisely** - `resource:id:attribute` pattern
+5. **Monitor hit rates** - Target >80% hit rate
+
+## Load Balancing
+
+### Algorithms
+
+**Round Robin** - Distribute evenly across servers
+```nginx
+upstream backend {
+    server backend1.example.com;
+    server backend2.example.com;
+    server backend3.example.com;
+}
+```
+
+**Least Connections** - Route to server with fewest connections
+```nginx
+upstream backend {
+    least_conn;
+    server backend1.example.com;
+    server backend2.example.com;
+}
+```
+
+**IP Hash** - Same client → same server (session affinity)
+```nginx
+upstream backend {
+    ip_hash;
+    server backend1.example.com;
+    server backend2.example.com;
+}
+```
+
+### Health Checks
+
+```typescript
+// Express health check endpoint
+app.get('/health', async (req, res) => {
+  const checks = {
+    uptime: process.uptime(),
+    timestamp: Date.now(),
+    database: await checkDatabase(),
+    redis: await checkRedis(),
+    memory: process.memoryUsage(),
+  };
+
+  const isHealthy = checks.database && checks.redis;
+  res.status(isHealthy ? 200 : 503).json(checks);
+});
+```
+
+## Asynchronous Processing
+
+### Message Queues for Long-Running Tasks
+
+```typescript
+// Producer - Add job to queue
+import Queue from 'bull';
+
+const emailQueue = new Queue('email', {
+  redis: { host: 'localhost', port: 6379 },
+});
+
+await emailQueue.add('send-welcome', {
+  userId: user.id,
+  email: user.email,
+});
+
+// Consumer - Process jobs
+emailQueue.process('send-welcome', async (job) => {
+  await sendWelcomeEmail(job.data.email);
+});
+```
+
+**Use Cases:**
+- Email sending
+- Image/video processing
+- Report generation
+- Data export
+- Webhook delivery
+
+## CDN (Content Delivery Network)
+
+**Impact:** 50%+ latency reduction for global users
+
+### Configuration
+
+```typescript
+// Cache-Control headers
+res.setHeader('Cache-Control', 'public, max-age=31536000, immutable'); // Static assets
+res.setHeader('Cache-Control', 'public, max-age=3600'); // API responses
+res.setHeader('Cache-Control', 'private, no-cache'); // User-specific data
+```
+
+**CDN Providers:**
+- Cloudflare (generous free tier, global coverage)
+- AWS CloudFront (AWS integration)
+- Fastly (real-time purging)
+
+## Horizontal vs Vertical Scaling
+
+### Horizontal Scaling (Scale Out)
+
+**Pros:**
+- Better fault tolerance
+- Unlimited scaling potential
+- Cost-effective (commodity hardware)
+
+**Cons:**
+- Complex architecture
+- Data consistency challenges
+- Network overhead
+
+**When to use:** High traffic, need redundancy, stateless applications
+
+### Vertical Scaling (Scale Up)
+
+**Pros:**
+- Simple architecture
+- No code changes needed
+- Easier data consistency
+
+**Cons:**
+- Hardware limits
+- Single point of failure
+- Expensive at high end
+
+**When to use:** Monolithic apps, rapid scaling needed, data consistency critical
+
+## Database Scaling Patterns
+
+### Read Replicas
+
+```
+Primary (Write) → Replica 1 (Read)
+               → Replica 2 (Read)
+               → Replica 3 (Read)
+```
+
+**Implementation:**
+```typescript
+// Write to primary
+await primaryDb.users.create(userData);
+
+// Read from replica
+const users = await replicaDb.users.findAll();
+```
+
+**Use Cases:**
+- Read-heavy workloads (90%+ reads)
+- Analytics queries
+- Reporting dashboards
+
+### Database Sharding
+
+**Horizontal Partitioning** - Split data across databases
+
+```typescript
+// Shard by user ID
+function getShardId(userId: string): number {
+  return hashCode(userId) % SHARD_COUNT;
+}
+
+const shardId = getShardId(userId);
+const db = shards[shardId];
+const user = await db.users.findById(userId);
+```
+
+**Sharding Strategies:**
+- **Range-based:** Users 1-1M → Shard 1, 1M-2M → Shard 2
+- **Hash-based:** Hash(userId) % shard_count
+- **Geographic:** EU users → EU shard, US users → US shard
+- **Entity-based:** Users → Shard 1, Orders → Shard 2
+
+## Performance Monitoring
+
+### Key Metrics
+
+**Application:**
+- Response time (p50, p95, p99)
+- Throughput (requests/second)
+- Error rate
+- CPU/memory usage
+
+**Database:**
+- Query execution time
+- Connection pool saturation
+- Cache hit rate
+- Slow query log
+
+**Tools:**
+- Prometheus + Grafana (metrics)
+- New Relic / Datadog (APM)
+- Sentry (error tracking)
+- OpenTelemetry (distributed tracing)
+
+## Performance Optimization Checklist
+
+### Database
+- [ ] Indexes on frequently queried columns
+- [ ] Connection pooling configured
+- [ ] N+1 queries eliminated
+- [ ] Slow query log monitored
+- [ ] Query execution plans analyzed
+
+### Caching
+- [ ] Redis cache for hot data
+- [ ] Cache TTL configured appropriately
+- [ ] Cache invalidation on writes
+- [ ] CDN for static assets
+- [ ] >80% cache hit rate achieved
+
+### Application
+- [ ] Async processing for long tasks
+- [ ] Response compression enabled (gzip)
+- [ ] Load balancing configured
+- [ ] Health checks implemented
+- [ ] Resource limits set (CPU, memory)
+
+### Monitoring
+- [ ] APM tool configured (New Relic/Datadog)
+- [ ] Error tracking (Sentry)
+- [ ] Performance dashboards (Grafana)
+- [ ] Alerting on key metrics
+- [ ] Distributed tracing for microservices
+
+## Common Performance Pitfalls
+
+1. **No caching** - Repeatedly querying same data
+2. **Missing indexes** - Full table scans
+3. **N+1 queries** - Fetching related data in loops
+4. **Synchronous processing** - Blocking on long tasks
+5. **No connection pooling** - Creating new connections per request
+6. **Unbounded queries** - No LIMIT on large tables
+7. **No CDN** - Serving static assets from origin
+
+## Resources
+
+- **PostgreSQL Performance:** https://www.postgresql.org/docs/current/performance-tips.html
+- **Redis Best Practices:** https://redis.io/docs/management/optimization/
+- **Web Performance:** https://web.dev/performance/
+- **Database Indexing:** https://use-the-index-luke.com/

package/dist/skills/backend-development/references/backend-security.md

@@ -0,0 +1,290 @@
+# Backend Security
+
+Security best practices, OWASP Top 10 mitigation, and modern security standards (2025).
+
+## OWASP Top 10 (2025 RC1)
+
+### New Entries (2025)
+- **Supply Chain Failures** - Vulnerable dependencies, compromised packages
+- **Mishandling of Exceptional Conditions** - Improper error handling exposing system info
+
+### Top Vulnerabilities & Mitigation
+
+#### 1. Broken Access Control
+**Risk:** Users access unauthorized resources (28% of vulnerabilities)
+
+**Mitigation:**
+- Implement RBAC (Role-Based Access Control)
+- Deny by default, explicitly allow
+- Log access control failures
+- Enforce authorization on backend (never client-side)
+- Use JWT with proper claims validation
+
+```typescript
+// Good: Server-side authorization check
+@UseGuards(JwtAuthGuard, RolesGuard)
+@Roles('admin')
+async deleteUser(@Param('id') id: string) {
+  // Verify user can access this resource
+  return this.usersService.delete(id);
+}
+```
+
+#### 2. Cryptographic Failures
+**Risk:** Sensitive data exposure, weak encryption
+
+**Mitigation:**
+- Use Argon2id for password hashing (replaces bcrypt as of 2025)
+- TLS 1.3 for data in transit
+- Encrypt sensitive data at rest (AES-256)
+- Use crypto.randomBytes() for tokens, not Math.random()
+- Never store passwords in plain text
+
+```python
+# Good: Argon2id password hashing
+from argon2 import PasswordHasher
+
+ph = PasswordHasher()
+hash = ph.hash("password123")  # Auto-salted, memory-hard
+ph.verify(hash, "password123")  # Verify password
+```
+
+#### 3. Injection Attacks
+**Risk:** SQL injection, NoSQL injection, command injection (6x increase 2020-2024)
+
+**Mitigation (98% vulnerability reduction):**
+- Use parameterized queries ALWAYS
+- Input validation with allow-lists
+- Escape special characters
+- Use ORMs properly (avoid raw queries)
+
+```typescript
+// Bad: Vulnerable to SQL injection
+const query = `SELECT * FROM users WHERE email = '${email}'`;
+
+// Good: Parameterized query
+const query = 'SELECT * FROM users WHERE email = $1';
+const result = await db.query(query, [email]);
+```
+
+#### 4. Insecure Design
+**Risk:** Flawed architecture, missing security controls
+
+**Mitigation:**
+- Threat modeling during design phase
+- Security requirements from start
+- Principle of least privilege
+- Defense in depth (multiple security layers)
+
+#### 5. Security Misconfiguration
+**Risk:** Default credentials, verbose errors, unnecessary features enabled
+
+**Mitigation:**
+- Remove default accounts
+- Disable directory listing
+- Use security headers (CSP, HSTS, X-Frame-Options)
+- Minimize attack surface
+- Regular security audits
+
+```typescript
+// Security headers middleware
+app.use(helmet({
+  contentSecurityPolicy: {
+    directives: {
+      defaultSrc: ["'self'"],
+      scriptSrc: ["'self'", "'unsafe-inline'"],
+    },
+  },
+  hsts: {
+    maxAge: 31536000,
+    includeSubDomains: true,
+  },
+}));
+```
+
+#### 6. Vulnerable Components
+**Risk:** Outdated dependencies with known vulnerabilities
+
+**Mitigation:**
+- Regular dependency updates (npm audit, pip-audit)
+- Use Dependabot/Renovate for automated updates
+- Monitor CVE databases
+- Software composition analysis (SCA) in CI/CD
+- Lock file integrity checks
+
+```bash
+# Check for vulnerabilities
+npm audit fix
+pip-audit --fix
+```
+
+#### 7. Authentication Failures
+**Risk:** Weak passwords, session hijacking, credential stuffing
+
+**Mitigation:**
+- MFA mandatory for admin accounts
+- Rate limiting on login endpoints (10 attempts/minute)
+- Strong password policies (12+ chars, complexity)
+- Session timeout (15 mins idle, 8 hours absolute)
+- FIDO2/WebAuthn for passwordless auth
+
+#### 8. Software & Data Integrity Failures
+**Risk:** CI/CD pipeline compromise, unsigned updates
+
+**Mitigation:**
+- Code signing for releases
+- Verify integrity of packages (lock files)
+- Secure CI/CD pipelines (immutable builds)
+- Checksum verification
+
+#### 9. Logging & Monitoring Failures
+**Risk:** Breaches undetected, insufficient audit trail
+
+**Mitigation:**
+- Log authentication events (success/failure)
+- Log access control failures
+- Centralized logging (ELK Stack, Splunk)
+- Alerting on suspicious patterns
+- Log rotation and retention policies
+
+#### 10. Server-Side Request Forgery (SSRF)
+**Risk:** Server makes malicious requests to internal resources
+
+**Mitigation:**
+- Validate and sanitize URLs
+- Allow-list for remote resources
+- Network segmentation
+- Disable unnecessary protocols (file://, gopher://)
+
+## Input Validation (Prevents 70%+ Vulnerabilities)
+
+### Validation Strategies
+
+**1. Type Validation**
+```typescript
+// Use class-validator with NestJS
+class CreateUserDto {
+  @IsEmail()
+  email: string;
+
+  @IsString()
+  @MinLength(12)
+  @Matches(/^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)/)
+  password: string;
+
+  @IsInt()
+  @Min(18)
+  age: number;
+}
+```
+
+**2. Sanitization**
+```typescript
+import DOMPurify from 'isomorphic-dompurify';
+
+// Sanitize HTML input
+const clean = DOMPurify.sanitize(userInput);
+```
+
+**3. Allow-lists (Preferred over Deny-lists)**
+```typescript
+// Good: Allow-list approach
+const allowedFields = ['name', 'email', 'age'];
+const sanitized = Object.keys(input)
+  .filter(key => allowedFields.includes(key))
+  .reduce((obj, key) => ({ ...obj, [key]: input[key] }), {});
+```
+
+## Rate Limiting
+
+### Token Bucket Algorithm (Industry Standard)
+
+```typescript
+import rateLimit from 'express-rate-limit';
+
+const limiter = rateLimit({
+  windowMs: 15 * 60 * 1000, // 15 minutes
+  max: 100, // 100 requests per window
+  standardHeaders: true,
+  legacyHeaders: false,
+  message: 'Too many requests, please try again later',
+});
+
+app.use('/api/', limiter);
+```
+
+### API-Specific Limits
+
+- **Authentication:** 10 attempts/15 min
+- **Public APIs:** 100 requests/15 min
+- **Authenticated APIs:** 1000 requests/15 min
+- **Admin endpoints:** 50 requests/15 min
+
+## Security Headers
+
+```typescript
+// Essential security headers (2025)
+{
+  'Strict-Transport-Security': 'max-age=31536000; includeSubDomains',
+  'Content-Security-Policy': "default-src 'self'",
+  'X-Frame-Options': 'DENY',
+  'X-Content-Type-Options': 'nosniff',
+  'Referrer-Policy': 'strict-origin-when-cross-origin',
+  'Permissions-Policy': 'geolocation=(), microphone=()',
+}
+```
+
+## Secrets Management
+
+### Best Practices
+
+1. **Never commit secrets** - Use .env files (gitignored)
+2. **Environment-specific** - Different secrets per environment
+3. **Rotation policy** - Rotate secrets every 90 days
+4. **Encryption at rest** - Encrypt secrets in secret managers
+5. **Least privilege** - Minimal permissions per secret
+
+### Tools
+
+- **HashiCorp Vault** - Multi-cloud, dynamic secrets
+- **AWS Secrets Manager** - Managed service, auto-rotation
+- **Azure Key Vault** - Integrated with Azure services
+- **Pulumi ESC** - Unified secrets orchestration (2025 trend)
+
+```typescript
+// Good: Secrets from environment
+const dbPassword = process.env.DB_PASSWORD;
+if (!dbPassword) throw new Error('DB_PASSWORD not set');
+```
+
+## API Security Checklist
+
+- [ ] Use HTTPS/TLS 1.3 only
+- [ ] Implement OAuth 2.1 + JWT for authentication
+- [ ] Rate limiting on all endpoints
+- [ ] Input validation on all inputs
+- [ ] Parameterized queries (prevent SQL injection)
+- [ ] Security headers configured
+- [ ] CORS properly configured (not `*` in production)
+- [ ] API versioning implemented
+- [ ] Error messages don't leak system info
+- [ ] Logging authentication events
+- [ ] MFA for admin accounts
+- [ ] Regular security audits (quarterly)
+
+## Common Security Pitfalls
+
+1. **Client-side validation only** - Always validate on server
+2. **Using Math.random() for tokens** - Use crypto.randomBytes()
+3. **Storing passwords with bcrypt** - Use Argon2id (2025 standard)
+4. **Trusting user input** - Validate and sanitize everything
+5. **Weak CORS configuration** - Don't use `*` in production
+6. **Insufficient logging** - Log all authentication/authorization events
+7. **No rate limiting** - Implement on all public endpoints
+
+## Resources
+
+- **OWASP Top 10 (2025):** https://owasp.org/www-project-top-ten/
+- **OWASP Cheat Sheets:** https://cheatsheetseries.owasp.org/
+- **CWE Top 25:** https://cwe.mitre.org/top25/
+- **NIST Guidelines:** https://www.nist.gov/cybersecurity