proagents 1.6.16 → 1.6.18

package/.proagents/examples/backend-nodejs/complete-conversation.md

@@ -1,601 +0,0 @@
-# Complete Conversation Example: Node.js API
-
-An example of building a backend feature using ProAgents workflow.
-
----
-
-## Starting Point
-
-**User has:** Express + TypeScript API with Prisma ORM and JWT auth.
-
-**Goal:** Add a rate limiting system with per-user quotas.
-
----
-
-## The Conversation
-
-### User Initiates
-
-```
-User: Add rate limiting to our API. I want different limits for different
-user tiers (free, pro, enterprise) and per-endpoint limits for expensive
-operations.
-```
-
-### AI Response: Analysis & Understanding
-
-```
-AI: I'll help you implement a tiered rate limiting system. Let me analyze
-your codebase first.
-
-[Running Analysis...]
-
-──────────────────────────────────────────────────────────
-Codebase Analysis Complete
-──────────────────────────────────────────────────────────
-
-Tech Stack:
-• Node.js 20.x + TypeScript 5.2
-• Express 4.18
-• Prisma ORM 5.x (PostgreSQL)
-• JWT authentication
-• Redis connected (for caching)
-
-Middleware Structure:
-src/middleware/
-├── auth.ts           # JWT validation, sets req.user
-├── validate.ts       # Zod schema validation
-├── errorHandler.ts   # Global error handler
-└── cors.ts           # CORS configuration
-
-Existing User Model:
-model User {
-  id        String   @id @default(uuid())
-  email     String   @unique
-  tier      UserTier @default(FREE)
-  // ... other fields
-}
-
-enum UserTier {
-  FREE
-  PRO
-  ENTERPRISE
-}
-
-I see you already have:
-• User tier in the database
-• Redis available for rate limit storage
-• Auth middleware that sets req.user
-
-This is a good foundation for tiered rate limiting. Let me gather the
-specific requirements.
-```
-
-### AI: Requirements Clarification
-
-```
-AI: I have some questions to define the rate limits:
-
-1. What are the default rate limits per tier?
-
-   My suggestion based on common patterns:
-   ┌────────────────────────────────────────────────────┐
-   │ Tier        │ Requests/min │ Requests/day │ Burst  │
-   ├────────────────────────────────────────────────────┤
-   │ FREE        │ 60           │ 1,000        │ 10     │
-   │ PRO         │ 300          │ 10,000       │ 50     │
-   │ ENTERPRISE  │ 1,000        │ 100,000      │ 200    │
-   └────────────────────────────────────────────────────┘
-
-2. Which endpoints need special (lower) limits?
-
-   Common expensive operations:
-   [ ] File uploads
-   [ ] Report generation
-   [ ] Bulk operations
-   [ ] Search with complex queries
-   [ ] Export endpoints
-
-3. What should happen when rate limited?
-   [ ] Return 429 with retry-after header (standard)
-   [ ] Return 429 with upgrade prompt
-   [ ] Gradually degrade (slower responses)
-
-4. Should rate limits reset at midnight or rolling window?
-   [ ] Rolling window (recommended)
-   [ ] Fixed daily reset
-```
-
-### User Provides Details
-
-```
-User:
-1. Your suggestions look good, use those numbers
-2. File uploads, report generation, and bulk operations need lower limits
-3. Return 429 with retry-after and include upgrade info for free users
-4. Rolling window
-```
-
-### AI: Implementation Plan
-
-```
-AI: Got it. Here's the implementation plan:
-
-──────────────────────────────────────────────────────────
-Implementation Plan: Rate Limiting System
-──────────────────────────────────────────────────────────
-
-Architecture:
-┌─────────────────────────────────────────────────────────┐
-│                    Request Flow                         │
-├─────────────────────────────────────────────────────────┤
-│                                                         │
-│  Request → Auth Middleware → Rate Limiter → Route       │
-│                    │                                    │
-│                    ↓                                    │
-│              ┌─────────────┐                           │
-│              │   Redis     │                           │
-│              │ (Counters)  │                           │
-│              └─────────────┘                           │
-│                                                         │
-└─────────────────────────────────────────────────────────┘
-
-Files to Create:
-
-src/middleware/
-├── rateLimit/
-│   ├── index.ts              # Main middleware export
-│   ├── rateLimiter.ts        # Core rate limiting logic
-│   ├── tierLimits.ts         # Tier configuration
-│   ├── endpointLimits.ts     # Per-endpoint overrides
-│   └── types.ts              # TypeScript interfaces
-
-src/services/
-└── rateLimitStore.ts         # Redis operations
-
-src/utils/
-└── rateLimitResponse.ts      # 429 response formatting
-
-Files to Modify:
-
-• src/app.ts                  # Apply global rate limiter
-• src/routes/files.ts         # Add stricter limits
-• src/routes/reports.ts       # Add stricter limits
-• src/routes/bulk.ts          # Add stricter limits
-• src/types/express.d.ts      # Extend Request type
-
-Database Changes:
-
-• Add RateLimitOverride table (for custom limits per user)
-• Migration: 20240120_add_rate_limit_overrides
-
-Shall I proceed with implementation?
-```
-
-### User Confirms
-
-```
-User: Yes, proceed.
-```
-
-### AI: Implementation
-
-```
-AI: Starting implementation...
-
-✓ Created types and configuration
-──────────────────────────────────────────────────────────
-// src/middleware/rateLimit/tierLimits.ts
-
-import { UserTier } from '@prisma/client';
-
-export interface TierLimit {
-  requestsPerMinute: number;
-  requestsPerDay: number;
-  burstLimit: number;
-}
-
-export const TIER_LIMITS: Record<UserTier, TierLimit> = {
-  FREE: {
-    requestsPerMinute: 60,
-    requestsPerDay: 1_000,
-    burstLimit: 10,
-  },
-  PRO: {
-    requestsPerMinute: 300,
-    requestsPerDay: 10_000,
-    burstLimit: 50,
-  },
-  ENTERPRISE: {
-    requestsPerMinute: 1_000,
-    requestsPerDay: 100_000,
-    burstLimit: 200,
-  },
-};
-
-// Expensive endpoint multipliers (lower = stricter)
-export const ENDPOINT_MULTIPLIERS: Record<string, number> = {
-  'POST:/api/files/upload': 0.1,      // 10% of normal limit
-  'POST:/api/reports/generate': 0.05,  // 5% of normal limit
-  'POST:/api/bulk/*': 0.02,            // 2% of normal limit
-};
-──────────────────────────────────────────────────────────
-
-✓ Created Redis store
-──────────────────────────────────────────────────────────
-// src/services/rateLimitStore.ts
-
-import { redis } from './redis';
-
-export class RateLimitStore {
-  private prefix = 'ratelimit:';
-
-  async increment(
-    userId: string,
-    window: 'minute' | 'day',
-    endpoint?: string
-  ): Promise<{ count: number; ttl: number }> {
-    const key = this.getKey(userId, window, endpoint);
-    const ttl = window === 'minute' ? 60 : 86400;
-
-    const multi = redis.multi();
-    multi.incr(key);
-    multi.expire(key, ttl);
-    multi.ttl(key);
-
-    const [count, , remainingTtl] = await multi.exec();
-
-    return {
-      count: count as number,
-      ttl: remainingTtl as number,
-    };
-  }
-
-  async getCount(
-    userId: string,
-    window: 'minute' | 'day',
-    endpoint?: string
-  ): Promise<number> {
-    const key = this.getKey(userId, window, endpoint);
-    const count = await redis.get(key);
-    return parseInt(count || '0', 10);
-  }
-
-  private getKey(
-    userId: string,
-    window: string,
-    endpoint?: string
-  ): string {
-    const base = `${this.prefix}${userId}:${window}`;
-    return endpoint ? `${base}:${endpoint}` : base;
-  }
-}
-
-export const rateLimitStore = new RateLimitStore();
-──────────────────────────────────────────────────────────
-
-✓ Created main rate limiter middleware
-──────────────────────────────────────────────────────────
-// src/middleware/rateLimit/rateLimiter.ts
-
-import { Request, Response, NextFunction } from 'express';
-import { rateLimitStore } from '@/services/rateLimitStore';
-import { TIER_LIMITS, ENDPOINT_MULTIPLIERS } from './tierLimits';
-import { RateLimitError } from '@/errors';
-
-interface RateLimitOptions {
-  windowMs?: number;
-  endpointKey?: string;
-}
-
-export function createRateLimiter(options: RateLimitOptions = {}) {
-  return async (req: Request, res: Response, next: NextFunction) => {
-    // Skip if no authenticated user (handled by auth)
-    if (!req.user) {
-      return next();
-    }
-
-    const userId = req.user.id;
-    const userTier = req.user.tier;
-    const limits = TIER_LIMITS[userTier];
-
-    // Get endpoint-specific multiplier
-    const endpointKey = options.endpointKey ||
-      `${req.method}:${req.baseUrl}${req.path}`;
-    const multiplier = getMultiplier(endpointKey);
-
-    // Calculate effective limits
-    const effectiveMinuteLimit = Math.floor(
-      limits.requestsPerMinute * multiplier
-    );
-    const effectiveDayLimit = Math.floor(
-      limits.requestsPerDay * multiplier
-    );
-
-    // Check minute window
-    const minuteResult = await rateLimitStore.increment(
-      userId,
-      'minute',
-      multiplier < 1 ? endpointKey : undefined
-    );
-
-    if (minuteResult.count > effectiveMinuteLimit) {
-      return sendRateLimitResponse(res, {
-        retryAfter: minuteResult.ttl,
-        limit: effectiveMinuteLimit,
-        window: 'minute',
-        userTier,
-      });
-    }
-
-    // Check daily window
-    const dayResult = await rateLimitStore.increment(userId, 'day');
-
-    if (dayResult.count > effectiveDayLimit) {
-      return sendRateLimitResponse(res, {
-        retryAfter: dayResult.ttl,
-        limit: effectiveDayLimit,
-        window: 'day',
-        userTier,
-      });
-    }
-
-    // Add rate limit headers
-    res.setHeader('X-RateLimit-Limit', effectiveMinuteLimit);
-    res.setHeader('X-RateLimit-Remaining',
-      Math.max(0, effectiveMinuteLimit - minuteResult.count)
-    );
-    res.setHeader('X-RateLimit-Reset',
-      Math.floor(Date.now() / 1000) + minuteResult.ttl
-    );
-
-    next();
-  };
-}
-
-function getMultiplier(endpointKey: string): number {
-  // Check exact match first
-  if (ENDPOINT_MULTIPLIERS[endpointKey]) {
-    return ENDPOINT_MULTIPLIERS[endpointKey];
-  }
-
-  // Check wildcard patterns
-  for (const [pattern, mult] of Object.entries(ENDPOINT_MULTIPLIERS)) {
-    if (pattern.endsWith('/*')) {
-      const prefix = pattern.slice(0, -1);
-      if (endpointKey.startsWith(prefix)) {
-        return mult;
-      }
-    }
-  }
-
-  return 1;
-}
-
-function sendRateLimitResponse(
-  res: Response,
-  info: {
-    retryAfter: number;
-    limit: number;
-    window: string;
-    userTier: string;
-  }
-) {
-  res.setHeader('Retry-After', info.retryAfter);
-
-  const response: any = {
-    error: 'Too Many Requests',
-    message: `Rate limit exceeded. Try again in ${info.retryAfter} seconds.`,
-    retryAfter: info.retryAfter,
-  };
-
-  // Add upgrade prompt for free users
-  if (info.userTier === 'FREE') {
-    response.upgrade = {
-      message: 'Upgrade to Pro for higher rate limits',
-      link: '/pricing',
-      proLimit: TIER_LIMITS.PRO.requestsPerMinute,
-    };
-  }
-
-  return res.status(429).json(response);
-}
-
-export const rateLimit = createRateLimiter();
-──────────────────────────────────────────────────────────
-
-✓ Applied to app.ts
-──────────────────────────────────────────────────────────
-// In src/app.ts - After auth middleware
-
-import { rateLimit } from '@/middleware/rateLimit';
-
-// Apply global rate limiter
-app.use(rateLimit);
-──────────────────────────────────────────────────────────
-
-✓ Added stricter limits to expensive routes
-──────────────────────────────────────────────────────────
-// In src/routes/files.ts
-
-import { createRateLimiter } from '@/middleware/rateLimit';
-
-const uploadLimiter = createRateLimiter({
-  endpointKey: 'POST:/api/files/upload',
-});
-
-router.post('/upload', uploadLimiter, async (req, res) => {
-  // ... upload logic
-});
-──────────────────────────────────────────────────────────
-
-✓ Created database migration
-──────────────────────────────────────────────────────────
--- Migration: 20240120_add_rate_limit_overrides
-
-CREATE TABLE rate_limit_overrides (
-  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
-  user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
-  requests_per_minute INT,
-  requests_per_day INT,
-  endpoint_pattern VARCHAR(255),
-  created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
-  expires_at TIMESTAMP WITH TIME ZONE,
-  reason TEXT,
-  UNIQUE(user_id, endpoint_pattern)
-);
-
-CREATE INDEX idx_rate_limit_overrides_user
-  ON rate_limit_overrides(user_id);
-──────────────────────────────────────────────────────────
-
-Implementation complete. Running tests...
-```
-
-### AI: Testing
-
-```
-AI: Running tests...
-
-──────────────────────────────────────────────────────────
-Test Results
-──────────────────────────────────────────────────────────
-
-Unit Tests:
-✓ rateLimitStore.increment increments counter (15ms)
-✓ rateLimitStore.increment sets TTL correctly (12ms)
-✓ createRateLimiter allows requests under limit (28ms)
-✓ createRateLimiter blocks requests over minute limit (32ms)
-✓ createRateLimiter blocks requests over daily limit (35ms)
-✓ createRateLimiter applies endpoint multipliers (25ms)
-✓ createRateLimiter sets correct headers (22ms)
-✓ FREE tier gets upgrade prompt in 429 response (30ms)
-✓ PRO tier does not get upgrade prompt (28ms)
-
-Integration Tests:
-✓ Rate limit persists across requests (145ms)
-✓ Rate limit resets after window (2015ms)
-✓ Different users have separate limits (95ms)
-✓ Expensive endpoints have lower limits (88ms)
-
-Load Tests:
-✓ Handles 1000 concurrent rate checks (234ms)
-
-Coverage: 96%
-
-All 14 tests passed.
-```
-
-### AI: Summary
-
-```
-AI: Rate limiting implementation complete!
-
-──────────────────────────────────────────────────────────
-Summary: Rate Limiting System
-──────────────────────────────────────────────────────────
-
-Features Implemented:
-✓ Tiered rate limits (FREE/PRO/ENTERPRISE)
-✓ Per-minute and per-day windows
-✓ Rolling window implementation
-✓ Per-endpoint overrides for expensive operations
-✓ Standard rate limit headers
-✓ 429 responses with retry-after
-✓ Upgrade prompts for free users
-✓ Redis-based distributed counting
-✓ Custom override support via database
-
-Rate Limits:
-┌────────────────────────────────────────────────────┐
-│ Tier        │ /min  │ /day    │ Upload │ Report  │
-├────────────────────────────────────────────────────┤
-│ FREE        │ 60    │ 1,000   │ 6      │ 3       │
-│ PRO         │ 300   │ 10,000  │ 30     │ 15      │
-│ ENTERPRISE  │ 1,000 │ 100,000 │ 100    │ 50      │
-└────────────────────────────────────────────────────┘
-
-Files Created: 6
-Files Modified: 4
-Test Coverage: 96%
-
-Ready to commit?
-```
-
-### User Commits
-
-```
-User: Yes, commit it.
-```
-
-### AI: Commit
-
-```
-AI: Committing changes...
-
-$ git add .
-$ git commit -m "feat(api): add tiered rate limiting system
-
-- Add rate limiting middleware with Redis storage
-- Implement FREE/PRO/ENTERPRISE tier limits
-- Add per-endpoint multipliers for expensive operations
-- Include rate limit headers on all responses
-- Return 429 with retry-after and upgrade prompts
-- Add rate limit override table for custom limits
-- Add comprehensive tests (96% coverage)
-
-Rate limits:
-- FREE: 60/min, 1000/day
-- PRO: 300/min, 10000/day
-- ENTERPRISE: 1000/min, 100000/day
-
-Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>"
-
-Committed: abc123f
-Branch: feature/rate-limiting
-```
-
----
-
-## API Documentation Generated
-
-```
-AI: I've also generated API documentation for the rate limiting:
-
-──────────────────────────────────────────────────────────
-Rate Limiting Documentation
-──────────────────────────────────────────────────────────
-
-All API requests include rate limit headers:
-
-Response Headers:
-• X-RateLimit-Limit: Maximum requests per minute
-• X-RateLimit-Remaining: Requests remaining in window
-• X-RateLimit-Reset: Unix timestamp when window resets
-
-429 Response:
-{
-  "error": "Too Many Requests",
-  "message": "Rate limit exceeded. Try again in 45 seconds.",
-  "retryAfter": 45,
-  "upgrade": {
-    "message": "Upgrade to Pro for higher rate limits",
-    "link": "/pricing",
-    "proLimit": 300
-  }
-}
-
-For custom rate limits, contact support.
-──────────────────────────────────────────────────────────
-```
-
----
-
-## Key Backend Patterns Shown
-
-1. **Middleware composition** - Rate limiter as Express middleware
-2. **Redis integration** - For distributed rate limit counting
-3. **Configuration-driven** - Tier limits in separate config file
-4. **Header standards** - Proper rate limit headers
-5. **Database flexibility** - Override table for exceptions
-6. **Testability** - High test coverage with unit and integration tests