@democratize-quality/mcp-server 1.2.0 → 1.2.1

package/src/skills/test-generation/SKILL.md

@@ -0,0 +1,309 @@
+---
+name: test-generation
+description: Generate executable API tests from test plans in multiple formats (Playwright, Jest, Postman). Automatically detects project configuration and creates tests with proper authentication, setup, and error handling. Use when user wants to generate tests, create test files, convert test plans to code, or needs automated API test creation.
+---
+
+# API Test Generation Skill
+
+Use this skill to convert API test plans into executable test code in Playwright, Jest, or Postman formats.
+
+## When to Use This Skill
+
+- User has a test plan and wants to generate executable tests
+- User mentions generating Playwright tests or Jest tests
+- User needs to create Postman collections
+- User wants to convert test documentation to code
+- User requests automated test generation
+
+## Core Workflow
+
+### Step 1: Project Setup Detection (REQUIRED FIRST STEP)
+
+**ALWAYS call `api_project_setup` tool FIRST** before generating any tests:
+
+```javascript
+const setupResult = await tools.api_project_setup({
+  outputDir: "./tests"  // or user-specified directory
+})
+```
+
+**The tool uses Smart Detection:**
+- ✅ **Has playwright.config.ts** → Auto-detects: Playwright + TypeScript
+- ✅ **Has playwright.config.js** → Auto-detects: Playwright + JavaScript
+- ✅ **Has jest.config.ts** → Auto-detects: Jest + TypeScript
+- ✅ **Has jest.config.js** → Auto-detects: Jest + JavaScript
+- ⚠️ **Has tsconfig.json only** → Asks user: Which framework?
+- ❓ **No config files** → Asks user: Framework + Language
+
+### Step 2: Handle Detection Results
+
+**Case A: Auto-Detected Configuration**
+```javascript
+if (setupResult.autoDetected) {
+  // Configuration found - proceed directly to generation
+  const framework = setupResult.config.framework;  // 'playwright' or 'jest'
+  const language = setupResult.config.language;    // 'typescript' or 'javascript'
+  
+  console.log(`✓ Detected ${framework} project with ${language}`);
+  // Proceed to Step 3
+}
+```
+
+**Case B: Partial Detection (TypeScript found, need framework)**
+```javascript
+if (setupResult.needsUserInput && setupResult.detected.hasTypeScript) {
+  // Ask user: "I found TypeScript. Which framework would you like to use?"
+  // Options: Playwright (recommended), Jest, Postman, All formats
+  // After user responds, store: language='typescript', framework=<user choice>
+}
+```
+
+**Case C: No Configuration (Empty project)**
+```javascript
+if (setupResult.needsUserInput && !setupResult.detected.hasTypeScript) {
+  // Ask user both:
+  // 1. Which test framework? Playwright / Jest / Postman / All
+  // 2. Which language? TypeScript (recommended) / JavaScript
+  // After user responds, store both values
+}
+```
+
+### Step 3: Extract Test Plan Content (When User Specifies Sections)
+
+When user requests specific sections (e.g., "generate tests for section 1"):
+
+```javascript
+// Read test plan file
+const testPlanContent = await readFile("./api-test-plan.md");
+
+// Parse sections using markdown headers (##)
+// Extract requested sections preserving structure
+
+// User says: "generate tests for section 1"
+// Extract: Section at index 0 (first ## after title)
+
+// User says: "tests for GET /api/v1/Users"
+// Extract: Section(s) matching pattern in title
+```
+
+**Include base URL** from plan overview in extracted content.
+
+### Step 4: Generate Tests with api_generator
+
+```javascript
+await tools.api_generator({
+  // Use extracted content OR full plan path
+  testPlanContent: extractedContent,  // for specific sections
+  // OR testPlanPath: "./api-test-plan.md",  // for full plan
+  
+  // Use detected/chosen configuration from Step 1
+  outputFormat: detectedConfig.framework,  // 'playwright', 'jest', 'postman', 'all'
+  language: detectedConfig.language,       // 'typescript' or 'javascript'
+  
+  // Pass project info from Step 1
+  projectInfo: {
+    hasTypeScript: detectedConfig.hasTypeScript,
+    hasPlaywrightConfig: detectedConfig.hasPlaywrightConfig,
+    hasJestConfig: detectedConfig.hasJestConfig
+  },
+  
+  // Additional parameters
+  outputDir: "./tests",
+  sessionId: "api-gen-" + Date.now(),
+  includeAuth: true,
+  includeSetup: true,
+  baseUrl: "https://api.example.com"  // optional override
+})
+```
+
+## Output Formats
+
+### Playwright Tests (TypeScript/JavaScript)
+- Browser-based API testing with request fixture
+- Full HTTP client with assertions
+- TypeScript types for better IDE support
+
+### Jest Tests (TypeScript/JavaScript)
+- Node.js API testing with axios
+- Popular testing framework
+- Easy integration with existing projects
+
+### Postman Collections
+- Import-ready JSON format
+- Environment variables included
+- Can be used in Postman or Newman
+
+### All Formats
+- Generate all three formats simultaneously
+- Maximum compatibility
+- Ready for different testing scenarios
+
+## Example Generation Scenarios
+
+### Scenario 1: New Empty Project
+```
+1. User asks to generate tests
+2. Call api_project_setup → No config detected
+3. Ask user: Framework? Language?
+4. User chooses: Playwright + JavaScript
+5. Call api_generator with choices
+6. Generate tests + setup instructions
+```
+
+### Scenario 2: Existing TypeScript Project
+```
+1. User asks to generate tests
+2. Call api_project_setup → Auto-detects Playwright + TypeScript
+3. Call api_generator with detected config
+4. Generate tests (no user input needed)
+```
+
+### Scenario 3: Specific Section Request
+```
+1. User: "Generate tests for section 2"
+2. Call api_project_setup → Detect config
+3. Read test plan and extract section 2
+4. Call api_generator with extracted content + config
+5. Generate tests for that section only
+```
+
+### Scenario 4: Override Auto-Detection
+```
+1. User: "Generate Jest tests in JavaScript"
+2. Call api_project_setup (may detect Playwright)
+3. User explicitly wants Jest + JS → Use user preference
+4. Call api_generator with outputFormat='jest', language='javascript'
+5. Generate requested format
+```
+
+## Session Management and Validation
+
+After generation, validate using API request tools:
+
+```javascript
+// Validate generated tests work
+await tools.api_request({
+  sessionId: "validation-session",
+  method: "POST",
+  url: "https://api.example.com/auth/login",
+  data: { email: "test@example.com", password: "test123" },
+  expect: { status: 200 },
+  extract: { token: "access_token" }
+})
+
+// Check session status
+await tools.api_session_status({
+  sessionId: "validation-session"
+})
+
+// Generate validation report
+await tools.api_session_report({
+  sessionId: "validation-session",
+  outputPath: "./validation-report.html"
+})
+```
+
+## Manual Test Creation (Fallback)
+
+When automatic generation needs refinement:
+
+```javascript
+// Create custom test files
+await tools.edit_createFile({
+  path: "./tests/custom-api-test.spec.ts",
+  content: `import { test, expect } from '@playwright/test';
+
+test.describe('Custom API Tests', () => {
+  test('should validate custom scenario', async ({ request }) => {
+    const response = await request.get('https://api.example.com/custom');
+    expect(response.status()).toBe(200);
+  });
+});`
+})
+```
+
+## Best Practices
+
+### DO:
+- Always call `api_project_setup` first
+- Let tool auto-detect configuration when possible
+- Extract specific sections when user requests them
+- Validate generated tests with api_request
+- Provide clear feedback about detected configuration
+- Use consistent sessionId for related operations
+- Generate comprehensive reports
+
+### DON'T:
+- Don't skip project setup detection
+- Don't guess configuration - detect or ask
+- Don't generate without confirmation if detection fails
+- Don't ignore user's explicit format preferences
+- Don't generate without preserving test plan structure
+
+## Error Handling
+
+If test generation fails:
+1. Check if project setup was called first
+2. Verify test plan format is correct
+3. Ensure configuration matches project structure
+4. Try manual file creation as fallback
+5. Provide clear error messages to user
+
+## Generated Test Structure
+
+### Playwright Example:
+```typescript
+import { test, expect } from '@playwright/test';
+
+test.describe('Books API', () => {
+  test('GET /api/v1/Books - should return all books', async ({ request }) => {
+    const response = await request.get('https://api.example.com/api/v1/Books');
+    expect(response.ok()).toBeTruthy();
+    expect(response.status()).toBe(200);
+    const books = await response.json();
+    expect(Array.isArray(books)).toBeTruthy();
+  });
+});
+```
+
+### Jest Example:
+```typescript
+import axios from 'axios';
+
+describe('Books API', () => {
+  const baseUrl = 'https://api.example.com';
+  
+  test('GET /api/v1/Books - should return all books', async () => {
+    const response = await axios.get(`${baseUrl}/api/v1/Books`);
+    expect(response.status).toBe(200);
+    expect(Array.isArray(response.data)).toBe(true);
+  });
+});
+```
+
+## Additional Tools Available
+
+Use these file manipulation tools when needed:
+- `search/fileSearch` - Find test files
+- `search/textSearch` - Search within files
+- `search/listDirectory` - List test directory
+- `search/readFile` - Read existing tests
+- `edit/createFile` - Create new test files
+- `edit/editFiles` - Modify existing tests
+
+## Troubleshooting
+
+### Project setup not detecting configuration
+- Verify config files exist in correct location
+- Check file names match expected patterns
+- Manually ask user for preferences
+
+### Generated tests don't match project structure
+- Ensure project setup was called first
+- Verify outputFormat matches project framework
+- Check language setting matches project
+
+### Tests fail to run after generation
+- Validate with api_request tool
+- Check API endpoint availability
+- Verify authentication configuration

package/src/skills/test-healing/SKILL.md

@@ -0,0 +1,405 @@
+---
+name: test-healing
+description: Debug and automatically fix failing API tests using intelligent analysis and healing strategies. Repairs authentication issues, endpoint changes, schema mismatches, and assertion failures. Use when tests are failing, broken, or need debugging after API changes.
+---
+
+# API Test Healing Skill
+
+Use this skill to automatically diagnose and fix failing API tests through systematic analysis and intelligent healing strategies.
+
+## When to Use This Skill
+
+- User mentions failing tests or broken tests
+- Tests are not passing after API changes
+- Authentication errors in API tests
+- Schema mismatches or validation failures
+- User needs to debug API test failures
+- Tests need updating after API deployment
+
+## Core Workflow
+
+### Step 1: Automated Healing (Primary Approach)
+
+**ALWAYS use `api_healer` tool first** for automated healing:
+
+```javascript
+// Heal specific test file
+await tools.api_healer({
+  testPath: "./tests/api-tests.spec.js",
+  testType: "auto",                    // jest, playwright, postman, auto
+  sessionId: "healing-session",
+  maxHealingAttempts: 3,
+  autoFix: true,
+  backupOriginal: true,
+  healingStrategies: [
+    "schema-update",
+    "endpoint-fix",
+    "auth-repair",
+    "data-correction",
+    "assertion-update"
+  ]
+})
+
+// Heal multiple test files
+await tools.api_healer({
+  testFiles: [
+    "./tests/auth.test.js",
+    "./tests/users.test.js",
+    "./tests/orders.test.js"
+  ],
+  autoFix: true,
+  analysisOnly: false  // set to true for analysis without fixing
+})
+```
+
+### Step 2: Healing Strategies
+
+The `api_healer` tool implements these automatic strategies:
+
+#### Schema Update
+- Detects API response schema changes
+- Updates test assertions to match current API structure
+- Fixes property name changes and type mismatches
+
+#### Endpoint Fix
+- Identifies 404 errors from changed endpoints
+- Attempts to discover correct endpoint URLs
+- Updates test configurations with working endpoints
+
+#### Authentication Repair
+- Fixes expired or invalid authentication tokens
+- Updates authentication methods and headers
+- Repairs OAuth flows and API key issues
+
+#### Data Correction
+- Fixes request payload validation errors
+- Updates test data to match current API requirements
+- Corrects data types and required fields
+
+#### Assertion Update
+- Updates test expectations based on actual API responses
+- Fixes assertion mismatches and validation rules
+- Aligns test expectations with current API behavior
+
+### Step 3: Manual Healing (When Automated Healing Needs Assistance)
+
+#### Analysis Mode
+```javascript
+// Use analysis-only mode first
+await tools.api_healer({
+  testPath: "./failing-test.js",
+  analysisOnly: true,
+  sessionId: "analysis-session"
+})
+```
+
+#### Diagnostic Testing
+```javascript
+// Test authentication manually
+await tools.api_request({
+  sessionId: "diagnostic-session",
+  method: "POST",
+  url: "https://api.example.com/auth/login",
+  data: { email: "test@example.com", password: "test123" },
+  expect: { status: [200, 401] },
+  extract: { token: "access_token" }
+})
+
+// Test failing endpoint with auth
+await tools.api_request({
+  sessionId: "diagnostic-session",
+  method: "GET",
+  url: "https://api.example.com/protected-resource",
+  headers: { "Authorization": "Bearer {{token}}" },
+  expect: { status: [200, 401, 403, 404] }
+})
+```
+
+#### Targeted Fixes
+```javascript
+// Apply specific healing strategies
+await tools.api_healer({
+  testPath: "./failing-test.js",
+  healingStrategies: ["auth-repair"],  // Target specific issues
+  maxHealingAttempts: 1,
+  autoFix: true
+})
+```
+
+## Common Failure Categories
+
+### 1. Authentication Failures
+**Symptoms:** 401 Unauthorized, 403 Forbidden responses
+
+**Causes:**
+- Expired tokens
+- Invalid credentials
+- Missing authorization headers
+
+**Healing Actions:**
+- Refresh authentication flow
+- Update token generation
+- Fix header format
+
+### 2. Request Format Issues
+**Symptoms:** 400 Bad Request, validation errors
+
+**Causes:**
+- Invalid JSON
+- Missing required fields
+- Incorrect data types
+
+**Healing Actions:**
+- Update request payload structure
+- Fix field mappings
+- Correct data types
+
+### 3. Response Validation Mismatches
+**Symptoms:** Test assertions failing on response content
+
+**Causes:**
+- API schema changes
+- New fields added/removed
+- Data format updates
+
+**Healing Actions:**
+- Update validation rules
+- Adjust expected response structure
+- Fix assertion patterns
+
+### 4. Endpoint Changes
+**Symptoms:** 404 Not Found, method not allowed errors
+
+**Causes:**
+- API versioning
+- Endpoint deprecation
+- URL structure changes
+
+**Healing Actions:**
+- Update endpoint URLs
+- Change HTTP methods
+- Handle API versioning
+
+### 5. Data Dependencies
+**Symptoms:** Tests failing due to missing/invalid test data
+
+**Causes:**
+- Test data cleanup
+- External service dependencies
+- Race conditions
+
+**Healing Actions:**
+- Improve test data setup
+- Add proper cleanup
+- Handle async operations
+
+## Debugging Methodology
+
+### 1. Parse Error Messages
+- Analyze validation failures, HTTP errors, timeout issues
+- Examine differences between expected and actual responses
+- Identify error patterns
+
+### 2. Compare Expected vs Actual
+- Look at response structure differences
+- Check data type mismatches
+- Identify missing or extra fields
+
+### 3. Trace Request Flow
+- Follow the complete request/response cycle
+- Identify break points in the flow
+- Check intermediate steps
+
+### 4. Check Dependencies
+- Verify prerequisite API calls work
+- Validate test data setup
+- Confirm authentication is valid
+
+## Session Management and Reporting
+
+```javascript
+// Check healing session status
+await tools.api_session_status({
+  sessionId: "healing-session"
+})
+
+// Generate healing report
+await tools.api_session_report({
+  sessionId: "healing-session",
+  outputPath: "./healing-report.html"
+})
+```
+
+## Best Practices
+
+### DO:
+- Use api_healer tool immediately for automated fixing
+- Create backups before making changes (default: true)
+- Analyze patterns in multiple failing tests
+- Apply targeted healing strategies when possible
+- Validate fixes by re-running tests
+- Document changes made during healing
+- Use session tracking for comprehensive analysis
+
+### DON'T:
+- Don't manually edit tests before trying automated healing
+- Don't ignore error patterns across multiple tests
+- Don't skip analysis mode for complex failures
+- Don't apply all strategies blindly
+- Don't forget to verify fixes work
+- Don't lose track of what was changed
+
+## Standard Test Pattern Examples
+
+### Jest Test Structure
+```javascript
+const axios = require('axios');
+
+describe('API Test Suite', () => {
+  const baseUrl = 'https://api.example.com/v1';
+  let authToken;
+
+  beforeAll(async () => {
+    // Setup code
+    const authResponse = await axios.post(`${baseUrl}/auth/login`, {
+      username: 'test',
+      password: 'password'
+    });
+    authToken = authResponse.data.token;
+  });
+
+  test('should get data successfully', async () => {
+    const response = await axios.get(`${baseUrl}/data`, {
+      headers: { 'Authorization': `Bearer ${authToken}` }
+    });
+    
+    expect(response.status).toBe(200);
+    expect(response.data).toHaveProperty('id');
+  });
+});
+```
+
+### Playwright Test Structure
+```typescript
+import { test, expect } from '@playwright/test';
+
+test.describe('API Test Suite', () => {
+  let authToken: string;
+
+  test.beforeAll(async ({ request }) => {
+    const response = await request.post('/auth/login', {
+      data: { username: 'test', password: 'password' }
+    });
+    const data = await response.json();
+    authToken = data.token;
+  });
+
+  test('should get data successfully', async ({ request }) => {
+    const response = await request.get('/data', {
+      headers: { 'Authorization': `Bearer ${authToken}` }
+    });
+    
+    expect(response.ok()).toBeTruthy();
+    const data = await response.json();
+    expect(data).toHaveProperty('id');
+  });
+});
+```
+
+## Common Fix Patterns
+
+### Authentication Fix
+```javascript
+// Before (failing)
+const response = await axios.get('/protected-endpoint');
+
+// After (fixed with proper auth)
+const response = await axios.get('/protected-endpoint', {
+  headers: {
+    'Authorization': `Bearer ${authToken}`,
+    'Content-Type': 'application/json'
+  }
+});
+```
+
+### Response Validation Fix
+```javascript
+// Before (failing due to wrong expectations)
+expect(response.data.items).toHaveLength(10);
+
+// After (fixed based on actual API response)
+expect(Array.isArray(response.data.items)).toBe(true);
+expect(response.data.items.length).toBeGreaterThan(0);
+```
+
+### Error Handling Improvement
+```javascript
+// Before (basic error handling)
+try {
+  const response = await axios.get('/endpoint');
+  expect(response.status).toBe(200);
+} catch (error) {
+  throw error;
+}
+
+// After (proper error handling)
+test('should handle 404 errors correctly', async () => {
+  await expect(axios.get('/nonexistent-endpoint'))
+    .rejects
+    .toMatchObject({
+      response: {
+        status: 404
+      }
+    });
+});
+```
+
+## Output Requirements
+
+When healing tests, provide:
+
+1. **Detailed failure analysis** - Categorize error types
+2. **Applied healing strategies** - List what was attempted
+3. **Fixed test files** - Show modified code with backups
+4. **Healing report** - Summary of changes and success rate
+5. **Recommendations** - Suggest how to prevent similar issues
+6. **Verification status** - Confirm tests now pass
+
+## Error Resolution Strategy
+
+- Make reasonable assumptions and fix issues automatically
+- Fix one issue at a time and re-test to verify
+- Use standard test patterns (Jest/Playwright conventions)
+- Preserve test coverage while updating for API changes
+- Improve test maintainability during fixes
+- If API is fundamentally broken, mark tests as skipped with comments
+- Continue until all tests pass or are properly documented
+
+## Additional Tools Available
+
+Use these tools for manual investigation when needed:
+- `search/fileSearch` - Find test files
+- `search/textSearch` - Search for error patterns
+- `search/readFile` - Read test file contents
+- `edit/editFiles` - Manually fix tests after analysis
+- `search/listDirectory` - List test directories
+
+## Troubleshooting
+
+### Automated healing not working
+- Check if test file path is correct
+- Verify test type detection is accurate
+- Try analysis mode first to understand issues
+- Apply targeted strategies instead of all
+
+### Tests still failing after healing
+- Use api_request for manual endpoint testing
+- Check if API is actually working
+- Validate authentication separately
+- Consider if API changes are breaking
+
+### Cannot identify failure cause
+- Use analysis-only mode
+- Check session status for more details
+- Generate healing report for comprehensive view
+- Test API endpoints manually with api_request