@democratize-quality/mcp-server 1.1.2 → 1.1.4

package/docs/test-agents.md

@@ -0,0 +1,673 @@
+# API Test Agents
+
+## Introduction
+
+Democratize Quality MCP Server comes with three powerful AI-driven API Test Agents: **🌐 api-planner**, **🌐 api-generator**, and **🌐 api-healer**.
+
+These agents can be used independently, sequentially, or in a chained workflow to produce comprehensive API test coverage for your applications. Using them together creates a complete test automation pipeline:
+
+- **🌐 api-planner** analyzes API schemas (REST/GraphQL) and produces comprehensive test plans
+- **🌐 api-generator** transforms test plans into executable tests (Playwright, Jest, Postman)
+- **🌐 api-healer** diagnoses and automatically repairs failing tests
+
+## Getting Started
+
+### Installation
+
+Install the MCP server with AI testing agents for GitHub Copilot:
+
+```bash
+npx @democratize-quality/mcp-server@latest --agents
+```
+
+**What this does:**
+- ✅ Installs the MCP server
+- ✅ Sets up 3 AI-powered testing agents
+- ✅ Configures VS Code integration automatically
+- ✅ Creates project folders (`.github/chatmodes/`, `.vscode/`)
+
+Once the agents are installed, you can use GitHub Copilot Chat to command these agents to build API tests for both REST and GraphQL APIs.
+
+### Prerequisites
+
+- Node.js 14+ installed
+- VS Code with GitHub Copilot extension
+- Active internet connection
+
+---
+
+## 🌐 api-planner
+
+The **planner agent** analyzes your API (REST or GraphQL) and produces a comprehensive test plan covering multiple scenarios and user flows.
+
+### Input
+
+- An API schema URL (OpenAPI/Swagger JSON/YAML, or GraphQL introspection endpoint)
+- Clear instructions about what to test (e.g., "Generate a plan for Books and Authors endpoints")
+- (Optional) Authentication requirements
+- (Optional) Specific test categories (functional, security, performance)
+
+### Usage
+
+**Example: REST API**
+
+```
+@🌐 api-planner analyze the OpenAPI spec at 
+https://fakerestapi.azurewebsites.net/swagger/v1/swagger.json 
+and create a comprehensive test plan focusing on Books and Authors endpoints
+```
+
+**Example: GraphQL API**
+
+```
+@🌐 api-planner analyze the GitHub GraphQL API at 
+https://api.github.com/graphql 
+and create a test plan for repository queries and mutations
+```
+
+### What It Does
+
+1. **Schema Analysis**: Fetches and parses the API schema
+2. **Endpoint Discovery**: Identifies all available endpoints/operations
+3. **Test Scenario Generation**: Creates test cases for:
+   - Happy paths (successful operations)
+   - Error cases (404, 400, 401, 403)
+   - Edge cases (empty lists, boundary values)
+   - Data validation scenarios
+4. **Sample Data Generation**: Creates realistic test data using 70+ semantic patterns
+5. **Optional Validation**: Can make actual API calls to validate endpoints
+
+### Output
+
+A Markdown test plan saved as `./api-test-reports/[api-name]-test-plan.md` containing:
+
+- Test scenarios organized by endpoint/operation
+- Sample request bodies with realistic data
+- Expected response structures
+- Error scenarios and edge cases
+- Response time metrics (if validation enabled)
+
+**Example Test Plan Section (REST):**
+
+```markdown
+### GET /api/v1/Books - Happy Path
+
+**Endpoint:** `GET https://fakerestapi.azurewebsites.net/api/v1/Books`
+
+**Expected Response (200 OK):**
+```json
+[
+  {
+    "id": 1,
+    "title": "The Great Gatsby",
+    "description": "A classic American novel",
+    "pageCount": 180,
+    "publishDate": "1925-04-10T00:00:00Z"
+  }
+]
+```
+
+**Example Test Plan Section (GraphQL):**
+
+```markdown
+### Query: getRepository - Happy Path
+
+**Endpoint:** `POST https://api.github.com/graphql`
+
+**Request Body:**
+```json
+{
+  "query": "query GetRepo($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { name description stargazerCount } }",
+  "variables": {
+    "owner": "microsoft",
+    "name": "playwright"
+  }
+}
+```
+
+**Expected Response:**
+```json
+{
+  "data": {
+    "repository": {
+      "name": "playwright",
+      "description": "Playwright is a framework for Web Testing and Automation",
+      "stargazerCount": 50000
+    }
+  }
+}
+```
+```
+
+---
+
+## 🌐 api-generator
+
+The **generator agent** transforms test plans into executable test code. It supports multiple frameworks and can generate tests in TypeScript or JavaScript.
+
+### Input
+
+- Test plan markdown file (generated by api-planner)
+- Target framework (Playwright, Jest, or Postman)
+- Programming language (TypeScript or JavaScript)
+- Output directory for tests
+
+### Usage
+
+**Example: Generate Playwright Tests**
+
+```
+@🌐 api-generator create Playwright tests in TypeScript from the test plan, 
+focusing on the Books API section
+```
+
+**Example: Generate Jest Tests**
+
+```
+@🌐 api-generator create Jest tests in JavaScript for the GraphQL operations 
+in the test plan
+```
+
+**Example: Generate Postman Collection**
+
+```
+@🌐 api-generator create a Postman collection from the entire test plan 
+with all endpoints and example requests
+```
+
+### What It Does
+
+1. **Project Detection**: Automatically detects framework and language from your project
+2. **Code Generation**: Creates executable tests based on the test plan
+3. **Selector Verification**: Validates endpoint URLs and request structures
+4. **Framework-Specific Code**: Generates idiomatic code for each framework
+5. **Type Safety**: Includes TypeScript types when applicable
+
+### Output
+
+**Playwright Tests:**
+```typescript
+// tests/books.spec.ts
+import { test, expect } from '@playwright/test';
+
+const BASE_URL = 'https://fakerestapi.azurewebsites.net';
+
+test.describe('Books API', () => {
+  test('GET /api/v1/Books - should return all books', async ({ request }) => {
+    const response = await request.get(`${BASE_URL}/api/v1/Books`);
+    expect(response.ok()).toBeTruthy();
+    expect(response.status()).toBe(200);
+    const books = await response.json();
+    expect(Array.isArray(books)).toBeTruthy();
+  });
+  
+  test('POST /api/v1/Books - should create a new book', async ({ request }) => {
+    const newBook = {
+      id: 201,
+      title: "Test Book",
+      pageCount: 350
+    };
+    const response = await request.post(`${BASE_URL}/api/v1/Books`, {
+      data: newBook
+    });
+    expect(response.status()).toBe(200);
+  });
+});
+```
+
+**Jest Tests:**
+```javascript
+// tests/books.test.js
+const axios = require('axios');
+
+const BASE_URL = 'https://fakerestapi.azurewebsites.net';
+
+describe('Books API', () => {
+  test('GET /api/v1/Books - should return all books', async () => {
+    const response = await axios.get(`${BASE_URL}/api/v1/Books`);
+    expect(response.status).toBe(200);
+    expect(Array.isArray(response.data)).toBeTruthy();
+  });
+  
+  test('POST /api/v1/Books - should create a new book', async () => {
+    const newBook = {
+      id: 201,
+      title: "Test Book",
+      pageCount: 350
+    };
+    const response = await axios.post(`${BASE_URL}/api/v1/Books`, newBook);
+    expect(response.status).toBe(200);
+  });
+});
+```
+
+**Postman Collection:**
+```json
+{
+  "info": {
+    "name": "Fake REST API",
+    "schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
+  },
+  "item": [
+    {
+      "name": "Books",
+      "item": [
+        {
+          "name": "Get All Books",
+          "request": {
+            "method": "GET",
+            "url": "{{baseUrl}}/api/v1/Books"
+          }
+        }
+      ]
+    }
+  ]
+}
+```
+
+### GraphQL Test Generation
+
+The generator automatically detects GraphQL operations and generates appropriate test code:
+
+**Playwright GraphQL Test:**
+```typescript
+test('Query: getRepository - should fetch repository details', async ({ request }) => {
+  const response = await request.post('https://api.github.com/graphql', {
+    headers: {
+      'Content-Type': 'application/json',
+      'Authorization': 'Bearer YOUR_TOKEN'
+    },
+    data: {
+      query: `query GetRepo($owner: String!, $name: String!) {
+        repository(owner: $owner, name: $name) {
+          name
+          description
+          stargazerCount
+        }
+      }`,
+      variables: {
+        owner: 'microsoft',
+        name: 'playwright'
+      }
+    }
+  });
+  
+  expect(response.status()).toBe(200);
+  const responseData = await response.json();
+  expect(responseData).toHaveProperty('data');
+  expect(responseData.data.repository.name).toBe('playwright');
+});
+```
+
+---
+
+## 🌐 api-healer
+
+When tests fail due to API changes, the **healer agent** automatically diagnoses and repairs them.
+
+### Input
+
+- Path to failing test file(s)
+- Test framework type (auto-detected if not specified)
+- Healing strategies (optional)
+
+### Usage
+
+**Example: Fix Failing Tests**
+
+```
+@🌐 api-healer the Books tests are failing with 404 errors. 
+Please analyze and fix them.
+```
+
+**Example: Fix Schema Mismatches**
+
+```
+@🌐 api-healer fix assertion errors in tests/users.spec.ts
+```
+
+### What It Does
+
+1. **Error Analysis**: Replays failing tests to identify issues
+2. **Root Cause Detection**: Determines why tests are failing:
+   - Endpoint URL changes
+   - Schema mismatches (new/removed fields)
+   - Data type changes
+   - Authentication issues
+   - Status code changes
+3. **Automatic Repair**: Applies fixes based on healing strategies
+4. **Verification**: Re-runs tests to ensure they pass
+5. **Backup Creation**: Saves original files before making changes
+
+### Healing Strategies
+
+| Strategy | What It Fixes |
+|----------|---------------|
+| `endpoint-fix` | Updates changed endpoint URLs and API versions |
+| `schema-update` | Corrects response schema validation (new/removed fields) |
+| `auth-repair` | Fixes authentication headers and token issues |
+| `data-correction` | Adjusts test data types (string vs number, etc.) |
+| `assertion-update` | Updates assertions to match new API behavior |
+
+### Output
+
+**Analysis Results:**
+```
+Analyzing test failures in tests/books.spec.ts...
+
+Issues found:
+❌ Endpoint changed: /api/Books → /api/v1/Books
+❌ Response schema updated: added "pageCount" field
+❌ ID format changed: number → string
+
+Applying fixes:
+✅ Updated all endpoint URLs to include /v1/ prefix
+✅ Updated response assertions to include pageCount
+✅ Fixed ID type in test data (42 → "42")
+✅ Backed up original file to tests/books.spec.ts.backup
+
+Re-running tests... ✅ All 15 tests now passing!
+```
+
+**Before Healing:**
+```typescript
+test('GET book by ID', async ({ request }) => {
+  const response = await request.get(`${BASE_URL}/api/Books/1`);
+  expect(response.status()).toBe(200);
+  const book = await response.json();
+  expect(book).toHaveProperty('id');
+  expect(book).toHaveProperty('title');
+  // Missing pageCount assertion
+});
+```
+
+**After Healing:**
+```typescript
+test('GET book by ID', async ({ request }) => {
+  const response = await request.get(`${BASE_URL}/api/v1/Books/1`);
+  expect(response.status()).toBe(200);
+  const book = await response.json();
+  expect(book).toHaveProperty('id');
+  expect(book).toHaveProperty('title');
+  expect(book).toHaveProperty('pageCount'); // Added
+});
+```
+
+---
+
+## Quick Start Examples
+
+### Example 1: Complete REST API Testing Workflow
+
+**Testing the Fake REST API** (https://fakerestapi.azurewebsites.net)
+
+**Step 1: Create Test Plan**
+```
+@🌐 api-planner analyze the OpenAPI spec at 
+https://fakerestapi.azurewebsites.net/swagger/v1/swagger.json 
+and create a comprehensive test plan focusing on Books and Authors endpoints
+```
+
+**Step 2: Generate Tests**
+```
+@🌐 api-generator create Playwright tests in TypeScript from the test plan, 
+focusing on the Books API section
+```
+
+**Step 3: Run Tests**
+```bash
+npx playwright test tests/books.spec.ts
+```
+
+**Step 4: Fix Failures (if any)**
+```
+@🌐 api-healer fix the failing tests in tests/books.spec.ts
+```
+
+### Example 2: Complete GraphQL API Testing Workflow
+
+**Testing GitHub's GraphQL API** (https://api.github.com/graphql)
+
+**Step 1: Create Test Plan**
+```
+@🌐 api-planner analyze the GitHub GraphQL API and create a test plan 
+for repository queries including name, description, and stargazer count
+```
+
+**Step 2: Generate Tests**
+```
+@🌐 api-generator create Playwright tests in TypeScript for the 
+GraphQL repository queries from the test plan
+```
+
+**Step 3: Run Tests**
+```bash
+GITHUB_TOKEN=your_token npx playwright test tests/github-api.spec.ts
+```
+
+**Step 4: Fix Schema Changes (if any)**
+```
+@🌐 api-healer fix tests/github-api.spec.ts to handle schema updates
+```
+
+### Example 3: Generate Postman Collection
+
+**Quick API Documentation Workflow**
+
+**Step 1: Generate Test Plan with Validation**
+```
+@🌐 api-planner create test plan from 
+https://fakerestapi.azurewebsites.net/swagger/v1/swagger.json
+with validation enabled to test actual endpoints
+```
+
+**Step 2: Generate Postman Collection**
+```
+@🌐 api-generator create Postman collection from the test plan 
+with all endpoints and example requests
+```
+
+**Step 3: Import and Share**
+- Import `postman/FakeRestAPI.postman_collection.json` into Postman
+- Share with your team for manual testing and documentation
+
+---
+
+## Artifacts and Conventions
+
+The agents follow a structured, auditable file organization:
+
+```
+project/
+  .github/
+    chatmodes/              # Agent definitions (AI prompts)
+      🌐 api-planner.chatmode.md
+      🌐 api-generator.chatmode.md
+      🌐 api-healer.chatmode.md
+  .vscode/
+    mcp.json                # MCP server configuration
+  api-test-reports/         # Generated test plans and reports
+    fakerest-test-plan.md
+    github-test-plan.md
+    validation-report.html
+  tests/                    # Generated test files
+    books.spec.ts
+    authors.spec.ts
+    github-api.spec.ts
+  postman/                  # Generated Postman collections
+    FakeRestAPI.postman_collection.json
+    environment.json
+```
+
+### Agent Definitions
+
+Agent definitions are markdown files containing:
+- Instructions for the AI agent
+- MCP tool references
+- Best practices and examples
+- Context about the testing workflow
+
+They are automatically created by the `--agents` installation and should be committed to version control.
+
+### Test Plans in `api-test-reports/`
+
+Test plans are structured documents describing:
+- API endpoints/operations
+- Test scenarios (happy path, errors, edge cases)
+- Sample request/response data
+- Expected outcomes
+- Validation results (if enabled)
+
+Test plans are human-readable and can be reviewed before code generation.
+
+### Tests in `tests/`
+
+Generated test files aligned with the test plan structure:
+- One test file per API resource/domain
+- Organized by test categories
+- Include setup/teardown when needed
+- Follow framework best practices
+
+---
+
+## Features
+
+### Supported API Types
+
+✅ **REST APIs** - OpenAPI 2.0/3.0, Swagger  
+✅ **GraphQL APIs** - Introspection, SDL schemas  
+✅ **Hybrid APIs** - Mix of REST and GraphQL endpoints
+
+### Supported Test Frameworks
+
+✅ **Playwright** - Modern web automation framework  
+✅ **Jest** - Popular JavaScript testing framework  
+✅ **Postman** - API testing and documentation tool
+
+### Smart Test Data Generation
+
+The planner uses **70+ semantic field patterns** to generate realistic test data:
+
+| Field Pattern | Generated Value |
+|---------------|-----------------|
+| `email`, `userEmail` | "user@example.com" |
+| `name`, `fullName` | "John Doe" |
+| `phone`, `phoneNumber` | "+1-555-123-4567" |
+| `createdAt`, `updatedAt` | "2025-10-25T12:00:00Z" |
+| `price`, `amount` | 583.39 |
+| `url`, `imageUrl` | "https://example.com/image.jpg" |
+| `description`, `summary` | Meaningful text content |
+| `status`, `state` | Valid enum values |
+
+### GraphQL-Specific Features
+
+✅ **Full Type System Support** - OBJECT, LIST, NON_NULL, ENUM, UNION, INTERFACE, SCALAR  
+✅ **Custom Scalar Detection** - DateTime, URL, Email, JSON  
+✅ **Circular Reference Protection** - Handles complex schemas safely  
+✅ **Query/Mutation/Subscription Support** - All GraphQL operations  
+✅ **Variable Extraction** - Generates typed variables  
+✅ **Fragment Support** - Reusable query fragments
+
+---
+
+## Advanced Usage
+
+### Custom Configuration
+
+Set environment variables for advanced control:
+
+```bash
+# Output directory for test plans and reports
+OUTPUT_DIR=./custom-reports
+
+# Enable debug mode
+MCP_FEATURES_ENABLEDEBUGMODE=true
+
+# Node environment
+NODE_ENV=production
+```
+
+### CI/CD Integration
+
+Use agents in automated pipelines:
+
+```yaml
+# .github/workflows/api-tests.yml
+name: API Tests
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      - uses: actions/setup-node@v3
+        with:
+          node-version: '18'
+      
+      - name: Install dependencies
+        run: npm ci
+      
+      - name: Run API tests
+        run: npx playwright test tests/
+        env:
+          API_TOKEN: ${{ secrets.API_TOKEN }}
+```
+
+### Multi-Environment Testing
+
+Generate tests for different environments:
+
+```
+@🌐 api-generator create Jest tests with environment-specific configurations
+for dev (api-dev.example.com), staging (api-staging.example.com), 
+and prod (api.example.com) environments
+```
+
+---
+
+## Video Tutorial
+
+<div style="position: relative; padding-bottom: 56.25%; height: 0;">
+  <iframe src="https://www.youtube.com/watch?v=aEpdALTMCos" frameborder="0" allowfullscreen 
+    style="position: absolute; top: 0; left: 0; width: 100%; height: 100%;">
+  </iframe>
+</div>
+
+*Watch a complete walkthrough of using the API Test Agents to build comprehensive test coverage.*
+
+---
+
+## Learn More
+
+- [Getting Started Guide](./getting-started.md) - Complete setup walkthrough
+- [Tool Reference](./api/tool-reference.md) - Detailed API documentation
+- [GraphQL Support Guide](../GRAPHQL-SUPPORT.md) - GraphQL-specific features
+- [API Tools Usage Guide](./api_tools_usage.md) - Advanced patterns and examples
+
+---
+
+## Community
+
+- 📦 [NPM Package](https://www.npmjs.com/package/@democratize-quality/mcp-server)
+- 🐛 [GitHub Issues](https://github.com/uppadhyayraj/democratize-quality-mcp-server/issues)
+- ⭐ [Star on GitHub](https://github.com/uppadhyayraj/democratize-quality-mcp-server)
+
+---
+
+## License
+
+MIT License
+
+Copyright © 2025 Democratize Quality
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+**THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.**
+
+---
+
+*This documentation is provided for informational purposes. Users are responsible for ensuring their API testing practices comply with applicable laws, terms of service, and ethical guidelines.*

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@democratize-quality/mcp-server",
-  "version": "1.1.2",
+  "version": "1.1.4",
   "main": "mcpServer.js",
   "bin": {
     "democratize-quality-mcp": "cli.js",
@@ -75,6 +75,7 @@
     "chrome-launcher": "^1.2.0",
     "chrome-remote-interface": "^0.33.3",
     "express": "^5.1.0",
+    "graphql": "^16.11.0",
     "json-rpc-2.0": "^1.7.1",
     "yaml": "^2.8.1",
     "zod": "^4.0.10"