playwright-ai-reporter 0.0.4

package/docs/PROVIDERS.md

@@ -0,0 +1,261 @@
+# Playwright AI Test Reporter - Provider-Based Architecture
+
+## Overview
+
+This reporter has been refactored into a generalized provider-based framework that supports multiple implementations for each service type. This architecture allows you to easily switch between different service providers without changing the core reporter code.
+
+## Provider Types
+
+### 1. AI Providers
+
+The reporter supports multiple AI providers for generating test failure suggestions and auto-healing:
+
+- **Azure OpenAI** - Uses Azure's OpenAI service with managed identity or PAT authentication
+- **OpenAI** - Direct integration with OpenAI's API
+- **Anthropic** - Claude models for AI suggestions
+- **Google AI** - Gemini models
+- **Mistral AI** - Mistral's AI models
+
+**Configuration:**
+
+```env
+AI_PROVIDER=azure-openai  # or: openai, anthropic, google-ai, mistral
+```
+
+### 2. Bug Tracker Providers
+
+Create bugs/issues automatically for failing tests:
+
+- **Azure DevOps** - Create work items in Azure DevOps
+- **GitHub** - Create issues in GitHub repositories
+- **Jira** - Create tickets in Jira
+
+**Configuration:**
+
+```env
+BUG_TRACKER_PROVIDER=github  # or: azure-devops, jira
+```
+
+### 3. Database Providers
+
+Store test run and test result data:
+
+- **SQLite** - Lightweight local database
+- **MySQL** - MySQL database
+- **PostgreSQL** - PostgreSQL database (coming soon)
+
+**Configuration:**
+
+```env
+DATABASE_PROVIDER=sqlite  # or: mysql, postgresql
+```
+
+### 4. PR Providers
+
+Auto-generate pull requests with fixes:
+
+- **GitHub** - Create PRs in GitHub repositories
+- **Azure DevOps** - Create PRs in Azure Repos
+
+**Configuration:**
+
+```env
+PR_PROVIDER=github  # or: azure-devops
+```
+
+### 5. Notification Providers
+
+Send notifications about test results:
+
+- **Email** - Send email notifications
+- **Slack** - Post to Slack channels (coming soon)
+- **Microsoft Teams** - Post to Teams channels (coming soon)
+
+**Configuration:**
+
+```env
+NOTIFICATION_PROVIDER=email  # or: slack, teams
+```
+
+## Usage
+
+### Basic Setup
+
+1. Copy `.env.example` to `.env`:
+
+```bash
+cp examples/env-configs/.env.example .env
+```
+
+2. Configure your providers in `.env`:
+
+```env
+AI_PROVIDER=azure-openai
+BUG_TRACKER_PROVIDER=github
+DATABASE_PROVIDER=sqlite
+PR_PROVIDER=github
+NOTIFICATION_PROVIDER=email
+```
+
+3. Add provider-specific configuration (API keys, endpoints, etc.)
+
+### Programmatic Configuration
+
+You can also configure providers programmatically:
+
+```typescript
+import {ProviderRegistry} from './providers/ProviderRegistry';
+
+// Initialize with custom configuration
+await ProviderRegistry.initialize({
+    ai: {
+        type: 'openai',
+        config: {apiKey: 'your-api-key'},
+    },
+    bugTracker: {
+        type: 'github',
+        config: {token: 'your-token'},
+    },
+    database: {
+        type: 'sqlite',
+        config: {dbPath: './test-results.db'},
+    },
+});
+
+// Get providers
+const aiProvider = await ProviderRegistry.getAIProvider();
+const bugTracker = await ProviderRegistry.getBugTrackerProvider();
+const database = await ProviderRegistry.getDatabaseProvider();
+```
+
+### Using Individual Factories
+
+You can also use individual factories directly:
+
+```typescript
+import {AIProviderFactory, AIProviderType} from './providers/ai/AIProviderFactory';
+
+// Create a specific provider
+const aiProvider = await AIProviderFactory.createProvider(AIProviderType.OpenAI);
+
+// Or create from environment variable
+const aiProvider = await AIProviderFactory.createFromEnv('AI_PROVIDER');
+```
+
+## Architecture
+
+```
+src/
+├── providers/
+│   ├── interfaces/           # Provider interfaces
+│   │   ├── IAIProvider.ts
+│   │   ├── IBugTrackerProvider.ts
+│   │   ├── IDatabaseProvider.ts
+│   │   ├── INotificationProvider.ts
+│   │   └── IPRProvider.ts
+│   ├── ai/                   # AI provider implementations
+│   │   ├── AzureOpenAIProvider.ts
+│   │   ├── OpenAIProvider.ts
+│   │   ├── AnthropicProvider.ts
+│   │   ├── GoogleAIProvider.ts
+│   │   ├── MistralProvider.ts
+│   │   └── AIProviderFactory.ts
+│   ├── bugTrackers/          # Bug tracker implementations
+│   │   ├── AzureDevOpsBugTracker.ts
+│   │   ├── GitHubBugTracker.ts
+│   │   └── JiraBugTracker.ts
+│   ├── databases/            # Database implementations
+│   │   ├── SQLiteProvider.ts
+│   │   └── MySQLProvider.ts
+│   ├── notifications/        # Notification implementations
+│   │   └── EmailNotificationProvider.ts
+│   ├── pr/                   # PR provider implementations
+│   │   ├── GitHubPRProvider.ts
+│   │   └── AzureDevOpsPRProvider.ts
+│   ├── factories/            # Factory classes
+│   │   ├── BugTrackerFactory.ts
+│   │   ├── DatabaseFactory.ts
+│   │   ├── NotificationFactory.ts
+│   │   └── PRProviderFactory.ts
+│   ├── ProviderRegistry.ts   # Central provider registry
+│   └── index.ts              # Provider exports
+```
+
+## Adding New Providers
+
+### 1. Create Provider Implementation
+
+```typescript
+// src/providers/ai/NewAIProvider.ts
+import {IAIProvider, AIMessage, AIModelConfig, AIResponse} from '../interfaces/IAIProvider';
+
+export class NewAIProvider implements IAIProvider {
+    getName(): string {
+        return 'New AI Provider';
+    }
+
+    async initialize(): Promise<void> {
+        // Initialize connection, load config, etc.
+    }
+
+    async generateCompletion(messages: AIMessage[], config?: AIModelConfig): Promise<AIResponse> {
+        // Implementation
+    }
+
+    async generateText(prompt: string, config?: AIModelConfig): Promise<string> {
+        // Implementation
+    }
+
+    async testConnection(): Promise<boolean> {
+        // Test the connection
+    }
+}
+```
+
+### 2. Update Factory
+
+```typescript
+// Add to AIProviderFactory.ts
+import { NewAIProvider } from './NewAIProvider';
+
+export enum AIProviderType {
+  // ... existing types
+  NewProvider = 'new-provider',
+}
+
+// Add case in createProvider method
+case AIProviderType.NewProvider:
+  provider = new NewAIProvider();
+  break;
+```
+
+### 3. Update Documentation
+
+Add the new provider to `.env.example` and this README.
+
+## Environment Variables Reference
+
+See example files in `examples/env-configs/` for a complete list of all supported environment variables for each provider type.
+
+## Features
+
+- **Flexible Provider System**: Easily switch between different service providers
+- **Factory Pattern**: Centralized provider creation and management
+- **Lazy Initialization**: Providers are only initialized when needed
+- **Type Safety**: Full TypeScript support with proper interfaces
+- **Environment-Based Config**: Configure via environment variables or programmatically
+- **Extensible**: Easy to add new providers without changing existing code
+
+## Testing
+
+Each provider can be tested independently:
+
+```typescript
+const provider = await AIProviderFactory.createProvider('openai');
+const isConnected = await provider.testConnection();
+console.log(`Connection status: ${isConnected}`);
+```
+
+## License
+
+See LICENSE file for details.

package/docs/QUICKSTART.md

@@ -0,0 +1,350 @@
+# Quick Start Guide - Playwright AI Test Reporter
+
+## Installation
+
+```bash
+npm install @azure/identity
+```
+
+## Configuration
+
+### 1. Create Environment File
+
+Copy one of the pre-configured example files:
+
+```bash
+# For GitHub users (recommended for open source)
+cp examples/env-configs/.env.github-stack .env
+
+# For Azure users (recommended for enterprise)
+cp examples/env-configs/.env.azure-stack .env
+
+# For Jira users
+cp examples/env-configs/.env.openai-jira .env
+
+# For minimal setup
+cp examples/env-configs/.env.anthropic-minimal .env
+```
+
+### 2. Configure Your Providers
+
+Edit `.env` and set your provider choices:
+
+```env
+# Minimal configuration for getting started
+AI_PROVIDER=azure-openai
+BUG_TRACKER_PROVIDER=github
+DATABASE_PROVIDER=sqlite
+
+# Enable features
+ENABLE_BUG_CREATION=true
+ENABLE_DATABASE_LOGGING=true
+ENABLE_AUTO_HEALING=true
+ENABLE_PR_CREATION=false
+```
+
+### 3. Add Provider-Specific Configuration
+
+#### For Azure OpenAI (Recommended for Enterprise)
+
+```env
+AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
+AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4
+AZURE_OPENAI_API_VERSION=2024-02-01
+```
+
+Uses Azure managed identity - no API key needed!
+
+#### For OpenAI (Quick Start)
+
+```env
+AI_PROVIDER=openai
+OPENAI_API_KEY=sk-...
+OPENAI_MODEL=gpt-4o
+```
+
+#### For GitHub Bug Tracking
+
+```env
+GITHUB_TOKEN=ghp_...
+GITHUB_OWNER=your-username
+GITHUB_REPO=your-repo
+```
+
+### 4. Validate Configuration
+
+Run the configuration validator:
+
+```bash
+npm run validate:config
+```
+
+This will check all your provider configurations and test connections.
+
+## Usage
+
+### Basic Setup in Playwright Config
+
+```typescript
+// playwright.config.ts
+import {defineConfig} from '@playwright/test';
+
+export default defineConfig({
+    reporter: [
+        [
+            './src/reporter.ts',
+            {
+                // Reporter options
+            },
+        ],
+    ],
+});
+```
+
+### Using the Provider Framework
+
+```typescript
+import {ProviderRegistry} from './src/providers/ProviderRegistry';
+
+// Initialize providers (loads from .env)
+await ProviderRegistry.initialize();
+
+// Get AI provider
+const ai = await ProviderRegistry.getAIProvider();
+const response = await ai.generateText('Analyze this test failure...');
+
+// Get bug tracker
+const bugTracker = await ProviderRegistry.getBugTrackerProvider();
+await bugTracker.createBug({
+    title: 'Test Failed',
+    description: 'Details...',
+    priority: BugPriority.High,
+});
+
+// Get database
+const db = await ProviderRegistry.getDatabaseProvider();
+await db.saveTestRun({
+    name: 'Test Run',
+    timestamp: new Date(),
+    totalTests: 10,
+    passedTests: 8,
+    failedTests: 2,
+});
+```
+
+### Using Individual Factories
+
+```typescript
+import {AIProviderFactory} from './src/providers/ai/AIProviderFactory';
+
+// Create specific provider
+const ai = await AIProviderFactory.createProvider('openai');
+
+// Or create from environment
+const ai = await AIProviderFactory.createFromEnv('AI_PROVIDER');
+```
+
+## Supported Providers
+
+### AI Providers
+
+- ✅ Azure OpenAI (recommended for enterprise)
+- ✅ OpenAI
+- ✅ Anthropic (Claude)
+- ✅ Google AI (Gemini)
+- ✅ Mistral AI
+
+### Bug Trackers
+
+- ✅ GitHub Issues
+- ✅ Azure DevOps Work Items
+- ✅ Jira
+
+### Databases
+
+- ✅ SQLite (default, no setup needed)
+- ✅ MySQL
+- ⏳ PostgreSQL (coming soon)
+
+### Notifications
+
+- ✅ Email (SMTP)
+- ⏳ Slack (coming soon)
+- ⏳ Microsoft Teams (coming soon)
+
+### PR Providers
+
+- ✅ GitHub
+- ✅ Azure DevOps
+
+## Common Configurations
+
+### Minimal Setup (SQLite + OpenAI)
+
+```env
+AI_PROVIDER=openai
+OPENAI_API_KEY=sk-...
+DATABASE_PROVIDER=sqlite
+ENABLE_DATABASE_LOGGING=true
+```
+
+### GitHub Integration
+
+```env
+AI_PROVIDER=openai
+OPENAI_API_KEY=sk-...
+
+BUG_TRACKER_PROVIDER=github
+GITHUB_TOKEN=ghp_...
+GITHUB_OWNER=your-username
+GITHUB_REPO=your-repo
+
+PR_PROVIDER=github
+# Uses same GITHUB_* vars
+
+ENABLE_BUG_CREATION=true
+ENABLE_PR_CREATION=false
+```
+
+### Azure DevOps Integration
+
+```env
+AI_PROVIDER=azure-openai
+AZURE_OPENAI_ENDPOINT=https://...
+AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4
+
+BUG_TRACKER_PROVIDER=azure-devops
+AZURE_DEVOPS_ORG_URL=https://dev.azure.com/your-org
+AZURE_DEVOPS_PROJECT=YourProject
+AZURE_DEVOPS_PAT=your-pat
+
+PR_PROVIDER=azure-devops
+# Uses same AZURE_DEVOPS_* vars
+
+ENABLE_BUG_CREATION=true
+```
+
+### Enterprise Setup (Azure + MySQL)
+
+```env
+AI_PROVIDER=azure-openai
+AZURE_OPENAI_ENDPOINT=https://...
+AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4
+
+DATABASE_PROVIDER=mysql
+MYSQL_HOST=mysql.example.com
+MYSQL_USER=testuser
+MYSQL_PASSWORD=secure-password
+MYSQL_DATABASE=test_results
+
+BUG_TRACKER_PROVIDER=azure-devops
+AZURE_DEVOPS_ORG_URL=https://dev.azure.com/enterprise
+AZURE_DEVOPS_PROJECT=QA
+
+NOTIFICATION_PROVIDER=email
+EMAIL_HOST=smtp.office365.com
+EMAIL_USER=notifications@company.com
+EMAIL_PASSWORD=app-password
+EMAIL_FROM=notifications@company.com
+EMAIL_TO=qa-team@company.com
+
+ENABLE_BUG_CREATION=true
+ENABLE_DATABASE_LOGGING=true
+ENABLE_NOTIFICATIONS=true
+```
+
+## Running Tests
+
+```bash
+# Run Playwright tests with the reporter
+npm run test:e2e
+
+# With UI
+npm run test:e2e:ui
+
+# With debugging
+npm run test:e2e:debug
+```
+
+## Workflow Example
+
+See `src/examples/ReporterWorkflow.ts` for a complete example of:
+
+1. Generating AI fix suggestions
+2. Creating bugs for failures
+3. Auto-generating PRs with fixes
+4. Saving results to database
+5. Sending notifications
+
+## Troubleshooting
+
+### Configuration Issues
+
+Run the validator:
+
+```bash
+npm run validate:config
+```
+
+### Provider Connection Failed
+
+Check:
+
+1. API keys are correct
+2. Endpoints are accessible
+3. Network/firewall settings
+4. Service status (Azure/OpenAI/etc.)
+
+### Azure OpenAI Authentication
+
+Ensure you're logged in with Azure CLI:
+
+```bash
+az login
+```
+
+Or Azure PowerShell:
+
+```powershell
+Connect-AzAccount
+```
+
+### Database Connection Issues
+
+For SQLite:
+
+- No setup needed, file created automatically
+
+For MySQL:
+
+- Check host, port, credentials
+- Ensure database exists
+- Verify network access
+
+## Next Steps
+
+1. ✅ Validate configuration with `npm run validate:config`
+2. ✅ Run a test to verify reporter works
+3. ✅ Check test-results directory for output
+4. ✅ Review generated prompts and fixes
+5. ✅ Configure additional providers as needed
+
+## Documentation
+
+- [PROVIDERS.md](./PROVIDERS.md) - Detailed provider documentation
+- [IMPLEMENTATION_SUMMARY.md](./IMPLEMENTATION_SUMMARY.md) - Architecture overview
+- [examples/env-configs/](../examples/env-configs/) - Pre-configured environment files
+- [examples/tests/](../examples/tests/) - Sample test files
+
+## Support
+
+For issues or questions:
+
+1. Check the documentation
+2. Run configuration validator
+3. Review example workflow
+4. Check provider-specific documentation
+
+## License
+
+See LICENSE file for details.