ai-qakit 1.0.0

package/templates/.claude/commands/qakit/ask.md

@@ -0,0 +1,57 @@
+# QA Ask Command
+
+Ask QA-related questions and get expert guidance.
+
+## Usage
+
+```
+/qakit:ask <question>
+```
+
+## Instructions
+
+You are a QA expert assistant. When the user asks a question:
+
+1. **Understand the Context**
+   - Identify if this is about test strategy, automation, bug investigation, or tooling
+   - Consider the project's tech stack and testing frameworks
+
+2. **Provide Expert Guidance**
+   - Give clear, actionable answers
+   - Include code examples when relevant
+   - Reference best practices and patterns
+
+3. **Suggest Next Steps**
+   - Recommend relevant QA commands if applicable
+   - Point to skills or workflows that might help
+
+## Topics You Can Help With
+
+- **Test Strategy**: Test planning, coverage strategies, risk-based testing
+- **Test Automation**: Framework selection, patterns, best practices
+- **Bug Investigation**: Root cause analysis, reproduction strategies
+- **API Testing**: REST, GraphQL, contract testing
+- **UI Testing**: Playwright, WebdriverIO, visual testing
+- **Performance Testing**: Load testing, benchmarking, profiling
+- **Accessibility Testing**: WCAG compliance, a11y tools
+- **Test Data**: Data generation, fixtures, factories
+- **CI/CD Integration**: Test pipelines, reporting
+- **Quality Metrics**: Coverage, defect density, test effectiveness
+
+## Example Questions
+
+- "How should I structure my E2E tests for a React app?"
+- "What's the best approach for testing microservices?"
+- "How do I set up contract testing between services?"
+- "What metrics should I track for test quality?"
+- "How can I improve test execution time?"
+
+## Response Format
+
+When answering:
+1. Provide a clear, concise answer
+2. Include relevant code examples
+3. Suggest related commands or skills
+4. Offer to dive deeper if needed
+
+$ARGUMENTS

package/templates/.claude/commands/qakit/automation/fix-flaky.md

@@ -0,0 +1,233 @@
+# Fix Flaky Tests
+
+Analyze and fix flaky tests in automation suite.
+
+## Usage
+
+```
+/qakit:automation/fix-flaky [test-file]
+```
+
+## Instructions
+
+### Step 1: Identify Flaky Tests
+
+Scan test results to find flaky tests:
+
+```
+🔍 Flaky Test Analysis
+
+Found 5 flaky tests (failed intermittently in last 10 runs):
+
+| Test | File | Failures | Pattern |
+|------|------|----------|---------|
+| should login successfully | login.spec.ts:45 | 3/10 | Timing |
+| should add to cart | cart.spec.ts:78 | 2/10 | Race condition |
+| should search products | search.spec.ts:23 | 4/10 | Selector |
+| should checkout | checkout.spec.ts:112 | 2/10 | Data dependency |
+| should load dashboard | dashboard.spec.ts:34 | 3/10 | Network |
+```
+
+### Step 2: Analyze Root Cause
+
+For each flaky test, identify root cause:
+
+#### Common Flaky Patterns
+
+| Pattern | Symptoms | Solution |
+|---------|----------|----------|
+| **Timing** | Fails on slow CI, passes locally | Add explicit waits |
+| **Race Condition** | Random failures, no pattern | Use proper synchronization |
+| **Selector** | Element not found intermittently | Use stable selectors |
+| **Data Dependency** | Fails when data changes | Isolate test data |
+| **Network** | Timeout errors | Add retry logic |
+| **State Leakage** | Fails after other tests | Ensure test isolation |
+
+### Step 3: Generate Fix Recommendations
+
+For each flaky test:
+
+```markdown
+## Flaky Test: should login successfully
+
+**File:** `tests/specs/auth/login.spec.ts:45`
+**Failure Rate:** 30% (3/10 runs)
+**Pattern:** Timing issue
+
+### Current Code
+```typescript
+test('should login successfully', async ({ page }) => {
+  await page.goto('/login');
+  await page.fill('[data-testid="email"]', 'user@example.com');
+  await page.fill('[data-testid="password"]', 'password123');
+  await page.click('[data-testid="submit"]');
+  
+  // ❌ Problem: No wait for navigation
+  expect(page.url()).toContain('/dashboard');
+});
+```
+
+### Root Cause
+- Missing wait for navigation after form submission
+- Assertion runs before page redirects
+
+### Recommended Fix
+```typescript
+test('should login successfully', async ({ page }) => {
+  await page.goto('/login');
+  await page.fill('[data-testid="email"]', 'user@example.com');
+  await page.fill('[data-testid="password"]', 'password123');
+  
+  // ✅ Wait for navigation
+  await Promise.all([
+    page.waitForURL('**/dashboard'),
+    page.click('[data-testid="submit"]'),
+  ]);
+  
+  // ✅ Use Playwright's auto-waiting assertion
+  await expect(page).toHaveURL(/.*dashboard/);
+});
+```
+
+### Additional Recommendations
+1. Use Page Object pattern for better maintainability
+2. Add retry configuration in playwright.config.ts
+3. Enable trace on first retry for debugging
+```
+
+### Step 4: Apply Fixes
+
+Generate fixed test code:
+
+```typescript
+// Before (Flaky)
+test('should add to cart', async ({ page }) => {
+  await page.goto('/products');
+  await page.click('.product-card:first-child .add-to-cart');
+  const cartCount = await page.textContent('.cart-count');
+  expect(cartCount).toBe('1');
+});
+
+// After (Stable)
+test('should add to cart', async ({ page }) => {
+  await page.goto('/products');
+  
+  // Wait for products to load
+  await page.waitForSelector('.product-card');
+  
+  // Click and wait for cart update
+  const addButton = page.locator('.product-card').first().locator('.add-to-cart');
+  await addButton.click();
+  
+  // Use auto-waiting assertion
+  await expect(page.locator('.cart-count')).toHaveText('1');
+});
+```
+
+### Step 5: Verify Fix
+
+After applying fixes:
+
+```bash
+# Run the specific test multiple times
+npx playwright test login.spec.ts --repeat-each=10
+
+# Results
+✅ 10/10 passed
+```
+
+## Common Fixes by Pattern
+
+### Timing Issues
+
+```typescript
+// ❌ Bad: No wait
+await page.click('button');
+expect(await page.textContent('.result')).toBe('Success');
+
+// ✅ Good: Auto-waiting assertion
+await page.click('button');
+await expect(page.locator('.result')).toHaveText('Success');
+```
+
+### Race Conditions
+
+```typescript
+// ❌ Bad: Click without waiting for response
+await page.click('button');
+
+// ✅ Good: Wait for network response
+await Promise.all([
+  page.waitForResponse('**/api/submit'),
+  page.click('button'),
+]);
+```
+
+### Selector Issues
+
+```typescript
+// ❌ Bad: Fragile selector
+await page.click('.btn.primary.large');
+
+// ✅ Good: Stable selector
+await page.click('[data-testid="submit-button"]');
+// or
+await page.getByRole('button', { name: 'Submit' }).click();
+```
+
+### Data Dependencies
+
+```typescript
+// ❌ Bad: Depends on existing data
+test('should edit user', async ({ page }) => {
+  await page.goto('/users/123');
+});
+
+// ✅ Good: Create test data
+test('should edit user', async ({ page, request }) => {
+  // Create user via API
+  const user = await request.post('/api/users', {
+    data: { name: 'Test User' }
+  });
+  
+  await page.goto(`/users/${user.id}`);
+  
+  // Cleanup
+  await request.delete(`/api/users/${user.id}`);
+});
+```
+
+### Network Issues
+
+```typescript
+// ❌ Bad: No timeout handling
+await page.goto('/slow-page');
+
+// ✅ Good: Extended timeout with retry
+await page.goto('/slow-page', { timeout: 60000 });
+
+// Or in config
+export default defineConfig({
+  use: {
+    navigationTimeout: 60000,
+  },
+  retries: 2,
+});
+```
+
+## Output
+
+- Fixed test files with stable patterns
+- Flaky test report with root cause analysis
+- Recommendations for preventing future flakiness
+
+## Prevention Tips
+
+1. **Always use auto-waiting assertions**
+2. **Use stable selectors (data-testid, roles)**
+3. **Isolate test data**
+4. **Enable retries in CI**
+5. **Add traces for debugging**
+6. **Review tests in code review**
+
+$ARGUMENTS

package/templates/.claude/commands/qakit/automation/health-check.md

@@ -0,0 +1,192 @@
+# Automation Health Check
+
+Analyze automation framework health and generate maintenance recommendations.
+
+## Usage
+
+```
+/qakit:automation/health-check [--full]
+```
+
+## Options
+
+| Option | Description |
+|--------|-------------|
+| `--full` | Run comprehensive analysis (slower) |
+
+## Instructions
+
+### Step 1: Detect Framework
+
+Scan project to identify automation framework:
+- `playwright.config.ts` → Playwright
+- `wdio.*.conf.ts` → WebdriverIO
+- `karate-config.js` → Karate
+- `cypress.config.ts` → Cypress
+
+### Step 2: Analyze Project Structure
+
+Check for standard structure compliance:
+
+```
+✅ Structure Analysis
+├── Page Objects: Found (15 files)
+├── Test Specs: Found (42 files)
+├── Support/Helpers: Found (8 files)
+├── Config Files: Found (3 files)
+├── Test Data: Found (5 files)
+└── Reports Directory: Found
+```
+
+### Step 3: Check Dependencies
+
+Analyze package.json for:
+- Outdated dependencies
+- Security vulnerabilities
+- Missing peer dependencies
+
+```
+📦 Dependencies Analysis
+├── @playwright/test: 1.40.0 → 1.42.0 (update available)
+├── typescript: 5.3.0 ✅ (up to date)
+├── dotenv: 16.3.0 ✅ (up to date)
+└── Security: No vulnerabilities found
+```
+
+### Step 4: Analyze Test Health
+
+Review test execution patterns:
+
+```
+🧪 Test Health Analysis
+├── Total Specs: 42
+├── Flaky Tests: 3 (7.1%)
+│   ├── login.spec.ts:45 - Flaky (failed 3/10 runs)
+│   ├── checkout.spec.ts:78 - Flaky (failed 2/10 runs)
+│   └── search.spec.ts:23 - Flaky (failed 2/10 runs)
+├── Slow Tests: 5 (>30s)
+│   ├── e2e-flow.spec.ts - 45s
+│   └── data-import.spec.ts - 38s
+├── Skipped Tests: 2
+└── Test Coverage: 78%
+```
+
+### Step 5: Check Configuration
+
+Validate configuration files:
+
+```
+⚙️ Configuration Analysis
+├── Environment Variables:
+│   ├── WEB_BASE_URL: ✅ Set
+│   ├── TEST_USER_EMAIL: ✅ Set
+│   ├── TEST_USER_PASSWORD: ✅ Set
+│   ├── API_BASE_URL: ⚠️ Not set (optional)
+│   └── BROWSERSTACK_USERNAME: ⚠️ Not set (optional)
+├── Config Files:
+│   ├── playwright.config.ts: ✅ Valid
+│   ├── .env.example: ✅ Present
+│   └── tsconfig.json: ✅ Valid
+└── CI/CD:
+    ├── GitHub Actions: ✅ Found
+    └── Test workflow: ✅ Configured
+```
+
+### Step 6: Code Quality Analysis
+
+Check code quality:
+
+```
+📝 Code Quality Analysis
+├── Linting: 3 warnings, 0 errors
+├── TypeScript: No type errors
+├── Page Object Patterns:
+│   ├── Base class usage: ✅ 100%
+│   ├── Selector encapsulation: ✅ 95%
+│   └── Semantic methods: ⚠️ 80% (review needed)
+├── Test Patterns:
+│   ├── Proper assertions: ✅ 98%
+│   ├── No hard waits: ⚠️ 3 instances found
+│   └── Data isolation: ✅ 100%
+└── Documentation:
+    ├── README.md: ✅ Present
+    ├── Page object docs: ⚠️ 60% documented
+    └── Test data docs: ❌ Missing
+```
+
+### Step 7: Generate Health Report
+
+```markdown
+# Automation Health Report
+
+**Generated:** 2024-01-15 10:30:00
+**Framework:** Playwright
+**Overall Health Score:** 78/100 (Good)
+
+## Summary
+
+| Category | Score | Status |
+|----------|-------|--------|
+| Structure | 95/100 | ✅ Excellent |
+| Dependencies | 85/100 | ✅ Good |
+| Test Health | 70/100 | ⚠️ Needs Attention |
+| Configuration | 90/100 | ✅ Good |
+| Code Quality | 75/100 | ⚠️ Needs Attention |
+| Documentation | 60/100 | ⚠️ Needs Improvement |
+
+## Critical Issues (Fix Immediately)
+
+1. **3 Flaky Tests Detected**
+   - `login.spec.ts:45` - Add explicit waits
+   - `checkout.spec.ts:78` - Fix race condition
+   - `search.spec.ts:23` - Stabilize selector
+
+## Warnings (Address This Sprint)
+
+1. **Outdated Dependencies**
+   - Update @playwright/test to 1.42.0
+   
+2. **Hard Waits Found**
+   - `checkout.spec.ts:34` - Replace `waitForTimeout(5000)`
+   - `search.spec.ts:56` - Replace `waitForTimeout(3000)`
+   
+3. **Missing Documentation**
+   - Add test data documentation
+   - Document 6 undocumented page objects
+
+## Recommendations
+
+### This Week
+- [ ] Fix 3 flaky tests
+- [ ] Remove hard waits
+- [ ] Update dependencies
+
+### This Sprint
+- [ ] Add missing documentation
+- [ ] Improve page object semantic methods
+- [ ] Set up optional env variables
+
+### This Month
+- [ ] Increase test coverage to 85%
+- [ ] Add visual regression tests
+- [ ] Set up BrowserStack integration
+```
+
+## Output
+
+- `reports/automation-health-report.md` - Full health report
+- Console summary with actionable items
+
+## Maintenance Schedule
+
+Based on health check, recommend maintenance tasks:
+
+| Frequency | Tasks |
+|-----------|-------|
+| **Daily** | Monitor CI failures, update flaky tracker |
+| **Weekly** | Review new features, check selector drift |
+| **Sprint** | Run linting, audit configs, rotate test data |
+| **Monthly** | Update dependencies, rotate secrets |
+| **Quarterly** | Benchmark performance, refactor POMs |
+
+$ARGUMENTS