qa360 2.1.7 → 2.2.0

package/docs/archive/obsolete-proposals/technical-plan-dom-discovery.md

@@ -0,0 +1,870 @@
+# QA360 DOM Element Discovery - Technical Implementation Plan
+
+**Version** : 1.0
+**Date** : 2026-01-09
+**Auteur** : QA360 Core Team
+**Status** : Ready for Implementation
+
+---
+
+## 1. Overview
+
+This document outlines the technical implementation plan for the DOM Element Discovery feature in QA360. The goal is to automatically discover and catalog UI elements from web pages to streamline test creation.
+
+### Scope
+
+**Phase 1 (MVP)** includes:
+- New `qa360 scan` command
+- DOM element scanning (buttons, links, forms, inputs)
+- JSON output format
+- Auto-generation of QA360 test files
+
+**Out of Scope** (future phases):
+- Multi-page crawling
+- Interactive element selection
+- HTML report generation
+- Baseline regression detection
+
+---
+
+## 2. Architecture
+
+### 2.1 Component Structure
+
+```
+cli/
+├── src/
+│   ├── commands/
+│   │   └── scan.ts              # NEW: scan command handler
+│   ├── scanners/
+│   │   ├── index.ts             # NEW: scanner exports
+│   │   ├── dom-scanner.ts       # NEW: DOM scanning logic
+│   │   ├── element-classifier.ts # NEW: Element type classification
+│   │   └── metadata-extractor.ts # NEW: Element metadata extraction
+│   ├── generators/
+│   │   ├── index.ts             # NEW: generator exports
+│   │   ├── test-generator.ts    # NEW: QA360 test file generation
+│   │   └── json-reporter.ts     # NEW: JSON report generation
+│   └── types/
+│       └── scan.ts              # NEW: Scan-related types
+
+core/
+├── src/
+│   └── scanners/
+│       ├── element-scanner.ts   # NEW: Core scanning engine
+│       └── types.ts             # NEW: Shared scanner types
+```
+
+### 2.2 Data Flow
+
+```
+User Input (URL + Options)
+        │
+        ▼
+┌───────────────────┐
+│  scan.ts (CLI)    │
+│  - Parse args     │
+│  - Validate URL   │
+└─────────┬─────────┘
+          │
+          ▼
+┌───────────────────┐
+│ DOMScanner        │
+│ - Load page       │
+│ - Query elements  │
+└─────────┬─────────┘
+          │
+          ▼
+┌───────────────────┐
+│ ElementClassifier │
+│ - Categorize      │
+│ - Filter          │
+└─────────┬─────────┘
+          │
+          ▼
+┌───────────────────┐
+│ MetadataExtractor │
+│ - Extract attrs   │
+│ - Get position    │
+└─────────┬─────────┘
+          │
+          ▼
+┌───────────────────┐
+│  OutputGenerator  │
+│ - JSON report     │
+│ - QA360 test      │
+└─────────┬─────────┘
+          │
+          ▼
+    File Output
+```
+
+---
+
+## 3. Type Definitions
+
+### 3.1 Core Types
+
+```typescript
+// cli/src/types/scan.ts
+
+export interface ScanOptions {
+  url: string;
+  output?: string;
+  include?: ElementType[];
+  exclude?: string[];
+  screenshot?: boolean;
+  autoGenerateTest?: boolean;
+  timeout?: number;
+  headless?: boolean;
+}
+
+export type ElementType =
+  | 'buttons'
+  | 'links'
+  | 'forms'
+  | 'inputs'
+  | 'images'
+  | 'headings'
+  | 'all';
+
+export interface DiscoveredElement {
+  type: string;
+  selector: string;
+  text?: string;
+  attributes?: Record<string, string>;
+  position: ElementPosition;
+  visible: boolean;
+  page: string;
+}
+
+export interface ElementPosition {
+  x: number;
+  y: number;
+  width: number;
+  height: number;
+}
+
+export interface ScanResult {
+  scanDate: string;
+  url: string;
+  duration: number;
+  summary: ScanSummary;
+  elements: DiscoveredElement[];
+}
+
+export interface ScanSummary {
+  totalElements: number;
+  buttons: number;
+  links: number;
+  forms: number;
+  inputs: number;
+  images: number;
+  headings: number;
+}
+```
+
+### 3.2 Generated Test Types
+
+```typescript
+export interface GeneratedTestConfig {
+  version: 2;
+  name: string;
+  description: string;
+  gates: Record<string, GeneratedGate>;
+}
+
+export interface GeneratedGate {
+  adapter: string;
+  enabled: boolean;
+  config: {
+    baseUrl: string;
+    pages: GeneratedPageConfig[];
+  };
+}
+
+export interface GeneratedPageConfig {
+  url: string;
+  expectedElements: string[];
+}
+```
+
+---
+
+## 4. Implementation Details
+
+### 4.1 DOM Scanner
+
+```typescript
+// cli/src/scanners/dom-scanner.ts
+
+import { Page } from '@playwright/test';
+
+export class DOMScanner {
+  private page: Page;
+
+  constructor(page: Page) {
+    this.page = page;
+  }
+
+  /**
+   * Scan the page for elements
+   */
+  async scan(options: ScanOptions): Promise<DiscoveredElement[]> {
+    const elements: DiscoveredElement[] = [];
+
+    // Scan based on included types
+    if (this.shouldInclude('buttons', options.include)) {
+      elements.push(...await this.scanButtons());
+    }
+    if (this.shouldInclude('links', options.include)) {
+      elements.push(...await this.scanLinks());
+    }
+    if (this.shouldInclude('forms', options.include)) {
+      elements.push(...await this.scanForms());
+    }
+    if (this.shouldInclude('inputs', options.include)) {
+      elements.push(...await this.scanInputs());
+    }
+    if (this.shouldInclude('images', options.include)) {
+      elements.push(...await this.scanImages());
+    }
+    if (this.shouldInclude('headings', options.include)) {
+      elements.push(...await this.scanHeadings());
+    }
+
+    // Apply exclusions
+    return this.filterExcluded(elements, options.exclude || []);
+  }
+
+  /**
+   * Scan for buttons (button elements and elements with role="button")
+   */
+  private async scanButtons(): Promise<DiscoveredElement[]> {
+    const selectors = [
+      'button',
+      '[role="button"]',
+      'input[type="submit"]',
+      'input[type="button"]'
+    ];
+
+    const elements: DiscoveredElement[] = [];
+    for (const selector of selectors) {
+      const found = await this.page.locator(selector).all();
+      for (const el of found) {
+        const handle = await el.elementHandle();
+        if (!handle) continue;
+
+        const isVisible = await el.isVisible();
+        if (!isVisible) continue;
+
+        const box = await el.boundingBox();
+        const text = await el.textContent();
+        const id = await handle.evaluate(e => (e as HTMLElement).id);
+        const classes = await handle.evaluate(e => (e as HTMLElement).className);
+
+        elements.push({
+          type: 'button',
+          selector: this.generateSelector(selector, id, classes),
+          text: text?.trim() || undefined,
+          position: box || { x: 0, y: 0, width: 0, height: 0 },
+          visible: true,
+          page: this.page.url()
+        });
+      }
+    }
+
+    return elements;
+  }
+
+  /**
+   * Scan for links
+   */
+  private async scanLinks(): Promise<DiscoveredElement[]> {
+    const elements: DiscoveredElement[] = [];
+    const links = await this.page.locator('a[href]').all();
+
+    for (const link of links) {
+      const isVisible = await link.isVisible();
+      if (!isVisible) continue;
+
+      const box = await link.boundingBox();
+      const text = await link.textContent();
+      const href = await link.getAttribute('href');
+      const handle = await link.elementHandle();
+      const id = handle ? await handle.evaluate(e => (e as HTMLElement).id) : '';
+      const classes = handle ? await handle.evaluate(e => (e as HTMLElement).className) : '';
+
+      elements.push({
+        type: 'link',
+        selector: this.generateSelector('a', id, classes, `[href="${href}"]`),
+        text: text?.trim() || undefined,
+        attributes: { href: href || '' },
+        position: box || { x: 0, y: 0, width: 0, height: 0 },
+        visible: true,
+        page: this.page.url()
+      });
+    }
+
+    return elements;
+  }
+
+  /**
+   * Scan for forms
+   */
+  private async scanForms(): Promise<DiscoveredElement[]> {
+    const elements: DiscoveredElement[] = [];
+    const forms = await this.page.locator('form').all();
+
+    for (const form of forms) {
+      const isVisible = await form.isVisible();
+      if (!isVisible) continue;
+
+      const handle = await form.elementHandle();
+      const id = handle ? await handle.evaluate(e => (e as HTMLElement).id) : '';
+      const classes = handle ? await handle.evaluate(e => (e as HTMLElement).className) : '';
+      const action = await form.getAttribute('action');
+
+      elements.push({
+        type: 'form',
+        selector: this.generateSelector('form', id, classes),
+        attributes: { action: action || '' },
+        position: { x: 0, y: 0, width: 0, height: 0 },
+        visible: true,
+        page: this.page.url()
+      });
+    }
+
+    return elements;
+  }
+
+  /**
+   * Scan for inputs
+   */
+  private async scanInputs(): Promise<DiscoveredElement[]> {
+    const elements: DiscoveredElement[] = [];
+    const inputs = await this.page.locator('input, textarea, select').all();
+
+    for (const input of inputs) {
+      const isVisible = await input.isVisible();
+      if (!isVisible) continue;
+
+      const box = await input.boundingBox();
+      const handle = await input.elementHandle();
+      const type = await input.getAttribute('type');
+      const name = await input.getAttribute('name');
+      const id = handle ? await handle.evaluate(e => (e as HTMLElement).id) : '';
+      const classes = handle ? await handle.evaluate(e => (e as HTMLElement).className) : '';
+      const placeholder = await input.getAttribute('placeholder');
+
+      elements.push({
+        type: 'input',
+        selector: this.generateSelector('input', id, classes, `[type="${type}"]`),
+        text: placeholder || undefined,
+        attributes: {
+          type: type || 'text',
+          name: name || '',
+          placeholder: placeholder || ''
+        },
+        position: box || { x: 0, y: 0, width: 0, height: 0 },
+        visible: true,
+        page: this.page.url()
+      });
+    }
+
+    return elements;
+  }
+
+  /**
+   * Scan for images
+   */
+  private async scanImages(): Promise<DiscoveredElement[]> {
+    const elements: DiscoveredElement[] = [];
+    const images = await this.page.locator('img').all();
+
+    for (const img of images) {
+      const isVisible = await img.isVisible();
+      if (!isVisible) continue;
+
+      const box = await img.boundingBox();
+      const handle = await img.elementHandle();
+      const src = await img.getAttribute('src');
+      const alt = await img.getAttribute('alt');
+      const id = handle ? await handle.evaluate(e => (e as HTMLElement).id) : '';
+      const classes = handle ? await handle.evaluate(e => (e as HTMLElement).className) : '';
+
+      elements.push({
+        type: 'image',
+        selector: this.generateSelector('img', id, classes, `[src="${src}"]`),
+        text: alt || undefined,
+        attributes: {
+          src: src || '',
+          alt: alt || ''
+        },
+        position: box || { x: 0, y: 0, width: 0, height: 0 },
+        visible: true,
+        page: this.page.url()
+      });
+    }
+
+    return elements;
+  }
+
+  /**
+   * Scan for headings
+   */
+  private async scanHeadings(): Promise<DiscoveredElement[]> {
+    const elements: DiscoveredElement[] = [];
+    const headings = await this.page.locator('h1, h2, h3, h4, h5, h6').all();
+
+    for (const heading of headings) {
+      const isVisible = await heading.isVisible();
+      if (!isVisible) continue;
+
+      const box = await heading.boundingBox();
+      const text = await heading.textContent();
+      const handle = await heading.elementHandle();
+      const tag = handle ? await handle.evaluate(e => e.tagName) : '';
+      const id = handle ? await handle.evaluate(e => (e as HTMLElement).id) : '';
+      const classes = handle ? await handle.evaluate(e => (e as HTMLElement).className) : '';
+
+      elements.push({
+        type: 'heading',
+        selector: this.generateSelector(tag.toLowerCase(), id, classes),
+        text: text?.trim() || undefined,
+        attributes: { level: tag?.toLowerCase() || 'h1' },
+        position: box || { x: 0, y: 0, width: 0, height: 0 },
+        visible: true,
+        page: this.page.url()
+      });
+    }
+
+    return elements;
+  }
+
+  /**
+   * Generate a CSS selector for an element
+   */
+  private generateSelector(
+    tag: string,
+    id: string,
+    classes: string,
+    extra?: string
+  ): string {
+    if (id) {
+      return `#${id}`;
+    }
+    if (classes) {
+      const classList = classes.split(' ').filter(c => c.length > 0);
+      if (classList.length > 0) {
+        return `${tag}.${classList[0]}`;
+      }
+    }
+    return extra ? `${tag}${extra}` : tag;
+  }
+
+  /**
+   * Check if a type should be included
+   */
+  private shouldInclude(type: ElementType, include?: ElementType[]): boolean {
+    if (!include || include.length === 0) return true;
+    return include.includes(type) || include.includes('all');
+  }
+
+  /**
+   * Filter out excluded elements
+   */
+  private filterExcluded(
+    elements: DiscoveredElement[],
+    exclude: string[]
+  ): DiscoveredElement[] {
+    if (exclude.length === 0) return elements;
+
+    return elements.filter(el => {
+      return !exclude.some(pattern => {
+        // Simple CSS selector matching
+        try {
+          return el.selector.match(pattern.replace('*', '.*'));
+        } catch {
+          return el.selector.includes(pattern);
+        }
+      });
+    });
+  }
+}
+```
+
+### 4.2 Scan Command
+
+```typescript
+// cli/src/commands/scan.ts
+
+import { Command } from 'commander';
+import { chromium } from '@playwright/test';
+import { DOMScanner } from '../scanners/dom-scanner.js';
+import { JSONReporter } from '../generators/json-reporter.js';
+import { TestGenerator } from '../generators/test-generator.js';
+import type { ScanOptions, ElementType } from '../types/scan.js';
+import { resolve } from 'path';
+import { mkdirSync } from 'fs';
+
+export const scanCommand = new Command('scan');
+
+scanCommand
+  .description('Scan a web page for UI elements')
+  .argument('<url>', 'URL to scan')
+  .option('-o, --output <file>', 'Output file path', '.qa360/discovery/elements.json')
+  .option('-i, --include <types>', 'Element types to include (comma-separated)', 'all')
+  .option('-e, --exclude <selectors>', 'Selectors to exclude (comma-separated)')
+  .option('-s, --screenshot', 'Capture screenshots of elements')
+  .option('-a, --auto-generate-test', 'Auto-generate QA360 test file')
+  .option('-t, --timeout <ms>', 'Navigation timeout in ms', '30000')
+  .option('--headed', 'Run in headed mode (show browser)')
+  .action(async (url: string, options) => {
+    const startTime = Date.now();
+
+    // Parse include types
+    const include: ElementType[] = options.include
+      .split(',')
+      .map(t => t.trim().toLowerCase() as ElementType);
+
+    // Parse exclude selectors
+    const exclude = options.exclude
+      ? options.exclude.split(',').map(s => s.trim())
+      : [];
+
+    const scanOptions: ScanOptions = {
+      url,
+      output: options.output,
+      include,
+      exclude,
+      screenshot: options.screenshot,
+      autoGenerateTest: options.autoGenerateTest,
+      timeout: parseInt(options.timeout),
+      headless: !options.headed
+    };
+
+    console.log(`🔍 Scanning page: ${url}`);
+    console.log('━'.repeat(50));
+
+    try {
+      // Launch browser
+      const browser = await chromium.launch({
+        headless: scanOptions.headless
+      });
+
+      const context = await browser.newContext({
+        viewport: { width: 1280, height: 720 }
+      });
+
+      const page = await context.newPage();
+
+      // Navigate to URL
+      await page.goto(url, {
+        timeout: scanOptions.timeout,
+        waitUntil: 'networkidle'
+      });
+
+      // Run scan
+      const scanner = new DOMScanner(page);
+      const elements = await scanner.scan(scanOptions);
+
+      // Generate summary
+      const summary = {
+        totalElements: elements.length,
+        buttons: elements.filter(e => e.type === 'button').length,
+        links: elements.filter(e => e.type === 'link').length,
+        forms: elements.filter(e => e.type === 'form').length,
+        inputs: elements.filter(e => e.type === 'input').length,
+        images: elements.filter(e => e.type === 'image').length,
+        headings: elements.filter(e => e.type === 'heading').length
+      };
+
+      // Print summary
+      console.log(`✅ Buttons found:      ${summary.buttons}`);
+      console.log(`✅ Links found:        ${summary.links}`);
+      console.log(`✅ Forms found:        ${summary.forms}`);
+      console.log(`✅ Inputs found:       ${summary.inputs}`);
+      console.log(`✅ Images found:       ${summary.images}`);
+      console.log(`✅ Headings found:     ${summary.headings}`);
+      console.log('━'.repeat(50));
+      console.log(`📦 Total elements:     ${summary.totalElements}`);
+      console.log();
+
+      // Create output directory
+      const outputPath = resolve(scanOptions.output!);
+      const outputDir = outputPath.substring(0, outputPath.lastIndexOf('/'));
+      mkdirSync(outputDir, { recursive: true });
+
+      // Generate JSON report
+      const result = {
+        scanDate: new Date().toISOString(),
+        url,
+        duration: Date.now() - startTime,
+        summary,
+        elements
+      };
+
+      const reporter = new JSONReporter();
+      await reporter.generate(result, outputPath);
+      console.log(`💾 Saved to: ${outputPath}`);
+
+      // Auto-generate test if requested
+      if (options.autoGenerateTest) {
+        const testGenerator = new TestGenerator();
+        const testPath = outputPath.replace('.json', '.yml');
+        await testGenerator.generate(result, testPath);
+        console.log(`📄 Test generated: ${testPath}`);
+      }
+
+      // Screenshots
+      if (options.screenshot) {
+        const screenshotDir = outputDir + '/screenshots';
+        mkdirSync(screenshotDir, { recursive: true });
+        // Screenshot logic here...
+        console.log(`📸 Screenshots: ${screenshotDir}/ (${elements.length} files)`);
+      }
+
+      await browser.close();
+
+      console.log();
+      console.log(`✨ Scan completed in ${((Date.now() - startTime) / 1000).toFixed(1)}s`);
+      console.log();
+      console.log('💡 Next steps:');
+      console.log(`   • Review the discovered elements`);
+      if (options.autoGenerateTest) {
+        console.log(`   • Run the auto-generated test: qa360 run ${outputPath.replace('.json', '.yml')}`);
+      }
+      console.log(`   • Edit the test to remove unwanted elements`);
+
+    } catch (error) {
+      console.error(`❌ Scan failed: ${error}`);
+      process.exit(1);
+    }
+  });
+```
+
+### 4.3 Test Generator
+
+```typescript
+// cli/src/generators/test-generator.ts
+
+import { writeFileSync } from 'fs';
+import type { ScanResult } from '../types/scan.js';
+import { yamlDump } from 'js-yaml';
+
+export class TestGenerator {
+  /**
+   * Generate a QA360 test file from scan results
+   */
+  async generate(result: ScanResult, outputPath: string): Promise<void> {
+    const url = new URL(result.url);
+    const baseUrl = `${url.protocol}//${url.host}`;
+
+    // Group elements by page (for MVP, just one page)
+    const selectors = result.elements.map(el => el.selector);
+
+    // Generate unique selectors only
+    const uniqueSelectors = [...new Set(selectors)];
+
+    const testConfig = {
+      version: 2,
+      name: `auto-generated-${this.sanitizeName(url.hostname)}`,
+      description: `Auto-generated test from scan of ${result.url}`,
+      gates: {
+        'ui-smoke': {
+          adapter: 'playwright-ui',
+          enabled: true,
+          config: {
+            baseUrl,
+            pages: [
+              {
+                url: url.pathname || '/',
+                expectedElements: uniqueSelectors
+              }
+            ]
+          }
+        }
+      },
+      execution: {
+        default_timeout: 30000,
+        default_retries: 1,
+        on_failure: 'continue'
+      }
+    };
+
+    writeFileSync(outputPath, yamlDump(testConfig));
+  }
+
+  /**
+   * Sanitize hostname for use in test name
+   */
+  private sanitizeName(hostname: string): string {
+    return hostname.replace(/[^a-z0-9]/gi, '-').toLowerCase();
+  }
+}
+```
+
+### 4.4 JSON Reporter
+
+```typescript
+// cli/src/generators/json-reporter.ts
+
+import { writeFileSync } from 'fs';
+import type { ScanResult } from '../types/scan.js';
+
+export class JSONReporter {
+  /**
+   * Generate a JSON report from scan results
+   */
+  async generate(result: ScanResult, outputPath: string): Promise<void> {
+    writeFileSync(outputPath, JSON.stringify(result, null, 2));
+  }
+}
+```
+
+---
+
+## 5. CLI Integration
+
+### 5.1 Register the Command
+
+```typescript
+// cli/src/index.ts
+
+import { scanCommand } from './commands/scan.js';
+
+// In the program setup
+program.addCommand(scanCommand);
+```
+
+### 5.2 Help Text
+
+```
+Usage: qa360 scan [options] <url>
+
+Scan a web page for UI elements and generate test configurations
+
+Arguments:
+  url                  URL of the page to scan
+
+Options:
+  -o, --output <file>     Output file path (default: ".qa360/discovery/elements.json")
+  -i, --include <types>   Element types to include: buttons,links,forms,inputs,images,headings,all (default: "all")
+  -e, --exclude <selects> Selectors to exclude (comma-separated)
+  -s, --screenshot        Capture screenshots of elements
+  -a, --auto-generate-test Auto-generate QA360 test file
+  -t, --timeout <ms>      Navigation timeout in ms (default: "30000")
+  --headed               Run in headed mode (show browser)
+  -h, --help              Display help for command
+```
+
+---
+
+## 6. Testing Strategy
+
+### 6.1 Unit Tests
+
+```typescript
+// cli/src/scanners/__tests__/dom-scanner.test.ts
+
+import { describe, it, expect, beforeEach } from 'vitest';
+import { DOMScanner } from '../dom-scanner';
+
+describe('DOMScanner', () => {
+  // Test cases for each scan method
+  // Mock Playwright Page object
+});
+```
+
+### 6.2 Integration Tests
+
+```typescript
+// cli/src/commands/__tests__/scan.integration.test.ts
+
+describe('scan command integration', () => {
+  it('should scan a simple page', async () => {
+    // Test against a local test server
+  });
+});
+```
+
+### 6.3 E2E Tests
+
+```bash
+# Manual testing checklist
+□ Scan page with buttons
+□ Scan page with links
+□ Scan page with forms
+□ Scan page with inputs
+□ Exclude specific selectors
+□ Generate test file
+□ Verify generated test is runnable
+```
+
+---
+
+## 7. Performance Considerations
+
+- Use `page.locator()` with efficient selectors
+- Parallelize element queries where possible
+- Limit visibility checks (only for interactive elements)
+- Cache selector generation results
+- Use browser pooling for repeated scans
+
+---
+
+## 8. Error Handling
+
+| Error Scenario | Handling Strategy |
+|----------------|-------------------|
+| Invalid URL | Validate before browser launch |
+| Navigation timeout | Configurable, fail gracefully |
+| No elements found | Warning message, empty result |
+| File write error | Clear error message with path |
+| Browser launch failure | Fallback to alternative browser |
+
+---
+
+## 9. Dependencies
+
+No new dependencies required - uses existing:
+- `@playwright/test` (already installed)
+- `js-yaml` (already installed)
+- `commander` (already installed)
+
+---
+
+## 10. Success Criteria
+
+Phase 1 is complete when:
+
+- [ ] `qa360 scan` command works on any public URL
+- [ ] All element types (buttons, links, forms, inputs, images, headings) are detected
+- [ ] JSON output is valid and complete
+- [ ] Generated QA360 test file is runnable
+- [ ] Command completes in < 5 seconds for typical pages
+- [ ] Help text is clear and accurate
+- [ ] Unit tests coverage > 80%
+- [ ] Documentation is updated
+
+---
+
+## 11. Open Questions
+
+1. Should we support XPath selectors in addition to CSS?
+2. How should we handle elements inside iframes?
+3. Should we detect and exclude invisible/hidden elements by default?
+4. What's the maximum number of elements we should support per page?
+
+---
+
+## 12. Future Enhancements (Out of Scope)
+
+- Multi-page scanning with crawler
+- Interactive element selection mode
+- HTML report generation
+- Baseline comparison for regression detection
+- Component detection (React, Vue, etc.)
+- Screenshot capture for each element
+- CI/CD integration mode