qa360 2.2.1 → 2.2.7

package/docs/pack-v2.md

@@ -1,558 +0,0 @@
-# QA360 Pack Configuration v2
-
-**QA360 Pack v2** is the next generation configuration format that adds comprehensive authentication profiles, dynamic gate naming, and improved flexibility while maintaining backward compatibility with v1 concepts.
-
-## Key Features
-
-- **Authentication Profiles**: Named, reusable authentication configurations
-- **Dynamic Gate Names**: Custom gate names instead of predefined types
-- **Multiple Auth Types**: JWT, OAuth2, API keys, Basic auth, UI login, TOTP, cloud provider credentials
-- **Per-Gate Auth Override**: Each gate can use its own auth profile
-- **Credential Caching**: Auth sessions are cached to avoid repeated logins
-
-## Pack v2 Structure
-
-```yaml
-version: 2                      # Required: version identifier
-name: "my-test-pack"            # Required: unique pack name
-description: "Optional description"
-profile: "strict"               # Optional: preset profile (strict, standard, loose)
-
-# Authentication configuration
-auth:
-  api: "default-api"            # Default auth for API gates
-  ui: "default-ui"              # Default auth for UI gates
-  profiles:                     # Named auth profiles
-    default-api:
-      type: bearer
-      config:
-        token: "${API_TOKEN}"
-
-    default-ui:
-      type: ui_login
-      config:
-        url: "https://example.com/login"
-        username: "${USER_EMAIL}"
-        password: "${USER_PASSWORD}"
-        username_selector: "#email"
-        password_selector: "#pass"
-        submit_selector: "button[type='submit']"
-
-# Quality gates (dynamic naming)
-gates:
-  api-health:
-    adapter: playwright-api
-    enabled: true
-    auth: "default-api"         # Use specific auth profile
-    config:
-      baseUrl: "https://api.example.com"
-      smoke:
-        - "GET /health -> 200"
-        - "GET /users -> 200"
-
-  ui-smoke:
-    adapter: playwright-ui
-    enabled: true
-    auth: "default-ui"
-    config:
-      baseUrl: "https://example.com"
-      pages:
-        - "https://example.com/dashboard"
-        - "https://example.com/settings"
-
-# Lifecycle hooks
-hooks:
-  beforeAll:
-    - run: "docker compose up -d"
-      timeout: 30000
-  afterAll:
-    - run: "docker compose down"
-
-# Execution options
-execution:
-  timeout: 30000
-  max_retries: 2
-  on_failure: continue
-
-# Variables (environment substitution)
-variables:
-  ENVIRONMENT: "staging"
-  BASE_URL: "https://staging.example.com"
-```
-
-## Authentication Types
-
-### Bearer Token
-
-```yaml
-auth:
-  profiles:
-    bearer-auth:
-      type: bearer
-      config:
-        token: "your-bearer-token"
-        # Or use environment variable
-        token: "${BEARER_TOKEN}"
-```
-
-### JWT (JSON Web Token)
-
-```yaml
-auth:
-  profiles:
-    jwt-auth:
-      type: jwt
-      config:
-        token: "eyJhbGciOiJIUzI1NiIs..."
-        header: "Authorization"
-        prefix: "Bearer "
-```
-
-### API Key
-
-```yaml
-auth:
-  profiles:
-    apikey-auth:
-      type: api_key
-      config:
-        header: "X-API-Key"
-        value: "${API_KEY}"
-        # Or query parameter
-        query_param: "api_key"
-```
-
-### Basic Authentication
-
-```yaml
-auth:
-  profiles:
-    basic-auth:
-      type: basic
-      config:
-        username: "${API_USER}"
-        password: "${API_PASS}"
-```
-
-### OAuth2 Client Credentials
-
-```yaml
-auth:
-  profiles:
-    oauth2-auth:
-      type: oauth2
-      config:
-        token_url: "https://auth.example.com/oauth/token"
-        client_id: "${CLIENT_ID}"
-        client_secret: "${CLIENT_SECRET}"
-        scope: "read write"
-```
-
-### UI Form Login (Playwright automation)
-
-```yaml
-auth:
-  profiles:
-    ui-login:
-      type: ui_login
-      config:
-        url: "https://example.com/login"
-        username: "${USER_EMAIL}"
-        password: "${USER_PASSWORD}"
-        username_selector: "input[name='email']"
-        password_selector: "input[name='password']"
-        submit_selector: "button[type='submit']"
-        # Optional: Session file for caching
-        session_file: ".qa360/session.json"
-        # Optional: Cache settings
-        cache:
-          enabled: true
-          ttl: 86400  # 24 hours
-```
-
-#### UI Login with TOTP (2FA)
-
-```yaml
-auth:
-  profiles:
-    ui-login-totp:
-      type: ui_login
-      config:
-        url: "https://example.com/login"
-        username: "${USER_EMAIL}"
-        password: "${USER_PASSWORD}"
-        username_selector: "#email"
-        password_selector: "#password"
-        submit_selector: "button[type='submit']"
-        # TOTP configuration
-        totp_secret: "${TOTP_SECRET}"
-        totp_selector: "input[name='code']"
-```
-
-### TOTP (Time-based One-Time Password)
-
-```yaml
-auth:
-  profiles:
-    totp-auth:
-      type: totp
-      config:
-        secret: "JBSWY3DPEHPK3PXP"  # Base32 encoded
-        digits: 6
-        period: 30
-```
-
-### GCP Application Default Credentials
-
-```yaml
-auth:
-  profiles:
-    gcp-auth:
-      type: gcp_adc
-      config:
-        project_id: "${GCP_PROJECT}"
-        scope: "https://www.googleapis.com/auth/cloud-platform"
-```
-
-### AWS IAM Credentials
-
-```yaml
-auth:
-  profiles:
-    aws-auth:
-      type: aws_iam
-      config:
-        access_key_id: "${AWS_ACCESS_KEY_ID}"
-        secret_access_key: "${AWS_SECRET_ACCESS_KEY}"
-        region: "us-east-1"
-        # Or use profile
-        profile: "default"
-```
-
-### Azure AD
-
-```yaml
-auth:
-  profiles:
-    azure-auth:
-      type: azure_ad
-      config:
-        tenant_id: "${AZURE_TENANT_ID}"
-        client_id: "${AZURE_CLIENT_ID}"
-        client_secret: "${AZURE_CLIENT_SECRET}"
-        scope: "https://management.azure.com/.default"
-```
-
-## Gate Configuration
-
-### API Smoke Test Gate
-
-```yaml
-gates:
-  api-health:
-    adapter: playwright-api
-    enabled: true
-    config:
-      baseUrl: "https://api.example.com"
-      smoke:
-        - "GET /health -> 200"
-        - "GET /users -> 200"
-        - "POST /users -> 201"
-      budgets:
-        p95_ms: 1000
-      timeout: 10000
-      retries: 2
-```
-
-### UI Smoke Test Gate
-
-```yaml
-gates:
-  ui-smoke:
-    adapter: playwright-ui
-    enabled: true
-    config:
-      baseUrl: "https://example.com"
-      pages:
-        - "https://example.com/dashboard"
-        - "https://example.com/settings"
-        - "https://example.com/profile"
-      budgets:
-        a11y_min: 80  # Minimum accessibility score
-      timeout: 30000
-```
-
-### Performance Gate
-
-```yaml
-gates:
-  perf-test:
-    adapter: k6-perf
-    enabled: true
-    config:
-      baseUrl: "https://example.com"
-      vus: 10           # Virtual users
-      duration: "30s"
-      budgets:
-        p95_ms: 2000
-        p99_ms: 5000
-```
-
-### Security Scan Gate
-
-```yaml
-gates:
-  sast-scan:
-    adapter: semgrep
-    enabled: true
-    config:
-      targets:
-        - "src/**/*.ts"
-        - "src/**/*.tsx"
-      rules:
-        - "security"
-        - "correctness"
-```
-
-## Gate Options
-
-```yaml
-gates:
-  my-gate:
-    adapter: playwright-api
-    enabled: true
-    # Execution options
-    options:
-      timeout: 30000          # Gate timeout in ms
-      retries: 3              # Number of retries
-      continue_on_failure: true  # Continue to next gate on failure
-      parallelism: 4          # Parallel test execution
-    # Budgets
-    budgets:
-      max_duration_ms: 60000  # Max gate duration
-      p95_ms: 2000           # 95th percentile latency
-      a11y_min: 90           # Min accessibility score
-      max_high: 5            # Max high-severity findings
-      max_critical: 0        # Max critical-severity findings
-```
-
-## Per-Gate Auth Override
-
-Each gate can override the default auth:
-
-```yaml
-auth:
-  api: "default-api"
-  profiles:
-    default-api:
-      type: bearer
-      config: { token: "${API_TOKEN}" }
-    admin-api:
-      type: bearer
-      config: { token: "${ADMIN_API_TOKEN}" }
-
-gates:
-  public-api:
-    adapter: playwright-api
-    auth: "default-api"    # Uses default-api profile
-    config: { ... }
-
-  admin-api:
-    adapter: playwright-api
-    auth: "admin-api"      # Uses admin-api profile
-    config: { ... }
-
-  no-auth:
-    adapter: playwright-api
-    # No auth specified - uses no authentication
-    config: { ... }
-```
-
-## Variables and Environment Substitution
-
-Use `${VARIABLE_NAME}` syntax for environment variable substitution:
-
-```yaml
-variables:
-  APP_URL: "https://${ENVIRONMENT}.example.com"
-  API_KEY: "${API_KEY}"
-
-auth:
-  profiles:
-    api-auth:
-      type: api_key
-      config:
-        value: "${variables.API_KEY}"
-
-gates:
-  api-test:
-    adapter: playwright-api
-    config:
-      baseUrl: "${variables.APP_URL}"
-```
-
-## Hooks
-
-```yaml
-hooks:
-  beforeAll:
-    - run: "docker compose up -d"
-      timeout: 30000
-      continue_on_error: false
-
-    - run: "npx wait-on http://localhost:3000"
-      timeout: 30000
-
-  afterAll:
-    - run: "docker compose down"
-      timeout: 30000
-
-  beforeEach:
-    - run: "echo 'Starting gate: ${GATE_NAME}'"
-
-  afterEach:
-    - run: "echo 'Gate finished: ${GATE_NAME}'"
-```
-
-## Complete Example
-
-```yaml
-version: 2
-name: "e-commerce-checks"
-description: "QA360 checks for e-commerce platform"
-
-auth:
-  api: "customer-api"
-  ui: "customer-ui"
-  profiles:
-    # Customer API authentication
-    customer-api:
-      type: bearer
-      config:
-        token: "${CUSTOMER_API_TOKEN}"
-      cache:
-        enabled: true
-        ttl: 3600
-
-    # Customer UI login
-    customer-ui:
-      type: ui_login
-      config:
-        url: "https://shop.example.com/login"
-        username: "${TEST_USER_EMAIL}"
-        password: "${TEST_USER_PASSWORD}"
-        username_selector: "#email"
-        password_selector: "#password"
-        submit_selector: "button[type='submit']"
-      cache:
-        enabled: true
-        ttl: 86400
-
-gates:
-  # API health checks
-  api-health:
-    adapter: playwright-api
-    enabled: true
-    config:
-      baseUrl: "https://api.shop.example.com"
-      smoke:
-        - "GET /health -> 200"
-        - "GET /products -> 200"
-        - "GET /categories -> 200"
-      timeout: 10000
-
-  # UI smoke tests
-  ui-smoke:
-    adapter: playwright-ui
-    enabled: true
-    config:
-      baseUrl: "https://shop.example.com"
-      pages:
-        - "https://shop.example.com/products"
-        - "https://shop.example.com/cart"
-        - "https://shop.example.com/profile"
-      budgets:
-        a11y_min: 85
-
-  # Accessibility focused
-  a11y-check:
-    adapter: playwright-ui
-    enabled: true
-    config:
-      baseUrl: "https://shop.example.com"
-      pages:
-        - "https://shop.example.com/products"
-        - "https://shop.example.com/checkout"
-      budgets:
-        a11y_min: 95
-
-hooks:
-  beforeAll:
-    - run: "echo 'Starting e-commerce checks'"
-
-execution:
-  timeout: 60000
-  on_failure: continue
-
-variables:
-  ENVIRONMENT: "production"
-```
-
-## Migration from v1 to v2
-
-### v1 Format
-
-```yaml
-version: 1
-name: "my-tests"
-gates:
-  - api_smoke
-  - ui
-targets:
-  api:
-    baseUrl: "https://api.example.com"
-    smoke:
-      - "GET /health -> 200"
-  web:
-    baseUrl: "https://example.com"
-    pages:
-      - "https://example.com/home"
-```
-
-### v2 Format (equivalent)
-
-```yaml
-version: 2
-name: "my-tests"
-gates:
-  api_smoke:
-    adapter: playwright-api
-    enabled: true
-    config:
-      baseUrl: "https://api.example.com"
-      smoke:
-        - "GET /health -> 200"
-
-  ui:
-    adapter: playwright-ui
-    enabled: true
-    config:
-      baseUrl: "https://example.com"
-      pages:
-        - "https://example.com/home"
-```
-
-## Best Practices
-
-1. **Use Environment Variables**: Never hardcode credentials in pack files
-2. **Enable Auth Caching**: Reduce login overhead by caching sessions
-3. **Name Gates Meaningfully**: Use descriptive gate names for better reporting
-4. **Set Appropriate Budgets**: Define realistic performance and accessibility budgets
-5. **Use Per-Gate Auth**: Override auth only when necessary
-6. **Leverage Hooks**: Use hooks for setup/teardown of test environments
-
-## See Also
-
-- [Pack v1 Reference](./pack-v1.md) - Original pack format
-- [Architecture](./ARCHITECTURE.md) - System architecture
-- [Adapters](./adapters-security.md) - Security adapter documentation