universal-dev-standards 5.4.0 → 5.6.0

package/bundled/core/browser-compatibility-standards.md

@@ -0,0 +1,220 @@
+# Browser Compatibility Standards
+
+> **Language**: English | [繁體中文](../locales/zh-TW/core/browser-compatibility-standards.md)
+
+**Version**: 1.0.0
+**Last Updated**: 2026-05-05
+**Applicability**: Frontend projects (web apps, progressive web apps, web components)
+**Scope**: universal
+**Industry Standards**: Browserslist, W3C WebDriver, WebDriver BiDi
+**References**: [caniuse.com](https://caniuse.com/), [Playwright browser support matrix](https://playwright.dev/docs/browsers)
+
+---
+
+## Purpose
+
+This standard defines supported browser and device matrices, testing automation strategies, and the release gate for browser compatibility (Dimension 9 in `release-readiness-gate.md`, Tier-3).
+
+Browser compatibility issues are among the most user-visible defects, yet they are systematically under-tested because teams assume "it works in Chrome." Without an explicit supported matrix and automated verification, regressions slip to production and affect large user segments.
+
+---
+
+## Support Tier Definitions
+
+| Tier | Definition | Release Gate |
+|------|-----------|--------------|
+| **Tier-1** (Supported) | Full feature parity + automated test coverage | 100% pass — blocks release if any test fails |
+| **Tier-2** (Partial support) | Best-effort; major flows must work | ≥ 95% pass — WARN if below, FAIL if < 90% |
+| **Tier-3** (Best effort) | Not officially supported; defects logged but not blocking | Advisory only |
+
+---
+
+## Default Browser Matrix
+
+Teams MUST declare their supported matrix explicitly. The defaults below cover the majority of web traffic (2025–2026 data):
+
+### Tier-1 (Default)
+
+| Browser | Versions | Platform |
+|---------|----------|---------|
+| Chrome | latest, latest-1 | Windows, macOS, Linux, Android |
+| Safari | latest, latest-1 | macOS, iOS |
+| Firefox | latest | Windows, macOS, Linux |
+| Edge | latest | Windows, macOS |
+
+### Tier-2 (Default)
+
+| Browser | Versions | Platform |
+|---------|----------|---------|
+| Chrome | latest-2, latest-3 | Desktop |
+| Safari | latest-2 | macOS, iOS |
+| Samsung Internet | latest | Android |
+| Opera | latest | Desktop |
+
+### Tier-3 (Default)
+
+| Browser | Notes |
+|---------|-------|
+| IE 11 | EOL; only if contractually required |
+| Chrome < latest-3 | Tracked as known limitation |
+
+### Device / Viewport Matrix
+
+| Category | Min Width | Representative Device |
+|----------|-----------|-----------------------|
+| Mobile (small) | 360px | Android (small) |
+| Mobile (standard) | 390px | iPhone 14 |
+| Tablet (portrait) | 768px | iPad |
+| Tablet (landscape) | 1024px | iPad landscape |
+| Desktop (small) | 1280px | Laptop 13" |
+| Desktop (standard) | 1440px | Laptop 15" / External monitor |
+| Desktop (wide) | 1920px | Full HD monitor |
+
+Minimum: test at **360px, 768px, 1280px** (mobile / tablet / desktop breakpoints).
+
+---
+
+## Automated Testing
+
+### Playwright Matrix Configuration
+
+```typescript
+// playwright.config.ts
+import { defineConfig, devices } from "@playwright/test";
+
+export default defineConfig({
+  projects: [
+    // Tier-1 browsers
+    { name: "chromium", use: { ...devices["Desktop Chrome"] } },
+    { name: "firefox", use: { ...devices["Desktop Firefox"] } },
+    { name: "webkit", use: { ...devices["Desktop Safari"] } },
+    { name: "edge", use: { ...devices["Desktop Edge"] } },
+    // Mobile Tier-1
+    { name: "mobile-chrome", use: { ...devices["Pixel 7"] } },
+    { name: "mobile-safari", use: { ...devices["iPhone 14"] } },
+    // Viewport coverage
+    { name: "tablet", use: { ...devices["iPad Pro 11"] } },
+  ],
+  // Tier-1 failure = build failure
+  reporter: [["html"], ["junit", { outputFile: "results/browser-compat.xml" }]],
+});
+```
+
+### CI Execution Strategy
+
+```bash
+# Run full Tier-1 matrix on release candidate
+npx playwright test --project=chromium,firefox,webkit,edge,mobile-chrome,mobile-safari,tablet
+
+# Run Tier-1 only on every PR (fast feedback)
+npx playwright test --project=chromium,firefox,webkit
+
+# Run Tier-2 on release candidate (results feed into sign-off as WARN/PASS)
+npx playwright test --project=samsung,opera
+```
+
+### Cloud Browser Testing
+
+For Tier-1 cross-OS testing (e.g., Safari on Windows-hosted CI), use cloud services:
+
+| Service | Use Case |
+|---------|---------|
+| BrowserStack Automate | Commercial projects; widest OS+browser matrix |
+| Sauce Labs | Enterprises with existing contract |
+| LambdaTest | Open-source / cost-sensitive projects |
+
+**Minimum cloud testing**: Safari latest + latest-1 on real iOS devices (Simulator is insufficient for WebKit bugs).
+
+---
+
+## Visual Regression (Optional but Recommended)
+
+Pixel-diff testing detects layout regressions across browsers:
+
+```bash
+# Using Playwright visual comparisons
+npx playwright test --update-snapshots  # update baseline
+npx playwright test                     # compare against baseline; fail if diff > threshold
+```
+
+Threshold recommendation: < 0.5% pixel diff for layout components; < 2% for complex interactive components.
+
+---
+
+## Release Gate Criteria
+
+This is **Dimension 9** in `release-readiness-gate.md` (Tier-3: required for frontend/web projects; `N/A` for CLI/backend-only).
+
+| Gate | Pass | Warn | Fail |
+|------|------|------|------|
+| Tier-1 browser matrix | 100% tests pass | — | Any test fails |
+| Tier-2 browser matrix | ≥ 95% pass | 90–95% | < 90% |
+| Viewport coverage (360/768/1280) | No layout breaks on any Tier-1 browser | — | Any critical flow unusable |
+
+### Evidence for Sign-off
+
+```
+| 9 | Browser / Device Compat | PASS | Playwright: 6 browsers × 7 viewports, 100% Tier-1; Tier-2: 97%; [junit report link] | QA Lead |
+```
+
+### `N/A` Criteria
+
+Mark as `N/A` when:
+- Project is CLI-only, pure backend API, or mobile native (not web)
+- Document rationale: `"N/A — backend API service, no browser UI"`
+
+---
+
+## Browserslist Configuration
+
+Commit a `.browserslistrc` to the repo root to ensure build tools (Babel, PostCSS, Autoprefixer) target the same browsers:
+
+```
+# .browserslistrc
+# Tier-1: production targets
+last 2 Chrome versions
+last 2 Firefox versions
+last 2 Safari versions
+last 2 Edge versions
+last 2 iOS versions
+last 2 ChromeAndroid versions
+
+# Tier-2: for reference (not in build targets by default)
+# last 4 Chrome versions
+# Samsung Internet >= 14
+```
+
+---
+
+## Anti-Patterns
+
+- **Testing only Chrome** — Chrome represents ~65% of desktop traffic; the remaining 35% is Safari/Firefox/Edge users who will find your bugs
+- **Using browser simulators for iOS Safari testing** — WebKit on Simulator diverges from real device WebKit; always test on real iOS for release candidates
+- **Not specifying a matrix** — implicit assumption of "all browsers" is impossible to test; an explicit Tier-1 matrix is better than implicit coverage of none
+- **Treating Tier-3 browser failures as blockers** — Tier-3 is best-effort; logging the issue is appropriate, not blocking the release
+- **Skipping mobile viewport testing** — mobile-first is standard; missing 360px tests will produce broken UX for the majority of mobile users
+
+---
+
+## Relationship to Other Standards
+
+- **`accessibility-standards.md`** — keyboard nav and screen reader tests run across Tier-1 browsers
+- **`e2e-testing.md`** — Playwright matrix config extends E2E tests to multi-browser
+- **`release-readiness-gate.md`** — Dimension 9 (Tier-3)
+- **`performance-standards.md`** — Core Web Vitals targets apply per Tier-1 browser
+
+---
+
+## Version History
+
+| Version | Date | Changes |
+|---------|------|---------|
+| 1.0.0 | 2026-05-05 | Initial release: Tier-1/2/3 matrix, Playwright config, cloud testing, release gate criteria |
+
+---
+
+## License
+
+This standard is released under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/).
+
+**Source**: [universal-dev-standards](https://github.com/AsiaOstrich/universal-dev-standards)

package/bundled/core/chaos-injection-tests.md

@@ -0,0 +1,116 @@
+# Chaos Injection Tests
+
+## Overview
+
+Chaos injection tests make failure scenarios machine-verifiable. Where `chaos-engineering-standards` describes the experiment methodology, this standard defines the specific tests required for AI agent systems — LLM timeouts, database disconnects, policy engine failures, and blast-radius containment.
+
+## Why AI Agent Systems Need Dedicated Chaos Tests
+
+Traditional software has a handful of external dependencies. AI agent systems compound this:
+
+- **LLM API**: high latency, rate limits, non-deterministic failures
+- **Policy engine** (OPA/Rego): security-critical — must fail closed
+- **Vector store / knowledge base**: retrieval failures affect output quality
+- **Database**: mid-operation disconnects can corrupt multi-step agent state
+- **Peer agents**: in multi-agent pipelines, one agent crash must not cascade
+
+Each of these failure modes needs a dedicated test, not just a comment in a runbook.
+
+## Requirements Summary
+
+| ID | Rule | Rationale |
+|----|------|-----------|
+| REQ-CIT-001 | Each external dependency needs a failure isolation test | Single dependency failure must not cascade |
+| REQ-CIT-002 | LLM client must handle timeout and rate-limit | LLM is the highest-risk dependency |
+| REQ-CIT-003 | Policy engine unavailability must default to DENY | Fail-open is a security vulnerability |
+| REQ-CIT-004 | DB disconnect mid-operation must roll back cleanly | Partial writes cause data corruption |
+| REQ-CIT-005 | Agent crash must not propagate to unrelated agents | Inter-agent blast radius must be bounded |
+
+## Injection Patterns
+
+### LLM Timeout
+
+```typescript
+it('surfaces TimeoutError when LLM does not respond in time', async () => {
+  const slowLlm = { complete: () => new Promise(() => {}) } // never resolves
+  const agent = new PlannerAgent({ llm: slowLlm, timeoutMs: 100 })
+  await expect(agent.run(input)).rejects.toThrow('TimeoutError')
+})
+```
+
+### LLM Rate Limit (429)
+
+```typescript
+it('retries with backoff on 429 and eventually surfaces RateLimitError', async () => {
+  const rateLimitedLlm = mockLlm({ status: 429, retryAfter: 1 })
+  const agent = new PlannerAgent({ llm: rateLimitedLlm })
+  await expect(agent.run(input)).rejects.toThrow('RateLimitError')
+  expect(rateLimitedLlm.callCount).toBeLessThanOrEqual(3) // respects retry policy
+})
+```
+
+### Policy Engine Down (Fail-Closed)
+
+```typescript
+it('returns DENY when OPA sidecar is unavailable', async () => {
+  const downOpa = { query: () => Promise.reject(new Error('ECONNREFUSED')) }
+  const guardian = new GuardianAgent({ opa: downOpa })
+  const result = await guardian.review(reviewable)
+  expect(result.decision).toBe('DENY')
+  expect(result.reason).toMatch(/policy engine unavailable/)
+})
+```
+
+### Database Disconnect
+
+```typescript
+it('rolls back transaction on mid-operation DB disconnect', async () => {
+  const db = createTestDb()
+  await seedRows(db, [{ id: 1, name: 'alice' }])
+  
+  // Force disconnect after first write in the transaction
+  let writeCount = 0
+  const hookedDb = hookAfterWrite(db, () => {
+    if (++writeCount === 1) db.close()
+  })
+  
+  await expect(runner.executeWithDb(hookedDb, plan)).rejects.toThrow()
+  
+  const freshDb = createTestDb()
+  const rows = freshDb.prepare('SELECT * FROM records').all()
+  expect(rows).toHaveLength(1) // original row preserved, partial write rolled back
+})
+```
+
+### Agent Crash Containment
+
+```typescript
+it('pipeline continues when one agent throws', async () => {
+  const crashingAgent = { run: () => { throw new Error('agent crash') } }
+  const pipeline = new Pipeline({ agents: { planner: crashingAgent, builder: realBuilder } })
+  
+  const result = await pipeline.run(input, { skipFailedAgents: true })
+  expect(result.completedAgents).toContain('builder')
+  expect(result.failedAgents).toContain('planner')
+})
+```
+
+## Safety Rules
+
+1. Never run chaos tests against production or shared staging databases
+2. Always clean up injected faults in `afterEach` or `finally` blocks
+3. Tag chaos tests (`@chaos`) to exclude from fast unit test runs in developer workflow
+4. Chaos tests may run in CI on a dedicated job, not in the standard unit test matrix
+
+## Anti-Patterns
+
+- **Catching and ignoring all errors in the main handler** — this hides chaos failures from assertions
+- **Not verifying database state after disconnect** — asserting the error is thrown is not enough; assert no partial data was written
+- **Fail-open policy engine handling** — any ambiguity in the policy path must resolve to DENY, not ALLOW
+
+## See Also
+
+- `chaos-engineering-standards.ai.yaml` — experiment methodology and SLO integration
+- `testing.ai.yaml` — general test structure
+- `secure-op.ai.yaml` — Fail-Closed principle for AI agents
+- `security-standards.ai.yaml` — security invariants

package/bundled/core/checkin-standards.md

@@ -1121,6 +1121,7 @@ auto_fix:
 - [Acceptance Criteria Traceability](acceptance-criteria-traceability.md) - AC coverage verification
 - [Change Batching Standards](change-batching-standards.md) - Batch threshold triggers
 - [Pipeline Integration Standards](pipeline-integration-standards.md) - Automated check-in
+- [Release Readiness Gate](release-readiness-gate.md) - **Check-in ≠ release readiness**: passing per-commit gates is necessary but not sufficient for production; see the 16-dimension release gate for what "done" means at release time
 
 ---