@odavl/guardian 0.1.0-rc1

package/flows/example-login-flow.json

@@ -0,0 +1,36 @@
+{
+  "id": "example-login-flow",
+  "name": "User Login Flow",
+  "description": "Test the login functionality - navigate to login page, fill credentials, and verify success",
+  "steps": [
+    {
+      "type": "navigate",
+      "target": "/login"
+    },
+    {
+      "type": "waitFor",
+      "target": "input[name='email']",
+      "timeout": 5000
+    },
+    {
+      "type": "type",
+      "target": "input[name='email']",
+      "value": "test@example.com"
+    },
+    {
+      "type": "type",
+      "target": "input[name='password']",
+      "value": "test123"
+    },
+    {
+      "type": "click",
+      "target": "button[type='submit']",
+      "waitForNavigation": true
+    },
+    {
+      "type": "waitFor",
+      "target": "[data-testid='dashboard']",
+      "timeout": 5000
+    }
+  ]
+}

package/flows/example-signup-flow.json

@@ -0,0 +1,44 @@
+{
+  "id": "example-signup-flow",
+  "name": "User Signup Flow",
+  "description": "Test the signup functionality - navigate to signup, fill form, submit, and verify confirmation",
+  "steps": [
+    {
+      "type": "navigate",
+      "target": "/signup"
+    },
+    {
+      "type": "waitFor",
+      "target": "form[data-testid='signup-form']",
+      "timeout": 5000
+    },
+    {
+      "type": "type",
+      "target": "input[name='name']",
+      "value": "Test User"
+    },
+    {
+      "type": "type",
+      "target": "input[name='email']",
+      "value": "newuser@example.com"
+    },
+    {
+      "type": "type",
+      "target": "input[name='password']",
+      "value": "securepass123"
+    },
+    {
+      "type": "click",
+      "target": "input[type='checkbox'][name='terms']"
+    },
+    {
+      "type": "submit",
+      "target": "form[data-testid='signup-form']"
+    },
+    {
+      "type": "waitFor",
+      "target": "[data-testid='signup-success']",
+      "timeout": 10000
+    }
+  ]
+}

package/guardian-contract-v1.md

@@ -0,0 +1,149 @@
+# ODAVL Guardian — Contract v1.0
+
+Status: Active
+
+Scope: Guardian MVP
+
+Version: v1.0
+
+Decision-Grade: Yes
+
+## 0) Purpose
+
+This contract defines the canonical, non-negotiable rules and guarantees for ODAVL Guardian’s Market Reality Testing Engine at MVP scope. It specifies final decisions, confidence semantics, evidence requirements, flow execution rules, honesty constraints, prohibited behaviors, verifiability, and stability expectations. All implementations, outputs, and operator-facing UX must conform to this contract.
+
+## 1) Definition
+
+ODAVL Guardian is a headless browser-based reality testing engine that:
+- Launches a real browser (Playwright/Chromium) to inspect a site via crawl and/or execute predefined flows.
+- Collects evidence artifacts (screenshots, logs, reports, traces, optional HAR) for verifiable outcomes.
+- Computes coverage and behavior metrics, forms a conservative confidence judgment, and issues a final decision.
+- Emits machine-readable `report.json`, human-readable `report.html`, `logs.txt`, and run directory with artifacts.
+
+Terminology:
+- "Coverage Confidence": Confidence derived from how much of the site was observed (depth/pages vs discovered).
+- "Behavioral Confidence": Confidence derived from observed runtime stability and health (navigation/http/page/console errors).
+- "Evidence Policy": The resolved set of required vs optional artifacts that must be present for a decision to be valid.
+- "Flow": A predefined deterministic sequence of real user interactions (navigate/click/type/submit/waitFor).
+
+## 2) Final Decisions
+
+Guardian issues exactly three decisions:
+- READY — Site is acceptable for MVP launch given the evidence and confidence model.
+- DO_NOT_LAUNCH — Launch is blocked due to critical failure or missing required evidence.
+- INSUFFICIENT_CONFIDENCE — Evidence and/or behavior do not justify readiness.
+
+Exit codes:
+- READY → 0
+- DO_NOT_LAUNCH → 1
+- INSUFFICIENT_CONFIDENCE → 1
+- TOOL ERROR → 2
+
+Decision drivers:
+- Any Flow FAIL → DO_NOT_LAUNCH.
+- Required evidence missing per Evidence Policy → DO_NOT_LAUNCH.
+- Overall confidence HIGH or MEDIUM → READY.
+- Overall confidence LOW → INSUFFICIENT_CONFIDENCE.
+
+## 3) Confidence Model
+
+Guardian separates confidence into two dimensions and a resolved overall level.
+
+### 3.1 Coverage Confidence
+- HIGH: Sufficient coverage (e.g., deep exploration and/or high coverage percentage).
+- MEDIUM: Moderate coverage.
+- LOW: Limited coverage (e.g., very few pages visited).
+
+### 3.2 Behavioral Confidence
+- HIGH: No critical runtime instability (no navigation failures, no page errors, no server 5xx), acceptable client logs.
+- MEDIUM: Minor issues (e.g., 4xx or console errors) without critical failures.
+- LOW: Critical instability (navigation failure, unhandled page errors, or server 5xx).
+
+### 3.3 Overall Confidence Resolution
+- HIGH: Sufficient coverage OR a successful Flow with no critical errors.
+- MEDIUM: Limited coverage BUT clean behavior (no page errors or critical failures); minor issues allowed.
+- LOW: Insufficient coverage AND/OR unstable behavior (critical failures present).
+
+Guardian must record a human-readable "reasoning" for the resolved confidence.
+
+## 4) Evidence Contract
+
+Guardian enforces an explicit Evidence Policy defining required vs optional artifacts.
+
+### 4.1 Policy Modes
+- normal (default):
+  - REQUIRED: screenshots (pages and flow steps), `report.json`, `report.html`.
+  - REQUIRED when enabled: `trace.zip`.
+  - OPTIONAL: `network.har`/`network.json`.
+- strict:
+  - REQUIRED: screenshots, `report.json`, `report.html`.
+  - REQUIRED when enabled: `trace.zip` and `network.har`/`network.json`.
+- custom (via CLI overrides):
+  - `--require-har` forces HAR required.
+  - `--optional-har` forces HAR optional.
+  - Precedence: `require-har` > `optional-har` > `--evidence`.
+
+### 4.2 Missing Evidence Handling
+- Missing REQUIRED evidence → DO_NOT_LAUNCH.
+- Missing OPTIONAL evidence → does not block READY; recorded as warnings and may reduce confidence.
+
+### 4.3 Evidence Reporting (report.json)
+Guardian must include:
+```
+evidence: {
+  policy: { mode: "normal|strict|custom", requirements: { screenshots: true, traceWhenEnabled: true, harWhenEnabled: true|false } },
+  present: { screenshots: true|false, trace: true|false|null, har: true|false|null },
+  requiredMissing: [ ... ],
+  optionalMissing: [ ... ],
+  warnings: [ { code, message } ]
+}
+```
+
+The HTML report must present the Evidence Policy, list required/optional missing items, and warnings.
+
+## 5) Flow Contract
+
+Flows are executed as real interactions through the browser and must:
+- Support step types: `navigate <url>`, `click <selector(s)>`, `type <selector> <value>`, `submit <selector>`, `waitFor <selector|url:pattern>`.
+- Capture a full-page screenshot for each step with deterministic naming (`flow-XX-type.png`).
+- Stop immediately on step failure; record `failedStepIndex`, the error, and mark Flow result FAIL.
+- Treat any Flow FAIL as REVENUE-BLOCKING: the final decision must be DO_NOT_LAUNCH.
+- Log step-by-step outcomes in `report.json` and `report.html`.
+
+## 6) Honesty & Transparency
+
+Guardian must:
+- Never overstate certainty; only issue READY when justified by the confidence model and evidence policy.
+- Explicitly list reasons for final decisions and confidence (human-readable) in outputs.
+- Disclose coverage limitations (e.g., LOW coverage warning) visibly in CLI and HTML.
+- Preserve conservative defaults that favor truthful reporting.
+
+## 7) Prohibited Behaviors
+
+Guardian must not:
+- Ignore missing REQUIRED evidence.
+- Fabricate or simulate evidence artifacts.
+- Issue READY when overall confidence is LOW.
+- Suppress critical errors (navigation failures, page errors, server 5xx) from decisions.
+- Alter artifacts or reports after generation except for permitted `--clean` on PASS.
+
+## 8) Verifiability
+
+Guardian must be verifiable via:
+- Reproducible commands and exit codes (PowerShell examples in README).
+- Stable artifact paths under `artifacts/run-YYYYMMDD-HHMMSS/` including `report.json`, `report.html`, `logs.txt`, screenshots, traces, and HAR/network when present.
+- Machine-readable fields capturing coverage, behavior, evidence, flow results, blockers, and final decisions.
+
+CLI must surface final decision, confidence summary (coverage + behavior + reasoning), evidence policy, and warnings.
+
+## 9) Stability Clause
+
+Guardian aims for stable, repeatable outcomes under identical inputs:
+- Flow runs must be deterministic with consistent PASS/FAIL given the same site state.
+- Crawl outcomes may vary in timing, but decisions should remain consistent when behavior and evidence are unchanged.
+- Evidence presence (trace/HAR) may vary by environment; the Evidence Policy governs whether such variance affects decisions.
+- Any non-deterministic behavior should be logged; the final decision must remain conservative.
+
+## Final Seal / Closing Statement
+
+This contract is canonical for ODAVL Guardian MVP. All implementations and outputs must adhere strictly to these rules. Operators rely on Guardian’s conservative, evidence-driven judgments; deviations from this contract are prohibited. Changes to this contract require explicit versioning and are not implied by code modifications.

package/guardian.config.json

@@ -0,0 +1,54 @@
+{
+  "baseUrl": "https://example.com",
+  "mode": "SAFE",
+  "maxPages": 25,
+  "maxDepth": 3,
+  "timeoutMs": 20000,
+  "slowThresholdMs": 6000,
+  "traceEnabled": true,
+  "harEnabled": true,
+  "safety": {
+    "denyUrlPatterns": [
+      "logout",
+      "signout",
+      "sign-out",
+      "delete",
+      "remove",
+      "admin",
+      "unsubscribe",
+      "checkout/confirm",
+      "checkout/complete",
+      "order/confirm",
+      "payment/confirm",
+      "deactivate",
+      "destroy",
+      "purge",
+      "terminate",
+      "cancel"
+    ],
+    "denySelectors": [
+      "[data-danger]",
+      "[data-destructive]",
+      "button:has-text(\"Delete\")",
+      "button:has-text(\"Remove\")",
+      "button:has-text(\"Confirm Payment\")",
+      "button:has-text(\"Place Order\")",
+      "button:has-text(\"Complete Purchase\")",
+      "button:has-text(\"Unsubscribe\")",
+      "button:has-text(\"Deactivate\")",
+      "button:has-text(\"Cancel Account\")",
+      ".btn-danger",
+      ".btn-delete",
+      ".payment-submit",
+      ".checkout-submit"
+    ],
+    "allowUrlPatterns": [],
+    "allowSelectors": [],
+    "blockFormSubmitsByDefault": true,
+    "blockPaymentsByDefault": true
+  },
+  "artifacts": {
+    "artifactsDir": "artifacts",
+    "runIdStrategy": "timestamp"
+  }
+}

package/guardian.policy.json

@@ -0,0 +1,12 @@
+{
+  "name": "Startup Policy",
+  "description": "Permissive policy for fast-moving startups. Fail only on CRITICAL issues.",
+  "failOnSeverity": "CRITICAL",
+  "maxWarnings": 999,
+  "maxInfo": 999,
+  "maxTotalRisk": 999,
+  "failOnNewRegression": false,
+  "failOnSoftFailures": false,
+  "softFailureThreshold": 999,
+  "requireBaseline": false
+}

package/guardian.profile.docs.yaml

@@ -0,0 +1,18 @@
+# Guardian profile template: Docs
+baseUrl: https://docs.example.com
+profile: docs
+mode: SAFE
+maxPages: 40
+maxDepth: 5
+slowThresholdMs: 7000
+traceEnabled: false
+harEnabled: false
+revenue:
+  enabled: false
+auth:
+  enabled: false
+regression:
+  enabled: true
+  maxScoreDrop: 12
+  maxCoverageDropPercent: 12
+policyPack: trust-first

package/guardian.profile.ecommerce.yaml

@@ -0,0 +1,17 @@
+# Guardian profile template: E-commerce
+baseUrl: https://shop.example.com
+profile: ecommerce
+mode: SAFE
+maxPages: 35
+maxDepth: 4
+slowThresholdMs: 6000
+traceEnabled: false
+harEnabled: false
+revenue:
+  enabled: true
+  pricingPath: /products
+regression:
+  enabled: true
+  maxScoreDrop: 6
+  maxCoverageDropPercent: 8
+policyPack: revenue-first

package/guardian.profile.marketing.yaml

@@ -0,0 +1,18 @@
+# Guardian profile template: Marketing
+baseUrl: https://www.example.com
+profile: marketing
+mode: SAFE
+maxPages: 20
+maxDepth: 3
+slowThresholdMs: 6000
+traceEnabled: false
+harEnabled: false
+revenue:
+  enabled: false
+auth:
+  enabled: false
+regression:
+  enabled: true
+  maxScoreDrop: 10
+  maxCoverageDropPercent: 12
+policyPack: trust-first

package/guardian.profile.saas.yaml

@@ -0,0 +1,21 @@
+# Guardian profile template: SaaS
+baseUrl: https://app.example.com
+profile: saas
+mode: SAFE
+maxPages: 25
+maxDepth: 3
+slowThresholdMs: 6000
+traceEnabled: false
+harEnabled: false
+auth:
+  enabled: true
+  email: ${GUARDIAN_AUTH_EMAIL}
+  password: ${GUARDIAN_AUTH_PASSWORD}
+revenue:
+  enabled: true
+  pricingPath: /pricing
+regression:
+  enabled: true
+  maxScoreDrop: 8
+  maxCoverageDropPercent: 10
+policyPack: revenue-first

package/package.json

@@ -0,0 +1,69 @@
+{
+  "name": "@odavl/guardian",
+  "version": "0.1.0-rc1",
+  "description": "ODAVL Guardian — Market Reality Testing Engine with Visual Diffs, Behavioral Signals, Auto-Discovery, Intelligence, and CI/CD Integration",
+  "license": "MIT",
+  "author": "ODAVL",
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/odavlstudio/odavlguardian"
+  },
+  "keywords": [
+    "testing",
+    "browser-testing",
+    "reality-testing",
+    "playwright",
+    "cli",
+    "guardian",
+    "market-testing",
+    "quality-assurance"
+  ],
+  "engines": {
+    "node": ">=18.0.0"
+  },
+  "bin": {
+    "guardian": "bin/guardian.js"
+  },
+  "files": [
+    "bin/",
+    "src/",
+    "flows/",
+    "policies/",
+    "guardian.config.json",
+    "guardian.policy.json",
+    "guardian.profile.docs.yaml",
+    "guardian.profile.ecommerce.yaml",
+    "guardian.profile.marketing.yaml",
+    "guardian.profile.saas.yaml",
+    "guardian-contract-v1.md",
+    "README.md",
+    "LICENSE",
+    "CHANGELOG.md"
+  ],
+  "scripts": {
+    "test": "node test/mvp.test.js",
+    "test:phase0": "node test/phase0-reality-lock.test.js",
+    "test:phase1": "node test/phase1-baseline-expansion.test.js",
+    "test:phase2": "node test/phase2-auto-attempts.test.js",
+    "test:phase2-old": "node test/phase2.test.js",
+    "test:attempt": "node test/attempt.test.js",
+    "test:reality": "node test/reality.test.js",
+    "test:baseline": "node test/baseline.test.js",
+    "test:baseline:junit": "node test/baseline-junit.test.js",
+    "test:snapshot": "node test/snapshot.test.js",
+    "test:soft-failures": "node test/soft-failures.test.js",
+    "test:criticality": "node test/market-criticality.test.js",
+    "test:discovery": "node test/discovery.test.js",
+    "test:phase5": "node test/phase5.test.js",
+    "test:phase5:visual": "node test/phase5-visual.test.js",
+    "test:phase5:evidence": "node test/phase5-evidence-run.test.js",
+    "test:phase5:all": "node test/phase5-visual.test.js && node test/phase5-evidence-run.test.js",
+    "test:phase6": "node test/phase6.test.js && node test/phase6-product.test.js",
+    "test:all": "node test/phase0-reality-lock.test.js && node test/mvp.test.js && node test/phase2.test.js && node test/attempt.test.js && node test/reality.test.js && node test/baseline.test.js && node test/baseline-junit.test.js && node test/snapshot.test.js && node test/soft-failures.test.js && node test/market-criticality.test.js && node test/discovery.test.js && node test/phase5.test.js && node test/phase5-visual.test.js && node test/phase5-evidence-run.test.js && node test/phase6.test.js",
+    "start": "node bin/guardian.js"
+  },
+  "dependencies": {
+    "express": "^5.2.1",
+    "playwright": "^1.48.2"
+  }
+}

package/policies/enterprise.json

@@ -0,0 +1,12 @@
+{
+  "name": "Enterprise Policy",
+  "description": "Strict policy for enterprise deployments. Zero tolerance for CRITICAL or WARNING issues.",
+  "failOnSeverity": "WARNING",
+  "maxWarnings": 0,
+  "maxInfo": 0,
+  "maxTotalRisk": 0,
+  "failOnNewRegression": true,
+  "failOnSoftFailures": true,
+  "softFailureThreshold": 0,
+  "requireBaseline": true
+}

package/policies/saas.json

@@ -0,0 +1,12 @@
+{
+  "name": "SaaS Policy",
+  "description": "Balanced policy for SaaS products. Fail on CRITICAL and detect regressions.",
+  "failOnSeverity": "CRITICAL",
+  "maxWarnings": 1,
+  "maxInfo": 999,
+  "maxTotalRisk": 999,
+  "failOnNewRegression": true,
+  "failOnSoftFailures": false,
+  "softFailureThreshold": 5,
+  "requireBaseline": false
+}

package/policies/startup.json

@@ -0,0 +1,12 @@
+{
+  "name": "Startup Policy",
+  "description": "Permissive policy for fast-moving startups. Fail only on CRITICAL issues.",
+  "failOnSeverity": "CRITICAL",
+  "maxWarnings": 999,
+  "maxInfo": 999,
+  "maxTotalRisk": 999,
+  "failOnNewRegression": false,
+  "failOnSoftFailures": false,
+  "softFailureThreshold": 999,
+  "requireBaseline": false
+}