opendevbrowser 0.0.17 → 0.0.19

package/skills/opendevbrowser-best-practices/artifacts/skill-runtime-surface-matrix.md

@@ -0,0 +1,58 @@
+# Skill Runtime Surface Matrix
+
+Canonical inventory of repo-local OpenDevBrowser skill packs and the runtime surfaces they must prove with real workflow execution.
+
+## Canonical skill packs
+
+| Pack | Type | Validator | Runtime surfaces | Real workflow proof lane | External boundary |
+|---|---|---|---|---|---|
+| `opendevbrowser-best-practices` | `governance` | `./skills/opendevbrowser-best-practices/scripts/validate-skill-assets.sh` | surface docs, workflow router, release evidence policy, CLI or tool or `/ops` or `/canvas` inventory | `node scripts/docs-drift-check.mjs`, `./skills/opendevbrowser-best-practices/scripts/run-robustness-audit.sh`, `node scripts/cli-smoke-test.mjs`, `node scripts/provider-direct-runs.mjs`, `node scripts/live-regression-direct.mjs`, `node scripts/canvas-competitive-validation.mjs` | extension unavailable at start, auth walls, anti-bot pressure, rate limits, upstream outage |
+| `opendevbrowser-continuity-ledger` | `doc_only` | `./skills/opendevbrowser-continuity-ledger/scripts/validate-skill-assets.sh` | continuity ownership, ledger template, reply protocol | validator output plus `skill-discovery` and `docs-drift`; no live browser probe | not applicable |
+| `opendevbrowser-data-extraction` | `workflow` | `./skills/opendevbrowser-data-extraction/scripts/validate-skill-assets.sh` | snapshot, DOM text/html extraction, pagination, quality gates, structured extraction | `node scripts/cli-smoke-test.mjs` plus `node scripts/product-video-fixture-live-probe.mjs` for structured extraction on a repo-local product page | extension unavailable at start if extension mode is audited separately |
+| `opendevbrowser-design-agent` | `browser_surface` | `./skills/opendevbrowser-design-agent/scripts/validate-skill-assets.sh` | `/canvas`, annotate send/stored semantics, design contract workflow, real-surface validation | `node scripts/live-regression-direct.mjs`, `node scripts/canvas-competitive-validation.mjs` | extension unavailable at start, optional Figma live smoke |
+| `opendevbrowser-form-testing` | `workflow` | `./skills/opendevbrowser-form-testing/scripts/validate-skill-assets.sh` | snapshot -> type/click -> validation loop, multi-step forms, challenge checkpoints, network correlation | `node scripts/cli-smoke-test.mjs` plus `node scripts/login-fixture-live-probe.mjs` for invalid-submit, MFA, and persistence branches | extension unavailable at start if extension mode is audited separately |
+| `opendevbrowser-login-automation` | `workflow` | `./skills/opendevbrowser-login-automation/scripts/validate-skill-assets.sh` | login branching, invalid credential handling, MFA step-up, session validation | `node scripts/login-fixture-live-probe.mjs` | none; proof lane is repo-local |
+| `opendevbrowser-product-presentation-asset` | `workflow` | `./skills/opendevbrowser-product-presentation-asset/scripts/validate-skill-assets.sh` | `product-video run`, asset-pack assembly, screenshots, evidence mapping | `node scripts/product-video-fixture-live-probe.mjs` | none; proof lane is repo-local |
+| `opendevbrowser-research` | `workflow` | `./skills/opendevbrowser-research/scripts/validate-skill-assets.sh` | `research run`, timebox resolution, multi-source artifact generation | `opendevbrowser research run --topic "<topic>" --source-selection auto --mode json` via `scripts/skill-runtime-audit.mjs` | auth walls, rate limits, upstream source failure |
+| `opendevbrowser-shopping` | `workflow` | `./skills/opendevbrowser-shopping/scripts/validate-skill-assets.sh` | `shopping run`, offer normalization, market analysis, direct provider workflows | `node scripts/provider-direct-runs.mjs` | auth walls, anti-bot challenges, rate limits, upstream outage |
+
+## Shared runtime families
+
+| Runtime family | Representative CLI surface | Representative tool surface | Real task | Proof lane |
+|---|---|---|---|---|
+| Session lifecycle and daemon control | `serve`, `status`, `launch`, `disconnect` | `opendevbrowser_launch`, `opendevbrowser_status`, `opendevbrowser_disconnect` | start a session, inspect state, then close it cleanly | `node scripts/cli-smoke-test.mjs`, `node scripts/live-regression-direct.mjs` |
+| Navigation and interaction | `goto`, `wait`, `click`, `hover`, `press`, `type`, `select`, `scroll`, `scroll-into-view` | matching `opendevbrowser_*` interaction tools | move through an interactive page and complete an action loop | `node scripts/cli-smoke-test.mjs`, `node scripts/login-fixture-live-probe.mjs` |
+| Pointer controls | `pointer-move`, `pointer-down`, `pointer-up`, `pointer-drag` | `opendevbrowser_pointer_move`, `opendevbrowser_pointer_down`, `opendevbrowser_pointer_up`, `opendevbrowser_pointer_drag` | low-level pointer movement for challenge-aware or canvas-aligned surfaces | `node scripts/login-fixture-live-probe.mjs` plus pointer-aware source inventory |
+| DOM and extraction | `dom-html`, `dom-text`, `dom-attr`, `dom-value`, `dom-visible`, `dom-enabled`, `dom-checked` | matching `opendevbrowser_dom_*` tools | extract structured fields and validate state transitions | `node scripts/cli-smoke-test.mjs`, `node scripts/product-video-fixture-live-probe.mjs` |
+| Targets and pages | `targets-list`, `target-use`, `target-new`, `target-close`, `page`, `pages`, `page-close` | matching `opendevbrowser_target_*` tools | switch tabs/pages during a real flow | `node scripts/cli-smoke-test.mjs` |
+| Diagnostics and export | `perf`, `screenshot`, `console-poll`, `network-poll`, `clone-page`, `clone-component`, `debug-trace-snapshot` | matching diagnostics/export tools | capture browser evidence after a real workflow step | `node scripts/cli-smoke-test.mjs`, `node scripts/live-regression-direct.mjs` |
+| Browser replay and desktop observation | `screencast-start`, `screencast-stop`, `desktop-status`, `desktop-capture-desktop` | matching replay/desktop tools | capture time-based browser evidence or read-only sibling desktop state during an investigation | `node scripts/live-regression-direct.mjs`, `node scripts/docs-drift-check.mjs` |
+| Workflow wrappers | `research run`, `shopping run`, `product-video run` | `opendevbrowser_research_run`, `opendevbrowser_shopping_run`, `opendevbrowser_product_video_run` | run end-to-end research, shopping, and product asset workflows | `scripts/skill-runtime-audit.mjs`, `scripts/provider-direct-runs.mjs`, `scripts/product-video-fixture-live-probe.mjs` |
+| Canvas and design relay | `canvas` CLI and `/canvas` commands | `opendevbrowser_canvas` | open a canvas session, mutate safely, render preview, and validate feedback | `node scripts/live-regression-direct.mjs`, `node scripts/canvas-competitive-validation.mjs` |
+| Extension `/ops` relay | `/ops` session, target, nav, interact, pointer, DOM, overlay, preview bridge envelopes | extension-backed tool parity via daemon | attach the extension, verify `/ops` readiness, and run real command loops | `node scripts/live-regression-direct.mjs`, `node scripts/docs-drift-check.mjs` |
+| Skill discovery and loading | bundled `skills/` directories and loader search paths | `opendevbrowser_skill_list`, `opendevbrowser_skill_load` | enumerate and load the canonical packs | `node scripts/skill-runtime-audit.mjs` |
+
+## Audit domains
+
+| Audit domain | Covers | Proof lanes | Contract tests | Targeted reruns |
+|---|---|---|---|---|
+| `skills-assets-discovery` | bundled skills, validator assets, loader discovery, install targets | `skill-discovery`, `docs-drift` | `tests/skill-loader.test.ts`, `tests/skill-workflow-packs.test.ts`, `tests/cli-skills-installer.test.ts` | `./skills/opendevbrowser-best-practices/scripts/validate-skill-assets.sh`, `node scripts/skill-runtime-audit.mjs --smoke --out artifacts/skill-runtime-audit/smoke.json` |
+| `cli-tools-surface` | CLI args/help, tool registry, surface inventory docs | `docs-drift`, `cli-smoke` | `tests/cli-help.test.ts`, `tests/cli-help-parity.test.ts`, `tests/daemon-commands.integration.test.ts` | `node scripts/docs-drift-check.mjs`, `node scripts/cli-smoke-test.mjs` |
+| `scripts-and-governance` | audit scripts, workflow router, drift and robustness policy | `docs-drift`, `best-practices-robustness`, `skill-discovery` | `tests/docs-drift-check.test.ts`, `tests/skill-runtime-audit.test.ts`, `tests/audit-zombie-files.test.ts` | `./skills/opendevbrowser-best-practices/scripts/run-robustness-audit.sh`, `node scripts/docs-drift-check.mjs` |
+| `browser-snapshot-interaction` | browser lifecycle, snapshot refs, DOM reads, click/type flows | `cli-smoke`, `login-fixture`, `product-video-fixture` | `tests/tools-workflows.test.ts`, `tests/tools-workflows-branches.test.ts`, `tests/browser-manager.test.ts` | `node scripts/cli-smoke-test.mjs`, `node scripts/login-fixture-live-probe.mjs --out artifacts/skill-runtime-audit/lanes/login-fixture.json` |
+| `canvas-annotate-design` | `/canvas`, annotate delivery, design-agent workflow, extension canvas runtime | `live-regression`, `canvas-competitive` | `tests/canvas-command-inventory.test.ts`, `tests/canvas-live-workflow-script.test.ts`, `tests/annotate-live-probe-script.test.ts` | `node scripts/live-regression-direct.mjs --out artifacts/skill-runtime-audit/lanes/live-regression.json`, `node scripts/canvas-competitive-validation.mjs --out artifacts/skill-runtime-audit/lanes/canvas-competitive.json` |
+| `extension-relay-cdp` | extension `/ops`, relay contracts, legacy CDP, target-session routing | `live-regression`, `docs-drift` | `tests/parity-matrix.test.ts`, `tests/relay-server.test.ts`, `tests/ops-browser-manager.test.ts` | `node scripts/live-regression-direct.mjs --out artifacts/skill-runtime-audit/lanes/live-regression.json`, `./skills/opendevbrowser-best-practices/scripts/odb-workflow.sh cdp-channel-check` |
+| `providers-macros-workflows` | providers, workflow wrappers, macro resolve or execute, product-video | `provider-direct`, `research-live`, `product-video-fixture` | `tests/provider-direct-runs.test.ts`, `tests/macro-resolve.test.ts`, `tests/cli-macro-resolve.test.ts` | `node scripts/provider-direct-runs.mjs --smoke --out artifacts/skill-runtime-audit/lanes/provider-direct.json`, `npm run test -- tests/macro-resolve.test.ts tests/cli-macro-resolve.test.ts` |
+| `challenges-and-guardrails` | challenge coordinator, browser-scoped `challengeAutomationMode` computer-use boundary, login/form challenge branches, blocker envelopes | `login-fixture`, `provider-direct`, `research-live`, `live-regression` | `tests/challenges-capability-matrix.test.ts`, `tests/browser-manager-challenge-runtime-handle.test.ts`, `tests/ops-browser-manager-challenge-runtime-handle.test.ts` | `node scripts/login-fixture-live-probe.mjs --out artifacts/skill-runtime-audit/lanes/login-fixture.json`, `node scripts/provider-direct-runs.mjs --smoke --out artifacts/skill-runtime-audit/lanes/provider-direct.json` |
+| `runtime-infrastructure` | bootstrap, runtime factory, daemon client, shared hub utilities | `cli-smoke`, `live-regression` | `tests/core-bootstrap.test.ts`, `tests/providers-runtime-factory.test.ts`, `tests/daemon-client.test.ts` | `npm run test -- tests/core-bootstrap.test.ts tests/providers-runtime-factory.test.ts tests/daemon-client.test.ts`, `node scripts/cli-smoke-test.mjs` |
+
+## Acceptance rules
+
+- A pack is `fail` when discovery/load parity fails, the validator fails, the required live probe is missing, or the real workflow probe reports a repo-owned defect.
+- A pack is `env_limited` only when the probe exists, ran, and hit a real external blocker.
+- `scripts/skill-runtime-audit.mjs` smoke mode may use isolated temp harnesses, but smoke `live-regression` failures must be rerun against the current configured daemon before they count as repo-owned defects.
+- Full-mode `provider-direct` and `live-regression` lanes must reuse the current configured daemon/environment so extension and auth-backed coverage is real when available.
+- Non-release provider-direct profiles may emit skipped coverage-gap advisories when reduced mode coverage omits provider ids; release-gate mode upgrades the same gap to a failure.
+- A shared runtime family is considered covered only when at least one live proof lane exercised that family during the same audit pass.
+- An audit domain is `fail` when any mapped proof lane or mapped pack reports a repo-owned defect.
+- An audit domain is `env_limited` when it has no repo defects but a mapped proof lane hit real external blockers.