zerocheck 0.1.6 → 0.1.8

package/README.md

@@ -2,14 +2,15 @@
 
 Zerocheck turns your team’s manual release checklist into repeatable browser tests.
 
-Paste a checklist, review one drafted check and browser verification result per actionable item, then save readable YAML in your repository. Run those same files locally, in GitHub Actions, or in a hosted browser. Results, recordings, attempts and healing history are available in the web app, with screenshots and a summary in your pull request.
+Paste a checklist, review one drafted check and browser verification result per actionable item, then save readable YAML in your repository. Run those same files locally, in GitHub Actions, or in a hosted browser. Results, recordings, attempts and healing history are available in the web app, with a summary and an authenticated evidence link in your pull request.
 
 ## Setup
 
 Requires Node.js 20.19 or newer and a Zerocheck project/account. Create a project in the web app’s Settings, then authorize a device or create a project token.
 
 ```sh
-npm install --save-dev zerocheck@0.1.6
+npm install --save-dev --save-exact zerocheck@0.1.8
+export ZEROCHECK_API=https://your-instance.example.com
 npx zerocheck login
 npx zerocheck init --project your-project-id --url http://localhost:3000
 npx zerocheck install
@@ -17,7 +18,7 @@ npx zerocheck install
 
 `install` downloads Chromium for local execution. Linux CI can use `zerocheck install --with-deps`. Hosted execution does not require a browser on your machine, but still uses the CLI and your Zerocheck account.
 
-For unattended use, set `ZEROCHECK_TOKEN` to the project token. It overrides the device token saved in `~/.zerocheck/config.json`. `ZEROCHECK_API` selects your Zerocheck server and defaults to `https://app.tryzerocheck.com`. Never commit tokens or resolved test credentials.
+Set `ZEROCHECK_API` to your customer instance before login; there is no default public service. Device login stores that instance with its token in `~/.zerocheck/config.json` (mode 0600). A different destination is rejected before any token is sent. Legacy unbound device tokens require logout and a new login. For unattended use, explicitly set both `ZEROCHECK_TOKEN` and `ZEROCHECK_API`; CI never borrows a saved device destination. Instance URLs must be HTTPS origins (HTTP localhost is permitted for development); API redirects are rejected. Never commit tokens or resolved test credentials.
 
 ## Import your release checklist
 
@@ -55,7 +56,7 @@ Answer missing inputs together in a JSON file keyed by the displayed item IDs:
 npx zerocheck import --draft IMPORT_ID --answers answers.json
 ```
 
-Only the answered unresolved items are drafted/verified again. Already completed items are preserved. Edit a drafted test’s expected behavior as a reviewed YAML change; changed YAML needs a new verification.
+Only the answered unresolved items are drafted/verified again. To retry a failed item that has no drafted definition, use an empty answer, for example `{"item-3":""}`. Already completed definitions and unselected items are preserved. An existing executable draft is verified unchanged rather than regenerated; missing credentials and configuration or ambiguous-step errors show their respective remedies. Edit a drafted test’s expected behavior as a reviewed YAML change; changed YAML needs a new verification.
 
 ## One repository format, named environments
 
@@ -82,11 +83,12 @@ environments:
   production:
     url: https://example.com
 execution:
-  retries: 1
 ```
 
 Only explicitly referenced secret variables are read for execution. Missing variables fail clearly. Listing, reading, saving and validating checks do not require execution secrets.
 
+Ordinary HTTP(S) redirects and embedded checkout flows need no domain list. A nonempty environment `allowed_origins` list is an optional exact-origin restriction: only your app origin and its listed origins can be interacted with. Omit it or leave it empty for unrestricted browsing; list only your app origin for app-only browsing. Credentials remain restricted separately: `secret_origins` grants each configured credential exact additional destinations during login steps only. If a browser restriction is set, it must also include those login destinations. Hosted private-network protections remain in effect.
+
 Each check is one `zerocheck/v1` YAML file:
 
 ```yaml
@@ -98,7 +100,11 @@ steps:
   - Verify the Release notes heading is visible
 ```
 
-Repository files are authoritative. The web app shows synced revisions and exact run snapshots; it does not maintain a second independently edited suite. A run always uses the selected files’ current contents. Empty, missing, duplicate or invalid selections fail before execution; `run` never invents starter checks.
+Repository files are authoritative for CLI-managed tests; tests created and saved in the web app are edited there. Sync cannot overwrite a web-managed definition with different contents. A CLI run uses the selected files’ current contents, and each saved result retains its exact revisions. Empty, missing, duplicate or invalid selections fail before execution; diagnostics list all selected path and YAML errors together. Correct the entire selection before running. Explicit authoring/save supplies an omitted `version`; reading or running an existing file never rewrites it. `run` never invents starter checks.
+
+Checks may end with `Verify` or a condition-based `Wait for`; timed waits cannot supply the outcome. `zerocheck validate` prints parsed steps and effective negative defaults without starting a browser. Quote an entire YAML step containing a literal `#`; plain step comments are rejected in test files and `zerocheck.yaml` login steps.
+
+Negative quoted-text Verify defaults to bounded phrase matching and a 3-second observation window. `Paid` does not match `Unpaid`; explicit `exact text`, credential comparisons and `for N seconds` retain their precise meaning. New authoring writes defaults explicitly; existing saved bytes/history remain unchanged. Results and PR comments report actual observation mode/window. Unsupported semantic negatives need a quoted-text correction, and visible unreadable frames cannot prove absence. Positive text/URL/request Wait conditions use deterministic checks; semantic Wait still uses AI and supplies no independent target evidence.
 
 ## Run and inspect
 
@@ -112,13 +118,13 @@ npx zerocheck results RUN_ID
 npx zerocheck doctor --env staging
 ```
 
-`local` is the default runner and can reach localhost and private networks available to your machine/CI job. `hosted` uses Zerocheck’s existing hosted browser and can reach targets available to that worker; it cannot reach your laptop’s localhost, and Zerocheck does not create a network tunnel. Runner location, environment and trigger are independent. Hosted interruptions produce an explicit error and require a new user-triggered run.
+`local` is the default runner and can reach localhost and private networks available to your machine/CI job. `hosted` uses Zerocheck’s existing hosted browser and can reach targets available to that worker; it cannot reach your laptop’s localhost, and Zerocheck does not create a network tunnel. Runner location, environment and trigger are independent. Local CLI/MCP requires a trusted single-user OS, Actions an isolated job, and hosted pilots a dedicated OS/container boundary without untrusted co-tenants; the browser control endpoint is local but not authenticated. Hosted interruptions produce an explicit error and require a new user-triggered run.
 
-Results retain every attempt. A failed attempt followed by a pass, or a transient grounding failure recovered in the same browser, is **flaky / passed on retry**, even when CI succeeds. Step recovery details remain visible. `--fail-on-flaky` makes blocking flaky checks fail CI. `blocks_merge: false` explicitly makes a check non-blocking; setup errors and incomplete execution still fail the command.
+Each selected check executes once in its own browser and retains its step trace. After a stuck browser is confirmed dead, the next independent check can start; whole-job cancellation or unconfirmed termination stops the selection. Historical records retain every older attempt. A transient grounding failure recovered within a check is reported as **passed after recovery** and marked flaky, even when CI succeeds. Step recovery details remain visible. `--fail-on-flaky` makes blocking flaky checks fail CI. `blocks_merge: false` explicitly makes a check non-blocking; setup errors and incomplete execution still fail the command.
 
-Exit codes: **0** completed within the selected merge policy; **1** blocking test failure (or strict flaky result); **2** setup, incomplete execution or infrastructure error. Import exits nonzero while any actionable item remains unverified.
+Exit codes: **0** completed within the selected merge policy; **1** blocking test failure (or strict flaky result); **2** setup, incomplete execution, infrastructure or result-publication error. A failed upload, including HTTP 401/409, preserves the local browser verdict and evidence, records the publication error separately, and adds a JUnit publication error. It never repeats the browser actions. Import exits nonzero while any actionable item remains unverified.
 
-Learned targets persist in `.zerocheck/cache`. Local CLI and MCP share this cache; hosted execution uses its own persistent cache. A healed target is separate from a retry, and the result records target changes and their validation. Forms and login do not block target repairs. The engine saves only independently validated targets after a complete unchanged check; model-only outcomes leave repairs pending. It never reruns a submission solely to validate healing. Recovery before dispatch stays in the current browser. A full restart after application activity needs a trusted isolated-state integration; `retries: 1` does not itself establish safe replay. Include `.zerocheck/` in `.gitignore`; `init` adds it.
+Learned targets persist in `.zerocheck/cache`. Local CLI and MCP share this cache; hosted execution uses its own persistent cache. A healed target is separate from a retry, and the result records target changes and their validation. Forms and login do not block target repairs. The engine saves only independently validated targets after a complete unchanged check; model-only evidence leaves the particular repair pending. A different independently proven input can still become reusable when the test also contains a semantic assertion. It never reruns a submission solely to validate healing. Recovery before dispatch stays in the current browser. The runner never restarts a whole check automatically. Provider transport failures can retry once within the remaining deadline; auth/configuration errors and failed expectations do not. Legacy `retries: 0` disables transient grounding retries; pre-dispatch target recovery remains available. Include `.zerocheck/` in `.gitignore`; `init` adds it.
 
 ## GitHub Actions and PR comments
 
@@ -126,11 +132,11 @@ Learned targets persist in `.zerocheck/cache`. Local CLI and MCP share this cach
 npx zerocheck init --github-actions
 ```
 
-The generated `.github/workflows/zerocheck.yml` pins the CLI version, restores the local target cache, runs staging checks, and runs the reporter and artifact upload even if checks fail. Set the repository secret `ZEROCHECK_TOKEN`, configure staging in `zerocheck.yaml`, and add only the test-secret variables referenced there. For localhost testing, add your app startup and fixture setup before the run step.
+The generated `.github/workflows/zerocheck.yml` pins the CLI version, restores the local target cache, runs staging checks, and runs the reporter and artifact upload even if checks fail. Set the repository secret `ZEROCHECK_TOKEN` and repository variable `ZEROCHECK_API` to the same customer instance, configure staging in `zerocheck.yaml`, and add only the test-secret variables referenced there. For localhost testing, add your app startup and fixture setup before the run step.
 
 The workflow uses `pull_request`, not `pull_request_target`. It grants `contents: read` and `pull-requests: write`; it does not execute fork code with elevated secrets. Fork or read-only-token jobs keep their job summary and explain why a PR comment could not be posted.
 
-The comment contains pass/fail/flaky results, failing steps, selected screenshots, recordings, environment, commit and a link to full results. One Zerocheck bot comment is updated per PR. Results from older commits cannot replace the current comment. `report` explicitly creates unguessable share links for the selected PR screenshots/recordings, so GitHub can render them without a Zerocheck login; other artifacts retain normal access controls.
+The comment contains pass/fail/flaky results, failing steps, environment, commit and a link to authenticated full results and private evidence. One Zerocheck bot comment is updated per PR. Results from older commits cannot replace the current comment. Neither report adapter publishes screenshots or recordings. Public artifact links, including old links, are disabled; private evidence is retained. Offline summaries still work when instance setup failed.
 
 ```sh
 npx zerocheck report --file .zerocheck/latest-run.json
@@ -149,13 +155,13 @@ Use the installed CLI as a stdio MCP server, with an explicit project directory:
   "mcpServers": {
     "zerocheck": {
       "command": "npx",
-      "args": ["--yes", "zerocheck@0.1.6", "mcp", "--project-dir", "/absolute/path/to/project"]
+      "args": ["--yes", "zerocheck@0.1.8", "mcp", "--project-dir", "/absolute/path/to/project"]
     }
   }
 }
 ```
 
-Authenticate with `zerocheck login` first or provide `ZEROCHECK_TOKEN` through the MCP host’s environment/secret settings. stdout contains only MCP protocol messages; diagnostics use stderr.
+Authenticate with `zerocheck login` first or provide both `ZEROCHECK_TOKEN` and `ZEROCHECK_API` through the MCP host’s environment/secret settings. stdout contains only MCP protocol messages; diagnostics use stderr.
 
 Tools: `list_checks`, `read_check`, `save_check`, `import_checklist`, `answer_import`, `adopt_import`, `verify_check`, `run_checks`, `get_results`, and `cancel`. Existing check replacements require their current `expectedRevision`. File operations stay within the configured project, including symbolic-link checks.
 
@@ -163,10 +169,12 @@ Browser operations return job handles. Poll `get_results` with `kind: "job"` and
 
 ## Data sent to Zerocheck
 
-Local execution sends selected check YAML, the page context needed for AI (including screenshots/accessibility content), and run results/artifacts to Zerocheck. The browser still runs on your machine. Hosted execution additionally sends the URL, login steps and explicitly configured test secrets needed to run the check. The CLI never uploads your entire process environment. Browser screenshots may contain visible application data, and `report` shares the selected PR assets as described above.
+Local execution sends selected check YAML, the page context needed for AI (including screenshots/accessibility content), and run results/artifacts to Zerocheck. The browser still runs on your machine. Hosted execution additionally sends the URL, login steps and explicitly configured test secrets needed to run the check. The CLI never uploads your entire process environment. Browser screenshots may contain visible application data. `report` links to authenticated evidence without creating public artifact URLs. Hosted server-owned credential references must use `ZEROCHECK_TEST_*`; locally resolved references may use other explicitly chosen test-variable names.
 
 ## Repository development
 
-The CLI bundles the shared engine source into `dist/index.js`; it has no unpublished engine-package dependency. From this repository, run `npm run build`, `npm run check`, and `npm test` in `packages/cli`. Tests include a real MCP client communicating with the bundled CLI over stdio. The generated workflow and documentation use version `0.1.6`; update that pin together with the package version for a release.
+The CLI bundles the shared engine source into `dist/index.js`; it has no unpublished engine-package dependency. From this repository, run `npm run build`, `npm run check`, and `npm test` in `packages/cli`. Tests include a real MCP client communicating with the bundled CLI over stdio. The generated workflow and documentation use version `0.1.8`; update that pin together with the package version for a release.
+
+Hosted runs and imports return a queued handle when browsers are occupied; CLI and MCP keep polling until they finish. The queue holds 20 waiting jobs for up to five minutes and never automatically resumes interrupted jobs after restart. Recent drafts is available in the web app’s Checks page.
 
-Hosted runs and imports return a queued handle when browsers are occupied; CLI and MCP keep polling until they finish. The queue holds 20 waiting jobs for up to five minutes and never automatically resumes interrupted jobs after restart. Recent imports is available in the web app’s Checks page.
+If a local run finishes but its private upload fails, keep its ID and retry only publication with `zerocheck results <id> --upload`. For a saved local import use `zerocheck import --upload <id>`. These commands preserve the browser verdict and evidence and do not execute tests. They require the original configured instance and project. Reconcile an unresolved hosted run with `zerocheck results <id>` before starting another; for an import use `zerocheck import --status <id>`. These reads do not adopt tests. A lost response is not permission to replay a checkout.