@plune-ai/cli 0.27.0 → 0.28.0

package/README.md

@@ -29,6 +29,8 @@ npx -y @plune-ai/cli run
 
 Requires **Node.js ≥ 20**.
 
+The completion cache uses `better-sqlite3`, an optional dependency with prebuilt binaries for Node 20 to 26 on Windows (x64, arm64), macOS and Linux. If the binary cannot be downloaded (proxy, offline mirror) and there is no C++ toolchain with Python to build it, npm skips it: the install still succeeds and `plune run` prints `completion cache disabled` and runs without the cache. To turn the cache on, fix the download or install the build tools (Visual Studio Build Tools on Windows), then run `npm rebuild better-sqlite3 -g`.
+
 ## Quickstart
 
 ```bash
@@ -113,29 +115,43 @@ the last resort, used only when `PLUNE_TOKEN` is set nowhere else. Nothing write
 
 | Command | Summary |
 | ------- | ------- |
-| `plune run` | Run the suite. Flags: `--dry-run` (price the run from the model's rates — no provider key, no network), `--only <id\|tag>` (repeatable), `--bail`, `--no-cache`, `--concurrency <n>`, `--format console\|json\|markdown`, `-o, --output <file>`. |
+| `plune run` | Run the suite. Flags: `--dry-run` (price the run from the model's rates — no provider key, no network), `--only <id\|tag>` (repeatable), `--bail`, `--no-cache`, `--cache-strict`, `--cache-max-age <duration>`, `--concurrency <n>`, `--format console\|json\|markdown`, `-o, --output <file>`. |
 | `plune report` | Re-render the most recent run. Flags: `--format`, `-o`. |
 | `plune diff <baseline> <current>` | Compare two `plune run --format json` outputs and report pass→fail regressions. Flags: `--fail-on-regression`, `--format`, `-o`. |
 | `plune init` | Scaffold `plune.yaml`, a sample dataset, and `.env.example`. Flags: `--yes` (non-interactive), `--force`. |
 | `plune login` | Save a [Plune platform](https://plune.ai/platform) API token so `sync` and `ingest` can reach it. The token is **checked against the API before it is saved**, so a wrong one fails here rather than two commands later. Get one at `https://beta.plune.ai` → Account → API tokens. Flags: `--token <token>` (omit to paste it or pipe it via stdin), `--skip-verify` (save without checking, for offline setup). In CI, skip this step: every platform command reads `PLUNE_TOKEN` from the environment first, and the saved login only when it is not set. |
 | `plune logout` | Remove the saved token. |
 | `plune sync` | Upload the latest local run to the platform. Flags: `--file <path>` to send a specific run JSON, `--fail-on-failures` to read the run's summary back once after the upload and exit with [the gate](#a-release-gate---fail-on-failures)'s code. Each upload carries an `Idempotency-Key` (SHA-256 of the body), so sending the same run again returns the same run rather than creating a second one. |
-| `plune run import <file>` | Turn a **JUnit XML** or **Playwright JSON** report, or an **Allure `allure-results` folder**, into a run in Plune. Needs no provider key — nothing is generated. Flags: `--format junit\|playwright-json\|allure` (detected from the file when omitted; a folder is Allure), `--key <externalKey>` to land several reports in one run (with `PLUNE_SHARED_RUN=1` — see below), `--create` to offer unmatched tests to the review queue, `--fail-on-missing-attachments` to exit 1 when a file a JUnit report names with `[[ATTACHMENT\|path]]`, or an Allure result names as a `source`, is not there (the results are sent either way), `--upload-concurrency <n>` for files uploaded at once ([1 to 32, default 16](#many-screenshots-upload-concurrency-and-retries)), `--fail-on-failures` to exit with [the gate](#a-release-gate---fail-on-failures)'s code on the summary the platform froze when this import finished the run. |
-| `plune run start` | Open a platform run — or join the one already carrying `--key` — and print its id. Flags: `--key <externalKey>` (generated when absent), `--json` for one machine-readable line. |
+| `plune run import <file>` | Turn a **JUnit XML** or **Playwright JSON** report, or an **Allure `allure-results` folder**, into a run in Plune. Needs no provider key — nothing is generated. Flags: `--format junit\|playwright-json\|allure` (detected from the file when omitted; a folder is Allure), `--name <text>` to call the run something other than the [folder-based default](#what-the-run-is-called), `--key <externalKey>` to land several reports in one run (with `PLUNE_SHARED_RUN=1` — see below), `--create` to offer unmatched tests to the review queue, `--fail-on-missing-attachments` to exit 1 when a file a JUnit report names with `[[ATTACHMENT\|path]]`, or an Allure result names as a `source`, is not there (the results are sent either way), `--upload-concurrency <n>` for files uploaded at once ([1 to 32, default 16](#many-screenshots-upload-concurrency-and-retries)), `--fail-on-failures` to exit with [the gate](#a-release-gate---fail-on-failures)'s code on the summary the platform froze when this import finished the run. |
+| `plune run start` | Open a platform run — or join the one already carrying `--key` — and print its id. Flags: `--key <externalKey>` (generated when absent), `--name <text>` to call the run something other than the [default](#what-the-run-is-called), `--json` for one machine-readable line. |
 | `plune run finish <id>` | Close a platform run. This is what the reporter tells you to do for a run it had to leave open. Flags: `--terminate` to record it as cut short, `--reason <text>`, `--fail-on-failures` to exit with [the gate](#a-release-gate---fail-on-failures)'s code on the summary the platform froze. |
-| `plune run exec -- <command>` | Open a run, run the command inside it, close the run — and exit with whatever the command returned. Sets `PLUNE_SHARED_RUN` for you, so anything reporting inside joins that run. Flags: `--key <externalKey>`, `--fail-on-failures` to exit with [the gate](#a-release-gate---fail-on-failures)'s code instead — or with the command's own when no result reached the run. |
+| `plune run exec -- <command>` | Open a run, run the command inside it, close the run — and exit with whatever the command returned. Sets `PLUNE_SHARED_RUN` for you, so anything reporting inside joins that run. Flags: `--key <externalKey>`, `--name <text>` (as on `run start`), `--fail-on-failures` to exit with [the gate](#a-release-gate---fail-on-failures)'s code instead — or with the command's own when no result reached the run. |
 | `plune run delete <id>` | Delete a run and everything it produced — its results and the review-queue entries it raised. Approved test cases and the audit log stay. **Recoverable for 30 days** (an owner of the project can put it back from Settings → Bin), then gone for good. No prompt: it is your data, and this command belongs in scripts. |
 | `plune run report` | Replay `.plune/pending-results.jsonl` — send what the reporter could not. The lines of one reporter run go back as one run, with the commit and branch the reporter recorded. Once every result has gone the file is renamed to `pending-results.<time>.sent.jsonl`; if anything did not go, it stays as it was. Flags: `--file <path>`. |
 | `plune ingest [dir]` | Record a [Cairn](https://github.com/plune-ai/cairn) run in Plune. Omit `[dir]` for the newest run under `./runs`, or name the directory holding `report.json`. Generated cases arrive as **review proposals** — nothing is created until a person approves it. |
 | `plune pull [file]` | Write the project's test cases as **one Markdown document** — Testomat's classical format, plus Plune's own columns — to `plune/cases.md` (or `[file]`). Refuses to overwrite a file git sees as modified; `--force` overrides, `--suite <id>` takes one suite or folder and what is under it. |
 | `plune push [file] [--dry-run]` | Send the document back. Cases are matched by `id`; a block without one becomes a draft; an unknown id is refused by line; **nothing is deleted**. Prints the report — created, updated, unchanged, refused, warnings — and with `--dry-run` writes nothing. Exit `4` when anything was refused. |
 | `plune requirements import <file.md>` | Import requirements from a **Markdown file** (an SRS, a PRD, a page exported from a wiki). With no mapping flags it lists the tables and sections the file has and exits `0`; with `--table` or `--sections` it is a **dry run** that writes nothing until you add `--yes`. See [Requirements from a Markdown file](#requirements-from-a-markdown-file). Needs a Plune platform that serves `POST /v1/projects/:id/requirements/import` (plune-ai/plune#1621). |
+| `plune verify` | Compare the project's **automated cases** with the Playwright tests in the code, **read-only** (plune-ai/plune#1291). Prints both sides: cases in Plune that the code no longer has (already `detached`, or one a full run would detach) and tests in the code that Plune has no case for. A test is matched by the id it names (`PluneId` annotation or `@P<id>` in its title), else by its `path-title` key. Nothing is written to Plune and no id is added to your source. Playwright only. Flags: `--from <file>` (a saved `playwright test --list --reporter=json`, instead of running Playwright), `--playwright-config <path>`, `--json`, `--fail-on-drift` (exit `1` when either side has something the other lacks). |
 | `plune plan grep <id>` | Print a [test plan](https://docs.plune.ai/platform/plans/) as **one `--grep` pattern** — `npx playwright test --grep "$(plune plan grep <id>)"` or `npx vitest run -t "$(plune plan grep <id>)"` runs exactly what the plan collects. Each case contributes its title path from the `path-title` key the reporter wrote (the file left out, `[ >#]+` between the segments — a space for Playwright, jest, mocha and vitest ≤ 4, ` > ` for vitest 5), escaped, joined with `\|`; a case without one goes in by title. The pattern is all that reaches stdout; the summary — how many cases, how many keyed — goes to stderr. An empty plan prints `(?!)`, which matches nothing, because an empty `--grep` would run everything. |
 | `plune run grep <run> [--failed]` | Print a run's **automated** cases as one `--grep` pattern, built the way `plan grep` builds a plan's — so the commands Plune shows for a run work as they are: `npx playwright test --grep "$(plune run grep <run-id> --failed)"` re-runs an automated run's failures where your tests run (Plune starts no CI), and `PLUNE_RUN=<key> npx playwright test --grep "$(plune run grep <key>)"` runs the automated half of a mixed run launched without a plan. `<run>` is the run's id (a UUID) or its key (`rk_` and ten hex); anything else is refused with exit `2` before anything is sent, and a key no run carries exits `1`. The cases are the ones the run expected — cases added on its page among them, run or not — or holds a result of; manual cases are left out. `--failed` keeps the cases whose latest result the platform counts as failed or broken, so a muted case's failure stays out, as from **Re-run N failed**. Only the pattern reaches stdout; how many cases were taken and how many left out goes to stderr; nothing taken prints `(?!)`. Needs a Plune that serves `GET /v1/runs/:id/cases` (plune-ai/plune#1333). |
 
 Global flags: `-c, --config <path>` · `-v, --verbose` · `--no-color`.
 
-**Exit codes:** `0` everything passed · `1` an assertion failed · `2` configuration or execution error · `3` `pull` refused to overwrite uncommitted changes · `4` `push` had refusals.
+**Exit codes:** `0` everything passed · `1` an assertion failed · `2` configuration or execution error · `3` `pull` refused to overwrite uncommitted changes · `4` `push` had refusals · `5` `plune run --cache-strict` found rows with no cache entry (it wins over `1` and `2`: the run is incomplete).
+
+### Completion cache
+
+`plune run` keeps model answers in `.plune/cache.db`, keyed by provider, model, temperature, `max_tokens`, the resolved prompt and the cache policy version. Never by time or run id.
+
+- **What is stored.** An answer is cached only when it has text, was not cut off by the token limit (`finish_reason` `length`), carries no provider error, and has the format the eval expects (a `json-schema` eval needs JSON in the answer). An answer that is well formed but wrong (the schema or a `contains` check fails) *is* cached: the model would say the same thing again.
+- **What is dropped.** A stored answer that no longer has the expected format is deleted and the model is asked once in the same run. A provider failure deletes nothing. A damaged entry or file is a miss, never a crash.
+- **Policy version.** `CACHE_POLICY_VERSION` is part of the key. When it is raised, older entries stop answering, are dropped, and the run says `cache policy changed, N entries are stale` once on stderr.
+- **`--cache-strict`.** Every row must be in the cache; the model is not called for completions (judge assertions such as `llm-judge` still call it). A miss ends the run with exit code `5` and lists the rows with no entry. Not combinable with `--no-cache` (exit `2`), and neither is `--cache-max-age`. An entry with the wrong format is left in place in this mode (there is no retry).
+- **`--cache-max-age <duration>`.** `90s`, `15m`, `12h`, `7d` or milliseconds. An older entry is a miss; a fresh one is a hit.
+- **Summary.** The console report and the JSON (`summary.cache`) show rows `from_cache`, `from_model`, `missed` and `errored` (the provider failed; the four add up to the rows run), and the tokens and cost the cache saved (`saved`, the sum of each row's `cache_saved`).
+
+Design: `docs/features/C7-completion-cache/adr/0001-completion-cache-hardening.md`.
 
 ### Requirements from a Markdown file
 
@@ -454,7 +470,7 @@ tests to compare against, so the flag means nothing there.
 A run nobody named is called after the directory, the minute it started and where it ran —
 `plune · 2026-09-15 15:26 · ci` — so a row in the list reads without being opened. The clock is the
 reporting machine's own; `ci` comes from the `CI` variable every hosted runner sets. To call it
-something else, set `PLUNE_RUN_TITLE`; `PLUNE_ENV` and `PLUNE_LABELS` mark where it ran and how, and
+something else, set `PLUNE_RUN_TITLE` (`--name <text>` on `plune run import`, `run start` and `run exec` does the same and wins); `PLUNE_ENV` and `PLUNE_LABELS` mark where it ran and how, and
 those stay marks rather than becoming part of the name.
 
 ## Optional: keep a history