cuttlely 1.1.0 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +8 -8
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -12,10 +12,10 @@ Requires Node.js 20 or newer, and Docker.
|
|
|
12
12
|
npx cuttlely start
|
|
13
13
|
```
|
|
14
14
|
|
|
15
|
-
That pulls `ghcr.io/cuttlely/cuttlely
|
|
15
|
+
That pulls `ghcr.io/cuttlely/cuttlely`, tagged with the same version as the `cuttlely` package you ran, keeps data in the Docker volume `cuttlely-data`, publishes port **43117**, waits until `http://127.0.0.1:43117/api/v1/ping` answers `pong`, prints `http://localhost:43117`, and opens it.
|
|
16
16
|
|
|
17
17
|
```bash
|
|
18
|
-
npx cuttlely start --port 8080 --tag 1.
|
|
18
|
+
npx cuttlely start --port 8080 --tag 1.3.0 --env-file .env --no-open
|
|
19
19
|
```
|
|
20
20
|
|
|
21
21
|
| Flag | Meaning |
|
|
@@ -86,13 +86,13 @@ npx cuttlely check --flow <flow id> --gate
|
|
|
86
86
|
| `--threshold` | Pass mark from 0 to 100. Omit it and the server uses 100. |
|
|
87
87
|
| `--gate` | Dataset, evaluators, and threshold saved on the flow. Do not combine it with `--dataset`. |
|
|
88
88
|
|
|
89
|
-
Do not combine those flags with `--fail-on-changed`.
|
|
89
|
+
Do not combine those flags with `--fail-on-changed`. The gate covers Publish of the current draft only. Roll back, Promote, and `PUT /api/v1/chatflows/:id` can still change Live. A harness cannot store a passing test set. `--dataset` can score a harness for that run only. `--gate` on a harness exits 2 before any check.
|
|
90
90
|
|
|
91
|
-
| Code | When
|
|
92
|
-
| ---- |
|
|
93
|
-
| 0 | The server verdict is passed. With `--gate`, a second read of the flow still
|
|
94
|
-
| 1 | The server verdict is failed.
|
|
95
|
-
| 2 | The same failures as the question check, a missing verdict, a usage error, a withheld dataset or evaluator id, or a `--gate`
|
|
91
|
+
| Code | When |
|
|
92
|
+
| ---- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
93
|
+
| 0 | The server verdict is passed. With `--gate`, a second read of the flow still stores this check as passed with an empty `block` (`publishGate.enabled` is true, `publishGate.result.checkId` is this check, and `publishGate.result.verdict` is `passed`). |
|
|
94
|
+
| 1 | The server verdict is failed. With `--gate`, the command does not read the flow again. |
|
|
95
|
+
| 2 | The same failures as the question check, a missing verdict, a usage error, or a withheld dataset or evaluator id. `--gate` also exits 2 when the confirmation read fails: a non-empty block, a different checkId, or a missing passed. Turning the passing test set off during a `--gate` check, after the server verdict is `passed`, exits 2. The message is `The saved test set changed during the check. Run it again.` when the server sends no `block` text. Changing the dataset, the evaluators, or the pass mark during that check exits 2 the same way, and a `block` from the server is printed instead. |
|
|
96
96
|
|
|
97
97
|
A 403 names the flow view and update permissions, plus `datasets:view` and `evaluators:view`. The summary prints the pass rate, the threshold, and the cases whose draft scores are not Pass. `--json` writes the full check, including each score.
|
|
98
98
|
|