primethink-cli 1.5.0__tar.gz → 1.7.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. {primethink_cli-1.5.0/primethink_cli.egg-info → primethink_cli-1.7.0}/PKG-INFO +7 -2
  2. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/README.md +5 -1
  3. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/SPECS.md +12 -7
  4. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/USER_GUIDE.md +100 -10
  5. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/docs/agent-tools.md +1 -1
  6. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/docs/cli-reference.md +219 -10
  7. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/primethink.py +2292 -17
  8. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/primethink_agent_tools/tools.py +24 -5
  9. {primethink_cli-1.5.0 → primethink_cli-1.7.0/primethink_cli.egg-info}/PKG-INFO +7 -2
  10. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/primethink_cli.egg-info/SOURCES.txt +1 -0
  11. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/primethink_cli.egg-info/requires.txt +1 -0
  12. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/primethink_mcp.py +68 -16
  13. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/pyproject.toml +4 -1
  14. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/skills/primethink-cli/SKILL.md +47 -3
  15. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_agent_tools.py +18 -0
  16. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_cli.py +141 -3
  17. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_install_developer_skill.py +28 -0
  18. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_live_app.py +188 -0
  19. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_mcp_server.py +54 -0
  20. primethink_cli-1.7.0/tests/test_suite.py +735 -0
  21. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/DEVELOPER.md +0 -0
  22. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/LICENSE +0 -0
  23. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/MANIFEST.in +0 -0
  24. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/docs/install.md +0 -0
  25. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/docs/primethink_help_llms.txt +0 -0
  26. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/install/homebrew/primethink-cli.rb +0 -0
  27. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/install/install.cmd +0 -0
  28. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/install/install.ps1 +0 -0
  29. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/install/install.sh +0 -0
  30. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/primethink_agent_tools/__init__.py +0 -0
  31. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/primethink_agent_tools/client.py +0 -0
  32. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/primethink_agent_tools/settings.py +0 -0
  33. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/primethink_cli.egg-info/dependency_links.txt +0 -0
  34. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/primethink_cli.egg-info/entry_points.txt +0 -0
  35. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/primethink_cli.egg-info/top_level.txt +0 -0
  36. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/pytest.ini +0 -0
  37. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/scripts/build.sh +0 -0
  38. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/scripts/deploy.sh +0 -0
  39. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/scripts/publish.sh +0 -0
  40. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/setup.cfg +0 -0
  41. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/setup.py +0 -0
  42. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/skills/__init__.py +0 -0
  43. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/__init__.py +0 -0
  44. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/conftest.py +0 -0
  45. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_agent_commands.py +0 -0
  46. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_env_config.py +0 -0
  47. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_install_skill.py +0 -0
  48. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_packaging.py +0 -0
  49. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_project_workflows.py +0 -0
  50. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_search_commands.py +0 -0
  51. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_task_commands.py +0 -0
  52. {primethink_cli-1.5.0 → primethink_cli-1.7.0}/tests/test_whoami.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: primethink-cli
3
- Version: 1.5.0
3
+ Version: 1.7.0
4
4
  Summary: PrimeThink CLI - A powerful tool for interacting with PrimeThink AI API
5
5
  Author-email: PrimeThink <support@primethink.ai>
6
6
  License-Expression: MIT
@@ -26,6 +26,7 @@ Requires-Dist: click>=8.0.0
26
26
  Requires-Dist: requests>=2.25.0
27
27
  Provides-Extra: mcp
28
28
  Requires-Dist: mcp<2,>=1.2.0; extra == "mcp"
29
+ Requires-Dist: pydantic>=2.0; extra == "mcp"
29
30
  Provides-Extra: agent-tools
30
31
  Requires-Dist: langchain-core>=0.3.0; extra == "agent-tools"
31
32
  Requires-Dist: pydantic>=2.0; extra == "agent-tools"
@@ -104,7 +105,7 @@ pt chat send --agent AGENT_ID --message "your message"
104
105
  | `pt chat sync-from` | Sync a chat's files to a local directory |
105
106
  | `pt chat sync` | Two-way sync between a chat folder and a local directory |
106
107
  | `pt chatdb init` | Initialize a chat's ChatDB (Live App data) store |
107
- | `pt chatdb list` | List entities in a chat's ChatDB store |
108
+ | `pt chatdb list` | List entities in a chat's ChatDB store (or an attached DB Collection via `--collection`/`--collection-id`) |
108
109
  | `pt chatdb get` | Get one ChatDB entity by ID |
109
110
  | `pt chatdb add` | Add one or more rows to a ChatDB entity |
110
111
  | `pt chatdb update` | Update ChatDB entities (single or bulk) |
@@ -146,6 +147,10 @@ pt chat send --agent AGENT_ID --message "your message"
146
147
  | `pt workspace list` / `create` / `rename` / `set-goal` / `delete` | Manage chat workspaces |
147
148
  | `pt workspace archive` / `pin` / `add-chat` / `remove-chat` | Organize chats within workspaces |
148
149
  | `pt tag list` / `create` / `assign` | List, create, and assign tags (task/agent/capability/collection) |
150
+ | `pt suite build` / `validate` / `inspect` | Build, check and describe a `.ptsuite` package (apps, tasks, agents, collections, tags, workspace) |
151
+ | `pt suite install` / `uninstall` | Plan (`--dry-run`) and install/upgrade a package idempotently from a file, folder, GitHub or the catalog; remove what it created |
152
+ | `pt suite export` | Turn live tasks and Live Apps (with their agents and collections) into a package folder |
153
+ | `pt suite catalog build` / `list` / `search` | Index a folder/GitHub repo of packages and browse it |
149
154
  | `pt group list` / `get` / `create` / `update` / `delete` | Manage groups (organizations) |
150
155
  | `pt group members` / `remove-member` / `invite` / `add-agent` | Manage group members, invites, and agents |
151
156
  | `pt settings list` / `get` / `set` / `delete` | Manage group/user settings and provider API keys (secrets never shown) |
@@ -65,7 +65,7 @@ pt chat send --agent AGENT_ID --message "your message"
65
65
  | `pt chat sync-from` | Sync a chat's files to a local directory |
66
66
  | `pt chat sync` | Two-way sync between a chat folder and a local directory |
67
67
  | `pt chatdb init` | Initialize a chat's ChatDB (Live App data) store |
68
- | `pt chatdb list` | List entities in a chat's ChatDB store |
68
+ | `pt chatdb list` | List entities in a chat's ChatDB store (or an attached DB Collection via `--collection`/`--collection-id`) |
69
69
  | `pt chatdb get` | Get one ChatDB entity by ID |
70
70
  | `pt chatdb add` | Add one or more rows to a ChatDB entity |
71
71
  | `pt chatdb update` | Update ChatDB entities (single or bulk) |
@@ -107,6 +107,10 @@ pt chat send --agent AGENT_ID --message "your message"
107
107
  | `pt workspace list` / `create` / `rename` / `set-goal` / `delete` | Manage chat workspaces |
108
108
  | `pt workspace archive` / `pin` / `add-chat` / `remove-chat` | Organize chats within workspaces |
109
109
  | `pt tag list` / `create` / `assign` | List, create, and assign tags (task/agent/capability/collection) |
110
+ | `pt suite build` / `validate` / `inspect` | Build, check and describe a `.ptsuite` package (apps, tasks, agents, collections, tags, workspace) |
111
+ | `pt suite install` / `uninstall` | Plan (`--dry-run`) and install/upgrade a package idempotently from a file, folder, GitHub or the catalog; remove what it created |
112
+ | `pt suite export` | Turn live tasks and Live Apps (with their agents and collections) into a package folder |
113
+ | `pt suite catalog build` / `list` / `search` | Index a folder/GitHub repo of packages and browse it |
110
114
  | `pt group list` / `get` / `create` / `update` / `delete` | Manage groups (organizations) |
111
115
  | `pt group members` / `remove-member` / `invite` / `add-agent` | Manage group members, invites, and agents |
112
116
  | `pt settings list` / `get` / `set` / `delete` | Manage group/user settings and provider API keys (secrets never shown) |
@@ -66,13 +66,13 @@ To obtain an API key, go to `Settings > API Keys` in the PrimeThink app and gene
66
66
  | `pt scheduled-job delete` | `DELETE /api/v1/scheduled_jobs/scheduled_job_in_chat/{job_id}` |
67
67
  | document download (used by `download-file` and `sync-from`) | `GET /api/v1/documents/{document_id}/download` |
68
68
  | `pt chatdb init` | `POST /api/v1/chats/{chat_id}/chatdb/init` |
69
- | `pt chatdb list` | `POST /api/v1/chats/{chat_id}/chatdb/list` |
70
- | `pt chatdb get` | `GET /api/v1/chats/{chat_id}/chatdb/entities/{entity_id}` |
71
- | `pt chatdb add` | `POST /api/v1/chats/{chat_id}/chatdb/entities` |
72
- | `pt chatdb update` | `PATCH /api/v1/chats/{chat_id}/chatdb/entities` |
73
- | `pt chatdb delete` | `DELETE /api/v1/chats/{chat_id}/chatdb/entities` |
69
+ | `pt chatdb list` | `POST /api/v1/chats/{chat_id}/chatdb/list` (body may add `collection_name`/`collection_id`) |
70
+ | `pt chatdb get` | `GET /api/v1/chats/{chat_id}/chatdb/entities/{entity_id}` (query: `collection_name`, `collection_id`) |
71
+ | `pt chatdb add` | `POST /api/v1/chats/{chat_id}/chatdb/entities` (body may add `collection_name`/`collection_id`) |
72
+ | `pt chatdb update` | `PATCH /api/v1/chats/{chat_id}/chatdb/entities` (body may add `collection_name`/`collection_id`) |
73
+ | `pt chatdb delete` | `DELETE /api/v1/chats/{chat_id}/chatdb/entities` (body may add `collection_name`/`collection_id`) |
74
74
  | `pt collection list` | `GET /api/v1/collections` |
75
- | `pt collection create` | `POST /api/v1/collections` (params: `name`, `description`, `type`, `public`) |
75
+ | `pt collection create` | `POST /api/v1/collections` (params: `name`, `description`, `type` = `collection`/`skill`/`external_source`/`db`, `public`) |
76
76
  | `pt collection get` | `GET /api/v1/collections/{collection_id}` |
77
77
  | `pt collection update` | `PATCH /api/v1/collections/{collection_id}` |
78
78
  | `pt collection reindex` | `POST /api/v1/collections/{collection_id}/reindex` |
@@ -101,7 +101,6 @@ To obtain an API key, go to `Settings > API Keys` in the PrimeThink app and gene
101
101
  | `pt capability delete` | `DELETE /api/v1/virtual-assistants/capabilities/{capability_id}` |
102
102
  | `pt capability archive` / `unarchive` | `PUT /api/v1/virtual-assistants/capabilities/{capability_id}/archived?archived=true\|false` |
103
103
  | `pt capability duplicate` | `POST /api/v1/virtual-assistants/capabilities/{capability_id}/duplicate` |
104
- | `pt capability resolve` | resolves capability codes/ids to ids via `GET /api/v1/virtual-assistants/capabilities` |
105
104
  | `pt models list` | `GET /api/v1/catalog/llm/models` |
106
105
  | `pt models embeddings` | `GET /api/v1/catalog/embeddings/models` |
107
106
  | `pt task list` | `GET /api/v1/tasks/` |
@@ -147,6 +146,12 @@ To obtain an API key, go to `Settings > API Keys` in the PrimeThink app and gene
147
146
  | `pt tag list` | `GET /api/v1/tags?model=` |
148
147
  | `pt tag create` | `POST /api/v1/tags` (`{model, name, tag_category}`) |
149
148
  | `pt tag assign` | `PUT /api/v1/tags/assignments` (`{model, owner_id, tag_ids}`) |
149
+ | `pt capability resolve` / `pt agent create --capability` | `GET /api/v1/virtual-assistants/capabilities?page=N&page_size=100` — every page (the endpoint returns only 10 without page parameters) |
150
+ | `pt suite build` / `validate` / `inspect` / `catalog build` | local only (no API) |
151
+ | `pt suite install SOURCE` | reads: `GET /api/v1/virtual-assistants/capabilities` (all pages), `…/virtual-assistants/types`, `/groups/current/settings`, `/users/me/settings`, `/tags?model=`, `/virtual-assistants?search=`, `/collections?search=`, `/tasks/?search=&status=all`, `/chat-workspaces`, `/memories?workspace_id=`, `/chats?search=`, `/chats/{id}/collections`, `/chats/{id}/users`, `/users/me/visible-users`, `/scheduled_jobs/scheduled_jobs_in_chat/{chat}`, `POST /chats/{id}/chatdb/list`; writes (not in `--dry-run`): `POST /tags`, `PUT /tags/assignments`, `POST/PATCH /virtual-assistants/capabilities[/{id}]`, `POST /virtual-assistants`, `PATCH /virtual-assistants/{id}`, `POST /virtual-assistants/{id}/collections/attach`, `POST /collections?name=&type=`, `POST /collections/{id}/documents`, `POST /tasks/`, `PATCH /tasks/{id}`, `POST /tasks/{id}/image`, the `@app` sync of `pt live-app publish`, `POST /tasks/{id}/versions`, `POST /tasks/{id}/collections-files`, `POST /chat-workspaces`, `PUT /chat-workspaces/{id}/goal`, `POST /memories?workspace_id=`, `POST /chats?copy_from_task_id=`, `POST /chats/{id}/collections/{cid}`, `DELETE /chats/{id}/collections`, `POST /chats/{id}/members`, `POST /chats/{id}/chatdb/entities`, `POST/PUT /scheduled_jobs/scheduled_job_in_chat[/{id}]` |
152
+ | `pt suite uninstall ID` | `DELETE /scheduled_jobs/scheduled_job_in_chat/{id}`, `DELETE /tasks/{id}`, `DELETE /virtual-assistants/{id}`, `DELETE /virtual-assistants/capabilities/{id}`; with flags `DELETE /collections/{id}`, `DELETE /chats/{id}` |
153
+ | `pt suite export` | `GET /tasks/{id}`, `GET /virtual-assistants/{id}`, `GET /virtual-assistants/types`, `GET /collections/{id}`, `GET /tasks/{id}/directories?path=/app`, `GET /documents/{id}/download` |
154
+ | `pt suite … github:owner/repo[@ref]` | `GET https://codeload.github.com/{owner}/{repo}/zip/{ref}`, or with `GITHUB_TOKEN` `GET https://api.github.com/repos/{owner}/{repo}/zipball/{ref}` |
150
155
  | `pt group list` | `GET /api/v1/groups/` |
151
156
  | `pt group get` | `GET /api/v1/groups/{group_id}` |
152
157
  | `pt group create` | `POST /api/v1/groups/` |
@@ -22,11 +22,12 @@ For a terse, complete listing of every command and option, see the [CLI Referenc
22
22
  14. [Finding Users](#finding-users)
23
23
  15. [Notifications](#notifications)
24
24
  16. [Scaffolding Live Apps](#scaffolding-live-apps)
25
- 17. [MCP Server](#mcp-server)
26
- 18. [Common Use Cases](#common-use-cases)
27
- 19. [Tips and Tricks](#tips-and-tricks)
28
- 20. [Troubleshooting](#troubleshooting)
29
- 21. [FAQ](#faq)
25
+ 17. [Suite Packages](#suite-packages)
26
+ 18. [MCP Server](#mcp-server)
27
+ 19. [Common Use Cases](#common-use-cases)
28
+ 20. [Tips and Tricks](#tips-and-tricks)
29
+ 21. [Troubleshooting](#troubleshooting)
30
+ 22. [FAQ](#faq)
30
31
 
31
32
  ## Introduction
32
33
 
@@ -77,7 +78,7 @@ pt version
77
78
 
78
79
  You should see output like:
79
80
  ```
80
- PrimeThink CLI v1.5.0
81
+ PrimeThink CLI v1.7.0
81
82
  ```
82
83
 
83
84
  ## Getting Started
@@ -713,6 +714,19 @@ pt chatdb delete 123 --entity-id 7 # asks for confirmation; --yes to sk
713
714
 
714
715
  `--data` takes a single JSON object; `--items` takes a JSON array for bulk operations. On update, `--merge` patches the existing row while `--replace` overwrites it, and `--if-unchanged-since TIMESTAMP` guards against overwriting a row that changed underneath you.
715
716
 
717
+ ### DB Collections
718
+
719
+ A **DB Collection** is a collection of type `db` that holds the same kind of entities, but can be shared between chats. Once one is attached to a chat, `list`, `get`, `add`, `update` and `delete` can target it instead of the chat's own ChatDB, by name or by ID:
720
+
721
+ ```bash
722
+ pt collection create --name project-db --type db # note the id, e.g. 42
723
+ # attach it to the chat in the web app, then:
724
+ pt chatdb add 123 --entity tasks --data '{"title": "a"}' --collection project-db
725
+ pt chatdb list 123 --entity tasks --collection-id 42
726
+ ```
727
+
728
+ Prefer `--collection-id` when two attached DB Collections share a name, or in scripts that should survive a rename. If you pass both, they must identify the same collection. A collection attached read-only accepts `list`/`get` but rejects writes. This mirrors `pt.db('project-db')` / `pt.db(42)` in a Live App.
729
+
716
730
  > Building the Live App itself (the front-end that uses this data) is covered by the **primethink-developer** skill; `pt chatdb` is the deterministic data plane underneath it.
717
731
 
718
732
  ## Semantic Search
@@ -1144,10 +1158,10 @@ pt eval delete 99 EVAL_ID
1144
1158
  ### 2. Configure how evaluations run
1145
1159
 
1146
1160
  ```bash
1147
- pt eval settings 99 --active --run-time daily --evaluator-agent-id 7 --pass-threshold 0.8
1161
+ pt eval settings 99 --active --run-time daily --evaluator-agent-id 7 --pass-threshold 80
1148
1162
  ```
1149
1163
 
1150
- `--run-time` is `manual`, `daily`, `weekly`, or `monthly`; `--pass-threshold` is the minimum score to count as a pass; `--message-delay-ms` throttles messages during a run.
1164
+ `--run-time` is `manual`, `daily`, `weekly`, or `monthly`; `--pass-threshold` is the minimum score to count as a pass, as a whole-number percentage from 1 to 100 (`80`, not `0.8`) — anything outside that range is rejected by the CLI before a request is made; `--message-delay-ms` throttles messages during a run.
1151
1165
 
1152
1166
  ### 3. Run it, then read the results
1153
1167
 
@@ -1270,8 +1284,8 @@ Defaults are `--framework react --tailwind --flowbite`. Flowbite depends on Tail
1270
1284
  After generation:
1271
1285
 
1272
1286
  1. Read the generated `README.md`; build and deployment steps differ by template.
1273
- 2. Rename the sample Chat DB entity so it is unique to your app.
1274
- 3. Replace the sample CRUD interface while retaining the PrimeThink runtime, theme, persistence, and real-time patterns.
1287
+ 2. Name every Chat DB entity for your app. The templates ship none, and a generic name such as `item` collides with a sibling app in the same chat.
1288
+ 3. Build the interface on the blank starter, keeping the PrimeThink wiring it ships: the runtime, the host-theme bridge, and the deployment configuration.
1275
1289
  4. Build if required, then upload the files described by the template README.
1276
1290
 
1277
1291
  The default Flowbite React starter is a Vite project and requires a local `npm install` and build. The other default starters are no-build, one-file HTML applications. The CLI itself intentionally runs neither package installation nor generated code.
@@ -1325,6 +1339,82 @@ pt live-app test ./my-live-app --chat-id "$(cat ./my-live-app/.chat-id)"
1325
1339
 
1326
1340
  Automated UI testing of a Live App is not part of the CLI. It is a deterministic, plan-driven workflow provided by the `primethink-developer` skill (`pt install-developer-skill`): the skill captures the running app's accessibility snapshot, authors a reviewable `tests/test_plan.yaml`, runs it with a bundled Playwright runner without an LLM in the execution loop, reads the structured results, and heals failing selectors before re-running.
1327
1341
 
1342
+ ## Suite Packages
1343
+
1344
+ A **suite package** (`.ptsuite`) installs a complete working set in one step: Live Apps,
1345
+ tasks with their schedules, the agents they run on, DB and document collections, tags,
1346
+ and a workspace with its shared rules and the chats to open. It is a folder with a
1347
+ `manifest.json` — shipped as that folder, zipped as a `.ptsuite` file, or kept in a
1348
+ GitHub repository.
1349
+
1350
+ ### Look before you install
1351
+
1352
+ ```bash
1353
+ pt suite inspect ./primethink.gtm-1.0.0.ptsuite
1354
+ ```
1355
+
1356
+ `inspect` shows the name and description in your language, what the package needs
1357
+ (capabilities, agent types, a minimum CLI version, settings such as `SERPER_API_KEY`), its
1358
+ install parameters, and everything it contains.
1359
+
1360
+ ### Install
1361
+
1362
+ Always plan first — a dry run only reads from PrimeThink:
1363
+
1364
+ ```bash
1365
+ pt suite install ./primethink.gtm-1.0.0.ptsuite --params gtm.params.json --model openai:gpt-5.4 --dry-run
1366
+ pt suite install ./primethink.gtm-1.0.0.ptsuite --params gtm.params.json --model openai:gpt-5.4 --yes
1367
+ ```
1368
+
1369
+ The plan lists every resource as *create*, *update* or *ok*. A **BLOCKED** line (a missing
1370
+ capability or required setting, a CLI too old) means nothing will be written; the message
1371
+ says how to fix it. `--model` is the model for agents the package does not pin
1372
+ (`pt models list --only-configured`).
1373
+
1374
+ Parameters come from the package (`inspect` lists them), as a JSON file (`--params`) or one
1375
+ at a time (`--param key=value`, where the value may be JSON).
1376
+
1377
+ ### Upgrade and re-run
1378
+
1379
+ Installing again is safe and is how you upgrade: the installer finds what it created
1380
+ (even from another machine), updates it in place and adds what is new. It never deletes a
1381
+ collection, never removes a capability from an agent, and never overwrites a reference
1382
+ row you have edited (seed rows are insert-only). After a daylight-saving change, run it
1383
+ again with `--reschedule` — PrimeThink stores schedules in UTC.
1384
+
1385
+ ### Sources and the catalog
1386
+
1387
+ ```bash
1388
+ pt suite install github:primethink-ai/primethink-catalog/suites/gtm@gtm@1.0.0
1389
+ pt suite catalog list --catalog github:primethink-ai/primethink-catalog
1390
+ pt suite catalog search "sales" --kind suite
1391
+ pt suite install gtm@1.0.0 # by name, through the catalog
1392
+ ```
1393
+
1394
+ For a private repository set `GITHUB_TOKEN`; it is sent only to GitHub.
1395
+
1396
+ ### Make your own
1397
+
1398
+ ```bash
1399
+ pt suite export --task-id 81 --task-id 82 -o ./my-suite --id acme.onboarding
1400
+ # add ./my-suite/icon.png (a PNG), review the files, then:
1401
+ pt suite build ./my-suite
1402
+ ```
1403
+
1404
+ `build` hashes every file into the manifest and zips it. `validate` refuses scripts,
1405
+ secrets, environment ids, and files that changed after the build. `pt suite catalog build
1406
+ DIR` indexes a folder of packages into `catalog.json`.
1407
+
1408
+ ### Remove
1409
+
1410
+ ```bash
1411
+ pt suite uninstall primethink.gtm --dry-run
1412
+ pt suite uninstall primethink.gtm --yes
1413
+ ```
1414
+
1415
+ Only what the install created is removed. Collections (your data) and chats stay unless
1416
+ you pass `--delete-data` / `--delete-chats`.
1417
+
1328
1418
  ## MCP Server
1329
1419
 
1330
1420
  Core API management operations can also be exposed over the [Model Context
@@ -60,7 +60,7 @@ resolve on 3.8. The `dev` extra installs the plugin's test dependencies only on
60
60
 
61
61
  ```bash
62
62
  # In the API image (production): one pinned line in requirements.txt
63
- primethink-cli[agent-tools]==1.5.0
63
+ primethink-cli[agent-tools]==1.7.0
64
64
 
65
65
  # From a checkout (dev): the dev extra already includes langchain-core
66
66
  pip install -e ".[dev]"
@@ -1,6 +1,6 @@
1
1
  # PrimeThink CLI — Command Reference
2
2
 
3
- Complete reference for every command in the PrimeThink CLI (`pt`), version 1.5.0.
3
+ Complete reference for every command in the PrimeThink CLI (`pt`), version 1.7.0.
4
4
 
5
5
  Commands are organized into noun groups: `profile`, `live-app`, `chat`, `collection`, `agent`, `task`, `search`, and `image`.
6
6
 
@@ -17,6 +17,14 @@ Commands are organized into noun groups: `profile`, `live-app`, `chat`, `collect
17
17
  - [`pt live-app new`](#pt-live-app-new)
18
18
  - [`pt live-app publish`](#pt-live-app-publish)
19
19
  - [`pt live-app test`](#pt-live-app-test)
20
+ - [Suite packages: `pt suite`](#suite-packages-pt-suite)
21
+ - [`pt suite build`](#pt-suite-build)
22
+ - [`pt suite validate`](#pt-suite-validate)
23
+ - [`pt suite inspect`](#pt-suite-inspect)
24
+ - [`pt suite install`](#pt-suite-install)
25
+ - [`pt suite uninstall`](#pt-suite-uninstall)
26
+ - [`pt suite export`](#pt-suite-export)
27
+ - [`pt suite catalog`](#pt-suite-catalog)
20
28
  - [Profiles: `pt profile`](#profiles-pt-profile)
21
29
  - [`pt profile add`](#pt-profile-add)
22
30
  - [`pt profile use`](#pt-profile-use)
@@ -221,7 +229,7 @@ Display the CLI version.
221
229
 
222
230
  ```bash
223
231
  pt version
224
- # PrimeThink CLI v1.5.0
232
+ # PrimeThink CLI v1.7.0
225
233
  ```
226
234
 
227
235
  ### `pt whoami`
@@ -507,6 +515,181 @@ iterating on. The status check alone would miss a run that exits `0` without pri
507
515
 
508
516
  ---
509
517
 
518
+ ## Suite packages: `pt suite`
519
+
520
+ A **suite package** bundles Live Apps, tasks, agents, custom capabilities, collections,
521
+ tags and a workspace into one declarative package — a folder with `manifest.json` at its
522
+ root, that folder zipped as a **`.ptsuite`** file, or the same folder in a GitHub
523
+ repository. Resources reference each other by package **key**, never by id; the installer
524
+ maps keys to ids in the target environment. Nothing in a package is executed on the
525
+ installing machine. Design: `docs/superpowers/specs/2026-09-28-ptsuite-package-design.md`.
526
+
527
+ ```bash
528
+ pt suite build DIR [-o FILE] [--no-zip]
529
+ pt suite validate SOURCE [--catalog C]
530
+ pt suite inspect SOURCE [--catalog C] [--json] [--help-text]
531
+ pt suite install SOURCE [--params FILE] [--param k=v ...] [--dry-run] [--yes]
532
+ [--agent-type NAME|ID] [--model M] [--reschedule] [--catalog C]
533
+ pt suite uninstall SUITE_ID [--delete-data] [--delete-chats] [--dry-run] [--yes]
534
+ pt suite export --task-id ID [--task-id ID ...] -o DIR [--id ID] [--version V]
535
+ pt suite catalog build DIR [--refresh]
536
+ pt suite catalog list [--catalog C] [--kind suite|app|task|agent] [--tag T] [--json]
537
+ pt suite catalog search TEXT [--catalog C] [--kind K] [--json]
538
+ ```
539
+
540
+ **`SOURCE`** is any of: a package folder; a `.ptsuite` file;
541
+ `github:owner/repo[/path][@ref]` (the ref may itself contain `@`, e.g.
542
+ `github:primethink-ai/primethink-catalog/suites/gtm@gtm@1.0.0`); or a catalog name
543
+ `name[@version]`, resolved through `--catalog` / `$PT_CATALOG` / the default
544
+ `github:primethink-ai/primethink-catalog`. Public repositories download from
545
+ `codeload.github.com`. With `GITHUB_TOKEN` set, the download goes to `api.github.com`
546
+ with the token, and GitHub's redirect to codeload drops it — the token is never sent
547
+ anywhere else.
548
+
549
+ ### Package layout
550
+
551
+ ```
552
+ manifest.json format, kind (suite|app|task|agent), id, version (semver), name, description
553
+ (a string or a language map with "en"), icon (PNG), help, help_url,
554
+ languages, tags, requires {cli, capabilities, agent_types, models},
555
+ prerequisites {settings[{name, scope, required, description}], permissions,
556
+ notes}, params, contents[{path, sha256, bytes}]
557
+ params.schema.json install parameters (JSON Schema subset: type, properties, required, items,
558
+ enum, pattern, minItems, default)
559
+ tags.json [{key, name, model: task|agent|capability|collection}]
560
+ capabilities/<key>.json custom api/mcp capabilities {name, code, type, access_type, options, …}
561
+ agents/<key>.json {name, public_description, description|description_file, type, model,
562
+ access_type, capabilities: [code | "@capabilities/<key>"], collections, tags}
563
+ collections/<key>.json {name, type: db|collection|skill|external_source, description, schema,
564
+ seed {entity: file.jsonl}, files, for_each, attach_to, tags}
565
+ apps/<key>/app.json task fields + {agent, extra_agents, evaluator_agent, collections, tags,
566
+ goal_file, initial_prompt_file, image, dist, schedule}
567
+ apps/<key>/dist/… the flat build uploaded to @app
568
+ tasks/<key>/task.json as app.json, without dist; schedule {nl, prompt_file|prompt, timezone}
569
+ workspace.json {name, goal|goal_file, memories|memories_file, chats[{key, app|task, name,
570
+ for_each, in_workspace, collections, only_collections, invite}]}
571
+ ```
572
+
573
+ - **Templates** — a collection or chat with `for_each: "<param>"` is expanded once per
574
+ item of that array parameter; `{field}` placeholders in `name`/`invite` come from the
575
+ item. `attach_to: ["apps/<key>", …]` attaches every instance to those apps/tasks.
576
+ - **`invite`** — emails, `{field}` from the item, or `@params.<name>` (a list parameter).
577
+ - **`only_collections: true`** on a chat detaches any collection not listed for it (an
578
+ introducer's Portal chat carries only its own collection).
579
+ - `requires.models` is informational (not checked).
580
+
581
+ ### `pt suite build`
582
+
583
+ Recomputes `manifest.contents` (SHA-256 and size of every file), validates, and writes a
584
+ deterministic zip (sorted entries, fixed timestamps: the same folder always produces the
585
+ same bytes). Final line: `Built FILE (N files, B bytes, sha256 …)`. `--no-zip` only
586
+ refreshes contents and validates (`Valid: id version`).
587
+
588
+ ### `pt suite validate`
589
+
590
+ Checks the manifest (format, kind, id, semver, name/description, the icon is a real PNG),
591
+ every file against `contents` (no unlisted, missing or modified file), every reference
592
+ (agents, collections, capabilities, tags, templates, chats), the apps' `dist` is flat with
593
+ `index.html`, images match their extension, seed rows carry a unique `seed_key`, no
594
+ environment ids (`virtual_assistant_id` …) appear, **no script or executable files**, and
595
+ **no secret-shaped strings** (API keys, tokens, private keys). A prerequisite setting that
596
+ carries a value is an error. Exit 1 on any error.
597
+
598
+ ### `pt suite inspect`
599
+
600
+ Name, description, requirements, prerequisites, parameters and contents, in the user's
601
+ language (`PT_LANG`, else `LANG`). `--json` prints the manifest (without `contents`), the
602
+ resource list, the parameter schema, errors and warnings. `--help-text` prints the
603
+ package's `HELP.md`.
604
+
605
+ ### `pt suite install`
606
+
607
+ Plans first, then applies (after a confirmation, or with `--yes`). `--dry-run` prints the
608
+ plan and sends **only reads** (GETs, plus `chatdb/list` to compare seed rows).
609
+
610
+ 1. **Prerequisites** — `requires.cli` (`>=X.Y.Z`), capability codes, agent types, and
611
+ required settings (checked by name at group/user scope; a missing one prints the exact
612
+ `pt settings set NAME <value> --scope group`). Any blocker stops the install before a write.
613
+ 2. **Tags** — created if missing, plus one `suite:<id>` tag per model used.
614
+ 3. **Capabilities** (custom) — created, or updated by code.
615
+ 4. **Agents** — found by provenance / name, else created with the type (`type`, else
616
+ `--agent-type`) and model (`model`, else `--model`). Capabilities are the **union** of
617
+ what the agent has and what the package wants: `PATCH` replaces the set, so the
618
+ installer never drops one.
619
+ 5. **Collections** — found by exact name (a different type is a blocker), else created;
620
+ `files` uploaded if missing.
621
+ 6. **Apps and tasks** — created or PATCHed with their fields; apps' `dist` synced into
622
+ `@app` (identical files are left unchanged) and a Production version created; the image
623
+ is uploaded on create.
624
+ 7. **Attachments** — collections on tasks (`POST /tasks/{id}/collections-files`) and RAG
625
+ collections on agents.
626
+ 8. **Workspace** — created or its goal updated; missing `workspace_constitution` memories added.
627
+ 9. **Chats** — each launched from its task (`POST /chats?copy_from_task_id=…`) into the
628
+ workspace unless `in_workspace: false`; collections attached; members invited (users
629
+ already in the chat are skipped).
630
+ 10. **Seed** — through the first chat that carries the collection. **Insert-only**: a row
631
+ whose `seed_key` exists is never updated or deleted.
632
+ 11. **Schedules** — a scheduled job in each chat of a task with a `schedule` (created, or
633
+ updated when its `nl` changed or with `--reschedule`, e.g. after a clock change).
634
+
635
+ Every created task and agent records `extra.ptsuite = {id, version, key}`, and every
636
+ resource gets the `suite:<id>` tag, so a re-install finds its resources **without** the
637
+ local lock file (a Deep1 sandbox loses its files). The lock file
638
+ `.ptsuite/<api host>/<id>.lock.json` (key → id, what this install created) is a cache and
639
+ the input to `uninstall`. Final line:
640
+ `Installed ID VERSION: N apps, N tasks, N agents, N collections, N chats`.
641
+
642
+ ### `pt suite uninstall`
643
+
644
+ Removes what the install **created** (from its lock file): scheduled jobs, apps, tasks,
645
+ agents and custom capabilities. Resources it found and updated are left alone.
646
+ Collections stay unless `--delete-data` (their rows remain server-side), chats unless
647
+ `--delete-chats`.
648
+
649
+ ### `pt suite export`
650
+
651
+ Turns live tasks/Live Apps into a package folder: each task becomes `apps/<key>` (page type
652
+ html; its `@app` files are downloaded into `dist/`) or `tasks/<key>`, its agents become
653
+ `agents/<key>.json` (+ description), custom api/mcp capabilities become
654
+ `capabilities/<key>.json` (review them for secrets), attached collections become
655
+ descriptors (no rows or files), tags become `tags.json`. Ids become keys. Add `icon.png`,
656
+ review, then `pt suite build`.
657
+
658
+ ### `pt suite catalog`
659
+
660
+ A catalog is a folder or GitHub repo of packages with a generated `catalog.json`.
661
+ `catalog build DIR` validates every package under DIR and writes the index (id, kind,
662
+ version, path, name, description, icon, tags, languages, requires, counts, and the
663
+ SHA-256 of each package's canonical zip — `install NAME` refuses a package that no longer
664
+ matches it). `--refresh` recomputes each package's contents first. `list` and `search`
665
+ read a catalog (every search word must match id, name, description or tags).
666
+ Versions are git tags `<slug>@<version>`: `pt suite install gtm@1.2.0` fetches
667
+ `github:<catalog>/<path>@gtm@1.2.0`.
668
+
669
+ | Step | Endpoint |
670
+ |---|---|
671
+ | capabilities | `GET /api/v1/virtual-assistants/capabilities?page=&page_size=` (all pages), `POST`/`PATCH …/capabilities/{id}` |
672
+ | agent types | `GET /api/v1/virtual-assistants/types` |
673
+ | settings | `GET /api/v1/groups/current/settings`, `GET /api/v1/users/me/settings` |
674
+ | tags | `GET /api/v1/tags?model=`, `POST /api/v1/tags`, `PUT /api/v1/tags/assignments` |
675
+ | agents | `GET /api/v1/virtual-assistants?search=`, `GET/PATCH /api/v1/virtual-assistants/{id}`, `POST /api/v1/virtual-assistants`, `POST …/{id}/collections/attach` |
676
+ | collections | `GET /api/v1/collections?search=`, `POST /api/v1/collections`, `GET …/{id}/directories`, `POST …/{id}/documents` |
677
+ | apps/tasks | `GET /api/v1/tasks/?search=&status=all`, `POST /api/v1/tasks/`, `GET/PATCH /api/v1/tasks/{id}`, `POST …/image`, `@app` sync (as `pt live-app publish`), `POST …/versions`, `POST …/collections-files` |
678
+ | workspace | `GET/POST /api/v1/chat-workspaces`, `PUT …/{id}/goal`, `GET/POST /api/v1/memories?workspace_id=` |
679
+ | chats | `POST /api/v1/chats?copy_from_task_id=`, `GET /api/v1/chats?search=`, `GET/POST …/{id}/collections[/{cid}]`, `DELETE …/{id}/collections`, `GET …/{id}/users`, `POST …/{id}/members`, `GET /api/v1/users/me/visible-users` |
680
+ | seed | `POST /api/v1/chats/{id}/chatdb/list`, `POST /api/v1/chats/{id}/chatdb/entities` (with `collection_id`) |
681
+ | schedules | `GET /api/v1/scheduled_jobs/scheduled_jobs_in_chat/{chat}`, `POST/PUT /api/v1/scheduled_jobs/scheduled_job_in_chat[/{id}]` |
682
+
683
+ ```bash
684
+ pt suite build ./gtm-suite/package/gtm -o primethink.gtm-1.0.0.ptsuite
685
+ pt suite inspect primethink.gtm-1.0.0.ptsuite
686
+ pt suite install primethink.gtm-1.0.0.ptsuite --params gtm.params.json --model openai:gpt-5.4 --dry-run
687
+ pt suite install primethink.gtm-1.0.0.ptsuite --params gtm.params.json --model openai:gpt-5.4 --yes
688
+ pt suite catalog list --catalog github:primethink-ai/primethink-catalog --kind app
689
+ ```
690
+
691
+ ---
692
+
510
693
  ## Profiles: `pt profile`
511
694
 
512
695
  Profiles let you store multiple API tokens (e.g. for different accounts or environments) and switch between them.
@@ -1021,6 +1204,13 @@ Entities are grouped by name (think tables); each row has an `id`. Initialize th
1021
1204
 
1022
1205
  > Unlike `pt chat`/`pt collection`, this group *does* use `-p` for `--profile` (there is no `--path` here).
1023
1206
 
1207
+ **Targeting a DB Collection.** By default every command below except `init` reads and writes the chat's own ChatDB. Pass `--collection NAME` and/or `--collection-id N` to target a **DB Collection** (a collection of type `db`, see `pt collection create --type db`) attached to the chat instead. The ID is unambiguous when several attached DB Collections share a name, and it survives a rename; when you pass both, they must identify the same collection or the API answers 404. Writes (`add`, `update`, `delete`) to a collection attached read-only are rejected with 403. The target is sent as `collection_name`/`collection_id` in the request body, or as query parameters for `get`. There is no `pt` command to attach a collection to a chat yet — attach it in the web app.
1208
+
1209
+ ```bash
1210
+ pt chatdb list 123 --entity tasks --collection project-db
1211
+ pt chatdb add 123 --entity tasks --data '{"title": "a"}' --collection-id 42
1212
+ ```
1213
+
1024
1214
  ### `pt chatdb init`
1025
1215
 
1026
1216
  Initialize the ChatDB store for a chat.
@@ -1047,6 +1237,8 @@ pt chatdb list CHAT_ID [OPTIONS]
1047
1237
  | `--offset` | Row offset |
1048
1238
  | `--page` | Page number |
1049
1239
  | `--page-size` | Results per page |
1240
+ | `--collection` | Target an attached DB Collection by name instead of the chat's own ChatDB |
1241
+ | `--collection-id` | Target an attached DB Collection by ID (≥ 1; unambiguous when names repeat) |
1050
1242
 
1051
1243
  Calls `POST /api/v1/chats/{chat_id}/chatdb/list`.
1052
1244
 
@@ -1060,9 +1252,14 @@ pt chatdb list 123 --entity todos --filters '{"done": false}' --limit 20
1060
1252
  Get a single ChatDB entity by its ID.
1061
1253
 
1062
1254
  ```bash
1063
- pt chatdb get CHAT_ID ENTITY_ID [--profile NAME] [--api-url URL]
1255
+ pt chatdb get CHAT_ID ENTITY_ID [--collection NAME] [--collection-id N] [--profile NAME] [--api-url URL]
1064
1256
  ```
1065
1257
 
1258
+ | Option | Description |
1259
+ |---|---|
1260
+ | `--collection` | Target an attached DB Collection by name instead of the chat's own ChatDB |
1261
+ | `--collection-id` | Target an attached DB Collection by ID (≥ 1; unambiguous when names repeat) |
1262
+
1066
1263
  Calls `GET /api/v1/chats/{chat_id}/chatdb/entities/{entity_id}`.
1067
1264
 
1068
1265
  ### `pt chatdb add`
@@ -1078,6 +1275,8 @@ pt chatdb add CHAT_ID --entity NAME [--data JSON] [--items JSON]
1078
1275
  | `--entity` | Entity name to add to (required) |
1079
1276
  | `--data` | A single row as a JSON object |
1080
1277
  | `--items` | Multiple rows as a JSON array (bulk insert) |
1278
+ | `--collection` | Target an attached DB Collection by name instead of the chat's own ChatDB |
1279
+ | `--collection-id` | Target an attached DB Collection by ID (≥ 1; unambiguous when names repeat) |
1081
1280
 
1082
1281
  Calls `POST /api/v1/chats/{chat_id}/chatdb/entities`.
1083
1282
 
@@ -1101,8 +1300,10 @@ pt chatdb update CHAT_ID [OPTIONS]
1101
1300
  | `--merge` / `--replace` | Merge `--data` into the existing row or replace it (server default applies if unset) |
1102
1301
  | `--items` | Bulk updates as a JSON array |
1103
1302
  | `--if-unchanged-since` | Only update if the row is unchanged since this timestamp (optimistic concurrency) |
1303
+ | `--collection` | Target an attached DB Collection by name instead of the chat's own ChatDB |
1304
+ | `--collection-id` | Target an attached DB Collection by ID (≥ 1; unambiguous when names repeat) |
1104
1305
 
1105
- Requires at least one of `--entity-id`/`--data` or `--items`. Calls `PATCH /api/v1/chats/{chat_id}/chatdb/entities`.
1306
+ Requires at least one of `--entity-id`/`--data` or `--items`; a collection target on its own does not count. Calls `PATCH /api/v1/chats/{chat_id}/chatdb/entities`.
1106
1307
 
1107
1308
  ```bash
1108
1309
  pt chatdb update 123 --entity-id 7 --data '{"done": true}' --merge
@@ -1113,9 +1314,17 @@ pt chatdb update 123 --entity-id 7 --data '{"done": true}' --merge
1113
1314
  Delete a single row (`--entity-id`) or several (`--ids`). **Destructive and irreversible** — prompts for confirmation unless `--yes` is passed.
1114
1315
 
1115
1316
  ```bash
1116
- pt chatdb delete CHAT_ID [--entity-id N | --ids 1,2,3] [--yes]
1317
+ pt chatdb delete CHAT_ID [--entity-id N | --ids 1,2,3] [--collection NAME] [--collection-id N] [--yes]
1117
1318
  ```
1118
1319
 
1320
+ | Option | Description |
1321
+ |---|---|
1322
+ | `--entity-id` | ID of a single row to delete |
1323
+ | `--ids` | Comma-separated row IDs to delete |
1324
+ | `--collection` | Target an attached DB Collection by name instead of the chat's own ChatDB |
1325
+ | `--collection-id` | Target an attached DB Collection by ID (≥ 1; unambiguous when names repeat) |
1326
+ | `--yes` | Skip the confirmation prompt |
1327
+
1119
1328
  Requires `--entity-id` or `--ids`. Calls `DELETE /api/v1/chats/{chat_id}/chatdb/entities`.
1120
1329
 
1121
1330
  ```bash
@@ -1162,7 +1371,7 @@ pt collection create --name NAME [OPTIONS]
1162
1371
  |---|---|
1163
1372
  | `--name` | Collection name (required) |
1164
1373
  | `--description` | Collection description |
1165
- | `--type` | Collection type: `collection`, `skill`, or `external_source` (server default: `collection`) |
1374
+ | `--type` | Collection type: `collection`, `skill`, `external_source`, or `db` (a DB Collection: structured entities that `pt chatdb --collection` and Live Apps' `pt.db()` read and write) (server default: `collection`) |
1166
1375
  | `--public` / `--private` | Make the collection public or private (server default: private) |
1167
1376
 
1168
1377
  Calls `POST /api/v1/collections` and prints the created collection as JSON.
@@ -1198,7 +1407,7 @@ pt collection update COLLECTION_ID [OPTIONS]
1198
1407
  |---|---|
1199
1408
  | `--name` | Collection name |
1200
1409
  | `--description` | Collection description |
1201
- | `--type` | Collection type: `collection`, `skill`, or `external_source` |
1410
+ | `--type` | Collection type: `collection`, `skill`, `external_source`, or `db` |
1202
1411
  | `--public` / `--private` | Make the collection public or private |
1203
1412
  | `--indexed` / `--not-indexed` | Enable or disable indexing |
1204
1413
  | `--ocr-instructions` | OCR instructions for document processing |
@@ -2472,7 +2681,7 @@ pt eval update TASK_ID EVAL_ID [--user-query …] [--ideal-response …] [--type
2472
2681
  pt eval delete TASK_ID EVAL_ID [--yes] # DESTRUCTIVE
2473
2682
 
2474
2683
  # Configure how evaluations run
2475
- pt eval settings TASK_ID [--active | --inactive] [--run-time manual|daily|weekly|monthly] [--evaluator-agent-id N] [--pass-threshold F] [--message-delay-ms N]
2684
+ pt eval settings TASK_ID [--active | --inactive] [--run-time manual|daily|weekly|monthly] [--evaluator-agent-id N] [--pass-threshold 1-100] [--message-delay-ms N]
2476
2685
 
2477
2686
  # Run an evaluation and inspect its runs/results
2478
2687
  pt eval run TASK_ID [--version N] [--model-override M]
@@ -2503,13 +2712,13 @@ pt eval delete-simulation TASK_ID SIM_ID [--yes] # DESTRUCTIVE
2503
2712
  | `simulations` | `GET /api/v1/tasks/{id}/simulations` | `list_task_simulations` |
2504
2713
  | `delete-simulation` | `DELETE /api/v1/tasks/{id}/simulations/{sim_id}` | `delete_task_simulation` |
2505
2714
 
2506
- `--type` for a test case is `exact` (exact-match), `similar` (fuzzy/semantic match), or `agent` (an evaluator agent judges the response). `--examples` takes a JSON object of good/bad example fields. `--chat-group` (default `1`) groups related test cases.
2715
+ `--pass-threshold` is the minimum score that counts as a pass, as a whole-number percentage from 1 to 100 (`80`, not `0.8`); the CLI rejects anything outside that range with a usage error before any request is made. `--type` for a test case is `exact` (exact-match), `similar` (fuzzy/semantic match), or `agent` (an evaluator agent judges the response). `--examples` takes a JSON object of good/bad example fields. `--chat-group` (default `1`) groups related test cases.
2507
2716
 
2508
2717
  ```bash
2509
2718
  # Build a two-case plan, make it evaluator-judged and daily, then run it
2510
2719
  pt eval add 99 --user-query "What's your return window?" --ideal-response "30 days" --type similar
2511
2720
  pt eval add 99 --user-query "Refund a gift?" --ideal-response "Yes, store credit" --type agent --evaluator-agent-id 7
2512
- pt eval settings 99 --active --run-time daily --evaluator-agent-id 7 --pass-threshold 0.8
2721
+ pt eval settings 99 --active --run-time daily --evaluator-agent-id 7 --pass-threshold 80
2513
2722
  pt eval run 99
2514
2723
  pt eval results 99 | jq '.[] | {case, passed, score}'
2515
2724
  ```