@zibby/cli 0.2.1 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/README.md +117 -16
  2. package/dist/bin/zibby.js +3 -3
  3. package/dist/commands/chat-agents.js +1 -1
  4. package/dist/commands/chat.js +62 -62
  5. package/dist/commands/credentials-api.js +1 -1
  6. package/dist/commands/credentials-file.js +1 -1
  7. package/dist/commands/creds.js +1 -1
  8. package/dist/commands/init.js +180 -82
  9. package/dist/commands/memory.js +48 -20
  10. package/dist/commands/run.js +52 -55
  11. package/dist/commands/template.js +9 -0
  12. package/dist/commands/uninstall.js +15 -14
  13. package/dist/commands/workflows/agent-helpers.js +7 -7
  14. package/dist/commands/workflows/generate.js +54 -32
  15. package/dist/commands/workflows/run-local.js +19 -19
  16. package/dist/commands/workflows/run.js +5 -5
  17. package/dist/commands/workflows/start.js +12 -12
  18. package/dist/package.json +2 -2
  19. package/dist/templates/zibby-workflow-claude/agents-md-block.md +65 -5
  20. package/dist/templates/zibby-workflow-claude/claude/agents/zibby-test-author.md +16 -1
  21. package/dist/templates/zibby-workflow-claude/claude/agents/zibby-workflow-builder.md +22 -2
  22. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-add-node.md +1 -1
  23. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-debug.md +1 -1
  24. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-delete.md +1 -1
  25. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-deploy.md +24 -14
  26. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-list.md +2 -2
  27. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-memory-cost.md +39 -0
  28. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-memory-pull.md +47 -0
  29. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-memory-remote-use-hosted.md +61 -0
  30. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-memory-stats.md +38 -0
  31. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-static-ip.md +8 -6
  32. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-tail.md +1 -1
  33. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-test-debug.md +2 -2
  34. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-test-generate.md +1 -1
  35. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-test-run.md +3 -2
  36. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-test-write.md +1 -1
  37. package/dist/templates/zibby-workflow-claude/claude/commands/zibby-trigger.md +10 -6
  38. package/dist/templates/zibby-workflow-claude/cursor/rules/zibby-workflows.mdc +76 -13
  39. package/dist/templates/zibby-workflow-claude/manifest.json +5 -1
  40. package/dist/utils/agent-credentials.js +4 -3
  41. package/dist/utils/apply-memory-sync-config.js +1 -0
  42. package/dist/utils/credentials-loader.js +1 -1
  43. package/dist/utils/hosted-memory-sync.js +1 -0
  44. package/package.json +2 -2
@@ -1,4 +1,4 @@
1
- <!-- BEGIN zibby-workflows zibby-template-version: 3 -->
1
+ <!-- BEGIN zibby-workflows zibby-template-version: 4 -->
2
2
  ## Zibby
3
3
 
4
4
  This project uses **Zibby** — there are two surfaces:
@@ -16,7 +16,7 @@ Both share `.zibby.config.mjs` at the project root.
16
16
  Files:
17
17
  ```
18
18
  <paths.workflows or .zibby/workflows>/<name>/
19
- ├── workflow.json name, entryClass, triggers, schemas
19
+ ├── workflow.json name, entryClass, triggers, schemas (manifest)
20
20
  ├── graph.mjs nodes + edges from START to END
21
21
  ├── nodes/
22
22
  │ ├── index.mjs barrel export
@@ -33,19 +33,42 @@ Each node has `async run(ctx)` where `ctx` provides:
33
33
  Common dev loop:
34
34
  ```
35
35
  zibby workflow new <name> # scaffold
36
- zibby workflow run <name> # one-shot local run (mirrors trigger flags)
36
+ zibby workflow run <name> # one-shot local run (preferred for the dev loop)
37
37
  zibby workflow run <name> -p k=v # with input
38
38
  zibby workflow deploy <name> # build + push to Zibby Cloud
39
39
  zibby workflow trigger <uuid> # invoke the cloud workflow
40
40
  zibby workflow logs <uuid> -t # tail live logs (docker-compose-style)
41
- zibby workflow list # find UUIDs and statuses
41
+ zibby workflow list # find UUIDs and statuses (local + cloud)
42
+ zibby workflow download <uuid> # pull the cloud workflow source back to .zibby/workflows/
42
43
  zibby workflow delete <uuid> # remove a deployed workflow
43
44
  ```
44
45
 
45
- `run` and `trigger` accept the same input flags (`-p key=value`, `--input '<json>'`, `--input-file path.json`) flip the verb to switch between local and cloud. `workflow start` exists too but is the long-lived dev server (Studio integration); for plain CLI iteration prefer `run`.
46
+ **`run` vs `start`.** `workflow run` is the one-shot CLI iteration command load the graph, execute it once, print the result, exit. That's the right primitive for the dev loop and for CI/CD. `workflow start` is a *long-lived* local dev server (default port 3848) used by Studio for replay/debug; for plain CLI iteration always prefer `run`.
47
+
48
+ `run` and `trigger` accept the same input flag surface — flip the verb to switch between local and cloud:
49
+ - `-p key=value` (repeatable) — highest precedence
50
+ - `--input '<json>'` — JSON string
51
+ - `--input-file path.json` — JSON file, lowest precedence
46
52
 
47
53
  Static outbound IPs (for customers behind firewalls): see `--dedicated-ip` flag on `deploy`.
48
54
 
55
+ #### Per-workflow env vars
56
+
57
+ Each deployed workflow has its own encrypted env-var bag (KMS-backed). Vars get injected into the Fargate task at trigger time, and **workflow env wins over project secrets on conflict**. Use this for per-pipeline credentials (different `ANTHROPIC_API_KEY` per workflow, a workflow-only `DATABASE_URL`, etc.).
58
+
59
+ ```
60
+ zibby workflow env list <uuid> # show key names (values never returned)
61
+ zibby workflow env set <uuid> ANTHROPIC_API_KEY=sk-… # add or rotate one key
62
+ zibby workflow env unset <uuid> OLD_KEY # remove one key
63
+ zibby workflow env push <uuid> --file .env [--file .env.prod] # bulk replace from .env files
64
+ ```
65
+
66
+ Fast path on first deploy — sync a `.env` in one shot:
67
+ ```
68
+ zibby workflow deploy my-pipeline --env .env [--env .env.prod]
69
+ ```
70
+ The CLI deploys, then runs `push` against the freshly-minted UUID.
71
+
49
72
  ---
50
73
 
51
74
  ### Tests
@@ -55,6 +78,7 @@ Files:
55
78
  test-specs/ source `.txt` specs (paths.specs)
56
79
  tests/ generated `.spec.js` (paths.generated; regenerated each run)
57
80
  test-results/ videos, traces, JSON results per run
81
+ .zibby/memory/.dolt/ local test memory DB (selectors, page model, history)
58
82
  playwright.config.js
59
83
  ```
60
84
 
@@ -64,6 +88,8 @@ Common dev loop:
64
88
  ```
65
89
  zibby test test-specs/<name>.txt # run a spec
66
90
  zibby test "go to example.com and ..." # inline, no file
91
+ zibby test <spec> --agent claude # override the configured agent (claude|cursor|codex|gemini)
92
+ zibby test --sources <ids> --execution <id> # cloud test cases (run from a stored execution)
67
93
  zibby generate -t ENG-1234 # generate specs from a Jira ticket
68
94
  zibby video # organize videos next to spec files
69
95
  zibby upload <spec-path> # upload existing artifacts to cloud
@@ -71,6 +97,39 @@ zibby upload <spec-path> # upload existing artifacts to cloud
71
97
 
72
98
  When debugging a failed test, watch the video at `test-results/<spec>/video.webm` — that's almost always faster than reading logs.
73
99
 
100
+ #### Test memory
101
+
102
+ `.zibby/memory/.dolt/` is a **local-first Dolt SQL database** (Git-for-data) that learns from every test run — selectors that worked, page-element fingerprints, navigation transitions, timing quirks, recorded insights. The runner auto-pulls before a run and auto-pushes after a passing run. Keying is **per-domain** (not per-spec), so any spec that hits `myapp.com` benefits from selectors learned by every other spec on the same domain.
103
+
104
+ When `zibby test` runs and `.zibby/memory/.dolt/` exists, the agent gets 5 MCP tools auto-exposed:
105
+
106
+ - `memory_get_test_history` — query recent runs (pass/fail/timing); filter by spec-path substring
107
+ - `memory_get_selectors` — query known selectors per page with stability metrics (success/fail counts)
108
+ - `memory_get_page_model` — query page structure (elements, roles, accessible names, best selector)
109
+ - `memory_get_navigation` — known page-to-page transitions (what click/submit produced what URL)
110
+ - `memory_save_insight` — save an observation. Categories: `selector_tip | timing | navigation | workaround | flaky | general`
111
+
112
+ > **AFTER completing the test, you MUST call `memory_save_insight` at least once.** Save any useful finding: reliable selectors, timing quirks, navigation patterns, workarounds. Be specific — future runs will read your insights. (Lifted from the memory skill's `promptFragment`.)
113
+
114
+ Local CLI:
115
+ ```
116
+ zibby memory stats # row counts, last commit, per-spec breakdown
117
+ zibby memory cost # real LLM token spend per spec / per domain
118
+ zibby memory compact # prune old runs + Dolt GC (--max-runs 50, --max-age 90d)
119
+ zibby memory reset -f # wipe the DB
120
+ ```
121
+
122
+ **Team sync.** Memory is local-first; opt into a shared remote so teammates' learnings flow back to you:
123
+
124
+ ```
125
+ zibby memory remote add aws://my-bucket/team/proj/main # BYO S3 / GCS / DoltHub / file:///
126
+ zibby memory remote use --hosted # OR: Zibby-managed S3 (signed-in only)
127
+ zibby memory pull # manual override (auto on test start)
128
+ zibby memory push # manual override (auto on passing test)
129
+ ```
130
+
131
+ Set `memorySync.remote` in `.zibby.config.mjs` (`'hosted'` or an `aws://...` URL) and `zibby init` auto-wires the remote — teammates clone the repo, run `zibby init`, and they're plugged into the same memory.
132
+
74
133
  ---
75
134
 
76
135
  ### How to invoke the CLI
@@ -97,6 +156,7 @@ Don't waste time on `npx @zibby/cli` — not always published.
97
156
  - Deploying & bundling: https://docs.zibby.app/workflows/deploying
98
157
  - Triggering & inputs: https://docs.zibby.app/workflows/triggers
99
158
  - Live log streaming: https://docs.zibby.app/workflows/logs
159
+ - Per-workflow env vars: https://docs.zibby.app/cloud/env-vars
100
160
  - Egress proxy / static IPs: https://docs.zibby.app/workflows/egress
101
161
  - Security & secrets: https://docs.zibby.app/workflows/security
102
162
  - Debugging: https://docs.zibby.app/workflows/debugging
@@ -1,4 +1,4 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  ---
3
3
  name: zibby-test-author
4
4
  description: Sub-agent that helps the user design and author Zibby test specs end-to-end. Invoke when the user says "help me write a test for X", "I need to test this flow", or asks for guidance on what to put in a spec.
@@ -53,12 +53,27 @@ A spec is unambiguous English with one action per line. See `/zibby-test-write`
53
53
 
54
54
  5. **Stop when the spec exercises the goal end-to-end.** Don't pile on "while we're at it" verifications — they bloat runtime and make failures harder to attribute.
55
55
 
56
+ ## Test memory (`.zibby/memory/.dolt/`)
57
+
58
+ When `zibby test` runs and `.zibby/memory/.dolt/` exists (initialized by `zibby memory init` or auto-created on first run with `-m` / a `memorySync.remote` config), the agent gets 5 MCP tools auto-exposed. They read from a local-first Dolt SQL DB that learns selectors, page model, navigation, and history **per-domain** across every spec hitting the same site:
59
+
60
+ - `memory_get_test_history` — recent runs (filter by spec-path substring) — pass/fail and timing
61
+ - `memory_get_selectors` — known selectors per page with stability metrics (success/fail counts)
62
+ - `memory_get_page_model` — page elements, ARIA roles, accessible names, best-known selector
63
+ - `memory_get_navigation` — known page-to-page transitions (what click/submit produced what URL)
64
+ - `memory_save_insight` — save observations: `selector_tip | timing | navigation | workaround | flaky | general`
65
+
66
+ > **Hard rule: after every test run, the agent MUST call `memory_save_insight` at least once.** Save reliable selectors, timing quirks, navigation patterns, workarounds — be specific. Future runs read these. (This is in the memory skill's prompt fragment; surface it to the user if they ask why their tests keep getting smarter.)
67
+
68
+ Team sync (optional): a project may have `memorySync.remote: 'hosted'` (Zibby-managed S3, signed-in only) or `'aws://...' / 'gs://...'` (BYO) configured in `.zibby.config.mjs`. If set, the runner auto-pulls before each run and auto-pushes after passing runs. Manual override: `zibby memory pull` / `zibby memory push`.
69
+
56
70
  ## Hard rules
57
71
 
58
72
  - **Never recommend `--headless` for first runs.** Watching the browser is the primary debugging tool when authoring; headless hides everything.
59
73
  - **Never recommend disabling video.** Videos are 99% of post-mortem signal; they're cheap.
60
74
  - **Don't write CSS selectors into specs.** Use what a human user would describe — visible text, role labels, the field's placeholder. Selectors belong in generated `.spec.js`, not the source.
61
75
  - **Don't suggest `npx playwright test` directly** to bypass Zibby for "speed". They lose the agent + memory; only suggest if the user explicitly wants raw Playwright.
76
+ - **Always call `memory_save_insight` at the end of a test run.** This is non-negotiable — without it, memory degrades to the seeded baseline and stops compounding.
62
77
 
63
78
  ## Reference
64
79
 
@@ -1,4 +1,4 @@
1
- <!-- zibby-template-version: 2 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  ---
3
3
  name: zibby-workflow-builder
4
4
  description: Sub-agent that walks the user through building, testing, and deploying a Zibby agent workflow end-to-end. Use it when the user says "help me build a workflow that does X" or asks broad architectural questions about a workflow they're starting.
@@ -49,7 +49,7 @@ The return value of `run()` is the node's output, available to downstream nodes
49
49
 
50
50
  4. **Generate the scaffold** if they don't have one yet:
51
51
  ```
52
- zibby workflow generate <slug>
52
+ zibby workflow new <slug>
53
53
  ```
54
54
  Then add nodes one at a time using the `/zibby-add-node` command.
55
55
 
@@ -64,6 +64,26 @@ The return value of `run()` is the node's output, available to downstream nodes
64
64
 
65
65
  6. **Stop when the workflow does the goal end-to-end.** Don't pile on speculative nodes.
66
66
 
67
+ ## Per-workflow env vars
68
+
69
+ Each deployed workflow has its own encrypted env-var bag (KMS-backed). Workflow env wins over project secrets on conflict.
70
+
71
+ - `zibby workflow env list <uuid>` — show key names (values never returned)
72
+ - `zibby workflow env set <uuid> ANTHROPIC_API_KEY=sk-…` — add or rotate one key
73
+ - `zibby workflow env unset <uuid> OLD_KEY` — remove one key
74
+ - `zibby workflow env push <uuid> --file .env [--file .env.prod]` — bulk replace from .env files (later files override)
75
+ - `zibby workflow deploy <slug> --env .env` — fast path: deploy + auto-`push` of .env to the new UUID
76
+
77
+ Use this for credentials specific to one workflow (per-pipeline `ANTHROPIC_API_KEY`, a workflow-only `DATABASE_URL`, an external webhook secret). Project-wide secrets stay on the project record.
78
+
79
+ ## Pulling a deployed workflow back to local
80
+
81
+ ```
82
+ zibby workflow download <uuid>
83
+ ```
84
+
85
+ Pulls the cloud workflow's source back into `.zibby/workflows/<name>/`. Useful when collaborators need the source from cloud (e.g. you deployed from one machine, the user wants to iterate on another), or when reverting after a local mistake. UUIDs come from `zibby workflow list`.
86
+
67
87
  ## Hard rules
68
88
 
69
89
  - **Never recommend `--force` flags or skipping checks** to make a deploy go faster. Build problems are signal.
@@ -1,4 +1,4 @@
1
- <!-- zibby-template-version: 2 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-add-node — scaffold a new node in a Zibby workflow
3
3
 
4
4
  You are helping the user add a new **node** to one of their Zibby agent workflows.
@@ -1,4 +1,4 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-debug — diagnose a failing or stuck Zibby workflow
3
3
 
4
4
  You are helping the user debug a workflow that didn't behave as expected.
@@ -1,4 +1,4 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-delete — delete a deployed Zibby workflow
3
3
 
4
4
  You are helping the user remove a workflow from Zibby Cloud.
@@ -1,4 +1,4 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-deploy — deploy a Zibby workflow to the cloud
3
3
 
4
4
  You are helping the user deploy a workflow they've been building locally.
@@ -47,31 +47,41 @@ Canonical docs: **https://docs.zibby.app/workflows/deploying**
47
47
 
48
48
  ## Optional flags worth knowing
49
49
 
50
+ `zibby workflow deploy` accepts:
50
51
  - `--project <id>` — skip the interactive project picker
51
52
  - `--api-key <key>` — use a PAT instead of the session token (for CI)
53
+ - `--env <path>` — sync a `.env` file into per-workflow env vars after deploy. Repeatable; later files override.
52
54
  - `--verbose` — print raw CodeBuild output during the build (helpful for debugging build failures)
53
- - `--dedicated-ip <action>` — opt this workflow into the dedicated egress addon (static outbound IP). See `/zibby-static-ip` for setup.
54
55
 
55
- ## Static outbound IP (dedicated egress) at deploy time
56
+ ### Seeding per-workflow env on first deploy
56
57
 
57
- If the user's workflow needs to call APIs that require IP allowlisting (corporate GitHub, GitLab Enterprise, paranoid SaaS firewalls), the workflow needs the **dedicated egress IP** addon enabled on their account, AND the workflow must opt in.
58
+ If the workflow needs its own `ANTHROPIC_API_KEY`, `DATABASE_URL`, etc., put them in a `.env` and pass `--env`:
58
59
 
59
- Three flags map to three things:
60
+ ```
61
+ zibby workflow deploy <name> --env .env
62
+ zibby workflow deploy <name> --env .env --env .env.prod # later files win
63
+ ```
64
+
65
+ After deploy, manage them surgically with `zibby workflow env set/unset/list/push <uuid>`. See `/zibby-list` to recover the UUID; full guide at https://docs.zibby.app/cloud/env-vars.
66
+
67
+ ## Static outbound IP (dedicated egress)
68
+
69
+ If the user's workflow needs to call APIs that require IP allowlisting (corporate GitHub, GitLab Enterprise, paranoid SaaS firewalls), the workflow needs the **dedicated egress IP** addon. The flag lives on the legacy alias `zibby deploy` (NOT `zibby workflow deploy`):
60
70
 
61
71
  | Flag | What it does |
62
72
  |------|-------------|
63
- | `--dedicated-ip status` | Show current addon state for the account (active / inactive / billing) |
64
- | `--dedicated-ip enable` | Enable the addon on the account (Pro subscription required, ~$50/mo). One-time per account. |
65
- | `--dedicated-ip use` | Mark THIS workflow as using the static egress IP (per-workflow opt-in, after `enable`) |
66
- | `--dedicated-ip unuse` | Stop routing this workflow through the static IP |
67
- | `--dedicated-ip disable` | Disable the addon for the whole account |
73
+ | `zibby deploy <name> --dedicated-ip status` | Show current addon state for the account (active / inactive / billing) |
74
+ | `zibby deploy <name> --dedicated-ip enable` | Enable the addon on the account (Pro subscription required, ~$50/mo). One-time per account. |
75
+ | `zibby deploy <name> --dedicated-ip use` | Mark THIS workflow as using the static egress IP (per-workflow opt-in, after `enable`) |
76
+ | `zibby deploy <name> --dedicated-ip unuse` | Stop routing this workflow through the static IP |
77
+ | `zibby deploy <name> --dedicated-ip disable` | Disable the addon for the whole account |
68
78
 
69
79
  Typical first-time flow when the user says "I need a static outbound IP":
70
- 1. `zibby workflow deploy <name> --dedicated-ip status` — check whether they have it
71
- 2. If inactive → `zibby workflow deploy <name> --dedicated-ip enable` — enables the account-wide addon (interactive billing prompt; prerequisite Pro subscription)
72
- 3. `zibby workflow deploy <name> --dedicated-ip use` — opts this specific workflow in
80
+ 1. `zibby deploy <name> --dedicated-ip status` — check whether they have it
81
+ 2. If inactive → `zibby deploy <name> --dedicated-ip enable` — enables the account-wide addon (interactive billing prompt; prerequisite Pro subscription)
82
+ 3. `zibby deploy <name> --dedicated-ip use` — opts this specific workflow in
73
83
  4. Regular `zibby workflow deploy <name>` from now on uses the static IP
74
84
 
75
85
  After `--dedicated-ip use`, every node in this workflow gets its outbound HTTP routed through the egress proxy, and `process.env.HTTP_PROXY` / `HTTPS_PROXY` are set in the sandbox automatically. Their static IPs are visible to customers via `https://docs.zibby.app/workflows/egress`.
76
86
 
77
- **Don't** run `--dedicated-ip enable` without confirming with the user — it has billing impact ($50/mo addon). Always confirm.
87
+ **Don't** run `--dedicated-ip enable` without confirming with the user — it has billing impact ($50/mo addon). Always confirm. See `/zibby-static-ip` for the deeper walkthrough.
@@ -1,9 +1,9 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-list — list workflows (local + cloud) with their UUIDs and statuses
3
3
 
4
4
  You are helping the user see what workflows exist — locally scaffolded and remotely deployed.
5
5
 
6
- Canonical docs: **https://docs.zibby.app/workflows/listing**
6
+ Canonical docs: **https://docs.zibby.app/cli-reference#workflow-list**
7
7
 
8
8
  ## Steps
9
9
 
@@ -0,0 +1,39 @@
1
+ <!-- zibby-template-version: 4 -->
2
+ # /zibby-memory-cost — show real LLM token spend across past test runs
3
+
4
+ You are helping the user see how many input/output/cache tokens their tests have actually burned, broken down per spec and per domain. This is real measured spend (read off run records in `.zibby/memory/.dolt/`), not an estimate.
5
+
6
+ Canonical docs: **https://docs.zibby.app/tests/memory**
7
+
8
+ ## What the command shows
9
+
10
+ ```
11
+ Bash(zibby memory cost)
12
+ ```
13
+
14
+ Per-spec and per-domain rollup of:
15
+ - Input tokens
16
+ - Output tokens
17
+ - Cache hit / cache write tokens (when the agent supports prompt caching)
18
+ - Estimated $ cost (uses current public model pricing)
19
+ - Recent-runs trend, so you can see if a spec is getting cheaper or more expensive over time
20
+
21
+ The numbers are pulled from `test_runs` rows in the Dolt DB — every test run records the agent's actual usage on completion.
22
+
23
+ ## When to invoke
24
+
25
+ - User asks "how much are my tests costing me?" or "which spec is the expensive one?"
26
+ - After enabling prompt caching to confirm cache hits are landing
27
+ - When deciding whether to swap to a cheaper agent on hot specs (`--agent` per run)
28
+ - When triaging a regression in test runtime — high token counts often correlate with the agent retrying
29
+
30
+ ## Caveats
31
+
32
+ - **Only counts what's in local memory.** Runs on machines that haven't pulled from the team remote won't appear. Run `/zibby-memory-pull` first if you want the full team picture.
33
+ - **Pricing is informational.** Public API pricing changes; treat the $ column as a guide, not a bill. The token counts themselves are exact.
34
+ - **Empty if you've never run a test with memory enabled.** Confirm the runs are in there with `/zibby-memory-stats` first.
35
+
36
+ ## Related
37
+
38
+ - `/zibby-memory-stats` — what's in the DB at all
39
+ - `/zibby-memory-pull` — refresh from team remote before reading cost
@@ -0,0 +1,47 @@
1
+ <!-- zibby-template-version: 4 -->
2
+ # /zibby-memory-pull — pull the team's latest test memory from the configured remote
3
+
4
+ You are helping the user fetch the team's latest learnings (selectors, page model, insights, run history) from the project's configured memory remote into local `.zibby/memory/.dolt/`.
5
+
6
+ Canonical docs: **https://docs.zibby.app/tests/memory**
7
+
8
+ ## When this is needed (vs. just runs automatically)
9
+
10
+ `zibby test` auto-pulls before every run when a remote is configured, and auto-pushes after every passing run. So most of the time the user doesn't need to invoke pull manually. Manual pull is for:
11
+
12
+ - Fresh clone of the repo — first sync to seed `.zibby/memory/.dolt/` from the remote
13
+ - After a teammate landed a big batch of new learnings and the user wants them before running anything
14
+ - Inspecting team memory (`/zibby-memory-stats`, `/zibby-memory-cost`) without running a test
15
+ - Reconciling after a manual conflict in the Dolt DB
16
+
17
+ ## How to run
18
+
19
+ ```
20
+ Bash(zibby memory pull)
21
+ ```
22
+
23
+ The CLI fetches from whatever remote `zibby memory remote info` reports — BYO S3/GCS/DoltHub URL or the Zibby-hosted backend. No flags.
24
+
25
+ ## Pre-flight: is a remote configured?
26
+
27
+ Before suggesting `pull`, check:
28
+
29
+ ```
30
+ Bash(zibby memory remote info)
31
+ ```
32
+
33
+ - **No remote configured** → pull errors out. Tell the user to either:
34
+ - Add their own: `zibby memory remote add aws://my-bucket/team/proj/main`
35
+ - Use the hosted one: `zibby memory remote use --hosted` (requires `zibby login`)
36
+ - See `/zibby-memory-remote-use-hosted` for the hosted path.
37
+ - **Hosted remote, signed out** → `zibby login` first.
38
+
39
+ ## After pulling
40
+
41
+ Confirm the pull landed with `/zibby-memory-stats` — row counts should jump (selectors, runs, insights) compared to before.
42
+
43
+ ## Related
44
+
45
+ - `zibby memory push` — manual push (auto on passing test, but sometimes you want to share now)
46
+ - `/zibby-memory-stats` — verify what came in
47
+ - `/zibby-memory-remote-use-hosted` — switch to the Zibby-managed S3 backend
@@ -0,0 +1,61 @@
1
+ <!-- zibby-template-version: 4 -->
2
+ # /zibby-memory-remote-use-hosted — switch this project's memory remote to Zibby-managed S3
3
+
4
+ You are helping the user point their `.zibby/memory/.dolt/` at Zibby's hosted S3 backend, instead of running their own S3 bucket / GCS / DoltHub repo.
5
+
6
+ Canonical docs: **https://docs.zibby.app/tests/memory**
7
+
8
+ ## What this does
9
+
10
+ ```
11
+ Bash(zibby memory remote use --hosted)
12
+ ```
13
+
14
+ Allocates a tenant-scoped prefix on Zibby-managed S3 for this project (keyed on the projectId in `.zibby.config.mjs`) and writes that as the local Dolt remote. After this, every `zibby test` run auto-pulls before and auto-pushes after — same as a BYO remote, just without the bucket plumbing.
15
+
16
+ ## Prerequisite: signed in
17
+
18
+ Hosted remote is **signed-in users only**. Verify:
19
+
20
+ ```
21
+ Bash(zibby status)
22
+ ```
23
+
24
+ If not signed in, run `zibby login` first. The CLI uses the saved session to derive the tenant prefix; it won't fall back to anonymous.
25
+
26
+ ## When to use hosted vs BYO
27
+
28
+ | | Hosted (`--hosted`) | BYO (`zibby memory remote add aws://...`) |
29
+ |---|---|---|
30
+ | Setup time | Zero — `--hosted` and you're done | Provision an S3 bucket, IAM, optional KMS |
31
+ | Who can read | Everyone with project access on Zibby | Whoever you grant in IAM |
32
+ | Where data lives | Zibby-managed AWS account | Your account |
33
+ | Compliance / data-residency | Limited regions | Wherever you want |
34
+ | Cost | Included in plan | Your S3 bill |
35
+
36
+ If the user has any data-residency requirement or a regulated workload, prefer BYO. Otherwise hosted is the path of least resistance.
37
+
38
+ ## Switching from BYO to hosted
39
+
40
+ `zibby memory remote use --hosted` overwrites the existing remote. If they had a BYO remote and might want to keep its history, run `zibby memory push` against the old remote first so nothing's lost — then switch.
41
+
42
+ ## After switching
43
+
44
+ 1. `zibby memory pull` — seed `.zibby/memory/.dolt/` from the hosted prefix (no-op the very first time per project)
45
+ 2. `/zibby-memory-stats` — confirm
46
+ 3. Commit `.zibby.config.mjs` if you set `memorySync.remote: 'hosted'` so teammates auto-wire on next `zibby init`
47
+
48
+ ## Reverting
49
+
50
+ ```
51
+ Bash(zibby memory remote remove)
52
+ ```
53
+
54
+ Drops the remote — memory becomes local-only again. The data on Zibby's S3 isn't deleted (it's still tenant-scoped), but nothing pushes or pulls until a new remote is configured.
55
+
56
+ ## Related
57
+
58
+ - `/zibby-memory-pull` — manual pull (auto on test start)
59
+ - `/zibby-memory-stats` — verify what's in the local DB
60
+ - `zibby memory remote info` — show current remote config
61
+ - `zibby memory remote add <url>` — BYO remote (S3/GCS/DoltHub/file:///)
@@ -0,0 +1,38 @@
1
+ <!-- zibby-template-version: 4 -->
2
+ # /zibby-memory-stats — inspect the local test memory database
3
+
4
+ You are helping the user see what's in their `.zibby/memory/.dolt/` test-memory DB — row counts per table, last commit, and per-spec breakdown.
5
+
6
+ Canonical docs: **https://docs.zibby.app/tests/memory**
7
+
8
+ ## What the command shows
9
+
10
+ ```
11
+ Bash(zibby memory stats)
12
+ ```
13
+
14
+ Prints a summary of the local Dolt database:
15
+ - **Test runs** — total runs recorded, pass/fail split, last run timestamp
16
+ - **Selectors** — total cached selectors, top pages by selector count
17
+ - **Page model** — pages mapped, total elements
18
+ - **Navigation** — known transitions
19
+ - **Insights** — count by category (`selector_tip | timing | navigation | workaround | flaky | general`)
20
+ - **Dolt status** — current branch, last commit hash, uncommitted changes
21
+
22
+ ## When to invoke
23
+
24
+ - User asks "what does Zibby know about my app?" or "show me what's in test memory"
25
+ - After running a few tests, to confirm the agent is actually persisting learnings
26
+ - Before a `zibby memory compact` to see how much there is to prune
27
+ - Before a `zibby memory remote add` to know what's about to ship to the team
28
+
29
+ ## Empty database?
30
+
31
+ If the user just ran `zibby memory init` (or it auto-initialized on first `zibby test`), most counts will be 0. That's fine — selectors and page model populate after the first successful run. Suggest running a test first.
32
+
33
+ ## Related commands
34
+
35
+ - `/zibby-memory-cost` — real LLM token spend per spec / per domain
36
+ - `/zibby-memory-pull` — pull team's latest learnings from the configured remote
37
+ - `zibby memory compact` — prune old runs (`--max-runs N`, `--max-age <days>`)
38
+ - `zibby memory reset -f` — wipe the DB (destructive — confirm first)
@@ -1,10 +1,12 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-static-ip — set up dedicated outbound static IP for a workflow
3
3
 
4
4
  You are helping the user route a workflow's outbound traffic through a static IP address — needed when the workflow calls APIs that require IP allowlisting (corporate GitLab/GitHub Enterprise, internal SaaS, firewalls).
5
5
 
6
6
  Canonical docs: **https://docs.zibby.app/workflows/egress**
7
7
 
8
+ > Note: the `--dedicated-ip` flag lives on the legacy alias `zibby deploy <name>`, NOT on `zibby workflow deploy <name>`. The two share a handler, but only `zibby deploy` exposes this flag in `--help`.
9
+
8
10
  ## What "static IP" means here
9
11
 
10
12
  By default, workflow tasks run on Fargate and their outbound traffic exits via AWS-managed IPs that rotate. With the **dedicated egress** addon enabled, the workflow's outbound traffic is routed through a Zibby-managed proxy whose IP is pinned and customer-allowlistable.
@@ -23,19 +25,19 @@ Two pieces:
23
25
 
24
26
  2. **Check current state:**
25
27
  ```
26
- Bash(zibby workflow deploy <name> --dedicated-ip status)
28
+ Bash(zibby deploy <name> --dedicated-ip status)
27
29
  ```
28
30
  Output tells you: addon active or inactive, this workflow currently using it or not, and the assigned IPs to publish to customers.
29
31
 
30
32
  3. **If addon is inactive — enable it** (only after explicit user confirmation):
31
33
  ```
32
- Bash(zibby workflow deploy <name> --dedicated-ip enable)
34
+ Bash(zibby deploy <name> --dedicated-ip enable)
33
35
  ```
34
36
  This is one-time per account. After this, the addon is active for ALL workflows in the account that opt in.
35
37
 
36
38
  4. **Opt this workflow in:**
37
39
  ```
38
- Bash(zibby workflow deploy <name> --dedicated-ip use)
40
+ Bash(zibby deploy <name> --dedicated-ip use)
39
41
  ```
40
42
  From now on, every deploy of this workflow + every triggered execution routes outbound through the static IP.
41
43
 
@@ -52,8 +54,8 @@ Two pieces:
52
54
 
53
55
  ## Reverting
54
56
 
55
- - `Bash(zibby workflow deploy <name> --dedicated-ip unuse)` — stop routing this workflow's egress through the static IP. Other opted-in workflows are unaffected.
56
- - `Bash(zibby workflow deploy <name> --dedicated-ip disable)` — disable the addon entirely (also stops billing).
57
+ - `Bash(zibby deploy <name> --dedicated-ip unuse)` — stop routing this workflow's egress through the static IP. Other opted-in workflows are unaffected.
58
+ - `Bash(zibby deploy <name> --dedicated-ip disable)` — disable the addon entirely (also stops billing).
57
59
 
58
60
  ## Tell the user the IPs
59
61
 
@@ -1,4 +1,4 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-tail — stream live logs from a Zibby workflow
3
3
 
4
4
  You are helping the user tail logs from a workflow execution.
@@ -1,4 +1,4 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-test-debug — diagnose a failing Zibby test
3
3
 
4
4
  You are helping the user figure out why a test failed.
@@ -34,7 +34,7 @@ Spec ambiguity is the most common cause. If the spec says "click the button" and
34
34
  ```
35
35
  Bash(zibby test test-specs/<name>.txt --verbose) # info-level logs
36
36
  Bash(zibby test test-specs/<name>.txt --debug) # all logs, lots
37
- Bash(zibby test test-specs/<name>.txt --headed) # see the browser
37
+ Bash(zibby test test-specs/<name>.txt) # default is headed — drop --headless to watch the browser
38
38
  ```
39
39
 
40
40
  ### 5. Re-execute one node from a prior session
@@ -1,4 +1,4 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-test-generate — generate test specs from a Jira ticket / requirements
3
3
 
4
4
  You are helping the user auto-generate test specs from a ticket description (Jira) or a free-text requirements doc. Zibby's `generate` command runs the configured AI agent against the codebase + ticket and produces `.txt` specs in `test-specs/`.
@@ -1,4 +1,4 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-test-run — execute a Zibby test spec
3
3
 
4
4
  You are helping the user run an existing test spec through Zibby. A spec is a `.txt` file describing what to test in plain language; Zibby's runner turns it into a Playwright execution and produces a video + JSON results.
@@ -40,9 +40,10 @@ Canonical docs: **https://docs.zibby.app/tests/running**
40
40
  - `--verbose` / `--debug` — escalate log levels
41
41
  - `-m, --mem` — enable test memory (Dolt-backed knowledge from prior runs)
42
42
  - `--sync` / `--no-sync` — force / skip cloud upload regardless of config
43
+ - `--sources <ids> --execution <id>` — run cloud-stored test cases from a specific execution (comma-separated IDs)
43
44
 
44
45
  ## Common failure modes
45
46
 
46
47
  - **"No spec found"** — path is relative to project root, not cwd. Check `paths.specs` in `.zibby.config.mjs`.
47
- - **"Browser crashed"** — usually the playwright browser cache is stale. Re-run with `--headed` once to refresh, then `--headless`.
48
+ - **"Browser crashed"** — usually the playwright browser cache is stale. Drop `--headless` once (default is headed) so you can see what's happening, then re-add `--headless` once it's healthy.
48
49
  - **MCP errors during `execute_live`** — the agent's MCP tool config may need refreshing. See `/zibby-test-debug`.
@@ -1,4 +1,4 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-test-write — author a new Zibby test spec
3
3
 
4
4
  You are helping the user write a new test spec. Specs are plain-language `.txt` files in `test-specs/` (configurable via `.zibby.config.mjs` `paths.specs`). Zibby's runner converts them to Playwright at execution time.
@@ -1,11 +1,11 @@
1
- <!-- zibby-template-version: 1 -->
1
+ <!-- zibby-template-version: 4 -->
2
2
  # /zibby-trigger — run a deployed Zibby workflow
3
3
 
4
4
  You are helping the user trigger a deployed workflow execution.
5
5
 
6
6
  A trigger creates a new ECS Fargate task that loads the workflow's bundle, runs the graph, and writes status + logs as it goes.
7
7
 
8
- Canonical docs: **https://docs.zibby.app/workflows/triggering**
8
+ Canonical docs: **https://docs.zibby.app/cloud/triggering**
9
9
 
10
10
  ## Steps
11
11
 
@@ -13,15 +13,19 @@ Canonical docs: **https://docs.zibby.app/workflows/triggering**
13
13
 
14
14
  2. **Construct the input.** Workflows take a JSON input that nodes can read via `ctx.input`. Ask the user what input the workflow expects (or read `workflow.json`'s `inputSchema` if present).
15
15
 
16
- 3. **Run the trigger:**
16
+ 3. **Run the trigger.** Three ways to pass input — they merge with this **precedence (highest → lowest)**:
17
+ 1. `-p key=value` (repeatable) — wins over everything; great for shell-friendly tweaks on top of a base payload
18
+ 2. `--input '<json>'` — full JSON payload as a string
19
+ 3. `--input-file path.json` — full JSON/YAML payload from a file (lowest precedence; `-p` and `--input` override individual keys)
20
+
17
21
  ```
18
22
  zibby workflow trigger <uuid> --input '{"key":"value"}'
19
- ```
20
- Or use `-p key=value` for individual params (more shell-friendly than embedded JSON):
21
- ```
22
23
  zibby workflow trigger <uuid> -p ticket=ENG-1234 -p priority=high
24
+ zibby workflow trigger <uuid> --input-file payload.json -p priority=urgent # mix
23
25
  ```
24
26
 
27
+ Same flag surface as `zibby workflow run` (local) — flip the verb and the same call shape goes from local to remote.
28
+
25
29
  4. **Tail the logs immediately:**
26
30
  ```
27
31
  zibby workflow logs <uuid> -t