@zibby/cli 0.4.16 → 0.4.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/bin/zibby.js +2 -2
- package/dist/commands/init.js +64 -64
- package/dist/commands/workflows/generate.js +108 -108
- package/dist/commands/workflows/schedule.js +10 -0
- package/dist/commands/workflows/validate-helpers.js +1 -0
- package/dist/commands/workflows/validate.js +4 -0
- package/dist/package.json +5 -3
- package/dist/templates/.claude/CLAUDE.md +425 -0
- package/dist/templates/.claude/commands/add-node.md +63 -0
- package/dist/templates/.claude/commands/add-skill.md +83 -0
- package/dist/templates/.claude/commands/new-workflow.md +61 -0
- package/dist/templates/.claude/commands/validate-workflow.md +67 -0
- package/package.json +5 -3
- package/templates/.claude/CLAUDE.md +425 -0
- package/templates/.claude/commands/add-node.md +63 -0
- package/templates/.claude/commands/add-skill.md +83 -0
- package/templates/.claude/commands/new-workflow.md +61 -0
- package/templates/.claude/commands/validate-workflow.md +67 -0
- package/templates/zibby-workflow-claude/agents-md-block.md +173 -0
- package/templates/zibby-workflow-claude/claude/agents/zibby-test-author.md +87 -0
- package/templates/zibby-workflow-claude/claude/agents/zibby-workflow-builder.md +101 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-add-node.md +75 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-debug.md +67 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-delete.md +37 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-deploy.md +87 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-list.md +30 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-memory-cost.md +39 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-memory-pull.md +47 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-memory-remote-use-hosted.md +61 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-memory-stats.md +38 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-static-ip.md +70 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-tail.md +53 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-test-debug.md +59 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-test-generate.md +39 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-test-run.md +49 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-test-write.md +46 -0
- package/templates/zibby-workflow-claude/claude/commands/zibby-trigger.md +56 -0
- package/templates/zibby-workflow-claude/claude/settings.json +10 -0
- package/templates/zibby-workflow-claude/cursor/rules/zibby-workflows.mdc +119 -0
- package/templates/zibby-workflow-claude/manifest.json +47 -0
|
@@ -0,0 +1,75 @@
|
|
|
1
|
+
<!-- zibby-template-version: 4 -->
|
|
2
|
+
# /zibby-add-node — scaffold a new node in a Zibby workflow
|
|
3
|
+
|
|
4
|
+
You are helping the user add a new **node** to one of their Zibby agent workflows.
|
|
5
|
+
|
|
6
|
+
## Context: what is a Zibby workflow?
|
|
7
|
+
|
|
8
|
+
A workflow is a graph of nodes that an AI agent (cursor / claude / codex / gemini) executes in a sandboxed ECS Fargate container.
|
|
9
|
+
- Each node is one `.mjs` file under `<workflow>/nodes/`
|
|
10
|
+
- The graph wires nodes together (`<workflow>/graph.mjs`)
|
|
11
|
+
- `<workflow>/workflow.json` declares the workflow's name, entry class, triggers
|
|
12
|
+
- Workflows live under `<workflowsBasePath>/<workflow-name>/` (default `.zibby/workflows/`, configured in the project root's `.zibby.config.mjs`)
|
|
13
|
+
|
|
14
|
+
For canonical, evolving docs see **https://docs.zibby.app/workflows/nodes**
|
|
15
|
+
|
|
16
|
+
## Steps for this command
|
|
17
|
+
|
|
18
|
+
1. **Identify the target workflow.** Look under the path configured in `.zibby.config.mjs` `paths.workflows` (default `.zibby/workflows/`). If multiple workflows exist, ask the user which one. If they're already `cd`'d inside one, infer from `${cwd}`.
|
|
19
|
+
|
|
20
|
+
2. **Get the node spec from the user.** Ask:
|
|
21
|
+
- Node name (kebab-case, e.g. `analyze-ticket`)
|
|
22
|
+
- One-sentence description of what it does
|
|
23
|
+
- Inputs (variables it reads from prior nodes)
|
|
24
|
+
- Outputs (variables it produces — these become available to downstream nodes)
|
|
25
|
+
|
|
26
|
+
3. **Create the node file** at `<workflow>/nodes/<name>.mjs`. Pattern from the existing `example.mjs`:
|
|
27
|
+
```js
|
|
28
|
+
export default {
|
|
29
|
+
id: '<name>',
|
|
30
|
+
description: '<one-sentence description>',
|
|
31
|
+
async run(ctx) {
|
|
32
|
+
// ctx.input — outputs from upstream nodes
|
|
33
|
+
// ctx.agent — call the configured AI agent
|
|
34
|
+
// ctx.shell — run shell commands in the sandbox
|
|
35
|
+
// return value becomes the node's output
|
|
36
|
+
return { /* ... */ };
|
|
37
|
+
},
|
|
38
|
+
};
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
4. **Register the node in `nodes/index.mjs`** — add the import + export.
|
|
42
|
+
|
|
43
|
+
5. **Wire into `graph.mjs`** — add the node id to the graph's `nodes` array, then add an `edge` from its predecessor (or from `START` if it's first) and an edge to `END` (or its successor).
|
|
44
|
+
|
|
45
|
+
6. **Update `workflow.json`** if the new node introduces an `outputSchema` the workflow's caller relies on. Most nodes don't need this.
|
|
46
|
+
|
|
47
|
+
7. **Test locally:**
|
|
48
|
+
```
|
|
49
|
+
zibby workflow run <workflow-name>
|
|
50
|
+
zibby workflow run <workflow-name> -p ticket=BUG-123 # with input
|
|
51
|
+
```
|
|
52
|
+
One-shot — exits when the run finishes. Same input flag surface as `zibby workflow trigger` (cloud).
|
|
53
|
+
|
|
54
|
+
8. **Deploy when ready:**
|
|
55
|
+
```
|
|
56
|
+
zibby workflow deploy <workflow-name>
|
|
57
|
+
```
|
|
58
|
+
Then `zibby workflow trigger <uuid>` and `zibby workflow logs <uuid> -t` to verify.
|
|
59
|
+
|
|
60
|
+
## Common pitfalls
|
|
61
|
+
|
|
62
|
+
- **Node name must match the file name and `id`** — mismatches cause silent skip.
|
|
63
|
+
- **`ctx.agent` calls block on the LLM** — large prompts can take 30+ seconds. Stream output for visibility.
|
|
64
|
+
- **Don't import npm packages inside `run()`** — declare deps in `<workflow>/package.json`. The deploy bundler installs them.
|
|
65
|
+
- **Failed nodes terminate the workflow** unless wrapped in try/catch and explicit `outputSchema.status: 'warn'`.
|
|
66
|
+
|
|
67
|
+
## When to consult the user vs proceed
|
|
68
|
+
|
|
69
|
+
Always ask before:
|
|
70
|
+
- Creating a node that calls external APIs (cost / data egress concern)
|
|
71
|
+
- Modifying `workflow.json` (changes the contract for downstream callers)
|
|
72
|
+
|
|
73
|
+
Proceed without asking when:
|
|
74
|
+
- Just adding a self-contained node and wiring it
|
|
75
|
+
- Tweaking the example/implementation in response to user spec
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
<!-- zibby-template-version: 4 -->
|
|
2
|
+
# /zibby-debug — diagnose a failing or stuck Zibby workflow
|
|
3
|
+
|
|
4
|
+
You are helping the user debug a workflow that didn't behave as expected.
|
|
5
|
+
|
|
6
|
+
Canonical docs: **https://docs.zibby.app/workflows/debugging**
|
|
7
|
+
|
|
8
|
+
## Diagnostic recipe
|
|
9
|
+
|
|
10
|
+
Apply in order. Stop at the first thing that explains the symptom.
|
|
11
|
+
|
|
12
|
+
### 1. Did the deploy succeed?
|
|
13
|
+
|
|
14
|
+
```
|
|
15
|
+
zibby workflow list
|
|
16
|
+
```
|
|
17
|
+
Find the workflow. If `bundleStatus` isn't `ready`, the deploy didn't finish. Re-run `zibby workflow deploy <name> --verbose` and read the CodeBuild output.
|
|
18
|
+
|
|
19
|
+
### 2. Did the trigger reach ECS?
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
zibby workflow trigger <uuid>
|
|
23
|
+
```
|
|
24
|
+
Look at the response — it should include a `Job ID` immediately. If you get an HTTP error, it's an auth or quota problem (CodeBuild concurrency, ECS task limit, etc.). Surface to the user.
|
|
25
|
+
|
|
26
|
+
### 3. Did the agent task START?
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
zibby workflow logs <uuid> -t
|
|
30
|
+
```
|
|
31
|
+
Within 30s of the trigger you should see `[setup] Fetching bundle...` then `zibby v<version>`. If silence past 30s:
|
|
32
|
+
- Maybe ECS couldn't pull the image — check CloudWatch alarm `zibby-sse-fanout-no-task-prod`
|
|
33
|
+
- Maybe the task started but its log stream is delayed — wait another 30s
|
|
34
|
+
- Maybe the workflow row hasn't been written yet (rare — would only affect the very first second)
|
|
35
|
+
|
|
36
|
+
### 4. Did the workflow execute the wrong path?
|
|
37
|
+
|
|
38
|
+
If the tail shows nodes running but in unexpected order, your `graph.mjs` edges are wrong. Common causes:
|
|
39
|
+
- Edge from `START` is missing — first node never runs
|
|
40
|
+
- Cycle in the graph — runtime errors with "cycle detected"
|
|
41
|
+
- Node id in `nodes/` array doesn't match the file's exported `id`
|
|
42
|
+
|
|
43
|
+
### 5. Did a node fail?
|
|
44
|
+
|
|
45
|
+
The tail will show `Error: Node '<name>' failed: <reason>`. Common reasons:
|
|
46
|
+
- Agent (LLM) returned malformed output that didn't match the node's `outputSchema`
|
|
47
|
+
- Node code threw an uncaught exception
|
|
48
|
+
- Shell command in the sandbox returned non-zero
|
|
49
|
+
|
|
50
|
+
For agent errors, look for `│ Prompt sent to LLM:` and `│ Response:` blocks in the tail. The model's reply is right there.
|
|
51
|
+
|
|
52
|
+
### 6. Did the task die without finishing?
|
|
53
|
+
|
|
54
|
+
Look for `[fanout] hard timeout` in the SSE fan-out logs (sse-fanout container) — means the task ran past the cap. Or the status in DDB stays `running` indefinitely (zombie row). Re-trigger.
|
|
55
|
+
|
|
56
|
+
### 7. Are you seeing logs from a stale execution?
|
|
57
|
+
|
|
58
|
+
`-t` on a workflow UUID auto-attaches to the **latest** existing execution at connect time, plus new ones triggered while it's open. If you're tailing an old failed run, drain it (Ctrl+C, re-run after triggering fresh).
|
|
59
|
+
|
|
60
|
+
## Quick reference: what each piece does
|
|
61
|
+
|
|
62
|
+
- **Trigger** → writes a row to `zibby-prod-executions` (DDB) + spawns an ECS task
|
|
63
|
+
- **Task** → pulls bundle from S3, runs `node graph.mjs`, writes logs to CloudWatch, updates DDB status as it progresses
|
|
64
|
+
- **SSE fan-out** → polls CloudWatch, fans events out to subscribers (`-t` clients)
|
|
65
|
+
- **Status** → moves through `starting → running → completed/failed/error`
|
|
66
|
+
|
|
67
|
+
If `status` in DDB is wrong (e.g. stuck `running` after the task is gone), it's an upstream zombie — separate from any workflow logic issue.
|
|
@@ -0,0 +1,37 @@
|
|
|
1
|
+
<!-- zibby-template-version: 4 -->
|
|
2
|
+
# /zibby-delete — delete a deployed Zibby workflow
|
|
3
|
+
|
|
4
|
+
You are helping the user remove a workflow from Zibby Cloud.
|
|
5
|
+
|
|
6
|
+
**This is destructive.** It removes the workflow record, its bundle in S3, and its routing — but does NOT delete in-flight executions or their CloudWatch logs (those age out per their retention policy). New triggers against the deleted UUID will fail.
|
|
7
|
+
|
|
8
|
+
Canonical docs: **https://docs.zibby.app/workflows/lifecycle**
|
|
9
|
+
|
|
10
|
+
## Steps
|
|
11
|
+
|
|
12
|
+
1. **Get the UUID.** If user gave a name, look it up:
|
|
13
|
+
```
|
|
14
|
+
Bash(zibby workflow list)
|
|
15
|
+
```
|
|
16
|
+
Find the matching `name` and grab its `uuid`.
|
|
17
|
+
|
|
18
|
+
2. **Confirm with the user.** Always confirm before deleting — show them the workflow's name, project, last-triggered timestamp. Don't proceed silently.
|
|
19
|
+
```
|
|
20
|
+
"Delete workflow 'pr-summarizer' (uuid abc-123, last run 2 days ago)? This cannot be undone."
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
3. **Run the delete:**
|
|
24
|
+
```
|
|
25
|
+
Bash(zibby workflow delete <uuid>)
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
4. **Clean up local files** if the user wants. The local `.zibby/workflows/<name>/` folder isn't auto-deleted — ask before removing:
|
|
29
|
+
```
|
|
30
|
+
rm -rf .zibby/workflows/<name>
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## When NOT to delete
|
|
34
|
+
|
|
35
|
+
- If the user might want to re-deploy later — keep the local folder, just stop triggering it.
|
|
36
|
+
- If there are running executions — the deploy is gone but those will keep running until they exit. Tell the user to wait or `Ctrl+C`-equivalent (kill the ECS task) if urgent.
|
|
37
|
+
- For a "hide from list" feeling without losing history — there's no soft-delete; it's all-or-nothing.
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
<!-- zibby-template-version: 4 -->
|
|
2
|
+
# /zibby-deploy — deploy a Zibby workflow to the cloud
|
|
3
|
+
|
|
4
|
+
You are helping the user deploy a workflow they've been building locally.
|
|
5
|
+
|
|
6
|
+
## What `zibby workflow deploy` does
|
|
7
|
+
|
|
8
|
+
1. Bundles the workflow's source (graph.mjs + nodes/ + package.json) into a tarball
|
|
9
|
+
2. Uploads it to S3 via a presigned URL
|
|
10
|
+
3. Triggers AWS CodeBuild to install deps + bake the bundle
|
|
11
|
+
4. Updates DynamoDB so future triggers run the new bundle
|
|
12
|
+
|
|
13
|
+
A successful deploy is required before `zibby workflow trigger <uuid>` works against the cloud.
|
|
14
|
+
|
|
15
|
+
Canonical docs: **https://docs.zibby.app/workflows/deploying**
|
|
16
|
+
|
|
17
|
+
## Steps for this command
|
|
18
|
+
|
|
19
|
+
1. **Identify the workflow.** If the user passes a name, use it. Otherwise list everything under `paths.workflows` (from `.zibby.config.mjs`) and ask.
|
|
20
|
+
|
|
21
|
+
2. **Pre-flight checks.** Read the workflow folder and confirm:
|
|
22
|
+
- `graph.mjs` exists and exports a graph
|
|
23
|
+
- `nodes/` has at least one node
|
|
24
|
+
- `workflow.json` is valid (must have `name`, `entryClass`, `triggers`)
|
|
25
|
+
- `package.json` declares all imports used in nodes (run a quick grep to spot missing deps)
|
|
26
|
+
|
|
27
|
+
3. **Run the deploy:**
|
|
28
|
+
```
|
|
29
|
+
zibby workflow deploy <workflow-name>
|
|
30
|
+
```
|
|
31
|
+
This is interactive if `--project` isn't passed. The user picks a project, the CLI handles auth via the saved session token.
|
|
32
|
+
|
|
33
|
+
4. **Watch the build.** The CLI streams CodeBuild output. If it succeeds, it prints the workflow's UUID. If it fails, the build logs show why — usually a missing dep in `package.json` or a syntax error in a node.
|
|
34
|
+
|
|
35
|
+
5. **Verify post-deploy:**
|
|
36
|
+
```
|
|
37
|
+
zibby workflow trigger <uuid> --input '{}'
|
|
38
|
+
zibby workflow logs <uuid> -t
|
|
39
|
+
```
|
|
40
|
+
Tail logs until the workflow reaches `completed` (or `failed` — diagnose from logs).
|
|
41
|
+
|
|
42
|
+
## Common failure modes
|
|
43
|
+
|
|
44
|
+
- **Build fails with module-not-found** → node imports a package not in `package.json`. Add it and redeploy.
|
|
45
|
+
- **Build succeeds but trigger fails immediately** → `entryClass` in `workflow.json` doesn't match a class exported by `graph.mjs`.
|
|
46
|
+
- **Workflow runs but a node fails** → tail the live logs and read the error. Most are in the agent's prompt/output handling.
|
|
47
|
+
|
|
48
|
+
## Optional flags worth knowing
|
|
49
|
+
|
|
50
|
+
`zibby workflow deploy` accepts:
|
|
51
|
+
- `--project <id>` — skip the interactive project picker
|
|
52
|
+
- `--api-key <key>` — use a PAT instead of the session token (for CI)
|
|
53
|
+
- `--env <path>` — sync a `.env` file into per-workflow env vars after deploy. Repeatable; later files override.
|
|
54
|
+
- `--verbose` — print raw CodeBuild output during the build (helpful for debugging build failures)
|
|
55
|
+
|
|
56
|
+
### Seeding per-workflow env on first deploy
|
|
57
|
+
|
|
58
|
+
If the workflow needs its own `ANTHROPIC_API_KEY`, `DATABASE_URL`, etc., put them in a `.env` and pass `--env`:
|
|
59
|
+
|
|
60
|
+
```
|
|
61
|
+
zibby workflow deploy <name> --env .env
|
|
62
|
+
zibby workflow deploy <name> --env .env --env .env.prod # later files win
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
After deploy, manage them surgically with `zibby workflow env set/unset/list/push <uuid>`. See `/zibby-list` to recover the UUID; full guide at https://docs.zibby.app/cloud/env-vars.
|
|
66
|
+
|
|
67
|
+
## Static outbound IP (dedicated egress)
|
|
68
|
+
|
|
69
|
+
If the user's workflow needs to call APIs that require IP allowlisting (corporate GitHub, GitLab Enterprise, paranoid SaaS firewalls), the workflow needs the **dedicated egress IP** addon. The flag lives on the legacy alias `zibby deploy` (NOT `zibby workflow deploy`):
|
|
70
|
+
|
|
71
|
+
| Flag | What it does |
|
|
72
|
+
|------|-------------|
|
|
73
|
+
| `zibby deploy <name> --dedicated-ip status` | Show current addon state for the account (active / inactive / billing) |
|
|
74
|
+
| `zibby deploy <name> --dedicated-ip enable` | Enable the addon on the account (Pro subscription required, ~$50/mo). One-time per account. |
|
|
75
|
+
| `zibby deploy <name> --dedicated-ip use` | Mark THIS workflow as using the static egress IP (per-workflow opt-in, after `enable`) |
|
|
76
|
+
| `zibby deploy <name> --dedicated-ip unuse` | Stop routing this workflow through the static IP |
|
|
77
|
+
| `zibby deploy <name> --dedicated-ip disable` | Disable the addon for the whole account |
|
|
78
|
+
|
|
79
|
+
Typical first-time flow when the user says "I need a static outbound IP":
|
|
80
|
+
1. `zibby deploy <name> --dedicated-ip status` — check whether they have it
|
|
81
|
+
2. If inactive → `zibby deploy <name> --dedicated-ip enable` — enables the account-wide addon (interactive billing prompt; prerequisite Pro subscription)
|
|
82
|
+
3. `zibby deploy <name> --dedicated-ip use` — opts this specific workflow in
|
|
83
|
+
4. Regular `zibby workflow deploy <name>` from now on uses the static IP
|
|
84
|
+
|
|
85
|
+
After `--dedicated-ip use`, every node in this workflow gets its outbound HTTP routed through the egress proxy, and `process.env.HTTP_PROXY` / `HTTPS_PROXY` are set in the sandbox automatically. Their static IPs are visible to customers via `https://docs.zibby.app/workflows/egress`.
|
|
86
|
+
|
|
87
|
+
**Don't** run `--dedicated-ip enable` without confirming with the user — it has billing impact ($50/mo addon). Always confirm. See `/zibby-static-ip` for the deeper walkthrough.
|
|
@@ -0,0 +1,30 @@
|
|
|
1
|
+
<!-- zibby-template-version: 4 -->
|
|
2
|
+
# /zibby-list — list workflows (local + cloud) with their UUIDs and statuses
|
|
3
|
+
|
|
4
|
+
You are helping the user see what workflows exist — locally scaffolded and remotely deployed.
|
|
5
|
+
|
|
6
|
+
Canonical docs: **https://docs.zibby.app/cli-reference#workflow-list**
|
|
7
|
+
|
|
8
|
+
## Steps
|
|
9
|
+
|
|
10
|
+
1. **Run the list command:**
|
|
11
|
+
```
|
|
12
|
+
Bash(zibby workflow list)
|
|
13
|
+
```
|
|
14
|
+
This shows both local (in `.zibby/workflows/`) and remote (deployed to Zibby Cloud) workflows. Each row has: name, UUID, project, last triggered.
|
|
15
|
+
|
|
16
|
+
2. **Filter on demand.** If the user wants only local or only remote:
|
|
17
|
+
```
|
|
18
|
+
zibby workflow list --local-only
|
|
19
|
+
zibby workflow list --remote-only --project <id>
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
## When you'd use this
|
|
23
|
+
|
|
24
|
+
- User asks "what workflows do I have?" → run it, show the result.
|
|
25
|
+
- You need a UUID to pass into `/zibby-trigger`, `/zibby-tail`, `/zibby-delete` and the user only knows the name → run it, look up the UUID.
|
|
26
|
+
- After a deploy to confirm the bundle landed.
|
|
27
|
+
|
|
28
|
+
## Output expectations
|
|
29
|
+
|
|
30
|
+
The output is human-readable text (not JSON). If you need to extract a specific UUID programmatically, parse the line for the workflow name. If the user has many workflows, ask which one they want — don't grep blind.
|
|
@@ -0,0 +1,39 @@
|
|
|
1
|
+
<!-- zibby-template-version: 4 -->
|
|
2
|
+
# /zibby-memory-cost — show real LLM token spend across past test runs
|
|
3
|
+
|
|
4
|
+
You are helping the user see how many input/output/cache tokens their tests have actually burned, broken down per spec and per domain. This is real measured spend (read off run records in `.zibby/memory/.dolt/`), not an estimate.
|
|
5
|
+
|
|
6
|
+
Canonical docs: **https://docs.zibby.app/tests/memory**
|
|
7
|
+
|
|
8
|
+
## What the command shows
|
|
9
|
+
|
|
10
|
+
```
|
|
11
|
+
Bash(zibby memory cost)
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Per-spec and per-domain rollup of:
|
|
15
|
+
- Input tokens
|
|
16
|
+
- Output tokens
|
|
17
|
+
- Cache hit / cache write tokens (when the agent supports prompt caching)
|
|
18
|
+
- Estimated $ cost (uses current public model pricing)
|
|
19
|
+
- Recent-runs trend, so you can see if a spec is getting cheaper or more expensive over time
|
|
20
|
+
|
|
21
|
+
The numbers are pulled from `test_runs` rows in the Dolt DB — every test run records the agent's actual usage on completion.
|
|
22
|
+
|
|
23
|
+
## When to invoke
|
|
24
|
+
|
|
25
|
+
- User asks "how much are my tests costing me?" or "which spec is the expensive one?"
|
|
26
|
+
- After enabling prompt caching to confirm cache hits are landing
|
|
27
|
+
- When deciding whether to swap to a cheaper agent on hot specs (`--agent` per run)
|
|
28
|
+
- When triaging a regression in test runtime — high token counts often correlate with the agent retrying
|
|
29
|
+
|
|
30
|
+
## Caveats
|
|
31
|
+
|
|
32
|
+
- **Only counts what's in local memory.** Runs on machines that haven't pulled from the team remote won't appear. Run `/zibby-memory-pull` first if you want the full team picture.
|
|
33
|
+
- **Pricing is informational.** Public API pricing changes; treat the $ column as a guide, not a bill. The token counts themselves are exact.
|
|
34
|
+
- **Empty if you've never run a test with memory enabled.** Confirm the runs are in there with `/zibby-memory-stats` first.
|
|
35
|
+
|
|
36
|
+
## Related
|
|
37
|
+
|
|
38
|
+
- `/zibby-memory-stats` — what's in the DB at all
|
|
39
|
+
- `/zibby-memory-pull` — refresh from team remote before reading cost
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
<!-- zibby-template-version: 4 -->
|
|
2
|
+
# /zibby-memory-pull — pull the team's latest test memory from the configured remote
|
|
3
|
+
|
|
4
|
+
You are helping the user fetch the team's latest learnings (selectors, page model, insights, run history) from the project's configured memory remote into local `.zibby/memory/.dolt/`.
|
|
5
|
+
|
|
6
|
+
Canonical docs: **https://docs.zibby.app/tests/memory**
|
|
7
|
+
|
|
8
|
+
## When this is needed (vs. just runs automatically)
|
|
9
|
+
|
|
10
|
+
`zibby test` auto-pulls before every run when a remote is configured, and auto-pushes after every passing run. So most of the time the user doesn't need to invoke pull manually. Manual pull is for:
|
|
11
|
+
|
|
12
|
+
- Fresh clone of the repo — first sync to seed `.zibby/memory/.dolt/` from the remote
|
|
13
|
+
- After a teammate landed a big batch of new learnings and the user wants them before running anything
|
|
14
|
+
- Inspecting team memory (`/zibby-memory-stats`, `/zibby-memory-cost`) without running a test
|
|
15
|
+
- Reconciling after a manual conflict in the Dolt DB
|
|
16
|
+
|
|
17
|
+
## How to run
|
|
18
|
+
|
|
19
|
+
```
|
|
20
|
+
Bash(zibby memory pull)
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
The CLI fetches from whatever remote `zibby memory remote info` reports — BYO S3/GCS/DoltHub URL or the Zibby-hosted backend. No flags.
|
|
24
|
+
|
|
25
|
+
## Pre-flight: is a remote configured?
|
|
26
|
+
|
|
27
|
+
Before suggesting `pull`, check:
|
|
28
|
+
|
|
29
|
+
```
|
|
30
|
+
Bash(zibby memory remote info)
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
- **No remote configured** → pull errors out. Tell the user to either:
|
|
34
|
+
- Add their own: `zibby memory remote add aws://my-bucket/team/proj/main`
|
|
35
|
+
- Use the hosted one: `zibby memory remote use --hosted` (requires `zibby login`)
|
|
36
|
+
- See `/zibby-memory-remote-use-hosted` for the hosted path.
|
|
37
|
+
- **Hosted remote, signed out** → `zibby login` first.
|
|
38
|
+
|
|
39
|
+
## After pulling
|
|
40
|
+
|
|
41
|
+
Confirm the pull landed with `/zibby-memory-stats` — row counts should jump (selectors, runs, insights) compared to before.
|
|
42
|
+
|
|
43
|
+
## Related
|
|
44
|
+
|
|
45
|
+
- `zibby memory push` — manual push (auto on passing test, but sometimes you want to share now)
|
|
46
|
+
- `/zibby-memory-stats` — verify what came in
|
|
47
|
+
- `/zibby-memory-remote-use-hosted` — switch to the Zibby-managed S3 backend
|
|
@@ -0,0 +1,61 @@
|
|
|
1
|
+
<!-- zibby-template-version: 4 -->
|
|
2
|
+
# /zibby-memory-remote-use-hosted — switch this project's memory remote to Zibby-managed S3
|
|
3
|
+
|
|
4
|
+
You are helping the user point their `.zibby/memory/.dolt/` at Zibby's hosted S3 backend, instead of running their own S3 bucket / GCS / DoltHub repo.
|
|
5
|
+
|
|
6
|
+
Canonical docs: **https://docs.zibby.app/tests/memory**
|
|
7
|
+
|
|
8
|
+
## What this does
|
|
9
|
+
|
|
10
|
+
```
|
|
11
|
+
Bash(zibby memory remote use --hosted)
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Allocates a tenant-scoped prefix on Zibby-managed S3 for this project (keyed on the projectId in `.zibby.config.mjs`) and writes that as the local Dolt remote. After this, every `zibby test` run auto-pulls before and auto-pushes after — same as a BYO remote, just without the bucket plumbing.
|
|
15
|
+
|
|
16
|
+
## Prerequisite: signed in
|
|
17
|
+
|
|
18
|
+
Hosted remote is **signed-in users only**. Verify:
|
|
19
|
+
|
|
20
|
+
```
|
|
21
|
+
Bash(zibby status)
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
If not signed in, run `zibby login` first. The CLI uses the saved session to derive the tenant prefix; it won't fall back to anonymous.
|
|
25
|
+
|
|
26
|
+
## When to use hosted vs BYO
|
|
27
|
+
|
|
28
|
+
| | Hosted (`--hosted`) | BYO (`zibby memory remote add aws://...`) |
|
|
29
|
+
|---|---|---|
|
|
30
|
+
| Setup time | Zero — `--hosted` and you're done | Provision an S3 bucket, IAM, optional KMS |
|
|
31
|
+
| Who can read | Everyone with project access on Zibby | Whoever you grant in IAM |
|
|
32
|
+
| Where data lives | Zibby-managed AWS account | Your account |
|
|
33
|
+
| Compliance / data-residency | Limited regions | Wherever you want |
|
|
34
|
+
| Cost | Included in plan | Your S3 bill |
|
|
35
|
+
|
|
36
|
+
If the user has any data-residency requirement or a regulated workload, prefer BYO. Otherwise hosted is the path of least resistance.
|
|
37
|
+
|
|
38
|
+
## Switching from BYO to hosted
|
|
39
|
+
|
|
40
|
+
`zibby memory remote use --hosted` overwrites the existing remote. If they had a BYO remote and might want to keep its history, run `zibby memory push` against the old remote first so nothing's lost — then switch.
|
|
41
|
+
|
|
42
|
+
## After switching
|
|
43
|
+
|
|
44
|
+
1. `zibby memory pull` — seed `.zibby/memory/.dolt/` from the hosted prefix (no-op the very first time per project)
|
|
45
|
+
2. `/zibby-memory-stats` — confirm
|
|
46
|
+
3. Commit `.zibby.config.mjs` if you set `memorySync.remote: 'hosted'` so teammates auto-wire on next `zibby init`
|
|
47
|
+
|
|
48
|
+
## Reverting
|
|
49
|
+
|
|
50
|
+
```
|
|
51
|
+
Bash(zibby memory remote remove)
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Drops the remote — memory becomes local-only again. The data on Zibby's S3 isn't deleted (it's still tenant-scoped), but nothing pushes or pulls until a new remote is configured.
|
|
55
|
+
|
|
56
|
+
## Related
|
|
57
|
+
|
|
58
|
+
- `/zibby-memory-pull` — manual pull (auto on test start)
|
|
59
|
+
- `/zibby-memory-stats` — verify what's in the local DB
|
|
60
|
+
- `zibby memory remote info` — show current remote config
|
|
61
|
+
- `zibby memory remote add <url>` — BYO remote (S3/GCS/DoltHub/file:///)
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
<!-- zibby-template-version: 4 -->
|
|
2
|
+
# /zibby-memory-stats — inspect the local test memory database
|
|
3
|
+
|
|
4
|
+
You are helping the user see what's in their `.zibby/memory/.dolt/` test-memory DB — row counts per table, last commit, and per-spec breakdown.
|
|
5
|
+
|
|
6
|
+
Canonical docs: **https://docs.zibby.app/tests/memory**
|
|
7
|
+
|
|
8
|
+
## What the command shows
|
|
9
|
+
|
|
10
|
+
```
|
|
11
|
+
Bash(zibby memory stats)
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Prints a summary of the local Dolt database:
|
|
15
|
+
- **Test runs** — total runs recorded, pass/fail split, last run timestamp
|
|
16
|
+
- **Selectors** — total cached selectors, top pages by selector count
|
|
17
|
+
- **Page model** — pages mapped, total elements
|
|
18
|
+
- **Navigation** — known transitions
|
|
19
|
+
- **Insights** — count by category (`selector_tip | timing | navigation | workaround | flaky | general`)
|
|
20
|
+
- **Dolt status** — current branch, last commit hash, uncommitted changes
|
|
21
|
+
|
|
22
|
+
## When to invoke
|
|
23
|
+
|
|
24
|
+
- User asks "what does Zibby know about my app?" or "show me what's in test memory"
|
|
25
|
+
- After running a few tests, to confirm the agent is actually persisting learnings
|
|
26
|
+
- Before a `zibby memory compact` to see how much there is to prune
|
|
27
|
+
- Before a `zibby memory remote add` to know what's about to ship to the team
|
|
28
|
+
|
|
29
|
+
## Empty database?
|
|
30
|
+
|
|
31
|
+
If the user just ran `zibby memory init` (or it auto-initialized on first `zibby test`), most counts will be 0. That's fine — selectors and page model populate after the first successful run. Suggest running a test first.
|
|
32
|
+
|
|
33
|
+
## Related commands
|
|
34
|
+
|
|
35
|
+
- `/zibby-memory-cost` — real LLM token spend per spec / per domain
|
|
36
|
+
- `/zibby-memory-pull` — pull team's latest learnings from the configured remote
|
|
37
|
+
- `zibby memory compact` — prune old runs (`--max-runs N`, `--max-age <days>`)
|
|
38
|
+
- `zibby memory reset -f` — wipe the DB (destructive — confirm first)
|
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
<!-- zibby-template-version: 4 -->
|
|
2
|
+
# /zibby-static-ip — set up dedicated outbound static IP for a workflow
|
|
3
|
+
|
|
4
|
+
You are helping the user route a workflow's outbound traffic through a static IP address — needed when the workflow calls APIs that require IP allowlisting (corporate GitLab/GitHub Enterprise, internal SaaS, firewalls).
|
|
5
|
+
|
|
6
|
+
Canonical docs: **https://docs.zibby.app/workflows/egress**
|
|
7
|
+
|
|
8
|
+
> Note: the `--dedicated-ip` flag lives on the legacy alias `zibby deploy <name>`, NOT on `zibby workflow deploy <name>`. The two share a handler, but only `zibby deploy` exposes this flag in `--help`.
|
|
9
|
+
|
|
10
|
+
## What "static IP" means here
|
|
11
|
+
|
|
12
|
+
By default, workflow tasks run on Fargate and their outbound traffic exits via AWS-managed IPs that rotate. With the **dedicated egress** addon enabled, the workflow's outbound traffic is routed through a Zibby-managed proxy whose IP is pinned and customer-allowlistable.
|
|
13
|
+
|
|
14
|
+
Two pieces:
|
|
15
|
+
1. **Account-level addon** — `~$50/mo`, requires Pro subscription. One-time toggle per account.
|
|
16
|
+
2. **Per-workflow opt-in** — once the addon is on, each workflow opts in individually (some workflows might not need it, no point routing them).
|
|
17
|
+
|
|
18
|
+
## Steps
|
|
19
|
+
|
|
20
|
+
1. **Confirm the user understands the cost.** Before any `--dedicated-ip enable`, explicitly say:
|
|
21
|
+
```
|
|
22
|
+
"This will enable a $50/mo dedicated-egress addon on your account. Confirm?"
|
|
23
|
+
```
|
|
24
|
+
If they don't have a Pro subscription, the enable call returns 402 — direct them to https://zibby.dev/billing first.
|
|
25
|
+
|
|
26
|
+
2. **Check current state:**
|
|
27
|
+
```
|
|
28
|
+
Bash(zibby deploy <name> --dedicated-ip status)
|
|
29
|
+
```
|
|
30
|
+
Output tells you: addon active or inactive, this workflow currently using it or not, and the assigned IPs to publish to customers.
|
|
31
|
+
|
|
32
|
+
3. **If addon is inactive — enable it** (only after explicit user confirmation):
|
|
33
|
+
```
|
|
34
|
+
Bash(zibby deploy <name> --dedicated-ip enable)
|
|
35
|
+
```
|
|
36
|
+
This is one-time per account. After this, the addon is active for ALL workflows in the account that opt in.
|
|
37
|
+
|
|
38
|
+
4. **Opt this workflow in:**
|
|
39
|
+
```
|
|
40
|
+
Bash(zibby deploy <name> --dedicated-ip use)
|
|
41
|
+
```
|
|
42
|
+
From now on, every deploy of this workflow + every triggered execution routes outbound through the static IP.
|
|
43
|
+
|
|
44
|
+
5. **Re-deploy the workflow** so the runtime picks up the change:
|
|
45
|
+
```
|
|
46
|
+
Bash(zibby workflow deploy <name>)
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
6. **Verify in a node** by adding a quick log:
|
|
50
|
+
```js
|
|
51
|
+
ctx.log(`HTTP_PROXY=${process.env.HTTP_PROXY}`);
|
|
52
|
+
```
|
|
53
|
+
The proxy URL should be set on the node's process. The IP that external services see is the dedicated one.
|
|
54
|
+
|
|
55
|
+
## Reverting
|
|
56
|
+
|
|
57
|
+
- `Bash(zibby deploy <name> --dedicated-ip unuse)` — stop routing this workflow's egress through the static IP. Other opted-in workflows are unaffected.
|
|
58
|
+
- `Bash(zibby deploy <name> --dedicated-ip disable)` — disable the addon entirely (also stops billing).
|
|
59
|
+
|
|
60
|
+
## Tell the user the IPs
|
|
61
|
+
|
|
62
|
+
After `enable`, the assigned outbound IPs are visible in `--dedicated-ip status` output. Surface them clearly so the user can paste into their customer's firewall allowlist:
|
|
63
|
+
```
|
|
64
|
+
Outbound static IPs (allowlist these in the customer's firewall):
|
|
65
|
+
54.252.121.111
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
## Don't confuse with the inbound static IPs
|
|
69
|
+
|
|
70
|
+
This is OUTBOUND (workflow → external API). There's also an INBOUND static-IP set for `https://logs-stream.zibby.app` (the SSE log endpoint customers tail with `zibby workflow logs -t`). That one is unrelated to this addon — see `https://docs.zibby.app/security/ip-allowlist`.
|
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
<!-- zibby-template-version: 4 -->
|
|
2
|
+
# /zibby-tail — stream live logs from a Zibby workflow
|
|
3
|
+
|
|
4
|
+
You are helping the user tail logs from a workflow execution.
|
|
5
|
+
|
|
6
|
+
`zibby workflow logs <jobId> -t` is the live-streaming variant (like `heroku logs --tail` or `docker compose logs -f`). Without `-t`, you get a one-shot fetch.
|
|
7
|
+
|
|
8
|
+
Canonical docs: **https://docs.zibby.app/workflows/logs**
|
|
9
|
+
|
|
10
|
+
## What `<jobId>` accepts
|
|
11
|
+
|
|
12
|
+
- A **workflow UUID** (`2b1ea07f-...`) → tails ALL currently-active executions of that workflow, plus any new ones triggered while the tail is open. Lines are interleaved by arrival time, prefixed with `(taskId)` so concurrent runs are distinguishable.
|
|
13
|
+
- An **execution ID** (`abc-...`) → tails just that single execution. Closes when the execution drains.
|
|
14
|
+
|
|
15
|
+
## Example flow
|
|
16
|
+
|
|
17
|
+
```
|
|
18
|
+
$ zibby workflow trigger 2b1ea07f-3ede-4bfd-a51d-431f0bab008e
|
|
19
|
+
Job ID: 569ef1ee-15c8-4ee7-af30-933c8bec7ea7
|
|
20
|
+
|
|
21
|
+
$ zibby workflow logs 2b1ea07f-3ede-4bfd-a51d-431f0bab008e -t
|
|
22
|
+
Streaming logs for workflow 2b1ea07f-...
|
|
23
|
+
Press Ctrl+C to stop.
|
|
24
|
+
|
|
25
|
+
┌─ Execution: 569ef1ee...7ea7 (task: 9e61c690)
|
|
26
|
+
└─ Streaming logs...
|
|
27
|
+
|
|
28
|
+
2026-05-04 06:16:27.621 (9e61c690) zibby v0.1.91
|
|
29
|
+
2026-05-04 06:16:27.997 (9e61c690) Workflow: hello-world
|
|
30
|
+
2026-05-04 06:16:33.055 (9e61c690) │ Prompt sent to LLM:
|
|
31
|
+
...
|
|
32
|
+
2026-05-04 06:16:56.389 (9e61c690) ✓ Workflow completed
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
## Tail behavior worth knowing
|
|
36
|
+
|
|
37
|
+
- **Auto-switch:** when the current execution drains and a new trigger lands, the tail seamlessly picks it up — no need to re-run the command.
|
|
38
|
+
- **Multi-attach:** if you trigger two workflows in parallel, the tail attaches to both. Lines from each are interleaved by timestamp.
|
|
39
|
+
- **Reconnect:** transient network blips reconnect silently. Long quiet windows (3+ min) are kept alive via 5s keepalives.
|
|
40
|
+
|
|
41
|
+
## Steps for this command
|
|
42
|
+
|
|
43
|
+
1. Identify the jobId. If user gives a workflow name (not UUID), look it up via `zibby workflow list` and use the UUID.
|
|
44
|
+
2. **Run in the background** — `-t` is a long-running stream. If you call `Bash` without `run_in_background: true` it'll block the chat for minutes:
|
|
45
|
+
```
|
|
46
|
+
Bash({ command: "zibby workflow logs <jobId> -t", run_in_background: true })
|
|
47
|
+
```
|
|
48
|
+
Then use `BashOutput` (or whatever your background-output tool is) to read new lines as they arrive.
|
|
49
|
+
3. If the user wants a **one-shot snapshot** (no follow), drop the `-t` flag and call Bash normally:
|
|
50
|
+
```
|
|
51
|
+
Bash({ command: "zibby workflow logs <jobId>" })
|
|
52
|
+
```
|
|
53
|
+
4. To stop the live tail: kill the background bash task. The CLI exits cleanly with `Stopped streaming.`
|