@zibby/skills 0.2.23 → 0.2.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/browser.d.ts +0 -17
- package/dist/browser.js +3 -3
- package/dist/code-scan.d.ts +1 -1
- package/dist/index.d.ts +0 -1
- package/dist/index.js +97 -97
- package/dist/jira.d.ts +75 -9
- package/dist/jira.js +4 -4
- package/dist/package.json +1 -1
- package/dist/report.d.ts +51 -276
- package/dist/trackers/index.js +12 -12
- package/dist/trackers/jira-adapter.js +11 -11
- package/package.json +1 -1
- package/docs/analysis.md +0 -109
- package/docs/apps/agent-ops.md +0 -130
- package/docs/apps/auth.md +0 -158
- package/docs/apps/deploy.md +0 -207
- package/docs/apps/goal-mode.md +0 -175
- package/docs/apps/index.md +0 -149
- package/docs/apps/managing.md +0 -121
- package/docs/cli-reference.md +0 -494
- package/docs/cloning-repositories.md +0 -285
- package/docs/cloud/bundles.md +0 -92
- package/docs/cloud/dedicated-egress.md +0 -140
- package/docs/cloud/editing-prompts.md +0 -79
- package/docs/cloud/env-vars.md +0 -144
- package/docs/cloud/limits.md +0 -81
- package/docs/cloud/logs.md +0 -104
- package/docs/cloud/triggering.md +0 -128
- package/docs/concepts/agents.md +0 -112
- package/docs/concepts/designing-agents.md +0 -247
- package/docs/concepts/graph.md +0 -83
- package/docs/concepts/sessions.md +0 -70
- package/docs/concepts/skills.md +0 -86
- package/docs/concepts/state.md +0 -106
- package/docs/concepts/sub-graphs.md +0 -227
- package/docs/custom-workflows.md +0 -358
- package/docs/get-started/deploy.md +0 -75
- package/docs/get-started/install.md +0 -60
- package/docs/get-started/run-locally.md +0 -94
- package/docs/get-started/trigger-and-logs.md +0 -90
- package/docs/get-started/use-from-agents.md +0 -153
- package/docs/get-started/your-first-workflow.md +0 -69
- package/docs/getting-started.md +0 -108
- package/docs/installation.md +0 -127
- package/docs/integrations/github.md +0 -73
- package/docs/integrations/gitlab.md +0 -43
- package/docs/integrations/jira.md +0 -71
- package/docs/integrations/lark.md +0 -41
- package/docs/integrations/linear.md +0 -43
- package/docs/integrations/notion.md +0 -33
- package/docs/integrations/plane.md +0 -46
- package/docs/integrations/sentry.md +0 -42
- package/docs/integrations/slack.md +0 -33
- package/docs/intro.md +0 -86
- package/docs/legacy/test-automation.md +0 -111
- package/docs/packages/agent-workflow.md +0 -88
- package/docs/packages/cli.md +0 -73
- package/docs/packages/core.md +0 -72
- package/docs/packages/mcp-browser.md +0 -110
- package/docs/packages/mcp-cli.md +0 -176
- package/docs/packages/memory.md +0 -223
- package/docs/packages/skills.md +0 -218
- package/docs/packages/ui-memory.md +0 -245
- package/docs/recipes/bug-autofix.md +0 -85
- package/docs/recipes/github-ai-scout.md +0 -61
- package/docs/recipes/index.md +0 -68
- package/docs/recipes/pipeline-supervisor.md +0 -57
- package/docs/recipes/sentry-triage.md +0 -93
- package/docs/recipes/test.md +0 -190
- package/docs/reviewing-results.md +0 -114
- package/docs/running-tests.md +0 -134
- package/docs/self-host/backup-restore.md +0 -53
- package/docs/self-host/custom-sidecars.md +0 -136
- package/docs/self-host/index.md +0 -75
- package/docs/self-host/storage.md +0 -50
- package/docs/self-host/troubleshooting.md +0 -48
- package/docs/self-host/upgrade.md +0 -81
- package/docs/skills/browser.md +0 -97
- package/docs/skills/chat-memory.md +0 -122
- package/docs/skills/core-tools.md +0 -80
- package/docs/skills/function-skill.md +0 -93
- package/docs/skills/github.md +0 -91
- package/docs/skills/index.md +0 -46
- package/docs/skills/jira.md +0 -99
- package/docs/skills/lark.md +0 -85
- package/docs/skills/memory.md +0 -92
- package/docs/skills/sentry.md +0 -80
- package/docs/skills/slack.md +0 -89
- package/docs/tests/memory.md +0 -131
- package/docs/triggering-workflows.md +0 -552
- package/docs/workflow-artifact-layout-evaluation.md +0 -119
- package/docs/workflow.md +0 -558
package/docs/running-tests.md
DELETED
|
@@ -1,134 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 5
|
|
3
|
-
title: Running Tests
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Running Tests
|
|
7
|
-
|
|
8
|
-
Zibby runs tests locally using an AI agent that drives a real browser. It records video, captures actions, and generates reusable Playwright scripts.
|
|
9
|
-
|
|
10
|
-
## Basic Usage
|
|
11
|
-
|
|
12
|
-
```bash
|
|
13
|
-
zibby test test-specs/login.txt --agent cursor
|
|
14
|
-
```
|
|
15
|
-
|
|
16
|
-
## From a Test Spec File
|
|
17
|
-
|
|
18
|
-
Create a plain-text file with test steps:
|
|
19
|
-
|
|
20
|
-
```text title="test-specs/login.txt"
|
|
21
|
-
1. Navigate to the login page
|
|
22
|
-
2. Enter email: test@example.com
|
|
23
|
-
3. Enter password: TestPass123
|
|
24
|
-
4. Click the Sign In button
|
|
25
|
-
5. Verify the dashboard page loads
|
|
26
|
-
6. Verify the user's name appears in the header
|
|
27
|
-
```
|
|
28
|
-
|
|
29
|
-
Run it:
|
|
30
|
-
|
|
31
|
-
```bash
|
|
32
|
-
zibby test test-specs/login.txt
|
|
33
|
-
```
|
|
34
|
-
|
|
35
|
-
## From Cloud Test Cases
|
|
36
|
-
|
|
37
|
-
After running analysis on a Jira ticket in the dashboard:
|
|
38
|
-
|
|
39
|
-
```bash
|
|
40
|
-
zibby test --sources TC001,TC002 --execution abc-123-def --sync
|
|
41
|
-
```
|
|
42
|
-
|
|
43
|
-
## What Happens During a Run
|
|
44
|
-
|
|
45
|
-
The default workflow executes three nodes:
|
|
46
|
-
|
|
47
|
-
1. **Preflight** — AI extracts a title and assertions from the test spec
|
|
48
|
-
2. **Execute Live** — AI drives a Playwright browser, following the test steps
|
|
49
|
-
3. **Generate Script** — AI produces a reusable `.spec.js` from recorded actions
|
|
50
|
-
|
|
51
|
-
```
|
|
52
|
-
Test Spec → Preflight → Execute Live → Generate Script → Done
|
|
53
|
-
```
|
|
54
|
-
|
|
55
|
-
## Choosing an AI Agent
|
|
56
|
-
|
|
57
|
-
```bash
|
|
58
|
-
# Cursor Agent (uses Cursor IDE or CURSOR_API_KEY)
|
|
59
|
-
zibby test test-specs/login.txt --agent cursor
|
|
60
|
-
|
|
61
|
-
# Claude (uses ANTHROPIC_API_KEY)
|
|
62
|
-
zibby test test-specs/login.txt --agent claude
|
|
63
|
-
|
|
64
|
-
# Codex (uses OPENAI_API_KEY + codex CLI)
|
|
65
|
-
zibby test test-specs/login.txt --agent codex
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
## Headless Mode
|
|
69
|
-
|
|
70
|
-
Run without a visible browser (for CI/CD):
|
|
71
|
-
|
|
72
|
-
```bash
|
|
73
|
-
zibby test test-specs/login.txt --headless
|
|
74
|
-
```
|
|
75
|
-
|
|
76
|
-
## Cloud Sync
|
|
77
|
-
|
|
78
|
-
Upload results to the [Zibby dashboard](https://zibby.app):
|
|
79
|
-
|
|
80
|
-
```bash
|
|
81
|
-
# Upload to dashboard
|
|
82
|
-
zibby test test-specs/login.txt --sync
|
|
83
|
-
|
|
84
|
-
# Organize into a collection
|
|
85
|
-
zibby test test-specs/login.txt --collection "Auth Tests" --sync
|
|
86
|
-
|
|
87
|
-
# With subfolder
|
|
88
|
-
zibby test test-specs/login.txt --collection "Auth Tests" --folder "Login" --sync
|
|
89
|
-
|
|
90
|
-
# Local only (no upload)
|
|
91
|
-
zibby test test-specs/login.txt --no-sync
|
|
92
|
-
```
|
|
93
|
-
|
|
94
|
-
The `cloudSync` setting in `.zibby.config.js` sets the default. CLI flags override it.
|
|
95
|
-
|
|
96
|
-
## Running a Single Node
|
|
97
|
-
|
|
98
|
-
Re-run just one step of the workflow (useful for debugging):
|
|
99
|
-
|
|
100
|
-
```bash
|
|
101
|
-
# Re-execute the browser test (reusing previous session)
|
|
102
|
-
zibby test test-specs/login.txt --node execute_live --session last
|
|
103
|
-
|
|
104
|
-
# Only regenerate the Playwright script
|
|
105
|
-
zibby test test-specs/login.txt --node generate_script --session last
|
|
106
|
-
```
|
|
107
|
-
|
|
108
|
-
## Custom Workflows
|
|
109
|
-
|
|
110
|
-
If you've created a custom workflow in `.zibby/graph.js`:
|
|
111
|
-
|
|
112
|
-
```bash
|
|
113
|
-
zibby test test-specs/login.txt --workflow QuickSmokeWorkflow
|
|
114
|
-
```
|
|
115
|
-
|
|
116
|
-
Supported formats: `QuickSmokeWorkflow`, `QuickSmoke`, `quick-smoke`.
|
|
117
|
-
|
|
118
|
-
## Opening Results in Browser
|
|
119
|
-
|
|
120
|
-
```bash
|
|
121
|
-
zibby test test-specs/login.txt --sync --open
|
|
122
|
-
```
|
|
123
|
-
|
|
124
|
-
The `--open` flag opens the dashboard results page after upload.
|
|
125
|
-
|
|
126
|
-
## Debugging
|
|
127
|
-
|
|
128
|
-
```bash
|
|
129
|
-
# Info-level logs
|
|
130
|
-
zibby test test-specs/login.txt --verbose
|
|
131
|
-
|
|
132
|
-
# Full debug logs
|
|
133
|
-
zibby test test-specs/login.txt --debug
|
|
134
|
-
```
|
|
@@ -1,53 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 3
|
|
3
|
-
title: Backup & restore
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Backup & restore
|
|
7
|
-
|
|
8
|
-
`backup.sh` and `restore.sh` ship next to the compose file in your install directory (the installer drops them there). Together they let a dead VM — or a fat-fingered `docker compose down -v` — never mean "start over and reconnect 50 repos".
|
|
9
|
-
|
|
10
|
-
## What a backup contains
|
|
11
|
-
|
|
12
|
-
One timestamped tarball holds the four things that together **are** your install:
|
|
13
|
-
|
|
14
|
-
1. **ScyllaDB volume** (`scylla-data`) — all metadata: integrations + their encrypted connect tokens, deployed agents (graph/env/schedule/prompts), repo→agent links, projects, members, access tokens, review-memory, execution history, stores.
|
|
15
|
-
2. **Object-store volume** — `seaweed-data` (SeaweedFS, the default) or `minio-data` on a legacy box. Run artifacts + per-node logs, datasets, marketplace icons. The active store is detected from `.env` (`ZIBBY_OBJECT_STORE`) / volume presence; `restore.sh` handles either.
|
|
16
|
-
3. **Sidecar data volumes** (`zibby-sidecar-*-data`) — content that exists **nowhere else** and cannot be re-derived: a knowledge base's indexed documents, a sidecar's encrypted OAuth token store. They are discovered by convention, so a sidecar you add later — including one you brought yourself — is included with no configuration. Skip them with `--skip-sidecar-data` (or `ZIBBY_BACKUP_SIDECAR_DATA=false` for scheduled backups) if the size is a real problem; those sidecars then restore **empty**, and `restore.sh` says so out loud.
|
|
17
|
-
4. **`.env`** — **critical**. It holds `ENCRYPTION_KEY`, without which every encrypted token/secret in ScyllaDB is unrecoverable, plus `JWT_SECRET`, the object-store root creds, and integration tokens. A ScyllaDB restore without the matching `ENCRYPTION_KEY` is useless, so `.env` travels **inside** the backup — which is also why the backup is highly sensitive.
|
|
18
|
-
|
|
19
|
-
## Take a backup
|
|
20
|
-
|
|
21
|
-
```bash
|
|
22
|
-
cd zibby-selfhosted
|
|
23
|
-
./backup.sh # → ./backups/zibby-selfhosted-<timestamp>.tar.gz
|
|
24
|
-
./backup.sh --out /mnt/backups # write elsewhere
|
|
25
|
-
./backup.sh --encrypt # encrypt at rest with age or gpg (prompts for a passphrase)
|
|
26
|
-
./backup.sh --s3 s3://mybucket/zibby # also upload with YOUR aws creds (nothing goes to Zibby)
|
|
27
|
-
```
|
|
28
|
-
|
|
29
|
-
Consistency + downtime: for a single-VM box the safe, simple approach wins — the script briefly **stops the writers** (control-plane + datastores + any running sidecars, whose single-writer file databases would otherwise be captured mid-write), `nodetool flush`es Scylla so the commitlog is on disk, tars the volumes read-only from a throwaway helper container, then restarts everything. Downtime is typically seconds to a few minutes (plus Scylla's ~1–2 min cold-boot on restart). A trap guarantees the stack restarts even if the backup errors out. Volume names are compose-project-prefixed and **discovered at runtime**, so it works regardless of your install-dir name.
|
|
30
|
-
|
|
31
|
-
Copy the tarball **off the box** — it is your only recovery path, and it contains the encryption key.
|
|
32
|
-
|
|
33
|
-
:::tip
|
|
34
|
-
An upgrade takes its own safety snapshot into `<install-dir>/backups/upgrade-<id>/` before touching anything, and there is an `auto-backup.sh` you can cron for scheduled snapshots.
|
|
35
|
-
:::
|
|
36
|
-
|
|
37
|
-
## Restore
|
|
38
|
-
|
|
39
|
-
Onto the same box after data loss, or onto a brand-new VM:
|
|
40
|
-
|
|
41
|
-
```bash
|
|
42
|
-
./restore.sh zibby-selfhosted-<timestamp>.tar.gz # interactive confirm
|
|
43
|
-
./restore.sh backup.tar.gz --dir /opt/zibby --force # scripted
|
|
44
|
-
./restore.sh backup.age # decrypts first (needs age/gpg)
|
|
45
|
-
```
|
|
46
|
-
|
|
47
|
-
`restore.sh` restores **both** data volumes and the `.env` (so the restored, encrypted data is readable). Volume names are discovered at runtime, or created if this is a fresh box.
|
|
48
|
-
|
|
49
|
-
:::danger Restore wipes current data
|
|
50
|
-
Restoring **overwrites** this install's volumes. The script refuses to run without an explicit confirmation (interactive `y/N`, or `--force`).
|
|
51
|
-
:::
|
|
52
|
-
|
|
53
|
-
A MinIO-era bundle restored onto a current (post-SeaweedFS-flip) release comes up on the legacy MinIO override; the next installer run auto-migrates it to SeaweedFS — see [Storage](./storage).
|
|
@@ -1,136 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 5
|
|
3
|
-
title: Custom (BYO) sidecars
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Bring your own sidecar
|
|
7
|
-
|
|
8
|
-
A **sidecar** is a long-lived service container the box runs next to your
|
|
9
|
-
agents — the platform's own vector-KB engine and OAuth token broker ship this
|
|
10
|
-
way. On a self-hosted box you can also register **your own** sidecar image.
|
|
11
|
-
That is the standard way to add a custom resident service to the platform —
|
|
12
|
-
most commonly a **thin MCP server that wraps an internal REST API**, so your
|
|
13
|
-
agents and the chat Copilot can call systems that only exist inside your
|
|
14
|
-
network.
|
|
15
|
-
|
|
16
|
-
No marketplace entry, no platform code change, no third-party registry: you
|
|
17
|
-
push a `docker save` tarball to your own box, the box pins its sha256, and the
|
|
18
|
-
sidecar flows through the exact same launch/health/idle-reap machinery as the
|
|
19
|
-
built-ins.
|
|
20
|
-
|
|
21
|
-
:::tip Wrapping a REST API? Try the OpenAPI bridge agent first.
|
|
22
|
-
|
|
23
|
-
If all you need is "my agents should be able to call this internal REST API",
|
|
24
|
-
you probably do **not** need to build an image at all. Deploy the **OpenAPI MCP
|
|
25
|
-
Bridge** agent from the marketplace, put your API's spec URL in its **Env** tab,
|
|
26
|
-
and every operation in the spec becomes an MCP tool — no code, no upload.
|
|
27
|
-
|
|
28
|
-
```json
|
|
29
|
-
OPENAPI_APIS = {"billing":{"specUrl":"https://internal/v2/api-docs","root":"https://internal"}}
|
|
30
|
-
```
|
|
31
|
-
|
|
32
|
-
It is a marketplace agent that carries its own sidecar, so deploying it is the
|
|
33
|
-
whole installation. Build your own image (below) when you need logic the bridge
|
|
34
|
-
can't express — a non-HTTP protocol, a stateful session, custom auth.
|
|
35
|
-
:::
|
|
36
|
-
|
|
37
|
-
## Push one
|
|
38
|
-
|
|
39
|
-
```bash
|
|
40
|
-
# 1. Build your image however you like; it must serve HTTP on one port.
|
|
41
|
-
docker build -t report-mcp:0.1.0 .
|
|
42
|
-
docker save report-mcp:0.1.0 | gzip > report-mcp.tar.gz
|
|
43
|
-
|
|
44
|
-
# 2. Push it to the box. Needs an OPERATOR access token
|
|
45
|
-
# (a restricted per-user token is refused).
|
|
46
|
-
zibby sidecar push report-mcp.tar.gz \
|
|
47
|
-
--name report-mcp --port 8080 \
|
|
48
|
-
--health-path /health \
|
|
49
|
-
--warm \
|
|
50
|
-
--api-url https://your-box.example.com --token $OPERATOR_PAT
|
|
51
|
-
```
|
|
52
|
-
|
|
53
|
-
`push` streams the tarball up (`POST /selfhost/sidecars/artifacts`), then
|
|
54
|
-
registers the declaration (`PUT /selfhost/sidecars/report-mcp`) pinned to the
|
|
55
|
-
sha256 the server computed from the uploaded bytes. The image is verified and
|
|
56
|
-
`docker load`ed **before** anything is persisted — a corrupt or tampered
|
|
57
|
-
tarball can never become a registered sidecar.
|
|
58
|
-
|
|
59
|
-
Manage what's registered:
|
|
60
|
-
|
|
61
|
-
```bash
|
|
62
|
-
zibby sidecar list # custom sidecars + the reserved built-in names
|
|
63
|
-
zibby sidecar remove <name> # unregister + stop (image reclaimed; data volume kept) (data volumes are kept)
|
|
64
|
-
```
|
|
65
|
-
|
|
66
|
-
## Reach it
|
|
67
|
-
|
|
68
|
-
- **From the control plane / the chat Copilot** — the container listens on the
|
|
69
|
-
box's infra network at `http://zibby-sidecar-<name>:<port>`. Agent run
|
|
70
|
-
containers deliberately **cannot** dial it directly (they are isolated on the
|
|
71
|
-
run network); access is brokered by the platform.
|
|
72
|
-
- **From a browser / third-party client** — every `--public-path` prefix is
|
|
73
|
-
served at `https://<box-origin>/sidecars/<name>/<path>` through the
|
|
74
|
-
control-plane reverse proxy. Anything not declared public stays
|
|
75
|
-
infra-network-only.
|
|
76
|
-
|
|
77
|
-
:::warning `--public-path` is opt-in, and most sidecars don't need it
|
|
78
|
-
A declared public path is reachable **anonymously** — the platform adds no auth
|
|
79
|
-
and no identity; your app owns that surface entirely. Agents and the chat
|
|
80
|
-
Copilot reach a sidecar over the infra network, so an MCP server for them needs
|
|
81
|
-
**no** public path. It exists for browser flows (an OAuth callback a provider
|
|
82
|
-
redirects to). Registering one is therefore refused unless the box opts in with
|
|
83
|
-
`SIDECAR_BYO_PUBLIC_PATHS=1` in its `.env` (then restart the control-plane).
|
|
84
|
-
:::
|
|
85
|
-
|
|
86
|
-
### Wire it into the Copilot
|
|
87
|
-
|
|
88
|
-
For the "internal API → MCP → chat" use case, register the sidecar and then
|
|
89
|
-
attach its MCP endpoint to the Copilot (in chat):
|
|
90
|
-
|
|
91
|
-
> connect yourself to the MCP at `http://zibby-sidecar-report-mcp:8080/mcp`
|
|
92
|
-
|
|
93
|
-
The Copilot can also drive the whole flow itself with the owner-only tools
|
|
94
|
-
`zibby_add_sidecar` (takes a downloadable `url` + pinned `sha256` instead of a
|
|
95
|
-
file upload), `zibby_list_sidecars`, and `zibby_remove_sidecar`.
|
|
96
|
-
|
|
97
|
-
## Declaration reference
|
|
98
|
-
|
|
99
|
-
| Flag | Meaning |
|
|
100
|
-
|---|---|
|
|
101
|
-
| `--name` | `[a-z][a-z0-9-]{1,30}`. Built-in names (`gbrain`, `pingcode`, …) are reserved. |
|
|
102
|
-
| `--port` | The HTTP port your app listens on inside the container. Required. |
|
|
103
|
-
| `--version` | Display/tag version (default: first 12 chars of the sha256). |
|
|
104
|
-
| `--health-path` | GET path returning 2xx when ready (default `/health`). |
|
|
105
|
-
| `--public-path` | Repeatable. Path prefix exposed through the public reverse proxy — **anonymously**. Disabled unless the box sets `SIDECAR_BYO_PUBLIC_PATHS=1`; see below. |
|
|
106
|
-
| `--warm` | Keep it always-on (pre-warmed at boot, never idle-reaped). Default: on-demand launch + idle reap. |
|
|
107
|
-
| `--env-key` | Repeatable. A box `.env` variable forwarded into the container when set. |
|
|
108
|
-
| `--request-config-key` | Repeatable. Per-agent config: resolved from the **calling agent's encrypted Env bag** and sent per request — one shared container can serve N projects with different upstream credentials. |
|
|
109
|
-
| `--data-path` | Container path persisted on a named volume (survives restarts; kept on remove). |
|
|
110
|
-
| `--memory-bytes` | Container memory cap. |
|
|
111
|
-
|
|
112
|
-
## What the box guarantees
|
|
113
|
-
|
|
114
|
-
- **sha256-pinned, always.** The tarball's hash is computed server-side at
|
|
115
|
-
upload and re-verified on every subsequent load — including the re-fetch
|
|
116
|
-
path: the verified tarball is kept in the box's own object store, so if the
|
|
117
|
-
docker image is ever pruned it is restored automatically without a re-upload.
|
|
118
|
-
- **Operator-only surface.** Upload/register/remove require the box
|
|
119
|
-
owner/admin token; restricted per-user tokens get `403`.
|
|
120
|
-
- **Isolation unchanged.** Custom sidecars get the same placement as built-in
|
|
121
|
-
ones: infra network, brokered access, no host mounts — the declaration
|
|
122
|
-
cannot ask for privileged options.
|
|
123
|
-
- **No shadowing, at BOTH layers.** A custom sidecar can never take over a
|
|
124
|
-
built-in sidecar's *name*, and its package can never take over a platform or
|
|
125
|
-
another sidecar's *image tag*: the archive's manifest is read **before**
|
|
126
|
-
`docker load` runs, and a package declaring `zibby-*` or another sidecar's
|
|
127
|
-
image is refused outright — even if that image isn't on the box yet.
|
|
128
|
-
- **Uninstall reclaims storage.** Removing a sidecar deletes its image; a
|
|
129
|
-
superseded image is reclaimed when you push a new version. The declared data
|
|
130
|
-
volume is **kept** (it holds your content) — the response tells you its name,
|
|
131
|
-
and `zibby sidecar remove <name> --purge-data` deletes it deliberately.
|
|
132
|
-
|
|
133
|
-
:::note Cloud
|
|
134
|
-
BYO sidecars are **self-host only** today. On cloud, wrap an external API as a
|
|
135
|
-
remote MCP server and attach it with `zibby_add_mcp` instead.
|
|
136
|
-
:::
|
package/docs/self-host/index.md
DELETED
|
@@ -1,75 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 1
|
|
3
|
-
title: Self-host overview
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Operating a self-hosted box
|
|
7
|
-
|
|
8
|
-
Run the entire Zibby agent platform on **one host**, inside your own VPC, with `docker compose`. This section is the **operator runbook** — for the person who holds the box's owner/admin token and is responsible for install, upgrade, backup, and storage. It is not the agent-authoring guide (that's the rest of these docs, and `zibby init` in your repo).
|
|
9
|
-
|
|
10
|
-
A self-hosted box is the **same codebase as cloud**, assembled differently: the control plane is `backend/server.js` (the Express server that wraps every Lambda handler, including the MCP server), the run executor is the host Docker engine instead of Fargate, and the datastore / object store are ScyllaDB Alternator + SeaweedFS instead of DynamoDB + S3.
|
|
11
|
-
|
|
12
|
-
## Install
|
|
13
|
-
|
|
14
|
-
One command, on a Linux/macOS host with **Docker Engine + Compose v2**:
|
|
15
|
-
|
|
16
|
-
```bash
|
|
17
|
-
curl -fsSL https://dl.zibby.app/selfhosted/latest/install.sh | bash
|
|
18
|
-
```
|
|
19
|
-
|
|
20
|
-
The installer downloads a signed release (a `docker save` image tarball + compose file + the marketplace-template bundle), **verifies it** (an Ed25519 signature over `SHA256SUMS`, then a per-file checksum — it fails closed on any mismatch and never runs unverified bytes), `docker load`s the images, and `docker compose up -d`s the stack. It is idempotent — re-running it is how you upgrade in place, and cached downloads are checksum-verified.
|
|
21
|
-
|
|
22
|
-
:::warning Compose v2 is required
|
|
23
|
-
`apt install docker.io` alone gives you the engine but **not** Compose v2. Install both — the installer aborts with a link if `docker compose version` fails. On a headless box also make sure `openssl` is present (the installer needs it to verify the release signature).
|
|
24
|
-
:::
|
|
25
|
-
|
|
26
|
-
### What comes up
|
|
27
|
-
|
|
28
|
-
The stack is a handful of containers on the `zibby-selfhosted` docker network:
|
|
29
|
-
|
|
30
|
-
| Container | Role |
|
|
31
|
-
|---|---|
|
|
32
|
-
| `zibby-control-plane` | the long-lived API + `/mcp` + `/hooks/*` server + the dashboard; drives the host Docker engine as the run executor over the mounted `docker.sock` |
|
|
33
|
-
| `zibby-scylla` | **ScyllaDB Alternator** (`:8000`) — speaks the DynamoDB API, so the backend is unchanged. All metadata: integrations + encrypted tokens, deployed agents, projects, members, executions, stores. |
|
|
34
|
-
| `zibby-minio` | the **object store** (`:9000`) for run sources + artifacts + logs. As of 0.1.126 this is **SeaweedFS** (the service keeps the historical name `minio` so every `S3_ENDPOINT` consumer is unchanged). Legacy boxes still run MinIO until they auto-migrate — see [Storage](./storage). |
|
|
35
|
-
| `zibby-bootstrap` | one-shot: provisions tables + buckets, syncs all marketplace templates, seeds the default project + token, then exits. |
|
|
36
|
-
| run containers | one ephemeral `zibby-agent` container per run, destroyed on exit — the same per-run isolation Fargate gives cloud. |
|
|
37
|
-
|
|
38
|
-
The control plane mounts `/var/run/docker.sock` — that socket **is** the run executor (no docker-in-docker).
|
|
39
|
-
|
|
40
|
-
### After install
|
|
41
|
-
|
|
42
|
-
The installer prints the access info and the seeded credential:
|
|
43
|
-
|
|
44
|
-
- **Dashboard** — `http://<host>:3001/` (and `/ui`). The self-host dashboard is a single vanilla HTML page served by the control plane; the cloud React app is not shipped on-prem.
|
|
45
|
-
- **API + CLI** — `http://<host>:3001`. Set `ZIBBY_API_URL` + `ZIBBY_API_KEY=<apiToken>` (the seeded `zby_…` project token printed at the end, or recover it with `docker logs zibby-bootstrap`).
|
|
46
|
-
- **MCP** — `http://<host>:3001/mcp`, `Authorization: Bearer <PAT>`. MCP needs a **user PAT** (`zby_pat_…`), minted with `zibby self-host token` — not the project token.
|
|
47
|
-
- **Health** — `http://<host>:3001/health`.
|
|
48
|
-
|
|
49
|
-
AI model keys are **per-project**, set in the dashboard (Settings → AI); the deploy modal collects a missing one. The box comes up keyless — there is deliberately no box-global model credential.
|
|
50
|
-
|
|
51
|
-
## Install knobs
|
|
52
|
-
|
|
53
|
-
Set these as environment variables or flags on the install command (they persist into `.env`, which is the runtime source of truth and is **reused untouched on upgrade**):
|
|
54
|
-
|
|
55
|
-
| Knob | Default | Effect |
|
|
56
|
-
|---|---|---|
|
|
57
|
-
| `--dir <path>` / `ZIBBY_DIR` | `./zibby-selfhosted` | install + data directory |
|
|
58
|
-
| `--url <u>` / `ZIBBY_DIST_URL` | `https://dl.zibby.app/selfhosted/latest` | release base URL |
|
|
59
|
-
| `--reconfigure` | — | re-run the external-URL / domain prompt |
|
|
60
|
-
| `CONTROL_PLANE_PORT` (`.env`) | `3001` | host port the control plane binds |
|
|
61
|
-
| `CONTROL_PLANE_BIND` (`.env`) | `0.0.0.0` | set to `127.0.0.1` to restrict the dashboard/API to loopback and front it with the bundled Caddy TLS proxy |
|
|
62
|
-
| `ZIBBY_EXTERNAL_URL` (`.env`/prompt) | unset | a domain → automatic Let's Encrypt HTTPS; an IP → self-signed HTTPS; `http://…` → plain HTTP via Caddy. Blank → plain HTTP on `CONTROL_PLANE_PORT`. |
|
|
63
|
-
| `MAX_CONCURRENT_RUNS` (`.env`) | `5` | run containers executing in parallel; the rest queue |
|
|
64
|
-
| `ZIBBY_SKIP_OBJECT_STORE_MIGRATION` | `0` | `1` opts a legacy MinIO box out of the auto-migration to SeaweedFS |
|
|
65
|
-
|
|
66
|
-
:::warning Plain-HTTP default binds in cleartext
|
|
67
|
-
With no external URL the dashboard/API bind to `0.0.0.0:3001` unencrypted — the operator token and login passcode cross the network in the clear. Either set `ZIBBY_EXTERNAL_URL` (→ auto-TLS via Caddy), firewall the port to trusted IPs, or set `CONTROL_PLANE_BIND=127.0.0.1`.
|
|
68
|
-
:::
|
|
69
|
-
|
|
70
|
-
## The runbook
|
|
71
|
-
|
|
72
|
-
- **[Upgrade & rollback](./upgrade)** — check the running version, the owner-only remote upgrade, and how auto-rollback works.
|
|
73
|
-
- **[Backup & restore](./backup-restore)** — snapshot the whole install to one tarball; restore onto the same or a fresh box.
|
|
74
|
-
- **[Storage](./storage)** — the object store, and the automatic MinIO → SeaweedFS migration.
|
|
75
|
-
- **[Troubleshooting](./troubleshooting)** — reading logs and the failure signatures worth recognizing.
|
|
@@ -1,50 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 4
|
|
3
|
-
title: Storage
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Storage
|
|
7
|
-
|
|
8
|
-
A self-hosted box keeps two stateful stores, both as single-box containers on named docker volumes:
|
|
9
|
-
|
|
10
|
-
- **ScyllaDB Alternator** (`zibby-scylla`, volume `scylla-data`) — the DynamoDB-compatible metadata store. All accounts, projects, agents, integrations, executions, and stores live here.
|
|
11
|
-
- **Object store** (`zibby-minio`, volume `seaweed-data` or legacy `minio-data`) — run sources, artifacts, per-node logs, datasets, and marketplace icons, over the S3 API.
|
|
12
|
-
|
|
13
|
-
Both survive `docker compose restart` / `down` (they're on named volumes) but **not** `down -v`. Back them up — see [Backup & restore](./backup-restore).
|
|
14
|
-
|
|
15
|
-
## MinIO → SeaweedFS
|
|
16
|
-
|
|
17
|
-
MinIO's upstream was archived in 2026-04, so as of **0.1.126** the object store is **SeaweedFS**. The service is still *named* `minio` in compose and DNS so every `S3_ENDPOINT` consumer is unchanged — only the engine behind it changed. (Full evaluation: `strategy/object-store-replacement.md` in the root repo.)
|
|
18
|
-
|
|
19
|
-
- **New installs** come up on SeaweedFS (volume `seaweed-data`). Nothing to do.
|
|
20
|
-
- **Upgrades from a MinIO box auto-migrate.** When the installer upgrades a legacy (pre-flip, `minio-data`) box, it keeps MinIO running for that `up`, then — once the upgraded stack is healthy — runs the migration automatically.
|
|
21
|
-
|
|
22
|
-
### How the auto-migration works
|
|
23
|
-
|
|
24
|
-
It is designed to **auto-attempt, never auto-brick** — MinIO's data is never modified, so every failure path rolls back to a working box:
|
|
25
|
-
|
|
26
|
-
1. **backup** — a full `backup.sh` snapshot first; it refuses to migrate without one.
|
|
27
|
-
2. **quiesce** — stop the S3 writers (control-plane + copilot-runtime) so the copy is consistent.
|
|
28
|
-
3. **copy + verify** — `migrate-object-store.sh` mirrors every bucket into a throwaway SeaweedFS container writing to the `seaweed-data` volume, then verifies per-bucket **object count + total bytes must match exactly**, plus a sha256 comparison of sampled objects (always including the largest). It never deletes anything.
|
|
29
|
-
4. **flip** — record `ZIBBY_OBJECT_STORE=seaweedfs` in `.env` and bring up the default (SeaweedFS) stack.
|
|
30
|
-
5. **health-gate** — the new control plane must become healthy **and** pass an authenticated S3 list against the migrated store.
|
|
31
|
-
6. **rollback** — any failure at any stage rolls back to MinIO (whose data was never touched) with a loud status line.
|
|
32
|
-
|
|
33
|
-
The **old `minio-data` volume is retained** as a rollback safety net. After a few days of verified operation you can reclaim it:
|
|
34
|
-
|
|
35
|
-
```bash
|
|
36
|
-
docker volume rm <project>_minio-data # irreversible — only after you trust SeaweedFS
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
### Controlling / retrying the migration
|
|
40
|
-
|
|
41
|
-
- **Opt out** (stay on MinIO for now): run the installer with `ZIBBY_SKIP_OBJECT_STORE_MIGRATION=1`.
|
|
42
|
-
- **Retry** after a failed auto-migration: re-run the installer, or manually —
|
|
43
|
-
```bash
|
|
44
|
-
cd zibby-selfhosted
|
|
45
|
-
docker compose stop control-plane copilot-runtime
|
|
46
|
-
bash migrate-object-store.sh --yes
|
|
47
|
-
# then flip: set ZIBBY_OBJECT_STORE=seaweedfs + COMPOSE_FILE=docker-compose.yml in .env
|
|
48
|
-
docker compose -f docker-compose.yml up -d
|
|
49
|
-
```
|
|
50
|
-
- **MinIO emergency fallback image** (if a legacy box lost its local MinIO image): the pinned tarball is at `https://dl.zibby.app/selfhosted/infra/` — verify the `.sha256`, then `docker load -i`. The box only ever pulls images from our own CDN, never a third-party registry.
|
|
@@ -1,48 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 5
|
|
3
|
-
title: Troubleshooting
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Troubleshooting
|
|
7
|
-
|
|
8
|
-
A self-hosted box is **one VM** — control-plane container + Scylla + object store + ephemeral run containers. When something is slow or failing, diagnose from the box's own logs; the reflex "it doesn't scale / the VM is overloaded" is almost always wrong (a `top` usually shows the box near-idle).
|
|
9
|
-
|
|
10
|
-
## First moves
|
|
11
|
-
|
|
12
|
-
```bash
|
|
13
|
-
top # CPU / memory — usually reveals it's idle
|
|
14
|
-
docker ps -a # are containers Up? did a run container Exit?
|
|
15
|
-
docker logs zibby-control-plane --tail 100 # the executor + reaper + API
|
|
16
|
-
docker logs zibby-bootstrap # first-boot template sync + the seeded token
|
|
17
|
-
curl -s http://localhost:3001/health # control-plane health
|
|
18
|
-
curl -s http://localhost:3001/selfhost/version # the running release (baked, authoritative)
|
|
19
|
-
```
|
|
20
|
-
|
|
21
|
-
To query the box's own API, use its `PROJECT_API_TOKEN` from `.env`:
|
|
22
|
-
|
|
23
|
-
```bash
|
|
24
|
-
curl -H "Authorization: Bearer <zby_...>" \
|
|
25
|
-
"http://localhost:3001/executions?projectId=selfhosted-project"
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
A failed run's `reapedReason` field tells you why it was reaped. (The cloud Zibby MCP/CLI talks to the *cloud* backend — it cannot reach a self-host box; you need the box's own token.)
|
|
29
|
-
|
|
30
|
-
## Failure signatures worth recognizing
|
|
31
|
-
|
|
32
|
-
**Runs "fail" with empty steps + `pull access denied for zibby-agent … repository does not exist`.** An agent-image tag mismatch after an upgrade — the executor is launching a tag the reused `.env` doesn't have. Fixed at the root in installer 0.1.53+; immediate unblock:
|
|
33
|
-
|
|
34
|
-
```bash
|
|
35
|
-
docker tag zibby-agent:slim zibby-agent:local
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
**The dashboard shows the new version but the API still behaves like the old one.** `docker compose up -d` can judge a service "up to date" when only the image *content* changed. The installer force-recreates the control-plane container to avoid this; if you hit it, re-run the installer (or `docker compose up -d --no-deps --force-recreate control-plane`). Because `/selfhost/version` reads the **baked** file, a genuine version mismatch is *visible* rather than hidden behind a lying env-var label.
|
|
39
|
-
|
|
40
|
-
**Control plane won't become healthy right after boot.** Scylla cold-boots slowly (~1–2 min). The installer waits up to ~5 min; if it's still unhealthy, read `docker logs zibby-control-plane` — a broken data path shows up there, not in `top`.
|
|
41
|
-
|
|
42
|
-
**An upgrade ended at `rolled_back`.** The box is safe (back on the prior version). Read `GET /selfhost/upgrade/status` `logTail` for the failing phase, and check `docker logs zibby-control-plane`. Re-attempt with a **pinned** version (see [Upgrade & rollback](./upgrade)).
|
|
43
|
-
|
|
44
|
-
## Where things live
|
|
45
|
-
|
|
46
|
-
- **Data** — docker volumes `scylla-data` + the object store volume. Back them up (see [Backup & restore](./backup-restore)); they survive restarts but not `down -v`.
|
|
47
|
-
- **`.env`** — the runtime source of truth (`JWT_SECRET`, `ENCRYPTION_KEY`, tokens, `MAX_CONCURRENT_RUNS`, `AGENT_IMAGE`). Reused untouched on upgrade.
|
|
48
|
-
- **Upgrade backups + logs** — `<install-dir>/backups/upgrade-<id>/`.
|
|
@@ -1,81 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 2
|
|
3
|
-
title: Upgrade & rollback
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Upgrade & rollback
|
|
7
|
-
|
|
8
|
-
## Check the running version
|
|
9
|
-
|
|
10
|
-
```bash
|
|
11
|
-
curl -s http://<host>:3001/selfhost/version
|
|
12
|
-
# {"version":"0.1.126"}
|
|
13
|
-
```
|
|
14
|
-
|
|
15
|
-
This reads the **immutable file `/app/SELFHOST_VERSION`**, baked into the image at build time from `selfhosted/package.json`. It ships inside the image layer, so it **always matches the code actually running** and cannot be spoofed by a runtime env / compose `.env` override. That is the point: if an upgrade did not actually swap the control-plane image (a silent-upgrade failure), the baked file still reports the old version — you *see* the mismatch instead of trusting a lying label.
|
|
16
|
-
|
|
17
|
-
The dashboard renders this same value as a small `v0.1.126` label in the sidebar, so the running release is always visible. (The dashboard has no upgrade button — upgrades are driven by the API endpoint or by re-running the installer, below.)
|
|
18
|
-
|
|
19
|
-
The latest published version is in the release manifest:
|
|
20
|
-
|
|
21
|
-
```bash
|
|
22
|
-
curl -s https://dl.zibby.app/selfhosted/latest/manifest.env
|
|
23
|
-
# ZIBBY_SELFHOSTED_VERSION=0.1.126
|
|
24
|
-
# IMAGE_TARBALL=zibby-selfhosted-0.1.126.tar.gz
|
|
25
|
-
# ...
|
|
26
|
-
```
|
|
27
|
-
|
|
28
|
-
## Two ways to upgrade
|
|
29
|
-
|
|
30
|
-
### 1. Re-run the installer (in place)
|
|
31
|
-
|
|
32
|
-
The installer is idempotent — re-running it pulls a newer release, verifies + `docker load`s the new images, re-pins `AGENT_IMAGE`, and **force-recreates** the control-plane container from the freshly loaded image:
|
|
33
|
-
|
|
34
|
-
```bash
|
|
35
|
-
curl -fsSL https://dl.zibby.app/selfhosted/latest/install.sh | bash
|
|
36
|
-
# or, from the install dir: cd zibby-selfhosted && bash install.sh
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
Your `.env` (JWT_SECRET, ENCRYPTION_KEY, tokens) and data volumes are preserved.
|
|
40
|
-
|
|
41
|
-
### 2. The owner-only remote upgrade (recommended)
|
|
42
|
-
|
|
43
|
-
`POST /selfhost/upgrade` runs a **safe, self-driving upgrade** with an automatic backup and rollback — you can trigger it remotely and it survives the control-plane restart. It is **owner-only**: the endpoint requires the workspace-owner PAT/session or the admin project token (a run token or a non-owner member is refused `403`).
|
|
44
|
-
|
|
45
|
-
```bash
|
|
46
|
-
curl -s -X POST http://<host>:3001/selfhost/upgrade \
|
|
47
|
-
-H "Authorization: Bearer <owner-or-admin-token>" \
|
|
48
|
-
-H "Content-Type: application/json" \
|
|
49
|
-
-d '{"version":"0.1.126"}'
|
|
50
|
-
# → 202 { "upgradeId": "...", "phase": "starting", ... }
|
|
51
|
-
```
|
|
52
|
-
|
|
53
|
-
Because a container can't recreate itself, the control plane launches a **detached helper** container (with the host docker socket) that outlives the control-plane bounce and drives the upgrade end to end:
|
|
54
|
-
|
|
55
|
-
1. **backup** — flush Scylla, tar the Scylla + object-store volumes, copy `.env`, into `<install-dir>/backups/upgrade-<id>/`.
|
|
56
|
-
2. **anchor** — pin the current control-plane image as `zibby-control-plane:rollback-<id>` so the new `docker load` can't garbage-collect the rollback target.
|
|
57
|
-
3. **download → install** — fetch the target release's `install.sh` (Ed25519 + checksum verified), `docker load`, re-pin `AGENT_IMAGE`, force-recreate.
|
|
58
|
-
4. **healthcheck** — poll the new control plane until container-healthy **and** `GET /selfhost/version` returns the target version.
|
|
59
|
-
5. **rollback** — on an unhealthy new control plane (or an installer that exited non-zero), re-tag the anchor back to `:local` and force-recreate → back to the working version.
|
|
60
|
-
|
|
61
|
-
Poll the status (owner-only, read-only):
|
|
62
|
-
|
|
63
|
-
```bash
|
|
64
|
-
curl -s http://<host>:3001/selfhost/upgrade/status \
|
|
65
|
-
-H "Authorization: Bearer <owner-or-admin-token>"
|
|
66
|
-
# { "upgrade": { "phase": "install", ... }, "logTail": "...", "history": [...] }
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
Phases progress `starting → backup → anchor → download → install → healthcheck → success`. A failed upgrade ends at `rolled_back` (the box is back on the prior, healthy version) or, if even rollback fails, `rollback_failed` (manual restore needed — the backup dir and rollback-anchor tag are named in the status). The control plane restarts mid-upgrade, so the poll may briefly fail to connect — that is expected.
|
|
70
|
-
|
|
71
|
-
:::warning Pin the exact version, not `latest`
|
|
72
|
-
`{"version":"latest"}` resolves through the CDN, which can serve a **stale cached** `latest` for a while after a publish. For a deterministic upgrade, pass the exact version you verified against `manifest.env` (e.g. `{"version":"0.1.126"}`). The helper also treats "healthy but the version didn't advance to the pinned target" as a failure and rolls back — a guard you only get with a pinned version.
|
|
73
|
-
:::
|
|
74
|
-
|
|
75
|
-
## If a run fails with empty steps after an upgrade
|
|
76
|
-
|
|
77
|
-
Symptom: reviews/runs "fail" with `steps: []` and the control-plane log shows `pull access denied for zibby-agent … repository does not exist`. That is an **agent-image tag mismatch** — the executor is trying to launch a tag the reused `.env` points at but the tarball loaded a different one. Installer 0.1.53+ fixes this at the root (it pins `AGENT_IMAGE` to the loaded tag and aliases `zibby-agent:local`). Immediate operator unblock:
|
|
78
|
-
|
|
79
|
-
```bash
|
|
80
|
-
docker tag zibby-agent:slim zibby-agent:local
|
|
81
|
-
```
|