@zibby/skills 0.2.22 → 0.2.24

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (90) hide show
  1. package/dist/browser.js +3 -10
  2. package/dist/code-scan.d.ts +1 -1
  3. package/dist/index.js +81 -88
  4. package/dist/jira.d.ts +75 -9
  5. package/dist/jira.js +4 -4
  6. package/dist/package.json +1 -1
  7. package/dist/report.d.ts +51 -276
  8. package/dist/trackers/index.js +12 -12
  9. package/dist/trackers/jira-adapter.js +11 -11
  10. package/package.json +1 -1
  11. package/docs/analysis.md +0 -109
  12. package/docs/apps/agent-ops.md +0 -130
  13. package/docs/apps/auth.md +0 -158
  14. package/docs/apps/deploy.md +0 -207
  15. package/docs/apps/goal-mode.md +0 -175
  16. package/docs/apps/index.md +0 -149
  17. package/docs/apps/managing.md +0 -121
  18. package/docs/cli-reference.md +0 -494
  19. package/docs/cloning-repositories.md +0 -285
  20. package/docs/cloud/bundles.md +0 -92
  21. package/docs/cloud/dedicated-egress.md +0 -140
  22. package/docs/cloud/editing-prompts.md +0 -79
  23. package/docs/cloud/env-vars.md +0 -144
  24. package/docs/cloud/limits.md +0 -81
  25. package/docs/cloud/logs.md +0 -104
  26. package/docs/cloud/triggering.md +0 -128
  27. package/docs/concepts/agents.md +0 -112
  28. package/docs/concepts/designing-agents.md +0 -247
  29. package/docs/concepts/graph.md +0 -83
  30. package/docs/concepts/sessions.md +0 -70
  31. package/docs/concepts/skills.md +0 -86
  32. package/docs/concepts/state.md +0 -106
  33. package/docs/concepts/sub-graphs.md +0 -227
  34. package/docs/custom-workflows.md +0 -358
  35. package/docs/get-started/deploy.md +0 -75
  36. package/docs/get-started/install.md +0 -60
  37. package/docs/get-started/run-locally.md +0 -94
  38. package/docs/get-started/trigger-and-logs.md +0 -90
  39. package/docs/get-started/use-from-agents.md +0 -153
  40. package/docs/get-started/your-first-workflow.md +0 -69
  41. package/docs/getting-started.md +0 -108
  42. package/docs/installation.md +0 -127
  43. package/docs/integrations/github.md +0 -73
  44. package/docs/integrations/gitlab.md +0 -43
  45. package/docs/integrations/jira.md +0 -71
  46. package/docs/integrations/lark.md +0 -41
  47. package/docs/integrations/linear.md +0 -43
  48. package/docs/integrations/notion.md +0 -33
  49. package/docs/integrations/plane.md +0 -46
  50. package/docs/integrations/sentry.md +0 -42
  51. package/docs/integrations/slack.md +0 -33
  52. package/docs/intro.md +0 -86
  53. package/docs/legacy/test-automation.md +0 -111
  54. package/docs/packages/agent-workflow.md +0 -88
  55. package/docs/packages/cli.md +0 -73
  56. package/docs/packages/core.md +0 -72
  57. package/docs/packages/mcp-browser.md +0 -110
  58. package/docs/packages/mcp-cli.md +0 -176
  59. package/docs/packages/memory.md +0 -223
  60. package/docs/packages/skills.md +0 -218
  61. package/docs/packages/ui-memory.md +0 -245
  62. package/docs/recipes/bug-autofix.md +0 -85
  63. package/docs/recipes/github-ai-scout.md +0 -61
  64. package/docs/recipes/index.md +0 -68
  65. package/docs/recipes/pipeline-supervisor.md +0 -57
  66. package/docs/recipes/sentry-triage.md +0 -93
  67. package/docs/recipes/test.md +0 -190
  68. package/docs/reviewing-results.md +0 -114
  69. package/docs/running-tests.md +0 -134
  70. package/docs/self-host/backup-restore.md +0 -53
  71. package/docs/self-host/custom-sidecars.md +0 -136
  72. package/docs/self-host/index.md +0 -75
  73. package/docs/self-host/storage.md +0 -50
  74. package/docs/self-host/troubleshooting.md +0 -48
  75. package/docs/self-host/upgrade.md +0 -81
  76. package/docs/skills/browser.md +0 -97
  77. package/docs/skills/chat-memory.md +0 -122
  78. package/docs/skills/core-tools.md +0 -80
  79. package/docs/skills/function-skill.md +0 -93
  80. package/docs/skills/github.md +0 -91
  81. package/docs/skills/index.md +0 -46
  82. package/docs/skills/jira.md +0 -99
  83. package/docs/skills/lark.md +0 -85
  84. package/docs/skills/memory.md +0 -92
  85. package/docs/skills/sentry.md +0 -80
  86. package/docs/skills/slack.md +0 -89
  87. package/docs/tests/memory.md +0 -131
  88. package/docs/triggering-workflows.md +0 -552
  89. package/docs/workflow-artifact-layout-evaluation.md +0 -119
  90. package/docs/workflow.md +0 -558
@@ -1,134 +0,0 @@
1
- ---
2
- sidebar_position: 5
3
- title: Running Tests
4
- ---
5
-
6
- # Running Tests
7
-
8
- Zibby runs tests locally using an AI agent that drives a real browser. It records video, captures actions, and generates reusable Playwright scripts.
9
-
10
- ## Basic Usage
11
-
12
- ```bash
13
- zibby test test-specs/login.txt --agent cursor
14
- ```
15
-
16
- ## From a Test Spec File
17
-
18
- Create a plain-text file with test steps:
19
-
20
- ```text title="test-specs/login.txt"
21
- 1. Navigate to the login page
22
- 2. Enter email: test@example.com
23
- 3. Enter password: TestPass123
24
- 4. Click the Sign In button
25
- 5. Verify the dashboard page loads
26
- 6. Verify the user's name appears in the header
27
- ```
28
-
29
- Run it:
30
-
31
- ```bash
32
- zibby test test-specs/login.txt
33
- ```
34
-
35
- ## From Cloud Test Cases
36
-
37
- After running analysis on a Jira ticket in the dashboard:
38
-
39
- ```bash
40
- zibby test --sources TC001,TC002 --execution abc-123-def --sync
41
- ```
42
-
43
- ## What Happens During a Run
44
-
45
- The default workflow executes three nodes:
46
-
47
- 1. **Preflight** — AI extracts a title and assertions from the test spec
48
- 2. **Execute Live** — AI drives a Playwright browser, following the test steps
49
- 3. **Generate Script** — AI produces a reusable `.spec.js` from recorded actions
50
-
51
- ```
52
- Test Spec → Preflight → Execute Live → Generate Script → Done
53
- ```
54
-
55
- ## Choosing an AI Agent
56
-
57
- ```bash
58
- # Cursor Agent (uses Cursor IDE or CURSOR_API_KEY)
59
- zibby test test-specs/login.txt --agent cursor
60
-
61
- # Claude (uses ANTHROPIC_API_KEY)
62
- zibby test test-specs/login.txt --agent claude
63
-
64
- # Codex (uses OPENAI_API_KEY + codex CLI)
65
- zibby test test-specs/login.txt --agent codex
66
- ```
67
-
68
- ## Headless Mode
69
-
70
- Run without a visible browser (for CI/CD):
71
-
72
- ```bash
73
- zibby test test-specs/login.txt --headless
74
- ```
75
-
76
- ## Cloud Sync
77
-
78
- Upload results to the [Zibby dashboard](https://zibby.app):
79
-
80
- ```bash
81
- # Upload to dashboard
82
- zibby test test-specs/login.txt --sync
83
-
84
- # Organize into a collection
85
- zibby test test-specs/login.txt --collection "Auth Tests" --sync
86
-
87
- # With subfolder
88
- zibby test test-specs/login.txt --collection "Auth Tests" --folder "Login" --sync
89
-
90
- # Local only (no upload)
91
- zibby test test-specs/login.txt --no-sync
92
- ```
93
-
94
- The `cloudSync` setting in `.zibby.config.js` sets the default. CLI flags override it.
95
-
96
- ## Running a Single Node
97
-
98
- Re-run just one step of the workflow (useful for debugging):
99
-
100
- ```bash
101
- # Re-execute the browser test (reusing previous session)
102
- zibby test test-specs/login.txt --node execute_live --session last
103
-
104
- # Only regenerate the Playwright script
105
- zibby test test-specs/login.txt --node generate_script --session last
106
- ```
107
-
108
- ## Custom Workflows
109
-
110
- If you've created a custom workflow in `.zibby/graph.js`:
111
-
112
- ```bash
113
- zibby test test-specs/login.txt --workflow QuickSmokeWorkflow
114
- ```
115
-
116
- Supported formats: `QuickSmokeWorkflow`, `QuickSmoke`, `quick-smoke`.
117
-
118
- ## Opening Results in Browser
119
-
120
- ```bash
121
- zibby test test-specs/login.txt --sync --open
122
- ```
123
-
124
- The `--open` flag opens the dashboard results page after upload.
125
-
126
- ## Debugging
127
-
128
- ```bash
129
- # Info-level logs
130
- zibby test test-specs/login.txt --verbose
131
-
132
- # Full debug logs
133
- zibby test test-specs/login.txt --debug
134
- ```
@@ -1,53 +0,0 @@
1
- ---
2
- sidebar_position: 3
3
- title: Backup & restore
4
- ---
5
-
6
- # Backup & restore
7
-
8
- `backup.sh` and `restore.sh` ship next to the compose file in your install directory (the installer drops them there). Together they let a dead VM — or a fat-fingered `docker compose down -v` — never mean "start over and reconnect 50 repos".
9
-
10
- ## What a backup contains
11
-
12
- One timestamped tarball holds the four things that together **are** your install:
13
-
14
- 1. **ScyllaDB volume** (`scylla-data`) — all metadata: integrations + their encrypted connect tokens, deployed agents (graph/env/schedule/prompts), repo→agent links, projects, members, access tokens, review-memory, execution history, stores.
15
- 2. **Object-store volume** — `seaweed-data` (SeaweedFS, the default) or `minio-data` on a legacy box. Run artifacts + per-node logs, datasets, marketplace icons. The active store is detected from `.env` (`ZIBBY_OBJECT_STORE`) / volume presence; `restore.sh` handles either.
16
- 3. **Sidecar data volumes** (`zibby-sidecar-*-data`) — content that exists **nowhere else** and cannot be re-derived: a knowledge base's indexed documents, a sidecar's encrypted OAuth token store. They are discovered by convention, so a sidecar you add later — including one you brought yourself — is included with no configuration. Skip them with `--skip-sidecar-data` (or `ZIBBY_BACKUP_SIDECAR_DATA=false` for scheduled backups) if the size is a real problem; those sidecars then restore **empty**, and `restore.sh` says so out loud.
17
- 4. **`.env`** — **critical**. It holds `ENCRYPTION_KEY`, without which every encrypted token/secret in ScyllaDB is unrecoverable, plus `JWT_SECRET`, the object-store root creds, and integration tokens. A ScyllaDB restore without the matching `ENCRYPTION_KEY` is useless, so `.env` travels **inside** the backup — which is also why the backup is highly sensitive.
18
-
19
- ## Take a backup
20
-
21
- ```bash
22
- cd zibby-selfhosted
23
- ./backup.sh # → ./backups/zibby-selfhosted-<timestamp>.tar.gz
24
- ./backup.sh --out /mnt/backups # write elsewhere
25
- ./backup.sh --encrypt # encrypt at rest with age or gpg (prompts for a passphrase)
26
- ./backup.sh --s3 s3://mybucket/zibby # also upload with YOUR aws creds (nothing goes to Zibby)
27
- ```
28
-
29
- Consistency + downtime: for a single-VM box the safe, simple approach wins — the script briefly **stops the writers** (control-plane + datastores + any running sidecars, whose single-writer file databases would otherwise be captured mid-write), `nodetool flush`es Scylla so the commitlog is on disk, tars the volumes read-only from a throwaway helper container, then restarts everything. Downtime is typically seconds to a few minutes (plus Scylla's ~1–2 min cold-boot on restart). A trap guarantees the stack restarts even if the backup errors out. Volume names are compose-project-prefixed and **discovered at runtime**, so it works regardless of your install-dir name.
30
-
31
- Copy the tarball **off the box** — it is your only recovery path, and it contains the encryption key.
32
-
33
- :::tip
34
- An upgrade takes its own safety snapshot into `<install-dir>/backups/upgrade-<id>/` before touching anything, and there is an `auto-backup.sh` you can cron for scheduled snapshots.
35
- :::
36
-
37
- ## Restore
38
-
39
- Onto the same box after data loss, or onto a brand-new VM:
40
-
41
- ```bash
42
- ./restore.sh zibby-selfhosted-<timestamp>.tar.gz # interactive confirm
43
- ./restore.sh backup.tar.gz --dir /opt/zibby --force # scripted
44
- ./restore.sh backup.age # decrypts first (needs age/gpg)
45
- ```
46
-
47
- `restore.sh` restores **both** data volumes and the `.env` (so the restored, encrypted data is readable). Volume names are discovered at runtime, or created if this is a fresh box.
48
-
49
- :::danger Restore wipes current data
50
- Restoring **overwrites** this install's volumes. The script refuses to run without an explicit confirmation (interactive `y/N`, or `--force`).
51
- :::
52
-
53
- A MinIO-era bundle restored onto a current (post-SeaweedFS-flip) release comes up on the legacy MinIO override; the next installer run auto-migrates it to SeaweedFS — see [Storage](./storage).
@@ -1,136 +0,0 @@
1
- ---
2
- sidebar_position: 5
3
- title: Custom (BYO) sidecars
4
- ---
5
-
6
- # Bring your own sidecar
7
-
8
- A **sidecar** is a long-lived service container the box runs next to your
9
- agents — the platform's own vector-KB engine and OAuth token broker ship this
10
- way. On a self-hosted box you can also register **your own** sidecar image.
11
- That is the standard way to add a custom resident service to the platform —
12
- most commonly a **thin MCP server that wraps an internal REST API**, so your
13
- agents and the chat Copilot can call systems that only exist inside your
14
- network.
15
-
16
- No marketplace entry, no platform code change, no third-party registry: you
17
- push a `docker save` tarball to your own box, the box pins its sha256, and the
18
- sidecar flows through the exact same launch/health/idle-reap machinery as the
19
- built-ins.
20
-
21
- :::tip Wrapping a REST API? Try the OpenAPI bridge agent first.
22
-
23
- If all you need is "my agents should be able to call this internal REST API",
24
- you probably do **not** need to build an image at all. Deploy the **OpenAPI MCP
25
- Bridge** agent from the marketplace, put your API's spec URL in its **Env** tab,
26
- and every operation in the spec becomes an MCP tool — no code, no upload.
27
-
28
- ```json
29
- OPENAPI_APIS = {"billing":{"specUrl":"https://internal/v2/api-docs","root":"https://internal"}}
30
- ```
31
-
32
- It is a marketplace agent that carries its own sidecar, so deploying it is the
33
- whole installation. Build your own image (below) when you need logic the bridge
34
- can't express — a non-HTTP protocol, a stateful session, custom auth.
35
- :::
36
-
37
- ## Push one
38
-
39
- ```bash
40
- # 1. Build your image however you like; it must serve HTTP on one port.
41
- docker build -t report-mcp:0.1.0 .
42
- docker save report-mcp:0.1.0 | gzip > report-mcp.tar.gz
43
-
44
- # 2. Push it to the box. Needs an OPERATOR access token
45
- # (a restricted per-user token is refused).
46
- zibby sidecar push report-mcp.tar.gz \
47
- --name report-mcp --port 8080 \
48
- --health-path /health \
49
- --warm \
50
- --api-url https://your-box.example.com --token $OPERATOR_PAT
51
- ```
52
-
53
- `push` streams the tarball up (`POST /selfhost/sidecars/artifacts`), then
54
- registers the declaration (`PUT /selfhost/sidecars/report-mcp`) pinned to the
55
- sha256 the server computed from the uploaded bytes. The image is verified and
56
- `docker load`ed **before** anything is persisted — a corrupt or tampered
57
- tarball can never become a registered sidecar.
58
-
59
- Manage what's registered:
60
-
61
- ```bash
62
- zibby sidecar list # custom sidecars + the reserved built-in names
63
- zibby sidecar remove <name> # unregister + stop (image reclaimed; data volume kept) (data volumes are kept)
64
- ```
65
-
66
- ## Reach it
67
-
68
- - **From the control plane / the chat Copilot** — the container listens on the
69
- box's infra network at `http://zibby-sidecar-<name>:<port>`. Agent run
70
- containers deliberately **cannot** dial it directly (they are isolated on the
71
- run network); access is brokered by the platform.
72
- - **From a browser / third-party client** — every `--public-path` prefix is
73
- served at `https://<box-origin>/sidecars/<name>/<path>` through the
74
- control-plane reverse proxy. Anything not declared public stays
75
- infra-network-only.
76
-
77
- :::warning `--public-path` is opt-in, and most sidecars don't need it
78
- A declared public path is reachable **anonymously** — the platform adds no auth
79
- and no identity; your app owns that surface entirely. Agents and the chat
80
- Copilot reach a sidecar over the infra network, so an MCP server for them needs
81
- **no** public path. It exists for browser flows (an OAuth callback a provider
82
- redirects to). Registering one is therefore refused unless the box opts in with
83
- `SIDECAR_BYO_PUBLIC_PATHS=1` in its `.env` (then restart the control-plane).
84
- :::
85
-
86
- ### Wire it into the Copilot
87
-
88
- For the "internal API → MCP → chat" use case, register the sidecar and then
89
- attach its MCP endpoint to the Copilot (in chat):
90
-
91
- > connect yourself to the MCP at `http://zibby-sidecar-report-mcp:8080/mcp`
92
-
93
- The Copilot can also drive the whole flow itself with the owner-only tools
94
- `zibby_add_sidecar` (takes a downloadable `url` + pinned `sha256` instead of a
95
- file upload), `zibby_list_sidecars`, and `zibby_remove_sidecar`.
96
-
97
- ## Declaration reference
98
-
99
- | Flag | Meaning |
100
- |---|---|
101
- | `--name` | `[a-z][a-z0-9-]{1,30}`. Built-in names (`gbrain`, `pingcode`, …) are reserved. |
102
- | `--port` | The HTTP port your app listens on inside the container. Required. |
103
- | `--version` | Display/tag version (default: first 12 chars of the sha256). |
104
- | `--health-path` | GET path returning 2xx when ready (default `/health`). |
105
- | `--public-path` | Repeatable. Path prefix exposed through the public reverse proxy — **anonymously**. Disabled unless the box sets `SIDECAR_BYO_PUBLIC_PATHS=1`; see below. |
106
- | `--warm` | Keep it always-on (pre-warmed at boot, never idle-reaped). Default: on-demand launch + idle reap. |
107
- | `--env-key` | Repeatable. A box `.env` variable forwarded into the container when set. |
108
- | `--request-config-key` | Repeatable. Per-agent config: resolved from the **calling agent's encrypted Env bag** and sent per request — one shared container can serve N projects with different upstream credentials. |
109
- | `--data-path` | Container path persisted on a named volume (survives restarts; kept on remove). |
110
- | `--memory-bytes` | Container memory cap. |
111
-
112
- ## What the box guarantees
113
-
114
- - **sha256-pinned, always.** The tarball's hash is computed server-side at
115
- upload and re-verified on every subsequent load — including the re-fetch
116
- path: the verified tarball is kept in the box's own object store, so if the
117
- docker image is ever pruned it is restored automatically without a re-upload.
118
- - **Operator-only surface.** Upload/register/remove require the box
119
- owner/admin token; restricted per-user tokens get `403`.
120
- - **Isolation unchanged.** Custom sidecars get the same placement as built-in
121
- ones: infra network, brokered access, no host mounts — the declaration
122
- cannot ask for privileged options.
123
- - **No shadowing, at BOTH layers.** A custom sidecar can never take over a
124
- built-in sidecar's *name*, and its package can never take over a platform or
125
- another sidecar's *image tag*: the archive's manifest is read **before**
126
- `docker load` runs, and a package declaring `zibby-*` or another sidecar's
127
- image is refused outright — even if that image isn't on the box yet.
128
- - **Uninstall reclaims storage.** Removing a sidecar deletes its image; a
129
- superseded image is reclaimed when you push a new version. The declared data
130
- volume is **kept** (it holds your content) — the response tells you its name,
131
- and `zibby sidecar remove <name> --purge-data` deletes it deliberately.
132
-
133
- :::note Cloud
134
- BYO sidecars are **self-host only** today. On cloud, wrap an external API as a
135
- remote MCP server and attach it with `zibby_add_mcp` instead.
136
- :::
@@ -1,75 +0,0 @@
1
- ---
2
- sidebar_position: 1
3
- title: Self-host overview
4
- ---
5
-
6
- # Operating a self-hosted box
7
-
8
- Run the entire Zibby agent platform on **one host**, inside your own VPC, with `docker compose`. This section is the **operator runbook** — for the person who holds the box's owner/admin token and is responsible for install, upgrade, backup, and storage. It is not the agent-authoring guide (that's the rest of these docs, and `zibby init` in your repo).
9
-
10
- A self-hosted box is the **same codebase as cloud**, assembled differently: the control plane is `backend/server.js` (the Express server that wraps every Lambda handler, including the MCP server), the run executor is the host Docker engine instead of Fargate, and the datastore / object store are ScyllaDB Alternator + SeaweedFS instead of DynamoDB + S3.
11
-
12
- ## Install
13
-
14
- One command, on a Linux/macOS host with **Docker Engine + Compose v2**:
15
-
16
- ```bash
17
- curl -fsSL https://dl.zibby.app/selfhosted/latest/install.sh | bash
18
- ```
19
-
20
- The installer downloads a signed release (a `docker save` image tarball + compose file + the marketplace-template bundle), **verifies it** (an Ed25519 signature over `SHA256SUMS`, then a per-file checksum — it fails closed on any mismatch and never runs unverified bytes), `docker load`s the images, and `docker compose up -d`s the stack. It is idempotent — re-running it is how you upgrade in place, and cached downloads are checksum-verified.
21
-
22
- :::warning Compose v2 is required
23
- `apt install docker.io` alone gives you the engine but **not** Compose v2. Install both — the installer aborts with a link if `docker compose version` fails. On a headless box also make sure `openssl` is present (the installer needs it to verify the release signature).
24
- :::
25
-
26
- ### What comes up
27
-
28
- The stack is a handful of containers on the `zibby-selfhosted` docker network:
29
-
30
- | Container | Role |
31
- |---|---|
32
- | `zibby-control-plane` | the long-lived API + `/mcp` + `/hooks/*` server + the dashboard; drives the host Docker engine as the run executor over the mounted `docker.sock` |
33
- | `zibby-scylla` | **ScyllaDB Alternator** (`:8000`) — speaks the DynamoDB API, so the backend is unchanged. All metadata: integrations + encrypted tokens, deployed agents, projects, members, executions, stores. |
34
- | `zibby-minio` | the **object store** (`:9000`) for run sources + artifacts + logs. As of 0.1.126 this is **SeaweedFS** (the service keeps the historical name `minio` so every `S3_ENDPOINT` consumer is unchanged). Legacy boxes still run MinIO until they auto-migrate — see [Storage](./storage). |
35
- | `zibby-bootstrap` | one-shot: provisions tables + buckets, syncs all marketplace templates, seeds the default project + token, then exits. |
36
- | run containers | one ephemeral `zibby-agent` container per run, destroyed on exit — the same per-run isolation Fargate gives cloud. |
37
-
38
- The control plane mounts `/var/run/docker.sock` — that socket **is** the run executor (no docker-in-docker).
39
-
40
- ### After install
41
-
42
- The installer prints the access info and the seeded credential:
43
-
44
- - **Dashboard** — `http://<host>:3001/` (and `/ui`). The self-host dashboard is a single vanilla HTML page served by the control plane; the cloud React app is not shipped on-prem.
45
- - **API + CLI** — `http://<host>:3001`. Set `ZIBBY_API_URL` + `ZIBBY_API_KEY=<apiToken>` (the seeded `zby_…` project token printed at the end, or recover it with `docker logs zibby-bootstrap`).
46
- - **MCP** — `http://<host>:3001/mcp`, `Authorization: Bearer <PAT>`. MCP needs a **user PAT** (`zby_pat_…`), minted with `zibby self-host token` — not the project token.
47
- - **Health** — `http://<host>:3001/health`.
48
-
49
- AI model keys are **per-project**, set in the dashboard (Settings → AI); the deploy modal collects a missing one. The box comes up keyless — there is deliberately no box-global model credential.
50
-
51
- ## Install knobs
52
-
53
- Set these as environment variables or flags on the install command (they persist into `.env`, which is the runtime source of truth and is **reused untouched on upgrade**):
54
-
55
- | Knob | Default | Effect |
56
- |---|---|---|
57
- | `--dir <path>` / `ZIBBY_DIR` | `./zibby-selfhosted` | install + data directory |
58
- | `--url <u>` / `ZIBBY_DIST_URL` | `https://dl.zibby.app/selfhosted/latest` | release base URL |
59
- | `--reconfigure` | — | re-run the external-URL / domain prompt |
60
- | `CONTROL_PLANE_PORT` (`.env`) | `3001` | host port the control plane binds |
61
- | `CONTROL_PLANE_BIND` (`.env`) | `0.0.0.0` | set to `127.0.0.1` to restrict the dashboard/API to loopback and front it with the bundled Caddy TLS proxy |
62
- | `ZIBBY_EXTERNAL_URL` (`.env`/prompt) | unset | a domain → automatic Let's Encrypt HTTPS; an IP → self-signed HTTPS; `http://…` → plain HTTP via Caddy. Blank → plain HTTP on `CONTROL_PLANE_PORT`. |
63
- | `MAX_CONCURRENT_RUNS` (`.env`) | `5` | run containers executing in parallel; the rest queue |
64
- | `ZIBBY_SKIP_OBJECT_STORE_MIGRATION` | `0` | `1` opts a legacy MinIO box out of the auto-migration to SeaweedFS |
65
-
66
- :::warning Plain-HTTP default binds in cleartext
67
- With no external URL the dashboard/API bind to `0.0.0.0:3001` unencrypted — the operator token and login passcode cross the network in the clear. Either set `ZIBBY_EXTERNAL_URL` (→ auto-TLS via Caddy), firewall the port to trusted IPs, or set `CONTROL_PLANE_BIND=127.0.0.1`.
68
- :::
69
-
70
- ## The runbook
71
-
72
- - **[Upgrade & rollback](./upgrade)** — check the running version, the owner-only remote upgrade, and how auto-rollback works.
73
- - **[Backup & restore](./backup-restore)** — snapshot the whole install to one tarball; restore onto the same or a fresh box.
74
- - **[Storage](./storage)** — the object store, and the automatic MinIO → SeaweedFS migration.
75
- - **[Troubleshooting](./troubleshooting)** — reading logs and the failure signatures worth recognizing.
@@ -1,50 +0,0 @@
1
- ---
2
- sidebar_position: 4
3
- title: Storage
4
- ---
5
-
6
- # Storage
7
-
8
- A self-hosted box keeps two stateful stores, both as single-box containers on named docker volumes:
9
-
10
- - **ScyllaDB Alternator** (`zibby-scylla`, volume `scylla-data`) — the DynamoDB-compatible metadata store. All accounts, projects, agents, integrations, executions, and stores live here.
11
- - **Object store** (`zibby-minio`, volume `seaweed-data` or legacy `minio-data`) — run sources, artifacts, per-node logs, datasets, and marketplace icons, over the S3 API.
12
-
13
- Both survive `docker compose restart` / `down` (they're on named volumes) but **not** `down -v`. Back them up — see [Backup & restore](./backup-restore).
14
-
15
- ## MinIO → SeaweedFS
16
-
17
- MinIO's upstream was archived in 2026-04, so as of **0.1.126** the object store is **SeaweedFS**. The service is still *named* `minio` in compose and DNS so every `S3_ENDPOINT` consumer is unchanged — only the engine behind it changed. (Full evaluation: `strategy/object-store-replacement.md` in the root repo.)
18
-
19
- - **New installs** come up on SeaweedFS (volume `seaweed-data`). Nothing to do.
20
- - **Upgrades from a MinIO box auto-migrate.** When the installer upgrades a legacy (pre-flip, `minio-data`) box, it keeps MinIO running for that `up`, then — once the upgraded stack is healthy — runs the migration automatically.
21
-
22
- ### How the auto-migration works
23
-
24
- It is designed to **auto-attempt, never auto-brick** — MinIO's data is never modified, so every failure path rolls back to a working box:
25
-
26
- 1. **backup** — a full `backup.sh` snapshot first; it refuses to migrate without one.
27
- 2. **quiesce** — stop the S3 writers (control-plane + copilot-runtime) so the copy is consistent.
28
- 3. **copy + verify** — `migrate-object-store.sh` mirrors every bucket into a throwaway SeaweedFS container writing to the `seaweed-data` volume, then verifies per-bucket **object count + total bytes must match exactly**, plus a sha256 comparison of sampled objects (always including the largest). It never deletes anything.
29
- 4. **flip** — record `ZIBBY_OBJECT_STORE=seaweedfs` in `.env` and bring up the default (SeaweedFS) stack.
30
- 5. **health-gate** — the new control plane must become healthy **and** pass an authenticated S3 list against the migrated store.
31
- 6. **rollback** — any failure at any stage rolls back to MinIO (whose data was never touched) with a loud status line.
32
-
33
- The **old `minio-data` volume is retained** as a rollback safety net. After a few days of verified operation you can reclaim it:
34
-
35
- ```bash
36
- docker volume rm <project>_minio-data # irreversible — only after you trust SeaweedFS
37
- ```
38
-
39
- ### Controlling / retrying the migration
40
-
41
- - **Opt out** (stay on MinIO for now): run the installer with `ZIBBY_SKIP_OBJECT_STORE_MIGRATION=1`.
42
- - **Retry** after a failed auto-migration: re-run the installer, or manually —
43
- ```bash
44
- cd zibby-selfhosted
45
- docker compose stop control-plane copilot-runtime
46
- bash migrate-object-store.sh --yes
47
- # then flip: set ZIBBY_OBJECT_STORE=seaweedfs + COMPOSE_FILE=docker-compose.yml in .env
48
- docker compose -f docker-compose.yml up -d
49
- ```
50
- - **MinIO emergency fallback image** (if a legacy box lost its local MinIO image): the pinned tarball is at `https://dl.zibby.app/selfhosted/infra/` — verify the `.sha256`, then `docker load -i`. The box only ever pulls images from our own CDN, never a third-party registry.
@@ -1,48 +0,0 @@
1
- ---
2
- sidebar_position: 5
3
- title: Troubleshooting
4
- ---
5
-
6
- # Troubleshooting
7
-
8
- A self-hosted box is **one VM** — control-plane container + Scylla + object store + ephemeral run containers. When something is slow or failing, diagnose from the box's own logs; the reflex "it doesn't scale / the VM is overloaded" is almost always wrong (a `top` usually shows the box near-idle).
9
-
10
- ## First moves
11
-
12
- ```bash
13
- top # CPU / memory — usually reveals it's idle
14
- docker ps -a # are containers Up? did a run container Exit?
15
- docker logs zibby-control-plane --tail 100 # the executor + reaper + API
16
- docker logs zibby-bootstrap # first-boot template sync + the seeded token
17
- curl -s http://localhost:3001/health # control-plane health
18
- curl -s http://localhost:3001/selfhost/version # the running release (baked, authoritative)
19
- ```
20
-
21
- To query the box's own API, use its `PROJECT_API_TOKEN` from `.env`:
22
-
23
- ```bash
24
- curl -H "Authorization: Bearer <zby_...>" \
25
- "http://localhost:3001/executions?projectId=selfhosted-project"
26
- ```
27
-
28
- A failed run's `reapedReason` field tells you why it was reaped. (The cloud Zibby MCP/CLI talks to the *cloud* backend — it cannot reach a self-host box; you need the box's own token.)
29
-
30
- ## Failure signatures worth recognizing
31
-
32
- **Runs "fail" with empty steps + `pull access denied for zibby-agent … repository does not exist`.** An agent-image tag mismatch after an upgrade — the executor is launching a tag the reused `.env` doesn't have. Fixed at the root in installer 0.1.53+; immediate unblock:
33
-
34
- ```bash
35
- docker tag zibby-agent:slim zibby-agent:local
36
- ```
37
-
38
- **The dashboard shows the new version but the API still behaves like the old one.** `docker compose up -d` can judge a service "up to date" when only the image *content* changed. The installer force-recreates the control-plane container to avoid this; if you hit it, re-run the installer (or `docker compose up -d --no-deps --force-recreate control-plane`). Because `/selfhost/version` reads the **baked** file, a genuine version mismatch is *visible* rather than hidden behind a lying env-var label.
39
-
40
- **Control plane won't become healthy right after boot.** Scylla cold-boots slowly (~1–2 min). The installer waits up to ~5 min; if it's still unhealthy, read `docker logs zibby-control-plane` — a broken data path shows up there, not in `top`.
41
-
42
- **An upgrade ended at `rolled_back`.** The box is safe (back on the prior version). Read `GET /selfhost/upgrade/status` `logTail` for the failing phase, and check `docker logs zibby-control-plane`. Re-attempt with a **pinned** version (see [Upgrade & rollback](./upgrade)).
43
-
44
- ## Where things live
45
-
46
- - **Data** — docker volumes `scylla-data` + the object store volume. Back them up (see [Backup & restore](./backup-restore)); they survive restarts but not `down -v`.
47
- - **`.env`** — the runtime source of truth (`JWT_SECRET`, `ENCRYPTION_KEY`, tokens, `MAX_CONCURRENT_RUNS`, `AGENT_IMAGE`). Reused untouched on upgrade.
48
- - **Upgrade backups + logs** — `<install-dir>/backups/upgrade-<id>/`.
@@ -1,81 +0,0 @@
1
- ---
2
- sidebar_position: 2
3
- title: Upgrade & rollback
4
- ---
5
-
6
- # Upgrade & rollback
7
-
8
- ## Check the running version
9
-
10
- ```bash
11
- curl -s http://<host>:3001/selfhost/version
12
- # {"version":"0.1.126"}
13
- ```
14
-
15
- This reads the **immutable file `/app/SELFHOST_VERSION`**, baked into the image at build time from `selfhosted/package.json`. It ships inside the image layer, so it **always matches the code actually running** and cannot be spoofed by a runtime env / compose `.env` override. That is the point: if an upgrade did not actually swap the control-plane image (a silent-upgrade failure), the baked file still reports the old version — you *see* the mismatch instead of trusting a lying label.
16
-
17
- The dashboard renders this same value as a small `v0.1.126` label in the sidebar, so the running release is always visible. (The dashboard has no upgrade button — upgrades are driven by the API endpoint or by re-running the installer, below.)
18
-
19
- The latest published version is in the release manifest:
20
-
21
- ```bash
22
- curl -s https://dl.zibby.app/selfhosted/latest/manifest.env
23
- # ZIBBY_SELFHOSTED_VERSION=0.1.126
24
- # IMAGE_TARBALL=zibby-selfhosted-0.1.126.tar.gz
25
- # ...
26
- ```
27
-
28
- ## Two ways to upgrade
29
-
30
- ### 1. Re-run the installer (in place)
31
-
32
- The installer is idempotent — re-running it pulls a newer release, verifies + `docker load`s the new images, re-pins `AGENT_IMAGE`, and **force-recreates** the control-plane container from the freshly loaded image:
33
-
34
- ```bash
35
- curl -fsSL https://dl.zibby.app/selfhosted/latest/install.sh | bash
36
- # or, from the install dir: cd zibby-selfhosted && bash install.sh
37
- ```
38
-
39
- Your `.env` (JWT_SECRET, ENCRYPTION_KEY, tokens) and data volumes are preserved.
40
-
41
- ### 2. The owner-only remote upgrade (recommended)
42
-
43
- `POST /selfhost/upgrade` runs a **safe, self-driving upgrade** with an automatic backup and rollback — you can trigger it remotely and it survives the control-plane restart. It is **owner-only**: the endpoint requires the workspace-owner PAT/session or the admin project token (a run token or a non-owner member is refused `403`).
44
-
45
- ```bash
46
- curl -s -X POST http://<host>:3001/selfhost/upgrade \
47
- -H "Authorization: Bearer <owner-or-admin-token>" \
48
- -H "Content-Type: application/json" \
49
- -d '{"version":"0.1.126"}'
50
- # → 202 { "upgradeId": "...", "phase": "starting", ... }
51
- ```
52
-
53
- Because a container can't recreate itself, the control plane launches a **detached helper** container (with the host docker socket) that outlives the control-plane bounce and drives the upgrade end to end:
54
-
55
- 1. **backup** — flush Scylla, tar the Scylla + object-store volumes, copy `.env`, into `<install-dir>/backups/upgrade-<id>/`.
56
- 2. **anchor** — pin the current control-plane image as `zibby-control-plane:rollback-<id>` so the new `docker load` can't garbage-collect the rollback target.
57
- 3. **download → install** — fetch the target release's `install.sh` (Ed25519 + checksum verified), `docker load`, re-pin `AGENT_IMAGE`, force-recreate.
58
- 4. **healthcheck** — poll the new control plane until container-healthy **and** `GET /selfhost/version` returns the target version.
59
- 5. **rollback** — on an unhealthy new control plane (or an installer that exited non-zero), re-tag the anchor back to `:local` and force-recreate → back to the working version.
60
-
61
- Poll the status (owner-only, read-only):
62
-
63
- ```bash
64
- curl -s http://<host>:3001/selfhost/upgrade/status \
65
- -H "Authorization: Bearer <owner-or-admin-token>"
66
- # { "upgrade": { "phase": "install", ... }, "logTail": "...", "history": [...] }
67
- ```
68
-
69
- Phases progress `starting → backup → anchor → download → install → healthcheck → success`. A failed upgrade ends at `rolled_back` (the box is back on the prior, healthy version) or, if even rollback fails, `rollback_failed` (manual restore needed — the backup dir and rollback-anchor tag are named in the status). The control plane restarts mid-upgrade, so the poll may briefly fail to connect — that is expected.
70
-
71
- :::warning Pin the exact version, not `latest`
72
- `{"version":"latest"}` resolves through the CDN, which can serve a **stale cached** `latest` for a while after a publish. For a deterministic upgrade, pass the exact version you verified against `manifest.env` (e.g. `{"version":"0.1.126"}`). The helper also treats "healthy but the version didn't advance to the pinned target" as a failure and rolls back — a guard you only get with a pinned version.
73
- :::
74
-
75
- ## If a run fails with empty steps after an upgrade
76
-
77
- Symptom: reviews/runs "fail" with `steps: []` and the control-plane log shows `pull access denied for zibby-agent … repository does not exist`. That is an **agent-image tag mismatch** — the executor is trying to launch a tag the reused `.env` points at but the tarball loaded a different one. Installer 0.1.53+ fixes this at the root (it pins `AGENT_IMAGE` to the loaded tag and aliases `zibby-agent:local`). Immediate operator unblock:
78
-
79
- ```bash
80
- docker tag zibby-agent:slim zibby-agent:local
81
- ```