@zibby/skills 0.2.22 → 0.2.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/browser.js +3 -10
- package/dist/code-scan.d.ts +1 -1
- package/dist/index.js +81 -88
- package/dist/jira.d.ts +75 -9
- package/dist/jira.js +4 -4
- package/dist/package.json +1 -1
- package/dist/report.d.ts +51 -276
- package/dist/trackers/index.js +12 -12
- package/dist/trackers/jira-adapter.js +11 -11
- package/package.json +1 -1
- package/docs/analysis.md +0 -109
- package/docs/apps/agent-ops.md +0 -130
- package/docs/apps/auth.md +0 -158
- package/docs/apps/deploy.md +0 -207
- package/docs/apps/goal-mode.md +0 -175
- package/docs/apps/index.md +0 -149
- package/docs/apps/managing.md +0 -121
- package/docs/cli-reference.md +0 -494
- package/docs/cloning-repositories.md +0 -285
- package/docs/cloud/bundles.md +0 -92
- package/docs/cloud/dedicated-egress.md +0 -140
- package/docs/cloud/editing-prompts.md +0 -79
- package/docs/cloud/env-vars.md +0 -144
- package/docs/cloud/limits.md +0 -81
- package/docs/cloud/logs.md +0 -104
- package/docs/cloud/triggering.md +0 -128
- package/docs/concepts/agents.md +0 -112
- package/docs/concepts/designing-agents.md +0 -247
- package/docs/concepts/graph.md +0 -83
- package/docs/concepts/sessions.md +0 -70
- package/docs/concepts/skills.md +0 -86
- package/docs/concepts/state.md +0 -106
- package/docs/concepts/sub-graphs.md +0 -227
- package/docs/custom-workflows.md +0 -358
- package/docs/get-started/deploy.md +0 -75
- package/docs/get-started/install.md +0 -60
- package/docs/get-started/run-locally.md +0 -94
- package/docs/get-started/trigger-and-logs.md +0 -90
- package/docs/get-started/use-from-agents.md +0 -153
- package/docs/get-started/your-first-workflow.md +0 -69
- package/docs/getting-started.md +0 -108
- package/docs/installation.md +0 -127
- package/docs/integrations/github.md +0 -73
- package/docs/integrations/gitlab.md +0 -43
- package/docs/integrations/jira.md +0 -71
- package/docs/integrations/lark.md +0 -41
- package/docs/integrations/linear.md +0 -43
- package/docs/integrations/notion.md +0 -33
- package/docs/integrations/plane.md +0 -46
- package/docs/integrations/sentry.md +0 -42
- package/docs/integrations/slack.md +0 -33
- package/docs/intro.md +0 -86
- package/docs/legacy/test-automation.md +0 -111
- package/docs/packages/agent-workflow.md +0 -88
- package/docs/packages/cli.md +0 -73
- package/docs/packages/core.md +0 -72
- package/docs/packages/mcp-browser.md +0 -110
- package/docs/packages/mcp-cli.md +0 -176
- package/docs/packages/memory.md +0 -223
- package/docs/packages/skills.md +0 -218
- package/docs/packages/ui-memory.md +0 -245
- package/docs/recipes/bug-autofix.md +0 -85
- package/docs/recipes/github-ai-scout.md +0 -61
- package/docs/recipes/index.md +0 -68
- package/docs/recipes/pipeline-supervisor.md +0 -57
- package/docs/recipes/sentry-triage.md +0 -93
- package/docs/recipes/test.md +0 -190
- package/docs/reviewing-results.md +0 -114
- package/docs/running-tests.md +0 -134
- package/docs/self-host/backup-restore.md +0 -53
- package/docs/self-host/custom-sidecars.md +0 -136
- package/docs/self-host/index.md +0 -75
- package/docs/self-host/storage.md +0 -50
- package/docs/self-host/troubleshooting.md +0 -48
- package/docs/self-host/upgrade.md +0 -81
- package/docs/skills/browser.md +0 -97
- package/docs/skills/chat-memory.md +0 -122
- package/docs/skills/core-tools.md +0 -80
- package/docs/skills/function-skill.md +0 -93
- package/docs/skills/github.md +0 -91
- package/docs/skills/index.md +0 -46
- package/docs/skills/jira.md +0 -99
- package/docs/skills/lark.md +0 -85
- package/docs/skills/memory.md +0 -92
- package/docs/skills/sentry.md +0 -80
- package/docs/skills/slack.md +0 -89
- package/docs/tests/memory.md +0 -131
- package/docs/triggering-workflows.md +0 -552
- package/docs/workflow-artifact-layout-evaluation.md +0 -119
- package/docs/workflow.md +0 -558
|
@@ -1,85 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 4
|
|
3
|
-
title: Bug-autofix agent
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# `bug-autofix` — ticket → fix PR → tracker writeback
|
|
7
|
-
|
|
8
|
-
The flagship orchestrator agent. It polls a tracker for new bugs, classifies each one, opens a fix PR for the autofixable ones, and writes the result back to the tracker — chaining three reusable building-block agents via **sub-graph dispatch**.
|
|
9
|
-
|
|
10
|
-
```
|
|
11
|
-
poll (this graph)
|
|
12
|
-
↓ found a ticket?
|
|
13
|
-
triage → sub-graph: ticket-triage → { severity, shouldAutofix, summary }
|
|
14
|
-
↓ severity ≥ AUTOFIX_MIN_SEVERITY AND shouldAutofix AND repo configured?
|
|
15
|
-
├─ yes → code_fix → sub-graph: code-fix → { pr_url, branch }
|
|
16
|
-
│ ↓
|
|
17
|
-
└─ no ───────────────────────────────────────────┐
|
|
18
|
-
↓
|
|
19
|
-
writeback → sub-graph: tracker-writeback (runs on BOTH branches)
|
|
20
|
-
↓
|
|
21
|
-
END
|
|
22
|
-
```
|
|
23
|
-
|
|
24
|
-
High-severity, concrete, autofixable bugs get a fix PR opened and the Jira ticket moved to *In Review*. Everything else (noise, vague, too-big, below threshold, or no repo configured) is triaged and a human is notified — no PR.
|
|
25
|
-
|
|
26
|
-
## The three building-block agents
|
|
27
|
-
|
|
28
|
-
`bug-autofix` is an orchestrator: each step is a separate, independently deployable agent it dispatches as a sub-graph. Deploy all four in the same project.
|
|
29
|
-
|
|
30
|
-
| Step | Building block | Input | Output |
|
|
31
|
-
|---|---|---|---|
|
|
32
|
-
| `triage` | `ticket-triage` | `{ ticket }` | severity (`CRITICAL…NOISE`), `shouldAutofix`, summary |
|
|
33
|
-
| `code_fix` | `code-fix` | `{ ticket, repo }` | `{ pr_url, branch }` — clones, fixes with an inline test-gate, opens a PR |
|
|
34
|
-
| `writeback` | `tracker-writeback` | `{ ticket, pr_url?, branch?, result }` | transitions the issue, comments the PR link, posts to Slack/Lark |
|
|
35
|
-
|
|
36
|
-
Each block is usable on its own — `ticket-triage` is a fine standalone classifier; `code-fix` is a standalone clone→fix→PR agent.
|
|
37
|
-
|
|
38
|
-
## Sub-graph dispatch
|
|
39
|
-
|
|
40
|
-
A child step is declared with `{ workflow }` on the node:
|
|
41
|
-
|
|
42
|
-
```js
|
|
43
|
-
graph.addNode('triage', {
|
|
44
|
-
workflow: 'ticket-triage', // child deploy slug
|
|
45
|
-
input: (state) => ({ ticket: state.poll.ticket }),
|
|
46
|
-
output: 'classify', // dot-path on the child's final state
|
|
47
|
-
});
|
|
48
|
-
```
|
|
49
|
-
|
|
50
|
-
The engine spawns the child as a **separate execution** (its own run row + Fargate task, or in-process when runtime tags match), links it to the parent for the Activity-tab tree and cancellation cascade, polls until terminal, and extracts `output` back into the parent state under the node name.
|
|
51
|
-
|
|
52
|
-
## Trigger
|
|
53
|
-
|
|
54
|
-
Cron poll (default) or per-ticket webhook. Each run processes ONE ticket; the next tick/webhook handles the next.
|
|
55
|
-
|
|
56
|
-
```json
|
|
57
|
-
{ "jql": "issuetype = Bug AND statusCategory != Done ORDER BY updated DESC" } // cron
|
|
58
|
-
{ "ticketKey": "PROJ-123" } // webhook
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
## Config (ENV tab)
|
|
62
|
-
|
|
63
|
-
| Var | Meaning |
|
|
64
|
-
|---|---|
|
|
65
|
-
| `REPO_URL` | Repo the fix targets. Unset → no autofix, notify-only. |
|
|
66
|
-
| `REPO_NAME` | Short repo name (default: derived from `REPO_URL`). |
|
|
67
|
-
| `REPO_BRANCH` | Base branch (default `main`). |
|
|
68
|
-
| `AUTOFIX_MIN_SEVERITY` | Routing floor for code-fix (default `MEDIUM`). |
|
|
69
|
-
|
|
70
|
-
Plus the children's own config: Jira connected (triage/poll/writeback), GitHub connected (code-fix), and `SLACK_CHANNEL` / `LARK_RECEIVE_ID` (writeback).
|
|
71
|
-
|
|
72
|
-
## Scope (v1)
|
|
73
|
-
|
|
74
|
-
- **In:** poll → triage → (autofix?) → code-fix → writeback, end-to-end to "PR opened + Jira written back".
|
|
75
|
-
- **Out:** deploy / verify / rollback; an auto re-dispatch loop; in-engine approval. The open PR is the human gate.
|
|
76
|
-
- **Tracker seam:** Jira is implemented; GitHub / Linear are extension points in the child templates.
|
|
77
|
-
|
|
78
|
-
## Deploy
|
|
79
|
-
|
|
80
|
-
```bash
|
|
81
|
-
zibby agent templates # browse the marketplace
|
|
82
|
-
zibby agent new bug-autofix -t bug-autofix # scaffold the orchestrator
|
|
83
|
-
# ...also scaffold + deploy ticket-triage, code-fix, tracker-writeback in the same project
|
|
84
|
-
zibby agent deploy bug-autofix
|
|
85
|
-
```
|
|
@@ -1,61 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 5
|
|
3
|
-
title: GitHub AI scout
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# `github-ai-scout` — daily AI-project radar
|
|
7
|
-
|
|
8
|
-
A daily scout agent that finds **new/trending AI projects on GitHub**, scores them against **your** rubric with an LLM, and posts a shortlist to Slack for a human to review. It proposes — it never stars, forks, or adds anything.
|
|
9
|
-
|
|
10
|
-
```
|
|
11
|
-
scan (this graph) → GitHub search REST API: your query + created:> + stars:>=
|
|
12
|
-
↓
|
|
13
|
-
score (LLM) → rank + filter candidates against your rubric → shortlist
|
|
14
|
-
↓
|
|
15
|
-
digest (this graph) → render a report-object → sub-graph: notify-slack
|
|
16
|
-
↓
|
|
17
|
-
END
|
|
18
|
-
```
|
|
19
|
-
|
|
20
|
-
Everything that defines *what* it scouts is a deploy-time input — the search query, the recency/star thresholds, and the scoring rubric. Point it at a different topic and a different rubric and it scouts that domain instead.
|
|
21
|
-
|
|
22
|
-
## Inputs
|
|
23
|
-
|
|
24
|
-
| Input | Default | What it does |
|
|
25
|
-
|---|---|---|
|
|
26
|
-
| `query` | `topic:ai topic:llm topic:agents topic:rag` | GitHub search query, WITHOUT date/stars filters (scan appends those). |
|
|
27
|
-
| `daysBack` | `30` | Only repos created within this many days. |
|
|
28
|
-
| `minStars` | `30` | Minimum stars. |
|
|
29
|
-
| `maxCandidates` | `30` | How many repos to fetch (max 100). |
|
|
30
|
-
| `shortlistSize` | `8` | How many to surface in Slack. |
|
|
31
|
-
| `rubric` | generic quality rubric | Plain-English scoring instruction — describe *your* taste. |
|
|
32
|
-
| `excludeRepos` | `[]` | `owner/repo` names to skip — dedup against repos you already track. |
|
|
33
|
-
| `slackChannel` | **required** | Channel id (`C012345`) or `#name` where the shortlist lands. |
|
|
34
|
-
|
|
35
|
-
## GitHub auth (optional)
|
|
36
|
-
|
|
37
|
-
The scan works **unauthenticated** for public search. To raise the rate limit, set a `GITHUB_TOKEN` env var on the project — the scan sends it as a Bearer token. No scopes needed; public read is enough.
|
|
38
|
-
|
|
39
|
-
## Slack setup
|
|
40
|
-
|
|
41
|
-
The digest dispatches to the **notify-slack** building-block agent (in-process sub-graph), which renders the shortlist as a native Block-Kit card. Deploy `notify-slack` in the same project, connect Slack, and set `slackChannel`.
|
|
42
|
-
|
|
43
|
-
## Trigger
|
|
44
|
-
|
|
45
|
-
Cron — typically **daily**. Each run is a fresh scan over the trailing `daysBack` window; use `excludeRepos` to keep the shortlist to NEW finds.
|
|
46
|
-
|
|
47
|
-
```json
|
|
48
|
-
{ "slackChannel": "#ai-radar" }
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
## What it does NOT do
|
|
52
|
-
|
|
53
|
-
It **proposes a shortlist for human review — never auto-adds.** No starring, no forking, no writing anywhere except the Slack post. A person decides what to do with each find.
|
|
54
|
-
|
|
55
|
-
## Deploy
|
|
56
|
-
|
|
57
|
-
```bash
|
|
58
|
-
zibby agent templates # browse the marketplace
|
|
59
|
-
zibby agent new github-ai-scout -t github-ai-scout # scaffold (also deploy notify-slack)
|
|
60
|
-
zibby agent deploy github-ai-scout
|
|
61
|
-
```
|
package/docs/recipes/index.md
DELETED
|
@@ -1,68 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 1
|
|
3
|
-
title: Agent Marketplace
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Agent Marketplace
|
|
7
|
-
|
|
8
|
-
Zibby ships a set of **ready-made agents** — production-ready automations you can deploy today. Each one is a real Zibby agent, eating its own dog food. Browse them, deploy with one command, and tune them to your project.
|
|
9
|
-
|
|
10
|
-
> An **Agent** is a deployed automation built on coding-agent CLIs. Build and ship one with the `zibby agent` CLI.
|
|
11
|
-
|
|
12
|
-
```
|
|
13
|
-
┌──────────────────┐
|
|
14
|
-
┌──────────────────►│ zibby agent new │ ◄── Build your own
|
|
15
|
-
│ │ zibby agent ... │
|
|
16
|
-
│ └──────────────────┘
|
|
17
|
-
│ ▲
|
|
18
|
-
│ │ uses the same primitives
|
|
19
|
-
│ │
|
|
20
|
-
│ ┌──────────────────────┐
|
|
21
|
-
│ Marketplace ────►│ bug-autofix │ ◄── Ticket → fix PR → writeback
|
|
22
|
-
│ agents built │ github-ai-scout │ ◄── Daily AI-project radar
|
|
23
|
-
│ on top of the │ pipeline-supervisor │ ◄── Zibby managing Zibby
|
|
24
|
-
│ same platform │ sentry-triage │ ◄── Incident routing
|
|
25
|
-
│ │ zibby test │ ◄── Browser testing
|
|
26
|
-
│ └──────────────────────┘
|
|
27
|
-
```
|
|
28
|
-
|
|
29
|
-
You don't have to use the marketplace. You can build whatever agent you want with `zibby agent new`. The marketplace just saves you from writing the obvious starter graphs for common cases.
|
|
30
|
-
|
|
31
|
-
## Available agents
|
|
32
|
-
|
|
33
|
-
| Agent | What it does | Best for |
|
|
34
|
-
|---|---|---|
|
|
35
|
-
| [Bug-autofix](./bug-autofix) | Polls a tracker, triages each bug, opens a fix PR for autofixable ones, writes the result back. Chains three reusable building-block agents (`ticket-triage` → `code-fix` → `tracker-writeback`). | End-to-end bug SDLC, automated remediation |
|
|
36
|
-
| [GitHub AI scout](./github-ai-scout) | Daily scan of new/trending AI projects on GitHub, LLM-scored against your rubric, shortlist posted to Slack | Tracking a fast-moving space without manual triage |
|
|
37
|
-
| [Pipeline supervisor](./pipeline-supervisor) | Watches the project's *other* agents, flags failing/slow ones, posts an improvement proposal to Slack/Lark | Zibby managing Zibby — agent fleet health |
|
|
38
|
-
| [Sentry triage](./sentry-triage) | Hourly: fetch unresolved Sentry issues, classify by severity, route via Slack/Lark (author DM + usergroup mention) | Automated incident routing without a human triager |
|
|
39
|
-
| [`zibby test`](./test) | Drives a browser via Cursor or Claude, runs assertions, generates a Playwright script + verification video | E2E test generation from plain-English specs |
|
|
40
|
-
|
|
41
|
-
Plus reusable **building-block agents** that the orchestrators compose via sub-graph dispatch — `ticket-triage`, `code-fix`, `tracker-writeback`, `notify-slack`, `notify-lark`, `notify-notion`. Each is independently deployable and usable on its own.
|
|
42
|
-
|
|
43
|
-
## Why a marketplace
|
|
44
|
-
|
|
45
|
-
Three reasons we ship ready-made agents alongside the platform:
|
|
46
|
-
|
|
47
|
-
1. **Proof of concept** — every marketplace agent IS a Zibby agent. If `bug-autofix` works, the platform works. You can see the actual graph definition and adapt it.
|
|
48
|
-
2. **Faster onboarding** — you don't need to design a full graph on day one. Deploy an agent, see the output, then build your own.
|
|
49
|
-
3. **Composable** — the orchestrators (`bug-autofix`) are built from smaller building-block agents dispatched as sub-graphs, so you can reuse the pieces in your own agents.
|
|
50
|
-
|
|
51
|
-
## Building your own agent
|
|
52
|
-
|
|
53
|
-
If you have an agent you'd want shipped as a built-in:
|
|
54
|
-
|
|
55
|
-
```bash
|
|
56
|
-
zibby agent new my-agent # scaffold
|
|
57
|
-
# ... build it out ...
|
|
58
|
-
zibby agent run my-agent # test locally
|
|
59
|
-
zibby agent deploy my-agent # ship to your cloud account
|
|
60
|
-
```
|
|
61
|
-
|
|
62
|
-
If it's broadly useful, we may pull it into the official marketplace. Open an issue or PR.
|
|
63
|
-
|
|
64
|
-
## Next
|
|
65
|
-
|
|
66
|
-
- **[Bug-autofix agent](./bug-autofix)** — the flagship orchestrator, walked through end-to-end
|
|
67
|
-
- **[Build your own agent](../get-started/your-first-workflow)** — scaffold and customize
|
|
68
|
-
- **[Concepts: graph](../concepts/graph)** — the primitives every agent is built on
|
|
@@ -1,57 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 6
|
|
3
|
-
title: Pipeline supervisor
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# `pipeline-supervisor` — Zibby managing Zibby
|
|
7
|
-
|
|
8
|
-
A scheduled supervisor agent that watches the project's *other* agents, finds the ones that are failing or slow, and posts a human-reviewable improvement proposal to Slack or Lark.
|
|
9
|
-
|
|
10
|
-
v1 is strictly **READ → PROPOSE → NOTIFY**. It never edits another agent's graph — that's the safe starting point. The auto-PATCH step is a deliberate TODO, not implemented.
|
|
11
|
-
|
|
12
|
-
```
|
|
13
|
-
scan_pipelines (deterministic + Zibby REST API, PAT-authed)
|
|
14
|
-
→ propose_improvements (LLM — one proposal per flagged agent)
|
|
15
|
-
→ notify (LLM + SKILLS.CHAT_NOTIFY — one review card)
|
|
16
|
-
```
|
|
17
|
-
|
|
18
|
-
If `scan_pipelines` flags nothing, the graph short-circuits straight to `notify` (which posts/skips without an LLM call on the proposer).
|
|
19
|
-
|
|
20
|
-
## How it reads other agents
|
|
21
|
-
|
|
22
|
-
A direct authed `GET /executions?projectId=<id>&limit=200` against the Zibby REST API (the same route the dashboard and remote MCP server use), carrying a **user personal access token** in `Authorization: Bearer`.
|
|
23
|
-
|
|
24
|
-
It must be a USER PAT (`zby_pat_…`), **not** the Fargate-injected `PROJECT_API_TOKEN`: every cross-agent read route requires a `userId` from the authorizer, and a project token carries none — so it 401s.
|
|
25
|
-
|
|
26
|
-
## Config (ENV tab)
|
|
27
|
-
|
|
28
|
-
Required:
|
|
29
|
-
|
|
30
|
-
- `ZIBBY_PAT` — user personal access token the supervisor reads executions with.
|
|
31
|
-
- `SLACK_CHANNEL` **or** `LARK_RECEIVE_ID` — where the review card goes.
|
|
32
|
-
|
|
33
|
-
Optional:
|
|
34
|
-
|
|
35
|
-
- `SUPERVISOR_PROJECT_ID` — project to supervise (defaults to the running project).
|
|
36
|
-
- `SLACK_MENTIONS` / `LARK_MENTIONS` — JSON array of mentions on the card.
|
|
37
|
-
|
|
38
|
-
## Input (per-run dials)
|
|
39
|
-
|
|
40
|
-
| Field | Default | Meaning |
|
|
41
|
-
|---|---|---|
|
|
42
|
-
| `lookbackHours` | 24 | Hours of execution history to scan |
|
|
43
|
-
| `minFailRate` | 0.4 | Flag an agent failing ≥ this fraction of recent runs |
|
|
44
|
-
| `targetWorkflowTypes` | — | Optional name filter (case-insensitive substring) |
|
|
45
|
-
| `maxPipelines` | 25 | Cap on distinct agents analyzed per run |
|
|
46
|
-
|
|
47
|
-
## Trigger
|
|
48
|
-
|
|
49
|
-
Cron — typically daily or hourly, depending on how active the supervised project is.
|
|
50
|
-
|
|
51
|
-
## Deploy
|
|
52
|
-
|
|
53
|
-
```bash
|
|
54
|
-
zibby agent templates # browse the marketplace
|
|
55
|
-
zibby agent new pipeline-supervisor -t pipeline-supervisor # scaffold
|
|
56
|
-
zibby agent deploy pipeline-supervisor
|
|
57
|
-
```
|
|
@@ -1,93 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 3
|
|
3
|
-
title: Sentry triage recipe
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# `sentry-triage` — agent-driven Sentry triage
|
|
7
|
-
|
|
8
|
-
An hourly Sentry triage agent that fetches unresolved issues, classifies them by severity, and **routes them to the right human**. Three nodes, end-to-end agent-driven, deployed from the marketplace in one click.
|
|
9
|
-
|
|
10
|
-
```
|
|
11
|
-
fetch_issues → classify → dispatch_alerts
|
|
12
|
-
(deterministic (LLM — (LLM —
|
|
13
|
-
+ Sentry API) severity) Slack/Lark, agent-driven routing)
|
|
14
|
-
```
|
|
15
|
-
|
|
16
|
-
## What it does
|
|
17
|
-
|
|
18
|
-
1. **fetch_issues** — calls Sentry's REST API for issues unresolved + unassigned + `lastSeen:-60m`. Hydrates each with `suspectCommits[]` (author email from Sentry's GitHub integration) for downstream routing.
|
|
19
|
-
2. **classify** — labels each issue `NOISE | LOW | MEDIUM | HIGH | CRITICAL` based on a configurable rubric (impact metric, surface area, payment paths, security tags). Skips below-threshold issues.
|
|
20
|
-
3. **dispatch_alerts** — the routing brain. Three layers of decisioning:
|
|
21
|
-
- **Free-form `DISPATCH_RULES`** in env (highest priority) — natural language like *"send to Sam for billing issues"*
|
|
22
|
-
- **Structured env vars** — `SLACK_CHANNEL`, `ROUTING_PREFER_AUTHOR`, `ROUTING_HIGH_SEVERITY_GROUP`
|
|
23
|
-
- **Defaults** — channel-only post, threshold `MEDIUM`
|
|
24
|
-
|
|
25
|
-
The agent uses [`slack_lookup_user_by_email`](../skills/slack), [`slack_list_usergroups`](../skills/slack), [`slack_search_users`](../skills/slack) (or the Lark equivalents) to resolve names → IDs, then `slack_post_message` / `lark_send_message` to deliver. Channel post, user DM, usergroup mention — same agent decides per-issue based on what you wrote in the rules.
|
|
26
|
-
|
|
27
|
-
## Deploy from the marketplace
|
|
28
|
-
|
|
29
|
-
```bash
|
|
30
|
-
zibby agent templates deploy sentry-triage --project <project-id>
|
|
31
|
-
```
|
|
32
|
-
|
|
33
|
-
Or via the dashboard: `/marketplace/workflows` → Sentry Triage → Deploy.
|
|
34
|
-
|
|
35
|
-
After deploy, configure ENV (Apps → agent → ENV tab):
|
|
36
|
-
|
|
37
|
-
| Env var | Required? | Default | What it does |
|
|
38
|
-
|---|---|---|---|
|
|
39
|
-
| `SLACK_CHANNEL` *or* `LARK_RECEIVE_ID` | Yes (one of) | — | Channel id (Slack `C…`) / chat id (Lark `oc_…`) for fallback posts |
|
|
40
|
-
| `SEVERITY_THRESHOLD` | No | `MEDIUM` | Skip anything below: `NOISE` / `LOW` / `MEDIUM` / `HIGH` / `CRITICAL` |
|
|
41
|
-
| `ROUTING_PREFER_AUTHOR` | No | `false` | If `true`, when a suspect commit author is known, DM them |
|
|
42
|
-
| `ROUTING_HIGH_SEVERITY_GROUP` | No | — | Slack usergroup handle (`@oncall`) mentioned on CRITICAL/HIGH |
|
|
43
|
-
| `SLACK_MENTIONS` *or* `LARK_MENTIONS` | No | `[]` | JSON array of mentions prepended on CRITICAL only |
|
|
44
|
-
| `DISPATCH_RULES` | No | — | Free-form natural-language override (see below) |
|
|
45
|
-
|
|
46
|
-
## DISPATCH_RULES — natural-language routing
|
|
47
|
-
|
|
48
|
-
When you set `DISPATCH_RULES`, the agent treats it as **authoritative**; the structured env vars become fallbacks for things the rules don't cover.
|
|
49
|
-
|
|
50
|
-
```
|
|
51
|
-
DISPATCH_RULES="
|
|
52
|
-
- CRITICAL bugs in /payment/ → DM Sam and post to #incidents
|
|
53
|
-
- HIGH severity → DM the suspect commit author if known, else post to #engineering
|
|
54
|
-
- Anything mentioning 'security' → also mention the @security usergroup
|
|
55
|
-
- Frontend bugs (zibby-frontend project) → only Sarah, never page on-call
|
|
56
|
-
- NOISE → skip entirely
|
|
57
|
-
"
|
|
58
|
-
```
|
|
59
|
-
|
|
60
|
-
The agent reads issue metadata (severity, message, tags, suspectCommit author email, project name) and applies rules in order. **Same rule + same issue is deterministic** — temperature 0, schema-enforced output, every dispatch records who got it and why under `dispatched[].recipient.{kind,id,label}`.
|
|
61
|
-
|
|
62
|
-
## Author-DM path
|
|
63
|
-
|
|
64
|
-
When `ROUTING_PREFER_AUTHOR=true` and Sentry has a `suspectCommits[0].author.email`:
|
|
65
|
-
|
|
66
|
-
```
|
|
67
|
-
1. agent reads issue.suspectCommits[0].authorEmail
|
|
68
|
-
2. → slack_lookup_user_by_email(email)
|
|
69
|
-
3a. ✓ returns {id, name} → slack_post_message(channel: <user-id>, text: …)
|
|
70
|
-
3b. ✗ users_not_found → channel fallback
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
Requires the [Sentry → GitHub integration](https://sentry.io/settings/integrations/github/) installed and Code Mappings configured. Without it, `suspectCommits[]` is empty and the agent falls back to channel-only routing automatically.
|
|
74
|
-
|
|
75
|
-
If you deployed your backend with `RELEASE_SHA` Sentry release-tracking on, suspect commits populate within ~minutes of new issues being created. (The platform-side wiring — `Sentry.init({release})` + `sentry-cli releases set-commits --auto` at deploy time — is what makes per-issue blame work; without it, every issue lands with `suspectCommits: []`.)
|
|
76
|
-
|
|
77
|
-
## Customize the prompts
|
|
78
|
-
|
|
79
|
-
Each node's prompt lives in its own module — fork the template, edit, redeploy:
|
|
80
|
-
|
|
81
|
-
```bash
|
|
82
|
-
zibby agent download <uuid>
|
|
83
|
-
# edit nodes/dispatch-node.js
|
|
84
|
-
zibby agent deploy ./sentry-triage # same UUID, new version
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
Or fork the whole template repo if you want long-term divergence — it's just a `@zibby/workflow-templates/sentry-triage/` directory in the published package.
|
|
88
|
-
|
|
89
|
-
## Cadence
|
|
90
|
-
|
|
91
|
-
Default: hourly cron, fires `sinceMinutes=60`. Change in the trigger config (Apps → agent → Triggers) — keep the SQL safe `since` between 5 and 1440 minutes (`inputSchema` enforces this).
|
|
92
|
-
|
|
93
|
-
→ Next: [`zibby test`](./test) (the browser-testing recipe) or [Build your own agent](../get-started/your-first-workflow).
|
package/docs/recipes/test.md
DELETED
|
@@ -1,190 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 2
|
|
3
|
-
title: Browser test recipe (zibby test)
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# `zibby test` — browser test recipe
|
|
7
|
-
|
|
8
|
-
The browser-test recipe takes a plain-English spec, drives a real browser via a coding agent (Cursor / Claude / Codex / Gemini), runs the assertions, and produces a Playwright script + verification video.
|
|
9
|
-
|
|
10
|
-
It's a worked example of what the Zibby platform does — every step is a regular agent node with Zod-validated handoff. You can read the source, fork it, or build your own variation.
|
|
11
|
-
|
|
12
|
-
## Quick start
|
|
13
|
-
|
|
14
|
-
```bash
|
|
15
|
-
# Inline spec
|
|
16
|
-
zibby test "Go to https://example.com and verify the title is 'Example Domain'"
|
|
17
|
-
|
|
18
|
-
# Spec file
|
|
19
|
-
zibby test test-specs/login.txt
|
|
20
|
-
|
|
21
|
-
# With a specific agent
|
|
22
|
-
zibby test test-specs/checkout.txt --agent claude
|
|
23
|
-
```
|
|
24
|
-
|
|
25
|
-
## What it produces
|
|
26
|
-
|
|
27
|
-
```
|
|
28
|
-
.zibby/output/sessions/<session-id>/
|
|
29
|
-
├── execute_live/
|
|
30
|
-
│ ├── result.json ← Zod-validated assertions + agent reasoning
|
|
31
|
-
│ └── browser-trace/ ← Playwright trace files
|
|
32
|
-
├── generate_script/
|
|
33
|
-
│ ├── result.json ← parsed script + metadata
|
|
34
|
-
│ └── generated.spec.js ← reusable Playwright test
|
|
35
|
-
└── video/
|
|
36
|
-
└── recording.webm ← visual verification
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
Open the session in [Zibby Studio](https://zibby.app/studio) to scrub through the run, swap the prompt, re-execute any node.
|
|
40
|
-
|
|
41
|
-
## The graph (this is just a Zibby agent)
|
|
42
|
-
|
|
43
|
-
Under the hood, `zibby test` is a 3-node graph:
|
|
44
|
-
|
|
45
|
-
```
|
|
46
|
-
┌──────────────┐ ┌──────────────────┐ ┌─────────────────┐
|
|
47
|
-
│ preflight │ → │ execute_live │ → │ generate_script │
|
|
48
|
-
│ │ │ │ │ │
|
|
49
|
-
│ extract │ │ agent drives │ │ produce │
|
|
50
|
-
│ assertions │ │ browser via MCP, │ │ Playwright │
|
|
51
|
-
│ from spec │ │ records video │ │ test file │
|
|
52
|
-
└──────────────┘ └──────────────────┘ └─────────────────┘
|
|
53
|
-
│ │ │
|
|
54
|
-
Zod out Zod out Zod out
|
|
55
|
-
(Assertions) (BrowserResult) (PlaywrightScript)
|
|
56
|
-
```
|
|
57
|
-
|
|
58
|
-
Each node is a real `WorkflowGraph` node. The agent in `execute_live` does its own tool loop (browser navigation, click, assertion checking) — Zibby just defines the contract.
|
|
59
|
-
|
|
60
|
-
## Customizing
|
|
61
|
-
|
|
62
|
-
**Use a different agent per run:**
|
|
63
|
-
```bash
|
|
64
|
-
zibby test test-specs/checkout.txt --agent claude # Claude Code
|
|
65
|
-
zibby test test-specs/checkout.txt --agent cursor # Cursor (default)
|
|
66
|
-
zibby test test-specs/checkout.txt --agent codex # OpenAI Codex
|
|
67
|
-
```
|
|
68
|
-
|
|
69
|
-
**Run only one node** (e.g. just regenerate the script from an existing run):
|
|
70
|
-
```bash
|
|
71
|
-
zibby test --session 1768974629717 --node generate_script
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
**Headless vs headed:**
|
|
75
|
-
```bash
|
|
76
|
-
zibby test test-specs/login.txt # headed (default — see the browser)
|
|
77
|
-
zibby test test-specs/login.txt --headless # headless mode (for CI)
|
|
78
|
-
```
|
|
79
|
-
|
|
80
|
-
**Cloud-stored test cases (run a saved execution):**
|
|
81
|
-
```bash
|
|
82
|
-
zibby test --sources <id1>,<id2> --execution <executionId>
|
|
83
|
-
```
|
|
84
|
-
|
|
85
|
-
## Test memory
|
|
86
|
-
|
|
87
|
-
The recipe ships with a learning loop: every run reads from and writes to a local Dolt DB at `.zibby/memory/.dolt/` (cross-spec, **per-domain**). Selectors that worked, page-element fingerprints, navigation transitions, and free-form insights are all persisted — the agent's 100th run on a site is sharper and cheaper than its first.
|
|
88
|
-
|
|
89
|
-
When `zibby test` runs and the DB exists, the agent gets 5 MCP tools auto-exposed:
|
|
90
|
-
|
|
91
|
-
- `memory_get_test_history` — recent runs (pass/fail/timing)
|
|
92
|
-
- `memory_get_selectors` — known selectors with stability metrics
|
|
93
|
-
- `memory_get_page_model` — page elements / roles / accessible names
|
|
94
|
-
- `memory_get_navigation` — known page-to-page transitions
|
|
95
|
-
- `memory_save_insight` — save observations (categories: `selector_tip | timing | navigation | workaround | flaky | general`). **Required at least once per run.**
|
|
96
|
-
|
|
97
|
-
```bash
|
|
98
|
-
zibby memory stats # what's in the DB
|
|
99
|
-
zibby memory cost # real LLM token spend per spec / per domain
|
|
100
|
-
zibby memory compact # prune old runs + GC
|
|
101
|
-
```
|
|
102
|
-
|
|
103
|
-
**Team sync.** Point the DB at a remote and teammates' learnings flow back to you on the next test run:
|
|
104
|
-
|
|
105
|
-
```bash
|
|
106
|
-
zibby memory remote add aws://my-bucket/team/proj/main # BYO (S3 / GCS / DoltHub / file:///)
|
|
107
|
-
zibby memory remote use --hosted # OR Zibby-managed S3 (signed-in only)
|
|
108
|
-
```
|
|
109
|
-
|
|
110
|
-
Or commit `memorySync.remote: 'hosted'` (or an `aws://` URL) into `.zibby.config.mjs` and `zibby init` auto-wires it for every teammate.
|
|
111
|
-
|
|
112
|
-
Auto-pull on test start, auto-push on test pass. Failing runs don't pollute team memory.
|
|
113
|
-
|
|
114
|
-
→ Full guide: [Test memory](../tests/memory). Schema and SDK: [`@zibby/ui-memory`](../packages/ui-memory).
|
|
115
|
-
|
|
116
|
-
## Forking the recipe
|
|
117
|
-
|
|
118
|
-
If the built-in recipe doesn't fit your case, scaffold a custom agent and copy the structure:
|
|
119
|
-
|
|
120
|
-
```bash
|
|
121
|
-
zibby agent new my-test-agent
|
|
122
|
-
```
|
|
123
|
-
|
|
124
|
-
Then in `graph.mjs`, define your own nodes:
|
|
125
|
-
|
|
126
|
-
```js
|
|
127
|
-
import { WorkflowGraph, z } from '@zibby/agent-workflow';
|
|
128
|
-
|
|
129
|
-
const AssertionsSchema = z.object({
|
|
130
|
-
assertions: z.array(z.string()),
|
|
131
|
-
baseUrl: z.string().url(),
|
|
132
|
-
});
|
|
133
|
-
|
|
134
|
-
const BrowserResultSchema = z.object({
|
|
135
|
-
passed: z.boolean(),
|
|
136
|
-
details: z.array(z.object({ assertion: z.string(), passed: z.boolean() })),
|
|
137
|
-
videoPath: z.string().optional(),
|
|
138
|
-
});
|
|
139
|
-
|
|
140
|
-
const graph = new WorkflowGraph();
|
|
141
|
-
|
|
142
|
-
graph.addNode('preflight', {
|
|
143
|
-
agent: 'claude',
|
|
144
|
-
prompt: ({ spec }) => `Extract assertions and base URL from: ${spec}`,
|
|
145
|
-
outputSchema: AssertionsSchema,
|
|
146
|
-
});
|
|
147
|
-
|
|
148
|
-
graph.addNode('execute_live', {
|
|
149
|
-
agent: 'cursor',
|
|
150
|
-
skills: ['browser'],
|
|
151
|
-
prompt: ({ preflight }) => `Navigate to ${preflight.baseUrl} and verify: ${preflight.assertions.join('; ')}`,
|
|
152
|
-
outputSchema: BrowserResultSchema,
|
|
153
|
-
});
|
|
154
|
-
|
|
155
|
-
graph.addEdge('preflight', 'execute_live');
|
|
156
|
-
graph.setEntryPoint('preflight');
|
|
157
|
-
|
|
158
|
-
export default graph;
|
|
159
|
-
```
|
|
160
|
-
|
|
161
|
-
That's the platform. The recipe is just a starter.
|
|
162
|
-
|
|
163
|
-
## CI/CD
|
|
164
|
-
|
|
165
|
-
```yaml
|
|
166
|
-
- name: Run Zibby test
|
|
167
|
-
env:
|
|
168
|
-
ZIBBY_USER_TOKEN: ${{ secrets.ZIBBY_USER_TOKEN }}
|
|
169
|
-
run: |
|
|
170
|
-
npx @zibby/cli test test-specs/checkout.txt --headless
|
|
171
|
-
```
|
|
172
|
-
|
|
173
|
-
For agents triggered remotely (rather than per-CI-run), use [`agent trigger`](../cloud/triggering) on a deployed graph.
|
|
174
|
-
|
|
175
|
-
## Why this is different from Playwright codegen / a basic LLM script
|
|
176
|
-
|
|
177
|
-
| | Playwright codegen | LLM-only script | Zibby test recipe |
|
|
178
|
-
|---|---|---|---|
|
|
179
|
-
| Plain-English input | ❌ | ✅ | ✅ |
|
|
180
|
-
| Real browser execution | ✅ | ❌ (just generates code) | ✅ |
|
|
181
|
-
| Coding-agent driven | ❌ | partial | ✅ Cursor / Claude / Codex |
|
|
182
|
-
| Multi-step verification | ❌ | ❌ | ✅ Zod-validated nodes |
|
|
183
|
-
| Replayable + debuggable | ❌ | ❌ | ✅ Studio |
|
|
184
|
-
| Vendor-neutral | N/A | locked to one LLM | swap agent per run |
|
|
185
|
-
|
|
186
|
-
## See also
|
|
187
|
-
|
|
188
|
-
- [Recipes overview](./)
|
|
189
|
-
- [Concepts: graph](../concepts/graph) — the primitives this recipe uses
|
|
190
|
-
- [Cloud triggering](../cloud/triggering) — fire agents from CI/CD
|
|
@@ -1,114 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 7
|
|
3
|
-
title: Reviewing Results
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# Reviewing Results
|
|
7
|
-
|
|
8
|
-
After running tests with `--sync`, all results are available in the Zibby dashboard for review.
|
|
9
|
-
|
|
10
|
-
## Test Runs
|
|
11
|
-
|
|
12
|
-
Go to your project's **Test Runs** page to see all executions.
|
|
13
|
-
|
|
14
|
-
Each run shows:
|
|
15
|
-
- Run name and timestamp
|
|
16
|
-
- Pass/fail status
|
|
17
|
-
- Video thumbnail (hover to preview)
|
|
18
|
-
- Source (local spec or cloud test case)
|
|
19
|
-
|
|
20
|
-
## Video Replay
|
|
21
|
-
|
|
22
|
-
Click on a test run to open the full replay view:
|
|
23
|
-
|
|
24
|
-
### Video Player
|
|
25
|
-
|
|
26
|
-
- Full video recording of the browser session
|
|
27
|
-
- Playback controls (play, pause, seek)
|
|
28
|
-
- Speed control (0.5x, 1x, 1.5x, 2x)
|
|
29
|
-
- Cinematic mode for full-screen viewing
|
|
30
|
-
|
|
31
|
-
### Action Timeline
|
|
32
|
-
|
|
33
|
-
A timeline below the video shows every action the AI performed:
|
|
34
|
-
|
|
35
|
-
- **Navigate** — page navigations
|
|
36
|
-
- **Click** — button and link clicks
|
|
37
|
-
- **Type** — keyboard input
|
|
38
|
-
- **Assert** — verification checks
|
|
39
|
-
- **Screenshot** — captured screenshots
|
|
40
|
-
|
|
41
|
-
Click any action to jump to that point in the video.
|
|
42
|
-
|
|
43
|
-
### Events Sidebar
|
|
44
|
-
|
|
45
|
-
A detailed list of all events with:
|
|
46
|
-
- Timestamps
|
|
47
|
-
- Action descriptions
|
|
48
|
-
- Element selectors
|
|
49
|
-
- AI-generated captions explaining each action
|
|
50
|
-
|
|
51
|
-
## Generated Scripts
|
|
52
|
-
|
|
53
|
-
The **Scripts** tab shows the auto-generated Playwright test script:
|
|
54
|
-
|
|
55
|
-
```javascript
|
|
56
|
-
const { test, expect } = require('@playwright/test');
|
|
57
|
-
|
|
58
|
-
test('User Login Flow', async ({ page }) => {
|
|
59
|
-
await page.goto('https://example.com/login');
|
|
60
|
-
await page.fill('[data-testid="email"]', 'test@example.com');
|
|
61
|
-
await page.fill('[data-testid="password"]', 'TestPass123');
|
|
62
|
-
await page.click('[data-testid="submit"]');
|
|
63
|
-
await expect(page).toHaveURL('/dashboard');
|
|
64
|
-
});
|
|
65
|
-
```
|
|
66
|
-
|
|
67
|
-
Click **Copy** to copy the script to your clipboard and add it to your test suite.
|
|
68
|
-
|
|
69
|
-
## Test Case Details
|
|
70
|
-
|
|
71
|
-
The **Test Case** tab displays the original test specification that was executed, with syntax highlighting.
|
|
72
|
-
|
|
73
|
-
## Analysis Results
|
|
74
|
-
|
|
75
|
-
For ticket analyses, the analysis page provides:
|
|
76
|
-
|
|
77
|
-
### Code Diff Viewer
|
|
78
|
-
|
|
79
|
-
- Side-by-side diff of all code changes
|
|
80
|
-
- File tree navigation
|
|
81
|
-
- **Create Pull Request** button to push changes to GitHub
|
|
82
|
-
|
|
83
|
-
### Test Cases
|
|
84
|
-
|
|
85
|
-
- Structured test cases with steps, data, and expected outcomes
|
|
86
|
-
- Editable — refine test cases before running them
|
|
87
|
-
- Auto-saves changes
|
|
88
|
-
|
|
89
|
-
### Pipeline View
|
|
90
|
-
|
|
91
|
-
Live status of each analysis step:
|
|
92
|
-
|
|
93
|
-
| Step | Description |
|
|
94
|
-
|---|---|
|
|
95
|
-
| Setup | Repository cloned and initialized |
|
|
96
|
-
| Analyze Ticket | AI reads and understands the ticket |
|
|
97
|
-
| Generate Code | Code changes produced |
|
|
98
|
-
| Generate Test Cases | Test cases written |
|
|
99
|
-
| Finalize | Report compiled and uploaded |
|
|
100
|
-
|
|
101
|
-
Each step shows logs that can be expanded for debugging.
|
|
102
|
-
|
|
103
|
-
## Collections
|
|
104
|
-
|
|
105
|
-
Organize runs into collections for grouping related tests:
|
|
106
|
-
|
|
107
|
-
- View collections in the **Collections** sidebar
|
|
108
|
-
- Each collection can have subfolders
|
|
109
|
-
- Drag runs between collections
|
|
110
|
-
- Filter and search within collections
|
|
111
|
-
|
|
112
|
-
## Sharing Results
|
|
113
|
-
|
|
114
|
-
Share a test run with your team by copying the URL from the browser. Anyone with access to the project can view the results.
|