@zibby/skills 0.2.22 → 0.2.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/browser.js +3 -10
- package/dist/code-scan.d.ts +1 -1
- package/dist/index.js +81 -88
- package/dist/jira.d.ts +75 -9
- package/dist/jira.js +4 -4
- package/dist/package.json +1 -1
- package/dist/report.d.ts +51 -276
- package/dist/trackers/index.js +12 -12
- package/dist/trackers/jira-adapter.js +11 -11
- package/package.json +1 -1
- package/docs/analysis.md +0 -109
- package/docs/apps/agent-ops.md +0 -130
- package/docs/apps/auth.md +0 -158
- package/docs/apps/deploy.md +0 -207
- package/docs/apps/goal-mode.md +0 -175
- package/docs/apps/index.md +0 -149
- package/docs/apps/managing.md +0 -121
- package/docs/cli-reference.md +0 -494
- package/docs/cloning-repositories.md +0 -285
- package/docs/cloud/bundles.md +0 -92
- package/docs/cloud/dedicated-egress.md +0 -140
- package/docs/cloud/editing-prompts.md +0 -79
- package/docs/cloud/env-vars.md +0 -144
- package/docs/cloud/limits.md +0 -81
- package/docs/cloud/logs.md +0 -104
- package/docs/cloud/triggering.md +0 -128
- package/docs/concepts/agents.md +0 -112
- package/docs/concepts/designing-agents.md +0 -247
- package/docs/concepts/graph.md +0 -83
- package/docs/concepts/sessions.md +0 -70
- package/docs/concepts/skills.md +0 -86
- package/docs/concepts/state.md +0 -106
- package/docs/concepts/sub-graphs.md +0 -227
- package/docs/custom-workflows.md +0 -358
- package/docs/get-started/deploy.md +0 -75
- package/docs/get-started/install.md +0 -60
- package/docs/get-started/run-locally.md +0 -94
- package/docs/get-started/trigger-and-logs.md +0 -90
- package/docs/get-started/use-from-agents.md +0 -153
- package/docs/get-started/your-first-workflow.md +0 -69
- package/docs/getting-started.md +0 -108
- package/docs/installation.md +0 -127
- package/docs/integrations/github.md +0 -73
- package/docs/integrations/gitlab.md +0 -43
- package/docs/integrations/jira.md +0 -71
- package/docs/integrations/lark.md +0 -41
- package/docs/integrations/linear.md +0 -43
- package/docs/integrations/notion.md +0 -33
- package/docs/integrations/plane.md +0 -46
- package/docs/integrations/sentry.md +0 -42
- package/docs/integrations/slack.md +0 -33
- package/docs/intro.md +0 -86
- package/docs/legacy/test-automation.md +0 -111
- package/docs/packages/agent-workflow.md +0 -88
- package/docs/packages/cli.md +0 -73
- package/docs/packages/core.md +0 -72
- package/docs/packages/mcp-browser.md +0 -110
- package/docs/packages/mcp-cli.md +0 -176
- package/docs/packages/memory.md +0 -223
- package/docs/packages/skills.md +0 -218
- package/docs/packages/ui-memory.md +0 -245
- package/docs/recipes/bug-autofix.md +0 -85
- package/docs/recipes/github-ai-scout.md +0 -61
- package/docs/recipes/index.md +0 -68
- package/docs/recipes/pipeline-supervisor.md +0 -57
- package/docs/recipes/sentry-triage.md +0 -93
- package/docs/recipes/test.md +0 -190
- package/docs/reviewing-results.md +0 -114
- package/docs/running-tests.md +0 -134
- package/docs/self-host/backup-restore.md +0 -53
- package/docs/self-host/custom-sidecars.md +0 -136
- package/docs/self-host/index.md +0 -75
- package/docs/self-host/storage.md +0 -50
- package/docs/self-host/troubleshooting.md +0 -48
- package/docs/self-host/upgrade.md +0 -81
- package/docs/skills/browser.md +0 -97
- package/docs/skills/chat-memory.md +0 -122
- package/docs/skills/core-tools.md +0 -80
- package/docs/skills/function-skill.md +0 -93
- package/docs/skills/github.md +0 -91
- package/docs/skills/index.md +0 -46
- package/docs/skills/jira.md +0 -99
- package/docs/skills/lark.md +0 -85
- package/docs/skills/memory.md +0 -92
- package/docs/skills/sentry.md +0 -80
- package/docs/skills/slack.md +0 -89
- package/docs/tests/memory.md +0 -131
- package/docs/triggering-workflows.md +0 -552
- package/docs/workflow-artifact-layout-evaluation.md +0 -119
- package/docs/workflow.md +0 -558
package/docs/packages/memory.md
DELETED
|
@@ -1,223 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 3
|
|
3
|
-
title: "@zibby/memory"
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# @zibby/memory
|
|
7
|
-
|
|
8
|
-
Version-controlled test memory database powered by [Dolt](https://www.dolthub.com/). Learns from every test run — selectors that worked, pages that were visited, patterns that failed, workarounds that helped.
|
|
9
|
-
|
|
10
|
-
```bash
|
|
11
|
-
npm install @zibby/memory
|
|
12
|
-
```
|
|
13
|
-
|
|
14
|
-
## Why Memory?
|
|
15
|
-
|
|
16
|
-
Without memory, every test run starts from scratch. The AI has no idea which selectors are stable, which pages have changed, or what workarounds were discovered in previous runs.
|
|
17
|
-
|
|
18
|
-
With `@zibby/memory`:
|
|
19
|
-
- **Selectors** — the AI knows which CSS/XPath selectors are reliable and which are flaky
|
|
20
|
-
- **Page models** — the AI has a map of page elements, roles, and structure before it even navigates
|
|
21
|
-
- **Navigation patterns** — the AI knows which URL transitions are valid
|
|
22
|
-
- **Test history** — the AI sees pass/fail trends and avoids repeating past failures
|
|
23
|
-
- **Insights** — the AI reads and writes tips (timing quirks, workarounds, selector alternatives)
|
|
24
|
-
|
|
25
|
-
## Setup
|
|
26
|
-
|
|
27
|
-
### 1. Install Dolt
|
|
28
|
-
|
|
29
|
-
Dolt is a version-controlled SQL database (Git for data):
|
|
30
|
-
|
|
31
|
-
```bash
|
|
32
|
-
# macOS
|
|
33
|
-
brew install dolt
|
|
34
|
-
|
|
35
|
-
# Linux
|
|
36
|
-
sudo bash -c 'curl -L https://github.com/dolthub/dolt/releases/latest/download/install.sh | bash'
|
|
37
|
-
```
|
|
38
|
-
|
|
39
|
-
### 2. Initialize Memory Database
|
|
40
|
-
|
|
41
|
-
```bash
|
|
42
|
-
zibby init --mem
|
|
43
|
-
```
|
|
44
|
-
|
|
45
|
-
This creates a Dolt database at `.zibby/memory/` with the schema for test runs, selectors, page models, navigation, and insights.
|
|
46
|
-
|
|
47
|
-
### 3. Enable Memory in Your Workflow
|
|
48
|
-
|
|
49
|
-
Add `SKILLS.MEMORY` to any node that should have memory access:
|
|
50
|
-
|
|
51
|
-
```javascript
|
|
52
|
-
import { SKILLS } from '@zibby/core';
|
|
53
|
-
|
|
54
|
-
export const executeLiveNode = {
|
|
55
|
-
name: 'execute_live',
|
|
56
|
-
skills: [SKILLS.BROWSER, SKILLS.MEMORY],
|
|
57
|
-
// ...
|
|
58
|
-
};
|
|
59
|
-
```
|
|
60
|
-
|
|
61
|
-
The built-in `execute_live` node already has memory enabled by default.
|
|
62
|
-
|
|
63
|
-
## How It Works
|
|
64
|
-
|
|
65
|
-
### During a Test Run
|
|
66
|
-
|
|
67
|
-
1. **Before execution** — the memory middleware loads relevant history:
|
|
68
|
-
- Previous runs for this spec (pass/fail, timing)
|
|
69
|
-
- Known selectors for the target pages
|
|
70
|
-
- Saved insights and tips
|
|
71
|
-
|
|
72
|
-
2. **During execution** — the AI can call memory tools:
|
|
73
|
-
- `memory_get_selectors` to find stable selectors
|
|
74
|
-
- `memory_get_page_model` to understand page structure
|
|
75
|
-
- `memory_save_insight` to record a finding
|
|
76
|
-
|
|
77
|
-
3. **After execution** — the result handler persists new data:
|
|
78
|
-
- Test result (pass/fail, duration)
|
|
79
|
-
- Selectors used and their success/failure
|
|
80
|
-
- Page model updates
|
|
81
|
-
- Navigation transitions discovered
|
|
82
|
-
|
|
83
|
-
### Version Control
|
|
84
|
-
|
|
85
|
-
Every persist operation creates a Dolt commit. You can:
|
|
86
|
-
|
|
87
|
-
```bash
|
|
88
|
-
# View memory history
|
|
89
|
-
cd .zibby/memory
|
|
90
|
-
dolt log
|
|
91
|
-
|
|
92
|
-
# Diff between runs
|
|
93
|
-
dolt diff HEAD~1 HEAD
|
|
94
|
-
|
|
95
|
-
# Branch for experiments
|
|
96
|
-
dolt branch experiment
|
|
97
|
-
dolt checkout experiment
|
|
98
|
-
```
|
|
99
|
-
|
|
100
|
-
## Database Schema
|
|
101
|
-
|
|
102
|
-
### `test_runs`
|
|
103
|
-
|
|
104
|
-
| Column | Type | Description |
|
|
105
|
-
|---|---|---|
|
|
106
|
-
| `session_id` | VARCHAR | Unique session identifier |
|
|
107
|
-
| `spec_path` | VARCHAR | Path to the test spec file |
|
|
108
|
-
| `passed` | BOOLEAN | Whether the test passed |
|
|
109
|
-
| `duration_ms` | INT | Total execution time |
|
|
110
|
-
| `agent_type` | VARCHAR | Which agent ran the test |
|
|
111
|
-
| `created_at` | DATETIME | Timestamp |
|
|
112
|
-
|
|
113
|
-
### `selectors`
|
|
114
|
-
|
|
115
|
-
| Column | Type | Description |
|
|
116
|
-
|---|---|---|
|
|
117
|
-
| `page_url` | VARCHAR | URL where this selector was used |
|
|
118
|
-
| `selector` | VARCHAR | The CSS/XPath selector string |
|
|
119
|
-
| `stable_id` | VARCHAR | Zibby stable ID (if available) |
|
|
120
|
-
| `success_count` | INT | Times this selector worked |
|
|
121
|
-
| `fail_count` | INT | Times this selector failed |
|
|
122
|
-
| `last_used` | DATETIME | Last usage timestamp |
|
|
123
|
-
|
|
124
|
-
### `page_model`
|
|
125
|
-
|
|
126
|
-
| Column | Type | Description |
|
|
127
|
-
|---|---|---|
|
|
128
|
-
| `url` | VARCHAR | Page URL |
|
|
129
|
-
| `element_role` | VARCHAR | ARIA role |
|
|
130
|
-
| `element_name` | VARCHAR | Accessible name |
|
|
131
|
-
| `selector` | VARCHAR | Best known selector |
|
|
132
|
-
| `updated_at` | DATETIME | Last update |
|
|
133
|
-
|
|
134
|
-
### `navigation`
|
|
135
|
-
|
|
136
|
-
| Column | Type | Description |
|
|
137
|
-
|---|---|---|
|
|
138
|
-
| `from_url` | VARCHAR | Source page URL |
|
|
139
|
-
| `to_url` | VARCHAR | Destination page URL |
|
|
140
|
-
| `trigger` | VARCHAR | What caused the navigation (click, submit, etc.) |
|
|
141
|
-
| `count` | INT | Times this transition was observed |
|
|
142
|
-
|
|
143
|
-
### `insights`
|
|
144
|
-
|
|
145
|
-
| Column | Type | Description |
|
|
146
|
-
|---|---|---|
|
|
147
|
-
| `category` | ENUM | `selector_tip`, `timing`, `navigation`, `workaround`, `flaky`, `general` |
|
|
148
|
-
| `content` | TEXT | The insight text |
|
|
149
|
-
| `spec_path` | VARCHAR | Related spec |
|
|
150
|
-
| `session_id` | VARCHAR | Session that created it |
|
|
151
|
-
| `created_at` | DATETIME | Timestamp |
|
|
152
|
-
|
|
153
|
-
## MCP Tools
|
|
154
|
-
|
|
155
|
-
The memory MCP server exposes five tools:
|
|
156
|
-
|
|
157
|
-
```
|
|
158
|
-
memory_get_test_history — Query recent test runs (filter by spec path)
|
|
159
|
-
memory_get_selectors — Query selectors with stability metrics (filter by page URL)
|
|
160
|
-
memory_get_page_model — Query page elements and roles (filter by URL)
|
|
161
|
-
memory_get_navigation — Query page-to-page transitions (filter by source URL)
|
|
162
|
-
memory_save_insight — Save a useful observation for future runs
|
|
163
|
-
```
|
|
164
|
-
|
|
165
|
-
### Example: AI Querying Memory
|
|
166
|
-
|
|
167
|
-
During execution, the AI might call:
|
|
168
|
-
|
|
169
|
-
```json
|
|
170
|
-
{
|
|
171
|
-
"tool": "memory_get_selectors",
|
|
172
|
-
"input": { "pageUrl": "myapp.com/login", "limit": 10 }
|
|
173
|
-
}
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
And receive:
|
|
177
|
-
|
|
178
|
-
```json
|
|
179
|
-
[
|
|
180
|
-
{ "selector": "[data-testid='email']", "success_count": 12, "fail_count": 0 },
|
|
181
|
-
{ "selector": "#login-email", "success_count": 8, "fail_count": 3 },
|
|
182
|
-
{ "selector": "input[name='email']", "success_count": 5, "fail_count": 1 }
|
|
183
|
-
]
|
|
184
|
-
```
|
|
185
|
-
|
|
186
|
-
The AI then prefers `[data-testid='email']` because it has the highest success rate.
|
|
187
|
-
|
|
188
|
-
## Middleware Integration
|
|
189
|
-
|
|
190
|
-
Memory provides automatic middleware that injects history into the node context:
|
|
191
|
-
|
|
192
|
-
```javascript
|
|
193
|
-
import { createMemoryMiddleware } from '@zibby/memory';
|
|
194
|
-
|
|
195
|
-
const middleware = createMemoryMiddleware();
|
|
196
|
-
|
|
197
|
-
const graph = new WorkflowGraph({ middleware: [middleware] });
|
|
198
|
-
```
|
|
199
|
-
|
|
200
|
-
The memory skill registers this middleware automatically when `SKILLS.MEMORY` is declared on a node.
|
|
201
|
-
|
|
202
|
-
## CLI Commands
|
|
203
|
-
|
|
204
|
-
```bash
|
|
205
|
-
# Initialize memory database
|
|
206
|
-
zibby init --mem
|
|
207
|
-
|
|
208
|
-
# View memory stats
|
|
209
|
-
zibby memory status
|
|
210
|
-
|
|
211
|
-
# Sync memory (push to Dolt remote)
|
|
212
|
-
zibby memory sync
|
|
213
|
-
```
|
|
214
|
-
|
|
215
|
-
## Exports
|
|
216
|
-
|
|
217
|
-
```javascript
|
|
218
|
-
import {
|
|
219
|
-
createMemoryMiddleware,
|
|
220
|
-
memoryEndRun,
|
|
221
|
-
memorySyncPush,
|
|
222
|
-
} from '@zibby/memory';
|
|
223
|
-
```
|
package/docs/packages/skills.md
DELETED
|
@@ -1,218 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 2
|
|
3
|
-
title: "@zibby/skills"
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# @zibby/skills
|
|
7
|
-
|
|
8
|
-
> Looking for skill-specific docs and examples? See [Skills reference](../skills/index.md).
|
|
9
|
-
|
|
10
|
-
Built-in skill definitions for Zibby's test automation framework.
|
|
11
|
-
|
|
12
|
-
```bash
|
|
13
|
-
npm install @zibby/skills
|
|
14
|
-
```
|
|
15
|
-
|
|
16
|
-
> Installed automatically as a dependency of `@zibby/cli`.
|
|
17
|
-
|
|
18
|
-
## What Are Skills?
|
|
19
|
-
|
|
20
|
-
A **skill** is a declarative description of an MCP (Model Context Protocol) server and the tools it exposes. Skills are the bridge between your agent nodes and external capabilities like browser automation, Jira, GitHub, Slack, and test memory.
|
|
21
|
-
|
|
22
|
-
Skills are **agent-agnostic** — the same skill definition works across Cursor, Claude, and Codex. The framework resolves the skill into the right MCP configuration for whichever agent is active.
|
|
23
|
-
|
|
24
|
-
## Built-in Skills
|
|
25
|
-
|
|
26
|
-
| Skill ID | MCP Server | Tools Provided |
|
|
27
|
-
|---|---|---|
|
|
28
|
-
| `browser` | `@zibby/mcp-browser` / `@playwright/mcp` | Browser navigation, clicking, typing, snapshots, video |
|
|
29
|
-
| `memory` | `@zibby/mcp-memory` | Test history, selector stability, page model, save insights |
|
|
30
|
-
| `jira` | `@zibby/mcp-jira` | Read/write Jira tickets |
|
|
31
|
-
| `github` | GitHub MCP server | Repository access, PR creation |
|
|
32
|
-
| `slack` | Slack MCP server | Send notifications, post results |
|
|
33
|
-
|
|
34
|
-
## Using Skills in Nodes
|
|
35
|
-
|
|
36
|
-
Declare skills in a node definition:
|
|
37
|
-
|
|
38
|
-
```javascript
|
|
39
|
-
import { SKILLS } from '@zibby/core';
|
|
40
|
-
|
|
41
|
-
export const executeLiveNode = {
|
|
42
|
-
name: 'execute_live',
|
|
43
|
-
skills: [SKILLS.BROWSER, SKILLS.MEMORY],
|
|
44
|
-
prompt: (state) => `Execute the test: ${state.testSpec}`,
|
|
45
|
-
outputSchema: ExecutionSchema,
|
|
46
|
-
};
|
|
47
|
-
```
|
|
48
|
-
|
|
49
|
-
When the agent runs:
|
|
50
|
-
1. The framework reads the node's `skills` array
|
|
51
|
-
2. For each skill, calls `skill.resolve()` to get the MCP server config
|
|
52
|
-
3. Injects the resolved MCP server into the agent's environment
|
|
53
|
-
4. Appends the skill's `promptFragment` to the prompt (if defined)
|
|
54
|
-
5. Runs skill middleware (if defined)
|
|
55
|
-
|
|
56
|
-
## Skill Anatomy
|
|
57
|
-
|
|
58
|
-
Every skill has this shape:
|
|
59
|
-
|
|
60
|
-
```javascript
|
|
61
|
-
{
|
|
62
|
-
id: 'browser', // Unique identifier
|
|
63
|
-
type: 'mcp', // 'mcp' or 'function'
|
|
64
|
-
serverName: 'playwright', // MCP server name
|
|
65
|
-
allowedTools: ['mcp__playwright__*'], // Tool patterns for Claude SDK
|
|
66
|
-
cursorKey: 'playwright-official', // Key in ~/.cursor/mcp.json
|
|
67
|
-
sessionEnvKey: 'ZIBBY_SESSION_INFO', // Env var with session path
|
|
68
|
-
envKeys: [], // Required env vars
|
|
69
|
-
description: 'Playwright Browser MCP',
|
|
70
|
-
|
|
71
|
-
// Prompt text appended to every node that uses this skill
|
|
72
|
-
promptFragment: 'Execute using browser tools...',
|
|
73
|
-
|
|
74
|
-
// Returns MCP server config { command, args, env }
|
|
75
|
-
resolve({ sessionPath, workspace }) {
|
|
76
|
-
return {
|
|
77
|
-
command: 'node',
|
|
78
|
-
args: ['/path/to/mcp-server.js', '--output-dir', sessionPath],
|
|
79
|
-
};
|
|
80
|
-
},
|
|
81
|
-
|
|
82
|
-
// Optional: middleware factory (called once per graph run)
|
|
83
|
-
async middleware() {
|
|
84
|
-
return async (nodeName, next, stateValues, state) => {
|
|
85
|
-
// Pre-node logic (e.g., load test history)
|
|
86
|
-
const result = await next();
|
|
87
|
-
// Post-node logic (e.g., persist insights)
|
|
88
|
-
return result;
|
|
89
|
-
};
|
|
90
|
-
},
|
|
91
|
-
|
|
92
|
-
// Tool schemas for compile-time validation
|
|
93
|
-
tools: [
|
|
94
|
-
{ name: 'tool_name', description: '...', input_schema: { ... } }
|
|
95
|
-
],
|
|
96
|
-
}
|
|
97
|
-
```
|
|
98
|
-
|
|
99
|
-
## Creating Custom Skills
|
|
100
|
-
|
|
101
|
-
### MCP Skill (wraps an external MCP server)
|
|
102
|
-
|
|
103
|
-
```javascript
|
|
104
|
-
import { skill } from '@zibby/skills';
|
|
105
|
-
|
|
106
|
-
export const linear = skill('linear', {
|
|
107
|
-
description: 'Linear issue tracker',
|
|
108
|
-
serverName: 'linear',
|
|
109
|
-
allowedTools: ['mcp__linear__*'],
|
|
110
|
-
envKeys: ['LINEAR_API_KEY'],
|
|
111
|
-
resolve() {
|
|
112
|
-
if (!process.env.LINEAR_API_KEY) return null;
|
|
113
|
-
return {
|
|
114
|
-
command: 'npx',
|
|
115
|
-
args: ['-y', '@anthropic/linear-mcp-server'],
|
|
116
|
-
env: { LINEAR_API_KEY: process.env.LINEAR_API_KEY },
|
|
117
|
-
};
|
|
118
|
-
},
|
|
119
|
-
});
|
|
120
|
-
```
|
|
121
|
-
|
|
122
|
-
Use it in a node:
|
|
123
|
-
|
|
124
|
-
```javascript
|
|
125
|
-
graph.addNode('create_issue', {
|
|
126
|
-
name: 'create_issue',
|
|
127
|
-
skills: ['linear'],
|
|
128
|
-
prompt: (state) => `Create a Linear issue for: ${state.bugReport}`,
|
|
129
|
-
outputSchema: IssueSchema,
|
|
130
|
-
});
|
|
131
|
-
```
|
|
132
|
-
|
|
133
|
-
### Function Skill (single tool, auto-bridged to MCP)
|
|
134
|
-
|
|
135
|
-
For simple tools that don't need a full MCP server:
|
|
136
|
-
|
|
137
|
-
```javascript
|
|
138
|
-
import { skill } from '@zibby/skills';
|
|
139
|
-
|
|
140
|
-
export const calculator = skill('calculator', {
|
|
141
|
-
description: 'Perform arithmetic calculations',
|
|
142
|
-
input: {
|
|
143
|
-
expression: 'string',
|
|
144
|
-
},
|
|
145
|
-
handler: async ({ expression }) => {
|
|
146
|
-
const result = eval(expression); // simplified example
|
|
147
|
-
return { result: String(result) };
|
|
148
|
-
},
|
|
149
|
-
});
|
|
150
|
-
```
|
|
151
|
-
|
|
152
|
-
The framework automatically spawns a lightweight MCP bridge server for function skills at runtime.
|
|
153
|
-
|
|
154
|
-
### Skill with Middleware
|
|
155
|
-
|
|
156
|
-
Middleware runs before and after every node that uses the skill:
|
|
157
|
-
|
|
158
|
-
```javascript
|
|
159
|
-
import { skill } from '@zibby/skills';
|
|
160
|
-
|
|
161
|
-
export const audit = skill('audit', {
|
|
162
|
-
description: 'Audit logging',
|
|
163
|
-
resolve() { return null; }, // No MCP server needed
|
|
164
|
-
async middleware() {
|
|
165
|
-
return async (nodeName, next, stateValues, state) => {
|
|
166
|
-
console.log(`[audit] Node ${nodeName} starting`);
|
|
167
|
-
const startTime = Date.now();
|
|
168
|
-
const result = await next();
|
|
169
|
-
console.log(`[audit] Node ${nodeName} completed in ${Date.now() - startTime}ms`);
|
|
170
|
-
return result;
|
|
171
|
-
};
|
|
172
|
-
},
|
|
173
|
-
});
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
## Browser Skill Details
|
|
177
|
-
|
|
178
|
-
The browser skill resolves to `@zibby/mcp-browser` if installed, otherwise falls back to `@playwright/mcp`:
|
|
179
|
-
|
|
180
|
-
```javascript
|
|
181
|
-
// Resolution priority:
|
|
182
|
-
// 1. MCP_BROWSER_PATH env var
|
|
183
|
-
// 2. @zibby/mcp-browser (enhanced: stable IDs, event recording)
|
|
184
|
-
// 3. @playwright/mcp (community fallback)
|
|
185
|
-
```
|
|
186
|
-
|
|
187
|
-
Default configuration:
|
|
188
|
-
- Video resolution: 1280x720
|
|
189
|
-
- Viewport: 1280x720
|
|
190
|
-
- Output directory: session path or `test-results/`
|
|
191
|
-
|
|
192
|
-
## Memory Skill Details
|
|
193
|
-
|
|
194
|
-
The memory skill provides five tools:
|
|
195
|
-
|
|
196
|
-
| Tool | Description |
|
|
197
|
-
|---|---|
|
|
198
|
-
| `memory_get_test_history` | Query recent test runs with pass/fail results |
|
|
199
|
-
| `memory_get_selectors` | Query known selectors with stability metrics |
|
|
200
|
-
| `memory_get_page_model` | Query page structure — elements, roles, selectors |
|
|
201
|
-
| `memory_get_navigation` | Query known page-to-page transitions |
|
|
202
|
-
| `memory_save_insight` | Save observations for future runs (selector tips, timing, workarounds) |
|
|
203
|
-
|
|
204
|
-
The memory skill also includes middleware that automatically loads relevant test history before node execution.
|
|
205
|
-
|
|
206
|
-
## Exports
|
|
207
|
-
|
|
208
|
-
```javascript
|
|
209
|
-
import { SKILLS } from '@zibby/skills';
|
|
210
|
-
// SKILLS.BROWSER, SKILLS.JIRA, SKILLS.GITHUB, SKILLS.SLACK, SKILLS.MEMORY
|
|
211
|
-
|
|
212
|
-
import { browserSkill, jiraSkill, githubSkill, slackSkill, memorySkill } from '@zibby/skills';
|
|
213
|
-
|
|
214
|
-
import { skill, functionSkill } from '@zibby/skills';
|
|
215
|
-
|
|
216
|
-
// Re-exported from @zibby/core
|
|
217
|
-
import { registerSkill, getSkill, hasSkill, getAllSkills, listSkillIds } from '@zibby/skills';
|
|
218
|
-
```
|
|
@@ -1,245 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
sidebar_position: 5
|
|
3
|
-
title: "@zibby/ui-memory"
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# @zibby/ui-memory
|
|
7
|
-
|
|
8
|
-
Version-controlled UI agent memory powered by [Dolt](https://www.dolthub.com/). Learns from every test run — selectors that worked, page-element fingerprints, navigation transitions, timing quirks, recorded insights. Used today by `zibby test`; designed to power any agent that drives a UI.
|
|
9
|
-
|
|
10
|
-
```bash
|
|
11
|
-
npm install @zibby/ui-memory
|
|
12
|
-
```
|
|
13
|
-
|
|
14
|
-
Current version: **1.1.0**
|
|
15
|
-
|
|
16
|
-
> Renamed from `@zibby/memory` to make the per-domain UI focus explicit. This package is the **UI test-memory** store (Dolt-backed, per-domain). For chat-style agent memory — facts, decisions, task history — see the [Chat memory skill](../skills/chat-memory.md), which defaults to a **mem0** semantic backend (with automatic fallback to Dolt) and can be pinned to Dolt explicitly.
|
|
17
|
-
|
|
18
|
-
## Why memory
|
|
19
|
-
|
|
20
|
-
Without memory, every test run starts from scratch. The agent has no idea which selectors are stable, which pages have changed, or what workarounds were discovered last week.
|
|
21
|
-
|
|
22
|
-
With `@zibby/ui-memory`:
|
|
23
|
-
|
|
24
|
-
- **Selectors** — the agent prefers selectors with high success / low fail counts
|
|
25
|
-
- **Page model** — known elements, ARIA roles, accessible names
|
|
26
|
-
- **Navigation** — known page-to-page transitions (which click produced which URL)
|
|
27
|
-
- **Test history** — pass/fail trends per spec, full timing
|
|
28
|
-
- **Insights** — categorized free-form notes the agent reads + writes (`selector_tip | timing | navigation | workaround | flaky | general`)
|
|
29
|
-
|
|
30
|
-
Critically, memory is keyed **per domain**, not per spec. A selector that one spec learned for `myapp.com/login` is available to every other spec hitting the same site.
|
|
31
|
-
|
|
32
|
-
## Setup
|
|
33
|
-
|
|
34
|
-
Most users get this automatically via `zibby init` — Dolt is bundled, the DB is initialized, and `zibby test` writes to it. The notes below are for direct package consumption.
|
|
35
|
-
|
|
36
|
-
### 1. Install Dolt (if not bundled)
|
|
37
|
-
|
|
38
|
-
```bash
|
|
39
|
-
# macOS
|
|
40
|
-
brew install dolt
|
|
41
|
-
|
|
42
|
-
# Linux
|
|
43
|
-
sudo bash -c 'curl -L https://github.com/dolthub/dolt/releases/latest/download/install.sh | bash'
|
|
44
|
-
```
|
|
45
|
-
|
|
46
|
-
### 2. Initialize the database
|
|
47
|
-
|
|
48
|
-
```bash
|
|
49
|
-
zibby memory init
|
|
50
|
-
```
|
|
51
|
-
|
|
52
|
-
Creates `.zibby/memory/.dolt/` with the schema for runs, selectors, page model, navigation, and insights.
|
|
53
|
-
|
|
54
|
-
### 3. Enable memory in your agent
|
|
55
|
-
|
|
56
|
-
Add `SKILLS.MEMORY` to any node that should have memory access:
|
|
57
|
-
|
|
58
|
-
```javascript
|
|
59
|
-
import { SKILLS } from '@zibby/core';
|
|
60
|
-
|
|
61
|
-
export const executeLiveNode = {
|
|
62
|
-
name: 'execute_live',
|
|
63
|
-
skills: [SKILLS.BROWSER, SKILLS.MEMORY],
|
|
64
|
-
// ...
|
|
65
|
-
};
|
|
66
|
-
```
|
|
67
|
-
|
|
68
|
-
The built-in `execute_live` node already has memory enabled.
|
|
69
|
-
|
|
70
|
-
## How it works
|
|
71
|
-
|
|
72
|
-
### Before each run
|
|
73
|
-
|
|
74
|
-
The runner auto-pulls from the configured remote (if any) and loads the relevant slice into the agent's context — selectors for the page, recent failures, applicable insights.
|
|
75
|
-
|
|
76
|
-
### During the run
|
|
77
|
-
|
|
78
|
-
The agent has 5 MCP tools auto-exposed:
|
|
79
|
-
|
|
80
|
-
| Tool | What it does |
|
|
81
|
-
|---|---|
|
|
82
|
-
| `memory_get_test_history` | Recent runs — filter by spec-path substring; returns pass/fail/timing |
|
|
83
|
-
| `memory_get_selectors` | Known selectors per page with success/fail counts |
|
|
84
|
-
| `memory_get_page_model` | Page elements (URL, ARIA role, accessible name, best selector) |
|
|
85
|
-
| `memory_get_navigation` | Known transitions (from URL → to URL via what trigger) |
|
|
86
|
-
| `memory_save_insight` | Save observation: `selector_tip | timing | navigation | workaround | flaky | general` |
|
|
87
|
-
|
|
88
|
-
### After the run
|
|
89
|
-
|
|
90
|
-
The result handler persists the new state — selectors used (with success/failure deltas), page-model updates, new navigation transitions discovered, and any insights the agent saved.
|
|
91
|
-
|
|
92
|
-
> **The agent is required to call `memory_save_insight` at least once at the end of every run.** This is in the memory skill's prompt fragment. Without insights, memory degrades to cached selectors and run rows; with them it compounds.
|
|
93
|
-
|
|
94
|
-
### Version control
|
|
95
|
-
|
|
96
|
-
Every persist creates a Dolt commit. You can:
|
|
97
|
-
|
|
98
|
-
```bash
|
|
99
|
-
cd .zibby/memory
|
|
100
|
-
dolt log
|
|
101
|
-
dolt diff HEAD~1 HEAD
|
|
102
|
-
dolt branch experiment
|
|
103
|
-
dolt checkout experiment
|
|
104
|
-
```
|
|
105
|
-
|
|
106
|
-
## Database schema
|
|
107
|
-
|
|
108
|
-
### `test_runs`
|
|
109
|
-
|
|
110
|
-
| Column | Type | Description |
|
|
111
|
-
|---|---|---|
|
|
112
|
-
| `session_id` | VARCHAR | Unique session identifier |
|
|
113
|
-
| `spec_path` | VARCHAR | Path to the test spec file |
|
|
114
|
-
| `passed` | BOOLEAN | Whether the test passed |
|
|
115
|
-
| `duration_ms` | INT | Total execution time |
|
|
116
|
-
| `agent_type` | VARCHAR | Which agent ran the test |
|
|
117
|
-
| `tokens_input` / `tokens_output` / `tokens_cache_*` | INT | LLM token usage (drives `zibby memory cost`) |
|
|
118
|
-
| `created_at` | DATETIME | Timestamp |
|
|
119
|
-
|
|
120
|
-
### `selectors`
|
|
121
|
-
|
|
122
|
-
| Column | Type | Description |
|
|
123
|
-
|---|---|---|
|
|
124
|
-
| `page_url` | VARCHAR | URL where this selector was used |
|
|
125
|
-
| `selector` | VARCHAR | The CSS/XPath selector string |
|
|
126
|
-
| `stable_id` | VARCHAR | Zibby stable ID (if available) |
|
|
127
|
-
| `success_count` | INT | Times this selector worked |
|
|
128
|
-
| `fail_count` | INT | Times this selector failed |
|
|
129
|
-
| `last_used` | DATETIME | Last usage timestamp |
|
|
130
|
-
|
|
131
|
-
### `page_model`
|
|
132
|
-
|
|
133
|
-
| Column | Type | Description |
|
|
134
|
-
|---|---|---|
|
|
135
|
-
| `url` | VARCHAR | Page URL |
|
|
136
|
-
| `element_role` | VARCHAR | ARIA role |
|
|
137
|
-
| `element_name` | VARCHAR | Accessible name |
|
|
138
|
-
| `selector` | VARCHAR | Best known selector |
|
|
139
|
-
| `updated_at` | DATETIME | Last update |
|
|
140
|
-
|
|
141
|
-
### `navigation`
|
|
142
|
-
|
|
143
|
-
| Column | Type | Description |
|
|
144
|
-
|---|---|---|
|
|
145
|
-
| `from_url` | VARCHAR | Source page URL |
|
|
146
|
-
| `to_url` | VARCHAR | Destination page URL |
|
|
147
|
-
| `trigger` | VARCHAR | What caused the navigation (click, submit, etc.) |
|
|
148
|
-
| `count` | INT | Times this transition was observed |
|
|
149
|
-
|
|
150
|
-
### `insights`
|
|
151
|
-
|
|
152
|
-
| Column | Type | Description |
|
|
153
|
-
|---|---|---|
|
|
154
|
-
| `category` | ENUM | `selector_tip`, `timing`, `navigation`, `workaround`, `flaky`, `general` |
|
|
155
|
-
| `content` | TEXT | The insight text |
|
|
156
|
-
| `spec_path` | VARCHAR | Related spec |
|
|
157
|
-
| `session_id` | VARCHAR | Session that created it |
|
|
158
|
-
| `created_at` | DATETIME | Timestamp |
|
|
159
|
-
|
|
160
|
-
## Team sync
|
|
161
|
-
|
|
162
|
-
Memory is local-first. Opt into a shared remote so teammates' learnings flow back:
|
|
163
|
-
|
|
164
|
-
```bash
|
|
165
|
-
# BYO — your S3 / GCS / DoltHub repo / file path
|
|
166
|
-
zibby memory remote add aws://my-bucket/team/proj/main
|
|
167
|
-
zibby memory remote add gs://bucket/team/proj/main
|
|
168
|
-
zibby memory remote add https://www.dolthub.com/repositories/<owner>/<repo>
|
|
169
|
-
zibby memory remote add file:///abs/path/to/local-shared
|
|
170
|
-
|
|
171
|
-
# OR Zibby-managed S3 (no plumbing; signed-in users only)
|
|
172
|
-
zibby memory remote use --hosted
|
|
173
|
-
```
|
|
174
|
-
|
|
175
|
-
Once configured:
|
|
176
|
-
|
|
177
|
-
- `zibby test` auto-pulls before runs and auto-pushes after passing runs
|
|
178
|
-
- `zibby memory pull` / `zibby memory push` for manual override
|
|
179
|
-
- `zibby memory remote info` to inspect, `zibby memory remote remove` to disconnect
|
|
180
|
-
|
|
181
|
-
To wire teammates in automatically, set `memorySync.remote` in `.zibby.config.mjs`:
|
|
182
|
-
|
|
183
|
-
```js
|
|
184
|
-
export default {
|
|
185
|
-
agent: { claude: { model: 'auto' } },
|
|
186
|
-
memorySync: {
|
|
187
|
-
remote: 'hosted', // or 'aws://my-bucket/team/proj/main' or null
|
|
188
|
-
},
|
|
189
|
-
};
|
|
190
|
-
```
|
|
191
|
-
|
|
192
|
-
`zibby init` reads this — when set to `'hosted'` and the user isn't signed in, init prompts for `zibby login` but never blocks. After login, the remote is wired and the next `zibby test` pulls.
|
|
193
|
-
|
|
194
|
-
### Hosted vs BYO
|
|
195
|
-
|
|
196
|
-
| | Hosted (`--hosted`) | BYO |
|
|
197
|
-
|---|---|---|
|
|
198
|
-
| Setup | `zibby memory remote use --hosted` | Provision bucket / IAM / KMS |
|
|
199
|
-
| Storage | Zibby-managed AWS account | Your account |
|
|
200
|
-
| Access | Anyone with project access on Zibby | Whoever your IAM grants |
|
|
201
|
-
| Compliance / data residency | Limited regions | Wherever you want |
|
|
202
|
-
| Cost | Included in plan | Your S3 bill |
|
|
203
|
-
|
|
204
|
-
## Middleware integration
|
|
205
|
-
|
|
206
|
-
Memory provides automatic middleware that injects history into the node context:
|
|
207
|
-
|
|
208
|
-
```javascript
|
|
209
|
-
import { createMemoryMiddleware } from '@zibby/ui-memory';
|
|
210
|
-
|
|
211
|
-
const middleware = createMemoryMiddleware();
|
|
212
|
-
const graph = new WorkflowGraph({ middleware: [middleware] });
|
|
213
|
-
```
|
|
214
|
-
|
|
215
|
-
The memory skill registers this middleware automatically when `SKILLS.MEMORY` is declared on a node.
|
|
216
|
-
|
|
217
|
-
## CLI commands
|
|
218
|
-
|
|
219
|
-
```bash
|
|
220
|
-
zibby memory init # initialize the DB
|
|
221
|
-
zibby memory stats # row counts, last commit, per-spec breakdown
|
|
222
|
-
zibby memory cost # real LLM token spend per spec / per domain
|
|
223
|
-
zibby memory compact # prune old runs + Dolt GC
|
|
224
|
-
zibby memory reset -f # wipe (destructive)
|
|
225
|
-
zibby memory pull / push # manual sync (auto on test start/end if remote configured)
|
|
226
|
-
zibby memory remote add <url> # BYO remote
|
|
227
|
-
zibby memory remote use --hosted# Zibby-managed S3
|
|
228
|
-
zibby memory remote info # show config
|
|
229
|
-
zibby memory remote remove # drop the remote
|
|
230
|
-
```
|
|
231
|
-
|
|
232
|
-
## Exports
|
|
233
|
-
|
|
234
|
-
```javascript
|
|
235
|
-
import {
|
|
236
|
-
createMemoryMiddleware,
|
|
237
|
-
memoryEndRun,
|
|
238
|
-
memorySyncPush,
|
|
239
|
-
} from '@zibby/ui-memory';
|
|
240
|
-
```
|
|
241
|
-
|
|
242
|
-
## See also
|
|
243
|
-
|
|
244
|
-
- [`zibby test` recipe](../recipes/test) — the primary consumer of memory
|
|
245
|
-
- [Test memory deep dive](../tests/memory) — usage-oriented walkthrough
|