@ui-sdk/svelte 0.1.0-beta.4 → 0.1.0-beta.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/docs/agents.md +66 -12
  2. package/package.json +3 -3
package/docs/agents.md CHANGED
@@ -77,16 +77,62 @@ You can give these tools to your own agent, or register them with a supported br
77
77
 
78
78
  ## Benchmark
79
79
 
80
- In a project-settings benchmark, agents renamed a project, updated members and notification settings, then saved. One used browser tools on a page built with another component library; the other used the agent bridge on the same screen built with ui-sdk.
80
+ We gave an agent one task on a project settings screen: rename the project, change its visibility, update two members and the notification settings, then save. We built the same screen with ui-sdk and with other component libraries, in React, Vue and Svelte.
81
81
 
82
- | | Browser tools | ui-sdk + agent bridge | Delta |
83
- | ---------- | ------------- | --------------------- | ----- |
84
- | Tool calls | 16 | **5** | −69% |
85
- | Text read | 39.8k | **6.8k** | −83% |
86
- | Time | 41 s | **16 s** | −61% |
87
- | Cost | $0.073 | **$0.032** | −56% |
82
+ On the other libraries, the agent used browser tools: it reads the page's accessibility tree, then clicks and types. On ui-sdk, it used the agent bridge.
88
83
 
89
- These are medians from three runs per approach. Both agents used Sonnet through Claude Code with the same task and system prompt, plus instructions for their available tools. Every run completed the task. Text read counts characters returned by the tools. Time and cost are what Claude Code reported for those runs.
84
+ ### React
85
+
86
+ | | Base UI + browser tools | Ark UI + browser tools | ui-sdk + agent bridge | Delta |
87
+ | ---------- | ----------------------- | ---------------------- | --------------------- | ----------- |
88
+ | Tool calls | 17 | 19 | **7** | 59% fewer |
89
+ | Tokens | 291.7k | 330.9k | **69.5k** | 76% fewer |
90
+ | Time | 53 s | 65 s | **31 s** | 41% faster |
91
+ | Cost | $0.139 | $0.144 | **$0.044** | 68% cheaper |
92
+
93
+ ### Vue
94
+
95
+ | | Reka UI + browser tools | ui-sdk + agent bridge | Delta |
96
+ | ---------- | ----------------------- | --------------------- | ----------- |
97
+ | Tool calls | 16 | **9** | 44% fewer |
98
+ | Tokens | 226.5k | **90.2k** | 60% fewer |
99
+ | Time | 56 s | **34 s** | 39% faster |
100
+ | Cost | $0.113 | **$0.051** | 55% cheaper |
101
+
102
+ ### Svelte
103
+
104
+ | | Bits UI + browser tools | ui-sdk + agent bridge | Delta |
105
+ | ---------- | ----------------------- | --------------------- | ----------- |
106
+ | Tool calls | 17 | **8** | 53% fewer |
107
+ | Tokens | 259.1k | **92.8k** | 64% fewer |
108
+ | Time | 56 s | **30 s** | 45% faster |
109
+ | Cost | $0.123 | **$0.056** | 54% cheaper |
110
+
111
+ These are medians from three runs per column. Every agent used Sonnet through Claude Code with the same task and system prompt, plus instructions for its tools. Every run completed the task. Time runs from the agent's start to its last message. Tokens and cost are what Claude Code reported for the run. Delta compares ui-sdk with the best other result in the row.
112
+
113
+ We also ran the ui-sdk screen with browser tools. It took about as much as the other libraries, so the difference comes from the tools.
114
+
115
+ ### Run It Yourself
116
+
117
+ The benchmark is in the repo, in [`benchmarks/agent`](https://github.com/vercel-labs/ui/tree/main/benchmarks/agent). It runs its agents with [Claude Code](https://claude.com/claude-code), so install it and sign in first.
118
+
119
+ ```sh
120
+ git clone https://github.com/vercel-labs/ui
121
+ cd ui
122
+ pnpm install
123
+ pnpm exec playwright install chromium
124
+ pnpm --filter agent-benchmark bench
125
+ ```
126
+
127
+ This opens every version of the React screen in its own browser window and starts one agent in each. When they finish, it prints a table with each run's tool calls, text read, time, tokens and cost, and whether the agent saved the right settings.
128
+
129
+ Pass `--framework vue` or `--framework svelte` for the other frameworks, and `--runs 3` for medians:
130
+
131
+ ```sh
132
+ pnpm --filter agent-benchmark bench --framework vue --runs 3
133
+ ```
134
+
135
+ The [README](https://github.com/vercel-labs/ui/tree/main/benchmarks/agent#readme) explains the task, what's measured, and how to add another library.
90
136
 
91
137
  ## Where to Start
92
138
 
@@ -410,10 +456,14 @@ Put several steps in one call to run them in order. If a step fails, `result.ok`
410
456
  For components with choices, such as a Select or Combobox, `query` with `within` lists the available values. For a Select labeled **Region**:
411
457
 
412
458
  ```ts
413
- const [region] = await agent.query({ within: "Region", text: "Europe" })
459
+ const found = await agent.query({ within: "Region", text: "Europe" })
460
+ if ("error" in found) {
461
+ throw new Error(found.error)
462
+ }
463
+ const [region] = found
414
464
  ```
415
465
 
416
- `region.items` holds up to 50 matching choices, each with a `value` and `label`, and `region.more` says how many matched beyond that. Leave out `text` to list the first 50. `query()` never opens anything. When a list's items only show while it's open, `items` is empty and `note` says so.
466
+ `query` returns `{ error }` when nothing on the page is named **Region**. Otherwise `region.items` holds up to 50 matching choices, each with a `value` and `label`, and `region.more` says how many matched beyond that. Leave out `text` to list the first 50. `query()` never opens anything. When a list's items only show while it's open, `items` is empty and `note` says so.
417
467
 
418
468
  `query` also finds components without listing the page. `agent.query({ text: "Europe" })` returns the components whose label or row text has that word, and lists whose choices match, with those choices under them. `kind` keeps one kind, such as `"select"`.
419
469
 
@@ -1383,7 +1433,7 @@ Each step is an `AgentStep`:
1383
1433
 
1384
1434
  Use a specific ref when a name is ambiguous. A step can reach a component revealed by an earlier step. The bridge can also look for a target inside closed accordion sections.
1385
1435
 
1386
- For choices, an argument can be the item's value or its label. Labels match regardless of case and accents, like component names; values match exactly. Ambiguous item labels are refused. When a list's items show only while it's open, `setValue` with a label asks `allow` with the label before it opens the list to read them, then again with the item's value. Numeric `setValue` actions also accept text that starts with a number, such as `"90 days"`.
1436
+ For choices, an argument can be the item's value or its label. Labels match regardless of case and accents, like component names; values match exactly. Ambiguous item labels are refused. When a list's items show only while it's open, `setValue` with a label asks `allow` with the label before it opens the list to read them, then again with the item's value. Numeric `setValue` actions also accept text that starts with a number, such as `"90 days"`. Text with grouped digits, such as `"1,499.5"`, is refused. A component that holds a list, such as a multiple Select, takes one value or label as a list of one.
1387
1437
 
1388
1438
  A `set` action's first argument is checked against the kind of the option it sets in the primitive's schema, like `setChecked` against `checked`. A `"boolean"` option takes only `true` or `false`, a `"number"` option takes a number or text that is one, like `"3"`, and an option read by an attribute parser takes what that parser reads, so a resizable group takes `"30,70"` as its layout but a checkbox refuses `"yes"`. A value the kind doesn't take fails the step, with the type it expects, and changes nothing.
1389
1439
 
@@ -1436,7 +1486,11 @@ A failed step stops the call. Earlier steps stay applied; there is no rollback.
1436
1486
  Finds components and their choices without listing the page, and returns a promise:
1437
1487
 
1438
1488
  ```ts
1439
- const [region] = await agent.query({ within: "Region", text: "Europe" })
1489
+ const found = await agent.query({ within: "Region", text: "Europe" })
1490
+ if ("error" in found) {
1491
+ throw new Error(found.error)
1492
+ }
1493
+ const [region] = found
1440
1494
  ```
1441
1495
 
1442
1496
  | Input | What It Does |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@ui-sdk/svelte",
3
- "version": "0.1.0-beta.4",
3
+ "version": "0.1.0-beta.5",
4
4
  "description": "UI SDK for Svelte.",
5
5
  "keywords": [
6
6
  "svelte",
@@ -381,7 +381,7 @@
381
381
  ],
382
382
  "peerDependencies": {
383
383
  "svelte": ">=5.33",
384
- "@ui-sdk/core": "^0.1.0-beta.4"
384
+ "@ui-sdk/core": "^0.1.0-beta.5"
385
385
  },
386
386
  "devDependencies": {
387
387
  "@sveltejs/package": "^2.5.8",
@@ -393,7 +393,7 @@
393
393
  "typescript": "~5.9.3",
394
394
  "vite": "^8.3.3",
395
395
  "vitest": "^5.0.3",
396
- "@ui-sdk/core": "0.1.0-beta.4"
396
+ "@ui-sdk/core": "0.1.0-beta.5"
397
397
  },
398
398
  "scripts": {
399
399
  "build": "node ../../scripts/build/build-svelte-package.mjs",