@qoniai/qoni 0.8.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +102 -2
- package/README.md +119 -72
- package/README.zh-CN.md +73 -44
- package/dist/index.cjs +355 -58
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +163 -42
- package/dist/index.d.ts +163 -42
- package/dist/index.js +355 -58
- package/dist/index.js.map +1 -1
- package/package.json +2 -2
- package/skills/qoni-sdk/SKILL.md +42 -20
package/CHANGELOG.md
CHANGED
|
@@ -1,6 +1,106 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
-
## 0.
|
|
3
|
+
## 0.9.0 - 2026-10-07
|
|
4
|
+
|
|
5
|
+
DoAnything interactions are now typed by what the user has to do, never by
|
|
6
|
+
business: `site_login`, `take_control`, `ask_user`, `confirmation`, plus the
|
|
7
|
+
system `wait`.
|
|
8
|
+
|
|
9
|
+
### Changed
|
|
10
|
+
|
|
11
|
+
- `Interaction` is a union distributed over `type`: `switch (interaction.type)`
|
|
12
|
+
narrows `payload` in each branch, including `handle.interaction` inside
|
|
13
|
+
`onInteraction`. `Interaction<"site_login">` still picks one member.
|
|
14
|
+
`TypedInteraction` names the union. Because it is now a type alias, not an
|
|
15
|
+
interface, `interface X extends Interaction` no longer compiles (TS2312):
|
|
16
|
+
use `type X = Interaction & { ... }`. Code that builds an `Interaction` from a
|
|
17
|
+
separate `type` and `payload` must narrow them together or cast the object.
|
|
18
|
+
- The wire's `clarification` is presented as `ask_user`, with payload
|
|
19
|
+
`{ question, answerType, options? }`. `answerType` is `text`, `number`,
|
|
20
|
+
`boolean`, `single_choice` or `multiple_choice`, and options are
|
|
21
|
+
`{ value, label }`. A clarification with choices becomes `single_choice`; one
|
|
22
|
+
without choices is `text`. A native `ask_user` keeps the `answer_type` it
|
|
23
|
+
sends, and every string option value is kept, including `""`.
|
|
24
|
+
Migration: compare `type` with `"ask_user"` instead of `"clarification"`
|
|
25
|
+
(TypeScript no longer compiles the old comparison); read
|
|
26
|
+
`payload.answerType` / `payload.options` instead of `payload.inputType`
|
|
27
|
+
(removed) or the answer action's `inputSchema` enum.
|
|
28
|
+
`InteractionTypes.Clarification` is deprecated and now equals
|
|
29
|
+
`InteractionTypes.AskUser`: it still matches interactions the SDK hands out,
|
|
30
|
+
but not a raw wire `type` or a `"clarification"` string you stored.
|
|
31
|
+
`ClarificationPayload` is a deprecated alias of `AskUserPayload`, which
|
|
32
|
+
requires `answerType`.
|
|
33
|
+
- `InteractionHandle.answer(value)` accepts the structured value its
|
|
34
|
+
`answerType` names (string, number, boolean, one option value, or an array of
|
|
35
|
+
option values) and throws `QoniValidationError` before sending a value that
|
|
36
|
+
does not fit.
|
|
37
|
+
- `InteractionHandle.confirmSignedIn()` saves the login opened by `openLogin()`
|
|
38
|
+
on the same interaction (`POST /profiles/{profileId}/confirm`, scope
|
|
39
|
+
`webagent.site_login:confirm`), then tells the agent to recheck. Sign-in is
|
|
40
|
+
`openLogin()` → `confirmSignedIn()`:
|
|
41
|
+
- It saves once and rechecks once per login session, however many handles
|
|
42
|
+
or concurrent calls ask; the deprecated `OpenLoginResult.confirm()` shares
|
|
43
|
+
the same save.
|
|
44
|
+
- A user who is not signed in yet, or 409 `no_pending_login`, still gets a
|
|
45
|
+
recheck: the agent raises a new `site_login` when needed.
|
|
46
|
+
- When the backend keeps the session but could not save it
|
|
47
|
+
(`probeResult: "release_failed"`), it throws a retryable `QoniError` coded
|
|
48
|
+
`site_login.save_failed` and does not tell the agent to go on; calling it
|
|
49
|
+
again saves again. Any other save failure is thrown as well.
|
|
50
|
+
- When `openLogin()` calls overlap, the latest call that succeeds owns the
|
|
51
|
+
session, whichever response arrives last.
|
|
52
|
+
- Reopening that returns the same backend session shares its save; when a
|
|
53
|
+
newer backend session replaces it, `confirm()` on the old result throws
|
|
54
|
+
`QoniValidationError` instead of saving. If the login is reopened while a
|
|
55
|
+
`confirmSignedIn()` is still saving, that call throws a retryable
|
|
56
|
+
`QoniError` coded `site_login.session_replaced` without telling the agent
|
|
57
|
+
to go on; call it again to save the newer login.
|
|
58
|
+
- Handles from the same `RunHandle` or `MonitorHandle` share the login
|
|
59
|
+
session, so a handle from a later callback saves it too. A RunHandle from
|
|
60
|
+
another `attach()` or process does not.
|
|
61
|
+
- `connectControl()` and `refreshControl()` resolve to a `ControlSession`
|
|
62
|
+
(`{ liveUrl, expiresAt?, status?, raw }`, the signed URL of the browser the
|
|
63
|
+
user now controls) instead of `void`. `await handle.connectControl()` is
|
|
64
|
+
unaffected; code that returns the call where `Promise<void>` is expected,
|
|
65
|
+
such as `const cb: () => Promise<void> = () => handle.connectControl()`, no
|
|
66
|
+
longer compiles (TS2322) — wrap it: `async () => { await handle.connectControl(); }`.
|
|
67
|
+
- `run.events()`, `wait({ onInteraction })` and `monitor.events()` keep an
|
|
68
|
+
interaction's type and content when the backend settles it with an empty
|
|
69
|
+
shell (a resolve labelled `clarification` with an empty question, or a
|
|
70
|
+
`site_login` with one empty site): only `status`, `resolvedAt` and `actions`
|
|
71
|
+
change. A settled update with content of its own, such as an expired
|
|
72
|
+
take-over with its reason, replaces the card as before. Run and monitor
|
|
73
|
+
handles keep about 256 interactions, forgetting settled cards first, and
|
|
74
|
+
logins once they are saved or their interaction ended; a pending card or a
|
|
75
|
+
login still in progress is never dropped.
|
|
76
|
+
|
|
77
|
+
### Added
|
|
78
|
+
|
|
79
|
+
- `QoniScopes.USER_PROFILE_READ` (`user.profile:read`), `USER_PROFILE_WRITE`
|
|
80
|
+
(`user.profile:write`), `USER_PAYMENT_USE` (`user.payment:use`),
|
|
81
|
+
`AGENT_MAIL_READ` (`agent.mail:read`) and `AGENT_MAIL_SEND`
|
|
82
|
+
(`agent.mail:send`). They belong to no product bundle: request them
|
|
83
|
+
explicitly, and allow them in the AccessKey policy. The constants only name
|
|
84
|
+
the scopes; requesting them does not by itself enable Profile, payment or
|
|
85
|
+
mailbox access in a run.
|
|
86
|
+
- Exported types `AskUserPayload`, `AskUserOption`, `AskUserAnswer`,
|
|
87
|
+
`AskUserAnswerType`, `InteractionBase`, `TypedInteraction` and
|
|
88
|
+
`ControlSession`.
|
|
89
|
+
|
|
90
|
+
### Fixed
|
|
91
|
+
|
|
92
|
+
- A scope written without its service prefix names every matching scope:
|
|
93
|
+
`profile:read` suggests both `gumem.profile:read` and `user.profile:read`.
|
|
94
|
+
|
|
95
|
+
### Deprecated
|
|
96
|
+
|
|
97
|
+
- `user` / `userId` on interactive delegation input: the authorization page
|
|
98
|
+
identifies the signed-in user, and the delegation service ignores them.
|
|
99
|
+
- `DelegateTokenResponse.grantedScopes`: the delegation service does not return
|
|
100
|
+
it. Read `qoni.genauth.introspectDelegationToken({ token }).data.scope`.
|
|
101
|
+
- `OpenLoginResult.confirm()`: `confirmSignedIn()` saves the login itself.
|
|
102
|
+
|
|
103
|
+
## 0.8.0 - 2026-10-04
|
|
4
104
|
|
|
5
105
|
### Added
|
|
6
106
|
|
|
@@ -150,7 +250,7 @@ compatible fixes.
|
|
|
150
250
|
- Optional abort signals on product run-detail and artifact-list wire reads;
|
|
151
251
|
existing calls without a signal remain compatible.
|
|
152
252
|
|
|
153
|
-
##
|
|
253
|
+
## Earlier changes
|
|
154
254
|
|
|
155
255
|
### Breaking
|
|
156
256
|
|
package/README.md
CHANGED
|
@@ -151,7 +151,7 @@ const run = await qoni.doAnything.run({
|
|
|
151
151
|
|
|
152
152
|
const result = await run.wait({
|
|
153
153
|
onScreenshot: (img, i) => fs.writeFileSync(`step-${i}.jpg`, img.bytes),
|
|
154
|
-
onInteraction: (i) => { //
|
|
154
|
+
onInteraction: (i) => { // site_login / take_control / ask_user / confirmation
|
|
155
155
|
if (i.can("confirm_signed_in")) return i.confirmSignedIn();
|
|
156
156
|
if (i.can("confirm")) return i.confirm();
|
|
157
157
|
},
|
|
@@ -286,18 +286,27 @@ const grant = await qoni.delegateToken({
|
|
|
286
286
|
scopes: ["gumem.memory:read", "webagent.do_anything:manage"],
|
|
287
287
|
});
|
|
288
288
|
|
|
289
|
+
// The callback will not carry grantId: save it with your business state, keyed by grantState.
|
|
290
|
+
await savePendingGrant(grant.data.grantState, {
|
|
291
|
+
grantId: grant.data.grantId,
|
|
292
|
+
businessState: "business-csrf-state",
|
|
293
|
+
});
|
|
289
294
|
redirectUserTo(grant.data.authorizationUrl);
|
|
290
295
|
```
|
|
291
296
|
|
|
292
|
-
The user visits `authorizationUrl` in Qoni Console/BFF. If they are not logged in, Console/BFF sends them through GenAuth login, confirms authorization, completes the grant internally, then redirects back to your `redirectUri` with a one-time `code`, your `state`, and `grant_state
|
|
297
|
+
The user visits `authorizationUrl` in Qoni Console/BFF. If they are not logged in, Console/BFF sends them through GenAuth login, confirms authorization, completes the grant internally, then redirects back to your `redirectUri` with a one-time `code`, your `state`, and `grant_state`. The callback does **not** carry `grantId`, so look up the record you saved by `grant_state`. The browser never receives a delegation token.
|
|
293
298
|
|
|
294
299
|
Your callback handler runs on the server and exchanges the callback fields with AK/SK:
|
|
295
300
|
|
|
296
301
|
```ts
|
|
302
|
+
const pending = await loadPendingGrant(String(req.query.grant_state));
|
|
303
|
+
// Your business state comes back unchanged; reject forged or mixed-up callbacks.
|
|
304
|
+
if (!pending || req.query.state !== pending.businessState) throw new Error("Invalid delegation callback");
|
|
305
|
+
|
|
297
306
|
const completed = await qoni.completeDelegateToken({
|
|
298
|
-
grantId:
|
|
307
|
+
grantId: pending.grantId,
|
|
299
308
|
code: String(req.query.code),
|
|
300
|
-
state: String(req.query.
|
|
309
|
+
state: String(req.query.grant_state), // grant_state, not your business state
|
|
301
310
|
});
|
|
302
311
|
|
|
303
312
|
await qoni.gumem.recall({
|
|
@@ -308,7 +317,7 @@ await qoni.gumem.recall({
|
|
|
308
317
|
|
|
309
318
|
`completeDelegateToken` is the recommended server callback helper. Root `qoni.delegateAgent` and `qoni.completeDelegateAgent` remain as deprecated compatibility names, but the current callback contract is `{ grantId, code, state }`; old `{ code, state }` completion is not supported.
|
|
310
319
|
|
|
311
|
-
## Original delegation style (0.8.0
|
|
320
|
+
## Original delegation style (0.8.0+)
|
|
312
321
|
|
|
313
322
|
```ts
|
|
314
323
|
const grant = await qoni.genauth.delegateAgent({
|
|
@@ -329,8 +338,8 @@ the authorization request directly; use `completeDelegateToken()` in the server
|
|
|
329
338
|
callback after consent. `agentKey` aliases `agent`; both labels must agree when
|
|
330
339
|
supplied together. Top-level `userId` retains precedence over `user.id`.
|
|
331
340
|
Expiry accepts integer seconds or whole-number
|
|
332
|
-
`s/m/h/d` durations, between 60 and 86400 seconds.
|
|
333
|
-
|
|
341
|
+
`s/m/h/d` durations, between 60 and 86400 seconds. These additions are available
|
|
342
|
+
in the published `@qoniai/qoni@0.8.0` release.
|
|
334
343
|
|
|
335
344
|
Legacy `mailbox.*`, `web.search`, and `web.extract` scopes, and `deniedScopes`, are unsupported and rejected.
|
|
336
345
|
They are never expanded into broader product permissions. Product scopes do not
|
|
@@ -348,7 +357,9 @@ await qoni.delegateToken({
|
|
|
348
357
|
});
|
|
349
358
|
```
|
|
350
359
|
|
|
351
|
-
Scope strings are validated locally before any request: a scope missing its service prefix (e.g. `do_anything:manage` instead of `webagent.do_anything:manage`) throws `QoniValidationError` immediately with the correct form in the message.
|
|
360
|
+
Scope strings are validated locally before any request: a scope missing its service prefix (e.g. `do_anything:manage` instead of `webagent.do_anything:manage`) throws `QoniValidationError` immediately with the correct form in the message — every candidate when the short form is ambiguous (`profile:read` → `gumem.profile:read` or `user.profile:read`).
|
|
361
|
+
|
|
362
|
+
The site-login, user Profile, payment and Agent mailbox scopes (`QoniScopes.SITE_LOGIN_*`, `USER_PROFILE_READ`, `USER_PROFILE_WRITE`, `USER_PAYMENT_USE`, `AGENT_MAIL_READ`, `AGENT_MAIL_SEND`) belong to no product: request them explicitly in `scopes`, and allow them in the AccessKey policy. The user Profile, payment and Agent mailbox constants only name the scopes; requesting them does not by itself enable Profile, payment or mailbox access in a run.
|
|
352
363
|
|
|
353
364
|
Use explicit scope strings when the requested permission boundary should be visible in code.
|
|
354
365
|
|
|
@@ -360,6 +371,10 @@ Use explicit scope strings when the requested permission boundary should be visi
|
|
|
360
371
|
| Search public web | `webagent.web_search:read`, `webagent.web_search:manage` | Agent can search public pages and read search results. |
|
|
361
372
|
| Run a bounded web task | `webagent.do_anything:read`, `webagent.do_anything:manage` | Agent can run a visible web task, read progress, stop it, and intervene when needed. |
|
|
362
373
|
| Create a monitor | `webagent.track:read`, `webagent.track:manage` | Agent can configure monitors, read events, run checks, and stop monitors. |
|
|
374
|
+
| Sign in to sites during a run | `webagent.site_login:request`, `webagent.site_login:confirm` | The user signs in through a live browser; the login is saved for later tasks. |
|
|
375
|
+
| Use the user's Profile | `user.profile:read`, `user.profile:write` | Agent can read the user's saved details (name, phone, shipping address, card brand and last 4) and save new ones the user provides. |
|
|
376
|
+
| Pay with the user's saved card | `user.payment:use` | After the user approves a payment, the saved card is filled in for them. |
|
|
377
|
+
| Use the Agent's own mailbox | `agent.mail:read`, `agent.mail:send` | Agent can read mail sent to its own address (such as verification codes) and send mail from it. |
|
|
363
378
|
|
|
364
379
|
Silent mode is usually appropriate for low-risk actions such as reading current-user memory, writing a confirmed summary, or searching public web pages.
|
|
365
380
|
|
|
@@ -634,81 +649,113 @@ the backend contract.
|
|
|
634
649
|
|
|
635
650
|
### Interactions (HITL)
|
|
636
651
|
|
|
637
|
-
When a run needs the user
|
|
638
|
-
|
|
639
|
-
|
|
640
|
-
`clarification` / `confirmation` / `take_control` / `wait`). Wrap it in a handle
|
|
641
|
-
and call the action methods; each posts to the endpoint the backend declared in
|
|
642
|
-
the interaction's `actions[]`, so you never hard-code a route:
|
|
652
|
+
When a run needs the user, it pauses and emits an `interaction` event whose
|
|
653
|
+
`event.data` is a typed `Interaction`. Interactions are typed by what the user
|
|
654
|
+
has to do, never by business, so one handler covers every task:
|
|
643
655
|
|
|
644
|
-
|
|
645
|
-
|
|
646
|
-
|
|
647
|
-
|
|
648
|
-
|
|
649
|
-
|
|
650
|
-
|
|
651
|
-
else if (i.can("release_control")) await i.releaseControl();
|
|
652
|
-
}
|
|
653
|
-
}
|
|
654
|
-
```
|
|
655
|
-
|
|
656
|
-
Methods: `answer(text)`, `skip()`, `confirm()`, `reject()`, `openLogin({ siteId?, profileId? })`,
|
|
657
|
-
`confirmSignedIn()`, `connectControl()`, `refreshControl()`, `releaseControl()`,
|
|
658
|
-
`retry()`, `switchProfile()`. `i.can(kind)` reports whether the interaction
|
|
659
|
-
currently offers that action; calling a method the backend did NOT declare
|
|
660
|
-
throws (so a "recovery" button only exists when it actually works).
|
|
656
|
+
| `type` | The user… | Methods |
|
|
657
|
+
| --- | --- | --- |
|
|
658
|
+
| `site_login` | signs in to a site in the live browser | `openLogin()`, `confirmSignedIn()` |
|
|
659
|
+
| `take_control` | operates the browser by hand (captcha, 2FA, …) | `connectControl()`, `refreshControl()`, `releaseControl()` |
|
|
660
|
+
| `ask_user` | answers a question shaped by `answerType` / `options` | `answer()`, `skip()` |
|
|
661
|
+
| `confirmation` | allows or rejects an action | `confirm()`, `reject()` |
|
|
662
|
+
| `wait` | nothing — the system is waiting (rate limit, external event) | `retry()` |
|
|
661
663
|
|
|
662
|
-
`
|
|
663
|
-
|
|
664
|
-
`
|
|
665
|
-
listed. Show the returned `liveUrl` to the user; after they sign in, persist the
|
|
666
|
-
login with `login.confirm()`, then tell the agent with `confirmSignedIn()`:
|
|
664
|
+
`Interaction` is a union distributed over `type`, so `switch (request.type)`
|
|
665
|
+
narrows `request.payload` in each branch. Each method posts to the endpoint the
|
|
666
|
+
backend declared in the interaction's `actions[]`, so you never hard-code a route:
|
|
667
667
|
|
|
668
668
|
```ts
|
|
669
|
-
import {
|
|
669
|
+
import type { InteractionHandle } from "@qoniai/qoni";
|
|
670
670
|
|
|
671
|
-
const
|
|
672
|
-
|
|
673
|
-
|
|
674
|
-
|
|
675
|
-
|
|
676
|
-
|
|
677
|
-
|
|
678
|
-
|
|
679
|
-
|
|
680
|
-
|
|
681
|
-
|
|
682
|
-
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
|
|
686
|
-
|
|
671
|
+
const handled = new Set<string>();
|
|
672
|
+
|
|
673
|
+
async function handleInteraction(handle: InteractionHandle) {
|
|
674
|
+
const request = handle.interaction;
|
|
675
|
+
// Creation, status updates and replays all call back: act once per pending id.
|
|
676
|
+
if (request.status !== "pending" || handled.has(request.id)) return;
|
|
677
|
+
handled.add(request.id);
|
|
678
|
+
|
|
679
|
+
switch (request.type) {
|
|
680
|
+
case "site_login": {
|
|
681
|
+
const login = await handle.openLogin(); // { liveUrl, ... } of a live login browser
|
|
682
|
+
await showToUser(login.liveUrl); // resolves when the user clicks "I'm signed in"
|
|
683
|
+
await handle.confirmSignedIn(); // saves the login, then the agent rechecks and continues
|
|
684
|
+
break;
|
|
685
|
+
}
|
|
686
|
+
case "take_control": {
|
|
687
|
+
// connectControl() resolves to the signed control URL, when the backend offers it
|
|
688
|
+
const control = handle.can("connect_control") ? await handle.connectControl() : undefined;
|
|
689
|
+
await showToUser(control?.liveUrl ?? request.payload.liveUrl);
|
|
690
|
+
await handle.releaseControl(); // hand the browser back
|
|
691
|
+
break;
|
|
692
|
+
}
|
|
693
|
+
case "ask_user": // payload: { question, answerType, options? }
|
|
694
|
+
await handle.answer(await askUser(request.payload));
|
|
695
|
+
break;
|
|
696
|
+
case "confirmation": // payload: { summary }
|
|
697
|
+
if (await approve(request.payload.summary)) await handle.confirm();
|
|
698
|
+
else await handle.reject();
|
|
699
|
+
break;
|
|
700
|
+
}
|
|
687
701
|
}
|
|
688
|
-
```
|
|
689
702
|
|
|
690
|
-
|
|
691
|
-
|
|
692
|
-
|
|
693
|
-
|
|
694
|
-
|
|
695
|
-
|
|
703
|
+
const result = await run.wait({ onInteraction: handleInteraction });
|
|
704
|
+
```
|
|
705
|
+
|
|
706
|
+
Interactions arrive one at a time: the run waits until the current one is
|
|
707
|
+
handled. The same interaction calls back again on every status change and on
|
|
708
|
+
reconnect replays, always with the same `id` and `type` — key your UI cards by
|
|
709
|
+
`request.id`. An update with content of its own replaces the card (a take-over
|
|
710
|
+
that became `active` carries its new `liveUrl`); when the backend settles an
|
|
711
|
+
interaction with an empty shell, the card keeps its original `payload`. A
|
|
712
|
+
`take_control` that arrives already `active` was started by the user, for
|
|
713
|
+
example from Console; show it if your UI offers control there. Methods: `answer(value)`, `skip()`, `confirm()`,
|
|
714
|
+
`reject()`, `openLogin({ siteId?, profileId? })`, `confirmSignedIn()`,
|
|
715
|
+
`connectControl()`, `refreshControl()`, `releaseControl()`, `retry()`,
|
|
716
|
+
`switchProfile()`. `connectControl()` and `refreshControl()` resolve to
|
|
717
|
+
`{ liveUrl, expiresAt?, status?, raw }`: the signed URL of the browser the user
|
|
718
|
+
now controls. `handle.can(kind)` reports whether the interaction currently
|
|
719
|
+
offers that action; calling a method the backend did NOT declare throws (so a
|
|
720
|
+
"recovery" button only exists when it actually works).
|
|
721
|
+
|
|
722
|
+
**`ask_user` answers are structured.** `answer()` takes the value matching
|
|
723
|
+
`payload.answerType` — `text` → string, `number` → number, `boolean` → boolean,
|
|
724
|
+
`single_choice` → one `options[].value`, `multiple_choice` → an array of option
|
|
725
|
+
values — and throws `QoniValidationError` before sending anything that does not
|
|
726
|
+
fit. The wire still calls this type `clarification`; the SDK presents it as
|
|
727
|
+
`ask_user` (choices arrive as `{ value, label }` options).
|
|
728
|
+
|
|
729
|
+
**Sign-in takes two calls.** `openLogin()` opens a live login browser for the
|
|
730
|
+
`site_login` interaction (pass `siteId` from `payload.sites[].siteId`, or omit it
|
|
731
|
+
when exactly one site is listed). Once the user is signed in, `confirmSignedIn()`
|
|
732
|
+
saves the login so later tasks reuse it, then tells the agent to recheck the site
|
|
733
|
+
and continue; if the user turns out not to be signed in, the agent raises a new
|
|
734
|
+
`site_login`. It saves once and rechecks once per login session, however many
|
|
735
|
+
times it is called. If the backend could not save the login yet, it throws a
|
|
736
|
+
retryable `QoniError` coded `site_login.save_failed` without telling the agent to
|
|
737
|
+
go on; call it again. The same holds for `site_login.session_replaced`, thrown
|
|
738
|
+
when the login was reopened while it was being saved. Handles from the same `RunHandle` (or `MonitorHandle`)
|
|
739
|
+
share the login session, so `confirmSignedIn()` also works on the handle of a
|
|
740
|
+
later callback; a RunHandle from a new `attach()` does not. Delegate
|
|
696
741
|
`QoniScopes.SITE_LOGIN_REQUEST` and `QoniScopes.SITE_LOGIN_CONFIRM`, and allow
|
|
697
|
-
both in the AccessKey policy.
|
|
698
|
-
|
|
699
|
-
|
|
700
|
-
|
|
701
|
-
`
|
|
702
|
-
`
|
|
703
|
-
|
|
704
|
-
|
|
705
|
-
`
|
|
742
|
+
both in the AccessKey policy. `openLogin()`'s result still offers `confirm()` for
|
|
743
|
+
callers that save the login themselves; it is deprecated.
|
|
744
|
+
|
|
745
|
+
The discriminant strings are also exported as constants so you can avoid bare
|
|
746
|
+
literals: `InteractionTypes` (`SiteLogin` / `TakeControl` / `AskUser` /
|
|
747
|
+
`Confirmation` / `Wait`; the deprecated `Clarification` now equals `AskUser`),
|
|
748
|
+
`InteractionStatuses`, and `ActionKinds` (the `handle.can(...)` action names).
|
|
749
|
+
Authorization scope strings have the same treatment — `QoniScopes` (every
|
|
750
|
+
individual scope), `QoniProductScopes` (the per-product `read`+`manage` pairs the
|
|
751
|
+
`products` sugar expands to), and `QoniScopeBundles` (curated bundles). Use them in
|
|
752
|
+
place of hand-typed strings:
|
|
706
753
|
|
|
707
754
|
```ts
|
|
708
755
|
import { InteractionTypes, ActionKinds, QoniScopes } from "@qoniai/qoni";
|
|
709
756
|
|
|
710
|
-
if (
|
|
711
|
-
await
|
|
757
|
+
if (handle.type === InteractionTypes.AskUser && handle.can(ActionKinds.answer)) {
|
|
758
|
+
await handle.answer("the blue button");
|
|
712
759
|
}
|
|
713
760
|
```
|
|
714
761
|
|
|
@@ -718,7 +765,7 @@ The 1:1 low-level methods live under `qoni.<product>.api.*` (e.g. `qoni.doAnythi
|
|
|
718
765
|
|
|
719
766
|
`event.raw.event` is the **raw wire event type** — an internal transport name (`run.action.started`, `run.section_completed`, …) whose naming is historical and inconsistent, exposed as a plain string. This is an implementation detail, not the product API: developers match on the curated `event.type` above. The full wire-type catalog is intentionally NOT exported — reach for `event.raw.event` (a string) only when you need an exact wire match the curated layer folds into `progress` (e.g. telling a specific browser action apart).
|
|
720
767
|
|
|
721
|
-
Browser Use scopes are reserved until their product runtime SDK methods are exported. Site Login scopes (`QoniScopes.SITE_LOGIN_REQUEST`, `QoniScopes.SITE_LOGIN_CONFIRM`) back `InteractionHandle.openLogin()` and `
|
|
768
|
+
Browser Use scopes are reserved until their product runtime SDK methods are exported. Site Login scopes (`QoniScopes.SITE_LOGIN_REQUEST`, `QoniScopes.SITE_LOGIN_CONFIRM`) back `InteractionHandle.openLogin()` and `confirmSignedIn()` on run interactions.
|
|
722
769
|
|
|
723
770
|
### Response Shape
|
|
724
771
|
|
package/README.zh-CN.md
CHANGED
|
@@ -144,7 +144,7 @@ const run = await qoni.doAnything.run({
|
|
|
144
144
|
|
|
145
145
|
const result = await run.wait({
|
|
146
146
|
onScreenshot: (img, i) => fs.writeFileSync(`step-${i}.jpg`, img.bytes),
|
|
147
|
-
onInteraction: (i) => { //
|
|
147
|
+
onInteraction: (i) => { // site_login / take_control / ask_user / confirmation
|
|
148
148
|
if (i.can("confirm_signed_in")) return i.confirmSignedIn();
|
|
149
149
|
if (i.can("confirm")) return i.confirm();
|
|
150
150
|
},
|
|
@@ -257,18 +257,27 @@ const grant = await qoni.delegateToken({
|
|
|
257
257
|
scopes: ["gumem.memory:read", "webagent.do_anything:manage"],
|
|
258
258
|
});
|
|
259
259
|
|
|
260
|
+
// 回调不会带回 grantId:按 grantState 保存 grantId 和业务 state
|
|
261
|
+
await savePendingGrant(grant.data.grantState, {
|
|
262
|
+
grantId: grant.data.grantId,
|
|
263
|
+
businessState: "business-csrf-state",
|
|
264
|
+
});
|
|
260
265
|
redirectUserTo(grant.data.authorizationUrl);
|
|
261
266
|
```
|
|
262
267
|
|
|
263
|
-
用户访问 `authorizationUrl` 进入 Qoni Console/BFF。若用户未登录,Console/BFF 会先走 GenAuth 登录;授权确认后由 Console/BFF 统一 complete,再回跳业务 `redirectUri`,只带一次性 `code`、业务 `state
|
|
268
|
+
用户访问 `authorizationUrl` 进入 Qoni Console/BFF。若用户未登录,Console/BFF 会先走 GenAuth 登录;授权确认后由 Console/BFF 统一 complete,再回跳业务 `redirectUri`,只带一次性 `code`、业务 `state` 和 `grant_state`,**不带 `grantId`**,需要按 `grant_state` 找回创建请求时保存的记录。浏览器不会拿到 delegation token。
|
|
264
269
|
|
|
265
270
|
业务 callback handler 在服务端用 AK/SK 兑换最终 token:
|
|
266
271
|
|
|
267
272
|
```ts
|
|
273
|
+
const pending = await loadPendingGrant(String(req.query.grant_state));
|
|
274
|
+
// 业务 state 原样回到回调里,和保存的值比对,防止回调被伪造或串用
|
|
275
|
+
if (!pending || req.query.state !== pending.businessState) throw new Error("Invalid delegation callback");
|
|
276
|
+
|
|
268
277
|
const completed = await qoni.completeDelegateToken({
|
|
269
|
-
grantId:
|
|
278
|
+
grantId: pending.grantId,
|
|
270
279
|
code: String(req.query.code),
|
|
271
|
-
state: String(req.query.
|
|
280
|
+
state: String(req.query.grant_state), // 传 grant_state,不是业务 state
|
|
272
281
|
});
|
|
273
282
|
|
|
274
283
|
await qoni.gumem.recall({
|
|
@@ -279,7 +288,7 @@ await qoni.gumem.recall({
|
|
|
279
288
|
|
|
280
289
|
`completeDelegateToken` 是推荐的服务端 callback helper。顶层 `qoni.delegateAgent` 和 `qoni.completeDelegateAgent` 仍保留为 deprecated 兼容命名,但当前 callback 契约是 `{ grantId, code, state }`;旧的 `{ code, state }` complete 签名不再兼容。
|
|
281
290
|
|
|
282
|
-
## 原始委托写法(0.8.0
|
|
291
|
+
## 原始委托写法(0.8.0 起)
|
|
283
292
|
|
|
284
293
|
```ts
|
|
285
294
|
const grant = await qoni.genauth.delegateAgent({
|
|
@@ -299,7 +308,7 @@ console.log(grant.grantId, grant.auditId);
|
|
|
299
308
|
用 `completeDelegateToken()` 兑换 token。`agentKey`、`userId` 分别兼容 `agent`、
|
|
300
309
|
`user.id`;同时提供两个 Agent 标签时必须一致,顶层 `userId` 保持原有的优先级。
|
|
301
310
|
有效期支持整数秒或整数 `s/m/h/d` 时长,
|
|
302
|
-
范围为 60–86400
|
|
311
|
+
范围为 60–86400 秒。这些新增能力已随 `@qoniai/qoni@0.8.0` 发布到 npm。
|
|
303
312
|
|
|
304
313
|
旧版 `mailbox.*`、`web.search`、`web.extract` scope 和 `deniedScopes` 不受支持,SDK 会拒绝,不会将它们
|
|
305
314
|
扩展成更宽的产品权限。产品 scope 不限制发送、删除邮件;任务描述不能代替禁止策略。
|
|
@@ -316,7 +325,9 @@ await qoni.delegateToken({
|
|
|
316
325
|
});
|
|
317
326
|
```
|
|
318
327
|
|
|
319
|
-
scope 字符串在发请求前就会本地预检:漏掉服务前缀(比如写成 `do_anything:manage` 而不是 `webagent.do_anything:manage`)会立即抛出 `QoniValidationError
|
|
328
|
+
scope 字符串在发请求前就会本地预检:漏掉服务前缀(比如写成 `do_anything:manage` 而不是 `webagent.do_anything:manage`)会立即抛出 `QoniValidationError`,错误消息里带正确写法;短写法有歧义时列出全部候选(`profile:read` → `gumem.profile:read` 或 `user.profile:read`)。
|
|
329
|
+
|
|
330
|
+
站点登录、用户 Profile、支付和 Agent 邮箱的 scope(`QoniScopes.SITE_LOGIN_*`、`USER_PROFILE_READ`、`USER_PROFILE_WRITE`、`USER_PAYMENT_USE`、`AGENT_MAIL_READ`、`AGENT_MAIL_SEND`)不属于任何产品:需要在 `scopes` 中显式申请,并在 AccessKey 权限策略中允许。用户 Profile、支付和 Agent 邮箱这几个常量只是 scope 名称:申请它们本身不会让任务获得 Profile、支付或邮箱能力。
|
|
320
331
|
|
|
321
332
|
需要让授权边界直接出现在业务代码里时,使用显式 scope 字符串。
|
|
322
333
|
|
|
@@ -328,6 +339,10 @@ scope 字符串在发请求前就会本地预检:漏掉服务前缀(比如
|
|
|
328
339
|
| 搜索公开网页 | `webagent.web_search:read`, `webagent.web_search:manage` | Agent 可以搜索公开网页并读取结果。 |
|
|
329
340
|
| 执行网页任务 | `webagent.do_anything:read`, `webagent.do_anything:manage` | Agent 可以执行有边界的网页任务、读取进度、停止任务,并在需要时介入控制。 |
|
|
330
341
|
| 创建长期监控 | `webagent.track:read`, `webagent.track:manage` | Agent 可以配置监控、读取事件、立即运行检查,并停止监控。 |
|
|
342
|
+
| 在任务中登录网站 | `webagent.site_login:request`, `webagent.site_login:confirm` | 用户在实时浏览器里亲自登录,登录状态保存后供之后的任务复用。 |
|
|
343
|
+
| 使用用户 Profile | `user.profile:read`, `user.profile:write` | Agent 可以读取用户保存的资料(姓名、手机号、收货地址、卡组织和后四位),并保存用户新提供的资料。 |
|
|
344
|
+
| 用用户保存的卡支付 | `user.payment:use` | 用户允许一笔支付后,替用户填入已保存的信用卡。 |
|
|
345
|
+
| 使用 Agent 自己的邮箱 | `agent.mail:read`, `agent.mail:send` | Agent 可以读取发到自己邮箱的邮件(例如验证码),并用它发送邮件。 |
|
|
331
346
|
|
|
332
347
|
低风险能力通常可以静默授权,例如读取当前用户自己的 GUMem、写入本次会话摘要、搜索公开网页、读取普通任务结果。
|
|
333
348
|
|
|
@@ -580,55 +595,69 @@ interface RunResult {
|
|
|
580
595
|
|
|
581
596
|
### 交互(HITL)
|
|
582
597
|
|
|
583
|
-
|
|
598
|
+
run 需要用户出手时会暂停,并发出一个 `interaction` 事件,`event.data` 是带类型的 `Interaction`。交互按「需要用户做什么」分类,和具体业务无关,所以一个处理函数就能覆盖所有任务:
|
|
584
599
|
|
|
585
|
-
|
|
586
|
-
|
|
587
|
-
|
|
588
|
-
|
|
589
|
-
|
|
590
|
-
|
|
591
|
-
|
|
592
|
-
else if (i.can("release_control")) await i.releaseControl();
|
|
593
|
-
}
|
|
594
|
-
}
|
|
595
|
-
```
|
|
596
|
-
|
|
597
|
-
方法:`answer(text)`、`skip()`、`confirm()`、`reject()`、`openLogin({ siteId?, profileId? })`、`confirmSignedIn()`、`connectControl()`、`refreshControl()`、`releaseControl()`、`retry()`、`switchProfile()`。`i.can(kind)` 判断该交互当前是否提供这个动作;调用后端没声明的方法会抛错(所以「恢复」按钮只有在真能用时才出现)。
|
|
600
|
+
| `type` | 用户要做的事 | 方法 |
|
|
601
|
+
| --- | --- | --- |
|
|
602
|
+
| `site_login` | 在实时浏览器里亲自登录某个网站 | `openLogin()`、`confirmSignedIn()` |
|
|
603
|
+
| `take_control` | 亲手操作浏览器(验证码、两步验证等) | `connectControl()`、`refreshControl()`、`releaseControl()` |
|
|
604
|
+
| `ask_user` | 回答一个问题,答案结构由 `answerType` / `options` 描述 | `answer()`、`skip()` |
|
|
605
|
+
| `confirmation` | 允许或拒绝一个操作 | `confirm()`、`reject()` |
|
|
606
|
+
| `wait` | 不需要做决定:系统在等待(限流、外部事件) | `retry()` |
|
|
598
607
|
|
|
599
|
-
`
|
|
608
|
+
`Interaction` 是按 `type` 分布的联合类型,`switch (request.type)` 之后每个分支里的 `request.payload` 会自动收窄。每个方法都 POST 到交互 `actions[]` 里后端声明的 endpoint,不需要写死路由:
|
|
600
609
|
|
|
601
610
|
```ts
|
|
602
|
-
import {
|
|
611
|
+
import type { InteractionHandle } from "@qoniai/qoni";
|
|
603
612
|
|
|
604
|
-
const
|
|
605
|
-
|
|
606
|
-
|
|
607
|
-
|
|
608
|
-
|
|
609
|
-
|
|
610
|
-
|
|
611
|
-
|
|
612
|
-
|
|
613
|
-
|
|
614
|
-
|
|
615
|
-
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
|
|
619
|
-
|
|
613
|
+
const handled = new Set<string>();
|
|
614
|
+
|
|
615
|
+
async function handleInteraction(handle: InteractionHandle) {
|
|
616
|
+
const request = handle.interaction;
|
|
617
|
+
// 创建、状态更新和断线回放都会回调:每个待处理的交互只处理一次
|
|
618
|
+
if (request.status !== "pending" || handled.has(request.id)) return;
|
|
619
|
+
handled.add(request.id);
|
|
620
|
+
|
|
621
|
+
switch (request.type) {
|
|
622
|
+
case "site_login": {
|
|
623
|
+
const login = await handle.openLogin(); // 打开受控登录浏览器,返回实时画面 login.liveUrl
|
|
624
|
+
await showToUser(login.liveUrl); // 用户点「我已登录」后返回
|
|
625
|
+
await handle.confirmSignedIn(); // 保存登录状态,Agent 复查后继续
|
|
626
|
+
break;
|
|
627
|
+
}
|
|
628
|
+
case "take_control": {
|
|
629
|
+
// 服务端提供开始接管时,connectControl() 返回签名后的控制地址
|
|
630
|
+
const control = handle.can("connect_control") ? await handle.connectControl() : undefined;
|
|
631
|
+
await showToUser(control?.liveUrl ?? request.payload.liveUrl);
|
|
632
|
+
await handle.releaseControl(); // 把浏览器交还给 Agent
|
|
633
|
+
break;
|
|
634
|
+
}
|
|
635
|
+
case "ask_user": // payload:{ question, answerType, options? }
|
|
636
|
+
await handle.answer(await askUser(request.payload));
|
|
637
|
+
break;
|
|
638
|
+
case "confirmation": // payload:{ summary }
|
|
639
|
+
if (await approve(request.payload.summary)) await handle.confirm();
|
|
640
|
+
else await handle.reject();
|
|
641
|
+
break;
|
|
642
|
+
}
|
|
620
643
|
}
|
|
644
|
+
|
|
645
|
+
const result = await run.wait({ onInteraction: handleInteraction });
|
|
621
646
|
```
|
|
622
647
|
|
|
623
|
-
`
|
|
648
|
+
交互按顺序一个一个出现:当前这个处理完,run 才会继续。同一个交互在每次状态变化和断线回放时都会再次回调,`id` 和 `type` 始终不变——界面卡片请按 `request.id` 更新。带有新内容的更新会替换卡片(例如接管变成 `active` 时带来新的 `liveUrl`);后端用空壳结束交互时,卡片保留原来的 `payload`。直接以 `active` 出现的 `take_control` 是用户自己发起的接管(例如在 Console 里),界面提供接管入口时再展示它。方法:`answer(value)`、`skip()`、`confirm()`、`reject()`、`openLogin({ siteId?, profileId? })`、`confirmSignedIn()`、`connectControl()`、`refreshControl()`、`releaseControl()`、`retry()`、`switchProfile()`。`connectControl()` 和 `refreshControl()` 返回 `{ liveUrl, expiresAt?, status?, raw }`:用户正在控制的那个浏览器的签名地址。`handle.can(kind)` 判断该交互当前是否提供这个动作;调用后端没声明的方法会抛错(所以「恢复」按钮只有在真能用时才出现)。
|
|
649
|
+
|
|
650
|
+
**`ask_user` 的回答是结构化的。** `answer()` 接收与 `payload.answerType` 对应的值:`text` → 字符串,`number` → 数字,`boolean` → 布尔,`single_choice` → 一个 `options[].value`,`multiple_choice` → 选项 value 数组;不匹配时在发送前就抛出 `QoniValidationError`。线上协议里这个类型仍叫 `clarification`,SDK 统一呈现为 `ask_user`(候选项是 `{ value, label }`)。
|
|
651
|
+
|
|
652
|
+
**登录只需两步。** `openLogin()` 为 `site_login` 交互打开一个实时登录浏览器(`siteId` 取自 `payload.sites[].siteId`,只列出一个站点时可以省略)。用户登录完成后调用 `confirmSignedIn()`:SDK 先保存这次登录,之后的任务直接复用,再通知 Agent 复查站点并继续;如果用户其实没有登录成功,Agent 会再发起一次 `site_login`。无论调用几次,同一个登录会话只保存一次、只通知 Agent 复查一次。后端暂时没能保存登录时,它抛出 `code` 为 `site_login.save_failed` 的可重试 `QoniError`,不会让 Agent 继续;再调用一次即可。保存期间用户又重开了登录时,抛出的是 `site_login.session_replaced`,处理方式相同。同一个 `RunHandle`(或 `MonitorHandle`)发出的句柄共享登录会话,所以在之后回调拿到的句柄上调用 `confirmSignedIn()` 也有效;重新 `attach()` 得到的 RunHandle 不共享。委托时需要申请 `QoniScopes.SITE_LOGIN_REQUEST` 和 `QoniScopes.SITE_LOGIN_CONFIRM`,并在 AccessKey 权限策略中允许这两个 scope。`openLogin()` 返回值上的 `confirm()` 仍然保留,供自己保存登录的调用方使用,已标记为弃用。
|
|
624
653
|
|
|
625
|
-
上面用到的判别字符串也都导出成了常量,避免裸写字面量:`InteractionTypes`(`
|
|
654
|
+
上面用到的判别字符串也都导出成了常量,避免裸写字面量:`InteractionTypes`(`SiteLogin` / `TakeControl` / `AskUser` / `Confirmation` / `Wait`;已弃用的 `Clarification` 现在等于 `AskUser`)、`InteractionStatuses`、`ActionKinds`(`handle.can(...)` 的动作名)。授权 scope 同样有:`QoniScopes`(每个单独 scope)、`QoniProductScopes`(`products` 语法糖展开成的 per-product `read`+`manage` 对)、`QoniScopeBundles`(精选 bundle)。用它们代替手写字符串:
|
|
626
655
|
|
|
627
656
|
```ts
|
|
628
657
|
import { InteractionTypes, ActionKinds, QoniScopes } from "@qoniai/qoni";
|
|
629
658
|
|
|
630
|
-
if (
|
|
631
|
-
await
|
|
659
|
+
if (handle.type === InteractionTypes.AskUser && handle.can(ActionKinds.answer)) {
|
|
660
|
+
await handle.answer("蓝色那个按钮");
|
|
632
661
|
}
|
|
633
662
|
```
|
|
634
663
|
|
|
@@ -638,7 +667,7 @@ if (i.type === InteractionTypes.Clarification && i.can(ActionKinds.answer)) {
|
|
|
638
667
|
|
|
639
668
|
原始 wire 事件类型(`run.action.started` 这种内部传输名,命名是历史遗留、也不一致)**不作为公开导出**——开发者按上面那套精选 `event.type` 匹配即可;极少数要精确 wire 的场景,`event.raw.event` 是个原始字符串,直接比对。
|
|
640
669
|
|
|
641
|
-
Browser Use 的 scope 已预留,但产品运行时 SDK 方法导出前不作为公开方法说明。Site Login 的 scope(`QoniScopes.SITE_LOGIN_REQUEST`、`QoniScopes.SITE_LOGIN_CONFIRM`)用于运行交互上的 `InteractionHandle.openLogin()` 和 `
|
|
670
|
+
Browser Use 的 scope 已预留,但产品运行时 SDK 方法导出前不作为公开方法说明。Site Login 的 scope(`QoniScopes.SITE_LOGIN_REQUEST`、`QoniScopes.SITE_LOGIN_CONFIRM`)用于运行交互上的 `InteractionHandle.openLogin()` 和 `confirmSignedIn()`。
|
|
642
671
|
|
|
643
672
|
### 返回结构
|
|
644
673
|
|