@qoniai/qoni 0.8.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,6 +1,106 @@
1
1
  # Changelog
2
2
 
3
- ## 0.8.0 - Unreleased
3
+ ## 0.9.0 - 2026-10-07
4
+
5
+ DoAnything interactions are now typed by what the user has to do, never by
6
+ business: `site_login`, `take_control`, `ask_user`, `confirmation`, plus the
7
+ system `wait`.
8
+
9
+ ### Changed
10
+
11
+ - `Interaction` is a union distributed over `type`: `switch (interaction.type)`
12
+ narrows `payload` in each branch, including `handle.interaction` inside
13
+ `onInteraction`. `Interaction<"site_login">` still picks one member.
14
+ `TypedInteraction` names the union. Because it is now a type alias, not an
15
+ interface, `interface X extends Interaction` no longer compiles (TS2312):
16
+ use `type X = Interaction & { ... }`. Code that builds an `Interaction` from a
17
+ separate `type` and `payload` must narrow them together or cast the object.
18
+ - The wire's `clarification` is presented as `ask_user`, with payload
19
+ `{ question, answerType, options? }`. `answerType` is `text`, `number`,
20
+ `boolean`, `single_choice` or `multiple_choice`, and options are
21
+ `{ value, label }`. A clarification with choices becomes `single_choice`; one
22
+ without choices is `text`. A native `ask_user` keeps the `answer_type` it
23
+ sends, and every string option value is kept, including `""`.
24
+ Migration: compare `type` with `"ask_user"` instead of `"clarification"`
25
+ (TypeScript no longer compiles the old comparison); read
26
+ `payload.answerType` / `payload.options` instead of `payload.inputType`
27
+ (removed) or the answer action's `inputSchema` enum.
28
+ `InteractionTypes.Clarification` is deprecated and now equals
29
+ `InteractionTypes.AskUser`: it still matches interactions the SDK hands out,
30
+ but not a raw wire `type` or a `"clarification"` string you stored.
31
+ `ClarificationPayload` is a deprecated alias of `AskUserPayload`, which
32
+ requires `answerType`.
33
+ - `InteractionHandle.answer(value)` accepts the structured value its
34
+ `answerType` names (string, number, boolean, one option value, or an array of
35
+ option values) and throws `QoniValidationError` before sending a value that
36
+ does not fit.
37
+ - `InteractionHandle.confirmSignedIn()` saves the login opened by `openLogin()`
38
+ on the same interaction (`POST /profiles/{profileId}/confirm`, scope
39
+ `webagent.site_login:confirm`), then tells the agent to recheck. Sign-in is
40
+ `openLogin()` → `confirmSignedIn()`:
41
+ - It saves once and rechecks once per login session, however many handles
42
+ or concurrent calls ask; the deprecated `OpenLoginResult.confirm()` shares
43
+ the same save.
44
+ - A user who is not signed in yet, or 409 `no_pending_login`, still gets a
45
+ recheck: the agent raises a new `site_login` when needed.
46
+ - When the backend keeps the session but could not save it
47
+ (`probeResult: "release_failed"`), it throws a retryable `QoniError` coded
48
+ `site_login.save_failed` and does not tell the agent to go on; calling it
49
+ again saves again. Any other save failure is thrown as well.
50
+ - When `openLogin()` calls overlap, the latest call that succeeds owns the
51
+ session, whichever response arrives last.
52
+ - Reopening that returns the same backend session shares its save; when a
53
+ newer backend session replaces it, `confirm()` on the old result throws
54
+ `QoniValidationError` instead of saving. If the login is reopened while a
55
+ `confirmSignedIn()` is still saving, that call throws a retryable
56
+ `QoniError` coded `site_login.session_replaced` without telling the agent
57
+ to go on; call it again to save the newer login.
58
+ - Handles from the same `RunHandle` or `MonitorHandle` share the login
59
+ session, so a handle from a later callback saves it too. A RunHandle from
60
+ another `attach()` or process does not.
61
+ - `connectControl()` and `refreshControl()` resolve to a `ControlSession`
62
+ (`{ liveUrl, expiresAt?, status?, raw }`, the signed URL of the browser the
63
+ user now controls) instead of `void`. `await handle.connectControl()` is
64
+ unaffected; code that returns the call where `Promise<void>` is expected,
65
+ such as `const cb: () => Promise<void> = () => handle.connectControl()`, no
66
+ longer compiles (TS2322) — wrap it: `async () => { await handle.connectControl(); }`.
67
+ - `run.events()`, `wait({ onInteraction })` and `monitor.events()` keep an
68
+ interaction's type and content when the backend settles it with an empty
69
+ shell (a resolve labelled `clarification` with an empty question, or a
70
+ `site_login` with one empty site): only `status`, `resolvedAt` and `actions`
71
+ change. A settled update with content of its own, such as an expired
72
+ take-over with its reason, replaces the card as before. Run and monitor
73
+ handles keep about 256 interactions, forgetting settled cards first, and
74
+ logins once they are saved or their interaction ended; a pending card or a
75
+ login still in progress is never dropped.
76
+
77
+ ### Added
78
+
79
+ - `QoniScopes.USER_PROFILE_READ` (`user.profile:read`), `USER_PROFILE_WRITE`
80
+ (`user.profile:write`), `USER_PAYMENT_USE` (`user.payment:use`),
81
+ `AGENT_MAIL_READ` (`agent.mail:read`) and `AGENT_MAIL_SEND`
82
+ (`agent.mail:send`). They belong to no product bundle: request them
83
+ explicitly, and allow them in the AccessKey policy. The constants only name
84
+ the scopes; requesting them does not by itself enable Profile, payment or
85
+ mailbox access in a run.
86
+ - Exported types `AskUserPayload`, `AskUserOption`, `AskUserAnswer`,
87
+ `AskUserAnswerType`, `InteractionBase`, `TypedInteraction` and
88
+ `ControlSession`.
89
+
90
+ ### Fixed
91
+
92
+ - A scope written without its service prefix names every matching scope:
93
+ `profile:read` suggests both `gumem.profile:read` and `user.profile:read`.
94
+
95
+ ### Deprecated
96
+
97
+ - `user` / `userId` on interactive delegation input: the authorization page
98
+ identifies the signed-in user, and the delegation service ignores them.
99
+ - `DelegateTokenResponse.grantedScopes`: the delegation service does not return
100
+ it. Read `qoni.genauth.introspectDelegationToken({ token }).data.scope`.
101
+ - `OpenLoginResult.confirm()`: `confirmSignedIn()` saves the login itself.
102
+
103
+ ## 0.8.0 - 2026-10-04
4
104
 
5
105
  ### Added
6
106
 
@@ -150,7 +250,7 @@ compatible fixes.
150
250
  - Optional abort signals on product run-detail and artifact-list wire reads;
151
251
  existing calls without a signal remain compatible.
152
252
 
153
- ## Unreleased
253
+ ## Earlier changes
154
254
 
155
255
  ### Breaking
156
256
 
package/README.md CHANGED
@@ -151,7 +151,7 @@ const run = await qoni.doAnything.run({
151
151
 
152
152
  const result = await run.wait({
153
153
  onScreenshot: (img, i) => fs.writeFileSync(`step-${i}.jpg`, img.bytes),
154
- onInteraction: (i) => { // login / clarify / confirm / take-control
154
+ onInteraction: (i) => { // site_login / take_control / ask_user / confirmation
155
155
  if (i.can("confirm_signed_in")) return i.confirmSignedIn();
156
156
  if (i.can("confirm")) return i.confirm();
157
157
  },
@@ -286,18 +286,27 @@ const grant = await qoni.delegateToken({
286
286
  scopes: ["gumem.memory:read", "webagent.do_anything:manage"],
287
287
  });
288
288
 
289
+ // The callback will not carry grantId: save it with your business state, keyed by grantState.
290
+ await savePendingGrant(grant.data.grantState, {
291
+ grantId: grant.data.grantId,
292
+ businessState: "business-csrf-state",
293
+ });
289
294
  redirectUserTo(grant.data.authorizationUrl);
290
295
  ```
291
296
 
292
- The user visits `authorizationUrl` in Qoni Console/BFF. If they are not logged in, Console/BFF sends them through GenAuth login, confirms authorization, completes the grant internally, then redirects back to your `redirectUri` with a one-time `code`, your `state`, and `grant_state`/`grantId`. The browser never receives a delegation token.
297
+ The user visits `authorizationUrl` in Qoni Console/BFF. If they are not logged in, Console/BFF sends them through GenAuth login, confirms authorization, completes the grant internally, then redirects back to your `redirectUri` with a one-time `code`, your `state`, and `grant_state`. The callback does **not** carry `grantId`, so look up the record you saved by `grant_state`. The browser never receives a delegation token.
293
298
 
294
299
  Your callback handler runs on the server and exchanges the callback fields with AK/SK:
295
300
 
296
301
  ```ts
302
+ const pending = await loadPendingGrant(String(req.query.grant_state));
303
+ // Your business state comes back unchanged; reject forged or mixed-up callbacks.
304
+ if (!pending || req.query.state !== pending.businessState) throw new Error("Invalid delegation callback");
305
+
297
306
  const completed = await qoni.completeDelegateToken({
298
- grantId: String(req.query.grantId ?? req.query.grant_id),
307
+ grantId: pending.grantId,
299
308
  code: String(req.query.code),
300
- state: String(req.query.state),
309
+ state: String(req.query.grant_state), // grant_state, not your business state
301
310
  });
302
311
 
303
312
  await qoni.gumem.recall({
@@ -308,7 +317,7 @@ await qoni.gumem.recall({
308
317
 
309
318
  `completeDelegateToken` is the recommended server callback helper. Root `qoni.delegateAgent` and `qoni.completeDelegateAgent` remain as deprecated compatibility names, but the current callback contract is `{ grantId, code, state }`; old `{ code, state }` completion is not supported.
310
319
 
311
- ## Original delegation style (0.8.0, unreleased)
320
+ ## Original delegation style (0.8.0+)
312
321
 
313
322
  ```ts
314
323
  const grant = await qoni.genauth.delegateAgent({
@@ -329,8 +338,8 @@ the authorization request directly; use `completeDelegateToken()` in the server
329
338
  callback after consent. `agentKey` aliases `agent`; both labels must agree when
330
339
  supplied together. Top-level `userId` retains precedence over `user.id`.
331
340
  Expiry accepts integer seconds or whole-number
332
- `s/m/h/d` durations, between 60 and 86400 seconds. Until published, use the 0.8.0
333
- source build for these additions.
341
+ `s/m/h/d` durations, between 60 and 86400 seconds. These additions are available
342
+ in the published `@qoniai/qoni@0.8.0` release.
334
343
 
335
344
  Legacy `mailbox.*`, `web.search`, and `web.extract` scopes, and `deniedScopes`, are unsupported and rejected.
336
345
  They are never expanded into broader product permissions. Product scopes do not
@@ -348,7 +357,9 @@ await qoni.delegateToken({
348
357
  });
349
358
  ```
350
359
 
351
- Scope strings are validated locally before any request: a scope missing its service prefix (e.g. `do_anything:manage` instead of `webagent.do_anything:manage`) throws `QoniValidationError` immediately with the correct form in the message.
360
+ Scope strings are validated locally before any request: a scope missing its service prefix (e.g. `do_anything:manage` instead of `webagent.do_anything:manage`) throws `QoniValidationError` immediately with the correct form in the message — every candidate when the short form is ambiguous (`profile:read` → `gumem.profile:read` or `user.profile:read`).
361
+
362
+ The site-login, user Profile, payment and Agent mailbox scopes (`QoniScopes.SITE_LOGIN_*`, `USER_PROFILE_READ`, `USER_PROFILE_WRITE`, `USER_PAYMENT_USE`, `AGENT_MAIL_READ`, `AGENT_MAIL_SEND`) belong to no product: request them explicitly in `scopes`, and allow them in the AccessKey policy. The user Profile, payment and Agent mailbox constants only name the scopes; requesting them does not by itself enable Profile, payment or mailbox access in a run.
352
363
 
353
364
  Use explicit scope strings when the requested permission boundary should be visible in code.
354
365
 
@@ -360,6 +371,10 @@ Use explicit scope strings when the requested permission boundary should be visi
360
371
  | Search public web | `webagent.web_search:read`, `webagent.web_search:manage` | Agent can search public pages and read search results. |
361
372
  | Run a bounded web task | `webagent.do_anything:read`, `webagent.do_anything:manage` | Agent can run a visible web task, read progress, stop it, and intervene when needed. |
362
373
  | Create a monitor | `webagent.track:read`, `webagent.track:manage` | Agent can configure monitors, read events, run checks, and stop monitors. |
374
+ | Sign in to sites during a run | `webagent.site_login:request`, `webagent.site_login:confirm` | The user signs in through a live browser; the login is saved for later tasks. |
375
+ | Use the user's Profile | `user.profile:read`, `user.profile:write` | Agent can read the user's saved details (name, phone, shipping address, card brand and last 4) and save new ones the user provides. |
376
+ | Pay with the user's saved card | `user.payment:use` | After the user approves a payment, the saved card is filled in for them. |
377
+ | Use the Agent's own mailbox | `agent.mail:read`, `agent.mail:send` | Agent can read mail sent to its own address (such as verification codes) and send mail from it. |
363
378
 
364
379
  Silent mode is usually appropriate for low-risk actions such as reading current-user memory, writing a confirmed summary, or searching public web pages.
365
380
 
@@ -634,81 +649,113 @@ the backend contract.
634
649
 
635
650
  ### Interactions (HITL)
636
651
 
637
- When a run needs the user — sign in to a site, answer a clarification, approve a
638
- plan, take over the browser, or wait — it emits an `interaction` event whose
639
- `event.data` is a typed `Interaction` (discriminated by `type`: `site_login` /
640
- `clarification` / `confirmation` / `take_control` / `wait`). Wrap it in a handle
641
- and call the action methods; each posts to the endpoint the backend declared in
642
- the interaction's `actions[]`, so you never hard-code a route:
652
+ When a run needs the user, it pauses and emits an `interaction` event whose
653
+ `event.data` is a typed `Interaction`. Interactions are typed by what the user
654
+ has to do, never by business, so one handler covers every task:
643
655
 
644
- ```ts
645
- for await (const event of run.events()) {
646
- if (event.type === "interaction") {
647
- const i = run.interactionHandle(event.data); // event.data is the Interaction
648
- if (i.type === "clarification" && i.can("answer")) await i.answer("the blue button");
649
- else if (i.can("confirm")) await i.confirm();
650
- else if (i.can("confirm_signed_in")) await i.confirmSignedIn();
651
- else if (i.can("release_control")) await i.releaseControl();
652
- }
653
- }
654
- ```
655
-
656
- Methods: `answer(text)`, `skip()`, `confirm()`, `reject()`, `openLogin({ siteId?, profileId? })`,
657
- `confirmSignedIn()`, `connectControl()`, `refreshControl()`, `releaseControl()`,
658
- `retry()`, `switchProfile()`. `i.can(kind)` reports whether the interaction
659
- currently offers that action; calling a method the backend did NOT declare
660
- throws (so a "recovery" button only exists when it actually works).
656
+ | `type` | The user… | Methods |
657
+ | --- | --- | --- |
658
+ | `site_login` | signs in to a site in the live browser | `openLogin()`, `confirmSignedIn()` |
659
+ | `take_control` | operates the browser by hand (captcha, 2FA, …) | `connectControl()`, `refreshControl()`, `releaseControl()` |
660
+ | `ask_user` | answers a question shaped by `answerType` / `options` | `answer()`, `skip()` |
661
+ | `confirmation` | allows or rejects an action | `confirm()`, `reject()` |
662
+ | `wait` | nothing — the system is waiting (rate limit, external event) | `retry()` |
661
663
 
662
- `openLogin()` starts an interactive login session for a `site_login`
663
- interaction of a run. Pass `siteId` (one of
664
- `i.interaction.payload.sites[].siteId`), or omit it when exactly one site is
665
- listed. Show the returned `liveUrl` to the user; after they sign in, persist the
666
- login with `login.confirm()`, then tell the agent with `confirmSignedIn()`:
664
+ `Interaction` is a union distributed over `type`, so `switch (request.type)`
665
+ narrows `request.payload` in each branch. Each method posts to the endpoint the
666
+ backend declared in the interaction's `actions[]`, so you never hard-code a route:
667
667
 
668
668
  ```ts
669
- import { QoniError } from "@qoniai/qoni";
669
+ import type { InteractionHandle } from "@qoniai/qoni";
670
670
 
671
- const login = await i.openLogin(); // { profileId, siteId, sessionId, liveUrl }
672
- showToUser(login.liveUrl); // the user signs in there
673
- await waitForUserToFinish();
674
- let result;
675
- try {
676
- result = await login.confirm(); // ends the session; saves the login if found
677
- } catch (error) {
678
- // 409: no pending login for this profile (session already ended, or profile gone).
679
- if (!(error instanceof QoniError && error.code === "no_pending_login")) throw error;
680
- }
681
- if (!result || result.active || result.probeResult === "no_session") {
682
- await i.confirmSignedIn(); // the agent rechecks; it asks again if still signed out
683
- } else {
684
- // The user was not signed in, and confirm() closed that session's browser.
685
- const retry = await i.openLogin({ siteId: login.siteId, profileId: login.profileId });
686
- showToUser(retry.liveUrl); // a new session for the same profile
671
+ const handled = new Set<string>();
672
+
673
+ async function handleInteraction(handle: InteractionHandle) {
674
+ const request = handle.interaction;
675
+ // Creation, status updates and replays all call back: act once per pending id.
676
+ if (request.status !== "pending" || handled.has(request.id)) return;
677
+ handled.add(request.id);
678
+
679
+ switch (request.type) {
680
+ case "site_login": {
681
+ const login = await handle.openLogin(); // { liveUrl, ... } of a live login browser
682
+ await showToUser(login.liveUrl); // resolves when the user clicks "I'm signed in"
683
+ await handle.confirmSignedIn(); // saves the login, then the agent rechecks and continues
684
+ break;
685
+ }
686
+ case "take_control": {
687
+ // connectControl() resolves to the signed control URL, when the backend offers it
688
+ const control = handle.can("connect_control") ? await handle.connectControl() : undefined;
689
+ await showToUser(control?.liveUrl ?? request.payload.liveUrl);
690
+ await handle.releaseControl(); // hand the browser back
691
+ break;
692
+ }
693
+ case "ask_user": // payload: { question, answerType, options? }
694
+ await handle.answer(await askUser(request.payload));
695
+ break;
696
+ case "confirmation": // payload: { summary }
697
+ if (await approve(request.payload.summary)) await handle.confirm();
698
+ else await handle.reject();
699
+ break;
700
+ }
687
701
  }
688
- ```
689
702
 
690
- `probeResult: "no_session"` means the backend no longer holds the login session —
691
- it may already have saved the login on its own once it detected the sign-in, or
692
- it restarted. Some browser providers instead reject `confirm()` with a
693
- `QoniError` coded `no_pending_login` (HTTP 409), which the backend also uses when
694
- the profile no longer exists for this user; the example handles it the same way,
695
- because the agent rechecks and asks again if the site is still signed out. Delegate
703
+ const result = await run.wait({ onInteraction: handleInteraction });
704
+ ```
705
+
706
+ Interactions arrive one at a time: the run waits until the current one is
707
+ handled. The same interaction calls back again on every status change and on
708
+ reconnect replays, always with the same `id` and `type` — key your UI cards by
709
+ `request.id`. An update with content of its own replaces the card (a take-over
710
+ that became `active` carries its new `liveUrl`); when the backend settles an
711
+ interaction with an empty shell, the card keeps its original `payload`. A
712
+ `take_control` that arrives already `active` was started by the user, for
713
+ example from Console; show it if your UI offers control there. Methods: `answer(value)`, `skip()`, `confirm()`,
714
+ `reject()`, `openLogin({ siteId?, profileId? })`, `confirmSignedIn()`,
715
+ `connectControl()`, `refreshControl()`, `releaseControl()`, `retry()`,
716
+ `switchProfile()`. `connectControl()` and `refreshControl()` resolve to
717
+ `{ liveUrl, expiresAt?, status?, raw }`: the signed URL of the browser the user
718
+ now controls. `handle.can(kind)` reports whether the interaction currently
719
+ offers that action; calling a method the backend did NOT declare throws (so a
720
+ "recovery" button only exists when it actually works).
721
+
722
+ **`ask_user` answers are structured.** `answer()` takes the value matching
723
+ `payload.answerType` — `text` → string, `number` → number, `boolean` → boolean,
724
+ `single_choice` → one `options[].value`, `multiple_choice` → an array of option
725
+ values — and throws `QoniValidationError` before sending anything that does not
726
+ fit. The wire still calls this type `clarification`; the SDK presents it as
727
+ `ask_user` (choices arrive as `{ value, label }` options).
728
+
729
+ **Sign-in takes two calls.** `openLogin()` opens a live login browser for the
730
+ `site_login` interaction (pass `siteId` from `payload.sites[].siteId`, or omit it
731
+ when exactly one site is listed). Once the user is signed in, `confirmSignedIn()`
732
+ saves the login so later tasks reuse it, then tells the agent to recheck the site
733
+ and continue; if the user turns out not to be signed in, the agent raises a new
734
+ `site_login`. It saves once and rechecks once per login session, however many
735
+ times it is called. If the backend could not save the login yet, it throws a
736
+ retryable `QoniError` coded `site_login.save_failed` without telling the agent to
737
+ go on; call it again. The same holds for `site_login.session_replaced`, thrown
738
+ when the login was reopened while it was being saved. Handles from the same `RunHandle` (or `MonitorHandle`)
739
+ share the login session, so `confirmSignedIn()` also works on the handle of a
740
+ later callback; a RunHandle from a new `attach()` does not. Delegate
696
741
  `QoniScopes.SITE_LOGIN_REQUEST` and `QoniScopes.SITE_LOGIN_CONFIRM`, and allow
697
- both in the AccessKey policy.
698
-
699
- The discriminant strings used above are also exported as constants so you can
700
- avoid bare literals: `InteractionTypes` (`site_login` / `clarification` /
701
- `confirmation` / `take_control` / `wait`), `InteractionStatuses`, and
702
- `ActionKinds` (the `i.can(...)` action names). Authorization scope strings have
703
- the same treatment — `QoniScopes` (every individual scope), `QoniProductScopes`
704
- (the per-product `read`+`manage` pairs the `products` sugar expands to), and
705
- `QoniScopeBundles` (curated bundles). Use them in place of hand-typed strings:
742
+ both in the AccessKey policy. `openLogin()`'s result still offers `confirm()` for
743
+ callers that save the login themselves; it is deprecated.
744
+
745
+ The discriminant strings are also exported as constants so you can avoid bare
746
+ literals: `InteractionTypes` (`SiteLogin` / `TakeControl` / `AskUser` /
747
+ `Confirmation` / `Wait`; the deprecated `Clarification` now equals `AskUser`),
748
+ `InteractionStatuses`, and `ActionKinds` (the `handle.can(...)` action names).
749
+ Authorization scope strings have the same treatment — `QoniScopes` (every
750
+ individual scope), `QoniProductScopes` (the per-product `read`+`manage` pairs the
751
+ `products` sugar expands to), and `QoniScopeBundles` (curated bundles). Use them in
752
+ place of hand-typed strings:
706
753
 
707
754
  ```ts
708
755
  import { InteractionTypes, ActionKinds, QoniScopes } from "@qoniai/qoni";
709
756
 
710
- if (i.type === InteractionTypes.Clarification && i.can(ActionKinds.answer)) {
711
- await i.answer("the blue button");
757
+ if (handle.type === InteractionTypes.AskUser && handle.can(ActionKinds.answer)) {
758
+ await handle.answer("the blue button");
712
759
  }
713
760
  ```
714
761
 
@@ -718,7 +765,7 @@ The 1:1 low-level methods live under `qoni.<product>.api.*` (e.g. `qoni.doAnythi
718
765
 
719
766
  `event.raw.event` is the **raw wire event type** — an internal transport name (`run.action.started`, `run.section_completed`, …) whose naming is historical and inconsistent, exposed as a plain string. This is an implementation detail, not the product API: developers match on the curated `event.type` above. The full wire-type catalog is intentionally NOT exported — reach for `event.raw.event` (a string) only when you need an exact wire match the curated layer folds into `progress` (e.g. telling a specific browser action apart).
720
767
 
721
- Browser Use scopes are reserved until their product runtime SDK methods are exported. Site Login scopes (`QoniScopes.SITE_LOGIN_REQUEST`, `QoniScopes.SITE_LOGIN_CONFIRM`) back `InteractionHandle.openLogin()` and `confirm()` on run interactions.
768
+ Browser Use scopes are reserved until their product runtime SDK methods are exported. Site Login scopes (`QoniScopes.SITE_LOGIN_REQUEST`, `QoniScopes.SITE_LOGIN_CONFIRM`) back `InteractionHandle.openLogin()` and `confirmSignedIn()` on run interactions.
722
769
 
723
770
  ### Response Shape
724
771
 
package/README.zh-CN.md CHANGED
@@ -144,7 +144,7 @@ const run = await qoni.doAnything.run({
144
144
 
145
145
  const result = await run.wait({
146
146
  onScreenshot: (img, i) => fs.writeFileSync(`step-${i}.jpg`, img.bytes),
147
- onInteraction: (i) => { // 登录 / 澄清 / 确认 / 接管
147
+ onInteraction: (i) => { // site_login / take_control / ask_user / confirmation
148
148
  if (i.can("confirm_signed_in")) return i.confirmSignedIn();
149
149
  if (i.can("confirm")) return i.confirm();
150
150
  },
@@ -257,18 +257,27 @@ const grant = await qoni.delegateToken({
257
257
  scopes: ["gumem.memory:read", "webagent.do_anything:manage"],
258
258
  });
259
259
 
260
+ // 回调不会带回 grantId:按 grantState 保存 grantId 和业务 state
261
+ await savePendingGrant(grant.data.grantState, {
262
+ grantId: grant.data.grantId,
263
+ businessState: "business-csrf-state",
264
+ });
260
265
  redirectUserTo(grant.data.authorizationUrl);
261
266
  ```
262
267
 
263
- 用户访问 `authorizationUrl` 进入 Qoni Console/BFF。若用户未登录,Console/BFF 会先走 GenAuth 登录;授权确认后由 Console/BFF 统一 complete,再回跳业务 `redirectUri`,只带一次性 `code`、业务 `state`、`grant_state`/`grantId`。浏览器不会拿到 delegation token。
268
+ 用户访问 `authorizationUrl` 进入 Qoni Console/BFF。若用户未登录,Console/BFF 会先走 GenAuth 登录;授权确认后由 Console/BFF 统一 complete,再回跳业务 `redirectUri`,只带一次性 `code`、业务 `state` 和 `grant_state`,**不带 `grantId`**,需要按 `grant_state` 找回创建请求时保存的记录。浏览器不会拿到 delegation token。
264
269
 
265
270
  业务 callback handler 在服务端用 AK/SK 兑换最终 token:
266
271
 
267
272
  ```ts
273
+ const pending = await loadPendingGrant(String(req.query.grant_state));
274
+ // 业务 state 原样回到回调里,和保存的值比对,防止回调被伪造或串用
275
+ if (!pending || req.query.state !== pending.businessState) throw new Error("Invalid delegation callback");
276
+
268
277
  const completed = await qoni.completeDelegateToken({
269
- grantId: String(req.query.grantId ?? req.query.grant_id),
278
+ grantId: pending.grantId,
270
279
  code: String(req.query.code),
271
- state: String(req.query.state),
280
+ state: String(req.query.grant_state), // 传 grant_state,不是业务 state
272
281
  });
273
282
 
274
283
  await qoni.gumem.recall({
@@ -279,7 +288,7 @@ await qoni.gumem.recall({
279
288
 
280
289
  `completeDelegateToken` 是推荐的服务端 callback helper。顶层 `qoni.delegateAgent` 和 `qoni.completeDelegateAgent` 仍保留为 deprecated 兼容命名,但当前 callback 契约是 `{ grantId, code, state }`;旧的 `{ code, state }` complete 签名不再兼容。
281
290
 
282
- ## 原始委托写法(0.8.0,尚未发布)
291
+ ## 原始委托写法(0.8.0 起)
283
292
 
284
293
  ```ts
285
294
  const grant = await qoni.genauth.delegateAgent({
@@ -299,7 +308,7 @@ console.log(grant.grantId, grant.auditId);
299
308
  用 `completeDelegateToken()` 兑换 token。`agentKey`、`userId` 分别兼容 `agent`、
300
309
  `user.id`;同时提供两个 Agent 标签时必须一致,顶层 `userId` 保持原有的优先级。
301
310
  有效期支持整数秒或整数 `s/m/h/d` 时长,
302
- 范围为 60–86400 秒。这些新增能力在发布前需要使用 0.8.0 源码构建。
311
+ 范围为 60–86400 秒。这些新增能力已随 `@qoniai/qoni@0.8.0` 发布到 npm。
303
312
 
304
313
  旧版 `mailbox.*`、`web.search`、`web.extract` scope 和 `deniedScopes` 不受支持,SDK 会拒绝,不会将它们
305
314
  扩展成更宽的产品权限。产品 scope 不限制发送、删除邮件;任务描述不能代替禁止策略。
@@ -316,7 +325,9 @@ await qoni.delegateToken({
316
325
  });
317
326
  ```
318
327
 
319
- scope 字符串在发请求前就会本地预检:漏掉服务前缀(比如写成 `do_anything:manage` 而不是 `webagent.do_anything:manage`)会立即抛出 `QoniValidationError`,错误消息里带正确写法。
328
+ scope 字符串在发请求前就会本地预检:漏掉服务前缀(比如写成 `do_anything:manage` 而不是 `webagent.do_anything:manage`)会立即抛出 `QoniValidationError`,错误消息里带正确写法;短写法有歧义时列出全部候选(`profile:read` → `gumem.profile:read` 或 `user.profile:read`)。
329
+
330
+ 站点登录、用户 Profile、支付和 Agent 邮箱的 scope(`QoniScopes.SITE_LOGIN_*`、`USER_PROFILE_READ`、`USER_PROFILE_WRITE`、`USER_PAYMENT_USE`、`AGENT_MAIL_READ`、`AGENT_MAIL_SEND`)不属于任何产品:需要在 `scopes` 中显式申请,并在 AccessKey 权限策略中允许。用户 Profile、支付和 Agent 邮箱这几个常量只是 scope 名称:申请它们本身不会让任务获得 Profile、支付或邮箱能力。
320
331
 
321
332
  需要让授权边界直接出现在业务代码里时,使用显式 scope 字符串。
322
333
 
@@ -328,6 +339,10 @@ scope 字符串在发请求前就会本地预检:漏掉服务前缀(比如
328
339
  | 搜索公开网页 | `webagent.web_search:read`, `webagent.web_search:manage` | Agent 可以搜索公开网页并读取结果。 |
329
340
  | 执行网页任务 | `webagent.do_anything:read`, `webagent.do_anything:manage` | Agent 可以执行有边界的网页任务、读取进度、停止任务,并在需要时介入控制。 |
330
341
  | 创建长期监控 | `webagent.track:read`, `webagent.track:manage` | Agent 可以配置监控、读取事件、立即运行检查,并停止监控。 |
342
+ | 在任务中登录网站 | `webagent.site_login:request`, `webagent.site_login:confirm` | 用户在实时浏览器里亲自登录,登录状态保存后供之后的任务复用。 |
343
+ | 使用用户 Profile | `user.profile:read`, `user.profile:write` | Agent 可以读取用户保存的资料(姓名、手机号、收货地址、卡组织和后四位),并保存用户新提供的资料。 |
344
+ | 用用户保存的卡支付 | `user.payment:use` | 用户允许一笔支付后,替用户填入已保存的信用卡。 |
345
+ | 使用 Agent 自己的邮箱 | `agent.mail:read`, `agent.mail:send` | Agent 可以读取发到自己邮箱的邮件(例如验证码),并用它发送邮件。 |
331
346
 
332
347
  低风险能力通常可以静默授权,例如读取当前用户自己的 GUMem、写入本次会话摘要、搜索公开网页、读取普通任务结果。
333
348
 
@@ -580,55 +595,69 @@ interface RunResult {
580
595
 
581
596
  ### 交互(HITL)
582
597
 
583
- 当 run 需要用户参与——登录某站、回答澄清、审批计划、接管浏览器、或等待——会发出一个 `interaction` 事件,`event.data` 是带类型的 `Interaction`(用 `type` 判别:`site_login` / `clarification` / `confirmation` / `take_control` / `wait`)。把它包成句柄再调动作方法即可;每个方法会 POST 到交互 `actions[]` 里后端声明的 endpoint,不需要写死路由:
598
+ run 需要用户出手时会暂停,并发出一个 `interaction` 事件,`event.data` 是带类型的 `Interaction`。交互按「需要用户做什么」分类,和具体业务无关,所以一个处理函数就能覆盖所有任务:
584
599
 
585
- ```ts
586
- for await (const event of run.events()) {
587
- if (event.type === "interaction") {
588
- const i = run.interactionHandle(event.data); // event.data 就是 Interaction
589
- if (i.type === "clarification" && i.can("answer")) await i.answer("蓝色那个按钮");
590
- else if (i.can("confirm")) await i.confirm();
591
- else if (i.can("confirm_signed_in")) await i.confirmSignedIn();
592
- else if (i.can("release_control")) await i.releaseControl();
593
- }
594
- }
595
- ```
596
-
597
- 方法:`answer(text)`、`skip()`、`confirm()`、`reject()`、`openLogin({ siteId?, profileId? })`、`confirmSignedIn()`、`connectControl()`、`refreshControl()`、`releaseControl()`、`retry()`、`switchProfile()`。`i.can(kind)` 判断该交互当前是否提供这个动作;调用后端没声明的方法会抛错(所以「恢复」按钮只有在真能用时才出现)。
600
+ | `type` | 用户要做的事 | 方法 |
601
+ | --- | --- | --- |
602
+ | `site_login` | 在实时浏览器里亲自登录某个网站 | `openLogin()`、`confirmSignedIn()` |
603
+ | `take_control` | 亲手操作浏览器(验证码、两步验证等) | `connectControl()`、`refreshControl()`、`releaseControl()` |
604
+ | `ask_user` | 回答一个问题,答案结构由 `answerType` / `options` 描述 | `answer()`、`skip()` |
605
+ | `confirmation` | 允许或拒绝一个操作 | `confirm()`、`reject()` |
606
+ | `wait` | 不需要做决定:系统在等待(限流、外部事件) | `retry()` |
598
607
 
599
- `openLogin()` 为运行中的 `site_login` 交互开启一次交互式登录会话。传入 `siteId`(取自 `i.interaction.payload.sites[].siteId`),交互只列出一个站点时可以省略。把返回的 `liveUrl` 展示给用户;用户登录完成后,先用 `login.confirm()` 把登录态写入 Profile,再调用 `confirmSignedIn()` 通知 Agent:
608
+ `Interaction` 是按 `type` 分布的联合类型,`switch (request.type)` 之后每个分支里的 `request.payload` 会自动收窄。每个方法都 POST 到交互 `actions[]` 里后端声明的 endpoint,不需要写死路由:
600
609
 
601
610
  ```ts
602
- import { QoniError } from "@qoniai/qoni";
611
+ import type { InteractionHandle } from "@qoniai/qoni";
603
612
 
604
- const login = await i.openLogin(); // { profileId, siteId, sessionId, liveUrl }
605
- showToUser(login.liveUrl); // 用户在这里登录
606
- await waitForUserToFinish();
607
- let result;
608
- try {
609
- result = await login.confirm(); // 结束这次会话;检测到登录态就写入 Profile
610
- } catch (error) {
611
- // 409:该 Profile 没有待确认的登录(会话已结束,或 Profile 已不存在)。
612
- if (!(error instanceof QoniError && error.code === "no_pending_login")) throw error;
613
- }
614
- if (!result || result.active || result.probeResult === "no_session") {
615
- await i.confirmSignedIn(); // Agent 重新检查;仍未登录会再次请求
616
- } else {
617
- // 用户没有登录成功,而 confirm() 已关闭这次会话的浏览器。
618
- const retry = await i.openLogin({ siteId: login.siteId, profileId: login.profileId });
619
- showToUser(retry.liveUrl); // 同一个 Profile 的新登录会话
613
+ const handled = new Set<string>();
614
+
615
+ async function handleInteraction(handle: InteractionHandle) {
616
+ const request = handle.interaction;
617
+ // 创建、状态更新和断线回放都会回调:每个待处理的交互只处理一次
618
+ if (request.status !== "pending" || handled.has(request.id)) return;
619
+ handled.add(request.id);
620
+
621
+ switch (request.type) {
622
+ case "site_login": {
623
+ const login = await handle.openLogin(); // 打开受控登录浏览器,返回实时画面 login.liveUrl
624
+ await showToUser(login.liveUrl); // 用户点「我已登录」后返回
625
+ await handle.confirmSignedIn(); // 保存登录状态,Agent 复查后继续
626
+ break;
627
+ }
628
+ case "take_control": {
629
+ // 服务端提供开始接管时,connectControl() 返回签名后的控制地址
630
+ const control = handle.can("connect_control") ? await handle.connectControl() : undefined;
631
+ await showToUser(control?.liveUrl ?? request.payload.liveUrl);
632
+ await handle.releaseControl(); // 把浏览器交还给 Agent
633
+ break;
634
+ }
635
+ case "ask_user": // payload:{ question, answerType, options? }
636
+ await handle.answer(await askUser(request.payload));
637
+ break;
638
+ case "confirmation": // payload:{ summary }
639
+ if (await approve(request.payload.summary)) await handle.confirm();
640
+ else await handle.reject();
641
+ break;
642
+ }
620
643
  }
644
+
645
+ const result = await run.wait({ onInteraction: handleInteraction });
621
646
  ```
622
647
 
623
- `probeResult: "no_session"` 表示服务端已不再持有这次登录会话:它可能在检测到用户登录后已经自动保存了登录态,也可能已经重启。部分浏览器提供方会改为让 `confirm()` 抛出 `code` 为 `no_pending_login` 的 `QoniError`(HTTP 409),服务端在 Profile 已不属于当前用户或已删除时也会返回这个错误码;示例按同样方式处理,因为 Agent 会重新检查,如果站点仍未登录会再次请求。委托时需要申请 `QoniScopes.SITE_LOGIN_REQUEST` 和 `QoniScopes.SITE_LOGIN_CONFIRM`,并在 AccessKey 权限策略中允许这两个 scope。
648
+ 交互按顺序一个一个出现:当前这个处理完,run 才会继续。同一个交互在每次状态变化和断线回放时都会再次回调,`id` 和 `type` 始终不变——界面卡片请按 `request.id` 更新。带有新内容的更新会替换卡片(例如接管变成 `active` 时带来新的 `liveUrl`);后端用空壳结束交互时,卡片保留原来的 `payload`。直接以 `active` 出现的 `take_control` 是用户自己发起的接管(例如在 Console 里),界面提供接管入口时再展示它。方法:`answer(value)`、`skip()`、`confirm()`、`reject()`、`openLogin({ siteId?, profileId? })`、`confirmSignedIn()`、`connectControl()`、`refreshControl()`、`releaseControl()`、`retry()`、`switchProfile()`。`connectControl()` 和 `refreshControl()` 返回 `{ liveUrl, expiresAt?, status?, raw }`:用户正在控制的那个浏览器的签名地址。`handle.can(kind)` 判断该交互当前是否提供这个动作;调用后端没声明的方法会抛错(所以「恢复」按钮只有在真能用时才出现)。
649
+
650
+ **`ask_user` 的回答是结构化的。** `answer()` 接收与 `payload.answerType` 对应的值:`text` → 字符串,`number` → 数字,`boolean` → 布尔,`single_choice` → 一个 `options[].value`,`multiple_choice` → 选项 value 数组;不匹配时在发送前就抛出 `QoniValidationError`。线上协议里这个类型仍叫 `clarification`,SDK 统一呈现为 `ask_user`(候选项是 `{ value, label }`)。
651
+
652
+ **登录只需两步。** `openLogin()` 为 `site_login` 交互打开一个实时登录浏览器(`siteId` 取自 `payload.sites[].siteId`,只列出一个站点时可以省略)。用户登录完成后调用 `confirmSignedIn()`:SDK 先保存这次登录,之后的任务直接复用,再通知 Agent 复查站点并继续;如果用户其实没有登录成功,Agent 会再发起一次 `site_login`。无论调用几次,同一个登录会话只保存一次、只通知 Agent 复查一次。后端暂时没能保存登录时,它抛出 `code` 为 `site_login.save_failed` 的可重试 `QoniError`,不会让 Agent 继续;再调用一次即可。保存期间用户又重开了登录时,抛出的是 `site_login.session_replaced`,处理方式相同。同一个 `RunHandle`(或 `MonitorHandle`)发出的句柄共享登录会话,所以在之后回调拿到的句柄上调用 `confirmSignedIn()` 也有效;重新 `attach()` 得到的 RunHandle 不共享。委托时需要申请 `QoniScopes.SITE_LOGIN_REQUEST` 和 `QoniScopes.SITE_LOGIN_CONFIRM`,并在 AccessKey 权限策略中允许这两个 scope。`openLogin()` 返回值上的 `confirm()` 仍然保留,供自己保存登录的调用方使用,已标记为弃用。
624
653
 
625
- 上面用到的判别字符串也都导出成了常量,避免裸写字面量:`InteractionTypes`(`site_login` / `clarification` / `confirmation` / `take_control` / `wait`)、`InteractionStatuses`、`ActionKinds`(`i.can(...)` 的动作名)。授权 scope 同样有:`QoniScopes`(每个单独 scope)、`QoniProductScopes`(`products` 语法糖展开成的 per-product `read`+`manage` 对)、`QoniScopeBundles`(精选 bundle)。用它们代替手写字符串:
654
+ 上面用到的判别字符串也都导出成了常量,避免裸写字面量:`InteractionTypes`(`SiteLogin` / `TakeControl` / `AskUser` / `Confirmation` / `Wait`;已弃用的 `Clarification` 现在等于 `AskUser`)、`InteractionStatuses`、`ActionKinds`(`handle.can(...)` 的动作名)。授权 scope 同样有:`QoniScopes`(每个单独 scope)、`QoniProductScopes`(`products` 语法糖展开成的 per-product `read`+`manage` 对)、`QoniScopeBundles`(精选 bundle)。用它们代替手写字符串:
626
655
 
627
656
  ```ts
628
657
  import { InteractionTypes, ActionKinds, QoniScopes } from "@qoniai/qoni";
629
658
 
630
- if (i.type === InteractionTypes.Clarification && i.can(ActionKinds.answer)) {
631
- await i.answer("蓝色那个按钮");
659
+ if (handle.type === InteractionTypes.AskUser && handle.can(ActionKinds.answer)) {
660
+ await handle.answer("蓝色那个按钮");
632
661
  }
633
662
  ```
634
663
 
@@ -638,7 +667,7 @@ if (i.type === InteractionTypes.Clarification && i.can(ActionKinds.answer)) {
638
667
 
639
668
  原始 wire 事件类型(`run.action.started` 这种内部传输名,命名是历史遗留、也不一致)**不作为公开导出**——开发者按上面那套精选 `event.type` 匹配即可;极少数要精确 wire 的场景,`event.raw.event` 是个原始字符串,直接比对。
640
669
 
641
- Browser Use 的 scope 已预留,但产品运行时 SDK 方法导出前不作为公开方法说明。Site Login 的 scope(`QoniScopes.SITE_LOGIN_REQUEST`、`QoniScopes.SITE_LOGIN_CONFIRM`)用于运行交互上的 `InteractionHandle.openLogin()` 和 `confirm()`。
670
+ Browser Use 的 scope 已预留,但产品运行时 SDK 方法导出前不作为公开方法说明。Site Login 的 scope(`QoniScopes.SITE_LOGIN_REQUEST`、`QoniScopes.SITE_LOGIN_CONFIRM`)用于运行交互上的 `InteractionHandle.openLogin()` 和 `confirmSignedIn()`。
642
671
 
643
672
  ### 返回结构
644
673