@aglit-ai/sdk 0.1.9 → 0.1.11

package/README.md

@@ -9,6 +9,188 @@ MIT-licensed SDK for building against the Aglit AI desktop API and tool runtime.
 
 > You must have the Aglit AI desktop app/API running locally (localhost). Set `SERVER_PORT`/`PORT`/`AGLIT_DESKTOP_BASE_URL` as needed.
 
+## Encrypted private website content
+
+The trusted `/_sites/bootstrap` shell can render a normal device-local website
+without exposing its HTML, scripts, assets or HTTP API bodies to the gateway:
+
+```ts
+import { createAuthenticatedSiteRuntimeFetch, mountEncryptedSite } from '@aglit-ai/sdk/browser';
+
+const response = await fetch('/_sites/context', { credentials: 'same-origin', cache: 'no-store' });
+if (!response.ok) throw new Error('Sign in to this site’s team');
+const context = await response.json();
+const runtimeFetch = await createAuthenticatedSiteRuntimeFetch({
+  deviceId: context.site.deviceId,
+  authenticatedEmail: context.user.email,
+});
+const surface = await mountEncryptedSite(document.querySelector('#site')!, {
+  siteName: context.site.name,
+  deviceId: context.site.deviceId,
+  agentId: context.site.agentId,
+  initialPath: new URLSearchParams(location.hash.slice(1)).get('path') ?? '/',
+  runtimeFetch,
+});
+// On teardown: await surface.destroy();
+```
+
+Serve the SDK's standalone `site-worker.js` as the trusted, authenticated
+`/_sites/worker.js`, with `Service-Worker-Allowed: /` and a JavaScript content type.
+Serve the bundled shell code at `/_sites/bootstrap.js`. The shell, worker,
+context and existing `/remote-access/api/...` endpoints remain protected by their
+canonical server authentication and route allowlists. All other site content is
+requested only through the generated `proxyPrivateSiteRequest` operation inside
+the existing signed and encrypted runtime channel. A bare site cookie never
+replaces that channel.
+
+The page holds the canonical endpoint crypto; the worker receives no keys. An
+in-memory MessageChannel binds the exact top-level shell, one-use initial iframe
+navigation, and browser-owned child client IDs. It returns decrypted responses
+locally to the iframe. It never fetches site content directly or stores content
+or keys in CacheStorage, IndexedDB, or a service-worker cache. Responses use
+`no-store` and a same-origin CSP; external content requests fail closed. The
+worker leaves only gateway-reserved `/_sites/*` and `/remote-access/api/*`
+requests to their existing authenticated handlers.
+
+The persistent shell must stay alive. A closed shell, rejected membership,
+missing endpoint verification, or unavailable crypto returns an error rather
+than plaintext fallback. A worker restart loses client custody, so the shell
+reconnects and reloads a newly owned iframe. Site documents share their site's
+origin and code trust; the iframe is a rendering surface, **not** an isolation
+boundary between the site's own scripts and its Aglit SDK capabilities.
+
+Shell links use `/_sites/bootstrap#path=<encoded-local-path>` so application
+paths and queries stay browser-local during worker navigation. Keep secrets out
+of ordinary URLs entered before the worker controls the page, as those initial
+network URLs are necessarily routing metadata.
+
+The first version supports bounded HTTP request bodies up to 1 MiB and individual
+responses up to 8 MiB. Open-ended media ranges are bounded to 1 MiB segments.
+Site cookies, worker registrations, external resources, WebSockets and unbounded
+HTTP streams are not forwarded by the content bridge. Use the direct WebRTC
+primitive below for live app/desktop video and input.
+
+## Private-site app and desktop streams
+
+Browser applications can import `@aglit-ai/sdk/browser` without loading local
+credential-file or IPC helpers. The default SDK entrypoint remains for local
+desktop/server integration.
+
+`createDirectRemoteStreamTransport(binding, { baseUrl, runtimeFetch })` connects
+to the existing authenticated direct WebRTC runtime. `mountAppStream`,
+`mountDesktopStream`, and `mountRemoteStream` provide framework-neutral video,
+connection status, and optional pointer/keyboard controls. They use the existing
+OpenAPI-generated signaling routes and direct ICE/TURN media, with the same
+`aglit-input-reliable` and `aglit-input-volatile` input channels as Aglit.
+
+```ts
+import { createAuthenticatedSiteRuntimeFetch, createDirectRemoteStreamTransport, mountAppStream } from '@aglit-ai/sdk/browser';
+
+// binding comes from your authorized backend's exact persisted game chat/window:
+// binding: { agentId, deviceId, sessionId, streamId,
+//   target: { streamType: 'app', appId, windowId } }
+const response = await fetch('/_sites/context', { credentials: 'same-origin', cache: 'no-store' });
+if (!response.ok) throw new Error('Sign in to this site’s team');
+const context = await response.json();
+if (context.site.deviceId !== binding.deviceId) throw new Error('Game device changed');
+const encryptedRuntimeFetch = await createAuthenticatedSiteRuntimeFetch({
+  deviceId: context.site.deviceId,
+  authenticatedEmail: context.user.email,
+  runtimeSessionId: binding.sessionId,
+});
+const surface = mountAppStream(document.querySelector('#game')!, {
+  title: "slurp's up!",
+  interactive: true,
+  transport: createDirectRemoteStreamTransport(binding, {
+    baseUrl: window.location.origin,
+    runtimeFetch: encryptedRuntimeFetch,
+    captureSpeakerAudio: true,
+  }),
+});
+await surface.connect();
+// During component/page cleanup:
+await surface.destroy();
+```
+
+For a desktop, use `mountDesktopStream` with
+`target: { streamType: 'desktop', displayId }` and omit `interactive`.
+Desktop streams are view-only in this version: interactive desktop mounting is
+rejected, and the direct transport rejects desktop control/input. Display video
+pixels can differ from logical native coordinates on Retina or scaled captures;
+desktop control requires the canonical verified logical display-bounds mapping.
+Supply a real persisted chat
+session, the original registered device, and an exact enumerated app window or
+display authorized by the backend. IDs select a target; they never grant access.
+The primitive does not launch apps or create account, team, or device grants.
+
+`createAuthenticatedSiteRuntimeFetch` enrolls this browser through the existing
+runtime auth/key flow. Obtain its account and device from authenticated
+`/_sites/context`, never a URL claim. It reuses canonical nonextractable browser
+keys, device identity verification and existing managed pins or fingerprint
+confirmation UI. A configured computer verification code is requested through
+a real code dialog; denied access and API-key requirements remain failures.
+The separate site cookie stays HttpOnly. No cookie or device key is copied into
+application JavaScript. The optional `runtimeSessionId` is an existing task/chat
+used to select the serving runtime generation; the auth flow allocates its own
+separate crypto session.
+
+The resulting **authenticated, encrypted runtime fetch** receives the unchanged
+generated `/remote-access/api/...` URLs on the site
+origin. The site gateway preserves their exact paths and queries, including
+URL-bound encryption proofs; it never rewrites them behind another API prefix.
+Passing ordinary `fetch` with a site cookie does not establish that runtime
+session. No broad local SDK token, browser cookie export, or provider-terminated
+media fallback is introduced.
+
+The adapter signs exact encrypted bytes and URLs with the canonical v2 proof,
+then accepts only authenticated nonce-bound successful responses. It has one
+account/device scope per page; reload after account, device or credential changes.
+
+App input is off by default. With `interactive: true`, the user explicitly selects
+**Take control** after the authorized direct input channel opens. Pointer input
+is mapped to the contained video rather than its letterbox. The default mode
+sends the canonical complete `keyEvent: press` gestures, including
+browser keyboard repeats, and complete clicks or left-button drags on pointer
+release. Interrupted pointer gestures are cancelled locally. It never leaves a
+native key or button waiting for a later release from a vanished viewer.
+
+For games that need continuous movement, explicitly set `keyboardMode: 'hold'`.
+Connection then requires the actual packaged native helper's advertised app
+held-key capability. It sends existing `down`/`up` events with one `keyHoldId`
+per physical hold; `keyHoldRenew: true` renewals preserve that identity and never repeat key-down
+callbacks. Simultaneous keys remain down together. Renewal runs every 250 ms
+only while the video has focus, the page is visible, and this surface has control.
+Blur, hidden pages, key-up and teardown stop renewal before sending releases.
+Queued down/renew packets expire before native admission; a renewal cannot
+create a hold. Native down/renew also requires the captured window to remain
+the foreground app's exact focused window, without focusing or changing that window.
+
+The native helper holds one exclusive app-keyboard owner per process,
+bound to its process birth and admitted exact window. App input and global
+keyboard events from another viewer cannot release or interrupt those keys.
+Existing authorized desktop pointer control may change focus; that stops app
+key renewal. A missing renewal expires its two-second native
+lease, checked every 100 ms; helper EOF/orderly shutdown also releases them. Cleanup uses the
+captured process identity and never focuses or resolves a replacement app.
+Expired/released hold IDs cannot be renewed into a new hold. Older runtimes
+reject held mode; they continue to support viewing and complete presses.
+Native activation and real game acceptance still require the new signed helper.
+An abrupt helper crash, SIGKILL or a suspended native process cannot run cleanup;
+the expiry guarantee requires the native helper to remain scheduled and healthy.
+Live button-held dragging and standalone modifier holds remain unsupported.
+Audio starts muted to respect browser playback requirements; a site can expose
+its own sound control using `surface.video.muted`. All viewers currently share
+the same running app and player state. Native held mode admits one input owner while
+keys are held; the Take control button itself grants no additional authority.
+
+Call `surface.destroy()` on unmount to abort signaling, cancel pending input, and
+delete only this exact viewer session. An auth/signaling failure closes the
+peer; create a new surface after restoring the authorized runtime session.
+Offer creation and local SDP readiness are bounded separately inside the full
+60-second startup deadline, matching the web viewer's browser compatibility path.
+
+See `examples/site-game.ts` for the minimal game wrapper.
+
 ## Install
 
 ```bash
@@ -94,6 +276,10 @@ socket config.
 
 Always include `x_aglit_ai_tool_reason` and pass structured arguments rather than raw shell strings. The IPC broker
 also enforces the same tool policies you see in the UI, including approval flows and logging.
+Scripts started from an Aglit chat terminal inherit `AGLIT_AGENT_ID`, `AGLIT_SESSION_ID`, and
+`AGLIT_SDK_MODULE_PATH`. Load the bundled SDK with `require(process.env.AGLIT_SDK_MODULE_PATH)`
+to avoid an older workspace copy. `executeTool` uses the inherited session ID automatically; an
+explicit request `_meta.sessionId` takes precedence.
 
 ## Examples
 
@@ -140,67 +326,23 @@ await executeTool({
 
 ### Browser Automation
 
-```ts
-// Navigate first
-const nav = await executeTool({
-  agentId: 'root-agent',
-  request: {
-    jsonrpc: '2.0',
-    id: 'browser-nav',
-    params: {
-      name: 'p1_browser',
-      arguments: {
-        action: 'navigate',
-        url: 'https://example.com',
-        x_aglit_ai_tool_reason: 'SDK example: warm up page',
-      },
-    },
-    context: { sessionId: 'sdk-browser', agentId: 'root-agent' },
-  },
-});
-
-const windowId = (nav.data?.result as any)?.windowId as string;
-
-// Inject JavaScript in that session
-await executeTool({
-  agentId: 'root-agent',
-  request: {
-    jsonrpc: '2.0',
-    id: 'browser-js',
-    params: {
-      name: 'p1_browser',
-      arguments: {
-        action: 'execute_javascript',
-        windowId,
-        code: 'document.title',
-        x_aglit_ai_tool_reason: 'SDK example: read page title',
-      },
-    },
-    context: { sessionId: 'sdk-browser', agentId: 'root-agent' },
-  },
-});
-```
-
-### Office App Automations (macOS)
+Use the SDK compatibility adapter for structured browser actions. The adapter executes
+only `p1_codex_browser` registry calls, preserves chat authorization and approvals,
+and returns the exact native tab ID. Page evaluation is read-only; use native
+click/fill actions to change a page.
 
 ```ts
-await executeTool({
-  agentId: 'root-agent',
-  request: {
-    jsonrpc: '2.0',
-    id: 'app-01',
-    params: {
-      name: 'p1_office',
-      arguments: {
-        tool: 'app',
-        app: 'calendar',
-        action: 'list',
-        x_aglit_ai_tool_reason: 'SDK example: list calendar events',
-      },
-    },
-    context: { sessionId: 'sdk-app', agentId: 'root-agent' },
-  },
+import { executeNativeControl, unwrapNativeControlResult } from '@aglit-ai/sdk';
+
+const call = (args: Record<string, unknown>, reason: string) => executeNativeControl({
+  toolName: 'p1_codex_browser', agentId: 'root-agent', sessionId: 'sdk-browser', args, reason,
 });
+const nav = unwrapNativeControlResult(await call(
+  { action: 'navigate', url: 'https://example.com' }, 'Open example.com'
+));
+if (!nav?.tabId) throw new Error('Native browser returned no exact tab ID');
+await call({ action: 'execute_javascript', tabId: nav.tabId, code: 'document.title' },
+  'Read the page title');
 ```
 
 ### Create Chat Sessions
@@ -240,3 +382,39 @@ The SDK resolves the current OpenAPI source files from `@aglit-ai/types`:
 - Desktop API: `@aglit-ai/types/schema/api.openapi.yaml`
 - Tool Builder API: `@aglit-ai/types/schema/tools-api.openapi.yaml`
 - Provider SPI API: `@aglit-ai/types/schema/provider-spi.openapi.yaml`
+
+## Native control migration
+
+`executeNativeControl({toolName, args, reason, agentId, sessionId})` adapts structured
+custom-tool browser/app actions to `p1_codex_browser` and `p1_codex_computer`. Existing
+native execute/capture/stream/focus calls pass through. The retired IDs remain rejected.
+
+```ts
+import { executeNativeControl } from '@aglit-ai/sdk';
+const result = await executeNativeControl({
+  toolName: 'p1_codex_browser',
+  args: {action: 'navigate', url: 'https://example.com'},
+  reason: 'Open the requested example page',
+  agentId, sessionId,
+});
+```
+
+The adapter keeps exact native targets, preserves typing/submit flags, returns original
+proof and familiar result fields, and never replays an uncertain mutation. Page evaluation
+is read-only. Authored DOM input plans use `aglitNativeClick(element)` and
+`aglitNativeFill(element, text)` during inspection; the adapter performs those actions
+through native input APIs afterward. No page mutation, CDP or terminal fallback is used.
+Custom runtimes package this same SDK adapter locally, so they need no new SDK dependency.
+
+Native control still requires the agent's existing unrestricted profiles and explicit
+access. Upload/file permissions remain separate native grants. Downloads return a saved
+path directly; old download progress stream IDs are unsupported. App window requests are
+verified against the private capture receipt before input; ambiguous/mismatched targets
+fail. Encoded recording and some legacy desktop-only APIs have no native equivalent.
+
+## Build and deployment
+
+Run `pnpm run build` before `pnpm run deploy`. Deploy requires the compiled
+entrypoints and bundled OpenAPI schemas, then publishes the already built SDK
+using the version committed in `package.json` and existing registry credentials.
+Deploy does not rebuild, bump versions or launch an interactive registry login.

package/browser.d.ts

@@ -0,0 +1 @@
+export * from './dist/browser';

package/browser.js

@@ -0,0 +1,2 @@
+// Separate entry keeps Node-only local credential and IPC modules out of browser bundles.
+module.exports = require('./dist/browser.js');