@mindstudio-ai/agent 0.1.171 → 0.1.173

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/llms.txt CHANGED
@@ -378,8 +378,11 @@ Scan text for personally identifiable information using Microsoft Presidio.
378
378
  Download a video file
379
379
  - Works with YouTube, TikTok, etc., by using ytdlp behind the scenes
380
380
  - Can save as mp4 or mp3
381
- - Input: `{ videoUrl: string, format: "mp4" | "mp3" }`
382
- - Output: `{ videoUrl: string }`
381
+ - `item` picks one item of a URL that holds several: a carousel, a playlist, a post with several videos. A link to one slide of a carousel downloads that slide. `itemCount` in the output says how many there are, so download item 1 and loop for the rest
382
+ - `start` and `duration` (seconds) download a clip. An mp4 clip is cut without re-encoding, so it can start a moment before `start` (for an exact cut, trim the result with `trimMedia`); an mp3 clip is exact
383
+ - Videos and audio only: a photo (a photo post, a photo in a carousel) fails as `video_unavailable`, and so does an item the URL doesn't have
384
+ - Input: `{ videoUrl: string, format: "mp4" | "mp3", item?: number, start?: number, duration?: number }`
385
+ - Output: `{ videoUrl: string, title?: string, durationSec?: number, itemCount?: number }`
383
386
 
384
387
  #### enhanceImageGenerationPrompt
385
388
  Generate or enhance an image generation prompt using a language model. Optionally generates a negative prompt.
@@ -2020,3 +2023,26 @@ The frontend consumes with `events.connect({ getToken, onEvent, onConnect })` fr
2020
2023
  **A channel is an audience, and a method decides who is in it.** Never put two users' data on one channel. Default to per-user channels with publish-time fan-out (as above) for membership-gated audiences — removing someone stops their events immediately; reserve a shared channel for genuinely broadcast content (a ticker, a live blog), where revocation waits out the grant TTL. For anonymous visitors `auth.userId` is null — key their channels on `session.visitorId` instead, or `user:null` becomes one channel shared by every anonymous user.
2021
2024
 
2022
2025
  **Events are at-most-once nudges.** Nothing is buffered while a client is disconnected and nothing replays on connect — clients refetch state in `onConnect` (subscribe for speed, reconcile for truth). `delivered: 0` means nobody is listening right now: normal, never an error. Every delivered frame carries the platform-stamped publish `id` (consumer dedupe key: `id` + `channel`); `publish` returns the same id. Payloads cap at 256k serialized chars, checked in the SDK before the network so oversize throws locally — when a publish carries data, publish before committing the write it announces; high-rate paths publish ids (`{ type, id }`) and let the client fetch. Channels are exact strings (letters, digits, `: _ - .` — no wildcards), ≤500 per publish, ≤100 per grant. Publishes and grants are scoped to the execution's environment (live/preview/dev) automatically, so a dev-session publish can never reach live users. Debug with `remy-admin events tail | publish | channels list`.
2026
+
2027
+ #### `browser` — a real Chrome the backend drives
2028
+
2029
+ Sign in to a site, work through its screens, read what it shows, keep screenshots. `browser.open()` starts a session: one browser with one page, open across calls until `close()`, its idle timeout (`idleTimeoutSeconds`, five minutes by default, up to 30) or 55 minutes. Backend only (hook token).
2030
+
2031
+ ```typescript
2032
+ import { browser } from '@mindstudio-ai/agent';
2033
+
2034
+ const session = await browser.open({ url: 'https://app.example.com/login', viewport: 'mobile' }); // or 'desktop' (default), { width, height }
2035
+ await session.type({ label: 'Email' }, process.env.DEMO_EMAIL!);
2036
+ await session.type({ label: 'Password' }, process.env.DEMO_PASSWORD!, { submit: true });
2037
+ await session.waitFor('Dashboard');
2038
+ const clicked = await session.click({ role: 'link', text: 'Settings' }); // { url, title, snapshot, matched, box }
2039
+ const shot = await session.screenshot({ store: Shots, boxes: ['e12'] }); // StoredFile & { width, height, deviceScaleFactor, boxes }; no store → { data: Buffer, ... }
2040
+ const later = await browser.session(session.id); // pick it up from another call (a cron resuming a run)
2041
+ await session.close();
2042
+ ```
2043
+
2044
+ Every action returns the page afterward, `{ url, title, snapshot }`: `snapshot` is an outline with a ref on everything actionable (`button "Save" [ref=e12]`). Targets: a ref from the latest snapshot (`'e12'`), or `{ text }`, `{ role, text }`, `{ label }`, `{ selector }`. Element actions add `matched` and `box` (where it was in the viewport as the action landed, CSS pixels). Pages render at 2x, so a screenshot is twice the viewport's size (its `deviceScaleFactor`). The rest: `goto(url)`, `snapshot()`, `select(target, option)`, `press(key)`, `scroll({ to?, by? })`, `screenshot({ fullPage?, cursor?, boxes? })` (`cursor` keeps the session's cursor in the shot, where it last acted; `boxes` takes targets, and the shot's `boxes` say where each is in it, measured as it's taken, `null` where the page hasn't one: draw highlights from these, since a page can move after an action), `evaluate(fn, arg?)` (sent as source: no closures, pass what it needs as `arg`). Input is the browser's own; password values never appear in a snapshot. Pages go out through the deployment's exit proxy, on Remy's cloud a US residential address held for the session (the platform's own hosts, the app's included, direct), and the user agent is Chrome's with `RemyBrowser/1.0`, where to report a session and an opaque reference after it; each session's open and close, with its sites, are in the workspace's audit log. Billed for browser time per hour, from ready to its end, and that traffic per MB, so close a session when its work is done; a workspace has 10 open at once unless its limit is raised. Actions on one session run one at a time, in call order.
2045
+
2046
+ **A model drives it** through `runTask({ tools: [...session.tools({ secrets: { password: process.env.X! } })] })`: one tool, `browser`, taking a list of steps (`snapshot`, `navigate`, `click`, `type`, `select`, `press`, `scroll`, `wait`) run in order, stopping at the first that fails, answering with each step's outcome and the page after the last. A `type` step can name a secret in place of its text, which the model never sees, and only the latest page stays in full in the task's transcript (`supersedes`). With `session.tools({ screenshots: Store })` the model also gets `browser_screenshot({ question?, fullPage?, cursor? })`, which saves a screenshot to that store and answers with its `key` and a written description (an `analyzeImage` call), for what the outline can't say; those answers all stay in the transcript. **A person watches it** when a method returns `session.grantView()` (after its own access checks) and the frontend calls `browser.view(element, { getToken, onEnd? })` from `@mindstudio-ai/interface`, which draws the page into a `<canvas>`; its `state` is `connecting`, `live`, then `ended` (with `onEnd(reason)`).
2047
+
2048
+ Errors are `MindStudioError` codes: `browser_action_failed` (the page's reason; take a fresh snapshot when a ref is gone), `browser_timeout`, `browser_session_closed` / `browser_session_not_found` (open a new one), `browser_unavailable` (try again shortly), `browser_at_capacity` (the workspace's limit: close finished sessions). A session drives its one page: new tabs and iframe content aren't reachable.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@mindstudio-ai/agent",
3
- "version": "0.1.171",
3
+ "version": "0.1.173",
4
4
  "description": "TypeScript SDK for MindStudio direct step execution",
5
5
  "type": "module",
6
6
  "main": "./dist/index.js",