@mindstudio-ai/remy 0.1.34 → 0.1.35

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (54) hide show
  1. package/dist/headless.js +578 -393
  2. package/dist/index.js +652 -385
  3. package/dist/prompt/sources/llms.txt +1618 -0
  4. package/dist/prompt/static/instructions.md +1 -1
  5. package/dist/prompt/static/team.md +1 -1
  6. package/dist/subagents/.notes-background-agents.md +60 -48
  7. package/dist/subagents/browserAutomation/prompt.md +14 -11
  8. package/dist/subagents/designExpert/data/sources/dev/index.html +901 -0
  9. package/dist/subagents/designExpert/data/sources/dev/serve.mjs +244 -0
  10. package/dist/subagents/designExpert/data/sources/dev/specimens-fonts.html +126 -0
  11. package/dist/subagents/designExpert/data/sources/dev/specimens-pairings.html +114 -0
  12. package/dist/subagents/designExpert/data/{fonts.json → sources/fonts.json} +0 -97
  13. package/dist/subagents/designExpert/data/sources/inspiration.json +392 -0
  14. package/dist/subagents/designExpert/prompt.md +36 -12
  15. package/dist/subagents/designExpert/prompts/animation.md +14 -6
  16. package/dist/subagents/designExpert/prompts/color.md +25 -5
  17. package/dist/subagents/designExpert/prompts/{icons.md → components.md} +17 -5
  18. package/dist/subagents/designExpert/prompts/frontend-design-notes.md +17 -122
  19. package/dist/subagents/designExpert/prompts/identity.md +15 -61
  20. package/dist/subagents/designExpert/prompts/images.md +35 -10
  21. package/dist/subagents/designExpert/prompts/layout.md +14 -9
  22. package/dist/subagents/designExpert/prompts/typography.md +39 -0
  23. package/package.json +2 -2
  24. package/dist/actions/buildFromInitialSpec.md +0 -15
  25. package/dist/actions/publish.md +0 -12
  26. package/dist/actions/sync.md +0 -19
  27. package/dist/compiled/README.md +0 -100
  28. package/dist/compiled/auth.md +0 -77
  29. package/dist/compiled/design.md +0 -251
  30. package/dist/compiled/dev-and-deploy.md +0 -69
  31. package/dist/compiled/interfaces.md +0 -238
  32. package/dist/compiled/manifest.md +0 -107
  33. package/dist/compiled/media-cdn.md +0 -51
  34. package/dist/compiled/methods.md +0 -225
  35. package/dist/compiled/msfm.md +0 -222
  36. package/dist/compiled/platform.md +0 -105
  37. package/dist/compiled/scenarios.md +0 -103
  38. package/dist/compiled/sdk-actions.md +0 -146
  39. package/dist/compiled/tables.md +0 -263
  40. package/dist/static/authoring.md +0 -101
  41. package/dist/static/coding.md +0 -29
  42. package/dist/static/identity.md +0 -1
  43. package/dist/static/instructions.md +0 -31
  44. package/dist/static/intake.md +0 -44
  45. package/dist/static/lsp.md +0 -4
  46. package/dist/static/projectContext.ts +0 -160
  47. package/dist/static/team.md +0 -39
  48. package/dist/subagents/designExpert/data/inspiration.json +0 -392
  49. package/dist/subagents/designExpert/prompts/instructions.md +0 -18
  50. /package/dist/subagents/designExpert/data/{compile-font-descriptions.sh → sources/compile-font-descriptions.sh} +0 -0
  51. /package/dist/subagents/designExpert/data/{compile-inspiration.sh → sources/compile-inspiration.sh} +0 -0
  52. /package/dist/subagents/designExpert/data/{inspiration.raw.json → sources/inspiration.raw.json} +0 -0
  53. /package/dist/subagents/designExpert/{prompts/tool-prompts → data/sources/prompts}/design-analysis.md +0 -0
  54. /package/dist/subagents/designExpert/{prompts/tool-prompts → data/sources/prompts}/font-analysis.md +0 -0
@@ -1,222 +0,0 @@
1
- # MSFM — MindStudio-Flavored Markdown
2
-
3
- Specs are written in MSFM, a strict superset of Markdown that adds **block annotations** and **inline annotations**. Annotations attach precision to prose so the compiler produces consistent results. A bare spec with no annotations is valid — each annotation makes compilation more deterministic.
4
-
5
- ## Block Annotations
6
-
7
- A tilde-fenced block (`~~~`) that attaches to the content immediately above it:
8
-
9
- ```markdown
10
- When requesting a new vendor, there are three areas of review:
11
- governance, legal, and accounting, with approvals flowing in that order.
12
-
13
- ~~~
14
- The spec says "three areas" but lists four names. From the process
15
- flowchart, it is three sequential stages:
16
-
17
- 1. Governance, Risk & Compliance (GRC) — one combined stage
18
- 2. Legal
19
- 3. Accounts Payable (AP)
20
-
21
- These are sequential. Each stage must complete before the next is
22
- notified. If any stage rejects, the entire request is rejected.
23
- ~~~
24
- ```
25
-
26
- **Rules:**
27
- - Starts and ends with `~~~` on its own line
28
- - Attaches to the nearest preceding block element (paragraph, heading, table, list)
29
- - Multiple annotations can follow the same element (read in order)
30
- - Can contain any Markdown content (use backtick code fences inside — they coexist with tilde fences)
31
-
32
- ## Inline Annotations
33
-
34
- Attach to a specific word or phrase with `[text]{content}`:
35
-
36
- ```markdown
37
- All invoices can be sent to the [Accounts Payable]{The AP team in
38
- this context refers to the internal accounts payable department, not
39
- a vendor's AP. Only users with the "ap" or "admin" role can process
40
- invoices.} team for processing against the PO.
41
- ```
42
-
43
- Good for definitions, units, and clarifications:
44
-
45
- ```markdown
46
- The PO [amount]{Total across all line items, in USD cents (integer).
47
- Does not include tax, shipping, or fees.} must not exceed the approved
48
- budget for the [cost center]{The organizational budget code that this
49
- purchase is charged to.}.
50
- ```
51
-
52
- ## Pointers
53
-
54
- When an annotation is too large for inline placement, use a pointer — an inline `{#id}` reference that points to a `~~~#id` block:
55
-
56
- ```markdown
57
- The AP team pays the vendor according to the [contract payment terms]{#payment-terms}.
58
-
59
- ~~~#payment-terms
60
- We do not model payment terms (net 30, net 60, etc.) on the PO or
61
- vendor. The dueDate on the invoice is the sole payment trigger.
62
-
63
- A production implementation would:
64
- - Store terms on the vendor record (net_30, net_60, due_on_receipt)
65
- - Auto-calculate dueDate from invoiceDate + terms
66
- - Support early payment discounts (2/10 net 30)
67
-
68
- Default behavior when no terms are specified: due on receipt.
69
- ~~~
70
- ```
71
-
72
- Keep the block co-located (right after the paragraph containing the pointer). A single block annotation can be referenced by multiple pointers — useful for concepts like "amounts are in USD cents" that apply in several places.
73
-
74
- ## Authoring Conventions
75
-
76
- **Specs are written in human language for humans.** The prose should describe what the app does the way you'd explain it to a colleague — not in code. No variable names, table names, column types, or function signatures in the prose. Technical details like column types, data representations, and implementation notes belong in annotations, where they serve as precision hints for the compiler without cluttering the readable spec.
77
-
78
- Good: "The app remembers every greeting it generates, along with who it was for."
79
- Bad: "One table: `greetings` with two columns: `name` (string) and `greeting` (string)."
80
-
81
- The annotation for the good version might be: `~~~\nOne table. Columns: the person's name (string) and the generated greeting text (string).\n~~~`
82
-
83
- **Use inline annotations liberally.** Inline annotations (`[text]{content}`) are the primary annotation tool — use them the way you'd use comments in Google Docs, attached to the specific word or phrase that needs clarification. When the prose says "the user submits a [request]{Must include vendor name, contact email, and tax ID. All fields required.} for review," the annotation is pinned to exactly the word that's ambiguous.
84
-
85
- Block annotations (`~~~...~~~`) are for longer notes that don't attach to a single word — implementation notes, edge case lists, multi-line technical details. But most annotations should be inline. A spec where every annotation is a block annotation under a heading is under-annotated — it means the prose itself isn't being examined for ambiguity at the word level.
86
-
87
- **Annotations support full markdown.** Use backticks for code and variable names, lists for enumerating options, code blocks for snippets. This keeps the prose clean while giving the compiler precise technical detail:
88
-
89
- ```markdown
90
- Each invoice has a [status]{One of: `pending_review`, `approved`, `rejected`, `paid`. Transitions:
91
- - `pending_review` → `approved` or `rejected` (by AP)
92
- - `approved` → `paid` (automatic on payment date)
93
- - `rejected` → `pending_review` (if resubmitted)} that tracks where it is in the review process.
94
- ```
95
-
96
- **Annotate ambiguity, not the obvious.** If a statement has only one reasonable interpretation, leave it alone. Annotations resolve genuine ambiguity — places where two engineers might implement different things.
97
-
98
- **Pin down edge cases.** The most valuable annotations answer "what happens when...":
99
- - What happens when a reviewer rejects?
100
- - What happens when the amount is exactly on the threshold?
101
- - What happens when no user has the required role?
102
- - What happens when this is called twice?
103
-
104
- **Specify data when it matters.** When "amount" could mean integer cents or decimal dollars, annotate the representation. When "status" could be any string, list the valid values. These are perfect for inline annotations: `The PO [amount]{Total across all line items, in USD cents (integer). Does not include tax, shipping, or fees.} must not exceed the budget.`
105
-
106
- **Let the spec breathe.** A spec with more annotation than prose is over-specified. Annotate the hard parts. Trust the compiler on the straightforward parts.
107
-
108
- ## Spec Structure
109
-
110
- A spec starts with YAML frontmatter followed by freeform Markdown. There's no mandated structure — use headings, prose, tables, lists, whatever makes sense for the domain.
111
-
112
- **Frontmatter fields:**
113
- - `name` (required) — display name for the spec file
114
- - `description` (optional) — short summary of what this file covers
115
- - `type` (optional) — defaults to `spec`. Other values: `design/color` (color palette definition), `design/typography` (font and type style definition), `roadmap` (feature roadmap item). The frontend renders these types with specialized editors.
116
- - `status` (roadmap only) — `done`, `in-progress`, or `not-started`
117
- - `requires` (roadmap only) — array of slugs for prerequisite roadmap items. Empty array means available now.
118
- - `effort` (roadmap only) — `quick`, `small`, `medium`, or `large`
119
-
120
- ```markdown
121
- ---
122
- name: Expense Tracker
123
- description: Core application spec
124
- ---
125
-
126
- # Expense Tracker
127
-
128
- [High-level description]
129
-
130
- ~~~
131
- [Clarifying annotations — roles, key constraints]
132
- ~~~
133
-
134
- ## [Domain Section]
135
-
136
- [Prose describing the workflow]
137
-
138
- ~~~
139
- [Edge cases, data representations, business rules]
140
- ~~~
141
- ```
142
-
143
- Color design spec example (3-5 brand colors with evocative names):
144
-
145
- ```markdown
146
- ---
147
- name: Colors
148
- type: design/color
149
- ---
150
-
151
- ```colors
152
- Midnight:
153
- value: "#000000"
154
- description: Primary background and dark surfaces
155
- Snow:
156
- value: "#F5F5F7"
157
- description: Primary text and foreground elements
158
- Smoke:
159
- value: "#86868B"
160
- description: Secondary text and supporting content
161
- ```
162
- ```
163
-
164
- Typography design spec example (fonts with source URLs, 1-2 anchor styles):
165
-
166
- ```markdown
167
- ---
168
- name: Typography
169
- type: design/typography
170
- ---
171
-
172
- ```typography
173
- fonts:
174
- Satoshi:
175
- src: https://api.fontshare.com/v2/css?f[]=satoshi@400,500,600,700&display=swap
176
-
177
- styles:
178
- Display:
179
- font: Satoshi
180
- size: 40px
181
- weight: 600
182
- letterSpacing: -0.03em
183
- lineHeight: 1.1
184
- case: uppercase
185
- description: Page titles and hero text
186
- Body:
187
- font: Satoshi
188
- size: 16px
189
- weight: 400
190
- lineHeight: 1.5
191
- description: Default reading text
192
- ```
193
- ```
194
-
195
- Roadmap item example (one file per feature in `src/roadmap/`):
196
-
197
- ```markdown
198
- ---
199
- name: Share & Export
200
- type: roadmap
201
- status: not-started
202
- description: Share haikus as image cards to social media or download as prints.
203
- requires: []
204
- effort: medium
205
- ---
206
-
207
- Share haikus as styled image cards on social media or download as prints.
208
- The card system generates images using the brand's typography and color
209
- palette, creating shareable assets that feel native to the app's identity.
210
-
211
- ~~~
212
- Use generateImage with Seedream to create styled cards. Card template
213
- applies brand typography and colors from the spec. Export as PNG via
214
- CDN transform at 2x resolution. Social sharing via Web Share API with
215
- clipboard fallback for unsupported browsers.
216
- ~~~
217
-
218
- ## History
219
-
220
- - **2026-03-22** — Built card generation using generateImage with Seedream.
221
- Added share button to haiku detail view.
222
- ```
@@ -1,105 +0,0 @@
1
- # MindStudio Platform
2
-
3
- A MindStudio app has three layers: a **spec** (natural language in `src/`), a **backend contract** (methods, tables, roles in `dist/`), and **interfaces** (web, API, bots, cron, etc. — also in `dist/`). The spec is the source of truth; the code is a derivation.
4
-
5
- `src/` is the authored source — natural language specs, brand guidelines, reference materials. No code. `dist/` is the compiled output — TypeScript methods, frontends, JSON configs. You can edit `dist/` directly, but `src/` is the reset point. Regenerate `dist/` from `src/` at any time.
6
-
7
- ## Directory Layout
8
-
9
- ```
10
- my-app/
11
- mindstudio.json ← manifest (declares everything)
12
-
13
- src/ ← authored source (no code)
14
- app.md backend spec (MSFM)
15
- references/ supporting material (PDFs, notes, diagrams)
16
- interfaces/
17
- @brand/ shared brand identity
18
- visual.md aesthetic direction, surfaces, spacing
19
- colors.md brand color palette (type: design/color)
20
- typography.md fonts and type styles (type: design/typography)
21
- voice.md tone, terminology, error messages
22
- assets/ logos, icons
23
- web.md web UI spec
24
- api.md API conventions
25
- cron.md scheduled job descriptions
26
- roadmap/ feature roadmap (one file per item, type: roadmap)
27
-
28
- dist/ ← compiled output (code + config)
29
- methods/ backend contract
30
- src/
31
- tables/ one file per table (TypeScript interfaces)
32
- common/ shared helpers (imported by methods, not methods themselves)
33
- *.ts one file per method (named async function export)
34
- .scenarios/ seed data scripts (dev only, not deployed)
35
- package.json backend dependencies
36
-
37
- interfaces/ interface projections
38
- web/ full project directory (Vite + React)
39
- web.json dev server config
40
- package.json
41
- src/
42
- api/interface.json REST API config
43
- discord/interface.json Discord bot config
44
- telegram/interface.json Telegram bot config
45
- cron/interface.json cron config
46
- webhook/interface.json webhook config
47
- email/interface.json email config
48
- mcp/interface.json MCP config
49
- ```
50
-
51
- ## What Goes Where
52
-
53
- | What | Where | Notes |
54
- |------|-------|-------|
55
- | Method handlers | `dist/methods/src/*.ts` | One file per method, named export |
56
- | Table definitions | `dist/methods/src/tables/*.ts` | One file per table |
57
- | Shared helpers | `dist/methods/src/common/*.ts` | Imported by methods, not invokable directly |
58
- | Scenarios | `dist/methods/.scenarios/*.ts` | Seed data for testing (not deployed) |
59
- | Backend dependencies | `dist/methods/package.json` | Only declared packages are available at runtime |
60
- | Web interface | `dist/interfaces/web/` | Full Vite + React project directory |
61
- | Interface configs | `dist/interfaces/*/interface.json` | One per non-web interface type |
62
- | Specs | `src/*.md` | Natural language, MSFM format |
63
- | Brand identity | `src/interfaces/@brand/` | visual.md (aesthetic), colors.md (palette), typography.md (fonts), voice.md (tone), assets/ |
64
- | Roadmap | `src/roadmap/*.md` | Feature roadmap items (type: roadmap). One file per feature with status, dependencies, and history. |
65
- | Reference material | `src/references/` | Context for the agent, not consumed by platform |
66
-
67
- ## The Two SDKs
68
-
69
- **Backend: `@mindstudio-ai/agent`** — used inside methods. Provides `db` (database), `auth` (access control), and platform capabilities (AI, integrations, connectors). Pre-installed in the sandbox; must also be declared in `dist/methods/package.json`.
70
-
71
- ```typescript
72
- import { db, auth } from '@mindstudio-ai/agent';
73
- ```
74
-
75
- **Frontend: `@mindstudio-ai/interface`** — used in the web interface. Typed RPC to backend methods.
76
-
77
- ```typescript
78
- import { createClient } from '@mindstudio-ai/interface';
79
-
80
- const api = createClient<{
81
- approveVendor(input: { vendorId: string }): Promise<{ vendor: Vendor }>;
82
- }>();
83
-
84
- const { vendor } = await api.approveVendor({ vendorId: '...' });
85
- ```
86
-
87
- ## What the Platform Provides
88
-
89
- - **Managed databases.** SQLite with typed schemas. Push a schema change and the platform diffs, migrates, and promotes atomically.
90
- - **Built-in auth.** Define roles in the manifest, call `auth.requireRole('admin')` in methods. Platform handles sessions, tokens, user resolution.
91
- - **Multiple interfaces, one codebase.** Web, API, Discord, Telegram, Cron, Webhook, Email, MCP — all invoke the same methods. Methods don't know which interface called them.
92
- - **Sandboxed execution.** Methods run in isolated sandboxes with npm packages pre-installed.
93
- - **Git-native deployment.** Push to default branch to deploy. Push to feature branch for preview. Rollback is a git revert.
94
-
95
- ## Minimum Viable App
96
-
97
- ```
98
- my-app/
99
- mindstudio.json
100
- dist/methods/
101
- src/hello.ts
102
- package.json
103
- ```
104
-
105
- One manifest, one method, one package.json. The method is accessible via API key.
@@ -1,103 +0,0 @@
1
- # Scenarios
2
-
3
- Scenarios are seed scripts that set up the dev database into a specific state for testing. Instead of manually creating data through the app, run a scenario and get a repeatable starting point. A scenario is just an async function that uses the same `db.push()` calls as methods — no new API to learn.
4
-
5
- ## Defining Scenarios
6
-
7
- In `mindstudio.json`:
8
-
9
- ```json
10
- {
11
- "scenarios": [
12
- {
13
- "id": "ap-overdue-invoices",
14
- "name": "AP: Overdue Invoices",
15
- "description": "AP user with two invoices past due date.",
16
- "path": "dist/methods/.scenarios/apOverdueInvoices.ts",
17
- "export": "apOverdueInvoices",
18
- "roles": ["ap"]
19
- },
20
- {
21
- "id": "empty-requester",
22
- "name": "Empty Requester",
23
- "description": "Brand new user, no data.",
24
- "path": "dist/methods/.scenarios/emptyRequester.ts",
25
- "export": "emptyRequester",
26
- "roles": ["requester"]
27
- }
28
- ]
29
- }
30
- ```
31
-
32
- | Field | Description |
33
- |-------|-------------|
34
- | `id` | Kebab-case identifier |
35
- | `name` | Display name (shown in dev panel) |
36
- | `description` | What state this scenario creates |
37
- | `path` | Path to the TypeScript file |
38
- | `export` | Named export (the async function) |
39
- | `roles` | Roles to impersonate after seeding |
40
-
41
- ## Writing a Scenario
42
-
43
- Scenarios live at `dist/methods/.scenarios/` — inside the methods package scope. `@mindstudio-ai/agent` resolves from `dist/methods/node_modules/` and table imports are relative.
44
-
45
- ```typescript
46
- // dist/methods/.scenarios/apOverdueInvoices.ts
47
-
48
- import { db } from '@mindstudio-ai/agent';
49
- import { Vendors } from '../src/tables/vendors';
50
- import { PurchaseOrders } from '../src/tables/purchase-orders';
51
- import { Invoices } from '../src/tables/invoices';
52
-
53
- export async function apOverdueInvoices() {
54
- const vendor = await Vendors.push({
55
- name: 'Acme Corp',
56
- contactEmail: 'billing@acme.com',
57
- status: 'approved',
58
- });
59
-
60
- const po = await PurchaseOrders.push({
61
- vendorId: vendor.id,
62
- requestedBy: 'user-requester-1',
63
- totalAmountCents: 500000,
64
- status: 'active',
65
- });
66
-
67
- await Invoices.push([
68
- {
69
- poId: po.id,
70
- invoiceNumber: 'INV-001',
71
- amountCents: 150000,
72
- dueDate: db.ago(db.days(5)),
73
- status: 'pending_review',
74
- },
75
- {
76
- poId: po.id,
77
- invoiceNumber: 'INV-002',
78
- amountCents: 100000,
79
- dueDate: db.ago(db.days(2)),
80
- status: 'approved',
81
- },
82
- ]);
83
- }
84
- ```
85
-
86
- An empty scenario is valid — it exists so you can switch to "clean slate" state:
87
-
88
- ```typescript
89
- export async function emptyRequester() {
90
- // No data — the truncate clears everything.
91
- }
92
- ```
93
-
94
- Shared setup code can go in `dist/methods/.scenarios/_helpers/`.
95
-
96
- ## How Scenarios Run
97
-
98
- When a scenario runs, the platform:
99
- 1. **Truncates** all tables (deletes all rows, preserves schema)
100
- 2. **Executes** the seed function (your `db.push()` calls populate the clean database)
101
- 3. **Impersonates** the roles from the scenario's `roles` field (the app renders from that user's perspective)
102
-
103
- This is deterministic — same scenario always produces the same state.
@@ -1,146 +0,0 @@
1
- # MindStudio Agent SDK
2
-
3
- `@mindstudio-ai/agent` provides access to 200+ AI models and 1,000+ actions through a single API key. No separate provider keys needed. MindStudio routes to the correct provider (OpenAI, Anthropic, Google, etc.) server-side.
4
-
5
- There is a huge amount of capability here: hundreds of text generation models (OpenAI, Anthropic, Google, Meta, Mistral, and more), dozens of image generation models (FLUX, DALL-E, Stable Diffusion, Ideogram, and more), video generation, text-to-speech, music generation, vision analysis, web scraping, 850+ OAuth connectors, and much more. The tables below are a summary. Always use `askMindStudioSdk` before writing SDK code.
6
-
7
- ## Usage in Methods
8
-
9
- Inside a MindStudio app method, create an instance with no arguments — credentials come from the execution environment:
10
-
11
- ```typescript
12
- import { MindStudioAgent } from '@mindstudio-ai/agent';
13
-
14
- const agent = new MindStudioAgent();
15
- ```
16
-
17
- Every action is a method on the agent instance:
18
-
19
- ```typescript
20
- const { content } = await agent.generateText({ message: 'Summarize this...' });
21
- ```
22
-
23
- Results are returned flat — output fields at the top level alongside metadata:
24
-
25
- ```typescript
26
- const result = await agent.generateText({ message: 'Hello' });
27
- result.content; // step-specific output
28
- result.$billingCost; // cost in credits (if applicable)
29
- ```
30
-
31
- ## Available Actions
32
-
33
- ### AI Generation
34
-
35
- | Action | What it does | Key input | Key output |
36
- |--------|-------------|-----------|------------|
37
- | `generateText` | Text generation with any LLM | `message`, `modelOverride?` | `content` |
38
- | `generateImage` | Image from text prompt | `prompt`, `modelOverride?` | `imageUrl` |
39
- | `generateVideo` | Video from text/image | `prompt`, `imageUrl?` | `videoUrl` |
40
- | `textToSpeech` | Text to spoken audio | `text`, `modelOverride?` | `audioUrl` |
41
- | `generateMusic` | Music from text description | `prompt` | `audioUrl` |
42
- | `generateLipsync` | Animate face to match audio | `imageUrl`, `audioUrl` | `videoUrl` |
43
- | `generateAsset` | HTML/PDF/PNG/video output | `prompt` | `assetUrl` |
44
- | `generateChart` | Chart from data | `data`, `chartType` | `imageUrl` |
45
-
46
- ### AI Analysis
47
-
48
- | Action | What it does | Key input | Key output |
49
- |--------|-------------|-----------|------------|
50
- | `analyzeImage` | Vision model analysis | `prompt`, `imageUrl` | `analysis` |
51
- | `analyzeVideo` | Video analysis | `prompt`, `videoUrl` | `analysis` |
52
- | `transcribeAudio` | Audio to text | `audioUrl` | `transcription` |
53
- | `extractText` | Extract text from documents/images | `url` | `text` |
54
- | `detectPII` | Find personal data | `text` | `entities` |
55
-
56
- ### Web & Search
57
-
58
- | Action | What it does | Key input | Key output |
59
- |--------|-------------|-----------|------------|
60
- | `scrapeUrl` | Extract page content | `url` | `markdown` |
61
- | `searchGoogle` | Google search | `query` | `results` |
62
- | `searchGoogleImages` | Image search | `query` | `results` |
63
- | `searchGoogleNews` | News search | `query` | `results` |
64
- | `searchPerplexity` | AI-powered search | `query` | `answer` |
65
- | `httpRequest` | Custom HTTP call | `url`, `method`, `headers?`, `body?` | `response` |
66
-
67
- ### Communication
68
-
69
- | Action | What it does | Key input | Key output |
70
- |--------|-------------|-----------|------------|
71
- | `sendEmail` | Send an email | `to`, `subject`, `body` | `messageId` |
72
- | `sendSMS` | Send a text message | `to`, `message` | `messageId` |
73
- | `postToSlackChannel` | Post to Slack | `channel`, `message` | — |
74
-
75
- ### Media Processing
76
-
77
- | Action | What it does |
78
- |--------|-------------|
79
- | `removeBackgroundFromImage` | Remove image background |
80
- | `upscaleImage` | Upscale image resolution |
81
- | `imageFaceSwap` | Swap faces in an image |
82
- | `imageRemoveWatermark` | Remove watermarks |
83
- | `mergeVideos` | Concatenate video clips |
84
- | `trimMedia` | Trim audio/video |
85
- | `addSubtitlesToVideo` | Auto-generate subtitles |
86
- | `extractAudioFromVideo` | Extract audio track |
87
- | `captureThumbnail` | Get video thumbnail |
88
-
89
- ### Files & Data
90
-
91
- | Action | What it does |
92
- |--------|-------------|
93
- | `uploadFile` | Upload a file to CDN |
94
- | `downloadVideo` | Download a video URL |
95
- | `getMediaMetadata` | Get dimensions, duration, etc. |
96
- | `convertPdfToImages` | PDF pages to PNG images |
97
-
98
- ### Third-Party Integrations (OAuth Connectors)
99
-
100
- 850+ additional actions from the MindStudio Connector Registry, covering services like HubSpot, Salesforce, Airtable, Google Workspace, Notion, Coda, and many more. These require OAuth connections set up by the user in MindStudio.
101
-
102
- Built-in connector methods include: ActiveCampaign, Airtable, Apollo, Coda, Facebook, Gmail, Google Docs/Sheets/Calendar/Drive, HubSpot, Hunter.io, Instagram, LinkedIn, Notion, X (Twitter), YouTube.
103
-
104
- For other services, use `runFromConnectorRegistry`:
105
-
106
- ```typescript
107
- // Discover available connectors
108
- const { connectors } = await agent.listConnectors();
109
-
110
- // Get action details
111
- const action = await agent.getConnectorAction('hubspot', 'create-contact');
112
-
113
- // Execute
114
- const result = await agent.runFromConnectorRegistry({
115
- serviceId: 'hubspot',
116
- actionId: 'create-contact',
117
- input: { email: 'user@example.com', firstName: 'Alice' },
118
- });
119
- ```
120
-
121
- ### Model Selection
122
-
123
- Override the default model for any AI action. Each model has its own config options (dimensions, seed, inference steps, etc.) so always use `askMindStudioSdk` to look up the correct config before specifying a model override:
124
-
125
- ```typescript
126
- const { content } = await agent.generateText({
127
- message: 'Hello',
128
- modelOverride: {
129
- model: 'claude-sonnet-4-6',
130
- temperature: 0.7,
131
- maxResponseTokens: 1024,
132
- },
133
- });
134
- ```
135
-
136
- ### Batch Execution
137
-
138
- Run up to 50 actions in parallel:
139
-
140
- ```typescript
141
- const result = await agent.executeStepBatch([
142
- { stepType: 'generateImage', step: { prompt: 'a sunset' } },
143
- { stepType: 'textToSpeech', step: { text: 'hello world' } },
144
- ]);
145
- // result.results[0].output, result.results[1].output
146
- ```