@superglue/cli 1.1.20 → 1.1.22

package/skills/superglue/references/sftp-smb.md

@@ -0,0 +1,156 @@
+# SFTP / FTP / SMB
+
+FTP, SFTP, and SMB share an identical operation interface. Only the URL protocol and connection details differ.
+
+## Step Configuration
+
+```typescript
+{
+  type: "request",
+  systemId: "my_sftp_server",
+  url: "sftp://<<my_sftp_server_username>>:<<my_sftp_server_password>>@<<my_sftp_server_host>>:<<my_sftp_server_port>>/basePath",
+  body: '{"operation": "get", "path": "data/report.csv"}'
+}
+```
+
+Configure the connection URL based on which credentials are stored in the system — use `sg system find` to check `storedCredentials` and reference them accordingly.
+
+`method`, `headers`, `queryParams`, and `pagination` are HTTP-only fields — omit them for file server steps.
+
+### Fields
+
+| Field      | Required | Notes                                                          |
+| ---------- | -------- | -------------------------------------------------------------- |
+| `url`      | yes      | Connection string with `<<credential>>` placeholders           |
+| `body`     | yes      | JSON: single operation object or array for batch               |
+| `systemId` | no       | Links system credentials for `<<systemId_credKey>>` resolution |
+
+### URL Formats
+
+```
+ftp://user:password@host:port/basePath       (port 21)
+ftps://user:password@host:port/basePath      (port 21, TLS)
+sftp://user:password@host:port/basePath      (port 22)
+smb://user:password@host/sharename/basePath  (port 445)
+smb://domain\user:password@host/sharename/   (domain auth)
+```
+
+SMB requires a share name as the first path segment. Additional segments become a base path prepended to all operation paths.
+
+### Body Format
+
+JSON string — single operation or array of operations (batch, executed sequentially on a single connection):
+
+```json
+{ "operation": "get", "path": "data/report.csv" }
+```
+
+```json
+[
+  { "operation": "mkdir", "path": "backup" },
+  { "operation": "get", "path": "data/report.csv" },
+  { "operation": "put", "path": "backup/report.csv", "content": "file::report.raw" }
+]
+```
+
+## Authentication
+
+Credentials are embedded in the connection URL using `<<systemId_credentialKey>>` placeholders. Nothing is injected automatically.
+
+```
+sftp://<<my_server_username>>:<<my_server_password>>@host/path
+```
+
+**SFTP private key auth:** use `credentials.privateKey` and `credentials.passphrase` on the system instead of URL password placeholders. Store the private key as a credential on the system.
+
+**SMB domain auth:** use `domain\user` format in the URL, or store `domain` as a system credential.
+
+Only store connection credentials on the system: `host`, `port`, `username`, `password`, `privateKey`, `passphrase`, `domain` (SMB), `home_dir`. Paths and share names belong in the URL, not in credentials.
+
+## File Server Runtime Details
+
+### Path Handling
+
+All operation paths are relative to the base path in the URL. Do not use absolute filesystem paths.
+
+If the system URL is `sftp://user:pass@host/home/sftptest/uploads`:
+
+- `report.csv` resolves to `/home/sftptest/uploads/report.csv`
+- `subdir/data.csv` resolves to `/home/sftptest/uploads/subdir/data.csv`
+
+Do not use `/home/sftptest/uploads/report.csv` — the base path is already set in the URL.
+
+If the URL has no base path (e.g. `sftp://user:pass@host`), paths are used as-is from the server root.
+
+SMB paths use forward slashes — the library handles Windows path conversion internally.
+
+### Operations
+
+| Operation | Required fields   | Returns                                                         |
+| --------- | ----------------- | --------------------------------------------------------------- |
+| `list`    | `path`            | Array of `{ name, path, size, type, modifyTime, ... }`          |
+| `get`     | `path`            | Auto-parsed file content (CSV to objects, JSON to parsed, etc.) |
+| `put`     | `path`, `content` | `{ success, message, size }`                                    |
+| `delete`  | `path`            | `{ success, message }`                                          |
+| `rename`  | `path`, `newPath` | `{ success, message }`                                          |
+| `mkdir`   | `path`            | `{ success, message }`                                          |
+| `rmdir`   | `path`            | `{ success, message }`                                          |
+| `exists`  | `path`            | `{ exists: boolean, path }`                                     |
+| `stat`    | `path`            | File metadata or `{ exists: false }`                            |
+
+### `get` — File Handling
+
+Downloaded files are added to the runtime file store. See `file-handling.md` for detection, parsing, aliasing rules, and the full `file::` reference syntax.
+
+- `data` contains auto-parsed content (CSV to objects, JSON to parsed, PDF to structured, etc.)
+- The step result exposes `stepFileKeys`
+- Later steps reference downloaded files via `file::<stepId>.raw`, `file::<stepId>.base64`, or `file::<stepId>.extracted`
+- Multi-file batch downloads use bracket notation: `file::<stepId>["report.csv"].raw`
+
+### `put` — Content Handling
+
+Use `file::<key>.raw` to preserve original bytes when uploading files from previous steps. See `file-handling.md` for all suffix options.
+
+| Content type                     | Behavior                             |
+| -------------------------------- | ------------------------------------ |
+| `file::<key>.raw` (RawFileBytes) | Written as exact original bytes      |
+| String                           | Written as-is                        |
+| Buffer                           | Written directly                     |
+| Object / Array                   | JSON-stringified with 2-space indent |
+
+### Return Values
+
+Single operation returns the result directly. Array of operations returns an array of results (one per operation, in order).
+
+### Connection Details
+
+**SFTP:** uses `ssh2-sftp-client`. Supports password and private key auth. Connection timeout configurable, default retries from server config.
+
+**FTP/FTPS:** uses `basic-ftp`. FTPS uses TLS with `rejectUnauthorized: false`. Connection timeout configurable.
+
+**SMB:** uses `@tryjsky/v9u-smb2`. Connection lifecycle: create client (with share, domain, credentials), execute operations, disconnect. Entire operation wrapped in a timeout race. For private systems via Secure Gateway, the host is `tunnelId.tunnel` (e.g. `my_tunnel.tunnel`).
+
+### Retry Behavior
+
+Default retries from server config. On final failure, error includes the full operations list. Each retry creates a fresh connection.
+
+## Common Pitfalls
+
+- Using absolute filesystem paths instead of paths relative to the URL base path
+- Forgetting that SMB requires a share name as the first URL path segment
+- Using `file::<key>` without a suffix in `put` content. Use `file::<key>.raw` to preserve original bytes
+- Missing the double-data access pattern for `get` results: `sourceData.stepId.data` gives you the parsed file content (through the step envelope)
+- Trying to `put` to a path where intermediate directories don't exist — use `mkdir` first or batch the operations
+
+## Error Recovery
+
+When a file server step fails and the cause is not obvious:
+
+1. **Read the error message** — superglue includes the operations list in the error. Check for typos in paths, missing fields, or unsupported operations.
+2. **Verify connection credentials** — use `sg system find` to confirm `storedCredentials` keys match the placeholders in your URL. Test with a simple `list` operation on the base path via `sg system call`.
+3. **Check path resolution** — if you get "file not found", verify the path is relative to the URL base path, not an absolute filesystem path. Use `list` on the parent directory to see what files exist.
+4. **Check permissions** — permission denied errors mean the user account lacks read/write access to the target path. Verify with the user that the account has the required grants.
+5. **Check protocol-specific issues:**
+   - **SFTP**: if the error mentions authentication, verify whether the server expects password or private key auth. If using private key, ensure `privateKey` and `passphrase` are stored as system credentials.
+   - **FTP/FTPS**: if the connection fails, try switching between `ftp://` and `ftps://` — some servers require TLS, others don't support it.
+   - **SMB**: verify the share name is correct (first path segment). If using domain auth, ensure the domain is included in the URL or stored as a credential. Check that the SMB port (445) is accessible.

package/skills/superglue/references/superglue-info.md

@@ -0,0 +1,91 @@
+# superglue Info
+
+## Company
+
+superglue is an open-source, AI-native integration platform that builds and runs deterministic multi-step workflows ("tools") connecting APIs, databases, and file servers. AI generates tool configurations during building — execution is 100% deterministic JavaScript with no LLMs involved.
+
+Developed by superglue (Y Combinator W25), founded by Adina Görres and Stefan Faistenauer in 2025, based in Munich and San Francisco.
+
+- Website: https://superglue.ai
+- Documentation: https://docs.superglue.cloud
+- GitHub: https://github.com/superglue-ai/superglue
+
+If the user is asking general questions (not building):
+
+- Company/team/pricing → https://superglue.ai/
+- Product/features → https://docs.superglue.cloud/getting-started/introduction
+- Open-source/code → https://github.com/superglue-ai/superglue
+
+## Authentication
+
+All superglue interfaces (API, SDK, CLI, MCP) require only an **API key**. Self-hosted deployments also need the API endpoint URL. There are no workspace IDs, project IDs, or other identifiers — the API key is the only credential needed.
+
+Users can generate and manage API keys in **Control Panel → API Keys**. The API endpoint is displayed on the same page.
+
+## Interfaces
+
+- **Web app** — primary UI for building, testing, and managing tools and systems
+- **TypeScript/Python SDK** — https://docs.superglue.cloud/sdk/overview
+- **REST API** — https://docs.superglue.cloud/api-reference/
+- **MCP Server** — https://docs.superglue.cloud/mcp/using-the-mcp
+- **CLI** — `npm install -g @superglue/cli` — https://docs.superglue.cloud/getting-started/cli-skills
+
+### Webhook Triggers
+
+Tools can be triggered via webhook: `POST {apiEndpoint}/v1/hooks/{toolId}?token={apiKey}`
+
+On superglue Cloud, the API endpoint is `https://api.superglue.cloud`. Self-hosted and enterprise deployments use their own API endpoint — users can find theirs under Control Panel → API Keys.
+
+### OAuth Callback URL
+
+OAuth flows require a callback URL: `{appEndpoint}/api/auth/callback`
+
+On superglue Cloud, the app endpoint is `https://app.superglue.cloud`. Self-hosted and enterprise deployments use their own app URL. If a user doesn't know their endpoints, they can check the browser URL bar for the app endpoint, and Control Panel → API Keys for the API endpoint.
+
+## Web App UI Layout
+
+Left sidebar navigation:
+
+- **Agent** — AI chat assistant for building and debugging tools/systems
+- **Landscape** — Visual graph showing all tools and their connected systems
+- **Tools** — List of saved tools. Click a tool to open its playground for editing, testing, and running.
+- **Systems** — List of connected external systems with credentials and documentation
+- **Control Panel** (expandable):
+  - Overview: Dashboard summary
+  - Runs: Execution history for all tool runs with status, errors, and step results
+  - Schedules: Manage scheduled/recurring tool runs
+  - API Keys: View and manage superglue API keys
+  - Organization: Team members and org settings
+  - Notifications: Configure alerts for tool run failures
+- **Docs** — Link to docs.superglue.cloud
+
+## Internals
+
+### Execution Pipeline
+
+`ToolExecutor.execute({ payload, credentials, options })`:
+
+1. **Validate** tool structure (id, steps array, URLs on request steps)
+2. **For each step in order:**
+   a. Build aggregated data: `{ ...originalPayload, ...previousStepResults }`
+   b. Resolve system credentials (refresh OAuth if needed), namespace as `systemId_key`
+   c. Run `dataSelector` → object means single execution, array means loop
+   d. For each item: merge `currentItem` into input and execute
+   e. Wrap result: `{ currentItem, data, success }`
+   f. On failure: abort if `failureBehavior !== "continue"`
+3. **Output transform** (if present): run JS function, validate against outputSchema
+4. **Response filters** (if present): remove/mask/fail on pattern matches
+
+### Strategy Routing
+
+Steps are routed to execution strategies by protocol (first match wins):
+
+1. Transform steps → if `config.type === "transform"`
+2. HTTP → URL starts with `http://` or `https://`
+3. PostgreSQL → URL starts with `postgres://` or `postgresql://`
+4. MSSQL/Azure SQL → URL starts with `mssql://` or `sqlserver://`
+5. Redis → URL starts with `redis://` or `rediss://`
+6. FTP/SFTP → URL starts with `ftp://`, `ftps://`, or `sftp://`
+7. SMB → URL starts with `smb://`
+
+All user-provided JS (data selectors, transforms, stop conditions) runs in an isolated Deno sandbox.

package/dist/skills/superglue/references/data-handling.md

@@ -1,303 +0,0 @@
-# Data Handling: Variables, Selectors & Transforms
-
-Most tool building and editing failures trace back to incorrect variable syntax, wrong data selector returns, or misunderstanding of the tool result envelope.
-
-## Variable Replacement (`<<>>` Syntax)
-
-The `<<expression>>` syntax injects dynamic values into step config fields (URL, headers, body, queryParams).
-
-### Simple Variables (top-level keys only)
-
-```
-<<userId>>           payload keys
-<<currentItem>>      whole current loop item
-<<page>>             pagination variable
-<<systemId_api_key>> system credential
-<<systemId_url>>     system base URL
-<<sg_auth_email>>    email of the authenticated user
-```
-
-**CRITICAL**: Simple `<<varName>>` only works for top-level keys. NO dots, NO nesting.
-
-- VALID: `<<userId>>`, `<<currentItem>>`, `<<page>>`, `<<stripe_api_key>>`, `<<stripe_url>>`, `<<sg_auth_email>>`
-- INVALID: `<<currentItem.id>>`, `<<sourceData.userId>>`, `<<user.name>>` — these FAIL at runtime
-
-### Arrow Function Expressions (for everything else)
-
-```
-<<(sourceData) => sourceData.currentItem.player_id>>
-<<(sourceData) => sourceData.getUsers.data.map(u => u.id)>>
-<<(sourceData) => JSON.stringify({ ids: sourceData.fetchUsers.data.map(u => u.id) })>>
-```
-
-### Resolution Order
-
-1. Direct lookup: check if expression is a top-level key in the merged variable object
-2. Arrow function: execute in Deno subprocess sandbox with full data as `sourceData`
-3. Failure: throws error listing available keys
-
-### Variable Sources
-
-All variables come from a single merged object:
-
-```javascript
-{
-  ...originalPayload,       // user-provided input
-  ...previousStepResults,   // keyed by stepId
-  ...systemCredentials,     // namespaced: systemId_credKey
-  ...systemUrls,            // namespaced: systemId_url
-  sg_auth_email,            // email of authenticated user (context variable)
-  currentItem,              // current loop item (if in a loop)
-  page, offset, cursor,     // pagination variables
-  limit, pageSize           // pagination size
-}
-```
-
-## Context Variables
-
-Context variables provide information about the execution environment:
-
-- `<<sg_auth_email>>` — Email address of the authenticated user who triggered the tool execution
-
-**Use case**: Personalize tool outputs based on who is running the tool. For example:
-
-- Different permissions in Tableau based on user
-- User-specific account lookups
-- Audit logging with user identity
-
-**Important**: `sg_auth_email` is NOT available in scheduled executions (cron jobs). If a tool references `<<sg_auth_email>>` and runs via scheduler, it will fail with a clear error message.
-
-### Return Type Handling
-
-- Objects/arrays returned from expressions → JSON.stringified automatically
-- Resolved values of `"undefined"` or `"null"` → stripped from headers and queryParams
-
-### Special: Basic Auth
-
-`"Authorization": "Basic <<user>>:<<pass>>"` is auto-converted to Base64. Do NOT manually encode.
-
-## Credential Resolution
-
-Credentials are available via `<<>>` syntax. Two sources, payload takes precedence:
-
-1. **System credentials**: Stored on System, namespaced as `<<systemId_credKey>>`
-
-   ```javascript
-   // System id="stripe", credentials={ api_key: "sk_..." }
-   // Available as: <<stripe_api_key>>
-   ```
-
-2. **Payload credentials**: Passed at runtime, accessed directly
-   ```javascript
-   // Payload: { user_access_token: "abc" }
-   // Available as: <<user_access_token>>
-   // Or: <<(sourceData) => sourceData.user_access_token>>
-   ```
-
-For OAuth systems, use `<<systemId_access_token>>` — the token value is auto-refreshed during tool execution, but the header itself must be present in the step config.
-
-## System URL Resolution
-
-System URLs are available via `<<systemId_url>>` syntax. This enables environment-agnostic tools:
-
-```javascript
-// System id="salesforce", url="https://mycompany.salesforce.com"
-// Available as: <<salesforce_url>>
-
-// Step config:
-{
-  "url": "<<salesforce_url>>/services/data/v58.0/sobjects/Account",
-  "headers": { "Authorization": "Bearer <<salesforce_access_token>>" }
-}
-```
-
-**Why use system URL variables?**
-
-- Same tool works across dev/prod environments without modification
-- When switching execution mode, the system URL resolves to the appropriate environment
-- Avoids hardcoding base URLs that differ between environments
-
-## Data Selectors
-
-Every step has a `dataSelector` — a JS function that controls execution mode and what `currentItem` is.
-
-### Returns OBJECT → Single Execution
-
-Step runs once. The object becomes `currentItem`.
-
-```javascript
-(sourceData) => ({ userId: sourceData.userId })
-(sourceData) => ({})   // run once with no specific input
-```
-
-Result stored as:
-
-```javascript
-sourceData.stepId = { currentItem: <object>, data: <response>, success: true }
-```
-
-### Returns ARRAY → Loop Execution
-
-Step runs once per array element. Each element becomes `currentItem`.
-
-```javascript
-(sourceData) => sourceData.getContacts.data.filter(c => c.active)
-(sourceData) => sourceData.userIds   // if userIds is an array from payload
-```
-
-Result stored as:
-
-```javascript
-sourceData.stepId = [
-  { currentItem: <item1>, data: <response1>, success: true },
-  { currentItem: <item2>, data: <response2>, success: true },
-  ...
-]
-```
-
-Empty arrays are valid — step just skips execution.
-
-### Accessing currentItem in Step Config
-
-```
-<<currentItem>>                                           // whole value
-<<(sourceData) => sourceData.currentItem.id>>             // property
-<<(sourceData) => sourceData.currentItem.name.toUpperCase()>>  // with transform
-```
-
-## Result Envelope
-
-**Every** step result (request or transform) is wrapped identically:
-
-### From Object Selector
-
-```javascript
-sourceData.stepId = {
-  currentItem: <selectorOutput>,
-  data: <apiResponse or transformResult>,
-  success: true
-}
-
-// Access:
-sourceData.stepId.data           // the response
-sourceData.stepId.currentItem    // what was passed in
-```
-
-### From Array Selector
-
-```javascript
-sourceData.stepId = [
-  { currentItem: <item1>, data: <response1>, success: true },
-  { currentItem: <item2>, data: <response2>, success: false, error: "..." },
-  ...
-]
-
-// Access:
-sourceData.stepId.map(item => item.data)                     // all responses
-sourceData.stepId.flatMap(item => item.data.results)          // nested arrays
-sourceData.stepId.filter(item => item.success).map(i => i.data) // only successes
-```
-
-### From Paginated Step
-
-Paginated steps merge all pages server-side into a **single** result. The result is always a single envelope, NOT an array — even though multiple pages were fetched.
-
-```javascript
-sourceData.stepId = {
-  data: <merged response from all pages>,
-  success: true
-}
-
-// Access:
-sourceData.stepId.data              // the merged response
-sourceData.stepId.data.items        // if the API wraps results in .items
-
-// WRONG — do NOT treat paginated results as an array of envelopes:
-sourceData.stepId.map(...)          // FAILS — stepId is an object, not an array
-```
-
-## sourceData Structure
-
-`sourceData` is the cumulative state available to data selectors, variable expressions, and transforms:
-
-```javascript
-sourceData = {
-  // Original payload fields at ROOT level (NOT under .payload)
-  userId: "abc",
-  date: "2024-01-15",
-  companies: ["acme", "globex"],
-
-  // Previous step results, keyed by step ID
-  getUsers: { currentItem: {}, data: { users: [...] }, success: true },
-  fetchDetails: [
-    { currentItem: { id: 1 }, data: { name: "Alice" }, success: true },
-    { currentItem: { id: 2 }, data: { name: "Bob" }, success: true },
-  ],
-
-  // Current item (only within a loop step's config)
-  currentItem: { id: 1 },
-
-  // Runtime file map — see file-handling skill for full reference
-  __files__: { ... }
-}
-```
-
-Treat `sourceData.__files__` as read-only. See the file-handling skill for file detection, the `RuntimeExecutionFile` shape, aliasing rules, and how to access files in transforms.
-
-## Three Transformation Points
-
-| Point                            | When it runs             | Purpose                              |
-| -------------------------------- | ------------------------ | ------------------------------------ |
-| `dataSelector`                   | Before step executes     | Controls input + single vs loop mode |
-| Transform step (`transformCode`) | As the step itself       | Reshape data between request steps   |
-| `outputTransform`                | After all steps complete | Shape final tool output              |
-
-## Transform Steps
-
-Transform steps use `type: "transform"` with a `transformCode` function. They do NOT have: systemId, url, method, headers, body, queryParams, or pagination.
-
-Results are wrapped in the **same envelope** as request steps:
-
-```javascript
-sourceData.formatForInsert = { currentItem: <dataSelector output>, data: <transformCode result>, success: true }
-```
-
-If the step has a dataSelector returning an array, the transform runs once per item. See tool-building guidelines for when to use transform steps vs dataSelectors vs outputTransform.
-
-### Returning Files From a Transform
-
-Transform steps can produce files by returning a reserved `__files__` key in the result. See the file-handling skill for the full shape, aliasing rules, and access patterns.
-
-## Output Transform
-
-Final transformation shaping the tool's output. Runs after all steps complete. Can access cumulative step result data and payloads.
-
-### Requirements
-
-- Function signature: `(sourceData) => { ... }`
-- **Must have a return statement**
-- Async/await IS supported (same Deno sandbox as transform steps)
-- NEVER include newlines or tabs in the code string
-
-## JS Code Constraints (all transform points)
-
-Runs in a Deno subprocess sandbox:
-
-- 8192 MB memory limit, 10 min timeout (transforms/selectors), 3s timeout (pagination stops)
-- Async/await IS supported in transforms
-- No filesystem access — `--deny-read`, `--deny-write`
-- No subprocess spawning — `--deny-run`
-- Network access IS available (`--allow-net`) — transforms can make HTTP requests
-- Environment variables are sandboxed — only safe vars (PATH, HOME, DENO_DIR) are visible, no server secrets
-- JSON-serializable I/O only
-- Must return a value (undefined returns null)
-
-### Serialization Safety
-
-All JS code is stored as a JSON string, then MessagePack-serialized to the Deno subprocess, then `eval()`'d. Patterns that break across this pipeline:
-
-- **Regex literals with `/`** — `/https?:\/\/([^/]+)/` corrupts. Use `new URL(str).hostname` or `str.split("//")[1].split("/")[0]` instead.
-- **Backslash-heavy patterns** — `\n`, `\t`, `\\` in regex/strings get misinterpreted. Prefer string methods (`.split()`, `.indexOf()`, `.includes()`) over regex.
-- **Literal newlines** — code string must be single-line. Use `;` between statements.
-
-**Rule of thumb**: prefer string methods over regex. When regex is unavoidable, use `new RegExp("pattern")` constructor over `/pattern/` literals.