@superglue/cli 1.1.11 → 1.1.12

package/dist/dist/skills/superglue/references/databases.md

@@ -0,0 +1,88 @@
+# Database Steps (PostgreSQL)
+
+## Step Configuration
+
+```typescript
+{
+  type: "request",
+  systemId: "my_postgres_db",
+  url: "postgres://<<my_postgres_db_user>>:<<my_postgres_db_password>>@<<my_postgres_db_host>>:<<my_postgres_db_port>>/<<my_postgres_db_database>>",
+  method: "POST",            // ignored for Postgres but required by schema
+  body: '{"query": "SELECT * FROM users WHERE id = $1", "params": [<<userId>>]}'
+}
+```
+
+Configure the connection url depending on which credentials are saved in the stored db system - reference them accordingly.
+
+## URL Format
+
+```
+postgres://user:password@host:port/database
+postgresql://user:password@host:port/database
+```
+
+All credential variables are resolved before connecting. Trailing slashes and extra slashes before query strings are cleaned.
+
+## Body Format
+
+JSON string with `query` and optional `params` (or `values`):
+
+```json
+{ "query": "SELECT * FROM users WHERE age > $1 AND status = $2", "params": [25, "active"] }
+```
+
+### Parameterized Queries
+
+**Always use parameterized queries** with `$1`, `$2`, etc. — prevents SQL injection.
+
+```json
+{ "query": "SELECT * FROM orders WHERE customer_id = $1", "params": ["<<customerId>>"] }
+```
+
+With expressions:
+
+```json
+{
+  "query": "INSERT INTO logs (message, level) VALUES ($1, $2)",
+  "params": ["<<(sourceData) => sourceData.message>>", "error"]
+}
+```
+
+With RETURNING:
+
+```json
+{
+  "query": "INSERT INTO users (name, email) VALUES ($1, $2) RETURNING *",
+  "params": [
+    "<<(sourceData) => sourceData.currentItem.name>>",
+    "<<(sourceData) => sourceData.currentItem.email>>"
+  ]
+}
+```
+
+## Connection Pooling
+
+- Pools cached by connection string, max 10 clients, 5s connection timeout
+- Idle pools cleaned up automatically
+- SSL auto-configured: enabled (with `rejectUnauthorized: false`) unless connecting to localhost without explicit sslmode
+
+## Retry Logic
+
+Default retries from server config. On final failure, error includes query text and params.
+
+## Return Value
+
+Returns `result.rows` — array of row objects with column names as keys.
+
+- `SELECT`: array of matching rows
+- `INSERT/UPDATE/DELETE` with `RETURNING`: array of returned rows
+- Mutations without `RETURNING`: empty array
+
+## Batch Operations via Loop
+
+Use a data selector returning an array to execute a query per item:
+
+```javascript
+// dataSelector: (sourceData) => sourceData.newUsers
+// body: {"query": "INSERT INTO users (name) VALUES ($1)", "params": ["<<(sourceData) => sourceData.currentItem.name>>"]}
+```

package/dist/dist/skills/superglue/references/file-servers.md

@@ -0,0 +1,136 @@
+# File Server Steps (FTP / SFTP / SMB)
+
+FTP, SFTP, and SMB share an identical operation interface. Only the URL protocol and connection details differ.
+
+## URL Formats
+
+```
+ftp://user:password@host:port/basePath       (port 21)
+ftps://user:password@host:port/basePath      (port 21, TLS)
+sftp://user:password@host:port/basePath      (port 22)
+smb://user:password@host/sharename/basePath  (port 445)
+smb://domain\user:password@host/sharename/   (domain auth)
+```
+
+All support `<<variable>>` syntax in the URL.
+
+SMB requires a share name as the first path segment. Additional segments become a base path prepended to all operation paths.
+
+## Path Handling
+
+**All operation paths are relative to the base path in the URL.** Do NOT use absolute filesystem paths.
+
+If the system URL is `sftp://user:pass@host/home/sftptest/uploads`:
+
+- `report.csv` → `/home/sftptest/uploads/report.csv`
+- `subdir/data.csv` → `/home/sftptest/uploads/subdir/data.csv`
+
+Do NOT use `/home/sftptest/uploads/report.csv` — the base path is already set in the URL.
+
+If the URL has no base path (e.g., `sftp://user:pass@host`), operation paths are used as-is from the server root.
+
+## Body Format
+
+JSON string — single operation or array of operations (batch):
+
+### Single
+
+```json
+{ "operation": "get", "path": "data/report.csv" }
+```
+
+### Batch (sequential, same connection)
+
+```json
+[
+  { "operation": "mkdir", "path": "backup" },
+  { "operation": "get", "path": "data/report.csv" },
+  { "operation": "put", "path": "backup/report.csv", "content": "data here" }
+]
+```
+
+## Supported Operations
+
+| Operation | Required fields   | Returns                                                                 |
+| --------- | ----------------- | ----------------------------------------------------------------------- |
+| `list`    | `path`            | Array of `{ name, path, size, type, modifyTime, ... }`                  |
+| `get`     | `path`            | Auto-parsed file content (CSV→objects, JSON→parsed, etc.) or raw string |
+| `put`     | `path`, `content` | `{ success, message, size }`                                            |
+| `delete`  | `path`            | `{ success, message }`                                                  |
+| `rename`  | `path`, `newPath` | `{ success, message }`                                                  |
+| `mkdir`   | `path`            | `{ success, message }`                                                  |
+| `rmdir`   | `path`            | `{ success, message }`                                                  |
+| `exists`  | `path`            | `{ exists: boolean, path }`                                             |
+| `stat`    | `path`            | File metadata or `{ exists: false }`                                    |
+
+### `get` — Auto-Parsing
+
+Downloaded files are all parsed automatically:
+
+- CSV, JSON, XML, Excel, PDF, etc. → parsed to JS objects
+- Falls back to UTF-8 string
+- SMB additionally detects binary files (returns `{ _binary: true, encoding: "base64", data: "..." }`)
+
+### `put` — Content Handling
+
+- String → written as-is
+- Buffer → written directly
+- Object/Array → `JSON.stringify` with 2-space indent
+
+## Connection Details
+
+### SFTP
+
+Uses `ssh2-sftp-client`. Supports password + private key auth (`credentials.privateKey`, `credentials.passphrase`).
+
+### FTP/FTPS
+
+Uses `basic-ftp`. FTPS uses TLS with `rejectUnauthorized: false`.
+
+### SMB
+
+Uses `@awo00/smb2`. Connection lifecycle: create client → authenticate (domain optional) → connect tree (share) → execute → disconnect/logoff/close. Entire operation wrapped in timeout race.
+
+All paths use forward slashes (`/`) — SMB library handles Windows conversion internally.
+
+Credentials must be injected into the URL using placeholders:
+
+```
+smb://<<systemId_username>>:<<systemId_password>>@fileserver.example.com/ShareName
+```
+
+For private systems via Secure Gateway, the host is `tunnelId.tunnel` (e.g., `my_tunnel.tunnel`).
+
+## Return Value
+
+Single operation → result directly. Multiple operations → array of results.
+
+## Common Patterns
+
+### Download and process
+
+```json
+{ "operation": "get", "path": "incoming/<<(sourceData) => sourceData.currentItem.filename>>" }
+```
+
+### Upload results
+
+```json
+{
+  "operation": "put",
+  "path": "output/results.json",
+  "content": "<<(sourceData) => JSON.stringify(sourceData.processData.data)>>"
+}
+```
+
+### List then loop
+
+Step 1 lists files, step 2 uses data selector to loop over them:
+
+```javascript
+// Step 1: list files
+// body: {"operation": "list", "path": "incoming"}
+
+// Step 2 dataSelector: (sourceData) => sourceData.listFiles.data.filter(f => f.type === "file")
+// body: {"operation": "get", "path": "<<(sourceData) => sourceData.currentItem.path>>"}
+```

package/dist/dist/skills/superglue/references/integration.md

@@ -0,0 +1,279 @@
+# Integrating superglue into Your Codebase
+
+After building and saving tools with the CLI, integrate them into your application.
+
+## Quick Reference: REST API (curl)
+
+For any language, use the REST API directly:
+
+```bash
+# Set your API key and endpoint
+export SUPERGLUE_API_KEY="your-api-key"
+export SUPERGLUE_API_ENDPOINT="https://api.superglue.cloud"
+
+# Run a tool (sync - waits for completion)
+curl -X POST "$SUPERGLUE_API_ENDPOINT/v1/tools/my-tool-id/run" \
+  -H "Authorization: Bearer $SUPERGLUE_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"inputs": {"userId": "123"}}'
+
+# Run a tool (async - returns immediately)
+curl -X POST "$SUPERGLUE_API_ENDPOINT/v1/tools/my-tool-id/run" \
+  -H "Authorization: Bearer $SUPERGLUE_API_KEY" \
+  -H "Content-Type: application/json" \
+  -d '{"inputs": {"userId": "123"}, "options": {"async": true}}'
+
+# Get run status/result
+curl "$SUPERGLUE_API_ENDPOINT/v1/runs/{runId}" \
+  -H "Authorization: Bearer $SUPERGLUE_API_KEY"
+
+# List tools
+curl "$SUPERGLUE_API_ENDPOINT/v1/tools?page=1&limit=50" \
+  -H "Authorization: Bearer $SUPERGLUE_API_KEY"
+
+# Get tool details
+curl "$SUPERGLUE_API_ENDPOINT/v1/tools/my-tool-id" \
+  -H "Authorization: Bearer $SUPERGLUE_API_KEY"
+
+# Cancel a running tool
+curl -X POST "$SUPERGLUE_API_ENDPOINT/v1/runs/{runId}/cancel" \
+  -H "Authorization: Bearer $SUPERGLUE_API_KEY"
+```
+
+### Response Format
+
+**Sync execution (200 OK):**
+
+```json
+{
+  "runId": "7f3e9c1a-2b4d-4e8f-9a3b-1c5d7e9f2a4b",
+  "toolId": "my-tool-id",
+  "status": "success",
+  "data": { "result": "..." },
+  "stepResults": [{ "stepId": "step-1", "success": true, "data": {} }],
+  "metadata": {
+    "startedAt": "2024-01-15T10:00:00Z",
+    "completedAt": "2024-01-15T10:00:05Z",
+    "durationMs": 5234
+  }
+}
+```
+
+**Async execution (202 Accepted):**
+
+```json
+{
+  "runId": "7f3e9c1a-2b4d-4e8f-9a3b-1c5d7e9f2a4b",
+  "toolId": "my-tool-id",
+  "status": "running"
+}
+```
+
+---
+
+## SDK Installation
+
+```bash
+# TypeScript/JavaScript
+npm install @superglue/client
+
+# Python
+pip install superglue-client
+```
+
+Both SDKs are auto-generated from the OpenAPI spec (JS via orval, Python via openapi-python-client).
+
+## TypeScript/JavaScript Usage
+
+The JS SDK exports: `configure`, `listTools`, `getTool`, `runTool`, `getRun`, `cancelRun`, `triggerWebhook`, `listRuns`.
+
+```typescript
+import { configure, runTool, getRun, listTools, getTool } from "@superglue/client";
+
+configure({
+  apiKey: process.env.SUPERGLUE_API_KEY!,
+  baseUrl: "https://api.superglue.cloud/v1",
+});
+```
+
+### Run a Tool (Sync)
+
+```typescript
+const response = await runTool("my-tool-id", {
+  inputs: { userId: "123", date: "2024-01-15" },
+});
+
+if (response.status === 200) {
+  const run = response.data;
+  console.log(run.status); // "success" | "failed"
+  console.log(run.data); // tool output
+} else {
+  console.error("Error:", response.data);
+}
+```
+
+The response is `{ data: Run, status: number, headers: Headers }`. The `Run` object has: `runId`, `toolId`, `status`, `data`, `error`, `stepResults`, `metadata`.
+
+### Async Execution (Fire-and-Forget)
+
+```typescript
+const response = await runTool("my-tool-id", {
+  inputs: { userId: "123" },
+  options: { async: true },
+});
+
+if (response.status === 202) {
+  const run = response.data;
+  console.log(`Run started: ${run.runId}`);
+
+  let latest = run;
+  while (latest.status === "running") {
+    await new Promise((r) => setTimeout(r, 1000));
+    const poll = await getRun(latest.runId);
+    latest = poll.data;
+  }
+}
+```
+
+### List and Discover Tools
+
+```typescript
+const { data: tools } = await listTools({ page: 1, limit: 50 });
+console.log(tools.data);
+
+const { data: tool } = await getTool("my-tool-id");
+console.log(tool.inputSchema);
+```
+
+## Python Usage
+
+The Python SDK uses `AuthenticatedClient` with httpx. Functions are in `superglue_client.api.tools` and `superglue_client.api.runs`.
+
+```python
+import os
+from superglue_client import AuthenticatedClient
+from superglue_client.api.tools import run_tool, list_tools, get_tool
+from superglue_client.api.runs import get_run
+from superglue_client.models.run_request import RunRequest
+from superglue_client.models.run_request_inputs import RunRequestInputs
+
+client = AuthenticatedClient(
+    base_url="https://api.superglue.cloud/v1",
+    token=os.environ["SUPERGLUE_API_KEY"],
+)
+```
+
+### Run a Tool (Sync)
+
+```python
+with client as c:
+    result = run_tool.sync(
+        "my-tool-id",
+        client=c,
+        body=RunRequest(
+            inputs=RunRequestInputs.from_dict({"userId": "123"}),
+        ),
+    )
+
+    if hasattr(result, "status") and result.status == "success":
+        print(result.data)
+    elif hasattr(result, "error"):
+        print(result.error)
+```
+
+`sync()` returns `Run | Error | None`. Use `sync_detailed()` for the full `Response` wrapper with status code and headers.
+
+### Async Python
+
+```python
+async with client as c:
+    result = await run_tool.asyncio(
+        "my-tool-id",
+        client=c,
+        body=RunRequest(
+            inputs=RunRequestInputs.from_dict({"userId": "123"}),
+        ),
+    )
+```
+
+## Webhook Triggers
+
+Trigger tools via HTTP POST without SDK. Webhooks execute asynchronously.
+
+```bash
+curl -X POST "https://api.superglue.cloud/v1/hooks/{toolId}?token={apiKey}" \
+  -H "Content-Type: application/json" \
+  -d '{"userId": "123"}'
+```
+
+### Webhook URL Pattern
+
+```
+POST {endpoint}/v1/hooks/{toolId}?token={apiKey}
+```
+
+- `endpoint` — Your superglue API endpoint
+- `toolId` — The saved tool ID (from `sg tool save`)
+- `apiKey` — Your superglue API key (passed as `token` query param)
+
+### Webhook Response
+
+Webhooks return `202 Accepted` immediately:
+
+```json
+{
+  "runId": "uuid-of-execution",
+  "status": "accepted",
+  "toolId": "my-tool-id"
+}
+```
+
+To get the result, poll the run status:
+
+```bash
+curl "https://api.superglue.cloud/v1/runs/{runId}" \
+  -H "Authorization: Bearer {apiKey}"
+```
+
+## Error Handling
+
+### Retry Pattern (TypeScript)
+
+```typescript
+async function runToolWithRetry(toolId: string, inputs: Record<string, any>, maxRetries = 3) {
+  for (let attempt = 1; attempt <= maxRetries; attempt++) {
+    const response = await runTool(toolId, { inputs });
+
+    if (response.status === 200 && response.data.status === "success") {
+      return response.data;
+    }
+
+    if (response.data.error?.includes("validation")) {
+      throw new Error(response.data.error);
+    }
+
+    if (attempt < maxRetries) {
+      await new Promise((r) => setTimeout(r, 1000 * Math.pow(2, attempt)));
+    }
+  }
+  throw new Error(`Tool ${toolId} failed after ${maxRetries} attempts`);
+}
+```
+
+## Environment Configuration
+
+```bash
+# Required
+SUPERGLUE_API_KEY=your-api-key
+
+# Optional (defaults to hosted)
+SUPERGLUE_API_ENDPOINT=https://api.superglue.cloud
+```
+
+## Best Practices
+
+1. **Store API keys in secrets** — Never commit to version control
+2. **Validate payloads client-side** — Check against `inputSchema` before calling
+3. **Handle errors gracefully** — Tools can fail due to external API issues
+4. **Monitor run history** — Use `sg run list` or the web UI to track executions
+5. **Use async mode for long-running tools** — Avoid timeouts on complex workflows

package/dist/dist/skills/superglue/references/redis.md

@@ -0,0 +1,190 @@
+# Redis Steps
+
+## Step Configuration
+
+```typescript
+{
+  type: "request",
+  systemId: "my_redis",
+  url: "redis://<<my_redis_username>>:<<my_redis_password>>@<<my_redis_host>>:<<my_redis_port>>/<<my_redis_database>>",
+  method: "POST",            // ignored for Redis but required by schema
+  body: '{"command": "GET", "args": ["<<keyName>>"]}'
+}
+```
+
+Configure the connection URL depending on which credentials are saved in the stored Redis system - reference them accordingly.
+
+## URL Format
+
+```
+redis://user:password@host:port/database
+rediss://user:password@host:port/database   (TLS)
+```
+
+- `redis://` for standard connections
+- `rediss://` for TLS-encrypted connections (e.g. Redis Cloud, AWS ElastiCache with encryption)
+- `/database` is the database number (0-15), defaults to 0 if omitted
+
+All credential variables are resolved before connecting. Trailing slashes are cleaned.
+
+## Body Format
+
+JSON string with `command` (required) and optional `args` array. Body can be a single command object or an array of commands (pipelined in a single round-trip):
+
+```json
+{ "command": "GET", "args": ["mykey"] }
+```
+
+Multiple commands:
+
+```json
+[
+  { "command": "GET", "args": ["user:1:name"] },
+  { "command": "HGETALL", "args": ["user:1"] }
+]
+```
+
+### Common Commands
+
+**String operations:**
+
+```json
+{ "command": "GET", "args": ["user:123:name"] }
+{ "command": "SET", "args": ["user:123:name", "Alice"] }
+{ "command": "MGET", "args": ["key1", "key2", "key3"] }
+{ "command": "INCR", "args": ["counter"] }
+```
+
+**Hash operations:**
+
+```json
+{ "command": "HGETALL", "args": ["user:123"] }
+{ "command": "HGET", "args": ["user:123", "email"] }
+{ "command": "HSET", "args": ["user:123", "email", "alice@example.com"] }
+{ "command": "HMGET", "args": ["user:123", "name", "email"] }
+```
+
+**List operations:**
+
+```json
+{ "command": "LRANGE", "args": ["queue:tasks", "0", "-1"] }
+{ "command": "LPUSH", "args": ["queue:tasks", "task1"] }
+{ "command": "RPOP", "args": ["queue:tasks"] }
+{ "command": "LLEN", "args": ["queue:tasks"] }
+```
+
+**Set operations:**
+
+```json
+{ "command": "SMEMBERS", "args": ["tags:post:1"] }
+{ "command": "SADD", "args": ["tags:post:1", "redis", "database"] }
+{ "command": "SISMEMBER", "args": ["tags:post:1", "redis"] }
+```
+
+**Sorted set operations:**
+
+```json
+{ "command": "ZRANGE", "args": ["leaderboard", "0", "-1", "WITHSCORES"] }
+{ "command": "ZADD", "args": ["leaderboard", "100", "player1"] }
+{ "command": "ZRANK", "args": ["leaderboard", "player1"] }
+```
+
+**Key operations:**
+
+```json
+{ "command": "KEYS", "args": ["user:*"] }
+{ "command": "EXISTS", "args": ["mykey"] }
+{ "command": "TTL", "args": ["session:abc"] }
+{ "command": "EXPIRE", "args": ["session:abc", "3600"] }
+{ "command": "DEL", "args": ["mykey"] }
+{ "command": "TYPE", "args": ["mykey"] }
+```
+
+**Scan (safe iteration):**
+
+```json
+{ "command": "SCAN", "args": ["0", "MATCH", "user:*", "COUNT", "100"] }
+{ "command": "HSCAN", "args": ["myhash", "0", "MATCH", "field*"] }
+```
+
+### Dynamic Arguments with Variables
+
+```json
+{ "command": "GET", "args": ["<<(sourceData) => `user:${sourceData.userId}:profile`>>"] }
+```
+
+```json
+{ "command": "HSET", "args": ["<<userKey>>", "<<fieldName>>", "<<fieldValue>>"] }
+```
+
+## Connection Management
+
+- A fresh connection is created per execution and closed after completion
+- TLS auto-configured for `rediss://` URLs (with `rejectUnauthorized: false`)
+- 5-second connection timeout, 30-second command timeout
+- Array commands (pipeline) run on a single connection for efficiency
+
+## Retry Logic
+
+Default retries from server config. On final failure, error includes command and args.
+
+## Return Value
+
+Returns the raw Redis response:
+
+- `GET` / `HGET`: string or null
+- `HGETALL`: object with field-value pairs (flattened array from Redis, parsed by ioredis)
+- `LRANGE` / `SMEMBERS` / `KEYS`: array of strings
+- `SET` / `DEL` / `EXPIRE`: "OK" or integer
+- `INCR` / `LLEN` / `SADD`: integer
+- `MGET` / `HMGET`: array (nulls for missing keys)
+- `SCAN` / `HSCAN`: array of [cursor, results]
+
+## Multiple Commands (Pipeline)
+
+Pass an array of commands to execute them in a single round-trip (like FTP/SFTP batch operations):
+
+```json
+[
+  { "command": "GET", "args": ["user:1:name"] },
+  { "command": "HGETALL", "args": ["user:1"] },
+  { "command": "LRANGE", "args": ["tasks", "0", "-1"] }
+]
+```
+
+Returns an array of results, one per command:
+
+```json
+[
+  { "command": "GET", "result": "Alice" },
+  { "command": "HGETALL", "result": ["name", "Alice", "email", "alice@example.com"] },
+  { "command": "LRANGE", "result": ["task3", "task2", "task1"] }
+]
+```
+
+If an individual command fails, that entry has `error` instead of `result`:
+
+```json
+[
+  { "command": "GET", "result": "Alice" },
+  { "command": "WRONGCMD", "error": "ERR unknown command 'WRONGCMD'" }
+]
+```
+
+A single command returns a single result. Multiple commands return an array — same pattern as FTP/SFTP operations.
+
+## Batch Operations via Loop
+
+Use a data selector returning an array to execute a command per item:
+
+```javascript
+// dataSelector: (sourceData) => sourceData.userIds
+// body: {"command": "HGETALL", "args": ["<<(sourceData) => `user:${sourceData.currentItem}`>>"]}
+```
+
+## Important Notes
+
+- All args must be strings (Redis protocol is text-based) - numbers are auto-coerced
+- Use `SCAN` instead of `KEYS` in production for large keyspaces
+- `HGETALL` returns an empty object `{}` for non-existent keys
+- Use array body for multi-command pipelines instead of separate steps