@superglue/cli 1.1.20 → 1.1.22
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/plugin.json +1 -1
- package/README.md +1 -7
- package/dist/cli.js +303 -74
- package/dist/skills/superglue/SKILL.md +313 -621
- package/dist/skills/superglue/references/file-handling.md +116 -90
- package/dist/skills/superglue/references/graphql.md +131 -0
- package/dist/skills/superglue/references/http.md +246 -0
- package/dist/skills/superglue/references/integration.md +4 -2
- package/dist/skills/superglue/references/mssql.md +207 -0
- package/dist/skills/superglue/references/postgres.md +154 -0
- package/dist/skills/superglue/references/redis.md +68 -118
- package/dist/skills/superglue/references/sftp-smb.md +156 -0
- package/dist/skills/superglue/references/superglue-info.md +91 -0
- package/package.json +1 -1
- package/skills/superglue/SKILL.md +313 -621
- package/skills/superglue/references/file-handling.md +116 -90
- package/skills/superglue/references/graphql.md +131 -0
- package/skills/superglue/references/http.md +246 -0
- package/skills/superglue/references/integration.md +4 -2
- package/skills/superglue/references/mssql.md +207 -0
- package/skills/superglue/references/postgres.md +154 -0
- package/skills/superglue/references/redis.md +68 -118
- package/skills/superglue/references/sftp-smb.md +156 -0
- package/skills/superglue/references/superglue-info.md +91 -0
- package/dist/skills/superglue/references/data-handling.md +0 -303
- package/dist/skills/superglue/references/databases.md +0 -253
- package/dist/skills/superglue/references/file-servers.md +0 -112
- package/dist/skills/superglue/references/http-apis.md +0 -248
- package/skills/superglue/references/data-handling.md +0 -303
- package/skills/superglue/references/databases.md +0 -253
- package/skills/superglue/references/file-servers.md +0 -112
- package/skills/superglue/references/http-apis.md +0 -248
|
@@ -73,12 +73,14 @@ curl -X POST "$SUPERGLUE_API_ENDPOINT/v1/runs/{runId}/cancel" \
|
|
|
73
73
|
|
|
74
74
|
## SDK Installation
|
|
75
75
|
|
|
76
|
+
**Always install the latest version** — older SDK versions may have known bugs (e.g. case-sensitivity issues on Linux in versions before 3.3.0).
|
|
77
|
+
|
|
76
78
|
```bash
|
|
77
79
|
# TypeScript/JavaScript
|
|
78
|
-
npm install @superglue/client
|
|
80
|
+
npm install @superglue/client@latest
|
|
79
81
|
|
|
80
82
|
# Python
|
|
81
|
-
pip install superglue-client
|
|
83
|
+
pip install --upgrade superglue-client
|
|
82
84
|
```
|
|
83
85
|
|
|
84
86
|
Both SDKs are auto-generated from the OpenAPI spec (JS via orval, Python via openapi-python-client).
|
|
@@ -0,0 +1,207 @@
|
|
|
1
|
+
# MSSQL
|
|
2
|
+
|
|
3
|
+
## Step Configuration
|
|
4
|
+
|
|
5
|
+
```typescript
|
|
6
|
+
{
|
|
7
|
+
type: "request",
|
|
8
|
+
systemId: "my_azure_sql",
|
|
9
|
+
url: "mssql://<<my_azure_sql_user>>:<<my_azure_sql_password>>@<<my_azure_sql_host>>:<<my_azure_sql_port>>/<<my_azure_sql_database>>",
|
|
10
|
+
body: '{"query": "SELECT * FROM users WHERE id = @param1", "params": [<<userId>>]}'
|
|
11
|
+
}
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Configure the connection URL based on which credentials are stored in the system — use `sg system find` to check `storedCredentials` and reference them accordingly.
|
|
15
|
+
|
|
16
|
+
`method`, `headers`, `queryParams`, and `pagination` are HTTP-only fields — omit them for database steps.
|
|
17
|
+
|
|
18
|
+
### Fields
|
|
19
|
+
|
|
20
|
+
| Field | Required | Notes |
|
|
21
|
+
| ---------- | -------- | -------------------------------------------------------------- |
|
|
22
|
+
| `url` | yes | Connection string with `<<credential>>` placeholders |
|
|
23
|
+
| `body` | yes | JSON string with `query` and optional `params` |
|
|
24
|
+
| `systemId` | no | Links system credentials for `<<systemId_credKey>>` resolution |
|
|
25
|
+
|
|
26
|
+
### URL Format
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
mssql://user:password@host:port/database
|
|
30
|
+
sqlserver://user:password@host:port/database
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Both `mssql://` and `sqlserver://` are supported.
|
|
34
|
+
|
|
35
|
+
**Azure SQL Database** — use the fully qualified server name:
|
|
36
|
+
|
|
37
|
+
```
|
|
38
|
+
mssql://myuser:mypassword@myserver.database.windows.net:1433/mydatabase
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Azure SQL usernames often include `@servername` suffix (e.g., `myuser@myserver`). Encode the `@` as `%40` in the URL:
|
|
42
|
+
|
|
43
|
+
```
|
|
44
|
+
mssql://myuser%40myserver:mypassword@myserver.database.windows.net:1433/mydatabase
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
**Connection parameters** can be appended as query strings:
|
|
48
|
+
|
|
49
|
+
```
|
|
50
|
+
mssql://user:password@host:port/database?encrypt=true&trustServerCertificate=false
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
| Parameter | Default | Notes |
|
|
54
|
+
| ------------------------ | ------- | --------------------------------------- |
|
|
55
|
+
| `encrypt` | `true` | Required for Azure SQL |
|
|
56
|
+
| `trustServerCertificate` | `false` | Whether to trust the server certificate |
|
|
57
|
+
| `database` | — | Can be in URL path or query parameter |
|
|
58
|
+
|
|
59
|
+
### Body Format
|
|
60
|
+
|
|
61
|
+
JSON string with `query` and optional `params` (or `values`):
|
|
62
|
+
|
|
63
|
+
```json
|
|
64
|
+
{
|
|
65
|
+
"query": "SELECT * FROM users WHERE age > @param1 AND status = @param2",
|
|
66
|
+
"params": [25, "active"]
|
|
67
|
+
}
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
## Authentication
|
|
71
|
+
|
|
72
|
+
### SQL Server Authentication (default)
|
|
73
|
+
|
|
74
|
+
Credentials are embedded directly in the connection URL using `<<systemId_credentialKey>>` placeholders. Nothing is injected automatically.
|
|
75
|
+
|
|
76
|
+
```
|
|
77
|
+
mssql://<<my_db_user>>:<<my_db_password>>@<<my_db_host>>:<<my_db_port>>/<<my_db_database>>
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
### Azure Active Directory Authentication
|
|
81
|
+
|
|
82
|
+
For Azure AD auth, store the following credential keys on the system alongside the standard connection fields:
|
|
83
|
+
|
|
84
|
+
**Entra ID User + Password** — authenticates as an Azure AD user. Requires an Azure app registration.
|
|
85
|
+
|
|
86
|
+
| Credential Key | Required | Notes |
|
|
87
|
+
| -------------- | -------- | ------------------------------------------------------ |
|
|
88
|
+
| `authType` | yes | `azure-active-directory-password` |
|
|
89
|
+
| `clientId` | yes | Application (client) ID from Azure app registration |
|
|
90
|
+
| `tenantId` | no | Directory (tenant) ID |
|
|
91
|
+
| `user` | yes | Azure AD username (also accepts `username`, or in URL) |
|
|
92
|
+
| `password` | yes | Azure AD password (or in URL) |
|
|
93
|
+
|
|
94
|
+
`user` and `password` can be stored as credentials or embedded in the URL — credentials take precedence.
|
|
95
|
+
|
|
96
|
+
```
|
|
97
|
+
URL: mssql://<<sys_user>>:<<sys_password>>@myserver.database.windows.net:1433/mydb
|
|
98
|
+
Credentials: { authType, clientId, tenantId, user, password, host, port, database }
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
**Service Principal** — authenticates as an Azure AD application. No user/password needed in the URL.
|
|
102
|
+
|
|
103
|
+
| Credential Key | Required | Notes |
|
|
104
|
+
| -------------- | -------- | ------------------------------------------------- |
|
|
105
|
+
| `authType` | yes | `azure-active-directory-service-principal-secret` |
|
|
106
|
+
| `clientId` | yes | Application (client) ID |
|
|
107
|
+
| `clientSecret` | yes | Client secret |
|
|
108
|
+
| `tenantId` | yes | Directory (tenant) ID |
|
|
109
|
+
|
|
110
|
+
```
|
|
111
|
+
URL: mssql://myserver.database.windows.net:1433/mydb
|
|
112
|
+
Credentials: { authType, clientId, clientSecret, tenantId, host, port, database }
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
The runtime authenticates entirely via the service principal credentials — no user or password is needed in the URL.
|
|
116
|
+
|
|
117
|
+
## MSSQL Runtime Details
|
|
118
|
+
|
|
119
|
+
### Schema Discovery
|
|
120
|
+
|
|
121
|
+
superglue does not auto-introspect MSSQL schemas. Before building tools, always query the schema manually with `sg system call` to discover tables and columns:
|
|
122
|
+
|
|
123
|
+
```json
|
|
124
|
+
{
|
|
125
|
+
"query": "SELECT TABLE_NAME, COLUMN_NAME, DATA_TYPE, IS_NULLABLE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = 'dbo' ORDER BY TABLE_NAME, ORDINAL_POSITION"
|
|
126
|
+
}
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
For Azure SQL or databases with non-default schemas, adjust the `TABLE_SCHEMA` filter or remove it to see all schemas.
|
|
130
|
+
Always verify table and column names against the actual schema before building tools.
|
|
131
|
+
|
|
132
|
+
### Parameterized Queries
|
|
133
|
+
|
|
134
|
+
Always use parameterized queries with `@param1`, `@param2`, etc. This prevents SQL injection. The params array maps positionally: `params[0]` maps to `@param1`, `params[1]` maps to `@param2`, etc.
|
|
135
|
+
|
|
136
|
+
```json
|
|
137
|
+
{ "query": "SELECT * FROM orders WHERE customer_id = @param1", "params": ["<<customerId>>"] }
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
With expressions:
|
|
141
|
+
|
|
142
|
+
```json
|
|
143
|
+
{
|
|
144
|
+
"query": "INSERT INTO logs (message, level) VALUES (@param1, @param2)",
|
|
145
|
+
"params": ["<<(sourceData) => sourceData.message>>", "error"]
|
|
146
|
+
}
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
With OUTPUT (MSSQL equivalent of RETURNING):
|
|
150
|
+
|
|
151
|
+
```json
|
|
152
|
+
{
|
|
153
|
+
"query": "INSERT INTO users (name, email) OUTPUT INSERTED.* VALUES (@param1, @param2)",
|
|
154
|
+
"params": [
|
|
155
|
+
"<<(sourceData) => sourceData.currentItem.name>>",
|
|
156
|
+
"<<(sourceData) => sourceData.currentItem.email>>"
|
|
157
|
+
]
|
|
158
|
+
}
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
### Return Value
|
|
162
|
+
|
|
163
|
+
Returns the recordset — array of row objects with column names as keys.
|
|
164
|
+
|
|
165
|
+
- `SELECT`: array of matching rows
|
|
166
|
+
- `INSERT/UPDATE/DELETE` with `OUTPUT`: array of returned rows
|
|
167
|
+
- Mutations without `OUTPUT`: empty array
|
|
168
|
+
|
|
169
|
+
### Batch Operations
|
|
170
|
+
|
|
171
|
+
Use a data selector returning an array to execute a query per item:
|
|
172
|
+
|
|
173
|
+
```javascript
|
|
174
|
+
// dataSelector: (sourceData) => sourceData.newUsers
|
|
175
|
+
// body: {"query": "INSERT INTO users (name) VALUES (@param1)", "params": ["<<(sourceData) => sourceData.currentItem.name>>"]}
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
### Connection Pooling
|
|
179
|
+
|
|
180
|
+
- Pools cached by connection string, max 10 clients per pool
|
|
181
|
+
- 30s connection timeout, configurable request timeout
|
|
182
|
+
- Idle pools cleaned up automatically
|
|
183
|
+
- SSL/TLS encryption enabled by default for Azure SQL
|
|
184
|
+
|
|
185
|
+
### Retry Behavior
|
|
186
|
+
|
|
187
|
+
Default retries from server config. On final failure, error includes query text and params.
|
|
188
|
+
|
|
189
|
+
## Common Pitfalls
|
|
190
|
+
|
|
191
|
+
- Using `$1`, `$2` (PostgreSQL syntax) instead of `@param1`, `@param2` — MSSQL uses named `@param` parameters
|
|
192
|
+
- Forgetting `OUTPUT INSERTED.*` on INSERT/UPDATE/DELETE when you need the result — without it the step returns an empty array
|
|
193
|
+
- Accessing step result directly instead of via `.data` — database results follow the same envelope as all steps: `sourceData.stepId.data`
|
|
194
|
+
- Not URL-encoding `@` in Azure SQL usernames — `myuser@myserver` must be `myuser%40myserver` in the connection URL
|
|
195
|
+
|
|
196
|
+
## Error Recovery
|
|
197
|
+
|
|
198
|
+
When an MSSQL step fails and the cause is not obvious:
|
|
199
|
+
|
|
200
|
+
1. **Read the error message** — superglue includes the query text and params in the error. Check for syntax errors, missing columns, or type mismatches.
|
|
201
|
+
2. **Verify connection credentials** — use `sg system find` to confirm `storedCredentials` keys match the placeholders in your URL. Test connectivity with a simple `SELECT 1` via `sg system call`.
|
|
202
|
+
3. **Check database-side access** — connection refused or permission denied errors typically mean the database user lacks access. Verify with the user that the credentials have the required grants.
|
|
203
|
+
4. **Check Azure-specific issues** — if connecting to Azure SQL:
|
|
204
|
+
- Confirm the client IP is allowed in the Azure SQL firewall rules
|
|
205
|
+
- Verify the username includes the `@servername` suffix if required, properly encoded as `%40`
|
|
206
|
+
- Confirm `encrypt=true` is set (required for Azure SQL)
|
|
207
|
+
5. **Verify parameter types** — MSSQL infers types from the values passed. Passing a string where a number is expected may cause implicit conversion errors. Ensure `params` values match expected column types.
|
|
@@ -0,0 +1,154 @@
|
|
|
1
|
+
# PostgreSQL
|
|
2
|
+
|
|
3
|
+
## Step Configuration
|
|
4
|
+
|
|
5
|
+
```typescript
|
|
6
|
+
{
|
|
7
|
+
type: "request",
|
|
8
|
+
systemId: "my_postgres_db",
|
|
9
|
+
url: "postgres://<<my_postgres_db_user>>:<<my_postgres_db_password>>@<<my_postgres_db_host>>:<<my_postgres_db_port>>/<<my_postgres_db_database>>",
|
|
10
|
+
body: '{"query": "SELECT * FROM users WHERE id = $1", "params": [<<userId>>]}'
|
|
11
|
+
}
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
Configure the connection URL based on which credentials are stored in the system — use `sg system find` to check `storedCredentials` and reference them accordingly.
|
|
15
|
+
|
|
16
|
+
`method`, `headers`, `queryParams`, and `pagination` are HTTP-only fields — omit them for database steps.
|
|
17
|
+
|
|
18
|
+
### Fields
|
|
19
|
+
|
|
20
|
+
| Field | Required | Notes |
|
|
21
|
+
| ---------- | -------- | -------------------------------------------------------------- |
|
|
22
|
+
| `url` | yes | Connection string with `<<credential>>` placeholders |
|
|
23
|
+
| `body` | yes | JSON string with `query` and optional `params` |
|
|
24
|
+
| `systemId` | no | Links system credentials for `<<systemId_credKey>>` resolution |
|
|
25
|
+
|
|
26
|
+
### URL Format
|
|
27
|
+
|
|
28
|
+
```
|
|
29
|
+
postgres://user:password@host:port/database
|
|
30
|
+
postgresql://user:password@host:port/database
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
Both `postgres://` and `postgresql://` are supported. Trailing slashes and extra slashes before query strings are cleaned automatically.
|
|
34
|
+
|
|
35
|
+
### Body Format
|
|
36
|
+
|
|
37
|
+
JSON string with `query` and optional `params` (or `values`):
|
|
38
|
+
|
|
39
|
+
```json
|
|
40
|
+
{ "query": "SELECT * FROM users WHERE age > $1 AND status = $2", "params": [25, "active"] }
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
## Authentication
|
|
44
|
+
|
|
45
|
+
Credentials are embedded directly in the connection URL using `<<systemId_credentialKey>>` placeholders. Nothing is injected automatically.
|
|
46
|
+
|
|
47
|
+
```
|
|
48
|
+
postgres://<<my_db_user>>:<<my_db_password>>@<<my_db_host>>:<<my_db_port>>/<<my_db_database>>
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Credential variables are resolved at runtime before connecting.
|
|
52
|
+
|
|
53
|
+
## PostgreSQL Runtime Details
|
|
54
|
+
|
|
55
|
+
### Schema Discovery
|
|
56
|
+
|
|
57
|
+
superglue does not auto-introspect Postgres schemas. Before building tools, always query the schema manually with `sg system call` to discover tables and columns:
|
|
58
|
+
|
|
59
|
+
```json
|
|
60
|
+
{
|
|
61
|
+
"query": "SELECT table_name, column_name, data_type, is_nullable FROM information_schema.columns WHERE table_schema = 'public' ORDER BY table_name, ordinal_position"
|
|
62
|
+
}
|
|
63
|
+
```
|
|
64
|
+
|
|
65
|
+
Always verify table and column names against the actual schema before building tools.
|
|
66
|
+
|
|
67
|
+
### Parameterized Queries
|
|
68
|
+
|
|
69
|
+
Always use parameterized queries with `$1`, `$2`, etc. — prevents SQL injection.
|
|
70
|
+
|
|
71
|
+
```json
|
|
72
|
+
{ "query": "SELECT * FROM orders WHERE customer_id = $1", "params": ["<<customerId>>"] }
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
With expressions:
|
|
76
|
+
|
|
77
|
+
```json
|
|
78
|
+
{
|
|
79
|
+
"query": "INSERT INTO logs (message, level) VALUES ($1, $2)",
|
|
80
|
+
"params": ["<<(sourceData) => sourceData.message>>", "error"]
|
|
81
|
+
}
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
With RETURNING:
|
|
85
|
+
|
|
86
|
+
```json
|
|
87
|
+
{
|
|
88
|
+
"query": "INSERT INTO users (name, email) VALUES ($1, $2) RETURNING *",
|
|
89
|
+
"params": [
|
|
90
|
+
"<<(sourceData) => sourceData.currentItem.name>>",
|
|
91
|
+
"<<(sourceData) => sourceData.currentItem.email>>"
|
|
92
|
+
]
|
|
93
|
+
}
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
### Return Value
|
|
97
|
+
|
|
98
|
+
Returns `result.rows` — array of row objects with column names as keys.
|
|
99
|
+
|
|
100
|
+
- `SELECT`: array of matching rows
|
|
101
|
+
- `INSERT/UPDATE/DELETE` with `RETURNING`: array of returned rows
|
|
102
|
+
- Mutations without `RETURNING`: empty array
|
|
103
|
+
|
|
104
|
+
### Batch Operations
|
|
105
|
+
|
|
106
|
+
Use a data selector returning an array to execute a query per item:
|
|
107
|
+
|
|
108
|
+
```javascript
|
|
109
|
+
// dataSelector: (sourceData) => sourceData.newUsers
|
|
110
|
+
// body: {"query": "INSERT INTO users (name) VALUES ($1)", "params": ["<<(sourceData) => sourceData.currentItem.name>>"]}
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
### Connection Pooling
|
|
114
|
+
|
|
115
|
+
- Pools cached by connection string, max 10 clients, 30s connection timeout
|
|
116
|
+
- Idle pools cleaned up automatically
|
|
117
|
+
|
|
118
|
+
### SSL / TLS
|
|
119
|
+
|
|
120
|
+
SSL is auto-configured based on the connection URL — you do not need to set it manually in most cases.
|
|
121
|
+
|
|
122
|
+
**Default behavior (no `sslmode` in URL):**
|
|
123
|
+
|
|
124
|
+
- `localhost` / `127.0.0.1` → SSL disabled
|
|
125
|
+
- All other hosts → superglue probes the server to detect SSL support and enables it if the server accepts SSL handshakes. If the probe fails or times out, SSL is enabled as a safe default.
|
|
126
|
+
|
|
127
|
+
**Explicit `sslmode` in URL** (appended as `?sslmode=...`):
|
|
128
|
+
|
|
129
|
+
- `sslmode=disable` → SSL off
|
|
130
|
+
- `sslmode=require` → SSL on, enforced
|
|
131
|
+
- `sslmode=verify-ca` / `sslmode=verify-full` → SSL on, enforced; AWS RDS CA bundle loaded if present on disk
|
|
132
|
+
- Any other sslmode value → SSL on, not enforced
|
|
133
|
+
|
|
134
|
+
Server certificate chain validation is not performed by default; `verify-ca` / `verify-full` only flip the `enforce` flag and opportunistically load the RDS CA bundle. If strict chain validation matters for your deployment, terminate TLS at a trusted proxy.
|
|
135
|
+
|
|
136
|
+
### Retry Behavior
|
|
137
|
+
|
|
138
|
+
Default retries from server config. On final failure, error includes query text and params.
|
|
139
|
+
|
|
140
|
+
## Common Pitfalls
|
|
141
|
+
|
|
142
|
+
- Using string interpolation instead of parameterized queries — always use `$1`, `$2` with `params` array
|
|
143
|
+
- Forgetting `RETURNING *` on INSERT/UPDATE/DELETE when you need the result — without it the step returns an empty array
|
|
144
|
+
- Accessing step result directly instead of via `.data` — database results follow the same envelope as all steps: `sourceData.stepId.data`
|
|
145
|
+
|
|
146
|
+
## Error Recovery
|
|
147
|
+
|
|
148
|
+
When a PostgreSQL step fails and the cause is not obvious:
|
|
149
|
+
|
|
150
|
+
1. **Read the error message** — superglue includes the query text and params in the error. Check for syntax errors, missing columns, or type mismatches.
|
|
151
|
+
2. **Verify connection credentials** — use `sg system find` to confirm `storedCredentials` keys match the placeholders in your URL. Test connectivity with a simple `SELECT 1` via `sg system call`.
|
|
152
|
+
3. **Check database-side access** — connection refused or permission denied errors typically mean the database user lacks access to the database, schema, or table. Verify with the user that the credentials have the required grants.
|
|
153
|
+
4. **Test the query directly** — use `sg system call` with a simplified version of the query to isolate whether the issue is the query itself, the parameters, or the connection.
|
|
154
|
+
5. **Check for SSL/TLS issues** — if the error mentions SSL or handshake failure, check the SSL / TLS section above. For non-localhost servers that don't support SSL, add `?sslmode=disable` to the URL. For servers that require SSL, add `?sslmode=require`.
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Redis
|
|
1
|
+
# Redis
|
|
2
2
|
|
|
3
3
|
## Step Configuration
|
|
4
4
|
|
|
@@ -11,9 +11,19 @@
|
|
|
11
11
|
}
|
|
12
12
|
```
|
|
13
13
|
|
|
14
|
-
Configure the connection URL
|
|
14
|
+
Configure the connection URL based on which credentials are stored in the system — use `sg system find` to check `storedCredentials` and reference them accordingly.
|
|
15
15
|
|
|
16
|
-
|
|
16
|
+
`method`, `headers`, `queryParams`, and `pagination` are HTTP-only fields — omit them for Redis steps.
|
|
17
|
+
|
|
18
|
+
### Fields
|
|
19
|
+
|
|
20
|
+
| Field | Required | Notes |
|
|
21
|
+
| ---------- | -------- | -------------------------------------------------------------- |
|
|
22
|
+
| `url` | yes | Connection string with `<<credential>>` placeholders |
|
|
23
|
+
| `body` | yes | JSON: single command object or array for pipeline |
|
|
24
|
+
| `systemId` | no | Links system credentials for `<<systemId_credKey>>` resolution |
|
|
25
|
+
|
|
26
|
+
### URL Format
|
|
17
27
|
|
|
18
28
|
```
|
|
19
29
|
redis://user:password@host:port/database
|
|
@@ -21,20 +31,18 @@ rediss://user:password@host:port/database (TLS)
|
|
|
21
31
|
```
|
|
22
32
|
|
|
23
33
|
- `redis://` for standard connections
|
|
24
|
-
- `rediss://` for TLS-encrypted connections (
|
|
25
|
-
- `/database` is the database number (defaults to 0
|
|
26
|
-
|
|
27
|
-
All credential variables are resolved before connecting. Trailing slashes are cleaned.
|
|
34
|
+
- `rediss://` for TLS-encrypted connections (Redis Cloud, AWS ElastiCache with encryption)
|
|
35
|
+
- `/database` is the database number (defaults to 0; max depends on server config, commonly 16)
|
|
28
36
|
|
|
29
|
-
|
|
37
|
+
### Body Format
|
|
30
38
|
|
|
31
|
-
JSON string with `command` (required) and optional `args` array
|
|
39
|
+
JSON string with `command` (required) and optional `args` array:
|
|
32
40
|
|
|
33
41
|
```json
|
|
34
42
|
{ "command": "GET", "args": ["mykey"] }
|
|
35
43
|
```
|
|
36
44
|
|
|
37
|
-
|
|
45
|
+
For pipelines, pass an array of command objects (executed in a single round-trip):
|
|
38
46
|
|
|
39
47
|
```json
|
|
40
48
|
[
|
|
@@ -43,115 +51,39 @@ Multiple commands:
|
|
|
43
51
|
]
|
|
44
52
|
```
|
|
45
53
|
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
**String operations:**
|
|
49
|
-
|
|
50
|
-
```json
|
|
51
|
-
{ "command": "GET", "args": ["user:123:name"] }
|
|
52
|
-
{ "command": "SET", "args": ["user:123:name", "Alice"] }
|
|
53
|
-
{ "command": "MGET", "args": ["key1", "key2", "key3"] }
|
|
54
|
-
{ "command": "INCR", "args": ["counter"] }
|
|
55
|
-
```
|
|
54
|
+
All args must be strings — numbers are auto-coerced.
|
|
56
55
|
|
|
57
|
-
|
|
56
|
+
Dynamic arguments with variables:
|
|
58
57
|
|
|
59
58
|
```json
|
|
60
|
-
{ "command": "
|
|
61
|
-
{ "command": "HGET", "args": ["user:123", "email"] }
|
|
62
|
-
{ "command": "HSET", "args": ["user:123", "email", "alice@example.com"] }
|
|
63
|
-
{ "command": "HMGET", "args": ["user:123", "name", "email"] }
|
|
59
|
+
{ "command": "GET", "args": ["<<(sourceData) => `user:${sourceData.userId}:profile`>>"] }
|
|
64
60
|
```
|
|
65
61
|
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
```json
|
|
69
|
-
{ "command": "LRANGE", "args": ["queue:tasks", "0", "-1"] }
|
|
70
|
-
{ "command": "LPUSH", "args": ["queue:tasks", "task1"] }
|
|
71
|
-
{ "command": "RPOP", "args": ["queue:tasks"] }
|
|
72
|
-
{ "command": "LLEN", "args": ["queue:tasks"] }
|
|
73
|
-
```
|
|
62
|
+
## Authentication
|
|
74
63
|
|
|
75
|
-
|
|
64
|
+
Credentials are embedded directly in the connection URL using `<<systemId_credentialKey>>` placeholders. Nothing is injected automatically.
|
|
76
65
|
|
|
77
|
-
```json
|
|
78
|
-
{ "command": "SMEMBERS", "args": ["tags:post:1"] }
|
|
79
|
-
{ "command": "SADD", "args": ["tags:post:1", "redis", "database"] }
|
|
80
|
-
{ "command": "SISMEMBER", "args": ["tags:post:1", "redis"] }
|
|
81
66
|
```
|
|
82
|
-
|
|
83
|
-
**Sorted set operations:**
|
|
84
|
-
|
|
85
|
-
```json
|
|
86
|
-
{ "command": "ZRANGE", "args": ["leaderboard", "0", "-1", "WITHSCORES"] }
|
|
87
|
-
{ "command": "ZADD", "args": ["leaderboard", "100", "player1"] }
|
|
88
|
-
{ "command": "ZRANK", "args": ["leaderboard", "player1"] }
|
|
67
|
+
redis://<<my_redis_username>>:<<my_redis_password>>@<<my_redis_host>>:<<my_redis_port>>/<<my_redis_database>>
|
|
89
68
|
```
|
|
90
69
|
|
|
91
|
-
|
|
70
|
+
## Redis Runtime Details
|
|
92
71
|
|
|
93
|
-
|
|
94
|
-
{ "command": "KEYS", "args": ["user:*"] }
|
|
95
|
-
{ "command": "EXISTS", "args": ["mykey"] }
|
|
96
|
-
{ "command": "TTL", "args": ["session:abc"] }
|
|
97
|
-
{ "command": "EXPIRE", "args": ["session:abc", "3600"] }
|
|
98
|
-
{ "command": "DEL", "args": ["mykey"] }
|
|
99
|
-
{ "command": "TYPE", "args": ["mykey"] }
|
|
100
|
-
```
|
|
72
|
+
### Return Values
|
|
101
73
|
|
|
102
|
-
|
|
74
|
+
| Command type | Returns |
|
|
75
|
+
| ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |
|
|
76
|
+
| `GET`, `HGET` | string or null |
|
|
77
|
+
| `HGETALL` | flat array of alternating field-value pairs, e.g. `["name", "Alice", "email", "alice@example.com"]`. Empty array for non-existent keys |
|
|
78
|
+
| `LRANGE`, `SMEMBERS`, `KEYS` | array of strings |
|
|
79
|
+
| `SET`, `DEL`, `EXPIRE` | `"OK"` or integer |
|
|
80
|
+
| `INCR`, `LLEN`, `SADD` | integer |
|
|
81
|
+
| `MGET`, `HMGET` | array (nulls for missing keys) |
|
|
82
|
+
| `SCAN`, `HSCAN` | array of `[cursor, results]` |
|
|
103
83
|
|
|
104
|
-
|
|
105
|
-
{ "command": "SCAN", "args": ["0", "MATCH", "user:*", "COUNT", "100"] }
|
|
106
|
-
{ "command": "HSCAN", "args": ["myhash", "0", "MATCH", "field*"] }
|
|
107
|
-
```
|
|
84
|
+
### Pipeline Results
|
|
108
85
|
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
```json
|
|
112
|
-
{ "command": "GET", "args": ["<<(sourceData) => `user:${sourceData.userId}:profile`>>"] }
|
|
113
|
-
```
|
|
114
|
-
|
|
115
|
-
```json
|
|
116
|
-
{ "command": "HSET", "args": ["<<userKey>>", "<<fieldName>>", "<<fieldValue>>"] }
|
|
117
|
-
```
|
|
118
|
-
|
|
119
|
-
## Connection Management
|
|
120
|
-
|
|
121
|
-
- A fresh connection is created per execution and closed after completion
|
|
122
|
-
- TLS auto-configured for `rediss://` URLs (currently uses `rejectUnauthorized: false`)
|
|
123
|
-
- 5-second connection timeout, 30-second command timeout
|
|
124
|
-
- Array commands (pipeline) run on a single connection for efficiency
|
|
125
|
-
|
|
126
|
-
## Retry Logic
|
|
127
|
-
|
|
128
|
-
Default retries from server config. On final failure, error includes command and args.
|
|
129
|
-
|
|
130
|
-
## Return Value
|
|
131
|
-
|
|
132
|
-
Returns the raw Redis response:
|
|
133
|
-
|
|
134
|
-
- `GET` / `HGET`: string or null
|
|
135
|
-
- `HGETALL`: flat array of alternating field-value pairs (e.g., `["name", "Alice", "email", "alice@example.com"]`)
|
|
136
|
-
- `LRANGE` / `SMEMBERS` / `KEYS`: array of strings
|
|
137
|
-
- `SET` / `DEL` / `EXPIRE`: "OK" or integer
|
|
138
|
-
- `INCR` / `LLEN` / `SADD`: integer
|
|
139
|
-
- `MGET` / `HMGET`: array (nulls for missing keys)
|
|
140
|
-
- `SCAN` / `HSCAN`: array of [cursor, results]
|
|
141
|
-
|
|
142
|
-
## Multiple Commands (Pipeline)
|
|
143
|
-
|
|
144
|
-
Pass an array of commands to execute them in a single round-trip (like FTP/SFTP batch operations):
|
|
145
|
-
|
|
146
|
-
```json
|
|
147
|
-
[
|
|
148
|
-
{ "command": "GET", "args": ["user:1:name"] },
|
|
149
|
-
{ "command": "HGETALL", "args": ["user:1"] },
|
|
150
|
-
{ "command": "LRANGE", "args": ["tasks", "0", "-1"] }
|
|
151
|
-
]
|
|
152
|
-
```
|
|
153
|
-
|
|
154
|
-
Returns an array of results, one per command:
|
|
86
|
+
A single command returns a single result. An array of commands returns an array of results:
|
|
155
87
|
|
|
156
88
|
```json
|
|
157
89
|
[
|
|
@@ -161,18 +93,13 @@ Returns an array of results, one per command:
|
|
|
161
93
|
]
|
|
162
94
|
```
|
|
163
95
|
|
|
164
|
-
If an individual command fails, that entry has `error` instead of `result`:
|
|
96
|
+
If an individual command in a pipeline fails, that entry has `error` instead of `result`:
|
|
165
97
|
|
|
166
98
|
```json
|
|
167
|
-
|
|
168
|
-
{ "command": "GET", "result": "Alice" },
|
|
169
|
-
{ "command": "WRONGCMD", "error": "ERR unknown command 'WRONGCMD'" }
|
|
170
|
-
]
|
|
99
|
+
{ "command": "WRONGCMD", "error": "ERR unknown command 'WRONGCMD'" }
|
|
171
100
|
```
|
|
172
101
|
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
## Batch Operations via Loop
|
|
102
|
+
### Batch Operations
|
|
176
103
|
|
|
177
104
|
Use a data selector returning an array to execute a command per item:
|
|
178
105
|
|
|
@@ -181,9 +108,32 @@ Use a data selector returning an array to execute a command per item:
|
|
|
181
108
|
// body: {"command": "HGETALL", "args": ["<<(sourceData) => `user:${sourceData.currentItem}`>>"]}
|
|
182
109
|
```
|
|
183
110
|
|
|
184
|
-
|
|
111
|
+
Prefer array body pipelines over loop-based batch when commands are independent and don't depend on each other's results.
|
|
112
|
+
|
|
113
|
+
### Connection Management
|
|
114
|
+
|
|
115
|
+
- Fresh connection created per execution, closed after completion
|
|
116
|
+
- TLS auto-configured for `rediss://` URLs (`rejectUnauthorized: false`)
|
|
117
|
+
- 5-second connection timeout, 30-second command timeout
|
|
118
|
+
- Pipeline commands run on a single connection
|
|
119
|
+
|
|
120
|
+
### Retry Behavior
|
|
121
|
+
|
|
122
|
+
Default retries from server config. On final failure, error includes command and args.
|
|
123
|
+
|
|
124
|
+
## Common Pitfalls
|
|
125
|
+
|
|
126
|
+
- Using `KEYS` in production on large keyspaces — use `SCAN` with `MATCH` pattern instead
|
|
127
|
+
- Expecting `HGETALL` to return an object — it returns a flat alternating array `[field, value, field, value, ...]`
|
|
128
|
+
- Accessing step result directly instead of via `.data` — Redis results follow the same envelope as all steps: `sourceData.stepId.data`
|
|
129
|
+
- Using separate steps for independent commands that could be pipelined in a single array body
|
|
130
|
+
|
|
131
|
+
## Error Recovery
|
|
132
|
+
|
|
133
|
+
When a Redis step fails:
|
|
185
134
|
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
-
|
|
135
|
+
1. **Read the error message** — superglue includes the command and args in the error. Check for unknown commands, wrong argument counts, or type errors (e.g. running a list command on a string key).
|
|
136
|
+
2. **Verify connection credentials** — use `sg system find` to confirm `storedCredentials` keys match the placeholders in your URL. Test with a simple `PING` command via `sg system call`.
|
|
137
|
+
3. **Check connectivity and access** — connection refused errors typically mean the Redis server is unreachable or requires TLS (`rediss://`). AUTH errors mean the password is wrong or the user lacks access.
|
|
138
|
+
4. **Check key types** — Redis commands are type-specific. Running `GET` on a hash key or `HGETALL` on a string key returns a `WRONGTYPE` error. Use `TYPE` command via `sg system call` to check a key's type before building.
|
|
139
|
+
5. **Verify TLS requirements** — if the error mentions connection reset or handshake failure, try switching between `redis://` and `rediss://`.
|