@superglue/cli 1.1.21 → 1.1.23

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (34) hide show
  1. package/.claude-plugin/plugin.json +1 -1
  2. package/README.md +1 -7
  3. package/dist/cli.js +307 -77
  4. package/dist/skills/superglue/SKILL.md +315 -631
  5. package/dist/skills/superglue/references/file-handling.md +180 -96
  6. package/dist/skills/superglue/references/graphql.md +131 -0
  7. package/dist/skills/superglue/references/http.md +251 -0
  8. package/dist/skills/superglue/references/integration.md +4 -2
  9. package/dist/skills/superglue/references/mssql.md +207 -0
  10. package/dist/skills/superglue/references/odbc.md +162 -0
  11. package/dist/skills/superglue/references/postgres.md +154 -0
  12. package/dist/skills/superglue/references/redis.md +68 -118
  13. package/dist/skills/superglue/references/sftp-smb.md +161 -0
  14. package/dist/skills/superglue/references/superglue-info.md +91 -0
  15. package/package.json +1 -1
  16. package/skills/superglue/SKILL.md +315 -631
  17. package/skills/superglue/references/file-handling.md +180 -96
  18. package/skills/superglue/references/graphql.md +131 -0
  19. package/skills/superglue/references/http.md +251 -0
  20. package/skills/superglue/references/integration.md +4 -2
  21. package/skills/superglue/references/mssql.md +207 -0
  22. package/skills/superglue/references/odbc.md +162 -0
  23. package/skills/superglue/references/postgres.md +154 -0
  24. package/skills/superglue/references/redis.md +68 -118
  25. package/skills/superglue/references/sftp-smb.md +161 -0
  26. package/skills/superglue/references/superglue-info.md +91 -0
  27. package/dist/skills/superglue/references/data-handling.md +0 -303
  28. package/dist/skills/superglue/references/databases.md +0 -253
  29. package/dist/skills/superglue/references/file-servers.md +0 -112
  30. package/dist/skills/superglue/references/http-apis.md +0 -248
  31. package/skills/superglue/references/data-handling.md +0 -303
  32. package/skills/superglue/references/databases.md +0 -253
  33. package/skills/superglue/references/file-servers.md +0 -112
  34. package/skills/superglue/references/http-apis.md +0 -248
@@ -0,0 +1,251 @@
1
+ # HTTP
2
+
3
+ ## Step Configuration
4
+
5
+ ```typescript
6
+ {
7
+ type: "request",
8
+ systemId: "my_api",
9
+ url: "<<my_api_url>>/v1/users",
10
+ method: "GET",
11
+ headers: { "Authorization": "Bearer <<my_api_access_token>>" },
12
+ queryParams: { "limit": "<<limit>>", "status": "active" },
13
+ body: '{"name": "<<(sourceData) => sourceData.currentItem.name>>"}',
14
+ pagination: { ... }
15
+ }
16
+ ```
17
+
18
+ All fields support `<<variable>>` and `<<(sourceData) => ...>>` expressions.
19
+
20
+ ### Fields
21
+
22
+ | Field | Required | Notes |
23
+ | ------------- | -------- | ------------------------------------------------------- |
24
+ | `url` | yes | Prefer `<<systemId_url>>/path` over hardcoded base URLs |
25
+ | `method` | yes | `GET`, `POST`, `PUT`, `DELETE`, `PATCH` |
26
+ | `headers` | no | Auth headers, content type, custom headers |
27
+ | `queryParams` | no | Key-value pairs appended to URL |
28
+ | `body` | no | Stripped for GET/HEAD/DELETE/OPTIONS |
29
+ | `pagination` | no | See Pagination section |
30
+ | `systemId` | no | Links system credentials and URL. Omit for public APIs |
31
+
32
+ ## Authentication
33
+
34
+ Credentials are never injected automatically. You must place auth headers explicitly using `<<systemId_credentialKey>>` syntax.
35
+
36
+ ```javascript
37
+ // Bearer token (including OAuth — token refresh is automatic, header is not)
38
+ { "Authorization": "Bearer <<systemId_access_token>>" }
39
+
40
+ // API key
41
+ { "X-API-Key": "<<systemId_api_key>>" }
42
+
43
+ // Basic Auth — auto-encoded to Base64, do NOT manually encode
44
+ { "Authorization": "Basic <<systemId_username>>:<<systemId_password>>" }
45
+
46
+ // Runtime credentials from payload
47
+ { "Authorization": "Bearer <<(sourceData) => sourceData.user_access_token>>" }
48
+ ```
49
+
50
+ ### SOAP Authentication
51
+
52
+ Some APIs (e.g. Salesforce) use SOAP XML login where credentials are embedded in the request body, not in headers. Multiple `<<>>` placeholders can appear adjacent — each is resolved independently and the results are concatenated.
53
+
54
+ ```typescript
55
+ {
56
+ method: "POST",
57
+ url: "https://test.salesforce.com/services/Soap/u/59.0",
58
+ headers: {
59
+ "Content-Type": "text/xml; charset=UTF-8",
60
+ "SOAPAction": "login"
61
+ },
62
+ body: '<?xml version="1.0" encoding="utf-8"?><env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:env="http://schemas.xmlsoap.org/soap/envelope/"><env:Body><n1:login xmlns:n1="urn:partner.soap.sforce.com"><n1:username><<systemId_username>></n1:username><n1:password><<systemId_password>><<systemId_security_token>></n1:password></n1:login></env:Body></env:Envelope>'
63
+ }
64
+ ```
65
+
66
+ `<<systemId_password>><<systemId_security_token>>` resolves to the password and token concatenated — no separator needed. This works in URL, headers, and body fields.
67
+
68
+ Headers starting with `x-` are treated as custom headers.
69
+ Modern APIs expect auth in headers, not query parameters, unless docs explicitly say otherwise.
70
+
71
+ ## HTTP Runtime Details
72
+
73
+ ### Request Behavior
74
+
75
+ - **GET, HEAD, DELETE, OPTIONS**: body is always stripped
76
+ - **POST, PUT, PATCH**: body with leading `{` is JSON-parsed; empty body becomes undefined
77
+ - Default headers: `Accept: */*`, Chrome-like `User-Agent`
78
+ - HTTPS: `rejectUnauthorized: false` (accepts self-signed certs)
79
+
80
+ ### Body Resolution
81
+
82
+ The runtime processes the body in this order:
83
+
84
+ 1. `<<>>` template variables are resolved — object/array returns are `JSON.stringify`'d before splicing
85
+ 2. The resulting string is JSON-parsed when possible
86
+ 3. `file::` references are resolved on the parsed object
87
+ 4. Final send format is determined:
88
+ - raw bytes when the entire body is `file::<key>.raw`
89
+ - base64 string for `file::<key>.base64`
90
+ - `multipart/form-data` when the `Content-Type` header requests it and body resolves to an object
91
+ - `JSON.stringify(...)` for objects
92
+ - plain string for everything else
93
+
94
+ File references resolve only in the body, not in headers, query params, or URLs.
95
+
96
+ **Double-encoding risk:** the runtime stringifies at both step 1 (expression into template) and step 4 (object into fetch body). When the body contains nested JSON (e.g. LLM APIs), have `<<>>` expressions return a string via `JSON.stringify(...)`, or use a preceding transform step.
97
+
98
+ ### File Uploads
99
+
100
+ **Raw body** — send a file as the entire HTTP body:
101
+
102
+ ```typescript
103
+ {
104
+ method: "PUT",
105
+ headers: { "Content-Type": "application/pdf" },
106
+ body: "file::document.raw"
107
+ }
108
+ ```
109
+
110
+ **Base64-in-JSON** — embed a base64-encoded file inside a structured payload:
111
+
112
+ ```typescript
113
+ {
114
+ method: "POST",
115
+ headers: { "Content-Type": "application/json" },
116
+ body: '{"fileBase64":"file::document.base64"}'
117
+ }
118
+ ```
119
+
120
+ **Multipart form** — set the `Content-Type` header and provide a JSON object of form fields:
121
+
122
+ ```typescript
123
+ {
124
+ method: "POST",
125
+ headers: { "Content-Type": "multipart/form-data" },
126
+ body: '{"file":"file::report.raw","description":"monthly export"}'
127
+ }
128
+ ```
129
+
130
+ The runtime builds a real `FormData` request: `file::...raw` values become file parts, scalars become text parts, non-file objects/arrays are JSON-stringified into text parts. The user-supplied `Content-Type` header is removed so `fetch` sets the boundary automatically.
131
+
132
+ Multipart restrictions:
133
+
134
+ - Body must resolve to a flat object of form fields
135
+ - Nested file refs inside objects are not supported
136
+ - Do not manually set a multipart boundary
137
+ - Use `.raw` for file parts, `.base64` for base64 text fields, `.extracted` for parsed content fields
138
+
139
+ ### Response Handling
140
+
141
+ All HTTP responses are read as raw bytes, then classified by byte-level detection (see `file-handling.md` for the full detection logic).
142
+
143
+ - **Binary types** (PDF, Excel, DOCX, ZIP, GZIP): treated as files, parsed content also populates `data`. Later steps can reference via `file::<stepId>.raw/base64/extracted`.
144
+ - **`application/octet-stream`**: always treated as a file.
145
+ - **Responses over 25 MB**: treated as files as a safety fallback.
146
+ - **RAW + `Content-Disposition: attachment`**: treated as a file.
147
+ - **Structured/text responses** (JSON, XML, CSV, YAML, HTML, text under size limit): parsed inline to `data` only.
148
+
149
+ ### Error Detection
150
+
151
+ Non-2xx responses throw with method, URL, response body preview (1000 chars), masked config, and retry count. Any response with status in the 2xx range is treated as success and returned to the step as-is — the runtime does not scan successful response bodies for error fields, so APIs that return HTTP 200 with `{ "error": "..." }` must be handled by your transform or by setting a `stopCondition`/assertion at the step level.
152
+
153
+ Bypass: set `failureBehavior: "continue"` to skip the non-2xx throw and return the response anyway.
154
+
155
+ ### Retry Behavior
156
+
157
+ **Connection/server errors:**
158
+
159
+ - Default 5 retries, configurable via `retries` on the step
160
+ - Retries on fetch-level errors (network/DNS/abort except timeout) and on 5xx responses
161
+ - Fixed delay between retries (default 1000ms, configurable via `retryDelay`)
162
+
163
+ **Rate limiting (429):**
164
+
165
+ - Counts against the same retry budget as 5xx/network errors
166
+ - Respects `Retry-After` header (seconds or HTTP-date)
167
+ - Without header: exponential backoff `10^n * 1000ms + jitter` (capped at 1hr per wait)
168
+ - Total wait across all 429 retries capped at `MAX_RATE_LIMIT_WAIT_MS` (1hr)
169
+
170
+ **Timeouts:** Request timeout aborts the fetch and throws immediately — timeouts are not retried.
171
+
172
+ ### Pagination
173
+
174
+ Makes multiple requests in a loop, merging results. Only configure after verifying the exact API pagination mechanism from docs.
175
+
176
+ **Configuration:**
177
+
178
+ ```typescript
179
+ pagination: {
180
+ type: "offsetBased" | "pageBased" | "cursorBased",
181
+ pageSize: "50",
182
+ cursorPath: "meta.next_cursor", // cursorBased only — dot-path to cursor
183
+ stopCondition: "(response, pageInfo) => !response.data.meta.next_cursor"
184
+ }
185
+ ```
186
+
187
+ **Variables** — the matching variable MUST appear in the request (URL, queryParams, headers, or body). Throws if missing.
188
+
189
+ | Type | Variable | Starts at | Increments by |
190
+ | ----------- | ---------------------------- | --------- | ------------------------ |
191
+ | pageBased | `<<page>>` | 1 | 1 |
192
+ | offsetBased | `<<offset>>` | 0 | pageSize |
193
+ | cursorBased | `<<cursor>>` | null | extracted via cursorPath |
194
+ | all | `<<limit>>` / `<<pageSize>>` | pageSize | — |
195
+
196
+ **Stop conditions** — JS expression evaluated in sandbox. Receives `(response, pageInfo)`:
197
+
198
+ - `response.data` = parsed API response body
199
+ - `response.headers` = response headers
200
+ - `pageInfo = { page, offset, cursor, totalFetched }`
201
+ - Return `true` to STOP
202
+
203
+ ```javascript
204
+ "!response.data.meta.next_cursor";
205
+ "response.data.items.length === 0";
206
+ "response.data.hasMore === false";
207
+ "pageInfo.totalFetched >= 1000";
208
+ ```
209
+
210
+ **Cursor extraction:** `cursorPath` is a simple dot-separated property path (e.g. `meta.next_cursor`, `data.0.id`) resolved against the parsed response body. It is not a JSONPath expression — wildcards, filters, and `$.` prefixes are not supported. For property names containing `@` (e.g. OData `@odata.nextLink`), use the name directly as the path. Null/undefined cursor stops pagination.
211
+
212
+ **Safety checks (when stopCondition is set):**
213
+
214
+ 1. Identical pages 1 and 2 (both with data): throws — pagination params are not being applied
215
+ 2. Both pages empty + stop did not trigger: throws — broken stop condition
216
+ 3. Duplicate consecutive response after page 2: auto-stops
217
+
218
+ **Without stopCondition (fallback):**
219
+
220
+ - Array shorter than pageSize: stop
221
+ - Duplicate response hash against any previously seen page: stop
222
+ - Non-array response: stop after first request
223
+ - Max 500 requests (vs 1000 max with a stopCondition)
224
+
225
+ **Paginated step results:** all pages merge into a single result. Access via `sourceData.stepId.data` — it is NOT an array of per-page results. Do not call `.map()` on a paginated step result. Returns single object if one result, array if multiple (unwrapped from single-element array).
226
+
227
+ ### File Output
228
+
229
+ When an HTTP step downloads a file (PDF, CSV, binary response), set `outputFile: true` on the step to make it downloadable by the user. Without the flag, downloaded files stay internal (only accessible to subsequent steps via `file::` references).
230
+
231
+ ## Common Pitfalls
232
+
233
+ - Hardcoding base URLs that match the system's base URL instead of using `<<systemId_url>>`
234
+ - Forgetting auth headers entirely — nothing is injected automatically, including for OAuth systems (only token refresh is automatic)
235
+ - Forgetting `outputFile: true` on steps that download files the user wants
236
+ - Putting pagination variables in the config but not setting a `pagination` block, or vice versa
237
+ - Using `<<currentItem.id>>` — must use arrow function syntax: `<<(sourceData) => sourceData.currentItem.id>>`
238
+ - Mixing multiple `<<>>` expressions in one body string when the API expects nested JSON — use a transform step instead
239
+ - Setting `method: "POST"` for GraphQL queries without `modify: false` — POST that only reads should not be marked as modifying
240
+
241
+ ## Error Recovery
242
+
243
+ When an HTTP step fails and the cause is not immediately clear:
244
+
245
+ 1. **Isolate endpoint vs system-level issues** — use `sg system call` to test a different endpoint on the same system. If multiple endpoints fail, the issue is likely global authentication or system configuration. If only one fails, the issue is endpoint-specific (wrong URL, missing params, incorrect method or authentication scopes).
246
+
247
+ 2. **Check credential scopes and system-side configuration** — a 403 or empty response often means the API key or OAuth token lacks required scopes, or the resource is restricted server-side. Use `sg system search-docs` or a web search to look up required scopes/permissions for the endpoint.
248
+
249
+ 3. **Verify request shape against API docs** — use `sg system search-docs` and web search to confirm the endpoint URL, required headers, query parameters, body format, and expected method. APIs change across versions. Search for the error message and the endpoint in docs and online.
250
+
251
+ 4. **Check for rate limiting** — 429 errors are retried automatically with backoff, but persistent 429s may mean the API key has hit a quota. Check API dashboard or docs for rate limit details.
@@ -73,12 +73,14 @@ curl -X POST "$SUPERGLUE_API_ENDPOINT/v1/runs/{runId}/cancel" \
73
73
 
74
74
  ## SDK Installation
75
75
 
76
+ **Always install the latest version** — older SDK versions may have known bugs (e.g. case-sensitivity issues on Linux in versions before 3.3.0).
77
+
76
78
  ```bash
77
79
  # TypeScript/JavaScript
78
- npm install @superglue/client
80
+ npm install @superglue/client@latest
79
81
 
80
82
  # Python
81
- pip install superglue-client
83
+ pip install --upgrade superglue-client
82
84
  ```
83
85
 
84
86
  Both SDKs are auto-generated from the OpenAPI spec (JS via orval, Python via openapi-python-client).
@@ -0,0 +1,207 @@
1
+ # MSSQL
2
+
3
+ ## Step Configuration
4
+
5
+ ```typescript
6
+ {
7
+ type: "request",
8
+ systemId: "my_azure_sql",
9
+ url: "mssql://<<my_azure_sql_user>>:<<my_azure_sql_password>>@<<my_azure_sql_host>>:<<my_azure_sql_port>>/<<my_azure_sql_database>>",
10
+ body: '{"query": "SELECT * FROM users WHERE id = @param1", "params": [<<userId>>]}'
11
+ }
12
+ ```
13
+
14
+ Configure the connection URL based on which credentials are stored in the system — use `sg system find` to check `storedCredentials` and reference them accordingly.
15
+
16
+ `method`, `headers`, `queryParams`, and `pagination` are HTTP-only fields — omit them for database steps.
17
+
18
+ ### Fields
19
+
20
+ | Field | Required | Notes |
21
+ | ---------- | -------- | -------------------------------------------------------------- |
22
+ | `url` | yes | Connection string with `<<credential>>` placeholders |
23
+ | `body` | yes | JSON string with `query` and optional `params` |
24
+ | `systemId` | no | Links system credentials for `<<systemId_credKey>>` resolution |
25
+
26
+ ### URL Format
27
+
28
+ ```
29
+ mssql://user:password@host:port/database
30
+ sqlserver://user:password@host:port/database
31
+ ```
32
+
33
+ Both `mssql://` and `sqlserver://` are supported.
34
+
35
+ **Azure SQL Database** — use the fully qualified server name:
36
+
37
+ ```
38
+ mssql://myuser:mypassword@myserver.database.windows.net:1433/mydatabase
39
+ ```
40
+
41
+ Azure SQL usernames often include `@servername` suffix (e.g., `myuser@myserver`). Encode the `@` as `%40` in the URL:
42
+
43
+ ```
44
+ mssql://myuser%40myserver:mypassword@myserver.database.windows.net:1433/mydatabase
45
+ ```
46
+
47
+ **Connection parameters** can be appended as query strings:
48
+
49
+ ```
50
+ mssql://user:password@host:port/database?encrypt=true&trustServerCertificate=false
51
+ ```
52
+
53
+ | Parameter | Default | Notes |
54
+ | ------------------------ | ------- | --------------------------------------- |
55
+ | `encrypt` | `true` | Required for Azure SQL |
56
+ | `trustServerCertificate` | `false` | Whether to trust the server certificate |
57
+ | `database` | — | Can be in URL path or query parameter |
58
+
59
+ ### Body Format
60
+
61
+ JSON string with `query` and optional `params` (or `values`):
62
+
63
+ ```json
64
+ {
65
+ "query": "SELECT * FROM users WHERE age > @param1 AND status = @param2",
66
+ "params": [25, "active"]
67
+ }
68
+ ```
69
+
70
+ ## Authentication
71
+
72
+ ### SQL Server Authentication (default)
73
+
74
+ Credentials are embedded directly in the connection URL using `<<systemId_credentialKey>>` placeholders. Nothing is injected automatically.
75
+
76
+ ```
77
+ mssql://<<my_db_user>>:<<my_db_password>>@<<my_db_host>>:<<my_db_port>>/<<my_db_database>>
78
+ ```
79
+
80
+ ### Azure Active Directory Authentication
81
+
82
+ For Azure AD auth, store the following credential keys on the system alongside the standard connection fields:
83
+
84
+ **Entra ID User + Password** — authenticates as an Azure AD user. Requires an Azure app registration.
85
+
86
+ | Credential Key | Required | Notes |
87
+ | -------------- | -------- | ------------------------------------------------------ |
88
+ | `authType` | yes | `azure-active-directory-password` |
89
+ | `clientId` | yes | Application (client) ID from Azure app registration |
90
+ | `tenantId` | no | Directory (tenant) ID |
91
+ | `user` | yes | Azure AD username (also accepts `username`, or in URL) |
92
+ | `password` | yes | Azure AD password (or in URL) |
93
+
94
+ `user` and `password` can be stored as credentials or embedded in the URL — credentials take precedence.
95
+
96
+ ```
97
+ URL: mssql://<<sys_user>>:<<sys_password>>@myserver.database.windows.net:1433/mydb
98
+ Credentials: { authType, clientId, tenantId, user, password, host, port, database }
99
+ ```
100
+
101
+ **Service Principal** — authenticates as an Azure AD application. No user/password needed in the URL.
102
+
103
+ | Credential Key | Required | Notes |
104
+ | -------------- | -------- | ------------------------------------------------- |
105
+ | `authType` | yes | `azure-active-directory-service-principal-secret` |
106
+ | `clientId` | yes | Application (client) ID |
107
+ | `clientSecret` | yes | Client secret |
108
+ | `tenantId` | yes | Directory (tenant) ID |
109
+
110
+ ```
111
+ URL: mssql://myserver.database.windows.net:1433/mydb
112
+ Credentials: { authType, clientId, clientSecret, tenantId, host, port, database }
113
+ ```
114
+
115
+ The runtime authenticates entirely via the service principal credentials — no user or password is needed in the URL.
116
+
117
+ ## MSSQL Runtime Details
118
+
119
+ ### Schema Discovery
120
+
121
+ superglue does not auto-introspect MSSQL schemas. Before building tools, always query the schema manually with `sg system call` to discover tables and columns:
122
+
123
+ ```json
124
+ {
125
+ "query": "SELECT TABLE_NAME, COLUMN_NAME, DATA_TYPE, IS_NULLABLE FROM INFORMATION_SCHEMA.COLUMNS WHERE TABLE_SCHEMA = 'dbo' ORDER BY TABLE_NAME, ORDINAL_POSITION"
126
+ }
127
+ ```
128
+
129
+ For Azure SQL or databases with non-default schemas, adjust the `TABLE_SCHEMA` filter or remove it to see all schemas.
130
+ Always verify table and column names against the actual schema before building tools.
131
+
132
+ ### Parameterized Queries
133
+
134
+ Always use parameterized queries with `@param1`, `@param2`, etc. This prevents SQL injection. The params array maps positionally: `params[0]` maps to `@param1`, `params[1]` maps to `@param2`, etc.
135
+
136
+ ```json
137
+ { "query": "SELECT * FROM orders WHERE customer_id = @param1", "params": ["<<customerId>>"] }
138
+ ```
139
+
140
+ With expressions:
141
+
142
+ ```json
143
+ {
144
+ "query": "INSERT INTO logs (message, level) VALUES (@param1, @param2)",
145
+ "params": ["<<(sourceData) => sourceData.message>>", "error"]
146
+ }
147
+ ```
148
+
149
+ With OUTPUT (MSSQL equivalent of RETURNING):
150
+
151
+ ```json
152
+ {
153
+ "query": "INSERT INTO users (name, email) OUTPUT INSERTED.* VALUES (@param1, @param2)",
154
+ "params": [
155
+ "<<(sourceData) => sourceData.currentItem.name>>",
156
+ "<<(sourceData) => sourceData.currentItem.email>>"
157
+ ]
158
+ }
159
+ ```
160
+
161
+ ### Return Value
162
+
163
+ Returns the recordset — array of row objects with column names as keys.
164
+
165
+ - `SELECT`: array of matching rows
166
+ - `INSERT/UPDATE/DELETE` with `OUTPUT`: array of returned rows
167
+ - Mutations without `OUTPUT`: empty array
168
+
169
+ ### Batch Operations
170
+
171
+ Use a data selector returning an array to execute a query per item:
172
+
173
+ ```javascript
174
+ // dataSelector: (sourceData) => sourceData.newUsers
175
+ // body: {"query": "INSERT INTO users (name) VALUES (@param1)", "params": ["<<(sourceData) => sourceData.currentItem.name>>"]}
176
+ ```
177
+
178
+ ### Connection Pooling
179
+
180
+ - Pools cached by connection string, max 10 clients per pool
181
+ - 30s connection timeout, configurable request timeout
182
+ - Idle pools cleaned up automatically
183
+ - SSL/TLS encryption enabled by default for Azure SQL
184
+
185
+ ### Retry Behavior
186
+
187
+ Default retries from server config. On final failure, error includes query text and params.
188
+
189
+ ## Common Pitfalls
190
+
191
+ - Using `$1`, `$2` (PostgreSQL syntax) instead of `@param1`, `@param2` — MSSQL uses named `@param` parameters
192
+ - Forgetting `OUTPUT INSERTED.*` on INSERT/UPDATE/DELETE when you need the result — without it the step returns an empty array
193
+ - Accessing step result directly instead of via `.data` — database results follow the same envelope as all steps: `sourceData.stepId.data`
194
+ - Not URL-encoding `@` in Azure SQL usernames — `myuser@myserver` must be `myuser%40myserver` in the connection URL
195
+
196
+ ## Error Recovery
197
+
198
+ When an MSSQL step fails and the cause is not obvious:
199
+
200
+ 1. **Read the error message** — superglue includes the query text and params in the error. Check for syntax errors, missing columns, or type mismatches.
201
+ 2. **Verify connection credentials** — use `sg system find` to confirm `storedCredentials` keys match the placeholders in your URL. Test connectivity with a simple `SELECT 1` via `sg system call`.
202
+ 3. **Check database-side access** — connection refused or permission denied errors typically mean the database user lacks access. Verify with the user that the credentials have the required grants.
203
+ 4. **Check Azure-specific issues** — if connecting to Azure SQL:
204
+ - Confirm the client IP is allowed in the Azure SQL firewall rules
205
+ - Verify the username includes the `@servername` suffix if required, properly encoded as `%40`
206
+ - Confirm `encrypt=true` is set (required for Azure SQL)
207
+ 5. **Verify parameter types** — MSSQL infers types from the values passed. Passing a string where a number is expected may cause implicit conversion errors. Ensure `params` values match expected column types.
@@ -0,0 +1,162 @@
1
+ # ODBC
2
+
3
+ ## Step Configuration
4
+
5
+ ```typescript
6
+ {
7
+ type: "request",
8
+ systemId: "my_sybase_db",
9
+ url: "odbc://<<my_sybase_db_username>>:<<my_sybase_db_password>>@<<my_sybase_db_host>>:<<my_sybase_db_port>>/<<my_sybase_db_database>>?DRIVER=<<my_sybase_db_driver>>",
10
+ body: '{"query": "SELECT * FROM users WHERE id = ?", "params": [<<userId>>]}'
11
+ }
12
+ ```
13
+
14
+ Configure the connection URL based on which credentials are stored in the system — use `sg system find` to check `storedCredentials` and reference them accordingly.
15
+
16
+ `method`, `headers`, `queryParams`, and `pagination` are HTTP-only fields — omit them for database steps.
17
+
18
+ ### Fields
19
+
20
+ | Field | Required | Notes |
21
+ | ---------- | -------- | -------------------------------------------------------------- |
22
+ | `url` | yes | Connection string with `<<credential>>` placeholders |
23
+ | `body` | yes | JSON string with `query` and optional `params` |
24
+ | `systemId` | no | Links system credentials for `<<systemId_credKey>>` resolution |
25
+
26
+ ### URL Format
27
+
28
+ ```
29
+ odbc://user:password@host:port/database?DRIVER=DriverName
30
+ odbc://user:password@host:port/database?DRIVER=FreeTDS&TDS_Version=5.0
31
+ ```
32
+
33
+ The path contains the **database name**. The ODBC driver is specified via the `DRIVER` query parameter (defaults to `FreeTDS` if omitted). This format is consistent with other superglue database strategies (postgres, mssql) where the path is always the database.
34
+
35
+ **Supported drivers:**
36
+
37
+ | Driver | Database | Default Port |
38
+ | --------- | ---------------------------- | ------------ |
39
+ | `FreeTDS` | SAP ASE (Sybase), SQL Server | 5000 |
40
+
41
+ Additional query parameters (beyond `DRIVER`) are passed through to the ODBC connection string as key=value pairs.
42
+
43
+ ### Body Format
44
+
45
+ JSON string with `query` and optional `params` (or `values`):
46
+
47
+ ```json
48
+ {
49
+ "query": "SELECT * FROM users WHERE age > ? AND status = ?",
50
+ "params": [25, "active"]
51
+ }
52
+ ```
53
+
54
+ ## Authentication
55
+
56
+ Credentials are embedded directly in the connection URL using `<<systemId_credentialKey>>` placeholders. Nothing is injected automatically.
57
+
58
+ ```
59
+ odbc://<<my_db_username>>:<<my_db_password>>@<<my_db_host>>:<<my_db_port>>/<<my_db_database>>?DRIVER=<<my_db_driver>>
60
+ ```
61
+
62
+ Credential variables are resolved at runtime before connecting.
63
+
64
+ ## ODBC Runtime Details
65
+
66
+ ### Schema Discovery
67
+
68
+ superglue does not auto-introspect ODBC database schemas. Before building tools, always query the schema manually with `sg system call` to discover tables and columns.
69
+
70
+ **SAP ASE (Sybase):**
71
+
72
+ ```json
73
+ {
74
+ "query": "SELECT o.name AS table_name, c.name AS column_name, t.name AS data_type, CASE WHEN c.status & 8 = 8 THEN 'YES' ELSE 'NO' END AS is_nullable FROM sysobjects o JOIN syscolumns c ON o.id = c.id JOIN systypes t ON c.usertype = t.usertype WHERE o.type = 'U' ORDER BY o.name, c.colid"
75
+ }
76
+ ```
77
+
78
+ Always verify table and column names against the actual schema before building tools.
79
+
80
+ ### Parameterized Queries
81
+
82
+ Always use parameterized queries with positional `?` placeholders — prevents SQL injection. Parameters are bound positionally from the `params` array.
83
+
84
+ For numeric or boolean parameters in query params arrays, use unquoted placeholders: `[<<numericParam>>]` not `["<<numericParam>>"]`. The quotes force string resolution.
85
+
86
+ ```json
87
+ { "query": "SELECT * FROM orders WHERE name = ? AND customer_id = ?", "params": ["<<customerName>>", <<customerId>>] }
88
+ ```
89
+
90
+ With expressions:
91
+
92
+ ```json
93
+ {
94
+ "query": "INSERT INTO logs (message, level) VALUES (?, ?)",
95
+ "params": ["<<(sourceData) => sourceData.message>>", "error"]
96
+ }
97
+ ```
98
+
99
+ ### Return Value
100
+
101
+ Returns an array of row objects with column names as keys.
102
+
103
+ - `SELECT`: array of matching rows
104
+ - `INSERT/UPDATE/DELETE`: empty array (SAP ASE does not support `RETURNING` — query the data separately if needed)
105
+
106
+ ### Batch Operations
107
+
108
+ Use a data selector returning an array to execute a query per item:
109
+
110
+ ```javascript
111
+ // dataSelector: (sourceData) => sourceData.newUsers
112
+ // body: {"query": "INSERT INTO users (name) VALUES (?)", "params": ["<<(sourceData) => sourceData.currentItem.name>>"]}
113
+ ```
114
+
115
+ ### Retry Behavior
116
+
117
+ Default retries from server config. On final failure, error includes a truncated query preview.
118
+
119
+ ## SAP ASE (Sybase) Specifics
120
+
121
+ ### Differences from SQL Server
122
+
123
+ SAP ASE uses Transact-SQL like SQL Server, but with key differences:
124
+
125
+ - **No `OUTPUT` clause** — use `SELECT @@identity` after INSERT to get the last inserted identity value
126
+ - **No `TOP` with `ORDER BY` guarantee** — use `SET ROWCOUNT n` before the query instead
127
+ - **`CONVERT` syntax differs** — `CONVERT(target_type, expression)` not `CONVERT(expression, style)`
128
+ - **String concatenation** — use `+` operator, not `CONCAT()` (unless ASE 16.0+)
129
+ - **Date functions** — use `GETDATE()` not `CURRENT_TIMESTAMP`, `DATEADD`/`DATEDIFF` syntax matches SQL Server
130
+
131
+ ### TDS Protocol Version
132
+
133
+ FreeTDS defaults to TDS 5.0 for Sybase connections. If you encounter protocol errors, you can specify the version via query parameter:
134
+
135
+ ```
136
+ odbc://user:pass@host:5000/mydb?DRIVER=FreeTDS&TDS_Version=5.0
137
+ ```
138
+
139
+ | TDS Version | Use Case |
140
+ | ----------- | -------------------------- |
141
+ | `5.0` | SAP ASE / Sybase (default) |
142
+ | `7.0` | SQL Server 7.0+ |
143
+ | `7.4` | SQL Server 2012+ |
144
+
145
+ ## Common Pitfalls
146
+
147
+ - Using `@param1`, `@param2` (MSSQL syntax) or `$1`, `$2` (PostgreSQL syntax) instead of positional `?` placeholders — ODBC uses `?` for all parameter binding
148
+ - Forgetting that SAP ASE has no `OUTPUT`/`RETURNING` clause — mutations return empty arrays; query the data separately
149
+ - Using `CONCAT()` on older ASE versions — use `+` for string concatenation instead
150
+ - Driver name mismatch — the `DRIVER` query parameter must exactly match the section name in `odbcinst.ini` (case-sensitive). Use `DRIVER=FreeTDS`, not `DRIVER=freetds` or `DRIVER=FREETDS`
151
+ - Accessing step result directly instead of via `.data` — database results follow the same envelope as all steps: `sourceData.stepId.data`
152
+
153
+ ## Error Recovery
154
+
155
+ When an ODBC step fails and the cause is not obvious:
156
+
157
+ 1. **Read the error message** — superglue includes the a truncated query preview in the error. Check for syntax errors, missing columns, or type mismatches.
158
+ 2. **Verify connection credentials** — use `sg system find` to confirm `storedCredentials` keys match the placeholders in your URL. Test connectivity with a simple `SELECT 1` via `sg system call`.
159
+ 3. **Check database-side access** — connection refused or permission denied errors typically mean the database user lacks access. Verify with the user that the credentials have the required grants.
160
+ 4. **Check driver availability** — if the error mentions "driver not found", the `DRIVER` query parameter does not match any registered driver in `/etc/odbcinst.ini`. Verify the driver name is correct and the driver is installed in the Docker image.
161
+ 5. **Test the query directly** — use `sg system call` with a simplified version of the query to isolate whether the issue is the query itself, the parameters, or the connection.
162
+ 6. **Check TDS version** — protocol errors may indicate a TDS version mismatch. Try adding `?TDS_Version=5.0` to the query parameters for SAP ASE connections.