@superglue/cli 1.1.21 → 1.1.23

package/skills/superglue/references/file-handling.md

@@ -1,160 +1,244 @@
 # File Handling
 
-How the runtime detects, parses, and exposes files across all protocol strategies (HTTP, FTP, SFTP, SMB).
+How the runtime detects, parses, and exposes files across all protocol strategies (HTTP, SFTP/FTP, SMB).
 
-## When Does a Response Become a File?
+## Step Configuration
 
-Every response is read as raw bytes. The runtime decides whether it routes to `producedFiles` (i.e. `sourceData.__files__`) or inline `data` using a priority chain — **not** based on size alone:
+File handling is not a standalone step type — it is a cross-cutting runtime behavior that applies to any step that produces or consumes files. The `file::<alias>.<suffix>` syntax is used in step config `body` / `content` fields to reference files from previous steps or user uploads.
 
-1. **Magic-byte detection** (primary) — the raw bytes are inspected for known binary signatures (ZIP `PK\x03\x04`, PDF `%PDF`, GZIP `\x1f\x8b`, plus internal ZIP structure checks for Excel/DOCX). If matched, the response is a file regardless of size or HTTP headers. A 1KB ZIP and a 500MB ZIP both go to `__files__`.
-2. **`Content-Disposition: attachment`** — any response with `attachment` or `filename=` in this header becomes a file, even if byte detection found structured text.
-3. **`application/octet-stream`** — explicit octet-stream MIME becomes a file, catching binary payloads whose bytes don't match a known signature.
-4. **25MB size fallback** (last resort) — only applies when byte detection returned `RAW` (unrecognized format), the response is larger than 25MB, **and** the `Content-Type` is not clearly text-like (`text/*`, `application/json`, `*+json`, `application/xml`, etc.). This is the only path where size matters.
+```typescript
+// Reference a file downloaded by a previous step
+{ "body": "file::downloadStep.raw" }
 
-Structured text responses (JSON, XML, CSV, YAML, HTML) that pass none of the above checks go to `data` only — no file entry.
+// Reference a user-uploaded file
+{ "body": '{"fileBase64": "file::document.base64"}' }
 
-| Classification      | Types                       | Where result lives                                             |
-| ------------------- | --------------------------- | -------------------------------------------------------------- |
-| **Binary**          | PDF, Excel, DOCX, ZIP, GZIP | `producedFiles` + `data` (extracted/parsed content)            |
-| **Structured text** | JSON, XML, CSV, YAML, HTML  | `data` only (parsed to JS objects)                             |
-| **RAW**             | Unrecognized                | File if attachment/octet-stream/25MB fallback, else raw string |
+// Reference a specific file from a multi-file step
+{ "content": 'file::batchDownload["report.csv"].raw' }
+```
 
-Binary responses expose `stepFileKeys` and populate `sourceData.__files__`. Structured text responses only populate `sourceData.stepId.data`.
+### file:: Reference Syntax
 
-File server `get` operations always produce files regardless of detected type — content goes into both `data` and `producedFiles`.
+Use `file::<alias>.<suffix>` in step config `body` / `content` fields:
 
-## Auto-Parsing
+| Suffix       | Returns                     | Use case                                |
+| ------------ | --------------------------- | --------------------------------------- |
+| `.raw`       | Original bytes (Uint8Array) | SFTP/FTP put, HTTP raw body upload      |
+| `.base64`    | Base64-encoded string       | APIs expecting base64 inside JSON/XML   |
+| `.extracted` | Parsed content              | Text fields needing parsed file content |
 
-| Type  | Parsed result                                                 |
-| ----- | ------------------------------------------------------------- |
-| JSON  | Native JS object/array                                        |
-| CSV   | Array of objects (header row = keys)                          |
-| XML   | JS object tree                                                |
-| HTML  | JS object tree                                                |
-| YAML  | Native JS object                                              |
-| Excel | Object keyed by sheet name → array of row objects (see below) |
-| PDF   | `{ textContent, structuredContent: [{ page, text }] }`        |
-| DOCX  | Markdown string                                               |
-| ZIP   | Object keyed by entry filename, each value recursively parsed |
-| GZIP  | Recursively parsed inner content                              |
-| RAW   | UTF-8 string fallback                                         |
+Constraints:
 
-Detection is heuristic for text formats. YAML requires `---`, `%YAML`, or multiple `key: value` lines to be recognized. Short or ambiguous YAML can fall through to RAW and appear as a plain string instead of a parsed object — both in `.data` and `.extracted`.
+- Bare `file::<key>` without a suffix is invalid and causes a runtime error
+- File references resolve only in body/content fields, not in headers, query params, or URLs
+- Multi-file steps use bracket notation: `file::stepId["report.csv"].raw`
+- Loop iterations use numeric index: `file::stepId[0].raw`
 
-### Excel rules
+## Authentication
 
-- Excel is NEVER a flat array. It is always `{ "SheetName": [ {row}, ... ] }` — even for single-sheet files.
-- Access rows via `extracted.SheetName` or `extracted["Sheet1"]`, never `extracted[0]`.
-- When producing xlsx, use namespace `http://schemas.openxmlformats.org/spreadsheetml/2006/main` (not the truncated path).
+N/A — file handling is a runtime behavior, not a protocol. Authentication is handled by the protocol skill that produces or consumes the file (http, sftp-smb, etc.).
 
-## file:: Reference Syntax
+## File Handling Runtime Details
 
-Use `file::<alias>.<suffix>` in step config `body` / `content` fields:
+### Detection Priority
 
-| Suffix       | Returns                     | Use case                                |
-| ------------ | --------------------------- | --------------------------------------- |
-| `.raw`       | Original bytes (Uint8Array) | SFTP/FTP put, HTTP raw body upload      |
-| `.base64`    | Base64-encoded string       | APIs expecting base64 inside JSON/XML   |
-| `.extracted` | Parsed content              | Text fields needing parsed file content |
+Every response is read as raw bytes. The runtime classifies it using this priority chain:
+
+1. **Magic-byte detection** — known binary signatures (ZIP `PK\x03\x04`, PDF `%PDF`, GZIP `\x1f\x8b`, plus internal ZIP structure checks for Excel/DOCX)
+2. **`Content-Disposition: attachment`** — `attachment` or `filename=` in this header forces file treatment
+3. **`application/octet-stream`** — explicit octet-stream MIME catches unlabeled binary payloads
+4. **25MB size fallback** — only when byte detection returned `RAW`, the response exceeds 25MB, and `Content-Type` is not text-like (`text/*`, `application/json`, `*+json`, `application/xml`)
 
-Rules:
+File server `get` operations always produce files regardless of detected type.
 
-- Bare `file::<key>` without a suffix is **NOT valid** — causes a runtime error
-- File references are resolved **only in body/content fields**, not in headers, query params, or URLs
-- For multi-file steps, use bracket notation: `file::stepId["report.csv"].raw`
-- For loop iterations: `file::stepId[0].raw`
-- Important: base64 access in transforms is capped at 500MB by design, so base64 encoded binary in transform steps is only available for files smaller than this
+### Classification
+
+| Classification      | Types                       | Destination                                                    |
+| ------------------- | --------------------------- | -------------------------------------------------------------- |
+| **Binary**          | PDF, Excel, DOCX, ZIP, GZIP | `producedFiles` + `data` (extracted/parsed content)            |
+| **Structured text** | JSON, XML, CSV, YAML, HTML  | `data` only (parsed to JS objects)                             |
+| **RAW**             | Unrecognized                | File if attachment/octet-stream/25MB fallback, else raw string |
 
-## File Aliasing
+Binary responses expose `stepFileKeys` and populate `sourceData.__files__`. Structured text responses only populate `sourceData.stepId.data`.
 
-When a step produces files, the runtime first chooses a **root alias**:
+### Auto-Parsing
 
-- non-loop step: `stepId`
-- loop iteration: `stepId[N]`
+| Type  | Parsed result                                                  |
+| ----- | -------------------------------------------------------------- |
+| JSON  | Native JS object/array                                         |
+| CSV   | Array of objects (header row = keys)                           |
+| XML   | JS object tree                                                 |
+| HTML  | JS object tree                                                 |
+| YAML  | Native JS object                                               |
+| Excel | Object keyed by sheet name, each value is array of row objects |
+| PDF   | `{ textContent, structuredContent: [{ page, text }] }`         |
+| DOCX  | Markdown string                                                |
+| ZIP   | Object keyed by entry filename, each value recursively parsed  |
+| GZIP  | Recursively parsed inner content                               |
+| RAW   | UTF-8 string fallback                                          |
 
-Then it decides whether that root needs a filename suffix:
+**Excel rules:**
 
-- if that step/iteration produced **exactly one file**, the alias is just the root
-- if that step/iteration produced **multiple files**, append `["filename.ext"]` to distinguish them
+- Excel is never a flat array. It is always `{ "SheetName": [ {row}, ... ] }` — even for single-sheet files
+- Access rows via `extracted.SheetName` or `extracted["Sheet1"]`, never `extracted[0]`
+- When producing xlsx, use namespace `http://schemas.openxmlformats.org/spreadsheetml/2006/main` (not the truncated path)
 
-Examples:
+### File Aliasing
 
-- one direct download: `downloadReport`
-- one loop iteration download: `downloadReport[0]`
-- one batch step returning multiple files: `downloadReport["summary.csv"]`
-- one loop iteration returning multiple files: `downloadReport[0]["summary.csv"]`
+When a step produces files, the runtime assigns an alias:
 
-This is about **one file vs many files at that step/iteration**, not about whether the runtime knows the filename. File server `get` operations know the filename, but single-file outputs still use the plain step ID (or `stepId[N]`) alias.
+| Scenario                    | Alias format                |
+| --------------------------- | --------------------------- |
+| Single file, non-loop       | `stepId`                    |
+| Single file, loop iteration | `stepId[N]`                 |
+| Multiple files, non-loop    | `stepId["filename.ext"]`    |
+| Multiple files, loop        | `stepId[N]["filename.ext"]` |
 
-Loop steps (those with a `dataSelector` that iterates) always use numeric indices — never filenames — even when each iteration fetches a named file. Filename brackets only appear when a single step or iteration returns multiple files in one response.
+Loop steps always use numeric indices — never filenames — even when each iteration fetches a named file. Filename brackets only appear when a single step or iteration returns multiple files in one response.
 
 These aliases are used both in `file::` references and as keys in `sourceData.__files__`.
 
-## Accessing Files in Transforms
+### stepFileKeys
 
-Inside `transformCode` and `outputTransform`, use `sourceData.__files__` with the **runtime alias**:
+Each step result exposes `stepFileKeys` — an array of runtime aliases for files produced by that step:
 
 ```javascript
-sourceData.__files__.downloadStep; // correct (step ID for single file)
-sourceData.__files__["doubled.csv"]; // wrong (local filename)
+sourceData.downloadStep.stepFileKeys; // ["downloadStep"]
+sourceData.batchDownload[0].stepFileKeys; // ["batchDownload[0]"]
 ```
 
-For multi-file aliases with bracket notation, prefer discovering aliases from `stepFileKeys` or
-`Object.keys(sourceData.__files__)` instead of hardcoding quoted alias strings inside the transform
-code string. This avoids JSON/JS escaping mistakes like:
+### Accessing Files in Transforms
+
+Inside `transformCode` and `outputTransform`, use `sourceData.__files__` with the runtime alias. Treat `sourceData.__files__` as read-only — the runtime rebuilds it after every step.
 
 ```javascript
-sourceData.__files__['fetchThreeFiles[".bashrc"]']; // brittle inside JSON-encoded transform strings
+sourceData.__files__.downloadStep; // single-file step
+sourceData.__files__["stepId[0]"]; // loop iteration
 ```
 
-Prefer:
+Available properties on each file object: `filename`, `contentType`, `size`, `raw`, `extracted`, `fileType`, `parseError`.
+
+**base64 access:** `base64` is a lazy getter — it is not stored on the file object by default. Access it directly:
+
+```javascript
+sourceData.__files__.myStep.base64; // computes and returns base64 string on first access
+```
+
+The base64 value is computed on first access and cached for subsequent reads. It will not appear in `Object.keys()` or `JSON.stringify()` output (non-enumerable). Throws if the file exceeds 500MB — use `.raw` for large files.
+
+Prefer discovering aliases dynamically via `stepFileKeys`:
 
 ```javascript
 (sourceData) => {
   return (sourceData.fetchThreeFiles.stepFileKeys || []).map(function (alias) {
     var f = sourceData.__files__[alias];
-    return {
-      alias: alias,
-      filename: f ? f.filename : null,
-      size: f ? f.size : null,
-    };
+    return { alias: alias, filename: f ? f.filename : null, size: f ? f.size : null };
   });
 };
 ```
 
-`sourceData.__files__` is **read-only**. To produce new files from a transform, return them via a `__files__` key:
+### Producing Files From Transforms
+
+Return a `__files__` key from a transform to produce new files. The **key name is the filename** — content type is inferred from the extension. Values can be plain strings, bytes, or full file objects:
 
 ```javascript
+// Simplest: key = filename, value = string content (auto-encoded to bytes)
 (sourceData) => ({
-  summary: "processed",
   __files__: {
-    "output.csv": {
-      filename: "output.csv",
-      contentType: "text/csv",
-      raw: new TextEncoder().encode("a,b\n1,2\n"),
-    },
+    "report.csv": Papa.unparse(sourceData.step1.data),
   },
-});
+})
+
+// Multiple files with different types
+(sourceData) => ({
+  __files__: {
+    "data.csv": Papa.unparse(sourceData.step1.data),
+    "config.yaml": yaml.dump(sourceData.step1.data),
+  },
+  summary: { rowCount: sourceData.step1.data.length },
+})
 ```
 
-Everything except `__files__` becomes the step's `data`. Single-file transforms get the step ID as alias; multi-file transforms use bracket notation.
+Supported value types for `__files__` entries:
 
-### `raw` rules
+- **String** — auto-encoded to UTF-8 bytes. Filename and content type from the key.
+- **Uint8Array / ArrayBuffer** — used as-is. Filename and content type from the key.
+- **Object with `raw`** — `raw` is normalized (string/bytes/array), `filename` defaults to key if missing, `contentType` inferred from key extension if missing.
+- **Full file object** — `{ filename, contentType, raw }` for explicit control (backwards compatible).
 
-- `raw` accepts: `Uint8Array | ArrayBuffer | number[] | string | object | null`.
-  - **String** → UTF-8 encoded. Pass a JSON string for structured data: `raw: JSON.stringify(data)`.
-  - **Object / array of objects** → automatically JSON-stringified to UTF-8 bytes. Works, but prefer explicit `JSON.stringify` with a `text/csv` or `application/json` contentType for clarity.
-  - **`null` / `undefined`** → produces an empty file (0 bytes).
-  - **`Uint8Array` / `ArrayBuffer` / `number[]`** → used as-is for binary content.
-- For binary formats (xlsx, zip, pdf): pass through original bytes via `sourceData.__files__.stepId.raw` — never construct the internal file structure as a JS object.
+Everything except `__files__` becomes the step's `data`. Single-file transforms get the step ID as alias; multi-file transforms use bracket notation.
 
-## stepFileKeys
+**Available libraries in transforms:** The following libraries are available for file generation in any transform code (step transforms and output transforms):
 
-Each step result exposes `stepFileKeys` — an array of runtime aliases for files produced by that step:
+- `Papa` — [papaparse](https://www.papaparse.com/). Use `Papa.unparse(rows)` to generate CSV from an array of objects. Handles quoting, escaping, and special characters automatically.
+- `XLSX` — [SheetJS](https://sheetjs.com/). Use `XLSX.utils.json_to_sheet(data)` + `XLSX.write(wb, { type: "array", bookType: "xlsx" })` to generate Excel files.
+- `yaml` — [js-yaml](https://github.com/nodeca/js-yaml). Use `yaml.dump(obj)` to generate YAML.
+
+Example — CSV with Papa:
 
 ```javascript
-sourceData.downloadStep.stepFileKeys; // ["downloadStep"]
-sourceData.batchDownload[0].stepFileKeys; // ["batchDownload[0]"]
+(sourceData) => ({
+  __files__: { "report.csv": Papa.unparse(sourceData.step1.data) },
+});
+```
+
+**Auto-parsing of transform-produced files:** if `extracted` is omitted from the file object, the runtime runs the full detection and parsing pipeline on the raw bytes (same as for HTTP/file server responses). This means a CSV file produced with `raw: "a,b\n1,2"` will automatically get `extracted: [{a: "1", b: "2"}]`. To skip auto-parsing and keep the raw content as-is, set `extracted` explicitly (e.g. `extracted: null`).
+
+**`raw` field rules:**
+
+- `Uint8Array` / `ArrayBuffer` / typed arrays — used as-is for binary content
+- String — UTF-8 encoded. Pass a JSON string for structured data: `raw: JSON.stringify(data)`
+- Object / array — auto JSON-stringified to UTF-8 bytes. Prefer explicit `JSON.stringify` with an appropriate `contentType` for clarity
+- `null` / `undefined` — produces an empty file (0 bytes)
+- For binary formats (xlsx, zip, pdf): pass through original bytes via `sourceData.__files__.stepId.raw` — never construct the internal file structure as a JS object
+
+### Returning Files as Tool Output
+
+To make files downloadable from the tool's API response, set `outputFile: true` on the step that produces the file. The output transform is only for shaping JSON data — it has nothing to do with files.
+
+```json
+{
+  "steps": [
+    {
+      "id": "fetchData",
+      "config": { "type": "request", "url": "https://api.example.com/report" }
+    },
+    {
+      "id": "generateReport",
+      "config": {
+        "type": "transform",
+        "transformCode": "(sourceData) => ({ __files__: { \"report.csv\": Papa.unparse(sourceData.fetchData.data) } })"
+      },
+      "outputFile": true
+    }
+  ],
+  "outputTransform": "(sourceData) => ({ rowCount: sourceData.fetchData.data.length })"
+}
 ```
 
-Use `stepFileKeys` to dynamically discover which file aliases are available from a step.
+Only steps with `outputFile: true` have their files stored to S3 and returned as downloadable artifacts. Steps without the flag keep their files internal (for inter-step passing only).
+
+This works on any step type — transform steps, HTTP steps, SFTP/FTP steps. Just set `outputFile: true`.
+
+Use `sg run download <runId>` to download file artifacts from the CLI.
+
+## Common Pitfalls
+
+- Using bare `file::<key>` without `.raw`, `.base64`, or `.extracted` suffix — causes a runtime error
+- Placing file references in headers or query params instead of body/content fields — they only resolve in body
+- Expecting Excel parsed output to be a flat array — it is always `{ "SheetName": [...] }` even for single-sheet files
+- Hardcoding bracket aliases like `sourceData.__files__["step[\"file.csv\"]"]` — prefer `stepFileKeys` or `Object.keys(sourceData.__files__)` to discover aliases dynamically
+- Trying to construct binary file internals (xlsx, zip) as JS objects — always pass through original bytes via `.raw`
+- Accessing `sourceData.__files__` in step configs — `file::` syntax is for step configs, `sourceData.__files__` is for transform code
+- Manually base64-encoding file content with `btoa()` — never do this. Return raw bytes (Uint8Array) or strings directly. The runtime handles encoding automatically. Use `XLSX.write(wb, { type: "array" })` not `type: "buffer"` followed by manual btoa.
+- Putting file logic in the output transform — the output transform is for JSON data only. Use `outputFile: true` on the step instead.
+
+## Error Recovery
+
+When a file-related step fails:
+
+1. **File input failure** - Ask the user whether the file is present in the session state and verify whether you can access the file at all
+2. **Check the file:: reference** — verify the alias matches a step that actually produces files. Use `stepFileKeys` on the producing step's result to see available aliases.
+3. **Verify the suffix** — bare `file::<key>` without `.raw`, `.base64`, or `.extracted` always fails. Choose the suffix based on how the consuming step needs the data.
+4. **Check the producing step** — if `stepFileKeys` is empty or undefined, the producing step may not have returned a file. Verify the response is binary or has `Content-Disposition: attachment` (for HTTP), or that the operation is `get` (for file servers).
+5. **Check aliasing for loops** — loop iterations use numeric indices (`stepId[0]`), not filenames. If the alias doesn't match, run the producing step and inspect the result to see the actual aliases assigned.

package/skills/superglue/references/graphql.md

@@ -0,0 +1,131 @@
+# GraphQL
+
+GraphQL APIs use HTTP POST under the hood. This skill covers the GraphQL-specific conventions; the http skill covers the underlying HTTP runtime behavior.
+
+## Step Configuration
+
+```typescript
+{
+  type: "request",
+  systemId: "my_graphql_api",
+  url: "<<my_graphql_api_url>>/graphql",
+  method: "POST",
+  headers: {
+    "Authorization": "Bearer <<my_graphql_api_access_token>>",
+    "Content-Type": "application/json"
+  },
+  body: '{"query": "query GetUser($id: ID!) { user(id: $id) { name email } }", "variables": {"id": "<<userId>>"}}'
+}
+```
+
+### Fields
+
+| Field      | Required | Notes                                                  |
+| ---------- | -------- | ------------------------------------------------------ |
+| `url`      | yes      | GraphQL endpoint — typically `/graphql`                |
+| `method`   | yes      | Always `POST`                                          |
+| `headers`  | yes      | Must include `Content-Type: application/json` and auth |
+| `body`     | yes      | JSON string with `query` and optional `variables`      |
+| `systemId` | no       | Links system credentials and URL                       |
+
+### Body Format
+
+The body is always a JSON string with:
+
+```json
+{
+  "query": "query { ... }",
+  "variables": { ... }
+}
+```
+
+For dynamic variables, use `<<>>` expressions:
+
+```json
+{
+  "query": "query GetOrders($customerId: ID!, $limit: Int) { orders(customerId: $customerId, first: $limit) { id total } }",
+  "variables": {
+    "customerId": "<<customerId>>",
+    "limit": "<<(sourceData) => sourceData.pageSize || 50>>"
+  }
+}
+```
+
+For complex variable objects, use a preceding transform step to build the variables, then reference with `<<(sourceData) => JSON.stringify(sourceData.prepStep.data)>>` as the entire body.
+
+## Authentication
+
+Same as HTTP — credentials are placed in headers using `<<systemId_credentialKey>>` syntax. Nothing is injected automatically.
+
+## GraphQL Runtime Details
+
+### Queries vs Mutations
+
+- **Queries** (read-only): set `modify: false` on the step (this is the default)
+- **Mutations** (write): set `modify: true`
+
+Both use `method: "POST"` — do not rely on HTTP method to distinguish read vs write.
+
+### Response Shape
+
+GraphQL APIs return `{ "data": { ... }, "errors": [ ... ] }`. The superglue step result wraps this in the standard envelope:
+
+```javascript
+sourceData.stepId.data; // → { data: { user: { name: "Alice" } }, errors: [...] }
+```
+
+Access the actual data via `sourceData.stepId.data.data.user` (double `.data` — one for the envelope, one for the GraphQL response).
+
+### Pagination
+
+GraphQL APIs typically use cursor-based pagination (Relay-style). Use superglue's `cursorBased` pagination:
+
+```typescript
+{
+  body: '{"query": "query($cursor: String) { users(first: 50, after: $cursor) { edges { node { id name } } pageInfo { endCursor hasNextPage } } }", "variables": {"cursor": "<<cursor>>"}}',
+  pagination: {
+    type: "cursorBased",
+    pageSize: "50",
+    cursorPath: "data.users.pageInfo.endCursor",
+    stopCondition: "response.data.data.users.pageInfo.hasNextPage === false"
+  }
+}
+```
+
+Note the `cursorPath` and `stopCondition` must account for the GraphQL `data` wrapper.
+
+### Schema Discovery
+
+Superglue automatically introspects GraphQL schemas during documentation fetching. When a system URL contains "graphql", introspection is the first strategy tried — the full schema is fetched and stored as system documentation.
+
+Use `sg system search-docs` to query the stored schema for types, fields, and arguments before building tools. This is more reliable than manual introspection and works even when the API has introspection disabled for external callers.
+
+If the schema is not available via `sg system search-docs` (e.g. the system was created without a GraphQL URL, or introspection was blocked), test manually with `sg system call`:
+
+```json
+{ "query": "{ __typename }" }
+```
+
+If that succeeds, try a targeted introspection for the types you need:
+
+```json
+{ "query": "{ __type(name: \"User\") { fields { name type { name kind } } } }" }
+```
+
+## Common Pitfalls
+
+- Forgetting `Content-Type: application/json` header — most GraphQL servers reject requests without it
+- Accessing `sourceData.stepId.data.user` instead of `sourceData.stepId.data.data.user` — the GraphQL response has its own `data` wrapper inside the step envelope
+- Building complex variable objects inline with multiple `<<>>` expressions — use a transform step to build the variables object
+- Using `cursorPath` or `stopCondition` that doesn't account for the GraphQL `data` wrapper (e.g. `users.pageInfo` instead of `data.users.pageInfo`)
+- Not setting `modify: true` on mutation steps
+
+## Error Recovery
+
+When a GraphQL step fails:
+
+1. **Check GraphQL-level errors** — even 200 responses can contain `errors` in the response body. Check `sourceData.stepId.data.errors` for field-level or validation errors.
+2. **Verify the endpoint works** — use `sg system call` with `{ "query": "{ __typename }" }` to confirm connectivity and auth before debugging the query itself.
+3. **Check schema compatibility** — fields, types, and arguments change between API versions. Use `sg system search-docs` to check the auto-introspected schema first. If unavailable, use targeted `__type` queries or web search to verify the current schema.
+4. **Verify variables match the query signature** — GraphQL type mismatches (e.g. passing a string where `ID!` is expected) cause validation errors. Check that variable types in the query match what you're passing.
+5. **Check for auth scope issues** — some GraphQL APIs return partial data with `null` fields and errors when the token lacks required scopes for specific fields.