@unstable-dev/unmeshed-mcp 0.1.0

package/knowledge/references/steps-knowledge.md

@@ -0,0 +1,834 @@
+# Unmeshed Step Types — Full Reference
+
+---
+
+## JAVASCRIPT
+
+Executes an inline JavaScript transformation. Input/output data mapping between steps.
+
+```json
+{
+  "type": "JAVASCRIPT",
+  "input": {
+    "script": "(steps, context) => {\n  // your logic here\n  return { result: value };\n}"
+  }
+}
+```
+
+**Rules:**
+- Script signature is always `(steps, context) => { ... }`
+- Access previous step output via `steps.<ref>.output.result.<field>`
+- Access workflow input via `context.input.<field>`
+- Access workflow/process metadata via `context.id`
+- Must `return` an object — this becomes `output.result`
+- Use `steps.__self.id` for current step ID
+- Use `steps.__self.startTime` for the step's start timestamp (milliseconds) — useful in WAIT steps
+- Use `steps.__self.executionList` — array of past execution attempts for this step (populated when an error policy retries). `steps.__self.executionList.length` = number of attempts so far.
+- `preExecutionScript` in configuration can be used for setup (rare)
+- To throw a test failure: `throw Error("Testcase failed")` — this causes the step and workflow to fail
+
+**Accessing a prior step's INPUT** (not output) from a later step:
+```javascript
+steps.<ref>.input.body.content.<field>   // HTTP step request body field
+steps.<ref>.input.<field>                // any step input field
+```
+**Accessing a prior JS/Python step's OUTPUT** from another step:
+```javascript
+steps.<ref>.output.result.<field>
+```
+**Accessing error output from an optional failed step:**
+```javascript
+const errorMessage = steps.my_optional_step.output.error;
+if (errorMessage.includes("some error text")) { ... }
+```
+
+**Return `__statePut` to update workflow state:**
+```javascript
+return {
+  "__statePut": { "count": (context.state.count || 0) + 1 }
+};
+```
+
+**Deep equality helper** (no JSON.stringify, handles nested objects):
+```javascript
+function deepEqual(obj1, obj2) {
+  if (obj1 === obj2) return true;
+  if (typeof obj1 !== 'object' || obj1 === null ||
+      typeof obj2 !== 'object' || obj2 === null) return false;
+  const keys1 = Object.keys(obj1);
+  const keys2 = Object.keys(obj2);
+  if (keys1.length !== keys2.length) return false;
+  for (let key of keys1) {
+    if (!keys2.includes(key)) return false;
+    if (!deepEqual(obj1[key], obj2[key])) return false;
+  }
+  return true;
+}
+```
+
+---
+
+## PYTHON
+
+Executes inline Python for heavier processing (file parsing, binary ops, filesystem access).
+
+```json
+{
+  "type": "PYTHON",
+  "input": {
+    "script": "def main(steps, context):\n  # your logic\n  return { 'key': 'value' }"
+  }
+}
+```
+
+**Rules:**
+- Entry point is always `def main(steps, context):`
+- `steps` and `context` are dicts — use `.get()` for safe access
+- Files available at `/app/files/<location>/<filename>`
+- Must return a dict
+- Standard Python libraries available (os, re, zlib, json, base64, mimetypes, etc.)
+- Output is under `steps.<ref>.output.result.<field>`
+- Use `steps["<ref>"].output.result["<field>"]` when referring to output from another Python or JavaScript step
+- Use `steps["__self"]["id"]` for current step ID inside Python
+- Use `context.get("id")` for the workflow process/run ID
+
+**File system patterns:**
+```python
+import os, base64, mimetypes
+
+def main(steps, context):
+    base_dir = "/app/files/my-folder"
+    attachments = []
+    for filename in os.listdir(base_dir):
+        path = os.path.join(base_dir, filename)
+        if os.path.isfile(path):
+            with open(path, "rb") as f:
+                raw = f.read()
+            encoded = base64.b64encode(raw).decode("utf-8")
+            mime, _ = mimetypes.guess_type(filename)
+            attachments.append({
+                "name": filename,
+                "contentType": mime or "application/octet-stream",
+                "type": "CONTENT_BYTES",
+                "contentBase64": encoded
+            })
+    return {"attachments": attachments}
+```
+
+---
+
+## preExecutionScript
+
+`preExecutionScript` is supported for Python and JavaScript steps. It runs before the step executes and can override the step definition input at runtime.
+
+Required fields:
+- `preExecutionScript`: the actual script to execute
+- `constructInputFromScript`: set to `true` to enable script-driven input construction
+- `scriptLanguage`: `JAVASCRIPT` or `PYTHON` (`JAVASCRIPT` is the default when omitted)
+
+The step schema fields are: `preExecutionScript`, `constructInputFromScript`, and `scriptLanguage`.
+
+When enabled, the script runs before the step and its returned keys are merged into the step definition. Any returned values override the actual step configuration during runtime.
+
+Example (JavaScript):
+```json
+{
+  "type": "HTTP",
+  "preExecutionScript": "(steps, context) => { return {\n      url: `https://api.example.com/users/${steps.create_user.output.result.id}`,\n      method: 'GET'\n    }; }",
+  "constructInputFromScript": true,
+  "scriptLanguage": "JAVASCRIPT",
+  "input": {
+    "method": "GET",
+    "url": "https://placeholder.example.com",
+    "headers": {
+      "Accept": "application/json"
+    },
+    "params": {}
+  }
+}
+```
+
+Example (Python):
+```json
+{
+  "type": "HTTP",
+  "preExecutionScript": "def main(steps, context):\n    return {\n      'url': f'https://api.example.com/users/{steps[\"create_user\"].output.result[\"id\"]}',\n      'method': 'GET'\n    }\n",
+  "constructInputFromScript": true,
+  "scriptLanguage": "PYTHON",
+  "input": {
+    "method": "GET",
+    "url": "https://placeholder.example.com",
+    "headers": {
+      "Accept": "application/json"
+    },
+    "params": {}
+  }
+}
+```
+
+Note:
+- `preExecutionScript` must return an object
+- returned fields override the step configuration
+- the default `scriptLanguage` is `JAVASCRIPT`
+
+## NOOP
+
+A pass-through step that holds static configuration/constants and exposes them as output.
+
+```json
+{
+  "type": "NOOP",
+  "input": {
+    "anyKey": "anyStaticValue",
+    "nestedObject": { "field": "value" },
+    "arrayField": ["item1", "item2"]
+  }
+}
+```
+
+**Rules:**
+- No computation — purely a data carrier
+- All `input` fields become directly accessible as `steps.<ref>.output.<field>`
+  (note: NOT wrapped in `.result` — access directly)
+- Ideal for job definitions, config objects, feature flags
+
+---
+
+## INTEGRATION
+
+Calls an external service. The `type` field in `input` determines which integration.
+
+```json
+{
+  "type": "INTEGRATION",
+  "input": {
+    "type": "<integration-type>",
+    "name": "<connection-name>",
+    "publishProperties": { /* integration-specific config */ },
+    "messageBody": { /* payload / prompts / operation */ }
+  }
+}
+```
+
+See `integrations.md` for full per-integration schemas.
+
+---
+
+## HTTP (native step — not INTEGRATION)
+
+A dedicated HTTP step type (distinct from the `http` INTEGRATION). Used for direct HTTP calls with optional polling/repeat support.
+
+### GET request
+```json
+{
+  "type": "HTTP",
+  "input": {
+    "method": "GET",
+    "url": "{{variables.dev_server}}/api/resource/{{steps.create.output.response.id}}",
+    "headers": {
+      "Content-Type": "application/json",
+      "Accept": "application/json",
+      "Authorization": "Bearer {{secrets.api_token}}"
+    },
+    "params": { "key": "value" },
+    "repeatUntilEnabled": null,
+    "repeatUntilCondition": { "script": "(steps, context) => { return false; }" },
+    "repeatIntervalSeconds": null,
+    "maxRepeatCount": null,
+    "includeFullResponseString": false,
+    "noEncode": false,
+    "extraLongTimeouts": false
+  }
+}
+```
+
+### POST / PUT request (with body)
+```json
+{
+  "type": "HTTP",
+  "input": {
+    "method": "POST",
+    "url": "{{variables.dev_server}}/api/resource",
+    "headers": { "Content-Type": "application/json", "Authorization": "Bearer {{secrets.api_token}}" },
+    "params": {},
+    "body": {
+      "type": "json",
+      "content": {
+        "title": "foo",
+        "body": "{{ steps.prev_step.output.response.someField }}",
+        "userId": 1,
+        "fullObject": "{{ steps.prev_step.output.response }}"
+      }
+    },
+    "repeatUntilEnabled": null,
+    "repeatUntilCondition": { "script": "(steps, context) => { return false; }" },
+    "repeatIntervalSeconds": null,
+    "maxRepeatCount": null,
+    "includeFullResponseString": false,
+    "noEncode": false,
+    "extraLongTimeouts": false
+  }
+}
+```
+
+### DELETE request
+```json
+{
+  "input": {
+    "method": "DELETE",
+    "url": "{{variables.dev_server}}/api/resource",
+    "params": { "id": "{{ steps.create.output.response.id }}" },
+    "body": { "type": "json", "content": {} }
+  }
+}
+```
+
+### Polling (repeat until condition)
+```json
+{
+  "input": {
+    "method": "GET",
+    "url": "https://api.example.com/status",
+    "params": { "count": "{{ context.state.count }}" },
+    "repeatUntilEnabled": true,
+    "repeatUntilCondition": {
+      "script": "(steps, context) => { return steps.__self.output?.response?.status === 'done'; }"
+    },
+    "repeatIntervalSeconds": 2,
+    "maxRepeatCount": 10,
+    "includeFullResponseString": false,
+    "noEncode": false,
+    "extraLongTimeouts": false
+  }
+}
+```
+
+**Rules:**
+- `type` is `"HTTP"` (uppercase, not `"INTEGRATION"`)
+- `ref` naming: `new_http_<n>`, `http_<n>`, or a descriptive name like `fetch_all`, `create_new`
+- Output: `steps.<ref>.output.response.<field>` (response body fields), `steps.<ref>.output.statusCode` (integer)
+- Access a prior step's INPUT from a later step: `steps.<ref>.input.body.content.<field>` (real pattern — useful for assertions)
+- `body`: for POST/PUT/DELETE — structure is `{ "type": "json", "content": { ... } }`. GET requests omit `body`.
+- `body.content` can include `{{...}}` template references to inject previous step outputs
+- `params`: query string key-value pairs (always include even if empty `{}`)
+- `{{variables.<name>}}`: inject environment/namespace variables (e.g. server URLs)
+- `{{secrets.<name>}}`: inject secrets (e.g. API tokens) — never hardcode credentials
+- URL path segments can include `{{steps.<ref>.output.response.<field>}}` directly in the URL string
+- `repeatUntilEnabled: true` + `repeatIntervalSeconds` + `maxRepeatCount` — polling mode; stops when condition returns `true`
+- `repeatUntilCondition.script`: use optional chaining (`?.`) for safety since early polls may have no response yet
+- When NOT polling: set `repeatUntilEnabled: null`, `repeatIntervalSeconds: null`, `maxRepeatCount: null`
+- `includeFullResponseString: false` — set `true` only when you need the raw response string
+- `extraLongTimeouts: false` — set `true` for very slow endpoints
+- Collect FOREACH loop results: `steps["http[0]"].output`, `steps["http[1]"].output`, etc.
+
+---
+
+## SQLITE
+
+A native SQLITE step that executes a SQL statement against a managed SQLite store in the current namespace.
+
+Example:
+```json
+{
+  "orgId": 1,
+  "namespace": "automated_tests",
+  "name": "update",
+  "type": "SQLITE",
+  "ref": "update",
+  "optional": false,
+  "createdBy": "system",
+  "updatedBy": "system",
+  "description": null,
+  "label": null,
+  "created": 1765971852407,
+  "updated": 1765971852407,
+  "configuration": {
+    "errorPolicyName": null,
+    "useCache": false,
+    "cacheKey": null,
+    "cacheTimeoutSeconds": 0,
+    "stream": false,
+    "streamAllStatuses": false,
+    "preExecutionScript": null,
+    "constructInputFromScript": false,
+    "scriptLanguage": null,
+    "jqTransformer": null,
+    "rateLimitMaxRequests": 0,
+    "rateLimitWindowSeconds": 0
+  },
+  "children": [],
+  "input": {
+    "storeName": "automated-test-db",
+    "sql": "update sample_table set name = :#newName where name = 'Alice' or name = :secondName;",
+    "parameters": {
+      "newName": "Bob",
+      "secondName": "Alex"
+    }
+  },
+  "output": null
+}
+```
+
+**Rules:**
+- `type` must be `SQLITE`
+- `storeName` specifies the SQLite store name in the current namespace
+- `sql` contains the SQL statement to execute
+- `parameters` is optional, and is used for parameterized SQL values
+- `preExecutionScript`, `constructInputFromScript`, and `scriptLanguage` behave the same way as other step types
+- SQL results are returned in `steps.<ref>.output` if the step is configured to expose output
+
+---
+
+## WORKER
+
+Executes code in external worker processes using the Unmeshed Polyglot SDK. Workers run outside the main Unmeshed engine and can be implemented in Java, Python, Go, TypeScript/NodeJs or other languages.
+
+Example:
+```json
+{
+  "orgId": 1,
+  "namespace": "automated_tests",
+  "name": "not_exist_x",
+  "type": "WORKER",
+  "ref": "worker_1",
+  "optional": false,
+  "createdBy": "system",
+  "updatedBy": "system",
+  "description": null,
+  "label": null,
+  "created": 1776247373914,
+  "updated": 1776247373914,
+  "configuration": {
+    "errorPolicyName": null,
+    "useCache": false,
+    "cacheKey": null,
+    "cacheTimeoutSeconds": 0,
+    "stream": false,
+    "streamAllStatuses": false,
+    "preExecutionScript": null,
+    "constructInputFromScript": false,
+    "scriptLanguage": null,
+    "jqTransformer": null,
+    "rateLimitMaxRequests": 0,
+    "rateLimitWindowSeconds": 0
+  },
+  "children": [],
+  "input": {},
+  "output": null
+}
+```
+
+**Rules:**
+- `type` must be `WORKER`
+- `name` specifies the worker queue name that the SDK worker is registered to
+- `input` contains the data passed to the worker function
+- Workers run asynchronously outside the main engine using the Unmeshed SDK
+- Worker implementations can be in Java (Spring Boot), Python, or other supported languages
+- Worker output is returned in `steps.<ref>.output` when the worker completes
+- `preExecutionScript`, `constructInputFromScript`, and `scriptLanguage` behave the same way as other step types
+- Workers must be registered and running for the step to execute successfully
+
+---
+
+## WAIT
+
+Pauses workflow execution until a specified time.
+
+```json
+{
+  "type": "WAIT",
+  "input": {
+    "script": "(steps, context) => {\n  return {\n    \"waitUntil\": steps.__self.startTime + (25 * 1000)\n  };\n}"
+  }
+}
+```
+
+**Rules:**
+- `input.script` is a JS function returning `{ waitUntil: <timestamp_ms> }`
+- `steps.__self.startTime` — start time of this WAIT instance in milliseconds (use for relative waits)
+- `steps.__self.start` — alternative property (also milliseconds)
+- Inside a WHILE loop body: use the WHILE step's `.updated` timestamp as the loop iteration reference:
+  ```javascript
+  (steps, context) => {
+    const lastIterationTime = steps.my_while_step.updated;
+    return { "waitUntil": lastIterationTime + 1000 };
+  }
+  ```
+- Inside FOREACH loops: use `steps.__self.executionList?.at(-1)?.scheduled` as reference
+- `waitUntil` is an absolute epoch timestamp in milliseconds
+- `ref` naming: `wait_<n>` or `wait`
+
+---
+
+## FOREACH
+
+Iterates over an array, executing its child steps for each element.
+
+```json
+{
+  "type": "FOREACH",
+  "children": [
+    { /* child step(s) — usually a LIST */ }
+  ],
+  "input": {
+    "inputArray": [1, 2, 3, 4, 5],
+    "concurrency": 1
+  }
+}
+```
+
+**Rules:**
+- `inputArray`: the array to iterate over — can be static or a `{{...}}` template reference
+- `concurrency`: how many iterations run in parallel — `1` = sequential, `N` = N at a time
+- Children execute for each element in the array
+- Access the current iteration value inside child scripts via `context.input` or loop context
+- After the loop, access individual iteration outputs: `steps["<ref>[0]"].output`, `steps["<ref>[1]"].output`, etc.
+- `ref` naming: use a descriptive name like `foreachstep` or `foreach_<n>`
+- Children are typically a single `LIST` containing the loop body steps
+
+**Collecting FOREACH results in JAVASCRIPT:**
+```javascript
+(steps, context) => {
+  const allOutputs = [];
+  allOutputs.push(steps["http[0]"].output);
+  allOutputs.push(steps["http[1]"].output);
+  // ... etc
+  return allOutputs;
+}
+```
+
+---
+
+## WHILE
+
+Loops while a condition is true, executing its child steps each iteration.
+
+```json
+{
+  "type": "WHILE",
+  "children": [
+    {
+      "type": "LIST",
+      "ref": "loop_body",
+      "children": [
+        { /* WAIT step (throttle iterations) */ },
+        { /* JAVASCRIPT step (do work + update state) */ }
+      ],
+      "input": {}
+    }
+  ],
+  "input": {
+    "script": "(steps, context) => {\n  return whileloop.iteration < 5;\n}"
+  }
+}
+```
+
+**Rules:**
+- `input.script` is a JS function returning a boolean — loop continues while `true`
+- `whileloop.iteration` — built-in counter starting at 0, increments each iteration
+- Child is typically a single `LIST` containing the loop body
+- Always include a `WAIT` step inside the loop body to prevent tight-loop runaway
+- Use `context.state.<key>` to read mutable state that persists across iterations
+- Use `__statePut` in a JAVASCRIPT step to write to state (see context.state below)
+- `steps.<while_ref>.updated` — timestamp of the last loop iteration (use in WAIT as base time)
+- `ref` naming: `new_while_<n>` or `while_<n>`
+
+**Loop body with state update:**
+```javascript
+// WAIT step — throttle using last iteration time
+(steps, context) => {
+  return { "waitUntil": steps.my_while_step.updated + 1000 };
+}
+
+// JAVASCRIPT step — increment counter in state
+(steps, context) => {
+  return {
+    "__statePut": {
+      "count": (context.state.count || 0) + 1
+    }
+  };
+}
+```
+
+---
+
+## context.state and __statePut
+
+Workflows support mutable in-run state accessible across all steps via `context.state`.
+
+**Reading state:**
+```javascript
+(steps, context) => {
+  const count = context.state.count || 0;
+  const lastId = context.state.lastProcessedId;
+  // ...
+}
+```
+
+**Writing state** (return `__statePut` from any JAVASCRIPT step):
+```javascript
+(steps, context) => {
+  return {
+    "__statePut": {
+      "count": (context.state.count || 0) + 1,
+      "lastProcessedId": someId
+    }
+  };
+}
+```
+
+**Rules:**
+- `context.state` is a plain object, readable in any step script
+- To write: return `{ "__statePut": { key: value } }` from a JAVASCRIPT step
+- `__statePut` merges into existing state (not a full replace)
+- State is scoped to the current workflow run — not persisted across runs (use PERSISTED_STATE for that)
+- Common use: loop counters, accumulating results, tracking IDs across iterations
+- Also accessible in HTTP step `params`: `"count": "{{ context.state.count }}"`
+
+---
+
+## {{variables.*}} and {{secrets.*}}
+
+Template placeholders available in HTTP step URLs, headers, params, and body content.
+
+```json
+{
+  "url": "{{variables.dev_server}}/api/resource",
+  "headers": {
+    "Authorization": "Bearer {{secrets.unmeshed_test_token}}"
+  }
+}
+```
+
+**Rules:**
+- `{{variables.<name>}}` — namespace/environment variables (e.g. base URLs, config values)
+- `{{secrets.<name>}}` — secret values (API tokens, passwords) — never hardcode these
+- Available in HTTP step `url`, `headers`, `params`, and `body.content`
+- Also available as `{{steps.<ref>.output.response.<field>}}` template refs in the same fields
+
+---
+
+## DECISION_ENGINE
+
+Routes based on a named decision table (rule engine).
+
+```json
+{
+  "type": "DECISION_ENGINE",
+  "input": {
+    "decisionRuleStrategy": "FIRST_MATCH",
+    "decisionTable": "<table-name>",
+    "decisionContext": {
+      "FieldName": "{{ steps.<ref>.output.results.<field> }}"
+    },
+    "decisionTableVersion": "",
+    "decisionOutputColumns": ["OutputCol1", "OutputCol2"]
+  }
+}
+```
+
+**Rules:**
+- `decisionTable` must reference a table configured in Unmeshed
+- `decisionRuleStrategy`: `FIRST_MATCH` (most common) or `ALL_MATCH`
+- `decisionContext` keys must match column names in the decision table
+- Context values can be `{{...}}` template expressions or empty string `""` for optional fields
+- Output columns define what fields are returned from the matched rule
+- Decision output accessed via `steps.<ref>.output.result.<ColumnName>` or `steps.<ref>.output.results.<ColumnName>` — defensively check both
+
+---
+
+## SWITCH
+
+Branches the workflow based on script logic. Must have `children` array with branch steps.
+
+```json
+{
+  "type": "SWITCH",
+  "children": [
+    { /* branch step — any type: LIST, INTEGRATION, NOOP, EXIT, JAVASCRIPT */ },
+    { /* another branch */ }
+  ],
+  "input": {
+    "script": "(steps, context) => {\n  if (condition) return 'branch-a';\n  return 'branch-b';\n}",
+    "responseMapping": [
+      { "targetRef": "<ref-of-branch-a>", "value": "branch-a" },
+      { "targetRef": "<ref-of-branch-b>", "value": "branch-b" },
+      { "targetRef": "<ref-of-default>", "defaultBranch": true }
+    ]
+  }
+}
+```
+
+**Rules:**
+- Script must return a string matching one of the `value` fields in `responseMapping`
+- `defaultBranch: true` entry is required — points to the fallback branch ref
+- Children can be **any step type** — `LIST`, `INTEGRATION`, `NOOP`, `EXIT`, `JAVASCRIPT`
+- Only the matched branch executes
+- The `ref` of each child must match the `targetRef` in `responseMapping`
+- A SWITCH branch child that is a single step does NOT need to be wrapped in a LIST
+- Real-world pattern: check for API errors (e.g. rate limiting) and branch to a skip path vs main path
+
+---
+
+## LIST
+
+A sequential container of steps. Used as a branch body inside SWITCH or PARALLEL, or as a standalone grouping.
+
+```json
+{
+  "type": "LIST",
+  "children": [
+    { /* step 1 */ },
+    { /* step 2 */ },
+    { /* step 3 */ }
+  ],
+  "input": {},
+  "output": null
+}
+```
+
+**Rules:**
+- Steps in `children` execute in order
+- Used as a named branch inside SWITCH (`ref` must match SWITCH `responseMapping`)
+- Used as a branch inside PARALLEL (each LIST runs concurrently with others)
+- Can contain any step types including nested SWITCH, PARALLEL, WAIT, FOREACH
+- `input` is always `{}` for LIST
+
+---
+
+## PARALLEL
+
+Executes multiple branches simultaneously. Children are typically LIST steps (each containing multiple steps), not just SUB_PROCESS.
+
+```json
+{
+  "type": "PARALLEL",
+  "children": [
+    {
+      "type": "LIST",
+      "ref": "branch_1",
+      "children": [ /* steps */ ],
+      "input": {}
+    },
+    {
+      "type": "LIST",
+      "ref": "branch_2",
+      "children": [ /* steps */ ],
+      "input": {}
+    }
+  ],
+  "input": {
+    "failIfAnyBranchFails": true
+  }
+}
+```
+
+**Rules:**
+- All children execute concurrently
+- `failIfAnyBranchFails: true` — workflow fails if any branch fails (strict mode)
+- `failIfAnyBranchFails: false` — continue even if branches fail (lenient mode)
+- Children are **LIST steps** (for multi-step branches) or **SUB_PROCESS steps** (for sub-workflow fan-out)
+- Each LIST branch has its own `ref` and `children`
+- Steps inside different PARALLEL branches can reference each other's outputs after the PARALLEL completes
+
+---
+
+## SUB_PROCESS
+
+Invokes another named Unmeshed workflow.
+
+```json
+{
+  "type": "SUB_PROCESS",
+  "input": {
+    "processName": "<workflow-name>",
+    "waitForCompletion": true,
+    "input": { /* optional input to pass */ }
+  }
+}
+```
+
+**Rules:**
+- `processName` must match the `name` of another workflow in the same org
+- `waitForCompletion: true` — waits for sub-process to finish before continuing
+- `waitForCompletion: false` — fire-and-forget
+- `input` is optional; omit or pass `{}` if no input needed
+- Common pattern: SUB_PROCESS followed by WAIT to allow async side effects (e.g. emails) to propagate before assertions
+
+---
+
+## PERSISTED_STATE
+
+Reads or writes persistent key-value state scoped to the org/namespace.
+
+```json
+{
+  "type": "PERSISTED_STATE",
+  "input": {
+    "operation": "READ",
+    "name": "<state-key>",
+    "path": "$"
+  }
+}
+```
+
+For UPSERT (write):
+```json
+{
+  "type": "PERSISTED_STATE",
+  "input": {
+    "operation": "UPSERT",
+    "name": "<state-key>",
+    "path": "$",
+    "value": "<value-to-store>"
+  }
+}
+```
+
+**Rules:**
+- `operation`: `"READ"` or `"UPSERT"`
+- `name`: the key — can be a static string or a `{{...}}` template expression
+- `path`: use `"$"` for root; supports JSONPath for nested access
+- `value`: only for UPSERT — the value to persist
+- READ output: `steps.<ref>.output.result.value` — returns `null` if key doesn't exist
+- Common pattern: READ → SWITCH on whether result is null → UPSERT after creating resource
+
+---
+
+## EXIT
+
+Terminates the workflow immediately with a given status.
+
+```json
+{
+  "type": "EXIT",
+  "input": {
+    "message": "Exit message",
+    "status": "COMPLETED"
+  }
+}
+```
+
+**Rules:**
+- `status`: `"COMPLETED"` or `"FAILED"`
+- Used inside SWITCH branches to short-circuit the workflow
+- Common pattern: gate check via PERSISTED_STATE → SWITCH → EXIT if flag is false
+- Can also be used mid-workflow as a temporary stub (e.g. `"message": "Exiting temporarily"`)
+
+---
+
+## Optional Steps
+
+Any step can be marked `"optional": true` to allow the workflow to continue even if that step fails.
+
+```json
+{
+  "optional": true,
+  ...
+}
+```
+
+**Rules:**
+- Default is `false` — step failure stops the workflow
+- Set `optional: true` for steps that may legitimately fail (e.g. external API calls that might be rate-limited, best-effort cleanup steps)
+- Real-world use: `get_all_tables` in an Airtable workflow marked optional so the SWITCH after it can check for rate-limit errors gracefully