@unstable-dev/unmeshed-mcp 0.1.0

package/knowledge/steps/SEND_RESPONSE.md

@@ -0,0 +1,301 @@
+# Send Response Step Schema
+
+`SEND_RESPONSE` lets a workflow send an HTTP response payload back to the
+waiting API caller before or during the rest of the workflow execution. Use it
+when an API-triggered process must control the HTTP status code and response
+body directly from inside the workflow.
+
+From the engine source:
+- The step copies a mapped payload into the **process-level output**.
+- It resolves the response payload from `input.outputMapping`, unless the
+  process definition has a top-level `outputMapping`, in which case that
+  process-level mapping is used instead.
+- If `httpStatusCode` is `2xx`, it streams a normal response to the waiting
+  caller.
+- If `httpStatusCode` is not `2xx`, it streams a `CustomHttpException` to the
+  waiting caller with `errorMessage` and the mapped output as `context`.
+- The step itself is still marked `COMPLETED` and proceeds through the normal
+  engine update path.
+
+Important:
+- `SEND_RESPONSE` is most meaningful for synchronous API/endpoint execution.
+- If there is no active waiting caller, the stream call has no visible external
+  effect, but the process-level output is still updated.
+
+## Definition Input Schema
+
+```json
+{
+  "httpStatusCode": 400,
+  "outputMapping": {
+    "result": "failed"
+  }
+}
+```
+
+| Field | Type | Required | Description |
+|---|---|---|---|
+| `httpStatusCode` | number or numeric string | No | HTTP status code to send back to the waiting caller. Defaults to `200`. |
+| `outputMapping` | object | No | Response payload to merge into process output and return to the caller. |
+
+Observed source behavior:
+- `httpStatusCode` may be a number like `400` or a numeric string like `"202"`.
+- If `httpStatusCode` is an invalid non-numeric string, the step fails with `output.error = "httpStatusCode should be a valid numerical status code"`.
+- If the **process definition** has a top-level `outputMapping`, that mapping is resolved and used instead of the step input `outputMapping`.
+
+## Runtime Output Schema
+
+The step record itself usually has an empty `output` map on success:
+
+```json
+{}
+```
+
+But it mutates the **process-level output**. For example, with `httpStatusCode`
+`200`:
+
+```json
+{
+  "result": "failed",
+  "__currentExecutionStartTime": 1778615728768
+}
+```
+
+When called through a synchronous endpoint with a non-2xx status, the caller
+receives an error payload like:
+
+```json
+{
+  "result": "failed",
+  "errorMessage": "Request failed with HTTP status code: 400",
+  "context": {
+    "result": "failed"
+  }
+}
+```
+
+When the status code is invalid, the step itself fails with:
+
+```json
+{
+  "error": "httpStatusCode should be a valid numerical status code"
+}
+```
+
+## Output Access Paths
+
+Use:
+- `context.output.<field>` conceptually at the process level, because
+  `SEND_RESPONSE` writes to the process output rather than to its own normal
+  `step.output.result`.
+- In run debugging, inspect the **process-level** `output` to see the mapped
+  response payload.
+- Inspect `stepRecords[].output.error` only when the step itself fails, for
+  example because `httpStatusCode` is invalid.
+
+Do not use:
+- `steps.<ref>.output.result`
+- `steps.<ref>.output.response`
+- `steps.<ref>.output.<field>` to read the successful response payload
+
+## Generation Rules
+
+- Use uppercase step type: `"SEND_RESPONSE"`.
+- Use `children: []`; SEND_RESPONSE is not a container.
+- Put `httpStatusCode` and `outputMapping` in `input`.
+- Use `outputMapping` to shape the response payload that should be returned to
+  the caller.
+- Prefer `2xx` status codes when the API call should succeed.
+- Use non-2xx status codes when the caller should receive an HTTP error with a
+  structured body.
+- Remember that `SEND_RESPONSE` changes the process-level output, not the step's
+  own output structure.
+- If the process definition already has a top-level `outputMapping`, remember
+  that source code gives that mapping precedence over the step input mapping.
+- Use templating inside `outputMapping` when you want to return data from
+  previous steps, for example values from `steps.http_1.output.response`.
+- Do not expect `SEND_RESPONSE` to behave like `EXIT`; it is for returning a
+  response, not intentionally ending the process with a final process status.
+
+## Debugging Rules
+
+When debugging a SEND_RESPONSE step:
+- Inspect `input.httpStatusCode`.
+- Inspect whether the process definition also has a top-level `outputMapping`.
+- Inspect the final **process-level** `output` instead of expecting payload
+  fields under the SEND_RESPONSE step record.
+- If the caller saw an HTTP error response, check whether `httpStatusCode` was
+  non-2xx.
+- If the caller saw `errorMessage: "Request failed with HTTP status code: ..."`
+  that came from the SEND_RESPONSE step runner's error streaming logic.
+- If the step failed outright, check whether `httpStatusCode` was an invalid
+  non-numeric string.
+- If later steps still ran, remember that SEND_RESPONSE is not EXIT; the step is
+  marked `COMPLETED` and follows the normal update path.
+
+Common failures:
+- Looking for successful payload fields under `steps.send_response_1.output`.
+- Assuming SEND_RESPONSE stops the workflow like EXIT.
+- Forgetting that process definition `outputMapping` overrides the step's own
+  `input.outputMapping`.
+- Passing `"bad-request"` instead of a numeric code like `400`.
+- Using SEND_RESPONSE in a background/asynchronous context and expecting an HTTP
+  caller to receive it immediately.
+
+## Minimal Process Definition Example
+
+```json
+{
+  "orgId": 1,
+  "namespace": "default",
+  "name": "send_response_test",
+  "version": 1,
+  "type": "API_ORCHESTRATION",
+  "description": "Call HTTP and then send a mapped response to the API caller.",
+  "configuration": null,
+  "steps": [
+    {
+      "orgId": 1,
+      "namespace": "default",
+      "name": "http",
+      "type": "HTTP",
+      "ref": "http_1",
+      "optional": false,
+      "createdBy": "system",
+      "updatedBy": "system",
+      "description": null,
+      "label": null,
+      "created": 1700000000000,
+      "updated": 1700000000000,
+      "configuration": {
+        "errorPolicyName": null,
+        "useCache": false,
+        "cacheKey": null,
+        "cacheTimeoutSeconds": 0,
+        "stream": false,
+        "streamAllStatuses": false,
+        "preExecutionScript": null,
+        "constructInputFromScript": false,
+        "scriptLanguage": null,
+        "jqTransformer": null,
+        "rateLimitMaxRequests": 0,
+        "rateLimitWindowSeconds": 0
+      },
+      "children": [],
+      "input": {
+        "method": "GET",
+        "url": "http://localhost:8080/api/test/get",
+        "headers": {
+          "Content-Type": "application/json",
+          "Accept": "application/json",
+          "Authorization": "Bearer {{secrets.test_token}}"
+        },
+        "params": {
+          "sampleKey": "sampleValue"
+        },
+        "repeatUntilEnabled": null,
+        "repeatUntilCondition": {
+          "script": "(steps, context) => {\n  return steps.__self.output.response.counter === 100;\n}"
+        },
+        "repeatIntervalSeconds": null,
+        "maxRepeatCount": null,
+        "includeFullResponseString": false,
+        "noEncode": false,
+        "extraLongTimeouts": false
+      },
+      "output": null
+    },
+    {
+      "orgId": 1,
+      "namespace": "default",
+      "name": "send_response",
+      "type": "SEND_RESPONSE",
+      "ref": "send_response_1",
+      "optional": false,
+      "createdBy": "system",
+      "updatedBy": "system",
+      "description": null,
+      "label": null,
+      "created": 1700000000000,
+      "updated": 1700000000000,
+      "configuration": {
+        "errorPolicyName": null,
+        "useCache": false,
+        "cacheKey": null,
+        "cacheTimeoutSeconds": 0,
+        "stream": false,
+        "streamAllStatuses": false,
+        "preExecutionScript": null,
+        "constructInputFromScript": false,
+        "scriptLanguage": null,
+        "jqTransformer": null,
+        "rateLimitMaxRequests": 0,
+        "rateLimitWindowSeconds": 0
+      },
+      "children": [],
+      "input": {
+        "httpStatusCode": 400,
+        "outputMapping": {
+          "result": "failed"
+        }
+      },
+      "output": null
+    }
+  ],
+  "defaultInput": null,
+  "defaultOutput": null,
+  "outputMapping": null,
+  "signature": null,
+  "metadata": null,
+  "tags": null,
+  "dependencies": null,
+  "dependents": null
+}
+```
+
+## Minimal Executed Step Example
+
+```json
+{
+  "id": 30050047,
+  "processId": 30050044,
+  "ref": "send_response_1",
+  "namespace": "default",
+  "name": "send_response",
+  "type": "SEND_RESPONSE",
+  "status": "COMPLETED",
+  "input": {
+    "outputMapping": {
+      "result": "failed"
+    },
+    "__currentExecutionStartTime": 1778615728762,
+    "httpStatusCode": 200
+  },
+  "output": {}
+}
+```
+
+## Example: Return Upstream HTTP Data To The Caller
+
+```json
+{
+  "httpStatusCode": 200,
+  "outputMapping": {
+    "counter": "{{steps.http_1.output.response.counter}}",
+    "randomId": "{{steps.http_1.output.response.randomId}}"
+  }
+}
+```
+
+## Source Notes
+
+Observed from:
+- `SendResponseStepRunnerActor`
+- `SendResponseStepRunnerActorTest`
+
+Key implementation details:
+- Default status code is `200`.
+- Numeric strings like `"202"` are accepted.
+- Invalid strings fail the step.
+- Success calls `ProcessExecutionHandler.streamResultFull(processContext)`.
+- Non-2xx calls `ProcessExecutionHandler.streamErrorFull(...)`.

package/knowledge/steps/SQLITE.md

@@ -0,0 +1,301 @@
+# SQLITE Step Schema
+
+`SQLITE` executes SQL statements against a named SQLite datastore managed by
+Unmeshed. It supports `SELECT`, `INSERT`, `UPDATE`, `DELETE`, and DDL
+statements. Use it for lightweight persistent storage, lookup tables,
+accumulating run data, or any scenario where a full external database is
+overkill.
+
+## Definition Input Schema
+
+```json
+{
+  "storeName": "my-database",
+  "sql": "select * from sample_table where name = :#name;",
+  "parameters": {
+    "name": "Alice"
+  }
+}
+```
+
+| Field | Type | Required | Description |
+|---|---|---|---|
+| `storeName` | string | Yes | Name of the SQLite datastore (managed via Unmeshed config). The store is created automatically if it does not exist. |
+| `sql` | string | Yes | SQL statement to execute. Use `:#paramName` syntax for parameterised bindings (see below). |
+| `parameters` | object | No | Key-value map of parameter bindings. Each key must match a `:#key` placeholder in `sql`. Values can be strings, numbers, or booleans. |
+
+### Parameter Binding Syntax
+
+Parameters use the `:#<name>` prefix inside the SQL string:
+
+```sql
+select * from users where name = :#name and age > :#minAge;
+```
+
+With parameters:
+```json
+{
+  "name": "Alice",
+  "minAge": 18
+}
+```
+
+- `:#name` → binds the value of `parameters.name`
+- `:#minAge` → binds the value of `parameters.minAge`
+- Always use parameter bindings for user-supplied values to prevent SQL injection.
+- Do not use string concatenation or template literals inside the SQL string.
+
+### Supported SQL Operations
+
+| Operation | Example |
+|---|---|
+| `SELECT` | `select * from users where id = :#id;` |
+| `INSERT` | `insert into users (name, age) values (:#name, :#age);` |
+| `UPDATE` | `update users set age = :#age where name = :#name;` |
+| `DELETE` | `delete from users where id = :#id;` |
+| `CREATE TABLE` | `create table if not exists users (id integer primary key autoincrement, name text, age integer);` |
+
+## Runtime Output Schema
+
+### SELECT (returns rows)
+
+```json
+{
+  "result": {
+    "rowsAffected": 0,
+    "rows": [
+      {
+        "name": "Alice",
+        "id": 1,
+        "age": 25
+      }
+    ]
+  }
+}
+```
+
+### INSERT / UPDATE / DELETE (returns affected count)
+
+```json
+{
+  "result": {
+    "rowsAffected": 1,
+    "rows": []
+  }
+}
+```
+
+### CREATE TABLE / DDL
+
+```json
+{
+  "result": {
+    "rowsAffected": 0,
+    "rows": []
+  }
+}
+```
+
+## Output Access Paths
+
+Use:
+- `steps.<ref>.output.result.rows` — array of row objects for SELECT queries.
+- `steps.<ref>.output.result.rows[0]` — first row of a SELECT result.
+- `steps.<ref>.output.result.rows[0].<column>` — specific column from the first row.
+- `steps.<ref>.output.result.rowsAffected` — number of rows affected (INSERT/UPDATE/DELETE).
+- `steps.<ref>.output.result.rows.length` — row count from a SELECT.
+
+Do not use:
+- `steps.<ref>.output.response` — SQLITE does not use `response`; that is for HTTP steps.
+- `steps.<ref>.output.results` — the field is `result` (singular), not `results`.
+- `steps.<ref>.output.result.data` — rows are at `result.rows`, not `result.data`.
+
+## Generation Rules
+
+- Use uppercase step type: `"SQLITE"`.
+- Use `children: []`; SQLITE is not a container.
+- `storeName` must be a valid SQLite datastore name configured in the org.
+- `sql` must end with a semicolon `;`.
+- Always use `:#paramName` parameter bindings — never interpolate values into SQL.
+- Parameters must be a flat key-value object (no nested objects or arrays).
+- For multi-step database operations (e.g. create table then insert), use
+  separate SQLITE steps in a LIST.
+- When reading SELECT results in a subsequent step, always guard against empty
+  rows: `steps.<ref>.output.result.rows[0] || {}`.
+- When using `steps.<ref>.output.result` reference from another step's input,
+  make sure to access `.rows` for the actual data.
+
+## Debugging Rules
+
+When debugging a SQLITE step:
+- Check `output.result.rowsAffected` — 0 for SELECT/DDL, > 0 for INSERT/UPDATE/DELETE.
+- Check `output.result.rows` — array of row objects for SELECT, empty `[]` for non-SELECT.
+- If `status: "FAILED"`, check the step `output.error` for the SQL error message.
+- Common SQL errors:
+  - `no such table` — the table does not exist; add a CREATE TABLE step first.
+  - `no such column` — column name mismatch between SQL and actual schema.
+  - Parameter binding mismatch — `:#paramName` in SQL without a matching key in `parameters`.
+
+Common failures:
+- Missing `storeName` — the step will fail because it doesn't know which database to use.
+- Missing semicolon at end of SQL — may cause parse errors.
+- Using `steps.<ref>.output.result` directly instead of `steps.<ref>.output.result.rows` for SELECT data.
+- Forgetting to create the table before INSERT — use a separate SQLITE step with `CREATE TABLE IF NOT EXISTS`.
+
+## Minimal Process Definition Example
+
+```json
+{
+  "orgId": 1,
+  "namespace": "default",
+  "name": "sqlite-query-example",
+  "version": 1,
+  "type": "API_ORCHESTRATION",
+  "description": "Query a SQLite table by name parameter.",
+  "configuration": null,
+  "steps": [
+    {
+      "orgId": 1,
+      "namespace": "default",
+      "name": "select",
+      "type": "SQLITE",
+      "ref": "select_1",
+      "optional": false,
+      "createdBy": "system",
+      "updatedBy": "system",
+      "description": null,
+      "label": null,
+      "created": 1700000000000,
+      "updated": 1700000000000,
+      "configuration": {
+        "errorPolicyName": null,
+        "useCache": false,
+        "cacheKey": null,
+        "cacheTimeoutSeconds": 0,
+        "stream": false,
+        "streamAllStatuses": false,
+        "preExecutionScript": null,
+        "constructInputFromScript": false,
+        "scriptLanguage": null,
+        "jqTransformer": null,
+        "rateLimitMaxRequests": 0,
+        "rateLimitWindowSeconds": 0
+      },
+      "children": [],
+      "input": {
+        "storeName": "my-database",
+        "sql": "select * from sample_table where name = :#name;",
+        "parameters": {
+          "name": "Alice"
+        }
+      },
+      "output": null
+    }
+  ],
+  "defaultInput": null,
+  "defaultOutput": null,
+  "outputMapping": null,
+  "signature": null,
+  "metadata": null,
+  "tags": null,
+  "dependencies": null,
+  "dependents": null
+}
+```
+
+## Minimal Executed Step Example
+
+```json
+{
+  "id": 765971854,
+  "processId": 765971852,
+  "ref": "select_1",
+  "namespace": "automated_tests",
+  "name": "select",
+  "type": "SQLITE",
+  "status": "COMPLETED",
+  "input": {
+    "storeName": "automated-test-db",
+    "sql": "select * from sample_table where name = :#name;",
+    "parameters": {
+      "name": "Alice"
+    },
+    "__currentExecutionStartTime": 1765971852500
+  },
+  "output": {
+    "result": {
+      "rowsAffected": 0,
+      "rows": [
+        {
+          "name": "Alice",
+          "id": 1,
+          "age": 25
+        }
+      ]
+    }
+  }
+}
+```
+
+## Example: Create Table Then Insert and Query
+
+```json
+{
+  "steps": [
+    {
+      "type": "SQLITE",
+      "ref": "create_table_1",
+      "name": "create_table",
+      "input": {
+        "storeName": "my-database",
+        "sql": "create table if not exists users (id integer primary key autoincrement, name text not null, age integer);",
+        "parameters": {}
+      },
+      "children": [],
+      "...": "..."
+    },
+    {
+      "type": "SQLITE",
+      "ref": "insert_user_1",
+      "name": "insert_user",
+      "input": {
+        "storeName": "my-database",
+        "sql": "insert into users (name, age) values (:#name, :#age);",
+        "parameters": {
+          "name": "Bob",
+          "age": 30
+        }
+      },
+      "children": [],
+      "...": "..."
+    },
+    {
+      "type": "SQLITE",
+      "ref": "query_users_1",
+      "name": "query_users",
+      "input": {
+        "storeName": "my-database",
+        "sql": "select * from users;",
+        "parameters": {}
+      },
+      "children": [],
+      "...": "..."
+    }
+  ]
+}
+```
+
+## Example: Using SQLITE Output in a JAVASCRIPT Step
+
+```json
+{
+  "type": "JAVASCRIPT",
+  "ref": "process_results_1",
+  "name": "process_results",
+  "input": {
+    "script": "(steps, context) => {\n  const rows = steps.select_1.output.result.rows || [];\n  return {\n    count: rows.length,\n    names: rows.map(r => r.name)\n  };\n}"
+  },
+  "children": [],
+  "...": "..."
+}
+```