@unstable-dev/unmeshed-mcp 0.1.0

package/knowledge/steps/WHILE.md

@@ -0,0 +1,328 @@
+# While Step Schema
+
+`WHILE` is a container step that executes exactly one child step repeatedly while
+a JavaScript condition returns `true`. When the condition returns `false`, the
+loop exits.
+
+Use WHILE when the same operation needs to repeat until a runtime condition is
+met, such as polling, paginating, transforming repeated inputs, or calling a
+subprocess until work is complete.
+
+## Definition Input Schema
+
+```json
+{
+  "script": "// (steps, context) will be provided as default inputs \n(steps, context) => {\n  return whileloop.iteration < 3;\n}"
+}
+```
+
+Required:
+- `script`: JavaScript function receiving `(steps, context)`.
+- Script must return boolean `true` or `false`.
+- `true` means execute the child step.
+- `false` means exit the loop.
+
+Injected keyword:
+- `whileloop.iteration`: zero-based counter for the current loop execution.
+
+The WHILE step must have exactly one child from the WHILE step's point of view.
+If an iteration needs multiple steps, wrap them in a child `LIST`. If an
+iteration should call another workflow, use a child `SUB_PROCESS`.
+
+## Runtime Output Schema
+
+The WHILE container output records the final condition result and final
+iteration count.
+
+```json
+{
+  "result": false,
+  "iteration": 3,
+  "logs": []
+}
+```
+
+The WHILE step execution list contains prior condition evaluations:
+
+```json
+[
+  {
+    "output": {
+      "result": true,
+      "iteration": 0,
+      "logs": []
+    }
+  },
+  {
+    "output": {
+      "result": true,
+      "iteration": 1,
+      "logs": []
+    }
+  },
+  {
+    "output": {
+      "result": true,
+      "iteration": 2,
+      "logs": []
+    }
+  }
+]
+```
+
+Child steps inside WHILE expose only the latest iteration on their top-level
+`output`. Earlier iteration outputs are stored in the child step's
+`executionList`.
+
+## Output Access Paths
+
+Use:
+- `steps.<while_ref>.output.result` for the final loop condition result.
+- `steps.<while_ref>.output.iteration` for the final iteration count.
+- `steps.<while_ref>.executionList` for condition outputs from prior loop checks.
+- `steps.<child_ref>.output` for the latest child iteration output.
+- `steps.<child_ref>.executionList` for earlier child iteration outputs.
+- `steps.<http_child_ref>.output.response` for the latest native HTTP child response.
+
+Do not use:
+- `steps.<while_ref>.output.response`
+- `steps.<while_ref>.output.<child_ref>`
+- `steps.<while_ref>.children.<child_ref>.output`
+- `steps["<child_ref>[0]"].output` for WHILE iteration output; that indexed syntax is for FOREACH, not WHILE.
+
+## Generation Rules
+
+- Use uppercase step type: `"WHILE"`.
+- Put JavaScript code in `input.script`.
+- The script must accept `(steps, context)`.
+- The script must return boolean `true` or `false`.
+- Use `whileloop.iteration` for the zero-based loop counter.
+- Include a clear false/exit condition to avoid endless loops.
+- WHILE should have exactly one child step.
+- Use a child `LIST` if one iteration needs multiple sequential steps.
+- Use a child `SUB_PROCESS` if the repeated operation is another workflow.
+- Nested WHILE loops are supported, but avoid unnecessary depth.
+- Use output/state carefully to carry data between iterations.
+- Prefer Unmeshed State Management when loop state must be process-level or durable.
+
+## State And Iteration Output Rules
+
+Inside loop child scripting steps:
+- `steps.__self` refers to the currently running step.
+- The previous iteration output is available to the step on the next run.
+- A script can carry state by reading its previous output and returning updated output.
+
+Example state update inside a JavaScript child step:
+
+```javascript
+(steps, context) => {
+  let currentSum = steps.__self?.output?.result?.sum || 0;
+  currentSum += 10;
+  return { sum: currentSum };
+}
+```
+
+To collect every iteration output after WHILE, add a JavaScript step after the
+loop and combine `executionList` plus the top-level final output:
+
+```javascript
+(steps, context) => {
+  const results = [];
+  const execs = steps.http_1.executionList ?? [];
+  for (const e of execs.slice(0, -1)) {
+    if (e.output) {
+      results.push(e.output);
+    }
+  }
+  const finalOutput = steps.http_1.output;
+  if (finalOutput) {
+    results.push(finalOutput);
+  }
+  return { while_iterations_output: results };
+}
+```
+
+## Debugging Rules
+
+When debugging a WHILE:
+- Inspect the WHILE step's `input.script`.
+- Inspect `output.result` and `output.iteration`.
+- Confirm the script eventually returns `false`.
+- Inspect `input.__whileLoop` on the executed WHILE step when available.
+- Inspect child step `executionList` for previous iteration outputs.
+- Remember that the child top-level `output` is only the latest iteration output.
+- If the process appears stuck, check whether the loop condition can ever become false.
+- If multiple operations are needed per iteration, verify the child is a `LIST`.
+
+Common failures:
+- Endless loop because the condition never returns `false`.
+- Script returns a non-boolean value.
+- Missing or invalid use of `whileloop.iteration`.
+- Multiple direct children under WHILE instead of one child `LIST`.
+- Using FOREACH indexed output syntax for WHILE iteration outputs.
+- Assuming child top-level output contains all iteration outputs.
+- Forgetting that the latest run output may be top-level while prior outputs live in `executionList`.
+
+## Minimal Process Definition Example
+
+```json
+{
+  "orgId": 1,
+  "namespace": "default",
+  "name": "while_test_gulam",
+  "version": 1,
+  "type": "API_ORCHESTRATION",
+  "description": null,
+  "configuration": null,
+  "steps": [
+    {
+      "orgId": 1,
+      "namespace": "default",
+      "name": "while",
+      "type": "WHILE",
+      "ref": "while_1",
+      "optional": false,
+      "createdBy": "system",
+      "updatedBy": "system",
+      "description": null,
+      "label": null,
+      "created": 1700000000000,
+      "updated": 1700000000000,
+      "configuration": {
+        "errorPolicyName": null,
+        "useCache": false,
+        "cacheKey": null,
+        "cacheTimeoutSeconds": 0,
+        "stream": false,
+        "streamAllStatuses": false,
+        "preExecutionScript": null,
+        "constructInputFromScript": false,
+        "scriptLanguage": null,
+        "jqTransformer": null,
+        "rateLimitMaxRequests": 0,
+        "rateLimitWindowSeconds": 0
+      },
+      "children": [
+        {
+          "orgId": 1,
+          "namespace": "default",
+          "name": "http",
+          "type": "HTTP",
+          "ref": "http_1",
+          "optional": false,
+          "createdBy": "system",
+          "updatedBy": "system",
+          "description": null,
+          "label": null,
+          "created": 1700000000000,
+          "updated": 1700000000000,
+          "configuration": {
+            "errorPolicyName": null,
+            "useCache": false,
+            "cacheKey": null,
+            "cacheTimeoutSeconds": 0,
+            "stream": false,
+            "streamAllStatuses": false,
+            "preExecutionScript": null,
+            "constructInputFromScript": false,
+            "scriptLanguage": null,
+            "jqTransformer": null,
+            "rateLimitMaxRequests": 0,
+            "rateLimitWindowSeconds": 0
+          },
+          "children": [],
+          "input": {
+            "method": "GET",
+            "url": "http://localhost:8080/api/test/get",
+            "headers": {
+              "Content-Type": "application/json",
+              "Accept": "application/json",
+              "Authorization": "Bearer {{secrets.test_token}}"
+            },
+            "params": {
+              "sampleKey": "sampleValue"
+            },
+            "repeatUntilEnabled": null,
+            "repeatUntilCondition": {
+              "script": "(steps, context) => {\n  return steps.__self.output.response.counter === 100;\n}"
+            },
+            "repeatIntervalSeconds": null,
+            "maxRepeatCount": null,
+            "includeFullResponseString": false,
+            "noEncode": false,
+            "extraLongTimeouts": false
+          },
+          "output": null
+        }
+      ],
+      "input": {
+        "script": "// (steps, context) will be provided as default inputs \n(steps, context) => {\n  return whileloop.iteration < 3;\n}"
+      },
+      "output": null
+    }
+  ],
+  "defaultInput": null,
+  "defaultOutput": null,
+  "outputMapping": null,
+  "signature": null,
+  "metadata": null,
+  "tags": null,
+  "dependencies": null,
+  "dependents": null
+}
+```
+
+## Minimal Executed Step Example
+
+```json
+{
+  "id": 592668723,
+  "processId": 592668721,
+  "ref": "while_1",
+  "namespace": "default",
+  "name": "while",
+  "type": "WHILE",
+  "status": "COMPLETED",
+  "input": {
+    "script": "// (steps, context) will be provided as default inputs \n(steps, context) => {\n  return whileloop.iteration < 3;\n}",
+    "__whileLoop": {
+      "ticketId": "while_1",
+      "iteration": 3,
+      "loopComplete": true,
+      "executedTickets": []
+    },
+    "__currentExecutionStartTime": 1778186238199
+  },
+  "output": {
+    "result": false,
+    "iteration": 3,
+    "logs": []
+  }
+}
+```
+
+Child HTTP latest output example:
+
+```json
+{
+  "ref": "http_1",
+  "parentRef": "while_1",
+  "type": "HTTP",
+  "input": {
+    "__iteration": 2,
+    "__tickets": [
+      "while_1:592668723:0",
+      "while_1:592668723:1",
+      "while_1:592668723:2"
+    ]
+  },
+  "output": {
+    "response": {
+      "counter": 6492,
+      "randomId": "5efb3a97-5207-4446-a10c-711823d0ed7e"
+    },
+    "statusCode": 200
+  }
+}
+```
+

package/knowledge/steps/WORKER.md

@@ -0,0 +1,233 @@
+# Worker Step Schema
+
+`WORKER` delegates execution to an external worker node instead of running code
+inside the Unmeshed Engine. Use it when work must run outside Unmeshed, such as
+when organization policy prevents Unmeshed from accessing data directly, when
+business logic already exists in an external service, or when built-in steps do
+not provide the required capability.
+
+Worker nodes are external clients that connect to Unmeshed, poll for work, run
+worker methods, and return results. Unmeshed still schedules, tracks, retries,
+reschedules, and times out worker steps according to process needs and error
+policies.
+
+## Definition Input Schema
+
+Worker input is the object passed to the worker method. If the worker method has
+an input parameter, all fields under the step `input` are used to build that
+object.
+
+```json
+{
+  "myString": "myValue"
+}
+```
+
+Minimal observed worker step:
+
+```json
+{
+  "type": "WORKER",
+  "name": "worker",
+  "namespace": "default",
+  "ref": "worker_1",
+  "optional": false,
+  "createdBy": "system",
+  "updatedBy": "system",
+  "created": 1700000000000,
+  "updated": 1700000000000,
+  "children": [],
+  "input": {},
+  "configuration": {
+    "rateLimitMaxRequests": null,
+    "rateLimitWindowSeconds": null
+  }
+}
+```
+
+Important:
+- Some worker SDK languages may fail if extra input fields are included.
+- Match the worker method input shape exactly when known.
+- If the worker method accepts no parameter, use `input: {}`.
+
+## Runtime Output Schema
+
+If the worker method returns an object, its fields are exposed directly under
+the step `output`.
+
+```json
+{
+  "myInteger": 1
+}
+```
+
+Observed manual/completion-style output:
+
+```json
+{
+  "__completionReason": "No reason provided",
+  "__actedBy": "gulam@unmeshed.com"
+}
+```
+
+Worker output is not automatically wrapped in `result`.
+
+## Output Access Paths
+
+Use:
+- `steps.<ref>.output.<field>` for fields returned by the worker method.
+- `steps.<ref>.output.__completionReason` for observed completion reason metadata.
+- `steps.<ref>.output.__actedBy` for observed actor metadata.
+- `stepRecords[].workerId` when debugging which worker handled the step.
+- `stepRecords[].executionList` when debugging polling/rescheduling behavior.
+
+Do not use:
+- `steps.<ref>.output.result` unless the worker itself explicitly returned a field named `result`.
+- `steps.<ref>.output.response`
+- `steps.<ref>.output.logs` unless the worker itself explicitly returned a field named `logs`.
+
+## Generation Rules
+
+- Use uppercase step type: `"WORKER"`.
+- Use `children: []`; WORKER is not a container.
+- Put only the worker method input fields in `input`.
+- Avoid adding extra input fields unless the worker method accepts them.
+- Worker methods can receive up to one input parameter.
+- Worker methods can optionally return a result object.
+- Returned object fields become direct output fields.
+- Use error policies/timeouts when work must complete within a threshold.
+- Do not generate worker implementation code inside the process definition; the worker runs externally.
+- Do not assume a worker is available unless the user confirms the worker service is connected.
+
+## Scheduling And Rescheduling
+
+Unlike regular built-in steps, worker nodes may return an incomplete result. If
+timeout thresholds have not been met, Unmeshed can reschedule the worker step so
+a client can execute it later.
+
+Debugging clues:
+- `executionList[].scheduled` shows when work was scheduled.
+- `executionList[].polled` shows when a worker/client picked it up.
+- `executionList[].executor` may identify the executor, or be `null`.
+- `runs` may be `0` if work was completed/updated without a normal worker run.
+- `workerId` may be `null` when no specific worker id is recorded.
+
+## Debugging Rules
+
+When debugging a WORKER:
+- Inspect the step `input` and compare it to the worker method's expected input shape.
+- Inspect direct fields under `output`.
+- Check `workerId`, `executionList`, `scheduled`, `polled`, `start`, and `updated`.
+- If the step did not run, check whether a worker node is connected and polling.
+- If the step timed out, inspect error policy/time thresholds.
+- If the worker failed due to deserialization, look for extra or missing input fields.
+- If output seems missing, confirm whether the worker method returned an object.
+
+Common failures:
+- Extra input fields cause SDK/object mapping failures.
+- Worker service is not connected or not polling.
+- Worker method name/configuration does not match the registered worker capability.
+- Looking for output at `output.result` instead of direct output fields.
+- Timeout/error policy fails the step while a worker is still running.
+- Worker returns incomplete output and the step is rescheduled.
+
+## Minimal Process Definition Example
+
+```json
+{
+  "orgId": 1,
+  "namespace": "default",
+  "name": "kebab-case-name",
+  "version": 1,
+  "type": "API_ORCHESTRATION",
+  "description": "Delegate work to an external worker.",
+  "configuration": null,
+  "steps": [
+    {
+      "type": "WORKER",
+      "name": "worker",
+      "namespace": "default",
+      "ref": "worker_1",
+      "optional": false,
+      "createdBy": "system",
+      "updatedBy": "system",
+      "created": 1700000000000,
+      "updated": 1700000000000,
+      "children": [],
+      "input": {},
+      "configuration": {
+        "rateLimitMaxRequests": null,
+        "rateLimitWindowSeconds": null
+      }
+    }
+  ],
+  "defaultInput": null,
+  "defaultOutput": null,
+  "outputMapping": null,
+  "signature": null,
+  "metadata": null,
+  "tags": null,
+  "dependencies": null,
+  "dependents": null
+}
+```
+
+## Minimal Executed Step Example
+
+```json
+{
+  "id": 28800043,
+  "processId": 28800041,
+  "ref": "worker_1",
+  "namespace": "default",
+  "name": "worker",
+  "type": "WORKER",
+  "status": "COMPLETED",
+  "input": {},
+  "output": {
+    "__completionReason": "No reason provided",
+    "__actedBy": "gulam@unmeshed.com"
+  },
+  "workerId": null,
+  "start": 0,
+  "schedule": 1778185168004,
+  "updated": 1778185197573,
+  "optional": false,
+  "executionList": [
+    {
+      "id": 0,
+      "scheduled": 1778185168004,
+      "polled": 1778185197572,
+      "start": 1778185197572,
+      "updated": 1778185197573,
+      "executor": null,
+      "ref": "worker_1",
+      "runs": 0,
+      "output": {}
+    }
+  ]
+}
+```
+
+## Example Worker Input And Output
+
+If the worker method input object has one field named `myString`, the step input
+should be:
+
+```json
+{
+  "myString": "myValue"
+}
+```
+
+If the worker method returns an object with one field named `myInteger`, later
+steps should read:
+
+```javascript
+(steps, context) => {
+  return {
+    valueFromWorker: steps.worker_1.output.myInteger
+  };
+}
+```
+