@unstable-dev/unmeshed-mcp 0.1.0

package/knowledge/SKILL.md

@@ -0,0 +1,359 @@
+---
+name: unmeshed-workflow
+description: >
+  Generate valid Unmeshed process workflow JSON definitions. Use this skill
+  whenever the user says "create a workflow", "build a workflow", "generate a
+  workflow", "design a workflow", or describes any automation, orchestration,
+  or multi-step process they want to run in Unmeshed. Also trigger when the
+  user asks to add steps to an existing workflow, modify an existing workflow
+  JSON, or convert a business process description into an Unmeshed-compatible
+  JSON. This skill must be used any time Unmeshed workflow generation or
+  editing is involved — even if the user's phrasing is indirect, like "I want
+  to automate X using Unmeshed" or "how would I build X in Unmeshed".
+---
+
+# Unmeshed Workflow Generator
+
+Converts plain-language process descriptions into valid Unmeshed
+`API_ORCHESTRATION` workflow JSON, following the exact schema and conventions
+derived from real Unmeshed workflow samples.
+
+Always read `references/step-types.md` and `references/integrations.md` before
+generating any workflow. Also read `assets/patterns.md` for ready-to-use
+scaffolds.
+
+---
+
+## Clarification Rules (ALWAYS follow before generating)
+
+Before generating any workflow JSON, ask the user for any missing answers to:
+
+1. **Workflow name** — a kebab-case identifier (e.g. `send-welcome-email`)
+2. **Namespace** — defaults to `"default"` if not specified
+3. **Description** — a one-sentence summary of what it does
+4. **Trigger / entry point** — what starts this workflow? (API call, form submission, schedule, another workflow, etc.)
+5. **Steps** — what should each step do, in order?
+6. **Integrations needed** — which external services are involved? (Google Sheets, Drive, Slack, LLM Claude, Notion, PostgreSQL, HTTP APIs, etc.)
+7. **Branching logic** — are there any if/else conditions, switch branches, or parallel paths?
+8. **Data flow** — what data passes from step to step? Ask about key field names if unclear.
+9. **Error handling** — should any steps retry on failure, or exit cleanly with a FAILED status?
+10. **Caching** — are any expensive integration calls (e.g. LLM) safe to cache? If so, what's the cache key?
+
+Do NOT generate JSON until you have enough answers to produce a correct workflow.
+If a question is not critical (e.g. description), use a sensible default and note it.
+
+---
+
+## Output Format
+
+Always output a single, complete, valid JSON object. Follow the top-level schema:
+
+```json
+{
+  "orgId": 1,
+  "namespace": "<namespace>",
+  "name": "<kebab-case-name>",
+  "version": 1,
+  "type": "API_ORCHESTRATION",
+  "description": "<description or null>",
+  "configuration": null,
+  "steps": [
+    /* array of step objects */
+  ],
+  "defaultInput": null,
+  "defaultOutput": null,
+  "outputMapping": null,
+  "signature": null,
+  "metadata": null,
+  "tags": null,
+  "dependencies": null,
+  "dependents": null
+}
+```
+
+**Top-level optional fields:**
+
+- `defaultInput`: provide a JSON object as an example payload for testing (useful for workflows with complex inputs)
+- `outputMapping`: a JQ expression string that reshapes the final workflow output; use when the consumer expects a specific response shape
+- `tags`: always `null` — do not populate this field
+
+After the JSON, always add a **Workflow Summary** section explaining:
+
+- What each step does in plain English
+- The data flow between steps
+- Any assumptions made
+- Any placeholders the user must replace (e.g. connection names, spreadsheet IDs, folder IDs)
+
+---
+
+## Step Construction Rules
+
+Every step — at every nesting level — must include this full structure:
+
+```json
+{
+  "orgId": 1,
+  "namespace": "<same as workflow>",
+  "name": "<snake_case_name>",
+  "type": "<STEP_TYPE>",
+  "ref": "<unique_ref_id>",
+  "optional": false,
+  "createdBy": "system",
+  "updatedBy": "system",
+  "description": "<string or null>",
+  "label": null,
+  "created": 1700000000000,
+  "updated": 1700000000000,
+  "configuration": {
+    /* standard config block */
+  },
+  "children": [],
+  "input": {
+    /* step-specific input */
+  },
+  "output": null
+}
+```
+
+**Standard `configuration` block** (use for every step unless noted):
+
+```json
+{
+  "errorPolicyName": null,
+  "useCache": false,
+  "cacheKey": null,
+  "cacheTimeoutSeconds": 0,
+  "stream": false,
+  "streamAllStatuses": false,
+  "preExecutionScript": null,
+  "constructInputFromScript": false,
+  "scriptLanguage": null,
+  "jqTransformer": null,
+  "rateLimitMaxRequests": 0,
+  "rateLimitWindowSeconds": 0
+}
+```
+
+**Configuration field guidance:**
+
+- `errorPolicyName`: set to a named error policy string (e.g. `"retry-3x"`) if the step should retry on failure; otherwise `null`
+- `useCache`: set to `true` to cache step output; pair with a `cacheKey` and `cacheTimeoutSeconds`
+- `cacheKey`: a string key (can include `{{...}}` template variables) — e.g. `"llm-classify-{{steps.map_input.output.result.category}}"`
+- `cacheTimeoutSeconds`: how long to keep the cached result (e.g. `3600` = 1 hour); only meaningful when `useCache: true`
+- `stream` / `streamAllStatuses`: set `true` for LLM streaming responses (advanced use)
+- `jqTransformer`: a JQ expression to post-process the raw step output before it is stored
+- `rateLimitMaxRequests` / `rateLimitWindowSeconds`: set both to non-zero to throttle a step (e.g. `10` requests per `60` seconds)
+
+**`ref` naming convention**: Use `<name>_<index>` for integrations/stateful steps (e.g. `google_sheets_1`, `llm_claude_1`). Use the plain `name` value for script/logic steps (e.g. `preprocessing`, `format_resume_link`).
+
+**Variable references** between steps must use the producer step's output shape:
+
+- Native `HTTP`: `{{steps.<ref>.output.response.<field>}}` or `steps.<ref>.output.response` in scripts
+- Native `HTTP` status code: `{{steps.<ref>.output.statusCode}}`
+- `JAVASCRIPT`: `{{steps.<ref>.output.result.<field>}}` or `steps.<ref>.output.result` in scripts
+- `PYTHON`: `{{steps.<ref>.output.result.<field>}}` or `steps.<ref>.output.result` in scripts
+- `NOOP`: `{{steps.<ref>.output.<field>}}` (not wrapped in `result`)
+- `INTEGRATION`: use the integration-specific documented path, commonly `output.result` or `output.results`
+
+Never reference a native `HTTP` step as `steps.<http_ref>.output.result`.
+For example, after an `HTTP` step with ref `fetch_posts_1`, use:
+
+```javascript
+(steps, context) => {
+  const posts = steps.fetch_posts_1.output.response || [];
+  return { count: Array.isArray(posts) ? posts.length : 0 };
+}
+```
+
+---
+
+## Step Types Reference
+
+Read `references/step-types.md` for full details on all step types.
+Read `references/integrations.md` for integration `input` schemas.
+
+### Quick Reference
+
+| Type              | Use when                                                                                                        |
+| ----------------- | --------------------------------------------------------------------------------------------------------------- |
+| `JAVASCRIPT`      | Data mapping, transformation, normalization, assertions                                                         |
+| `PYTHON`          | File processing, filesystem access, binary/base64 ops                                                           |
+| `NOOP`            | Static config/constants passed as output                                                                        |
+| `INTEGRATION`     | Calling external services (Google, Slack, LLM, MongoDB, Redis, Airtable, Jira, Outlook, Notion, HTTP, DB, etc.) |
+| `HTTP`            | Direct HTTP calls with optional polling/repeat-until support                                                    |
+| `WAIT`            | Pause execution for N seconds (e.g. wait for email delivery)                                                    |
+| `FOREACH`         | Iterate over an array, running child steps per element                                                          |
+| `WHILE`           | Loop while a condition is true; use `whileloop.iteration` counter                                               |
+| `DECISION_ENGINE` | Rule-table based routing                                                                                        |
+| `SWITCH`          | Conditional branching (if/else, multi-branch, rate-limit handling)                                              |
+| `LIST`            | Sequential group of steps — branch body inside SWITCH or PARALLEL                                               |
+| `PARALLEL`        | Run multiple LIST branches or SUB_PROCESS steps simultaneously                                                  |
+| `SUB_PROCESS`     | Invoke another named workflow                                                                                   |
+| `PERSISTED_STATE` | Read/write persistent key-value state                                                                           |
+| `EXIT`            | Terminate workflow with a status and message                                                                    |
+| `FAIL`            | Immediately fail the workflow/branch with a reason                                                              |
+| `DEPENDSON`       | Wait for an external process or step to reach a target status (cross-workflow sync)                             |
+| `UPDATE_STEP`     | Programmatically update the status/output of a step in another running workflow (callbacks, approvals)          |
+| `SQLITE`          | Execute SQL (SELECT/INSERT/UPDATE/DELETE/DDL) against a named SQLite datastore                                  |
+
+---
+
+## Common Patterns
+
+Read `assets/patterns.md` for full scaffolds. Quick reference:
+
+### Pattern A — Linear pipeline
+
+`NOOP (config) → JAVASCRIPT (preprocess) → INTEGRATION → JAVASCRIPT (format) → INTEGRATION`
+
+### Pattern B — LLM classification then routing
+
+`JAVASCRIPT (map fields) → INTEGRATION (llm-claude) → DECISION_ENGINE`
+
+### Pattern C — State-gated branch (create or reuse resource)
+
+`PERSISTED_STATE (read) → SWITCH → [LIST branch new, LIST branch existing]`
+
+### Pattern D — Parallel sub-process fan-out
+
+`SWITCH (gate check) → PARALLEL (children: SUB_PROCESS steps)`
+
+### Pattern E — Classify → Store always → Conditionally notify
+
+`JAVASCRIPT → INTEGRATION (llm) → DECISION_ENGINE → JAVASCRIPT (format) → INTEGRATION (sheets) → SWITCH (notify?)`
+
+### Pattern F — HTTP API Call → Normalize → Store
+
+`JAVASCRIPT (build payload) → HTTP → JAVASCRIPT (normalize) → INTEGRATION (db/sheets)`
+
+### Pattern G — Error-Aware Exit Branches
+
+`JAVASCRIPT (validate) → SWITCH → [EXIT on invalid, LIST on valid]`
+
+### Pattern H — WAIT for Async Side Effect
+
+`SUB_PROCESS (trigger action) → WAIT (N seconds) → INTEGRATION (verify) → JAVASCRIPT (assert)`
+
+### Pattern I — FOREACH Loop
+
+`FOREACH (inputArray, concurrency) → LIST → [steps] → JAVASCRIPT (collect results) → JAVASCRIPT (assert)`
+
+### Pattern J — PARALLEL with inline LIST branches
+
+`PARALLEL → [LIST branch_1, LIST branch_2, ...]` — branches are inline, not sub-processes
+
+### Pattern K — Rate-Limit Aware Integration
+
+`INTEGRATION (optional:true, retry) → SWITCH (check for 429 error) → [skip branch, main branch]`
+
+### Pattern L — WHILE Loop with State Counter
+
+`WHILE (iteration < N) → LIST → [WAIT (throttle), JAVASCRIPT (__statePut)]`
+
+### Pattern M — Error Policy Retry Check
+
+`STEP (optional:true, errorPolicyName) → JAVASCRIPT (assert executionList.length or check output.error)`
+
+### Pattern N — Variables and Secrets in HTTP
+
+Use `{{variables.*}}` for base URLs/config and `{{secrets.*}}` for tokens — never hardcode
+
+### Pattern O — Cross-Branch Synchronisation with DEPENDSON
+
+`PARALLEL → [LIST (WAIT → HTTP), LIST (WAIT → DEPENDSON waits for HTTP)]` — use DEPENDSON to sync across sibling PARALLEL branches
+
+### Pattern P — Guard with FAIL
+
+`JAVASCRIPT (validate) → SWITCH → [LIST (happy path), FAIL (error path)]` — FAIL terminates the branch/process immediately
+
+### Pattern Q — External Callback with UPDATE_STEP
+
+`Workflow A: HTTP → WAIT (paused)` ... `Workflow B: UPDATE_STEP (completes WAIT in A with callback data)`
+
+### Pattern R — Human Task Approval with UPDATE_STEP
+
+`INTEGRATION (send approval email) → WAIT (paused)` ... `Approval workflow: UPDATE_STEP (BY_COMPLETION_TOKEN, completes WAIT)`
+
+### Pattern S — SQLITE CRUD Pipeline
+
+`SQLITE (create table) → SQLITE (insert row) → SQLITE (select) → JAVASCRIPT (process rows)`
+
+---
+
+## Key Rules
+
+- `name` fields are `snake_case` or `kebab-case` consistently within the workflow
+- `ref` values must be **unique** across the entire workflow including nested children
+- `optional: true` on a step allows the workflow to continue even if that step fails — use for best-effort steps or rate-limit-prone integrations
+- SWITCH `children` can be **any step type** — `LIST`, `INTEGRATION`, `NOOP`, `EXIT`, `JAVASCRIPT`. Do NOT wrap single-step branches in LIST unnecessarily.
+- SWITCH `input.responseMapping` must have a `defaultBranch: true` entry
+- PARALLEL children can be **LIST steps** (inline multi-step branches) OR **SUB_PROCESS steps** (sub-workflow fan-out) — not just SUB_PROCESS
+- PARALLEL `input` must include `"failIfAnyBranchFails": true` (strict) or `false` (lenient)
+- FOREACH `input` requires `inputArray` and `concurrency`; access iteration outputs as `steps["<ref>[N]"].output`
+- WHILE `input.script` returns a boolean; use `whileloop.iteration` for the loop counter; always include a WAIT inside the loop body
+- WAIT inside a WHILE loop: use `steps.<while_ref>.updated + N` as the base time (not `steps.__self.startTime`)
+- `context.state.<key>` — read mutable in-run state; write via returning `{ "__statePut": { key: value } }` from any JAVASCRIPT step
+- `{{variables.<name>}}` and `{{secrets.<name>}}` — inject env vars and secrets in HTTP URLs, headers, params, body
+- `steps.__self.executionList.length` — number of retry attempts so far (populated by error policies)
+- `steps.<ref>.output.error` — error message string from a failed optional step; check with `.includes()`
+- `steps.<ref>.input.<field>` — access a prior step's INPUT (not output) from a later step; e.g. `steps.http_step.input.body.content.userId`
+- WAIT `input.script` must return `{ waitUntil: steps.__self.startTime + (N * 1000) }`
+- HTTP (native step type, uppercase) supports `repeatUntilEnabled` polling; use for internal API calls with retry logic
+- SUB_PROCESS `input` must include `processName` and `waitForCompletion`
+- PERSISTED_STATE `input` must include `operation` (`READ` or `UPSERT`), `name`, and `path`
+- EXIT `input` must include `message` and `status` (`"COMPLETED"` or `"FAILED"`)
+- FAIL `input` must include `reason` (string) — always produces a `FAILED` status; use for error branches, guard failures
+- FAIL vs EXIT: FAIL always fails and uses `reason`; EXIT supports both `COMPLETED`/`FAILED` and uses `message` + `status`
+- DEPENDSON `input` must include `dependsOnStatement` (SQL-style expression) and `intervalSeconds` (polling interval)
+- DEPENDSON statement syntax: `STEP('process-name', 'step-ref', 'STATUS')` or `PROCESS('process-name', 'STATUS')` — combine with `AND` / `OR`
+- DEPENDSON output: check `steps.<ref>.output.__repeatConditionMatched` (bool) and `steps.<ref>.output.__dependsOnOutput.result` (bool)
+- DECISION_ENGINE `input` must include `decisionTable`, `decisionContext`, `decisionRuleStrategy` (`FIRST_MATCH` or `ALL_MATCH`), and `decisionOutputColumns`
+- DECISION_ENGINE output: `FIRST_MATCH` returns `steps.<ref>.output.result.<column>` (single object); `ALL_MATCH` returns `steps.<ref>.output.result` as an array
+- UPDATE_STEP `input` must include `inputMatchType` (`BY_PROCESS_ID_REF`, `BY_CORRELATION_ID_REF`, `BY_REQUEST_ID_REF`, `BY_STEP_ID`, `BY_COMPLETION_TOKEN`)
+- UPDATE_STEP: `stepStatus` must be `COMPLETED`, `FAILED`, or `RUNNING` (defaults to `COMPLETED`); `output` is merged into the target step's output
+- UPDATE_STEP: `maxWaitSeconds` (default 24h) controls how long the step polls before failing; engine polls every 30s
+- UPDATE_STEP: target steps must be in an updatable status (SCHEDULED, RUNNING) — already terminal steps cannot be updated
+- SQLITE `input` must include `storeName` (database name) and `sql` (SQL statement ending with `;`)
+- SQLITE uses `:#paramName` parameter binding syntax — never interpolate values into SQL strings
+- SQLITE SELECT output: `steps.<ref>.output.result.rows` (array of row objects), `steps.<ref>.output.result.rowsAffected` (always 0 for SELECT)
+- SQLITE INSERT/UPDATE/DELETE output: `steps.<ref>.output.result.rowsAffected` (count), `steps.<ref>.output.result.rows` (empty `[]`)
+- SQLITE `parameters` must be a flat key-value object — each key matches a `:#key` placeholder in `sql`
+- LLM Claude output is under `.results` (plural) — always defensively read: `steps.<ref>.output.results || steps.<ref>.output.result || {}`
+- MongoDB output is under `.results.<field>` (directly on results, not nested in `.result`)
+- Redis GET of a JSON object exposes fields directly on `.results` (e.g. `.results.name`, `.results.age`)
+- Slack `input.type` is `"slack-messaging"` — build message string in a preceding JAVASCRIPT step
+- Notion operations all go in `messageBody.operation`; `publishProperties` is `{}`
+- Jira operations all go in `publishProperties.action`; `messageBody` is `{}`
+- Airtable `pageSize` and `maxRecords` are strings: `"100"` not `100`
+- Outlook `receivedFromMinutesAgo` is a string: `"60"` not `60`; attachments land at `/app/files/_outlook_attachments_/`
+- Always add WAIT after SUB_PROCESS when the sub-process triggers async effects (emails, webhooks) before reading results
+- When a step should always run (e.g. log/store), put it **before** a SWITCH — the SWITCH only handles conditional paths after
+- Always list placeholders in the Workflow Summary (connection names, IDs, folder IDs, etc.)
+
+---
+
+## Data Flow Cheatsheet
+
+```
+context.input.<field>                        ← workflow API input
+steps.<ref>.output.response.<field>          ← native HTTP response body
+steps.<ref>.output.statusCode                ← native HTTP status code
+steps.<ref>.output.result.<field>            ← JAVASCRIPT / PYTHON / most INTEGRATION output
+steps.<ref>.output.results.<field>           ← LLM Claude output (note: results plural)
+steps.<ref>.output.<field>                   ← NOOP output (no .result wrapper)
+steps.<ref>.output.spreadsheetId            ← Google Sheets CREATE output
+steps.<ref>.output.result.rows              ← PostgreSQL / MySQL SELECT output
+steps.<ref>.output.id                       ← Google Drive upload output (file ID)
+steps.<ref>.output.webViewLink              ← Google Drive upload output (share URL)
+steps.<ref>.output.result.<column>          ← DECISION_ENGINE FIRST_MATCH output
+steps.<ref>.output.result[N].<column>       ← DECISION_ENGINE ALL_MATCH output (array)
+steps.<ref>.output.__dependsOnOutput.result ← DEPENDSON boolean result (true/false)
+steps.<ref>.output.__repeatConditionMatched ← DEPENDSON matched before cancel (true/false)
+steps.<ref>.output.reason                   ← FAIL step reason string
+steps.<ref>.output.__updateStepResults       ← UPDATE_STEP map of processId:stepId → status message
+steps.<ref>.output.__updateStepMetadata      ← UPDATE_STEP array of update detail objects
+steps.<ref>.output.result.rows               ← SQLITE SELECT row data
+steps.<ref>.output.result.rowsAffected       ← SQLITE write affected count
+steps.__self.id                              ← current step ID (inside scripts)
+context.id                                   ← workflow process/run ID
+```