@datanimbus/dnio-mcp 1.0.0

package/docs/tools/data-pipes.md

@@ -0,0 +1,286 @@
+# Data Pipes (Flows)
+
+Tools registered by `src/tools/data-pipes.js`. A **data pipe** (also called a **flow**) is a directed execution graph: a single **trigger node** (`inputNode`) followed by N **process nodes**, wired via `onSuccess` / `onError` arrays.
+
+Backed by `DataPipesClient` (`src/clients/data-pipes.js`) for flow CRUD, and reuses `PluginsClient` (`plugins.listInstalled` with category filters) for trigger/process discovery.
+
+Endpoint pattern: `/api/a/bm/${app}/flow[/…]`.
+
+## Mental model
+
+```
+┌─────────────┐        ┌───────────────┐        ┌────────────────┐
+│ inputNode   │ ──→    │ process node  │ ──→    │ V1_RESPONSE    │
+│ (trigger)   │        │ (parse, fetch,│        │ (optional end) │
+│             │        │  save, code…) │        │                │
+└─────────────┘        └───────────────┘        └────────────────┘
+       │                       │
+       └─→ onError…            └─→ onSuccess[].condition (branches)
+```
+
+- The trigger is one of `V1_HTTP_SERVER`, `V1_TIMER`, `V1_TIMER_MULTIPLE`.
+- Each node's `onSuccess` is an array of `{ _id, condition?, name?, color? }`. Multiple entries form parallel branches with conditions.
+- Each node's `mappings[]` describe how its inputs come from previous nodes' outputs (see Mappings below).
+- A flow that ends in `V1_RESPONSE` returns an HTTP body; otherwise it just terminates silently.
+
+## Always-full PUT
+
+The platform rejects partial flow updates. Every mutating tool in this domain follows the same pattern: **GET the current flow → mutate in memory → PUT the full document**. The tools handle the get/put internally so the LLM never needs to call `get_flow` first (though it can, for inspection).
+
+## Server-side plugin resolution
+
+`create_flow` and `add_node_to_flow` always **re-fetch** the plugin from the platform's `/my-node` endpoint on every call, so the persisted node always carries the canonical `inputSchema`, `outputSchema`, `errorSchema`, and `nodeId`. The LLM does not need to round-trip the full plugin object between tool calls — passing `{ _id: '<installed-plugin-id>' }` or `{ type: 'V1_PARSE_JSON' }` is enough. The full plugin object from `list_process_plugins` also works (and is used as-is when complete) — but if any of the schema fields is missing/empty, the tool re-resolves from the platform regardless. This guarantees the canonical schemas and prevents the LLM from accidentally clearing them by sending a slimmed-down object.
+
+## Mappings: simple vs full form
+
+Every mapping in `add_node_to_flow.mappings` and `update_node_in_flow.mappings` accepts **two forms** (mix freely in one array):
+
+**Simple form** — what the LLM should normally use:
+
+```json
+{ "from": "fetch_students.data", "to": "data" }
+```
+
+The tool walks the source node's `outputSchema` (via `findNodeRef(flow, 'fetch_students')`) and the current node's `inputSchema` to build the full DNIO mapping object — `target` (with `_id`, `type`, `dataPath`, `dataPathSegs`), `source[]` (with `key`, `name`, `nodeId`, `dataPath`, `dataPathSegs`, `_id`, `type`, `subType`), `expression: { type: 'simple', value: " {{nodeId['key']}}" }`, plus `children`, `key`, `name`, `derivedFrom`. You don't construct any of that by hand.
+
+For hardcoded expressions with no source, omit `from` and pass `expression`:
+
+```json
+{ "to": "authToken", "expression": "{{CONSTANTS.TOKEN}}" }
+```
+
+**Full form** — for advanced cases (nested children, iterators):
+
+Pass an entry that already has `target` as an object and `source` as an array. The tool detects this and uses it as-is. Use this when you need `children[]` for nested objects or `iterator` for arrays of objects.
+
+## Plugin selection strategy
+
+The tool descriptions explicitly tell the agent:
+
+1. Search for an installed first-class plugin matching the task (`Save File` → `V1_SAVE_FILE`, `Parse JSON` → `V1_PARSE_JSON`, etc.).
+2. If nothing matches but the marketplace has it: `list_marketplace_plugins` → `install_plugins` → re-list.
+3. **Only** if no built-in covers the task and the logic is trivial / glue / one-off: fall back to `V1_CODEBLOCK` with custom code in `options.code`.
+
+## Tool reference
+
+### `list_trigger_plugins`
+
+| | |
+|---|---|
+| **Purpose** | List installed plugins where `category=TRIGGER`. Source for `triggerPlugin` in `create_flow`. |
+| **Inputs** | (none) |
+| **Behaviour** | Calls `dnioClient.plugins.listInstalled(app, {filter: {category: 'TRIGGER'}, select: '-code'})`. |
+| **Returns** | `{ app, count, plugins: [<trimmed plugin object>], usage }`. |
+
+### `list_process_plugins`
+
+| | |
+|---|---|
+| **Purpose** | List installed plugins where `category=PROCESS`. Source for `pluginObject` in `add_node_to_flow`. |
+| **Inputs** | `group` (optional, e.g. `'DataService'`, `'HTTP'`, `'Misc'`), `search` (optional). |
+| **Behaviour** | Same client call as triggers but `category: 'PROCESS'`. Filtering by group/search is client-side. |
+| **Returns** | Same shape as triggers. |
+| **Description** | Includes the plugin-selection strategy verbatim so the LLM sees it at tool-call time. |
+
+### `list_flows`
+
+| | |
+|---|---|
+| **Purpose** | Browse flows in the selected app. |
+| **Inputs** | `name` (optional substring), `status` (optional, e.g. `'Draft'`, `'Active'`). |
+| **Endpoint** | `GET /api/a/bm/${app}/flow?filter={"app":"${app}"[, "status":"…"]}&count=-1&select=_id,name,status,version,inputNode.type,description` |
+| **Returns** | `{ app, count, flows: [{ flowId, name, status, version, triggerType, description }] }` |
+
+### `get_flow`
+
+| | |
+|---|---|
+| **Purpose** | Fetch the full flow document. The mutating tools fetch internally too — this is for inspection. |
+| **Inputs** | `flowId` (required, e.g. `FLOW6156`). |
+| **Endpoint** | `GET /api/a/bm/${app}/flow/${flowId}` |
+| **Returns** | The full flow JSON. |
+
+### `create_flow`
+
+| | |
+|---|---|
+| **Purpose** | Create a new flow with one trigger node and zero or more initial process nodes. |
+| **Inputs** | `name` (required), `description` (optional), `skipAuth` (optional, default true), `triggerPlugin` (required object), `triggerOptions` (required object), `initialNodes` (optional array). |
+| **Validation** | Rejects if `triggerPlugin.category !== 'TRIGGER'`. Rejects if any `initialNodes[i].pluginObject.connectorType !== 'NONE'` and no `connector` is provided. |
+| **Behaviour** | Builds `inputNode` from `triggerPlugin` + `triggerOptions`, slugifying the trigger label for `_id` (e.g. `http_server`, `timer`). For each initial node, builds the full node entry from its plugin (copying `inputSchema`, `outputSchema`, `errorSchema`, `category`, `group`, `icon`, `label`, `version`, `type`, `connectorType`), wires sequentially via `onSuccess`, places coordinates left-to-right at `x = (i+1) * 200`. POSTs `/flow`. |
+| **Endpoint** | `POST /api/a/bm/${app}/flow` |
+| **Returns** | `{ flowId, status, flow }` |
+
+#### Trigger options shapes
+
+| Trigger type | `triggerOptions` |
+|---|---|
+| `V1_HTTP_SERVER` | `{ method: 'POST', path: '/upload' }` |
+| `V1_TIMER` | `{ cron: '0 9 * * *', timezone: 'Asia/Kolkata', holidayList?: '…' }` |
+| `V1_TIMER_MULTIPLE` | `{ cronList: [{ cron, timezone }, …] }` |
+
+### `add_node_to_flow`
+
+| | |
+|---|---|
+| **Purpose** | Append a process node to an existing flow and wire it after another node. |
+| **Inputs** | `flowId`, `pluginObject` (full plugin from `list_process_plugins`), `name` (slug becomes `_id`), `afterNodeId` (the node whose `onSuccess` should point at this new one — use the inputNode's `_id` to attach right after the trigger), `options` (optional, merged on top of platform defaults), `mappings` (optional), `connector` (optional, `{_id}`), `coordinates` (optional, defaults to `{x: maxX(flow) + 200, y: 0}`), `branchCondition` (optional, see below). |
+| **Validation** | Rejects if `pluginObject.connectorType !== 'NONE'` and no `connector` is provided. Rejects if `afterNodeId` doesn't exist in the flow. |
+| **Behaviour** | Fetches flow → builds new node → resolves `_id` (slugified `name`, dedup-suffixed if collision) → appends to `nodes[]` → wires the edge → PUTs full document. |
+| **Endpoint** | `GET /flow/${flowId}` then `PUT /flow/${flowId}` |
+| **Returns** | `{ addedNodeId, flow }` |
+
+#### Wiring rules
+
+- Default (no `branchCondition`): **replace** `afterNodeId.onSuccess` with `[{ _id: <new node> }]`.
+- With `branchCondition`: **append** `{ _id: <new node>, condition, name, color? }` to `afterNodeId.onSuccess` so multiple conditional branches coexist.
+
+### `update_node_in_flow`
+
+| | |
+|---|---|
+| **Purpose** | Edit an existing node's options, mappings, coordinates, name, or connector. |
+| **Inputs** | `flowId`, `nodeId` (or `inputNode._id` for the trigger), `options?`, `mappings?`, `coordinates?`, `name?`, `connector?`. |
+| **Behaviour** | Fetches flow → finds node → merges `options` partially (does NOT touch keys not in the patch) → replaces `mappings` entirely if provided → updates the rest as given → PUTs. |
+| **Returns** | `{ updatedNodeId, flow }` |
+
+### `connect_nodes`
+
+| | |
+|---|---|
+| **Purpose** | Wire `onSuccess` / `onError` between existing nodes. Use to re-route or to add additional branches. |
+| **Inputs** | `flowId`, `fromNodeId`, `toNodeId`, `edge` (optional, `'onSuccess'` default), `condition` (optional, lodash-style expression), `branchName` (required if `condition` is set), `branchColor` (optional hex without `#`), `mode` (optional `'append'` or `'replace'`). |
+| **Default mode** | `replace` if no `condition`, `append` if `condition` is set. |
+| **Behaviour** | GET → mutate `fromRef.node[edge]` → PUT. |
+| **Returns** | `{ from, to, edge, mode, flow }` |
+
+### `remove_node_from_flow`
+
+| | |
+|---|---|
+| **Purpose** | Delete a node and clean up dangling references in every other node's `onSuccess`/`onError`. |
+| **Inputs** | `flowId`, `nodeId`, `force` (optional boolean). |
+| **Validation** | Refuses if `nodeId === inputNode._id` (cannot remove the trigger). Refuses if removing this node would leave the inputNode pointing to nothing while other nodes still exist, unless `force: true`. |
+| **Behaviour** | GET → splice from `nodes[]` → strip refs from every node's edges → PUT. |
+| **Returns** | `{ removedNodeId, flow }` |
+
+### `publish_flow`
+
+| | |
+|---|---|
+| **Purpose** | Move a flow out of `Draft` state. Until published, a flow can't be added to a deployment group. |
+| **Inputs** | `flowId`. |
+| **Endpoint** | `PUT /api/a/bm/${app}/flow/utils/${flowId}/publish` with body `{ app }` (server-injected). |
+| **Returns** | `{ flowId, result }` |
+
+## Mappings
+
+Each entry in `node.mappings[]` describes how a field on the **current node's input** is filled from previous nodes' outputs. Shape:
+
+```json
+{
+  "key": "data",
+  "name": "data",
+  "target": {
+    "_id": "data",
+    "dataPath": "data",
+    "dataPathSegs": ["data"],
+    "type": "Buffer"
+  },
+  "source": [
+    {
+      "_id": "http_receiver.data",
+      "nodeId": "http_receiver",
+      "key": "data",
+      "name": "data",
+      "dataPath": "data",
+      "dataPathSegs": ["data"],
+      "type": "Buffer"
+    }
+  ],
+  "expression": {
+    "type": "simple",
+    "value": "{{http_receiver['data']}}"
+  },
+  "children": [],
+  "iterator": null,
+  "derivedFrom": null
+}
+```
+
+- `target` — field on this node's input.
+- `source[]` — previous-node fields being read (can be empty for hardcoded expressions).
+- `expression.value` — lodash-style template (see Runtime globals).
+- `children[]` — recursive, for nested objects.
+- `iterator` — set to a source array reference; `children[]` then describes the per-item mapping.
+
+## Runtime globals (in CODEBLOCK code, mapping `expression.value`, and branch `condition`)
+
+The platform exposes these globals everywhere expressions are evaluated:
+
+- **Every node's `_id`** is reachable two ways:
+  - Top-level bare variable in `{{ }}` expressions: `{{fetch_transactions['data']}}`, `{{route['data']['outputFormat']['_id']}}`.
+  - Via `node['<id>']` inside CODEBLOCK code: `node['parse_body'].data.records.length`.
+  - Outputs always live under `.data`.
+- **`CONSTANTS`** — flow-level constants on the flow document. Example: `{{CONSTANTS.TOKEN}}`.
+- **`ENV`** — deployment env vars. Example: `{{ENV.SOME_VAR}}`.
+
+Common patterns:
+
+```
+{{CONSTANTS.TOKEN}}                                              ← header value
+JSON.stringify({"id": {{route['data']['outputFormat']['_id']}}}) ← literal-with-interp
+_.isNull({{prev.data.content}})                                  ← branch condition
+{{fetch_transactions['data']['status']}} == "SUCCESS"            ← branch equality
+```
+
+The same documentation block is appended verbatim to the descriptions of `add_node_to_flow`, `update_node_in_flow`, and `connect_nodes`, so the LLM always has it in context when authoring expressions.
+
+## CODEBLOCK contract
+
+`V1_CODEBLOCK` nodes execute custom JavaScript. The function signature is fixed:
+
+```js
+async function executeCode(inputData, node, connectorConfig) {
+  try {
+    let data = {};
+    // your logic here
+    return data;            // becomes outputSchema.data downstream
+  } catch (err) {
+    logger.error(err);      // 'logger' is provided by the platform
+    throw err;              // routes to onError
+  }
+}
+```
+
+Critical: the function MUST return `{ data: <whatever> }` (or assign to `data` inside the function and return it as shown). Anything else is dropped — the platform reads `outputSchema.data` only.
+
+`V1_CODEBLOCK` has `connectorType=NONE`; do not pass a `connector`.
+
+## Branching example
+
+```
+add_node_to_flow(
+  pluginObject: <V1_RESPONSE>,
+  name: 'no_txn_response',
+  afterNodeId: 'fetch_transactions',
+  branchCondition: {
+    condition: '_.isEmpty({{fetch_transactions["data"]["records"]}})',
+    name: 'No Transactions',
+    color: 'FF9800'
+  }
+)
+
+add_node_to_flow(
+  pluginObject: <V1_DATASERVICE_PUT>,
+  name: 'update_txns',
+  afterNodeId: 'fetch_transactions',
+  branchCondition: {
+    condition: '!_.isEmpty({{fetch_transactions["data"]["records"]}})',
+    name: 'Has Transactions',
+    color: '4CAF50'
+  }
+)
+```
+
+After both, `fetch_transactions.onSuccess` has two entries with conditions — the runtime evaluates them and routes accordingly.

package/docs/tools/data-services.md

@@ -0,0 +1,105 @@
+# Data Services (Definitions)
+
+Tools registered by `src/tools/services.js`. These manage **data-service definitions** — the schemas + hooks + workflow config that describe a Kubernetes-deployable record store. (CRUD on the records of an existing service is in [`records.md`](records.md).)
+
+Backed by `ServicesClient` (`src/clients/services.js`).
+
+Endpoint pattern: `/api/a/sm/${app}/service[/…]`.
+
+## What gets auto-injected on create/update
+
+Three things are injected server-side into `create_data_service` (and `app` into `update_data_service`) so the LLM never has to remember them:
+
+1. **`payload.app = registry.selectedApp`** — the platform's create endpoint requires `app` in the body. Without this, the platform returns *"App data not found for undefined"*. The tool always overrides whatever the LLM sent.
+2. **Connectors** — if `payload.connectors.data._id` or `payload.connectors.file._id` is missing, the tool calls `dnioClient.connectors.list(app)`, picks the connector flagged `options.default && options.isValid !== false` for category `DB` (data) and `STORAGE` (file), and fills them in. The response surfaces `connectorsAutoFilled` so the user sees what was attached.
+3. **Roles & field permission map** — if `payload.role` is not already set with `roles[]` and `fields`, the tool generates the three standard roles (No Access / Manage / View) with random IDs (format `PNA_<10 digits>` for No Access, `P<10 digits>` for the others) and walks `payload.definition` to build a parallel `fields` map where every leaf field gets `{_t, _p}` with `_p` containing all three role IDs mapped to `"R"`. Object-type fields recurse into nested children with no `_t`/`_p` at the parent level. The response surfaces `rolesAutoFilled` with the generated role IDs.
+
+`update_data_service` only injects `app`. Connectors and roles are not regenerated on update — the existing values are preserved.
+
+## Tool reference
+
+### `get_data_service_spec`
+
+| | |
+|---|---|
+| **Purpose** | Returns the canonical creation spec — field types, hook formats, workflow config options, connector schema, and a full example payload. **MUST be called before `create_data_service`** so the LLM understands the required shape. |
+| **Inputs** | (none) |
+| **Source** | Static constant `DATA_SERVICE_CREATION_SPEC` in `src/utils/constants.js`. |
+| **Returns** | The spec object. |
+
+The spec covers:
+- Supported field types: `String`, `Number`, `Boolean`, `Date`, `Object`, `Array` and the properties each accepts.
+- The required first field (`_id` with prefix/counter).
+- `preHooks` and `workflowHooks` shapes.
+- `stateModel` (state-machine field) and `workflowConfig` (maker-checker approval).
+- Defaults (`schemaFree`, `permanentDeleteData`, `disableInsights`, `enableSearchIndex`, `allowedFileTypes`).
+- A connector section explaining the `connectors.data` / `connectors.file` requirement and that defaults will auto-attach.
+- A complete `examplePayload`.
+
+### `create_data_service`
+
+| | |
+|---|---|
+| **Purpose** | Create a new data service. |
+| **Inputs** | `data` (required, JSON string matching the spec). |
+| **Behaviour** | 1. Reject if no app is selected. 2. Parse the JSON. 3. Force `payload.app = registry.selectedApp`. 4. Auto-fill `connectors` if missing. 5. Auto-fill `role` (with roles + fields) if missing. 6. POST. |
+| **Endpoint** | `POST /api/a/sm/${app}/service` |
+| **Returns** | `{ result, connectorsAutoFilled?, rolesAutoFilled? }` — the auto-fill blocks are omitted when the user supplied them. |
+
+### `get_data_service`
+
+| | |
+|---|---|
+| **Purpose** | Fetch a data service definition by ID. Use `select` to fetch only specific fields. |
+| **Inputs** | `serviceId` (required, e.g. `SRVC13120`), `select` (optional CSV), `draft` (optional boolean — fetches the unsaved draft version when true). |
+| **Endpoint** | `GET /api/a/sm/${app}/service/${serviceId}?filter={"app":"${app}"}[&draft=true][&select=…]` |
+| **Returns** | The data-service document. |
+| **Common select patterns** | Documented inline in the tool description: `_id,name,description,status,attributeCount,version` (overview); `_id,name,definition,schemaFree` (schema); `_id,name,preHooks,workflowHooks` (hooks); `_id,name,workflowConfig,stateModel` (workflow); a longer one for full-edit. |
+
+### `list_data_services`
+
+| | |
+|---|---|
+| **Purpose** | List all data services in the selected app. |
+| **Inputs** | `select` (optional CSV, default `_id,name,description,status,attributeCount`), `statusFilter` (optional, one of `Active`, `Pending`, `Draft`, `Stopped`). |
+| **Endpoint** | `GET /api/a/sm/${app}/service?filter={"app":"${app}"[, "status":"…"]}&count=-1[&select=…]` |
+| **Returns** | The array. |
+
+### `update_data_service`
+
+| | |
+|---|---|
+| **Purpose** | Update an existing data service. The platform requires the **full** payload, not a partial patch — call `get_data_service` first, mutate, send back. After updating, call `deploy_data_service` to make the changes live. |
+| **Inputs** | `serviceId` (required), `data` (required, JSON string). |
+| **Behaviour** | Force `payload.app = registry.selectedApp`. Does **not** auto-fill connectors or roles. |
+| **Endpoint** | `PUT /api/a/sm/${app}/service/${serviceId}` |
+| **Returns** | The updated document. |
+
+### `deploy_data_service`
+
+| | |
+|---|---|
+| **Purpose** | Deploy a data service after creation or update. Triggers the actual Kubernetes deployment. **Must** be called after `create_data_service` or `update_data_service` to make the service usable. |
+| **Inputs** | `serviceId` (required). |
+| **Endpoint** | `PUT /api/a/sm/${app}/service/utils/${serviceId}/deploy` |
+| **Returns** | `{ serviceId, status: 'Deployment initiated', result }` |
+
+### `start_stop_data_service`
+
+| | |
+|---|---|
+| **Purpose** | Start or stop a deployed data service (toggles the running pod). |
+| **Inputs** | `serviceId` (required), `action` (required, `start` or `stop`). |
+| **Endpoint** | `PUT /api/a/sm/${app}/service/utils/${serviceId}/${start\|stop}` |
+| **Returns** | `{ serviceId, action, result }` |
+
+## Typical sequence
+
+```
+get_data_service_spec               # learn the shape
+create_data_service(data)           # tool injects app, connectors, role
+deploy_data_service(serviceId)      # K8s rollout
+# … service appears in list_services / select_app once active
+update_data_service(serviceId, …)   # if you need to change the schema
+deploy_data_service(serviceId)      # apply the change
+```

package/docs/tools/deployment-groups.md

@@ -0,0 +1,152 @@
+# Deployment Groups (Kubernetes Deployment Lifecycle)
+
+Tools registered by `src/tools/deployment-groups.js`. A **deployment group** is the unit of actual Kubernetes deployment: it bundles one or more **published** data pipes (flows) under a name and, when started, spins up real K8s pods/services for every flow in the group.
+
+Backed by `DeploymentGroupsClient` (`src/clients/deployment-groups.js`).
+
+Endpoint pattern: `/api/a/bm/${app}/deployment/group[/…]`.
+
+## Hard rule: one flow, one group
+
+A flow can belong to **at most one** deployment group. The platform enforces this; the tools enforce it client-side too for clear error messages:
+
+- `list_available_flows` returns only flows that are **published and not currently bound to any group**.
+- `create_deployment_group` and `add_flows_to_deployment_group` validate every requested flow against `list_available_flows` and reject the entire call atomically if any are unavailable.
+- `add_flows_to_deployment_group` additionally checks the target group's existing `deployments[]` to catch duplicates within the same group.
+- Removing a flow from a group (or deleting the group) frees that flow up again.
+
+## K8s timing
+
+`start` / `stop` / `sync` / `delete` are async on the K8s side. Tools return as soon as the platform accepts the request — they do **not** poll for pod readiness. Settle time is ~10 seconds. Re-fetch via `get_deployment_group` if you need to confirm.
+
+## Tool reference
+
+### `list_available_flows`
+
+| | |
+|---|---|
+| **Purpose** | List flows in the selected app that can be added to a deployment group right now. Source of truth for `create_deployment_group` / `add_flows_to_deployment_group`. |
+| **Inputs** | (none) |
+| **Endpoint** | `GET /api/a/bm/${app}/deployment/group/utils/available/flows?filter={"app":"${app}"}` |
+| **Returns** | `{ app, count, flows: [{ _id, name, type }] }` |
+
+### `list_deployment_groups`
+
+| | |
+|---|---|
+| **Purpose** | List deployment groups in the selected app, with their status and bound flows. |
+| **Inputs** | `name` (optional substring), `status` (optional, e.g. `'Running'`, `'Stopped'`, `'Pending'`). |
+| **Endpoint** | `GET /api/a/bm/${app}/deployment/group?filter={"app":"${app}"[, "status":"…"]}&count=-1` |
+| **Returns** | `{ app, count, groups: [{ groupId, name, status, deploymentCount, deployments: [{ _id, name, version, type }] }] }` |
+
+### `get_deployment_group`
+
+| | |
+|---|---|
+| **Purpose** | Fetch the full document for a single group. |
+| **Inputs** | `groupId` (required, e.g. `DG2943`). |
+| **Endpoint** | `GET /api/a/bm/${app}/deployment/group/${groupId}` |
+| **Returns** | The full group JSON. |
+
+### `create_deployment_group`
+
+| | |
+|---|---|
+| **Purpose** | Create a new group bundling one or more published flows. |
+| **Inputs** | `name` (required), `flowIds` (required array, length ≥ 1). |
+| **Validation** | Atomic. Rejects if `flowIds` contains duplicates. Calls `list_available_flows`; rejects the whole call if any requested flow isn't available, with explicit guidance pointing at `list_deployment_groups`. |
+| **Behaviour** | For each valid flow, fetches the flow document via `dnioClient.dataPipes.get(app, flowId)` to read `inputNode` + `version`. Builds `deployments[]` entries shaped `{ _id, name, type: 'FLOW', version, inputNode }`. POSTs. |
+| **Endpoint** | `POST /api/a/bm/${app}/deployment/group` |
+| **Returns** | `{ groupId, status, group }` |
+
+### `add_flows_to_deployment_group`
+
+| | |
+|---|---|
+| **Purpose** | Append flows to an existing group. |
+| **Inputs** | `groupId` (required), `flowIds` (required array). |
+| **Validation** | Same atomic checks: no duplicates in input, no flows already in this group, all flows in `list_available_flows`. |
+| **Behaviour** | GET group → fetch each flow → append entries → PUT. |
+| **Endpoint** | `GET … /${groupId}` then `PUT … /${groupId}` |
+| **Returns** | `{ groupId, addedFlowIds, group }` |
+
+### `remove_flows_from_deployment_group`
+
+| | |
+|---|---|
+| **Purpose** | Detach flows from a group. The removed flows become available again. |
+| **Inputs** | `groupId`, `flowIds` (required array), `force` (optional boolean). |
+| **Behaviour** | GET → filter `deployments[]` → if none of the requested flowIds were in the group, return error. PUT. If the group is now empty and `force` is not set, the response includes a warning that the next `start` will fail. |
+| **Endpoint** | `GET … /${groupId}` then `PUT … /${groupId}` |
+| **Returns** | `{ groupId, removedCount, remainingCount, group, warning? }` |
+
+### `rename_deployment_group`
+
+| | |
+|---|---|
+| **Purpose** | Change the name of a group. Does not affect deployments or running pods. |
+| **Inputs** | `groupId`, `name`. |
+| **Behaviour** | GET → mutate `name` → PUT. |
+| **Returns** | `{ groupId, name, group }` |
+
+### `start_deployment_group`
+
+| | |
+|---|---|
+| **Purpose** | Spin up Kubernetes pods + services for every flow in the group. |
+| **Inputs** | `groupId`. |
+| **Endpoint** | `PUT /api/a/bm/${app}/deployment/group/utils/${groupId}/start` |
+| **Returns** | `{ groupId, action: 'start', note: 'Kubernetes deployment initiated; may take ~10s to settle.', result }` |
+
+### `stop_deployment_group`
+
+| | |
+|---|---|
+| **Purpose** | Tear down K8s pods. Group definition + bound flows are preserved (flows stay unavailable to other groups until removed or the group is deleted). |
+| **Inputs** | `groupId`. |
+| **Endpoint** | `PUT /api/a/bm/${app}/deployment/group/utils/${groupId}/stop` |
+| **Returns** | `{ groupId, action: 'stop', result }` |
+
+### `sync_deployment_group`
+
+| | |
+|---|---|
+| **Purpose** | Re-pull the latest **published** version of every flow in the group and reconcile the running pods. Use after editing + re-publishing a flow that's already bound to a running group. Drafts are ignored. |
+| **Inputs** | `groupId`. |
+| **Endpoint** | `PUT /api/a/bm/${app}/deployment/group/utils/${groupId}/sync` |
+| **Returns** | `{ groupId, action: 'sync', result }` |
+
+### `delete_deployment_group`
+
+| | |
+|---|---|
+| **Purpose** | Permanently delete a group. K8s torn down, group definition removed, bound flows freed. |
+| **Inputs** | `groupId`, `confirm` (optional boolean — required `true` to actually delete). |
+| **Behaviour** | Two-phase. With `confirm: false` (default), GETs the group and returns `{ error: 'confirmation required', groupId, freedFlows: [{_id, name, version}], group }` so the user can see exactly which flows would be freed. With `confirm: true`, calls DELETE. |
+| **Endpoint** | `DELETE /api/a/bm/${app}/deployment/group/${groupId}` |
+| **Returns** | When confirmed: `{ groupId, action: 'deleted', result }`. |
+
+### `get_deployment_group_yamls`
+
+| | |
+|---|---|
+| **Purpose** | Fetch the actual Kubernetes Deployment + Service YAMLs the platform generated. Useful for debugging or showing the user. |
+| **Inputs** | `groupId`. |
+| **Endpoint** | `GET /api/a/bm/${app}/deployment/group/utils/${groupId}/yamls?filter={"app":"${app}"}` |
+| **Returns** | `{ groupId, yamls }` |
+
+## Typical sequence
+
+```
+list_available_flows                                                  # see what's eligible
+create_deployment_group(name, flowIds: [FLOW6156])                    # bundle
+start_deployment_group(groupId)                                       # K8s up
+get_deployment_group_yamls(groupId)                                   # inspect generated K8s objects
+# … later, after editing + republishing a flow inside the group:
+sync_deployment_group(groupId)                                        # roll forward
+# … to release a flow:
+remove_flows_from_deployment_group(groupId, [flowId])
+# … or to tear it all down:
+stop_deployment_group(groupId)
+delete_deployment_group(groupId, confirm: true)
+```

package/docs/tools/plugins.md

@@ -0,0 +1,94 @@
+# Plugins (Workflow Nodes)
+
+Tools registered by `src/tools/plugins.js`. **Plugins** are reusable code blocks that act as **nodes inside data pipes** — parsing, HTTP calls, data-service operations, file ops, custom logic, etc.
+
+The platform separates a global **marketplace catalog** (cross-app) from per-app **installed** plugins. A plugin must be installed into the selected app before flow tools can use it.
+
+Backed by `PluginsClient` (`src/clients/plugins.js`).
+
+## Lifecycle
+
+```
+list_marketplace_plugins           →  catalog of available plugins
+       │
+       ▼
+install_plugins(marketIds)         →  installs into the selected app
+       │
+       ▼
+list_installed_plugins             →  list / get / inspect the installed copies
+       │
+       ├─ update_plugins(ids)      →  refresh to latest marketplace version
+       │
+       └─ uninstall_plugins(ids)   →  destructive, may break flows that use them
+```
+
+**Critical: marketplace `_id` ≠ installed `_id`.** The marketplace returns short hex `_id` values (e.g. `6859067143a8a209fc83a000`); after install, the platform assigns a different long-hex `_id` for the per-app copy. Use `marketId` for `install_plugins`; use `_id` from `list_installed_plugins` for everything else.
+
+## Tool reference
+
+### `list_marketplace_plugins`
+
+| | |
+|---|---|
+| **Purpose** | Browse the cross-app catalog of plugins. |
+| **Inputs** | `search` (optional, substring on label or type). |
+| **Endpoint** | `GET /api/a/bm/${app}/marketplace/node?page=1&count=-1&sort=label&filter={}&select=label,type,version,icon` |
+| **Returns** | `{ app, count, plugins: [{ marketId, type, label, version, icon }], usage }` |
+
+### `install_plugins`
+
+| | |
+|---|---|
+| **Purpose** | Install one or more marketplace plugins into the selected app. Idempotent — installing an already-installed plugin returns the existing instance. |
+| **Inputs** | `marketIds` (required array of strings, min length 1; pass even one as an array). |
+| **Endpoint** | `POST /api/a/bm/${app}/my-node/utils/install` with body `{ marketIds: [...] }` |
+| **Returns** | `{ app, requested, installed, failed, results: [...] }`. The tool buckets results by status code so the LLM can report which installs succeeded/failed. |
+
+### `list_installed_plugins`
+
+| | |
+|---|---|
+| **Purpose** | List plugins installed in the selected app, or fetch full details for a single one. |
+| **Inputs** | `pluginId` (optional, installed `_id`), `details` (optional boolean, default false), `search` (optional). |
+| **Endpoint** | `GET /api/a/bm/${app}/my-node?count=-1&filter={"app":"${app}"[, "_id":"…"]}&select=…` |
+| **Behaviour** | When `details: false` the select is `label,type,version,icon` (summary). When `details: true` the select is `_id,app,nodeId,version,category,group,type,label,icon,connectorType,inputSchema,outputSchema,errorSchema,code` — large payload, includes the full executeCode source. When `pluginId` is set the response collapses to a single plugin object; otherwise an array. |
+| **Returns** | When listing: `{ app, count, details, plugins }`. When `pluginId` given: `{ app, plugin }`. |
+
+### `update_plugins`
+
+| | |
+|---|---|
+| **Purpose** | Refresh installed plugins to the latest version from the marketplace. Idempotent. |
+| **Inputs** | `ids` (required array, **installed** `_id` values, not marketplace IDs). |
+| **Endpoint** | `POST /api/a/bm/${app}/my-node/utils/update` with body `{ app, ids }` (the `app` is server-injected). |
+| **Returns** | `{ app, requested, result }` |
+
+### `uninstall_plugins`
+
+| | |
+|---|---|
+| **Purpose** | Uninstall plugins from the selected app. **Destructive** — data pipes that reference uninstalled plugins may break. |
+| **Inputs** | `ids` (required array, installed `_id`). |
+| **Endpoint** | `POST /api/a/bm/${app}/my-node/utils/uninstall` with body `{ app, ids }` |
+| **Returns** | `{ app, requested, result }` |
+
+## Plugin categories
+
+Plugins are split into two categories the agent cares about:
+
+- `TRIGGER` — starter nodes for flows (`V1_HTTP_SERVER`, `V1_TIMER`, `V1_TIMER_MULTIPLE`). Drive `list_trigger_plugins` in the data-pipes domain.
+- `PROCESS` — downstream nodes used after the trigger (`V1_PARSE_JSON`, `V1_SAVE_FILE`, `V1_RESPONSE`, `V1_DATASERVICE_*`, `V1_CALL_FLOW`, `V1_CODEBLOCK`, …). Drive `list_process_plugins` in the data-pipes domain.
+
+Plugins are filtered by category there — see [`data-pipes.md`](data-pipes.md).
+
+## Typical sequence
+
+```
+list_marketplace_plugins(search: 'parse')
+  → user picks marketIds
+install_plugins([marketId])
+list_installed_plugins                 # confirm + grab the new _id
+list_installed_plugins(pluginId, details: true)   # if you need the schema/code
+```
+
+After install, the plugins are usable by `add_node_to_flow` in the data-pipes domain.