@readymadeit/mcp 0.0.0-stage → 0.1.0

package/bundle/docs/content/services/file-source.md

@@ -0,0 +1,67 @@
+# File Source
+
+Lets the user select one or more local files and emits their contents into the pipeline.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `hookup.to/service/file-source` |
+
+---
+
+## What it does
+
+File Source presents a file input in its UI panel. When the user selects files, the service reads each one as text and emits the results downstream via `app.next()`. If a single file is selected, its text content is emitted directly. If multiple files are selected, an array of text strings is emitted.
+
+A facade's `file-pick` widget can hand it a file too: name the File Source as the widget's `action.serviceUuid`, and as its `progressServiceUuid` so the widget knows when the file has gone on. The file is read as text and emitted the same way — e.g. a picked `.sql` dump passed on to an hkp-node [SQL](./sql.md) service in `import` mode, as the [SQL Import](../boards/sql-import-board.md) board does.
+
+---
+
+### In chunks
+
+With a `chunkSize`, each file is still read whole, but goes on as an **array of
+pieces** of at most that many characters, in order — an array even when the file
+fits in one piece, so what follows does not change shape with the size of the
+file. Several files picked at once go on as an array of such arrays.
+
+The pieces are still one value. What sends them on one by one is a
+[Spread](./spread.md) after the File Source; a [Gather](./gather.md) on the far
+side puts them back together. That is how a file larger than a deployed board's
+frame limit (`HKP_COORDINATOR_MAX_FRAME_BYTES`) still reaches hkp-node:
+
+```
+ui:   file-source (chunkSize: 500000) → spread
+node: gather (join(params.items, '')) → …
+```
+
+`chunkSize` counts characters, not bytes. For ASCII text — base64, CSV, most SQL
+dumps — the two are the same; other text takes up to three bytes per character
+in UTF-8, and a frame also carries the part's wrapping and any JSON escaping, so
+leave room below the limit. A piece never ends between the two halves of a
+surrogate pair.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `chunkSize` | `number` | `0` | Characters per piece; `0` sends each file whole |
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | None (triggered by the file picker) |
+| **Output** | `String` — text content of a single file; `JSON` — `string[]`, the text contents when multiple files are selected; with a `chunkSize`, `string[]` of pieces per file |
+
+---
+
+## Notes
+
+Files are read as plain text (`FileReader.readAsText`). For binary files (images, audio), use [Image Picker](./image-picker.md) or [Chunked File Provider](./chunked-file-provider.md) instead.

package/bundle/docs/content/services/filesystem.md

@@ -0,0 +1,157 @@
+# Filesystem
+
+Reads from and writes to the local file system on an hkp-rt server.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| hkp-rt | `filesystem` |
+| Node.js (hkp-node) | `filesystem` |
+
+Not available in the browser runtime, which cannot reach the file system.
+
+Neither reaches the whole machine. On hkp-rt a path lies inside a **folder the
+runtime gives boards** — one by default, more if the person or the operator adds
+them — because a board is something somebody else may have written, and the
+machine it runs on also holds the app's vault, its settings and a server's
+secrets. On hkp-node, which is multi-tenant, a board names a path **inside a
+volume** the runtime owns, and nothing it can write resolves outside one. The
+rest of this page describes the hkp-rt service; the hkp-node one is described
+below.
+
+---
+
+## What it does
+
+Filesystem reads and writes files inside the folders its runtime gives boards.
+The operation is determined by the incoming pipeline data or by configuration.
+
+---
+
+## Configuration
+
+| Property | Type | Description |
+|---|---|---|
+| `path` | `string` | A file or folder inside the folders boards are given. Relative paths are taken from the default one |
+| `operation` | `"read" \| "write"` | What a request naming none means |
+
+### Where the files are
+
+| Runtime is | Default folder | Adding more |
+|---|---|---|
+| the desktop app's (`"remote": "embedded"`) | `~/.hkp/cpp/files` | *Folders boards may use* in the app's settings |
+| a standalone hkp-rt | `HKP_FILES_DIR`, else `~/.hkp/cpp/files` | `HKP_FILES_ALLOW` |
+| the iOS and Android apps' | none | — |
+
+A path outside every folder is refused, and so is one that only looks inside:
+`..` and symlinks are followed before the path is compared. A name that arrives
+with the data — `path` in a read request, `meta.path` beside uploaded bytes — is
+taken **inside** the configured `path`, whatever it says, so the data picks a
+file and never the folder. A refusal is logged as `file-refused` and a write
+answers `{ "status": "failure", "error": "…" }`.
+
+Write `"recordings"` rather than `"/Users/me/recordings"`: a relative path means
+the same thing on every machine the board is opened on.
+
+---
+
+## Input / Output
+
+### Read mode
+
+| | Shape |
+|---|---|
+| **Input** | Any value (trigger only) |
+| **Output** | File contents as a string (text) or byte array (binary) |
+
+### Write mode
+
+| | Shape |
+|---|---|
+| **Input** | String or byte array to write |
+| **Output** | Write confirmation or `null` |
+
+---
+
+## Typical use
+
+### Load a configuration file at startup
+
+```
+timer (one-shot) → filesystem (read, path: "config/settings.json") → map → configure services
+```
+
+### Append log entries to a file
+
+```
+data source → map (format log line) → filesystem (write, path: "logs/output.log")
+```
+
+### Process a dataset file
+
+```
+filesystem (read) → map (parse CSV/JSON) → buffer → process
+```
+
+
+---
+
+## On hkp-node
+
+The node service answers the same requests [Storage](./storage.md) does, because
+it is what a Storage is usually pointing at:
+
+```json
+{ "op": "write", "path": "episodes/one.mp3", "binary": "…the bytes…" }
+```
+
+`read`, `write`, `list`, `stat` and `delete`, with the operation coming from the
+request rather than from a mode — one instance therefore answers a read and a
+write, which is what lets it be swapped for another backend without the services
+around it noticing.
+
+### Configuration
+
+| Property | Type | Description |
+|---|---|---|
+| `volume` | `string` | Which library this instance reads and writes. Empty is the board's own |
+| `operation` | `string` | What a request naming none means |
+| `path` | `string` | What a request naming none means |
+
+### Where the files are
+
+```
+<root>/<sha256(owner)>/<volume>/<path>
+```
+
+The owner is hashed and the rest is not, and the asymmetry is the point. An
+owner comes from a token and may contain anything, so it is derived into a path
+rather than used as one. A volume and a path are written by a board, can
+therefore be checked, and are worth keeping legible — the reason to put bytes on
+a disk rather than in a record is that they can then be found, copied, played and
+backed up by somebody who is not this runtime.
+
+What is checked is narrow, because these names become a real path: a segment is
+letters, digits, dot, dash and underscore, a leading dot is not a name, and
+nothing resolves outside the volume it named. A board that gets it wrong is told.
+
+`HKP_FILES_DIR` says where the root is; empty keeps everything in memory, the
+way `HKP_STORE_DIR` and `HKP_DB_DIR` do.
+
+### Sharing a volume
+
+An unnamed volume belongs to the board, derived from its name. Two boards share
+one by **naming the same volume** — a board that renders audio and a board that
+publishes it, working over one library — exactly as two boards share tables by
+naming the same [database](./sql.md).
+
+### Answers
+
+A read comes back as `meta` beside `binary`, which is also a response envelope:
+put an endpoint in front of a Storage and a file is served as what it is, content
+type and all. A miss is a `404` in `meta.status` rather than an exception, and a
+`stat` for a file that is not there is a `200` saying `exists: false` — absence
+is an answer to that question, not a failure.

package/bundle/docs/content/services/filter.md

@@ -0,0 +1,133 @@
+# Filter
+
+Evaluates one or more boolean expressions against the pipeline data and either passes it through or stops the pipeline.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `hookup.to/service/filter` |
+
+---
+
+## What it does
+
+Filter evaluates a set of expressions against the current pipeline value.
+If the result is truthy, the original input is passed to the next service
+unchanged. If the result is falsy, the service returns `null`, stopping
+the pipeline at that point.
+
+When the input is an array, each element is tested individually. Elements
+that pass the condition are kept; elements that fail are removed. If all
+elements are removed the service returns `null`.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `conditions` | `string \| string[]` | `[]` | One or more expression strings to evaluate |
+| `aggregator` | `"and" \| "or"` | `"and"` | How multiple conditions are combined |
+
+### `conditions`
+
+Each condition is a JavaScript expression string evaluated against the
+pipeline input. The expression receives the following scope:
+
+- **`params`** — the current pipeline value (the full input object or
+  array element being tested)
+- All built-in functions from the expression scope (see the
+  [Map service](./map.md#expression-scope) for the full list)
+
+A condition should evaluate to a truthy or falsy value. Examples:
+
+```
+params.score > 80
+params.status === "active"
+params.tags && params.tags.includes("urgent")
+```
+
+### `aggregator`
+
+When multiple conditions are provided:
+
+| Value | Behaviour |
+|---|---|
+| `"and"` (default) | All conditions must be truthy for data to pass |
+| `"or"` | Any single truthy condition allows data to pass |
+
+> **Note:** With a single condition the aggregator has no effect.
+
+> **Note — empty conditions:** If no conditions are configured the
+> service blocks everything and always returns `null`.
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | Any JSON value |
+| **Output (pass)** | Original input, unchanged |
+| **Output (block)** | `null` — stops the pipeline |
+| **Array input** | Elements are tested individually; non-matching elements are removed |
+
+---
+
+## Examples
+
+### Threshold guard
+
+Allow only values where a numeric property exceeds a threshold:
+
+```json
+{
+  "conditions": "params.temperature > 37.5",
+  "aggregator": "and"
+}
+```
+
+### Multiple conditions with AND
+
+Require both a minimum score and an active status:
+
+```json
+{
+  "conditions": [
+    "params.score >= 60",
+    "params.status === \"active\""
+  ],
+  "aggregator": "and"
+}
+```
+
+### Any-of with OR
+
+Pass through if either of two urgent signals is set:
+
+```json
+{
+  "conditions": [
+    "params.priority === \"high\"",
+    "params.overdue === true"
+  ],
+  "aggregator": "or"
+}
+```
+
+### Filtering an array
+
+If the pipeline value is an array (e.g. from a Buffer), only elements
+matching the condition are kept:
+
+```json
+{
+  "conditions": "params.value > 0"
+}
+```
+
+Input `[{ "value": 3 }, { "value": -1 }, { "value": 7 }]`
+→ Output `[{ "value": 3 }, { "value": 7 }]`

package/bundle/docs/content/services/flat-map.md

@@ -0,0 +1,62 @@
+# FlatMap
+
+Applies an expression to each item in an array and concatenates all results into a single flat array.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `hookup.to/service/flat-map` |
+
+---
+
+## What it does
+
+FlatMap wraps the Map service's expression evaluator with a flattening step. For each element in the input (or for the single input if it is not an array), the `expression` is evaluated with `params` set to that element. All results are concatenated (`Array.flat(1)`); `null` and `undefined` results are dropped.
+
+If the expression returns an array for each item, FlatMap effectively expands nested arrays into a single level — hence the name.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `expression` | `string` | `""` | Expression evaluated per input item. Should return an array or a value. Uses the same expression syntax and built-in functions as the [Map](./map.md) service. |
+
+If `expression` is empty, the input is passed through unchanged.
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | Any value or array of values |
+| **Output** | Flat array of all non-null expression results |
+
+---
+
+## Example: extract all tags from a list of posts
+
+```json
+{ "expression": "params.tags" }
+```
+
+Input:
+```json
+[
+  { "title": "A", "tags": ["js", "react"] },
+  { "title": "B", "tags": ["css"] }
+]
+```
+
+Output: `["js", "react", "css"]`
+
+---
+
+## Expression scope
+
+FlatMap uses the same expression engine and built-in functions as [Map](./map.md). The variable `params` refers to the current item being processed.

package/bundle/docs/content/services/game-of-life.md

@@ -0,0 +1,93 @@
+# Game of Life
+
+Simulates Conway's Game of Life on an infinite sparse grid and renders each generation as canvas draw commands.
+
+---
+
+## Available in
+
+| Runtime | Service IDs |
+|---|---|
+| Browser | `hookup.to/service/game-of-life`, `hookup.to/service/game-of-life-renderer` |
+
+---
+
+## Two-service pipeline
+
+The simulation is split across two services for separation of concerns:
+
+| Service | Responsibility |
+|---|---|
+| **Game of Life** | Pure simulation — advances the grid by one generation per tick and emits a frame descriptor |
+| **Game of Life Renderer** | Pure rendering — converts the frame descriptor into canvas draw commands |
+
+Connect them in sequence, then pipe the Renderer's output into a [Canvas](./canvas.md) service.
+
+```
+Timer → Game of Life → Game of Life Renderer → Canvas
+```
+
+---
+
+## Game of Life
+
+**Service ID:** `hookup.to/service/game-of-life`
+
+### What it does
+
+Maintains a sparse infinite grid using a `Set<string>` of `"row,col"` keys. On each `process()` call it advances by one generation using standard Conway rules (birth at 3 neighbours; survival at 2 or 3; otherwise death). The initial state is the **Gosper Glider Gun** — the classic infinite-growth pattern.
+
+### Output shape
+
+```typescript
+{
+  generation: number;
+  objects: GolObject[];
+}
+```
+
+Each `GolObject` is a maximal connected component (8-connectivity) of alive cells:
+
+```typescript
+{
+  row: number;      // top-left row of the bounding box
+  col: number;      // top-left col of the bounding box
+  matrix: number[][];  // [rowIndex][colIndex], 1=alive 0=dead
+}
+```
+
+### Configuration
+
+| Property | Type | Description |
+|---|---|---|
+| `reset` | `boolean` | Resets the grid to the initial Gosper Glider Gun seed |
+
+---
+
+## Game of Life Renderer
+
+**Service ID:** `hookup.to/service/game-of-life-renderer`
+
+### What it does
+
+Receives a `GolFrame`, culls cells outside the visible viewport, and emits an array of `{ type: "rect", ... }` draw commands compatible with the [Canvas](./canvas.md) service. The viewport pans smoothly toward the centre of mass of all alive cells using exponential blending.
+
+Objects are color-coded by size:
+- Large structures (≥ 20 cells, e.g. the gun) — deep blue
+- Mid-size (5–19 cells, e.g. gliders in transit) — deep green
+- Small / still-lifes (< 5 cells) — deep red
+
+### Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `cellSize` | `number` | `6` | Size of each cell in pixels |
+| `canvasWidth` | `number` | `800` | Width of the target canvas in pixels |
+| `canvasHeight` | `number` | `600` | Height of the target canvas in pixels |
+
+### Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | `GolFrame` from the Game of Life service |
+| **Output** | Array of Canvas rect draw commands |

package/bundle/docs/content/services/gather.md

@@ -0,0 +1,135 @@
+# Gather
+
+Waits for every part a Spread sent on and turns them back into one value — as an array, or as whatever its nested pipeline makes of them.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `gather` |
+| Node.js (hkp-node) | `gather` |
+| Python (hkp-python) | `gather` |
+| C++ (hkp-rt) | `gather` |
+
+Put it in whichever runtime the whole is wanted in — the browser, when a server
+spread a result for a panel to show; hkp-node, when a browser spread a file for
+an import.
+
+---
+
+## What it does
+
+Gather is the other half of [Spread](./spread.md). Spread turns one array into
+as many values as it has items, each sent on by itself; Gather waits until it
+has seen them all and turns them back into one.
+
+```
+ui:   file-source (chunkSize) → spread     one value per chunk
+node: gather → measure                     one value per file
+```
+
+What makes that worth doing is the space between the two. A value leaving a
+runtime travels as one message, and a coordinator drops a message above its
+frame limit (`HKP_COORDINATOR_MAX_FRAME_BYTES`) and logs it as `frame-dropped`.
+A file too large to send at once can still be sent as chunks, each under the
+limit, and arrive whole. `boards/frame-limit-test-board.json` shows both.
+
+### A part says which whole it belongs to
+
+Spread sends each item as `{ item, group, index, total }`. Gather collects the
+parts of one `group`, puts each at its `index`, and once it holds `total` of
+them the group is complete. Parts may arrive in any order; a part sent twice
+counts once.
+
+Until a group is complete Gather passes **nothing** on (`null`): the run that
+brought the part ends there. The run that brings the last part is the one the
+whole goes on in.
+
+**Anything that is not a part passes through untouched**, so the same Gather
+sits on a board that sometimes sends a value whole.
+
+### What the group becomes
+
+With no `pipeline`, the items go on as an array, in the order they were spread.
+
+With one, the pipeline is given `{ group, items }` and its answer is what goes
+on — the same arrangement as the reducer of [Tracks](./tracks.md): a pipeline,
+not a list of strategies. Joining text is one [Map](./map.md) term:
+
+```json
+{
+  "uuid": "gather",
+  "serviceId": "gather",
+  "state": {
+    "pipeline": [
+      {
+        "uuid": "join",
+        "serviceId": "map",
+        "state": { "mode": "replace", "template": { "=": "join(params.items, '')" } }
+      }
+    ]
+  }
+}
+```
+
+A pipeline that stops — returns `null` — stops Gather too. The pipeline runs as
+a run of its own beneath the one that brought the last part, like a
+[SubService](./sub-service.md)'s.
+
+### Collecting holds the group in memory
+
+Which is what a frame limit set to bound memory would otherwise have prevented.
+So a group is bounded, and one that never completes is let go:
+
+- **`maxBytes`** — a group whose parts add up to more is refused: what it held
+  is dropped, the parts still on their way are ignored, and the board log
+  records `gather.refused`.
+- **`maxGroups`** — how many groups may be incomplete at once. A part that
+  would start one more is refused, logged as `gather.refused`, and the rest of
+  that group with it. Nothing already arriving is given up to make room. The
+  most one Gather holds is therefore `maxGroups × maxBytes`, which is what to
+  lower on a server that others' boards run on.
+- **`timeoutMs`** — a group still incomplete that long after its first part is
+  let go, logged as `gather.timed-out`. Checked whenever a part arrives.
+
+What a group is called is bounded too: a part naming a `group` longer than 256
+characters is not a part, and passes through. Spread names its groups by UUID.
+
+Whichever limit it was, `error` says what happened, in the service's state and as a
+notification — a facade `notice` on `error` shows it — because a group that
+never arrives looks, from outside, exactly like nothing having been sent.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `pipeline` | `array` | `[]` | The nested services that make one value of `{ group, items }`; none passes the items on as an array |
+| `maxBytes` | `number` | `67108864` | The most one group may hold (64 MiB, the coordinator's default frame limit) |
+| `maxGroups` | `number` | `16` | How many groups may be incomplete at once |
+| `timeoutMs` | `number` | `60000` | How long a group may take to complete |
+| `bypass` | `boolean` | `false` | Pass every input through untouched |
+| `stopPropagation` | `boolean` | `false` | Pass nothing on, even a complete group |
+| `pending` | `number` | — | Read-only: how many groups are incomplete |
+| `error` | `string` | `""` | Read-only: the last group refused or let go, and why |
+
+Each part that arrives notifies `{ group, received, total, pending }`.
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | `JSON` — a part, `{ item, group, index, total }`, as a Spread sends it; anything else passes through |
+| **Output** | `JSON` — the group's items as an array, or what the pipeline made of them (e.g. a `String`); `Null` until the group is complete |
+
+---
+
+## Typical uses
+
+- A file past the frame limit: Browser File Source (`chunkSize`) → Spread → **Gather** (Map `join(params.items, '')`) → SQL import
+- Collecting what a Spread sent through other services back into one list: … → Spread → Map (`overwrite` mode, rewriting `item` and keeping the rest of the part) → **Gather** → …