@readymadeit/mcp 0.0.0-stage → 0.1.0

package/bundle/docs/content/services/storage.md

@@ -0,0 +1,155 @@
+# Storage
+
+A place on the board where a board keeps bytes — audio it rendered, a file
+somebody uploaded, an export — addressed by path and backed by whatever the
+board points it at.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Node.js (hkp-node) | `storage` |
+
+---
+
+## What it does
+
+Storage is the board's name for a place things are kept. What actually holds
+them is its **nested pipeline**: [Filesystem](./filesystem.md) today, something
+speaking S3 or SQL the day a board needs that, with nothing in front of it
+changing. That indirection is the whole service. Without it, one backend's
+vocabulary spreads through every board that keeps anything.
+
+It is deliberately not a record store. [Store](./store.md) keeps JSON a board
+wrote and [SQL](./sql.md) keeps rows a board queries; what neither can hold is a
+byte stream something outside the board will read.
+
+## The contract is a request, not a set of paths
+
+A request says what it wants done, to what, and with what:
+
+```json
+{ "op": "write", "path": "episodes/one.mp3", "binary": "…the bytes…" }
+```
+
+| `op` | Does | Answers with |
+|---|---|---|
+| `read` | reads one file | `meta` + `binary`, or a 404 |
+| `write` | writes one file | what was written: path, size, modified |
+| `list` | every file under a prefix | `files`, `count` |
+| `stat` | asks whether a file is there | `exists`, and its size if it is |
+| `delete` | removes one file | whether there was one to remove |
+
+Bytes may arrive as `binary` (a `Uint8Array` from another service or an HTTP
+body), as `base64` (for a caller with only JSON to send), or as `body` — a
+string or an object, for the boards that write a feed or a manifest and would
+otherwise need an encoder service in between.
+
+The same request stands whether it arrived as a value in a pipeline, as JSON
+over a mount, or as a plain HTTP call, which is what keeps this from being an
+API somebody has to learn twice.
+
+### An HTTP request is read as the same thing
+
+| Request | Means |
+|---|---|
+| `GET /episodes/one.mp3` | `read` |
+| `GET /episodes/` or `GET /` | `list` |
+| `HEAD /episodes/one.mp3` | `stat` |
+| `PUT` / `POST` | `write`, with the body as the bytes |
+| `DELETE` | `delete` |
+
+So a browser, a podcast client or `curl` reaches a store without being told
+anything, and a read comes back as what it is — `audio/mpeg` for an mp3 — with
+range requests answered so a player can seek. See
+[the response envelope](./http.md#what-a-handler-may-answer-with).
+
+**A listing is JSON, and stays JSON.** Opening the endpoint itself in a browser
+shows the listing as data rather than as a page, because a store answers callers
+rather than readers; what turns it into something to click is a panel, or a feed.
+A single file opened in a browser does play — an `audio/mpeg` response is all a
+browser needs.
+
+A listing is also the raw material for a document some other program does
+understand. A board that wants one publishes it beside the store rather than
+asking the store to speak that language: `sql` over `json_each($rows)` joins the
+listing into the document — an M3U playlist, a feed — a `map` gives it a content
+type, and a second endpoint publishes it from a slot. The store keeps its
+one job, and the board gains an address it can hand to a media player. The
+library unit of the reading-radio boards does exactly this.
+
+A caller with a storage request to make can also POST it as JSON to the same
+endpoint; a body carrying an `op` is read as the request rather than as bytes
+to keep.
+
+## Configuration
+
+| Property | Type | Description |
+|---|---|---|
+| `pipeline` | `array` | The backend. Required — a store with none says so rather than inventing one |
+| `prefix` | `string` | Confines every path to one part of the store |
+| `readOnly` | `boolean` | Answers reads and refuses everything else |
+| `operation` | `string` | What a request naming no `op` means (default `read`) |
+
+### `readOnly` is what makes a public endpoint safe
+
+A [mount](../concepts/mounts.md) is unauthenticated by design, so the instance
+behind one answers reads and nothing else. The instance a board writes through
+is a different one, reachable only from inside. Two instances over one place,
+with the policy in the board where it can be read, rather than a flag buried in
+whatever happens to hold the bytes.
+
+`prefix` is the other half of that: a board can hand out an endpoint over its
+episodes without handing out one over everything else it kept.
+
+It is **invisible from outside**. A caller asks for `one.mp3`, so a listing
+answers `one.mp3` — not `episodes/one.mp3`, which asked for at that same endpoint
+would resolve to `episodes/episodes/one.mp3`. A path a listing gives out has to
+be one that can be asked for again.
+
+## Two boards sharing one place
+
+Storage keeps nothing itself, so what two boards share is whatever their
+backends name — for Filesystem, a **volume**. Naming the same volume is how a
+board that renders audio and a board that publishes it work over one library, the
+same way two boards share tables by naming the same [database](./sql.md).
+
+## Example
+
+A public endpoint over a library, read-only:
+
+```json
+{
+  "uuid": "listen",
+  "serviceId": "http-server-subservices",
+  "state": {
+    "mountName": "listen",
+    "pipeline": [
+      {
+        "instanceId": "episodes",
+        "serviceId": "storage",
+        "state": {
+          "readOnly": true,
+          "prefix": "episodes",
+          "pipeline": [
+            { "instanceId": "files", "serviceId": "filesystem", "state": { "volume": "radio" } }
+          ]
+        }
+      }
+    ]
+  }
+}
+```
+
+## Demo board
+
+`storage-demo-board.json` — writes what you type into a volume and serves it
+back at an address.
+
+## See also
+
+- [Filesystem](./filesystem.md) — the backend this uses by default
+- [HTTP](./http.md) — the endpoint in front of it, and what it may answer with
+- [Store](./store.md) — records rather than files

package/bundle/docs/content/services/store.md

@@ -0,0 +1,255 @@
+# Store
+
+What a board remembers between runs — keyed, durable, and scoped to the board.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Node.js (hkp-node) | `store` |
+
+---
+
+## What it does
+
+Store keeps records for one board and hands them back later. Each record has a
+key, a value, and the times it first arrived and was last written.
+
+It is deliberately **not a database**. It is the missing half of two patterns
+that otherwise cannot be expressed in a single board:
+
+- **Dump cheaply, process expensively.** Something arrives, is stored, and costs
+  nothing more. A timer later reads the batch and does the expensive work once.
+  Without somewhere to put it, the only choices are to process every arrival
+  the moment it lands, or to lose it.
+- **Work that outlives whoever started it.** Somebody clicks a button, a model
+  takes a minute, and they close the tab. The answer lands here, and is still
+  here when they come back.
+
+### Which records a `store` sees
+
+Records belong to **the board, and the tenant that owns it**. Both come from the
+runtime rather than from the service's configuration — a service is told its own
+state and nothing about who asked for it, so a board cannot widen its own scope
+by asking. Two runtimes of one board share one table; two boards do not, and two
+people never do.
+
+This is why **every `store` on a board reads and writes the same records without
+being configured to.** A board with three of them — one writing, one listing,
+one releasing — needs no shared identifier, because there is nothing to
+identify: `key` names a *record within* a table, never the table itself.
+
+Nesting does not change it either: a `store` inside a `sub-service` sees the
+same table as one beside it.
+
+### More than one table on a board
+
+`namespace` subdivides the board's table. Leave it empty — the default — and
+there is one table, which is what most boards want. Set it where a board keeps
+two unrelated sets of things that would otherwise arrive in one `list`:
+
+```json
+{ "serviceId": "store", "state": { "mode": "put",  "namespace": "enquiries" } },
+{ "serviceId": "store", "state": { "mode": "list", "namespace": "enquiries" } },
+{ "serviceId": "store", "state": { "mode": "list", "namespace": "invoices"  } }
+```
+
+A namespace only ever *narrows* what the runtime already granted, so unlike the
+tenant and the board it is safe for a board to choose. Clearing one namespace
+leaves the others alone, and clearing the board's own table leaves every
+namespace in it untouched.
+
+---
+
+## Modes
+
+| Mode | What it does | Emits |
+|---|---|---|
+| `put` (default) | Writes a record | `{ key, value, createdAt, updatedAt }` |
+| `get` | Reads one back | the record, or **nothing at all** on a miss |
+| `list` | Records of the board, oldest first | `{ records: [...], count }` |
+| `delete` | Removes one | `{ key, deleted }` |
+| `clear` | Empties the table | `{ cleared }` |
+| `release` | Hands named records to the pipeline, one pass **per record** | the record |
+| `ack` | Settles the record this run was carrying — it finally leaves | `{ key, acknowledged }` |
+| `requeue` | Puts stranded records back where anyone can take them | nothing |
+
+**A `get` miss stops the pipeline.** That is the cache signal: whatever follows
+is the "go and fetch it" path, so it runs only when the lookup found nothing.
+
+```
+store (get) → http-client (fetch it) → store (put) → ...
+```
+
+---
+
+## A queue with a checkpoint in it
+
+The last three modes turn the table into a work queue that something outside the
+board — usually a person — decides the pace of.
+
+`release` takes a list of keys and sends each of those records down the rest of
+the pipeline, **one pass per record**. Everything else stays where it was: the
+queue is what has *not* been dealt with. One pass per record rather than one
+batch, because what follows an approval is per-item work — read this document,
+write this row — and a board should not have to unpack a batch to do it.
+
+The keys arrive as **input**, so whoever decides sends them with the pipeline's
+`process` entry point: a facade button, an HTTP call to a mount, or a service
+upstream that picked them itself.
+
+### Nothing is lost when the work fails
+
+A released record is **leased, not deleted**. It leaves the queue, but stays on
+disk until an `ack` says the work finished:
+
+```
+store(release) → … → http-client(Baserow) → store(ack)
+                                            ↑ the success point
+```
+
+Where you put the `ack` is how the board says what "done" means. That has to be
+declared, because **the pipeline cannot report its own outcome**: a service that
+answers late returns nothing from its pass, so success and failure look
+identical to whatever called it.
+
+An `ack` with no key settles the record **this run** was carrying. `release`
+hands each record to a run of its own and writes that run onto the record, and
+the run id threads through every service after it — including the ones that
+answer long after the pass that started them. So the acknowledgement finds its
+record without the board carrying the key through every step, which the services
+in between would otherwise drop. Naming a key outright still works where a board
+prefers it.
+
+Anything that never reaches the `ack` stays in flight: out of the queue, so
+nobody takes it twice, and still there. `requeue` is how it goes back.
+
+Every record says which state it is in, so one table can show both:
+
+```json
+{ "type": "data-table",
+  "source": { "serviceUuid": "queue", "path": "records" },
+  "columns": ["state", "key", "value.subject"] }
+```
+
+| `show` | Returns |
+|---|---|
+| `"waiting"` (default) | Records nobody is dealing with — the queue |
+| `"in-flight"` | Released and not yet acknowledged |
+| `"all"` | Both, which is what a single table wants |
+
+`state` is reported rather than left to be inferred from whether a lease is
+present — a column showing a timestamp, blank when absent, is not something a
+reader should have to decode.
+
+Acting on a mixed selection is safe: `release` passes over what is already in
+flight, so a second click cannot start the same work twice, and `requeue` passes
+over what was never handed out. Records already settled are passed over without
+complaint — two people looking at one queue is the normal case, not an error.
+
+**There is no lease timeout yet.** A record whose pipeline died stays in flight
+until somebody returns it. That is deliberate for now: this queue exists because
+a human is in the loop, and the same human can decide a stranded record is worth
+another go.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `mode` | `string` | `"put"` | One of the eight above |
+| `namespace` | `string` | `""` | Which of the board's tables. Empty is the board's own |
+| `key` | `string` | `""` | The key to act on, when every pass means the same slot |
+| `keyFrom` | `string` | `""` | Dotted path into the input to take the key from, e.g. `meta.messageId` |
+| `valueFrom` | `string` | `""` | Dotted path to the part worth keeping; the whole input by default |
+| `limit` | `number` | `0` | `list`: how many records to take, oldest first. `0` = all |
+| `show` | `string` | `"waiting"` | `list`: `waiting`, `in-flight`, or `all` |
+| `lastCount` | `number` | — | Read-only: what the last pass saw |
+
+**Where the key comes from**, in order: `keyFrom` if it leads anywhere, then a
+`key` field on the input, then a bare string input (for a lookup), then the
+configured `key`. A `put` with no key at all gets one that sorts by when it was
+made, so an unnamed dump keeps its arrival order.
+
+An input shaped `{ key, value }` is understood as a record handed over as one:
+the wrapper is not what gets stored.
+
+`release` and `requeue` name their records on the **input**, as
+`{ "keys": [...] }` or the keys themselves. Whole records work too, so a service
+upstream can hand them over without unpacking.
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | anything to keep (`put`), or the key(s) to act on (`get`, `delete`, `release`, `requeue`) |
+| **Output** | see the mode table above |
+
+**The outcome is pushed, not returned.** The disk answers asynchronously and
+the pipeline does not wait, so the service returns nothing, stops the push, and
+calls the rest of the pipeline itself once the record is written or read (the
+same inversion-of-control path `http-client` takes). For a board this is
+invisible.
+
+Anything that goes wrong — an unwritable directory, a missing key on a mode that
+needs one — is reported as `{ error }` and passes nothing on. A board that
+carried on here would be acting on a record that was never written.
+
+---
+
+## Where it lives
+
+Records are one JSON file each, under a directory the runtime owns:
+
+```
+~/.hkp/node-<instance>/store/<sha256(owner)>/<sha256(board)>/[<sha256(namespace)>/]<sha256(key)>.json
+```
+
+A namespace is a directory *inside* the board's own, so the board's default
+table keeps the path it always had, and a namespaced one cannot be reached from
+outside the board. Records are files ending in `.json` and namespaces are
+directories, so listing one never sees the other.
+
+Set `HKP_STORE_DIR` to move it. `HKP_STORE_DIR=""` keeps records **in memory**
+— they last as long as the process and no longer, which is what tests and
+throwaway runs want.
+
+The names in that path are derived, never used directly: every part of it comes
+from outside, so a key called `../../etc/passwd` cannot escape the root, and
+`Key` and `key` — two keys here — cannot become one file on a case-insensitive
+filesystem. The real names are inside the file.
+
+Records carry whatever a board put in them, which for these workflows is
+correspondence — so directories are `0o700` and files `0o600`, the owner's to
+read and nobody else's. Writes go to a temporary name and are renamed into
+place, so a crash mid-write costs the record being written rather than the
+table.
+
+---
+
+## Example
+
+The attached demo board is the cheap-dump loop end to end: a webhook drops
+enquiries in without processing them, and a timer picks up the batch later.
+
+```json
+{
+  "node": [
+    { "serviceId": "http-server-subservices", "state": { "pipeline": [
+      { "serviceId": "store", "state": { "mode": "put", "keyFrom": "body.id" } }
+    ] } },
+    { "serviceId": "timer", "state": {
+      "periodic": true, "periodicValue": 1, "periodicUnit": "m", "start": true } },
+    { "serviceId": "store", "state": { "mode": "list", "limit": 10 } },
+    { "serviceId": "monitor" }
+  ]
+}
+```
+
+POST an enquiry to the mount and nothing expensive happens. A minute later the
+batch is there to work through.

package/bundle/docs/content/services/sub-service.md

@@ -0,0 +1,152 @@
+# SubService
+
+A service that contains its own ordered pipeline — the primary way to build a higher-level block out of smaller ones.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `sub-service` |
+| hkp-node | `sub-service` |
+| hkp-python | `sub-service` |
+| hkp-rt | `sub-service` |
+
+The browser runtime adds a second mode of its own; see
+[Browser Sub-Service](./browser-sub-service.md).
+
+---
+
+## What it does
+
+A runtime is an ordered list of services, and that order is the wiring. A
+SubService is one entry in that list holding a list of its own: input goes in at
+the head of the nested pipeline, and what comes out the end is the SubService's
+own output.
+
+That is the whole of it by default — which makes it a way of naming a group of
+services and reusing them. Two further settings turn it into a
+[scope](../concepts/scopes.md): a boundary that says whether its answer leaves
+and what its contents can see.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `pipeline` | `PipelineEntry[]` | `[]` | The ordered services inside |
+| `bypass` | `boolean` | `false` | Skip the pipeline; the input passes through |
+| `stopPropagation` | `boolean` | `false` | Whether what the pipeline produced leaves this service |
+| `scope` | `{ slots }` | `{ slots: "own" }` | What the services inside can see |
+
+### `stopPropagation`
+
+Absent means `false`, so a board that says nothing about it passes its result
+along exactly as before.
+
+Set, it closes **both** ways out: the value `process()` answers with, *and*
+anything the pipeline emitted without being asked — a Timer tick, or a service
+that answered nothing and came back later. A scope holding something
+asynchronous needs both, or it goes on driving the board after its own answers
+have stopped.
+
+It composes with `bypass`; the two together are a
+[`stopper`](./stopper.md).
+
+### `scope.slots`
+
+Which cells a [`hold`](./hold.md) inside this pipeline reaches.
+
+| Value | Which cells |
+|---|---|
+| `"own"` (default) | This service's, so two copies of one pipeline do not collide and a slot named inside means nothing outside |
+| `"inherit"` | The runtime's, so a slot named inside is the same cell as one named beside it |
+
+The default is the reusable case: a block whose slot names leaked into its
+surroundings would clash with a second copy of itself. Two Holds that must meet
+across the boundary need it to say `"inherit"`.
+
+### What a configure replaces
+
+In every runtime, a configure naming `pipeline` **replaces the whole pipeline**:
+the services inside are rebuilt from the entries given, so whatever an entry no
+longer says is gone rather than left over from before. Whatever runs inside
+restarts — a Timer's schedule, say. The one exception is a pipeline that differs
+only in names, which the browser renames in place without rebuilding.
+
+Every other field is applied **only when present**: a configure that does not
+mention `stopPropagation` or `scope` leaves them as they were. To put one back,
+name it with its default.
+
+[Blocks](../concepts/blocks.md) rely on both: a use is refreshed by configuring
+it with its whole definition, its own fields filled in with the defaults above.
+
+---
+
+## Addressing what is inside
+
+A service in a nested pipeline is named by the path through the services holding
+it, dot-separated:
+
+```json
+{ "serviceUuid": "list.kept-articles", "path": "rows" }
+```
+
+This works for everything a board does with a service — reading one in a facade
+widget, processing one from a button, configuring one, a notice, and a
+[`hkp-mount://`](../concepts/mounts.md) reference. Nested services report under
+their address too, so what a widget binds to is what the service says.
+
+Entering a nested service runs it and what follows it *inside that pipeline*,
+not in the runtime's own list.
+
+See [Scopes](../concepts/scopes.md) for the whole arrangement and why it exists.
+
+---
+
+## Pipeline entry shape
+
+```json
+{
+  "serviceId": "hookup.to/service/monitor",
+  "instanceId": "shown",
+  "serviceName": "What was held",
+  "state": {}
+}
+```
+
+`instanceId` is unique within its own pipeline, which is why an address is the
+path and not the bare name.
+
+---
+
+## Typical uses
+
+**A reusable block.** A group of services given one name, dropped into a board
+as one entry:
+
+```
+http-client → SubService (parse → filter → shape) → sql
+```
+
+**Two flows on one runtime.** Each its own scope, neither reaching the other:
+
+```
+read  (stopPropagation: true)   →  nothing continues
+  └ rss
+
+list                            ←  entered by the facade
+  └ tracks → sql → … → http-server-subservices
+```
+
+**An endpoint's document, held between passes.** The scope owns the cells, so a
+second endpoint on the same runtime may use the same slot name:
+
+```
+SubService (scope.slots: "own")
+  └ sql → map → http-server-subservices
+                  ├─ onProcess: Hold (slot: document, op: write)
+                  └─ onRequest: Hold (slot: document, op: read)
+```

package/bundle/docs/content/services/switch.md

@@ -0,0 +1,67 @@
+# Switch
+
+Routes its input into the first of several sub-pipelines whose condition holds: the structured if/else of a board.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `hookup.to/service/switch` |
+
+---
+
+## What it does
+
+Each **case** pairs a condition with a pipeline. The Switch evaluates the conditions in order against its input, and the first that is truthy runs its pipeline on that input; what the pipeline answers is what the Switch answers. When no case matches, the `default` pipeline runs; with no default either, the input passes through unchanged. A matched case with an empty pipeline also passes the input through.
+
+Conditions are expressions over `params`, the input: `params.kind == 'audio'`, `params.count > 3`, or, for an input that is a plain value, `params == 'swing'`.
+
+---
+
+## A case is a branch, not a scope
+
+A case's pipeline belongs to the pipeline the Switch sits in, the way the body of an `if` belongs to the code around it:
+
+- **Slots.** Its services hold values in the [slots](./hold.md#where-its-cells-live) of whatever holds the Switch, not in cells of their own. A value set outside the Switch is visible inside every case. A [scope](../concepts/scopes.md) is the thing that keeps cells to itself; a case does not.
+- **Activity.** What happens inside a case is reported outward under `<switch>.<instanceId>`, so a panel or the board overview outside sees it, and a facade can address a service in a case by that path. Because the case is not part of the address, an `instanceId` must be unique across all of a Switch's cases.
+- **Cancelling.** A scope that is [cancelled](./browser-sub-service.md#commands) ends the passes running inside its Switches' cases too.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `cases` | `{ when, pipeline }[]` | `[]` | The cases, tried in order. `when` is an expression over `params`; `pipeline` is a list of pipeline entries (see [Browser Sub-Service](./browser-sub-service.md#pipeline-entry-shape)). |
+| `default` | pipeline entries | `[]` | Runs when no case matches. |
+| `ignoreInnerResult` | `boolean` | `false` | Answer with the original input instead of the case pipeline's result: the case runs for its effects. |
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | Any value; conditions see it as `params` |
+| **Output** | The matched pipeline's result (or the input, with `ignoreInnerResult`); the input unchanged when nothing matched and there is no default |
+
+---
+
+## Example
+
+The [Nested Rhythm](../boards/nested-rhythm-demo-board.md) board picks one of four drum patterns per bar. A [Hold](./hold.md) reads the chosen pattern's name out of a slot, and the Switch routes the bar into the pattern with that name:
+
+```json
+{
+  "serviceId": "hookup.to/service/switch",
+  "instanceId": "patterns",
+  "state": {
+    "cases": [
+      { "when": "params == 'straight'", "pipeline": [ { "serviceId": "sub-service", "instanceId": "straight", "state": { … } } ] },
+      { "when": "params == 'shuffle'",  "pipeline": [ { "serviceId": "sub-service", "instanceId": "shuffle",  "state": { … } } ] }
+    ]
+  }
+}
+```