@readymadeit/mcp 0.0.0-stage → 0.1.0

package/bundle/docs/content/services/imap-email.md

@@ -0,0 +1,104 @@
+# IMAP Email
+
+Listens on an IMAP mailbox and pushes each new incoming email as a structured object into the pipeline.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| hkp-node | `imap-email` |
+
+---
+
+## What it does
+
+The IMAP Email service connects to any IMAP server and uses the IDLE command to receive server-push notifications when new mail arrives. Each new message is fetched, converted into an email envelope object, and emitted downstream — making it a **source service** that drives the pipeline without needing an upstream trigger.
+
+The service keeps the connection up by itself. A connection that goes away — a socket
+reset when a laptop lid closes, a server restart, a network that came back on a different
+address — is reconnected automatically, retrying with an exponential back-off from 2 seconds
+up to a minute until it succeeds. IDLE is renewed every five minutes so that a connection
+which died without saying so is noticed rather than waited on.
+
+Reconnecting resumes where the last connection left off: mail that arrived while the
+connection was down is delivered on the next connection, and mail already delivered is not
+delivered twice. Only the very first connection to a mailbox starts at its current end, so
+existing mail is not replayed. If the server resets its UIDVALIDITY the mailbox is treated as
+new again.
+
+Whether the service should be listening is part of its state, so a board saved (or deployed)
+while listening starts listening again on load — nobody has to be watching for it to come
+back up.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `host` | `string` | `""` | IMAP server hostname (e.g. `imap.gmail.com`) |
+| `port` | `number` | `993` | IMAP server port |
+| `username` | `string` | `""` | Login username / email address |
+| `password` | `string` | `""` | Login password or app-specific password |
+| `tls` | `boolean` | `true` | Use TLS (recommended — disable only for plain-text port 143) |
+| `mailbox` | `string` | `"INBOX"` | Mailbox folder to watch |
+| `enabled` | `boolean` | `false` | Whether to listen; persisted, so a saved board reconnects on load |
+| `connect` | `boolean` | — | Send `{ connect: true }` to start, `{ connect: false }` to stop |
+| `disconnect` | `boolean` | — | Send `{ disconnect: true }` to stop the connection |
+
+### Read-only state fields
+
+| Field | Type | Description |
+|---|---|---|
+| `running` | `boolean` | Whether a connection is up right now |
+| `status` | `string` | `disconnected`, `connecting`, `connected` or `reconnecting` |
+| `reconnectAttempts` | `number` | Attempts since the connection was last up; `0` while connected |
+| `error` | `string` | Last error message, if any |
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | Any value — passed through unchanged when the pipeline is triggered externally |
+| **Output (push)** | `{ subject, from, to, date, messageId, references, inReplyTo, uid, text }` — emitted for each new email |
+
+### Output object fields
+
+| Field | Description |
+|---|---|
+| `subject` | Email subject line |
+| `from` | Sender address(es), comma-separated |
+| `to` | Recipient address(es), comma-separated |
+| `date` | ISO 8601 arrival date string |
+| `messageId` | Message-ID header, without its angle brackets |
+| `references` | The References chain, oldest first — `references[0]` is the message that began the thread |
+| `inReplyTo` | In-Reply-To header, or `""` when the message starts a thread |
+| `uid` | IMAP UID of the message, stable within the mailbox |
+| `text` | Plain-text body, empty when the message could not be parsed |
+
+**Which conversation a message belongs to is stated by the sending client**, in
+`references` and `inReplyTo`, and reading it there is much steadier than
+recovering it from a subject: clients rewrite subjects, and the reply prefix
+differs by language (`RE:`, `AW:`, `SV:`, `Antw:`). The root of the chain is the
+identity every reply in a thread shares:
+
+```
+references[0] || inReplyTo || messageId
+```
+
+A message that starts a thread has neither header, so it is its own root.
+Angle brackets are stripped everywhere, because they are header syntax rather
+than part of the identity — left on, the same message compares unequal
+depending on which header it was read from.
+
+---
+
+## Typical uses
+
+- IMAP Email → Monitor — log every incoming email to the console
+- IMAP Email → Map → HTTP Uploader — forward email metadata to a webhook
+- IMAP Email → Filter → Notification — alert on emails matching a subject pattern

package/bundle/docs/content/services/injector.md

@@ -0,0 +1,129 @@
+# Injector
+
+Injects a configured static value into the pipeline, replacing or providing the current data.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `hookup.to/service/injector` |
+| hkp-rt | `injector` |
+| hkp-node | `injector` |
+
+---
+
+## What it does
+
+Injector stores a value (set via its UI or configuration) and emits it
+on every pipeline trigger instead of the incoming data. If no value has
+been configured, it falls back to passing the incoming data through
+unchanged.
+
+The most common use is to provide a constant payload — a fixed API
+request body, a default configuration object, or a test fixture — that
+can be swapped in and out without changing the upstream pipeline.
+
+Injector can also be triggered imperatively: sending
+`command.action: "inject"` with a value causes it to fire immediately,
+bypassing the normal pipeline flow.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `inject` | `any` | `undefined` | Value to store and emit. Setting this fires an immediate `app.next()` |
+| `injectBinary` | `string` | `undefined` | hkp-rt and hkp-node only: base64 payload to store and emit as binary |
+| `recentInjection` | `any` | `undefined` | Restore the last injected value on board load (set automatically) |
+| `plainText` | `boolean` | `false` | Treat the stored value as a plain string rather than parsed JSON |
+
+### Behaviour
+
+| Condition | Output |
+|---|---|
+| `recentInjection` is set | Emits `recentInjection` on every trigger |
+| `recentInjection` is unset | Passes incoming `params` through unchanged |
+
+Setting `inject` both stores the value as `recentInjection` AND
+immediately calls `app.next()` with the value, triggering the downstream
+pipeline without waiting for an upstream event.
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | Any value (used as fallback when no injection is set) |
+| **Output** | The configured `recentInjection` value, or the input if none is set |
+
+---
+
+## hkp-rt specifics
+
+The hkp-rt service behaves identically, with the differences that come
+from living behind a REST API rather than in the same process as the UI:
+
+- A JSON string is injected as `String`, any other JSON value as `JSON`.
+- Files are injected as base64 through `injectBinary` and arrive
+  downstream as `BinaryData`. Text files still use `inject`.
+- The reported state carries `recentInjectionSize` (byte count) instead
+  of `recentInjection` after a binary injection, so that large payloads
+  are not echoed back to the UI.
+- Injections are pushed downstream asynchronously (through the runtime's
+  io_context), so `configure` returns before the pipeline has run.
+
+Run the example config:
+
+```bash
+./build/hkp-rt/exe/hkp-rt 8887 127.0.0.1 boards/runtime-configs/injector-example.json
+```
+
+---
+
+## hkp-node specifics
+
+Same behaviour, and the same REST-vs-in-process differences as hkp-rt above:
+
+- `inject` and `injectBinary` push the injected value through the rest of
+  the pipeline immediately — the runtime runs the services after Injector
+  and reports the result over the runtime's websocket — rather than
+  waiting for the next call to reach the service.
+- `injectBinary` is decoded from base64 into a `Buffer`. hkp-node's
+  pipeline carries binary payloads as `Uint8Array` (the same shape
+  `http-server` and `document-extract` use), not as a distinct binary
+  type.
+- The reported state carries `recentInjectionSize` (byte count) instead
+  of `recentInjection` after a binary injection, for the same reason as
+  hkp-rt: a large payload is not echoed back to the UI.
+
+See the Node runtime in the demo board below for a working example
+(needs hkp-node, kept under the name `node` in *Manage runtime servers*).
+
+---
+
+## Example: fixed API request body
+
+Store a JSON object that is always sent by the downstream Fetcher:
+
+```json
+{
+  "recentInjection": { "query": "SELECT * FROM events LIMIT 100", "format": "json" }
+}
+```
+
+## Example: fire a one-time trigger
+
+Send a specific payload immediately via `configure`:
+
+```json
+{
+  "inject": { "action": "reset", "timestamp": 0 }
+}
+```
+
+This stores the value AND fires it immediately through the downstream
+pipeline.

package/bundle/docs/content/services/input.md

@@ -0,0 +1,90 @@
+# Input
+
+Connects to a remote stream and injects each arriving message as a new pipeline trigger.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `hookup.to/service/input` |
+
+---
+
+## What it does
+
+Input opens a persistent connection to a URL and, on every message
+received, runs the full downstream pipeline with that message as its
+input. It is the counterpart to the [Output](./output.md) service, and
+the standard way to pull a continuous data stream from another runtime
+or external source into a browser pipeline.
+
+The connection is established as soon as a URL is configured and
+re-established automatically if it drops.
+
+---
+
+## Modes
+
+| Mode | Protocol | Use case |
+|---|---|---|
+| `eventsource` (default) | HTTP Server-Sent Events (SSE) | One-way streaming from server to browser; auto-reconnects |
+| `websocket` | WebSocket | Bidirectional, low-latency messaging |
+
+The mode is detected automatically when a `ws://` or `wss://` URL is
+provided. A URL starting with `http://` or `https://` always uses
+`eventsource`.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `url` | `string` | `""` | The stream endpoint URL |
+| `mode` | `"eventsource" \| "websocket"` | `"eventsource"` | Connection mode (auto-detected from URL scheme) |
+| `parseEventsAsJSON` | `boolean` | `true` | Parse message bodies as JSON before emitting |
+| `bypass` | `boolean` | `false` | When `true`, explicitly disconnects and stops receiving |
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | Ignored — Input generates its own pipeline triggers |
+| **Output** | Each incoming message, optionally parsed as JSON |
+
+When `parseEventsAsJSON` is `true` (the default) and a message body is
+valid JSON, the parsed object is emitted. If parsing fails the raw
+string is emitted as a fallback.
+
+---
+
+## Typical use: pairing with Output
+
+Place an **Output** service at the end of one pipeline to push data over a
+WebSocket, and an **Input** service at the start of another to receive it:
+
+```
+Board A                    Board B (or anything else listening)
+──────────────────         ────────────────────────────
+Timer → Map → Output ─WS─► Input → Process → Canvas
+                    url="ws://localhost:9000/stream"
+```
+
+This is for crossing *out* of a board — to a separate board, or to software that
+is not part of Readymade at all. Runtimes **within one board** are already chained in
+board order, and passing data from one to the next needs no services; see the
+Architecture page.
+
+Output configuration on Runtime A:
+```json
+{ "url": "ws://localhost:9000/stream", "mode": "websocket", "flow": "stop" }
+```
+
+Input configuration on Runtime B:
+```json
+{ "url": "ws://localhost:9000/stream", "mode": "websocket" }
+```

package/bundle/docs/content/services/iterator.md

@@ -0,0 +1,186 @@
+# Iterator
+
+Runs a nested pipeline once per item. Iteration you can see in the board.
+
+> Not to be confused with the browser's [Looper](./looper.md), which records a
+> stream of values and replays them at their original timing. Different idea,
+> similar-sounding name.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `iterator` |
+| Node.js (hkp-node) | `iterator` |
+| Python (hkp-python) | `iterator` |
+| C++ (hkp-rt) | `iterator` |
+
+The same board reads the same in each. Where they differ:
+
+- **C++** collects what an item answers as JSON, as a [Tracks](./tracks.md)
+  branch's is: there is no data type for a list of audio buffers. An item
+  answering with audio or bytes contributes nothing, as one that stopped does.
+  An input that is not JSON is one item, handed over untouched.
+- **Browser**: a service inside that throws is reported by the runtime, which
+  carries on, so `failed` counts only what escapes it.
+
+---
+
+## What it does
+
+A pipeline pass carries **one** value. That is a problem for anything producing
+several — a poll finding four conversations, a query returning twenty rows.
+
+Without Iterator there are only bad answers. Hand the whole array to the next
+service, and it — and everything after it — has to loop internally. Or let the
+service that found the items call the rest of the pipeline itself, which buries
+iteration inside whichever service happened to need it first, where no other
+service can reuse it and nobody reading the board can see it.
+
+Iterator is the third answer. The service that produced the array says only what
+it found; iterating over it is a service of its own, sitting in the board where
+you can see it.
+
+```
+timer → conversations (actionable) → iterator ┐
+                                              ├─ text-generation  (decide)
+                                              └─ conversations    (transition)
+```
+
+The timer ticks once. The two services **inside** the Iterator run once per
+waiting conversation.
+
+### A single item is an array of one
+
+An input that is not an array is iterated as a list with one element. A board
+that grows from "the one that arrived" to "the four that were waiting" does not
+change shape, and neither does one fed by something that sometimes returns a
+single row.
+
+An object counts as one item, not as its values.
+
+---
+
+## What comes back
+
+The results of each pass, as an array.
+
+**An item whose pipeline stopped contributes nothing.** `null` means *nothing to
+pass on* here exactly as it does everywhere else, so a sub-pipeline shaped like
+a filter makes the Iterator a filter:
+
+```
+iterator ┐
+         ├─ map      (work out whether this one matters)
+         └─ filter   (null for the ones that do not)
+```
+
+**Nothing collected at all stops the outer pipeline.** No items, or every item
+stopped, and nothing is passed on — rather than continuing with an empty array.
+
+### One item at a time
+
+Items are taken in turn, each awaited before the next begins. A pass that calls
+a model is then one request in flight rather than ten at once, which is what an
+inference provider's rate limit wants. The cost is that a poll takes as long as
+its items put together — ten conversations at twenty seconds each is a
+three-minute pass.
+
+Because the runtime awaits, a service that takes its time is still just a
+service: what follows it in the nested pipeline runs with its answer, and a
+[Join](./join.md) around it carries the item past it.
+
+---
+
+## One item failing does not end the loop
+
+Nine conversations should not go unprocessed because the tenth had a malformed
+address. A pass that throws is counted and reported, and the loop carries on.
+
+Every pass reports `{ items, results, failed }`, and the same counts are
+readable as `lastItems` / `lastFailed` — so an Iterator that is quietly losing
+items is visible rather than silent.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `pipeline` | `array` | `[]` | The nested services, in order |
+| `itemsFrom` | `string` | `""` | Dotted path to the array in the input; the input itself by default |
+| `limit` | `number` | `0` | Take at most this many items. `0` = all |
+| `bypass` | `boolean` | `false` | Pass the input through untouched |
+| `lastItems` | `number` | — | Read-only: how many the last pass iterated |
+| `lastFailed` | `number` | — | Read-only: how many of those threw |
+
+`itemsFrom` is how an Iterator reaches into a result that says more than the
+list itself. [Conversations](./conversations.md)' `actionable` answers
+`{ conversations, count }` rather than a bare array, so that it can say how many
+it found; `itemsFrom: "conversations"` takes the list out of it without the
+service having to flatten its answer for whatever comes next.
+
+An Iterator with an empty pipeline, or one that is bypassed, passes its input
+straight through — the same thing an empty SubService does, so a half-built
+board stays legible rather than going quiet.
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | an array, or a single item |
+| **Output** | the collected results, or **nothing at all** when none were collected |
+
+---
+
+## Runs and logs
+
+Each item runs as a **run of its own**, descended from the run that reached the
+Iterator. Without that, ten items processed on one timer tick would all be
+logged as that one tick, and *what happened to this one* would be unanswerable.
+With it, each item's journey through the nested pipeline is its own trace with
+the tick as its parent.
+
+Nested services log to the board's log like any other, and report their
+notifications out through the Iterator.
+
+---
+
+## Example
+
+Acting on every conversation waiting for a decision:
+
+```json
+{
+  "uuid": "per-conversation",
+  "serviceId": "iterator",
+  "serviceName": "Iterator",
+  "state": {
+    "itemsFrom": "conversations",
+    "pipeline": [
+      {
+        "serviceId": "conversations",
+        "uuid": "read-thread",
+        "state": { "mode": "thread" }
+      },
+      {
+        "serviceId": "text-generation",
+        "uuid": "decide",
+        "state": { "backend": "server", "model": "…" }
+      },
+      {
+        "serviceId": "conversations",
+        "uuid": "record-decision",
+        "state": {
+          "mode": "transition",
+          "states": ["init", "waiting-approval", "waiting-reply", "done"]
+        }
+      }
+    ]
+  }
+}
+```

package/bundle/docs/content/services/join.md

@@ -0,0 +1,153 @@
+# Join
+
+Splits off a piece of work and gets the answer back **beside** what you already
+had, rather than instead of it.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Node.js (hkp-node) | `join` |
+| Python (hkp-python) | `join` |
+
+---
+
+## What it does
+
+Most services replace their input. `text-generation` answers with an answer, not
+with the question and the answer. That is right for the service and wrong for the
+pipeline around it: whatever the input was carrying — which conversation this is,
+which record it came from — is gone by the time the answer arrives, and whatever
+has to file the answer no longer knows where it belongs.
+
+```
+input ──┬──────────────────────────────► merged output
+        └── nested pipeline ── result ──┘
+```
+
+The carrier never goes anywhere, because the nested pipeline is a **detour**
+rather than the road.
+
+The alternatives are all worse. Teaching every service to pass its input through
+makes every output a pile of everything that ever touched it. Asking a model to
+echo an id back makes correctness a matter of the model being careful. Keeping
+the id somewhere on the side stops working the moment two runs overlap.
+
+---
+
+## A detour that takes its time
+
+The runtime awaits every service, so a nested `text-generation` taking a minute
+is waited for and its answer merged like any other. Nothing about a slow detour
+needs saying in the board.
+
+That is the reason the pipeline awaits at all. A service that answered late by
+calling the rest of the pipeline itself left no input for anything to re-join
+with — the question was gone by the time the answer existed.
+
+---
+
+## `as` is the safe way to use it
+
+Naming a key puts the result there, and nothing the nested pipeline produces can
+collide with what the input was carrying:
+
+```json
+{ "as": "extraction" }
+```
+
+```
+in  { conversationId, emails }
+out { conversationId, emails, extraction: { … } }
+```
+
+Merging at the top level is available for the cases where the two shapes are
+known to be disjoint. Then `mode` decides a collision: `overwrite` (default)
+lets the nested answer win, `add` protects what the input was carrying.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `pipeline` | `array` | `[]` | The detour, in order |
+| `as` | `string` | `""` | Put the result under this key. Empty merges at the top level |
+| `mode` | `string` | `"overwrite"` | Top-level merge only: `overwrite` or `add` |
+| `onFailure` | `string` | `"stop"` | A detour that failed: `stop`, or `merge` its `{ error, service }` in place of the result — under `as`, or under `failure` without one |
+| `bypass` | `boolean` | `false` | Pass the input through untouched |
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | anything |
+| **Output** | the input and the result together, or **nothing** when the detour produced nothing |
+
+A nested pipeline that stops stops this one too. Continuing with the input alone
+would hand the services downstream something indistinguishable from a merge that
+worked — which is how an empty payload gets filed under a name claiming
+otherwise.
+
+A scalar or array on either side has no fields to combine: with `as` the input
+is kept under `input`, and without it the half that has fields wins.
+
+---
+
+## When the detour fails
+
+A detour can end with nothing for two reasons. A service **chose to stop** — a
+filter, a claim that found nothing to take — and that is flow: the pipeline
+stops, whatever `onFailure` says. Or a service **failed**, and said why in an
+`error` notification. By default that stops the pipeline too, and the reason
+exists only in a notification nobody may have been listening to.
+
+With `onFailure: "merge"` the failure goes on instead, where the result would
+have gone, so the services after the Join can record it beside the thing it
+happened to:
+
+```
+in  { link, title }
+out { link, title, brief: { error: "no value stored for anthropic.key", service: "write" } }
+```
+
+The failure always arrives under a name: `as`, or `failure` when the Join
+merges at the top level. Spread into the input like a result, it could be
+hidden by an `error` the input already carried, or replace a field of its own.
+
+A board then routes on it — a map that sends `brief.error` one way and
+`brief.text` the other — and a card, a row or a log can say *failed, and why*
+rather than nothing at all. Only a service directly in the detour is heard; one
+inside a scope of its own reports to that scope. Two runs through one Join at
+the same time never hear each other's failures: each hears its own detour.
+
+---
+
+## Example
+
+Asking a database a question about the record in hand, without losing the record:
+
+```json
+{
+  "uuid": "with-history",
+  "serviceId": "join",
+  "serviceName": "Join",
+  "state": {
+    "as": "history",
+    "pipeline": [
+      {
+        "serviceId": "sql",
+        "uuid": "past-bookings",
+        "state": {
+          "mode": "query",
+          "statement": "SELECT * FROM booking WHERE guest = $guest ORDER BY sentAt DESC LIMIT 5"
+        }
+      }
+    ]
+  }
+}
+```