@readymadeit/mcp 0.0.0-stage → 0.1.0

package/bundle/docs/content/services/sort.md

@@ -0,0 +1,48 @@
+# Sort
+
+Sorts an array of objects by a named property in ascending or descending order.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `hookup.to/service/sort` |
+
+---
+
+## What it does
+
+Sort receives an array of objects and returns a new sorted array. Sorting is done by comparing the value of the `by` property across each object. Non-array inputs are passed through unchanged.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `by` | `string` | `""` | Name of the property to sort by |
+| `dir` | `"asc"` \| `"desc"` | `"desc"` | Sort direction |
+
+If `by` is empty or the input is not an array, the input is returned unchanged.
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | Array of objects |
+| **Output** | Sorted copy of the array |
+
+---
+
+## Example
+
+```json
+{ "by": "score", "dir": "desc" }
+```
+
+Input: `[{ "name": "A", "score": 3 }, { "name": "B", "score": 7 }, { "name": "C", "score": 1 }]`
+Output: `[{ "name": "B", "score": 7 }, { "name": "A", "score": 3 }, { "name": "C", "score": 1 }]`

package/bundle/docs/content/services/sound.md

@@ -0,0 +1,87 @@
+# Sound
+
+Synthesises audio from note events using the Web Audio API, with synth and drum modes.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `hookup.to/service/sound` |
+
+---
+
+## What it does
+
+Sound receives note events from the pipeline and plays them through the browser's Web Audio API. Two generator modes are available:
+
+| Mode | Description |
+|---|---|
+| `synth` | Oscillator-based synthesis. Converts note names (e.g. `"C4"`) to Hz using standard MIDI tuning (A4 = 440 Hz). Configurable waveform and envelope (see *Envelope*). Also accepts `NoteFrame` objects from the Game of Life pipeline. |
+| `drums` | Synthesises kick, snare, and hi-hat patterns using only the Web Audio API — no sample files required. Note names are mapped to drum types via `drumMap`. |
+
+The service passes the incoming data through unchanged after triggering audio playback.
+
+All Sound instances on a page share one audio context. It is opened when the first Sound is created, not when the first note plays, because opening one blocks while the audio device starts (about 150 ms measured in Chromium). A context opened before the page has had a user gesture starts suspended, and is resumed on the first gesture anywhere on the page. It is closed a few seconds after the last Sound goes, so a pipeline rebuilt around its Sounds does not reopen the device.
+
+A note is started the moment its input arrives. It is heard after the latency the browser reports: the context's own buffering plus the output device's. The panel shows that figure ("Heard 12 ms after playing"). A Bluetooth device typically adds far more than wired speakers.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `generator.type` | `"synth"` \| `"drums"` | `"synth"` | Generator mode |
+| `volume` | `number` | `1.0` | Master volume (0–1) |
+| `waveType` | `string` | `"sine"` | Oscillator waveform for `synth` mode. One of `sine`, `triangle`, `square`, `sawtooth`, `organ`, `soft`, `fifth` |
+| `noteDuration` | `number` | `0.3` | How long a `synth` note is held before its release, in `noteDurationUnit` |
+| `noteDurationUnit` | `"s"` \| `"beats"` | `"s"` | Seconds, or beats of a tempo (see *Envelope*) |
+| `attack` | `number` | `0.01` | Seconds a `synth` note takes to rise to full volume |
+| `release` | `number` | `0.03` | Seconds a `synth` note takes to fade once it is no longer held |
+| `tempoSlot` | `string` | `"tempo"` | The slot a tempo in BPM is read from when the input carries none |
+| `drumMap` | `object` | `{ C4: "kick", D4: "snare", E4: "hihat" }` | Map of note name → drum type for `drums` mode |
+| `trigger` | `string` \| `null` | `null` | What to play for an input that names no note of its own, such as a Timer tick: a note name, or in `drums` mode also `"kick"`, `"snare"` or `"hihat"`. Unset, such an input plays nothing. A note the input names always wins. |
+
+### Envelope
+
+A `synth` note rises from silence to full volume over `attack`, is held until
+`noteDuration` has passed since it started, and then fades out over `release`.
+The release comes after the note, so a long one carries it into whatever is
+played next: a pad is a slow attack, a note held for the length of its chord
+and a long release. A note let go before its attack is over fades from where it
+got to.
+
+With `noteDurationUnit` `"beats"` the note is held for beats of a tempo, read
+as each note starts. The tempo is the input's `tempo` where it carries one (a
+[Timeline](./timeline.md) counting beats puts it on every frame), otherwise
+the one held in the slot `tempoSlot` names, as [Timer](./timer.md#beats) reads
+it, and 120 BPM when there is neither. A Sound inside a
+[Tracks](./tracks.md) track cannot see the slots around it, so there the tempo
+has to come with the input.
+
+### Custom waveforms
+
+`organ`, `soft`, and `fifth` are custom waveforms defined by their Fourier partial coefficients, providing more complex timbres than the standard oscillator shapes.
+
+---
+
+## Input shapes
+
+| Shape | Description |
+|---|---|
+| `{ note: "C4" }` | Single note |
+| `[{ note: "C4" }, { note: "E4" }]` | Chord (multiple simultaneous notes) |
+| `{ notes: [{ note: "C4" }, …], tempo? }` | Chord, held in beats of the `tempo` it carries |
+| `{ notes: [{ frequency, velocity, duration }] }` | `NoteFrame` from [Game of Life](./game-of-life.md) (`synth` only) |
+| Anything else | Plays `trigger`, if one is set |
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | Note object, array of note objects, or `NoteFrame` |
+| **Output** | Pass-through (original input unchanged) |

package/bundle/docs/content/services/speech-synth.md

@@ -0,0 +1,46 @@
+# Speech Synth
+
+Speaks a text string aloud using the browser's Web Speech Synthesis API.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `hookup.to/service/speech-synth` |
+
+---
+
+## What it does
+
+Speech Synth receives a text string from the pipeline and speaks it using `window.speechSynthesis`. The available voices are populated from the browser's synthesis engine and can be selected in the UI panel.
+
+The service does not pass the text downstream — it consumes it for speech output and returns `null`.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `voices` | `SpeechSynthesisVoice[]` | `[]` | Available voices (populated by the UI from `speechSynthesis.getVoices()`) |
+
+Voice selection is made through the UI, not programmatically through config.
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | `string` — the text to speak |
+| **Output** | `null` — speech output is a side effect; nothing is forwarded downstream |
+
+Non-string inputs emit an error notification and return `null`.
+
+---
+
+## Browser compatibility
+
+Speech Synthesis is available in all major browsers but voice availability and quality vary by platform and locale. On some browsers (e.g. Chrome) voices are loaded asynchronously; the service notifies the UI when the voice list is updated.

package/bundle/docs/content/services/speech-to-text.md

@@ -0,0 +1,172 @@
+# Speech To Text
+
+Transcribes spoken audio to text with a local open-source Whisper model — in the Python runtime via faster-whisper, or in the C++ runtime via embedded sherpa-onnx.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Python (hkp-python) | `speech-to-text` |
+| C++ (hkp-rt) | `speech-to-text` |
+
+Both runtimes consume the same `FloatRingBuffer` input and emit the same
+transcript JSON, so a board can move the service between them unchanged. The
+hkp-rt version adds a `server` backend (an OpenAI-compatible transcription
+endpoint) alongside its in-process `local` backend.
+
+---
+
+## What it does
+
+Speech To Text consumes a `FloatRingBuffer` of raw audio samples
+(16 kHz mono float32), runs it through a Whisper-family model via
+[faster-whisper](https://github.com/SYSTRAN/faster-whisper), and emits the
+transcript as a JSON object to the next service or runtime. Everything runs
+locally — no audio leaves your machine.
+
+The natural producer is the browser **Audio Input** service in its
+`pcm` output format, which captures the microphone, downsamples to
+16 kHz mono, and ships the recording to the Python runtime as a
+`FloatRingBuffer` over the YAS binary wire format.
+
+---
+
+## Prerequisites
+
+The ML dependencies are an optional extra of hkp-python:
+
+```
+uv sync --extra asr    # in hkp-python
+```
+
+The first transcription downloads the selected model from Hugging Face
+(~250 MB for `small`); subsequent runs use the local cache. No GPU is
+required — the default `int8` compute type is fast on CPU.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `model` | `string` | `"small"` | Model size: `tiny`, `base`, `small`, `medium`, `large-v3`, `distil-large-v3` |
+| `language` | `string` | `"auto"` | ISO language code (e.g. `"en"`, `"de"`) or `"auto"` to detect |
+| `computeType` | `string` | `"int8"` | Quantization: `int8`, `int8_float16`, `float16`, `float32` |
+| `device` | `string` | `"auto"` | `auto`, `cpu`, or `cuda` |
+
+The model loads lazily on first use and reloads when `model`, `device`,
+or `computeType` change.
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | `FloatRingBuffer` — 16 kHz mono float32 samples |
+| **Output** | `{ "text": string, "language": string, "languageProbability": number, "durationMs": number, "segments": [{ "start": number, "end": number, "text": string }] }` |
+
+Non-audio input produces `{ "error": "..." }` instead of crashing the
+pipeline; a missing `[asr]` extra is reported the same way with an
+install hint.
+
+---
+
+## Status notifications
+
+The service reports progress through notifications so UIs stay
+observable during long operations:
+
+| `status` | Meaning |
+|---|---|
+| `idle` | Ready; last transcription finished |
+| `loading` | Downloading / loading the model (see `detail`) |
+| `transcribing` | Inference in progress |
+| `error` | Something failed (input, dependency, or model) |
+
+A facade `status-indicator` widget can bind to `path: "status"` and
+color-map these values directly.
+
+---
+
+## Example
+
+The **Voice Notes** demo board wires three runtimes together —
+record in the browser, transcribe in Python, display in the browser:
+
+```json
+{
+  "recorder": [
+    {
+      "serviceId": "hookup.to/service/audio-input",
+      "state": { "format": "pcm" }
+    }
+  ],
+  "python": [
+    {
+      "serviceId": "speech-to-text",
+      "state": { "model": "small", "language": "auto" }
+    }
+  ],
+  "display": [
+    { "serviceId": "hookup.to/service/monitor" }
+  ]
+}
+```
+
+Start the Python runtime (`hkp-python`), keep it under the name `python` in
+*Manage runtime servers*, open the
+board, hit **Record**, speak, then **Stop & Transcribe**.
+
+---
+
+## hkp-rt (C++ runtime)
+
+The C++ runtime runs the same service with two backends selected by the
+`backend` state:
+
+- **`local`** (default) — a Whisper ONNX model runs in-process via embedded
+  [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx). Compiled in when the
+  runtime is built with `-DHKP_SPEECH_ENABLED=ON` (the default on desktop;
+  off for iOS/Android, which fall back to the server backend).
+- **`server`** — a thin client for an OpenAI-compatible
+  `POST /v1/audio/transcriptions` endpoint. Always available.
+
+### Model
+
+Download a sherpa-onnx Whisper model and point the three path keys at its files:
+
+```
+curl -sL https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-whisper-tiny.tar.bz2 | tar xj
+```
+
+Use a **multilingual** model (e.g. `sherpa-onnx-whisper-tiny`), not an
+English-only `.en` build — the decoder expects a language token that `.en`
+models don't emit.
+
+### Configuration (hkp-rt)
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `backend` | `string` | `"local"` | `local` or `server` |
+| `serverUrl` | `string` | `"http://127.0.0.1:8081"` | Server backend: base URL |
+| `model` | `string` | `"whisper-1"` | Server backend: model name |
+| `encoderPath` | `string` | `""` | Local backend: Whisper encoder ONNX (`~` expands) |
+| `decoderPath` | `string` | `""` | Local backend: Whisper decoder ONNX |
+| `tokensPath` | `string` | `""` | Local backend: tokens file |
+| `language` | `string` | `"auto"` | ISO code (e.g. `"en"`) or `"auto"` to detect |
+| `numThreads` | `number` | `2` | Local backend: inference threads |
+| `sampleRate` | `number` | `16000` | Rate of the incoming samples; resampled to 16 kHz internally |
+
+A single `FloatRingBuffer` hop carries at most ~88200 samples (the runtime's
+fixed ring-buffer size), i.e. ~5.5 s at 16 kHz — the service transcribes one
+such utterance per call.
+
+### Example (hkp-rt)
+
+`boards/runtime-configs/speech-to-text-example.json` wires the microphone straight into
+the transcriber: `core-input` → `speech-to-text` (local) → `monitor`. It pairs
+naturally with **text-generation** and **text-to-speech** to form a fully local
+voice loop inside one C++ runtime.

package/bundle/docs/content/services/spotify.md

@@ -0,0 +1,77 @@
+# Spotify
+
+Reads playback state and track metadata from the Spotify Web API.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `hookup.to/service/spotify` |
+
+---
+
+## What it does
+
+Spotify connects to the Spotify Web API using an OAuth access token and
+emits the current playback state — track name, artist, album art URL,
+progress, and playback status — on each pipeline trigger.
+
+---
+
+## Prerequisites
+
+- A Spotify Premium account
+- A valid OAuth 2.0 access token with the `user-read-playback-state`
+  scope (obtained via the Spotify Authorisation Code flow or similar)
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `accessToken` | `string` | `""` | Spotify OAuth access token |
+| `refreshToken` | `string` | `""` | OAuth refresh token for automatic renewal |
+| `clientId` | `string` | `""` | Spotify app client ID (needed for token refresh) |
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | Any value — used as a trigger; the value itself is discarded |
+| **Output** | Spotify playback state object |
+
+The output object mirrors the Spotify API's
+[Get Playback State](https://developer.spotify.com/documentation/web-api/reference/get-information-about-the-users-current-playback)
+response, which includes:
+
+```json
+{
+  "is_playing": true,
+  "progress_ms": 45000,
+  "item": {
+    "name": "Track Name",
+    "artists": [{ "name": "Artist" }],
+    "album": {
+      "name": "Album",
+      "images": [{ "url": "https://..." }]
+    },
+    "duration_ms": 240000
+  }
+}
+```
+
+---
+
+## Typical use
+
+Pair with a Timer to poll playback state periodically and display the
+current track on a Canvas:
+
+```
+Timer (1s) → Spotify → Map (extract track name + art) → Canvas
+```

package/bundle/docs/content/services/spread.md

@@ -0,0 +1,111 @@
+# Spread
+
+Sends each item of an array on by itself — one value in, as many out as it has items.
+
+---
+
+## Available in
+
+| Runtime | Service ID |
+|---|---|
+| Browser | `spread` |
+| Node.js (hkp-node) | `spread` |
+| Python (hkp-python) | `spread` |
+
+Not in hkp-rt. Spread continues the run itself once per item, and each item's
+result has to leave the runtime on its own. hkp-rt has no way to do that from
+inside a call: a pass a service starts there runs nested in the call it is
+making, so only the call's own answer leaves the runtime, and a pass scheduled
+for later loses the run and the order. An array produced in hkp-rt can be spread
+by a runtime after it.
+
+---
+
+## What it does
+
+A pipeline pass carries one value, and so does a value leaving a runtime: an
+array of a hundred chunks crosses to the next runtime as **one** message, as
+large as the file it was cut from. Spread sends each item down the rest of the
+board on its own — the services after it, and the runtimes after this one — so
+each crosses as a message of its own.
+
+That is what gets a large value past a coordinator's frame limit
+(`HKP_COORDINATOR_MAX_FRAME_BYTES`): a [File Source](./file-source.md) with a
+`chunkSize` cuts a file into an array of pieces, Spread sends each piece as its
+own frame, and a [Gather](./gather.md) on the far side puts them back together.
+
+```
+ui:   file-source (chunkSize) → spread     one value per chunk
+node: gather → …                           one value per file
+```
+
+### Not an Iterator
+
+An [Iterator](./iterator.md) runs a pipeline of its own once per item and comes
+back with the answers, so one value still leaves it. Spread holds no pipeline:
+what follows it on the board is what each item goes through, and nothing comes
+back. It returns `null` and continues the run itself, once per item — the
+[pull pattern](../concepts/runtime.md) — so every item goes on in the run that
+reached it, with the caller that run carries.
+
+### Each item goes on wrapped
+
+Every item is sent as
+
+```json
+{ "item": "…the item, untouched…", "group": "<id per array>", "index": 0, "total": 4 }
+```
+
+`group` is the same for every item of one array and different for the next;
+`index` and `total` say where the item sat. That is what a Gather needs to put
+the array back in order once it has seen every part. A service in between reads
+the item as `params.item`.
+
+### A single item is an array of one
+
+As for an Iterator: a non-array input is sent on as one part with `total: 1`,
+so a board fed sometimes one value and sometimes several does not change shape.
+Nothing (`null`, or an empty array) sends nothing on.
+
+### Items go on in order
+
+Each item is sent on and the services after Spread awaited before the next one
+goes, so they see the items in order and a hundred chunks are not all in flight
+at once. Order across a runtime boundary is not something a Gather relies on —
+it orders by `index`.
+
+### Where to put it on a deployed board
+
+Put Spread in the browser runtime **where the run begins** — the one a facade
+control or a picked file starts. A browser runtime that the coordinator calls
+into, because a runtime before it produced something, answers the coordinator
+once per call, so only the first item would go on from there.
+
+In hkp-node and hkp-python this does not arise: each item is handed on the way
+a Timer's tick is, as a result of the runtime's own, in the run that reached the
+Spread. There it can sit anywhere in the board.
+
+---
+
+## Configuration
+
+| Property | Type | Default | Description |
+|---|---|---|---|
+| `itemsFrom` | `string` | `""` | Dotted path to the array in the input; the input itself by default |
+| `lastTotal` | `number` | — | Read-only: how many items the last array had |
+
+---
+
+## Input / Output
+
+| | Shape |
+|---|---|
+| **Input** | `JSON` — an array, or a single item (any type) |
+| **Output** | `JSON` — one `{ item, group, index, total }` per item, each sent on by itself; `Null` from the service's own pass |
+
+---
+
+## Typical uses
+
+- Getting a file past the frame limit: File Source (`chunkSize`) → **Spread** → hkp-node Gather (`join`) → …
+- Handling query rows one by one further down the board: … → **Spread** (`itemsFrom: "rows"`) → Map (`params.item`) → …