@readymadeit/mcp 0.0.0-stage → 0.1.0

package/bundle/docs/content/concepts/scopes.md

@@ -0,0 +1,256 @@
+# Scopes
+
+A boundary inside a runtime: a group of services that runs as one flow, keeps what it holds to itself, and says for itself whether its answer leaves.
+
+---
+
+## The problem it solves
+
+A runtime is an ordered list, and that order is the wiring. One board, one chain.
+But a board often has **two flows on one runtime** — kept together because they
+share a database, or a unit, or simply because splitting them across runtimes
+would cost a round trip for nothing.
+
+The RSS reader is the plain case. Fetching feeds is one flow; keeping an article
+and publishing the reading list is another. They share a database and nothing
+else: a refresh must not reach the statement that saves an article, or every
+round would insert sixty rows nobody asked for.
+
+Before scopes, the way to say that was a [`stopper`](../services/stopper.md)
+between them:
+
+```
+feeds → Stopper → record-article → kept-articles → … → feed-serve
+```
+
+That works, and it says very little. It states that nothing continues past a
+point, and leaves the reader to infer that everything *above* it belongs to one
+flow, that everything *below* is entered from somewhere else, and that a service
+added in the wrong place would silently join the wrong half. The boundary is a
+convention held up by list order.
+
+A scope says it of itself:
+
+```
+read  (stopPropagation: true)        list
+  └ feeds                              └ record-article → kept-articles → … → feed-serve
+```
+
+---
+
+## A scope is a SubService
+
+There is no `scope` service. A scope is a
+[`sub-service`](../services/sub-service.md) using either of two capabilities —
+and one that uses neither is the SubService that has always existed.
+
+```json
+{
+  "uuid": "read",
+  "serviceId": "sub-service",
+  "serviceName": "Reading the feeds",
+  "state": {
+    "stopPropagation": true,
+    "scope": { "slots": "own" },
+    "pipeline": [ … ]
+  }
+}
+```
+
+Giving it a separate service id was considered and rejected: identical mechanics
+would fork the implementation across four runtimes and make board authors choose
+between two things that behave the same. If the word should be visible in a
+board, it is a `serviceName` — which costs nothing and is per instance.
+
+---
+
+## `stopPropagation`
+
+Whether what the pipeline produced leaves the service. **Default `false`, and an
+absent field means `false`**, so every board written before scopes goes on
+passing its result along.
+
+It closes **both** ways out, and that is the whole of the work:
+
+| Route | What it is |
+|---|---|
+| The answer | `process()` returns nothing, so the services after this one do not run |
+| The push | what the pipeline emitted *without being asked* — a Timer tick, or a service that answered nothing and came back later — is dropped rather than carried past |
+
+The second one is easy to miss and is usually the one that matters. An `rss`
+service returns nothing from `process` and pushes its articles once the feeds
+answer; a scope that closed only the first route would let every one of those
+articles through. A scope that holds anything asynchronous needs both closed, or
+it goes on driving the board long after its own answers have stopped.
+
+The panel puts this as **Output**, with `continues` and `stops` rather than
+the board's own true and false: a boolean named after one of its two states
+makes a reader work out which way round the name runs before they can answer
+the question it asks. Changing it rebuilds nothing — the flag is read on the
+way out of every call and every push, so saying it is all it takes, and a
+scope told to keep its answer to itself does not lose the Timer it was holding
+in the telling.
+
+It composes with `bypass`, which a SubService already had:
+
+| `bypass` | `stopPropagation` | |
+|---|---|---|
+| — | — | the pipeline runs, its result continues |
+| — | ✓ | the pipeline runs, nothing continues |
+| ✓ | — | the pipeline is skipped, the input continues |
+| ✓ | ✓ | the pipeline is skipped, nothing continues — a Stopper |
+
+---
+
+## `scope`
+
+What the services inside can see, as one block rather than keys added one at a
+time. Today it has one member.
+
+```json
+"scope": { "slots": "own" }
+```
+
+`slots` decides which cells a [`hold`](../services/hold.md) inside this scope
+reaches — its own, or the runtime's.
+
+**`own` is the default**, and the reason is reuse. A SubService is the primary
+way to build a higher-level block out of smaller ones, and a block whose slot
+names leaked into its surroundings would collide with a second copy of itself.
+Two Holds that must meet *across* a scope boundary need that boundary to say so:
+
+```json
+"scope": { "slots": "inherit" }
+```
+
+The same rule holds one level up: an endpoint owns the cells its two entry
+pipelines share, which is why two endpoints on a runtime may both call a slot
+`document` without meeting.
+
+A scope's panel says which of the two it got, under a **Slots** line that also
+says how many cells there are. The line is all it shows until it is opened,
+because a cell holds whatever a pipeline put in it — an article, a frame, a
+document — and a panel that drew every value would be mostly somebody else's
+data. Open it and the slot *names* appear, which is what a board is written in
+terms of; open a name and that one cell's value appears under it.
+
+Inherited cells are drawn greyed and a scope's own in the panel's text colour,
+because the difference is about who else can reach them — an inherited cell is
+shared with everything else out there naming that slot, so it is not this
+scope's to account for.
+
+Which of the two it is sits at the end of that same line, as a setting wearing
+what every other setting in a service panel wears — unlabelled, because the
+line is already headed Slots and the two values say themselves what they mean.
+It is the one thing about a scope's cells a panel offers to change, because the
+rest is what other pipelines left behind. Changing it **re-points the cells and
+rebuilds nothing** — the scope asks for its store on every lookup, so a Timer
+or a socket inside goes on running. A scope's own cells outlive the trip, too:
+they live on the service rather than on the pipeline, so a value held before a
+detour through the inherited ones is still there on the way back.
+
+Each slot carries a **bin**, and that is the only thing the panel does to a
+cell. What is in one is the pipelines' to write; what is worth doing from
+outside is taking a cell away, so that a board can be put back to before
+anything ran without hunting for the service that filled it. Removing is not
+the same as emptying — an empty cell is one a board named and nothing has
+filled yet, and a Hold reading either answers the same nothing — but it leaves
+no stale reading behind, and the next write brings the name back.
+
+This is all a browser-runtime reading, and the reason is distance: the store is
+an object in the same process as the panel, so it can be listed, watched and
+reached into directly. A scope on a REST runtime would have to report its cells
+in its state before a panel could draw them.
+
+`secrets` and `logging` are the members most likely to follow — which
+credentials a scope's children can resolve, and logging turned on for one scope
+and nothing else. Neither exists yet. The block is shaped for them so they do
+not arrive as scattered sibling keys.
+
+---
+
+## Addressing what is inside
+
+A runtime's services are a flat list, and a uuid names one of them. A scope's
+are not in that list, so they are named by the path through the services holding
+them:
+
+```json
+{ "serviceUuid": "list.kept-articles", "path": "rows" }
+```
+
+`serviceUuid` is the address; `path` still means the field inside that service's
+state. They stay separable, which is what lets an address nest without becoming
+ambiguous — `read.list.feed-doc` is three services, not a field called `list`.
+
+**The separator is a dot, and that is forced rather than chosen.** A service
+address is carried in a URL path segment
+(`/runtimes/<id>/services/<address>`), which a slash would split. The same
+constraint already decided the separator between a unit's name and its runtime
+ids.
+
+Everything a board does with a service works at an address: a facade widget
+reading one, a button processing one, a `configure` action, a notice, and a
+[`hkp-mount://`](./mounts.md) reference naming an endpoint inside a scope.
+
+Two consequences worth knowing:
+
+- **A service reports under its address, always.** Each boundary prefixes its
+  own uuid as a notification passes outward, because an instanceId is unique
+  only inside its own pipeline. There is no moment — not even while a nested
+  service is being configured — when it speaks under a name the board cannot
+  dial.
+- **Entering a scope at one of its services runs the rest of that scope.**
+  Processing `list.record-article` runs it and what follows it *inside* `list`,
+  which is written down, rather than "whatever happens to come next" in the
+  runtime's own list.
+
+---
+
+## Where a scope is entered
+
+A scope that passes nothing on cannot be reached by the chain, which is the
+point — so it is entered by being addressed. In the RSS reader, the facade's
+Save and Remove buttons process `list.record-article`, and nothing else ever
+enters that half of the board.
+
+That is the real gain over a stopper. With a stopper, "the reading list is
+entered from the facade" is true but unwritten: it follows from where the
+stopper sits. With scopes, the second flow is a thing with a name, and what runs
+when you enter it is what is written inside it.
+
+---
+
+## What runs where
+
+All four runtimes carry the same contract, so a board means the same thing
+wherever it runs.
+
+| | Browser | hkp-node | hkp-python | hkp-rt |
+|---|---|---|---|---|
+| `stopPropagation` | both routes | both routes | the answer only¹ | both routes |
+| `scope.slots` | ✓ | ✓ | ✓ | ✓ |
+| Scoped addressing | ✓ | ✓ | ✓ | ✓ |
+
+¹ hkp-python has no second route out: its SubService never carries a nested
+service's unprompted output past itself, so there is nothing there to close.
+
+An endpoint inside a scope works on the runtimes that have endpoints — a nested
+pipeline has no server of its own, so it claims its address on the runtime
+around it. The name it derives from falls back to the scoped address, so two
+copies of one scope do not take each other's callers; a board that named its
+mount keeps the address it published before being scoped.
+
+---
+
+## When not to reach for one
+
+A scope is a boundary, not a folder. Wrap something in one because its answer
+should stop, or because its cells should be its own — not to tidy a long list.
+A SubService that does neither is exactly what it always was, and the board
+gains a level of nesting for nothing.
+
+And a scope is not [Tracks](../services/tracks.md). Tracks is one call fanned
+out over several pipelines and reduced back to one answer; a scope is many calls
+over time sharing what they left behind. Two pipelines that must both run on the
+same input are tracks. Two flows with different triggers are scopes.

package/bundle/docs/content/concepts/service.md

@@ -0,0 +1,227 @@
+# Service
+
+The unit of work: one thing a board does, with configuration you can see, a panel you can drive it from, and a place in an ordered list that is its wiring.
+
+---
+
+## The contract
+
+A service takes input, does something, and returns output. Two methods carry
+almost everything:
+
+| Method | When it is called |
+|---|---|
+| `configure(config)` | on load, and whenever anything changes its settings — a user edit, a facade widget, the coordinator handing over a resolved address |
+| `process(input)` | with the output of the service before it, or `undefined`/`Null` when nothing came before |
+
+and one more that makes a board saveable:
+
+| `getState()` / `getConfiguration()` | what this service was configured with — which is what gets written into the board |
+
+In TypeScript that is `ServiceRuntimeContract` in `hkp-frontend/src/types.ts`; in
+hkp-node it is `HostedService`; in C++ it is the `Service` base class
+(`hkp-rt/lib/include/service.h`). The three are the same shape deliberately, so
+the same concept can exist on more than one runtime.
+
+**State is configuration, not output.** A Monitor's messages never appear in its
+state: what it *shows* is a notification, what it *is* is its settings. That
+distinction is why a browser attached to a deployed board receives notifications
+as their own kind of message rather than reading them out of state.
+
+---
+
+## What a service may do
+
+Beyond returning a value, a service reaches its host — `AppImpl` in the browser,
+`RuntimeHost` in node, the `Service` base in C++:
+
+| Capability | Browser | Meaning |
+|---|---|---|
+| Notify | `app.notify(this, payload)` | tell whoever is watching — panels, facade widgets. Dropped when nobody is |
+| Log | `app.log(this, level, event, data)` | record something about the run. Survives with nobody attached, which is what an unwatched board needs |
+| Emit / pull | `app.next(this, result)` | run the services *after* this one with this value, without re-running this one |
+| Nest | `app.createSubService(...)` | own a pipeline of your own |
+| Reach the board | `app.coordinator` | ask a question that spans runtimes — see `concepts/mounts.md` |
+| Keep something | `app.storeServiceData` / `restoreServiceData` | small durable per-service values |
+
+`notify` and `log` are the pair worth getting right: one is for a person looking
+now, the other is for answering "what happened?" later.
+
+---
+
+## Modes
+
+A service may have **modes** — named variants that change what `process` does
+while staying one service. Modes are configuration, not separate services: they
+are how a concept keeps its related behaviours together instead of splitting
+into near-duplicates.
+
+The design rule is in `CLAUDE.md`: *scoped by concept, not by technique*. Input
+handles event streams and WebSockets — different technology, same idea, one
+service. Complexity belongs in composition, not inside a service.
+
+---
+
+## Bypass
+
+Every service can be bypassed. A bypassed service is **skipped in the pipeline**
+but still there, still configured, still saved — the loop reads
+`if (svc && !svc.bypass)`. For services that hold a resource (a server, a
+socket, an audio device) bypass is also the on/off switch: hkp-rt's
+`onBypassChanged` starts and stops the server behind the service.
+
+---
+
+## The panel
+
+A service may ship a UI component (`createUI`). A service that ships none gets
+`PlaceholderUI`, which renders its configuration as editable fields — so
+**every** service has a panel, and nothing is a black box.
+
+- For **browser** services the service and its panel share one JS process, so
+  keeping them in sync is a direct method call.
+- For **remote** services the panel talks over REST: configure, then read back
+  `getState`. Optimistic UI is the norm — apply locally, send, reconcile.
+- Panels are resolved by `serviceId` through the UI registry
+  (`runtime/browser/UIRegistry.ts`, `runtime/rest/UIRegistry.ts`), falling back
+  to the runtime's own registry entry, then to the placeholder.
+
+---
+
+## Identity: `serviceId` vs `uuid`
+
+| | What it is |
+|---|---|
+| `serviceId` | *what kind* of service this is — `hookup.to/service/timer`, `http-client`, `monitor`. Chosen by the implementation, shared by every instance |
+| `uuid` | *which instance* — unique within the board, and what everything else refers to: facade widgets, mount references, sub-pipeline entries |
+| `serviceName` | the label a person sees; editable, and not an identifier |
+
+Browser service ids are historically URL-shaped (`hookup.to/service/...`), while
+the other runtimes use plain names (`http-client`). Where a service exists on
+more than one runtime, the ids are being aligned so a board can move — see
+`plans/TODO-CONSOLIDATION.md`.
+
+---
+
+## State that isn't the service's own
+
+Two prefixes in service state mean something the service did not decide:
+
+- **`__hkp…`** — a property whose meaning is defined *outside* the service
+  holding it. Generic board machinery reads and rewrites it. `__hkpMount` is the
+  current example. The prefix is reserved: a service must not use such a name for
+  anything else.
+- **`{{secret.alias}}`** — a reference to a value the board never holds. It is
+  never substituted into state, so `getState` reports the reference and saving
+  writes back what was configured. The value exists only inside the call that
+  uses it (`core/secrets.ts`).
+
+Both exist for the same reason: a board is a document meant to be shared, and
+neither an address that was true on one machine nor a credential belongs in it.
+
+---
+
+## Nesting
+
+A `sub-service` holds an ordered pipeline of its own and runs it as one step of
+the outer one. This is the primary mechanism for building reusable
+higher-level blocks, and every runtime has it:
+`runtime/browser/services/BrowserSubService.tsx`,
+`hkp-node/src/services/sub-service.ts`, `hkp-rt/lib/src/services/sub_service.h`.
+
+Nested services are configured through their host service rather than as
+top-level entries — `pipeline`, `appendService`, `removeService`,
+`configureService` in the host's own configuration — and they carry
+`instanceId` rather than `uuid`, which forking rewrites too.
+
+---
+
+## Structured flow, not wires
+
+There is no wire UI, and this is a decision rather than an omission. Explicit
+wires are like `goto`: they work, and they make a flow hard to reason about. The
+ordered service list *is* the flow, and branching is expressed with services:
+
+- **Filter** stops propagation when a predicate fails.
+- **Switch** pattern-matches and routes into sub-pipelines.
+- **Looper** repeats sub-services until a predicate stops it.
+- **Cache** early-returns on a hit, and on a miss pulls the services behind it.
+
+The four control-flow mechanisms those rely on — stop, early return, pull,
+deferred emit — and which runtimes implement each, are in
+`concepts/runtime.md`.
+
+### Nest; don't reach
+
+A service *can* reach another one: `app.getServiceById(uuid)` hands over the live
+instance, `configureServiceInRuntime` reconfigures one on another runtime, and
+the **Configurator** service calls `configure()` on a service named by
+`targetServiceUuid`. A panel can use this to control something elsewhere.
+
+But a service that names another service's id is a service that cannot be
+reused, moved or read on its own, and the id it holds is one more field the
+generic machinery has to know by name (`KNOWN_REFERENCE_FIELDS` in
+`core/forkBoard.ts`, so a fork can rewrite it). Two modules that were supposed to
+know nothing about each other now do.
+
+**Design services that nest instead.** A service that owns a pipeline gets its
+answer by *running* one, and the pipeline it runs knows nothing about it:
+
+- `sub-service` — a pipeline as one step.
+- `join` (hkp-node) — a pipeline as a **detour**: the result is merged *beside* the input
+  rather than replacing it, so the carrier (which conversation, which record)
+  survives. Two services that would otherwise have to find each other — one
+  producing a value, one filing it where it belongs — become one grouped step
+  that takes a value in and hands a bigger value out.
+- `switch` — pipelines as branches; `looper` — a pipeline repeated;
+  `http-server-subservices` — a pipeline as the handler for a request;
+  `board-service` — an entire saved board as a step.
+
+The test when writing a service: **does it need to know anything about the board
+around it?** If the answer is a service uuid, there is usually a nesting that
+computes the value where it is needed instead — and a service that takes input
+and returns output composes with everything, while a service that names another
+one composes with nothing.
+
+---
+
+## Adding one
+
+Every runtime has a registry that says what it can create, and a service is only
+finished when all of its pieces exist:
+
+| Piece | Browser | hkp-node | hkp-rt |
+|---|---|---|---|
+| Implementation | `runtime/browser/services/base/*.ts` | `src/services/*.ts` | `lib/src/services/*.h` |
+| Descriptor + factory | `runtime/browser/services/*.tsx` | `ServiceRegistryEntry` | `serviceId()` static |
+| Registration | `runtime/browser/registry/Default.ts` | the service map in `src/server.ts` | `lib/src/registry.cpp` |
+| Panel | `*UI.tsx` in the same folder | frontend `runtime/rest/ui/` | frontend `runtime/rest/ui/` |
+| Docs + demo board | `docs/content/services/<slug>.md`, `boards/<slug>-demo-board.json` | same | same |
+
+The scaffolding skills do this end to end: `/new-browser-service`,
+`/new-node-service`, `/new-cpp-service`.
+
+Every service file starts with a **Service Documentation** header block —
+service id, name, modes, key config, IO, how it treats arrays, binary and
+MixedData. It is the first thing to read, and the first thing to write.
+
+---
+
+## Rough index into the source
+
+| Concern | Where |
+|---|---|
+| Contract types | `hkp-frontend/src/types.ts` (`ServiceRuntimeContract`, `ServiceInstance`, `ServiceModule`, `AppImpl`) |
+| Browser service host | `hkp-frontend/src/runtime/browser/BrowserRuntimeApp.ts`, `BrowserRuntimeScope.ts` |
+| Browser registry | `hkp-frontend/src/runtime/browser/BrowserRegistry.tsx`, `registry/Default.ts` |
+| Panels | `hkp-frontend/src/runtime/browser/UIRegistry.ts`, `services/PlaceholderUI.tsx`, `ui-components/service/` |
+| Node contract | `hkp-node/src/types.ts` (`HostedService`, `RuntimeHost`) |
+| C++ base | `hkp-rt/lib/include/service.h`, `lib/src/service.cpp` |
+| Data types across runtimes | `hkp-frontend/src/runtime/rest/Data.ts`, `hkp-rt/lib/include/types/data.h` |
+| Secret references | `hkp-frontend/src/core/secrets.ts` |
+| Per-service docs | `docs/content/services/*.md` |
+
+---
+
+See also: **Board** (`concepts/board.md`), **Runtime** (`concepts/runtime.md`),
+**Mounts** (`concepts/mounts.md`).