@dcasegr/skein 0.0.0-stage → 0.1.1

package/README.md

@@ -1,3 +1,523 @@
-# Temporary Holding Version
+# skein
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+A skein is a deterministic WASI machine over a content-addressed graph. Every
+package that reaches it is appended to its log as received, every step a
+program takes is recorded, and replaying the log rebuilds the same graph,
+byte for byte. What a skein does depends on the apps installed in it, and
+the same history runs under any host: a server, a laptop, a browser tab.
+
+This repository is the kernel, the boundary programs, the node host and the
+client. Apps live in their own repositories (below). The public explainer is
+https://skein.nexus; these docs are the reference.
+
+## What it is
+
+Three layers:
+
+| layer | what | where |
+|---|---|---|
+| **kernel** | the machine (store, log, scheduler, WASI, fuel) and its four tables, plus the signer import (`wallet`). One implementation, Zig over wasmtime; it also builds for the browser | `kernel-zig/` |
+| **apps** | everything else, each a tree of WASI programs installed under its own name: the front door, messagebox, resolve, wallet, chain, static sites, overlays, the shell, chat, your own | `programs/` here (the boundary programs and the wallet); the rest in sibling repos |
+| **hosts** | whatever drives a kernel from outside: transports, providers, a store and the signer. The host judges nothing and routes nothing | `src/host/` (the node host, `skein-host`); `web/kernel/` (the browser host) |
+
+The kernel's four tables:
+
+| table | holds | written by |
+|---|---|---|
+| objects | blocks by CID (dag-cbor, git-raw, bitcoin, raw) | the kernel's `objects` operation; programs `put`/`keep` |
+| heads | name → root, each with an owner (the app the name is under) | the kernel's `head` operation; programs `advance`, only under their own app's name |
+| dispatch | rows `(transport, address, sender, program)`: boxes, HTTP paths, libp2p topics. A row is the permission | the kernel's `dispatch` operation |
+| address book | key → transport and address; providers are `local` rows | the kernel's `peers` operation |
+
+The four admin operations are the kernel's own, taken on admin messages from
+the owner (or a key the owner delegated with another row). So is the
+claim: a skein started from the default image has no owner until a message
+reaches its claim row, and then the kernel writes the sender's admin rows
+(#127: the owner is the claim's sender, never a key in its body) and removes
+the claim row. No program writes a kernel table. An app writes only heads under its own name; reads are
+global by CID; a call to another app hands back CIDs, not data.
+
+## Run a skein locally
+
+Needs Node 26 and Zig 0.16.0 through `mise`. The owner's admin messages
+(installing an app, a dispatch row, the address book, a directory into
+`main`) are `bin/skein install|uninstall|routes|grant|peers|host|claim|deploy`:
+the client signs them in its own process with the operator's key
+(`SKEIN_OPERATOR_KEY`, default `~/.skein/operator.key`, which `skein-host
+run` makes) and sends them — over the host's control socket on the host
+machine (`--instance <handle>`), else on one BRC-104 session with the
+instance's origin (#142). A browser wallet sends the same messages from the
+management page. The `bin/skein` client's other commands
+use a wallet reachable over HTTP (default `http://127.0.0.1:3322`; the dev
+setup runs `1sat serve wallet-api`, `scripts/host/wallets.sh`).
+
+That HTTP wallet is deny-by-default. The client (origin `skein-client`,
+`SKEIN_ORIGINATOR`) needs these grants
+from the owner's wallet (`scripts/host/grants.sh` grants them for the dev
+stack):
+
+| protocol | level | counterparty |
+|---|---|---|
+| `identity key retrieval` | 1 | |
+| `server hmac` | 2 | `self` |
+| `messagebox` | 1 | |
+| `auth message signature` | 2 | the instance's front-door key |
+| `message encryption` | 2 | the instance's front-door key |
+
+The front-door key is column 5 of `skein-host list`; a new instance is a
+new counterparty. A missing grant fails the command with the wallet's
+refusal, which names the `1sat permissions grant …` command for it.
+
+```
+git clone https://github.com/shruggr/skein && cd skein
+npm install
+(cd kernel-zig && mise exec -- zig build --release)   # kernel-zig/zig-out/bin/skein-kernel; fetches skein-sdk by URL+hash
+bin/skein-host add martha --image default            # an instance from the default image: identity derived from ~/.skein/master.key, no owner yet
+bin/skein-host run                                    # the host on :8100; martha at http://martha.localhost:8100 or /@martha
+bin/skein claim --instance martha                     # the claim, signed with ~/.skein/operator.key: you, its sender, own martha; the claim row goes
+```
+
+The default image (`images/default`, docs/BOOTSTRAP.md) is the same for
+every user skein: the front door, the messagebox, the wallet, and one row
+to the kernel, `claim`, from anyone: the first claim's sender owns it
+(#127), so claim it first thing. The host signs no user's claim; `skein
+claim --messagebox <url>` names where the instance reaches its owner. A
+second claim is refused. It has no apps (#152). After the claim, install
+the chain, git and site apps by message, and root's route `/` to the site:
+
+```
+git clone -q --branch v0.6.0 https://github.com/shruggr/skein-chain ../skein-chain
+git clone -q --branch v0.2.0 https://github.com/shruggr/skein-git ../skein-git
+git clone -q --branch v0.10.0 https://github.com/shruggr/skein-site ../skein-site
+bin/skein install ../skein-chain --instance martha
+bin/skein install ../skein-git --instance martha
+bin/skein install ../skein-site --instance martha
+bin/skein routes add --prefix --filters site.get --settings '{"root":"www"}' / --instance martha
+```
+
+Installing a newer version the same way upgrades the app and keeps its
+state (docs/APPS.md, "Reconfiguration").
+
+To have apps at birth, boot from an image directory that lists them: the
+default image's `bin/` and `etc/` with an `etc/apps.json` naming each app
+as a directory (relative to the image) or a release tarball by its tree's
+git id (images/README.md):
+
+```json
+{ "install": [
+    { "url": "https://github.com/shruggr/skein-chain/archive/refs/tags/v0.6.0.tar.gz", "hash": "13a517e964c4785330d8b7ef77796e149657cbbb" },
+    { "url": "https://github.com/shruggr/skein-git/archive/refs/tags/v0.2.0.tar.gz", "hash": "0894fefd71d7aa1739d77b9d503421b4d55e497a" },
+    { "url": "https://github.com/shruggr/skein-site/archive/refs/tags/v0.10.0.tar.gz", "hash": "2a7b9a492beec6c1c865fc9e33628e18bd36d312" } ],
+  "routes": [ { "transport": "http", "address": "/", "prefix": true, "filters": ["site.get"], "root": "www" } ] }
+```
+
+```
+bin/skein-host add martha --image ./my-image
+```
+
+A plain `skein-host add martha` (no `--image`) is code genesis, with the
+owner in it: `SKEIN_OWNER`, else `~/.skein/owner.identity` (the owner
+wallet's identity key, one line of hex). The tests and the dev stack use it.
+
+- State lives in `$SKEIN_HOME` (default `~/.skein`): `master.key` (the one
+  secret every instance key derives from), `host.db` (instances, queues, the
+  fuel ledger), `instances/<handle>/runtime.db` (each instance's store),
+  `logs/host.log`.
+- BRC-169 on the host's own origin (`SKEIN_ROUTER_ORIGIN`) is the host
+  skein's onboarding app (shruggr/skein-onboard, #113): the router carries
+  `/manifest.json`, `/.well-known/metanet-handles/resolve` and `/search`,
+  `/bsvalias/id/…`, `POST /account/register` and `POST /account/profile`
+  into the host skein. The app registers handles (a mailbox instance from
+  the instance manager, signed over `register <name>@<domain>`), has the
+  host's certifier (a child of `master.key`, never in an instance) sign each
+  handle certificate under a serial of its own, records every issue, and
+  answers resolve, search and the manifest from those records. The handle
+  domain, the host's name and note are the app's config, written at the
+  host skein's birth from the host's settings (below).
+  Revocation is not implemented (the host has no wallet): the certificate's
+  revocation outpoint is BRC-52's disabled sentinel.
+- Code genesis (no `--image`) is written at its first start: the owner, the
+  owner's admin rows, the boundary programs (front door, messagebox,
+  resolve) and their rows, and the host's providers in its address book.
+  It has no shell and no chat loop: those are apps, installed next. `skein-host add <h> --boot <dir>` boots
+  from a system tree instead (docs/BOOTSTRAP.md). A new log format means a
+  new store: move `runtime.db` aside and start again.
+- The full dev stack (owner and inference wallets, mailbox instances, address
+  books, the inference peer) is `scripts/host/up.sh`; see
+  `scripts/host/README.md`.
+- The host page is http://127.0.0.1:4600 (every instance, a link to its
+  origin). A skein from the default image serves its management page, the
+  explorer included, at its origin. `bin/skein-dev log | ls | show <cid>` and
+  `kernel-zig/zig-out/bin/skein-kernel dump | replay | fuel <store>` read a
+  store from the command line.
+
+## Run a host
+
+### Two shapes: the host loads the kernel, or runs it as a process
+
+The kernel is one zig source built two ways (kernel-zig/README.md, "The
+machine and the native host"). The node host runs either, chosen by an
+instance's store:
+
+| store (host.db `store`) | the machine | who uses it |
+|---|---|---|
+| `sqlite:<path>` — a set of the host's ONE SQLite database (`SKEIN_STORE_SQLITE`, default `$SKEIN_HOME/store.sqlite` when no native kernel is installed) | the **wasm kernel** (`kernel-zig/zig-out/web/skein-kernel.wasm`), loaded in the host's own process: one machine per active instance on a worker thread, none for an idle one; the host chain held by one more machine; storage node's built-in `node:sqlite` | the npm package (`npx skein-host`, `npx open-exchange`): node alone, any platform node runs on, no toolchain |
+| a file, `lmdb:<dir>` (`SKEIN_STORE`), `rocksdb:<dir>` (`SKEIN_STORE_ROCKSDB`) | the **native kernel** (`kernel-zig/zig-out/bin/skein-kernel`): `serve` per instance, or the one `host` process over RocksDB | a checkout with the kernel built — the hosted shape (the rack) |
+
+The same log gives the same state on both: `kernel-zig/equiv/wasm-host.ts`
+replays every store the wasm host writes with the native `skein-kernel
+replay` and compares the state CIDs.
+
+The package: `npm pack` (its `prepack` builds `dist/` with tsc and the wasm
+kernel with `zig build web`); it carries no native binary. `bin/skein-host.js`
+and `bin/skein.js` run `dist/` (or a checkout's `src/`).
+
+A host runs skeins for others too. It has one of its own, the **host
+skein**: the operator's instance, where the host keeps what is state or
+conversation (docs/ARCH.md, "The host skein").
+
+```
+# ~/.skein/host.env — the host's settings: every SKEIN_* line; the environment wins
+SKEIN_ROUTER_ORIGIN=https://id.example.com
+SKEIN_INSTANCE_ORIGIN=https://{handle}.example.com
+SKEIN_HANDLE_DOMAIN=example.com
+SKEIN_HOST_NAME="Example host"
+SKEIN_HTTP=fetch
+```
+
+```
+bin/skein-host run        # the first run: operator.key, master.key, host.db and the host skein; then the host
+```
+
+- `run` reads `$SKEIN_HOME/host.env` (every `SKEIN_*` line; the environment
+  wins). On its first run it makes the operator's key
+  (`SKEIN_OPERATOR_KEY`, default `$SKEIN_HOME/operator.key`; a file you put
+  there is used instead: hex or WIF, mode 0600) and the **host skein**
+  (handle `host`) from the host image — the default image with the
+  onboarding app (shruggr/skein-onboard v0.4.0, its release tarball,
+  fetched and checked by its tree) — its genesis naming the operator's key
+  as its owner: its admin rows, its explorer row and the onboarding app's
+  routes are there at birth, with no claim. The onboarding app's
+  config comes from the settings (`domain` from `SKEIN_HANDLE_DOMAIN`, else
+  the domain of `SKEIN_ROUTER_ORIGIN`; `origin`; `name` from
+  `SKEIN_HOST_NAME`; `note` from `SKEIN_HOST_NOTE`). It prints the host
+  skein's identity and URL, then serves. Later runs start what exists; a
+  changed host.env does not reach into the host skein (`skein install
+  <a skein-onboard checkout> --instance host --config <file.json>` changes
+  the app's config). `skein-host init` is the first run's part alone. Its
+  address book alone names the **instance manager**, the host's provider
+  that creates, starts and stops instances; the manager acts for no one
+  else.
+- `skein-host grant <your wallet's identity key> --apps` gives a browser
+  wallet the host skein's admin rows and its apps' owner rows (signed with
+  the operator's key), so the management page manages the host skein too.
+- With the onboarding app installed, anyone with a wallet gets a skein: a
+  BRC-104 session to the host skein and `POST /onboard/call {"fn":
+  "onboard.create", "args": {"handle": "alice", "claim": …}}`, `claim` the
+  wallet's own signed claim naming no recipient (#127; skein-onboard's
+  README has its shape). The answer, on the same connection, is `{handle,
+  identity, url}`: a new instance from the default image, its claim
+  forwarded by the instance manager as its first entry — owned by the
+  signer, already claimed when its URL first answers. Its owner installs apps into it from there (next sections). A
+  handle that is taken is refused (409).
+- **The management page.** Open the host skein's origin
+  (`http://host.localhost:8100/`) with a BRC-100 wallet in the browser: it
+  creates a skein for your key through that route and opens the new
+  skein's own copy of the page. Nothing is written into your wallet: the
+  page lists your skeins from the host (`GET <router>/skeins?key=<your
+  identity key>`, the skeins whose head `grants` names your key root). There it reads
+  the skein through its explorer and installs apps: the git app from the
+  image, then any app by repository URL and commit id — you see the rows each
+  asks for before your wallet signs. Deploy by hash needs the host to carry
+  requests beyond itself: run it with `SKEIN_HTTP=fetch`. Every skein from
+  the default image serves the same page (shruggr/skein-site).
+- **A handle from the page** (#103). On the same page, "Register a handle"
+  asks the host for `<name>@<domain>` for your key (the domain is the
+  router's host name, `id.skein.nexus` in production; the page finds the
+  router through `/.well-known/skein-host`, which the router answers at every
+  skein's origin): your wallet signs the request, the host creates your
+  mailbox instance and answers with the BRC-52 handle certificate, and your
+  wallet keeps it (`acquireCertificate`). "Your handles" lists those
+  certificates (`listCertificates`) and the mailbox each resolves to. The
+  labels `id` and `host` and every instance's handle are refused.
+- **Your profile** (#104). Each of your handles has a Profile form: a name
+  and an avatar (an image inscription, by outpoint). Your wallet signs the
+  profile record (the OpNS one: DAG-CBOR `{domain, name?, avatar?}`), and
+  the page writes it to your mailbox instance as its owner. The host serves
+  it with the handle when it is resolved or found; the page shows it as
+  signed by the handle's key. "Find a handle" searches the host by handle
+  or name.
+- **One key for every instance** (#160, #161, #162). A host can be
+  provisioned with ONE private key instead of the master key:
+  `SKEIN_KEY_FILE=<path>` (64 hex digits, read and never made) or
+  `SKEIN_KEY=<hex>`; set together with `SKEIN_MASTER_KEY` /
+  `SKEIN_MASTER_KEY_FILE` the host refuses to start. Every instance then
+  signs with that key directly (`skein-host add` records the same identity
+  on every row), and every instance's libp2p peer key is the key's one child
+  ([2, "skein instance"], key ID `libp2p`): a program that asks its signer
+  for `libp2p:<handle>` gets it, whatever the handle. One libp2p node per key
+  carries all of them: gossip goes to every instance that takes the topic
+  (accepted if any accepts); an inbound stream goes to the instance its first
+  frame's `topic` names (a frame naming none: the one instance declaring the
+  protocol, else `{kind: "error", message: "topic required"}`). Unset: the
+  master key's derivation per instance, one node per instance, as before.
+- **Born owned** (#164). A host program on skein's host as a library calls
+  `Router.createInstance(handle, owner, {image, owned: true})`: an image
+  instance whose genesis names `owner` root, as the host skein is born —
+  no claim, no claim route (the CLI has no flag for it).
+- **Path-only handles** (#161). A token id `<txid>_<vout>` is a handle too:
+  its instance has no hostname and is reached at `/@<txid>_<vout>/…` on the
+  host's origin (its `url` says so). An overlay request at the bare host —
+  `POST /submit` (its `X-Topics`) or `POST /lookup` (its `service`
+  `ls_color_<id>`, or a `topic` `tm_color_<id>`) — goes to the enabled
+  path-only instance whose handle is that `<id>`, so a wallet keeps one
+  provider URL; topics naming two instances, or one here and one not, are 400.
+  Any other name (#137: `tm_color`, `ls_color`, …) goes to the
+  enabled instance whose row names it in its `topics` (host.db: what the
+  instance manager's `create` or `start` named), whether or not it has run
+  yet; two instances naming the same name are 400. Nothing maps: the
+  hostname's instance (or the bare host) as before.
+- **An instance creates instances** (#162). `SKEIN_MANAGER_FOR=<handle>[,…]`
+  gives those instances the instance manager in their address book at birth
+  (as the host skein has it) and has the manager take their messages: `create
+  {handle, image?, config?, claim?, beef?, topics?}`, `start {handle,
+  topics?}`, `stop {handle}`. `topics` (#137): the overlay names, gossip
+  topics and lookup services alike (`["tm_color", "ls_color"]`),
+  the host routes to the instance at the bare host — plain non-empty
+  strings, no duplicates; stored with its row; a `start` naming them
+  replaces the row's, one naming none keeps them.
+  `image` is `default`, `mailbox`, or an image by name: the directory
+  `$SKEIN_HOME/images/<name>` (`SKEIN_IMAGES_DIR` names another directory;
+  the layout `skein-host add --image <dir>` takes, a link to an app
+  repository's `image/` included). `config` is the new instance's apps'
+  config at birth, `{<app>: {…}}`, merged as an install's `--config`. On a
+  one-key host a create with no `claim` is claimed by the host's own key
+  (its owner), so the instance is live at once; on a master-key host the
+  owner's signed claim stays required. `beef` (skein-amm#4): an Atomic BEEF
+  (bytes) the host hands into the new instance once it is claimed, published
+  and following headers — into its receive row `/skein/overlay/beef/1.0.0`
+  (the host's own entry, never served to peers), as a fetched BEEF is, from
+  the host's own peer; the host names
+  no topic (the instance's engine judges it under the topics its config
+  declares). The answer gains `delivered: {verdict, reason?}`; a BEEF the
+  door turns away (unmined, a bad BUMP) is reported there, not retried, and
+  the instance stays created. `stop` also takes the instance off its
+  libp2p node (its beacons stop beating); `start` declares it again.
+- `scripts/host/up.sh` brings up the dev stack.
+- Block headers for the chain app: run the host with
+  `SKEIN_HEADERS_URL=http://127.0.0.1:8083/chaintracks/v2/tip/stream`
+  (Arcade's chaintracks); every instance with the chain app installed is
+  subscribed to it (scripts/host/README.md, the feeds). The host fills the
+  default image with the whole header chain from that chaintracks' history
+  at start and appends every header the feed brings, so a new skein is born
+  with the chain (#132, docs/BOOTSTRAP.md, "The image's chain part").
+
+## Talk to it
+
+An instance is an HTTP server at its own origin. Every request is appended
+as received and the instance's front door (a program) runs BRC-103/104 on it
+inside the VM; the host holds the connection until the request's thread
+comes to rest, and the answer is signed on the session.
+
+```
+bin/skein install shell --instance martha       # the shell app (from martha's catalog, cloned by her git app): box shell/run
+bin/skein install chat --instance martha        # the chat app: box chat
+bin/skein whoami
+bin/skein import ~/some/dir                      # tree objects into the kernel's objects operation; prints the tree CID
+bin/skein run --tree <cid> -- 'ls | head -3'     # box shell/run (the shell app: the shell over a tree)
+bin/skein inbox --wait                           # the answer, from your mailbox instance
+bin/skein chat --new --wait 'what is here?'      # box chat (the chat app; needs the inference peer, bin/skein-infer)
+```
+
+- In a browser: a skein from the default image serves the management page
+  at its origin (`/`). It talks to that skein, and to every skein on its
+  host whose grants name your key root, with the same requests as the commands below,
+  signed by your wallet. Its Inbox lists a box of any mailbox you give it
+  (by default the mailbox your handle certificate resolves to) and syncs its
+  `metanet_inbox` into that wallet with `@1sat/actions`' `syncMetanetInbox`.
+- Messages are BRC-33 (`/sendMessage`, `/listMessages`,
+  `/acknowledgeMessage`) on a BRC-104 session, bodies dag-cbor
+  (`application/cbor`, BRC-231) or JSON. A message's id is the CID of its
+  record; an answer names it in `replyTo`.
+- What an instance sends you goes to your mailbox instance (an instance with
+  only the front door and the messagebox): a registration makes one (#113),
+  or `skein-host add <h> --mailbox --owner <key>` on a host with no host
+  skein.
+- libp2p: an instance with libp2p rows gets a node in the host; topic
+  messages and stream frames go through the same front door.
+- App routes live under `/<app>/` (an overlay's `/<app>/submit`,
+  `/<app>/lookup`); `{fn, args}` to an app's box calls a function.
+
+The wire contract is docs/MESSAGES.md.
+
+## Install an app
+
+```
+bin/skein install <name> --instance martha                         # a name from martha's catalog (her site's www/catalog.json)
+bin/skein install https://github.com/<owner>/<repo>#<commit> http://martha.localhost:8100   # any repository and commit, at an origin
+bin/skein install <name> --instance martha --dry-run               # the prompt and the messages, nothing sent
+bin/skein uninstall <app> --instance martha
+```
+
+`install` prints the prompt — what the app asks for (its head, its rows,
+its reads, its start) — and sends the owner's messages. A repository is
+installed **by hash**: one message to the skein's git app, `{fn:
+"git.clone", args: {url, hash}}`; the app fetches that one commit through
+the host's fetch provider (the host runs with `SKEIN_HTTP=fetch`), checks it
+against the hash, keeps its tree in the store and answers `{tree, app}`
+(the app record, built in the VM). The client plans from the stored tree and
+sends `head` (`<app>/app` → the app record), one `dispatch` per row, and the
+app's `start` — no `objects`: the blocks are already there. Installing
+again is the upgrade. A directory (`skein install <dir>`) is read on your
+machine and the objects the skein lacks are sent. The management page does
+the same from your wallet. docs/APPS.md §3.
+
+## Build an app
+
+An app is a tree with `etc/app.json`. A complete minimal one is
+`programs/test/app-demo/`: a counter with one interface, a box, a cron tick
+and an HTTP route. Its manifest, with the `peek` function and the
+description left out:
+
+```json
+{
+  "kind": "app",
+  "name": "app-demo",
+  "version": "0.1.0",
+  "programs": { "demo": "bin/app-demo.wasm" },
+  "provides": [{ "interface": "demo.counter/1", "functions": {
+    "get": { "writes": false, "args": {}, "answer": { "count": "int" } },
+    "add": { "writes": true, "args": { "by": "int", "note?": "string" }, "answer": { "count": "int" } } } }],
+  "requires": [],
+  "dispatch": [
+    { "address": "app-demo", "sender": "*", "program": "demo" },
+    { "address": "tick", "sender": "$cron", "program": "demo" },
+    { "transport": "http", "address": "/call", "sender": "session", "program": "demo", "fn": "call" }
+  ],
+  "start": { "body": { "kind": "app-demo-start" } },
+  "stop": { "body": { "kind": "app-demo-stop" } }
+}
+```
+
+- **The name rule.** The app's programs advance only heads under
+  `app-demo/…`; its record is `app-demo/app`, its own state the record's
+  `state` link.
+- **Rows.** `transport` defaults to `mailbox` (a box). An `http` address is
+  relative to `/<app>/`. Senders: `*`, `event`, `session` (http), `$owner`,
+  `$self`, `$<provider>` (`$cron`, `$status`), or a key in hex.
+- **Programs** are WASI preview1 modules or WASI 0.2 components. In Zig,
+  depend on skein-sdk by tag:
+
+  ```
+  zig fetch --save=skein_sdk https://github.com/shruggr/skein-sdk/archive/refs/tags/v0.7.2.tar.gz
+  ```
+
+  and dispatch `{fn, args}` with the SDK's `app.serve`
+  (`programs/test/app-demo/main.zig`). Build with `zig build` for
+  `wasm32-wasi` and commit the module under `bin/`.
+
+docs/APPS.md is the specification (the manifest fields, install, calling an
+app, the security model, worked examples for an overlay and the chain app).
+
+## Write an overlay
+
+An overlay is an app built on shruggr/skein-overlay: the engine program plus
+your topic managers and lookup services, wired from `config.overlay` in its
+manifest, serving BRC-22/24 at `<base>/submit` and `<base>/lookup`, its
+BRC-23 base URL `https://<handle>.<host>/<app>` (`/@<handle>/<app>` on the
+host's origin without wildcard DNS). It
+needs the chain app installed (`requires: ["chain/1"]`). The engine exports
+the `topic`, `lookup` and `sk` Zig modules to build against; its README and
+docs/OVERLAY.md there have the walkthrough.
+
+## Build and test
+
+```
+cd kernel-zig && mise exec -- zig build          # first: the host tests run kernel-zig/zig-out/bin/skein-kernel
+cd kernel-zig && mise exec -- zig build test     # the kernel's unit tests
+npm test                                         # the host, client, formats (node --test src/**/*.test.ts)
+TMPDIR=/tmp/sk kernel-zig/equiv/run.sh           # equivalence: shell, git, replay exactness, apps, browser, the wasm host
+SKEIN_TEST_STORE=sqlite npm test                 # the host suite with every test host on the wasm kernel
+```
+
+- Gotcha: after a kernel change, a stale `zig-out/bin/skein-kernel` makes
+  `npm test` fail with "kernel probe: exited (1)" and can hang the host
+  suite. Run `zig build` first.
+- CI (`.github/workflows/test.yml`) runs all four on Ubuntu for every push
+  and pull request, with the browser and component parts of the equivalence
+  suite skipped (no Playwright, Chromium, wasm-tools or adapter there).
+- `equiv/run.sh` clones the pinned sibling apps (skein-chain, skein-overlay,
+  skein-site, skein-shell, skein-chat, skein-onboard, skein-git);
+  `SKEIN_CHAIN_DIR`, `SKEIN_OVERLAY_DIR`, `SKEIN_SITE_DIR`,
+  `SKEIN_SHELL_DIR`, `SKEIN_CHAT_DIR`, `SKEIN_ONBOARD_DIR`, `SKEIN_GIT_DIR`
+  point it at local checkouts. The git app's case (`git-clone.ts`) serves a
+  local repository with `git http-backend`. `npm test` installs the
+  shell app and the chat app where a test runs them (src/testapps.ts: the
+  same pins, fetched once into `$TMPDIR/skein-apps`). The browser cases need Playwright and
+  Chromium, else they are skipped with a note.
+- The programs here: `scripts/build-programs.sh && scripts/pin-programs.sh`.
+  Developing against a local skein-sdk: clone it next to this checkout and
+  prefix a build with `scripts/sdk-local.sh`.
+
+## Repositories
+
+| repo | what | version |
+|---|---|---|
+| [shruggr/skein](https://github.com/shruggr/skein) | kernel, boundary programs (front door, messagebox, resolve), the wallet program, the node host, the client | log format 8 |
+| [shruggr/skein-sdk](https://github.com/shruggr/skein-sdk) | the Zig package every program is written against: the `skein` imports, codecs, `app`, `files`, `chain`, wallet library, WIT | v0.6.0 |
+| [shruggr/skein-chain](https://github.com/shruggr/skein-chain) | the chain app: the one writer of `chain/state`, ingest a BEEF, the only broadcaster; born with the image's header chain (#132) | v0.4.0 |
+| [shruggr/skein-overlay](https://github.com/shruggr/skein-overlay) | the overlay engine app: BRC-22/24 over topic managers and lookup services | v0.9.2 |
+| [shruggr/skein-shell](https://github.com/shruggr/skein-shell) | the shell app: `run`, and the shell itself (brush, coreutils, the toolset, python's stdlib) | v0.1.1 |
+| [shruggr/skein-chat](https://github.com/shruggr/skein-chat) | the chat app: the turn loop (`chat`), its `bash` calls in the shell app's shell | v0.1.0 |
+| [shruggr/skein-onboard](https://github.com/shruggr/skein-onboard) | the onboarding app, installed in the host skein: creates a skein for a wallet through the instance manager (the wallet's signed claim forwarded, #127); registration over the registrant's session (#135) | v0.3.3 |
+| [shruggr/skein-git](https://github.com/shruggr/skein-git) | the git app: clones one commit by hash into the store, in the VM, and builds its app record (deploy by hash) | v0.1.2 |
+| [shruggr/skein-site](https://github.com/shruggr/skein-site) | the management site, an app (#125: its page at `/site/`, served from its own tree by skein-sdk's `files`; the owner may add a row for `/`): your skeins from the host, create a skein, install apps, the explorer, the Inbox, handles and their profiles | v0.7.6 |
+| [shruggr/skein-nexus](https://github.com/shruggr/skein-nexus) | the source of https://skein.nexus | |
+
+This repository's layout:
+
+```
+kernel-zig/      the kernel: skein-kernel serve | replay | shell | dump | fuel; equiv/ (the equivalence suite)
+images/          the default image (#89): a system tree with no owner and the claim row, no apps (#152); the host image's own part (#142); images/README.md
+programs/        frontdoor, messagebox, resolve, wallet (Zig over skein-sdk); test/ (fixtures, app-demo)
+wasm/            the committed modules the kernel pins (kernel-zig/src/programs.zig), see wasm/README.md
+src/host/        the node host: skein-host (cli.ts), the HTTP transport, providers, feeds, broadcaster, libp2p node, signer, install
+src/runtime/     the formats and a store reader in TypeScript (no machine)
+src/client/      bin/skein: import, run, chat, inbox, dispatch
+src/peers/       bin/skein-infer (the inference peer), a remote cron provider
+src/dev/         bin/skein-dev (log, ls, show over a store file)
+web/             the browser host (web/kernel/) and a front-end demo
+scripts/         program builds and pins, sdk-local.sh; host/ (the dev stack)
+```
+
+## Docs
+
+| doc | for |
+|---|---|
+| [docs/ARCH.md](docs/ARCH.md) | the architecture: the kernel, the host, the browser host, the apps |
+| [docs/VM.md](docs/VM.md) | the machine: records, heads, the four tables, imports, `emit`, calls, requests, fuel, time, the index |
+| [docs/MESSAGES.md](docs/MESSAGES.md) | the wire: the front door, sessions, routes, the log format, the messagebox, outbound, providers, broadcast, scheduling, libp2p |
+| [docs/APPS.md](docs/APPS.md) | apps: the manifest, install, calling an app, security, worked examples |
+| [docs/BOOTSTRAP.md](docs/BOOTSTRAP.md) | genesis: the system tree, packets, checkpoints, installing |
+| [docs/WALLET.md](docs/WALLET.md) | the wallet program, the chain state it reads, proofs, SPV |
+| [docs/OVERLAY.md](docs/OVERLAY.md) | pointer to skein-overlay's docs |
+| [docs/SKILLS.md](docs/SKILLS.md) | which skill scripts run in the VM |
+| [kernel-zig/README.md](kernel-zig/README.md) | building and running the kernel, components, the browser build, equivalence |
+| [scripts/host/README.md](scripts/host/README.md) | the dev host: commands, ports, environment, Arcade, libp2p settings |
+| [src/client/README.md](src/client/README.md) | the client |
+| [docs/OPEN.md](docs/OPEN.md) | open questions still live from building |
+
+Vocabulary: the **host** is transports + providers + store + signer (there
+is no "router"; the dispatch table routes); the **kernel** is the machine,
+the four tables and the signer import; **admin operations** are kernel
+operations, not programs; every other thing is an **app** under its own
+name; a **box** is a mailbox address in the dispatch table; **emit** is the
+one way out (a message to a key in the address book — unsigned: a skein
+receives mail only on a BRC-104 session to its own front door or over
+libp2p, #126 step 4 — or an event).
+
+## Contributing
+
+The tracker is issue [#31](https://github.com/shruggr/skein/issues/31):
+decisions, conventions and what is ready to build. Where a doc and an issue
+disagree, the issue is right.

package/bin/skein-host.js

@@ -0,0 +1,22 @@
+#!/usr/bin/env node
+// skein-host for the npm package (and any checkout): src/host/cli.ts built to dist/ (node strips no
+// types under node_modules), else the checkout's TypeScript. No shell, no toolchain: node alone
+// (the kernel is the wasm file the package ships). As bin/skein-host does: SKEIN_HOME (default
+// ~/.skein), and for run | add | event | init SKEIN_OWNER / SKEIN_INFER from owner.identity /
+// infer.identity under it when unset.
+import { existsSync, readFileSync } from "node:fs";
+import { homedir } from "node:os";
+import { join } from "node:path";
+
+const root = new URL("..", import.meta.url);
+const home = process.env.SKEIN_HOME || join(homedir(), ".skein");
+process.env.SKEIN_HOME = home;
+const cmd = process.argv[2] ?? "";
+if (["run", "add", "event", "init"].includes(cmd)) {
+  for (const [k, f] of [["SKEIN_OWNER", "owner.identity"], ["SKEIN_INFER", "infer.identity"]]) {
+    if (!process.env[k] && existsSync(join(home, f))) process.env[k] = readFileSync(join(home, f), "utf8").trim();
+  }
+}
+const built = new URL("dist/host/cli.js", root);
+const { main } = await import(existsSync(built) ? built.href : new URL("src/host/cli.ts", root).href);
+process.exitCode = await main(process.argv.slice(2), { vars: process.env, out: (l) => process.stdout.write(`${l}\n`), err: (l) => process.stderr.write(`${l}\n`) });

package/bin/skein.js

@@ -0,0 +1,8 @@
+#!/usr/bin/env node
+// skein — the client — for the npm package (and any checkout): dist/client/cli.js, else src/client/cli.ts.
+import { existsSync } from "node:fs";
+
+const root = new URL("..", import.meta.url);
+const built = new URL("dist/client/cli.js", root);
+const { main } = await import(existsSync(built) ? built.href : new URL("src/client/cli.ts", root).href);
+main(process.argv.slice(2)).then((code) => process.exit(code), (e) => { process.stderr.write(`skein: ${e?.message ?? e}\n`); process.exit(1); });

package/dist/client/admin-cli.d.ts

@@ -0,0 +1,10 @@
+import { type Vars } from "../host/hostenv.ts";
+export declare const ADMIN_USAGE = "usage (the operator's messages, signed with SKEIN_OPERATOR_KEY, default $SKEIN_HOME/operator.key):\n  skein install <catalog-name | url#commit | dir> <where> [--config <file.json>] [--dry-run]\n                                                     an app (docs/APPS.md \u00A73): a name from the instance's catalog or a repository\n                                                     and commit, cloned by the instance's git app; head <app>/app, its rows, start\n  skein uninstall <app> <where> [--dry-run]          its stop, then its routes removed (the heads are left)\n  skein routes add [--transport mailbox|event|http|libp2p] [--prefix] [--filters f,g] [--fn f] [--settings json] <address> [<handler>] <where> [--dry-run]\n                                                     root's own route (#143): <handler> a program record CID, a genesis program,\n                                                     or <app>.<role>; an http route with --filters and no handler is a read route\n                                                     (the site at /: routes add --prefix --filters site.get --settings '{\"root\":\"www\"}' /)\n  skein routes remove [--transport t] [--prefix] <address> <where> [--dry-run]\n  skein grant add|remove <key> [--role root|<app>.<role>] <where> [--dry-run]     a grant (#143; root by default)\n  skein peers add <key> <address> [--transport mailbox|libp2p|local] [--handle h@d] <where> [--dry-run]\n  skein peers remove <key> <where> [--dry-run]      an address-book entry (#70)\n  skein host add|remove (<host-key> --x sats [--rates json] | --from <host origin>) <where> [--dry-run]\n                                                     the host row (#130)\n  skein claim [--messagebox url] [--handle h@d] <where> [--dry-run]               an image's claim (#127, #143): root to the key that sends it\n  skein deploy <dir> [--only glob,glob | --all] <where> [--dry-run]               a directory into main (objects, head)\n<where>: --instance <handle> (this host machine: over its control socket) | <origin> (one BRC-104 session)\n         | --store <runtime.db> (read only: planned and printed, nothing sent)";
+export declare const ADMIN_COMMANDS: readonly ["install", "uninstall", "routes", "grant", "peers", "host", "claim", "deploy"];
+export interface AdminEnv {
+    vars: Vars;
+    out(line: string): void;
+    err(line: string): void;
+}
+/** `skein <command> …` for the admin commands. */
+export declare function adminMain(cmd: string, argv: string[], env: AdminEnv): Promise<number>;