@jr2/orchestrator 0.4.1 → 0.5.1

package/docs/guides/mirror-the-kit-images.md

@@ -0,0 +1,176 @@
+---
+title: Mirror the Kit images
+description: Copy the Kit images and the Pinned image into your own registry, and make the cluster pull them from there.
+---
+
+# Mirror the Kit images
+
+Copy the Kit images and the Pinned image into your own registry, and make the cluster pull them from there.
+
+An installed Kit builds none of its own images. The nodes pull the three
+[Kit images](../reference/glossary.md#kit-image), `jr2-harness`, `jr2-operator` and `jr2-orchestrator`, from
+`ghcr.io/snapwich`. They pull the [Pinned image](../reference/glossary.md#pinned-image), the Envoy image of the
+[Custodian](../reference/glossary.md#custodian), from `docker.io`. Mirror these four images when the nodes cannot reach
+those registries, or when the cluster pulls from one registry only.
+
+`jr2 up` also checks each model that an [Agent](../reference/glossary.md#agent) names, from a pod in the cluster. That
+pod uses the Harness Kit image and the Envoy image, so the nodes also pull the images of that pod from the mirror.
+
+[`jr2 kit push`](../reference/cli.md#kit-push) copies the images. [`kitRegistry`](../reference/config.md#kitregistry)
+tells `jr2 up` where the nodes pull them from:
+
+```ts
+import { defineConfig } from "@jr2/orchestrator";
+
+export default defineConfig({
+  git: {
+    credentials: [{ match: "*", token: "JR2_GIT_TOKEN", sshKey: "jr2-git-ssh" }],
+  },
+  // The registry that holds the copy of the Kit images, such as registry.example.com/jr2.
+  // When JR2_KIT_REGISTRY is not set, the cluster pulls them from ghcr.io/snapwich.
+  kitRegistry: process.env.JR2_KIT_REGISTRY,
+});
+```
+
+## Before you start
+
+- An [Instance](../reference/glossary.md#instance), with its dependencies installed at the
+  [Kit version](../reference/glossary.md#kit-version) that you want to mirror.
+- Docker with Buildx on your computer. `jr2 kit push` uses `docker buildx imagetools`.
+- A registry that you can push to, and that the nodes can pull from. Log in to it with `docker login`. In the commands
+  on this page, `<registry>` is the registry and its path, such as `registry.example.com/jr2`.
+- Your computer can pull from `ghcr.io` and `docker.io`.
+
+## Copy the images
+
+Run `jr2 kit push` in the Instance folder:
+
+```sh
+jr2 kit push <registry>
+```
+
+```text
+…
+jr2 kit push — mirroring the v<version> kit images to <registry>
+harness: copied ghcr.io/snapwich/jr2-harness:<version> → <registry>/jr2-harness:<version>
+operator: copied ghcr.io/snapwich/jr2-operator:<version> → <registry>/jr2-operator:<version>
+orchestrator: copied ghcr.io/snapwich/jr2-orchestrator:<version> → <registry>/jr2-orchestrator:<version>
+custodian: copied docker.io/envoyproxy/envoy:<tag>@<digest> → <registry>/envoy:<tag>@<digest>
+mirrored 4 image(s), 0 already present
+{"registry":"<registry>","version":"<version>","images":{"harness":"<registry>/jr2-harness:<version>","operator":"<registry>/jr2-operator:<version>","orchestrator":"<registry>/jr2-orchestrator:<version>","custodian":"<registry>/envoy:<tag>@<digest>"}}
+```
+
+`<version>` is the Kit version. `<tag>` and `<digest>` are the tag and the digest of the Envoy image that this Kit
+version pins. The first line (`…`) is the `.env` line, when `.env` sets a variable that your shell does not set. The
+last line is on stdout. The other lines are on stderr.
+
+`jr2 kit push` copies the images from registry to registry. It copies each platform of an image, with the same digests,
+and it does not put the images on your computer.
+
+`jr2 kit push` does not read the Instance. In the Instance folder, the global `jr2` runs the Instance's own `@jr2/cli`
+([Handoff](../reference/glossary.md#handoff)), so the command copies the images of the Instance's Kit version. Outside
+an Instance, it copies the images of the version of your global `jr2`.
+
+A version tag of jr2 never moves to different content. Thus `jr2 kit push` does not copy an image that is in the
+registry already. You can run it again at any time:
+
+```text
+…
+jr2 kit push — mirroring the v<version> kit images to <registry>
+harness: already present — <registry>/jr2-harness:<version>
+operator: already present — <registry>/jr2-operator:<version>
+orchestrator: already present — <registry>/jr2-orchestrator:<version>
+custodian: already present — <registry>/envoy:<tag>@<digest>
+mirrored 0 image(s), 4 already present
+…
+```
+
+## Pull from the mirror
+
+Set `kitRegistry` in `jr2.config.ts`, as in the example. The example reads it from `.env`. Add this line to `.env`:
+
+```text
+JR2_KIT_REGISTRY=<registry>
+```
+
+Give `kitRegistry` the same value that you gave `jr2 kit push`, with no `/` at the end. The nodes then pull
+`<registry>/jr2-harness:<version>`, `<registry>/jr2-operator:<version>`, `<registry>/jr2-orchestrator:<version>` and
+`<registry>/envoy:<tag>@<digest>`.
+
+`jr2 version --local` shows where the Kit images come from:
+
+```sh
+jr2 version --local
+```
+
+```text
+→ .env: JR2_KIT_REGISTRY, …
+cli:       <version>    <instance folder>/node_modules/@jr2/cli
+global:    <global version>    <global folder>
+kit:       <version>    <instance folder>/node_modules/@jr2/orchestrator  ok
+instance:  my-instance  <instance folder>  (installed kit → <registry>)
+node:      v24.18.0
+```
+
+`<instance folder>` is the path of your Instance folder. `<global version>` and `<global folder>` are the version and
+the folder of your global `jr2`, which runs the Instance's own `@jr2/cli`. `node` gives your version of Node.js.
+
+Then converge:
+
+```sh
+jr2 up
+```
+
+```text
+…
+images: installed kit — the published v<version> Harness, operator and Orchestrator, pulled from <registry> (kitRegistry)
+custodian image: <registry>/envoy:<tag>@<digest> (pinned, deployed — never built)
+…
+converged — `jr2 run <workflow>` when ready
+```
+
+## Mirror each new Kit version
+
+The mirror holds the images of the versions that you copied, and no others. `jr2 up` never pushes to `kitRegistry`.
+Thus, when you upgrade the Kit version, run `jr2 kit push <registry>` again before `jr2 up`. Run it in the Instance
+folder after you install the new version. See [Upgrade the Kit version](upgrade-the-kit-version.md).
+
+One mirror can serve each Instance on the cluster, and each cluster of your organization. It can hold the images of more
+than one Kit version.
+
+`kitRegistry` is not `registry`. [`registry`](../reference/config.md#registry) is where `jr2 up` pushes the images that
+it builds. `kitRegistry` is where the nodes pull the Kit images and the Pinned image. You can set one, the other, or
+both.
+
+jr2 sets no image pull secret on its pods. The nodes must pull from the mirror with no pull secret.
+
+## Rules
+
+- `jr2 kit push` needs the registry:
+
+  ```text
+  jr2 kit push: no target registry
+  usage: jr2 kit push <registry>
+  …
+  ```
+
+  It exits with code 2.
+
+- `push` is the only subcommand of `jr2 kit`. `jr2 kit` refuses any other word with `unknown kit subcommand: <word>`,
+  and exits with code 2. With no word, it prints `jr2 kit: no subcommand`, and exits with code 2.
+- When the image is not at its published place, `jr2 kit push` stops:
+  `error: <image>: <source> is not there, so nothing can be mirrored to <target>`. `<image>` is `harness`, `operator`,
+  `orchestrator` or `custodian`. `<source>` is its published reference, and `<target>` is its reference in the mirror.
+  The next lines of the error say why. Check that `<version>` is a released Kit version.
+- When a copy fails, `jr2 kit push` stops, and exits with code 1. It gives
+  `error: Command failed: docker buildx imagetools create -t <target> <source>`, then the lines of Docker. Check that
+  `docker login` has logged you in to the registry, and that the registry is reachable.
+
+## Read next
+
+- [Converge](../concepts/converge.md)
+- [ADR-0044](../reference/decisions/0044-kit-images-live-at-a-canonical-home-a-self-host-mirrors-it.md): the canonical
+  place of the Kit images, and the mirror.
+- [ADR-0059](../reference/decisions/0059-a-harness-holds-stand-ins-and-the-custodian-holds-the-keys.md): the Custodian
+  and its Envoy image.
+- [Move from kind to a shared cluster](move-from-kind-to-a-shared-cluster.md)

package/docs/guides/move-from-kind-to-a-shared-cluster.md

@@ -0,0 +1,302 @@
+---
+title: Move from kind to a shared cluster
+description: Converge an Instance into a shared cluster, with a registry, Sandbox nodes, priority classes and a quota.
+---
+
+# Move from kind to a shared cluster
+
+Converge your [Instance](../reference/glossary.md#instance) into a shared Kubernetes cluster, after it runs on kind.
+
+The [Workflows](../reference/glossary.md#workflow) of the Instance do not change. [`jr2 up`](../reference/cli.md#up)
+converges into the cluster of the kubectl context, so you give it a different context. A shared cluster also needs some
+facts that kind does not need. These facts are a registry for the images, the platforms of the nodes, the nodes for the
+[Sandboxes](../reference/glossary.md#sandbox), and the priority classes. This `jr2.config.ts` gives all of them:
+
+```ts
+import { defineConfig } from "@jr2/orchestrator";
+
+export default defineConfig({
+  git: {
+    credentials: [{ match: "*", token: "JR2_GIT_TOKEN", sshKey: "jr2-git-ssh" }],
+  },
+  // Where jr2 up pushes the images that it builds, such as registry.example.com/team.
+  registry: process.env.JR2_REGISTRY,
+  // The platforms to build for, such as linux/arm64. When JR2_PLATFORMS is not set, jr2 up reads them from the nodes.
+  platforms: process.env.JR2_PLATFORMS?.split(","),
+  // The Sandboxes run only on the nodes of the agents pool. Its nodes have the taint pool=agents:NoSchedule.
+  sandbox: {
+    nodeSelector: { pool: "agents" },
+    tolerations: [{ key: "pool", operator: "Equal", value: "agents", effect: "NoSchedule" }],
+  },
+  // The PriorityClasses of the cluster. jr2 up creates none for a key that you set here.
+  priorityClasses: { control: "platform-critical", sandbox: "batch-low" },
+});
+```
+
+Add the fields to your own `jr2.config.ts`. Keep the fields that it has, such as `secrets` or `harness`. Each section
+below explains one field. Set only the fields that your cluster needs.
+
+## Before you start
+
+- An Instance that converges on kind. See [Iterate on a Machine on kind](iterate-on-a-machine-on-kind.md).
+- A kubectl context for the shared cluster. In the commands on this page, `<context>` is its name.
+- An image registry that you can push to from your computer, and that the nodes can pull from. In the commands on this
+  page, `<registry>` is its prefix, such as `registry.example.com/team`.
+- Docker on your computer, logged in to the registry with `docker login`.
+
+## Check the cluster
+
+`jr2 up` checks the cluster before it applies the Instance. The cluster must have these properties:
+
+- Kubernetes 1.35 or later. Each Sandbox mounts the runtime of jr2 as an image volume.
+- Pod-level resources. A [Workspace](../reference/glossary.md#workspace) reserves its whole
+  [Size](../reference/glossary.md#size) as one pod-level limit. The `PodLevelResources` feature gate must be on in the
+  API server and in the kubelets.
+- A container runtime that mounts image volumes, such as containerd 2.1 or later, or CRI-O 1.31 or later. `jr2 up` does
+  not check the runtime. With an older runtime, the first Sandbox fails, and its pod events give the cause.
+- Your kubectl user can apply objects outside the namespace of the Instance. `jr2 up` applies the operator in the
+  `jr2-system` namespace, with its CustomResourceDefinitions, ClusterRoles and ClusterRoleBindings. It also applies two
+  PriorityClasses, unless you name your own.
+
+All the Instances on a cluster share one operator. `jr2 up` does not downgrade it. See
+[Converge](../concepts/converge.md#kit-version-and-handoff).
+
+## Set the registry
+
+On kind, `jr2 up` loads each image that it builds into the nodes with `kind load`. A shared cluster is not kind, so
+`jr2 up` pushes each image to [`registry`](../reference/config.md#registry), and the nodes pull it from there. These are
+the [Sandbox Images](../reference/glossary.md#sandbox-image) that `jr2 up` builds: `images/default` and each `file:`
+URL. When no registered Machine makes a [Sandbox](../reference/glossary.md#sandbox), `jr2 up` builds no image, and the
+Instance needs no `registry`.
+
+The example reads `registry` from `JR2_REGISTRY`. Put this line in `.env`:
+
+```text
+JR2_REGISTRY=<registry>
+```
+
+`jr2 up` reads `.env`. A variable of the shell wins over `.env`.
+
+When `registry` is set, `jr2 up` pushes to it on kind too. To converge into kind again, remove `JR2_REGISTRY` from
+`.env`, or set it only in the shell for the shared cluster.
+
+jr2 sets no image pull secret on its pods. The nodes must pull from the registry with no pull secret, for example with
+credentials on the nodes or with anonymous pulls.
+
+The [Kit images](../reference/glossary.md#kit-image) do not go to `registry`. The image of the
+[Orchestrator](../reference/glossary.md#orchestrator) is one of them. The nodes pull them from `ghcr.io/snapwich`, and
+the Envoy image of the [Custodian](../reference/glossary.md#custodian) from `docker.io`. When the nodes cannot reach
+those registries, see [Mirror the Kit images](mirror-the-kit-images.md).
+
+`jr2 up` also checks each model that an [Agent](../reference/glossary.md#agent) names, from a pod in the cluster. That
+pod uses the Harness Kit image and the Envoy image of the Custodian, so it needs no other image.
+
+## Choose the platforms
+
+`jr2 up` builds each image for the platforms of the nodes. It reads the architecture of each node on which a pod can
+run, and of each [Sandbox node](../reference/glossary.md#sandbox-node). jr2 releases its images for `linux/amd64` and
+`linux/arm64` only. `jr2 up` skips a node of a different architecture, and tells you. This output is from a cluster with
+one `amd64` node and one `s390x` node:
+
+```text
+platforms: linux/amd64 (from 2 schedulable node(s))
+  skipping node arch s390x — the kit publishes no Kit image for it, so no pod there could run
+```
+
+When the cluster has nodes of the two architectures, `jr2 up` builds each image for both, and pushes it as one manifest
+list. This needs `registry`.
+
+Set [`platforms`](../reference/config.md#platforms) when the nodes do not give the correct set:
+
+- A node group that scales up from zero has no node to read yet.
+- A node group that needs no image, such as an amd64 GPU group beside arm64 nodes, adds a platform that you do not want.
+
+`platforms` replaces the set that `jr2 up` reads from the nodes. It does not add to that set. The example reads it from
+`JR2_PLATFORMS`, as a list with commas, such as `JR2_PLATFORMS=linux/arm64`.
+
+When you set `platforms`, `jr2 up` does not read the nodes. Thus it also does not check the Sandbox nodes or the Size.
+See [Choose the Sandbox nodes](#choose-the-sandbox-nodes).
+
+When a platform is not the platform of your computer, Docker emulates it with QEMU. `jr2 up` stops before the build when
+your computer cannot do this, and gives the command that installs QEMU.
+
+## Choose the Sandbox nodes
+
+By default, a Sandbox runs on each node on which an ordinary pod can run. That is a node that is not cordoned and has no
+`NoSchedule` or `NoExecute` taint. You do not add a label for jr2.
+
+To put the Sandboxes on a node pool of their own, set [`sandbox`](../reference/config.md#sandbox).
+[`sandbox.nodeSelector`](../reference/config.md#sandbox-nodeselector) gives the labels that a Sandbox node must have.
+[`sandbox.tolerations`](../reference/config.md#sandbox-tolerations) gives the taints that a Sandbox tolerates. Both have
+the shapes of the Kubernetes pod spec. The Repo cache runs on the same nodes as the Sandboxes.
+
+When you do not set `platforms` and a Workflow uses a Workspace, `jr2 up` gives the Sandbox nodes that it finds. In this
+output, the cluster has one node, `jr2-control-plane`:
+
+```text
+sandbox nodes: jr2-control-plane
+```
+
+When it finds none, it gives a warning and continues. It names each node and the reason that the node is not a Sandbox
+node:
+
+```text
+warning: no Sandbox node right now — a Sandbox stays Pending until one appears:
+  jr2-control-plane: lacks the label pool that sandbox.nodeSelector requires
+  a Sandbox lands where an ordinary pod lands; to admit a tainted or labeled node, set `sandbox: { nodeSelector, tolerations }` in jr2.config.ts (raw pod-spec shapes, ADR-0052).
+```
+
+The Repo cache runs only on the Sandbox nodes, so it has no pod either. `jr2 up` says so later in its output:
+`repo cache: WARNING — rollout succeeded but no pod matched app=jr2-repo-cache (nothing verified)`.
+
+When no Sandbox node can hold the Size of a Workspace now, `jr2 up` also gives a warning and continues. In this output,
+the Workflow `hold` asks for a Size of 512Gi of memory and 64 cpus:
+
+```text
+warning: no Sandbox node right now has the allocatable for a Size of memory 512Gi, cpu 64 (workflow "hold") — its Sandbox stays Pending until one does
+```
+
+The Workspace stays in [Placing](../reference/glossary.md#placing) until a Sandbox node can hold its Sandbox, for
+example a node that an autoscaler adds.
+
+Write `sandbox` in `jr2.config.ts` itself. Do not read it from `process.env`. The Orchestrator reads `jr2.config.ts`
+again in the cluster, where your shell and `.env` are not available.
+
+## Use the priority classes of the cluster
+
+`jr2 up` creates two PriorityClasses by default. `jr2-control` is for the Orchestrator, the operator, the Repo cache and
+the [Instance Effector](../reference/glossary.md#instance-effector). `jr2-sandbox` is for the Sandboxes and the
+[Instance Harness](../reference/glossary.md#instance-harness). An ordinary pod cannot preempt a Sandbox, and a Sandbox
+that waits for a node preempts no pod.
+
+When the cluster has its own priority scheme, set [`priorityClasses`](../reference/config.md#priorityclasses) to the
+names of its classes. `jr2 up` creates no class for a key that you set, and tells you which classes it uses:
+
+```text
+priority: the Instance's own — control "platform-critical", sandbox "batch-low"
+```
+
+`jr2 up` does not create a class that you name, so it must exist in the cluster. Write `priorityClasses` in
+`jr2.config.ts` itself. Do not read it from `process.env`, for the same reason as `sandbox`.
+
+> **Caution**
+>
+> `sandbox` and `priorityClasses` apply in each cluster that the folder converges into. Thus a folder that sets them works
+> only in a cluster that has those classes and those labels. On kind, `jr2 up` fails, because Kubernetes creates no pod
+> that names a class that the cluster does not have. Also, a Sandbox waits for a node with the label `pool: agents`.
+
+## Work under a quota
+
+jr2 writes no ResourceQuota, and it has no limit of its own on the number of Workspaces. The owner of the cluster can
+put a ResourceQuota on the namespace of the Instance. Each Sandbox requests its whole Size, so the quota counts the
+whole Size of each Sandbox.
+
+When the quota refuses the pod of a Sandbox, the Workspace waits in Placing. It does the same when no node has room. jr2
+sets no time limit on this wait. `jr2 run` shows the reason, in the words of the quota. In this output, the quota
+`team-quota` permits 512Mi of memory limits, and the Size asks for 1Gi:
+
+```text
+…
+  → active "placing"
+  placing: waits for quota — pods "ws-5e817334-x-0-61ru0u" is forbidden: exceeded quota: team-quota, requested: limits.memory=1Gi, used: limits.memory=2Gi, limited: limits.memory=512Mi
+```
+
+For a Workspace that is not the root of the Workflow, the line adds the path of the child after `placing`.
+
+[`jr2 runs`](../reference/cli.md#runs) marks each run that has a Workspace in Placing. To see why it waits, run
+`jr2 status <run id>`. `<run id>` is the run id or a prefix of it (see [run ids](../reference/cli.md#run-ids)). To limit
+the time that a Workflow waits, put `after` on the state of the [Machine](../reference/glossary.md#machine) that holds
+the Workspace.
+
+## Converge
+
+Run `jr2 up` with the context of the shared cluster. For the first converge into a namespace, `jr2 up` asks you before
+it continues. In CI, add `--yes`.
+
+```sh
+jr2 up --context <context>
+```
+
+```text
+→ .env: JR2_REGISTRY, …
+jr2 up — instance "my-instance" → context <context> / namespace my-instance
+typecheck: tsc --noEmit (the instance's own compiler)
+…
+kubernetes: v1.36.1 (floor 1.35, ADR-0037)
+first contact: deploy instance "my-instance" to context <context> (create namespace "my-instance")? [y/N] y
+platforms: linux/amd64 (from 1 schedulable node(s))
+sandbox nodes: jr2-control-plane
+images: installed kit — the published v<version> Harness, operator and Orchestrator, pulled from ghcr.io/snapwich
+…
+priority: the Instance's own — control "platform-critical", sandbox "batch-low"
+sandbox image "default": building <registry>/jr2-sandbox-my-instance-default:6389bdf72d50-amd64
+  no USER declared — the pod's uid-1000 fallback applies
+  push → <registry>
+…
+assembly: publishing 6355c2bb032a (no current Assembly)
+assembly: 6355c2bb032a is current — workflows: hold, ping
+…
+converged — `jr2 run <workflow>` when ready
+```
+
+In this output, the cluster runs Kubernetes v1.36.1 and has one node, `jr2-control-plane`, with the label
+`pool: agents`. The Workflows are `ping` and `hold`, a Workflow with a Workspace. `<version>` is the
+[Kit version](../reference/glossary.md#kit-version) of the Instance. `6389bdf72d50` changes when the inputs of the image
+change. `6355c2bb032a` is the hash of the [Assembly](../reference/glossary.md#assembly) of the Instance. Your version of
+Kubernetes, your nodes and your hashes are different.
+
+The other commands take the same `--context`, such as `jr2 run ping --context <context>`.
+
+These things also change in a shared cluster:
+
+- The pods must reach the model. A LAN address of your computer that works from kind can fail from a shared cluster. See
+  [Use a local model](use-a-local-model.md) and [Use a hosted provider or a gateway](use-a-hosted-provider.md).
+- The cluster needs the git credentials. See [Give the cluster git credentials](give-the-cluster-git-credentials.md).
+- The cluster needs your Secrets. `jr2.config.ts` can name a Secret in the namespace of the Instance, with `valueFrom`
+  in `secrets`, with `valueFrom` in `harness.env`, or with `harness.envFrom`. A Secret that you made on kind is not in
+  the shared cluster. Make each of these Secrets in the namespace of the shared cluster before you converge.
+
+## Rules
+
+- On a cluster that is not kind, `jr2 up` refuses to build a Sandbox Image when you do not set `registry`. A kind
+  cluster is a context whose name starts with `kind-`:
+  ``error: context <context> is not a kind cluster and no `registry` is configured — set `registry` in jr2.config.ts (from env) so the image can be pushed (ADR-0019)``
+- `jr2 up` refuses a cluster earlier than Kubernetes 1.35:
+  `refusing: <context> runs Kubernetes <kubernetes version>; jr2 needs Kubernetes 1.35 or newer (every Sandbox mounts jr2's runtime as an image volume, ADR-0037). Nothing was applied.`
+- `jr2 up` refuses a cluster that does not keep pod-level resources:
+  `refusing: <context> (Kubernetes <kubernetes version>) drops pod-level resources — the PodLevelResources feature gate is off, and a Workspace's Size cannot hold (ADR-0060). Enable the gate on the API server and kubelets. The namespace was created; nothing else was applied.`
+- `jr2 up` refuses a namespace that a different Instance owns:
+  `refusing: namespace "<namespace>" on <context> belongs to another instance ("<instance>")`. Give a different
+  namespace with `-n`, or a different context.
+- A first converge into a namespace needs an answer. With no terminal and no `--yes`, `jr2 up` stops:
+  `first contact: deploy instance "<name>" to context <context> (<what>)? — not a tty; pass --yes to proceed non-interactively`
+- `platforms` holds `linux/amd64` and `linux/arm64` only:
+  ``error: `platforms` names <platforms> — the kit releases images for linux/amd64, linux/arm64 and nothing else, so an image built for anything else could not be joined by a Kit image in the same pod. Set `platforms` in jr2.config.ts (docker platform strings, e.g. "linux/arm64") to name the set to build for.``
+- When no node has a released platform, `jr2 up` refuses:
+  ``error: this cluster's schedulable nodes report <architectures> — the kit releases images for linux/amd64, linux/arm64 and nothing else, so an image built for anything else could not be joined by a Kit image in the same pod. Set `platforms` in jr2.config.ts (docker platform strings, e.g. "linux/arm64") to name the set to build for.``
+- When your computer cannot emulate a platform of the build, `jr2 up` stops before the build:
+  `error: this host cannot build for <missing platforms>: a foreign architecture's RUN steps need qemu binfmt emulation, and none is registered (this builder targets <builder platforms>). Install it, then re-run:`,
+  then `docker run --privileged --rm tonistiigi/binfmt --install <arches>`. `<missing platforms>` are the platforms that
+  your builder cannot build, `<builder platforms>` are the platforms that it can build, and `<arches>` are the
+  architectures of the missing platforms.
+- `jr2 up` refuses a Secret that `jr2.config.ts` names and that is not in the namespace. For `valueFrom` in `secrets`:
+  `error: secrets[<i>].valueFrom names Secret "<name>", which does not exist in the instance's namespace — create it first: kubectl create secret generic <name> --from-literal=<key>=... (ADR-0019, ADR-0059)`.
+  For `harness.envFrom`:
+  `error: harness.envFrom references Secret "<name>", which does not exist in namespace "<namespace>" — create it first: kubectl -n <namespace> create secret generic <name> --from-literal=KEY=...`.
+- `priorityClasses` has the keys `control` and `sandbox` only, and each is a name that is not empty:
+  `error: <file> at priorityClasses<path>: <problem> — { control?, sandbox? }, each the name of an existing PriorityClass (ADR-0060)`
+
+## Read next
+
+- [Converge](../concepts/converge.md)
+- [Workspaces](../concepts/workspaces.md): the Size, the Sandbox nodes, and Placing.
+- [ADR-0019](../reference/decisions/0019-one-converging-command-against-the-current-context.md): one command against the
+  current context.
+- [ADR-0045](../reference/decisions/0045-the-platform-joins-the-image-address-and-the-cluster-chooses-it.md): the
+  platforms.
+- [ADR-0052](../reference/decisions/0052-a-sandbox-lands-where-an-ordinary-pod-lands-and-the-cache-follows-it.md): the
+  Sandbox nodes.
+- [ADR-0060](../reference/decisions/0060-a-workspace-states-its-size-and-jr2-reserves-all-of-it.md): the Size and the
+  priority classes.
+- [ADR-0064](../reference/decisions/0064-a-workspace-waits-for-capacity.md): the wait for capacity and the quota.
+- [Mirror the Kit images](mirror-the-kit-images.md)

package/docs/guides/open-the-console.md

@@ -0,0 +1,91 @@
+---
+title: Open the Console
+description: Forward the port of the Orchestrator, open the Console, and unlock it with the Instance token.
+---
+
+# Open the Console
+
+Open the [Console](../reference/glossary.md#console) in a browser, and paste the Instance token to start runs and answer
+[Gates](../reference/glossary.md#gate).
+
+The [Orchestrator](../reference/glossary.md#orchestrator) serves the Console on port 4000, in the cluster. To reach it
+from your computer, forward the port with `kubectl`. In the commands on this page, replace `<namespace>` with the
+namespace of your [Instance](../reference/glossary.md#instance). It is the [`name`](../reference/config.md#name) in
+`jr2.config.ts`. When `jr2.config.ts` sets no `name`, it is the name of the Instance folder. No `jr2` command opens the
+Console.
+
+## Forward the port
+
+```sh
+kubectl -n <namespace> port-forward svc/jr2-orchestrator 4000:4000
+```
+
+```text
+Forwarding from 127.0.0.1:4000 -> 4000
+Forwarding from [::1]:4000 -> 4000
+```
+
+The command stays in the foreground. Keep it open while you use the Console. Give the other commands in a second
+terminal. To use a different port on your computer, change the first number, such as `8080:4000`. For a cluster that is
+not the current kubectl context, add `--context <context>`, with the name of its kubectl context.
+
+## Open the page in a browser
+
+Open `http://localhost:4000` in a browser.
+
+Without the token, the Console shows the runs, and you cannot change them:
+
+- The Fleet rail on the left lists each [Workflow](../reference/glossary.md#workflow), with the number of its live runs.
+  Click the caret of a Workflow to list its runs.
+- The center shows the [Machine](../reference/glossary.md#machine) of the Workflow that you select. When you select a
+  run, the Console shows the states that the run is in.
+- The Activity tab of the right drawer lists the type of each [Emit](../reference/glossary.md#emit) of the runs of the
+  Workflow, the newest first.
+
+The address shows the Workflow that you select. Open `http://localhost:4000/workflows/<workflow>` to go to one Workflow
+directly. Replace `<workflow>` with its name, such as `ping`.
+
+## Get the Instance token
+
+`jr2 up` makes the [Instance token](../concepts/secrets-and-trust.md#the-tokens) when it first converges the Instance,
+and keeps it after that. It is in the Kubernetes Secret `jr2-instance`:
+
+```sh
+kubectl -n <namespace> get secret jr2-instance -o jsonpath='{.data.JR2_INSTANCE_TOKEN}' | base64 -d
+```
+
+The command prints the token, with no line break after it.
+
+> **Caution**
+>
+> The Instance token starts runs, answers Gates and cancels runs. Keep it as you keep a password.
+
+## Paste the token
+
+Paste the token into the box `instance token` in the nav of the Console. Push Enter. The badge after the box shows `…`
+while the Console checks the token, then `live`. With a live token, the Console shows more:
+
+- A `start` button for each Workflow in the Fleet rail. It opens a form for the run input of the Workflow. A Workflow
+  that declares no run input gets a box for JSON.
+- The Attention tab of the right drawer. It shows a card for each open Gate of the selected Workflow, with a form to
+  send each event that the Gate takes. Select `all workflows` to see the Gates of each Workflow.
+
+The Console keeps the token in the browser tab. A reload keeps it. When you close the tab, the token is gone.
+
+## Rules
+
+- A token that the Orchestrator does not accept reads `invalid`. The Console then shows no `start` button and no
+  Attention tab.
+- When the Orchestrator refuses the token later, with the status 401, the Console goes back to view only.
+- Only the Instance token unlocks the Console. A [Sandbox](../reference/glossary.md#sandbox) token reads `invalid`.
+- The port forward stops when you stop its command, or when the Orchestrator pod restarts. Then run the command again.
+
+## Read next
+
+- [Architecture](../concepts/architecture.md)
+- [ADR-0032](../reference/decisions/0032-the-console-unlocks-with-the-instance-token.md): why the token unlocks the
+  Console.
+- [ADR-0014](../reference/decisions/0014-observation-is-open-run-state-is-not.md): what the Console shows with no token.
+- [`GET /`](../reference/http-api.md#get) and the [open band](../reference/http-api.md#band-open)
+- [Answer a Gate](answer-a-gate.md)
+- [Start a run](start-a-run.md)