@jr2/orchestrator 0.4.1 → 0.6.0

package/docs/get-started/install.md

@@ -0,0 +1,130 @@
+---
+title: Install
+description: Install the tools that jr2 needs and the jr2 CLI, and make a kind cluster on your computer.
+---
+
+# Install
+
+At the end of this page, you have the [jr2 CLI](../reference/glossary.md#jr2-cli) and a kind cluster on your computer.
+The next pages use them to make an [Instance](../reference/glossary.md#instance) and run it.
+
+## Before you start
+
+This is the first page of the tutorial. You need a computer with these tools:
+
+- [Node.js](https://nodejs.org/) 24 or later. Node.js includes npm.
+- [Docker](https://docs.docker.com/get-started/get-docker/). Make sure that the Docker daemon runs.
+- [kind](https://kind.sigs.k8s.io/docs/user/quick-start/#installation), which runs a Kubernetes cluster in Docker.
+- [kubectl](https://kubernetes.io/docs/tasks/tools/), which speaks to the cluster.
+
+kind makes the cluster from Docker containers. `jr2 up` builds an image with Docker and loads it into the kind cluster.
+kubectl is the tool that `jr2 up`, `jr2 run` and the other commands use to speak to the cluster.
+
+The kind cluster uses the Linux kernel of Docker: the kernel of your computer, or on macOS and Windows, the kernel of
+the virtual machine of Docker. jr2 needs Landlock in that kernel: Linux 6.2 or later, with `landlock` in its list of
+security modules. `jr2 up` refuses to converge without Landlock.
+
+## Check Node.js
+
+Run:
+
+```sh
+node --version
+```
+
+The version must be `v24` or later:
+
+```text
+v24.18.0
+```
+
+Your version can be different.
+
+## Install the jr2 CLI
+
+Run:
+
+```sh
+npm i -g @jr2/cli
+```
+
+npm installs the `jr2` command and the `@jr2/orchestrator` package that it uses. npm prints its progress and a summary
+of the packages that it added. This page does not show that output. The next step checks the install.
+
+## Check the jr2 CLI
+
+Run:
+
+```sh
+jr2 --version
+```
+
+The CLI shows its version, its folder and the version of Node.js. Outside an Instance, it shows no Instance:
+
+```text
+cli:       <version>    /tmp/jr2-dist/npm-global/lib/node_modules/@jr2/cli
+instance:  none — no jr2.config.ts walking up from cwd
+node:      v24.18.0
+```
+
+`<version>` is the version of jr2 that you installed, such as `0.4.1`. The folder is in the global folder of your npm,
+so it can be different.
+
+## Make the kind cluster
+
+Run:
+
+```sh
+kind create cluster --name jr2
+```
+
+kind makes a cluster with one node. It sets your current kubectl context to the new cluster:
+
+```text
+Creating cluster "jr2" ...
+…
+Set kubectl context to "kind-jr2"
+…
+```
+
+The context of the cluster is `kind-jr2`. `jr2 up` converges into the current kubectl context, so the next pages use
+this cluster.
+
+## Check the Kubernetes version
+
+Run:
+
+```sh
+kubectl version
+```
+
+The `Server Version` must be `v1.35` or later:
+
+```text
+Client Version: v1.35.7
+Kustomize Version: v5.7.1
+Server Version: v1.36.1
+```
+
+Your versions can be different. If the server version is earlier than `v1.35`, delete the cluster with
+`kind delete cluster --name jr2`. Then install a newer kind release. Make the cluster again. `jr2 up` refuses an older
+cluster with this message, where `<server version>` is the `Server Version` of the cluster:
+
+```text
+refusing: kind-jr2 runs Kubernetes <server version>; jr2 needs Kubernetes 1.35 or newer (every Sandbox mounts jr2's runtime as an image volume, ADR-0037). Nothing was applied.
+```
+
+## What happened
+
+You installed the jr2 CLI globally. It is the command that you use for each step of this tutorial. Each Instance also
+installs its own copy of `@jr2/cli`. In an Instance folder, the global `jr2` gives the command to that copy. Thus the
+version that runs is the version that the Instance pins. The glossary calls this a
+[Handoff](../reference/glossary.md#handoff).
+
+You also made a kind cluster. jr2 needs no setup in the cluster other than Kubernetes 1.35 or later. `jr2 up` puts in it
+all that an Instance needs. [Converge](../concepts/converge.md) explains what `jr2 up` makes. The
+[CLI reference](../reference/cli.md) gives each command and its options.
+
+## Next
+
+[Create an Instance](create-an-instance.md).

package/docs/guides/add-tools-to-the-sandbox.md

@@ -0,0 +1,308 @@
+---
+title: Add tools to the Sandbox
+description: Put the programs that your Agents use into the Sandbox Image, or run a service in a User Container.
+---
+
+# Add tools to the Sandbox
+
+Put the programs that your [Agents](../reference/glossary.md#agent) and
+[`command`](../reference/authoring-surface/command.md) [Effects](../reference/glossary.md#effect) use into the
+[Sandbox Image](../reference/glossary.md#sandbox-image).
+
+A [Workspace](../reference/glossary.md#workspace) runs its body in a [Sandbox](../reference/glossary.md#sandbox): a pod
+of its own. The Sandbox Image is the image of the pod's `harness` container. The Agent's
+[Working tools](../reference/glossary.md#working-tools) run in that container, and so do the `command` Effects and the
+shell that you get with `kubectl exec`. Thus a program that is in the image is a program that they can run.
+
+Use this table to choose:
+
+| You want                                       | Do this                                                                                                        |
+| ---------------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
+| The same tools for each Workspace              | Change `images/default/Dockerfile`.                                                                            |
+| Different tools for the Workspaces of one file | Put a docker context beside the workflow module, and name it in the `image` option.                            |
+| An image that you build and push yourself      | Write its registry ref in the `image` option.                                                                  |
+| A service that runs beside the Agent           | Name its image in the `user` option. It runs in the [User Container](../reference/glossary.md#user-container). |
+
+## Add a tool to the default image
+
+`jr2 init` writes `images/default/Dockerfile`. Each Workspace that names no `image` in
+[`workspace()`](../reference/authoring-surface/workspace.md) uses it. This example adds pnpm to the file that `jr2 init`
+wrote:
+
+```dockerfile
+FROM node:24-slim
+
+RUN apt-get update \
+  && apt-get install -y --no-install-recommends git ca-certificates \
+  && rm -rf /var/lib/apt/lists/*
+
+# Your agents' toolchain goes here — compilers, CLIs, language servers, dotfiles.
+
+# pnpm on the system PATH, at a fixed version, so that each Sandbox gets the same pnpm.
+RUN npm install -g pnpm@10.9.0
+```
+
+The example does not show the comments at the start of the file. Keep them in your file.
+
+Install a tool at a fixed version, as the example does. Then each build of the image gets the same version.
+
+Run [`jr2 up`](../reference/cli.md#up). It builds the image. Then it loads the image into kind, or pushes it to the
+[`registry`](../reference/config.md#registry) of `jr2.config.ts`:
+
+```sh
+jr2 up
+```
+
+```text
+…
+sandbox image "default": building jr2-sandbox-my-instance-default:fce23ddc3cce-amd64
+  no USER declared — the pod's uid-1000 fallback applies
+  kind load → cluster "jr2"
+…
+```
+
+- The tag is the hash of the files in the folder, then the architecture of the nodes. Your hash is different. A change
+  to a file in the folder changes it. The architecture is such as `amd64`.
+- On kind, `jr2 up` loads the image into the cluster. The next line names the cluster.
+- When no file in the folder changed, `jr2 up` does not build the image again. The line then ends with
+  `(fresh — build skipped; --force to rebuild anyway)`.
+- `jr2 up` builds the Sandbox Images only when a registered [Machine](../reference/glossary.md#machine) has a
+  `workspace()`. Otherwise it prints `sandbox images: skipped (no registered Machine composes a Sandbox)`.
+
+A Sandbox that runs now keeps its image. The next Sandbox gets the new image. `jr2 up` removes the old image at the end
+of a later converge, when no Sandbox uses it. On a kind node, it keeps the old image for one more converge. It does not
+remove an image from a registry. [`jr2 gc`](../reference/cli.md#gc) does the same work when you ask for it (see
+[Clean up](clean-up.md)).
+
+To see the tool, open a shell in a Sandbox (see [Look inside a Sandbox](look-inside-a-sandbox.md)), and run it:
+
+```sh
+pnpm --version
+```
+
+```text
+10.9.0
+```
+
+## What the image must have
+
+The image is yours. jr2 adds no layer to it, and does not read the Dockerfile. When the pod starts, jr2 mounts its own
+runtime at `/opt/jr2`, and starts the [Harness](../reference/glossary.md#harness) from there. Thus:
+
+- The `ENTRYPOINT` and the `CMD` of the image do not run. To run a process all the time, use a User Container (see
+  [Run a service beside the Agent](#run-a-service-beside-the-agent)).
+- jr2 adds `/opt/jr2/bin` at the end of `PATH`. Thus a tool of the same name in your image comes first.
+- The `USER` and the `HOME` of the image stay as you set them. When an image that `jr2 up` built sets no `USER`, the pod
+  runs as uid 1000, with `HOME=/home/jr2`.
+
+The image must have these, or the Sandbox does not start:
+
+- A glibc base no older than the glibc of jr2's node, such as a current Debian or Ubuntu. An Alpine base has musl, and
+  jr2's runtime does not run on it.
+- `git` on the system `PATH`. The Harness clones, makes the [Worktrees](../reference/glossary.md#worktree) and commits
+  with it.
+- A `HOME` that the image's `USER` can write.
+- A `USER` that is a number and is not `0`, such as `USER 1000`. An image that `jr2 up` builds can also have no `USER`.
+  The `harness` container does not run as root.
+- Nothing at `/opt/jr2` or `/work`, and nothing that listens on port `8080`.
+
+The image does not need a git author. When the image sets none, a commit in the `harness` container has the author
+`jr2 Agent <agent@jr2.invalid>`. jr2 puts this default at git's system level, under the image's own `/etc/gitconfig`, so
+each git author that the image sets is used in its place: in `/etc/gitconfig`, in the `HOME` of its `USER`, or in
+`GIT_AUTHOR_NAME` and `GIT_AUTHOR_EMAIL`. When the image sets `GIT_CONFIG_SYSTEM` or `GIT_CONFIG_NOSYSTEM`, or its git
+reads another system file than `/etc/gitconfig`, jr2 does not set the default.
+
+Before the Harness starts, a `preflight` step in the pod checks the image. When the check fails, the run faults. Read
+the log of the step with `kubectl -n <namespace> logs <sandbox> -c preflight`. `<namespace>` is the namespace of the
+Instance, such as `my-instance`. [Look inside a Sandbox](look-inside-a-sandbox.md) tells how to find `<sandbox>`.
+
+## Give one Workflow its own image
+
+Put a docker context in a folder beside the workflow module. Name the folder in the `image` option of `workspace()`. In
+this example, `workflows/pytest.ts` runs the tests of a Python [Repo](../reference/glossary.md#repo), with the image in
+`workflows/python/`:
+
+```dockerfile
+# A Sandbox Image for a Python project. The build context is this folder.
+FROM python:3.13-slim
+
+# git is part of the floor: the Harness clones, makes the Worktrees and commits with it.
+RUN apt-get update \
+  && apt-get install -y --no-install-recommends git ca-certificates \
+  && rm -rf /var/lib/apt/lists/*
+
+# The tools of the project.
+RUN pip install --no-cache-dir pytest ruff
+```
+
+```ts
+import { z } from "zod";
+import { command, fail, isReady, jr2Setup, workspace, worktree, type Workspaced } from "@jr2/orchestrator";
+
+const runInput = z.object({ branch: z.string() });
+type BodyInput = Workspaced<z.infer<typeof runInput>, "app">;
+
+// The Worktree that the tests run in.
+const testTree = (context: BodyInput) => `${context.workspace.repos.app}/test` as const;
+
+const body = jr2Setup({
+  types: {} as { input: BodyInput; context: BodyInput },
+  actors: {
+    cut: worktree(),
+    // python and pytest come from the Sandbox Image in ./python.
+    tests: command(["python", "-m", "pytest"], { timeout: "15m" }),
+    fail: fail(),
+  },
+}).createMachine({
+  id: "pytest",
+  context: ({ input }) => input,
+  on: { "workspace.lost": ".lost" },
+  initial: "cutting",
+  states: {
+    cutting: {
+      invoke: {
+        src: "cut",
+        input: ({ context }) => ({ path: testTree(context), branch: context.branch }),
+        onDone: [{ guard: ({ event }) => isReady(event.output), target: "testing" }, { target: "notCut" }],
+      },
+    },
+    testing: {
+      invoke: {
+        src: "tests",
+        input: ({ context }) => ({ worktree: testTree(context) }),
+        onDone: [
+          {
+            guard: ({ event }) => event.output.outcome === "exited" && event.output.code === 0,
+            target: "passed",
+          },
+          { target: "failed" },
+        ],
+      },
+    },
+    passed: { type: "final" },
+    // Not final states: each one fails the run, and the run ends `failed`.
+    failed: { invoke: { src: "fail", input: { reason: "pytest did not pass" } } },
+    notCut: { invoke: { src: "fail", input: { reason: "the Worktree was not cut" } } },
+    lost: { invoke: { src: "fail", input: { reason: "the Sandbox was lost" } } },
+  },
+});
+
+export const machine = workspace(body, {
+  input: runInput,
+  // The docker context in the folder beside this file. jr2 up builds it.
+  image: import.meta.resolve("./python"),
+  repos: { app: "https://github.com/you/app.git" },
+});
+```
+
+- `import.meta.resolve("./python")` gives a `file:` URL to the folder. The folder is the build context. A `COPY` in the
+  Dockerfile reads from it.
+- jr2 reads only the `.ts` files in `workflows/` as [Workflows](../reference/glossary.md#workflow). A folder there is
+  not a Workflow.
+- An Agent in the body gets the same tools. Its Working tools run in the same container as the `command`.
+- `jr2 up` finds the folder from the registered Machines, and builds it. Its line is
+  `sandbox image "python": building jr2-sandbox-my-instance-python:<hash>-<arch>`. The name of the folder is the name in
+  the tag.
+
+A Machine that you register from a package, such as `task` from [`@jr2/machines`](../reference/machines.md), takes an
+image from [`customize()`](../reference/authoring-surface/customize.md): `customize(task, { image })`. The value has the
+same two forms, and it replaces the image of the Machine's `workspace()`. See
+[Bind a shipped Machine's Open parts](bind-open-parts.md).
+
+A Machine that you publish as an npm package carries its docker context in the package. See
+[Publish a Machine as an npm package](publish-a-machine-as-an-npm-package.md).
+
+## Use an image from a registry
+
+When you build and push the image yourself, write its ref in place of the `file:` URL:
+`image: "registry.example.com/python-sandbox:1.0"`. A value that does not start with `file:` is a registry ref.
+
+- jr2 does not build the image, does not read it, and does not remove it.
+- The nodes of the cluster pull it. jr2 gives the pod no pull credential.
+- The image must be for the architecture of the nodes.
+- The image must have each item in [What the image must have](#what-the-image-must-have). jr2 does not supply uid 1000
+  for a registry image: an image with no `USER` runs as root, and the pod does not start.
+- For each change, push a new tag. Then change the `image` value to the new tag. A node that has a tag does not pull it
+  again, unless the tag is `latest`.
+
+## Run a service beside the Agent
+
+The User Container is a third container in the Sandbox pod. Use it for a process that must run without an Agent, such as
+an sshd for a person. This image gives a person a shell beside the Agent:
+
+```dockerfile
+# A User Container: a shell for a person, beside the Agent, on the same Worktrees.
+FROM alpine:3.21
+
+RUN apk add --no-cache git
+
+# The Worktrees under /work belong to a different user. Tell git to trust them.
+RUN git config --system --add safe.directory '*'
+
+# jr2 does not replace this command. It must keep running: when it ends, the pod is not ready.
+CMD ["sh", "-c", "while true; do sleep 3600; done"]
+```
+
+Save it as `workflows/shell/Dockerfile`. Name it in the `workspace()` options with
+`user: import.meta.resolve("./shell")`. `user` takes the same two forms as `image`, and `jr2 up` builds a `file:`
+context in the same way. Open the shell with `kubectl -n <namespace> exec -it <sandbox> -c user -- sh`.
+
+The User Container is not a second Sandbox Image:
+
+- Its own `ENTRYPOINT` and `CMD` run. Its process must keep running: when it ends, the pod is not ready.
+- jr2 gives it `/work`, which it can write, and `/repos` and `/opt/jr2`, which it can only read. jr2 gives it no other
+  mount, no environment variable and no command.
+- It has no floor: an Alpine base is correct here. It runs as the `USER` of its image, and root is permitted.
+- Its user can be different from the user that owns the Worktrees, for example root. Thus an image that runs git needs
+  `git config --system --add safe.directory '*'`, as the example has.
+- The pod has one network. The User Container cannot listen on the ports of jr2's containers: `8080`, `8082` and
+  `15021`, and `8081` and `15001` on loopback.
+- To give it a part of the [Size](../reference/glossary.md#size), write `user: { image, resources }`. See
+  [Size a Workspace](size-a-workspace.md).
+
+A `push`, an `http` Effect whose body is a Worktree file, and a `command` that names an Effect secret run in a different
+container, the [Sandbox Effector](../reference/glossary.md#sandbox-effector). By default it uses the Sandbox Image. The
+`effector` option gives it an image of its own. See
+[Run a command or call a URL with an Effect](run-a-command-or-call-a-url.md) and
+[Push and sign a branch](push-and-sign-a-branch.md).
+
+## Rules
+
+- `workspace()` refuses an `image` that is empty or not a string:
+  ``workspace(): `image` must be a non-empty string (got <value>) — either a `file:` URL to a docker context this module ships (`import.meta.resolve("./image")`) or a registry ref (ADR-0037).``
+- `workspace()` and `customize()` refuse a `user` that is not an image or `{ image, resources? }`. `<where>` is
+  `workspace()` or `customize()`:
+  ``<where>: `user` must be a non-empty string (an image), or { image, resources? } (got <value>) — either a `file:` URL to a docker context this module ships (`import.meta.resolve("./image")`) or a registry ref (ADR-0037, ADR-0060).``
+  and `<where>: user.<key> is not accepted — the User Container is { image, resources? } (ADR-0005, ADR-0060)`.
+- `customize()` refuses `image` or `user` for a Machine that has no `workspace()`:
+  ``customize(): `image`/`user` say what a SANDBOX is made of, how big it is and which Repos it attaches, and machine "<id>" composes none — only a workspace() wrapper carries those seats, that Size and those slots (ADR-0049, ADR-0037, ADR-0051, ADR-0060).``
+- `jr2 up` refuses a `file:` URL to a folder that does not exist. `<path>` is the folder:
+  ``the Sandbox Image context <path> cannot be read (<code>) — a `workspace()` names its image with `import.meta.resolve("./<dir>")`, so that directory must exist beside the Machine's own module (ADR-0037/0049).``
+- `jr2 up` refuses an `images/default` folder that has no `Dockerfile`:
+  ``the Instance's `images/default` has no Dockerfile (<path>) — that directory IS a build context (ADR-0037), and it is the Sandbox Image every `workspace()` that names none falls back to.``
+- An image that `jr2 up` built, with a `USER` that is not a number or is `0`, faults the run before the pod is made.
+  `<image>` is the `image` value, or `default`:
+  ``the Sandbox Image "<image>" declares `USER <user>`, which cannot run a jr2 seat: every jr2-owned container is `runAsNonRoot` with no `runAsUser`, so the kubelet needs a NUMERIC non-zero uid it can check without reading the image (ADR-0005). Change the Dockerfile's last `USER` to that uid (e.g. `USER 1000`, or drop the line entirely and jr2 supplies uid 1000 with a writable HOME — ADR-0037), then re-run `jr2 up`.``
+- A registry image that runs as root faults the run. The text starts with
+  `Sandbox "<sandbox>" cannot start: its image runs as ROOT, and every jr2-owned seat is hardened with runAsNonRoot (ADR-0005).`
+- An image that does not have the floor stops the `preflight` step. Its log says:
+  `jr2: this Sandbox Image does not meet the floor (ADR-0037): a glibc base no older than jr2's node (musl is out entirely), git on the system PATH, a writable HOME for the image's USER, and /opt/jr2 + /work + :8080 unclaimed. jr2 vendors the rest.`
+  The run faults with a text that starts with
+  ``Sandbox "<sandbox>" never reached Ready (last phase: <phase>) — if its Sandbox Image is new, check the preflight: `kubectl logs <sandbox> -c preflight` ``.
+- The [Orchestrator](../reference/glossary.md#orchestrator) refuses a `file:` image that the last `jr2 up` did not
+  build. Run `jr2 up` again:
+  ``no Sandbox Image for <url> (context digest <digest>) — this instance's last converge built <n> image(s), none of them this one. Re-run `jr2 up`: it walks the registered Machines and builds every `file:` context they carry (ADR-0037/0049).``
+
+## Read next
+
+- [Workspaces](../concepts/workspaces.md): the pod, the Sandbox Image and the User Container.
+- [Converge](../concepts/converge.md): how `jr2 up` builds and loads the images.
+- [Look inside a Sandbox](look-inside-a-sandbox.md)
+- Reference: [`workspace`](../reference/authoring-surface/workspace.md),
+  [`SandboxOptions`](../reference/authoring-surface/type-sandboxoptions.md),
+  [`UserContainer`](../reference/authoring-surface/type-usercontainer.md), [`jr2 up`](../reference/cli.md#up)
+- [ADR-0037](../reference/decisions/0037-an-instance-builds-its-sandbox-images-jr2-injects-the-harness.md): the Instance
+  builds its Sandbox Images, and jr2 injects the Harness.
+- [ADR-0005](../reference/decisions/0005-sandbox-pod-composition.md): the containers of the Sandbox pod.
+- [ADR-0049](../reference/decisions/0049-a-machine-carries-its-parts-and-composes-by-invoke.md): a Machine carries its
+  images.