@orchestraworks/cli 0.5.0 → 0.5.2

package/skills/orchestra-fdk/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: orchestra-fdk
-description: Use when building and applying an Orchestra factory with the factory development kit for a developer's codebase - installing the kit, signing in, the bug-fix fast path (one agent, one bug, a pull request), writing the engineering-team starter, explaining its plan, the first apply without tokens, proving the machines locally first and applying them, the "Set up" item that runs the repository from a git bundle, the role grant, then connecting the code host and the tracker, and starting Study. Reads the orchestra skill for the platform's ideas and references/environment.md for the machines.
+description: Use when building and applying an Orchestra factory with the factory development kit for a developer's codebase - installing the kit, signing in, looking at the account to tell a new factory from a new environment for an existing one, the bug-fix fast path (one agent, one bug, a pull request), writing the engineering-team starter, explaining its plan, choosing how the agents reach a model, the first apply without tokens, proving the machines locally first and applying them, the "Set up" item that runs the repository from a git bundle, the steps the code cannot declare, connecting the code host and the tracker, and starting the Bootstrap. Reads the orchestra skill for the platform's ideas and references/environment.md for the machines.
 ---
 
 # Build and apply an Orchestra factory
@@ -14,6 +14,10 @@ The `orchestra` skill explains every idea named here, how the CLI works, and wha
 When this skill says "orchestra skill, section N", read that section of `orchestra/SKILL.md` in your skill
 folder. Use it to answer a case the steps do not cover, and never promise one of its "Cannot yet" lines.
 
+The CLI's own help is the full list of commands and flags: `orchestra --help` and
+`orchestra <noun> <verb> --help`. The kit's reference, `docs/REFERENCE.md`, ships in the package;
+`references/authoring.md` says how to find it. This skill names a flag only where a step needs it.
+
 Ask questions with your question tool: `AskUserQuestion` in Claude Code, its counterpart in Codex. Ask one
 thing at a time. Never ask the developer to paste a token into the chat.
 
@@ -21,12 +25,12 @@ thing at a time. Never ask the developer to paste a token into the chat.
 
 `orchestra fdk init` without `--template` writes the bug-fix starter: one agent that takes a bug as its
 first message and answers with a pull request and a report. Take this path when the developer wants one bug
-fixed in a GitHub repository. Do steps 1 and 2 below first. Then:
+fixed in a GitHub repository. Do steps 1 to 3 below first. Then:
 
 1. Make a new folder beside the codebase, `<codebase>/../orchestra-factory`, called `<factory>`. In it, run
    `npm init -y`, then `npm install @orchestraworks/cli`.
 2. Write the starter. `<owner>/<name>` is the GitHub repository. Add `--model-key anthropic` or
-   `--model-key openai` only when the organization brings its own model key.
+   `--model-key openai` only when the organization brings its own model key to the model gateway (step 6).
 
    ```bash
    npx orchestra fdk init --repository <owner>/<name>
@@ -40,12 +44,15 @@ fixed in a GitHub repository. Do steps 1 and 2 below first. Then:
    `ORCHESTRA_FDK_MODEL_GATEWAY_HOST`, `ORCHESTRA_FDK_IMAGE_PULL_ACCESS_KEY_ID` and
    `ORCHESTRA_FDK_IMAGE_PULL_SECRET_ACCESS_KEY`, plus `ORCHESTRA_FDK_ANTHROPIC_API_KEY` or
    `ORCHESTRA_FDK_OPENAI_API_KEY` with `--model-key`. Ask the developer to write them into a file only they
-   can read, as in step 9, and load it in each command that runs `orchestra`:
+   can read, as in step 6, and load it in each command that runs `orchestra`:
    `set -a; . ~/.config/orchestra/fdk-inputs.env; set +a`. Docker on this computer must also be signed in
    to the image registry, with the details in the same note.
 6. Run `npx orchestra fdk diff` and show the developer the change list. On their yes, run
-   `npx orchestra fdk apply --require-approval never`. It builds and pushes the image, then prints a GitHub
-   App install link and waits until the developer installs the app. Give them the link.
+   `npx orchestra fdk apply --require-approval never --install-timeout 0`. It builds and pushes the image,
+   prints a GitHub App install link, and stops without waiting. The link is shown once. Give it to the
+   developer, and ask them to say when the app is installed. Then run the same apply again: it finds the
+   installation and applies the rest. If the link was lost, add `--remint-install-links` to the apply. That
+   revokes the old link and prints a new one.
 7. Send the bug. The agent's id is `<factory name>-bugfix`, where the factory name is the one `init` printed:
 
    ```bash
@@ -75,21 +82,22 @@ Do them in order. Each step ends with a check. Do not go on until the check pass
 Before every apply, run the check in `references/environment.md` under "Before any cloud apply". It
 removes every local-only credential from the factory code.
 
+A *clean diff* is a `npx orchestra fdk diff` that creates, updates and deletes no agent, environment or
+pipeline. While a token or key is loaded, each secret it fills shows `value-write-pending` on every diff, even
+right after an apply. That is expected, and a clean diff may show it.
+
 The factory is applied three times. The first apply comes before any token or input, so the developer sees
 the agents and pipelines first. The second apply adds the machine setup, after you prove it on this computer.
 The third apply adds the code host and tracker tokens and the inputs.
 
-After the first apply, the steps follow the Onboarding order:
+After the first apply, the steps follow the order of the starter's README, "After apply":
 
-1. Set up the machines (step 6).
-2. Set up: make the repository run from a bundle (step 7).
-3. Give the Manager Agent its role (step 8).
-4. Connect the code (step 9).
-5. Connect the tracker (step 9).
-6. Study (step 11).
-7. Experiment, round 1 (step 11).
-8. Promote (later, the Manager Agent asks).
-9. Invite the team (later).
+1. Set up the machines (step 8).
+2. Grant the Manager Agent its role and create the score metrics (step 9).
+3. Set up: make the repository run from a bundle (step 10).
+4. Connect the code and the tracker (steps 11 and 12).
+5. The Bootstrap (step 13).
+6. Commissioning (later, when the Bootstrap passes).
 
 ### 1. Install the kit
 
@@ -97,14 +105,21 @@ The kit is the `orchestra` command and the library the factory code imports. It
 skill was linked, in one of two places. Keep it and its skills current with the command that matches:
 
 - Globally, with `npm install -g @orchestraworks/cli`. Update it with `npm update -g @orchestraworks/cli`.
+  Then `orchestra` runs in any folder.
 - In the factory folder, with `npm install @orchestraworks/cli` there. Update it with
   `npm update @orchestraworks/cli`, run in that folder. Run the command there as `npx orchestra`.
 
+Use one way to run the command in each place:
+
+- Where no factory folder with the kit exists yet (steps 2 to 4), run
+  `npx --package @orchestraworks/cli orchestra`. With a global install, plain `orchestra` works too.
+- In a factory folder after its `npm install`, run `npx orchestra`.
+
 1. Find the developer's codebase folder. Call it `<codebase>`.
-2. Run `orchestra --help`, or `npx orchestra --help` in the factory folder.
+2. Run `npx --package @orchestraworks/cli orchestra --help`.
 
-**Check:** `orchestra --help` prints the command list, and `orchestra fdk init --help` lists `--template`.
-Node 22 or later is needed.
+**Check:** the help prints the command list, and `fdk init --help` lists `--template`. Node 22 or later is
+needed.
 
 The kit's full authoring reference is `references/authoring.md` in this skill's folder. Read it when you need
 a building block this skill does not cover.
@@ -112,16 +127,17 @@ a building block this skill does not cover.
 ### 2. Sign in
 
 1. Tell the developer: "A browser window will open. Sign in to Orchestra there and confirm."
-2. Run, with a command timeout of at least five minutes:
+2. Run `npx --package @orchestraworks/cli orchestra auth login`, with a command timeout of at least five
+   minutes. The CLI opens the browser
+   and waits for the answer on a local port of this computer, so the browser must run on this same
+   computer. When the developer belongs to one organization, the login selects it. When they belong to
+   several, the login stops and asks you to name one. Ask the developer which, then run:
 
    ```bash
-   orchestra auth login --org <organization>
+   npx --package @orchestraworks/cli orchestra auth login --org <organization>
    ```
 
-   `<organization>` is the organization name from the hand-over note. The CLI opens the browser and waits
-   for the answer on a local port of this computer, so the browser must run on this same computer. It
-   signs in to `https://api.orchestraworks.ai` unless `ORCHESTRA_BASE_URL` or `--base-url` says otherwise.
-   Its first line names the address. This sign-in renews itself, so a long run stays signed in.
+   This sign-in renews itself, so a long run stays signed in. The address it signs in to is its first line.
 3. Use this browser sign-in. `orchestra auth login --no-browser` asks for a token at a hidden prompt
    instead, and a typed token cannot renew itself: it stops working when it expires. Use it only for a short
    run, with a token someone hands the developer. Users cannot create their own API keys in the console
@@ -130,11 +146,69 @@ a building block this skill does not cover.
 4. Read the two lines the login prints: `Signed in as <email>.` and
    `Logged in to organization <id> (<slug>).` Tell the developer both. Note the organization id.
 
-To check the sign-in later, see the orchestra skill, section 16 (How the CLI works).
-
-**Check:** the login printed the developer's email and the organization from the hand-over note.
-
-### 3. Write the factory
+**Check:** the login printed the developer's email and the organization the developer named.
+
+### 3. Look before you ask: a new factory or a new environment
+
+An organization usually holds one factory, or none yet. Read what it holds before you ask the developer
+anything. These commands only read. `--all-pages` follows every page, and prints the rows as one array:
+
+```bash
+npx --package @orchestraworks/cli orchestra agents list --all-pages --output json
+npx --package @orchestraworks/cli orchestra environments list --all-pages --output json
+npx --package @orchestraworks/cli orchestra pipelines list --all-pages --output json
+npx --package @orchestraworks/cli orchestra workflows list --all-pages --output json
+```
+
+A row that an apply made carries the label `sfdk.managedBy`, whose value is the factory's name.
+
+1. Infer what the developer most likely wants:
+   - *No agents and no pipelines:* the account is empty. Infer **a new factory**.
+   - *Rows labeled with a factory name:* a factory exists. Infer **a new environment for that factory**,
+     such as another repository, a bigger machine, or a machine of the team's own.
+   - *Rows with no label:* someone made them by hand or with the CLI. Infer a new factory. If a row's slug
+     is one the starter declares, such as `manager-agent`, the apply updates that row: say so before step 7.
+2. Tell the developer what you found, in one or two sentences, for example: "This organization already has
+   the factory `factory`, with the agents Manager Agent, Engineer Agent lg, md, sm and bare."
+3. Ask once, with your question tool, with the inferred choice first and marked as the recommendation.
+   Then follow the branch below for the answer.
+
+**A new factory.** Go on with step 4, or with the bug-fix fast path.
+
+The starters build one factory per organization. When the organization already holds a factory and the
+developer wants another one, make a separate organization for it:
+
+1. Create it: `npx --package @orchestraworks/cli orchestra organizations create --name "<name>"`. Read
+   `organizations create --help` for its other flags. When the command refuses, the developer creates the
+   organization in the console at `https://app.orchestraworks.ai`.
+2. Sign in to it as step 2 says, with `--org <the new organization>`.
+3. Start again at step 3, in the new organization.
+
+**A new environment for the existing factory.** Do not run `orchestra fdk init`. Edit the factory that
+exists instead:
+
+1. Find the existing factory's folder: the folder with `fdk.json` and `main.ts`. Ask the developer where it
+   is when it is not on this computer.
+2. Load the factory's inputs, the same way every earlier apply loaded them. The engineering-team starter
+   reads `~/.config/orchestra/tokens.env`; the bug-fix starter reads `~/.config/orchestra/fdk-inputs.env`.
+   Load the file with `set -a; . <file>; set +a`, and never print it. Ask the developer when the file is
+   not there. An apply without these inputs removes the token rules the factory has now.
+3. With the inputs loaded, run `npm install` and `npx orchestra fdk diff` in that folder. It must be a clean
+   diff before you edit anything. If it is not, show the changes to the developer and ask what to keep.
+4. Add the environment, and the agent that works on it:
+   - Engineering-team starter: call `declareEnvironment` in `main.ts`, as the existing agents do.
+   - Bug-fix starter: add a `new Environment(factory, "<name>", …)` beside `runtime` in `main.ts`, and an
+     `Agent` on it.
+5. Prove the new environment with `references/environment.md`, sections 1 to 6, then come back here. Prove
+   the environment and the agent you added, not the existing ones.
+6. Apply as step 12 says, with the inputs loaded: diff, explain, approve, apply. The diff after it is clean.
+7. Give the new agent its first work, and wait for it. Engineering-team starter: create an item in the
+   right pipeline and assign it to the new agent, as step 10 does, then wait as step 10 says. Bug-fix
+   starter: start a session with the new agent and wait, as the fast path's points 7 to 9 say.
+
+**Check:** the developer confirmed one of the two choices.
+
+### 4. Write the factory
 
 Ask one question, with these three choices:
 
@@ -144,24 +218,30 @@ Ask one question, with these three choices:
 
 What each answer does:
 
-- *Engineering team:* go on below.
+- *Engineering team:* go on below. This starter connects Bitbucket and Jira only. For a GitHub repository,
+  take the bug-fix fast path instead.
 - *Todo app sample:* write it as below, with `--template todo-factory` in point 2. It works on the public
-  `orchestraworks/todo-app`. Then go on with step 4.
+  `orchestraworks/todo-app`. Then go on with step 5.
 - *I already have a factory.ts:* copy their folder into `<factory>` (below). It needs `fdk.json` and a
-  `"type": "module"` package. Install the kit in it as in point 3, then go on with step 4, using their
+  `"type": "module"` package. Install the kit in it as in point 3, then go on with step 5, using their
   own inputs.
 
 For the engineering team:
 
 1. Make a new, empty folder beside the codebase: `<codebase>/../orchestra-factory`. Call it `<factory>`.
+   Do not install the kit in it before point 2.
 2. In `<factory>`, write the starter:
 
    ```bash
-   orchestra fdk init --template engineering-team
+   npx --package @orchestraworks/cli orchestra fdk init --template engineering-team
    ```
 
+   Run it in the new, empty folder, with no kit installed there: the starter brings its own
+   `package.json`.
+
    It writes `README.md`, `main.ts`, `inputs.json`, `package.json`, `fdk.json`, `tsconfig.json`,
-   `agents/` and `lib/`. It overwrites nothing. If a file is in the way, it writes nothing and names it.
+   `agents/`, `lib/` and `onboarding/`. It overwrites nothing. If a file is in the way, it writes nothing
+   and names it.
 3. In `<factory>`, install what the starter's `package.json` names, the kit among them:
 
    ```bash
@@ -170,11 +250,11 @@ For the engineering team:
 
    From now on, run `npx orchestra` in `<factory>`.
 4. Read `<factory>/README.md` whole. It says what the starter declares, and why.
-5. Leave `inputs.json` empty for now. Step 9 fills it in.
+5. Leave `inputs.json` empty for now. Step 11 fills it in.
 
 **Check:** `npm run typecheck` passes in `<factory>`.
 
-### 4. Read the plan and explain it
+### 5. Read the plan and explain it
 
 Before anything reaches the platform, show the developer what the factory is. Explain it from what the
 code produced, not from memory.
@@ -189,34 +269,28 @@ code produced, not from memory.
 
 **What each kind of thing is, and why it is there.**
 
-- *Agents* do the work: orchestra skill, section 2. There are four. The Manager Agent sets up the factory and
-  grades the others. The three Engineer Agents (lg, md and sm) fix bugs and build features. Say which AI model
+- *Agents* do the work: orchestra skill, section 2. There are five. The Manager Agent sets up the factory
+  and grades the others. The three Engineer Agents (lg, md and sm) fix bugs and build features. Engineer
+  Agent bare takes only the Bootstrap's baseline replays, to show what the study adds. Say which AI model
   each one uses.
-- *Environments* are the machines the agents work on: sections 4 and 5. Say each one's size and its network
-  rules.
+- *Environments* are the machines the agents work on: sections 4 and 5. There are four: Engineer Agent bare
+  uses the same one as Engineer Agent lg. Say each one's size and its network rules.
 - *Pipelines* are where work items live: sections 9 and 10.
 - *The warning* says the repository and the tracker are not connected yet. Part four explains it.
 
-**The gateway key.** It is the one key in the first apply. Explain it from the orchestra skill, section 7, in
-two parts:
+**The gateway key.** The starter's code as written sends every agent's model calls through the model
+gateway, with one gateway key: orchestra skill, section 7. The key is a key to `models.orchestraworks.ai`,
+stored as a secret. Our network edge adds it to each model call, so no agent and no person sees it.
 
-- *What it is for.* The model gateway is a hosted service that lets the organization manage model spending in
-  one place. Every agent's model calls go through it, and the organization sees what each key spends and can
-  cap it. Say where they see and set this today.
-- *What it is.* A key to `models.orchestraworks.ai`, stored as a secret. Our network edge adds it to each model
-  call, so no agent and no person sees it. The gateway swaps it for the provider's own key and charges usage to
-  the organization's credit at list price plus 10%. Without it, an agent's first message fails with "Invalid
-  API key".
-
-If they want to bring their own provider keys instead, the same section says what that costs.
+Say that this is one of three options. Step 6 asks which one the developer has.
 
 **The pipelines and their stages.** Show this table, built from the `pipeline` entries in the plan: the
 name, the item prefix and the stages, in order.
 
 | Pipeline | Item prefix | Stages | What it is for |
 |---|---|---|---|
-| Onboarding | `onbd` | Todo, In progress, Needs review, Done | Setting up the factory: Set up, Study, the Experiment rounds, Promote and every Replay item. It goes away after the Engineer Agents start real work. |
-| Bugs | `bug` | Triage, Parked, In progress, In review, Done, Won't fix | Bugs. Parked holds a kind of bug the Engineer Agents have not yet proven they can fix. |
+| Onboarding | `onbd` | Todo, In progress, Needs review, Done | Setting up the factory: Set up, the Bootstrap and every Replay item. It goes away at commissioning. |
+| Bugs | `bug` | Triage, Parked, In progress, In review, Done, Won't fix | Bugs. Parked holds a bug that waits until the Bootstrap passes, or one a person set aside. |
 | Features | `feat` | Backlog, Todo, In progress, In review, Done | New features. |
 | Support | `supp` | Triage, Answering, Waiting on requester, Resolved, Closed | Questions from the team's users. |
 
@@ -232,39 +306,122 @@ connects them and the warning goes away.
 
 **Check:** the developer has no more questions about the plan.
 
-### 5. The first apply (no tokens)
+### 6. Choose how the agents reach a model
+
+Each agent needs a way to reach its AI model. There are three options, and the developer uses the one that
+matches what they already have. Ask which they have, with your question tool, or infer it from what is set
+up, such as an existing factory's model Secret or `ModelProviderKey`. Then follow that option. For plan
+questions, send the developer to **Settings › Plan & usage** in the console.
+
+**A Claude or Codex subscription.** The developer signs in a real terminal on their own computer. Run from
+your shell, its browser step hangs. You never read or copy the value.
+
+- *Claude.* `claude setup-token` prints a long-lived token. The developer saves it in the private file
+  below, as `ORCHESTRA_FDK_CLAUDE_CODE_OAUTH_TOKEN`. The factory stores it as a Secret, and the network edge
+  adds it to calls to `api.anthropic.com`. The machine holds only a placeholder token and an empty API key.
+  Declare the Claude sign-in recipe from `references/authoring.md`, "Secrets are recipes, never values".
+  Never put the token in the environment's settings: they are stored and shown in plain text.
+- *Codex.* A personal access token, which begins `at-`. A ChatGPT Business or Enterprise member makes it
+  at `chatgpt.com/admin/access-tokens`, when the workspace owner allows it. Or a separate ChatGPT sign-in:
+  `CODEX_HOME=<dir> codex login`, never a copy of the developer's own `~/.codex` sign-in. A separate
+  sign-in must live on a `/home` volume that every machine using it shares, because each renewal
+  replaces the sign-in. After the copy, a new `codex login` in `~/.codex` signs the cloud copy out.
+  Keep a Codex sign-in off the model gateway: the gateway refuses its requests with `model_not_found`.
+  Not released yet: the worker settings that put a Codex sign-in on a machine. Until they are, use the
+  model gateway or an OpenAI API key for a `codex` agent.
+
+An agent on a subscription turns the model gateway off: remove `enableModelGateway: true` from its model
+settings.
+
+**An Orchestra account, through the model gateway.** One endpoint reaches OpenAI, Anthropic and open
+models. The organization can also route models: `orchestra model-configs` sets which providers serve a
+model and in which order, so a call falls back to the next provider, and `orchestra model-aliases` gives a
+model the organization's own name. The organization can add its own provider key to the gateway as a
+`ModelProviderKey` (orchestra skill, section 7). The starter's code as written uses this option, so it
+needs no edit.
+
+**Their own provider API key.** The key is a Secret that the network edge adds to calls straight to the
+provider, with no gateway in between. For Anthropic, add `x-api-key: ${secret}` on `api.anthropic.com`;
+for OpenAI, add `authorization: Bearer ${secret}` on `api.openai.com`. Read the key from a shell variable
+such as `ORCHESTRA_FDK_ANTHROPIC_API_KEY`, and turn `enableModelGateway` off for the agent.
+`references/authoring.md` shows the Secret and the rule.
+
+For a subscription or an own key, the apply reads the token or key from the shell, so it must be saved
+before step 7:
+
+1. Ask the developer to make a file only they can read, outside the codebase:
+
+   ```bash
+   mkdir -p ~/.config/orchestra && touch ~/.config/orchestra/tokens.env && chmod 600 ~/.config/orchestra/tokens.env
+   ```
+
+2. Ask them to add one line to it, filled in by them, such as `ORCHESTRA_FDK_CLAUDE_CODE_OAUTH_TOKEN=...`.
+   Step 11 adds the Bitbucket and Jira tokens to the same file later.
+3. Load the file into each apply's shell without printing it: `set -a; . ~/.config/orchestra/tokens.env;
+   set +a`.
+
+Then edit `<factory>/main.ts` and `<factory>/lib/environment.ts`, and run `npm run typecheck` and
+`npx orchestra fdk synth`.
+
+**Check:** the developer chose an option, and the plan shows it: a gateway rule for the model gateway, or a
+Secret and a rule on the provider's host for a subscription or an own key. For those two, the file exists
+with mode 600 (`stat`). Do not read its contents.
+
+### 7. The first apply (no code host or tracker tokens)
 
 `orchestra fdk apply` makes the platform match the code. The developer approves the change list first.
 
-1. Use a shell that has not loaded the tokens. If `ORCHESTRA_FDK_BITBUCKET_TOKEN` or `ORCHESTRA_FDK_JIRA_TOKEN` is set,
+1. For a subscription or an own key (step 6), load the token or key from the private file. Use a shell without the
+   Bitbucket and Jira tokens: if `ORCHESTRA_FDK_BITBUCKET_TOKEN` or `ORCHESTRA_FDK_JIRA_TOKEN` is set,
    unset it for this step.
 2. In `<factory>`, run `npx orchestra fdk diff`. It changes nothing.
-3. Show the developer the change list in plain words: the four agents, their four machines, the gateway
-   key and its secret, and the four pipelines.
+3. Show the developer the change list in plain words: the five agents, their four machines, the keys and
+   their secrets, and the four pipelines.
 4. Ask them to approve it, with your question tool.
-5. On a yes, run `npx orchestra fdk apply --require-approval never` in `<factory>`. The flag is needed because your
-   shell has no terminal to type "y" at. It is safe only after the developer approved the same change
-   list.
+5. On a yes, run `npx orchestra fdk apply --require-approval never` in `<factory>`. The flag is needed
+   because your shell has no terminal to type "y" at. It is safe only after the developer approved the
+   same change list.
 
-**Check:** a second `npx orchestra fdk diff` shows no change, and `<factory>/fdk.out/result.json` exists.
+An organization on the `individual` plan cannot store a key or add one at the network edge, so this apply
+stops with a plan error. Tell the developer, and point them to **Settings › Plan & usage** in the console.
 
-### 6. Set up the machines
+**Check:** a second `npx orchestra fdk diff` is a clean diff, and `<factory>/fdk.out/result.json` exists.
+
+### 8. Set up the machines
 
 Read and follow `references/environment.md`, in this skill's folder. It finds how the codebase is installed,
 tested and started. It proves that on this computer first, in the local environment, and the cloud is the
 fallback. It writes what worked into `<factory>/lib/environment.ts` and `<factory>/lib/setup.sh`.
 
-Then apply the machine setup. No token is loaded yet.
+Then apply the machine setup, in a shell loaded as in step 7: the model token or key of step 6 if there
+is one, and no Bitbucket or Jira token yet.
 
 1. In `<factory>`, run `npx orchestra fdk diff`.
-2. Explain what changed, in plain words: the setup script, the settings and the ports. Say that each new
-   machine gets them, and that running machines keep their old setup.
+2. Explain what changed, in plain words: the setup script, the settings and the ports.
+   Say that each new machine gets them, and that running machines keep their old setup.
 3. Ask the developer to approve it, with your question tool.
 4. On a yes, run `npx orchestra fdk apply --require-approval never` in `<factory>`.
 
-**Check:** the checks in `references/environment.md` pass, and a second `npx orchestra fdk diff` shows no change.
+**Check:** the checks in `references/environment.md` pass, and a second `npx orchestra fdk diff` is a clean
+diff.
+
+### 9. The steps the code cannot declare
+
+Do these before Set up starts, in the README's order: the first two steps of "After apply: steps the code
+cannot declare" in `<factory>/README.md`. It holds the exact commands.
+
+1. **Grant the Manager Agent the `agent-manager` role**, for the whole organization. The Manager Agent
+   works on items in every pipeline, reads the Engineer Agents' sessions, and changes the Engineer Agents.
+   An agent starts with only `agent-worker`. The role list you send replaces the old one, so keep every
+   entry it has now. Ask the developer to approve: "Give the Manager Agent the agent-manager role in the
+   whole organization?" On a yes, send the update with `--require-approval never`.
+2. **Create the score metrics**: `bash onboarding/create-score-metrics.sh`, once, in `<factory>`. The
+   Manager Agent writes each replay's scores into them.
+
+**Check:** `npx orchestra organization-memberships get <membership id>` lists both roles, and
+`npx orchestra metrics list --output json` lists the five `replay-` metrics.
 
-### 7. Set up: run the repository from a bundle
+### 10. Set up: run the repository from a bundle
 
 The Manager Agent has no access to the code host yet. So it gets the code as a *git bundle*: one file that
 holds the repository's history. This item is also the cloud's confirmation of what you proved on this
@@ -273,18 +430,18 @@ computer.
 1. Find the default branch: `git -C <codebase> symbolic-ref refs/remotes/origin/HEAD`.
 2. Cut the bundle outside the codebase:
    `git -C <codebase> bundle create <codebase>/../orchestra-setup.bundle <default branch>`.
-3. Upload it: `npx orchestra files create --file <codebase>/../orchestra-setup.bundle --output json`. Note the
-   file id.
+3. Upload it: `npx orchestra files create --file <codebase>/../orchestra-setup.bundle --output json`. Note
+   the file id.
 4. The kit cannot declare items. Create them in the Onboarding pipeline, in this order:
 
    ```bash
    npx orchestra work-items create --pipeline onboarding --title "Set up" --description "<what done means>" --output json
-   npx orchestra work-items create --pipeline onboarding --title "Study" --output json
-   npx orchestra work-items create --pipeline onboarding --title "Experiment, round 1" --output json
+   npx orchestra work-items create --pipeline onboarding --title "Bootstrap" --description "<the threshold>" --output json
    ```
 
    The Set up description says:
-   - Get the bundle with `orchestra files get-content <file id>`, and clone the code from it.
+   - Get the bundle with `orchestra files get-content <file id> --output-file setup.bundle`, and clone the
+     code from it under `/work`.
    - Install with the internet. Then run the stack with local settings only, such as the local Supabase
      stack's. Write those settings files, such as `.env.local`, from what `references/environment.md`
      found.
@@ -298,39 +455,24 @@ computer.
    - Done means the stack is up and validation passes. The tests do not decide Done.
    - Prove that an Engineer Agent can run the tests from the same bundle.
 
-   Leave "Study" and "Experiment, round 1" unassigned in Todo.
-5. Start Set up: `npx orchestra work-items assign <set-up item id> --agent-id manager-agent`. Assigning starts
-   the Manager Agent's conversation on the item.
+   The Bootstrap description holds the two values of "The Bootstrap threshold" in `<factory>/README.md`.
+   Leave the Bootstrap unassigned in Todo.
+5. Start Set up: `npx orchestra work-items assign <set-up item id> --agent-id manager-agent`. Assigning
+   starts the Manager Agent's conversation on the item.
 6. Watch it: `npx orchestra work-items get <id> --output json` lists the item's `sessions`. Run
-   `npx orchestra sessions wait <session id> --timeout 3600`. The developer can watch the same session in the console at
+   `npx orchestra sessions wait <session id> --timeout 540` in the foreground, and run it again when it
+   times out, as the fast path says. The developer can watch the same session in the console at
    `https://app.orchestraworks.ai`.
 7. When it ends, read the agent's report with the developer. Fix a failed step in the environment: go back
-   to `references/environment.md`, prove the fix on this computer, apply it as in step 6, then assign the
-   item again to wake the agent.
+   to `references/environment.md`, prove the fix on this computer, apply it as in step 8, then assign the
+   item again to wake the agent. Note the time before you assign. The session can still show the last
+   turn's `COMPLETE`, and then `sessions wait` returns at once. So run
+   `npx orchestra sessions get <session id> --output json` first, and wait only once its `lastActiveAt` is
+   later than that time.
 
 **Check:** the report shows the stack up, validation passing at a machine endpoint, and the test result.
 
-### 8. Give the Manager Agent its role
-
-The Manager Agent works on items in every pipeline, reads the Engineer Agents' sessions, and changes the
-Engineer Agents. An agent starts with only `agent-worker`. The `agent-manager` role allows the rest. The
-kit cannot grant a role to an agent, so do it with the CLI:
-
-1. Find the organization id: `npx orchestra organizations list --output json`.
-2. Find the Manager Agent's membership:
-   `npx orchestra organization-memberships list --organization-id <org id> --q manager-agent --output json`.
-   Note its `id` and its `roleAssignments`.
-3. Find the role id: `npx orchestra roles list --limit 100 --output json`. Note the `id` of the role whose
-   `name` is `agent-manager`.
-4. Write `grant.json` in `<factory>`: `{"roleAssignments": [...]}` holding every assignment from point 2
-   unchanged, plus `{"roleId": "<agent-manager role id>"}` with no `projectId`. The list replaces the old
-   one, so keep the old entries, `agent-worker` included.
-5. Ask the developer to approve: "Give the Manager Agent the agent-manager role in the whole organization?"
-6. On a yes: `npx orchestra organization-memberships update <membership id> --body @grant.json --require-approval never`.
-
-**Check:** `npx orchestra organization-memberships get <membership id>` lists both roles.
-
-### 9. Connect the code and the tracker
+### 11. Connect the code and the tracker
 
 The agents reach Bitbucket and Jira with two personal API tokens. The platform stores each one as a
 *Secret*, a value the agents never see: the network edge adds it to their requests. You never see the
@@ -339,7 +481,7 @@ values either. Connect the code first, then the tracker.
 1. Fill in `<factory>/inputs.json`. Read the values from `<codebase>` where you can, and ask for the rest:
    - `repositoryUrl`: the repository's HTTPS address, from `git -C <codebase> remote get-url origin`.
      Write it as `https://bitbucket.org/<workspace>/<repository>`.
-   - `repositoryDefaultBranch`: the default branch from step 7.
+   - `repositoryDefaultBranch`: the default branch from step 10.
    - `jiraSite`: the Jira site host, such as `acme.atlassian.net`. Ask.
    - `jiraEmail`: the Atlassian account email the Jira token belongs to. Ask.
 2. Ask the developer to create two tokens:
@@ -347,13 +489,14 @@ values either. Connect the code first, then the tracker.
      with the Bitbucket scopes to read and write repositories and pull requests.
    - The tracker: a Jira API token, at the same page, without scopes. A token with scopes works only
      through `api.atlassian.com`, which this starter does not add the token to.
-3. Ask the developer to write both into a file only they can read, outside the codebase:
+3. Ask the developer to add both to the private file of step 6. If it does not exist yet, they make it
+   first:
 
    ```bash
    mkdir -p ~/.config/orchestra && touch ~/.config/orchestra/tokens.env && chmod 600 ~/.config/orchestra/tokens.env
    ```
 
-   with these two lines, filled in by them:
+   Then they add these two lines, filled in by them:
 
    ```
    ORCHESTRA_FDK_BITBUCKET_TOKEN=...
@@ -363,28 +506,30 @@ values either. Connect the code first, then the tracker.
 4. Load the file into a command's shell without printing it:
    `set -a; . ~/.config/orchestra/tokens.env; set +a`.
 
-You may use the same tokens on this computer to read past pull requests and issues, for example for the
-Study item. Load the file the same way and send `Authorization: Bearer $ORCHESTRA_FDK_BITBUCKET_TOKEN` to
-`https://api.bitbucket.org/2.0/`. For Jira, use Basic auth with `jiraEmail` and `$ORCHESTRA_FDK_JIRA_TOKEN` on
+You may use the same tokens on this computer to read past pull requests and issues. Load the file the same
+way and send `Authorization: Bearer $ORCHESTRA_FDK_BITBUCKET_TOKEN` to `https://api.bitbucket.org/2.0/`.
+For Jira, use Basic auth with `jiraEmail` and `$ORCHESTRA_FDK_JIRA_TOKEN` on
 `https://<jiraSite>/rest/api/3/`. Never print a token, and never write one into a file in `<codebase>` or
 `<factory>`.
 
+Other code hosts and trackers connect through a *connection*: orchestra skill, section 17.
+
 **Check:** the file exists and has mode 600 (`stat`). Do not read its contents. With the tokens loaded,
 `npx orchestra fdk synth` writes an empty `fdk.out/warnings.json`.
 
-### 10. The token apply
+### 12. The token apply
 
 1. In `<factory>`, with the tokens loaded, run `npx orchestra fdk diff`.
-2. Explain only what changed since step 6, in plain words:
+2. Explain only what changed since step 8, in plain words:
    - *The token secrets.* The Bitbucket token (twice: once for the API, once for git) and the Jira token
      are stored as secrets. No one sees their values.
    - *The Manager Agent's machine.* It gets rules that add the Bitbucket token on `api.bitbucket.org` and
      `bitbucket.org`, and the Jira token on the Jira site. So it can clone, open pull requests and read
      issues.
    - *The Engineer Agents' machines.* They get deny rules for Bitbucket and for the Jira site. Say why:
-     the Engineer Agents first redo past issues, called replays, to show how well they work. A replay
-     must not see the real fix. So they cannot reach Bitbucket or Jira until commissioning, when they
-     start real work. They get the code as a file the Manager Agent uploads.
+     in the Bootstrap, the Engineer Agents redo past pull requests, called replays, to show how well they
+     work. A replay must not see the real fix. So they cannot reach Bitbucket or Jira until commissioning,
+     when they start real work. They get the code as a file the Manager Agent uploads.
    - The Manager Agent's instructions now name the repository and the tracker.
 3. Ask the developer to approve it, with your question tool.
 4. On a yes, run `npx orchestra fdk apply --require-approval never` in `<factory>`, with the tokens loaded.
@@ -392,21 +537,25 @@ Study item. Load the file the same way and send `Authorization: Bearer $ORCHESTR
 Run every later apply with the tokens loaded. An apply without them takes the token rules off the
 Manager Agent's machine again.
 
-**Check:** a second `npx orchestra fdk diff`, with the tokens loaded, shows no change.
+**Check:** a second `npx orchestra fdk diff`, with the tokens loaded, is a clean diff.
+
+### 13. The Bootstrap
 
-### 11. Study, then Experiment
+The Bootstrap teaches the Engineer Agents this codebase, then tests them on past pull requests they have
+not seen. `<factory>/README.md`, "Bootstrap", says each step and the threshold.
 
-1. Assign Study: `npx orchestra work-items assign <study item id> --agent-id manager-agent`. Watch it as in
-   step 7.
-2. Once Study is in Done, assign "Experiment, round 1" to `manager-agent` the same way.
+1. Once Set up is in Done, assign the Bootstrap:
+   `npx orchestra work-items assign <bootstrap item id> --agent-id manager-agent`. Watch it as in step 10.
+2. The Manager Agent creates the Replay items and scores them. It moves the Bootstrap to Done on a pass,
+   or to Needs review after two rounds that do not pass. Read its report with the developer.
 
-Promote and Invite the team come after the experiments. The Manager Agent asks for them in Needs review.
+Commissioning comes after a pass: `<factory>/README.md`, "Commissioning".
 
-**Check:** Study is in Done, and "Experiment, round 1" has a session.
+**Check:** the Bootstrap has a session, and the Onboarding pipeline lists its Replay items.
 
 ## Beyond the steps
 
-- What the platform can and cannot do yet: the orchestra skill, sections 1 to 18. Read the section for the idea
-  in question.
+- What the platform can and cannot do yet: the orchestra skill, sections 1 to 19. Read the section for the
+  idea in question.
 - Where an agent's knowledge goes (instructions, platform skills, codebase skills, the customer's own skills,
   versions): the orchestra skill, section 19.