@flow-engineer/flow 0.3.1 → 0.3.3

package/README.md

@@ -408,13 +408,18 @@ Each run's line carries its id (`FAIL  latte-correction #1 (whatsapp) trn_…: 6
   "runs": [
     { "runId": "trn_…", "scenario": "latte-correction", "channel": "whatsapp", "run": 1, "verdict": "failed",
       "error": null, "ended": "goal_met", "turns": 6, "watch": "https://…",
-      "failedChecks": [{ "kind": "judge", "text": "The agent asks Sam to confirm before placing the order.", "reason": "…" }] }
-  ]
+      "failedChecks": [{ "kind": "judge", "text": "The agent asks Sam to confirm before placing the order.", "reason": "…" }],
+      "invalid": null }
+  ],
+  "invalid": [],
+  "sessionInvalid": null
 }
 ```
 
 `outcome` is `passed` (exit 0), `failed` or `invalid` (1), `errored` or `transport` (2), `refused` (3) or `interrupted` (130). A run's `verdict` is `errored` whenever the cloud names an `error` for it (such as `judge_failed`, `simulator_failed` or `spend_cap`), whatever it `ended` as: the cloud could not play or judge it, so it says nothing about the agent. Its line reads `ERROR` instead of `PASS` or `FAIL`, and the run's `error` is that code (else `null`). `error` is `{ "message", "status", "type", "retryAfterSeconds" }` when the session did not finish (`type` is the cloud's code, such as `spend_cap`; the others are `null` when it did not say).
 
+A scenario the cloud cannot play (a key it does not know, a value it cannot read) does not stop the others: its runs are `errored` with `error` `scenario_invalid` and `invalid` `{ "path", "reason" }` (the reason starts `unsupported:` when the cloud does not know a key), its line reads `INVALID` with the file and what is wrong, and the top-level `invalid` lists each such scenario once as `{ "scenario", "path", "reason" }`. When the cloud ends the whole session `scenario_invalid` (a tools or checks file it cannot read, or an older cloud), `sessionInvalid` is `{ "reason", "olderCloud", "sent" }`: what the cloud said after the code (else `null`), whether that suggests a cloud older than this client, and the scenarios sent as `{ "scenario", "path" }`; the same lines are printed.
+
 `--only latte-correction,order-fails` runs only those scenarios (file names without `.yaml`); every tool file still goes. Once the session has opened, every exit (0, 1, 2, 3 or 130) writes `.flow/last-session.json` in the agent's folder and prints its path; `flow new` git-ignores `.flow/`. It holds what each run ended as and the judge's reasons, never the messages (a reason can quote one, which is why the file stays on your machine; the suite is synthetic by `--synthetic`):
 
 ```json
@@ -429,6 +434,8 @@ Each run's line carries its id (`FAIL  latte-correction #1 (whatsapp) trn_…: 6
 }
 ```
 
+A run of a scenario the cloud could not play also has `invalid` (`{ "path", "reason" }`), and a session the cloud ended `scenario_invalid` has `sessionInvalid`, as in `--json`.
+
 `ended` is `finished`, `stopped` (the session broke off or the cloud stopped it) or `interrupted`. `agent` is the cloud's id for the agent (a hosted cloud names it), which `flow show` reads. `only` is the scenarios the session was limited to, `null` for the whole suite. `verdict` is `passed`, `failed`, `errored` (the cloud could not play or judge the run) or `not_played` (the session ended first; its `runId` is `null`).
 
 `flow show <run-id>` prints one run's record from the cloud, read with `FLOW_FRAMEWORK_CLOUD_KEY`: the verdict, every check with the judge's reason, then the conversation in time order with each tool call the simulator played (its input, output and what it changed). `flow show last` picks the first run in `.flow/last-session.json` that did not pass (its first run when all passed) and names the others. `--json` prints the record exactly as the cloud answered it. It exits 0 when it printed; 1 for a wrong flag, no key or last session, no such run (404) or one past the 30 days runs are kept (410); 2 when the cloud did not answer or answered 5xx; 3 when it refused the key (401, 403) or is busy (429).

package/coding-agents/skill.md

@@ -352,6 +352,8 @@ run again. `--channel telegram|whatsapp|imessage` picks the channel; a channel f
 open yet exits 3 with its `next`. When only observe's part of setup fails (traces and the test cloud),
 `link` still prints the link and exits 0, with one warning saying what is missing and the server's
 fix (`warnings` in `--json`): pass it on to the owner. `login`, `setup` and `link` also run on their own.
+When the owner names a team's observe workspace, pass `--workspace <name|acc_id>` to `setup` or `link`
+once: its `acc_` id is kept in `.env` (`FLOW_FRAMEWORK_WORKSPACE`) for later runs, and `status` shows it.
 
 In `src/main.ts`, put `fm.telegram()` from `@flow-engineer/flow/flow-messaging` in the agent's
 channels and call `fm.start(agent, { cursorFile: ".flow/flow-messaging-cursor" })`: with no webhook

package/coding-agents/test-skill.md

@@ -183,13 +183,18 @@ connects out to the cloud, which plays the person and the tools and judges.
    `runId`, `scenario`, `channel`, `run`, `verdict`, `error` (the cloud's code when it could not play
    or judge the run, such as `judge_failed`; the run is then `errored` and its line reads `ERROR`,
    whatever it ended as), `ended`, `turns`, `watch` and `failedChecks` (`kind`, `text` and the judge's
-   `reason`). Each session that opened also leaves
+   `reason`), and `invalid` (`{ path, reason }` for a scenario the cloud could not play, its error
+   `scenario_invalid` and its line `INVALID`; else `null`); top-level `invalid` lists those scenarios
+   once each (`scenario`, `path`, `reason`: fix the file; `unsupported: ...` means the cloud does not
+   know that key), and `sessionInvalid` (`reason`, `olderCloud`, `sent`) is set when the cloud ended
+   the whole session `scenario_invalid`. Each session that opened also leaves
    `.flow/last-session.json` in the agent's folder (its path is printed on every exit): `sessionId`,
    `agent`, `dashboard`, `suiteHash`, `endedAt`, `ended` (`finished`, `stopped` or `interrupted`),
    `only` (the scenarios it was limited to, else `null`) and per run `scenario` (file name without
    `.yaml`), `channel`, `run`, `runId`, `verdict` (`passed`, `failed`, `errored`: the cloud could not
    play or judge it, such as a failed tool simulator or person model, which is not a failure of your
-   agent, or `not_played`: the session ended first) and `failedChecks` (with reasons).
+   agent, or `not_played`: the session ended first), `failedChecks` (with reasons) and `invalid`
+   when the cloud could not play the scenario, plus `sessionInvalid` as in `--json`.
    Read these instead of parsing the printed lines.
    Exit status: 0 all passed; 1 a run failed, or nothing was sent (a wrong flag, a test file that does
    not parse, no key); 2 the connection or the cloud failed, or runs errored and none failed; 3 the

package/dist/testing/src/account.d.ts

@@ -1,4 +1,4 @@
 export declare const ACCOUNT_COMMANDS: Set<string>;
-export declare const ACCOUNT_LINES = "  flow login [--wait | --no-wait] [--timeout <s>] [--force] [--no-browser] [--json]\n                            Sign in to flow.engineer: opens the browser, prints the link and code,\n                            and waits up to --timeout (default 120) for approval; --wait goes on\n                            waiting for the same code, --no-wait returns at once\n  flow logout [--json]      End this machine's sign-in and delete its token\n  flow whoami [--json]      Show the account you are signed in as\n  flow setup [--replace] [--rotate <keys> [--now]] [--display-name <name>] [--ci] [--json]\n                            Write this agent's keys (its Flow key FLOW_KEY, which also pays for the\n                            model on the account's trial credit when there is no provider key, test\n                            cloud, traces, Flow Messaging) and each open channel's sender into\n                            ./.env (in an agent's folder: package.json); --rotate key,cloud,traces,\n                            messaging or all makes those keys again (the old FLOW_KEY works 24 h\n                            more; --now ends it at once)\n  flow link [--channel telegram|whatsapp|imessage] [--display-name <name>] [--timeout <s>] [--no-browser] [--json]\n                            Print the link that opens a chat with this agent from a phone; signs in\n                            and runs setup first when this folder is not set up yet. --display-name\n                            (here and on setup) renames the agent: package.json's displayName and\n                            its prompt, agent/AGENTS.md; the phone link then says \"Hi <name>\"\n  flow status [--json]      What this folder has and lacks: FLOW_KEY and the model key (never their\n                            values) with the trial credit left, the\n                            sign-in, each channel's sender in .env, and, signed in, which channels\n                            are open now and their free-trial counts";
+export declare const ACCOUNT_LINES = "  flow login [--wait | --no-wait] [--timeout <s>] [--force] [--no-browser] [--json]\n                            Sign in to flow.engineer: opens the browser, prints the link and code,\n                            and waits up to --timeout (default 120) for approval; --wait goes on\n                            waiting for the same code, --no-wait returns at once\n  flow logout [--json]      End this machine's sign-in and delete its token\n  flow whoami [--json]      Show the account you are signed in as\n  flow setup [--replace] [--rotate <keys> [--now]] [--workspace <name|acc_id>] [--display-name <name>] [--ci] [--json]\n                            Write this agent's keys (its Flow key FLOW_KEY, which also pays for the\n                            model on the account's trial credit when there is no provider key, test\n                            cloud, traces, Flow Messaging) and each open channel's sender into\n                            ./.env (in an agent's folder: package.json); --rotate key,cloud,traces,\n                            messaging or all makes those keys again (the old FLOW_KEY works 24 h\n                            more; --now ends it at once). --workspace (here and on link) puts the\n                            agent in that observe workspace of yours, kept in .env as\n                            FLOW_FRAMEWORK_WORKSPACE; a new one makes the cloud and traces keys again\n  flow link [--channel telegram|whatsapp|imessage] [--workspace <name|acc_id>] [--display-name <name>] [--timeout <s>] [--no-browser] [--json]\n                            Print the link that opens a chat with this agent from a phone; signs in\n                            and runs setup first when this folder is not set up yet. --display-name\n                            (here and on setup) renames the agent: package.json's displayName and\n                            its prompt, agent/AGENTS.md; the phone link then says \"Hi <name>\"\n  flow status [--json]      What this folder has and lacks: FLOW_KEY and the model key (never their\n                            values) with the trial credit left, the workspace, the\n                            sign-in, each channel's sender in .env, and, signed in, which channels\n                            are open now and their free-trial counts";
 /** Runs one account command; resolves to the exit status. */
 export declare function accountCli(command: string, args: string[]): Promise<number>;

package/dist/testing/src/account.js

@@ -30,20 +30,22 @@ export const ACCOUNT_LINES = `  flow login [--wait | --no-wait] [--timeout <s>]
                             waiting for the same code, --no-wait returns at once
   flow logout [--json]      End this machine's sign-in and delete its token
   flow whoami [--json]      Show the account you are signed in as
-  flow setup [--replace] [--rotate <keys> [--now]] [--display-name <name>] [--ci] [--json]
+  flow setup [--replace] [--rotate <keys> [--now]] [--workspace <name|acc_id>] [--display-name <name>] [--ci] [--json]
                             Write this agent's keys (its Flow key FLOW_KEY, which also pays for the
                             model on the account's trial credit when there is no provider key, test
                             cloud, traces, Flow Messaging) and each open channel's sender into
                             ./.env (in an agent's folder: package.json); --rotate key,cloud,traces,
                             messaging or all makes those keys again (the old FLOW_KEY works 24 h
-                            more; --now ends it at once)
-  flow link [--channel telegram|whatsapp|imessage] [--display-name <name>] [--timeout <s>] [--no-browser] [--json]
+                            more; --now ends it at once). --workspace (here and on link) puts the
+                            agent in that observe workspace of yours, kept in .env as
+                            FLOW_FRAMEWORK_WORKSPACE; a new one makes the cloud and traces keys again
+  flow link [--channel telegram|whatsapp|imessage] [--workspace <name|acc_id>] [--display-name <name>] [--timeout <s>] [--no-browser] [--json]
                             Print the link that opens a chat with this agent from a phone; signs in
                             and runs setup first when this folder is not set up yet. --display-name
                             (here and on setup) renames the agent: package.json's displayName and
                             its prompt, agent/AGENTS.md; the phone link then says "Hi <name>"
   flow status [--json]      What this folder has and lacks: FLOW_KEY and the model key (never their
-                            values) with the trial credit left, the
+                            values) with the trial credit left, the workspace, the
                             sign-in, each channel's sender in .env, and, signed in, which channels
                             are open now and their free-trial counts`;
 const USAGE = `Usage:\n${ACCOUNT_LINES}
@@ -100,7 +102,7 @@ export async function accountCli(command, args) {
                     : {}),
                 ...(command === "setup" ? { replace: { type: "boolean", default: false }, rotate: { type: "string" }, now: { type: "boolean", default: false }, ci: { type: "boolean", default: false } } : {}),
                 ...(command === "link" ? { channel: { type: "string", default: "telegram" } } : {}),
-                ...(command === "link" || command === "setup" ? { "display-name": { type: "string" } } : {}),
+                ...(command === "link" || command === "setup" ? { "display-name": { type: "string" }, workspace: { type: "string" } } : {}),
             },
         }));
     }
@@ -131,6 +133,12 @@ export async function accountCli(command, args) {
         console.error(`flow setup: --now goes with --rotate key (or all): it ends the old FLOW_KEY at once.\n\n${USAGE}`);
         return EXIT.usage;
     }
+    const workspace = typeof values.workspace === "string" ? values.workspace.trim() : undefined;
+    // flow.engineer takes a workspace's acc_ id or its name: 1 to 100 characters, no control characters.
+    if (workspace !== undefined && (!workspace || workspace.length > 100 || /[\u0000-\u001f\u007f]/.test(workspace))) {
+        console.error(`flow ${command}: --workspace takes a workspace's name or its acc_ id, 1 to 100 characters.\n\n${USAGE}`);
+        return EXIT.usage;
+    }
     const json = values.json === true;
     // Straight to the file descriptors, never buffered: a coding agent's cut-off command keeps every line.
     const out = (report) => writeAll(1, `${JSON.stringify(report)}\n`);
@@ -162,9 +170,9 @@ export async function accountCli(command, args) {
         else if (command === "status")
             await status(json, out, say);
         else if (command === "setup")
-            await setupEnv({ replace: values.replace === true, rotate, now: values.now === true, ci: values.ci === true }, json, out, say);
+            await setupEnv({ replace: values.replace === true, rotate, now: values.now === true, ci: values.ci === true, workspace }, json, out, say);
         else
-            await link({ channel, noBrowser: values["no-browser"] === true, timeoutSeconds, renamed }, json, out, say);
+            await link({ channel, noBrowser: values["no-browser"] === true, timeoutSeconds, renamed, workspace }, json, out, say);
         return EXIT.ok;
     }
     catch (error) {
@@ -499,6 +507,19 @@ const OPEN_HINT = {
     whatsapp: "Open it and send the message it fills in.",
     imessage: "Open it on an iPhone or a Mac and send the message it fills in.",
 };
+/** The observe workspace this folder's agent was put in by --workspace (its acc_ id); unset: the personal one. */
+const WORKSPACE_ENV = "FLOW_FRAMEWORK_WORKSPACE";
+/** The answer's workspace_id, under the name setupEnv gives it only when a workspace was asked for. */
+const WORKSPACE_FIELD = "workspace_kept";
+/** --workspace names another workspace than the one .env's cloud and traces keys are in (none named yet: the personal one). */
+function movesWorkspace(workspace, value) {
+    if (workspace === undefined)
+        return false;
+    const stored = value(WORKSPACE_ENV);
+    if (stored)
+        return workspace.toLowerCase() !== stored.toLowerCase();
+    return Boolean(value(SETUP_ENV.cloud_key) || value(SETUP_ENV.traces_key));
+}
 async function setupEnv(options, json, out, say, forLink) {
     if (options.ci) {
         const steps = [
@@ -527,6 +548,14 @@ async function setupEnv(options, json, out, say, forLink) {
     const name = named.replace(/^@[^/]*\//, "").replace(/[^A-Za-z0-9 ._-]/g, "-").slice(0, 64) || "my-agent";
     const all = options.rotate.includes("all");
     const rotated = new Set(all ? Object.values(ROTATE).filter(Boolean) : options.rotate.map((k) => ROTATE[k]));
+    // The workspace asked for: --workspace, else the one an earlier --workspace kept in .env.
+    const workspace = options.workspace ?? (before.get(WORKSPACE_ENV) || undefined);
+    // Another workspace: the cloud and traces keys in .env are the old one's, so they are made again
+    // there (Messaging's key and FLOW_KEY are not bound to a workspace).
+    const moving = movesWorkspace(options.workspace, (n) => before.get(n));
+    if (moving)
+        for (const field of ["cloud_key", "traces_key"])
+            rotated.add(field);
     const rotatedEnv = new Set([...rotated].map((field) => (field === "flow_key" ? FLOW_KEY : SETUP_ENV[field])));
     // An empty value (`cp .env.example .env`) is missing: the server mints it. A model key counts in
     // the shell too (npm start reads both): FLOW_KEY is made only when it is not named as had, and
@@ -542,19 +571,22 @@ async function setupEnv(options, json, out, say, forLink) {
     // FLOW_FRAMEWORK_TRY_URL is one channel's page: link's channel when it is open, else the server's (Telegram's).
     const withTryUrl = (fields) => {
         const code = fields.join_code;
+        // The workspace the answer put the agent in, kept only when one was asked for.
+        if (workspace !== undefined && typeof fields.workspace_id === "string" && fields.workspace_id)
+            fields = { ...fields, [WORKSPACE_FIELD]: fields.workspace_id };
         if (!forLink || typeof fields[`${forLink.channel}_sender`] !== "string" || typeof code !== "string" || !code)
             return fields;
         const pages = fields.try_urls;
         const page = typeof pages?.[forLink.channel] === "string" ? pages[forLink.channel] : tryLink(url, code, forLink.channel);
         return { ...fields, try_url: page };
     };
-    const chosenPage = (n) => forLink !== undefined && n === SETUP_ENV.try_url;
+    const chosenPage = (n) => (forLink !== undefined && n === SETUP_ENV.try_url) || n === WORKSPACE_ENV;
     let answer;
     // For link: the server's refusal when Messaging's part came (a join code) and only observe's failed.
     let observeFailed;
     try {
         // Every channel: each comes back with its shared sender only while it is open.
-        answer = await setup(url, token, { name, hostname: hostname(), channels: [...CHANNELS], have, ...(rotate ? { rotate } : {}), ...(rotateNow ? { rotate_now: true } : {}) });
+        answer = await setup(url, token, { name, hostname: hostname(), channels: [...CHANNELS], have, ...(workspace !== undefined ? { workspace } : {}), ...(rotate ? { rotate } : {}), ...(rotateNow ? { rotate_now: true } : {}) });
     }
     catch (error) {
         // Texting needs only Messaging's part, so link goes on without observe's (traces and the test
@@ -579,6 +611,12 @@ async function setupEnv(options, json, out, say, forLink) {
     const had = new Map(before);
     // --replace takes every new value but a model key: only --rotate key replaces FLOW_KEY.
     const written = writeEnv(file, lines, before, envEntries(withTryUrl(fields)), (n) => rotatedEnv.has(n) || (options.replace && !MODEL_KEY_NAMES.includes(n)) || chosenPage(n));
+    // An older flow.engineer ignores the field and sets the agent up in the personal workspace.
+    if (workspace !== undefined && !observeFailed && (typeof answer.workspace_id !== "string" || !answer.workspace_id)) {
+        const stop = new Stop("workspace_unsupported", "flow.engineer does not support --workspace yet; the agent was set up in your personal workspace.", undefined, EXIT.refused);
+        stop.warnings = warnings;
+        throw stop;
+    }
     if (typeof answer.join_code !== "string" || !answer.join_code) {
         const kept = written.filter((w) => w.action === "added" || w.action === "replaced").map((w) => w.name);
         const stop = new Stop("no_join_code", `flow.engineer gave no join code${kept.length ? ` (the keys it made are in .env: ${kept.join(", ")})` : ""}, so there is no phone link yet.`, answer.next?.[0] ?? NEXT.setup, EXIT.refused);
@@ -586,7 +624,8 @@ async function setupEnv(options, json, out, say, forLink) {
         throw stop;
     }
     if (observeFailed) {
-        const missing = [SETUP_ENV.cloud_key, SETUP_ENV.traces_key].filter((n) => !before.get(n));
+        // Moving workspaces, the old workspace's keys still in .env do not count.
+        const missing = [SETUP_ENV.cloud_key, SETUP_ENV.traces_key].filter((n) => !before.get(n) || (moving && before.get(n) === had.get(n)));
         const why = observeFailed.why.replace(/\.$/, "");
         warnings.push(`Traces and tests won't work${missing.length ? ` until .env has ${missing.join(" and ")}` : ""}: ${why}.${observeFailed.fix ? ` Then: ${observeFailed.fix}` : ""}`);
     }
@@ -621,6 +660,7 @@ async function setupEnv(options, json, out, say, forLink) {
             whatsapp_link: answer.whatsapp_link ?? null, imessage_link: answer.imessage_link ?? null, try_urls: links,
             direct_links: Object.fromEntries(Object.entries(direct).map(([c, d]) => [c, d.link])), join_message: Object.values(direct)[0]?.message ?? null,
             trial, model_key: model.report,
+            workspace: typeof answer.workspace_id === "string" && answer.workspace_id ? { id: answer.workspace_id, name: answer.workspace_name ?? null } : null,
             // FLOW_KEY's state, never its value: made in this call, with its id, mode and scopes.
             flow_key: { set: Boolean(before.get(FLOW_KEY) || process.env[FLOW_KEY]?.trim()), minted: flowKey?.minted === true,
                 ...(flowKey?.minted ? { kid: flowKey.kid ?? null, mode: flowKey.mode ?? null, scopes: flowKey.scopes ?? [], previous_expires_at: flowKey.previous_expires_at ?? null } : {}) },
@@ -633,6 +673,8 @@ async function setupEnv(options, json, out, say, forLink) {
         say(`  ${w.name}=${w.value}  ${w.action}${w.action === "kept" ? (w.name === FLOW_KEY ? " (--rotate key makes a new one)" : " (yours differs; --replace takes the new one)") : ""}`);
     if (previous)
         say(previous);
+    if (answer.workspace_id)
+        say(`Workspace: ${answer.workspace_name ? `${answer.workspace_name} (${answer.workspace_id})` : answer.workspace_id}`);
     if (answer.dashboard_url)
         say(`Its dashboard: ${answer.dashboard_url}`);
     // Said by link instead, once, with its link.
@@ -669,6 +711,7 @@ function objectField(fields, name) {
  */
 function envEntries(fields) {
     const entries = Object.entries(SETUP_ENV).map(([field, envName]) => [envName, fields[field]]);
+    entries.push([WORKSPACE_ENV, fields[WORKSPACE_FIELD]]);
     const key = objectField(fields, "flow_key");
     if ((key?.env ?? FLOW_KEY) === FLOW_KEY && typeof key?.value === "string")
         entries.push([FLOW_KEY, key.value]);
@@ -745,7 +788,8 @@ async function status(json, out, say) {
                 const named = JSON.parse(readFileSync(join(process.cwd(), "package.json"), "utf8")).name ?? basename(process.cwd());
                 const name = named.replace(/^@[^/]*\//, "").replace(/[^A-Za-z0-9 ._-]/g, "-").slice(0, 64) || "my-agent";
                 const have = [...Object.values(SETUP_ENV).filter((n) => n !== SETUP_ENV.try_url), ...MODEL_KEY_NAMES];
-                server = await setup(url, current.token, { name, hostname: hostname(), channels: [...CHANNELS], have });
+                const workspace = value(WORKSPACE_ENV);
+                server = await setup(url, current.token, { name, hostname: hostname(), channels: [...CHANNELS], have, ...(workspace ? { workspace } : {}) });
             }
         }
         catch {
@@ -764,11 +808,13 @@ async function status(json, out, say) {
         ...(model.report.set && !model.next ? ["npm start"] : []),
         ...(!joinCode || !value(SETUP_ENV.messaging_key) ? [NEXT.link] : CHANNELS.filter((c) => channels[c].open && !channels[c].sender).map((c) => `${NEXT.link} --channel ${c}`)),
     ];
+    const workspace = value(WORKSPACE_ENV) || "personal";
     if (json)
-        out({ model_key: model.report, flow_key: { set: Boolean(value(FLOW_KEY)) }, signed_in: account ? { email: account.email, account: account.account } : false, join_code: joinCode, messaging_key: Boolean(value(SETUP_ENV.messaging_key)), channels, next });
+        out({ model_key: model.report, flow_key: { set: Boolean(value(FLOW_KEY)) }, workspace, signed_in: account ? { email: account.email, account: account.account } : false, join_code: joinCode, messaging_key: Boolean(value(SETUP_ENV.messaging_key)), channels, next });
     if (model.report.env && model.report.env !== FLOW_KEY)
         say(value(FLOW_KEY) ? `${FLOW_KEY} is set.` : `${FLOW_KEY}: not set yet (${RUN} setup writes it).`);
     say(model.line);
+    say(`Workspace: ${workspace}`);
     say(account ? `Signed in as ${account.email}.` : current ? "Not signed in here (or flow.engineer could not be reached)." : "Not signed in.");
     for (const c of CHANNELS) {
         const { sender, open, trial } = channels[c];
@@ -828,7 +874,8 @@ async function link(options, json, out, say) {
     let trial;
     let had;
     let warnings = [];
-    if (!value(SETUP_ENV.join_code) || !value(SETUP_ENV.messaging_key) || (needsSender && !value(SENDER_ENV[options.channel]))) {
+    const newWorkspace = options.workspace !== undefined && (movesWorkspace(options.workspace, value) || !value(WORKSPACE_ENV));
+    if (newWorkspace || !value(SETUP_ENV.join_code) || !value(SETUP_ENV.messaging_key) || (needsSender && !value(SENDER_ENV[options.channel]))) {
         agentFolder(process.cwd());
         if (!(await signedInAccount(loginUrl()))) {
             say("Not signed in to flow.engineer yet: signing in first.");
@@ -836,7 +883,7 @@ async function link(options, json, out, say) {
         }
         say("Setting up this agent's keys in .env.");
         had = value;
-        ({ answer: { model_credit: credit, trial }, warnings } = await setupEnv({ replace: false, rotate: [], ci: false }, json, out, say, { channel: options.channel }));
+        ({ answer: { model_credit: credit, trial }, warnings } = await setupEnv({ replace: false, rotate: [], ci: false, workspace: options.workspace }, json, out, say, { channel: options.channel }));
         value = dotenvValue();
     }
     const joinCode = value(SETUP_ENV.join_code);

package/dist/testing/src/cli.js

@@ -44,7 +44,7 @@ import { CHAT_LINE, chatCli } from "./chat.js";
 import { httpAgent } from "./http-agent.js";
 import { createInterface } from "node:readline/promises";
 import { displayNameOf, folderName, initHere, newStandalone, packageRoot } from "./new.js";
-import { EXIT, finishedOutcome, loadEnv, networkCode, reportError, runVerdict, stoppedOutcome } from "./report.js";
+import { EXIT, finishedOutcome, invalidLines, invalidScenarios, loadEnv, networkCode, reportError, runVerdict, sessionInvalid, sessionInvalidLines, stoppedOutcome } from "./report.js";
 import { LAST_SESSION, lastSessionIn, SHOW_LINE, showCli } from "./show.js";
 import { loadSuite, parseScenario, parseToolFile, suiteHash } from "./suite.js";
 const NEW_LINE = `  flow new <name> [--display-name <name>]
@@ -105,7 +105,7 @@ Exit status:
 Results: once the session has opened, every exit writes .flow/last-session.json in the agent's
 folder: the session id, the cloud's agent id, dashboard link, suite hash and time, and per run its
 scenario, channel, number, run id, verdict (passed, failed, errored or not_played) and failed checks
-with the judge's reasons. Never the messages. The folder is git-ignored and the suite synthetic.
+with the judge's reasons (and, for a scenario the cloud could not play, its file and what is wrong). Never the messages. The folder is git-ignored and the suite synthetic.
 
 Environment (also read from the agent folder's .env, or --dotenv; variables already set win;
 the CHINWAG_ names from before the rename still work):
@@ -251,6 +251,8 @@ let sessionId = "";
 let dashboard = "";
 let agentId = "";
 const seen = [];
+/** Set when the cloud ended the whole session scenario_invalid. */
+let endedInvalid = null;
 if (command !== undefined && (command !== "test" || rest.length))
     invalid(`flow: unknown command "${positionals.join(" ")}".\n\n${USAGE}`);
 const envProblem = loadEnv(values.dotenv);
@@ -381,6 +383,10 @@ catch (error) {
         end("transport", { received: error.runs, error: reportError(error, reason) });
     }
     console.error(reason);
+    endedInvalid = sessionInvalid(error, files.filter((f) => f.path.startsWith("scenarios/")).map((f) => ({ scenario: f.path.slice("scenarios/".length, -".yaml".length), path: f.path })));
+    if (endedInvalid)
+        for (const line of sessionInvalidLines(endedInvalid))
+            console.error(line);
     // The cloud stopped a session that had opened: what finished is kept, the rest is not_played.
     saveSession("stopped", seen, dashboard, console.error);
     end(stoppedOutcome(error, sessionId !== ""), { received: seen, error: reportError(error, reason) });
@@ -402,6 +408,8 @@ for (const channel of [...new Set(summary.runs.map((r) => r.channel))]) {
     const runs = summary.runs.filter((r) => r.channel === channel);
     say(`  ${channel}: ${runs.filter((r) => runVerdict(r) === "passed").length} of ${runs.length} passed`);
 }
+for (const line of invalidLines(invalidScenarios(mergedRuns(summary.runs).map((m) => ({ scenario: m.scenario, invalid: m.summary?.invalid })))))
+    say(line);
 if (summary.lost)
     say(`The cloud lost session ${summary.lost.sessionId} part way; a second session played its unfinished runs. The first: ${summary.lost.dashboard}`);
 say(`Cost: ${usd(total((r) => r.agentUsage.costUsd))} agent (${tokens(total((r) => r.agentUsage.input))} in, ${tokens(total((r) => r.agentUsage.output))} out) + ${usd(total((r) => r.testUsage.costUsd))} test models.\n${summary.dashboard}`);
@@ -432,8 +440,11 @@ function mergedRuns(received) {
         fileOf.set(scenario.id, id);
         for (let run = 1; run <= perScenario; run++) {
             for (const channel of scenario.channels) {
-                const r = byRun.get(key(scenario.id, channel, run));
+                // A scenario the cloud could not read is named by its file, not its id.
+                const r = byRun.get(key(scenario.id, channel, run)) ?? byRun.get(key(id, channel, run));
                 byRun.delete(key(scenario.id, channel, run));
+                if (r)
+                    byRun.delete(key(r.scenario, channel, run));
                 out.push(r ? merged(id, r) : { scenario: id, channel, run, verdict: "not_played" });
             }
         }
@@ -466,7 +477,9 @@ function saveSession(ended, received, link, print) {
             runId: m.summary?.runId ?? null,
             verdict: m.verdict,
             failedChecks: failedChecks(m.summary),
+            ...(m.summary?.invalid ? { invalid: m.summary.invalid } : {}),
         })),
+        ...(endedInvalid ? { sessionInvalid: endedInvalid } : {}),
     };
     const path = join(dir, LAST_SESSION);
     try {
@@ -496,6 +509,7 @@ function end(outcome, options = {}) {
             turns: m.summary?.turns ?? null,
             watch: m.summary?.watch || null,
             failedChecks: failedChecks(m.summary),
+            invalid: m.summary?.invalid ?? null,
         })) : [];
         const count = (verdict) => runs.filter((r) => r.verdict === verdict).length;
         const cost = (pick) => received.reduce((n, r) => n + pick(r), 0);
@@ -511,6 +525,8 @@ function end(outcome, options = {}) {
             counts: { runs: runs.length, passed: count("passed"), failed: count("failed"), errored: count("errored"), notPlayed: count("not_played") },
             costUsd: { agent: cost((r) => r.agentUsage.costUsd), test: cost((r) => r.testUsage.costUsd) },
             runs,
+            invalid: invalidScenarios(runs),
+            sessionInvalid: endedInvalid,
         };
         writeAll(`${JSON.stringify(report)}\n`);
     }

package/dist/testing/src/client.d.ts

@@ -49,6 +49,8 @@ export interface ConnectOptions {
  * code (`capacity: ...` gives capacity), when it starts with one. */
 export declare class CloudEnded extends Error {
     readonly code?: string;
+    /** The cloud's error as it sent it. */
+    readonly error: string;
     constructor(error: string);
 }
 /** The tunnel, the stream, or the connection failed after the session opened. `runs` are the ones already received. The CLI prints them and exits 2. */

package/dist/testing/src/client.js

@@ -15,9 +15,12 @@ import { businessRuns } from "./business.js";
  * code (`capacity: ...` gives capacity), when it starts with one. */
 export class CloudEnded extends Error {
     code;
+    /** The cloud's error as it sent it. */
+    error;
     constructor(error) {
         super(`the cloud stopped the session: ${error}`);
         this.name = "CloudEnded";
+        this.error = error;
         this.code = /^([a-z_]+):/.exec(error)?.[1];
     }
 }

package/dist/testing/src/flow-login.d.ts

@@ -71,6 +71,9 @@ export interface FlowKey {
 }
 export interface SetupAnswer {
     agent_id: string;
+    /** The observe workspace (acc_...) the agent and its cloud and traces keys are in; absent from an older server. */
+    workspace_id?: string;
+    workspace_name?: string;
     /** Only the keys minted in this call come back (those the request did not say it has). */
     cloud_key?: string;
     traces_key?: string;
@@ -148,6 +151,7 @@ export declare function setup(url: string, token: string, request: {
     hostname: string;
     channels: string[];
     have: string[];
+    workspace?: string;
     rotate?: true | string[];
     rotate_now?: true;
 }): Promise<SetupAnswer>;

package/dist/testing/src/flow-login.js

@@ -6,8 +6,8 @@
 //                                                authorization_pending, slow_down, expired_token, access_denied, invalid_grant
 //   POST /cli/revoke (bearer) {}              ends this token
 //   GET  /cli/whoami (bearer)                 -> {account, email, token}
-//   POST /cli/setup (bearer) {name, hostname, channels, have, rotate?, rotate_now?}
-//        -> {agent_id, cloud_key?, traces_key?, messaging_key?, flow_key?, model_credit?, join_code, account, app_id,
+//   POST /cli/setup (bearer) {name, hostname, channels, have, workspace?, rotate?, rotate_now?}
+//        -> {agent_id, workspace_id?, workspace_name?, cloud_key?, traces_key?, messaging_key?, flow_key?, model_credit?, join_code, account, app_id,
 //            dashboard_url, try_url, <channel>_sender?, <channel>_link? (telegram, whatsapp, imessage: each only while
 //            that shared sender is open), try_urls?, trial?, minted, next}; a 502 carries every field that worked too;
 //        flow_key is the agent's one Flow key, {env: "FLOW_KEY", value: "flow_test_...", kid, mode, scopes, agent_id,
@@ -18,6 +18,9 @@
 //        the reason in warnings; every folder's setup makes its own key, the credit is the account's;
 //        rotate is true (every key) or the answer fields to make again, such as ["flow_key"]; rotate_now: true
 //        ends the old FLOW_KEY at once instead of 24 h later (previous_expires_at);
+//        workspace is an observe workspace's acc_ id or name (case-insensitive), 1 to 100 characters; absent: the
+//        personal one; workspace_id and workspace_name are the workspace the agent and its cloud and traces keys are in
+//        (whenever observe answered; an older server sends neither); 403 not_a_member, 409 workspace_ambiguous (pass the id);
 //        429 slow_down with Retry-After; 403 for a trial limit (sandbox_allowance_used, ...), its next prose
 //        Any answer may carry warnings: up to 20 strings to show the owner as they are.
 //   GET  <model URL>/credit (bearer FLOW_KEY) -> {limit_usd, spent_usd, left_usd, default_model, models};

package/dist/testing/src/print-run.js

@@ -1,4 +1,4 @@
-import { runVerdict } from "./report.js";
+import { runVerdict, unsupported } from "./report.js";
 const usd = (n) => `$${n.toFixed(4)}`;
 /** Prints each run once. A run already shown as it finished is not printed again with the summary. */
 export function runPrinter(log = console.log) {
@@ -8,6 +8,11 @@ export function runPrinter(log = console.log) {
             return;
         printed.add(r.runId);
         const verdict = runVerdict(r);
+        // A scenario the cloud could not play: its file and what is wrong in it, not a run of the agent.
+        if (r.invalid) {
+            log(`INVALID  ${r.scenario} #${r.run} (${r.channel}) ${r.runId}: the cloud could not play this scenario: ${r.invalid.path}: ${r.invalid.reason}${unsupported(r.invalid.reason)}`);
+            return;
+        }
         const gaveUp = verdict === "passed" && r.ended === "gave_up" ? " (passed, the person gave up)" : "";
         const retried = r.retried ? "; tried once more" : "";
         // An errored run: the cloud could not play or judge it, so its checks say nothing about the agent.

package/dist/testing/src/protocol.d.ts

@@ -246,6 +246,12 @@ export interface RunSummary {
     /** The cloud's code when it could not play or judge the run (judge_failed, spend_cap, simulator_failed, ...).
      * The run then says nothing about the agent, whatever `passed` and `ended` say. */
     error?: string;
+    /** With error scenario_invalid: the scenario file the cloud could not play, and what is wrong in it
+     * (`unsupported: ...` when the cloud does not know a key). */
+    invalid?: {
+        path: string;
+        reason: string;
+    };
 }
 export interface SessionSummary {
     sessionId: string;

package/dist/testing/src/report.d.ts

@@ -26,6 +26,27 @@ export interface ReportRun {
         text: string;
         reason: string;
     }>;
+    /** The scenario file the cloud could not play and what is wrong in it (error scenario_invalid); else null. */
+    invalid: {
+        path: string;
+        reason: string;
+    } | null;
+}
+/** A scenario the cloud could not play, once per scenario. */
+export interface InvalidScenario {
+    scenario: string;
+    path: string;
+    reason: string;
+}
+/** The cloud ended the whole session scenario_invalid: what it said after the code (null when nothing),
+ * whether that looks like a cloud older than this client, and the scenario files this session sent. */
+export interface SessionInvalid {
+    reason: string | null;
+    olderCloud: boolean;
+    sent: Array<{
+        scenario: string;
+        path: string;
+    }>;
 }
 /** `flow test --json`: one document on stdout, whatever the exit. */
 export interface JsonReport {
@@ -50,6 +71,10 @@ export interface JsonReport {
         test: number;
     };
     runs: ReportRun[];
+    /** Each scenario the cloud could not play; the other scenarios played. */
+    invalid: InvalidScenario[];
+    /** Set when the cloud ended the whole session scenario_invalid; else null. */
+    sessionInvalid: SessionInvalid | null;
 }
 /** A run's verdict: errored when the cloud names an error for it or ended it run_failed, since the cloud
  * could not play or judge it and that says nothing about the agent (a judge_failed run can end goal_met). */
@@ -65,6 +90,27 @@ export declare function finishedOutcome(runs: Array<{
 }>): Outcome;
 /** What stopped a session, or the open, as an outcome. `opened`: the session had opened. */
 export declare function stoppedOutcome(error: unknown, opened: boolean): Outcome;
+/** The scenarios the cloud could not play, once each, from the runs it reported (a run per channel and --runs). */
+export declare function invalidScenarios(runs: Array<{
+    scenario: string;
+    invalid?: {
+        path: string;
+        reason: string;
+    } | null;
+}>): InvalidScenario[];
+/** The lines that name each scenario the cloud could not play. */
+export declare function invalidLines(invalid: InvalidScenario[]): string[];
+/** A reason naming a key the cloud does not know: the client may be newer than the cloud. */
+export declare function unsupported(reason: string): string;
+/** A session the cloud ended scenario_invalid; null for any other stop. A tools/ or checks/ file with
+ * its reason is the suite's own problem; anything else (no reason, or a scenario, which a current
+ * cloud reports as that scenario's run) suggests a cloud older than this client. */
+export declare function sessionInvalid(error: unknown, sent: Array<{
+    scenario: string;
+    path: string;
+}>): SessionInvalid | null;
+/** The lines that say why the session ended scenario_invalid. */
+export declare function sessionInvalidLines(info: SessionInvalid): string[];
 /** The error as the JSON report says it: the cloud's status, code and Retry-After when it answered. */
 export declare function reportError(error: unknown, message: string): ReportError;
 /** The connection's error code, from the error or what caused it. */

package/dist/testing/src/report.js

@@ -38,6 +38,48 @@ export function stoppedOutcome(error, opened) {
     // Something on this machine: the agent would not build before the open, or broke the session after it.
     return opened ? "failed" : "invalid";
 }
+/** The scenarios the cloud could not play, once each, from the runs it reported (a run per channel and --runs). */
+export function invalidScenarios(runs) {
+    const out = new Map();
+    for (const r of runs)
+        if (r.invalid)
+            out.set(`${r.scenario}\n${r.invalid.path}`, { scenario: r.scenario, path: r.invalid.path, reason: r.invalid.reason });
+    return [...out.values()];
+}
+/** The lines that name each scenario the cloud could not play. */
+export function invalidLines(invalid) {
+    if (!invalid.length)
+        return [];
+    const lines = [`${invalid.length} ${invalid.length === 1 ? "scenario" : "scenarios"} could not be played (fix ${invalid.length === 1 ? "it" : "them"} and run again):`];
+    for (const i of invalid)
+        lines.push(`  ${i.scenario}: ${i.path}: ${i.reason}${unsupported(i.reason)}`);
+    return lines;
+}
+/** A reason naming a key the cloud does not know: the client may be newer than the cloud. */
+export function unsupported(reason) {
+    return reason.startsWith("unsupported: ") ? " (the test cloud does not know this key; it may be older than this client)" : "";
+}
+/** A session the cloud ended scenario_invalid; null for any other stop. A tools/ or checks/ file with
+ * its reason is the suite's own problem; anything else (no reason, or a scenario, which a current
+ * cloud reports as that scenario's run) suggests a cloud older than this client. */
+export function sessionInvalid(error, sent) {
+    if (!(error instanceof CloudEnded))
+        return null;
+    const match = /^scenario_invalid(?::\s*(.*))?$/s.exec(error.error.trim());
+    if (!match)
+        return null;
+    const reason = match[1]?.trim() || null;
+    const suiteFile = reason !== null && /^(?:\S*\/)?(?:tools|checks)\/[^:]+:\s*\S/.test(reason);
+    return { reason, olderCloud: !suiteFile, sent };
+}
+/** The lines that say why the session ended scenario_invalid. */
+export function sessionInvalidLines(info) {
+    const lines = [`The test cloud could not play this suite${info.reason ? `: ${info.reason}` : " and gave no reason."}`];
+    if (info.olderCloud)
+        lines.push("the test cloud may be older than this client");
+    lines.push(`Scenarios sent: ${info.sent.map((s) => `${s.scenario} (${s.path})`).join(", ") || "none"}`);
+    return lines;
+}
 /** The error as the JSON report says it: the cloud's status, code and Retry-After when it answered. */
 export function reportError(error, message) {
     const refusal = postErrorOf(error);

package/dist/testing/src/show.d.ts

@@ -1,3 +1,4 @@
+import { type SessionInvalid } from "./report.ts";
 /** Where every session's results go, in the agent's folder; `flow new` git-ignores .flow/. */
 export declare const LAST_SESSION: string;
 /** The last session's file in dir: LAST_SESSION, else, before a run has written it, the one from before the rename (.chinwag/). */
@@ -29,7 +30,15 @@ export interface LastSession {
             text: string;
             reason?: string;
         }>;
+        /** The scenario file the cloud could not play and what is wrong in it (error scenario_invalid). */
+        invalid?: {
+            path: string;
+            reason: string;
+        };
     }>;
+    /** The cloud ended the whole session scenario_invalid: its reason, whether the cloud looks older
+     * than the client, and the scenario files sent. */
+    sessionInvalid?: SessionInvalid;
 }
 export declare const SHOW_LINE = "  flow show <run-id|last> [--json]\n                            Print one run's record from the test cloud: its checks with the\n                            judge's reasons, the transcript and the tool calls (flow show --help)";
 /** `flow show`: returns the exit status. */

package/docs/agents.md

@@ -63,9 +63,12 @@ before you hand it back; run them yourself, and do not offer them to the owner a
   and `init`, `setup`, `link` and `status` say so too, with the way to get one first in `next`.
   Each folder's `setup` makes its own `FLOW_KEY` (the credit is the account's);
   `setup --rotate key` replaces a leaked one (the old one works 24 h more; `--now` ends it at once).
+- `setup --workspace <name|acc_id>` (and `link --workspace`) puts the agent, its cloud and traces keys in
+  that observe workspace of yours and keeps its `acc_` id in `.env` as `FLOW_FRAMEWORK_WORKSPACE`, which later
+  runs send; another workspace makes those two keys again (403 `not_a_member`, 409 `workspace_ambiguous`: pass the id).
 - `status` reports `FLOW_KEY` and the model key (never their values; `flow_key.set` in `--json`; with
   `FLOW_KEY` the model's key, the credit left from Flow, `model_key.flow_credit`; a key Flow does not
-  know gets `setup --rotate key`), the sign-in, each channel's sender in `.env`
+  know gets `setup --rotate key`), the workspace (`FLOW_FRAMEWORK_WORKSPACE`, else `personal`), the sign-in, each channel's sender in `.env`
   and, signed in, which channels are open now with their trial counts. It writes nothing.
 - `chat` builds the agent from `src/agent.ts` and writes as its first test person. Each reply is
   printed (`agent: ...`) once the turn is over; `--json` gives one JSON line per thing it sends.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@flow-engineer/flow",
-  "version": "0.3.1",
+  "version": "0.3.3",
   "description": "An open-source runtime for chat agents that feel like a person texting back.",
   "license": "Apache-2.0",
   "repository": {