stageflow 0.42.0 → 0.44.0

package/dist/guide/knowledge/topics/connections.js

@@ -0,0 +1,66 @@
+export const connectionsTopic = {
+    id: "connections",
+    title: "Connections and stage channels",
+    summary: "Host-level messaging connections (AgentMail, Slack, Telegram, iMessage, WhatsApp), how a stage sends with a pinned recipient, the one-channel-per-stage limit, secrets, live checks, and why an unknown send is never resent.",
+    body: `# Connections and stage channels
+
+A **connection** is a host-level link to a messaging provider. A stage lists it under \`channels:\` with the recipient fixed, and gets one tool, \`send_message\`. An agent can also answer people on one connection (see the agents topic).
+
+## Providers
+
+| Provider | What \`to\` is | Notes |
+|---|---|---|
+| AgentMail | One email or a list of up to 20, optional \`cc\` (up to 20). No \`bcc\`. | Uses one existing inbox (\`config.inbox\`). Plain text only, up to 64 KiB, optional subject. Key needs inbox read and send. |
+| Slack | One id: channel \`C…\`, private \`G…\`, DM \`D…\` or user \`U…\`. Names like \`#general\` are refused. | Bot token (\`chat:write\`). A private channel needs the bot invited. |
+| Telegram | Numeric chat id or a public channel \`@username\`. | The person must start the bot first. Text up to 4096 characters. |
+| iMessage (Photon) | E.164 phone or Apple ID email, one recipient. | Free/Pro Photon projects: the person must be a registered project user and must text their assigned line once first. |
+| WhatsApp (Photon) | E.164 phone, one recipient. | Free text only within 24 hours of the person's last message; otherwise pin an approved \`template\`. Send-only for everyone. |
+
+iMessage and WhatsApp need the optional \`@spectrum-ts/*\` packages; without them sends fail with \`CHANNEL_PROVIDER_UNSUPPORTED\`.
+
+## Stage YAML
+
+\`\`\`yaml
+channels:
+  - connection: ops-slack
+    to: C0123ABCD
+    operations: [send]
+\`\`\`
+
+- \`connection\` (id), \`to\` (pinned recipient), \`operations\` (\`[send]\` is the only value today) are required. Optional: \`alias\`, \`cc\` (providers that support it), \`template\` (WhatsApp only).
+- **One channel per stage** (\`MAX_CHANNELS_PER_STAGE\` is 1). A second entry is rejected at validate, in the console, at \`start_run\`, at run start and at send, including one inherited from a pipeline-level default. To notify on two providers, use two stages. One AgentMail connection can still reach several people through its \`to\` and \`cc\` lists.
+- \`channels\` is a body key: put it in the stage file or inline entry, not on a \`uses:\` wrapper.
+- A pipeline-level \`channels:\` is inherited by every stage; \`channels: []\` on a stage drops it.
+- The recipient is chosen by the author. The agent can never pick a different one.
+- Pi backend only. On the Claude backend a stage with \`channels:\` fails at start.
+- Tell the agent in \`system_prompt\` what to send. It calls \`send_message\` with an \`operationKey\`.
+
+## Secrets
+
+- A secret is never a value in YAML. In \`connections.yaml\` it is \`env:NAME\` (read from the Host environment or \`NAME_FILE\`) or \`store:...\` (written by the console or tools into \`channel-secrets.json\`, mode 0600).
+- \`create_connection\` and \`update_connection\` store secret values; no tool, API or screen ever returns them. Secrets never reach run state, logs, envelopes, artifacts or the stage worker.
+- Do not repeat a secret back to the user.
+
+## Where connections live
+
+Host-level: \`$STAGEFLOW_HOME/connections.yaml\` (default \`~/.stageflow/connections.yaml\`), shared by every project on the Host. A \`connections.yaml\` next to a project's \`stageflow.yaml\` wins for that project. \`STAGEFLOW_CONNECTIONS_FILE\` overrides both. Console and tool edits apply without a restart; a hand edit needs a Host restart. Limits: 256 KiB, 100 connections, 20 shared credentials.
+
+## Checks
+
+1. \`validate\`: the connection exists and is enabled, the provider supports the operation, the target is valid. No network, no secret read.
+2. Run start (also \`preflight\` and \`sf doctor\`): a live check of each connection. Failure blocks the run with \`channel_check_failed\`. It never sends a message.
+3. The Host re-checks enabled connections every 5 minutes; \`test_connection\` checks values without saving.
+
+## Sending and retries
+
+- The Host sends with the pinned target. A repeated \`operationKey\` with the same content returns the first result without sending again; the same key with different content is \`CHANNEL_OPERATION_CONFLICT\`.
+- A send that may have reached the provider is \`CHANNEL_SEND_OUTCOME_UNKNOWN\` and is **never resent**, even after a restart. Only definite failures (refused connection, DNS/TLS error, 4xx) may be retried, under a new key.
+- Accepted means the provider took the message, not that anyone read it.
+
+## Tools
+
+\`list_connections\` (secret fields show only \`env\` or \`store\`), \`create_connection\`, \`update_connection\` (enable, rename, config, replace secrets), \`test_connection\`, \`validate_connection_target\` (checks a recipient against provider rules, no network), \`remove_connection\` (removes the entry and its stored secrets; destructive).
+
+Email accounts (\`email.yaml\`, IMAP/SMTP, used by email triggers and the stage \`email\` permissions) are separate from connections. An IMAP account cannot be an agent channel; AgentMail can.
+`,
+};

package/dist/guide/knowledge/topics/consoleMap.d.ts

@@ -0,0 +1,2 @@
+import type { GuideKnowledgeTopic } from "../index.js";
+export declare const consoleMapTopic: GuideKnowledgeTopic;

package/dist/guide/knowledge/topics/consoleMap.js

@@ -0,0 +1,55 @@
+export const consoleMapTopic = {
+    id: "console_map",
+    title: "Console map: where things live",
+    summary: "The operator console's sidebar sections and hash routes (#/inbox, #/pipelines, #/agents, #/triggers, #/connections, #/memories, #/evals, #/settings, …) and what to do on each page.",
+    body: `# Console map
+
+The console runs at the Host's address (\`sf ui\`, default \`http://127.0.0.1:3847\`). Pages are hash routes, so a link looks like \`#/pipelines\`. When you tell a user where to look, give the sidebar label and the route.
+
+## Sidebar
+
+**Top**
+- **Guide** — \`#/guide\`. This chat.
+- **Overview** — \`#/overview\`. What needs you, runs in flight, system health (agent slots, providers, connections needing attention, triggers armed, disk), KPIs and spend for 24h / 7d / 30d.
+- **Inbox** — \`#/inbox\` (home page). Tabs: **Needs you** (pipeline gates and agent approvals; answer here), **Failed** (\`#/inbox?tab=failed\`, retry a stage), **Done today** (\`#/inbox?tab=done_today\`).
+
+**Activity**
+- **Pipeline runs** — \`#/runs\`. Run list with filters. A run: \`#/runs/<runId>\`; a stage: \`#/runs/<runId>/stages/<stageId>\` (Transcript, Events, Envelope, Artifacts, and Retry / Resume / Abandon / answer panel); the envelope: \`#/runs/<runId>/stages/<stageId>/envelope\`; an artifact: \`#/runs/<runId>/artifacts?path=…\`.
+- **Agent sessions** — \`#/agent-sessions\`. Sessions across all agents.
+
+**Build**
+- **Pipelines** — \`#/pipelines\`. List; **New pipeline** opens the Workshop. One pipeline: \`#/pipelines/<id>\` opens the pipeline editor (stage graph, inspector with the stage **Channels** section, YAML panel, Problems panel with quick fixes; tabs Editor, Runs, History).
+- **Agents** — \`#/agents\`. Directory with status (Enabled, Disabled, Needs setup) and reach. New: \`#/agents/new\` (form plus author chat; always created disabled). One agent: \`#/agents/<id>\` (Enable/Disable, web chat card, Sessions, Configuration); edit: \`#/agents/<id>/edit\` (tools, approvals, limits, channel, schedule, web chat, version history, Remove = archive). A session: \`#/agents/<id>/sessions/<sessionId>\`; one turn: \`…/runs/<runId>\` (Steps, Waterfall, Messages, Logs, Configuration).
+- **Tasks** — \`#/tasks\`. Task files with their last run; one task: \`#/tasks/<id>\`.
+- **Catalog** — \`#/catalog?tab=stages\`. Stage files (\`#/stages/<path>\` opens one) and **Skills** (\`#/catalog?tab=skills\` or \`#/skills\`; install a skill here).
+- **Evals** — \`#/evals\`. List with status (never run, running, passing, failing, regressed, saturated) and **Try the sample eval**. New: \`#/evals/new\`. One eval: \`#/evals/<id>\`, edit \`#/evals/<id>/edit\`, a run \`#/evals/<id>/runs/<n>\` (tabs Overview, Cases, Graders, Config), a case \`…/case/<caseId>\`, compare \`#/evals/<id>/compare\`. Datasets: \`#/datasets\`, \`#/datasets/<id>\`.
+
+**Connect**
+- **Triggers** — \`#/triggers\`. Create (N), Fire (F, only enabled triggers with a catalog task), enable/disable (E). One trigger: \`#/triggers/<id>\`. The form edits \`event.source\` but not \`event.config\`.
+- **Connections** — \`#/connections\`. Communication channels, email accounts and MCP tool servers. **Add connection**, Test connection. One connection: \`#/connections/<id>\` (status, "Use in a stage" snippet, Enable/Disable, replace key, Disconnect).
+- **Memories** — \`#/memories\`. Create a memory, edit its cover, open, correct or delete cards. One memory: \`#/memories/<id>\`.
+
+## Not in the sidebar
+
+- **Start a run** — \`#/new\` (also the rail button). Pick a task and a pipeline; shows provider and validity before starting. \`?pipeline=\` and \`?task=\` preselect.
+- **Workshop** — \`#/workshop\`. Design a pipeline by chat; opened from Pipelines (New pipeline, or Open in Workshop). Saves to \`workshop/<pipeline-id>/\` unless told otherwise.
+- **Settings** — \`#/settings\` (rail footer). Sections: General (Workshop model, disk), Providers (connect, replace key, sign in again, disconnect), Concurrency (agent slots), MCP server (this console's own MCP address), Project MCP (add servers, Check, Sign in, MCP secrets), Notifications (desktop notification when a gate or approval opens).
+- **Connect** — \`#/connect\`. Guided provider setup.
+- **Public web chat** — \`/chat/<slug>\` on the Host (not a console page). Live only while the agent and its web chat are enabled.
+
+## Where to send the user for common jobs
+
+| Job | Page |
+|---|---|
+| Answer a question a stage asked, approve an agent action | Inbox \`#/inbox\` |
+| See why a run failed | \`#/runs/<runId>\`, open the stage, Transcript and Envelope |
+| Connect a model provider | Settings → Providers, or \`#/connect\` |
+| Add an MCP server or MCP secret | Settings → Project MCP |
+| Add Slack / email / Telegram / iMessage / WhatsApp | Connections → Add connection |
+| Install a skill | Catalog → Skills |
+| Change a stage's channel recipient | Pipeline editor → select stage → Channels |
+| Enable an agent, share its web chat link | \`#/agents/<id>\` |
+| Delete a memory card | \`#/memories/<id>\` |
+| Run or compare an eval | \`#/evals/<id>\` |
+`,
+};

package/dist/guide/knowledge/topics/evals.d.ts

@@ -0,0 +1,2 @@
+import type { GuideKnowledgeTopic } from "../index.js";
+export declare const evalsTopic: GuideKnowledgeTopic;

package/dist/guide/knowledge/topics/evals.js

@@ -0,0 +1,72 @@
+export const evalsTopic = {
+    id: "evals",
+    title: "Evals: checking a stage or an agent",
+    summary: "Stage vs agent evals, datasets and scenarios, graders and which target kind each applies to, repeats and statistics, cost cap, judge providers, the engine install, read_only tool policy, the two consents for real tools, and sorting failures.",
+    body: `# Evals
+
+An eval (\`evals/<id>.eval.yaml\`) runs one **target** over a **dataset** of cases, grades every result with **graders**, repeats each case, and lets you compare runs. Results are stored and survive restarts. Saving an eval never changes past runs.
+
+## Target
+
+- Stage: \`target: { kind: stage, pipeline: <path>, stage: <id> }\` or \`{ kind: stage, file: <stage file> }\`. Paths are relative to the eval file. Each case becomes a normal stage run (hidden from the Runs list), with \`input\` from the case and \`goal\` from the case, the eval's \`task.goal\`, or "Evaluate case <id>".
+- Agent: \`target: { kind: agent, agent: <id> }\`. Each case is a conversation in a hidden eval session on the agent's current revision. No \`task\` block on agent targets.
+
+## Dataset
+
+A file in \`datasets/\`: \`.dataset.jsonl\` (editable in the console), \`.dataset.json\` or \`.dataset.csv\` (read-only in the console; convert to JSONL). Each case: \`id\` (required, stable), \`input\` (required object), optional \`title\`, \`expected\`, \`tags\`, \`goal\`, \`metadata\`. For a stage, \`input\` must match the stage's \`io.input.schema\`; cases that do not are skipped (\`dataset.input_mismatch\`, \`eval.cases_skipped\`). Fewer than 5 cases warns. Limits: 5,000 cases.
+
+For an agent, \`input\` is a **scenario**: \`mode: script\` with \`script\` (1–40 user messages), or \`mode: simulated\` with \`opening\`, \`persona\`, \`goal\` and optional \`facts\`. Optional \`tool_mocks\`. \`expected\` may hold \`outcome\`, \`tools\`, \`forbidden_tools\`.
+
+## Graders
+
+Common keys: \`id\`, \`type\`, \`weight\` (default 1), \`must_pass\` (a failure fails the result). 1 to 20 graders.
+
+| Type | Stage | Agent | What it checks |
+|---|---|---|---|
+| \`schema\` | yes | no | Success envelope whose payload matches \`io.output.schema\`. |
+| \`field\` | yes | yes | Values at paths (\`exists\`, \`equals\`, \`contains\`, \`matches\`, \`in\`, \`length_gte\`, …). |
+| \`reference\` | yes | yes | Compares with the case's \`expected\` (\`each_mentioned\` or \`equals\`). |
+| \`rubric\` | yes | yes | A judge model grades against your words (\`pass_fail\` or \`1_to_5\`). Needs a judge. |
+| \`code\` | yes | yes | A JavaScript function body; runs on the Host with your permissions, 5 s limit. |
+| \`tool_calls\` | no | yes | \`must_call\`, \`must_not_call\`, \`must_ask_first\`, \`max_calls\`, \`args\`. |
+| \`conversation_check\` | no | yes | \`max_turns\`, \`max_agent_questions_per_turn\`, \`final_reply\` checks. |
+| \`goal_reached\` | no | yes | A judge decides whether the scenario goal was met. Needs a judge. |
+
+A grader on the wrong kind is \`eval.grader_not_applicable\`. There is no \`command\` or \`groundedness\` grader.
+
+## Defaults and statistics
+
+\`repeats\` 1–10 (default 3), \`pass_threshold\` (default 0.8), \`max_cost_usd\` (default 2; agents 1), \`concurrency\` 1–8 (default 4; agents 2), \`judge\`, \`model\` (override the target's model), \`user_model\` (agent simulator, defaults to the judge). A result passes when no \`must_pass\` grader failed and its weighted score reaches \`pass_threshold\`. The run passes when its pass rate reaches \`pass_threshold\`. Reported: pass rate, a 95% interval by case, reliability (pass on every repeat), pass@k, flaky cases, latency and cost. Compare pairs two runs of the same eval by case: improved, regressed, inconclusive, or not enough data (fewer than 5 common cases).
+
+## Cost
+
+Every run has a cost cap (\`max_cost_usd\`) covering model calls and judge calls. When reached, the run stops as \`budget_stopped\`, remaining results are skipped, and it does not pass. For agent evals the cap is checked before each conversation starts, so a run can exceed it by up to \`concurrency\` conversations. A conversation can cost 4 to 10 times a stage case.
+
+## Judges and the engine
+
+- Judge and simulator models must be \`anthropic/…\`, \`openai/…\` or \`openrouter/…\`, with an API key (Pi credential of type \`api_key\` or \`ANTHROPIC_API_KEY\` / \`OPENAI_API_KEY\` / \`OPENROUTER_API_KEY\`). OAuth logins cannot judge. The stage itself can use any provider.
+- Pick a judge different from the target's model (\`eval.judge_same_as_stage\` warns).
+- The engine (promptfoo 0.124.0) is installed on demand: \`sf eval engine install\` (about 1.2 GB, needs Node 22.22+ and network once). Without it a run fails with \`EVAL_ENGINE_UNAVAILABLE\`.
+- Only **one eval run is active per Host**. A second start fails with \`EVAL_RUN_ACTIVE\`.
+
+## Agent eval safety
+
+- \`conversation.tools\` defaults to \`read_only\`: read tools run; write, edit, bash, destructive or external Stageflow tools, unlisted MCP tools and browser tools are **stubbed** and recorded, not executed. List safe MCP tools in \`conversation.read_only_tools\` (built-in and Stageflow tools cannot be listed). \`mocked\` returns canned results from \`tool_mocks\`.
+- \`tools: real\` needs two consents: \`conversation.allow_real_tools: true\` in the file **and** a per-run consent (\`allow_real_tools\` on \`run_eval\`, a checkbox in the console, \`--allow-real-tools\` on the CLI). Without the run consent: \`EVAL_REAL_TOOLS_NOT_CONFIRMED\`. Real tools act on the project and services from this machine.
+- Approvals in evals never ask a person: \`record_only\` (default) records \`would_ask_first\`; \`deny\`; \`auto_approve_read\`.
+- Always blocked in eval sessions: \`run_eval\`, \`sort_eval_failures\`, \`cancel_eval_run\`, \`agent_chat\`, \`fire_trigger\`; under \`read_only\`/\`mocked\` also \`probe_project_mcp\` and sub-agent calls.
+- Eval sessions never send on channels, run schedules or use web chat, and are hidden from session lists, the Inbox and counts.
+
+## What evals cannot do
+
+No checkout or repository for cases. A stage that asks a person (HITL gate) records \`EVAL_STAGE_NEEDS_INPUT\`. Pipeline-entry overlays (entry-level \`mcp\`, \`secrets\`, \`requires\`, pipeline \`channels\`, \`schemas\`, \`include\`) are not applied when a pipeline stage is evaluated, so it can behave differently than inside its pipeline. No whole-pipeline evals, no scheduled evals, no judge calibration, no deleting an eval from the console.
+
+## Sort failures
+
+After a run, \`sort_eval_failures\` (manual only) suggests for each failed result a bucket (\`agent_wrong\`, \`grader_wrong\`, \`case_wrong\`, \`unclear\`) and a failure type. Classifier \`jev\` (TypeSafe; needs a TypeSafe key, about $0.04 per million input tokens, outside the cost cap) or \`judge\` (the run's judge model). Nothing is tagged until someone accepts (\`accept_eval_failure_suggestions\`).
+
+## Tools
+
+Read (free): \`list_evals\`, \`get_eval\`, \`validate_eval\`, \`list_eval_runs\`, \`get_eval_run\`, \`get_eval_results\`, \`get_eval_result\`, \`compare_eval_runs\`, \`get_eval_failure_suggestions\`, \`get_eval_failure_sort_status\`, \`get_eval_run_export\`. Write: \`create_eval\`, \`update_eval\`, \`create_dataset\`, \`update_dataset_cases\`, \`convert_dataset\`, \`install_eval_sample\`, \`put_eval_review\`, \`accept_eval_failure_suggestions\`, \`dismiss_eval_failure_suggestions\`. Costs money: \`run_eval\`, \`try_eval_case\`, \`sort_eval_failures\`. Also \`cancel_eval_run\`, \`delete_eval_run\` (destructive).
+`,
+};

package/dist/guide/knowledge/topics/limitations.d.ts

@@ -0,0 +1,2 @@
+import type { GuideKnowledgeTopic } from "../index.js";
+export declare const limitationsTopic: GuideKnowledgeTopic;

package/dist/guide/knowledge/topics/limitations.js

@@ -0,0 +1,84 @@
+export const limitationsTopic = {
+    id: "limitations",
+    title: "Limitations: what Stageflow does not do",
+    summary: "Known gaps and sharp edges, each with what the user sees and the workaround: missing deletes, memory and channel limits, schedule triggers without a task, edit_pipeline keys, agent and eval restrictions, operator-only actions, backend gaps.",
+    body: `# Limitations
+
+Each item: the limit, what the user sees, and the workaround.
+
+## Deleting and renaming
+
+- **No delete for catalog triggers.** There is no tool, API route or console delete. Workaround: \`set_trigger_enabled\` false; to remove it, delete the \`*.trigger.yaml\` file by hand.
+- **No delete for pipelines, stage files, tasks, evals or datasets** through tools or the API. Workaround: delete the files by hand. What can be deleted: runs, eval runs, memory cards, connections, console MCP servers; agents are archived.
+- **No delete for a whole memory, and its description cannot change after create.** No tool or route exists. Workaround: create a new memory with a new id, attach it instead, and stop using the old one.
+- **Ids are fixed.** Agent ids cannot change (\`agent.id_immutable\`); \`update_trigger\` cannot change a trigger id; changing a connection id makes a new connection.
+
+## Memories
+
+- **Guide and operators cannot add a memory card.** \`write_memory_card\` only replaces an existing card (\`MEMORY_CARD_NOT_FOUND\` otherwise); the console cannot create cards either. Only stage and agent sessions add cards. Workaround: put seed facts on the cover page, or let the first run add cards.
+- **Sessions cannot delete cards** or read previous copies. Only one previous copy is kept and there is no restore.
+- **Caps:** 64,000 characters per cover or card, 200 cards (\`MEMORY_TOO_LARGE\`, \`MEMORY_TOO_MANY_CARDS\`).
+- **No per-stage memories.** A pipeline's memories go to every stage.
+
+## Channels and connections
+
+- **One channel per stage.** A second entry, or one inherited from a pipeline default, fails with "a stage can use one channel for now". Workaround: two stages, or one AgentMail connection with several \`to\`/\`cc\` recipients.
+- **Stage channels are send-only.** A stage never receives messages. WhatsApp is send-only for agents too. An IMAP email account cannot be an agent channel.
+- **Connection secrets cannot be read back.** Tools and screens show only whether a secret comes from \`env\` or \`store\`. Workaround: replace it with \`update_connection\`.
+- **An unknown send outcome is never resent** (\`CHANNEL_SEND_OUTCOME_UNKNOWN\`). Workaround: confirm with the recipient, then send deliberately with a new key.
+- **Channels and memory tools are Pi-only.** On the Claude backend such stages fail at start.
+
+## Triggers
+
+- **A schedule trigger needs a catalog task.** \`create_trigger\` accepts a schedule without \`task\`, but every automatic fire fails with \`trigger.task_required\`. The console blocks it. Workaround: always set \`task\`.
+- **Dynamic triggers cannot be fired from the console** (Fire is disabled). Workaround: \`fire_trigger\` with an inline \`task\`.
+- **Only some event sources run.** GitHub (pull requests only, first poll fires nothing), signed webhook, IMAP email. Any other \`event.source\` is never watched. Email \`match\` is exact (no substring) and has no attachments; a new email trigger may need a Host restart. Webhooks need a reachable Host (loopback by default). The console cannot edit \`event.config\`; set it with \`create_trigger\`/\`update_trigger\` or YAML.
+- **Automatic triggers need a running Host** (\`sf ui\`, \`sf mcp\` or Docker).
+
+## Authoring tools
+
+- **\`edit_pipeline\` only applies these patch keys:** \`id\`, \`model\`, \`agent\`, \`stages\`, \`schemas\`, \`requires\`, \`memories\`, \`channels\`. Others (\`browser\`, \`include\`) are silently ignored, and \`model\` cannot be cleared. \`stages\` replaces the whole list. Workaround: edit the YAML in the pipeline editor or by hand.
+- **\`edit_stage\` merges fields and cannot remove one.** Workaround: \`write_catalog_file\` with the full stage YAML.
+- **\`create_stage\` writes the Host default model** into the stage when none is passed, overriding the pipeline default.
+- **Standalone task tools have no \`input\` argument** (\`create_task_file\`, \`update_task_file\`). Workaround: \`create_task\`/\`edit_task\` with \`input\` inside \`body\`, or edit the file.
+- **No tool to log in to a model provider, add a project MCP server, set an MCP secret or install a skill.** These are done in Settings, the Catalog page or the CLI.
+
+## Agents
+
+- **A new agent always starts disabled.** Workaround: \`enable_agent\` after validating.
+- **An agent cannot grant beyond what it holds, cannot weaken an ask-first it is bound by, and cannot edit, restore or archive itself.** The tool returns a refusal naming the extra grants.
+- **Not functional yet:** \`model.backend: claude\` and \`execution: sandbox\` (sessions fail), \`tools.browser\` (stored, unused), \`approvals.approvers.channel_ids\` (ignored).
+- **\`bash\` is not path-confined** and always asks first.
+- **Restarts:** running or waiting turns fail with \`agent.host_restarted\`; pending approvals expire.
+- **Cursor models** are refused for agents unless the Host opts in.
+
+## Evals
+
+- **The engine must be installed** (\`sf eval engine install\`, Node 22.22+, about 1.2 GB). Otherwise \`EVAL_ENGINE_UNAVAILABLE\`.
+- **One eval run per Host.** A second start fails with \`EVAL_RUN_ACTIVE\`. Workaround: wait or \`cancel_eval_run\`.
+- **Judges only from \`anthropic\`, \`openai\`, \`openrouter\`, with an API key** (not OAuth).
+- **No checkout, no gates:** cases run without a repository; a stage that asks a person ends \`EVAL_STAGE_NEEDS_INPUT\`.
+- **Pipeline overlays are not applied** to an evaluated pipeline stage (entry-level \`mcp\`, \`secrets\`, \`requires\`, pipeline \`channels\`, \`schemas\`, \`include\`).
+- **Not built:** whole-pipeline evals, scheduled evals, judge calibration, \`command\` and \`groundedness\` graders, deleting an eval in the console.
+- **Agent eval cost cap** is checked before each conversation, so it can be passed by up to \`concurrency\` conversations.
+
+## Operator-only actions
+
+- **Only a person can answer a pipeline gate (\`answer_gate\`), decide a feedback loop at its limit (\`decide_feedback_loop\`), or approve/deny an agent action.** Guide and other agents cannot. Workaround: point the user to the Inbox (\`#/inbox\`).
+
+## Pipelines and runs
+
+- **No run-level cost cap for pipelines.** Use cheaper models, \`timeout_ms\`, and small repair limits.
+- **Route \`if\` is limited:** only required scalar fields, no \`exists\`, no else branch, no array indexes.
+- **\`on: [failed]\` does not launch a stage after failure.**
+- **A clone chain fails on an empty list or one longer than \`clone_cap\`.**
+- **Validation does not check provider keys, env vars or live MCP connections.** Those fail at run start.
+
+## MCP, skills, browser
+
+- **OAuth-signed-in MCP servers do not work in pipeline stages.** Use a bearer-token secret.
+- **MCP elicitation is not supported.**
+- **Browser \`allow_domains\` is not enforced for profile stages**; passkeys and native prompts do not work; a login stage fails on a Host with no screen and no live view; another run waits for a leased profile with no time limit.
+- **Hosted multi-user use is not built.** Browser profiles and connections use a single local owner.
+`,
+};

package/dist/guide/knowledge/topics/mcpSkillsBrowser.d.ts

@@ -0,0 +1,2 @@
+import type { GuideKnowledgeTopic } from "../index.js";
+export declare const mcpSkillsBrowserTopic: GuideKnowledgeTopic;

package/dist/guide/knowledge/topics/mcpSkillsBrowser.js

@@ -0,0 +1,57 @@
+export const mcpSkillsBrowserTopic = {
+    id: "mcp_skills_browser",
+    title: "MCP servers, skills and browser sessions",
+    summary: "Giving stages and agents extra tools: project MCP servers (.mcp.json and console servers, secrets, OAuth), skills (binding and install), and browser sessions (profiles, login check, human login stage, limits).",
+    body: `# MCP servers, skills and browser sessions
+
+## MCP servers
+
+MCP servers give a stage or an agent tools in other systems (issue trackers, docs, web fetch, Playwright). Two sources, merged:
+
+- the project \`.mcp.json\` next to \`stageflow.yaml\` (\`{ "mcpServers": { "name": { ... } } }\`);
+- servers saved in the console (**Settings → Project MCP**, stored on the Host). A console server wins on a name clash. Project-file servers cannot be removed from the console.
+
+A server is stdio (\`command\`, \`args\`, \`env\`, \`cwd\`) or http (\`url\`, \`headers\`, \`httpTransport\`). The name \`stageflow\` is reserved. Link-local and cloud-metadata URLs are refused.
+
+**Secrets.** Tokens are moved out of the saved config into a secret store and replaced with \`\${NAME}\` placeholders. Values are write-only. Placeholders naming provider keys or Stageflow control tokens are refused unless allow-listed.
+
+**Who fills \`\${NAME}\`:**
+- Agents and the Check button: console secrets, plus Host variables named in \`STAGEFLOW_MCP_ENV_ALLOW\`.
+- Pipeline stages: only console secrets the stage lists under \`secrets:\` (they fill MCP placeholders and are not put in the stage environment), plus the stage's curated environment.
+- Repository-bound stages: never console servers or console secrets; they read the repository's own \`.mcp.json\`.
+
+**OAuth.** Remote servers can be signed in from the console (copy the redirected address back into the dialog). Signed-in servers work for agents and Check only, **not for pipeline stages**; give a stage a bearer-token secret instead.
+
+**Using it.**
+- Stage: \`mcp: [linear]\` on the pipeline entry (or the stage file) plus \`secrets: [LINEAR_TOKEN]\` if needed. \`validate\` checks names and that stdio commands are on PATH; connect failures and unset variables fail at run time. Default connect timeout 30 s. Large tool results are written to an artifact file and the model sees a preview. MCP elicitation (a server asking the user a question) is not supported.
+- Agent: \`tools.mcp: { linear: ["*"] }\` or a list of tool names. Grant id \`<server>.<tool>\`, usable in \`approvals.ask_first\`. Servers start lazily per session. An unknown server only warns (\`agent.mcp_server_unknown\`).
+- Tools: \`list_project_mcp\` (read), \`probe_project_mcp\` (connects to the server). There is no tool to add or remove a project MCP server or set an MCP secret; that is done in Settings → Project MCP.
+
+## Skills
+
+A skill is an instruction bundle (\`SKILL.md\`) the agent loads.
+
+- Stage: \`skill: <name>\` on the stage **body** (stage file or inline), not on a \`uses:\` wrapper. One skill per stage.
+- Agent: \`tools.skills: [names]\`.
+- Lookup order for a stage: skills shipped with the run (\`start_run.skills\`), then \`<checkout>/.pi/skills/<name>/\` (needs workspace trust for repository bindings), then the Host's Pi skills directory.
+- A missing skill fails the stage before the agent starts, or fails the agent session.
+- Install on the Host: console **Catalog → Skills** (Install), or \`sf skills install --from-zip <url> --skill-name <name>\`. \`list_skills\` shows name, description and origin. There is no tool to install a skill.
+
+## Browser sessions
+
+A stage with \`browser:\` drives a real Chrome through \`agent-browser\` (must be installed on the Host: \`npm i -g agent-browser && agent-browser install\`). The agent uses it through bash and the bundled \`browser\` skill.
+
+- \`profile\`: a saved login on the Host. YAML names it, never a path. No profile means a throw-away browser.
+- One shared browser per run and profile; each stage gets its own tab; tabs share cookies. A profile is leased to one run at a time: another run that needs it waits (no time limit) until the holder run ends.
+- \`headed\` (default true) or \`false\`. Use one mode for all stages that share a profile.
+- \`allow_domains\`: enforced only for stages without a profile. For profile stages it is only a prompt instruction plus an audit, not a security boundary.
+- \`check\`: the Host opens \`check.url\` and decides logged in / out / unknown from \`logged_in_url\` and \`logged_out_url\` globs; the stage must emit \`{ logged_in, url }\`.
+- Human login stage: \`verify: [{ type: browser_login }]\`, \`gate_kinds: [confirm]\`, \`browser.check\` set, \`headed\` not false, usually with \`on_verify_fail: { mode: repair, ... }\`. The operator logs in in a visible window or the console's live view, then confirms.
+- Login pattern: check → login (only the check→login edge has \`if: logged_in eq false\`) → work, where work joins check and login with no \`if\` on its inbound edges. Putting an \`if\` on check→work silently skips work when the login stage runs.
+- A Host with no screen and no live view (CLI-only run on a headless Linux) fails a login stage before the agent starts.
+- Passkeys, hardware keys, camera, location and notification prompts do not work; permission prompts are denied.
+- \`path\`, \`scope\`, \`secret\`, \`launch_args\`, \`executable_path\` are rejected in YAML; the Host decides those.
+- Many sites forbid automation in their terms. Keep volume low and stay read-only where possible.
+- Agents: \`tools.browser\` is stored but not wired to anything yet.
+`,
+};

package/dist/guide/knowledge/topics/memories.d.ts

@@ -0,0 +1,2 @@
+import type { GuideKnowledgeTopic } from "../index.js";
+export declare const memoriesTopic: GuideKnowledgeTopic;

package/dist/guide/knowledge/topics/memories.js

@@ -0,0 +1,63 @@
+export const memoriesTopic = {
+    id: "memories",
+    title: "Memories: durable notes for pipelines and agents",
+    summary: "Host memories with a cover page and keyed cards, attaching them to a pipeline or an agent, which memory tools a session gets, size caps, and what cannot be changed or deleted.",
+    body: `# Memories
+
+A memory is a named store on the Host (\`$STAGEFLOW_HOME/memories/<id>/\`). It survives runs and conversations. It has:
+
+- a **cover page** (\`body.md\`): one free-text page, replaced as a whole;
+- **cards**: keyed notes, one per subject (a person, a ticket, a release, a path). The key is the subject's own id (an email, a ticket number, a version), 1 to 1024 characters.
+
+A memory also has an \`id\` (\`[a-z][a-z0-9_-]{0,63}\`) and a \`description\` (1 to 500 characters) set at creation.
+
+## Create and attach
+
+1. Create the memory first: console **Memories**, or \`create_memory\` with \`id\` and \`description\`. The cover starts empty.
+2. Attach it:
+   - to a **pipeline** with the top-level \`memories: [id]\` list (tool: \`edit_pipeline\` with \`patch: { memories: [...] }\`; an empty list removes it). Every stage in that pipeline gets it. A stage cannot list memories of its own (\`pipeline.invalid_shape\`).
+   - to an **agent** with its \`memories: [id]\` field (tool: \`save_agent\`). Sub-agents do not inherit it.
+3. A missing, unreadable or mismatched id blocks validation and run start (\`pipeline.memory_unavailable\`) or agent save and enable (\`agent.memory_unavailable\`). A duplicate id in the list is \`pipeline.invalid_memories\`.
+
+## What a session gets
+
+A stage of a pipeline with memories (Pi backend) and every session of an agent with memories get six tools:
+
+- \`read_memory\`, \`replace_memory\` for the cover;
+- \`add_memory_card\` (refuses a key that exists), \`update_memory_card\` (refuses a key that is missing), \`read_memory_card\`, \`search_memory_cards\` (case-insensitive word scan of keys and bodies, at most 20 hits, key plus snippet).
+
+The prompt lists only the memory ids and descriptions, not the cover or card text; the session must read them with the tools. The session never gets a file path, and Pi stages with memories are blocked from reaching the memory store through the shell. A session cannot delete a card and cannot read a previous copy. The Claude backend has no memory tools: a pipeline with \`memories\` fails at stage start there.
+
+## Caps
+
+- Cover and each card: at most 64,000 characters. A larger write is refused (\`MEMORY_TOO_LARGE\`).
+- At most 200 cards per memory (\`MEMORY_TOO_MANY_CARDS\`).
+- Two cover replacements at the same time: the last write wins.
+- One previous copy of the cover and of each card is kept, not a full history, and there is no restore action.
+
+## What people and tools can do
+
+| Action | Session (stage or agent) | Operator in console | Guide / operator tools |
+|---|---|---|---|
+| Create a memory | no | yes | \`create_memory\` |
+| Read cover and card keys | yes | yes | \`get_memory\`, \`list_memories\` |
+| Replace the cover | yes | yes | \`write_memory\` |
+| Add a new card | yes | no | no |
+| Replace an existing card | yes | yes | \`write_memory_card\` (refuses a missing card) |
+| Delete a card | no | yes (also removes its previous copy; cannot be undone) | \`delete_memory_card\` (destructive) |
+| Change the description | no | no | no |
+| Delete a whole memory | no | no | no |
+
+So cards are written by the sessions that do the work. To seed facts before the first run, put them on the cover page, or let the first run add cards.
+
+## Memory or task context?
+
+- Facts for one run go in the task (\`context\`, \`constraints\`, \`input\`).
+- Facts that must persist and be updated across runs or conversations go in a memory: contacts already messaged, per-customer notes, release history.
+- Name the memory and write a clear description: it is all the session sees before it calls a tool.
+
+## Error codes
+
+\`MEMORY_INVALID_ID\`, \`MEMORY_INVALID_DESCRIPTION\`, \`MEMORY_INVALID_KEY\`, \`MEMORY_EXISTS\`, \`MEMORY_NOT_FOUND\`, \`MEMORY_NO_PREVIOUS\`, \`MEMORY_CARD_EXISTS\`, \`MEMORY_CARD_NOT_FOUND\`, \`MEMORY_TOO_MANY_CARDS\`, \`MEMORY_MISMATCH\`, \`MEMORY_UNREADABLE\`, \`MEMORY_TOO_LARGE\`, \`MEMORY_NOT_ATTACHED\`.
+`,
+};

package/dist/guide/knowledge/topics/modelsAndProviders.d.ts

@@ -0,0 +1,2 @@
+import type { GuideKnowledgeTopic } from "../index.js";
+export declare const modelsAndProvidersTopic: GuideKnowledgeTopic;

package/dist/guide/knowledge/topics/modelsAndProviders.js

@@ -0,0 +1,56 @@
+export const modelsAndProvidersTopic = {
+    id: "models_and_providers",
+    title: "Models and providers",
+    summary: "How a stage's model is resolved (stage, pipeline, stageflow.yaml), what must be connected before a run works, model ids and the catalog, agent and eval models, and where credentials live.",
+    body: `# Models and providers
+
+Stages and agents run on **Pi**. A model id is \`provider/model\` (for example \`anthropic/claude-sonnet-4-5\` or \`openrouter/…\`). A provider must be connected on the Host before runs that use it can succeed.
+
+## Connecting a provider
+
+- Console: **Settings → Providers**, or the **Connect** page (\`#/connect\`). API key or OAuth, depending on the provider.
+- CLI: \`sf providers login <id> --type api_key --api-key-env VAR\` (raw keys on the command line are refused), \`sf providers status\`.
+- At Host boot: \`STAGEFLOW_PROVIDER_<ID>_API_KEY\` or \`_FILE\`. The Host writes this key back at every boot, so it returns after a restart even if someone disconnected the provider in the console.
+- Credentials live in \`<credential root>/agent/auth.json\` (default \`~/.stageflow/agent/auth.json\`). They are never put into a stage's environment.
+- There is no tool to log in, log out or add a provider key. Guide can only look: \`list_providers\`, \`list_models\`, \`get_health\`.
+
+## How a stage's model is chosen
+
+Resolved at pipeline load:
+
+\`\`\`text
+stage.model ?? pipeline.model ?? stageflow.yaml model
+\`\`\`
+
+- Pipeline \`model\` is read only from the root pipeline file, not from included fragments. When it fills a stage, \`validate\` warns \`pipeline.model_applies\` (just information).
+- If all three are empty, the pipeline does not load: "model is required (set on stage, pipeline, or stageflow.yaml)". There is no silent fallback for stages.
+- Validation does not check that the provider is connected. A missing key shows up when the run starts.
+- \`agent\` (pipeline or manifest) picks the backend, \`pi\` by default. It is separate from \`model\`. The Claude backend lacks \`send_message\` and memory tools.
+- \`create_stage\` without a \`model\` writes the Host's current default model into the new stage file, so that stage no longer follows a pipeline default. Pass the model you want. \`edit_stage\` merges fields and cannot remove one; to let the pipeline default apply, rewrite the stage file without \`model\` using \`write_catalog_file\`.
+
+## The model catalog
+
+One catalog feeds every picker, \`list_models\` and the default model.
+
+- Only connected providers appear. Lists are fetched live and cached for 30 seconds, with a built-in fallback when a provider is unreachable.
+- Any \`provider/model\` id can be typed; an id no connected provider lists is saved and runs as written, with a warning.
+- The catalog default (used by pickers, by \`create_stage\`, and by the Workshop) is: \`stageflow.yaml\` \`model\`, then the Workshop model in Settings, then the catalog's pick (a free OpenRouter model when connected, else the first free model, else the first model; never a Cursor model while another exists). This does not change stage resolution above.
+- Cursor models (\`cursor/…\`) need a Cursor API key. Agents refuse Cursor models unless the Host sets \`STAGEFLOW_AGENTS_ALLOW_CURSOR_NATIVE_TOOLS=1\`, because Cursor runs its own file and shell tools and grants cannot hold (\`agent.model_native_tools\`).
+
+## Agents
+
+\`model: { backend: pi, id, thinking }\`. \`id\` defaults to the project default model. \`thinking\` is \`off\`, \`low\`, \`medium\` (default) or \`high\` for models that support it. An unknown id only warns (\`agent.model_unknown\`).
+
+## Evals
+
+The stage or agent may use any provider. Judges and the agent simulator must be \`anthropic\`, \`openai\` or \`openrouter\` with an API key (OAuth logins cannot judge). The eval's \`defaults.model\` (or a run override) replaces the target's model for that eval.
+
+## Workshop chat model
+
+A new Workshop chat uses \`stageflow.yaml\` \`model\`, then the Settings Workshop model, then \`cursor/auto\`.
+
+## Cost
+
+Run and agent cost come from provider token reports. A provider that reports no cost adds nothing to totals. Overview "spend" sums stage attempt costs. Agent \`limits.cost_usd\` and eval \`max_cost_usd\` are the cost brakes; pipelines have no run-level cost cap field.
+`,
+};

package/dist/guide/knowledge/topics/overview.d.ts

@@ -0,0 +1,2 @@
+import type { GuideKnowledgeTopic } from "../index.js";
+export declare const overviewTopic: GuideKnowledgeTopic;

package/dist/guide/knowledge/topics/overview.js

@@ -0,0 +1,56 @@
+export const overviewTopic = {
+    id: "overview",
+    title: "Building blocks and which one to pick",
+    summary: "What pipelines, stages, tasks, agents, triggers, connections, memories, MCP, skills and evals are, how they relate, and how to choose between them.",
+    body: `# Building blocks and which one to pick
+
+Stageflow runs configurable multi-stage workflows and project agents, with a local console. Work can be anything repeated: releases, research, content review, operations runbooks, support, software changes. Nothing in the product is tied to one domain.
+
+## The pieces
+
+- **Pipeline** (\`*.pipeline.yaml\`). An ordered graph of stages. Each stage runs in a fresh agent session, does one job and hands a typed result (an envelope) to the next stage. Wiring is declared on the source stage with \`route\`.
+- **Stage**. One step of a pipeline: a system prompt, a model, a required input and output schema (\`io\`), and optional checks (\`verify\`), questions to a person (\`gate_kinds\`), tools (MCP, skill, browser), a send channel and secrets. A stage can live inline in the pipeline file or in its own YAML file referenced with \`uses:\`.
+- **Task** (\`*.task.yaml\`). The input a pipeline run starts from: \`id\`, \`goal\`, optional \`context\`, \`constraints\`, \`input\` (structured data for the entry stage) and an optional working folder or repository. A run is always one pipeline plus one task.
+- **Run**. One execution of a pipeline with a task. Runs are stored on the Host under \`$STAGEFLOW_HOME\` (default \`~/.stageflow/\`), not in the project.
+- **Agent** (\`agents/<id>.agent.yaml\`). A configured assistant that holds a conversation over many turns: instructions, a model, the tools it may use, ask-first approvals, limits, and how people reach it (console, public web chat page, one inbound channel, or a schedule).
+- **Trigger** (\`*.trigger.yaml\`). Starts a pipeline run without someone clicking. Kinds: \`manual\` (fired on request), \`schedule\` (cron), \`event\` (GitHub pull requests, a signed webhook, or an IMAP mailbox).
+- **Connection**. A host-level link to a messaging provider (AgentMail, Slack, Telegram, iMessage via Photon, WhatsApp via Photon). A stage uses it to send one message to a recipient fixed in YAML. An agent can also answer on it.
+- **Memory**. A named store on the Host with a cover page and keyed cards. Attached to a whole pipeline (every stage gets it) or to an agent. Sessions read and write it with memory tools.
+- **MCP servers and skills**. Extra tools for a stage or an agent: MCP servers from the project \`.mcp.json\` or saved in the console, and installed skills.
+- **Eval** (\`evals/<id>.eval.yaml\`) with a **dataset**. Runs one stage or one agent over many cases, grades each output, repeats each case, and compares runs.
+
+## How they relate
+
+A trigger points at a pipeline and (usually) a task. The pipeline's stages may list MCP servers, a skill, a browser profile, one channel and secrets. The pipeline may attach memories. An eval points at one stage or one agent plus a dataset. An agent may list MCP tools, skills, Stageflow tools, sub-agents, memories and one connection.
+
+## Which one to pick
+
+**Pipeline vs agent**
+- Pick a **pipeline** when the work has known steps in a fixed order, each step can be described with an input and an output, and you want checks between steps. Runs are repeatable and inspectable stage by stage.
+- Pick an **agent** when someone needs to talk back and forth, the next step depends on the conversation, or people should reach it by chat, email, Slack, Telegram, iMessage or a web page.
+- An agent can start pipeline runs if it is granted the \`runs.control\` tool group (\`start_run\`), so a conversational front end can hand fixed work to a pipeline.
+
+**One stage vs many stages**
+- Start with a one-stage pipeline when the job is a single step. Split into stages when a step has its own clear output, needs a different model or tools, needs a person to approve before continuing, or should be checked on its own.
+- Many small stages cost more model sessions; one huge stage is hard to check and to recover.
+
+**Task vs trigger**
+- A **task** is what to do. A **trigger** is when to do it. You always need a pipeline; you need a catalog task for anything started from the console's Start a run, for a manual trigger fired from the console, and for a schedule trigger.
+- A trigger without a task (dynamic mode) only works when whoever fires it supplies the task: GitHub, webhook and email adapters build one from the event, or a manual fire passes one inline.
+
+**Memory vs task context**
+- Put facts for one run in the task (\`context\`, \`constraints\`, \`input\`).
+- Use a **memory** when facts must survive across runs or conversations and be updated by the sessions themselves (a contact list, per-ticket notes, what was already sent).
+
+**Connection vs MCP**
+- Use a **connection** when a stage should notify someone and the recipient must be fixed. The agent cannot change the recipient.
+- Use an **MCP server** when the stage or agent must read or act in another system (an issue tracker, docs, a database).
+
+**Eval**
+- Add an eval when a wrong answer is costly or you are about to change a prompt, model or tools and want to know if it got better or worse.
+
+## Who can do what
+
+Guide and other agents can create and edit almost everything with tools. Only a person can answer a pipeline gate, decide a feedback loop at its limit, or approve or deny an agent's ask-first action (tool group \`gates.decide\` is operator-only).
+`,
+};

package/dist/guide/knowledge/topics/pipelines.d.ts

@@ -0,0 +1,2 @@
+import type { GuideKnowledgeTopic } from "../index.js";
+export declare const pipelinesTopic: GuideKnowledgeTopic;